From b0e6eadf1cc60af32ac770df64092a758cebf169 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 03:11:55 -0400 Subject: [PATCH 01/84] B6: receipt - main's CI green Co-Authored-By: Claude Opus 5.5 --- .../consolidation-20261009/B6-ci-green.md | 34 +++++++++++++++++++ 1 file changed, 34 insertions(+) create mode 100644 docs/refactor/consolidation-20261009/B6-ci-green.md diff --git a/docs/refactor/consolidation-20261009/B6-ci-green.md b/docs/refactor/consolidation-20261009/B6-ci-green.md new file mode 100644 index 0000000000..0938dd5f17 --- /dev/null +++ b/docs/refactor/consolidation-20261009/B6-ci-green.md @@ -0,0 +1,34 @@ +# B6: main's CI green + +**Head audited:** `openhcs` `main` at `84e7a2904` (#1193). **Rules:** [00-RULES.md](00-RULES.md). **Input:** the last completed runs on main: Integration Tests 38116770859 (`b468e7359`, W), Website 38110269836, Documentation 38116770933. + +## What is wrong + +**Three workflows are red on main, and Documentation has not passed on main in the last 1,000 runs. Most jobs fail for a handful of shared causes that only CI exposes (wheel install, `--import-mode=importlib`, Windows, path filters), so local runs never see them.** + +Causes measured on the W run, before B5 merged (to be re-measured on this PR's CI): + +- **Website:** the job installs a hand-written roster of five first-party packages instead of OpenHCS, then imports `openhcs.authoring.session.operations`, whose graph needs `numba` (and the rest of OpenHCS's dependencies). The `paths:` filter lists `openhcs/__init__.py` only, so refactors that broke the import never triggered it. +- **unit-tests:** collection stops at 17 errors. 14 test modules import a sibling test module by bare name (`from test_function_outputs import …`); CI runs `--import-mode=importlib` from the installed wheel, where the test directory is not on `sys.path`. The rest were fixed by B5. +- **Documentation:** `scripts/validate_docs.py docs/source` reports 950 findings: 828 "authority changed after review", 78 "authority file does not exist", 34 "source changed after its editorial audit" over 140 pages in `docs/audits/*.json`, and 10 missing `openhcs/skills/use-openhcs/references/*.md` paths in the MCP knowledge-base manifest. +- **Integration (other jobs):** listed per job under Remaining, with owner, as measured on this PR. + +## Target + +- Each workflow installs what it imports through the one candidate installer (`scripts/install_ci_candidate.py`), never a roster. +- Test helpers are imported through the `tests` package. +- Root causes fixed in production, packaging or workflows; no skip, xfail, `continue-on-error` or loosened check. + +## Guards + +- `tests/unit/test_ci_import_guards.py`: no test module imports a test-tree module by bare name. +- `tests/unit/test_ci_package_boundaries.py`: the website installs the OpenHCS candidate through `install_ci_candidate`. +- `tests/unit/test_build_website.py`: the website workflow triggers on `openhcs/**`. + +## Done when + +Website, Documentation and Integration Tests are green on this PR's head, or each remaining red job names its exact cause and owning surface below. + +## Remaining + +(filled from this PR's CI) From 6bccc4d6d72c247658b889d3b4e570b2dea964af Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 03:11:55 -0400 Subject: [PATCH 02/84] B6: CI imports - test helpers through the tests package; website installs the OpenHCS candidate Under --import-mode=importlib (run_installed_tests.py) the test directory is not on sys.path, so 14 modules importing a sibling test module by bare name stopped unit-test collection. The website validated against a five-package roster and then imported openhcs, whose graph needs numba; it now installs the candidate through install_ci_candidate and triggers on openhcs/**. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/website-pages.yml | 71 ++++--------------- .../test_mcp_dev_client_sampling_results.py | 2 +- .../agent/test_mcp_dev_client_ui_renderers.py | 2 +- .../agent/test_mcp_plate_stream_progress.py | 2 +- .../test_selected_plane_reopen_stream.py | 2 +- tests/unit/test_aggregate_source_scope.py | 2 +- ...utomatic_image_role_materialization_435.py | 2 +- tests/unit/test_build_website.py | 5 +- tests/unit/test_ci_import_guards.py | 38 ++++++++++ tests/unit/test_ci_package_boundaries.py | 7 +- .../unit/test_derived_role_publication_435.py | 2 +- .../test_gray_to_color_binding_axis_432.py | 2 +- ..._gray_to_color_payload_axis_compile_432.py | 4 +- .../unit/test_named_checkpoint_publication.py | 2 +- .../test_rescale_stream_plane_declaration.py | 2 +- ...t_selected_plane_stream_materialization.py | 4 +- tests/unit/test_snapshot_deadline.py | 2 +- tests/unit/test_viewer_spatial_sample.py | 2 +- 18 files changed, 74 insertions(+), 79 deletions(-) create mode 100644 tests/unit/test_ci_import_guards.py diff --git a/.github/workflows/website-pages.yml b/.github/workflows/website-pages.yml index b0a85da525..c7163890a4 100644 --- a/.github/workflows/website-pages.yml +++ b/.github/workflows/website-pages.yml @@ -4,34 +4,36 @@ on: pull_request: paths: - "website/**" - - "openhcs/resources/assets/openhcs-*.svg" + - "openhcs/**" + - "external/**" + - "scripts/install_ci_candidate.py" + - "scripts/validate_local_release_floors.py" - "docs/source/_static/ui.png" - "scripts/build_website.py" - "scripts/capture_media_gallery.py" - "scripts/gallery_catalog.py" - "scripts/website_gallery_projection.py" - - "scripts/validate_local_release_floors.py" - "docs/media_gallery_capture.md" - "tests/unit/test_build_website.py" - "tests/unit/test_capture_media_gallery.py" - - "openhcs/__init__.py" - "pyproject.toml" - ".github/workflows/website-pages.yml" push: branches: [main] paths: - "website/**" - - "openhcs/resources/assets/openhcs-*.svg" + - "openhcs/**" + - "external/**" + - "scripts/install_ci_candidate.py" + - "scripts/validate_local_release_floors.py" - "docs/source/_static/ui.png" - "scripts/build_website.py" - "scripts/capture_media_gallery.py" - "scripts/gallery_catalog.py" - "scripts/website_gallery_projection.py" - - "scripts/validate_local_release_floors.py" - "docs/media_gallery_capture.md" - "tests/unit/test_build_website.py" - "tests/unit/test_capture_media_gallery.py" - - "openhcs/__init__.py" - "pyproject.toml" - ".github/workflows/website-pages.yml" workflow_dispatch: @@ -46,7 +48,7 @@ concurrency: jobs: validate: runs-on: ubuntu-latest - timeout-minutes: 10 + timeout-minutes: 30 steps: - name: Check out repository uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 @@ -58,57 +60,12 @@ jobs: with: python-version: "3.12" - - name: Install website test dependencies + - name: Install the OpenHCS candidate the website imports run: | - python -m pip install pytest - mapfile -t local_project_paths < <( - python - <<'PY' - from pathlib import Path - - from packaging.utils import canonicalize_name - - from scripts.validate_local_release_floors import ( - discover_local_projects, - read_project, - ) - - project = read_project(Path("pyproject.toml")) - required_names = { - canonicalize_name(name) - for name in ( - "metaclass-registry", - "polystore", - "python-introspect", - "zmqruntime", - ) - } - declared_names = { - canonicalize_name(requirement.name) - for requirement in project.dependencies - } - undeclared_names = required_names - declared_names - if undeclared_names: - raise SystemExit( - "Missing website dependencies in project metadata: " - + ", ".join(sorted(undeclared_names)) - ) - local_projects = { - candidate.canonical_name: candidate - for candidate in discover_local_projects() - } - missing_sources = required_names - local_projects.keys() - if missing_sources: - raise SystemExit( - "Missing website dependency submodules: " - + ", ".join(sorted(missing_sources)) - ) - print( - *(local_projects[name].path.parent for name in sorted(required_names)), - sep="\n", - ) - PY - ) - python -m pip install "${local_project_paths[@]}" + python -m pip install --upgrade pip build pytest + python -m scripts.install_ci_candidate \ + --dependency-source submodules \ + --wheel-directory "$RUNNER_TEMP/openhcs-candidate-wheels" - name: Test, build, and validate landing page run: | diff --git a/tests/unit/agent/test_mcp_dev_client_sampling_results.py b/tests/unit/agent/test_mcp_dev_client_sampling_results.py index ffb7621857..169aca3cec 100644 --- a/tests/unit/agent/test_mcp_dev_client_sampling_results.py +++ b/tests/unit/agent/test_mcp_dev_client_sampling_results.py @@ -82,7 +82,7 @@ def command_render(response, argv): @pytest.mark.parametrize("structured", [False, True]) @pytest.mark.parametrize("selected_case", [False, True]) def test_original_session_framing_to_generated_sampling_command(structured, selected_case, monkeypatch): - from test_mcp_dev_client_pipeline_results import ControlledWireSession + from tests.unit.agent.test_mcp_dev_client_pipeline_results import ControlledWireSession import openhcs.mcp.dev_client_core as ingress value = selected(sample()) if selected_case else sample() diff --git a/tests/unit/agent/test_mcp_dev_client_ui_renderers.py b/tests/unit/agent/test_mcp_dev_client_ui_renderers.py index 5f8a8a6057..f7dd2edd0e 100644 --- a/tests/unit/agent/test_mcp_dev_client_ui_renderers.py +++ b/tests/unit/agent/test_mcp_dev_client_ui_renderers.py @@ -60,7 +60,7 @@ UiActionCatalogRenderOptions, WidgetTreeRenderOptions, ) -from test_mcp_dev_client_renderers import SERVER, framed, sample_dto +from tests.unit.agent.test_mcp_dev_client_renderers import SERVER, framed, sample_dto def render(payload, options=None) -> str: diff --git a/tests/unit/agent/test_mcp_plate_stream_progress.py b/tests/unit/agent/test_mcp_plate_stream_progress.py index 87289aa9c1..9c71292a29 100644 --- a/tests/unit/agent/test_mcp_plate_stream_progress.py +++ b/tests/unit/agent/test_mcp_plate_stream_progress.py @@ -30,7 +30,7 @@ from openhcs.mcp.server import build_server from openhcs.mcp.socket import McpSocketTransport, wait_for_socket -from test_persisted_output_reopen import NoRuntimeBridge, declared_output +from tests.unit.agent.test_persisted_output_reopen import NoRuntimeBridge, declared_output from tests.unit.test_streaming_service import FakeViewer diff --git a/tests/unit/agent/test_selected_plane_reopen_stream.py b/tests/unit/agent/test_selected_plane_reopen_stream.py index 333e000a04..f6416bf0ec 100644 --- a/tests/unit/agent/test_selected_plane_reopen_stream.py +++ b/tests/unit/agent/test_selected_plane_reopen_stream.py @@ -39,7 +39,7 @@ ) from openhcs.runtime.napari_viewer_server import NapariBatchPayload, NapariImagePayload -from test_persisted_output_reopen import NoRuntimeBridge, declared_output +from tests.unit.agent.test_persisted_output_reopen import NoRuntimeBridge, declared_output @pytest.fixture diff --git a/tests/unit/test_aggregate_source_scope.py b/tests/unit/test_aggregate_source_scope.py index d312a8b1b3..318d135176 100644 --- a/tests/unit/test_aggregate_source_scope.py +++ b/tests/unit/test_aggregate_source_scope.py @@ -35,7 +35,7 @@ from polystore.filemanager import FileManager from polystore.memory import MemoryStorageBackend -from test_materialization_core import _viewer_stream_backend_kwargs +from tests.unit.test_materialization_core import _viewer_stream_backend_kwargs def _source(sites): diff --git a/tests/unit/test_automatic_image_role_materialization_435.py b/tests/unit/test_automatic_image_role_materialization_435.py index 392d0ebae8..bb666aa7f8 100644 --- a/tests/unit/test_automatic_image_role_materialization_435.py +++ b/tests/unit/test_automatic_image_role_materialization_435.py @@ -11,7 +11,7 @@ from polystore.filemanager import FileManager from polystore.memory import MemoryStorageBackend -from test_function_outputs import context_stub, function_step_plan, record_output_path +from tests.unit.test_function_outputs import context_stub, function_step_plan, record_output_path from openhcs.constants.constants import Backend from openhcs.core.aligned_image_payload import AlignedImageSliceContext, ImagePayloadBundleContext from openhcs.core.artifacts import ArtifactOutputPlan, ArtifactSpec, ImageArtifactType diff --git a/tests/unit/test_build_website.py b/tests/unit/test_build_website.py index 526f7f91f5..3a7563a35c 100644 --- a/tests/unit/test_build_website.py +++ b/tests/unit/test_build_website.py @@ -949,8 +949,9 @@ def test_website_source_and_workflow_follow_package_metadata_authorities(): assert RELEASE_VERSION_TOKEN in source_html assert "0.5.21" not in source_html assert "0.5.22" not in source_html - assert workflow.count(' - "openhcs/__init__.py"') == 2 - assert workflow.count(' - "openhcs/resources/assets/openhcs-*.svg"') == 2 + # The site imports openhcs (version, assets, session operations), so any + # change under the package re-validates it. + assert workflow.count(' - "openhcs/**"') == 2 for page_name in ("privacy.html", "support.html", "terms.html"): page_source = (REPO_ROOT / "website" / page_name).read_text(encoding="utf-8") assert CONTACT_EMAIL_TOKEN in page_source diff --git a/tests/unit/test_ci_import_guards.py b/tests/unit/test_ci_import_guards.py new file mode 100644 index 0000000000..ccc9036f75 --- /dev/null +++ b/tests/unit/test_ci_import_guards.py @@ -0,0 +1,38 @@ +"""CI runs pytest with ``--import-mode=importlib`` against the installed wheel +(``scripts/run_installed_tests.py``). In that mode a test directory is not on +``sys.path``, so a test module can reach shared helpers only through the +``tests`` package, never by a sibling module's bare name.""" + +from __future__ import annotations + +import ast +from pathlib import Path + +TESTS_ROOT = Path(__file__).resolve().parents[1] + + +def _test_tree_module_names() -> frozenset[str]: + return frozenset(path.stem for path in TESTS_ROOT.rglob("*.py")) - {"__init__"} + + +def test_test_modules_import_helpers_through_the_tests_package() -> None: + local_names = _test_tree_module_names() + offenders: list[str] = [] + for path in sorted(TESTS_ROOT.rglob("*.py")): + tree = ast.parse(path.read_text(encoding="utf-8"), filename=str(path)) + for node in ast.walk(tree): + if isinstance(node, ast.ImportFrom) and node.level == 0 and node.module: + names = (node.module,) + elif isinstance(node, ast.Import): + names = tuple(alias.name for alias in node.names) + else: + continue + for name in names: + if name.split(".")[0] in local_names: + offenders.append( + f"{path.relative_to(TESTS_ROOT.parent)}:{node.lineno}: {name}" + ) + assert not offenders, ( + "Import test helpers as tests..; bare sibling imports " + "fail under --import-mode=importlib:\n" + "\n".join(offenders) + ) diff --git a/tests/unit/test_ci_package_boundaries.py b/tests/unit/test_ci_package_boundaries.py index e25b647bf8..6aca9c9726 100644 --- a/tests/unit/test_ci_package_boundaries.py +++ b/tests/unit/test_ci_package_boundaries.py @@ -476,14 +476,13 @@ def test_website_source_validation_uses_local_candidate_owners() -> None: install_step = next( step for step in validate_job["steps"] - if step.get("name") == "Install website test dependencies" + if step.get("name") == "Install the OpenHCS candidate the website imports" ) install_script = install_step["run"] assert checkout_step["with"]["submodules"] == "recursive" - assert "discover_local_projects" in install_script - assert 'python -m pip install "${local_project_paths[@]}"' in install_script - assert 'python -m pip install "${declared_requirements[@]}"' not in install_script + assert "python -m scripts.install_ci_candidate" in install_script + assert "--dependency-source submodules" in install_script def test_publish_workflow_consumes_release_requirements_authority() -> None: diff --git a/tests/unit/test_derived_role_publication_435.py b/tests/unit/test_derived_role_publication_435.py index 25ccb1f56e..4acdffdfef 100644 --- a/tests/unit/test_derived_role_publication_435.py +++ b/tests/unit/test_derived_role_publication_435.py @@ -10,7 +10,7 @@ from polystore.filemanager import FileManager from polystore.memory import MemoryStorageBackend -from test_function_outputs import ( +from tests.unit.test_function_outputs import ( context_stub, function_step_plan, record_output_path, ) from openhcs.core.dataset_sources.source_schema import SourceSchemaFilenameParser diff --git a/tests/unit/test_gray_to_color_binding_axis_432.py b/tests/unit/test_gray_to_color_binding_axis_432.py index ce48aaac8f..7e940241bd 100644 --- a/tests/unit/test_gray_to_color_binding_axis_432.py +++ b/tests/unit/test_gray_to_color_binding_axis_432.py @@ -30,7 +30,7 @@ CellProfilerOutputRecorder, ) from openhcs.processing.backends.cellprofiler.color import GrayToColorModule, gray_to_color -from test_cellprofiler_generic_special_input_binding import _compile_public_step +from tests.unit.test_cellprofiler_generic_special_input_binding import _compile_public_step from tests.unit.cellprofiler_runtime_test_support import cellprofiler_runtime_adapter_for_test from openhcs.domains.microscopy.axes import Microscopy from openhcs.core.axes import ColourAxis diff --git a/tests/unit/test_gray_to_color_payload_axis_compile_432.py b/tests/unit/test_gray_to_color_payload_axis_compile_432.py index b2a214eddc..c4d80351e3 100644 --- a/tests/unit/test_gray_to_color_payload_axis_compile_432.py +++ b/tests/unit/test_gray_to_color_payload_axis_compile_432.py @@ -25,8 +25,8 @@ ColorToGrayMode, ImageChannelType, color_to_gray, gray_to_color, split_color_to_gray, ) -from test_gray_to_color_binding_axis_432 import _execute_bound_stack, _source_plane -from test_payload_axis_compile_gate import _compiled_pattern, _session +from tests.unit.test_gray_to_color_binding_axis_432 import _execute_bound_stack, _source_plane +from tests.unit.test_payload_axis_compile_gate import _compiled_pattern, _session from openhcs.core.payload_axes import ColourSampleAxisSpec from openhcs.core.axes import ColourAxis diff --git a/tests/unit/test_named_checkpoint_publication.py b/tests/unit/test_named_checkpoint_publication.py index b8d29ba9a4..5561e66be1 100644 --- a/tests/unit/test_named_checkpoint_publication.py +++ b/tests/unit/test_named_checkpoint_publication.py @@ -11,7 +11,7 @@ import pytest import tifffile -from test_artifact_publication_journey import _plate, progress_events +from tests.unit.test_artifact_publication_journey import _plate, progress_events from openhcs.core.artifacts import ( QaCheckpoint, ImageArtifactType, MainFlowPlaneProjectionOutputSpec, MainFlowStackOutputSpec, diff --git a/tests/unit/test_rescale_stream_plane_declaration.py b/tests/unit/test_rescale_stream_plane_declaration.py index 5ac112ad41..9fc43bdbb5 100644 --- a/tests/unit/test_rescale_stream_plane_declaration.py +++ b/tests/unit/test_rescale_stream_plane_declaration.py @@ -39,7 +39,7 @@ ) # Reuse the original provider-free compiler and runtime fixtures, not a second ABI. -from test_cellprofiler_generic_special_input_binding import _compile_public_step +from tests.unit.test_cellprofiler_generic_special_input_binding import _compile_public_step from tests.unit.cellprofiler_runtime_test_support import ( cellprofiler_runtime_adapter_for_test, ) diff --git a/tests/unit/test_selected_plane_stream_materialization.py b/tests/unit/test_selected_plane_stream_materialization.py index f461b7cdf6..081bfd6b17 100644 --- a/tests/unit/test_selected_plane_stream_materialization.py +++ b/tests/unit/test_selected_plane_stream_materialization.py @@ -55,12 +55,12 @@ ) from openhcs.processing.materialization import materialize -from test_function_artifact_materialization import ( +from tests.unit.test_function_artifact_materialization import ( _context, _plan, streaming_config_stub, ) -from test_materialization_core import _viewer_stream_backend_kwargs +from tests.unit.test_materialization_core import _viewer_stream_backend_kwargs from openhcs.domains.microscopy.axes import Microscopy diff --git a/tests/unit/test_snapshot_deadline.py b/tests/unit/test_snapshot_deadline.py index b0dbb1a6da..09460a9580 100644 --- a/tests/unit/test_snapshot_deadline.py +++ b/tests/unit/test_snapshot_deadline.py @@ -8,7 +8,7 @@ import pytest -from test_napari_render_snapshot import queued_viewer +from tests.unit.test_napari_render_snapshot import queued_viewer from openhcs.agent.dto.execution import ExecutionConnectionSpec from openhcs.agent.dto.viewer import ViewerWindowSnapshotRequest from openhcs.agent.services import viewer_window_service as service_module diff --git a/tests/unit/test_viewer_spatial_sample.py b/tests/unit/test_viewer_spatial_sample.py index 12bce6de08..903e7d7b6c 100644 --- a/tests/unit/test_viewer_spatial_sample.py +++ b/tests/unit/test_viewer_spatial_sample.py @@ -5,7 +5,7 @@ import numpy as np import pytest -from test_viewer_feature_measurement import native_route +from tests.unit.test_viewer_feature_measurement import native_route from openhcs.agent.dto.execution import ExecutionConnectionSpec from openhcs.agent.dto.viewer import ViewerWindowImageSampleRequest From 62752cf3373f12947564acd9f9033c903b6d6775 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 03:16:37 -0400 Subject: [PATCH 03/84] B6: process reaping works on Windows; GUI smoke imports session operations lazily reap_spawned_processes called os.getpgid unconditionally, an AttributeError at teardown of every reaping test on Windows. smoke_installed_gui.py gained a module-level openhcs.authoring import, so the dependency-light installer contract tests (pytest, packaging, build only) could not import it. Co-Authored-By: Claude Opus 5.5 --- scripts/smoke_installed_gui.py | 2 +- tests/conftest.py | 39 +++++++++++++++++++--------------- 2 files changed, 23 insertions(+), 18 deletions(-) diff --git a/scripts/smoke_installed_gui.py b/scripts/smoke_installed_gui.py index dcc5c5ba82..e2010984c5 100644 --- a/scripts/smoke_installed_gui.py +++ b/scripts/smoke_installed_gui.py @@ -1,7 +1,6 @@ """Operate an installed OpenHCS GUI through MCP outside the source checkout.""" from __future__ import annotations -from openhcs.authoring.session.operations.datasets import ShowDatasetCode import argparse import faulthandler @@ -565,6 +564,7 @@ def run_installed_live_mcp_smoke( MainWindowWidgetIdentity, PlateManagerWidgetIdentity, ) + from openhcs.authoring.session.operations.datasets import ShowDatasetCode from openhcs.mcp.dev_client import McpDevClient from openhcs.mcp.dev_client_commanding import CapabilityBackedCommandSpec from openhcs.mcp.dev_client_commands.ui import ( diff --git a/tests/conftest.py b/tests/conftest.py index 068fef6fc7..91a08b3619 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -151,25 +151,30 @@ def reap_spawned_processes(monkeypatch) -> Iterator[None]: marker = uuid.uuid4().hex monkeypatch.setenv(SPAWN_MARKER_VARIABLE, marker) yield - own_group = os.getpgid(0) survivors = _marked_processes(marker) for process in survivors: - try: - group = os.getpgid(process.pid) - if group != own_group: - os.killpg(group, signal.SIGTERM) - else: - process.terminate() - except (ProcessLookupError, psutil.NoSuchProcess): - continue + _stop_marked_process(process, force=False) _gone, alive = psutil.wait_procs(survivors, timeout=10) for process in alive: - try: - group = os.getpgid(process.pid) - if group != own_group: - os.killpg(group, signal.SIGKILL) - else: - process.kill() - except (ProcessLookupError, psutil.NoSuchProcess): - continue + _stop_marked_process(process, force=True) psutil.wait_procs(alive, timeout=10) + + +def _stop_marked_process(process: psutil.Process, *, force: bool) -> None: + """Signal a detached POSIX session's whole group, else the process itself. + + Windows has no process groups; every descendant carries the marker, so + stopping each marked process covers the tree there. + """ + try: + if os.name == "posix": + group = os.getpgid(process.pid) + if group != os.getpgid(0): + os.killpg(group, signal.SIGKILL if force else signal.SIGTERM) + return + if force: + process.kill() + else: + process.terminate() + except (ProcessLookupError, psutil.NoSuchProcess): + return From c9de5fab0d79a960cae8eb354fd374e48d922931 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 03:22:01 -0400 Subject: [PATCH 04/84] B6: docs path check matches whole repository paths REPOSITORY_SOURCE_PATH had no left boundary, so packaging/codex/openhcs/skills/... matched as openhcs/skills/... and the MCP knowledge-base manifest reported ten existing references as missing. The pattern now starts at a path boundary and covers packaging/. Co-Authored-By: Claude Opus 5.5 --- scripts/validate_docs.py | 3 ++- tests/unit/test_validate_documentation_audit.py | 9 +++++++++ 2 files changed, 11 insertions(+), 1 deletion(-) diff --git a/scripts/validate_docs.py b/scripts/validate_docs.py index 8205bb185d..1c36d3d499 100644 --- a/scripts/validate_docs.py +++ b/scripts/validate_docs.py @@ -65,7 +65,8 @@ MARKDOWN_FENCE = re.compile(r"^\s*```(?:python|py)\s*$", re.IGNORECASE) LITERAL_INCLUDE = re.compile(r"^\s*\.\.\s+literalinclude::\s+(?P\S+)\s*$") REPOSITORY_SOURCE_PATH = re.compile( - r"(?P(?:benchmark|docs|external|openhcs|scripts)/[A-Za-z0-9_.\-/]+" + r"(?(?:benchmark|docs|external|openhcs|packaging|scripts)/[A-Za-z0-9_.\-/]+" r"\.(?:jsonl|json|md|py|rst|toml|yaml|yml))(?![A-Za-z0-9_])" ) EXTERNAL_URL = re.compile(r"https?://\S+") diff --git a/tests/unit/test_validate_documentation_audit.py b/tests/unit/test_validate_documentation_audit.py index 8b3dc9f474..66cf6054c0 100644 --- a/tests/unit/test_validate_documentation_audit.py +++ b/tests/unit/test_validate_documentation_audit.py @@ -26,6 +26,15 @@ def test_repository_paths_retain_the_full_jsonl_suffix(): ) +def test_repository_paths_match_whole_paths_only(): + target = "packaging/codex/openhcs/skills/use-openhcs/references/viewer-qa.md" + matches = [ + match.group("target") + for match in REPOSITORY_SOURCE_PATH.finditer(f'"source_path": "{target}"') + ] + assert matches == [target] + + def _entry( path: str, source: bytes = b"Index\n=====\n", From 5544c445fcb3610fc1d2caf1b960d625771543a7 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 03:25:09 -0400 Subject: [PATCH 05/84] B6: website job installs the dev extra its pytest configuration needs pytest.ini's addopts carry --max-worker-restart (pytest-xdist), so pytest without the dev extra rejects its own configuration. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/website-pages.yml | 7 ++++--- tests/conftest.py | 1 + tests/unit/test_ci_package_boundaries.py | 5 ++++- 3 files changed, 9 insertions(+), 4 deletions(-) diff --git a/.github/workflows/website-pages.yml b/.github/workflows/website-pages.yml index c7163890a4..e05d73bcbd 100644 --- a/.github/workflows/website-pages.yml +++ b/.github/workflows/website-pages.yml @@ -60,12 +60,13 @@ jobs: with: python-version: "3.12" - - name: Install the OpenHCS candidate the website imports + - name: Install the OpenHCS candidate and the repository's test extra run: | - python -m pip install --upgrade pip build pytest + python -m pip install --upgrade pip build python -m scripts.install_ci_candidate \ --dependency-source submodules \ - --wheel-directory "$RUNNER_TEMP/openhcs-candidate-wheels" + --wheel-directory "$RUNNER_TEMP/openhcs-candidate-wheels" \ + --extras dev - name: Test, build, and validate landing page run: | diff --git a/tests/conftest.py b/tests/conftest.py index 91a08b3619..7bdc8c468b 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -26,6 +26,7 @@ default_download_policy=ImageJArchiveDownloadPolicy(allow_download=False) ) + def _missing_native_extensions() -> tuple[str, ...]: """Native modules declared by their sources (``.cpp``) but not built. diff --git a/tests/unit/test_ci_package_boundaries.py b/tests/unit/test_ci_package_boundaries.py index 6aca9c9726..ee9db39b8b 100644 --- a/tests/unit/test_ci_package_boundaries.py +++ b/tests/unit/test_ci_package_boundaries.py @@ -476,13 +476,16 @@ def test_website_source_validation_uses_local_candidate_owners() -> None: install_step = next( step for step in validate_job["steps"] - if step.get("name") == "Install the OpenHCS candidate the website imports" + if step.get("name") + == "Install the OpenHCS candidate and the repository's test extra" ) install_script = install_step["run"] assert checkout_step["with"]["submodules"] == "recursive" assert "python -m scripts.install_ci_candidate" in install_script assert "--dependency-source submodules" in install_script + # pytest.ini's addopts need the dev extra's plugins (pytest-xdist). + assert "--extras dev" in install_script def test_publish_workflow_consumes_release_requirements_authority() -> None: From 6669b34013e9e72a2116b4dd0425ec500ee49f6a Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 03:25:35 -0400 Subject: [PATCH 06/84] B6: pin PolyStore and pyqt-reactive docs without orphaned validation logs Lockstep with OpenHCSDev/PolyStore#38 and OpenHCSDev/PyQT-reactive#21: the Documentation workflow's sphinx -W builds of both failed on run records in no toctree. Co-Authored-By: Claude Opus 5.5 --- external/PolyStore | 2 +- external/pyqt-reactive | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/external/PolyStore b/external/PolyStore index 7ac9d01110..43cfd8714f 160000 --- a/external/PolyStore +++ b/external/PolyStore @@ -1 +1 @@ -Subproject commit 7ac9d011100b18f4ad1a54635ee94f52a47efb30 +Subproject commit 43cfd8714fff0369114982feb88960dcb7d7b682 diff --git a/external/pyqt-reactive b/external/pyqt-reactive index 269e896ba8..fe2a8cac69 160000 --- a/external/pyqt-reactive +++ b/external/pyqt-reactive @@ -1 +1 @@ -Subproject commit 269e896ba8ec3050ae305e4ac9c46a1c01c2cb9b +Subproject commit fe2a8cac69c2834ba07202c0c28273e0683043c6 From f37e60a28ebac7764456c5ad04e26f5891601962 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 03:26:04 -0400 Subject: [PATCH 07/84] B6: require pycodify 0.1.4, the release of the pinned source The pinned pycodify differs from v0.1.3 in src/pycodify/formatters.py, so pypi-dependency-readiness failed. Lockstep with OpenHCSDev/pycodify#3; the job stays red until the owner tags v0.1.4 and PyPI serves it. Co-Authored-By: Claude Opus 5.5 --- external/pycodify | 2 +- pyproject.toml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/external/pycodify b/external/pycodify index 108a8edbf2..0c6a6fbb61 160000 --- a/external/pycodify +++ b/external/pycodify @@ -1 +1 @@ -Subproject commit 108a8edbf25de258168756bac22c7572f018fa90 +Subproject commit 0c6a6fbb6111ab890d3da3fa01ce43c5aa020965 diff --git a/pyproject.toml b/pyproject.toml index 3139640895..b71c07fc48 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -100,7 +100,7 @@ dependencies = [ # External dependencies (PyPI versions for production builds) "zmqruntime>=0.6,<0.7", - "pycodify>=0.1.3,<0.2", + "pycodify>=0.1.4,<0.2", "objectstate>=1.3.1,<2", "python-introspect>=0.2.5,<0.3", "metaclass-registry>=0.3.0,<0.4", From ca7f0867593c3fc7d93c8e4ce4fe41aca3c5a423 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 03:31:07 -0400 Subject: [PATCH 08/84] B6: test configuration reads CPU-only mode from its owner conftest compared OPENHCS_CPU_ONLY to 'true' itself, while OpenHCSProcessEnvironment.cpu_only_mode also accepts 1/yes/on. Co-Authored-By: Claude Opus 5.5 --- tests/conftest.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tests/conftest.py b/tests/conftest.py index 7bdc8c468b..ea7aecaf00 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -8,6 +8,7 @@ from pathlib import Path from openhcs._source_dependencies import ensure_source_checkout_external_paths +from openhcs.utils.environment import OpenHCSProcessEnvironment ensure_source_checkout_external_paths() @@ -64,7 +65,7 @@ def pytest_sessionstart(session): # Conditionally import pytest-qt only when not in CPU-only mode -CPU_ONLY_MODE = os.getenv("OPENHCS_CPU_ONLY", "false").lower() == "true" +CPU_ONLY_MODE = OpenHCSProcessEnvironment.cpu_only_mode() if not CPU_ONLY_MODE: pytest_plugins = ["pytestqt"] else: From e59901e7626868e92d943f95b64b8fd4554f9406 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 03:35:26 -0400 Subject: [PATCH 09/84] B6: re-review 141 stale documentation pages against the refactored code Every page whose editorial audit went stale (a page or authority digest changed since review: 828 changed authorities, 78 deleted ones, 34 edited pages) was re-read against the diff of each changed authority since its reviewed version. Pages naming deleted mechanisms now name their replacements (StepSnapshot -> ResolvedPipelineDefinition, MicroscopeHandler -> DatasetSource, PURE_2D -> Pure2DContract, compiled_plate_execution -> bundle_execution, RuntimeSliceProjectionStrategy -> RuntimeSliceProjectableValue, Plate Manager -> Datasets where reviewed, and others); deleted authorities point at their current owners; findings and validation record the review; digests were pinned after the comparison (docs/audits/README.md step 4). Co-Authored-By: Claude Opus 5.5 --- README.md | 35 +- docs/audits/architecture_a_m.json | 1284 +++++---- docs/audits/architecture_n_z_development.json | 1879 +++++++------ docs/audits/project_surface.json | 86 +- docs/audits/user_docs.json | 2395 +++++++++-------- docs/source/api/index.rst | 39 +- docs/source/appendices/glossary.rst | 26 +- .../architecture/abstraction_lattices.rst | 29 +- .../analysis_consolidation_system.rst | 27 +- .../architecture/artifact_contract_system.rst | 19 +- .../architecture/cellprofiler_interop.rst | 5 +- .../compilation_system_detailed.rst | 4 +- .../component_system_integration.rst | 7 +- .../component_validation_system.rst | 24 +- .../source/architecture/concurrency_model.rst | 33 +- .../declarative_window_system.rst | 4 +- .../experimental_analysis_system.rst | 8 +- .../architecture/external_foundations.rst | 19 +- .../external_integrations_overview.rst | 8 +- .../architecture/gpu_resource_management.rst | 4 +- .../architecture/list_item_preview_system.rst | 2 +- docs/source/architecture/mcp_distribution.rst | 70 +- .../measurement_equivalence_system.rst | 39 +- .../microscope_handler_integration.rst | 91 +- .../multiprocessing_coordination_system.rst | 31 +- .../source/architecture/nominal_ownership.rst | 7 +- .../orchestrator_cleanup_guarantees.rst | 2 +- .../orchestrator_configuration_management.rst | 13 +- .../pipeline_compilation_system.rst | 49 +- .../architecture/plate_manager_services.rst | 2 +- .../architecture/processing_semantics.rst | 34 +- .../progress_runtime_projection_system.rst | 32 +- docs/source/architecture/quick_start.rst | 16 +- docs/source/architecture/roi_system.rst | 8 +- .../architecture/runtime_value_system.rst | 50 +- docs/source/architecture/source_model.rst | 10 +- .../source/architecture/special_io_system.rst | 5 +- .../streaming_boundary_and_wrappers.rst | 13 +- docs/source/architecture/system_overview.rst | 25 +- .../architecture/ui_services_architecture.rst | 28 +- .../zmq_execution_service_extracted.rst | 50 +- .../zmq_server_browser_system.rst | 14 +- docs/source/concepts/building_intuition.rst | 2 +- .../concepts/component_system_concepts.rst | 2 +- docs/source/concepts/core_model.rst | 6 +- docs/source/concepts/data_dimensions.rst | 22 +- docs/source/concepts/domain_fundamentals.rst | 8 +- docs/source/concepts/function_library.rst | 6 +- docs/source/concepts/function_patterns.rst | 2 +- docs/source/concepts/module_structure.rst | 20 +- docs/source/concepts/pipelines_and_steps.rst | 11 +- docs/source/concepts/storage_system.rst | 3 +- .../development/ast_refactoring_workflow.rst | 2 +- .../callable_artifact_authoring.rst | 16 +- .../source/development/compiler_extension.rst | 10 +- .../development/extension_workflows.rst | 12 +- .../development/git_worktree_testing.rst | 6 +- .../mcp_knowledge_base_manifest.json | 2 +- docs/source/development/mcp_release.rst | 15 +- .../respecting_codebase_architecture.rst | 2 +- .../runtime_system_assembly_rules.rst | 9 +- .../development/runtime_value_extension.rst | 15 +- .../development/source_binding_extension.rst | 11 +- .../getting_started/getting_started.rst | 6 - .../guide_for_biologists/basic_interface.rst | 24 +- .../configuration_reference.rst | 10 +- .../domain_expert_onboarding.rst | 4 +- .../guide_for_biologists/image_sources.rst | 66 +- docs/source/guide_for_biologists/intro.rst | 4 +- .../guide_for_biologists/intro_stitching.rst | 6 +- .../troubleshooting_FAQ.rst | 8 +- docs/source/guides/complete_examples.rst | 10 +- docs/source/guides/example_corpus_map.rst | 2 +- docs/source/guides/fiji_viewer_management.rst | 24 +- .../source/guides/memory_type_integration.rst | 5 +- docs/source/guides/omero_integration.rst | 2 +- .../guides/pipeline_compilation_workflow.rst | 2 +- docs/source/guides/testing_guide.rst | 13 + docs/source/guides/viewer_management.rst | 10 +- docs/source/installation.rst | 2 +- .../reference/experimental_config_syntax.rst | 2 +- .../user_guide/analysis_consolidation.rst | 5 +- docs/source/user_guide/code_ui_editing.rst | 10 +- .../user_guide/custom_function_management.rst | 4 +- docs/source/user_guide/custom_functions.rst | 9 +- docs/source/user_guide/mcp_clients.rst | 5 +- .../user_guide/real_time_visualization.rst | 4 +- 87 files changed, 3796 insertions(+), 3149 deletions(-) diff --git a/README.md b/README.md index dcb09fd10f..39f759dec9 100644 --- a/README.md +++ b/README.md @@ -98,7 +98,7 @@ graph LR ### 🛡️ Compile-Time Validation -Configuration is resolved once into step snapshots and a compilation session. Typed plans then validate sources, artifacts, materialization, memory contracts, and worker requirements before execution begins. Errors surface immediately, not after hours of processing. +Configuration is resolved once into a resolved pipeline definition and per-context compilation sessions. Typed plans then validate sources, artifacts, materialization, memory contracts, and worker requirements before execution begins. Errors surface immediately, not after hours of processing. @@ -199,7 +199,7 @@ graph TD | [**PyQT-reactive**](https://github.com/OpenHCSDev/PyQT-reactive) | UI form generation | React-style reactive forms from dataclasses with cross-window sync and flash animations | | [**pycodify**](https://github.com/OpenHCSDev/pycodify) | Code ↔ object conversion | Python source as serialization format — type-preserving, diffable, editable, with collision handling | | [**python-introspect**](https://github.com/OpenHCSDev/python-introspect) | Signature analysis | Pure-Python function/dataclass introspection for automatic UI generation and contract analysis | -| [**metaclass-registry**](https://github.com/OpenHCSDev/metaclass-registry) | Plugin discovery | Zero-boilerplate registry system powering microscope handler and storage backend auto-discovery | +| [**metaclass-registry**](https://github.com/OpenHCSDev/metaclass-registry) | Plugin discovery | Zero-boilerplate registry system powering dataset source and storage backend auto-discovery | --- @@ -219,7 +219,7 @@ graph TD | OMERO | Remote image and metadata access | | OpenHCS format | Native generated and materialized plates | -Source handlers are auto-detected and extensible through `metaclass-registry`. +Dataset sources are auto-detected and extensible through `metaclass-registry`. @@ -309,24 +309,27 @@ python -m pip install "openhcs[gui,mcp,viz]" ```python # Or lower a CellProfiler pipeline into public OpenHCS declarations +import queue from pathlib import Path from objectstate import ensure_global_config_context from openhcs.core.config import GlobalPipelineConfig from openhcs.core.orchestrator.orchestrator import PipelineOrchestrator +from openhcs.core.progress import set_progress_queue from openhcs.interop.cellprofiler.pipeline_import import import_cellprofiler_pipeline -plate_path = Path("/data/plate").resolve() +dataset_root = Path("/data/plate").resolve() ensure_global_config_context(GlobalPipelineConfig, GlobalPipelineConfig()) steps, pipeline_config = import_cellprofiler_pipeline( "analysis.cppipe", - source_root=plate_path, + source_root=dataset_root, ) orchestrator = PipelineOrchestrator( - plate_path, + dataset_root, pipeline_config=pipeline_config, ).initialize() +set_progress_queue(queue.Queue()) # compilation emits progress events execution_bundle = orchestrator.compile_pipelines(steps) ``` @@ -360,16 +363,18 @@ digests. Archived receipts without the latter two digests remain readable but cannot verify those files. These routes do not create a second pipeline runner or job-status system. `openhcs-benchmark run-measured --plate PLATE --pipeline-source-file PIPELINE.py ---output-dir EVIDENCE --run-id ID --wait-timeout-ms 120000` submits that -pipeline through ordinary headless job control, then calls the same receipt -finalizer as MCP. Use `--execution-plate PREPARED_PLATE` when the input workspace -has been prepared separately; the evidence directory must be empty. +--output-dir EVIDENCE --run-id ID --wait-timeout-ms 120000` runs that +pipeline through the OpenHCS session, then calls the same receipt finalizer as +MCP. The source plate is never written: the run mirrors it into an execution +workspace under `EVIDENCE/workspace` (or `--workspace-root`), or uses +`--execution-plate PREPARED_PLATE` when the input workspace has been prepared +separately; the evidence directory must be empty. Use `--observation-scope outcomes` to retain per-axis success and provenance without transferring runtime arrays; this does not compare output values. -After an ordinary headless job completes, the expert -`openhcs_finalize_measured_pipeline_run` tool can validate its runtime -observation and retain that receipt from the exact submitted pipeline and -server completion record. It does not submit another job. +After the OpenHCS session finishes an ordinary run that exported a runtime +observation, the expert `openhcs_finalize_measured_pipeline_run` tool can +validate that observation and retain the receipt; the submission and server +record come from the session. It does not submit another run.
📦 All installation options @@ -432,7 +437,7 @@ Supported on Python 3.11 and 3.12. See [Glencoe Software](https://www.glencoesof ``` PipelineConfig + list[FunctionStep] ↓ resolve once -StepSnapshot + CompilationSession +ResolvedPipelineDefinition + CompilationSession ↓ derive and validate typed CompiledStepPlan objects ↓ package diff --git a/docs/audits/architecture_a_m.json b/docs/audits/architecture_a_m.json index a4e273211a..446ecc6101 100644 --- a/docs/audits/architecture_a_m.json +++ b/docs/audits/architecture_a_m.json @@ -65,7 +65,7 @@ }, { "path": "docs/source/architecture/abstraction_lattices.rst", - "source_sha256": "aab6e8a69b89dc4f866cd83a639a4120a9ebbdfa366bd228fe8cda4202ea51b7", + "source_sha256": "01f3f213b569c46818674c2bf207749f196c91fa561ce43453203b6763036ee6", "audience": [ "OpenHCS maintainers", "extension authors", @@ -75,53 +75,68 @@ "diataxis": "explanation", "authority": [ { - "path": "openhcs/core/registry_strategies.py", - "sha256": "a41186665abbd308f224d0bb41d6618cd9d89b08acaf54ec2819f2024a6e0804", - "role": "Defines the shared enum, nominal-type, and most-derived context selection authorities discussed by the page." + "path": "external/metaclass-registry/src/metaclass_registry/strategies.py", + "sha256": "5c00cd4d80f9f0ceffbc2ab95ad8c6abd44a8a66a974959445ef93df08dd4f0d", + "role": "Defines EnumKeyedStrategyMixin (for_enum_member), NominalTypeKeyedStrategyMixin and MostDerivedContextStrategyMixin." }, { "path": "openhcs/interop/cellprofiler/module_declarations.py", - "sha256": "89dd65abc003c464ba6c489f52d3a4fa47be1d5f8b87d26ad72b7537f188cf8b", - "role": "Defines CellProfiler module leaves as owners of module identity, lowering, headless-import validation, artifact semantics, and their declaration-derived function catalogue." + "sha256": "6bbf41c158496a02439be2ee5aef12d7712fb632419c226a64c9259a55438b95", + "role": "Defines CellProfilerModule leaves and require_module as owners of module identity, lowering, headless-import validation and artifact semantics." }, { "path": "openhcs/interop/cellprofiler/module_artifact_declarations.py", - "sha256": "666ec8c9adfd6a9015adf2b226df671534bb7e547cb0d76f1d16c40ac61e30a6", + "sha256": "f70b52114fb5d5e2c2fc49cd9a2b46986bde48ce02d82f1a34a67f072ca2ab20", "role": "Defines the nominal interactive-module family that rejects headless import." }, { "path": "openhcs/core/callable_contract.py", - "sha256": "6a39b5e171901131ecb7280a78241e8ea72409a5003eabba3db34eb4851991d7", - "role": "Projects declaration-owned callable metadata, including memory contracts and artifact relations, at the nominal boundary." + "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", + "role": "Declares CallableContract, the declaration-owned callable metadata at the nominal boundary." }, { - "path": "openhcs/microscopes/microscope_base.py", - "sha256": "ae19ce4f6e1cd2e60946eceb908722e1ff760d7654a775d8372abbc4ac1fc81f", - "role": "Defines the microscope-handler registry and generic factory selection boundary." + "path": "openhcs/core/artifacts.py", + "sha256": "65eb0ec5c903366243ca8bd5ea07412e8d34bdb81e6015fa76b3c4c59838733f", + "role": "Declares the ArtifactSpecRelation family for cross-artifact semantics." + }, + { + "path": "openhcs/core/dataset_sources/source.py", + "sha256": "097fde4d2b74031eacf53a88e79f499417e8c1c97ba0753e8a9df831b307efa6", + "role": "Defines the DatasetSource registry and its layout detection and workspace projection boundary." + }, + { + "path": "openhcs/core/dataset_sources/source_bindings_source.py", + "sha256": "fccda877f39e7b2f99d6cc95143480e5955186264e316e832ca9a69c2ae90b8c", + "role": "Defines SourceBindingsSource, which composes a dataset source with SourceBindingsConfig declarations." + }, + { + "path": "openhcs/core/runtime_plane_projection.py", + "sha256": "3c88ed042a7f1c65c9495a1c62777e008f1e59c4742ec0563b2849d77ca87cdd", + "role": "Defines RuntimePlaneAxis and the RuntimePlaneAxisStrategy family used as the strategy example." }, { "path": "openhcs/agent/capabilities.py", - "sha256": "7fe9ab91490dedd6f750a57386e91835cf1067a27d10079966510aa9821adaf0", - "role": "Defines capability identity, transport availability, effects, and surface metadata on registered declarations." + "sha256": "eaebe33af01d057d1d21b3701f958bc96ede3ec5d0bddddca9917c1164ef0042", + "role": "Defines AgentCapabilityDeclaration and CapabilityTransport members carrying their transport semantics." }, { "path": "tests/unit/test_cellprofiler_static_deletion_gates.py", - "sha256": "83d81d01ef7d828b2d0fe971a8487aa4cd049bc69e488f8f82aca2e9f02c2cbf", + "sha256": "785ea0bd5da907137672735ca631007ee1273393605f3d5e263ac929fbe36c5a", "role": "Guards deletion of superseded CellProfiler mirrors and compatibility surfaces." }, { "path": "tests/unit/test_cellprofiler_pipeline_import.py", - "sha256": "353f0c1d8e0bc3552d2e418d792794ab0631daa16c78a3e10191b1cc1ea07d73", + "sha256": "6b1d201ec5c4ba620fa2367a9cbef1f6fbdd25edbdda9bd7b723ebe3a3363412", "role": "Verifies that importing an enabled interactive-only module fails explicitly." } ], "findings": [ - "The 972-line page mixed a current mental model with benchmark inventories, migration checklists, WIP prescriptions, and stale proposed collapses; it now explains the current owners and treats commit history only as historical evidence.", - "Transport initialization instructions were selected again inside the MCP server adapter; CapabilityTransport members now carry the registered nominal rendering strategy used by every consumer.", - "Enabled interactive-only CellProfiler modules could otherwise disappear as non-executable declarations; the nominal module family now validates headless import and rejects them explicitly." + "The page named MicroscopeHandler and SourceBindingsHandler; G4 replaced them with the DatasetSource family and SourceBindingsSource.", + "The strategy selection mixins moved from the deleted openhcs/core/registry_strategies.py into metaclass-registry; the page now says where they come from." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/abstraction_lattices.rst in full and compared it against the changed authorities: openhcs/core/registry_strategies.py (deleted; mixins now in metaclass_registry.strategies), openhcs/microscopes/microscope_base.py (deleted; DatasetSource family in openhcs/core/dataset_sources), module_declarations.py, module_artifact_declarations.py, callable_contract.py, openhcs/agent/capabilities.py and the two CellProfiler tests; replaced the microscope-handler owner and SourceBindingsHandler with DatasetSource and SourceBindingsSource, attributed the strategy mixins to metaclass-registry, and confirmed CellProfilerModule.require_module, ArtifactSpecRelation, RuntimePlaneAxisStrategy, for_enum_member and CapabilityTransport against current source. Historical commit references are left as history.", "Re-read docs/source/architecture/abstraction_lattices.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/registry_strategies.py; openhcs/interop/cellprofiler/module_declarations.py; openhcs/core/callable_contract.py; openhcs/microscopes/microscope_base.py; openhcs/agent/capabilities.py; tests/unit/test_cellprofiler_static_deletion_gates.py.", "Focused feature-lattice QA passed 65 tests covering framework resolution, execution scope, callable contracts, compiled execution, validation, and worker policy.", @@ -148,7 +163,7 @@ }, { "path": "docs/source/architecture/analysis_consolidation_system.rst", - "source_sha256": "3f3fa8f3f0ee441d4c2aa42bd79c2da3dc04635f4958b64ce49bcbfc25a618db", + "source_sha256": "d5e535d38edfd520b1fe69191d7547b4f02521631eb6fd71ea3de09bfbea7d83", "audience": [ "OpenHCS runtime maintainers", "analysis-output integrators" @@ -157,47 +172,48 @@ "diataxis": "explanation", "authority": [ { - "path": "openhcs/core/orchestrator/analysis_consolidation.py", - "sha256": "3ef9c20c347c1ac724353c84b24f9d7e13513aba7b9c5c01e194171b7e72bb52", - "role": "Projects execution-ledger CSV content with each result directory's compiled storage context and the global output-plate destination." + "path": "openhcs/domains/microscopy/analysis_consolidation.py", + "sha256": "292f7138a00dc22dba5b8d8af5de6140d247b955723cc807cc982ffec7a7dd2d", + "role": "Defines AnalysisConsolidationHook, its saved/reused output observations, the per-directory destinations and the FileManager summary writer." + }, + { + "path": "openhcs/core/post_execute.py", + "sha256": "42183491eae6a5a17fd68c179970b1682343f1f73f355291f2ba4b02c784905d", + "role": "Defines the PostExecuteHook family: binding from the global config, observation combine, and running after every partition succeeds." }, { "path": "openhcs/processing/backends/analysis/consolidate_analysis_results.py", - "sha256": "aad546fdc069653d809f9ef6867ad2ae351276ea9ab66b377af7cd6ab2428d60", - "role": "Defines nominal analysis sources and summary writers, in-memory merging, rendering, and consolidation policy." + "sha256": "508cdbf635705bd4b133569c7e745b2d163eb77871dd8be2dd2d994a6535c73a", + "role": "Defines AnalysisTableSource, AnalysisSummaryWriter, the inclusion selection, in-memory merging and summary rendering." }, { - "path": "openhcs/core/config.py", - "sha256": "ef81b50ee22b8da69513fd50d6e45a929117fa4bfaa62a20ef08a4c02531c043", - "role": "Declares AnalysisConsolidationConfig and its standalone consolidation policy." + "path": "openhcs/domains/microscopy/config.py", + "sha256": "79b8baefdbf22b85c199807f62b8234daca6904633a574886661a1c2e3d03a9b", + "role": "Declares AnalysisConsolidationConfig and PlateMetadataConfig." }, { - "path": "tests/unit/test_orchestrator_lane_planning.py", - "sha256": "09c39f5cba68f8c15a0def8a09843b62d6da5c5a01e7558a9263ce4d311c078d", - "role": "Exercises orchestrator lane planning and post-execution analysis boundaries used by consolidation." + "path": "external/PolyStore/src/polystore/base.py", + "sha256": "8f2082ade15072bc8e8eba363c75a1d1cff7c67c74845e6485f69eab244634dc", + "role": "Declares DataSink.contextual_save_kwargs used for backend save context." }, { "path": "tests/unit/test_function_artifact_materialization.py", - "sha256": "ba4f8f5f165ab87c3000d6d371a6f25cf4ec6a5d39b5047abaa392c827fe9ae8", - "role": "Proves consolidation consumes runtime CSV values, uses the compiled backend destination, ignores non-table artifacts, and propagates failures." + "sha256": "e92bd682c105778d8bde9cb0f63ff1baed9f4c082a05d2c7f8f04bf539bd699f", + "role": "Proves the hook consumes saved CSV content, uses the compiled backend destination, skips a disabled config, and propagates failures." }, { "path": "tests/unit/test_routed_channel_selection.py", - "sha256": "f0f7e10c99f6568048817f38ee10f648bb0b6fb4945cc0fae0f988761557745f", - "role": "Executes a native plate with separate analysis destinations and verifies per-directory and merged output summaries without writing a summary into the input plate." - }, - { - "path": "openhcs/processing/materialization/core.py", - "sha256": "1f5a89a1eb50766ef093df3fece16d6bb03422b92d062fe8ae6471656b1ebd45", - "role": "Owns writer runtime-path selection before rendering and the shared ROI target request." + "sha256": "3e4df60b48b699f18e0de29920be778bd03dc98f767fca8bb64a42d6536c4f41", + "role": "Executes a native plate with separate analysis destinations and verifies per-directory and merged summaries without writing into the input plate." } ], "findings": [ - "The page described discovery as a results-directory scan, while current orchestration consumes exact typed runtime observation records.", - "Runtime consolidation formerly reopened virtual output addresses through the host filesystem and discarded failures; it now consumes execution-ledger content and writes through the compiled backend." + "The page named consolidate_analysis_outputs in the deleted openhcs/core/orchestrator/analysis_consolidation.py; consolidation is now AnalysisConsolidationHook, a microscopy PostExecuteHook bound from the global config.", + "The page said consolidation selection is passed into runtime materialisation so writers can skip rendering; no caller passes an output path filter now, and the hook instead records the CSV text the writer already saved, filtered by the inclusion selection." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/analysis_consolidation_system.rst in full and compared it against the changed authorities: openhcs/core/orchestrator/analysis_consolidation.py (moved to openhcs/domains/microscopy/analysis_consolidation.py as a post-execute hook), consolidate_analysis_results.py, openhcs/core/config.py (consolidation config moved to openhcs/domains/microscopy/config.py), the three tests and openhcs/processing/materialization/core.py; rewrote the flow paragraph for AnalysisConsolidationHook and PostExecuteHook, removed the obsolete output-selection/skip-rendering paragraph, and confirmed the writer, DataSink save context, output-root destination and failure propagation against current source. Dropped test_orchestrator_lane_planning.py and materialization/core.py, which no longer own a stated claim.", "Re-read docs/source/architecture/analysis_consolidation_system.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/orchestrator/analysis_consolidation.py; openhcs/core/config.py; tests/unit/test_orchestrator_lane_planning.py.", "Focused feature-lattice QA passed 65 tests covering framework resolution, execution scope, callable contracts, compiled execution, validation, and worker policy.", @@ -215,7 +231,7 @@ }, { "path": "docs/source/architecture/artifact_contract_system.rst", - "source_sha256": "eb9380b348a7fccfb4385956bc1997b4dc4bf23aaf1f45abe2fd2fd8c1342d83", + "source_sha256": "270407772ecb71fad671363045561f7f736a535fc0b500410005687a04497b72", "audience": [ "OpenHCS compiler maintainers", "callable and artifact extension authors" @@ -225,35 +241,52 @@ "authority": [ { "path": "openhcs/core/artifacts.py", - "sha256": "bb958c74c98cb96c97c5ac1ea68ac997673aac6afcad57bdf0c9496065e895fa", - "role": "Defines artifact families, specifications, references, relations, and materialisation declarations." + "sha256": "65eb0ec5c903366243ca8bd5ea07412e8d34bdb81e6015fa76b3c4c59838733f", + "role": "Defines the ArtifactType families, ArtifactSpec/ArtifactSpecRef, ArtifactSpecRelation leaves and materialisation declarations." }, { "path": "openhcs/core/callable_contract.py", - "sha256": "6a39b5e171901131ecb7280a78241e8ea72409a5003eabba3db34eb4851991d7", - "role": "Defines callable artifact input and output ABI ownership and relation validation." + "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", + "role": "Defines CallableContract and its ordered artifact input and output collections." + }, + { + "path": "openhcs/core/function_patterns.py", + "sha256": "95b9056c6980e4f7943f909aeae8262252c9aa175247a534befc8d4cc459fdd2", + "role": "Defines InvocationArtifactInputEdgePlan, its main_flow_projection and storage plan, and MainFlowInputProjection." + }, + { + "path": "openhcs/core/pipeline/function_contracts.py", + "sha256": "13bcaf7c7a62654f512d2cc739edccd2deeff35d99970ca493f3e827afbda91a", + "role": "Defines the special_inputs, artifact_inputs and artifact_outputs decorators." }, { "path": "openhcs/core/pipeline/path_planner.py", - "sha256": "0ed8bb8d843ff41382f4de25367a74c9535a39425c6fc328091ad3025d526151", + "sha256": "677d106969aae4d818745bff29b6f84e487ec79a044c8eaf26bea1916236fc6c", "role": "Implements artifact satisfaction and exact compiled path planning." }, + { + "path": "tests/unit/test_implicit_main_flow_compilation.py", + "sha256": "9f8f3c44f096ce0318612df7d51affbb21a7bbc837c9846d599dbe20c0a8998b", + "role": "Compiles grouped native output into a CellProfiler Threshold consumer and asserts the main-flow edge, group scope and execution scope." + }, { "path": "tests/unit/test_callable_contract.py", - "sha256": "493f76a54f5fe024ca8f10c53b1f810b8c369626ad7141205b42ef3a78282137", + "sha256": "cc220f53107a08b0406b6e0dd49896e3e4c3c0af6e527b0ef00d4fdaba8f0b2c", "role": "Verifies callable contract construction, validation, and artifact semantics." }, { "path": "tests/unit/test_artifact_input_projection_plan.py", - "sha256": "a777c8fd900f15ca5f6a990fb48b36e1497a3555e5575e68487d7cb7b74cdd65", + "sha256": "45e9b6d379c6b2f95bfe9bc5821de66f157d4f828b00ddb74ea3c65a7d64bbd7", "role": "Verifies occurrence-specific artifact input projection plans." } ], "findings": [ - "The artifact-family inventory omitted spatial graphs, and an embedded extension procedure crossed the explanation/how-to boundary." + "The page described a consumes_main_flow flag on invocation input edges; the field is gone and an edge consumes main flow exactly when it carries a main_flow_projection.", + "The regression description said the test asserts consumes_main_flow=True; it now asserts a main-flow projection on the edge." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/artifact_contract_system.rst in full and compared it against the changed authorities: openhcs/core/artifacts.py, openhcs/core/callable_contract.py, openhcs/core/pipeline/path_planner.py, tests/unit/test_callable_contract.py, tests/unit/test_artifact_input_projection_plan.py; replaced the removed consumes_main_flow flag with main_flow_projection in the occurrence and implicit main-flow sections and the regression description, tidied the special_inputs sentence, and confirmed every other named declaration (ArtifactType families, relation leaves, SourceImageProvenance, ArtifactPlanKeySelector, SelectedPlaneImageOutput, RuntimeValueStore, StepMaterializationConfig) exists with the stated role.", "Re-read docs/source/architecture/artifact_contract_system.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/artifacts.py; openhcs/core/callable_contract.py; openhcs/core/pipeline/path_planner.py; tests/unit/test_callable_contract.py; tests/unit/test_artifact_input_projection_plan.py.", "Focused feature-lattice QA passed 65 tests covering framework resolution, execution scope, callable contracts, compiled execution, validation, and worker policy.", @@ -270,87 +303,60 @@ }, { "path": "docs/source/architecture/batch_workflow_service.rst", - "source_sha256": "5b35c78968bea4782ff67974f90428b37d7f00d79ef00509155f2b6d79917798", + "source_sha256": "5fe054e9e50b3615028bd270372778e384a15a71028c196f5ddddd313942fe4e", "audience": [ - "OpenHCS desktop and execution-service maintainers" + "OpenHCS session and execution-service maintainers" ], - "user_need": "Understand how Plate Manager composes compile, run, progress, status, and cleanup services.", + "user_need": "Understand how the session compiles and runs datasets for every client.", "diataxis": "explanation", "authority": [ { - "path": "openhcs/pyqt_gui/services/plate_manager_batch_workflow.py", - "sha256": "974aebec62dc0f283f92d8153594771d1206f02decdfd8ea5faae0a563676699", - "role": "Implements the Plate Manager batch facade and compile-before-execute lifecycle." - }, - { - "path": "openhcs/pyqt_gui/widgets/shared/services/compile_batch_workflow_service.py", - "sha256": "1b37faeb90ecbc326727bef124ba990a067dffca430955772f8f002356a451f8", - "role": "Implements compile-only batch submission and result collection." + "path": "openhcs/authoring/session/operations/datasets.py", + "sha256": "208274effc2e790775d90528211fa4f640eb01f7ca3fdd9c927c3bf71e39b632", + "role": "Declares CompileDatasets and its initialised, idle and non-empty-pipeline availability checks." }, { - "path": "openhcs/pyqt_gui/widgets/shared/services/batch_workflow_components.py", - "sha256": "abd29f375ab12fdc56058f1c860ce26c3b52129658795eb1c95581d83b7480bd", - "role": "Declares the lazily composed focused workflow services." + "path": "openhcs/authoring/session/compilation.py", + "sha256": "219db74f43cc83c5c07f5bb4b2a461e4979d22dca48f00b4ea041852f35b2ac5", + "role": "Declares DatasetPipelineRequest snapshots and the CompiledDataset result." }, { - "path": "tests/unit/pyqt_gui/test_batch_workflow_compile_engine.py", - "sha256": "a8295fb6edbd0229128025327c975d7c916b791f5774df4a030e6c9778f3c4cb", - "role": "Verifies batch compilation, artifact retention, status, and failure convergence." - }, - { - "path": "openhcs/pyqt_gui/main.py", - "sha256": "a156a006a3c40f43107b9b1f6ecb97ed150879eb88ded8ef89e25214309ae33a", - "role": "Coordinates post-paint endpoint preparation, projects the browser-owned endpoint snapshot into both status-bar surfaces, and routes endpoint termination to the client lifecycle owner." + "path": "openhcs/authoring/session/compile_batch.py", + "sha256": "0cbe5a6b86a0fb5e9b92c4fc45f858c979031ac93594b317b9761c28f5aaf9ab", + "role": "Runs compile-only batches and compile-before-execution through BatchSubmitWaitEngine." }, { - "path": "external/pyqt-reactive/src/pyqt_reactive/services/zmq_server_scan_service.py", - "sha256": "ae12092dfd8b85a3354b87a708f3c32aef787c2f9fd0402158d32d14b654c7de", - "role": "Owns the immutable endpoint observation snapshot, derives per-port status without stored connection flags, and projects startup presentation through the lifecycle declaration." + "path": "external/zmqruntime/src/zmqruntime/execution/batch_submit_wait.py", + "sha256": "c447b2ccbc372a5336f5a3b32844b896dc5d58566cb3de31640ae3be2379edee", + "role": "Defines BatchSubmitWaitEngine, the submit-then-wait batch driver." }, { - "path": "tests/unit/pyqt_gui/test_main_zmq_window_routing.py", - "sha256": "d02fe132fda83540f26ed4b7a79fc1409fabbf0ff6ce72f3d1ed8681258e9f6a", - "role": "Proves startup callbacks only commit observations, one snapshot drives status text and the indicator, and endpoint termination descends to the client lifecycle owner." + "path": "openhcs/authoring/session/session.py", + "sha256": "587047b63972df89ec46a01ed2c92ff3ef9577335f58a980297db02094e4e8e0", + "role": "Implements run_datasets (reserve, reset, connect, compile all, submit each with compile_artifact_id) and the endpoint status binding." }, { - "path": "openhcs/agent/services/endpoint_function_catalog_service.py", - "sha256": "977a74003819f2547548d322f4c5f0cb148b23fec20f31be4fa462bfcfb7ea04", - "role": "Owns the shared asynchronous endpoint catalogue, exact revision state, and selected-reference projection for desktop and local MCP consumers." + "path": "openhcs/authoring/session/submission.py", + "sha256": "710ac218d1db01f231b67bf0fe229dc03fb428bc6655b23671f3a026be91c673", + "role": "Submits each execution with its compile_artifact_id and auxiliary observation-export parameters." }, { - "path": "openhcs/pyqt_gui/widgets/shared/services/zmq_client_service.py", - "sha256": "8bb3b33ee64869a225e7ec23be3dcf4f5bf3622dd801f7e77bdd7826980cbd8e", - "role": "Owns the persistent OpenHCS execution-client lifecycle and retires its exact session when that endpoint terminates." + "path": "openhcs/authoring/session/progress.py", + "sha256": "cd34f68461b50506600da7854a3bf57646f2f0ff7288d0627bc40cb852a16825", + "role": "Defines ExecutionProgress: one registry-mutation subscription, dirty marking and interval rebuilds on the session main thread." }, { - "path": "external/zmqruntime/src/zmqruntime/client.py", - "sha256": "fc8bdf68455673921d79742204bf6c88e42b52bbe407d10ef4a831e084ca86a8", - "role": "Declares EndpointClientSession as the authoritative transport-client and application-admission proof." - }, - { - "path": "openhcs/pyqt_gui/widgets/shared/services/execution_state.py", - "sha256": "73d03020ffe437d8aaf4b6a5eeb4f43b91130de747d411648cb7fa2e4df27777", - "role": "Owns active batch membership, terminal outcomes and terminal row supersession." - }, - { - "path": "tests/unit/pyqt_gui/test_compile_terminal_supersession.py", - "sha256": "0b04d9c843ce2dcf16833b1c53b5853637802aa35c2d23de992c3db6c2bb2465", - "role": "Tests recompilation superseding row activity while retaining batch outcomes." - }, - { - "path": "tests/unit/pyqt_gui/test_concurrent_initialization_progress.py", - "sha256": "c8e960fa83adb73f3533cf4c75acafd59249a1993f4e650733101fdd3d62ddb8", - "role": "Tests other-plate initialisation without replacing active execution progress." + "path": "tests/unit/test_u1_session_guards.py", + "sha256": "98de1f6239607e92d2aa35bd5e95f3e1c1c42887cc238e1291ffc830ca2cd4a5", + "role": "Guards that session state lives in the Qt-free session package and its operations are declared classes." } ], "findings": [ - "The lifecycle explanation still described attach-only startup and omitted asynchronous endpoint-catalogue prewarming; it now follows the persistent endpoint and shared projection owners.", - "The status explanation assigned the indicator to client connection lifecycle even though the browser snapshot now owns every endpoint-status projection; the page now names the shared snapshot boundary.", - "GUI cleanup previously gated client-owner teardown on application admission, leaving an in-progress endpoint attempt alive; the explanation now records unconditional owner teardown and typed cancellation.", - "Clarified current endpoint catalogue reads after remote registration and retained asynchronous/cache ownership." + "The page had already been rewritten for the session in U1, but its audit record still named the deleted Plate Manager batch workflow, compile batch, execution-state and client-service modules and their tests; the audience and user need also still described Plate Manager." ], - "disposition": "revise", + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/batch_workflow_service.rst in full and compared it against the changed authorities: the deleted openhcs/pyqt_gui batch workflow, compile batch, components, client service and execution-state modules and their tests (replaced by openhcs/authoring/session), openhcs/pyqt_gui/main.py, test_main_zmq_window_routing.py, endpoint_function_catalog_service.py, zmqruntime client.py and test_concurrent_initialization_progress.py; checked CompileDatasets availability, DatasetPipelineRequest/CompiledDataset, BatchSubmitWaitEngine, run_datasets ordering and compile_artifact_id submission, ExecutionProgress listener and interval rebuild, and the endpoint status binding against the session package. The page text is accurate; the record now points at the session owners.", "Read the complete page and challenged accuracy, freshness, duplication, boundary, flow, and navigation against the listed current authorities.", "python scripts/validate_docs.py docs/source/architecture passed, and .venv Sphinx -E -W --keep-going rendered all 152 active pages without warnings.", "A focused 15-file architecture regression selection passed all 462 tests, including the authorities named for this page.", @@ -375,7 +381,7 @@ }, { "path": "docs/source/architecture/cellprofiler_interop.rst", - "source_sha256": "ca51e69ffeb5e594de80ab27c0c71bcb1d78a76fd059ab093afbab9b7a9b0040", + "source_sha256": "462d00bb6421b6de9772e6a8953288b45ee096aab92f630fcbc6a592c11ac8d2", "audience": [ "CellProfiler users", "OpenHCS interoperability and compiler maintainers" @@ -385,37 +391,52 @@ "authority": [ { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "b547c838b2c86fe7cc5fe12d6cbf6a6715e83055c2d8c1a109ac22566d22c51e", + "sha256": "3439d96f3fbfcb104a53019dc150049021d9a8132bd27096f1ccf510eefbe37b", "role": "Implements .cppipe parsing, declaration-owned headless validation, and lowering into PipelineConfig and FunctionStep declarations." }, { "path": "openhcs/interop/cellprofiler/module_declarations.py", - "sha256": "89dd65abc003c464ba6c489f52d3a4fa47be1d5f8b87d26ad72b7537f188cf8b", + "sha256": "6bbf41c158496a02439be2ee5aef12d7712fb632419c226a64c9259a55438b95", "role": "Owns module-specific callable, setting, headless-import, scope, artifact, and function-catalogue declarations." }, { "path": "openhcs/interop/cellprofiler/module_artifact_declarations.py", - "sha256": "666ec8c9adfd6a9015adf2b226df671534bb7e547cb0d76f1d16c40ac61e30a6", + "sha256": "f70b52114fb5d5e2c2fc49cd9a2b46986bde48ce02d82f1a34a67f072ca2ab20", "role": "Defines the nominal interactive-module family that rejects headless import." }, + { + "path": "openhcs/interop/cellprofiler/compile_time_contracts.py", + "sha256": "75d24fb50658de135b4d777e78aacda84d15769a76d31043343f18645cb92f3a", + "role": "Defines CellProfilerInvocationContractProviderFactory, which walks the ResolvedPipelineDefinition steps with a forward artifact context." + }, + { + "path": "openhcs/core/pipeline/compilation_session.py", + "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", + "role": "Declares ResolvedPipelineDefinition, the resolved step sequence the provider walks." + }, + { + "path": "openhcs/constants/input_source.py", + "sha256": "fccc1854051fbe47371df2cfb737080bdca14d932825ef7ddc9a0e1ed1cf0dc9", + "role": "Declares the InputSource members PREVIOUS_STEP and PIPELINE_START." + }, { "path": "tests/unit/test_cellprofiler_artifact_declarations.py", - "sha256": "627acfb57647e135e44511a8b24e3496c2a59b046f1f9b89c5ade1023cfce382", + "sha256": "fe894cd2b592d0755c106be3363eeaf078bc26d0b698a7ccfe6662ef0c0c475b", "role": "Verifies typed image, object, measurement, relationship, table, grid, graph, and special artifact declarations." }, { "path": "tests/integration/test_cellprofiler_official30_zmq.py", - "sha256": "bc5abfddf076996a050b0601c46b6039d2d812ab57cd303847b241e477176f53", + "sha256": "dbd6e113a7135901f936e9bc8278d727d678e9971a7c951fced9080adfd9d6df", "role": "Defines the executable Official30 import, execution, equivalence, and viewer evidence boundary." }, { "path": "tests/unit/test_cellprofiler_pipeline_import.py", - "sha256": "353f0c1d8e0bc3552d2e418d792794ab0631daa16c78a3e10191b1cc1ea07d73", + "sha256": "6b1d201ec5c4ba620fa2367a9cbef1f6fbdd25edbdda9bd7b723ebe3a3363412", "role": "Verifies that direct import rejects an enabled interactive-only module explicitly." }, { "path": "benchmark/cellprofiler_reference_exports.py", - "sha256": "604f9a95f94eeb10365437b875a0ff3d558dbb2fda13e7e951242d417a69b49c", + "sha256": "58cdcb7572266d562314ccae5bf9d98b5f3a4b19944e8a068f89b617219952f3", "role": "Derives digest-bound benchmark-only exporter pipelines from terminal importer contracts and owns numeric-image and categorical-label comparison semantics." }, { @@ -425,12 +446,11 @@ } ], "findings": [ - "The artifact inventory omitted spatial graphs, and a module-authoring procedure duplicated the task-oriented developer guide inside an explanation.", - "The explanation did not state that setting-backed output identities remain in public invocation keywords independently of downstream observation and are consumed before the raw runtime call.", - "Enabled interactive-only modules could otherwise be silently omitted as non-executable declarations; the owning nominal module family now rejects headless import explicitly." + "The compile-time reconstruction section said the provider walks a resolved StepSnapshot sequence; StepSnapshot and openhcs/core/pipeline/step_snapshot.py are gone, and the provider walks the steps of the ResolvedPipelineDefinition." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/cellprofiler_interop.rst in full and compared it against the changed authorities: pipeline_import.py, module_declarations.py, module_artifact_declarations.py, test_cellprofiler_artifact_declarations.py, the Official30 integration test, test_cellprofiler_pipeline_import.py and benchmark/cellprofiler_reference_exports.py; replaced the removed StepSnapshot with ResolvedPipelineDefinition and confirmed the import flow, interactive-module rejection, InputSource members, FunctionStepExecutionScope.PLATE, the provider's named methods, the CPA SQLite/properties route and CellProfilerReferenceExportPlan against current source.", "Re-read docs/source/architecture/cellprofiler_interop.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/interop/cellprofiler/pipeline_import.py; openhcs/interop/cellprofiler/module_declarations.py; tests/unit/test_cellprofiler_artifact_declarations.py; tests/integration/test_cellprofiler_official30_zmq.py.", "Focused feature-lattice QA passed 65 tests covering framework resolution, execution scope, callable contracts, compiled execution, validation, and worker policy.", @@ -455,42 +475,42 @@ "authority": [ { "path": "openhcs/core/pipeline_document.py", - "sha256": "0849800b3e6474b8176fe35bc4aaebc63cfa85bd77251000c50da5f4c928b1b2", + "sha256": "8c554c8275c665f308eb098955bda517aa804eeb34418b5e0f41d412650ce7b5", "role": "Defines the nominal pipeline code document and its rendering, parsing, and application boundary." }, { "path": "openhcs/pyqt_gui/services/ui_bridge_object_state.py", - "sha256": "747c284c37e420092ffda9abbb8275fc7deaaff7f3676cecf3f5b1f324314744", + "sha256": "e548b367507b3fb6830ce9ecb844e3b756243af96fe38e0d35de5da0b974c1fd", "role": "Projects live ObjectState scopes, fields, revisions, document capabilities, and mutation receipts through the UI bridge." }, { "path": "openhcs/agent/dto/ui_bridge.py", - "sha256": "585b1701b01b92bfce91437a60881afa769e36de9126807c3f4fcaf0ffc006fb", + "sha256": "918b30f34e7d096b19ce04017d9a991d0a10e4088c310797e497fb2444fc37c0", "role": "Carries generic endpoint application and operation-lifecycle declarations through UI bridge descriptors and request/response envelopes." }, { "path": "openhcs/agent/services/ui_bridge_transport.py", - "sha256": "32e759baf65b34989c99223d8fa9e64800ab566cd05160df2bc125b04dea5953", + "sha256": "1c2fb99aef62969cfabb9d22ccb954e8894ee72fd2e66e625304ffdb96a41b95", "role": "Validates the schema, bridge protocol, application identity, and request identity before an MCP response is accepted." }, { "path": "openhcs/pyqt_gui/services/ui_bridge_server.py", - "sha256": "667f582fffd4b489ca7274f8962a759c3d33d24b10ec8e6d1aa5cac01233b067", + "sha256": "4b0664fa62d24557294324b631ee3e39052c84e578d40ec408197bce5adcedac", "role": "Advertises the shared application identity and rejects incompatible requests before dispatching them into live UI state." }, { "path": "external/zmqruntime/src/zmqruntime/messages.py", - "sha256": "0df6261b31d3e2758ced4c7ee5933dde0e866b8e30d7927c3fcb2641a8d97d44", + "sha256": "ba7b9bac8f0202690f1e2bd17f445fe1741f43642f0a36e4a806bb577c2d1aa1", "role": "Owns generic exact endpoint-application compatibility and the typed mismatch error." }, { "path": "external/python-introspect/src/python_introspect/annotation_types.py", - "sha256": "dc126879a8a581cea0758eec4df18ec20c47ca4b1e135abde23699b65b790bde", + "sha256": "004a3ce2e214a184aaa3230ceaa89ccaa5f188c1c3d885aacad5222726c34b7a", "role": "Owns declaration-derived enum discovery, accepted input strings, and member coercion across transparent annotation wrappers." }, { "path": "openhcs/pyqt_gui/services/ui_agent_bridge.py", - "sha256": "fa8e7a1cd300c6e5cfde5ac155d3e0fd0a7324632fb150280f57c828d8dfcecd", + "sha256": "ce650c52475764657bbc0221109879594f9cf1482de8245d6c141f00474d5d23", "role": "Projects ObjectState branch and snapshot restoration through the attached-agent surface." }, { @@ -500,27 +520,27 @@ }, { "path": "tests/unit/pyqt_gui/test_ui_agent_bridge.py", - "sha256": "1feee4bb413738f04e81cbdf26244dc0d585a68921e6c7749a941a08518007a1", + "sha256": "de6758859d9743e826369162855327e4a84056cafdd7ddfdc33868d2a3093353", "role": "Verifies read-only rejection, semantic no-op application, and branch restoration through the UI bridge." }, { "path": "openhcs/pyqt_gui/services/function_step_code_document.py", - "sha256": "3cb9ee3bf0411b0cd9ddf254934dd402de12abb630bc764cd724a9492f912814", + "sha256": "67b00a1964680d65a371e238c862775069bb882da2ccb75773adfed4981b90f7", "role": "Defines the step-editor code-document projection and apply semantics." }, { "path": "tests/unit/pyqt_gui/test_pipeline_code_document_transactionality.py", - "sha256": "b7e546e904b897f13d76f0c523e751afa541a75c627349ba79a3a87313d30f4c", + "sha256": "0bb9b43d867a4e26f13b51531a2b7a847c6c3685841f8e002e209200a37366b6", "role": "Verifies revision checks, validation, snapshots, rollback, and transactional application." }, { "path": "openhcs/pyqt_gui/widgets/step_parameter_editor.py", - "sha256": "54e0bdd74f8d781c2bfdf1da559bbe7e2393f19ef74826d374a2281e8eb96258", + "sha256": "d7d6a2af7d3dafbd3edd57d4a3c52a1f40812e21b95820d1e2190a57e9a97c0c", "role": "Applies edited function patterns through generic reconciliation before rebuilding parameter panes." }, { "path": "openhcs/serialization/pycodify_formatters.py", - "sha256": "316ff58e34dcb87816082ead03347cd280fe72b4e3ab8ea31affc1627e548c23", + "sha256": "fe06f3ab873b537b7b2c0b13599ffd11030ae6a8ac504caea9e3dcfa1d4695b4", "role": "Supplies OpenHCS semantic equality and clean/full choices to shared parameter projection." }, { @@ -545,7 +565,7 @@ }, { "path": "tests/unit/test_pycodify_formatters.py", - "sha256": "445479f81fd70ce7f288d5b7537b9bbb85d1b5dd724d27baabf8e4c4e508128c", + "sha256": "2f0c6683a5255d01442ae33f676f601d5e3f1e3e04d1b2c2fc945b3c564796ce", "role": "Verifies clean/full parameter order and semantic default handling." }, { @@ -554,16 +574,10 @@ "role": "Verifies numeric value preservation and compact presentation." } ], - "findings": [ - "The page duplicated the task-oriented attached-agent sequence and rules already owned by the code/UI user guide, and bare knowledge identifiers were not navigable RTD links.", - "The page omitted the generic owner's dynamic writability and declaration-normalised no-op contracts.", - "The field projection list omitted declaration-derived enum inputs exposed by ObjectState field help.", - "The process boundary did not previously reject an MCP/UI pairing whose OpenHCS versions were incompatible despite sharing an older bridge protocol.", - "Explained signature-ordered parameter projection, retained child-state reconciliation, and generic numeric presentation ownership.", - "Clarified that function child-state retention is conditional on an unchanged editable callable surface; explicit keyword surface changes rebuild the child at its stable scope." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/code_ui_interconversion.rst in full and compared it against the changed authorities: openhcs/core/pipeline_document.py (PipelineDocumentAuthority renamed PipelineDocumentCodec; PipelineDocument unchanged), ui_bridge_object_state.py, openhcs/agent/dto/ui_bridge.py, ui_bridge_transport.py, ui_bridge_server.py, zmqruntime messages.py, python-introspect annotation_types.py, ui_agent_bridge.py and its test, function_step_code_document.py (now FunctionStepDocumentCodec), the code-document transactionality test, step_parameter_editor.py (the pickled .step settings files are gone, which the page never described), pycodify_formatters.py and its test; checked the schema/bridge-protocol/application identity checks on both bridge ends, the field-mutation request token without base revision, enum values in field help, code-document revision apply, and Plate Manager naming. Every claim the page makes still holds; no text change.", "Read the complete page and challenged accuracy, freshness, duplication, boundary, flow, and navigation against the listed current authorities.", "python scripts/validate_docs.py docs/source/architecture passed, and .venv Sphinx -E -W --keep-going rendered all 152 active pages without warnings.", "A focused 15-file architecture regression selection passed all 462 tests, including the authorities named for this page.", @@ -594,18 +608,18 @@ "authority": [ { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "ace8ae0c3e54bbe604b97cec7c3973a1abcf90e1dc517c2adda947dc2c35e882", + "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", "role": "Lowers declarations into typed step plans, including declaration-derived framework-device assignments." }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "3c840a89baccc8665d415175458280fd904c1a837b7fd879e303d7dbca129a1c", - "role": "Defines the scoped CompilationSession authority." + "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", + "role": "Defines the per-context CompilationSession and ResolvedPipelineDefinition." }, { "path": "openhcs/core/compiled_execution.py", - "sha256": "941115f2dca2e53538d3141a59f3ae1f928a87cf741fcfa177f7cd64a5f41aea", - "role": "Assembles and validates worker-safe execution bundles without a process-global GPU registry." + "sha256": "85972d418f0958e59f3185f07caf25a103bc9e101e481b2b63903377cc05e397", + "role": "Defines CompiledExecutionBundle with its runtime and pickle-safe transport contexts." } ], "findings": [ @@ -613,6 +627,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/compilation_service.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/compiled_execution.py; the page only names the session, typed-plan and execution-bundle model and redirects to pipeline_compilation_system, all of which still exist (CompilationSession, CompiledStepPlan, CompiledExecutionBundle), so the redirect stays unchanged.", "Re-read docs/source/architecture/compilation_service.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/pipeline/compiler.py; openhcs/core/pipeline/compilation_session.py; openhcs/core/compiled_execution.py.", "Focused feature-lattice QA passed 65 tests covering framework resolution, execution scope, callable contracts, compiled execution, validation, and worker policy.", @@ -623,7 +638,7 @@ }, { "path": "docs/source/architecture/compilation_system_detailed.rst", - "source_sha256": "f6b652564854eafd1e1d6670a2f6290c799d5aea05ddafff3fe6c8e7587af7bb", + "source_sha256": "d866a71b0092d3887ea91d83cbd76deb44fcf4302adf162794157dede8724ad4", "audience": [ "OpenHCS maintainers following the former detailed compiler URL" ], @@ -632,25 +647,31 @@ "authority": [ { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "ace8ae0c3e54bbe604b97cec7c3973a1abcf90e1dc517c2adda947dc2c35e882", + "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", "role": "Implements the current compiler stages, including declaration-derived framework-device assignment." }, { - "path": "openhcs/core/pipeline/step_snapshot.py", - "sha256": "d81a70b84d5c68fb738603653516d20da882fb51d74d41a6dca238823e3279cb", - "role": "Defines immutable resolved step snapshots consumed during compilation." + "path": "openhcs/core/pipeline/compilation_session.py", + "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", + "role": "Defines ResolvedPipelineDefinition, which replaced resolved step snapshots, and CompilationSession." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "6d46571e6f2d069b98a92b41d711b2e20bfab9f0493cf73d3f150d5be08e7f96", + "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", "role": "Defines typed step-plan fields, exact framework footprints, and framework-local device bindings." + }, + { + "path": "openhcs/core/compiled_execution.py", + "sha256": "85972d418f0958e59f3185f07caf25a103bc9e101e481b2b63903377cc05e397", + "role": "Defines CompiledExecutionBundle." } ], "findings": [ - "The former fixed-phase narrative is historical; the active URL correctly redirects to current StepSnapshot, CompilationSession, typed-plan, and bundle ownership." + "The redirect named StepSnapshot as part of the current architecture; openhcs/core/pipeline/step_snapshot.py was deleted and compilation now reads ResolvedPipelineDefinition." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/compilation_system_detailed.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/step_snapshot.py (deleted in 0adb68108; resolved steps now live on ResolvedPipelineDefinition in compilation_session.py), openhcs/core/compiled_step_plan.py; replaced StepSnapshot with ResolvedPipelineDefinition in the redirect text and confirmed the redirect target page exists.", "Re-read docs/source/architecture/compilation_system_detailed.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/pipeline/compiler.py; openhcs/core/pipeline/step_snapshot.py; openhcs/core/compiled_step_plan.py.", "Focused feature-lattice QA passed 65 tests covering framework resolution, execution scope, callable contracts, compiled execution, validation, and worker policy.", @@ -670,13 +691,18 @@ "authority": [ { "path": "external/ObjectState/src/objectstate/lazy_factory.py", - "sha256": "f81ce722695dbab98fd02658f2015e5cb93c1e1e69547d24249e62d896e2a828", + "sha256": "64ae924c176757863a84eef19b9df0eff516e32443c2661134ec2ae72f234bf9", "role": "Implements generic lazy dataclass creation and resolution owned by ObjectState." }, { "path": "openhcs/core/config.py", - "sha256": "ef81b50ee22b8da69513fd50d6e45a929117fa4bfaa62a20ef08a4c02531c043", - "role": "Declares OpenHCS domain configuration that consumes ObjectState mechanics." + "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "role": "Declares the kernel OpenHCS configuration dataclasses that consume ObjectState mechanics." + }, + { + "path": "openhcs/domains/microscopy/config.py", + "sha256": "79b8baefdbf22b85c199807f62b8234daca6904633a574886661a1c2e3d03a9b", + "role": "Declares the microscopy domain's configuration sections composed into the same ObjectState mechanics." } ], "findings": [ @@ -684,6 +710,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/component_configuration_framework.rst in full and compared it against the changed authorities: external/ObjectState/src/objectstate/lazy_factory.py, openhcs/core/config.py (domain sections such as analysis consolidation moved to openhcs/domains/microscopy/config.py, now added as an authority); the page only states that ObjectState owns lazy configuration and that OpenHCS documents its domain declarations, and redirects to component_system_integration and external_foundations, both of which exist, so the redirect stays unchanged.", "Re-read docs/source/architecture/component_configuration_framework.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: external/ObjectState/src/objectstate/lazy_factory.py; openhcs/core/config.py.", "Focused feature-lattice QA passed 65 tests covering framework resolution, execution scope, callable contracts, compiled execution, validation, and worker policy.", @@ -703,7 +730,7 @@ "authority": [ { "path": "external/python-introspect/src/python_introspect/signature_analyzer.py", - "sha256": "73600e546753640964d4ba5308fdf808ed3a4a6f085417d8ac0745d9e2748ed7", + "sha256": "edbe51f6c92b2c334dc4533bbeaa252be649c1ac738d2ebfca4364f2708479dc", "role": "Implements generic callable and signature analysis." }, { @@ -713,22 +740,21 @@ }, { "path": "external/metaclass-registry/src/metaclass_registry/core.py", - "sha256": "ed4dd1dfd0e4e05e916fec39f19b4b8bd526e776aa0c96e3abe9b1ffbdff0505", - "role": "Implements generic nominal registration and family discovery." + "sha256": "72c8284e8a32dd193f13f36a425289d5293a32f4a381046db523aa87182e0be8", + "role": "Implements generic nominal registration through AutoRegisterMeta." }, { - "path": "openhcs/constants/constants.py", - "sha256": "4cd6c29abd8b8e3ecc396b424d2b8b61e819a215fc1d770d3f6852b260a0691d", - "role": "Declares the OpenHCS component enum families derived from the domain component configuration." + "path": "openhcs/domains/microscopy/axes.py", + "sha256": "531695809259324032e0b88632813323b4fe6a9a176ad6f636aeef8dec332399", + "role": "Declares the microscopy axis family, the OpenHCS domain declaration registered through metaclass-registry." } ], "findings": [ - "The legacy page treated processor metaprogramming as one extracted owner even though signature analysis and nominal registration belong to separate packages.", - "python-introspect now also owns generic callable-declaration projection; the transition page now names that boundary.", - "The unreferenced OpenHCS-local dynamic-interface registry contradicted the extracted ownership boundary and was removed." + "The constants.py authority no longer declares component enum families: G1 deleted them, and the domain declaration is now the microscopy AxisFamily in openhcs/domains/microscopy/axes.py. The redirect text itself names no deleted owner and stays accurate." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/component_processor_metaprogramming.rst in full and compared it against the changed authorities: signature_analyzer.py, metaclass_registry/core.py, openhcs/constants/constants.py; confirmed signature analysis, callable-declaration projection and AutoRegisterMeta registration remain in the external packages, and replaced the deleted component-enum authority with the microscopy axis family declaration; no page edit was needed.", "Read the complete page and challenged accuracy, freshness, duplication, boundary, flow, and navigation against the listed current authorities.", "python scripts/validate_docs.py docs/source/architecture passed, and .venv Sphinx -E -W --keep-going rendered all 152 active pages without warnings.", "On 2026-08-21, re-read the transition against signature_analyzer.py and callable_declaration.py after python-introspect 0.1.10.", @@ -742,7 +768,7 @@ }, { "path": "docs/source/architecture/component_system_integration.rst", - "source_sha256": "5c4c60b9b9a02dfb0dd0f56ecc2bf4528e877314f14f76d0591ec112958e84c8", + "source_sha256": "28c20ffb0731f03ab08ed28bd2a431af411a8d30be02002bdfb3dcdc0931d0f4", "audience": [ "OpenHCS compiler, source-model, and processing maintainers" ], @@ -750,36 +776,43 @@ "diataxis": "explanation", "authority": [ { - "path": "openhcs/constants/constants.py", - "sha256": "4cd6c29abd8b8e3ecc396b424d2b8b61e819a215fc1d770d3f6852b260a0691d", - "role": "Declares the nominal component enum families and their distinct domains." + "path": "openhcs/core/axes.py", + "sha256": "8d93133639c576050f089fb8aab99937c8982aff07e742fea44f8a12e62bb7b2", + "role": "Declares Axis, AxisFamily and the PartitionAxis role whose single member partitions compiled contexts." }, { - "path": "openhcs/core/component_set.py", - "sha256": "b3f7dbee802766aab3002256db044025abca8c4ce60be4648f7c1de7b13b41cc", - "role": "Implements ordered enum-aware component set operations without copied name tables." + "path": "openhcs/core/config.py", + "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "role": "Declares the step-level variable_components and group_by fields and the pipeline-level sequential_components field." }, { "path": "openhcs/core/components/component_values.py", - "sha256": "3813d6f7d6d66fc21ea881e3dfcb72f54bd1bfe36212895f00be8f09e59d49dd", - "role": "Carries one complete immutable value projection keyed by the canonical component declaration." + "sha256": "17d9e89c5b04eaef3f592efa9fd3677bdcec4b58b5f91092da2f1c1c64321a36", + "role": "Carries one complete immutable AxisValues projection keyed by the active family's axes." }, { "path": "openhcs/core/components/parser_metaprogramming.py", - "sha256": "5eaf7c10d4d5667deb723c5b8d92410183ae06ce7b3092ddb71a88e9832d4b7e", - "role": "Wraps complete component values in a nominal filename result and exposes string keys only at explicit boundaries." + "sha256": "e17a7df08e4c2506971c3039ed0ea3bc918c515d761fbeca54bc1efc156a51c3", + "role": "Wraps complete component values in FilenameParseResult and exposes string keys only at explicit boundaries." + }, + { + "path": "openhcs/core/dataset_sources/interfaces.py", + "sha256": "dd88254bfd5da8b0fdf0a038e358f36b49db5b8cf0b93213d56dc928af965f00", + "role": "Defines MetadataComponentValueSet, the metadata-handler projection of complete axis values." }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "ace8ae0c3e54bbe604b97cec7c3973a1abcf90e1dc517c2adda947dc2c35e882", - "role": "Consumes component declarations to construct execution axes, stacks, grouping, and typed plans." + "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "role": "Consumes the axis declarations to construct execution-axis contexts, stacks, grouping, and typed plans." } ], "findings": [ - "The page did not explain how complete per-plate and per-filename component values cross internal boundaries without per-axis method or string-key mirrors." + "The page listed ``multiprocessing_axis`` as a component declaration; no such declaration exists. The parallel axis is the family's single PartitionAxis member (well in the microscopy family).", + "The constants.py and component_set.py authorities no longer own component declarations after G1 deleted the component enums; replaced them with openhcs/core/axes.py and the config fields' owner, and added the dataset-source interface that defines MetadataComponentValueSet." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/component_system_integration.rst in full and compared it against the changed authorities: constants.py, component_set.py, component_values.py, parser_metaprogramming.py, compiler.py; verified AxisValues, FilenameParseResult, MetadataComponentValueSet, variable_components/group_by/sequential_components and AxisFamily.partition_axis in current source, and replaced the nonexistent multiprocessing_axis declaration with the family's PartitionAxis.", "Re-read docs/source/architecture/component_system_integration.rst in full against repository base 32ca17c17 and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/constants/constants.py; openhcs/core/component_set.py; openhcs/core/pipeline/compiler.py.", "Focused feature-lattice QA passed 65 tests covering framework resolution, execution scope, callable contracts, compiled execution, validation, and worker policy.", @@ -792,32 +825,47 @@ }, { "path": "docs/source/architecture/component_validation_system.rst", - "source_sha256": "c9d15356dc02939a216b2bf7bad60b43b2a244ce30d1eda4e61556c26b80ee02", + "source_sha256": "96dd35dc7c5ef2788463cf6f8ef6f085390aa4f5865391cc51eb5d794c4a0c20", "audience": [ "OpenHCS configuration and compiler maintainers" ], "user_need": "Look up the semantic component combinations rejected before runtime.", "diataxis": "reference", "authority": [ + { + "path": "openhcs/core/axes.py", + "sha256": "8d93133639c576050f089fb8aab99937c8982aff07e742fea44f8a12e62bb7b2", + "role": "Defines AxisFamily.named decoding, variable_axes as every axis but the partition axis, and the Ungrouped declaration." + }, { "path": "openhcs/core/components/validation.py", - "sha256": "21ff922447de83ed3f62e54c25df7966419b4895be23a8a3c5d42a3981df554c", - "role": "Defines shared enum conversion and component-combination validators." + "sha256": "3efbc509d3559a5bf3255312b652e6b8944427e2fa10fa7ca353753082099fe6", + "role": "Implements GenericValidator step checks and dictionary-pattern key checks against dataset values." }, { "path": "openhcs/core/pipeline/funcstep_contract_validator.py", - "sha256": "73cb5250a5855625903f8b7deada667b83fba594d66de0ed4a68ec76afa3db19", - "role": "Validates callable grouping, required axes, function-step components, and declared input, output, and execution memory." + "sha256": "4a167f620be9ff6e6524101652b6dfa25984b3a389b19da3f5ce78c14d5947d4", + "role": "Validates allowed group_by roles, required callable axis roles, and declared memory contracts per callable." + }, + { + "path": "openhcs/core/pipeline/compiler.py", + "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "role": "Rejects pipeline sequential axes that are also a step's variable axes." }, { "path": "tests/unit/test_funcstep_contract_validator.py", - "sha256": "2188b297564b7c756af16f1138e304a1a7dadbdea28f7c440bc7b484d9eb615c", - "role": "Covers accepted and rejected component, grouping, callable-contract, and execution-memory combinations." + "sha256": "a208ca59a60b2d5bbcc9955e99592fd480368995336f6ef86a0a06f8f6991747", + "role": "Covers accepted and rejected grouping, required-role, and callable-contract combinations." } ], - "findings": [], - "disposition": "keep", + "findings": [ + "The page told consumers to use nominal enum members; G1 deleted the component enums and consumers now use the axis classes.", + "The check list was imprecise: variable axes are checked against the family's variable axes (not a source universe), dictionary-pattern keys against dataset values, callable requirements are axis roles, and the sequential check rejects overlap with a step's variable axes; there is no separate multiprocessing topology check.", + "The opening framed the declarations as microscope dimensions; the kernel is domain-blind and the microscopy family is one declaration." + ], + "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/component_validation_system.rst in full and compared it against the changed authorities: validation.py, funcstep_contract_validator.py, test_funcstep_contract_validator.py; traced GenericValidator.validate_step, validate_dict_pattern_keys, validate_allowed_group_by, validate_required_axis_roles and the compiler's sequential-components check, and rewrote the check list and consumer guidance to match.", "Re-read docs/source/architecture/component_validation_system.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/components/validation.py; openhcs/core/pipeline/funcstep_contract_validator.py; tests/unit/test_funcstep_contract_validator.py.", "Focused feature-lattice QA passed 65 tests covering framework resolution, execution scope, callable contracts, compiled execution, validation, and worker policy.", @@ -827,7 +875,7 @@ }, { "path": "docs/source/architecture/concurrency_model.rst", - "source_sha256": "db244918e5b7ef7ba93ae17d1855954c630e577b56813f8ded29dafd4e7889bc", + "source_sha256": "6c3bd9dbb4c18735e7bcccf48777ddf0f8ea646ae8ab9d3e5a6d2ce8fd0120bb", "audience": [ "OpenHCS runtime and multiprocessing maintainers" ], @@ -836,51 +884,59 @@ "authority": [ { "path": "openhcs/core/compiled_execution.py", - "sha256": "941115f2dca2e53538d3141a59f3ae1f928a87cf741fcfa177f7cd64a5f41aea", - "role": "Defines the worker-safe CompiledExecutionBundle and its context-derived axis projection." + "sha256": "85972d418f0958e59f3185f07caf25a103bc9e101e481b2b63903377cc05e397", + "role": "Defines the worker-safe CompiledExecutionBundle and its pickle-safe transport snapshot." }, { "path": "openhcs/core/context/processing_context.py", - "sha256": "c7b79f50661c528261ce8c577fd67c753c84a5e499aa87cd5e540b53417351a2", + "sha256": "81c5c0bf411040bac2d93418093fff7c97c5ffd9ad942272c0f71c2982c96a44", "role": "Owns the compiled multiprocessing-axis identity for one processing context." }, { - "path": "openhcs/core/orchestrator/worker_lanes.py", - "sha256": "2e2a17b52e06f692c924554c387ed7aed9d25b3174c60e69d28bed6906f430b0", - "role": "Groups runtime contexts into worker lanes from their declared axis identity." + "path": "openhcs/core/orchestrator/bundle_execution.py", + "sha256": "f5ba09a2ec7c688b1ca08167ab398cc976a9e2c39b301ae98f3c5fd560a25c85", + "role": "Runs one compiled bundle, then the parent-owned dataset-scoped steps and post-execute hooks." }, { - "path": "tests/unit/test_orchestrator_lane_planning.py", - "sha256": "09c39f5cba68f8c15a0def8a09843b62d6da5c5a01e7558a9263ce4d311c078d", - "role": "Verifies that worker-lane grouping remains correct when context mapping keys do not encode axis identity." + "path": "openhcs/core/orchestrator/worker_lanes.py", + "sha256": "d915c01a0ae2dd010d6b972e275669fef63bb3aefd100f1b66a1e34653673db0", + "role": "Defines WorkerLaneExecutionContext, the fork-inherited bundle state, and lane grouping from context-owned axis identity." }, { "path": "openhcs/core/orchestrator/worker_execution.py", - "sha256": "3c952465268e4b6c44676a853bb9c99d2d04a0490c7d398d2cdaf8b3b5f966d7", - "role": "Owns executor construction, submission, result collection, cancellation, and footprint-aware cleanup." + "sha256": "23a3ef2a70378dd1bb5c413a56e36bbb3fa013142741c9b45407bfc47de8c4a4", + "role": "Owns executor construction, the WorkerCommand family served by process workers, result collection, cancellation, and cleanup." }, { "path": "openhcs/core/orchestrator/cancellation.py", "sha256": "f825b602fb020a23d602e1f93ed9cecea881eca16e07882fd7a91ab519445244", - "role": "Owns the exact execution cancellation scope, pre-entry request handoff, and cooperative safe-boundary signal." + "role": "Owns the exact execution cancellation scope and cooperative safe-boundary signal." }, { "path": "openhcs/core/worker_start_policy.py", "sha256": "9493c8729ddfc7457a3a58efb605457d5aa4a54cd42c3f4ef1bd9f28593d9475", "role": "Owns multiprocessing start-method selection from the typed compiled GPU footprint." }, + { + "path": "tests/unit/test_orchestrator_lane_planning.py", + "sha256": "3d5c2f3662b8d4c330582af0c4a649b8f4d9582e064d9cf3159798c8934a2bbb", + "role": "Verifies that worker-lane grouping remains correct when context mapping keys do not encode axis identity." + }, { "path": "tests/unit/test_worker_start_policy.py", - "sha256": "a3eaadb10898f10e50391310d65d338d60e9df2f4f6dec8b791be7dc75d31fc9", + "sha256": "5a6f5e1ef54c49bc7f49c8b7baa530e685a8b76daf37679488ec50d9fb5fc37c", "role": "Verifies platform, server, override, and GPU start-method decisions." } ], "findings": [ - "The cancellation explanation did not distinguish generic terminal status ownership, cooperative thread boundaries, and process-worker interruption.", - "The execution-unit explanation left axis identity implicit and did not distinguish context-owned identity from context mapping keys." + "The page named ``compiled_plate_execution``, which G7 renamed to ``bundle_execution``.", + "The execution-unit text said the configured multiprocessing axis yields one context per well and that workers receive one context at a time; the partition axis is the family's PartitionAxis, and each worker executes a lane of the contexts it owns.", + "Process workers now serve a typed WorkerCommand family and forward progress over their parent connection; the page described only a shared queue.", + "Output consolidation is a microscopy-domain post-execute hook, and terminal steps are dataset-scoped; the page called them plate steps and core consolidation." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/concurrency_model.rst in full and compared it against the changed authorities: compiled_execution.py, processing_context.py, worker_lanes.py, test_orchestrator_lane_planning.py, worker_execution.py, test_worker_start_policy.py; confirmed CompiledExecutionBundle, WorkerLaneExecutionContext, resolve_worker_start_context, the cancellation signal and AXIS_COMPLETED, renamed compiled_plate_execution to bundle_execution, and corrected the lane, worker-command, progress and post-execute claims.", "Re-read docs/source/architecture/concurrency_model.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/compiled_execution.py; openhcs/core/orchestrator/worker_execution.py; openhcs/core/orchestrator/cancellation.py; openhcs/core/worker_start_policy.py; tests/unit/test_worker_start_policy.py.", "Focused feature-lattice QA passed 65 tests covering framework resolution, execution scope, callable contracts, compiled execution, validation, and worker policy.", @@ -902,25 +958,26 @@ "authority": [ { "path": "external/ObjectState/src/objectstate/lazy_factory.py", - "sha256": "f81ce722695dbab98fd02658f2015e5cb93c1e1e69547d24249e62d896e2a828", + "sha256": "64ae924c176757863a84eef19b9df0eff516e32443c2661134ec2ae72f234bf9", "role": "Implements reusable lazy dataclass configuration mechanics." }, { "path": "external/ObjectState/src/objectstate/context_manager.py", - "sha256": "a01b0fc8aeb9b5b0888346f725f92cbfd96b90963bf20001b5414a5e6cd6c0e6", + "sha256": "25b1a7aa1dc6abc96b6ff91dd14e27fc3804c06f8c45a2c7a3ca6a3dd01b35a0", "role": "Implements generic scoped context resolution." }, { "path": "openhcs/core/config.py", - "sha256": "ef81b50ee22b8da69513fd50d6e45a929117fa4bfaa62a20ef08a4c02531c043", - "role": "Declares the OpenHCS global, pipeline, step, and standalone configuration hierarchy." + "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "role": "Declares GlobalPipelineConfig and the step and standalone configuration dataclasses OpenHCS composes over ObjectState." } ], "findings": [ - "The transition page did not route readers to the exact ObjectState owner narrative or the current OpenHCS system boundary." + "The changes to lazy_factory.py, context_manager.py and config.py (axis-family typing, partition filter, viewer slot modes) do not touch the ownership the redirect states; the page stays a redirect, and the config.py role now names the declarations it actually holds instead of a separate pipeline config class, which the injection system generates." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/configuration_framework.rst in full and compared it against the changed authorities: lazy_factory.py, context_manager.py, openhcs/core/config.py; confirmed ObjectState still owns lazy configuration and context resolution, the linked system_overview and external_foundations pages exist, and no page edit was needed.", "Re-read docs/source/architecture/configuration_framework.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: external/ObjectState/src/objectstate/lazy_factory.py; external/ObjectState/src/objectstate/context_manager.py; openhcs/core/config.py.", "Focused feature-lattice QA passed 65 tests covering framework resolution, execution scope, callable contracts, compiled execution, validation, and worker policy.", @@ -940,20 +997,21 @@ "authority": [ { "path": "external/ObjectState/src/objectstate/context_manager.py", - "sha256": "a01b0fc8aeb9b5b0888346f725f92cbfd96b90963bf20001b5414a5e6cd6c0e6", + "sha256": "25b1a7aa1dc6abc96b6ff91dd14e27fc3804c06f8c45a2c7a3ca6a3dd01b35a0", "role": "Implements reusable configuration-context registration and resolution." }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "3c840a89baccc8665d415175458280fd904c1a837b7fd879e303d7dbca129a1c", - "role": "Consumes already resolved configuration in a scoped compiler session." + "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", + "role": "Consumes an already resolved GlobalPipelineConfig in a scoped compiler session." } ], "findings": [ - "The transition page did not link the exact ObjectState architecture owner or the current OpenHCS resolution path." + "The changes to context_manager.py and compilation_session.py do not alter the ownership the redirect states; the compiler session still receives resolved configuration, so the page stays a redirect without edits." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/context_system.rst in full and compared it against the changed authorities: context_manager.py, compilation_session.py; confirmed ObjectState owns context management and the compiler session consumes a resolved GlobalPipelineConfig; no page edit was needed.", "Read the complete page and challenged accuracy, freshness, duplication, boundary, flow, and navigation against the listed current authorities.", "python scripts/validate_docs.py docs/source/architecture passed, and .venv Sphinx -E -W --keep-going rendered all 152 active pages without warnings." ] @@ -999,7 +1057,7 @@ "authority": [ { "path": "openhcs/processing/custom_functions/manager.py", - "sha256": "aa5aa5a7032cd38c170d29d800175022883f04df7ce2d3f2fb4f6decad43f97d", + "sha256": "f87d622c4e03db3b6d7f977daa86187b892f59133be264ffa381472785247e56", "role": "Owns persisted custom-function lifecycle and projects validated declarations through the custom runtime registry." }, { @@ -1009,10 +1067,11 @@ } ], "findings": [ - "The architecture URL no longer owns a mechanism and is intentionally a transition to the maintained user workflow." + "The manager.py changes keep the persisted lifecycle the redirect points to; no claim was invalidated, and the architecture URL remains a transition to the user workflow page." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/custom_function_registration_system.rst in full and compared it against the changed authority manager.py; confirmed the user_guide/custom_function_management target exists and CustomFunctionManager still owns persisted custom functions through the runtime registry; no page edit was needed.", "Re-read docs/source/architecture/custom_function_registration_system.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/processing/custom_functions/manager.py; openhcs/core/steps/function_step.py.", "Focused feature-lattice QA passed 65 tests covering framework resolution, execution scope, callable contracts, compiled execution, validation, and worker policy.", @@ -1022,7 +1081,7 @@ }, { "path": "docs/source/architecture/declarative_window_system.rst", - "source_sha256": "480757c11ae720000f495ecfd073bc695a59f3ccf60a1c4069e0990a28828f96", + "source_sha256": "9dd05739e15cd02e259e4537e18c8f84485ccfc1440e0fa51500c516003785bd", "audience": [ "OpenHCS UI maintainers following the former declarative-window URL" ], @@ -1041,8 +1100,13 @@ }, { "path": "openhcs/agent/ui_bridge_identities.py", - "sha256": "5913d14b99340174af87b2a7cafa3cdd080a331553c0cf59392ade7d253f1234", - "role": "Owns the nominal declarations for stable OpenHCS widget and window identities, presentation titles, and complete stable-window family projection." + "sha256": "66dcf838d08cac2004ab8f73db777f73270cc520697281cf4f94a5fc6af4dc64", + "role": "Owns the nominal stable widget and window identities, including session-view windows whose title comes from the rendered view." + }, + { + "path": "openhcs/authoring/session/views.py", + "sha256": "8ce6cac27293ac9105f4d2ebc76d10eebeba1ef90e0e8954c0a117910e17e3b3", + "role": "Declares the SessionView family, each view's title, and the operations its window binds." }, { "path": "openhcs/pyqt_gui/services/ui_window_ids.py", @@ -1051,15 +1115,16 @@ }, { "path": "tests/unit/test_build_website.py", - "sha256": "4a8952ba0f51590ffd9f1d787986653802506807079a7e0d118343264252e0e0", - "role": "Verifies that the stable UI window reference derives its identities and titles from the nominal registry." + "sha256": "e88fc8ae73c5610dfb0a9948271a016e666451228ecade92aa1b2892da106059", + "role": "Verifies that the stable UI window reference derives its identities and titles from the identity registry." } ], "findings": [ - "The legacy mechanism is extracted; the final page is correctly constrained to owner-package links and the OpenHCS integration boundary." + "Since U1, the plate-manager and pipeline-editor windows render session views and take their titles from them, and pipeline and execution actions are session operations rather than widget actions; the retained-ownership sentence now says so." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/declarative_window_system.rst in full and compared it against the changed authorities: ui_bridge_identities.py, test_build_website.py; confirmed the pyqt-reactive window-manager and scope-window-factory pages and sources exist, traced UiSessionViewWidgetIdentityDeclaration to SessionView.rendered_in, and revised the OpenHCS-retained ownership sentence.", "Read the complete page and challenged accuracy, freshness, duplication, boundary, flow, and navigation against the listed current authorities.", "python scripts/validate_docs.py docs/source/architecture passed, and .venv Sphinx -E -W --keep-going rendered all 152 active pages without warnings.", "On 2026-08-28, re-read the complete redirect after stable window strings moved to nominal identity declarations; OpenHCS still owns product identities while pyqt-reactive owns generic lifecycle.", @@ -1080,20 +1145,21 @@ "authority": [ { "path": "openhcs/core/function_patterns.py", - "sha256": "fe76286a2d378a39fd6061273363ccebbdc88f1dde6b1d36ed54b9cc28539e4a", - "role": "Defines callable, sequential-chain, and dictionary-fan-out patterns and derives execution memory through nested declarations." + "sha256": "95b9056c6980e4f7943f909aeae8262252c9aa175247a534befc8d4cc459fdd2", + "role": "Defines callable, sequential-chain, and dictionary-fan-out patterns and their grouped source keys." }, { "path": "tests/unit/test_function_patterns.py", - "sha256": "2b18e6f40f3f5e4a9673f954544b0113eba710675573fcda3fac6ef18d5eb725", + "sha256": "d0160652d548748901a8b95d21abde9dca050259fd2f74d4a85cb60a50441175", "role": "Verifies dictionary fan-out, composition, and invalid pattern rejection." } ], "findings": [ - "The case-study URL is historical and now serves only as a stable route to the current conceptual owner." + "The function_patterns.py and test changes (axis-typed grouping, payload-axis proofs) do not affect the redirect; dictionary fan-out still exists and concepts/function_patterns remains its owner page." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/dict_pattern_case_study.rst in full and compared it against the changed authorities: function_patterns.py, test_function_patterns.py; confirmed NormalizedFunctionPattern still carries grouped dictionary keys and the redirect target exists; no page edit was needed.", "Re-read docs/source/architecture/dict_pattern_case_study.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/function_patterns.py; tests/unit/test_function_patterns.py.", "Focused feature-lattice QA passed 65 tests covering framework resolution, execution scope, callable contracts, compiled execution, validation, and worker policy.", @@ -1112,7 +1178,7 @@ "authority": [ { "path": "external/ObjectState/src/objectstate/lazy_factory.py", - "sha256": "f81ce722695dbab98fd02658f2015e5cb93c1e1e69547d24249e62d896e2a828", + "sha256": "64ae924c176757863a84eef19b9df0eff516e32443c2661134ec2ae72f234bf9", "role": "Implements dynamic lazy dataclass generation, validation, and resolution." }, { @@ -1122,17 +1188,18 @@ } ], "findings": [ - "The transition page did not identify the exact ObjectState architecture owner or the current OpenHCS system route." + "The lazy_factory.py change does not move lazy-dataclass generation out of ObjectState; the redirect is still accurate and needed no edit." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/dynamic_dataclass_factory.rst in full and compared it against the changed authority external/ObjectState/src/objectstate/lazy_factory.py; confirmed ObjectState still owns lazy dataclass generation and the linked pages exist; no page edit was needed.", "Read the complete page and challenged accuracy, freshness, duplication, boundary, flow, and navigation against the listed current authorities.", "python scripts/validate_docs.py docs/source/architecture passed, and .venv Sphinx -E -W --keep-going rendered all 152 active pages without warnings." ] }, { "path": "docs/source/architecture/experimental_analysis_system.rst", - "source_sha256": "9b143aff0b24ea2c3d9dbebe9f4789f764c3e81d1258747ee2fe33eed8e4e943", + "source_sha256": "5fb63b58a0a9d3c63672167141dccdcb3d11ec786b044386d136b4cc7d5730a6", "audience": [ "OpenHCS experimental-analysis maintainers", "desktop analysis integrators" @@ -1141,37 +1208,43 @@ "diataxis": "explanation", "authority": [ { - "path": "openhcs/core/config.py", - "sha256": "ef81b50ee22b8da69513fd50d6e45a929117fa4bfaa62a20ef08a4c02531c043", - "role": "Declares standalone ExperimentalAnalysisConfig options and all directory-workflow filenames." + "path": "openhcs/domains/microscopy/config.py", + "sha256": "79b8baefdbf22b85c199807f62b8234daca6904633a574886661a1c2e3d03a9b", + "role": "Declares standalone ExperimentalAnalysisConfig with all five directory-workflow filenames, outside the GlobalConfigSection family, and NormalizationMethod's member-owned calculations." }, { "path": "openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py", - "sha256": "316be268b761bfa74ed1df137f2ba014a1f1c273aa9abd598a8a2e10388afb79", - "role": "Coordinates the workbook-declared format strategy, replicate-local normalization, exports, and configuration-projected run_directory boundary." + "sha256": "2705d459204557c4f7092df3fd21ed687dcf56449ed361485382b1c0ce196444", + "role": "Coordinates the workbook-declared format strategy, replicate-local normalization, exports, and the configuration-projected run_directory boundary." + }, + { + "path": "openhcs/processing/backends/experimental_analysis/result_formats.py", + "sha256": "6951f41a2ff2bdac1945c7e56584a578c111c078cac1d684030c00f95a23f528", + "role": "Defines the scope-keyed CX5 and MetaXpress ExperimentalResultFormatStrategy leaves for result reading and plate population." }, { - "path": "openhcs/formats/experimental_result_formats.py", - "sha256": "ce46b895e4caafec7d026112cacfdfc496de0cb4cd5978cceb94db02b98fe288", - "role": "Defines the scope-keyed CX5 and MetaXpress leaves that own result reading, feature extraction, and plate population." + "path": "openhcs/processing/backends/experimental_analysis/analysis.py", + "sha256": "17a701e0740eac3e11319fe73f10302bbcab603218349bfba4e07014aabc27bb", + "role": "Owns layout parsing, exclusion projection, replicate averaging, and table/plate-grid export." }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "a156a006a3c40f43107b9b1f6ecb97ed150879eb88ded8ef89e25214309ae33a", - "role": "Routes the desktop analysis action through ExperimentalAnalysisEngine rather than the deprecated wrapper." + "sha256": "e43085c1eaf0c836ce34e1a1dae7740488d91bdfea56844e7ddd4e6925561cf6", + "role": "Routes the desktop analysis action through ExperimentalAnalysisEngine.run_directory with the default configuration." }, { "path": "tests/unit/test_experimental_analysis_engine.py", - "sha256": "23b5d73def871a4c1f1e60625faf56d4664a315f90ddbddff7fa1ceab2117c17", - "role": "Verifies configured path projection, exclusions, normalization-method delegation, mirror deletion, and an end-to-end MetaXpress directory run." + "sha256": "641fadecf8c940cc2316a06771c74db5f3f2db9dafdfbfebdc535760d6d776ec", + "role": "Verifies configured path projection, exclusions, normalization-method delegation, and an end-to-end MetaXpress directory run." } ], "findings": [ - "The audit exposed a source-level mirror: the desktop hardcoded four filenames and called a deprecated wrapper; the owner was repaired and the page now describes the configuration-owned engine path.", - "Source verification found three parallel CX5 and MetaXpress dispatch systems plus caller-owned normalization selection. They were collapsed into scope-keyed result-format leaves and NormalizationMethod member behavior before documenting the final boundary." + "openhcs/formats/experimental_result_formats.py was moved (G2) to openhcs/processing/backends/experimental_analysis/result_formats.py, and ExperimentalAnalysisConfig and NormalizationMethod moved from core/config.py into the microscopy domain config; the page now says the config is a domain declaration.", + "The page contrasted the desktop path with a deprecated compatibility wrapper that no longer exists; the clause was removed." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/experimental_analysis_system.rst in full and compared it against the changed authorities: core/config.py, unified_analysis_engine.py, the moved formats/experimental_result_formats.py, pyqt_gui/main.py, test_experimental_analysis_engine.py; traced ExperimentalAnalysisConfig and NormalizationMethod to domains/microscopy/config.py, the result-format strategies to result_formats.py and layout parsing to analysis.py, confirmed the desktop action calls ExperimentalAnalysisEngine().run_directory, and removed the stale deprecated-wrapper clause.", "Re-read docs/source/architecture/experimental_analysis_system.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/config.py; openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py; openhcs/formats/experimental_result_formats.py; openhcs/pyqt_gui/main.py; tests/unit/test_experimental_analysis_engine.py.", "Focused feature-lattice QA passed 65 tests covering framework resolution, execution scope, callable contracts, compiled execution, validation, and worker policy.", @@ -1188,7 +1261,7 @@ }, { "path": "docs/source/architecture/external_foundations.rst", - "source_sha256": "2c95e2864cf17a5a6195c2fc3c498ac99eb4cf450a841f7a8d2417a94d53f1cb", + "source_sha256": "032486c75f7c51f03c8868814044e42624517ff2c8e6f785cf9e88d27066ef4d", "audience": [ "OpenHCS maintainers", "first-party package integrators" @@ -1198,7 +1271,7 @@ "authority": [ { "path": "pyproject.toml", - "sha256": "6217b5f029796edcd83aa47fb8632d37aa7552344755c578fa4f181a6b23282d", + "sha256": "3592cc37444e2e1f5bb0284e939643039c0d513217861af8133dd8bd6e8a4325", "role": "Pins the eight first-party package constraints and documents their installed integration surface." }, { @@ -1213,12 +1286,12 @@ }, { "path": "external/ObjectState/README.md", - "sha256": "1d72de341fc17c96587a75bba4a913787321bf598b848474a833b34a0547842e", + "sha256": "e13c149f1130f43eab46cca59986ca0e2de5ce29f5b7d48f8e8aebb9fc9e127a", "role": "States ObjectState ownership of lazy state, contexts, provenance, and editing mechanics." }, { "path": "external/arraybridge/README.md", - "sha256": "b108a144e2f797425a1734f745a73ac7a215bd5e58b3f6fe4342b18e7d16b892", + "sha256": "2fe1a330ca101dadb6cea5ac6f0482a5ffe8c7e0e4b9bd7bc8bc2220e106415a", "role": "States ArrayBridge ownership of array conversion, memory mechanics, and callable input, output, and execution metadata." }, { @@ -1226,11 +1299,31 @@ "sha256": "0a8a5f1a61552b057997e94e4c89a19b5615875f307949e0d11784179f2a94f2", "role": "Owns framework-neutral array geometry inspection without device-to-host conversion." }, + { + "path": "external/arraybridge/src/arraybridge/array_operations.py", + "sha256": "ea8c4e21ffbef5b5d502f9e5b7779bd5a341cec4408337d16ba89613f4305636", + "role": "Owns native array operations as one family leaf per MemoryType declaration." + }, + { + "path": "external/arraybridge/src/arraybridge/types.py", + "sha256": "08f7ff4df8807bb3340e00a02c6276b1c702a0f00a2eedacd701142736c5d804", + "role": "Owns declaration-derived child-process environments for optional array frameworks." + }, { "path": "external/metaclass-registry/README.md", - "sha256": "668dab2cc7c536778ca0010f957c85335eda5eb2392e60c62d08b11d79009c37", + "sha256": "d89a6b35231f21cb398d02b484b0fc1c3a6e72c29831ab19b32852d201f6988f", "role": "States metaclass-registry ownership of reusable nominal registration and host-logging-safe discovery imports." }, + { + "path": "external/metaclass-registry/src/metaclass_registry/strategies.py", + "sha256": "5c00cd4d80f9f0ceffbc2ab95ad8c6abd44a8a66a974959445ef93df08dd4f0d", + "role": "Owns the shared registered-strategy and nominal selection mixins moved out of OpenHCS." + }, + { + "path": "external/metaclass-registry/src/metaclass_registry/caches.py", + "sha256": "aa5681adb13a5424f7e8ee5b0fa601b23409f7093dc3e755c23c0b16e7a16e21", + "role": "Owns the bounded cache and the process-local cache family." + }, { "path": "external/PolyStore/README.md", "sha256": "c054ab7a02e743a089ea7e258444ca2982318b86af27457877cc60156a3831cf", @@ -1243,53 +1336,54 @@ }, { "path": "external/pyqt-reactive/src/pyqt_reactive/services/window_snapshot.py", - "sha256": "c21e85912ab6715cc9c97bc796b9e5ed03610bf96f786d923557f18a876081f9", + "sha256": "bced886e6bbfd69ae9bdebf950027b809e72c23eddadcaa5fcf7151adf7ae913", "role": "Owns safe Qt-rendered capture and optional bounded observations of actual flash-renderer frames." }, - { - "path": "external/pyqt-reactive/docs/architecture/window_snapshot.rst", - "sha256": "76100ca0e5342138a0abe0b7e026939f33a06da9d9bff6edff3cc26f165c6378", - "role": "Documents the generic capture scopes and the application integration boundary." - }, { "path": "external/python-introspect/README.md", - "sha256": "d861f8aff2e79d9e77fbb2286b2f353c5b568412482a57efce9a488e58d543fe", + "sha256": "059dcfaafc0b2222e8e64b7fe072d24bddfb5a347a6f25113e715ed7946e44a1", "role": "States python-introspect ownership of callable/signature analysis and signature-derived declaration projection." }, + { + "path": "external/python-introspect/src/python_introspect/jsonable.py", + "sha256": "8fe06533a6dce179b7d77792b6ac768c2a64a1e9e79bbe926893498fae035bb4", + "role": "Owns the to_jsonable encoder paired with the dataclass_from_mapping decoder." + }, + { + "path": "external/python-introspect/src/python_introspect/public_api.py", + "sha256": "f3093140a9a7c65ff8e94e3d0f3ab85d720eaa6ad840e0f44b4725f5b73e6059", + "role": "Owns declared public-name derivation and the lazy_exports helper." + }, { "path": "external/zmqruntime/README.md", - "sha256": "c8f7fa1b36636d6cbcee78bf259f2e2511ef7e19a5533ae246548167e35b3eb9", + "sha256": "03d1d715f46d2bd9c4b4b4142fd4622764c3109b2671801fae5163415e41718c", "role": "States ZMQRuntime ownership of generic process and transport lifecycle." }, { "path": "external/zmqruntime/src/zmqruntime/client.py", - "sha256": "fc8bdf68455673921d79742204bf6c88e42b52bbe407d10ef4a831e084ca86a8", - "role": "Owns exact spawned endpoint-process adapters, retained exit observation, automatic child reaping, and owner groups without process rediscovery." + "sha256": "7d3cce31d5abaa070927f02fe87d919687619e2cf4adbc37ab82483b6b8fc212", + "role": "Owns exact spawned endpoint-process adapters, retained exit observation, automatic child reaping, and owner groups." + }, + { + "path": "external/zmqruntime/src/zmqruntime/process_launch.py", + "sha256": "a9b913aee1c08967cc86fa7def39e19f2bac49c444c6c6db8d375d77e1a31b60", + "role": "Owns the per-platform background-process launch policy moved out of pyqt-reactive." }, { "path": "external/pycodify/README.md", "sha256": "f32db990f5e1feb01ee3d7adbe1195fd691c8964acd789759901d00170cecf7e", "role": "States pycodify ownership of Python-source serialisation and formatter composition." - }, - { - "path": "external/arraybridge/src/arraybridge/types.py", - "sha256": "7b94760d57e54008d7adf4e4d37dca45d8a2c288a9dc8400be00c26148cf7624", - "role": "Owns declaration-derived child-process environments for optional array frameworks." } ], "findings": [ - "ArrayBridge 0.3 moved callable input, output, and execution memory metadata keys into its decorator contract; the catalogue now distinguishes package ownership from OpenHCS callable-contract projection and compiled resource planning.", - "python-introspect 0.1.10 added signature-derived callable-declaration projection to its generic owner boundary.", - "ArrayBridge 0.3.1 added framework-neutral array geometry so semantic consumers can inspect shape without implicitly transferring device arrays to the host.", - "Qt-rendered snapshot capture moved to pyqt-reactive; OpenHCS retains window discovery, authorisation, and domain transport.", - "Exact spawned endpoint-process ownership moved to ZMQRuntime; OpenHCS retains viewer persistence and application shutdown policy.", - "ZMQRuntime 0.2.15 made each retained endpoint process observe and reap its own terminal exit without caller polling.", - "ZMQRuntime 0.2.17 made owner-requested connection-attempt cancellation a typed generic result and disconnects a connection that wins a late readiness race.", - "The package catalogue did not name ArrayBridge child-process environment ownership or metaclass-registry host-logging-safe discovery imports.", - "The OMERO boundary assigned deployment to the alpha web application instead of the OpenHCS bundle and lifecycle owner." + "L1 moved the registry strategy/selection mixins and the cache family into metaclass-registry; the page still said OpenHCS owned the shared selection mixins.", + "L2 moved the to_jsonable encoder and the public-API/lazy-export helpers into python-introspect; the catalogue did not list them.", + "L3 moved the background-process launch policy into ZMQRuntime, and L5 added ArrayBridge's per-memory-type array operations; neither was in the catalogue.", + "The pyqt-reactive window_snapshot.rst authority was a documentation file; it was dropped in favour of the source owner already listed." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/external_foundations.rst in full and compared it against the changed authorities: pyproject.toml (zmqruntime 0.6, polystore 0.6, objectstate 1.3.1, python-introspect 0.2.5, metaclass-registry 0.3.0, arraybridge 0.4.1, pyqt-reactive 0.4 bounds; still eight first-party packages), the package READMEs, window_snapshot.py, client.py and arraybridge types.py; added the L1, L2, L3 and L5 ownership moves to the catalogue and confirmed the PolyStore, OMERO and pyqt-reactive claims against current source.", "Re-read docs/source/architecture/external_foundations.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: pyproject.toml; external/ObjectState/README.md; external/arraybridge/README.md; external/metaclass-registry/README.md; external/PolyStore/README.md; external/pyqt-reactive/README.md; external/python-introspect/README.md; external/zmqruntime/README.md; external/pycodify/README.md.", "Focused feature-lattice QA passed 65 tests covering framework resolution, execution scope, callable contracts, compiled execution, validation, and worker policy.", @@ -1328,7 +1422,7 @@ }, { "path": "docs/source/architecture/external_integrations_overview.rst", - "source_sha256": "cd950b20705e1661d76f3c2ba9a66e68a37a88020228083327b77a9e40b688f1", + "source_sha256": "b2e0a00a4927035f23c2efa7900cf2f30ec968c0acc7cf7dded57adddbf76895", "audience": [ "OpenHCS integrators", "architecture reviewers" @@ -1338,27 +1432,37 @@ "authority": [ { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "b547c838b2c86fe7cc5fe12d6cbf6a6715e83055c2d8c1a109ac22566d22c51e", - "role": "Implements CellProfiler lowering and declaration-owned headless validation before public OpenHCS pipeline declarations are emitted." + "sha256": "3439d96f3fbfcb104a53019dc150049021d9a8132bd27096f1ccf510eefbe37b", + "role": "Implements CellProfiler lowering to FunctionStep and PipelineConfig declarations with declaration-owned headless validation." + }, + { + "path": "openhcs/interop/cellprofiler/pipeline_importer.py", + "sha256": "618cc6bce994d1c5d323a7397930afbe6e7b3c65f6ecc9da53a94e1a61ff6c63", + "role": "Registers the .cppipe suffix as a PipelineImporter family member from interop." + }, + { + "path": "openhcs/core/pipeline_import.py", + "sha256": "b48f0a37e621f36fe256138472c4bda1e87d2d0d45be83361a40ce6cbfea4e55", + "role": "Defines the kernel's suffix-keyed PipelineImporter family." }, { "path": "openhcs/interop/cellprofiler/module_declarations.py", - "sha256": "89dd65abc003c464ba6c489f52d3a4fa47be1d5f8b87d26ad72b7537f188cf8b", + "sha256": "6bbf41c158496a02439be2ee5aef12d7712fb632419c226a64c9259a55438b95", "role": "Defines the nominal module validation hook queried by the importer." }, { "path": "openhcs/interop/cellprofiler/module_artifact_declarations.py", - "sha256": "666ec8c9adfd6a9015adf2b226df671534bb7e547cb0d76f1d16c40ac61e30a6", + "sha256": "f70b52114fb5d5e2c2fc49cd9a2b46986bde48ce02d82f1a34a67f072ca2ab20", "role": "Defines the interactive-module family that rejects headless import." }, { - "path": "openhcs/microscopes/microscope_base.py", - "sha256": "ae19ce4f6e1cd2e60946eceb908722e1ff760d7654a775d8372abbc4ac1fc81f", - "role": "Defines registered microscope discovery and source-specific declaration ownership." + "path": "openhcs/core/dataset_sources/source.py", + "sha256": "097fde4d2b74031eacf53a88e79f499417e8c1c97ba0753e8a9df831b307efa6", + "role": "Defines the registered DatasetSource family that microscope handlers implement." }, { "path": "external/PolyStore/src/polystore/base.py", - "sha256": "ae641affae21b61448816a14bd68b7f2f3fc0082f0f1fbe51079ed32520450b1", + "sha256": "8f2082ade15072bc8e8eba363c75a1d1cff7c67c74845e6485f69eab244634dc", "role": "Defines the generic storage backend boundary consumed by OpenHCS integrations." }, { @@ -1373,27 +1477,27 @@ }, { "path": "external/zmqruntime/src/zmqruntime/execution/server.py", - "sha256": "34e9e92c1a8a975a7a716d0c869b540e1ea8376c90cc71aa27d9614d08517a0d", + "sha256": "3afc85924853fdb245b2d7ed1e2d68e4a4262edcd84f36c355b9bfd9c4fad926", "role": "Implements the generic remote-execution lifecycle kept outside OpenHCS domain code." }, { "path": "openhcs/runtime/zmq_server_hooks.py", - "sha256": "b10f2a9a409ea1b97ad04ab4fa1fcea2fb41b9fbeaa43f7f31f0371be6ef484c", - "role": "Maps generic targeted interruption onto the exact OpenHCS orchestrator and its owned workers." + "sha256": "c3027f3495c5ec188a0f9176db7fd0fcbb22451892248c9fd41d73347ac236dd", + "role": "Maps generic targeted cancellation onto the exact OpenHCS orchestrator and its owned workers." }, { "path": "tests/unit/test_cellprofiler_pipeline_import.py", - "sha256": "353f0c1d8e0bc3552d2e418d792794ab0631daa16c78a3e10191b1cc1ea07d73", + "sha256": "6b1d201ec5c4ba620fa2367a9cbef1f6fbdd25edbdda9bd7b723ebe3a3363412", "role": "Verifies explicit rejection of an interactive-only module during direct import." } ], "findings": [ - "The overview named cancellation ownership but did not explain the generic terminal-state and OpenHCS cooperative-interruption boundary.", - "The OMERO summary assigned deployment to the web plugin instead of the OpenHCS deployment bundle and lifecycle owner.", - "The CellProfiler summary omitted the declaration-owned headless-import boundary for interactive-only modules." + "The page named ``MicroscopeHandler`` implementations; G4 deleted microscope_base.py and handlers now implement the kernel DatasetSource family.", + "The .cppipe importer is now a member of the kernel's PipelineImporter family registered from interop; the page did not say how the generic code reaches it." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/external_integrations_overview.rst in full and compared it against the changed authorities: pipeline_import.py, module_declarations.py, module_artifact_declarations.py, the deleted microscope_base.py, polystore/base.py, zmqruntime execution/server.py, zmq_server_hooks.py, test_cellprofiler_pipeline_import.py; replaced MicroscopeHandler with the DatasetSource family, named the PipelineImporter registration, and confirmed the CellProfiler, OMERO and cancellation claims and every linked page.", "Read the complete page and challenged accuracy, freshness, duplication, boundary, flow, and navigation against the listed current authorities.", "python scripts/validate_docs.py docs/source/architecture passed, and .venv Sphinx -E -W --keep-going rendered all 152 active pages without warnings.", "On 2026-08-25, re-read the complete explanation against the canonical Compose declaration, plugin compatibility owner, and PolyStore boundary; corrected OMERO deployment ownership.", @@ -1445,25 +1549,26 @@ "authority": [ { "path": "openhcs/runtime/fiji_stream_visualizer.py", - "sha256": "436a64ca81b9552c6a97acc9eea12cd509c7d746c5647c737060224cf2ea2632", - "role": "Implements the OpenHCS Fiji visualization adapter over compiled streaming declarations." + "sha256": "e2f24498a085bd5a3dc850b061386a0c0e3b0d3fc165413b5a9012b0825e47cc", + "role": "Implements the OpenHCS Fiji visualization adapter, keyed by the FijiViewer family member, over the shared managed-viewer lifecycle." }, { "path": "external/PolyStore/src/polystore/fiji_stream.py", - "sha256": "62a60a36f1da5668f60bc0b4e3a5d0e65008042b2361787f4909224dd9b51db9", + "sha256": "b1bd2ff8fc047e84dd24691d5fce0e60b25cdab06063b5889a95acca1d5ba0c1", "role": "Implements generic Fiji streaming storage mechanics." }, { "path": "external/zmqruntime/src/zmqruntime/viewer_protocol.py", - "sha256": "5580efffddca2f723b11a652f482fbdb52ebae657620fad95217b313997b9fc9", + "sha256": "118829a406428e38f0c04a3a6f54103dc110f4a330f69264900f3a4bb55e62d7", "role": "Defines the generic viewer-control transport protocol." } ], "findings": [ - "The former architecture page no longer owns streaming facts and correctly redirects to the current wrapper and package boundaries." + "The fiji_stream_visualizer.py change (FijiViewer family member replacing ViewerType.FIJI, start logic moved into the shared lifecycle) and the library changes do not affect the redirect; the page names no deleted owner and both linked pages exist." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/fiji_streaming_system.rst in full and compared it against the changed authorities: fiji_stream_visualizer.py, polystore/fiji_stream.py, zmqruntime/viewer_protocol.py; confirmed the redirect targets streaming_boundary_and_wrappers and external_foundations exist and the Fiji adapter still lives in OpenHCS over PolyStore and ZMQRuntime; no page edit was needed.", "Read the complete page and challenged accuracy, freshness, duplication, boundary, flow, and navigation against the listed current authorities.", "python scripts/validate_docs.py docs/source/architecture passed, and .venv Sphinx -E -W --keep-going rendered all 152 active pages without warnings.", "2026-09-22 PR #126 PolyStore-release follow-up: re-read the complete page against the PolyStore 0.2.18 `fiji_stream.py` change, which removed the subclass destructor and now relies on the shared `StreamingBackend` teardown authority for idempotent, destructor-safe cleanup. The page is a redirect stub with no subclass destructor or Fiji-internal lifecycle prose, so its reference disposition and navigation remain accurate. Refreshed only this compared evidence hash; details are retained in `docs/audits/pr126_authority_delta_review_20260922.md`." @@ -1480,8 +1585,8 @@ "authority": [ { "path": "external/pyqt-reactive/src/pyqt_reactive/animation/flash_mixin.py", - "sha256": "37cd2b32047109fc16255a3f9a9938ae553d5f8bd5987cd10460c7ba500f1486", - "role": "Implements reusable flash propagation, widget-geometry masking, paint acknowledgement, and animation coordination." + "sha256": "5b44c084784499a03c8eccffe37dbf31efc0e3379a6c99907272507f9100a492", + "role": "Implements reusable flash propagation, widget-geometry masking, label padding derived from animation config, and paint-acknowledged animation coordination." }, { "path": "external/pyqt-reactive/src/pyqt_reactive/animation/flash_config.py", @@ -1490,15 +1595,16 @@ }, { "path": "external/pyqt-reactive/tests/test_flash_destruction_lifetime.py", - "sha256": "a567e8b978182773d316e9a7dc7934b80da2708e2804bd8e84898b4b1a030b37", - "role": "Verifies Qt-owned cleanup receivers survive native sender destruction and Python garbage collection." + "sha256": "3d7f50d0677cc7deee95f879e9a5007dc7551453449f8299dc585499cea86cf3", + "role": "Verifies Qt-owned cleanup survives cycle collection and that a pending flash registration ends with the widgets it closes over." } ], "findings": [ - "The compatibility page lacked the exact owner-page link and pointed twice at the package-level boundary." + "The pyqt-reactive flash changes since the last review (pending registrations dropped on widget destruction, label padding derived from animation config) stay inside the generic owner; the redirect's owner link and OpenHCS boundary links remain correct, so only evidence roles changed." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/flash_animation_system.rst in full and compared it against the changed authorities: external/pyqt-reactive/src/pyqt_reactive/animation/flash_mixin.py and external/pyqt-reactive/tests/test_flash_destruction_lifetime.py; confirmed the pyqt-reactive flash page and the ui_services_architecture and external_foundations targets still exist and updated the evidence roles; the page text needed no change.", "Read the complete page and challenged accuracy, freshness, duplication, boundary, flow, and navigation against the listed current authorities.", "python scripts/validate_docs.py docs/source/architecture passed, and .venv Sphinx -E -W --keep-going rendered all 152 active pages without warnings.", "On 2026-09-06, re-read the complete redirect and canonical pyqt-reactive flash page against the native widget raster surface, shared mask projection, resolved-change notification ownership and unchanged FlashConfig timing. The redirect still points to the package that owns these generic mechanisms. Native RGB/BGR coverage passed 240 checks across 100%, 125% and 150% scaling; 460 reactive tests, nine OpenHCS form integration tests and five real MCP reset/edit checks passed. Reactive main CI 34044272928 passed every platform job.", @@ -1517,20 +1623,21 @@ "authority": [ { "path": "openhcs/core/function_patterns.py", - "sha256": "fe76286a2d378a39fd6061273363ccebbdc88f1dde6b1d36ed54b9cc28539e4a", - "role": "Defines callable, sequential-chain, and dictionary-fan-out patterns and derives execution memory through nested declarations." + "sha256": "95b9056c6980e4f7943f909aeae8262252c9aa175247a534befc8d4cc459fdd2", + "role": "Normalizes callable, (callable, kwargs), list-chain, and dict patterns into named invocations and compiles them with their memory and artifact plans." }, { "path": "tests/unit/test_function_patterns.py", - "sha256": "2b18e6f40f3f5e4a9673f954544b0113eba710675573fcda3fac6ef18d5eb725", - "role": "Verifies composition, flattening, validation, and dictionary pattern semantics." + "sha256": "d0160652d548748901a8b95d21abde9dca050259fd2f74d4a85cb60a50441175", + "role": "Verifies invocation identity for list and dict patterns, normalization, tuple-leaf validation, and compiled pattern semantics." } ], "findings": [ - "The former architecture URL is retained only as a stable transition to the current concept page." + "The redirect still points at the current concept page; function_patterns.py changed only in compiled-invocation detail. Evidence roles updated to the current module's normalized and compiled pattern classes." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/function_pattern_system.rst in full and compared it against the changed authorities: openhcs/core/function_patterns.py and tests/unit/test_function_patterns.py; confirmed the module still accepts callable, (callable, kwargs), list and dict patterns and that docs/source/concepts/function_patterns.rst exists; the page text needed no change.", "Re-read docs/source/architecture/function_pattern_system.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/function_patterns.py; tests/unit/test_function_patterns.py.", "Focused feature-lattice QA passed 65 tests covering framework resolution, execution scope, callable contracts, compiled execution, validation, and worker policy.", @@ -1549,31 +1656,31 @@ "authority": [ { "path": "external/python-introspect/src/python_introspect/signature_analyzer.py", - "sha256": "73600e546753640964d4ba5308fdf808ed3a4a6f085417d8ac0745d9e2748ed7", - "role": "Implements generic wrapped-target and parameter signature analysis." + "sha256": "edbe51f6c92b2c334dc4533bbeaa252be649c1ac738d2ebfca4364f2708479dc", + "role": "Implements generic wrapped-target and parameter signature analysis used by the OpenHCS editors." }, { "path": "external/python-introspect/src/python_introspect/callable_declaration.py", "sha256": "3bb8b474624fe6b2964a5c1f9a6aa540b2703500bfa124e7808995cb789d6951", - "role": "Orders supplied parameters by signature with optional default omission and caller-owned equality." + "role": "Provides callable_declaration_kwargs, which FunctionStep and the code serializer use to order supplied parameters by signature." }, { "path": "openhcs/core/callable_contract.py", - "sha256": "6a39b5e171901131ecb7280a78241e8ea72409a5003eabba3db34eb4851991d7", - "role": "Adds OpenHCS processing, runtime, artifact, scope, and ArrayBridge memory semantics to analysed callables." + "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", + "role": "Adds OpenHCS processing, runtime, artifact, scope, and ArrayBridge memory semantics on top of python-introspect annotation helpers." }, { "path": "tests/unit/test_callable_contract.py", - "sha256": "493f76a54f5fe024ca8f10c53b1f810b8c369626ad7141205b42ef3a78282137", - "role": "Verifies OpenHCS callable contracts projected from declaration-owned metadata, including execution memory." + "sha256": "cc220f53107a08b0406b6e0dd49896e3e4c3c0af6e527b0ef00d4fdaba8f0b2c", + "role": "Verifies OpenHCS callable contracts projected from declaration-owned metadata, including execution memory and runtime batching." } ], "findings": [ - "The transition page did not separate the extracted signature-analysis owner from OpenHCS domain callable semantics.", - "The extracted owner now also projects callable declarations from signature defaults." + "Re-checked the redirect against the rewritten callable_contract.py: generic signature analysis and declaration projection still come from python-introspect (SignatureAnalyzer, callable_declaration_kwargs, annotation helpers), and OpenHCS-specific callable semantics stay in callable_contract. No page text was wrong; evidence roles were refreshed." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/function_reference_pattern.rst in full and compared it against the changed authorities: external/python-introspect/src/python_introspect/signature_analyzer.py, openhcs/core/callable_contract.py and tests/unit/test_callable_contract.py; grepped OpenHCS imports of python_introspect and confirmed docs/source/architecture/processing_semantics.rst exists; the page text needed no change.", "Re-read docs/source/architecture/function_reference_pattern.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: external/python-introspect/src/python_introspect/signature_analyzer.py; openhcs/core/callable_contract.py; tests/unit/test_callable_contract.py.", "Focused feature-lattice QA passed 65 tests covering framework resolution, execution scope, callable contracts, compiled execution, validation, and worker policy.", @@ -1599,20 +1706,21 @@ "authority": [ { "path": "external/metaclass-registry/src/metaclass_registry/core.py", - "sha256": "ed4dd1dfd0e4e05e916fec39f19b4b8bd526e776aa0c96e3abe9b1ffbdff0505", + "sha256": "72c8284e8a32dd193f13f36a425289d5293a32f4a381046db523aa87182e0be8", "role": "Implements AutoRegisterMeta, RegistryConfig, families, and generic registry ownership." }, { - "path": "openhcs/core/registry_strategies.py", - "sha256": "a41186665abbd308f224d0bb41d6618cd9d89b08acaf54ec2819f2024a6e0804", - "role": "Defines OpenHCS generic selection over registered domain declarations." + "path": "external/metaclass-registry/src/metaclass_registry/strategies.py", + "sha256": "5c00cd4d80f9f0ceffbc2ab95ad8c6abd44a8a66a974959445ef93df08dd4f0d", + "role": "Provides the selection strategy mixins that OpenHCS declaration families import directly since the OpenHCS strategy module was deleted." } ], "findings": [ - "The transition page did not link the current metaclass-registry owner inventory directly." + "The listed authority openhcs/core/registry_strategies.py was deleted (b221f8491); its strategy mixins now live in metaclass_registry.strategies, which strengthens the page's claim that metaclass-registry owns the reusable mechanism. Authority replaced; page text unchanged." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/function_registry_system.rst in full and compared it against the changed authorities: external/metaclass-registry/src/metaclass_registry/core.py and the deleted openhcs/core/registry_strategies.py (moved to metaclass_registry.strategies and caches in b221f8491); confirmed no OpenHCS importer re-exports the old module and the nominal_ownership and external_foundations targets exist; the page text needed no change.", "Read the complete page and challenged accuracy, freshness, duplication, boundary, flow, and navigation against the listed current authorities.", "python scripts/validate_docs.py docs/source/architecture passed, and .venv Sphinx -E -W --keep-going rendered all 152 active pages without warnings.", "On 2026-08-25, re-read the complete page against metaclass-registry 0.2.1, ArrayBridge 0.3.3, the public native-catalogue projection, declaration-derived child environments, and the updated package floors; revised the page where those semantics are user-relevant and confirmed its remaining claims against current source and tests." @@ -1620,7 +1728,7 @@ }, { "path": "docs/source/architecture/gpu_resource_management.rst", - "source_sha256": "998a32e50c5ed6c59b44f3f7b0fc686b19fe619c63b459e0be7b273c555bc637", + "source_sha256": "812a3f6e083891191e39168946919f3633f4b5f80ea66de5fe65b40483c68ef9", "audience": [ "OpenHCS compiler and runtime maintainers", "GPU integration authors" @@ -1630,48 +1738,53 @@ "authority": [ { "path": "external/arraybridge/src/arraybridge/decorators.py", - "sha256": "99823105b0ae59054df1d84013633b8b6628d0a6dc3326383199ac7c65c69bed", + "sha256": "28f9bf4770bd4cc1ef34045fdef28065063ede029296f62b8a6c4140cc88e40e", "role": "Owns callable input, output, and execution memory metadata keys and their decorator projection." }, { "path": "external/arraybridge/src/arraybridge/types.py", - "sha256": "7b94760d57e54008d7adf4e4d37dca45d8a2c288a9dc8400be00c26148cf7624", - "role": "Declares framework capabilities, callable memory metadata, and the child-process import environment owned by each MemoryType member." + "sha256": "08f7ff4df8807bb3340e00a02c6276b1c702a0f00a2eedacd701142736c5d804", + "role": "Declares MemoryContractAttribute keys and each MemoryType member's import policy, device discovery, device scope, allocator cleanup, and subprocess_environment projection." }, { "path": "openhcs/core/callable_contract.py", - "sha256": "6a39b5e171901131ecb7280a78241e8ea72409a5003eabba3db34eb4851991d7", + "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", "role": "Projects declaration-owned input, output, and execution memory into immutable callable contracts." }, { "path": "openhcs/core/framework_device_resolver.py", "sha256": "be96430174bc576c0b8083879c579ef3fb9b83c4690234751459c2d7bc5556a8", - "role": "Resolves each required GPU framework through its ArrayBridge declaration and rejects CPU-only requests before imports." + "role": "Resolves each required GPU framework through its ArrayBridge declaration and rejects requests while GPU imports are disabled, before any framework import." }, { "path": "openhcs/core/pipeline/framework_device_assignment.py", - "sha256": "f77f3fb2d30c4af8cab6210dbe294f9fb5e246dad0a6bbf88bcaf1e2b58dead7", - "role": "Resolves the pipeline-wide framework footprint and projects exact bindings onto each step plan." + "sha256": "cdfde10f996ab682952a1d4948d3de2d23d48353eae41f3124770e19fc28a193", + "role": "Resolves the pipeline-wide framework footprint and projects exact bindings onto each step plan and compiled invocation." }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "ace8ae0c3e54bbe604b97cec7c3973a1abcf90e1dc517c2adda947dc2c35e882", - "role": "Invokes callable validation and framework-device assignment before assembling the execution bundle." + "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "role": "Calls assign_framework_devices on each context's step plans before bundle assembly." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "6d46571e6f2d069b98a92b41d711b2e20bfab9f0493cf73d3f150d5be08e7f96", - "role": "Owns framework-local device bindings, derived GPU footprints, execution scopes, and cleanup requests." + "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", + "role": "Owns FrameworkDeviceAssignment bindings, merged cleanup footprints, and the requires_gpu projection." }, { "path": "openhcs/core/orchestrator/worker_execution.py", - "sha256": "3c952465268e4b6c44676a853bb9c99d2d04a0490c7d398d2cdaf8b3b5f966d7", - "role": "Consumes compiled footprints for worker execution and lane-appropriate framework cleanup." + "sha256": "23a3ef2a70378dd1bb5c413a56e36bbb3fa013142741c9b45407bfc47de8c4a4", + "role": "Releases each axis's compiled footprint after process and inline contexts and the merged footprint only after thread-backed lanes join." }, { "path": "openhcs/core/worker_start_policy.py", "sha256": "9493c8729ddfc7457a3a58efb605457d5aa4a54cd42c3f4ef1bd9f28593d9475", - "role": "Projects the compiled GPU footprint into CUDA-safe multiprocessing start decisions." + "role": "Projects WorkerStartExecutionFacts into one start decision that requires spawn for GPU execution unless unsafe fork is explicit." + }, + { + "path": "openhcs/utils/environment.py", + "sha256": "2d8738e593b3e92c966089772624f47d17d38bb27fb90974707af659c91e8ad8", + "role": "Owns OPENHCS_CPU_ONLY and the CPU-only mode that --no-gpu enables before framework imports." }, { "path": "tests/unit/test_framework_device_resolver.py", @@ -1680,22 +1793,22 @@ }, { "path": "tests/unit/test_framework_execution_scope.py", - "sha256": "888a7693d387b0d5007c9868f0b0344fa3df26620f35e40b45832f0931ef7126", + "sha256": "ff92362a71f31b4b893a3816b1fdd396ed30307f45c7084270b4929c1aaa7aa4", "role": "Verifies execution-memory device scopes and exact per-lane allocator cleanup." }, { "path": "tests/unit/test_worker_start_policy.py", - "sha256": "a3eaadb10898f10e50391310d65d338d60e9df2f4f6dec8b791be7dc75d31fc9", + "sha256": "5a6f5e1ef54c49bc7f49c8b7baa530e685a8b76daf37679488ec50d9fb5fc37c", "role": "Verifies GPU-sensitive worker start policy and explicit unsafe overrides." } ], "findings": [ - "The former process-level GPU scheduler and GPU-memory validator were deleted; capability now descends from ArrayBridge declarations into a pipeline-wide framework assignment and exact per-step bindings, including execution memory.", - "Worker execution now consumes the compiled framework/device footprint for device scopes and cleanup instead of a scalar GPU identifier or copied framework registry.", - "The explanation did not state that MemoryType declarations project framework import requirements into fresh catalogue and execution-server processes." + "The page described parent/child framework import consistency with the jargon word 'admission'; reworded to plain terms.", + "Re-checked every ownership claim against the consolidation changes to callable_contract, compiler, compiled_step_plan and worker_execution: device resolution, per-step bindings, CPU-only rejection, subprocess_environment use by catalogue preparation (registry_service) and the execution-server client, worker-start policy and per-lane cleanup all remain as described. Added openhcs/utils/environment.py as the owner of the CPU-only switch." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/gpu_resource_management.rst in full and compared it against the changed authorities: arraybridge decorators.py and types.py, openhcs/core/callable_contract.py, openhcs/core/pipeline/framework_device_assignment.py, openhcs/core/pipeline/compiler.py, openhcs/core/compiled_step_plan.py, openhcs/core/orchestrator/worker_execution.py and the worker-start and execution-scope tests; traced assign_framework_devices from the compiler, FrameworkDeviceResolver's gpu_imports_disabled check, MemoryType.subprocess_environment() callers, _release_runtime_resources per axis and after threaded lanes, and resolve_worker_start_context; replaced one jargon phrase and refreshed evidence roles.", "Re-read docs/source/architecture/gpu_resource_management.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: external/arraybridge/src/arraybridge/decorators.py; external/arraybridge/src/arraybridge/types.py; openhcs/core/callable_contract.py; openhcs/core/framework_device_resolver.py; openhcs/core/pipeline/framework_device_assignment.py; openhcs/core/pipeline/compiler.py; openhcs/core/compiled_step_plan.py; openhcs/core/orchestrator/worker_execution.py; openhcs/core/worker_start_policy.py; tests/unit/test_framework_device_resolver.py; tests/unit/test_framework_execution_scope.py; tests/unit/test_worker_start_policy.py.", "Focused feature-lattice QA passed 65 tests covering framework resolution, execution scope, callable contracts, compiled execution, validation, and worker policy.", @@ -1756,27 +1869,36 @@ "sha256": "8e1243404095691030e9ebb95bb5611679f96c66c6b6819ae71e4b41249e74bc", "role": "Defines the public FunctionStep declaration named by the architecture invariants." }, + { + "path": "openhcs/core/config.py", + "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "role": "Declares GlobalPipelineConfig and the step variable_components and group_by fields, with group_by defaulting from the active AxisFamily." + }, + { + "path": "openhcs/core/processing_contracts.py", + "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", + "role": "Defines the ProcessingContract family named as an independent declaration in the invariants." + }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "3c840a89baccc8665d415175458280fd904c1a837b7fd879e303d7dbca129a1c", - "role": "Defines the scoped compiler authority named by the navigation map." + "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", + "role": "Defines CompilationSession, the scoped compiler state named by the invariants." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "6d46571e6f2d069b98a92b41d711b2e20bfab9f0493cf73d3f150d5be08e7f96", - "role": "Defines typed compiled plan ownership, including declaration-derived framework footprints and exact device bindings." + "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", + "role": "Defines typed CompiledStepPlan fields, including framework footprints and device bindings." }, { "path": "openhcs/core/compiled_execution.py", - "sha256": "941115f2dca2e53538d3141a59f3ae1f928a87cf741fcfa177f7cd64a5f41aea", - "role": "Defines the worker-safe runtime bundle boundary without a process-global GPU registry." + "sha256": "85972d418f0958e59f3185f07caf25a103bc9e101e481b2b63903377cc05e397", + "role": "Defines the CompiledExecutionBundle that runtime workers consume." } ], - "findings": [ - "The landing page cited an old documentation plan as a completed-disposition authority; active navigation now stands on current pages and source boundaries only." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/index.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compilation_session.py, openhcs/core/compiled_step_plan.py and openhcs/core/compiled_execution.py; verified each architecture invariant against the current declarations and that every toctree target file exists; the page text needed no change.", "Re-read docs/source/architecture/index.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/steps/function_step.py; openhcs/core/pipeline/compilation_session.py; openhcs/core/compiled_step_plan.py; openhcs/core/compiled_execution.py.", "Focused feature-lattice QA passed 65 tests covering framework resolution, execution scope, callable contracts, compiled execution, validation, and worker policy.", @@ -1786,7 +1908,7 @@ }, { "path": "docs/source/architecture/list_item_preview_system.rst", - "source_sha256": "59ec9b7019bcc8baae04e3cded674beb98cc808c040fd82f47852b253467d5ec", + "source_sha256": "fb9177bbbabb4168c1284df0209497cfb23fa9aaf981a248be6e0a15d33134c8", "audience": [ "OpenHCS UI maintainers following the extracted list-preview URL" ], @@ -1796,7 +1918,7 @@ { "path": "external/pyqt-reactive/src/pyqt_reactive/strategies/preview_formatting.py", "sha256": "dc1a423246ca40f13633847d5d3ffde3916e53c1a8a8a92de893d3e80ad8fda7", - "role": "Owns preview detail, collection bounds, and wrapping declarations." + "role": "Owns FormattingConfig preview detail, collection bounds, and wrapping declarations." }, { "path": "external/pyqt-reactive/src/pyqt_reactive/protocols/preview_formatter.py", @@ -1805,33 +1927,33 @@ }, { "path": "openhcs/core/config.py", - "sha256": "ef81b50ee22b8da69513fd50d6e45a929117fa4bfaa62a20ef08a4c02531c043", - "role": "Declares the WellFilterConfig family preview policy and WellFilterMode member operators." + "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "role": "Declares PartitionFilterConfig's show_preview_without_partition_filter policy and PartitionFilterMode members' preview_prefix operators." }, { "path": "openhcs/pyqt_gui/widgets/config_preview_formatters.py", - "sha256": "67b2cbdddc7ef1bced7d549ce5bd29ed7222cacb4d8ea255476ec36f5cab6f62", - "role": "Projects OpenHCS well-filter declarations through the generic preview API." + "sha256": "490bd6f32183c44028ae06a4a5ca092dd38b44c198ae623d232dc62c5f502183", + "role": "format_partition_filter_config projects the partition-filter declarations through the generic preview label API." }, { "path": "openhcs/pyqt_gui/services/reactor_providers.py", - "sha256": "e937c310e6dd7f06bf8e2cfc227ce442310f6d249fc0abed4ed0e588b4121fe8", - "role": "Registers the OpenHCS domain formatter with the generic preview service." + "sha256": "aecf1c77636095408486cab53a95cb0fc044fcf2ead0d7b72ac674842cf69abe", + "role": "Registers format_partition_filter_config for PartitionFilterConfig with the generic preview service." }, { "path": "openhcs/pyqt_gui/config.py", "sha256": "a3eece2e5b2926fc6fc0edd197221da451ab7c0d0d6f4ffc8bb2ff865ad74bd4", - "role": "Embeds the persisted generic list-preview policy in UIConfig." + "role": "Embeds FormattingConfig as UIConfig.list_previews." }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "a156a006a3c40f43107b9b1f6ecb97ed150879eb88ded8ef89e25214309ae33a", - "role": "Applies the same preview policy to both managers at startup and after configuration save." + "sha256": "e43085c1eaf0c836ce34e1a1dae7740488d91bdfea56844e7ddd4e6925561cf6", + "role": "_apply_list_preview_config applies list_previews to Plate Manager and Pipeline Editor at startup and on UI config change." }, { "path": "tests/unit/pyqt_gui/test_main_config_propagation.py", - "sha256": "291dc45ca20ead6025c255a8109a2e6c55d90bd7858e80ad12484dafa733d7c5", - "role": "Verifies live preview policy propagation through the main configuration lifecycle." + "sha256": "303d90b43bee7bd531a1c7664fbf9707a2e7be1696a5147614f93c40e5c7021f", + "role": "Verifies the list preview config reaches both real lists." }, { "path": "tests/unit/pyqt_gui/test_source_binding_preview_policy.py", @@ -1841,7 +1963,7 @@ { "path": "external/pyqt-reactive/src/pyqt_reactive/widgets/shared/list_item_delegate.py", "sha256": "282c8fe0c88f9bfd51df7eb95c739a8d428d2e9d99478796bb073c698f5907a7", - "role": "Owns per-row wrapping overrides, inherited view defaults, shared native/MCP toggling, and delegate paint acknowledgement." + "role": "Owns PreviewWrapMode per-row wrapping overrides, inherited view defaults, and delegate paint acknowledgement." }, { "path": "external/pyqt-reactive/src/pyqt_reactive/widgets/shared/styled_text_layout.py", @@ -1856,7 +1978,7 @@ { "path": "external/pyqt-reactive/src/pyqt_reactive/services/widget_tree_projection.py", "sha256": "05d5221d64e74ff14483f203975f17543381d959c30ee5918c200fa8e9957547", - "role": "Owns row action declarations, automatic selection and explicit preview toggling." + "role": "Declares TogglePreviewAction (item_preview_toggle), shared by native clicks and the MCP row action." }, { "path": "external/pyqt-reactive/tests/test_list_preview_wrapping.py", @@ -1865,13 +1987,12 @@ } ], "findings": [ - "The compatibility page lacked the exact owner-page link and duplicated a package-level boundary link.", - "The page did not identify the OpenHCS declarations that own well-filter preview labels, empty-selection visibility, and include or exclude operators.", - "Added the persisted generic preview policy and its shared two-manager application boundary; retained declaration-owned field selection.", - "Clarified row-arrow overrides and repeated graphical guides, distinguishing shared wrapping defaults from row-owned display choices." + "After the axis-family rename (WellFilterConfig to PartitionFilterConfig), the page still said an enabled member stays visible 'without an explicit well selection'; the declaration is show_preview_without_partition_filter. Corrected to 'partition filter'.", + "Evidence roles named the deleted WellFilterConfig/WellFilterMode; updated to the partition-filter declarations and the current formatter name." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/list_item_preview_system.rst in full and compared it against the changed authorities: openhcs/core/config.py, openhcs/pyqt_gui/widgets/config_preview_formatters.py, openhcs/pyqt_gui/services/reactor_providers.py, openhcs/pyqt_gui/main.py and tests/unit/pyqt_gui/test_main_config_propagation.py; confirmed PartitionFilterConfig/PartitionFilterMode preview declarations, the single formatter registration, UIConfig.list_previews, _apply_list_preview_config for both managers, and TogglePreviewAction; corrected one stale well-selection phrase.", "Read the complete page and challenged accuracy, freshness, duplication, boundary, flow, and navigation against the listed current authorities.", "python scripts/validate_docs.py docs/source/architecture passed, and .venv Sphinx -E -W --keep-going rendered all 152 active pages without warnings.", "On 2026-08-24, re-read the complete reference and its generic and domain authorities after moving preview semantics onto the WellFilterConfig family and WellFilterMode members; the page now names the projection boundary without duplicating a registry.", @@ -1885,7 +2006,7 @@ }, { "path": "docs/source/architecture/mcp_distribution.rst", - "source_sha256": "11d1d54561689e08a1b6c8ceaedfa1f4de7fd82dfa25e20e1b2ae813528ee0ce", + "source_sha256": "671e3066e2e1ff97544e6c08b35f0f088d5de4a85abdb912aee735d7c54e8c56", "audience": [ "OpenHCS MCP maintainers", "packaging and hosted-deployment integrators" @@ -1895,28 +2016,43 @@ "authority": [ { "path": "openhcs/agent/capabilities.py", - "sha256": "7fe9ab91490dedd6f750a57386e91835cf1067a27d10079966510aa9821adaf0", - "role": "Declares capability identity, services, side effects, security, transport availability, and surface profiles." + "sha256": "eaebe33af01d057d1d21b3701f958bc96ede3ec5d0bddddca9917c1164ef0042", + "role": "Declares AgentCapabilityDeclaration fields (service, side effects, security, data exposure, transport availability, progress heartbeat and worker-thread safety), CapabilityTransport semantics, local surface profiles, and one tool per headless session operation plus openhcs_session_events." + }, + { + "path": "openhcs/mcp/server.py", + "sha256": "e1f236011df715da3d17574c0b80cbf50fcae45e3f82491c22fb57e98c6bb662", + "role": "Binds the selected declaration registry into FastMCP with outputContract metadata and structured output, runs non-worker-safe capabilities through the main-thread dispatcher, and binds the session to the main thread." + }, + { + "path": "openhcs/mcp/execution.py", + "sha256": "2fcf0db56f5bd613844c8361962c04715aaa9bfb1b0f1ab191543a99522a06ad", + "role": "Runs the MCP SDK loop on a worker thread while a Qt core application and McpMainThreadDispatcher own main-thread calls." + }, + { + "path": "openhcs/mcp/stdio.py", + "sha256": "ec2935e01e9dc7c0ea56781c174f490528002bd736ac19062bd3a848f7397e7e", + "role": "Owns protocol stdin and stdout reservation, routes application output to stderr, and runs the transport through McpTransportExecutor." }, { - "path": "openhcs/mcp/dev_client_commanding.py", - "sha256": "e1c987624c76be157a6524cda985498351da3c77ef19c99b85e05dd5c2e9b970", - "role": "Validates one-shot development commands against the selected surface before starting an MCP subprocess." + "path": "tests/unit/agent/test_mcp_stdio_transport.py", + "sha256": "ffad185f736a7ead7b6e2e7699fb3c9b034c24534af14ea8848e0eb6989cfe1a", + "role": "Proves protocol isolation, cold native imports after handshake, and process-handle restoration." }, { - "path": "openhcs/mcp/dev_client.py", - "sha256": "cb91fcaf3040ff9bc41d6145b71aac8249b42b0369ce8547531cccf7f76dcb3a", - "role": "Validates commands before dispatch through an initialized persistent development session." + "path": "tests/unit/agent/test_mcp_distribution_runtime.py", + "sha256": "9ddc7dd1386813dc8199ff64ab1ecb0941527fbf62a9cab6577679730f94ea36", + "role": "Proves building the local and hosted MCP servers imports no PyQt6." }, { "path": "openhcs/mcp/dev_client_core.py", - "sha256": "6b05721536e92ed0f9ce4fdd0b6ebd4dbc75201277d1df1d625bf87456d5a57e", - "role": "Resolves tool identity and compatible profiles from the capability declarations at the shared call boundary." + "sha256": "41739b744b0e4287ca7d14cd133371d669e9bff12f4b622313996ae9bb5597da", + "role": "Resolves tool identity and compatible profiles from capability declarations and renews the command deadline on each response or progress notification." }, { - "path": "openhcs/mcp/server.py", - "sha256": "ed65a008b52db505393413f0fbadf6b096c648ff48340c7e2dda8584750f6c68", - "role": "Binds the selected declaration registry into the local stdio MCP server." + "path": "openhcs/mcp/dev_client_session.py", + "sha256": "2dceca7937087d05f72af1d037122ea393721d61d690803628a7b14ca70fd980", + "role": "Implements execute-source as add-datasets, set-pipeline, then initialize, compile and run stages followed through session events." }, { "path": "openhcs/mcp/control_timeout.py", @@ -1924,19 +2060,14 @@ "role": "Projects UI-bridge and viewer connection timeout declarations into bounded MCP caller policies." }, { - "path": "openhcs/agent/dto/ui_bridge.py", - "sha256": "585b1701b01b92bfce91437a60881afa769e36de9126807c3f4fcaf0ffc006fb", - "role": "Declares the UI bridge connection and its default control timeout." - }, - { - "path": "openhcs/agent/dto/viewer.py", - "sha256": "293ad9fcbd7570e74170e65f9e9d0c51eb78ead57f41213a2878ad6a95d055e3", - "role": "Declares viewer control requests and their default timeout." + "path": "openhcs/mcp/context.py", + "sha256": "1292c9bb7173e6afe5aa6237b861c5f39bc2e854f364fd722c75a3c6b76f534e", + "role": "Composes local stdio authoring and the headless session with the saved desktop execution endpoint while keeping hosted HTTP self-contained." }, { - "path": "openhcs/mcp/context.py", - "sha256": "7bb92e1357e02a0676243370179e5ff471d4e15c4c88b9a9f1d69b79f505c7cf", - "role": "Composes local stdio authoring with the persisted desktop execution endpoint while keeping hosted HTTP self-contained." + "path": "openhcs/authoring/session/session.py", + "sha256": "587047b63972df89ec46a01ed2c92ff3ef9577335f58a980297db02094e4e8e0", + "role": "Defines the headless Session whose datasets and pipelines live in the MCP process's ObjectState and whose events clients follow." }, { "path": "openhcs/pyqt_gui/config.py", @@ -1946,31 +2077,11 @@ { "path": "openhcs/core/config_document.py", "sha256": "37f66fd971806e059da6194c003e20a7ad08f69066eec4af7ebb11b4b70a08db", - "role": "Derives one typed nested field from the canonical configuration Python document and its dataclass schema." - }, - { - "path": "tests/unit/agent/test_mcp_distribution_runtime.py", - "sha256": "4f28f8eff46a5de20726bc80316f3fec276e51646b6dd133dbef7b39e6c00004", - "role": "Proves cold local MCP composition does not import PyQt6." - }, - { - "path": "tests/unit/test_config_document.py", - "sha256": "0ee12db4633ab39b34f630e4ee1b0c48b40d8226f91fddc504bb3311b9612cda", - "role": "Proves explicit, defaulted, and invalid typed field projections from canonical configuration source." - }, - { - "path": "openhcs/mcp/stdio.py", - "sha256": "4a5299915413293e5c2492d4e3889614231a1286f93a976094e4d2eba77ea9c0", - "role": "Owns protocol stdin and stdout reservation, isolates application input and native-library probes, and routes application output to stderr." - }, - { - "path": "tests/unit/agent/test_mcp_stdio_transport.py", - "sha256": "4f7c439168dacc350d7fc31618f10dfd7ce7478d52bd5ba13ed54e720291aff2", - "role": "Proves persistent protocol isolation, cold native imports after handshake, and process-handle restoration on normal and exceptional exits." + "role": "Derives one typed nested field from a configuration Python document and its dataclass schema." }, { "path": "openhcs/mcp/http.py", - "sha256": "56430a939d65607847486d417c693c91523bf5571cb969389c37ff917f4793c4", + "sha256": "c9de5befe7bbbcd6948749e4eafccb1364c0eec78930d55381ac9be0569a3618", "role": "Builds the stateless hosted Streamable HTTP server and rejects mutating hosted declarations." }, { @@ -1980,8 +2091,8 @@ }, { "path": "openhcs/mcp/client_registration.py", - "sha256": "28459b95d3b155778ce178f36302cb830958b74f2603980641ec492d81c41a9a", - "role": "Projects one stable launcher into registered local-client configurations." + "sha256": "55b701bb77a3de19243fa9a6276f784fad8329e2d435fb26fcc7fdce1e060731", + "role": "Projects one stable launcher into registered local-client configurations and synchronises the skill bundle into clients that declare a skill directory." }, { "path": "tests/unit/agent/test_hosted_mcp_semantic_ownership.py", @@ -1989,79 +2100,44 @@ "role": "Guards declaration-owned hosted transport selection and absence of copied tool lists." }, { - "path": "openhcs/desktop_deployment.py", - "sha256": "d5393386a14b693af3356484bdcb560e8a3cc0faea061352f46854c0cfead051", - "role": "Owns Windows stable-launcher publication and bounded deferral while a valid launcher is live." + "path": "openhcs/desktop/deployment.py", + "sha256": "4c6a7d63bf5ea759cc3eecd46e93fa335304113873869474f5fe6823f60c298f", + "role": "Defines DesktopDeployment and its Windows and macOS leaves, including bounded deferral of the live Windows launcher." }, { - "path": "tests/unit/test_desktop_deployment.py", - "sha256": "258a657c79a5ad96eacc06a9d4436018ffa10d7ca6d4ae56d5630e0a81c52238", + "path": "tests/unit/desktop/test_desktop_deployment.py", + "sha256": "6730f5346c91dc8b4f80656b9ab02012124b5ec9bdc4311f3f8d4932ac951ede", "role": "Proves live-launcher sharing violations defer only the cache projection and all unsafe failures remain fatal." }, { - "path": "openhcs/desktop_installation.py", - "sha256": "244ade0711356f68df9f376e2517dfe1c9fb77cc960c517f7c4e8ee418b5711f", - "role": "Nominally declares and validates native install policy, then projects one exact package selection for installers and staged updates." + "path": "openhcs/desktop/installation.py", + "sha256": "b2124aaa3e4ef4a310f91a51c1879de41de1818b25fc619cc7cdad888c4202b6", + "role": "Declares DESKTOP_INSTALL_PROFILE and projects one exact package selection for installers and staged updates." }, { "path": "scripts/render_installer_contract.py", - "sha256": "da9045e1012cb0a71ff0dfd2f2d9fc98232299f5a2f019b01228e344cf3b40e7", + "sha256": "1be39dfcc514e2f6f544e6907b3af0992485d3752877502fc761f8da760a7785", "role": "Derives the native contract from installer policy, project entry points, brand identity, and the exact release version." }, { "path": "external/pyqt-reactive/src/pyqt_reactive/services/window_snapshot.py", - "sha256": "c21e85912ab6715cc9c97bc796b9e5ed03610bf96f786d923557f18a876081f9", - "role": "Owns safe Qt-rendered capture and optional bounded observations of actual flash-renderer frames." + "sha256": "bced886e6bbfd69ae9bdebf950027b809e72c23eddadcaa5fcf7151adf7ae913", + "role": "Declares the widget and window capture scopes and owns safe Qt-rendered capture." }, { "path": "openhcs/pyqt_gui/services/ui_bridge_windows.py", - "sha256": "edb992542dffb88a9cc4557dbd6647b76d5b68584dca27a4c9486ef545be74e9", - "role": "Resolves authorised OpenHCS windows, projects generic snapshot results, and distinguishes focus from accepted target navigation in application DTOs." - }, - { - "path": "tests/unit/agent/test_mcp_server.py", - "sha256": "b99b6ca27be471f95ddf0852f8f94ce02797a459bf98bf6e0fa4851c0bb8c698", - "role": "Proves MCP registration, declaration-derived development-client surface preflight, and rejection of undeclared native capture." - }, - { - "path": "openhcs/demo/synthetic_data.py", - "sha256": "7a8f4a79e716ad944d5ceb1ed14e62cab9211aee65b1c0dc571e097e0f4d3c09", - "role": "Owns per-generator MT19937 state without changing seeded image generation or caller randomness." - }, - { - "path": "openhcs/agent/services/synthetic_plate_service.py", - "sha256": "2860714a9f1f2e66df403bc72331195d912cb180600889a1b4f4207642cc776b", - "role": "Owns complete synthetic generation, post-write plate inspection, and the typed result." - }, - { - "path": "tests/unit/agent/test_synthetic_plate_mcp_progress.py", - "sha256": "281197bf81d03a8e477026df293dbb385342d2a68c3d10c2897f6504ea85c74d", - "role": "Exercises real MCP subprocess progress and delayed post-inspection completion against an inactivity deadline, with a heartbeat-disabled timeout control." - }, - { - "path": "tests/unit/test_synthetic_generator_randomness.py", - "sha256": "6ab3efd7f4e083744dc4dda26d97c60b6a7649dd44432362020027b92271717f", - "role": "Proves pixel equivalence with the original global RNG leaf, caller-state isolation, and deterministic concurrent generators." - }, - { - "path": "openhcs/agent/services/llm_context_service.py", - "sha256": "f26b697684b185bd5ebe122de14f6211cf5377cc21f31c917cca53cea6b4e592", - "role": "Composes operating-guide sections from nominal context facets and bounds returned guidance." + "sha256": "e76298f78e3cbd949e8426a2b42cd6768cc54770a5f45c9dd6c9f788b257785e", + "role": "Resolves authorised OpenHCS windows, projects snapshot results, and distinguishes focus from accepted target navigation." }, { "path": "external/pyqt-reactive/src/pyqt_reactive/services/window_navigation.py", "sha256": "88da77ed07ea220b7c6874e71029b5bd50805fb2c3048701f9df773fe17257f2", - "role": "Owns typed target admission, the deferred exact-membership recheck, and optional terminal execution/exposure facts." - }, - { - "path": "external/pyqt-reactive/src/pyqt_reactive/widgets/shared/manager_selection_controller.py", - "sha256": "900fcec4c0a8e61b60130cc7a53ef8b0fd157d906182f96aa85fe57e5f008d46", - "role": "Owns item identity resolution and ordinary Qt list selection." + "role": "Owns typed target navigation and the deferred exact-membership recheck." }, { "path": "external/pyqt-reactive/src/pyqt_reactive/services/widget_tree_projection.py", "sha256": "05d5221d64e74ff14483f203975f17543381d959c30ee5918c200fa8e9957547", - "role": "Owns row action declarations, automatic selection and explicit preview toggling." + "role": "Owns bounded tree projection, row action declarations, automatic selection and explicit preview toggling." }, { "path": "external/pyqt-reactive/src/pyqt_reactive/core/deferred_callback.py", @@ -2069,57 +2145,53 @@ "role": "Owns QObject-parented delivery, target-destruction cancellation and callback retirement." }, { - "path": "external/pyqt-reactive/tests/test_deferred_callback.py", - "sha256": "1c59913d1b19a9725e5735bbd342dc115fdf4b555863427169b5001324e83b1c", - "role": "Real Qt regressions cover callback retirement, owner destruction, widget cancellation and persistent row identity." + "path": "openhcs/agent/services/synthetic_plate_service.py", + "sha256": "eb9e9ef693a877d65280004f8f85f1b6d6ab9be1c51aa3d2a0e1ed96630faa2d", + "role": "Owns complete synthetic generation, post-write plate inspection, and the typed result." }, { - "path": "benchmark/cellprofiler_benchmark_cli.py", - "sha256": "2b92755c556c1d66d28ffa0af6e2ac61cac5af4631514382290a2af0f907c26b", - "role": "Owns the registry-derived installed benchmark execution and report command surface." + "path": "openhcs/demo/synthetic_data.py", + "sha256": "be80973ecd87b4b3b36b7a7283d962a5d7cafd491cfa16eafe4622e09391463a", + "role": "Owns per-generator random state without changing seeded image generation or caller randomness." }, { - "path": "benchmark/contracts/run_receipt.py", - "sha256": "1704e1ab47411d25d382b95871c96c1e60c94db14b8ffd55f4dc6c8a917758b9", - "role": "Owns the shared comparison-run declaration, lifecycle enum, and validated receipt serialization and parsing." + "path": "openhcs/agent/services/llm_context_service.py", + "sha256": "ca5efbd7bfbd6a8aded1695b05abecba160fb817c04a14bc7c4a2df449ac5eaa", + "role": "Composes operating-guide sections from AuthoringContextSection facets and bounds returned guidance." }, { - "path": "benchmark/contracts/control.py", - "sha256": "0bce24c29ea9dfb7b50c88937b616118078505db6f497312fff87766397e5ce9", - "role": "Declares the lightweight typed benchmark inspection request, result, and artifact contracts imported by capability discovery." + "path": "benchmark/mcp_extension.py", + "sha256": "056ab5834fa3c37df1a36a37d80f2a821fb4b59c39892d7737e62fa6a57cb53b", + "role": "Declares the benchmark list, inspect, report, and measured-run finalise/inspect/report capabilities." }, { "path": "benchmark/control.py", - "sha256": "4706078a8fbeb516a52bc51600751ae86c39c3ef7e397a2f8cf13d49204db941", - "role": "Implements contained read-only receipt, streamed observation-progress, and structured-artifact inspection without importing benchmark execution modules." + "sha256": "83fe9016625d7d8721905b8a153a28c3dcaaf2b9e925a3c97711c634868156b7", + "role": "Implements contained read-only receipt, observation-progress, and artifact inspection without importing benchmark execution modules." }, { - "path": "tests/unit/test_benchmark_control.py", - "sha256": "ee75209799216c90cfa3a585dd9b7af99c99405f93e9fb5e4f900ea26d9d3ea0", - "role": "Proves receipt lifecycle, path policy, typed MCP projection, artifact discovery, and lightweight capability imports." + "path": "benchmark/control_service.py", + "sha256": "a1256b349ffdc8d04cab927326d97821462b61f57b29e564b61cc627302be49c", + "role": "Finalises measured-run evidence from the session's finished execution." + }, + { + "path": "benchmark/cellprofiler_benchmark_cli.py", + "sha256": "4025c591644e29f3df61e476ecae2143690e014336ea9f889aadeff4e692445e", + "role": "Owns the installed openhcs-benchmark execution, report and list-cases command surface." } ], "findings": [ - "The MCP adapter owned a separate 750-millisecond default and two-second cap for UI and viewer requests; it now derives both bounds from the corresponding connection declarations.", - "The page claimed multiple transport projections but described only local stdio; it now explains the declaration-filtered, read-only hosted HTTP lane and its security owner.", - "The desktop profile boundary now states that both UI-owned and independent headless execution are available while retaining separate state owners.", - "The Windows distribution explanation now distinguishes the stable launcher cache from the separately published current-environment authority and its bounded sharing-violation retry.", - "The page did not describe the screenshot privacy boundary; it now separates generic bounded Qt rendering from OpenHCS window discovery, authorisation, DTOs, and transport.", - "The native installer contract is now documented as a generated projection instead of a manually mirrored product declaration.", - "Service-local stdout guards could not protect the persistent stdio protocol from delayed imports or background native writes; the transport now owns the process channel for its full lifetime.", - "The window projection previously allowed focus to imply successful target navigation; it now reports navigation only from the registered nominal driver and returns a typed error for an unsupported target.", - "Dynamic Qt target lookup bypassed pyqt-reactive's live-window projection after discovery, so a deleted C++ window could escape through its stale Python wrapper; target resolution now uses the generic lifecycle boundary.", - "The development client allowed a command excluded by its selected profile to reach the MCP server and degrade into a low-level unknown-tool response; preflight and compatible profile names now derive from the capability declarations.", - "Constructing the complete UIConfig in a local MCP process pulled Qt-only shortcut validation across the headless boundary; MCP now projects the unique execution endpoint field from the canonical saved document and dataclass schema.", - "Local and hosted initialization instructions were dispatched again inside the MCP server; the CapabilityTransport declaration now delegates to a transport-owned nominal strategy.", - "Claude Desktop registration repeated Windows and macOS path decisions inside one target; registered platform leaves now own those paths while the target retains client identity, detection, and configuration format.", - "Separate UI and viewer timeout profile classes had become behaviorally identical after timeout derivation moved to the connection contracts; the generated binders now use the sole command policy directly.", - "The persistent development client wrapped a progress-aware MCP request in a second absolute wall-clock timeout; the duplicate ceiling was removed and the explanation now names transport inactivity as the client boundary.", - "Clarified generic row-action ownership, explicit preview toggling and selection as the automatic default.", - "The distribution explanation did not distinguish the packaged benchmark execution CLI from the expert local MCP capability that only inspects an existing authorized run directory." + "The local-lane section said the MCP server does not import PyQt; the stdio transport now keeps a QCoreApplication on the main thread and runs the SDK loop on a worker thread (McpTransportExecutor). Corrected, keeping the true claim that it hosts no GUI.", + "The liveness section said source-backed orchestrator-session creation runs on the main thread and that the event loop cannot emit progress meanwhile; that capability is gone (U1). Non-worker-safe capabilities now run through the main-thread dispatcher while the off-main SDK loop keeps emitting liveness. Rewrote the paragraph.", + "execute-source no longer creates a session and submits it for status polling; it adds the dataset to the headless session, sets its pipeline, and initializes, compiles and runs it, following openhcs_session_events (--no-wait returns on acceptance). Rewrote the session/execution paragraph around the per-operation session tools.", + "DesktopDeploymentAuthority was renamed DesktopDeployment and moved to openhcs.desktop.deployment (73a2aece2); corrected the name and replaced the three deleted desktop authorities.", + "Client registration now also synchronises the OpenHCS skill bundle into clients with a documented skill directory (Codex), reported separately; added this and narrowed 'agent instructions' to 'server instructions'.", + "The headless-session sentence implied no ObjectState state; the session keeps its datasets and pipelines in the MCP process's own ObjectState. Clarified that it creates no rows or history in a running desktop application.", + "Measured-run finalisation consumes the session's finished execution; corrected. Removed 'canonical' filler (three places)." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/mcp_distribution.rst in full and compared it against the changed authorities: openhcs/agent/capabilities.py, openhcs/mcp/server.py, stdio.py, context.py, http.py, client_registration.py, dev_client*.py, the deleted openhcs/desktop_deployment.py and desktop_installation.py (moved to openhcs.desktop in 73a2aece2), ui_bridge_windows.py, synthetic-plate and benchmark control owners and tests; verified every named symbol by grep, traced McpTransportExecutor, the binder's dispatcher path, SessionJourney.run_source and skill synchronisation; corrected the PyQt/threading, session-execution, desktop-deployment, client-registration and finalisation passages.", "Read the complete page and challenged accuracy, freshness, duplication, boundary, flow, and navigation against the listed current authorities.", "python scripts/validate_docs.py docs/source/architecture passed, and .venv Sphinx -E -W --keep-going rendered all 152 active pages without warnings.", "A focused 15-file architecture regression selection passed all 462 tests, including the authorities named for this page.", @@ -2172,7 +2244,7 @@ }, { "path": "docs/source/architecture/measurement_equivalence_system.rst", - "source_sha256": "f071f7effe7d94ef86cfcfaa24f479249fce185cba690c670ff247597aafa948", + "source_sha256": "55ba2895f65352f0b5404bb1730d42ff8a755b3e495777f66c9e592e407bcf96", "audience": [ "OpenHCS equivalence maintainers", "CellProfiler parity reviewers" @@ -2182,28 +2254,48 @@ "authority": [ { "path": "openhcs/core/equivalence/policy.py", - "sha256": "aaae502d1bb818ba38761374f02d8d4b5cf78a798bb2965f157216eba72cef79", - "role": "Declares tolerances, dialect, missing-value, stability, and feature-specific equivalence policy." + "sha256": "790d4fa07feac199d95acab616924e35ee95cc7861136e0de66c97683e3647e2", + "role": "Declares RuntimeEquivalencePolicy tolerances, MeasurementDialect selection, name mode, stability, and feature-specific numeric tolerances." }, { "path": "openhcs/core/equivalence/keys.py", - "sha256": "d2b2c2702dd5ad75c7df32905e7abc3ee4bb453c427503387fd210c2631dab5e", - "role": "Defines semantic measurement feature, row, source, and aggregate identities." + "sha256": "0ae79e4cc4e70804f9d2e948078ade7c271e142473821731b38eeccaef064b5d", + "role": "Defines RuntimeMeasurementFeatureKey and the semantic row, source, and aggregate identities." + }, + { + "path": "openhcs/core/equivalence/measurement_features.py", + "sha256": "18ff75cdd4dad0f88870b90b9a91f2ea352372773bcc9a74cee288d4c110f05b", + "role": "Defines the RuntimeMeasurementFeatureSemanticProfile most-derived strategy family." }, { - "path": "openhcs/core/equivalence/report.py", - "sha256": "de6381c0f804db72af16c9f96905b886aa9fa7b5138a34f867e871b701b62c36", - "role": "Defines typed difference records and the final RuntimeEquivalenceReport." + "path": "openhcs/core/equivalence/relationships.py", + "sha256": "fa257ac5280204b3f749fb581f98e80afea203dbd11fdb39c3cc4186d42625b5", + "role": "Defines the registered object-instance-key plane alignment strategies used for relationship comparison." + }, + { + "path": "openhcs/core/measurement_dialect.py", + "sha256": "96cac83e09e22eeaa02dbd14da2f9553c1f19f51962e83731936ba33429c4578", + "role": "Defines the registered MeasurementDialect family and PlainMeasurementDialect." + }, + { + "path": "benchmark/equivalence/report.py", + "sha256": "2711073c4039c38efcf69e2a07a88dac41bf2600337268019a896469f1a84b03", + "role": "Defines typed difference records and RuntimeEquivalenceReport outside the product package." + }, + { + "path": "tests/unit/test_equivalence_tooling_boundary.py", + "sha256": "91542052bd6ba03b0bac8899f09bf5e6f86db2341a3a0ff60b48f57caf1ce415", + "role": "Guards that no product module imports benchmark and that openhcs.core.equivalence keeps only production-reachable modules." }, { "path": "benchmark/adapters/openhcs.py", - "sha256": "177d91604f9be782b3fb9ed8641e0fb80d1c5182ad48d2e37e88c8bf3dc75596", - "role": "Builds ordinary benchmark runtime snapshots and strict CellProfiler policy, preserves unmodified fields of caller-declared path-planning, VFS, and consolidation policies while selecting benchmark overrides, and applies digest-bound reference-export comparisons." + "sha256": "8f84d9b70037c59558c2534b1ef49dd888c0f1d77d75eae44646c58173fb001d", + "role": "Builds benchmark runtime snapshots, the strict CellProfiler policy at 1e-6 tolerances, and digest-bound reference-export comparisons." }, { "path": "benchmark/openhcs_measured_run.py", - "sha256": "1e0cccc518fb6bec9c92ea8057dd7adfc7305b2d90a6da8f152c1b97601878ab", - "role": "Wraps the ordinary compile-and-execute operation with observation validation, execution-ID admission, phase timing, endpoint provenance, and the completed-run receipt; repeated measurements may reuse one connected client." + "sha256": "b7b2a3fd7b795db416005ad8b0b1661a960df0792c5d47e70de286d4cd08f9af", + "role": "Wraps the ordinary compile-and-execute submission with observation validation, phase timing from completed server records, endpoint provenance, and the completed-run receipt." }, { "path": "benchmark/contracts/run_artifacts.py", @@ -2212,37 +2304,42 @@ }, { "path": "benchmark/timing.py", - "sha256": "00b16a0e95fbc6164e6ab761f1a397441cc71b2699eaf046f46745658a448a8e", - "role": "Owns benchmark phase declarations, disjoint additive totals, and validated durations from completed ordinary server jobs." + "sha256": "5a98e0a02184d0b7f42651361d55fccaad0fc9fcc7112d985cbfe60317718f2d", + "role": "Owns benchmark phase declarations and additive totals that exclude nested runtime observations and benchmark validation phases." }, { "path": "benchmark/control_service.py", - "sha256": "bf2b39696a49d8c87cf74d39770d63013972b9c8cf884863c467ccdf46eb12d3", - "role": "Finalises benchmark evidence from the ordinary execution service's exact completed submission and job record." + "sha256": "a1256b349ffdc8d04cab927326d97821462b61f57b29e564b61cc627302be49c", + "role": "Finalises benchmark evidence from the OpenHCS session's finished execution: its request, endpoint handshake, and completion record." + }, + { + "path": "benchmark/control.py", + "sha256": "83fe9016625d7d8721905b8a153a28c3dcaaf2b9e925a3c97711c634868156b7", + "role": "Derives unreceipted_artifacts and retained_evidence_valid during measured-run inspection." }, { "path": "openhcs/runtime/zmq_execution_observation.py", - "sha256": "e52b2e9784daa396b493f1f569fbcbb0d394e7e3c03d1a0c67d6594778ce64d3", - "role": "Declares ordinary value and outcome exports, compiled axis-owned value expectations, producing execution identity, and exclusive evidence-file creation." + "sha256": "3cc43d3eee6c4701c23fec183ecd895a71acdfcc14acfe2afca62625ccb7381b", + "role": "Declares value and outcome exports with required compiled axis membership and rejects exports from other schema versions." }, { "path": "benchmark/contracts/measured_run_receipt.py", - "sha256": "67c71873d050e05c0e4a1e231c46b700a765c12cade51269553cc7a5cb96989e", - "role": "Declares the typed single-pipeline success receipt and its stable serialisation boundary without mirroring runtime job status." + "sha256": "c04b69926f1994fd7528eadf172a31ef33cf409380fef60b659d5f27f522f065", + "role": "Declares the typed single-pipeline receipt, accepting archived receipts without retained-artifact digests and requiring them on current receipts." }, { "path": "benchmark/manifests/official30_portable_axis1.json", - "sha256": "7f11549c0bc870f9ce2792afc9176f52b6a2b83236e47b33cbedb4a2cac73267", + "sha256": "1be9bb67ea05d78fd1e0b8991b994ff5635da2fff0358302b8e0df888e62888e", "role": "Defines the exact portable 30-case acceptance corpus." }, { "path": "tests/integration/test_cellprofiler_official30_zmq.py", - "sha256": "bc5abfddf076996a050b0601c46b6039d2d812ab57cd303847b241e477176f53", - "role": "Implements the headless and non-persistent Napari acceptance routes over all manifest cases." + "sha256": "dbd6e113a7135901f936e9bc8278d727d678e9971a7c951fced9080adfd9d6df", + "role": "Implements the headless and non-persistent Napari acceptance routes over all manifest cases, plus Fiji variant tests." }, { "path": "benchmark/cellprofiler_reference_exports.py", - "sha256": "604f9a95f94eeb10365437b875a0ff3d558dbb2fda13e7e951242d417a69b49c", + "sha256": "58cdcb7572266d562314ccae5bf9d98b5f3a4b19944e8a068f89b617219952f3", "role": "Owns terminal-artifact exporter derivation, exact output-inventory selection, numeric-image tolerance, categorical-label equality, and explicit single-plane label projection." }, { @@ -2250,28 +2347,23 @@ "sha256": "8b84a9706af7c9226ef93a96a3fdb9c1c16a8f220fbee4a65a53f6e15bc7cb61", "role": "Proves digest and inventory failures remain fail-closed and verifies the two declared per-artifact comparison contracts." }, - { - "path": "tests/unit/test_openhcs_adapter.py", - "sha256": "2f117fab8c889fb50a2ee734fe802b0ac6cbcbe9945a33d24c1482fbcd7eb955", - "role": "Proves validated reference-export comparisons and retention of caller-declared path-planning, VFS, and consolidation settings not overridden by benchmark policy." - }, - { - "path": "tests/unit/test_openhcs_measured_run.py", - "sha256": "ebb8b7741eec1130969325ed30f425752f6b47aaa59a5c03655002bc3bb53690", - "role": "Proves ordinary non-CellProfiler observation validation, execution-ID admission, pre-connect rejection, and distinct receipts for repeated measurements on one client." - }, { "path": "tests/unit/test_benchmark_timing.py", - "sha256": "0b25284f3404072758fed50c9818e41f8d25e9c9b7fdacb193afd198cf741e1d", - "role": "Proves completed-job duration validation and that overlapping progress and server intervals are excluded from additive phase totals." + "sha256": "a6d64635ab230da628c417a08e482f95b1c2a90ba6b9389abb21a459ce42da89", + "role": "Proves completed-job duration validation and the additive-total exclusions." } ], "findings": [ - "The canonical Linux acceptance command hardcoded one developer's /home cache path; it now uses the XDG cache root with a portable home fallback.", - "Measured-run inspection now names retained_evidence_valid and states that it verifies retained receipt and observation evidence, not pipeline image or table outputs." + "The page named RuntimeMeasurementDialect, which no longer exists; dialects are now the registered MeasurementDialect family (PlainMeasurementDialect in the kernel, CellProfilerMeasurementDialect in interop). Corrected.", + "RuntimeEquivalenceReport and the comparison half moved from openhcs/core/equivalence to benchmark/equivalence (f16177ed7); the page did not say where comparison lives. Added the product/benchmark boundary and replaced the deleted report.py authority.", + "The page said archived outcome-only exports without compiled axis membership remain readable and archived observation exports keep an all-axis expectation; the legacy readers were deleted and exports from other schema versions are now rejected. Corrected both passages.", + "Agent finalisation is supplied by the OpenHCS session's finished execution (U1), not an execution service and submission. Corrected.", + "Additive phase totals now also exclude benchmark validation phases; added.", + "'The two integration tests' was imprecise now that the module also holds Fiji variant tests; named the headless and Napari tests. Removed 'canonical' filler." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/measurement_equivalence_system.rst in full and compared it against the changed authorities: openhcs/core/equivalence/policy.py and keys.py, the deleted openhcs/core/equivalence/report.py (moved to benchmark/equivalence in f16177ed7), benchmark/adapters/openhcs.py, openhcs_measured_run.py, timing.py, control_service.py, openhcs/runtime/zmq_execution_observation.py, measured_run_receipt.py, the Official30 manifest and integration test, reference exports and their tests; corrected the dialect name, the comparison-package boundary, archived-export readability, the session-based finalisation path and the additive-total rule.", "Read the complete page and challenged accuracy, freshness, duplication, boundary, flow, and navigation against the listed current authorities.", "python scripts/validate_docs.py docs/source/architecture passed, and .venv Sphinx -E -W --keep-going rendered all 152 active pages without warnings.", "A focused 15-file architecture regression selection passed all 462 tests, including the authorities named for this page.", @@ -2309,13 +2401,13 @@ "authority": [ { "path": "external/arraybridge/src/arraybridge/decorators.py", - "sha256": "99823105b0ae59054df1d84013633b8b6628d0a6dc3326383199ac7c65c69bed", - "role": "Owns callable input, output, and execution memory metadata keys and decorator semantics." + "sha256": "28f9bf4770bd4cc1ef34045fdef28065063ede029296f62b8a6c4140cc88e40e", + "role": "Owns the memory decorators that write callable input, output, and execution memory metadata." }, { "path": "external/arraybridge/src/arraybridge/types.py", - "sha256": "7b94760d57e54008d7adf4e4d37dca45d8a2c288a9dc8400be00c26148cf7624", - "role": "Declares reusable memory types together with framework capability and device behaviour." + "sha256": "08f7ff4df8807bb3340e00a02c6276b1c702a0f00a2eedacd701142736c5d804", + "role": "Declares MemoryType members with framework capability and device behaviour, and MemoryContractAttribute's three metadata keys." }, { "path": "external/arraybridge/src/arraybridge/converters.py", @@ -2324,13 +2416,13 @@ }, { "path": "openhcs/core/callable_contract.py", - "sha256": "6a39b5e171901131ecb7280a78241e8ea72409a5003eabba3db34eb4851991d7", - "role": "Projects ArrayBridge metadata into immutable callable input, output, and execution memory contracts." + "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", + "role": "Reads ArrayBridge metadata through MemoryContractAttribute into immutable callable input, output, and execution memory contracts." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "6d46571e6f2d069b98a92b41d711b2e20bfab9f0493cf73d3f150d5be08e7f96", - "role": "Derives exact step framework footprints and owns framework-local device bindings." + "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", + "role": "Owns FrameworkDeviceAssignment framework-local device bindings and footprint cleanup." }, { "path": "openhcs/core/framework_device_resolver.py", @@ -2339,11 +2431,11 @@ } ], "findings": [ - "The transition page did not link the exact ArrayBridge owner inventory or the current OpenHCS planning explanation.", - "The transition pointed to the deleted GPU-memory validator; current OpenHCS ownership is callable-contract projection, compiled framework bindings, and declaration-derived capability resolution." + "Re-checked the redirect against the consolidation changes to callable_contract.py and compiled_step_plan.py and the current ArrayBridge declarations; ArrayBridge still owns memory types, conversion and the input/output/execution metadata keys, and OpenHCS still owns contract projection, compiled framework-device plans and runtime adapters. No page text was wrong; evidence roles were refreshed." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/memory_type_system.rst in full and compared it against the changed authorities: external/arraybridge/src/arraybridge/decorators.py, types.py, openhcs/core/callable_contract.py and openhcs/core/compiled_step_plan.py; confirmed MemoryContractAttribute's INPUT/OUTPUT/EXECUTION keys, FrameworkDeviceAssignment, openhcs/core/runtime_adapters.py and the gpu_resource_management and external_foundations targets; the page text needed no change.", "Re-read docs/source/architecture/memory_type_system.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: external/arraybridge/src/arraybridge/decorators.py; external/arraybridge/src/arraybridge/types.py; external/arraybridge/src/arraybridge/converters.py; openhcs/core/callable_contract.py; openhcs/core/compiled_step_plan.py; openhcs/core/framework_device_resolver.py.", "Focused feature-lattice QA passed 65 tests covering framework resolution, execution scope, callable contracts, compiled execution, validation, and worker policy.", @@ -2359,57 +2451,69 @@ }, { "path": "docs/source/architecture/microscope_handler_integration.rst", - "source_sha256": "7568ee810dd5ff9f13643ee2434a239e6e79d42d0e3dd57d2690b406d41b254f", + "source_sha256": "485cbdb1bbd21e8d5266ed571a66435ba97b294b56f271fc00841a5340b80534", "audience": [ "OpenHCS source-model maintainers", "microscope integration authors" ], - "user_need": "Understand how registered microscope handlers own detection, metadata, source projection, and backend compatibility.", + "user_need": "Understand how registered dataset sources own detection, metadata, source projection, and backend compatibility.", "diataxis": "explanation", "authority": [ { - "path": "openhcs/microscopes/microscope_base.py", - "sha256": "ae19ce4f6e1cd2e60946eceb908722e1ff760d7654a775d8372abbc4ac1fc81f", - "role": "Defines the handler root, registry-backed factory, uniform plate context, detection, and declaration-owned workspace-backend replay boundary." + "path": "openhcs/core/dataset_sources/source.py", + "sha256": "097fde4d2b74031eacf53a88e79f499417e8c1c97ba0753e8a9df831b307efa6", + "role": "Defines the DatasetSource family keyed by source_name, its selection-role mixins, detection_order, create hook, workspace backend registration, and virtual-workspace metadata recording." }, { - "path": "openhcs/microscopes/openhcs.py", - "sha256": "a8d416d4ac89e01f2a41a559632f62b67a06e6294833ea9d0af82fdec6787e7e", - "role": "Replays a persisted workspace through the source handler type recorded by the canonical OpenHCS metadata." + "path": "openhcs/core/dataset_sources/choice.py", + "sha256": "7fb3c7ce701f0d629f288b79090044f43ac8ce1b51ca4f50937534407f226f1a", + "role": "Defines DatasetSourceChoice, AutoDetectedSource, and open(), which selects the source class, applies declared-file selection, calls create, and binds dataset_root." }, { - "path": "openhcs/microscopes/microscope_interfaces.py", - "sha256": "a603fa1c58ba005f8ba22713e46efe4e33aee0f762ba188467482ae3ae429200", - "role": "Declares the single complete metadata-component projection implemented by microscope metadata handlers." + "path": "openhcs/core/dataset_sources/discovery.py", + "sha256": "d3c2cd7a43dce30ab3f74a9fcfb1f1d7be389f4ab101f867ac5a2fe418a16c93", + "role": "Discovers kernel members and imports the active AxisFamily's extension_modules without naming a domain module." }, { - "path": "openhcs/core/source_bindings.py", - "sha256": "e0afbbe40cce3847bcc2314910d0083ecc579a5a0d6e4278c3a5df4c62e7eb01", - "role": "Declares named source bindings independently of concrete microscope handlers." + "path": "openhcs/core/dataset_sources/interfaces.py", + "sha256": "dd88254bfd5da8b0fdf0a038e358f36b49db5b8cf0b93213d56dc928af965f00", + "role": "Defines MetadataHandler.component_value_set and MetadataComponentValueSet keyed by the active family's axes." }, { - "path": "tests/unit/test_microscope_handler_identity.py", - "sha256": "3ed9e436b4d9ce2770f5eeed5c884c7621f22eab665cd3cc9280df95d3e10ba8", - "role": "Verifies registered identity, explicit construction, auto-detection, and invalid handler failure." + "path": "openhcs/core/dataset_sources/openhcs_format.py", + "sha256": "a8fa6a597104075d888a5b14cc4b24a9676d1a3918c30b5df2bad4a9abb26078", + "role": "Reopens a persisted workspace by looking up the recorded source_name and re-registering that source's workspace backends." }, { - "path": "tests/unit/test_bioformats_microscope_handler.py", - "sha256": "d12e4e85a13bc28f7d67d1547ce8a347897c89436a129aebf45bc34cd5a193d6", - "role": "Verifies generic plate-folder binding and BioFormats handler metadata behaviour." + "path": "openhcs/core/config.py", + "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "role": "Declares PipelineConfig.dataset_source as a DatasetSourceChoice class defaulting to AutoDetectedSource." + }, + { + "path": "openhcs/microscopes/bioformats.py", + "sha256": "bda05411746852ea12ffe11b49e5298cb475e2137cd1a09cf9659b52a89f40e4", + "role": "Shows a decoder-owned source metadata handler replaced by the workspace metadata handler after the workspace is written." }, { - "path": "tests/unit/agent/test_plate_inspection_service.py", - "sha256": "fafd340d60d66e21ca0ac42cdc1a7f0fee5672788fa219cf7f7a7fefe8841194", - "role": "Verifies automatic inspection can reopen both native and SourceBindings virtual workspaces from persisted metadata." + "path": "tests/unit/test_dataset_source_family.py", + "sha256": "4cb651b475e5e033ab94824567b8a35b0ee02f313254082e03e35cd96585d829", + "role": "Verifies every registered source declares identity, selection role and backends, and that metadata detection and the factory use declared identity." + }, + { + "path": "tests/unit/test_dataset_source_guards.py", + "sha256": "adfeac23f6196a0c5393dc15b71df04fda0ec34e5ae23bfdaa22e0a165f68170", + "role": "Guards that the kernel imports no domain package and that the deleted microscope-handler names do not return." } ], "findings": [ - "The audit exposed a factory-side concrete class-name branch for OpenHCS plate context and an embedded extension procedure; the root now binds plate_folder uniformly and the page links to the task guide.", - "Prepared SourceBindings workspaces could not be inspected through automatic OpenHCS metadata detection because replay reconstructed an ingestion instance without its original configuration; the handler declaration now owns backend replay directly.", - "Microscope metadata exposed one method per concrete axis; handlers now return one complete component value set keyed by the canonical declaration." + "The page documented MicroscopeHandler, create_microscope_handler, MicroscopeHandler.__registry__, the Microscope enum with AUTO, and a factory binding plate_folder; G4 (c6d9a52ce) replaced these with the DatasetSource family keyed by source_name, PipelineConfig.dataset_source holding a DatasetSourceChoice (AutoDetectedSource or a registered source), selection-role mixins, and DatasetSourceChoice.open binding dataset_root. Rewrote the page in the current vocabulary and retitled it 'Dataset source integration' (URL unchanged).", + "Discovery is no longer plain package discovery of openhcs/microscopes: the kernel discovers its own members and imports the active AxisFamily's extension_modules. Added this and placed the extension boundary on those modules.", + "Removed 'canonical' and 'original' filler wording.", + "Four listed authorities were deleted by G4 (microscope_base.py, microscopes/openhcs.py, microscope_interfaces.py, test_microscope_handler_identity.py); replaced with the openhcs/core/dataset_sources owners and the family and guard tests." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/microscope_handler_integration.rst in full and compared it against the changed authorities: the deleted openhcs/microscopes/microscope_base.py, openhcs.py and microscope_interfaces.py (moved to openhcs/core/dataset_sources in c6d9a52ce), openhcs/core/source_bindings.py and the Bio-Formats and plate-inspection tests; traced DatasetSource registration and discovery, DatasetSourceChoice.open, detection_order, PipelineConfig.dataset_source, component_value_set, and workspace replay through the recorded source name; rewrote the page in DatasetSource vocabulary.", "Re-read docs/source/architecture/microscope_handler_integration.rst in full against repository base 32ca17c17 and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/microscopes/microscope_base.py; openhcs/core/source_bindings.py; tests/unit/test_microscope_handler_identity.py; tests/unit/test_bioformats_microscope_handler.py.", "Focused feature-lattice QA passed 65 tests covering framework resolution, execution scope, callable contracts, compiled execution, validation, and worker policy.", @@ -2424,7 +2528,7 @@ }, { "path": "docs/source/architecture/multiprocessing_coordination_system.rst", - "source_sha256": "816ae37a0aa964c708bf3b7855583b1e0ce706571a231df60caf1405eecee5ee", + "source_sha256": "d8e12a688c5e7b27a98ee51b9e9261176a4a6181a9e7321ef02f3068bf2d2f76", "audience": [ "OpenHCS compiler and multiprocessing maintainers" ], @@ -2433,23 +2537,33 @@ "authority": [ { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "ace8ae0c3e54bbe604b97cec7c3973a1abcf90e1dc517c2adda947dc2c35e882", - "role": "Compiles source execution-axis and framework-device facts into frozen contexts and typed plans." + "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "role": "Discovers partition-axis values through the active AxisFamily and compiles one frozen ProcessingContext per value." }, { - "path": "openhcs/core/compiled_execution.py", - "sha256": "941115f2dca2e53538d3141a59f3ae1f928a87cf741fcfa177f7cd64a5f41aea", - "role": "Assembles and validates the one worker-safe execution bundle without a process-global GPU registry." + "path": "openhcs/core/orchestrator/bundle_execution.py", + "sha256": "f5ba09a2ec7c688b1ca08167ab398cc976a9e2c39b301ae98f3c5fd560a25c85", + "role": "Validates compiled contexts and worker-safe steps before worker setup and builds the worker-lane execution plan." + }, + { + "path": "openhcs/core/orchestrator/worker_lanes.py", + "sha256": "d915c01a0ae2dd010d6b972e275669fef63bb3aefd100f1b66a1e34653673db0", + "role": "Plans deterministic worker lanes from compiled axis contexts and installs the bundle in workers through ForkInheritedWorkerExecutionState." }, { "path": "openhcs/core/orchestrator/worker_execution.py", - "sha256": "3c952465268e4b6c44676a853bb9c99d2d04a0490c7d398d2cdaf8b3b5f966d7", - "role": "Installs the bundle, submits one task per compiled axis, collects typed results, and performs footprint-aware cleanup." + "sha256": "23a3ef2a70378dd1bb5c413a56e36bbb3fa013142741c9b45407bfc47de8c4a4", + "role": "Submits one task per active lane, collects per-axis ExecutionResult values, ships or inherits the bundle, and terminates owned process workers on cancellation." + }, + { + "path": "openhcs/core/compiled_execution.py", + "sha256": "85972d418f0958e59f3185f07caf25a103bc9e101e481b2b63903377cc05e397", + "role": "Defines CompiledExecutionBundle with runtime contexts and pickle-safe transport snapshots." }, { "path": "openhcs/core/orchestrator/cancellation.py", "sha256": "f825b602fb020a23d602e1f93ed9cecea881eca16e07882fd7a91ab519445244", - "role": "Defines the parent-owned cancellation scope observed directly by inline and threaded lanes." + "role": "Defines the execution-scoped cooperative cancellation signal observed by inline and threaded lanes." }, { "path": "openhcs/core/worker_start_policy.py", @@ -2458,10 +2572,14 @@ } ], "findings": [ - "The page incorrectly described cancellation as a cross-process channel instead of a parent-owned interruption boundary." + "Step 5 said the executor submits one task per compiled axis; PooledWorkerLaneRunner submits one task per active worker lane, and a lane can own several axis contexts. Corrected.", + "The page attributed axis discovery to a 'microscope/source model' and component identities to 'nominal component declarations'; after G1/G4 the dataset source discovers the active AxisFamily's partition-axis values. Corrected the vocabulary.", + "'Worker-lane authority' and 'the execution server's worker ownership boundary' were vague; named ForkInheritedWorkerExecutionState (fork-inherited or shipped bundle) and the executor resources that terminate their own processes.", + "Bundle validation lives in orchestrator/bundle_execution.py, not compiled_execution.py; added it and worker_lanes.py as authorities." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/multiprocessing_coordination_system.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compiler.py, openhcs/core/compiled_execution.py and openhcs/core/orchestrator/worker_execution.py; traced _axis_values_to_process through AxisFamily.active().partition_axis(), validate_bundle_execution, CompiledContextLanePlanner, PooledWorkerLaneRunner._submit_lanes, ExecuteShippedLane and ProcessWorkerExecutorResources.cancel_execution; corrected the task granularity, axis vocabulary, bundle installation and cancellation wording.", "Re-read docs/source/architecture/multiprocessing_coordination_system.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/pipeline/compiler.py; openhcs/core/compiled_execution.py; openhcs/core/orchestrator/worker_execution.py; openhcs/core/orchestrator/cancellation.py; openhcs/core/worker_start_policy.py.", "Focused feature-lattice QA passed 65 tests covering framework resolution, execution scope, callable contracts, compiled execution, validation, and worker policy.", diff --git a/docs/audits/architecture_n_z_development.json b/docs/audits/architecture_n_z_development.json index 3d583b9137..2470b38114 100644 --- a/docs/audits/architecture_n_z_development.json +++ b/docs/audits/architecture_n_z_development.json @@ -10,12 +10,12 @@ "authority": [ { "path": "openhcs/runtime/napari_viewer_server.py", - "sha256": "64ccfef65a10e04878ef0806ae72d0446540efcbfc72af46c4e2ab785418a84c", - "role": "Detached Napari endpoint startup and application event-loop integration." + "sha256": "038d7a76eeac8d69079591cf6eb7736af5415b28a7a067d6677bd15d115d11c7", + "role": "Detached Napari endpoint: accepted stream batches, layer routing, variable-size display strategies, and dimension-label overlays." }, { "path": "openhcs/runtime/napari_streaming_handlers.py", - "sha256": "86f4ab319c7c68b626e7e2a1adef81eb3d7244f044b5b3e4dddd78bbee2aa4b6", + "sha256": "be585484075783f7717776d2b8ba163aab8a69aa24c560ca3844214485bd9eef", "role": "Napari route handlers that project typed received batches onto native viewer state." }, { @@ -25,10 +25,11 @@ } ], "findings": [ - "This legacy URL owns no current mechanism and should only route readers to the current integration and owner documentation." + "The Napari endpoint and handler rewrites (domain-blind viewers, G6) did not move ownership: the transport contract stays in zmqruntime and the OpenHCS wrappers stay in openhcs/runtime, so the redirect to streaming_boundary_and_wrappers and external_foundations remains correct. Only the endpoint evidence role was refreshed." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/napari_integration_architecture.rst in full and compared it against the changed authorities: openhcs/runtime/napari_viewer_server.py and openhcs/runtime/napari_streaming_handlers.py; confirmed both redirect targets exist and the modules remain the OpenHCS Napari wrappers; the page text needed no change.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", "tests/unit/test_napari_streaming_handlers.py and tests/unit/test_viewer_protocol.py exercise the named OpenHCS and transport boundaries", "On 2026-08-25, re-read the complete redirect against the changed Napari endpoint owner; bounded payload projection and atomic layer isolation do not change this legacy route's current-owner destination.", @@ -47,12 +48,12 @@ "authority": [ { "path": "openhcs/core/viewer_streaming_service.py", - "sha256": "46982fd46f47b962033e02d82afb6c3ce2abbafdabfbce99e591d9c6dff2fbab", - "role": "Compiled output streaming, viewer settlement, state evidence, and lifecycle cleanup." + "sha256": "c96eca530f767002cfcc9aca4fe75c93d7a5848a7f42df9e5d336bf6712ed51e", + "role": "Compiled output streaming parametrized by viewer type, viewer settlement, and lifecycle cleanup." }, { "path": "openhcs/runtime/napari_stream_visualizer.py", - "sha256": "a9ab90f3ddbfc2393683b4baf163fdb7030384b8985e8fee5b5aea20614ce2c1", + "sha256": "44087b2dbdc988e5854841fe2e47830a02bcb9e0c187e613d4cd250e8d763c2b", "role": "OpenHCS Napari visualiser declaration and launch adapter." }, { @@ -62,10 +63,11 @@ } ], "findings": [ - "The old page no longer owns streaming facts and must remain a compatibility route to the current boundary." + "The streaming service and Napari visualiser changes since the last review do not move the transport or wrapper boundary; the redirect to streaming_boundary_and_wrappers and external_foundations remains correct. Evidence roles refreshed only." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/napari_streaming_system.rst in full and compared it against the changed authorities: openhcs/core/viewer_streaming_service.py and openhcs/runtime/napari_stream_visualizer.py; confirmed both redirect targets exist and the modules remain the OpenHCS streaming service and Napari adapter; the page text needed no change.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", "tests/unit/test_streaming_service.py and tests/unit/test_napari_streaming_handlers.py verify the current route", "On 2026-08-25, re-read the complete page against metaclass-registry 0.2.1, ArrayBridge 0.3.3, the public native-catalogue projection, declaration-derived child environments, and the updated package floors; revised the page where those semantics are user-relevant and confirmed its remaining claims against current source and tests.", @@ -74,7 +76,7 @@ }, { "path": "docs/source/architecture/nominal_ownership.rst", - "source_sha256": "ab2d1a92cb778414fb8624533ef6de74cf92e70ba87c92a495bbe8ae70faf7be", + "source_sha256": "088cabe6cf370dc0c01a3927334fbb054ec7604a9256bab1ff052b7390299378", "audience": [ "OpenHCS maintainers and extension authors" ], @@ -82,38 +84,48 @@ "diataxis": "explanation", "authority": [ { - "path": "openhcs/core/registry_strategies.py", - "sha256": "a41186665abbd308f224d0bb41d6618cd9d89b08acaf54ec2819f2024a6e0804", - "role": "OpenHCS enum, nominal-type, and most-derived strategy selection implementations." + "path": "external/metaclass-registry/src/metaclass_registry/strategies.py", + "sha256": "5c00cd4d80f9f0ceffbc2ab95ad8c6abd44a8a66a974959445ef93df08dd4f0d", + "role": "EnumKeyedStrategyMixin, NominalTypeKeyedStrategyMixin and MostDerivedContextStrategyMixin selection semantics described under Registry roots." }, { "path": "external/metaclass-registry/src/metaclass_registry/core.py", - "sha256": "ed4dd1dfd0e4e05e916fec39f19b4b8bd526e776aa0c96e3abe9b1ffbdff0505", - "role": "Generic AutoRegisterMeta registry ownership and configuration mechanics." + "sha256": "72c8284e8a32dd193f13f36a425289d5293a32f4a381046db523aa87182e0be8", + "role": "AutoRegisterMeta __registry__, RegistryConfig and RegistryFamily ownership." }, { - "path": "tests/unit/test_registry_owner_iteration.py", - "sha256": "3e042f8980dfac25cf7e27d42f386fc6a63329d4ca7f21348a21589ab56a3046", - "role": "Proves semantic-control parameters and strategy iteration derive from their nominal declaration owners." + "path": "openhcs/core/runtime_plane_projection.py", + "sha256": "3c88ed042a7f1c65c9495a1c62777e008f1e59c4742ec0563b2849d77ca87cdd", + "role": "RuntimeSliceProjectableValue ABC on which owned runtime values declare their slice projection." + }, + { + "path": "openhcs/core/processing_contracts.py", + "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", + "role": "ProcessingContract subclass family owning per-callable execution over the plane axis." }, { "path": "openhcs/processing/backends/lib_registry/openhcs_registry.py", - "sha256": "aa3079341cb2af8788938e14efee6f9f6c4b6c01874227e74c6340850d525884", - "role": "Derives the public native callable catalogue while retaining explicit reconstruction from each callable declaration." + "sha256": "3fff5c3549ef8e1073d75e6d5544b2cd75fbb7cc9b17263207760ffca45cf885", + "role": "Native callable catalogue derived from each callable's nominal catalogue declaration and the public module surface." }, { "path": "openhcs/interop/cellprofiler/module_declarations.py", - "sha256": "89dd65abc003c464ba6c489f52d3a4fa47be1d5f8b87d26ad72b7537f188cf8b", - "role": "CellProfiler module declarations that own callable names, catalogue placement, and headless-import eligibility." + "sha256": "6bbf41c158496a02439be2ee5aef12d7712fb632419c226a64c9259a55438b95", + "role": "CellProfilerModule declarations owning callable names, catalogue placement, and headless-import eligibility." + }, + { + "path": "tests/unit/test_registry_owner_iteration.py", + "sha256": "1bbff81d29c9108c00382ccd1151c374ae0d68de38427034202ebb995abc5765", + "role": "Proves semantic-control parameters and strategy iteration derive from their nominal declaration owners." } ], "findings": [ - "The explanation omitted nominal callable catalogue-module ownership and the CellProfiler package's declaration-derived projection.", - "The catalogue ownership rule did not distinguish the public browsable surface from explicitly transported private decorated callables.", - "The ownership explanation did not include declaration-owned CellProfiler headless-import eligibility." + "The example 'RuntimeSliceProjectionStrategy selects behavior by nominal runtime type' named a class deleted by G3; owned runtime values now subclass RuntimeSliceProjectableValue and RuntimeSliceProjection handles only foreign values. Example rewritten.", + "The selection mixins were described as OpenHCS models while openhcs/core/registry_strategies.py had been deleted (b221f8491); they are now imported from metaclass_registry.strategies, which the page now says." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/nominal_ownership.rst in full and compared it against the changed authorities: deleted openhcs/core/registry_strategies.py (now metaclass_registry/strategies.py), metaclass_registry/core.py, tests/unit/test_registry_owner_iteration.py, lib_registry/openhcs_registry.py, interop/cellprofiler/module_declarations.py; verified ArtifactType, ProcessingContract, CellProfilerModule, the three mixins and the equivalence MostDerivedContextStrategyMixin users exist; replaced the deleted RuntimeSliceProjectionStrategy example and named the mixins' new home.", "Re-read docs/source/architecture/nominal_ownership.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/registry_strategies.py; external/metaclass-registry/src/metaclass_registry/core.py; tests/unit/test_registry_owner_iteration.py; openhcs/processing/backends/lib_registry/openhcs_registry.py; openhcs/interop/cellprofiler/module_declarations.py.", "Focused feature-lattice QA passed 159 tests covering the N-Z architecture and developer workflow authorities.", @@ -201,7 +213,7 @@ }, { "path": "docs/source/architecture/orchestrator_cleanup_guarantees.rst", - "source_sha256": "1fabf4a602e5856b9ac754b390478f7c2859a6bc6e088e7c55a4e5901ccb54a7", + "source_sha256": "2c2e37216d38b8574fac036c04d210d96efc2d8ec3c6718ec21dcfbdfdb89926", "audience": [ "OpenHCS runtime maintainers" ], @@ -209,63 +221,57 @@ "diataxis": "explanation", "authority": [ { - "path": "openhcs/core/orchestrator/compiled_plate_execution.py", - "sha256": "80998f355c3f4be9027cb91cabf581f51efd870fb86960f5f6ecd615bcdc89ef", - "role": "Parent compiled-plate execution, terminal steps, viewer settlement, evidence, and cleanup ordering." + "path": "openhcs/core/orchestrator/bundle_execution.py", + "sha256": "f5ba09a2ec7c688b1ca08167ab398cc976a9e2c39b301ae98f3c5fd560a25c85", + "role": "Parent compiled-bundle execution: executor shutdown, BrokenProcessPool handling, parent resource release, reverse-order viewer bootstrap rollback, and viewer settlement ordering." }, { "path": "openhcs/core/orchestrator/worker_execution.py", - "sha256": "3c952465268e4b6c44676a853bb9c99d2d04a0490c7d398d2cdaf8b3b5f966d7", - "role": "Worker task execution and owner-bound runtime/framework cleanup for process, inline, and thread lanes." + "sha256": "23a3ef2a70378dd1bb5c413a56e36bbb3fa013142741c9b45407bfc47de8c4a4", + "role": "Inline, process and thread lane resources: per-axis image-cache release, process-local runtime/framework cleanup, and thread-lane deferral to the parent." }, { - "path": "openhcs/core/orchestrator/orchestrator.py", - "sha256": "843e994d7b16592f43a15ce36c33a51ead534cf22df792838ce2b800621114b7", - "role": "PipelineOrchestrator configuration ownership, resolution, compiler setup, and executor lifecycle." + "path": "openhcs/core/orchestrator/cancellation.py", + "sha256": "f825b602fb020a23d602e1f93ed9cecea881eca16e07882fd7a91ab519445244", + "role": "Execution cancellation scopes that retain pre-entry requests and finish per execution." }, { "path": "openhcs/runtime/viewer_protocol.py", - "sha256": "30441eab97da9ffcb5b9dc33d65c2ecc56c426e79bb2b27eedc78a68aaf13bc3", - "role": "Distinguishes external viewer clients from owned viewer processes during failed-bootstrap rollback." + "sha256": "c4bdae4bae6dee412f0d27a9cf75a95db53b3bd95c03cee7d046be16b52fe5b8", + "role": "rollback_failed_bootstrap distinguishing connected external viewers from OpenHCS-owned viewer processes." }, { - "path": "openhcs/core/orchestrator/cancellation.py", - "sha256": "f825b602fb020a23d602e1f93ed9cecea881eca16e07882fd7a91ab519445244", - "role": "Owns cancellation scopes so pre-entry requests are retained and sequential executions do not share stale cancellation state." + "path": "openhcs/core/compiled_step_plan.py", + "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", + "role": "Framework/device assignment merge and cleanup_loaded used for the compiled framework/device footprint." }, { "path": "external/arraybridge/src/arraybridge/types.py", - "sha256": "7b94760d57e54008d7adf4e4d37dca45d8a2c288a9dc8400be00c26148cf7624", - "role": "Framework-local device scopes, installed-framework cleanup, and device discovery owned by ArrayBridge memory declarations." + "sha256": "08f7ff4df8807bb3340e00a02c6276b1c702a0f00a2eedacd701142736c5d804", + "role": "Per-framework cleanup hooks (CuPy, Torch) owned by ArrayBridge memory declarations." }, { - "path": "tests/unit/test_framework_device_resolver.py", - "sha256": "ba1814ae8eae05a39b594a81b4fc0d1444bf1334574bf4efe49aa829eec43a65", - "role": "Regression proof for independent framework namespaces and exact compiled device bindings." + "path": "openhcs/core/context/processing_context.py", + "sha256": "81c5c0bf411040bac2d93418093fff7c97c5ffd9ad942272c0f71c2982c96a44", + "role": "release_execution_image_cache clears the context-local image-stack cache without discarding observations." }, { "path": "tests/unit/test_orchestrator_lane_planning.py", - "sha256": "09c39f5cba68f8c15a0def8a09843b62d6da5c5a01e7558a9263ce4d311c078d", + "sha256": "3d5c2f3662b8d4c330582af0c4a649b8f4d9582e064d9cf3159798c8934a2bbb", "role": "Proves partial viewer acquisition and readiness failures roll back every acquired viewer." }, - { - "path": "openhcs/core/context/processing_context.py", - "sha256": "c7b79f50661c528261ce8c577fd67c753c84a5e499aa87cd5e540b53417351a2", - "role": "Owns explicit release of the context-local image-stack cache without discarding observations." - }, { "path": "tests/unit/test_runtime_stack_cache_lifetime.py", - "sha256": "bfcfbfb485de0ccdc24251020db9cc66eba2b0f31d019553814b4e6f1fde48a2", + "sha256": "a63226b25afc995817b496c1cf153ddd324786cb5e746c49399082b0ad2f573c", "role": "Tests context-local image-cache lifetime across success and failure." } ], "findings": [ - "The cleanup ordering omitted exact terminal-state ownership, cooperative safe boundaries, and closure of the execution cancellation scope.", - "Global GPU scheduler cleanup was removed; cleanup now follows each compiled framework/device footprint at its owning process boundary.", - "Viewer bootstrap previously returned its acquired set only after every viewer was created, so a later acquisition failure could leak earlier viewer processes." + "Authority openhcs/core/orchestrator/compiled_plate_execution.py was renamed to bundle_execution.py by G7 (4cd901c9f); the page's 'compiled plate execution boundary' now reads 'compiled-bundle execution boundary' to match the current owner. Cleanup ordering, lane ownership and viewer rollback claims were re-verified and remain accurate." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/orchestrator_cleanup_guarantees.rst in full and compared it against the changed authorities: compiled_plate_execution.py (renamed to bundle_execution.py, 4cd901c9f), worker_execution.py, orchestrator.py, viewer_protocol.py, arraybridge types.py, test_orchestrator_lane_planning.py, processing_context.py, test_runtime_stack_cache_lifetime.py; checked executor shutdown and BrokenProcessPool handling, reverse-order viewer rollback with external-client handling, thread-lane deferral of runtime release, per-axis image-cache release and cancellation-scope finish; renamed the boundary in the opening sentence. Dropped orchestrator.py and test_framework_device_resolver.py as authorities; added compiled_step_plan.py for the device-footprint cleanup.", "Re-read docs/source/architecture/orchestrator_cleanup_guarantees.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/orchestrator/compiled_plate_execution.py; openhcs/core/orchestrator/worker_execution.py; openhcs/core/orchestrator/orchestrator.py; openhcs/core/orchestrator/cancellation.py; external/arraybridge/src/arraybridge/types.py; tests/unit/test_framework_device_resolver.py.", "Focused feature-lattice QA passed 159 tests covering the N-Z architecture and developer workflow authorities.", @@ -281,7 +287,7 @@ }, { "path": "docs/source/architecture/orchestrator_configuration_management.rst", - "source_sha256": "1cf4d86ac3a102396af9ad8515466dc6cb55dd81c7c53190503210349b7f6b5b", + "source_sha256": "64ec638e772a8d62e5a55a13429dd832adc64d0b2dabd581acf19aeec5efc07e", "audience": [ "OpenHCS compiler and configuration maintainers" ], @@ -290,23 +296,31 @@ "authority": [ { "path": "openhcs/core/orchestrator/orchestrator.py", - "sha256": "843e994d7b16592f43a15ce36c33a51ead534cf22df792838ce2b800621114b7", - "role": "PipelineOrchestrator configuration ownership, resolution, compiler setup, and executor lifecycle." + "sha256": "a61658b107f170c192eea505966c268ab6826826f03586bd64562a44800e1ccc", + "role": "PipelineOrchestrator: __objectstate_delegate__ to pipeline_config, dataset_root identity, and initialize enforcing the source role's availability check." }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "ace8ae0c3e54bbe604b97cec7c3973a1abcf90e1dc517c2adda947dc2c35e882", - "role": "Compilation order, declaration resolution, callable warm-up, framework-local device assignment, and bundle construction." + "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "role": "Compiler consuming ObjectState-resolved steps and the resolved global configuration once per compilation." + }, + { + "path": "openhcs/core/dataset_sources/source.py", + "sha256": "097fde4d2b74031eacf53a88e79f499417e8c1c97ba0753e8a9df831b307efa6", + "role": "DatasetSource family and SourceSelectionRole.require_available_source, which owns the local-directory availability contract." }, { "path": "external/ObjectState/src/objectstate/object_state.py", - "sha256": "735ce860ab650b281534e24e3c7354f37b64bb06832cf186d6b1f626f87108fe", - "role": "Generic editable ObjectState and snapshot boundary." + "sha256": "03f818c3045b6a07024444ab2b5552fb6be9eab32dd151bb165b6f904d2863a3", + "role": "Generic editable ObjectState scope, resolution and snapshot mechanics." } ], - "findings": [], - "disposition": "keep", + "findings": [ + "The availability paragraph named a 'microscope handler's source role' and a 'plate path'; after G4 the orchestrator opens a DatasetSource from configuration, whose SourceSelectionRole.require_available_source owns availability, and the orchestrator identity is dataset_root. The paragraph now names DatasetSource, SourceSelectionRole and the dataset root." + ], + "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/orchestrator_configuration_management.rst in full and compared it against the changed authorities: orchestrator.py, pipeline/compiler.py, objectstate/object_state.py; checked the pipeline_config ObjectState delegation, once-per-compilation resolution, and the initialize availability check (now dataset_source.open plus SourceSelectionRole.require_available_source); corrected the availability paragraph's handler/plate-path vocabulary and added dataset_sources/source.py as authority.", "Re-read docs/source/architecture/orchestrator_configuration_management.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/orchestrator/orchestrator.py; openhcs/core/pipeline/compiler.py; external/ObjectState/src/objectstate/object_state.py.", "Focused feature-lattice QA passed 159 tests covering the N-Z architecture and developer workflow authorities.", @@ -429,32 +443,31 @@ "authority": [ { "path": "external/python-introspect/src/python_introspect/signature_analyzer.py", - "sha256": "73600e546753640964d4ba5308fdf808ed3a4a6f085417d8ac0745d9e2748ed7", - "role": "Generic callable signature-analysis ownership." + "sha256": "edbe51f6c92b2c334dc4533bbeaa252be649c1ac738d2ebfca4364f2708479dc", + "role": "SignatureAnalyzer: the generic callable signature analysis the redirect assigns to python-introspect." }, { "path": "external/python-introspect/src/python_introspect/callable_declaration.py", "sha256": "3bb8b474624fe6b2964a5c1f9a6aa540b2703500bfa124e7808995cb789d6951", - "role": "Orders supplied parameters by signature with optional default omission and caller-owned equality." + "role": "Generic callable-declaration projection: orders supplied parameters by signature with optional default omission." }, { "path": "external/python-introspect/docs/source/index.rst", - "sha256": "c98c4bcc2623fb87c1b4eb0858819ed11efa3618fd0fa56e8df3dccba415d6a9", - "role": "Current owner-package narrative for the extracted mechanism referenced by docs/source/architecture/parser_metaprogramming_system.rst." + "sha256": "63f46ce920b1c34b3c48fff239f45dce9496d4b46830380bad8618c2d10a873c", + "role": "Owner-package documentation the redirect points readers toward." }, { "path": "openhcs/processing/backends/lib_registry/registry_service.py", - "sha256": "046d3d088f685a993ddad6b5923cf63beef84b201127b72ae32af6721dd5ad63", - "role": "Canonical callable catalogue preparation, declaration-local projection, lookup caching, and complete derived-view clearing." + "sha256": "daef8925338fddf8504a9faa066af42456a13fe4602976bf64be06547a45fcc8", + "role": "OpenHCS consumer: imports SignatureAnalyzer from python-introspect for callable catalogue preparation instead of keeping its own signature parser." } ], "findings": [ - "Reusable parser and signature mechanics have moved to python-introspect; this page is only a compatibility route.", - "The redirect repeated the same external-foundations destination instead of stating the package policy once.", - "python-introspect now also owns generic callable-declaration projection." + "The record still carried disposition revise from an earlier prose fix; the page is a pure redirect to the python-introspect owner, so its disposition is redirect." ], - "disposition": "revise", + "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/parser_metaprogramming_system.rst in full and compared it against the changed authorities: python_introspect/signature_analyzer.py, python-introspect docs index, lib_registry/registry_service.py; confirmed python-introspect still owns signature analysis and declaration projection, registry_service.py now imports SignatureAnalyzer from it, and the external_foundations target exists; no prose change needed. Disposition corrected from revise to redirect, since the page is only a route.", "Re-read docs/source/architecture/parser_metaprogramming_system.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: external/python-introspect/src/python_introspect/signature_analyzer.py; external/python-introspect/docs/source/index.rst; openhcs/processing/backends/lib_registry/registry_service.py.", "Focused feature-lattice QA passed 159 tests covering the N-Z architecture and developer workflow authorities.", @@ -478,25 +491,26 @@ "authority": [ { "path": "openhcs/core/source_bindings.py", - "sha256": "e0afbbe40cce3847bcc2314910d0083ecc579a5a0d6e4278c3a5df4c62e7eb01", - "role": "Public typed source-binding declaration vocabulary." + "sha256": "b1474cfd969bd2ab3af6caba7e6e36f300f59fba5eeaaada08fd2254aef36598", + "role": "Public typed source-binding declaration vocabulary (SourceBindingsConfig and binding kinds)." }, { "path": "openhcs/core/source_binding_selection.py", - "sha256": "5afcabc6dae1181ae219294baedefe3c1b7ef41bbe03d44dd779a126c9405df1", - "role": "Nominal source selection, matching, and component projection strategies." + "sha256": "f97a4475aa26e465679b89ff9a1fe96690ab7631649d71650608fc7a8aa4bf5a", + "role": "Source candidate matching, anchor-pattern policies, and source-universe requests that replaced pattern detection." }, { - "path": "openhcs/microscopes/source_bindings_handler.py", - "sha256": "88f7c624310ea31a4cf5de884fdae295e4675a578f8a2821a837140e4e4b07b2", - "role": "Arbitrary-folder ingestion and declared source-binding projection." + "path": "openhcs/core/dataset_sources/source_bindings_source.py", + "sha256": "fccda877f39e7b2f99d6cc95143480e5955186264e316e832ca9a69c2ae90b8c", + "role": "SourceBindingsSource: the dataset source that ingests arbitrary image folders through source-binding declarations." } ], "findings": [ - "The former pattern-detection model has been replaced by typed source declarations and strategies." + "Authority openhcs/microscopes/source_bindings_handler.py was deleted by G4 (dataset-source family); arbitrary-folder ingestion now lives in openhcs/core/dataset_sources/source_bindings_source.py. The page names no module, so only the record changed." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/pattern_detection_system.rst in full and compared it against the changed authorities: source_bindings.py, source_binding_selection.py, and the deleted microscopes/source_bindings_handler.py (G4, c6d9a52ce); confirmed typed source matching still owns the model and the source_model target page exists; replaced the deleted handler authority with dataset_sources/source_bindings_source.py; no prose change needed.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", "tests/unit/test_source_bindings.py and tests/unit/test_source_projection.py passed against the current source model" ] @@ -512,30 +526,31 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "ef81b50ee22b8da69513fd50d6e45a929117fa4bfaa62a20ef08a4c02531c043", - "role": "Pipeline, global, step, processing, and source configuration declarations." + "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "role": "ProcessingConfig field docstrings: variable_components is the stack-axis authority and group_by partitions already assembled values, used for dictionary-pattern branch selection." }, { - "path": "openhcs/core/pipeline/artifact_planning.py", - "sha256": "8f2a9fe778df5699218185b2b52f9537a57a929f64ae2a43915d14e04a2e7135", - "role": "Exact ArtifactSpecRef planning and nominal artifact strategy selection." + "path": "openhcs/core/processing_contracts.py", + "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", + "role": "ProcessingContract family, which moved here from the library registry module (G5)." }, { - "path": "openhcs/core/runtime_stores.py", - "sha256": "5f323ec8edd4221ee30e34ed0c9c09e0f4e45ef502749c1a2d331b7a5452af99", - "role": "RuntimeValueStore recording, replacement, query, and observation semantics." + "path": "openhcs/core/pipeline/artifact_planning.py", + "sha256": "492024a74d5ce40dc326b162b3ac45edb0e55515a526a4f6512a71a8e4b11a77", + "role": "Artifact producer/consumer graph and output materialization planning behind the artifact-contract route." }, { - "path": "openhcs/processing/backends/lib_registry/unified_registry.py", - "sha256": "70462cddcad1bfee447ce0786a9e4a85abbd297726d59a1208156af0ef3a3743", - "role": "Processing-contract execution, registry-owned catalogue admission and cache semantics, and external-signature authoring exclusions." + "path": "openhcs/core/runtime_stores.py", + "sha256": "b6a7231bc2964d472070731018f6a4935e5f23cf8f238a232045c32244e6545a", + "role": "RuntimeValueStore and runtime artifact queries behind the runtime-value route." } ], "findings": [ - "The previous URL combined independent grouping, callable, artifact, and path semantics and now serves only as a migration route." + "ProcessingContract moved from lib_registry/unified_registry.py to openhcs/core/processing_contracts.py (G5); the authority now points at the new owner. The page names only the class, so its prose remains correct." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/pattern_grouping_and_special_outputs.rst in full and compared it against the changed authorities: core/config.py, pipeline/artifact_planning.py, core/runtime_stores.py, lib_registry/unified_registry.py; checked the group_by/variable_components claims against the ProcessingConfig docstrings, confirmed ProcessingContract still exists (now in core/processing_contracts.py) and that all four target pages exist; replaced unified_registry.py with processing_contracts.py as authority; no prose change needed.", "Re-read docs/source/architecture/pattern_grouping_and_special_outputs.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/config.py; openhcs/core/pipeline/artifact_planning.py; openhcs/core/runtime_stores.py; openhcs/processing/backends/lib_registry/unified_registry.py.", "Focused feature-lattice QA passed 159 tests covering the N-Z architecture and developer workflow authorities.", @@ -548,7 +563,7 @@ }, { "path": "docs/source/architecture/pipeline_compilation_system.rst", - "source_sha256": "a8b349cd9e24e5ca683416666c05ce71e9c74fbdddd4f216f20c1b0837ac36a6", + "source_sha256": "17eed1d9929c9a983d8a8b3f69c6862fea341ba5485e1f166a59accc2ff7fbc5", "audience": [ "OpenHCS compiler, runtime, and extension maintainers" ], @@ -557,77 +572,65 @@ "authority": [ { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "ace8ae0c3e54bbe604b97cec7c3973a1abcf90e1dc517c2adda947dc2c35e882", - "role": "Compilation order, declaration resolution, callable warm-up, framework-local device assignment, and bundle construction." + "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "role": "_resolve_pipeline_once ObjectState resolution, AxisCompilationRequest context projection, stage order, worker assignment, and bundle construction." }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "3c840a89baccc8665d415175458280fd904c1a837b7fd879e303d7dbca129a1c", - "role": "Resolved step/session identity and typed plan assembly invariants." - }, - { - "path": "openhcs/core/compiled_execution.py", - "sha256": "941115f2dca2e53538d3141a59f3ae1f928a87cf741fcfa177f7cd64a5f41aea", - "role": "CompiledExecutionBundle and runtime-environment authority; execution-wide worker facts and context-derived axis identities remain separate from per-step framework devices." - }, - { - "path": "openhcs/core/context/processing_context.py", - "sha256": "c7b79f50661c528261ce8c577fd67c753c84a5e499aa87cd5e540b53417351a2", - "role": "Owns the compiled multiprocessing-axis identity carried by each runtime context." - }, - { - "path": "tests/unit/test_compiled_execution.py", - "sha256": "a7ef4c3f011e0770d8e7a48928aa25c6ca550095c1f80146a25170d449ecc31f", - "role": "Verifies that bundle axis projection derives from context declarations rather than mapping-key syntax." + "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", + "role": "ResolvedPipelineDefinition scope/provenance validation and CompilationSession fields and missing-plan errors." }, { - "path": "openhcs/core/orchestrator/compiled_plate_execution.py", - "sha256": "80998f355c3f4be9027cb91cabf581f51efd870fb86960f5f6ecd615bcdc89ef", - "role": "Parent execution ordering for plate-scoped work, completed-output metadata projection, and viewer settlement." + "path": "openhcs/core/compiled_step_plan.py", + "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", + "role": "CompiledStepPlan typed field inventory and per-step framework-device assignment." }, { - "path": "openhcs/core/steps/function_outputs.py", - "sha256": "e02275c0c1d8ac00633d97f22d54bc5569dddc4f11971a0106743b94a78c8d5b", - "role": "Declaration-owned OpenHCS metadata targets, typed projection persistence, and completed-plate deduplication across compiled axes." + "path": "openhcs/core/compiled_execution.py", + "sha256": "85972d418f0958e59f3185f07caf25a103bc9e101e481b2b63903377cc05e397", + "role": "CompiledExecutionBundle contexts, worker assignments and context-derived axis ids; CompiledRuntimeEnvironmentPlan start-method reason, server mode, GPU and worker facts." }, { - "path": "tests/unit/test_compilation_session.py", - "sha256": "4224c87a6b67e4d8dde2a5c6b786ea7998474819de56da2b9d954aa2b479feb4", - "role": "Regression proof for one-to-one step, state, snapshot, context, and plan identity." + "path": "openhcs/core/callable_contract.py", + "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", + "role": "CallableContract.from_prepared_callable, process-local resolve_runtime_callable cache, and AXIS/PLATE FunctionStepExecutionScope." }, { - "path": "openhcs/core/callable_contract.py", - "sha256": "6a39b5e171901131ecb7280a78241e8ea72409a5003eabba3db34eb4851991d7", - "role": "Callable-declared input, output, and execution framework roles consumed by compilation." + "path": "openhcs/core/function_patterns.py", + "sha256": "95b9056c6980e4f7943f909aeae8262252c9aa175247a534befc8d4cc459fdd2", + "role": "CompiledFunctionInvocation binds its runtime callable at construction and rebuilds it after process transport." }, { - "path": "openhcs/core/compiled_step_plan.py", - "sha256": "6d46571e6f2d069b98a92b41d711b2e20bfab9f0493cf73d3f150d5be08e7f96", - "role": "Per-step framework-device assignment and typed compiled-plan ownership." + "path": "openhcs/processing/backends/lib_registry/registry_service.py", + "sha256": "daef8925338fddf8504a9faa066af42456a13fe4602976bf64be06547a45fcc8", + "role": "prepare_in_current_process endpoint catalogue preparation and registered_callable owner projection." }, { - "path": "openhcs/core/pipeline/framework_device_assignment.py", - "sha256": "f77f3fb2d30c4af8cab6210dbe294f9fb5e246dad0a6bbf88bcaf1e2b58dead7", - "role": "Pipeline-wide declaration-derived framework footprint and per-step binding selection." + "path": "openhcs/core/orchestrator/bundle_execution.py", + "sha256": "f5ba09a2ec7c688b1ca08167ab398cc976a9e2c39b301ae98f3c5fd560a25c85", + "role": "Parent ordering of plate-scoped steps, completed-dataset metadata finalization, and viewer settlement." }, { - "path": "openhcs/core/framework_device_resolver.py", - "sha256": "be96430174bc576c0b8083879c579ef3fb9b83c4690234751459c2d7bc5556a8", - "role": "Compile-time framework-local device discovery and fail-closed availability validation." + "path": "openhcs/core/steps/function_outputs.py", + "sha256": "89405feaa9092caa7ac3699de649eca5516c7bc54a757a24ce881264c21af262", + "role": "OpenHCSMetadataTarget declarations, typed projection persistence, and completed-plate deduplication across axes." }, { - "path": "tests/unit/test_framework_device_resolver.py", - "sha256": "ba1814ae8eae05a39b594a81b4fc0d1444bf1334574bf4efe49aa829eec43a65", - "role": "Regression proof for independent framework namespaces and exact per-step selection." + "path": "tests/unit/test_compilation_session.py", + "sha256": "381adf3a87729928a225fd1db5a35f8c802340fb2a1e583501884dc5500f6b9d", + "role": "Regression proof that axis sessions share one resolved pipeline and own their axis plans." } ], "findings": [ - "The typed-plan inventory still described a global GPU assignment and did not explain declaration-owned native catalogue placement.", - "The execution explanation did not state when multi-axis output metadata becomes a complete plate projection.", - "The execution-bundle explanation did not distinguish context-owned axis identity from runtime-context mapping keys." + "The 'Resolve once' section described a StepSnapshot tuple and an ObjectState map on ResolvedPipelineDefinition; StepSnapshot no longer exists and the definition now holds resolved steps with per-index scope ids and saved provenance, resolved without registering editor states.", + "The warm-up paragraph named prepare_compiled_context_callables(), deleted by db8b3650b; registered catalogue preparation now happens at endpoint readiness through RegistryService.prepare_in_current_process(), and compilation binds callables through CallableContract.from_prepared_callable() and CompiledFunctionInvocation.", + "The session field list named a step ObjectState map, snapshots and 'plate' facts, and the session was said to validate one-to-one step/state/snapshot identity; the session now holds the shared ResolvedPipelineDefinition, dataset scope and path resolver, and the checks are scope/provenance completeness plus missing-plan errors.", + "The typed-plan inventory listed source load plans and execution memory roles; CompiledStepPlan no longer has a source load plan and records only input/output memory types.", + "Authority openhcs/core/orchestrator/compiled_plate_execution.py was renamed to bundle_execution.py (G7, 4cd901c9f)." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/pipeline_compilation_system.rst in full and compared it against the changed authorities: pipeline/compiler.py, compilation_session.py, compiled_execution.py, context/processing_context.py, test_compiled_execution.py, compiled_plate_execution.py (renamed bundle_execution.py), steps/function_outputs.py, test_compilation_session.py, callable_contract.py, compiled_step_plan.py, framework_device_assignment.py; verified each named class/function exists (FunctionStepTransportAuthority, RegistryService.registered_callable, AxisCompilationRequest, CompiledRuntimeEnvironmentPlan, RuntimeArtifactBatch, OpenHCSMetadataTarget, FunctionStepExecutionScope AXIS/PLATE); rewrote the resolve-once, preparation, session and plan-inventory passages that named deleted StepSnapshot, prepare_compiled_context_callables and source load plans. Dropped the framework-device resolver/assignment files and test_framework_device_resolver.py as authorities since the page states only that bindings stay on step plans.", "Re-read docs/source/architecture/pipeline_compilation_system.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/pipeline/compiler.py; openhcs/core/pipeline/compilation_session.py; openhcs/core/compiled_execution.py; openhcs/core/orchestrator/compiled_plate_execution.py; openhcs/core/steps/function_outputs.py; tests/unit/test_compilation_session.py; openhcs/core/callable_contract.py; openhcs/core/compiled_step_plan.py; openhcs/core/pipeline/framework_device_assignment.py; openhcs/core/framework_device_resolver.py; tests/unit/test_framework_device_resolver.py.", "Focused feature-lattice QA passed 159 tests covering the N-Z architecture and developer workflow authorities.", @@ -645,82 +648,81 @@ }, { "path": "docs/source/architecture/plate_manager_services.rst", - "source_sha256": "06abc350333f3c0d51f3418db49de6c51d7eb3f5d27d523e1a113a5b2b2b943d", + "source_sha256": "428c259d79266a0ddd55efb210a8fe0bac05ccce1d373789c1cd2ff2b8bfe757", "audience": [ - "OpenHCS desktop workflow maintainers" + "OpenHCS session and client (GUI, MCP, script) maintainers" ], - "user_need": "Locate the service that owns a Plate Manager compile, submit, control, progress, or debug responsibility.", + "user_need": "Locate the session module that owns a dataset, pipeline, compile, run, control, progress, or debug responsibility.", "diataxis": "reference", "authority": [ { - "path": "openhcs/pyqt_gui/services/plate_manager_batch_workflow.py", - "sha256": "974aebec62dc0f283f92d8153594771d1206f02decdfd8ea5faae0a563676699", - "role": "Plate Manager compile-all-then-execute-all facade and lifecycle." + "path": "openhcs/authoring/session/session.py", + "sha256": "587047b63972df89ec46a01ed2c92ff3ef9577335f58a980297db02094e4e8e0", + "role": "Session class, MainThread/DispatcherThread, DatasetAccess and Renderer ports, dataset add/delete/split and work/mutation checks." }, { - "path": "openhcs/pyqt_gui/widgets/shared/services/batch_workflow_components.py", - "sha256": "abd29f375ab12fdc56058f1c860ce26c3b52129658795eb1c95581d83b7480bd", - "role": "Lazy owner of Plate Manager compile, submit, control, progress, debug, and measurement services." + "path": "openhcs/authoring/session/operations/__init__.py", + "sha256": "d890789ee230059f6e77fea0c991c38362dd2c726e6ade1af7f203607c85cb90", + "role": "SessionOperation, HeadlessOperation and RendererOperation family roots." }, { - "path": "tests/unit/pyqt_gui/test_plate_manager_widget.py", - "sha256": "b02e81c4a73213577df02a1effbd6843935f5c92f459d6c61ab2998bd489ad58", - "role": "Regression evidence for Plate Manager service composition and UI delegation." + "path": "openhcs/authoring/session/views.py", + "sha256": "8ce6cac27293ac9105f4d2ebc76d10eebeba1ef90e0e8954c0a117910e17e3b3", + "role": "SessionView family with DatasetListView and PipelineStepsView." }, { - "path": "openhcs/pyqt_gui/widgets/shared/services/plate_manager_workflows.py", - "sha256": "d640bf4b2710009dc3f92719ec97cf09fe3d93ed7eb1bebaf2b02913e8fafbfd", - "role": "Owns all-plates and selected-plates code-document mutation semantics." + "path": "openhcs/authoring/session/events.py", + "sha256": "6f0b27e3c6d43de6f301de4c9bb80b713b433c7e10c351eda5b35c57d91ece98", + "role": "SessionEvent family and event log with subscribe and events-after reads." }, { - "path": "openhcs/pyqt_gui/services/pipeline_object_state_binding.py", - "sha256": "afb190ac9ac38e800c8fe04497bbb78a2f1b6381a21eb14890e3061c61ae5b0b", - "role": "Owns incremental pipeline binding and complete declaration-graph replacement." + "path": "openhcs/authoring/session/compile_batch.py", + "sha256": "0cbe5a6b86a0fb5e9b92c4fc45f858c979031ac93594b317b9761c28f5aaf9ab", + "role": "Compile batches that check work is allowed and mark datasets compile-pending before connecting." }, { - "path": "openhcs/core/steps/function_step.py", - "sha256": "8e1243404095691030e9ebb95bb5611679f96c66c6b6819ae71e4b41249e74bc", - "role": "Owns complete FunctionStep declaration equality through canonical callable-pattern projection." + "path": "openhcs/authoring/session/dataset_document.py", + "sha256": "305da3c0bf7bb15a0f251a101696b721f9151d8620450a6b3e12413b79218574", + "role": "DatasetDocumentScope family and payload scope-id check for selected code documents." }, { - "path": "openhcs/pyqt_gui/widgets/shared/services/pipeline_editor_workflows.py", - "sha256": "6664126ac4daa851cacff5014e04a0c09a56bbc76f62661ff97d713e9137d447", - "role": "Applies a complete Pipeline Editor document and commits the reconciled ObjectState graph." + "path": "openhcs/authoring/session/pipelines.py", + "sha256": "245bed768c49ae17b26252d41a1040e59c46d59e0894b553309cd5ec8a9a1793", + "role": "Pipeline declaration reconciliation and saved-baseline commits." }, { - "path": "openhcs/agent/services/endpoint_function_catalog_service.py", - "sha256": "977a74003819f2547548d322f4c5f0cb148b23fec20f31be4fa462bfcfb7ea04", - "role": "Owns the shared asynchronous endpoint catalogue used by desktop startup, the Function Selector, and local MCP authoring." + "path": "openhcs/agent/capabilities.py", + "sha256": "eaebe33af01d057d1d21b3701f958bc96ede3ec5d0bddddca9917c1164ef0042", + "role": "SessionOperationInvocation deriving MCP tools from headless operations, and the openhcs_session_events capability." }, { - "path": "openhcs/pyqt_gui/main.py", - "sha256": "a156a006a3c40f43107b9b1f6ecb97ed150879eb88ded8ef89e25214309ae33a", - "role": "Coordinates post-paint endpoint preparation and callable-catalogue prewarming." + "path": "openhcs/core/pipeline_import.py", + "sha256": "b48f0a37e621f36fe256138472c4bda1e87d2d0d45be83361a40ce6cbfea4e55", + "role": "PipelineImporter family keyed by file suffix." }, { - "path": "tests/unit/pyqt_gui/test_pipeline_editor_widget.py", - "sha256": "8f28719c5bd9eb5681e67af8e1f7aceb445bcae97c12a13348f943c8b6bafa35", - "role": "Proves exact reconciled editor state using the installed synthetic demo pipeline declaration." + "path": "openhcs/interop/cellprofiler/dataset_scope.py", + "sha256": "0a3dc691a04c95bdb115fe1586374af1ea8401787b600e2c109c1ad8acfcaf43", + "role": "CellProfiler dataset scope pairing a source folder with one .cppipe." }, { - "path": "openhcs/demo/synthetic_plate_pipeline.py", - "sha256": "3f8916ff71938ac5b53c7f55329b037653519e9592cf564257f78dde158fbd11", - "role": "Installed synthetic pipeline declaration used by the Pipeline Editor widget regression fixture." + "path": "openhcs/pyqt_gui/session_rendering.py", + "sha256": "deee8c93c5bfd41c5ee25abe959938426ed17120275b3b6ee9ccc25b2dd0d690", + "role": "QtSessionEventRelay GUI subscriber." }, { - "path": "openhcs/pyqt_gui/services/plate_manager_state_projection.py", - "sha256": "814651335543000eb3855975a80af311b6290926da233d02ab5e95ba0daa82ba", - "role": "Owns lightweight current row activity and the richer result relation projection." + "path": "openhcs/mcp/server.py", + "sha256": "e1f236011df715da3d17574c0b80cbf50fcae45e3f82491c22fb57e98c6bb662", + "role": "MCP binds the session to DispatcherThread over McpMainThreadDispatcher." } ], "findings": [ - "The reference omitted code-document mutation scope and the identity distinction between incremental edits and complete pipeline replacement.", - "The service map omitted Pipeline Editor saved-baseline commits and the asynchronous endpoint-owned function-catalogue projection.", - "Callable occurrence identity now excludes wrapper and explicit-default differences that do not change a FunctionStep declaration.", - "Cleanup previously skipped the client owner before endpoint admission; the reference now records one teardown boundary for admitted clients and in-progress attempts." + "U1 (0631adc7c) replaced the Plate Manager service stack with the openhcs/authoring/session package and rewrote this page without a review; every Plate Manager authority (plate_manager_batch_workflow.py, batch_workflow_components.py, pipeline_object_state_binding.py, plate_manager_state_projection.py) was deleted. Authorities, audience and user need now follow the session package.", + "Re-checked the rewritten page's module map and invariants against the session package; replaced the jargon 'admission checks' with 'work and mutation checks' (require_work_allowed / require_definition_mutation_allowed). The file name still says plate_manager_services while the page documents the session; a rename is left to the lead." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/plate_manager_services.rst in full and compared it against the changed authorities: the page itself (rewritten in U1), deleted plate_manager_batch_workflow.py, batch_workflow_components.py, pipeline_object_state_binding.py and plate_manager_state_projection.py, changed plate_manager_workflows.py, pipeline_editor_workflows.py, endpoint_function_catalog_service.py, pyqt_gui/main.py, the Plate Manager and Pipeline Editor widget tests, and demo/synthetic_plate_pipeline.py; verified every named module in openhcs/authoring/session, SessionOperation/SessionView/SessionEvent families, DatasetListView, PipelineStepsView, DatasetDocumentScope, PipelineImporter keyed by suffix, the CellProfiler .cppipe dataset scope, QtSessionEventRelay, openhcs_session_events, DispatcherThread use in the GUI and MCP, compile-artifact ids on submission and compile-pending reservation; replaced all authorities with session-package owners.", "Re-read docs/source/architecture/plate_manager_services.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/pyqt_gui/services/plate_manager_batch_workflow.py; openhcs/pyqt_gui/widgets/shared/services/batch_workflow_components.py; tests/unit/pyqt_gui/test_plate_manager_widget.py; openhcs/pyqt_gui/widgets/shared/services/plate_manager_workflows.py; openhcs/pyqt_gui/services/pipeline_object_state_binding.py; openhcs/pyqt_gui/widgets/shared/services/pipeline_editor_workflows.py; openhcs/agent/services/endpoint_function_catalog_service.py; openhcs/pyqt_gui/main.py; tests/unit/pyqt_gui/test_pipeline_editor_widget.py; openhcs/demo/synthetic_plate_pipeline.py.", "Focused feature-lattice QA passed 159 tests covering the N-Z architecture and developer workflow authorities.", @@ -752,25 +754,26 @@ "authority": [ { "path": "external/metaclass-registry/src/metaclass_registry/core.py", - "sha256": "ed4dd1dfd0e4e05e916fec39f19b4b8bd526e776aa0c96e3abe9b1ffbdff0505", - "role": "Generic AutoRegisterMeta registry ownership and configuration mechanics." + "sha256": "72c8284e8a32dd193f13f36a425289d5293a32f4a381046db523aa87182e0be8", + "role": "AutoRegisterMeta, RegistryConfig and LazyDiscoveryDict: the generic registration mechanism the redirect assigns to metaclass-registry." }, { - "path": "external/metaclass-registry/docs/source/patterns.rst", - "sha256": "cb67e1b458cb073d226ad6691f4ed107bd35b887ad65452157a7283009707402", - "role": "Current owner-package narrative for the extracted mechanism referenced by docs/source/architecture/plugin_registry_advanced.rst." + "path": "external/metaclass-registry/src/metaclass_registry/strategies.py", + "sha256": "5c00cd4d80f9f0ceffbc2ab95ad8c6abd44a8a66a974959445ef93df08dd4f0d", + "role": "Enum-keyed, nominal-type-keyed and most-derived context strategy mixins, imported by OpenHCS families from metaclass_registry.strategies." }, { - "path": "openhcs/core/registry_strategies.py", - "sha256": "a41186665abbd308f224d0bb41d6618cd9d89b08acaf54ec2819f2024a6e0804", - "role": "OpenHCS enum, nominal-type, and most-derived strategy selection implementations." + "path": "external/metaclass-registry/docs/source/patterns.rst", + "sha256": "cb67e1b458cb073d226ad6691f4ed107bd35b887ad65452157a7283009707402", + "role": "Owner-package documentation of advanced registration patterns the redirect points readers toward." } ], "findings": [ - "Generic registration mechanics were extracted and this page is now a compatibility route." + "Authority openhcs/core/registry_strategies.py was deleted (b221f8491); the enum, nominal-type and most-derived strategy mixins now live in metaclass_registry.strategies, which strengthens the redirect's claim that metaclass-registry owns the reusable mechanism. Only the record changed." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/plugin_registry_advanced.rst in full and compared it against the changed authorities: metaclass_registry/core.py and the deleted openhcs/core/registry_strategies.py (moved to metaclass_registry/strategies.py in b221f8491); confirmed OpenHCS modules import AutoRegisterMeta and the strategy mixins from metaclass_registry and that nominal_ownership and external_foundations exist; no prose change needed.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", "Current imports and tests/unit/test_registry_owner_iteration.py confirm the extracted owner and OpenHCS consumer boundary", "On 2026-08-25, re-read the complete page against metaclass-registry 0.2.1, ArrayBridge 0.3.3, the public native-catalogue projection, declaration-derived child environments, and the updated package floors; revised the page where those semantics are user-relevant and confirmed its remaining claims against current source and tests." @@ -787,25 +790,26 @@ "authority": [ { "path": "external/metaclass-registry/src/metaclass_registry/core.py", - "sha256": "ed4dd1dfd0e4e05e916fec39f19b4b8bd526e776aa0c96e3abe9b1ffbdff0505", - "role": "Generic AutoRegisterMeta registry ownership and configuration mechanics." + "sha256": "72c8284e8a32dd193f13f36a425289d5293a32f4a381046db523aa87182e0be8", + "role": "AutoRegisterMeta, RegistryConfig and LazyDiscoveryDict: the generic registration mechanism the redirect assigns to metaclass-registry." }, { - "path": "external/metaclass-registry/docs/source/index.rst", - "sha256": "f707777948f83a5bba22528814b43520e61c76da730d0b93e0c5f3f91d6c5b12", - "role": "Current owner-package narrative for the extracted mechanism referenced by docs/source/architecture/plugin_registry_system.rst." + "path": "external/metaclass-registry/src/metaclass_registry/strategies.py", + "sha256": "5c00cd4d80f9f0ceffbc2ab95ad8c6abd44a8a66a974959445ef93df08dd4f0d", + "role": "Strategy mixins OpenHCS families now import from metaclass_registry.strategies instead of an OpenHCS module." }, { - "path": "openhcs/core/registry_strategies.py", - "sha256": "a41186665abbd308f224d0bb41d6618cd9d89b08acaf54ec2819f2024a6e0804", - "role": "OpenHCS enum, nominal-type, and most-derived strategy selection implementations." + "path": "external/metaclass-registry/docs/source/index.rst", + "sha256": "f707777948f83a5bba22528814b43520e61c76da730d0b93e0c5f3f91d6c5b12", + "role": "Owner-package documentation the redirect points readers toward." } ], "findings": [ - "Generic registration mechanics were extracted and this page is now a compatibility route." + "Authority openhcs/core/registry_strategies.py was deleted (b221f8491); its strategy mixins moved into metaclass_registry.strategies. Only the record changed." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/plugin_registry_system.rst in full and compared it against the changed authorities: metaclass_registry/core.py and the deleted openhcs/core/registry_strategies.py (moved to metaclass_registry/strategies.py in b221f8491); confirmed the redirect's ownership claim and its two target pages; no prose change needed.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", "Current imports and registry tests confirm the extracted owner", "On 2026-08-25, re-read the complete page against metaclass-registry 0.2.1, ArrayBridge 0.3.3, the public native-catalogue projection, declaration-derived child environments, and the updated package floors; revised the page where those semantics are user-relevant and confirmed its remaining claims against current source and tests." @@ -813,7 +817,7 @@ }, { "path": "docs/source/architecture/processing_semantics.rst", - "source_sha256": "9d1a0203f2a4236ea102cacd8e48735a24d93d55d3d5c807ed69a2f30d451278", + "source_sha256": "67a1c1bccacef1b3ccdbc97cfa9a95fec6b52dc0b6dd535d92db3e11876aca59", "audience": [ "Pipeline authors and OpenHCS processing maintainers" ], @@ -822,41 +826,49 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "ef81b50ee22b8da69513fd50d6e45a929117fa4bfaa62a20ef08a4c02531c043", - "role": "Pipeline, global, step, processing, and source configuration declarations." + "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "role": "ProcessingConfig variable_components, group_by and input_source field semantics." }, { - "path": "openhcs/processing/backends/lib_registry/unified_registry.py", - "sha256": "70462cddcad1bfee447ce0786a9e4a85abbd297726d59a1208156af0ef3a3743", - "role": "Processing-contract execution, registry-owned catalogue admission and cache semantics, and external-signature authoring exclusions." + "path": "openhcs/core/processing_contracts.py", + "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", + "role": "ProcessingContract family: Pure2DContract, Pure3DContract, FlexibleContract, VolumetricToSliceContract and collapses_input_plane_axis." }, { "path": "openhcs/core/callable_contract.py", - "sha256": "6a39b5e171901131ecb7280a78241e8ea72409a5003eabba3db34eb4851991d7", - "role": "Callable ABI including input/output/execution memory roles, runtime-owned parameter projection, and artifact contracts." + "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", + "role": "CallableContract memory types, allowed group_by roles, variable-component requirement, execution scope, runtime adapter, image mode and ImagePayloadConsumption." }, { - "path": "openhcs/core/runtime_slice_projection.py", - "sha256": "8a1cb65d3025ed8a87ee59af26deac9765a8f59b21bf2cd61b6f96d82999641d", - "role": "Nominal runtime slice projection registry and aggregation behaviour." + "path": "openhcs/core/payload_axes.py", + "sha256": "5d05309c16f8704a7d384a258961336cb22d6a1b366b36d02c454361982412b1", + "role": "Declared tensor payload dimensions (plane axis, spatial domain, other axis specs)." }, { - "path": "openhcs/core/compiled_step_plan.py", - "sha256": "6d46571e6f2d069b98a92b41d711b2e20bfab9f0493cf73d3f150d5be08e7f96", - "role": "Per-step callable memory footprint and framework-device binding authority." + "path": "openhcs/core/runtime_plane_projection.py", + "sha256": "3c88ed042a7f1c65c9495a1c62777e008f1e59c4742ec0563b2849d77ca87cdd", + "role": "RuntimeSliceProjectableValue: owned values declare slice count and per-slice projection." }, { - "path": "openhcs/core/pipeline/framework_device_assignment.py", - "sha256": "f77f3fb2d30c4af8cab6210dbe294f9fb5e246dad0a6bbf88bcaf1e2b58dead7", - "role": "Declaration-derived framework footprint and per-step binding selection." + "path": "openhcs/core/runtime_slice_projection.py", + "sha256": "8f45a17d6aabd4fae1df55e8d3cda2efdd8df2156357c69133f06ccf1cfb2d96", + "role": "RuntimeSliceProjection foreign-value handling and the missing-declaration error." + }, + { + "path": "openhcs/core/compiled_step_plan.py", + "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", + "role": "CompiledStepPlan storage of resolved callable facts and framework-local device bindings." } ], "findings": [ - "Callable ownership now includes an execution-memory role and compiler-selected framework-local device bindings.", - "The callable section used the narrower runtime-bound term after artifact, configuration, context, and adapter inputs became one runtime-owned authoring exclusion." + "The intro said image data is transported as 3D arrays; after G3 payloads are tensors with declared axes (PayloadAxes: plane axis, spatial domain, other declared axes). Rewritten.", + "The processing-contract section and common-mistakes list used enum-member names PURE_2D/PURE_3D/FLEXIBLE/VOLUMETRIC_TO_SLICE; G5 made contracts ProcessingContract subclasses (Pure2DContract, Pure3DContract, FlexibleContract, VolumetricToSliceContract) declared by class on the decorator. FLEXIBLE was said to delegate to the others; FlexibleContract now subclasses both and switches on its slice-by-slice control.", + "The runtime-projection section said RuntimeSliceProjection selects a registered strategy per value family; owned values now implement RuntimeSliceProjectableValue themselves and RuntimeSliceProjection handles only foreign values.", + "ProcessingContract moved from lib_registry/unified_registry.py to core/processing_contracts.py; the authority list follows it." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/processing_semantics.rst in full and compared it against the changed authorities: core/config.py, lib_registry/unified_registry.py, callable_contract.py, runtime_slice_projection.py, compiled_step_plan.py, framework_device_assignment.py; verified the ProcessingConfig field semantics, the four ProcessingContract subclasses and keys, contract=Pure2DContract decorator use, CallableContract fields, FunctionStepExecutionScope and ImagePayloadConsumption; rewrote the payload-rank intro, contract names and the runtime-projection paragraph. Replaced unified_registry.py and framework_device_assignment.py with processing_contracts.py, payload_axes.py and runtime_plane_projection.py as authorities.", "Re-read docs/source/architecture/processing_semantics.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/config.py; openhcs/processing/backends/lib_registry/unified_registry.py; openhcs/core/callable_contract.py; openhcs/core/runtime_slice_projection.py; openhcs/core/compiled_step_plan.py; openhcs/core/pipeline/framework_device_assignment.py.", "Focused feature-lattice QA passed 159 tests covering the N-Z architecture and developer workflow authorities.", @@ -871,7 +883,7 @@ }, { "path": "docs/source/architecture/progress_runtime_projection_system.rst", - "source_sha256": "6cd52fe8420d85bf7429ab128e3ba3a92a42bd158cc35767b9c1673ef51a6441", + "source_sha256": "c659560a49b0afd79636c3209940a51209db5c9662a696d48ce7e08cb076a147", "audience": [ "OpenHCS runtime, debugger, and UI maintainers" ], @@ -880,51 +892,69 @@ "authority": [ { "path": "openhcs/core/progress/projection.py", - "sha256": "f07ac1c460d0761c2b70c834aae57f77994ac43bb3ef5cd688e4196ccc5d9fce", - "role": "OpenHCS plate-runtime states and typed domain projection over generic progress." + "sha256": "0c931a096020b5622bcfbf92c856c7cb3ead193805b7276f5fb89bfb5613f8cb", + "role": "SubjectRuntimeState family (labels, terminal and server-lifecycle policy), SubjectRuntimeIdentity, state_counts and ExecutionRuntimeProjection heartbeat reconciliation." + }, + { + "path": "openhcs/core/progress/runtime_tree.py", + "sha256": "016279f4135e2c9b10cb841e57ba0297c4197a195623f8a746ed90ab06533ce5", + "role": "Runtime tree node kinds subject, worker, partition, step and compilation." + }, + { + "path": "openhcs/core/progress/types.py", + "sha256": "beabbfeb8a6ed727b4bb09f6155de60c4e8b89e71e7ed053bb85574aa1675419", + "role": "ProgressEvent, set_progress_queue and the PATTERN_GROUP, AXIS_COMPLETED and SUCCESS phases." + }, + { + "path": "openhcs/core/progress/debug_projection.py", + "sha256": "f3fc8a8628b3d6a739efdd9449fd71d56ea3ba5780c3d72f3344ed56516aeed4", + "role": "RuntimeProjectionBuilder and DebugRuntimeProjectionBuilder over one event snapshot." }, { "path": "openhcs/core/debug.py", - "sha256": "072259811bc50bfe71cce9a7cac3e9867bfd853bd34ab5a74b01d9fe14b68f36", - "role": "Nominal debug command, boundary, snapshot, replay, and paused-worker declarations." + "sha256": "95d365afefc46ee50da5abb8d5d246b3980feb32b85839f91e874cb7e689caaa", + "role": "Debug command, boundary-event, cursor, session, snapshot, artifact identity, snapshot store, warm-replay and paused-worker declarations." }, { "path": "openhcs/core/debug_session_projection.py", "sha256": "0a630eb03a83d741a4ed55838adb8ee65dd4fde9be07878aeba5e7a8c86c7378", - "role": "Typed debug-session phase selection from exact runtime context." + "role": "DebugSessionPhaseDeclarationBase phase selection." }, { - "path": "external/zmqruntime/src/zmqruntime/progress/projection.py", - "sha256": "72b22f765dabbcf6983c619c8cc5404bc52518186ef72d85a6f6c6ab65cb426f", - "role": "Generic progress projection kernel." + "path": "openhcs/authoring/session/progress.py", + "sha256": "cd34f68461b50506600da7854a3bf57646f2f0ff7288d0627bc40cb852a16825", + "role": "ExecutionProgress: dirty-marking registry subscription and interval rebuild on the session main thread, published as session events." }, { - "path": "external/zmqruntime/src/zmqruntime/execution/progress_stream.py", - "sha256": "b2dd8e27f8fcac010beef3f8532fd9d584d7bae8c3f5591dcee0e5a235d6213a", - "role": "Owns the immutable latest event and monotonic sequence observation retained per execution." + "path": "openhcs/agent/capabilities.py", + "sha256": "eaebe33af01d057d1d21b3701f958bc96ede3ec5d0bddddca9917c1164ef0042", + "role": "openhcs_session_events and openhcs_session_datasets headless readers." }, { - "path": "openhcs/agent/services/execution_session_service.py", - "sha256": "1a9a70d2924b1854338612b212f00888bacba5cd33eb23608be294bb06c76399", - "role": "Retains the exact submitting client and projects its authoritative progress observation." + "path": "openhcs/core/orchestrator/bundle_execution.py", + "sha256": "f5ba09a2ec7c688b1ca08167ab398cc976a9e2c39b301ae98f3c5fd560a25c85", + "role": "execute_dataset_scoped_steps and the parent ordering before the pipeline SUCCESS event." }, { - "path": "openhcs/agent/dto/execution.py", - "sha256": "1fbdc008937e3a043cf077e2243b40f6a0efba34d9166716c423af9b85445684", - "role": "Declares the typed public job status and delegates terminal semantics to the generic lifecycle declaration." + "path": "openhcs/runtime/zmq_progress.py", + "sha256": "740f8167a93565fb595b8b184776901457bb455c52eff68a89fd5974e602bc24", + "role": "ZMQCompilerProgressQueue and ZMQProgressEmitter for execution-server events." }, { - "path": "tests/unit/agent/test_agent_services.py", - "sha256": "0c3a5225c1e832d9fbd5a423ef0cfd495451c0872b6b718d5fd66726f0758fdf", - "role": "Proves same-client reuse, exact observation serialization, terminal caching, and one-time client release." + "path": "external/zmqruntime/src/zmqruntime/progress/projection.py", + "sha256": "ce72275812c552779df7e8bc9412659087a9a929f585b1994978615c68e8f33e", + "role": "Generic progress projection kernel the OpenHCS adapter specializes." } ], "findings": [ - "The canonical-owner section used checkout-local external/zmqruntime paths that cannot resolve in published documentation.", - "Link checking showed that the equivalent deep Read the Docs pages are not deployed; the page now links to the exact owner sources at the pinned zmqruntime release." + "G7 renamed the progress vocabulary from plates to subjects: PlateRuntimeStateDeclarationBase is now the SubjectRuntimeState family, runtime identity is SubjectRuntimeIdentity(execution_id, subject_id), and the tree path is subject -> worker -> partition -> step (was plate -> worker -> well -> step). Invariants corrected.", + "Consumption steps 6-7 and the background-callback invariant described a Plate Manager GUI-thread Qt timer; after U1 the session's ExecutionProgress marks the projection dirty and rebuilds it on the main thread at an interval, publishing session events. Corrected.", + "execute_plate_scoped_steps() no longer exists; the parent calls execute_dataset_scoped_steps(). Corrected.", + "Authority openhcs/agent/services/execution_session_service.py was deleted (U1, 880cb2c55); the headless path is now the session (authoring/session/progress.py, agent/capabilities.py). Dropped the agent DTO, progress_stream and agent-services test authorities the page no longer relies on." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/progress_runtime_projection_system.rst in full and compared it against the changed authorities: the page itself (edited in U1), core/progress/projection.py, core/debug.py, zmqruntime progress/projection.py and execution/progress_stream.py, deleted agent/services/execution_session_service.py, agent/dto/execution.py, tests/unit/agent/test_agent_services.py; verified every named class/function (all debug declarations, RuntimeProjectionBuilder, ExecutionProgress, LiveMeasurementProgressPayload, ZMQ progress emitters, phases) and corrected the plate-to-subject vocabulary, tree path, Qt-timer coalescing description and the dataset-scoped step function name.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", "tests/unit/progress/test_projection.py and tests/unit/progress/test_debug_projection.py passed; published owner URLs now replace checkout-local paths", "Sphinx linkcheck resolved both pinned zmqruntime owner-source links", @@ -941,7 +971,7 @@ }, { "path": "docs/source/architecture/quick_start.rst", - "source_sha256": "4dcc151050f02da75a6819db0be0053a1dcbefcf236b34998536ad9fbece915d", + "source_sha256": "8785a34f2a2723bb302210d4da339932e3c956c808b30476267903ab5f2ce5bf", "audience": [ "New OpenHCS users, developers, and coding agents" ], @@ -950,51 +980,70 @@ "authority": [ { "path": "openhcs/core/pipeline_document.py", - "sha256": "0849800b3e6474b8176fe35bc4aaebc63cfa85bd77251000c50da5f4c928b1b2", - "role": "PipelineDocument construction and atomic configuration-step identity." + "sha256": "8c554c8275c665f308eb098955bda517aa804eeb34418b5e0f41d412650ce7b5", + "role": "PipelineDocumentCodec.from_values and the optional pipeline_config / required pipeline_steps rule." }, { - "path": "openhcs/core/steps/function_step.py", - "sha256": "8e1243404095691030e9ebb95bb5611679f96c66c6b6819ae71e4b41249e74bc", - "role": "Public FunctionStep callable-pattern and nested step-configuration declaration." + "path": "openhcs/core/steps/abstract.py", + "sha256": "63734f0c5198272b08f128ae974ada725c8201cb44efd0ce32500fd280b879bf", + "role": "Step constructor accepting processing_config and source_bindings nested declarations." }, { - "path": "openhcs/agent/authoring_contexts.py", - "sha256": "6b68414c90e98d905f0b792440f12fad5dbd78abcd0c2c500b19fa6279184939", - "role": "Declaration-owned progressive MCP authoring contexts and knowledge targets." + "path": "openhcs/core/processing_contracts.py", + "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", + "role": "Pure2DContract and Pure3DContract named in the minimal example and the locality summary." + }, + { + "path": "openhcs/core/source_bindings.py", + "sha256": "b1474cfd969bd2ab3af6caba7e6e36f300f59fba5eeaaada08fd2254aef36598", + "role": "SourceBindingsConfig fields and the selector, filter and metadata-rule types used in the folder example." + }, + { + "path": "openhcs/core/dataset_sources/source_bindings_source.py", + "sha256": "fccda877f39e7b2f99d6cc95143480e5955186264e316e832ca9a69c2ae90b8c", + "role": "SourceBindingsSource, selected when a non-empty source-binding declaration is present." }, { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "b547c838b2c86fe7cc5fe12d6cbf6a6715e83055c2d8c1a109ac22566d22c51e", - "role": "Public CellProfiler pipeline lowering and declaration-owned headless-import validation." + "sha256": "3439d96f3fbfcb104a53019dc150049021d9a8132bd27096f1ccf510eefbe37b", + "role": "import_cellprofiler_pipeline signature and headless-import rejection of interactive modules." }, { - "path": "tests/unit/test_cellprofiler_pipeline_import.py", - "sha256": "353f0c1d8e0bc3552d2e418d792794ab0631daa16c78a3e10191b1cc1ea07d73", - "role": "Proves interactive CellProfiler modules fail headless import instead of disappearing from the lowered pipeline." + "path": "openhcs/agent/capabilities.py", + "sha256": "eaebe33af01d057d1d21b3701f958bc96ede3ec5d0bddddca9917c1164ef0042", + "role": "MCP tool names used in the first-session sequence (health check, authoring context, capability and knowledge search, config schema, pipeline drafts)." }, { - "path": "tests/unit/test_pipeline_document.py", - "sha256": "2c18222ac2a34ee267788063a374112b17874c64693bfe542669c4183bc90c36", - "role": "Proves missing pipeline_config selects PipelineConfig() while explicit invalid values and missing steps remain rejected." + "path": "openhcs/agent/authoring_contexts.py", + "sha256": "a714fab8481102a5ab7800a465fcbdf7267d1ed4468cb3803b75e03f2f29ff33", + "role": "first_use authoring context and the Official30 recipe knowledge target." + }, + { + "path": "openhcs/authoring/session/session.py", + "sha256": "587047b63972df89ec46a01ed2c92ff3ef9577335f58a980297db02094e4e8e0", + "role": "Session shared by the desktop application and MCP for progress, cancellation and results." }, { - "path": "openhcs/agent/services/llm_context_service.py", - "sha256": "f26b697684b185bd5ebe122de14f6211cf5377cc21f31c917cca53cea6b4e592", - "role": "Renders the declaration-composed operating guides, discovery boundaries and workflow instructions." + "path": "tests/unit/test_pipeline_document.py", + "sha256": "4df57a9c4f12a009222981bf1ac28554405874f3533656b61344ee94056d1bab", + "role": "Proves missing pipeline_config selects PipelineConfig() while explicit invalid values and missing steps remain rejected." }, { - "path": "tests/unit/agent/test_progressive_authoring_context.py", - "sha256": "533f8c704914112479c8adcb78ee14a006f750d5894b334c29cbd1380bc899d7", - "role": "Checks bounded guide rendering, nominal route projection and operating/profile boundaries." + "path": "tests/unit/test_cellprofiler_pipeline_import.py", + "sha256": "6b1d201ec5c4ba620fa2367a9cbef1f6fbdd25edbdda9bd7b723ebe3a3363412", + "role": "Proves interactive CellProfiler modules fail headless import instead of disappearing." } ], "findings": [ - "The pipeline-source boundary previously implied both assignments were mandatory; it now records that pipeline_steps is required and an omitted pipeline_config selects PipelineConfig().", - "The CellProfiler route did not state that interactive modules fail headless import rather than being silently omitted." + "The declaration-path diagram named StepSnapshot, which no longer exists; it now names ResolvedPipelineDefinition.", + "The ProcessingContract summary used the enum spellings PURE_2D/PURE_3D while the page's own example uses Pure2DContract (G5 class family); corrected to the class names.", + "The unsupported-folder section named SourceBindingsHandler and a 'registered microscope handler'; G4 replaced it with SourceBindingsSource in the DatasetSource family.", + "The source-binding code example used InputSource.PIPELINE_START without importing InputSource; the import is added.", + "The page said 'MCP execution services' own progress and cancellation; after U1 the shared Session does, for both the desktop application and MCP." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/quick_start.rst in full and compared it against the changed authorities: the page itself (edited in G5/G1), pipeline_document.py, agent/authoring_contexts.py, cellprofiler/pipeline_import.py, test_cellprofiler_pipeline_import.py, test_pipeline_document.py, agent/services/llm_context_service.py, test_progressive_authoring_context.py; verified every MCP tool name in agent/capabilities.py, the analysis-strategy section id in the knowledge manifest, the code-example imports (Microscopy.Channel, stack_percentile_normalize, Lazy*Config, PipelineDocumentCodec) and the MFD preset paths; corrected StepSnapshot, PURE_2D/PURE_3D, SourceBindingsHandler, the missing InputSource import and the progress owner. scripts/validate_docs.py reports no code-block or import finding for the page.", "Reviewed the complete page against PipelineDocumentAuthority and the public pipeline declarations after the compatibility change.", "Passed the documentation validator and tests/unit/test_pipeline_document.py for parsing, defaulting, rejection, and canonical rendering behavior.", "On 2026-08-27, re-read and revised the complete explanation against the current CellProfiler importer and its focused regression; interactive modules now fail headless import explicitly instead of being silently omitted.", @@ -1006,7 +1055,7 @@ }, { "path": "docs/source/architecture/roi_system.rst", - "source_sha256": "15c214ff7075f5e87c56620dcf2730405fdd6845c83d6c379409caca919dff61", + "source_sha256": "6a999d9dc4dbe7fac3c1f08f3078ef52f6be8d09fadc594b7902ed980d6a0adb", "audience": [ "OpenHCS ROI, viewer, and materialisation maintainers" ], @@ -1015,28 +1064,41 @@ "authority": [ { "path": "openhcs/napari_roi_manager/widgets/_roi_manager.py", - "sha256": "9ae7518576902c4a6d0384e387a08ae73528b39cf3878039c1fc70cfaf6c8a19", - "role": "Native Napari Shapes-layer ROI manager projection and selection behaviour." + "sha256": "841a7ead6e0b577ac224526e1bc1c6ca2fcef42ced5420c632e5501befb45ea3", + "role": "ROI manager bound to the active native Shapes layer through connect_layer, with observer release on widget destruction." }, { "path": "openhcs/processing/materialization/core.py", - "sha256": "1f5a89a1eb50766ef093df3fece16d6bb03422b92d062fe8ae6471656b1ebd45", - "role": "Nominal artifact materialisation and ROI projection strategies." + "sha256": "744573f1bafa67b7a81be5af425012ada47741292519863e250b79f1c778b265", + "role": "ROI output writers, polyline graph-edge writer with its 2D-only rejection, and projected filename resolution." + }, + { + "path": "openhcs/processing/materialization/options.py", + "sha256": "271346eda98bfcabe2614696e1d2499327b84dcd8245f3ad61025cc18ddcae0f", + "role": "SpatialGraphROIOptions and SWCOptions declarations." + }, + { + "path": "openhcs/core/artifacts.py", + "sha256": "65eb0ec5c903366243ca8bd5ea07412e8d34bdb81e6015fa76b3c4c59838733f", + "role": "ArtifactOutputPlan, ArtifactSpecRelation and ObjectArtifactSubjectBinding." }, { "path": "external/PolyStore/src/polystore/roi.py", - "sha256": "a3bd2db5696d4aca9f8a7d00bf4b58bfe627564eed5aeb17aa89b8194ba13375", - "role": "Generic ROI values and persistence mechanics." + "sha256": "fa945e9a3f7272e72f3ea022c7a3ae6cd30520ed7924591dd3f3cad5d0e71fc0", + "role": "Generic ROI value types and ImageJ ROI archive persistence." }, { "path": "tests/unit/test_napari_roi_manager_integration.py", - "sha256": "de66b3d7aeb49b100ac5bb3c3acdf1c14afc19fb3681071b4356f6d7949a52dc", + "sha256": "a4b7dee556f0ceccaa212751e29eb8160d3d0c08c741760715ee8372b0c0e927", "role": "Regression proof for the native Shapes-layer ROI manager seam." } ], - "findings": [], - "disposition": "keep", + "findings": [ + "Flow defect: 'Spatial graphs have two independent registered projections:' was followed by the unrelated projected-filename paragraph before the two options it introduced; the paragraph now precedes the object-label sentence. Ownership and naming claims were re-verified against the changed widget, materialization core and PolyStore ROI owner and remain accurate." + ], + "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/roi_system.rst in full and compared it against the changed authorities: napari_roi_manager/widgets/_roi_manager.py (observer release on destroy), processing/materialization/core.py (new ROIOutput/PointROIOutput writers), polystore/roi.py, test_napari_roi_manager_integration.py; verified SpatialGraphROIOptions, SWCOptions, SpatialGraph, ArtifactOutputPlan, ArtifactSpecRelation, ObjectArtifactSubjectBinding and connect_layer exist and that the polyline writer rejects 3D graphs; reordered one misplaced paragraph. Added options.py and core/artifacts.py as authorities for the named declarations.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", "Source inspection confirmed native Shapes ownership and nominal materialisation strategies; the Napari ROI integration test covers the UI seam", "On 2026-08-24, rechecked the complete explanation after materialization outputs gained nominal text behavior; no ROI ownership or projection claim changed.", @@ -1048,7 +1110,7 @@ }, { "path": "docs/source/architecture/runtime_value_system.rst", - "source_sha256": "130ae142dd1d147f24317d06c900d6701482fd492d580ee819082c8a756f7703", + "source_sha256": "43f47fa2b2c534958a47c42c887badc4492f83165772503e98ae2a994a18f3e5", "audience": [ "OpenHCS runtime and artifact maintainers" ], @@ -1057,51 +1119,74 @@ "authority": [ { "path": "openhcs/core/runtime_artifact_values.py", - "sha256": "9ac62ec4c4a6a6f7f375026ea112cc2e43f09ee6e6ca270a3d955be8ef5347cf", - "role": "RuntimeValue, ArtifactKey, location, and nominal payload wrapping." + "sha256": "930752071d37ef1a8669812e7a629e151690b3f33bf4a29dba92d1d479b9074d", + "role": "RuntimeValue, ArtifactKey, execution scope, and the output-normalization entry that calls the artifact type's contextualize_output_from_projector." }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "5f323ec8edd4221ee30e34ed0c9c09e0f4e45ef502749c1a2d331b7a5452af99", - "role": "RuntimeValueStore recording, replacement, query, and observation semantics." + "sha256": "b6a7231bc2964d472070731018f6a4935e5f23cf8f238a232045c32244e6545a", + "role": "RuntimeValueStore recording, replacement, RuntimeArtifactQuery resolution, and observation semantics." + }, + { + "path": "openhcs/core/runtime_plane_projection.py", + "sha256": "3c88ed042a7f1c65c9495a1c62777e008f1e59c4742ec0563b2849d77ca87cdd", + "role": "RuntimeSliceProjectableValue ABC and its indexed, invariant and identity-projectable subclasses." }, { "path": "openhcs/core/runtime_slice_projection.py", - "sha256": "8a1cb65d3025ed8a87ee59af26deac9765a8f59b21bf2cd61b6f96d82999641d", - "role": "Nominal runtime slice projection registry and aggregation behaviour." + "sha256": "8f45a17d6aabd4fae1df55e8d3cda2efdd8df2156357c69133f06ccf1cfb2d96", + "role": "RuntimeSliceProjection foreign-value list and rejection of undeclared owned values." + }, + { + "path": "openhcs/core/artifacts.py", + "sha256": "65eb0ec5c903366243ca8bd5ea07412e8d34bdb81e6015fa76b3c4c59838733f", + "role": "ArtifactType.contextualize_output root (unchanged) and its image, object-label, measurement, spatial-grid and spatial-graph overrides." }, { "path": "openhcs/core/runtime_image_values.py", - "sha256": "aec6e31b907c2ca5a69b3faf5fe3366654d9506b1191f90a7cbeb8ca1dbef08c", - "role": "Owns semantic image payload metadata, its serialized projection, geometry use, masks, source provenance, and context projection." + "sha256": "4801bdad5dd2bde9998e108836479923b2bb87023785eeff395936f3471fdf03", + "role": "ImagePayload metadata (mask, axes, plane axis, provenance) and contextualize_image_output for image sources." }, { - "path": "external/arraybridge/src/arraybridge/array_geometry.py", - "sha256": "0a8a5f1a61552b057997e94e4c89a19b5615875f307949e0d11784179f2a94f2", - "role": "Owns framework-neutral shape inspection without moving device-resident array data." + "path": "openhcs/core/runtime_object_labels.py", + "sha256": "a8ca5cc37f7049bb0ed06f1600afab9943d515e5ccb43dac0fb39f8f1bc7fbf7", + "role": "ObjectLabelValue.contextualize_image_output plane-axis rule for label-rendered images." }, { - "path": "openhcs/core/steps/function_runtime.py", - "sha256": "b1449120bebe7cf4898c055d841dc258c7d91d976d9e5ff7396aed81ad5011e7", - "role": "Runtime callable invocation and output-context strategy implementation." + "path": "openhcs/core/projected_image_output.py", + "sha256": "84ade707b65fab8b4b66006b8f01dd045920252f5b5cabe58bea8626165aae83", + "role": "SourceProjectedImageOutput and SelectedPlaneImageOutput source-plane selection." + }, + { + "path": "openhcs/core/callable_contract.py", + "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", + "role": "CallableContract.resolve_returned_plan_values binding returned positions to exact output plan refs." + }, + { + "path": "external/arraybridge/src/arraybridge/array_geometry.py", + "sha256": "0a8a5f1a61552b057997e94e4c89a19b5615875f307949e0d11784179f2a94f2", + "role": "Framework-neutral shape inspection without moving device-resident array data." }, { "path": "tests/unit/test_function_artifact_outputs.py", - "sha256": "600465c47acd7b69c1c2036a05173d9018e1d5c118ff71474a95d86fd14fda49", - "role": "Proves special artifacts bypass image-axis projection during runtime materialisation." + "sha256": "4c2b42cd9f31cc3cd63682039bb691cdd0af5bec3ca5b6f23f2ebea0fdbf0fac", + "role": "Proves image-output contextualization and that named artifacts bypass unrelated source-axis projection." }, { "path": "tests/unit/test_runtime_slice_projection.py", - "sha256": "4066679ba1293f7865f5c8aa9e0a1ee7cff9124b270180d458f2df1a7bf3e685", - "role": "Proves declared image slices are projected without implicit device-to-host conversion." + "sha256": "d47cad11b66f1bb3a7e77796bf5cca83599ef8ef4f876ac685f06e002dc9552a", + "role": "Proves undeclared value types are rejected and declared slices project without implicit device-to-host conversion." } ], "findings": [ - "The explanation did not state that declared special artifacts bypass image-axis projection through their nominal context strategy.", - "GPU integration exposed shape inspection that implicitly converted CuPy payloads to NumPy; generic geometry now belongs to ArrayBridge and semantic projection consumes it without transferring the data." + "RuntimeReturnedOutputMatcher was deleted; CallableContract.resolve_returned_plan_values now binds returned positions to plan refs.", + "RuntimeSliceProjectionStrategy (a registry of per-type strategies) was replaced by the RuntimeSliceProjectableValue ABC that each value family implements; RuntimeSliceProjection only lists foreign types. The section described the deleted registry.", + "FunctionOutputContextStrategy, UnchangedFunctionOutputContextStrategy and ImageOutputSourceContextStrategy were deleted (G3/G5); contextualization is now ArtifactType.contextualize_output with image dispatch to the source value's contextualize_image_output.", + "SelectedPlaneImageOutput preserves source-plane identity, not specifically channel identity; ImagePayload now also carries declared payload axes (G3)." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/runtime_value_system.rst in full and compared it against the changed authorities: runtime_artifact_values.py, runtime_stores.py, runtime_slice_projection.py, runtime_image_values.py, steps/function_runtime.py and the two test files; verified every named class/function exists, replaced the deleted output matcher, slice-projection strategy registry and function-output context strategies with their current owners (resolve_returned_plan_values, RuntimeSliceProjectableValue, ArtifactType.contextualize_output and contextualize_image_output), and confirmed the persisted-metadata reload fallback in source_workspace_projection.py still holds.", "Re-read docs/source/architecture/runtime_value_system.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/runtime_artifact_values.py; openhcs/core/runtime_stores.py; openhcs/core/runtime_slice_projection.py; openhcs/core/steps/function_runtime.py; tests/unit/test_function_artifact_outputs.py.", "Focused feature-lattice QA passed 159 tests covering the N-Z architecture and developer workflow authorities.", @@ -1176,7 +1261,7 @@ }, { "path": "docs/source/architecture/serialization_boundaries.rst", - "source_sha256": "28083415985860019b3ee15d0c69e9c07c903584a3890cd53b7a1676a6718c74", + "source_sha256": "5af8b42af40d836966f05ac2cf7cc943c7e34535687cd78ceb711a62e032bd56", "audience": [ "OpenHCS serialization and transport maintainers" ], @@ -1184,14 +1269,19 @@ "diataxis": "explanation", "authority": [ { - "path": "openhcs/serialization/json.py", - "sha256": "fa1e2aeae1e5d6896944d1516c17db0ac486157b335d7be06fcb553ab3203f1f", - "role": "Bounded JSON singledispatch projection for callables, registered types, dataclasses, paths, enums, and collections." + "path": "external/python-introspect/src/python_introspect/jsonable.py", + "sha256": "8fe06533a6dce179b7d77792b6ac768c2a64a1e9e79bbe926893498fae035bb4", + "role": "to_jsonable singledispatch encoder: scalars, mappings, sequences, paths, enums, callables by import path, dataclasses, and AutoRegisterMeta types by registry key." + }, + { + "path": "external/python-introspect/src/python_introspect/dataclass_projection.py", + "sha256": "e77d4f22b02e9d636e401c27643051a436b41b2f222bf17a0784e1c18a8e8425", + "role": "dataclass_from_mapping decoder paired with to_jsonable." }, { "path": "openhcs/core/image_file_serialization.py", - "sha256": "d3aec14d46696ac82d8cc0a7554f26550bdde76afd51c4e1a5d3be27e8545d5f", - "role": "Nominal image-container format and pixel-semantics registry." + "sha256": "644a01f8cccdf8e3d663202e3207d8750b4cf2f03f42c5bcc82dcaffb2c3fb5d", + "role": "ImageFileFormat registry: suffix matching, host preparation, read/write, source metadata, intensity scale, and declared pixel-band axes." }, { "path": "external/pycodify/src/pycodify/core.py", @@ -1199,9 +1289,12 @@ "role": "Executable Python rendering and formatter composition." } ], - "findings": [], + "findings": [ + "Authority openhcs/serialization/json.py was deleted (L2, c0a53a1e0); the JSON codec now lives only in python_introspect.jsonable and dataclass_projection, which the page already names. Authority replaced; prose unchanged." + ], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/serialization_boundaries.rst in full and compared it against the changed authorities: openhcs/serialization/json.py (deleted, L2) and openhcs/core/image_file_serialization.py; checked every to_jsonable dispatch case in python_introspect/jsonable.py, the dataclass_from_mapping decoder, OpenHCS to_jsonable.register users, and ImageFileFormat suffix/prepare/read/write/pixel-semantics members; no prose change was needed.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", "tests/unit/agent/test_agent_serialization.py and tests/unit/test_image_file_serialization.py cover the named boundaries" ] @@ -1266,7 +1359,7 @@ }, { "path": "docs/source/architecture/source_model.rst", - "source_sha256": "ba603eb36228385593c44bec049517320dfd29013ffb49cbdce81bc675ec11e0", + "source_sha256": "8d0e4c61c920be48ae0de60aea6ba29eb850d4cdb564646ba53422129aa69384", "audience": [ "Pipeline authors and OpenHCS source/compiler maintainers" ], @@ -1275,30 +1368,41 @@ "authority": [ { "path": "openhcs/core/source_bindings.py", - "sha256": "e0afbbe40cce3847bcc2314910d0083ecc579a5a0d6e4278c3a5df4c62e7eb01", - "role": "Public typed source-binding declaration vocabulary." + "sha256": "b1474cfd969bd2ab3af6caba7e6e36f300f59fba5eeaaada08fd2254aef36598", + "role": "Public source-binding declaration vocabulary and compiled binding/universe plans." }, { "path": "openhcs/core/source_binding_workspace.py", - "sha256": "7dce6e45b9e846713b246561340d43de378e251e35f6ef68d5527fccc1b31ad3", - "role": "Compiled virtual source workspace and binding projection." + "sha256": "4d28066a9f73b283fc637662e05fb2632c6fa18af164476c2899fce58cd647fe", + "role": "Source-binding workspace projector and materialization of the virtual workspace." }, { - "path": "openhcs/microscopes/source_bindings_handler.py", - "sha256": "88f7c624310ea31a4cf5de884fdae295e4675a578f8a2821a837140e4e4b07b2", - "role": "Arbitrary-folder ingestion and declared source-binding projection." + "path": "openhcs/core/dataset_sources/source_bindings_source.py", + "sha256": "fccda877f39e7b2f99d6cc95143480e5955186264e316e832ca9a69c2ae90b8c", + "role": "SourceBindingsSource: recursive folder inventory and declared source-binding projection." }, { - "path": "external/PolyStore/src/polystore/filemanager.py", - "sha256": "877a2b10731b12904b1784a5b42c9e39688e3a84f7f86018c1c2beee96ea2296", - "role": "Generic FileManager storage and backend composition." + "path": "openhcs/core/dataset_sources/choice.py", + "sha256": "7fb3c7ce701f0d629f288b79090044f43ac8ce1b51ca4f50937534407f226f1a", + "role": "AutoDetectedSource falls back to the registered declared-file source only when detection fails and bindings are non-empty." + }, + { + "path": "openhcs/core/source_projection.py", + "sha256": "deee0bf8b0f5c19bcbbf511fd5355767e65cfa0666c4c2b38cf5b709ae342919", + "role": "SourcePlaneProjection, SourceArtifactProjection, SourceProjectionSet and typed SourceDatasetDiagnostic." + }, + { + "path": "external/PolyStore/src/polystore/base.py", + "sha256": "8f2082ade15072bc8e8eba363c75a1d1cff7c67c74845e6485f69eab244634dc", + "role": "BackendBase.resolve_listed_address and optional physical_source_path versus DataSource.source_path." } ], "findings": [ - "The source explanation did not distinguish raw selector coordinates from projected component identity or state that the binding tuple, rather than a mirrored enabled flag, owns compiled-plan activity." + "SourceBindingsHandler and its module openhcs/microscopes/source_bindings_handler.py were replaced by SourceBindingsSource in the G4 dataset-source family; the fallback is chosen by AutoDetectedSource, not a handler factory." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/source_model.rst in full and compared it against the changed authorities: openhcs/core/source_bindings.py, openhcs/core/source_binding_workspace.py, openhcs/microscopes/source_bindings_handler.py (deleted, G4) and PolyStore filemanager.py; checked every named declaration, compiled plan, projection, diagnostic and PolyStore method exists, corrected the arbitrary-folder fallback to AutoDetectedSource/SourceBindingsSource, and confirmed CompiledStepPlan carries no separate source-binding enabled flag.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", "tests/unit/test_source_bindings.py, tests/unit/test_source_projection.py, and tests/unit/test_source_binding_workspace.py cover declarations, projections, and generic-folder ingestion", "On 2026-08-22, re-read the complete explanation against the current source-binding declarations, workspace projector, compiler grouping, and runtime projection; the focused OpenHCS documentation-drift suite passed 238 tests." @@ -1306,7 +1410,7 @@ }, { "path": "docs/source/architecture/special_io_system.rst", - "source_sha256": "e61acca7a6cb7e2c7cf41948d399b850695c9c245c46d0d5c85d1d5b0d0bfaf1", + "source_sha256": "3e48a170133fa5154e6cbb7fe7466855a28064eb30e45ecf0fd42d0e212e651e", "audience": [ "OpenHCS maintainers migrating old special-input and special-output code" ], @@ -1315,30 +1419,37 @@ "authority": [ { "path": "openhcs/core/callable_contract.py", - "sha256": "6a39b5e171901131ecb7280a78241e8ea72409a5003eabba3db34eb4851991d7", - "role": "Callable ABI including input/output/execution memory roles, runtime-owned parameter projection, and artifact contracts." + "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", + "role": "CallableContract input/output declarations and runtime-owned parameter projection." + }, + { + "path": "openhcs/core/pipeline/function_contracts.py", + "sha256": "13bcaf7c7a62654f512d2cc739edccd2deeff35d99970ca493f3e827afbda91a", + "role": "special_inputs decorator: records parameter names as runtime-managed callable metadata and excludes them from user parameters; no special_outputs decorator exists." }, { "path": "openhcs/core/pipeline/artifact_planning.py", - "sha256": "8f2a9fe778df5699218185b2b52f9537a57a929f64ae2a43915d14e04a2e7135", - "role": "Exact ArtifactSpecRef planning and nominal artifact strategy selection." + "sha256": "492024a74d5ce40dc326b162b3ac45edb0e55515a526a4f6512a71a8e4b11a77", + "role": "ArtifactGraph and ArtifactSpecRef planning during compilation." }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "5f323ec8edd4221ee30e34ed0c9c09e0f4e45ef502749c1a2d331b7a5452af99", - "role": "RuntimeValueStore recording, replacement, query, and observation semantics." + "sha256": "b6a7231bc2964d472070731018f6a4935e5f23cf8f238a232045c32244e6545a", + "role": "RuntimeValueStore recording, replacement, query, and observation during execution." }, { "path": "openhcs/interop/cellprofiler/module_artifact_contracts.py", - "sha256": "bf2e8f6c66a181218c8fd9afa9132922e86295f65d6102d976e1b36e41e17ee5", - "role": "Invocation-specific CellProfiler artifact contract derivation." + "sha256": "982680cd36409ea6d90a07d4e60893b162847f5525ed516891e995877dc32fd1", + "role": "CellProfilerModuleArtifactContracts: per-invocation artifact bindings from CellProfiler setting declarations." } ], "findings": [ - "The string-keyed special-I/O architecture is superseded; this URL now exists only to route legacy terminology." + "The string-keyed special-I/O architecture is superseded; this URL exists only to route legacy terminology.", + "The unreviewed page edit that dropped special_outputs left a broken sentence ('It names Python input parameters; they do not own ...'); fixed the grammar and stated that special_outputs no longer exists. Added the special_inputs declaration owner as an authority." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/special_io_system.rst in full, including its unreviewed special_outputs removal, and compared it against the changed authorities: callable_contract.py, pipeline/artifact_planning.py, runtime_stores.py, interop/cellprofiler/module_artifact_contracts.py; confirmed each named class and the special_inputs decorator exist with the roles the page assigns and that no special_outputs declaration remains; fixed the edited sentence.", "Re-read docs/source/architecture/special_io_system.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/callable_contract.py; openhcs/core/pipeline/artifact_planning.py; openhcs/core/runtime_stores.py; openhcs/interop/cellprofiler/module_artifact_contracts.py.", "Focused feature-lattice QA passed 159 tests covering the N-Z architecture and developer workflow authorities.", @@ -1364,18 +1475,26 @@ }, { "path": "openhcs/pyqt_gui/widgets/step_parameter_editor.py", - "sha256": "54e0bdd74f8d781c2bfdf1da559bbe7e2393f19ef74826d374a2281e8eb96258", - "role": "OpenHCS FunctionStep editing actions and domain-specific parameter policy." + "sha256": "d7d6a2af7d3dafbd3edd57d4a3c52a1f40812e21b95820d1e2190a57e9a97c0c", + "role": "FunctionStep editor widget: config-tree, code-document round trip, and OpenHCS parameter policy over the pyqt-reactive form." + }, + { + "path": "openhcs/authoring/session/step_scopes.py", + "sha256": "2d34e0765fb3fb3e20f33d23b3c2dcb9e9b213abd0c5d3ea7238a0adf3e16926", + "role": "StepEditorScope: FunctionStep scope identity used by the editor." }, { "path": "openhcs/pyqt_gui/services/window_handlers.py", - "sha256": "5080a81191a8cfc78ce826db7e241bd41d3df7607deea32128a06449e04a3aa8", - "role": "OpenHCS domain window-route registration over pyqt-reactive, with configuration-window presentation derived from its stable UI identity." + "sha256": "1523b0874cbd4f7048d713afef84311c18291918c432bbbe7305488c1baa03a6", + "role": "OpenHCS window-route registration, including definition-mutation guards and compiled-inspection actions through the plate session." } ], - "findings": [], + "findings": [ + "Authority changes (U1 session interface, D1 dead UI code) moved StepEditorScope to openhcs/authoring/session/step_scopes.py, routed mutation/inspection guards through plate_manager.session, and deleted the pickled .step load/save controller. The page names none of these details, so its boundary claims stay true; added step_scopes.py as the scope-registration owner." + ], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/step-editor-generalization.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/widgets/step_parameter_editor.py and openhcs/pyqt_gui/services/window_handlers.py; confirmed scope registration, code-document actions and compile/inspection routes still live in OpenHCS while form mechanics stay in pyqt-reactive; no prose change was needed.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", "Current imports place generic form/window mechanics in pyqt-reactive and OpenHCS actions in the named adapters", "Re-reviewed the public form-build completion, failure, and signal lifecycle against the complete dependent page and the updated pyqt-reactive owner documentation.", @@ -1398,7 +1517,7 @@ "authority": [ { "path": "external/PolyStore/src/polystore/filemanager.py", - "sha256": "877a2b10731b12904b1784a5b42c9e39688e3a84f7f86018c1c2beee96ea2296", + "sha256": "a9e9f3a8205216efc769accad06ab64d5373e6726733b2a1f2e3881613d1880e", "role": "Generic FileManager storage and backend composition." }, { @@ -1408,16 +1527,14 @@ }, { "path": "pyproject.toml", - "sha256": "6217b5f029796edcd83aa47fb8632d37aa7552344755c578fa4f181a6b23282d", - "role": "Current Python support, dependency, package discovery, and entry-point authority." + "sha256": "3592cc37444e2e1f5bb0284e939643039c0d513217861af8133dd8bd6e8a4325", + "role": "Declares polystore and arraybridge as separate runtime dependencies of OpenHCS." } ], - "findings": [ - "The transition page linked only PolyStore, grammatically assigned two packages to one owner, and duplicated its OpenHCS boundary link.", - "The redirect used raw Read the Docs URLs instead of the configured first-party PolyStore and ArrayBridge intersphinx inventories." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/storage_and_memory_system.rst in full and compared it against the changed authorities: external/PolyStore/src/polystore/filemanager.py (submodule advance) and pyproject.toml (polystore 0.6, arraybridge 0.4.1, JAX and openhcs-basicpy dependency changes); the redirect makes no version, backend or API claim those changes affect, and both packages remain separate dependencies with intersphinx targets in docs/source/conf.py; no prose change was needed.", "Re-read docs/source/architecture/storage_and_memory_system.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: external/PolyStore/src/polystore/filemanager.py; external/arraybridge/src/arraybridge/converters.py; pyproject.toml.", "Focused feature-lattice QA passed 159 tests covering the N-Z architecture and developer workflow authorities.", @@ -1443,7 +1560,7 @@ }, { "path": "docs/source/architecture/streaming_boundary_and_wrappers.rst", - "source_sha256": "b4943eeac6a59501b60ebfa16b0b093bc415b5bf8e4124409371b0d3641cbbfd", + "source_sha256": "f5c2031f28cff9e7b61a4b0c3ba0afb467568fa48c2e0c52468f6a83ec092242", "audience": [ "OpenHCS streaming, viewer, and runtime maintainers" ], @@ -1452,105 +1569,104 @@ "authority": [ { "path": "openhcs/core/viewer_streaming_service.py", - "sha256": "46982fd46f47b962033e02d82afb6c3ce2abbafdabfbce99e591d9c6dff2fbab", - "role": "Compiled output streaming, viewer settlement, state evidence, and lifecycle cleanup." + "sha256": "c96eca530f767002cfcc9aca4fe75c93d7a5848a7f42df9e5d336bf6712ed51e", + "role": "Compiled output streaming through StreamComponentMessageExtraAuthority and viewer settlement." }, { "path": "openhcs/core/streaming_config_declarations.py", - "sha256": "96ad52912439f8dd6b395b73b041313243f86f98f60eff8a2c3d43f97c7be340", - "role": "Registered viewer streaming configuration declarations compiled by OpenHCS." + "sha256": "fc6178165f7ddf80ce7f213774993fa16bbc85ec3538f7daa06ad9c4c17f6c1d", + "role": "ViewerFamily classes (NapariViewer, FijiViewer) and the ViewerType boundary enum derived from them." }, { - "path": "openhcs/runtime/viewer_protocol.py", - "sha256": "30441eab97da9ffcb5b9dc33d65c2ecc56c426e79bb2b27eedc78a68aaf13bc3", - "role": "OpenHCS viewer persistence policy, exact process ownership, platform policy, readiness, and settlement contracts." + "path": "openhcs/core/compiled_step_plan.py", + "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", + "role": "CompiledStepPlan.producer_identity_for_main_flow and producer_identity_for_artifact." }, { - "path": "openhcs/runtime/viewer_controls.py", - "sha256": "3227e6bf60e9b43fd792cc4299b6b50721741f012a0f7f856268b35022fd6062", - "role": "Declares caller payload controls, runtime projection policy, global shape budgets, and one atomic layer-isolation request." + "path": "openhcs/core/runtime_object_labels.py", + "sha256": "a8ca5cc37f7049bb0ed06f1600afab9943d515e5ccb43dac0fb39f8f1bc7fbf7", + "role": "ObjectLabelValue.contextualize_image_output plane-axis and provenance rule for label-rendered images." }, { - "path": "openhcs/runtime/napari_viewer_server.py", - "sha256": "64ccfef65a10e04878ef0806ae72d0446540efcbfc72af46c4e2ab785418a84c", - "role": "Applies bounded shape projection and preflighted layer isolation inside the owning Napari Qt process." + "path": "openhcs/core/orchestrator/bundle_execution.py", + "sha256": "f5ba09a2ec7c688b1ca08167ab398cc976a9e2c39b301ae98f3c5fd560a25c85", + "role": "Viewer bootstrap with rollback of the acquired set, settle-then-read-state order, and stopping of execution-owned viewers." }, { - "path": "openhcs/agent/services/viewer_window_service.py", - "sha256": "257718ca7da768132be2283326730f65d9b399f0ef852b2c7e1e9e48263f9eed", - "role": "Projects one typed bulk viewer response into agent results without client-side state mirroring or per-layer round trips." + "path": "openhcs/runtime/viewer_protocol.py", + "sha256": "c4bdae4bae6dee412f0d27a9cf75a95db53b3bd95c03cee7d046be16b52fe5b8", + "role": "ManagedViewerLifecycleMixin, ViewerProcessPlatform Linux policy, ViewerSettleProgress phases, and the execution-owned EndpointProcessGroup." }, { - "path": "openhcs/runtime/zmq_execution_server.py", - "sha256": "812ac9fc2363afa892410dbfebb461d4bc6f44c663e3ac4490e8edf41cf0dee2", - "role": "Releases execution-session process resources when the execution server stops." + "path": "openhcs/runtime/viewer_controls.py", + "sha256": "df0ac0aee66ba52a8b1706a1a99e03acc1300cfaf5cc74c2f93614a1c77465f6", + "role": "ViewerPayloadControlOptions, ViewerPayloadProjectionOptions, SUMMARY shape projection and ViewerLayerIsolationControlOptions." }, { - "path": "openhcs/core/execution_visualizer.py", - "sha256": "88d4332fbf10d63107d56675753b16a7afab49d0f16571949ae0bf2eda2be23c", - "role": "Declares the nominal application viewer contract consumed by compiled execution." + "path": "openhcs/runtime/napari_viewer_server.py", + "sha256": "038d7a76eeac8d69079591cf6eb7736af5415b28a7a067d6677bd15d115d11c7", + "role": "NapariImagePayload intake validation, bounded shape projection and preflighted layer isolation in the Napari Qt process." }, { - "path": "openhcs/core/orchestrator/compiled_plate_execution.py", - "sha256": "80998f355c3f4be9027cb91cabf581f51efd870fb86960f5f6ecd615bcdc89ef", - "role": "Acquires required viewers transactionally and rolls back the exact acquired set on launch or readiness failure." + "path": "openhcs/runtime/napari_streaming_handlers.py", + "sha256": "be585484075783f7717776d2b8ba163aab8a69aa24c560ca3844214485bd9eef", + "role": "Accepted intake-cycle and route error retention through settlement." }, { - "path": "external/PolyStore/src/polystore/filemanager.py", - "sha256": "877a2b10731b12904b1784a5b42c9e39688e3a84f7f86018c1c2beee96ea2296", - "role": "Generic FileManager storage and backend composition." + "path": "openhcs/agent/services/viewer_window_service.py", + "sha256": "cfe222586a221cb317099fc245216778f2ce15589604e5ce03410998dec4bb1a", + "role": "Projects one typed bulk viewer response into agent results without per-layer round trips." }, { - "path": "external/zmqruntime/src/zmqruntime/execution/client.py", - "sha256": "1b9453ea2be540b29849770cc3fe82f284f501e0e140797dd48c2c257ec36f12", - "role": "Generic execution client submit, wait, progress, and connection lifecycle." + "path": "openhcs/runtime/zmq_execution_server.py", + "sha256": "e24bdcb3b086cc60197ceb63f29a6f6b639a93cad175218941148a207eb285f2", + "role": "Releases execution-session process resources when the execution server stops." + }, + { + "path": "external/PolyStore/src/polystore/streaming/identity.py", + "sha256": "a8e48d76c8fd222515121e2ebc879dccc820e0ebf5b576a8728c1a7ffd35dc4e", + "role": "StreamProducerIdentity and its projection_key receiver route." }, { "path": "external/zmqruntime/src/zmqruntime/client.py", - "sha256": "fc8bdf68455673921d79742204bf6c88e42b52bbe407d10ef4a831e084ca86a8", - "role": "Provides exact viewer-process ownership, terminal observation, automatic child reaping, and the generic group used for execution-owned viewers." + "sha256": "7d3cce31d5abaa070927f02fe87d919687619e2cf4adbc37ab82483b6b8fc212", + "role": "EndpointProcessGroup, exact viewer-process ownership and terminal child reaping." }, { "path": "tests/unit/test_viewer_protocol.py", - "sha256": "f2226d3674b56cd46d2f93ad6764df96616104485d5f8ae15469882854492eaa", - "role": "Proves execution cleanup stops exact non-persistent processes while persistent viewers remain available for typed lifecycle shutdown." + "sha256": "04aa9e7e8bdc2ce6f3cd0705e531b35672c2369d5c46bb319b2eec17ee21dc75", + "role": "Proves execution cleanup stops exact non-persistent processes while persistent viewers remain available." }, { "path": "tests/unit/test_viewer_result_element_selection.py", - "sha256": "58480f4da61b427ec1c55560a77686e74df70404fbbf77e701ba386ad9f89a03", - "role": "Proves bulk isolation validates every route and navigation before any visibility, selection, or axis mutation." + "sha256": "b1dae580f0291f53be9453df6f6c6df5458046bd3ff947b550f7ddea94b29f96", + "role": "Proves bulk isolation validates every route and navigation before any mutation." }, { "path": "tests/unit/test_napari_streaming_handlers.py", - "sha256": "a338d6f2ef3094fa429f3d640709c7231eb741a7bf2859318bf4bbfa196afc07", - "role": "Proves one global ROI-member budget and metadata-only shape projection across payload records." + "sha256": "93244d4d09c5cd6d7a2645fca2482578ad52646f3bb9e6de5ebe505b157fbfea", + "role": "Proves one global ROI-member budget and metadata-only shape projection." }, { - "path": "tests/unit/test_orchestrator_lane_planning.py", - "sha256": "09c39f5cba68f8c15a0def8a09843b62d6da5c5a01e7558a9263ce4d311c078d", - "role": "Proves multi-viewer bootstrap rollback after partial acquisition and readiness failure." - }, - { - "path": "openhcs/runtime/napari_streaming_handlers.py", - "sha256": "86f4ab319c7c68b626e7e2a1adef81eb3d7244f044b5b3e4dddd78bbee2aa4b6", - "role": "Owns accepted intake-cycle and route error retention through settlement." + "path": "tests/unit/test_napari_accepted_work_settlement.py", + "sha256": "50cde8ceede31c3ef07db0c0aed69865d058617fd77f34d834bb48252a47fcf1", + "role": "Tests accepted display failures, settlement reporting and recovery." }, { - "path": "tests/unit/test_napari_accepted_work_settlement.py", - "sha256": "a6248305f0d4b27b92b3eb81fe5b75a31073695ad3b5ebfab1d76071d9d305e9", - "role": "Tests actual accepted display failures, settlement reporting and recovery." + "path": "tests/unit/test_orchestrator_lane_planning.py", + "sha256": "3d5c2f3662b8d4c330582af0c4a649b8f4d9582e064d9cf3159798c8934a2bbb", + "role": "Proves multi-viewer bootstrap rollback after partial acquisition and readiness failure." } ], "findings": [ - "Link checking showed that the viewer-streaming deep Read the Docs page is not deployed; the owner link now follows the maintained ZMQRuntime source instead of hard-coding a stale package release.", - "Execution-server cleanup registered persistent viewers in its transient process group and stopped them before post-run inspection; persistence policy now limits that exact group to non-persistent viewers.", - "A terminated retained endpoint process previously depended on later caller polling for child reaping; ZMQRuntime now retains one terminal observer per exact process.", - "ROI summaries transported full shape geometry and applied the per-record maximum independently; the runtime now composes one response-wide budget with metadata-only summary projection.", - "Layer isolation required state inspection followed by one navigation round trip per mounted layer; the viewer now preflights and applies the complete typed projection in one control operation.", - "Multi-viewer bootstrap did not retain incremental acquisition evidence, allowing a later launch or readiness failure to leak an earlier owned viewer." + "FunctionStepOutputProducerIdentityAuthority no longer exists; producer identity is derived by CompiledStepPlan.producer_identity_for_main_flow/producer_identity_for_artifact.", + "ObjectLabelImageOutputSourceContextStrategy was deleted; the label-rendered image rule is now ObjectLabelValue.contextualize_image_output.", + "openhcs/core/orchestrator/compiled_plate_execution.py was renamed bundle_execution.py (G7); the page's 'compiled plate execution' wording and the authority were updated.", + "Dropped authorities the page does not rely on (FileManager, zmqruntime execution client, execution_visualizer.py) and added PolyStore's StreamProducerIdentity owner." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/streaming_boundary_and_wrappers.rst in full and compared it against the changed authorities: viewer_streaming_service.py, streaming_config_declarations.py, viewer_protocol.py, viewer_controls.py, napari_viewer_server.py, viewer_window_service.py, zmq_execution_server.py, compiled_plate_execution.py (renamed bundle_execution.py, G7), zmqruntime client.py and the two viewer tests; checked every named class exists, corrected the producer-identity and label-image context owners, and confirmed the bootstrap rollback, settle/read-state/stop order, Linux XCB/MIT-SHM/vblank policy and settle phases in current code.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", "tests/unit/test_streaming_service.py and tests/unit/test_viewer_protocol.py exercise the compiled stream, readiness, settlement, and platform policies", "Sphinx linkcheck resolved the then-pinned zmqruntime viewer-streaming owner-source link", @@ -1588,25 +1704,26 @@ "authority": [ { "path": "openhcs/core/pipeline_document.py", - "sha256": "0849800b3e6474b8176fe35bc4aaebc63cfa85bd77251000c50da5f4c928b1b2", - "role": "PipelineDocument construction and atomic configuration-step identity." + "sha256": "8c554c8275c665f308eb098955bda517aa804eeb34418b5e0f41d412650ce7b5", + "role": "PipelineDocument and PipelineDocumentCodec: the declaration entry point of the end-to-end flow." }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "ace8ae0c3e54bbe604b97cec7c3973a1abcf90e1dc517c2adda947dc2c35e882", - "role": "Compilation order, declaration resolution, callable warm-up, framework-local device assignment, and bundle construction." + "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "role": "PipelineCompiler phases from declaration resolution to framework device assignment." }, { "path": "openhcs/core/compiled_execution.py", - "sha256": "941115f2dca2e53538d3141a59f3ae1f928a87cf741fcfa177f7cd64a5f41aea", - "role": "CompiledExecutionBundle and runtime-environment authority; execution-wide worker facts remain separate from per-step framework devices." + "sha256": "85972d418f0958e59f3185f07caf25a103bc9e101e481b2b63903377cc05e397", + "role": "CompiledExecutionBundle, worker start and runtime-environment plans handed to execution." } ], "findings": [ - "The previous broad integration narrative was replaced by the current typed system overview." + "The previous broad integration narrative was replaced by the current typed system overview; this URL only routes to it. B6 re-check: the target page and the authorities still exist (PipelineDocumentAuthority was renamed PipelineDocumentCodec, which this redirect does not name)." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/system_integration.rst in full and compared it against the changed authorities: openhcs/core/pipeline_document.py (PipelineDocumentAuthority renamed PipelineDocumentCodec), openhcs/core/pipeline/compiler.py and openhcs/core/compiled_execution.py; the redirect states no API detail and its target system_overview.rst exists; no prose change was needed.", "Re-read docs/source/architecture/system_integration.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/pipeline_document.py; openhcs/core/pipeline/compiler.py; openhcs/core/compiled_execution.py.", "Focused feature-lattice QA passed 159 tests covering the N-Z architecture and developer workflow authorities.", @@ -1617,7 +1734,7 @@ }, { "path": "docs/source/architecture/system_overview.rst", - "source_sha256": "87f6eec65645b2fd4a01de961126c765230ae0eb3480a1e3b4a7b3fb3618b28f", + "source_sha256": "cff7b61a5cdcf7f49a883725ccb25da48ded119783c7758952d3015eba85e362", "audience": [ "OpenHCS users, maintainers, and integrators" ], @@ -1626,63 +1743,68 @@ "authority": [ { "path": "openhcs/core/pipeline_document.py", - "sha256": "0849800b3e6474b8176fe35bc4aaebc63cfa85bd77251000c50da5f4c928b1b2", - "role": "PipelineDocument construction and atomic configuration-step identity." + "sha256": "8c554c8275c665f308eb098955bda517aa804eeb34418b5e0f41d412650ce7b5", + "role": "PipelineDocument: PipelineConfig plus the FunctionStep list that all front ends converge on." }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "3c840a89baccc8665d415175458280fd904c1a837b7fd879e303d7dbca129a1c", - "role": "Resolved step/session identity and typed plan assembly invariants." + "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", + "role": "ResolvedPipelineDefinition (resolved steps, scope ids, field provenance) and the per-axis CompilationSession." + }, + { + "path": "openhcs/core/compiled_step_plan.py", + "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", + "role": "CompiledStepPlan typed fields, including framework-local device assignment." + }, + { + "path": "openhcs/core/pipeline/framework_device_assignment.py", + "sha256": "cdfde10f996ab682952a1d4948d3de2d23d48353eae41f3124770e19fc28a193", + "role": "Declaration-derived framework footprint and per-step device selection." }, { "path": "openhcs/core/compiled_execution.py", - "sha256": "941115f2dca2e53538d3141a59f3ae1f928a87cf741fcfa177f7cd64a5f41aea", - "role": "CompiledExecutionBundle and runtime-environment authority; execution-wide worker facts remain separate from per-step framework devices." + "sha256": "85972d418f0958e59f3185f07caf25a103bc9e101e481b2b63903377cc05e397", + "role": "CompiledExecutionBundle contexts, transport contexts, worker assignments and runtime-environment plan." }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "5f323ec8edd4221ee30e34ed0c9c09e0f4e45ef502749c1a2d331b7a5452af99", + "sha256": "b6a7231bc2964d472070731018f6a4935e5f23cf8f238a232045c32244e6545a", "role": "RuntimeValueStore recording, replacement, query, and observation semantics." }, { - "path": "pyproject.toml", - "sha256": "6217b5f029796edcd83aa47fb8632d37aa7552344755c578fa4f181a6b23282d", - "role": "Current Python support, dependency, package discovery, and entry-point authority." - }, - { - "path": "external/python-introspect/README.md", - "sha256": "d861f8aff2e79d9e77fbb2286b2f353c5b568412482a57efce9a488e58d543fe", - "role": "States python-introspect ownership of callable/signature analysis and signature-derived declaration projection." + "path": "openhcs/core/runtime_plane_projection.py", + "sha256": "3c88ed042a7f1c65c9495a1c62777e008f1e59c4742ec0563b2849d77ca87cdd", + "role": "RuntimeSliceProjectableValue: value families own their slice projection." }, { - "path": "openhcs/core/compiled_step_plan.py", - "sha256": "6d46571e6f2d069b98a92b41d711b2e20bfab9f0493cf73d3f150d5be08e7f96", - "role": "Typed step-plan authority including framework-local device bindings." + "path": "external/python-introspect/src/python_introspect/jsonable.py", + "sha256": "8fe06533a6dce179b7d77792b6ac768c2a64a1e9e79bbe926893498fae035bb4", + "role": "python-introspect's to_jsonable JSON encoder." }, { - "path": "openhcs/core/pipeline/framework_device_assignment.py", - "sha256": "f77f3fb2d30c4af8cab6210dbe294f9fb5e246dad0a6bbf88bcaf1e2b58dead7", - "role": "Declaration-derived framework footprint and per-step device selection." + "path": "external/python-introspect/README.md", + "sha256": "059dcfaafc0b2222e8e64b7fe072d24bddfb5a347a6f25113e715ed7946e44a1", + "role": "States python-introspect ownership of callable/signature analysis and declaration projection." }, { "path": "external/arraybridge/src/arraybridge/types.py", - "sha256": "7b94760d57e54008d7adf4e4d37dca45d8a2c288a9dc8400be00c26148cf7624", - "role": "Owns declaration-derived child-process environments for optional array frameworks." + "sha256": "08f7ff4df8807bb3340e00a02c6276b1c702a0f00a2eedacd701142736c5d804", + "role": "Declaration-derived child-process environments for optional array frameworks." }, { "path": "external/metaclass-registry/src/metaclass_registry/discovery.py", - "sha256": "f92c94a448701a8dec278c8c6ce4b0a48c49ffc7df48620c5f2a51bc1734d24b", - "role": "Owns registry discovery imports that preserve host-owned root logging." + "sha256": "757958272d11633e7b854c7104c07f0d90845c37fee6ab0c3646fb14c2ed8daa", + "role": "Registry discovery imports that preserve host-owned root logging." } ], "findings": [ - "Execution-wide GPU registry ownership was removed; framework-local bindings now remain on the compiled step plans that derive them.", - "The extracted-package owner summary understated ArrayBridge's callable memory-role, framework-device, and execution-scope mechanics.", - "The extracted-package summary now includes python-introspect's callable-declaration projection boundary.", - "The system overview omitted declaration-derived child import environments and logging-safe discovery from the extracted-package boundary." + "StepSnapshot was deleted: ResolvedPipelineDefinition now holds resolved steps with step scope ids and per-field provenance, and CompilationSession holds that definition instead of snapshot tuples and an ObjectState map. The flow diagram and the resolution/compilation sections described the deleted type and its index invariants.", + "Runtime slice projection is owned by each value family (RuntimeSliceProjectableValue), not by projection strategies.", + "python-introspect now also owns the JSON codec after openhcs/serialization/json.py was deleted (L2); the owner list omitted it." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/system_overview.rst in full and compared it against the changed authorities: pipeline_document.py, pipeline/compilation_session.py, compiled_execution.py, runtime_stores.py, compiled_step_plan.py, framework_device_assignment.py, pyproject.toml and the python-introspect/arraybridge/metaclass-registry owners; replaced StepSnapshot with ResolvedPipelineDefinition, restated the session invariant to what CompilationSession enforces, moved slice projection to value families, added the python-introspect JSON codec, and confirmed the CompiledStepPlan fields and bundle environment facts.", "Re-read docs/source/architecture/system_overview.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/pipeline_document.py; openhcs/core/pipeline/compilation_session.py; openhcs/core/compiled_execution.py; openhcs/core/runtime_stores.py; pyproject.toml; openhcs/core/compiled_step_plan.py; openhcs/core/pipeline/framework_device_assignment.py.", "Focused feature-lattice QA passed 159 tests covering the N-Z architecture and developer workflow authorities.", @@ -1719,25 +1841,26 @@ "authority": [ { "path": "pyproject.toml", - "sha256": "6217b5f029796edcd83aa47fb8632d37aa7552344755c578fa4f181a6b23282d", - "role": "Current Python support, dependency, package-discovery, and installed entry-point authority." + "sha256": "3592cc37444e2e1f5bb0284e939643039c0d513217861af8133dd8bd6e8a4325", + "role": "No Textual dependency or entry point; openhcs-gui gui-script points at the PyQt application." }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "a156a006a3c40f43107b9b1f6ecb97ed150879eb88ded8ef89e25214309ae33a", - "role": "Desktop composition and live UIConfig application boundary." + "sha256": "e43085c1eaf0c836ce34e1a1dae7740488d91bdfea56844e7ddd4e6925561cf6", + "role": "PyQt desktop main window composition." }, { "path": "tests/unit/pyqt_gui/test_ui_config_lifecycle.py", - "sha256": "eeb7c260d3ae8bc7b5e6d94917b1e5e9a214feac9e2a0c2651552950cfe3a179", - "role": "Prevents the removed Textual package authority from returning alongside the supported PyQt lifecycle." + "sha256": "5fd4194e70de5210a6b51283693a1d25c6a6a1278b8a869c7deebcbd583503e0", + "role": "Guards that openhcs/textual_tui does not return." } ], "findings": [ - "The transition page still described deprecated Textual sources as retained after their implementation had been removed." + "The deprecated Textual implementation remains removed; the page routes readers to the PyQt desktop. B6 re-check found the claims unchanged by the pyproject and main-window changes." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/tui_system.rst in full and compared it against the changed authorities: pyproject.toml, openhcs/pyqt_gui/main.py and tests/unit/pyqt_gui/test_ui_config_lifecycle.py; confirmed no Textual dependency, package or entry point exists, openhcs-gui still launches openhcs.pyqt_gui, the textual_tui guard test remains and docs/archive still exists; no prose change was needed.", "Re-read docs/source/architecture/tui_system.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: pyproject.toml; openhcs/pyqt_gui/main.py; tests/unit/pyqt_gui/test_ui_config_lifecycle.py.", "On 2026-08-22, revised the transition page after deleting the excluded Textual package and passed 29 focused website and UI-lifecycle tests.", @@ -1766,7 +1889,7 @@ }, { "path": "docs/source/architecture/ui_services_architecture.rst", - "source_sha256": "f398733f3501923efcbd7e41fc8da2681c9ffe1e78c2d1997fc65df5e731de3a", + "source_sha256": "207497e08bf32257edc1eb223a012340d5e2b09e3772c6c9498b5413e599decc", "audience": [ "OpenHCS desktop, ObjectState, and UI-service maintainers" ], @@ -1780,12 +1903,12 @@ }, { "path": "openhcs/runtime/zmq_config.py", - "sha256": "842b25ffd542f99923dafb2f67d23bac0263f38d85e64ef1bdbaba3da671b792", - "role": "OpenHCS-owned ZMQConfig subtype and process-level transport fields." + "sha256": "7c96c39d684d450ae045d1de1186dd27c5d67e042996857d5cdfcda6c2d4d069", + "role": "OpenHCSZMQConfig: ExecutionTransportConfig subtype that sets OpenHCS application names." }, { "path": "openhcs/pyqt_gui/services/window_handlers.py", - "sha256": "5080a81191a8cfc78ce826db7e241bd41d3df7607deea32128a06449e04a3aa8", + "sha256": "1523b0874cbd4f7048d713afef84311c18291918c432bbbe7305488c1baa03a6", "role": "Registers OpenHCS domain window routes, including declaration-owned config ObjectState scopes and stable configuration-window presentation, over pyqt-reactive." }, { @@ -1795,12 +1918,12 @@ }, { "path": "tests/unit/pyqt_gui/test_ui_config_lifecycle.py", - "sha256": "eeb7c260d3ae8bc7b5e6d94917b1e5e9a214feac9e2a0c2651552950cfe3a179", + "sha256": "5fd4194e70de5210a6b51283693a1d25c6a6a1278b8a869c7deebcbd583503e0", "role": "Proves one absolute environment-selected UIConfig cache path and rejects relative paths." }, { "path": "external/pyqt-reactive/src/pyqt_reactive/services/ui_thread_dispatch.py", - "sha256": "f2dbe7e8ce8ac0ad3116001d7dfcd91affd516a34615775ce4ce9921480e40bd", + "sha256": "f6daf3e8588888fff7490775fa048bac9daca189e16afe25188791b2985b8535", "role": "Owns bounded callable dispatch, outcomes, cancellation, and shutdown on the Qt application thread." }, { @@ -1819,19 +1942,24 @@ "role": "Exercises real QApplication shutdown in a subprocess and asserts native parent and child destruction while Python references remain alive." }, { - "path": "openhcs/agent/ui_bridge_actions.py", - "sha256": "28c494cc9335ac84482787f1949d9937fd818864407d56ca23ba585a3b0156a1", - "role": "Owns main-window action behaviour, admission, effects and recovery warnings." + "path": "openhcs/authoring/session/operations/application.py", + "sha256": "628fb1dd209d62d1d5830e710cf6e0ea6804f576569489da7df42f974652ffd8", + "role": "CheckForUpdates, RestartApplication and ExitApplication session operations with availability, side effects and reconnect warnings." + }, + { + "path": "openhcs/pyqt_gui/services/ui_bridge_session_actions.py", + "sha256": "033aee9296f6f1c75f01d1fc3a39e5aeb0e7c13c15a7e4396872d3d014673123", + "role": "SessionOperationActionProvider projects session operation classes as UI-bridge actions." }, { - "path": "openhcs/pyqt_gui/services/desktop_restart.py", - "sha256": "24ca24c3afc0038a268280a73ae7615045eb107ee541a59c13fcdf07005603a2", - "role": "Owns explicit session-preserving restart admission and detached relaunch." + "path": "openhcs/desktop/update.py", + "sha256": "7bd5ab7abffdc20a9f6f85fb2e5f169ecda462e50b210b175d918453dbebfb3c", + "role": "DesktopRestartSession capture and restoration shared by restart, update and version-mismatch recovery." }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "a156a006a3c40f43107b9b1f6ecb97ed150879eb88ded8ef89e25214309ae33a", - "role": "Applies the same preview policy to both managers at startup and after configuration save." + "sha256": "e43085c1eaf0c836ce34e1a1dae7740488d91bdfea56844e7ddd4e6925561cf6", + "role": "OpenHCSMainWindow.set_ui_config consumer reconciliation and rollback, and preview-policy propagation to both managers." }, { "path": "external/pyqt-reactive/src/pyqt_reactive/strategies/preview_formatting.py", @@ -1840,7 +1968,7 @@ }, { "path": "tests/unit/pyqt_gui/test_main_config_propagation.py", - "sha256": "291dc45ca20ead6025c255a8109a2e6c55d90bd7858e80ad12484dafa733d7c5", + "sha256": "303d90b43bee7bd531a1c7664fbf9707a2e7be1696a5147614f93c40e5c7021f", "role": "Verifies live preview policy propagation through the main configuration lifecycle." }, { @@ -1882,17 +2010,22 @@ "path": "external/pyqt-reactive/tests/test_deferred_callback.py", "sha256": "1c59913d1b19a9725e5735bbd342dc115fdf4b555863427169b5001324e83b1c", "role": "Real Qt regressions cover callback retirement, owner destruction, widget cancellation and persistent row identity." + }, + { + "path": "openhcs/pyqt_gui/widgets/plate_manager.py", + "sha256": "bf77012db6c0984deebf09bc5d696d8ea8f273ff535ad964354b6ddbfe839791", + "role": "Plate Manager renders the Session dataset list, holds no session state, and forwards transport and progress settings to the session." } ], "findings": [ - "The page named the removed ScopeWindowFactory instead of ScopeWindowRegistry and incorrectly assigned the OpenHCSZMQConfig subtype itself to the generic transport package.", - "The UI configuration explanation did not record the single absolute process-local cache override used by isolated launches.", - "The window-composition explanation did not record that process configuration scopes share one stable window while the requested ObjectState scope and field select the owning tab.", - "Added list-preview policy to the existing UI configuration lifecycle without duplicating generic settings.", - "Clarified row-arrow overrides and repeated graphical guides, distinguishing shared wrapping defaults from row-owned display choices." + "MainWindowAction and openhcs/agent/ui_bridge_actions.py were deleted (U1); main-window actions are SessionOperation classes projected by SessionOperationActionProvider.", + "openhcs/pyqt_gui/services/desktop_restart.py was deleted (D1/U5); DesktopRestartSession now lives in openhcs/desktop/update.py. Removed the 'accepted receipt' wording.", + "OpenHCSZMQConfig now subclasses zmqruntime's ExecutionTransportConfig, not ZMQConfig.", + "Plate Manager no longer owns selection or orchestrator delegation; it renders the Session and forwards transport/progress settings to it." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/ui_services_architecture.rst in full and compared it against the changed authorities: runtime/zmq_config.py, services/window_handlers.py, pyqt_gui/main.py, ui_thread_dispatch.py, the two UI config tests, agent/ui_bridge_actions.py (deleted, U1) and services/desktop_restart.py (deleted); verified every named class exists, replaced MainWindowAction with session operations, moved DesktopRestartSession to openhcs/desktop/update.py, corrected the OpenHCSZMQConfig base, and restated Plate Manager as a renderer of Session state; UIConfig fields, set_ui_config ordering and application shutdown were confirmed in code.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", "tests/unit/test_ui_zmq_config_authority.py passed; source inspection confirmed ScopeWindowRegistry routing and OpenHCSZMQConfig inheritance", "On 2026-08-22, re-read the complete explanation against UIConfig cache ownership and lifecycle tests; the focused OpenHCS documentation-drift suite passed 238 tests.", @@ -1943,7 +2076,7 @@ }, { "path": "docs/source/architecture/zmq_execution_service_extracted.rst", - "source_sha256": "dcaf4e63eb604312edadcb3442479ac6e369842231bb5caeb3e5404ecf693155", + "source_sha256": "a30938b8339fc2a6941b9164dd917257d7dff9250c4ce1139ec24a962361315f", "audience": [ "OpenHCS contributors following the former local ZMQ execution-service URL" ], @@ -1952,47 +2085,47 @@ "authority": [ { "path": "external/zmqruntime/src/zmqruntime/execution/client.py", - "sha256": "1b9453ea2be540b29849770cc3fe82f284f501e0e140797dd48c2c257ec36f12", + "sha256": "dec1edbc173d93d20d07564e4487a58e0c714588b6e2d01a4b75b5b03d59b57c", "role": "Generic execution client submit, wait, progress, and connection lifecycle." }, { "path": "openhcs/runtime/zmq_execution_server.py", - "sha256": "812ac9fc2363afa892410dbfebb461d4bc6f44c663e3ac4490e8edf41cf0dee2", + "sha256": "e24bdcb3b086cc60197ceb63f29a6f6b639a93cad175218941148a207eb285f2", "role": "OpenHCS compiled-execution adapter over the generic execution server." }, { "path": "openhcs/runtime/zmq_execution_observation.py", - "sha256": "e52b2e9784daa396b493f1f569fbcbb0d394e7e3c03d1a0c67d6594778ce64d3", + "sha256": "3cc43d3eee6c4701c23fec183ecd895a71acdfcc14acfe2afca62625ccb7381b", "role": "Owns the ordinary value/outcome export schemas, producing execution identity, and exclusive evidence-file creation." }, { "path": "openhcs/core/compiled_execution.py", - "sha256": "941115f2dca2e53538d3141a59f3ae1f928a87cf741fcfa177f7cd64a5f41aea", + "sha256": "85972d418f0958e59f3185f07caf25a103bc9e101e481b2b63903377cc05e397", "role": "CompiledExecutionBundle and runtime-environment authority; execution-wide worker facts remain separate from per-step framework devices." }, { "path": "external/zmqruntime/src/zmqruntime/messages.py", - "sha256": "0df6261b31d3e2758ced4c7ee5933dde0e866b8e30d7927c3fcb2641a8d97d44", + "sha256": "ba7b9bac8f0202690f1e2bd17f445fe1741f43642f0a36e4a806bb577c2d1aa1", "role": "Declares endpoint application identity, generic exact compatibility, and member-owned execution lifecycle semantics." }, { "path": "external/zmqruntime/src/zmqruntime/execution/lifecycle.py", - "sha256": "23596c31b9efe812f1ab66b4300ee1bb19125ea9bef6e51ddcd8acaafb0f4ed6", + "sha256": "5e984c7a5b154a2ed67028871451a8f8a1343633510d7b8ad1cf2ad5468272b3", "role": "Owns thread-safe terminal transitions and targeted cancellation outcomes." }, { "path": "external/zmqruntime/tests/test_execution.py", - "sha256": "eadc940b9c04b221b993892442bcc71bd489de0687696bc037b3de931c0fffe5", + "sha256": "8d4bf962418d8338c165a583d391a9691aa5a167a76e41542adb8b20f3a21a17", "role": "Proves immutable per-execution progress observations alongside terminal cancellation, readiness cleanup, and lifecycle guarantees." }, { "path": "external/zmqruntime/src/zmqruntime/client.py", - "sha256": "fc8bdf68455673921d79742204bf6c88e42b52bbe407d10ef4a831e084ca86a8", + "sha256": "7d3cce31d5abaa070927f02fe87d919687619e2cf4adbc37ab82483b6b8fc212", "role": "Retains the exact readiness handshake as the authoritative connected-endpoint observation while preserving the established client readiness extension point." }, { "path": "external/zmqruntime/src/zmqruntime/transport_modes.py", - "sha256": "a4287d0045e69506b4f8b570aeaa14fd6b6b25a61c86467f67ac2ee8bdc74d83", + "sha256": "12bec1c13d87bb8f05b049dc76d3cb7c130414c9e69d6220228ceeb0087f9ad8", "role": "Owns transport-specific endpoint operations, including IPC deadlines and data/control pair availability." }, { @@ -2007,76 +2140,96 @@ }, { "path": "openhcs/runtime/zmq_execution_client.py", - "sha256": "31081f2a323a81c060aede1201a4bf1a1b39a51f37928c281a419ffaceadfe60", + "sha256": "147fb663d3ea60b1348063440e05b77d54b40634d1ae81cf4fb4f3b5457fbc99", "role": "Spawns the execution server with the ArrayBridge-projected child environment and carries typed execution transport." }, { "path": "external/zmqruntime/src/zmqruntime/timeouts.py", - "sha256": "bb5121a03070f0f2a0ed76170a2324512ed4a4d64536ecc7ba8bdc6a4b8961d0", + "sha256": "192f0e5ad6905cf60a197ba1a207c30eb3c9a98e6a45425ae20a7711f4b46c92", "role": "Declares the reusable monotonic operation deadline consumed by connection and control boundaries." }, { - "path": "openhcs/agent/services/execution_session_service.py", - "sha256": "1a9a70d2924b1854338612b212f00888bacba5cd33eb23608be294bb06c76399", - "role": "Retains the exact submitting client, projects its authoritative progress, caches terminal status, and releases the client once." + "path": "openhcs/authoring/session/submission.py", + "sha256": "710ac218d1db01f231b67bf0fe229dc03fb428bc6655b23671f3a026be91c673", + "role": "ExecutionSubmission records each accepted submission with its request and endpoint and follows status to terminal." + }, + { + "path": "openhcs/authoring/session/session.py", + "sha256": "587047b63972df89ec46a01ed2c92ff3ef9577335f58a980297db02094e4e8e0", + "role": "Session owns the one execution client and keeps FinishedExecution (request, endpoint, ExecutionRecord) per execution." + }, + { + "path": "openhcs/authoring/session/execution_control.py", + "sha256": "1ee26c917eb106809092f9cb9f906ee8f5e32bbb53700826b369646879eb2544", + "role": "Session stop: endpoint shutdown (graceful or forced) and cancellation of active datasets." + }, + { + "path": "openhcs/runtime/zmq_orchestrator_environment.py", + "sha256": "5301811d7a37fa500f4c511ce9e57ac3620b172331f0ffc76a29e3a63797bf85", + "role": "Opens execution_subject_id when declared, otherwise subject_id." + }, + { + "path": "external/zmqruntime/src/zmqruntime/execution/config.py", + "sha256": "a3f3ba1f16fbedeb282825fbc381d4cc788fa8ac1600fd50fa1a27b5c65421db", + "role": "ExecutionTransportConfig submission and control timeout budgets inherited by OpenHCSZMQConfig." }, { "path": "tests/unit/test_zmq_submission_timeout.py", - "sha256": "b525a1cc113bdae987c7f202369cc251ca1129188bd8d3adb834aec44f4c8393", + "sha256": "acea5dcb72c820ed8c1f225dc1960dfd615ad3f002ef6aec8e78b75bf8e50b3e", "role": "Proves a total submit deadline prevents execute dispatch after slow startup and preserves the declared connection inactivity timeout." }, { "path": "openhcs/runtime/function_catalog_preparation.py", - "sha256": "63abd2cc72de8baacff723e4a574a74f0f6ef422b17eb32ca5fabead689505fc", + "sha256": "c4710d17e92924c685e7b1f41817de0b3ae6656f5a0ea70d12b8a3308d0cb512", "role": "Owns persistent registry preparation and the live endpoint's single asynchronous catalogue-preparation operation." }, { "path": "openhcs/runtime/zmq_execution_server_launcher.py", - "sha256": "a9110cb65bfe8119e8f60816305f9d5a4bc7e94377eb78c2d13c073cba16eaf5", + "sha256": "09c3dd01d5bfd0fe2d08d6d8ffe0c9c806a817940d2df2d176ae5108cf0a1708", "role": "Executes dedicated persistent catalogue preparation before importing or constructing an execution server." }, { "path": "tests/unit/test_zmq_execution_server_process.py", - "sha256": "710f6f723e90679c21a87e10710db8faaf8556ef8376cb93b7ba891f01855e83", + "sha256": "a3c9a5c03de440b49dd8575879d01514dffa0da32cbbfe15c0d363711685d964", "role": "Proves capability-preparation mode does not construct or bind the execution server." }, { "path": "openhcs/processing/backends/lib_registry/registry_service.py", - "sha256": "046d3d088f685a993ddad6b5923cf63beef84b201127b72ae32af6721dd5ad63", + "sha256": "daef8925338fddf8504a9faa066af42456a13fe4602976bf64be06547a45fcc8", "role": "Prepares the canonical catalogue in a child environment projected by ArrayBridge declarations." }, { "path": "openhcs/processing/backends/lib_registry/unified_registry.py", - "sha256": "70462cddcad1bfee447ce0786a9e4a85abbd297726d59a1208156af0ef3a3743", + "sha256": "b1e9253021b9c3ab14a070b7d0f10909f641ba07784c794ad95d2ce36041b1d3", "role": "Processing-contract execution, registry-owned catalogue admission and cache semantics, and external-signature authoring exclusions." }, { "path": "openhcs/processing/backends/lib_registry/openhcs_registry.py", - "sha256": "aa3079341cb2af8788938e14efee6f9f6c4b6c01874227e74c6340850d525884", + "sha256": "3fff5c3549ef8e1073d75e6d5544b2cd75fbb7cc9b17263207760ffca45cf885", "role": "Derives the public native callable catalogue while retaining explicit reconstruction from each callable declaration." }, { "path": "tests/unit/test_library_registry_discovery.py", - "sha256": "a7ee2d3a2002ea291876c27aa2dce51a889cea6be9a3055602c76f1e9cf76f1b", + "sha256": "e612d181d0053bacb38cb7bd3a73e8af8b3f7cdbe3789c1b7deb8f5f6a655170", "role": "Proves source-revision cache invalidation, stable optional-runtime admission, declaration-owned projection, runtime-owned external-signature exclusions, and no partial catalogue after failure." }, { "path": "tests/unit/test_cellprofiler_processing_backend.py", - "sha256": "42c779c73969d25cff9b48700599b2af1366cec1cfa57c94a8c0b950b9487c63", + "sha256": "d5d53e859e04cafd3c815296e9c58c3642f9e788a45d5604ab54a9ce0c892c27", "role": "Proves distinct persistent cache identities for CPU-only and GPU-capable function-catalogue admission contexts." }, { - "path": "openhcs/pyqt_gui/services/desktop_restart.py", - "sha256": "24ca24c3afc0038a268280a73ae7615045eb107ee541a59c13fcdf07005603a2", + "path": "openhcs/desktop/restart.py", + "sha256": "53028fa6380292520ca42945b12c6844019dd91a77aef3d15e20108830a18c0b", "role": "Carries the saved desktop session and ObjectState history across a detached process restart." }, { - "path": "openhcs/pyqt_gui/services/zmq_version_restart.py", - "sha256": "86d4bf273c75417b2c4f5c38ac83a66e3487bb233be88a608140a0bdb4c8bc67", + "path": "openhcs/desktop/zmq_version_restart.py", + "sha256": "84b4f13432862bc9d61dbbcd19d4b54a0240fb243f9ddd2fb94a7b68f5808acb", "role": "Owns the one state-preserving mismatch decision, endpoint replacement, and desktop restart transaction." }, { - "path": "openhcs/pyqt_gui/services/desktop_restart_worker.py", + "path": "openhcs/desktop/restart_worker.py", "sha256": "52d8a8f223d49dbd9ef42951fe507dfbea803b33ff4dfce3e198f4b1a9fe4ed7", "role": "Waits for the current UI process to exit before launching the captured restart command." }, @@ -2086,38 +2239,38 @@ "role": "Proves the execution server advertises the OpenHCS version declaration in its handshake." }, { - "path": "tests/unit/pyqt_gui/test_desktop_restart.py", - "sha256": "aafaa6e7a46859f50b44232e12f8db606e0dd4631340a834694a0fe795b5ff70", + "path": "tests/unit/desktop/test_desktop_restart.py", + "sha256": "50425f39e481a6fff384b9732bc0f2cd68f223d3ac980234a9142b6db981b48f", "role": "Proves the themed mismatch decision and state-preserving version-restart transaction." }, { - "path": "tests/unit/pyqt_gui/test_zmq_version_restart.py", - "sha256": "052699f393497ab23ec49d23fa136d969245c95c0973d6e9898aedfb860c43f9", + "path": "tests/unit/desktop/test_zmq_version_restart.py", + "sha256": "f1968ec55295c577e118ada34ba80d987f50d69df30105b8b3a65a1a774c0018", "role": "Proves success, replacement failure, executor shutdown, capture failure, and restart-launch failure preserve one transaction owner." }, { - "path": "tests/unit/pyqt_gui/test_desktop_restart_worker.py", - "sha256": "a9fbda7d6c78972d6300cd9fe67c2e88497b6b802a9b48057568007bb33b0e1c", + "path": "tests/unit/desktop/test_desktop_restart_worker.py", + "sha256": "e20c6d3f3122bd09a304b7896245c35ba4fc14bb71943afbd68c9428c5374a90", "role": "Proves parent-exit gating, Windows handle cleanup, exact restart arguments, and fail-closed launch outcomes." }, { "path": "scripts/smoke_installed_desktop.py", - "sha256": "f745f93e245e5c1b02f6e0ece3a4a0e2af086c3cab77282e3907a167f5ce0a7f", + "sha256": "d5ce5e2a2b3590be2101b487b2957b270f386dcf47b4c197f1ce4c2ab6aa9d65", "role": "Executes the GUI acceptance probe and packaged restart worker from the installed environment while deriving the outer process ceiling from the probe's timing authority." }, { "path": "tests/installer/test_installed_desktop_smoke.py", - "sha256": "5ac9f0e9e0e2ffe73cbab940da132421424e355c354b7476a8a072090c5aad00", + "sha256": "bf6fa5db1ff253c4ee3df16b3d46bdc64e87d54171770287baeed3516d44bb05", "role": "Proves platform smoke orchestration passes the exact installed Python, install root, environment, and declaration-derived GUI operation and process-exit budgets to the installed verifiers." }, { "path": "openhcs/processing/custom_functions/runtime_registry.py", - "sha256": "ee935ccc09f3a967d84c9ebd08511ad729d53e73d5bceb52074b39a1cd6ead9c", + "sha256": "160fb2fbad1b4d2f1ffaf784bacc5d7355d78c2720e718083cfc8c05748384df", "role": "Atomic process-local projection of persisted custom-function declarations without a copied global catalogue." }, { "path": "external/zmqruntime/src/zmqruntime/transport.py", - "sha256": "6ddaf01cb423565391186fb26e5d0826c6e41aead7dce09ee0d6c8270df0bcd1", + "sha256": "e486a4215d4b977553030e897589bccac637dfe4e9abb28cfb7594139b2196c2", "role": "Owns typed transport endpoints, configured data/control pairs, exact occupancy, proof-gated stale cleanup, and forced local release." }, { @@ -2127,37 +2280,26 @@ }, { "path": "external/zmqruntime/src/zmqruntime/execution/progress_stream.py", - "sha256": "b2dd8e27f8fcac010beef3f8532fd9d584d7bae8c3f5591dcee0e5a235d6213a", + "sha256": "f01e1b98768b288bb921b757f1e51359e7fb2eddb2dbda18f54e4b11bbc4ccea", "role": "Owns the immutable latest event and monotonic sequence wire observation retained per execution." }, - { - "path": "tests/unit/agent/test_agent_services.py", - "sha256": "0c3a5225c1e832d9fbd5a423ef0cfd495451c0872b6b718d5fd66726f0758fdf", - "role": "Proves same-client status reuse, progress serialization, terminal caching, and one-time client release." - }, { "path": "external/arraybridge/src/arraybridge/types.py", - "sha256": "7b94760d57e54008d7adf4e4d37dca45d8a2c288a9dc8400be00c26148cf7624", + "sha256": "08f7ff4df8807bb3340e00a02c6276b1c702a0f00a2eedacd701142736c5d804", "role": "Owns the framework child environment consumed by catalogue preparation and server startup." } ], "findings": [ - "The transition page now records generic compatibility ownership, the retained handshake, and the OpenHCS restart projection without introducing a second connection-state authority.", - "The page did not previously describe the generic terminal cancellation authority or the exact OpenHCS interruption hook.", - "The transition page lacked the total submission-deadline boundary and the distinction between known pre-dispatch and unknown post-dispatch timeout outcomes.", - "The page did not record that cold persistent catalogue preparation must run before execution-server import to prevent recursive preparer launch.", - "The page did not record that optional-backend admission must be proven once and remain stable across cache preparation and later catalogue reads.", - "The catalogue transition needed the nominal module owner, source-revision cache context, atomic failure, and complete derived-view clearing boundaries.", - "The transition reference did not record the generic endpoint owner for data/control port-pair projection and exact occupancy.", - "OpenHCS still selected IPC-versus-TCP cleanup behavior locally; pair cleanup and forced release now descend through the ZMQRuntime transport declaration.", - "The endpoint-version restart transaction lacked direct failure-path evidence for executor shutdown, saved-session capture, and detached worker launch.", - "Owner-requested connection-attempt cancellation was not a typed result, so GUI teardown could only surface it as a generic asynchronous failure; ZMQRuntime now owns the explicit cancellation outcome.", - "The execution reference omitted the shared MemoryType child environment and the public-only native catalogue projection.", - "The catalogue cache used one physical document for distinct framework-admission contexts, allowing concurrent CPU-only and GPU-capable preparation to replace each other's valid result.", - "Cancelling a cold catalogue request could leave its isolated capability-discovery child alive; the server-owned preparation now carries one cancellation authority to the exact process and joins it before backend cleanup." + "openhcs/agent/services/execution_session_service.py and its headless job store were deleted (U1); headless MCP and the desktop now submit through Session/ExecutionSubmission. The retained-client, job-status and per-job cancellation paragraphs described the deleted service.", + "The per-job headless cancellation capability no longer exists; Session stop shuts down the endpoint and cancels active datasets, while the client keeps a cancel_execution control request.", + "plate_path/execution_plate_path were renamed subject_id/execution_subject_id (G7); the selected-pipeline rejection and read-path checks belonged to the deleted service and were dropped.", + "The submission budget is declared by zmqruntime's ExecutionTransportConfig, which OpenHCSZMQConfig now only extends with application names.", + "Replaced 'receipt' wording with plain 'evidence'.", + "Desktop restart and version-mismatch restart modules and their tests moved from openhcs/pyqt_gui/services to openhcs/desktop (D1/U5); authorities were remapped, and the agent-services test that covered the deleted job store was dropped." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/zmq_execution_service_extracted.rst in full and compared it against the changed authorities: zmq_execution_server.py, zmq_execution_observation.py, zmq_execution_client.py, compiled_execution.py, function_catalog_preparation.py, the zmqruntime execution/messages/client/transport/timeout owners, tests/unit/test_zmq_submission_timeout.py and agent/services/execution_session_service.py (deleted, U1); verified every named class and the --prepare-capabilities mode exist, rewrote the headless job, cancellation and execution-subject paragraphs for Session/ExecutionSubmission and subject_id/execution_subject_id, moved the submission budget to ExecutionTransportConfig, and confirmed the compatibility check before submit and the benchmark execution-ID check.", "Re-read docs/source/architecture/zmq_execution_service_extracted.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: external/zmqruntime/src/zmqruntime/execution/client.py; openhcs/runtime/zmq_execution_server.py; openhcs/core/compiled_execution.py; external/zmqruntime/src/zmqruntime/messages.py; external/zmqruntime/src/zmqruntime/execution/lifecycle.py; external/zmqruntime/tests/test_execution.py; external/zmqruntime/src/zmqruntime/client.py; external/zmqruntime/src/zmqruntime/transport_modes.py; external/zmqruntime/tests/test_transport.py; openhcs/runtime/zmq_application.py; openhcs/runtime/zmq_execution_client.py; external/zmqruntime/src/zmqruntime/timeouts.py; openhcs/agent/services/execution_session_service.py; tests/unit/test_zmq_submission_timeout.py; openhcs/runtime/function_catalog_preparation.py; openhcs/runtime/zmq_execution_server_launcher.py; tests/unit/test_zmq_execution_server_process.py; openhcs/processing/backends/lib_registry/registry_service.py; openhcs/processing/backends/lib_registry/unified_registry.py; openhcs/processing/backends/lib_registry/openhcs_registry.py; tests/unit/test_library_registry_discovery.py; openhcs/pyqt_gui/services/desktop_restart.py; tests/unit/test_zmq_application.py; tests/unit/pyqt_gui/test_desktop_restart.py; openhcs/processing/custom_functions/runtime_registry.py.", "Focused feature-lattice QA passed 159 tests covering the N-Z architecture and developer workflow authorities.", @@ -2200,7 +2342,7 @@ }, { "path": "docs/source/architecture/zmq_server_browser_system.rst", - "source_sha256": "2ceb609f8b720a6862577d5fb1e18658d1040bc7e7507c03642a3b8471a9fa6e", + "source_sha256": "aaf89e33f9fae47a31e0ea5e55cc466d1b9f91159d90209d1eda9e5e5f8a01dc", "audience": [ "OpenHCS server-browser and progress UI maintainers" ], @@ -2214,7 +2356,7 @@ }, { "path": "openhcs/pyqt_gui/widgets/shared/server_browser/progress_projection.py", - "sha256": "4b9c4bd90fbe6ed4ac55f5fe1d23cf2a46c3a7f743218d685df8ac2f27be7998", + "sha256": "d49ef50d9bebca37c25abd5d603ddce77680fe21757c2bbd3b02533042c11bb9", "role": "Browser-facing typed execution progress projection." }, { @@ -2229,17 +2371,17 @@ }, { "path": "openhcs/pyqt_gui/widgets/shared/server_browser/live_tree_sync.py", - "sha256": "81dbee5b7f471082bcfeb8ca63e753840ad7134aff7101fd14f1c821a121d1bb", + "sha256": "dfb301b3e3e465e40205c4f3cf29eb7b47333751f70cc6d26a3838c5eea848c0", "role": "Renders responsive, startup, and launching-viewer rows from current lifecycle snapshots without retaining parallel endpoint state." }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "a156a006a3c40f43107b9b1f6ecb97ed150879eb88ded8ef89e25214309ae33a", + "sha256": "e43085c1eaf0c836ce34e1a1dae7740488d91bdfea56844e7ddd4e6925561cf6", "role": "Projects the configured endpoint snapshot into its dedicated right-hand indicator, and routes endpoint termination to the client lifecycle owner." }, { "path": "tests/unit/pyqt_gui/test_main_zmq_window_routing.py", - "sha256": "d02fe132fda83540f26ed4b7a79fc1409fabbf0ff6ce72f3d1ed8681258e9f6a", + "sha256": "8568dcd39ed44cc1e065c0d7389b9f92dacc80e984f13ca2944d102baf679027", "role": "Proves startup callbacks do not write presentation directly, endpoint snapshots leave workflow messages untouched, and endpoint termination descends to the client lifecycle owner." }, { @@ -2253,23 +2395,28 @@ "role": "Owns typed multi-endpoint shutdown outcomes and generic progress-tracker retirement." }, { - "path": "openhcs/pyqt_gui/widgets/shared/services/zmq_client_service.py", - "sha256": "8bb3b33ee64869a225e7ec23be3dcf4f5bf3622dd801f7e77bdd7826980cbd8e", - "role": "Owns the persistent OpenHCS execution-client lifecycle and retires its exact session when that endpoint terminates." + "path": "openhcs/authoring/session/execution_client.py", + "sha256": "f56f23fc860dab554584cb5fe44d29332c73f7bb7a650c167702efeb986363b7", + "role": "ZMQClientService: the session-owned execution client, its EndpointClientSession, and retirement when its exact endpoint terminates." }, { "path": "external/zmqruntime/src/zmqruntime/client.py", - "sha256": "fc8bdf68455673921d79742204bf6c88e42b52bbe407d10ef4a831e084ca86a8", + "sha256": "7d3cce31d5abaa070927f02fe87d919687619e2cf4adbc37ab82483b6b8fc212", "role": "Declares EndpointClientSession as the authoritative transport-client and application-admission proof." + }, + { + "path": "openhcs/core/progress/runtime_tree.py", + "sha256": "016279f4135e2c9b10cb841e57ba0297c4197a195623f8a746ed90ab06533ce5", + "role": "RuntimeExecutionTopology, RuntimeTreeProjectionBuilder and the subject/worker/partition/step/compilation node kinds with mean or explicit percent policies." } ], "findings": [ - "The component inventory named a deleted ServerTreePopulation class and described responsibilities now split between LiveServerTreeSync and ServerRowPresenter.", - "The reference treated the browser only as an execution-progress consumer and omitted its generic endpoint-snapshot authority and the OpenHCS status projections derived from it.", - "The launching-viewer projection retained stale row payloads and rows after the authoritative viewer lifecycle no longer contained the endpoint." + "openhcs/pyqt_gui/widgets/shared/services/zmq_client_service.py was moved into the session as openhcs/authoring/session/execution_client.py (U1); the progress subscriber is the Session's ZMQClientService, not the Plate Manager's.", + "G7 renamed the progress tree nodes: the execution path is subject -> worker -> partition -> step, not plate -> worker -> well -> step; aggregation policies were updated to match." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/zmq_server_browser_system.rst in full and compared it against the changed authorities: server_browser/progress_projection.py, server_browser/live_tree_sync.py, pyqt_gui/main.py, tests/unit/pyqt_gui/test_main_zmq_window_routing.py, zmqruntime client.py and services/zmq_client_service.py (moved, U1); verified every named class and module path exists, corrected the progress subscriber owner and the subject/worker/partition/step tree path and aggregation kinds from core/progress/runtime_tree.py, and confirmed the main window still projects the endpoint snapshot into its indicator and manager button states.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", "The complete pyqt-reactive suite passed 320 tests, including shared non-overlapping browser and log-viewer scan lifecycle coverage.", "tests/unit/pyqt_gui/test_progress_tree_aggregation.py passed; tests/unit/test_configuration_mirror_deletion.py proves the obsolete server_tree_population.py remains absent", @@ -2297,45 +2444,44 @@ "authority": [ { "path": "openhcs/mcp/client_registration.py", - "sha256": "28459b95d3b155778ce178f36302cb830958b74f2603980641ec492d81c41a9a", - "role": "Nominal MCP client registration target registry." + "sha256": "55b701bb77a3de19243fa9a6276f784fad8329e2d435fb26fcc7fdce1e060731", + "role": "Declares McpClientRegistrationTarget.__registry__ keyed by target_id." }, { "path": "openhcs/agent/services/synthetic_plate_service.py", - "sha256": "2860714a9f1f2e66df403bc72331195d912cb180600889a1b4f4207642cc776b", - "role": "Production synthetic-plate request service delegating deterministic fixture generation to the installed demo package." + "sha256": "eb9e9ef693a877d65280004f8f85f1b6d6ab9be1c51aa3d2a0e1ed96630faa2d", + "role": "Production synthetic-plate service delegating generation to openhcs.demo.synthetic_data." }, { - "path": "website/assets/agent/cold-start-workflow-record.json", - "sha256": "52d126027b39a05cd5e799e4aca28e45fd8edd251c7b523204c33c14fa7de65c", - "role": "Retained baseline schema, client and model identity, operation mode, trace hashes, evidence, and acceptance-gate results." + "path": "openhcs/demo/synthetic_data.py", + "sha256": "be80973ecd87b4b3b36b7a7283d962a5d7cafd491cfa16eafe4622e09391463a", + "role": "Installed synthetic-plate generator; writes a time-derived ImageXpress UniquePlateIdentifier." }, { - "path": "tests/unit/test_installed_mcp_demo.py", - "sha256": "f5d4a719a7902b364b551a3cbc4a816ccd7f46e5d9abbc87bf2d53cb0180053b", - "role": "Proves the installed MCP workflow, progress-aware inactivity watchdog, hard ceiling, and terminal completion." + "path": "openhcs/mcp/installed_demo.py", + "sha256": "5debdb5ba24fbe29f48f065f9b3f473022c17209724e618f2a0b36190fb82913", + "role": "Declares the deterministic fixture request (1x1 grid, 192 px tiles, 10% overlap, 1 px stage error, 2 wavelengths, 1 Z, 12 cells, 0.95 shared, A01, ImageXpress, seed 1)." }, { - "path": "openhcs/demo/synthetic_data.py", - "sha256": "7a8f4a79e716ad944d5ceb1ed14e62cab9211aee65b1c0dc571e097e0f4d3c09", - "role": "Installed deterministic synthetic-plate implementation used by the production service and MCP demo." + "path": "website/assets/agent/cold-start-workflow-record.json", + "sha256": "52d126027b39a05cd5e799e4aca28e45fd8edd251c7b523204c33c14fa7de65c", + "role": "Retained baseline record with the schema groups, nullable registration_target_id and operation mode." }, { - "path": "openhcs/mcp/installed_demo.py", - "sha256": "4ea6c0730db6d3938eea1cac558dd46ea5c956ef151a5d68b8ea956e8bf57655", - "role": "Implements the installed workflow rehearsal and derives activity from exact execution progress observations." + "path": "tests/unit/test_installed_mcp_demo.py", + "sha256": "8babc443f7ec59cebebd3772633dde9ae9c49b125066cd5e7c7d527dc74c2a09", + "role": "Proves the installed demo runs the session journey and validates settled viewer state." }, { "path": "tests/unit/test_synthetic_generator_randomness.py", "sha256": "6ab3efd7f4e083744dc4dda26d97c60b6a7649dd44432362020027b92271717f", - "role": "Proves seeded synthetic image equivalence and isolated randomness across interleaved and concurrent generation." + "role": "Proves seeded synthetic image equivalence and isolated randomness." } ], - "findings": [ - "The production synthetic service now delegates to the installed openhcs.demo package rather than an importable tests package." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/agent_workflow_validation.rst in full and compared it against the changed authorities: openhcs/mcp/client_registration.py, synthetic_plate_service.py, tests/unit/test_installed_mcp_demo.py, openhcs/demo/synthetic_data.py, openhcs/mcp/installed_demo.py; the client skill sync and session-based execute path do not touch the protocol, and the fixture request values, the registry the page names, the time-derived HTD identifier, the ZMQ Server Manager and Live Results windows and the retained baseline record all still match; updated the test's role because its old progress-polling watchdog tests were replaced by the session journey.", "Re-read docs/source/development/agent_workflow_validation.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/mcp/client_registration.py; openhcs/agent/services/synthetic_plate_service.py; website/assets/agent/cold-start-workflow-record.json; tests/unit/test_installed_mcp_demo.py; openhcs/demo/synthetic_data.py.", "Focused feature-lattice QA passed 159 tests covering the N-Z architecture and developer workflow authorities.", @@ -2350,7 +2496,7 @@ }, { "path": "docs/source/development/ast_refactoring_workflow.rst", - "source_sha256": "79299114e82cb13d549d24dce90abc381cd249e2c2d4e10c116d2086b73e4686", + "source_sha256": "723e0d6cc10e795822010d36c73319567e263ecb31a8ba5264368996a15fdf25", "audience": [ "OpenHCS documentation and source maintainers" ], @@ -2359,20 +2505,17 @@ "authority": [ { "path": "scripts/validate_docs.py", - "sha256": "f048e8c882ad701ae7f238cda02526292e82c0ee317e2f817e75d75db3b9e882", - "role": "Executable documentation and audit validation contract." - }, - { - "path": "openhcs/core/pipeline/compiler.py", - "sha256": "ace8ae0c3e54bbe604b97cec7c3973a1abcf90e1dc517c2adda947dc2c35e882", - "role": "Compilation order, declaration resolution, callable warm-up, framework-local device assignment, and bundle construction." + "sha256": "d48be3462998b8277bff6f7e9eea8785e01d937a6d1a300127772c680a66f094", + "role": "Parses RST and Markdown Python blocks with ast and checks imports, literalincludes and repository paths, the validation step 6 names." } ], "findings": [ - "The integrated setup procedure was duplicated in docs/development_setup.md; the Markdown entry now points to this canonical how-to and unique optional-extra and packaging guidance was consolidated here." + "Step 6 began with a lowercase \"run\"; capitalised to match the other steps.", + "openhcs/core/pipeline/compiler.py supports no claim on this page (it only refers generically to the compiler/declaration authority) and was dropped from the authorities." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/ast_refactoring_workflow.rst in full and compared it against the changed authorities: scripts/validate_docs.py (repository-path pattern now also matches .jsonl and rejects trailing identifier characters) and openhcs/core/pipeline/compiler.py; neither change affects the workflow, the validate_docs.py step still holds, and the compiler authority was dropped as irrelevant.", "Re-read docs/source/development/ast_refactoring_workflow.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: scripts/validate_docs.py; openhcs/core/pipeline/compiler.py.", "Focused feature-lattice QA passed 159 tests covering the N-Z architecture and developer workflow authorities.", @@ -2383,7 +2526,7 @@ }, { "path": "docs/source/development/callable_artifact_authoring.rst", - "source_sha256": "60006087b96c5e1c84063923e457fabbd54a040db4bad4e9e96c2fc6bfeb62b3", + "source_sha256": "06b84f60a5bb09683e64772fd564ad3478286fe6104747fa6c2410ddcb0ff3a8", "audience": [ "OpenHCS callable and artifact extension authors" ], @@ -2392,47 +2535,69 @@ "authority": [ { "path": "openhcs/core/callable_contract.py", - "sha256": "6a39b5e171901131ecb7280a78241e8ea72409a5003eabba3db34eb4851991d7", - "role": "Callable ABI including input/output/execution memory roles, runtime-owned parameter projection, and artifact contracts." + "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", + "role": "Declares CallableContract fields (required_axis_roles, allowed_group_by_roles, memory roles), resolve_returned_output, the return partition and the artifact-parameter binding errors quoted by the page." }, { "path": "openhcs/core/artifacts.py", - "sha256": "bb958c74c98cb96c97c5ac1ea68ac997673aac6afcad57bdf0c9496065e895fa", - "role": "Artifact type, specification, reference, relation, and plan declarations." + "sha256": "65eb0ec5c903366243ca8bd5ea07412e8d34bdb81e6015fa76b3c4c59838733f", + "role": "Declares ArtifactSpec, MainFlowStackOutputSpec, the artifact types, ObjectMeasurementSubjectRelation and MaterializationSourceIdentityRelation." }, { - "path": "openhcs/interop/cellprofiler/module_artifact_contracts.py", - "sha256": "bf2e8f6c66a181218c8fd9afa9132922e86295f65d6102d976e1b36e41e17ee5", - "role": "Invocation-specific CellProfiler artifact contract derivation." + "path": "openhcs/core/processing_contracts.py", + "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", + "role": "Declares Pure2DContract and Pure3DContract used in the examples." }, { - "path": "tests/unit/test_artifact_spec_collection.py", - "sha256": "09d66f7b67e2b281c911ba935d6a1b3ff2c790da3977184256302d24b7a6d548", - "role": "Regression proof for role-neutral artifact specs, exact refs, relations, and collection validation." + "path": "openhcs/core/pipeline/function_contracts.py", + "sha256": "13bcaf7c7a62654f512d2cc739edccd2deeff35d99970ca493f3e827afbda91a", + "role": "Declares artifact_inputs, artifact_outputs, special_inputs, execution_scope and runtime_bound_parameters." }, { - "path": "external/arraybridge/src/arraybridge/types.py", - "sha256": "7b94760d57e54008d7adf4e4d37dca45d8a2c288a9dc8400be00c26148cf7624", - "role": "Memory role taxonomy and framework execution-device scopes." + "path": "openhcs/core/runtime_stores.py", + "sha256": "b6a7231bc2964d472070731018f6a4935e5f23cf8f238a232045c32244e6545a", + "role": "Declares RuntimeArtifactBatch.records and StoredRuntimeValue for plate-scope callables." + }, + { + "path": "openhcs/core/orchestrator/bundle_execution.py", + "sha256": "f5ba09a2ec7c688b1ca08167ab398cc976a9e2c39b301ae98f3c5fd560a25c85", + "role": "Requires a plate-scope callable to declare exactly one SpecialArtifactType output." + }, + { + "path": "openhcs/processing/materialization/core.py", + "sha256": "744573f1bafa67b7a81be5af425012ada47741292519863e250b79f1c778b265", + "role": "Point ROI writer: rejects missing fields, duplicate labels, non-finite coordinates and an empty archive." + }, + { + "path": "openhcs/interop/cellprofiler/module_artifact_contracts.py", + "sha256": "982680cd36409ea6d90a07d4e60893b162847f5525ed516891e995877dc32fd1", + "role": "Declares CellProfilerModuleArtifactContracts resolving setting-backed specs into the invocation CallableContract." }, { "path": "external/arraybridge/src/arraybridge/decorators.py", - "sha256": "99823105b0ae59054df1d84013633b8b6628d0a6dc3326383199ac7c65c69bed", - "role": "Decorators that declare input, output, and execution framework roles." + "sha256": "28f9bf4770bd4cc1ef34045fdef28065063ede029296f62b8a6c4140cc88e40e", + "role": "Declares the input, output and execution framework roles written by the memory decorators." + }, + { + "path": "tests/unit/test_artifact_spec_collection.py", + "sha256": "60ba79abffde80987809e2ef0621983e798780ad21cd4e98db6b29f21372c697", + "role": "Proves role-neutral artifact specs, exact refs, relations and collection validation." }, { "path": "tests/unit/test_callable_contract.py", - "sha256": "493f76a54f5fe024ca8f10c53b1f810b8c369626ad7141205b42ef3a78282137", + "sha256": "cc220f53107a08b0406b6e0dd49896e3e4c3c0af6e527b0ef00d4fdaba8f0b2c", "role": "Proves exact enum validation and contract-derived runtime-owned parameter exclusions." } ], "findings": [ - "The callable ABI guide omitted the execution-memory role and its distinction from input/output conversion boundaries.", - "The callable-authoring how-to did not state that an enum-annotated public kwarg requires a member of the exact declared enum.", - "The how-to did not state that contract-owned runtime parameters project into the shared authoring exclusion boundary." + "The contract summary said contracts carry required variable components and allowed grouping; after G1 they carry required_axis_roles and allowed_group_by_roles.", + "Prose still named the deleted enum members PURE_2D and PURE_3D although the code examples already use Pure2DContract; after G5 the prose names the contract classes.", + "The rewritten special_inputs sentence was an overlong unwrapped line; rewrapped.", + "Replaced the jargon \"registration receipt\" with \"registration result\"." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/callable_artifact_authoring.rst in full and compared it against the changed authorities: the page, openhcs/core/callable_contract.py, openhcs/core/artifacts.py, module_artifact_contracts.py, tests/unit/test_artifact_spec_collection.py, the ArrayBridge types and decorators, tests/unit/test_callable_contract.py; grepped every API the page names (SettingToKeywordBinding, SelectedPlaneImageOutput, canonical/trailing return specs, RuntimeArtifactBatch.records, PointROIOptions features, the custom-function capability and guide names) and the quoted error texts, confirmed the plate executor's single SpecialArtifactType rule and the empty-Points rejection, corrected the axis-role and processing-contract names, and added processing_contracts.py, function_contracts.py, runtime_stores.py, bundle_execution.py and materialization/core.py as owners while dropping arraybridge/types.py.", "Re-read docs/source/development/callable_artifact_authoring.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/callable_contract.py; openhcs/core/artifacts.py; openhcs/interop/cellprofiler/module_artifact_contracts.py; tests/unit/test_artifact_spec_collection.py; external/arraybridge/src/arraybridge/types.py; external/arraybridge/src/arraybridge/decorators.py; tests/unit/test_callable_contract.py.", "Focused feature-lattice QA passed 159 tests covering the N-Z architecture and developer workflow authorities.", @@ -2458,22 +2623,22 @@ "authority": [ { "path": "openhcs/interop/cellprofiler/module_declarations.py", - "sha256": "89dd65abc003c464ba6c489f52d3a4fa47be1d5f8b87d26ad72b7537f188cf8b", + "sha256": "6bbf41c158496a02439be2ee5aef12d7712fb632419c226a64c9259a55438b95", "role": "CellProfiler module registry, catalogue placement, and headless-import eligibility." }, { "path": "openhcs/interop/cellprofiler/module_artifact_contracts.py", - "sha256": "bf2e8f6c66a181218c8fd9afa9132922e86295f65d6102d976e1b36e41e17ee5", + "sha256": "982680cd36409ea6d90a07d4e60893b162847f5525ed516891e995877dc32fd1", "role": "Invocation-specific CellProfiler artifact contract derivation." }, { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "b547c838b2c86fe7cc5fe12d6cbf6a6715e83055c2d8c1a109ac22566d22c51e", + "sha256": "3439d96f3fbfcb104a53019dc150049021d9a8132bd27096f1ccf510eefbe37b", "role": "Public CellProfiler pipeline lowering and declaration-owned headless-import validation." }, { "path": "tests/unit/test_cellprofiler_pipeline_import.py", - "sha256": "353f0c1d8e0bc3552d2e418d792794ab0631daa16c78a3e10191b1cc1ea07d73", + "sha256": "6b1d201ec5c4ba620fa2367a9cbef1f6fbdd25edbdda9bd7b723ebe3a3363412", "role": "Proves interactive CellProfiler modules fail headless import instead of disappearing from the lowered pipeline." }, { @@ -2483,7 +2648,7 @@ }, { "path": "openhcs/processing/backends/lib_registry/openhcs_registry.py", - "sha256": "aa3079341cb2af8788938e14efee6f9f6c4b6c01874227e74c6340850d525884", + "sha256": "3fff5c3549ef8e1073d75e6d5544b2cd75fbb7cc9b17263207760ffca45cf885", "role": "Derives the public native callable catalogue while retaining explicit reconstruction from each callable declaration." }, { @@ -2492,14 +2657,10 @@ "role": "Lazy package projection derived from CellProfilerModule declared function names." } ], - "findings": [ - "CellProfilerModule now also owns callable catalogue placement; the backend package lazily projects its declared names.", - "The how-to did not explicitly require active output identity keywords to survive public source transport instead of observation-driven pruning.", - "The catalogue-authoring workflow did not state that only the public declared union is browsable.", - "The module-authoring workflow omitted declaration-owned headless-import validation." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/cellprofiler_module_authoring.rst in full and compared it against the changed authorities: module_declarations.py (axis-family imports, MeasurementDialect, lazy registry preparation, private _require_declared_callable with object-identity check, prepare_declared_callable), module_artifact_contracts.py (target_units_for_invocation replaces target_contract_for_invocation), pipeline_import.py, tests/unit/test_cellprofiler_pipeline_import.py and openhcs_registry.py (LazyDiscoveryDict declaration discovery, declared_callable_contract). Confirmed every name the page relies on still exists at its owner: AutoRegisterMeta, CellProfilerModule.__registry__, for_callable_contract (import identity then object identity), declared_function_names and the lazy CellProfilerBackendModule projection, validate_pipeline_import called by pipeline_import before lowering, CellProfilerModuleArtifactContracts, SettingToKeywordBinding.input/output, module_blocks_for_invocation, SourceBindingsConfig, ModuleBlock and CallableContract; no page change was needed.", "Re-read docs/source/development/cellprofiler_module_authoring.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/interop/cellprofiler/module_declarations.py; openhcs/interop/cellprofiler/module_artifact_contracts.py; openhcs/interop/cellprofiler/pipeline_import.py; tests/unit/test_cellprofiler_pipeline_import.py; tests/unit/test_cellprofiler_public_boundary_contracts.py; openhcs/processing/backends/lib_registry/openhcs_registry.py; openhcs/processing/backends/cellprofiler/__init__.py.", "Focused feature-lattice QA passed 159 tests covering the N-Z architecture and developer workflow authorities.", @@ -2513,7 +2674,7 @@ }, { "path": "docs/source/development/compiler_extension.rst", - "source_sha256": "4bb46f10ebfdab22de8f3f5b833e7f74b78d4d846ee50c671296e4aad33953ff", + "source_sha256": "f6374ec2abb6d45312729e9ac0f26e8e7ef562119c99bdd779091059d3a56a8b", "audience": [ "OpenHCS compiler extension authors" ], @@ -2522,37 +2683,37 @@ "authority": [ { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "3c840a89baccc8665d415175458280fd904c1a837b7fd879e303d7dbca129a1c", - "role": "Resolved step/session identity and typed plan assembly invariants." + "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", + "role": "ResolvedPipelineDefinition and CompilationSession fields, one-time artifact admission before axis fan-out, and the missing-plan failure of plan(index)." }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "ace8ae0c3e54bbe604b97cec7c3973a1abcf90e1dc517c2adda947dc2c35e882", + "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", "role": "Compilation order, declaration resolution, callable warm-up, framework-local device assignment, and bundle construction." }, { "path": "openhcs/core/pipeline/path_planner.py", - "sha256": "0ed8bb8d843ff41382f4de25367a74c9535a39425c6fc328091ad3025d526151", + "sha256": "677d106969aae4d818745bff29b6f84e487ec79a044c8eaf26bea1916236fc6c", "role": "Compiled source satisfaction, artifact edge, path, grouping, and plan decisions." }, { "path": "tests/unit/test_compilation_session.py", - "sha256": "4224c87a6b67e4d8dde2a5c6b786ea7998474819de56da2b9d954aa2b479feb4", - "role": "Regression proof for one-to-one step, state, snapshot, context, and plan identity." + "sha256": "381adf3a87729928a225fd1db5a35f8c802340fb2a1e583501884dc5500f6b9d", + "role": "Regression proof that axis sessions share one resolved pipeline definition while owning their own plans." }, { "path": "external/arraybridge/src/arraybridge/types.py", - "sha256": "7b94760d57e54008d7adf4e4d37dca45d8a2c288a9dc8400be00c26148cf7624", + "sha256": "08f7ff4df8807bb3340e00a02c6276b1c702a0f00a2eedacd701142736c5d804", "role": "Framework declarations, device discovery, and device-scope ownership." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "6d46571e6f2d069b98a92b41d711b2e20bfab9f0493cf73d3f150d5be08e7f96", + "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", "role": "Typed per-step framework-device binding authority." }, { "path": "openhcs/core/pipeline/framework_device_assignment.py", - "sha256": "f77f3fb2d30c4af8cab6210dbe294f9fb5e246dad0a6bbf88bcaf1e2b58dead7", + "sha256": "cdfde10f996ab682952a1d4948d3de2d23d48353eae41f3124770e19fc28a193", "role": "Compiler projection from declared framework roles to per-step bindings." }, { @@ -2567,10 +2728,11 @@ } ], "findings": [ - "Framework extensions needed an explicit declaration-to-CompiledStepPlan path after deletion of the global GPU scheduler and validator." + "The Invariants section described StepSnapshot and a CompilationSession that validates steps, snapshots, states and plate scope; StepSnapshot was deleted. ResolvedPipelineDefinition now carries the enabled steps with ObjectState scope ids and provenance and admits artifact contracts once before axis fan-out, and CompilationSession binds it to one ProcessingContext, its plan map, source projection and dataset scope (CompilationDatasetScope). Rewrote the section to match." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/compiler_extension.rst in full and compared it against the changed authorities: compilation_session.py (StepSnapshot and snapshot validation deleted; ResolvedPipelineDefinition and CompilationDatasetScope added), compiler.py, path_planner.py, tests/unit/test_compilation_session.py, arraybridge types.py, compiled_step_plan.py and framework_device_assignment.py. Rewrote the Invariants section for the current session model; confirmed CompiledStepPlan, ArtifactSpecRef, assign_framework_devices, FrameworkDeviceResolver and the absence of a global GPU scheduler.", "Re-read docs/source/development/compiler_extension.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/pipeline/compilation_session.py; openhcs/core/pipeline/compiler.py; openhcs/core/pipeline/path_planner.py; tests/unit/test_compilation_session.py; external/arraybridge/src/arraybridge/types.py; openhcs/core/compiled_step_plan.py; openhcs/core/pipeline/framework_device_assignment.py; openhcs/core/framework_device_resolver.py; tests/unit/test_framework_device_resolver.py.", "Focused feature-lattice QA passed 159 tests covering the N-Z architecture and developer workflow authorities.", @@ -2592,27 +2754,22 @@ "authority": [ { "path": "openhcs/agent/services/object_state_field_help_service.py", - "sha256": "d1f130b63e7b36aaee23b6333048de777b21fb40b7ecd85dcc7687c79dc74ce6", + "sha256": "5aa83508fa68cb69160d34ca64b495f5a34a401f358ce69ce290e24c96371a4c", "role": "Declaration-reflected ObjectState field help and configuration schema projection." }, { "path": "openhcs/agent/services/knowledge_base_service.py", - "sha256": "88e10e18c66eb3a8bc2ec5dd33493a9473510c0b0a9d355502d668042f32fdd9", + "sha256": "dfc26135b16f3827ced85b122cc1555c1325da1c5e805b421afcee9cdd7d91f3", "role": "Source-backed knowledge catalogue, search, section reads, and lazy Official30 projection." }, { "path": "scripts/validate_docs.py", - "sha256": "f048e8c882ad701ae7f238cda02526292e82c0ee317e2f817e75d75db3b9e882", - "role": "Executable documentation and audit validation contract." - }, - { - "path": "docs/audits/README.md", - "sha256": "aeb08ec4e4a0e8cc7dd6e0be9d0de9c1dc159793f5643e91aa6c543cac145c4b", - "role": "Audit-ledger schema and evidence rules only; it is not used as product-fact authority." + "sha256": "d48be3462998b8277bff6f7e9eea8785e01d937a6d1a300127772c680a66f094", + "role": "Executable documentation and audit validation contract, including the project.readme entry and digest gate." }, { "path": "scripts/gallery_catalog.py", - "sha256": "7f493e0cb305e14ee4cf7d3e995c67df4200186a8a8aaf00eef0bc66d8d191cf", + "sha256": "7fd3fce2b8fdae1abf049bb4f8a3a310ab598a7f2c507578425be537903ccfca", "role": "Owns nominal gallery scenario identity, publication targets, representative images, copy, accessibility text, dimensions, contextual and stable-window capture targets, and derivative contracts." }, { @@ -2622,16 +2779,16 @@ }, { "path": "tests/unit/test_build_website.py", - "sha256": "4a8952ba0f51590ffd9f1d787986653802506807079a7e0d118343264252e0e0", + "sha256": "e88fc8ae73c5610dfb0a9948271a016e666451228ecade92aa1b2892da106059", "role": "Proves every scenario owns a valid representative static projection, stable-window figures derive from registered UI identities, documentation-only scenarios do not expand the website cards, and Sphinx branding resolves from package authority." } ], "findings": [ - "The guide did not state that material RST changes must refresh the page-level audit record and source digest.", - "The root README becomes the package-index long description but was outside both the documented review workflow and the executable audit gate." + "The record listed docs/audits/README.md as an authority; it is documentation, and scripts/validate_docs.py already owns the enforced ledger and project.readme rules the page states, so it was dropped. Page text needed no change." ], - "disposition": "revise", + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/documentation_authoring.rst in full and compared it against the changed authorities: object_state_field_help_service.py (UI-bridge scope listing through UiBridgeService.invoke; JsonValue from python_introspect), knowledge_base_service.py (comparison-manifest parsing moved to openhcs.agent.knowledge_manifest_schema), scripts/validate_docs.py (path regex now accepts .jsonl), scripts/gallery_catalog.py (SessionOperation actions) and tests/unit/test_build_website.py. Confirmed openhcs_describe_config_schema, the openhcs-gallery directive, the napari-roi-navigation scenario, the documentation publication target, the project.readme gate, reference/configuration.rst and all four listed test files exist; no page claim changed. Dropped the docs/audits/README.md authority as a documentation file.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", "Config-help and knowledge-service tests provide the listed projection gates; the audit validator enforces exact active-source coverage and SHA-256 freshness.", "On 2026-08-24, re-read the complete guide after deriving the package-index README from project.readme and requiring it in the audit; 10 focused audit regressions passed.", @@ -2645,7 +2802,7 @@ }, { "path": "docs/source/development/extension_workflows.rst", - "source_sha256": "02e13f4611803605d430da15b4a1fa875a16a708b4b167fa0efc19f03df672eb", + "source_sha256": "8fc4f9ad9a36c88d51a8e41285e7e8ccb5a6af7ec0c349c41bf1295f0f0cbcac", "audience": [ "OpenHCS extension authors" ], @@ -2653,13 +2810,13 @@ "diataxis": "how-to", "authority": [ { - "path": "openhcs/core/registry_strategies.py", - "sha256": "a41186665abbd308f224d0bb41d6618cd9d89b08acaf54ec2819f2024a6e0804", - "role": "OpenHCS enum, nominal-type, and most-derived strategy selection implementations." + "path": "external/metaclass-registry/src/metaclass_registry/strategies.py", + "sha256": "5c00cd4d80f9f0ceffbc2ab95ad8c6abd44a8a66a974959445ef93df08dd4f0d", + "role": "Enum-keyed, nominal-type, and most-derived-context strategy family mixins used when behaviour varies by key or context." }, { "path": "pyproject.toml", - "sha256": "6217b5f029796edcd83aa47fb8632d37aa7552344755c578fa4f181a6b23282d", + "sha256": "3592cc37444e2e1f5bb0284e939643039c0d513217861af8133dd8bd6e8a4325", "role": "Current Python support, dependency, package discovery, and entry-point authority." }, { @@ -2668,59 +2825,59 @@ "role": "Orders supplied parameters by signature with optional default omission and caller-owned equality." }, { - "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "3c840a89baccc8665d415175458280fd904c1a837b7fd879e303d7dbca129a1c", - "role": "Resolved step/session identity and typed plan assembly invariants." + "path": "external/python-introspect/src/python_introspect/jsonable.py", + "sha256": "8fe06533a6dce179b7d77792b6ac768c2a64a1e9e79bbe926893498fae035bb4", + "role": "singledispatch to_jsonable projection for dataclasses, mappings, sequences, callables, registered types, enums, paths and scalars, paired with dataclass_from_mapping." }, { - "path": "openhcs/serialization/json.py", - "sha256": "fa1e2aeae1e5d6896944d1516c17db0ac486157b335d7be06fcb553ab3203f1f", - "role": "Bounded JSON singledispatch projection for callables, registered types, dataclasses, paths, enums, and collections." + "path": "openhcs/core/pipeline/compilation_session.py", + "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", + "role": "ResolvedPipelineDefinition and CompilationSession consumed by compiler phases." }, { "path": "external/arraybridge/src/arraybridge/types.py", - "sha256": "7b94760d57e54008d7adf4e4d37dca45d8a2c288a9dc8400be00c26148cf7624", + "sha256": "08f7ff4df8807bb3340e00a02c6276b1c702a0f00a2eedacd701142736c5d804", "role": "Declares framework capabilities, callable memory metadata, and the child-process import environment owned by each MemoryType member." }, { "path": "openhcs/core/callable_contract.py", - "sha256": "6a39b5e171901131ecb7280a78241e8ea72409a5003eabba3db34eb4851991d7", + "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", "role": "OpenHCS callable declaration projection for input, output, and execution frameworks." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "6d46571e6f2d069b98a92b41d711b2e20bfab9f0493cf73d3f150d5be08e7f96", + "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", "role": "Typed per-step framework-device binding authority." }, { "path": "openhcs/core/pipeline/framework_device_assignment.py", - "sha256": "f77f3fb2d30c4af8cab6210dbe294f9fb5e246dad0a6bbf88bcaf1e2b58dead7", + "sha256": "cdfde10f996ab682952a1d4948d3de2d23d48353eae41f3124770e19fc28a193", "role": "Compiler projection from declared framework roles to per-step bindings." }, { "path": "benchmark/cellprofiler_benchmark_cli.py", - "sha256": "2b92755c556c1d66d28ffa0af6e2ac61cac5af4631514382290a2af0f907c26b", + "sha256": "4025c591644e29f3df61e476ecae2143690e014336ea9f889aadeff4e692445e", "role": "Owns registered benchmark command declarations and the derived installed parser." }, { "path": "benchmark/contracts/run_receipt.py", - "sha256": "1704e1ab47411d25d382b95871c96c1e60c94db14b8ffd55f4dc6c8a917758b9", + "sha256": "4cd61bd33f8f6199b3cc4130ed091dddde9c3a23975e132b1bee06c3c6a43cd7", "role": "Owns the common comparison-run declaration and typed persisted lifecycle receipt." }, { "path": "tests/unit/test_benchmark_control.py", - "sha256": "ee75209799216c90cfa3a585dd9b7af99c99405f93e9fb5e4f900ea26d9d3ea0", + "sha256": "093b584eeb37cbb12089d5af082ad45acd779e9d2bdd5b3feaea08234fc22183", "role": "Proves registered CLI derivation and declaration-derived receipt construction." } ], "findings": [ - "The owner map lacked ArrayBridge framework/device discovery and OpenHCS per-step binding compilation.", - "The extension map omitted python-introspect's ownership of generic callable-declaration projection.", - "The extension map did not assign child framework import environments to ArrayBridge MemoryType declarations.", - "The owner map omitted the registered benchmark command family and the shared comparison-run declaration/receipt boundary." + "The Compiler/runtime split said compiler phases consume steps through StepSnapshot, which was deleted; they now consume ResolvedPipelineDefinition and CompilationSession.", + "The Serialization boundaries section still attributed to_jsonable to OpenHCS after openhcs/serialization/json.py was deleted in favour of python-introspect's to_jsonable/dataclass_from_mapping pair (L2); the owner table had already been updated but the prose contradicted it.", + "The record cited the deleted openhcs/core/registry_strategies.py and openhcs/serialization/json.py; their owners are now metaclass_registry/strategies.py and python_introspect/jsonable.py." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/extension_workflows.rst in full and compared it against the changed authorities: deleted openhcs/core/registry_strategies.py (strategy mixins moved to metaclass-registry, b221f8491), deleted openhcs/serialization/json.py (python-introspect codec, c0a53a1e0), compilation_session.py (StepSnapshot removed), arraybridge types.py, callable_contract.py, compiled_step_plan.py, framework_device_assignment.py, the benchmark CLI and run-receipt modules, and pyproject.toml. Corrected the StepSnapshot reference and the to_jsonable owner; confirmed ImageFileFormat, BenchmarkCliCommand, ComparisonSuiteRunDeclaration, ComparisonSuiteRunReceipt, MeasuredPipelineRunReceipt, ArtifactKey, CompiledExecutionBundle and MemoryType.import_environment exist at their owners.", "Re-read docs/source/development/extension_workflows.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/registry_strategies.py; pyproject.toml; openhcs/core/pipeline/compilation_session.py; openhcs/serialization/json.py; external/arraybridge/src/arraybridge/types.py; openhcs/core/callable_contract.py; openhcs/core/compiled_step_plan.py; openhcs/core/pipeline/framework_device_assignment.py.", "Focused feature-lattice QA passed 159 tests covering the N-Z architecture and developer workflow authorities.", @@ -2749,7 +2906,7 @@ }, { "path": "docs/source/development/git_worktree_testing.rst", - "source_sha256": "5a2487f59d2befe94a05219b3a382f2c60e2c773054cb90f8ac2bfdde0ab9e63", + "source_sha256": "53191366b967995617ea27cc7bb1a44dcd095f57826effd69ff6d3513b5a7e79", "audience": [ "OpenHCS contributors comparing or testing branches" ], @@ -2763,20 +2920,26 @@ }, { "path": "pyproject.toml", - "sha256": "6217b5f029796edcd83aa47fb8632d37aa7552344755c578fa4f181a6b23282d", + "sha256": "3592cc37444e2e1f5bb0284e939643039c0d513217861af8133dd8bd6e8a4325", "role": "Current Python support, dependency, package discovery, and entry-point authority." }, { "path": "tests/pytest_integration_options.py", "sha256": "de36c47ead5e45020fcd6f72cf5ab20bab195962d22ab8e69cde604a324a9ab8", "role": "Executable authority for accepted integration-test option names and values." + }, + { + "path": "tests/conftest.py", + "sha256": "82d01adebe952964b61c0cad85024d0b3ba60a4a6dba1a11f6aa715a9f046664", + "role": "Test-harness parsing of OPENHCS_CPU_ONLY, which accepts only 'true' when skipping pytest-qt." } ], "findings": [ - "The old guide recommended shared editable environments, incorrectly claimed tracked test data was shared, duplicated the integration option catalogue, and suggested broad process/temp deletion plus manual lock-file removal." + "The focused-test and comparison commands set OPENHCS_CPU_ONLY=1; tests/conftest.py only recognises the value 'true' when deciding whether to load pytest-qt, so the commands now use OPENHCS_CPU_ONLY=true, which both the test harness and OpenHCSProcessEnvironment accept." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/git_worktree_testing.rst in full and compared it against the changed authorities: pyproject.toml (first-party floors moved to zmqruntime 0.6, objectstate 1.3.1, python-introspect 0.2.5, metaclass-registry 0.3, arraybridge 0.4.1, polystore 0.6, pyqt-reactive 0.4; JAX extra now jax[cuda12-local]; openhcs-basicpy added); the page states no version floors. Checked the eight editable installs against .gitmodules, the dev and gui extras, tests/integration/test_main.py, and every --it-* option and value against tests/pytest_integration_options.py; corrected OPENHCS_CPU_ONLY=1 to OPENHCS_CPU_ONLY=true to match tests/conftest.py.", "Re-read docs/source/development/git_worktree_testing.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: .gitmodules; pyproject.toml; tests/pytest_integration_options.py.", "Focused feature-lattice QA passed 159 tests covering the N-Z architecture and developer workflow authorities.", @@ -2811,23 +2974,21 @@ "authority": [ { "path": "pyproject.toml", - "sha256": "6217b5f029796edcd83aa47fb8632d37aa7552344755c578fa4f181a6b23282d", + "sha256": "3592cc37444e2e1f5bb0284e939643039c0d513217861af8133dd8bd6e8a4325", "role": "Current Python support, dependency, package discovery, and entry-point authority." }, { - "path": "openhcs/core/compiled_execution.py", - "sha256": "941115f2dca2e53538d3141a59f3ae1f928a87cf741fcfa177f7cd64a5f41aea", - "role": "CompiledExecutionBundle and runtime-environment authority; execution-wide worker facts remain separate from per-step framework devices." - }, - { - "path": "docs/source/development/index.rst", - "sha256": "fd425732b5660df4f8f110174f6e773f070160f73828bb84f7dd641965184fdd", - "role": "Navigation or authoring-policy evidence only; product claims are also grounded in non-documentation authorities." + "path": ".gitmodules", + "sha256": "b70034bf0b9661f2528f002e08c03042d6cece29de79bd96c11d2d5a943684bd", + "role": "Recorded first-party submodule set behind the recursive-clone and install-extracted-packages-first instruction." } ], - "findings": [], + "findings": [ + "The record cited openhcs/core/compiled_execution.py and the page itself as authorities although the page makes no compiled-bundle claim and a page cannot prove its own claims; replaced them with .gitmodules, which owns the recursive clone and extracted-package statement. Page text needed no change." + ], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/index.rst in full and compared it against the changed authorities: pyproject.toml (first-party dependency floors, JAX extra, openhcs-basicpy) and openhcs/core/compiled_execution.py; the page states neither versions nor compiled-bundle facts. Confirmed every toctree target exists and the recursive-clone guidance matches .gitmodules; dropped the unrelated compiled_execution.py and self-referential page authorities.", "Re-read docs/source/development/index.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: pyproject.toml; openhcs/core/compiled_execution.py; docs/source/development/index.rst.", "Focused feature-lattice QA passed 159 tests covering the N-Z architecture and developer workflow authorities.", @@ -2864,12 +3025,12 @@ "authority": [ { "path": "scripts/capture_media_gallery.py", - "sha256": "987f0b3fe09f98fff0e7878053867311af7b65004569bd75937f627464e063b6", + "sha256": "f4808f79d960a5cfcdb40edd37780120189f1c21e833e51f5a4878af36fec9f1", "role": "Owns the typed capture manifest, host-tool discovery, trusted UI-bridge session, declaration-selected contextual fixture preparation, X11 capture, deterministic derivative construction, validation, and command-line surface." }, { "path": "scripts/gallery_catalog.py", - "sha256": "7f493e0cb305e14ee4cf7d3e995c67df4200186a8a8aaf00eef0bc66d8d191cf", + "sha256": "7fd3fce2b8fdae1abf049bb4f8a3a310ab598a7f2c507578425be537903ccfca", "role": "Owns the nominal gallery scenario lattice, stable-window and contextual capture targets, publication targets, derivative inventory, source evidence, card rendering, and generated release record." }, { @@ -2879,7 +3040,7 @@ }, { "path": "tests/unit/test_capture_media_gallery.py", - "sha256": "aabfe3defb657b72150caaa3737f06352056e8580060633999a4296818ee8dcd", + "sha256": "c1ee8fe70c63a4308c8c4b38b2733e73c4e9c7719695c305b723e7892c8dc04b", "role": "Proves manifest bounds, path containment, no-overwrite rules, typed UI-bridge snapshot ingestion, contextual target and scope projection, flash-lifecycle waiting, X11 capture commands, deterministic FFmpeg projection, atomic builds, and independent derivative validation." }, { @@ -2888,12 +3049,10 @@ "role": "Consumes the same scenario declarations to produce individual RTD workflow figures and the complete stable-window reference without creating a documentation-only media registry." } ], - "findings": [ - "The active capture workflow lived outside docs/source, so Sphinx navigation and the documentation drift ledger did not own it.", - "The original title and opening described a tool rather than leading a competent maintainer toward the publication goal." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/media_gallery_capture.rst in full and compared it against the changed authorities: scripts/capture_media_gallery.py (UI-bridge calls now go through UiBridgeService.invoke with typed operation classes, to_jsonable moved to python_introspect), scripts/gallery_catalog.py (contextual actions are SessionOperation classes such as ShowDatasetImages and ShowLiveResults instead of PlateManagerAction values; StepEditorScope moved to openhcs.authoring.session.step_scopes) and tests/unit/test_capture_media_gallery.py (native-frame snapshot requests). None of these internals are named by the page; re-checked every subcommand and option (doctor, catalog, plan, build, validate, capture-still, capture-scenario-still, capture-ui-window-references, capture-ui-context-references with --fixture-root, record-window with --no-mouse), manifest schema_version 2, the 30-second trim limit, the documentation-fixture preparation path, the UI-bridge environment variables, the multi-plate-overview and lazy-inheritance scenario ids, and gallery_catalog --check; all remain accurate.", "Re-read the complete how-to against scripts/capture_media_gallery.py, scripts/gallery_catalog.py, scripts/website_gallery_projection.py, and tests/unit/test_capture_media_gallery.py; every capture, transformation, ownership, validation, and no-overwrite claim matches its current owner.", "Passed python -m scripts.capture_media_gallery catalog and verified the declaration-owned scenario inventory; python -m scripts.gallery_catalog --check confirmed the checked-in release projection.", "Passed 42 focused capture and website tests, including a real FFmpeg derivative build and independent revalidation.", @@ -2908,7 +3067,7 @@ }, { "path": "docs/source/development/mcp_development.rst", - "source_sha256": "829194108df1e11d2b8aaae97da3a0ce7cfd6c5a2842e58ef3ab3ad305a88de7", + "source_sha256": "f0c6b83bb855c6455138e2ade37b4afbc30bd2a71e0a50f549f6cdad1ea89893", "audience": [ "OpenHCS MCP developers" ], @@ -2917,149 +3076,143 @@ "authority": [ { "path": "openhcs/mcp/dev_client.py", - "sha256": "cb91fcaf3040ff9bc41d6145b71aac8249b42b0369ce8547531cccf7f76dcb3a", - "role": "Current development-client parser, persistent shell entry point, and persistent-session selected-surface preflight." + "sha256": "c40f72cb9ba7f62b574d3f9e7b96470b5a78db60c87f6a4a99897449ae875714", + "role": "Current development-client parser, persistent shell entry point with --command and --stop-on-error, and the --surface option." }, { "path": "openhcs/mcp/dev_client_commanding.py", - "sha256": "e1c987624c76be157a6524cda985498351da3c77ef19c99b85e05dd5c2e9b970", + "sha256": "abb1d9a82c4f36e171db61c0cf11eefa06fde10cc9b65a1ca031be138d46c234", "role": "One-shot command orchestration and selected-surface preflight before subprocess construction." }, { "path": "openhcs/mcp/dev_client_core.py", - "sha256": "6b05721536e92ed0f9ce4fdd0b6ebd4dbc75201277d1df1d625bf87456d5a57e", - "role": "MCP dev-client session, declaration-derived command surface validation, receipt decoding, skip-reason-owned summary status, timeout, and workflow-owned terminal-state criteria." + "sha256": "41739b744b0e4287ca7d14cd133371d669e9bff12f4b622313996ae9bb5597da", + "role": "MCP dev-client session, selected-workflow argument projection, revision-token terminal-state criteria with idempotent initialization, and transport timeout." }, { "path": "openhcs/runtime/zmq_config.py", - "sha256": "842b25ffd542f99923dafb2f67d23bac0263f38d85e64ef1bdbaba3da671b792", - "role": "Declares the execution-submission deadline separately from short execution control deadlines." + "sha256": "7c96c39d684d450ae045d1de1186dd27c5d67e042996857d5cdfcda6c2d4d069", + "role": "OpenHCSZMQConfig, the OpenHCS execution transport configuration named by the page." + }, + { + "path": "external/zmqruntime/src/zmqruntime/execution/config.py", + "sha256": "a3f3ba1f16fbedeb282825fbc381d4cc788fa8ac1600fd50fa1a27b5c65421db", + "role": "Declares execution_submission_timeout_ms separately from the shorter control_timeout_ms." }, { "path": "openhcs/mcp/dev_client_commands/knowledge_pipeline.py", - "sha256": "91f04c09fbd3d3b83d962c221bee45b8f897a5cab236559abf964303a5fdb5cd", + "sha256": "dc64de05b47e7ce538431b67d55319b723ba637f54033ec6675395d7c1e61a3c", "role": "Projects the declaration-owned execution submission deadline into the execute-source development command." }, { "path": "openhcs/mcp/control_timeout.py", "sha256": "da74437a45c2c7bd3e249b202717c7adfd178433f667d60327fa513db233d0ee", - "role": "Projects the UI-bridge and viewer declarations into bounded development-client control timeouts." + "role": "Projects the UI-bridge and viewer declarations into bounded development-client control timeouts whose default equals their maximum." }, { - "path": "openhcs/agent/dto/ui_bridge.py", - "sha256": "585b1701b01b92bfce91437a60881afa769e36de9126807c3f4fcaf0ffc006fb", - "role": "Declares the default UI bridge connection timeout." + "path": "openhcs/agent/dto/viewer.py", + "sha256": "24435279b0d31bfd62ea8c29c07cbadc564a3427fd2b06c5e4abf20992f27f12", + "role": "Declares the five-second default viewer control timeout and the layer-isolation request." }, { - "path": "openhcs/agent/dto/viewer.py", - "sha256": "293ad9fcbd7570e74170e65f9e9d0c51eb78ead57f41213a2878ad6a95d055e3", - "role": "Declares the default viewer control timeout." + "path": "openhcs/agent/services/ui_bridge_service.py", + "sha256": "c7bdf8b5caec1c21c986cdc2224524839fadfa8764dafcd05125d90c72351899", + "role": "Declares the default UI bridge connection spec whose timeout bounds UI development commands." }, { "path": "openhcs/mcp/dev_client_commands/ui.py", - "sha256": "6c1142ce723a5d28ef14726b10868e11ede4cd043b7ea40a622fcdcc1eecc8a4", + "sha256": "be3bc9096e4a711cc6583667596ba25a60428711f2883248e82d5ac66d6dd28b", "role": "Owns declaration-derived UI development commands, including selected-workflow ordering and the typed output-directory projection for window snapshots." }, { "path": "openhcs/mcp/dev_client_commands/viewer.py", - "sha256": "9d854b05a76d67dc2ba4c4f0873a77b95630269f60d4564f4630ab1e4e3e572e", + "sha256": "b5e6551624d81a27507efe0553bafa4c20e6a533d00fd14146fb01b7a024e3de", "role": "Declares the development-client viewer state, payload, ROI-summary, navigation, and isolation command surfaces." }, + { + "path": "openhcs/agent/capabilities.py", + "sha256": "eaebe33af01d057d1d21b3701f958bc96ede3ec5d0bddddca9917c1164ef0042", + "role": "Projects each SessionOperation as an openhcs_ tool and declares openhcs_session_events, openhcs_session_datasets and the local surface profiles." + }, + { + "path": "openhcs/authoring/session/session.py", + "sha256": "587047b63972df89ec46a01ed2c92ff3ef9577335f58a980297db02094e4e8e0", + "role": "Session submission budget and runtime-observation export validation (writable root, no existing path, outcomes require a path)." + }, + { + "path": "openhcs/agent/dto/session.py", + "sha256": "cfad720d060c36c2985d956fbf1f0ca838476ba1521acbdc0c95b4e8c7a746c6", + "role": "Typed run request fields runtime_observation_export_path, runtime_observation_export_scope and submit_timeout_ms, and event_sequence results." + }, { "path": "tests/unit/agent/test_mcp_server.py", - "sha256": "b99b6ca27be471f95ddf0852f8f94ce02797a459bf98bf6e0fa4851c0bb8c698", + "sha256": "4e6f6b1761b2c0316c99984baad6ea2c97e5c76c3f155111b8e498d7358e3175", "role": "Regression proof for selected-surface preflight, stale-row rejection, idempotent initialization, and every declared or malformed bridge-receipt continuation outcome." }, { "path": "openhcs/mcp/bootstrap.py", - "sha256": "64b2ef9a4f4fe50154b4c921a7617b2e7c9dcf8bb475df7423e37e8395a8b250", - "role": "Installed stdio bootstrap, surface selection, and stale-process boundary." + "sha256": "0e8e94cf54a32a1772e52a77d1c89a9ee02c6d2a58eb04f6e18fad2ce79d0394", + "role": "Installed stdio bootstrap, surface selection, OPENHCS_MCP_VERBOSE, and stale-process boundary." }, { "path": "openhcs/mcp/stdio.py", - "sha256": "4a5299915413293e5c2492d4e3889614231a1286f93a976094e4d2eba77ea9c0", + "sha256": "ec2935e01e9dc7c0ea56781c174f490528002bd736ac19062bd3a848f7397e7e", "role": "Reserves protocol stdin and stdout before server construction and owns process-lifetime application I/O isolation." }, { "path": "tests/unit/agent/test_mcp_stdio_transport.py", - "sha256": "4f7c439168dacc350d7fc31618f10dfd7ce7478d52bd5ba13ed54e720291aff2", + "sha256": "ffad185f736a7ead7b6e2e7699fb3c9b034c24534af14ea8848e0eb6989cfe1a", "role": "Exercises persistent sessions after application output, cold native imports after handshake, and process-handle restoration." }, { "path": "tests/unit/test_mcp_dev_client_persistent_session.py", - "sha256": "043281c5b330e99fdb6e7941fd66655ee956256378b3676a393a11a44e61e65d", + "sha256": "5d3b517601b8a32b53a8519b65ee03fd1a7b51b1fb6e18203a58b50bd28f064f", "role": "Regression proof for persistent dev-client parsing, reuse, errors, and stop-on-error semantics." }, { "path": "openhcs/runtime/zmq_execution_client.py", - "sha256": "31081f2a323a81c060aede1201a4bf1a1b39a51f37928c281a419ffaceadfe60", + "sha256": "147fb663d3ea60b1348063440e05b77d54b40634d1ae81cf4fb4f3b5457fbc99", "role": "Owns the total headless submission budget and the pre-dispatch timeout boundary documented for MCP developers." }, - { - "path": "openhcs/agent/services/execution_session_service.py", - "sha256": "1a9a70d2924b1854338612b212f00888bacba5cd33eb23608be294bb06c76399", - "role": "Retains the submitting execution client, distinguishes dispatch outcomes, and projects its exact progress observation." - }, { "path": "tests/unit/test_zmq_submission_timeout.py", - "sha256": "b525a1cc113bdae987c7f202369cc251ca1129188bd8d3adb834aec44f4c8393", + "sha256": "acea5dcb72c820ed8c1f225dc1960dfd615ad3f002ef6aec8e78b75bf8e50b3e", "role": "Regression proof that deadline expiry after slow connection prevents execute dispatch." }, { "path": "openhcs/agent/services/ui_bridge_transport.py", - "sha256": "32e759baf65b34989c99223d8fa9e64800ab566cd05160df2bc125b04dea5953", + "sha256": "1c2fb99aef62969cfabb9d22ccb954e8894ee72fd2e66e625304ffdb96a41b95", "role": "Rejects response envelopes from an incompatible desktop application or bridge protocol." }, { "path": "openhcs/pyqt_gui/services/ui_bridge_server.py", - "sha256": "667f582fffd4b489ca7274f8962a759c3d33d24b10ec8e6d1aa5cac01233b067", + "sha256": "4b0664fa62d24557294324b631ee3e39052c84e578d40ec408197bce5adcedac", "role": "Rejects stale MCP request contracts before they reach live UI state." }, { "path": "tests/unit/pyqt_gui/test_ui_agent_bridge.py", - "sha256": "1feee4bb413738f04e81cbdf26244dc0d585a68921e6c7749a941a08518007a1", + "sha256": "de6758859d9743e826369162855327e4a84056cafdd7ddfdc33868d2a3093353", "role": "Proves stale protocol and application versions fail before UI operation dispatch." }, - { - "path": "external/zmqruntime/src/zmqruntime/execution/progress_stream.py", - "sha256": "b2dd8e27f8fcac010beef3f8532fd9d584d7bae8c3f5591dcee0e5a235d6213a", - "role": "Owns the immutable event and monotonic sequence wire observation described for asynchronous jobs." - }, - { - "path": "openhcs/agent/dto/execution.py", - "sha256": "1fbdc008937e3a043cf077e2243b40f6a0efba34d9166716c423af9b85445684", - "role": "Declares the public typed execution status and progress observation result." - }, - { - "path": "tests/unit/agent/test_agent_services.py", - "sha256": "0c3a5225c1e832d9fbd5a423ef0cfd495451c0872b6b718d5fd66726f0758fdf", - "role": "Proves exact progress projection from the submitting client and one-time release at terminal status." - }, { "path": "openhcs/mcp/server.py", - "sha256": "ed65a008b52db505393413f0fbadf6b096c648ff48340c7e2dda8584750f6c68", - "role": "Owns declaration-derived progress execution and verbose delayed thread-stack diagnostics across the full invocation." + "sha256": "e1f236011df715da3d17574c0b80cbf50fcae45e3f82491c22fb57e98c6bb662", + "role": "Owns the mcp_server_stale guard, declaration-derived progress execution, and verbose delayed thread-stack diagnostics across the full invocation." }, { - "path": "openhcs/pyqt_gui/config.py", - "sha256": "a3eece2e5b2926fc6fc0edd197221da451ab7c0d0d6f4ffc8bb2ff865ad74bd4", - "role": "Owns the isolated UI configuration cache override forwarded to the development MCP child." + "path": "openhcs/agent/ui_bridge_environment.py", + "sha256": "30f459623ed48c6d653725080453630ef9813f875dd8e63d171147f5e73caacf", + "role": "Declares OPENHCS_UI_BRIDGE_DESCRIPTOR, OPENHCS_UI_BRIDGE_DESCRIPTOR_DIR and the UIConfigCacheEnvironment override forwarded to the development MCP child." } ], "findings": [ - "The development guide described bounded UI and viewer calls without identifying their connection declarations as the timeout authority or stating the five-second bound.", - "The development workflow did not explain that submit_timeout_ms spans cold startup or distinguish a known pre-dispatch timeout from an unknown post-dispatch outcome.", - "The workflow did not explain how to recover when a client-owned MCP process and desktop bridge were launched from incompatible OpenHCS environments.", - "The selected-workflow waiter could accept a terminal Plate Manager row from an earlier operation without proving dispatch receipt or a state revision.", - "Reusing a live UI exposed that idempotent initialization can complete without changing the Plate Manager revision; the workflow criterion now owns that exception while compile and run retain the stale-row guard.", - "Capability-local stdout guards left delayed imports and background native output able to corrupt the persistent JSON-RPC stream; process stdout is now reserved once by the stdio transport.", - "The command examples omitted the bounded ROI-summary and atomic viewer-isolation probes needed to validate current viewer agent workflows.", - "The how-to did not state that the development client validates commands against the selected declaration-derived surface before starting or dispatching; the selected-profile workflow now fails early with compatible registered profiles.", - "The UI window-snapshot command could not select the request DTO's declared output directory even though the underlying capability and viewer command supported it; the CLI now projects that field through the shared request-option authority.", - "The guide called --timeout-seconds a generic client timeout even though standard progress renews it; it now distinguishes transport inactivity from operation-owned maximum duration." + "The record cited openhcs/agent/services/execution_session_service.py, deleted when headless MCP became a session (880cb2c55, U1); the page had already been updated to describe SessionOperation tools, event_sequence, openhcs_session_events and openhcs_session_datasets, so their owners (capabilities.py, authoring/session/session.py, agent/dto/session.py) replace it.", + "The record still cited the asynchronous-job progress-observation owners (zmqruntime progress_stream.py, agent/dto/execution.py, tests/unit/agent/test_agent_services.py) after the page dropped its openhcs_get_execution_status paragraph; they were removed from the authorities.", + "The execution-submission deadline the page attributes to OpenHCSZMQConfig is now declared on the inherited zmqruntime ExecutionTransportConfig; the zmqruntime config was added as its owner. Page text needed no further change." ], - "disposition": "revise", + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/mcp_development.rst in full and compared it against the changed authorities: the page's own U1 update (session-operation tools replacing the job-status paragraph; selected-workflow initialize_datasets), dev_client.py, dev_client_commanding.py, dev_client_core.py, zmq_config.py, knowledge_pipeline.py, agent/dto/ui_bridge.py, agent/dto/viewer.py, dev_client_commands/ui.py and viewer.py, bootstrap.py, stdio.py, zmq_execution_client.py, ui_bridge_transport.py, ui_bridge_server.py, progress_stream.py, agent/dto/execution.py, server.py, the deleted execution_session_service.py and their tests. Confirmed every dev-client command and option in the examples, the full/desktop/core/authoring surface names, initialize_datasets as a UiSelectedPlateWorkflowKind value, the revision-token and idempotent-initialization wait criteria, the 5000 ms UI and viewer control bounds, the 30000 ms submission default and pre-dispatch timeout error, the openhcs_ session tools, event_sequence, and the runtime-observation export checks; replaced the deleted and no-longer-relevant authorities.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", "Source search confirmed every documented dev-client command; persistent-session and MCP-server parser tests cover the command shapes", "Focused submission tests proved that slow startup cannot dispatch an execute request after the caller's budget expires", @@ -3112,28 +3265,34 @@ "authority": [ { "path": "openhcs/agent/services/knowledge_base_service.py", - "sha256": "88e10e18c66eb3a8bc2ec5dd33493a9473510c0b0a9d355502d668042f32fdd9", - "role": "Source-backed knowledge catalogue, search, section reads, and lazy Official30 projection." + "sha256": "dfc26135b16f3827ced85b122cc1555c1325da1c5e805b421afcee9cdd7d91f3", + "role": "Source-backed knowledge catalogue, search, section reads, the 12,000-character default bound, and on-demand uncached Official30 conversion through import_cellprofiler_pipeline." }, { "path": "openhcs/agent/knowledge_manifest.py", - "sha256": "2136e24d452774f8ae92a2644940da547152bb7e2d995a9fdb476ba4bcea0107", - "role": "Canonical allowlist mapping document identities to source-backed knowledge content." + "sha256": "2689e052a31a075475e48bcc288a5ce6ec43417a607486f67b892f794f61ff44", + "role": "Resolves the knowledge manifest and Python sources from the selected checkout or the packaged resources, and exposes manifest source paths to the MCP stale-source watchlist." + }, + { + "path": "openhcs/agent/capabilities.py", + "sha256": "eaebe33af01d057d1d21b3701f958bc96ede3ec5d0bddddca9917c1164ef0042", + "role": "Declares the openhcs://knowledge resource and the three knowledge tools with their knowledge, knowledge-document, and knowledge-search dev-client commands." }, { "path": "tests/unit/agent/test_knowledge_base_service.py", - "sha256": "7b4ab158ced3ec3355d314d04e9baf0c8a4e8b2e60ce84aa56b2723c44f563c1", - "role": "Regression proof for canonical document IDs, search, bounded reads, and section selection." + "sha256": "ce4636580893a8736fd4607c6ed0f431d25c97e0c44e2b95912af3fc36b1d750", + "role": "Regression proof for document IDs, search, bounded reads, and section selection." }, { "path": "tests/unit/agent/test_official30_knowledge_examples.py", - "sha256": "6b81450dc01a33370fdad7a9a3b88bc87c3f85eddcd8b872373da9a89172c7a4", + "sha256": "e8c5b917eceaed404e4a4858ede3acfa7cf3864255e27b82f3d6b8c4799ada9e", "role": "Regression proof for lazy exact-case Official30 source conversion and structured failure." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/mcp_knowledge_base.rst in full and compared it against the changed authorities: knowledge_base_service.py (comparison-manifest parsing moved to openhcs.agent.knowledge_manifest_schema.ComparisonManifestSnapshot; per-case module inventories are now cached by source mtime), knowledge_manifest.py (comparison_manifest_type_for_root) and both test files. Confirmed the generated Python conversion is still recomputed per request and never stored (the new cache holds only module-name inventories), the 12,000 default bound, the four MCP surface names and dev-client commands with --limit, --max-chars and --section-id, the four document ids in mcp_knowledge_base_manifest.json, benchmark/manifests/official30_portable_axis1.json, the first-use capability names, and every cited documentation path; no page change was needed. Added openhcs/agent/capabilities.py for the surface names.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", "tests/unit/agent/test_knowledge_base_service.py passed; source inspection confirmed lazy selected-section Official30 conversion", "On 2026-08-24, re-read the complete page against the changed declaration, dependency, UI state, knowledge, and test authorities named by this record; the documented behavior and ownership boundaries remain accurate.", @@ -3143,7 +3302,7 @@ }, { "path": "docs/source/development/mcp_release.rst", - "source_sha256": "df77e028492fbfa6856545a9eb138959a3ad7535de2878ede2583fa2b39f5ab1", + "source_sha256": "257396cda52f8bad5a3e32dc849007a939f4cd36ecbf03de298ba37083e2f9b7", "audience": [ "OpenHCS release maintainers" ], @@ -3152,7 +3311,7 @@ "authority": [ { "path": "openhcs/__init__.py", - "sha256": "4cb0632444fde9d3e430cc3ccce9c2b5c917e731e4b430356c5f48943de4abb1", + "sha256": "4ba9e81be71e75623f89835e547f37c621bf998264ffb9f6e9adc2895db14928", "role": "Single OpenHCS package version authority." }, { @@ -3197,17 +3356,17 @@ }, { "path": "tests/unit/test_validate_local_release_floors.py", - "sha256": "c9016348d86f7585ee08a4e1432e10c8071c612736aaf171dcd7e495efe14ff1", + "sha256": "36b9e9aa7a4d7ac78c7eaa229750582a025ea8bffe43e3b2ccd2350fe496c242", "role": "Proves metadata discovery and nominal requirement-to-candidate compatibility across stable and pre-1.0 release series." }, { "path": ".github/workflows/publish.yml", - "sha256": "5039b3878de77647a1abf839923ffb0f675478263ffc9a8840139710d00a6ca9", + "sha256": "918ea1bbc262845ecb7e6bffefcf0d6af0cd2ddb598f7be9ee8635d9d3c34bb7", "role": "Exact-tag installer, installed-GUI startup, PyPI, GitHub Release, recovery, and MCP Registry workflow consuming the central release requirements." }, { "path": "tests/unit/test_mcp_release_workflow.py", - "sha256": "801c4b41931b1e725e8e42d8c0c30a78cd43cd63baeb6de5ed7bfb27d0273292", + "sha256": "5433d76ab87c00f0047e43b65c7a590aab7a8e397a0a1fd7cf6dc3b01e062854", "role": "Proves immutable tag resolution, exact-SHA checkout, publication ordering, recovery, Registry policy, and publication-independent native candidate ownership." }, { @@ -3217,12 +3376,12 @@ }, { "path": ".github/workflows/integration-tests.yml", - "sha256": "c1ec94eab493bc46986c55b8b1428d9a0a5c0eaf5789f2dddbc06df2c0843ff7", + "sha256": "5017eb98536006a0cec4f34573a35c1a547c3ad89889e276e88a8fb6f992991c", "role": "Cross-platform source-candidate and published-dependency readiness, installed-wheel live GUI/MCP operation with retained native screenshots, three-stage native-installer workflow proof, signing-helper parse, and staged-update validation matrix." }, { "path": "scripts/resolve_successful_job_baseline.py", - "sha256": "8e6541a53dac376176a63cb449e9166a425ef673954847cccd96fe444d60ad48", + "sha256": "be32836149414a8edf9ea73e1316907847376c86156b758b786b21fbd96a3283", "role": "Derives the cumulative Python quality boundary from the latest successful code-quality job on an ancestor workflow run." }, { @@ -3237,7 +3396,7 @@ }, { "path": "tests/installer/test_windows_simple_installer.py", - "sha256": "3ab5b37fb4deee277f9deee41dc61deab5235e245e5ff37c98f32b06fde65697", + "sha256": "c83a5de13f423f7fd83624cabe38dec2901cb4ccfdc367cbca8e7c0460cbe6e7", "role": "Proves the Windows native wizard waits for rendered controls, synchronizes window composition, and retains each declared installer evidence state." }, { @@ -3262,7 +3421,7 @@ }, { "path": "scripts/smoke_installed_gui.py", - "sha256": "a5af7f55eab45b7ab43772a9ff8cf758e4940c5ad58857611cc51dbe6004bd89", + "sha256": "d9bf24b9e5c631b4eb6a3de0fc7a79a5ee56a848693ed6692b60dc9c570d4efc", "role": "Owns the installed-GUI acceptance timing and single terminal completion, applies the declared logging configuration, constructs the application on declaration-allocated endpoints, proves the background function catalogue and advertised endpoint log are usable, drives the GUI through packaged MCP, validates retained window snapshots, and cleans up exact owned resources outside the checkout." }, { @@ -3292,7 +3451,7 @@ }, { "path": "tests/installer/test_installer_signing.py", - "sha256": "c8f9d050c4337086766e732bb09b39bd7cd61d0f7c4866fd66f6417d99c4ca1b", + "sha256": "32622638bced1f620e21b3f251df8a70da27f5fc3cbafd98291c092d617fdb42", "role": "Regression proof for workflow signing policy and both native signing-helper contracts." }, { @@ -3302,38 +3461,17 @@ }, { "path": "tests/unit/test_ci_package_boundaries.py", - "sha256": "691efcdafe66255c55ae11411b441930bdb58161cd74e2580c2531a486863303", + "sha256": "df85419ab722292f54a520bf61bd1a11ecc08b7637ffa2abdf70a16f16088e04", "role": "Proves publication consumes central release requirements and that candidate and published GUI matrices retain their typed screenshot evidence." } ], "findings": [ - "The introduction called a version-specific 0.5.22 readiness plan the current checklist even though OpenHCS and the release workflow have advanced.", - "The pinned Jaunch signing guide link used an anchor that no longer exists in the target document.", - "Installer recovery now enforces an existing remote release tag before any build, and signing claims require the native helpers and integration matrix as authorities.", - "The release-candidate matrix now has one published-dependency readiness gate backed by fetched submodule release tags and exact installer-visible wheel requirements.", - "The PyPI installation matrix waited for the exact dependency wheels but then queried each runner's potentially stale package index instead of consuming that projection; every install lane now reuses the hash-pinned wheel requirements.", - "The macOS disk-image helper retained only the backing device and could leave its synthesized APFS volume mounted; the attachment boundary now carries and releases both exact device identities.", - "The release-candidate section did not identify the dedicated GUI source gate or distinguish it from installed-package evidence.", - "The per-push Python quality comparison could forget a red change after an unrelated follow-up push; it now retains changes since the most recent ancestor whose code-quality job succeeded.", - "Local tagging, manual publication, and installer recovery were independent fail-open paths; one readiness authority and one immutable tag-resolution job now bind them to the same exact commit and CI evidence.", - "PyPI recovery used --skip-existing, which could hide independently rebuilt artifact conflicts; existing release files now stop publication.", - "Workflow dependencies used movable major-version tags and deprecated Node 20 action releases; every external action now uses a current full-length commit pin maintained through grouped Dependabot updates.", - "Release-candidate installer lanes inherited workstation pip configuration and package indexes; native installation, reinstall, and staged update now suppress those external settings while CI injects an unreachable index as proof.", - "Published OpenHCS 0.7.26 could resolve a nominally valid but startup-incompatible pyqt-reactive version; installed-wheel publication gates and the scheduled published-release canary now exercise the real GUI ready boundary.", - "Six first-party requirements had no breaking-series ceiling; the package metadata now bounds every extracted dependency, and the metadata-derived release validator rejects an unbounded future candidate.", - "The installed-GUI probe reused the default execution endpoint, allowing an existing desktop server to influence an otherwise isolated package canary; it now allocates the endpoint through the selected transport declaration.", - "Installed GUI and MCP package checks terminated before either operated the other; the source-wheel, publication, and scheduled canaries now keep the packaged GUI live while the packaged desktop MCP discovers it and invokes a declared UI action.", - "The fixed desktop candidate was operated before publication only on Linux, while Windows and macOS evidence remained behind the unpublished-dependency gate; a complementary source-candidate matrix now proves Windows and both macOS architectures alongside the Linux wheel lane.", - "The native source-candidate matrix still depended on published first-party tags, so the real installer and staged-update jobs were skipped exactly while those dependency releases were being prepared; the native lane now builds its complete wheelhouse from metadata-discovered submodules while the separate PyPI lane remains fail-closed on publication readiness.", - "Cross-platform GUI canaries proved readiness and MCP actions but discarded the rendered UI, leaving native theming regressions outside retained evidence; candidate and published-package jobs now upload typed, hash-verified window snapshots.", - "The installed-GUI canary constructed the application below the production logging boundary, so a successful endpoint could advertise an empty log; the probe now applies its declared logging configuration and fails unless the exact advertised log contains output.", - "Native installer acceptance invoked worker mode without proving that either shipping wizard could start or drive installation; each supported lane now activates the shipping wizard and retains its welcome, progress, and completion states.", - "The Windows installer evidence harness captured its welcome frame when only the top-level handle existed, before child controls painted; capture now requires the real primary control and synchronizes recursive rendering with desktop composition.", - "Integration and Documentation workflows allowed superseded commits on the same event and ref to consume the full hosted matrix; their declarations now cancel predecessors while keeping manual dispatches independent.", - "Post-PyPI recovery rebuilt the wheel and source distribution, so GitHub could receive semantically equivalent but byte-distinct archives; recovery now materializes the PyPI-owned files and verifies their declared hashes before attaching them." + "The tag and recovery workflow descriptions said publish.yml builds and smoke tests one OpenHCS wheel; publish.yml now builds stable-ABI wheels for Linux (x86_64, aarch64), Windows and macOS (x86_64, arm64) with cibuildwheel, smoke-tests each native extension, builds the sdist separately, validates every wheel, and smoke tests the Linux x86_64 wheel. The PyPI-file recovery path now materializes several wheels.", + "The installed-GUI smoke no longer invokes a PlateManagerAction code action; it invokes the show_dataset_code session operation (ShowDatasetCode) on the Plate Manager window." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/mcp_release.rst in full and compared it against the changed authorities: openhcs/__init__.py (version 0.8.7, microscopy axis-family activation; the version-authority claim holds), .github/workflows/publish.yml and tests/unit/test_mcp_release_workflow.py (new build-native-wheels job, sdist built separately, per-wheel validation, Linux x86_64 smoke selection), .github/workflows/integration-tests.yml (parity-scope gate, Official30 parity on Python 3.12 Linux and 3.14 on all three OSes, run_installed_tests.py coverage), scripts/resolve_successful_job_baseline.py (branch filter removed; ancestor rule unchanged), scripts/smoke_installed_gui.py (typed MCP payloads, ShowDatasetCode), tests/installer/test_windows_simple_installer.py, tests/installer/test_installer_signing.py, tests/unit/test_validate_local_release_floors.py and tests/unit/test_ci_package_boundaries.py. Corrected the single-wheel publication and recovery descriptions and the GUI smoke action name; confirmed the Python 3.14 wheel-candidate lane still excludes the Centrosome oracle and the remaining gate, recovery and signing claims.", "Re-read docs/source/development/mcp_release.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/__init__.py; scripts/sync_mcp_release_metadata.py; scripts/release.py; scripts/release_readiness.py; scripts/validate_local_release_floors.py; scripts/install_ci_candidate.py; .github/workflows/publish.yml; tests/unit/test_mcp_release_workflow.py; tests/unit/test_release_readiness.py; .github/workflows/integration-tests.yml; .github/dependabot.yml; tests/unit/test_ci_action_pinning.py; packaging/installers/windows/Sign-Installer.ps1; packaging/installers/macos/Sign-Installer.sh; tests/installer/test_installer_signing.py.", "Focused feature-lattice QA passed 159 tests covering the N-Z architecture and developer workflow authorities.", @@ -3442,7 +3580,7 @@ }, { "path": "tests/integration/test_main.py", - "sha256": "1a0c532eb631a89b77f92a4202a7b55033c86d7b872350156f106d276b061e9f", + "sha256": "05331ebbe7a79554dfe7348e22ecf1ab3bbc0c58c43f9e3b9ca027f340a7dec0", "role": "Direct and ZMQ compile-before-execute integration harness with explicit OMERO lifecycle cleanup and no browser side effect." }, { @@ -3457,20 +3595,14 @@ }, { "path": ".github/workflows/integration-tests.yml", - "sha256": "c1ec94eab493bc46986c55b8b1428d9a0a5c0eaf5789f2dddbc06df2c0843ff7", + "sha256": "5017eb98536006a0cec4f34573a35c1a547c3ad89889e276e88a8fb6f992991c", "role": "Runs explicitly selected live OMERO variants and preserves component and table-service diagnostics on failure." } ], - "findings": [ - "The how-to omitted declaration-level tests for OMERO text, table-service, and image-plane semantics and did not require direct plus ZMQ live execution or a cold deployment.", - "Legacy setup documentation inferred readiness from a hardcoded container name; the how-to now follows the instance manager and PolyStore table-service authorities.", - "The how-to did not distinguish the packaged Compose lifecycle from host-owned Docker daemon startup.", - "The address-testing boundary did not distinguish concrete plane identities from symbolic pattern-discovery templates.", - "A live cold-start test could fail on a transient Docker daemon probe even though the same runner became ready moments later; the instance lifecycle now owns a bounded readiness wait.", - "The Python 3.12 cold deployment exposed Tables-0 disabling itself before the managed repository existed; tests now distinguish generic repository readiness from packaged deployment recovery." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/omero_testing.rst in full and compared it against the changed authorities: tests/integration/test_main.py (partition-filter config renames, Microscopy axis family, DatasetSourceChoice, per-test allocated ports, execute_compiled_bundle, subject_id, PipelineDocumentCodec) and .github/workflows/integration-tests.yml (parity-scope gate, run_installed_tests.py coverage). The page names none of the renamed test internals; confirmed the harness still registers OMEROLocalBackend and runs direct and ZMQ modes without opening a browser, the omero-tests-linux lane still builds submodule candidate wheels without a publication dependency and keeps Tables-0 and component diagnostics, and OMEROWellAddress, OMEROPlaneAddress, OMEROPlaneFilenameTemplate and FilenameParseResult exist at their owners.", "Re-read docs/source/development/omero_testing.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: external/PolyStore/src/polystore/omero_local.py; external/PolyStore/tests/test_omero_tables.py; tests/integration/test_main.py; openhcs/omero/docker-compose.yml; openhcs/runtime/omero_instance_manager.py; .github/workflows/integration-tests.yml.", "Focused feature-lattice QA passed 159 tests covering the N-Z architecture and developer workflow authorities.", @@ -3519,14 +3651,9 @@ "user_need": "Identify the archive status and reach the current signature/form extension boundaries.", "diataxis": "reference", "authority": [ - { - "path": "pyproject.toml", - "sha256": "6217b5f029796edcd83aa47fb8632d37aa7552344755c578fa4f181a6b23282d", - "role": "Current Python support, dependency, package discovery, and entry-point authority." - }, { "path": "external/python-introspect/src/python_introspect/signature_analyzer.py", - "sha256": "73600e546753640964d4ba5308fdf808ed3a4a6f085417d8ac0745d9e2748ed7", + "sha256": "edbe51f6c92b2c334dc4533bbeaa252be649c1ac738d2ebfca4364f2708479dc", "role": "Generic callable signature-analysis ownership." }, { @@ -3540,12 +3667,10 @@ "role": "Reusable parameter-form lifecycle and value projection." } ], - "findings": [ - "The active orphan did not explicitly distinguish its compatibility-route status from current developer guidance.", - "The current generic parameter-analysis owner now also projects callable declarations from signature defaults." - ], - "disposition": "redirect", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/parameter_analysis_audit.rst in full and compared it against the changed authorities: pyproject.toml (dependency floors only) and external/python-introspect/src/python_introspect/signature_analyzer.py (current file); signature and docstring analysis still lives in python-introspect, generic forms in pyqt-reactive, no Textual UI remains in openhcs/, and the archived audit path exists. Dropped pyproject.toml because the orphan redirect states no package facts.", "Re-read docs/source/development/parameter_analysis_audit.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: pyproject.toml; external/python-introspect/src/python_introspect/signature_analyzer.py; external/pyqt-reactive/src/pyqt_reactive/forms/parameter_form_manager.py.", "Focused feature-lattice QA passed 159 tests covering the N-Z architecture and developer workflow authorities.", @@ -3584,7 +3709,7 @@ "authority": [ { "path": "openhcs/core/debug.py", - "sha256": "072259811bc50bfe71cce9a7cac3e9867bfd853bd34ab5a74b01d9fe14b68f36", + "sha256": "95d365afefc46ee50da5abb8d5d246b3980feb32b85839f91e874cb7e689caaa", "role": "Nominal debug command, boundary, snapshot, replay, and paused-worker declarations." }, { @@ -3594,18 +3719,19 @@ }, { "path": "openhcs/agent/services/runtime_server_service.py", - "sha256": "34b61a154718e3e613245e84cdeca1c89a5fbbbddd6a5bb26f76a146d62c4f65", + "sha256": "ec521df2fbea765dedda5a64dbdbde9537cfa931823eace795d3aca86905fff1", "role": "Bounded runtime-server and paused-debug inspection service." }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "ace8ae0c3e54bbe604b97cec7c3973a1abcf90e1dc517c2adda947dc2c35e882", + "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", "role": "Compilation order, declaration resolution, callable warm-up, framework-local device assignment, and bundle construction." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/pipeline_debugging_guide.rst in full and compared it against the changed authorities: openhcs/core/debug.py (DebugJsonCodec removed, DebugProgressContext now a declared progress-context payload, plate_id renamed subject_id), openhcs/agent/services/runtime_server_service.py and openhcs/core/pipeline/compiler.py (CompilationPlateScope and plate_path renamed to dataset scope and dataset_root). The page names none of the renamed fields; confirmed openhcs_inspect_pipeline_source_artifact_plan, openhcs_inspect_debug_runtime_values and its runtime-debug-values command, debug_session_id, pipeline_debug_toolbar, FunctionStep.debug_pause, the Run to Pause, Step and Inspect Runtime actions, StepMaterializationConfig, RuntimeValueStore, CompiledStepPlan, CompilationSession and CompiledExecutionBundle still exist with the described roles.", "Re-read docs/source/development/pipeline_debugging_guide.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/debug.py; openhcs/core/debug_session_projection.py; openhcs/agent/services/runtime_server_service.py; openhcs/core/pipeline/compiler.py.", "Focused feature-lattice QA passed 159 tests covering the N-Z architecture and developer workflow authorities.", @@ -3616,7 +3742,7 @@ }, { "path": "docs/source/development/pyclesperanto_simple_implementation.rst", - "source_sha256": "5cecda2276f76c744ad6a215f8ecd58c82c2ce6ce4859d6d5990f39ba67ffc3c", + "source_sha256": "14d112b12e6984faed5c19c022a14cab430c5157a110c949ed0fd3bc0598da13", "audience": [ "OpenHCS GPU-processing callable authors" ], @@ -3631,31 +3757,36 @@ { "path": "openhcs/processing/backends/lib_registry/pyclesperanto_registry.py", "sha256": "9049165370dbea917e643395c516e08e94509777d9bf8edf9476cf1e75bccc77", - "role": "Declaration-owned pyclesperanto callable-library discovery." + "role": "Declaration-owned pyclesperanto callable-library discovery and the runtime-owned output_image, output_image_matrix and device parameters." }, { "path": "openhcs/processing/backends/processors/pyclesperanto_processor.py", - "sha256": "0504e1434a489005a1e0c3d7edeeb306acf1b4f9aba463e45ab0c817970a9652", + "sha256": "ae7b91db142b898f47831fcb3e44c67758e9e2f8c2d77ada3b347dadbc34af47", "role": "OpenHCS processing callables using the pyclesperanto integration." }, { "path": "external/arraybridge/src/arraybridge/types.py", - "sha256": "7b94760d57e54008d7adf4e4d37dca45d8a2c288a9dc8400be00c26148cf7624", + "sha256": "08f7ff4df8807bb3340e00a02c6276b1c702a0f00a2eedacd701142736c5d804", "role": "Pyclesperanto framework device discovery, execution scope, and cleanup authority." }, { "path": "external/arraybridge/src/arraybridge/decorators.py", - "sha256": "99823105b0ae59054df1d84013633b8b6628d0a6dc3326383199ac7c65c69bed", + "sha256": "28f9bf4770bd4cc1ef34045fdef28065063ede029296f62b8a6c4140cc88e40e", "role": "Memory decorator that declares pyclesperanto input, output, and execution roles." }, { "path": "openhcs/core/memory/decorators.py", - "sha256": "d9f9cf0be7593dc0ebfc57f258aebed75bcf9cfa508ed1e942137366d819766a", - "role": "OpenHCS metadata-preserving wrapper around ArrayBridge memory decorators." + "sha256": "76f234785f4865257fd19c935e0aace82d1f5586198737fec619ee243b79cbf6", + "role": "OpenHCS metadata-preserving wrapper around ArrayBridge memory decorators, accepting a ProcessingContract class as contract." + }, + { + "path": "openhcs/core/processing_contracts.py", + "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", + "role": "ProcessingContract class family including Pure3DContract used in the example." }, { "path": "openhcs/core/callable_contract.py", - "sha256": "6a39b5e171901131ecb7280a78241e8ea72409a5003eabba3db34eb4851991d7", + "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", "role": "Compiler-visible projection of all declared callable memory roles." }, { @@ -3665,7 +3796,7 @@ }, { "path": "openhcs/core/steps/function_runtime.py", - "sha256": "b1449120bebe7cf4898c055d841dc258c7d91d976d9e5ff7396aed81ad5011e7", + "sha256": "9f99f76c18669177abad9bfa4921c031875f4f9c3079f18aa3855f444b0bbe3b", "role": "Runtime conversion and invocation inside the compiled execution-framework device scope." }, { @@ -3675,15 +3806,14 @@ }, { "path": "tests/unit/test_unified_registry_runtime_payloads.py", - "sha256": "e731379d3e48fd5c98709203668e09baa195bb0c9301a2b041e1c799fa8140e3", + "sha256": "76e02cc26b40f5af984d31e874eab1b18d334259fb5cd44bc9fe5945c474b139", "role": "Proves pyclesperanto output buffers and device handles remain runtime-owned while algorithm parameters stay authored." } ], - "findings": [ - "The how-to lacked an exact decorator import, a minimal declaration example, framework-local device behavior, and a navigable ArrayBridge owner link." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/pyclesperanto_simple_implementation.rst in full and compared it against the changed authorities: the page itself (example already moved from ProcessingContract.PURE_3D to the Pure3DContract class, G5), pyclesperanto_processor.py (shared operations moved to image_stack_operations), arraybridge types.py and decorators.py, openhcs/core/memory/decorators.py, callable_contract.py, function_runtime.py and test_unified_registry_runtime_payloads.py. Confirmed the example imports (openhcs.core.memory.pyclesperanto, openhcs.core.processing_contracts.Pure3DContract), that the OpenHCS decorator accepts a ProcessingContract subclass as contract, and the three runtime-owned PyclesperantoRegistry parameters; added processing_contracts.py as the owner of the example contract.", "Re-read docs/source/development/pyclesperanto_simple_implementation.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: external/arraybridge/src/arraybridge/converters.py; openhcs/processing/backends/lib_registry/pyclesperanto_registry.py; openhcs/processing/backends/processors/pyclesperanto_processor.py; external/arraybridge/src/arraybridge/types.py; external/arraybridge/src/arraybridge/decorators.py; openhcs/core/memory/decorators.py; openhcs/core/callable_contract.py; openhcs/core/framework_device_resolver.py; openhcs/core/steps/function_runtime.py; tests/unit/test_framework_device_resolver.py; tests/unit/test_unified_registry_runtime_payloads.py.", "On 2026-08-27, re-read and revised the complete how-to after PyclesperantoRegistry began owning external output-buffer and device exclusions; the page now states which signature parameters stay out of authored forms and code, and the focused runtime-payload regression passed.", @@ -3712,15 +3842,14 @@ }, { "path": "pyproject.toml", - "sha256": "6217b5f029796edcd83aa47fb8632d37aa7552344755c578fa4f181a6b23282d", + "sha256": "3592cc37444e2e1f5bb0284e939643039c0d513217861af8133dd8bd6e8a4325", "role": "Current Python support, dependency, package discovery, and entry-point authority." } ], - "findings": [ - "The packaging paragraph attributed runtime dependency resolution to python -m build; it now distinguishes recording requirements in the wheel from resolving them during installation." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/repository_setup.rst in full and compared it against the changed authorities: pyproject.toml (first-party floors moved to zmqruntime 0.6, objectstate 1.3.1, python-introspect 0.2.5, metaclass-registry 0.3, arraybridge 0.4.1, polystore 0.6, pyqt-reactive 0.4; JAX GPU extra now jax[cuda12-local]; openhcs-basicpy added as an ordinary dependency); challenged the eight-submodule count against .gitmodules, the 3.11-3.14 range against requires-python, the centrosome parity oracle's python_version < 3.14 marker, the named extras (dev, gui, dev-gui, cellprofiler-compat, viz, gpu, docs), setup.py deferring metadata to pyproject.toml, and the CUDA 12 statement for the gpu extra; all remain accurate and the page states no version floors.", "Re-read docs/source/development/repository_setup.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: .gitmodules; pyproject.toml.", "Focused feature-lattice QA passed 159 tests covering the N-Z architecture and developer workflow authorities.", @@ -3746,7 +3875,7 @@ }, { "path": "docs/source/development/respecting_codebase_architecture.rst", - "source_sha256": "2f9665a3a063e62b401f3d44c24f596baf16d3496d0953efc269e563bda9d48b", + "source_sha256": "7db323adce801d8a8a38d8ba8d77493486870ba9a573a161f234ec1e4fdc1465", "audience": [ "OpenHCS maintainers and reviewers" ], @@ -3754,9 +3883,14 @@ "diataxis": "how-to", "authority": [ { - "path": "openhcs/core/registry_strategies.py", - "sha256": "a41186665abbd308f224d0bb41d6618cd9d89b08acaf54ec2819f2024a6e0804", - "role": "OpenHCS enum, nominal-type, and most-derived strategy selection implementations." + "path": "external/metaclass-registry/src/metaclass_registry/strategies.py", + "sha256": "5c00cd4d80f9f0ceffbc2ab95ad8c6abd44a8a66a974959445ef93df08dd4f0d", + "role": "Declares EnumKeyedStrategyMixin, NominalTypeKeyedStrategyMixin and MostDerivedContextStrategyMixin and their registry-derived selection, which the selection rules section names." + }, + { + "path": "external/metaclass-registry/src/metaclass_registry/core.py", + "sha256": "72c8284e8a32dd193f13f36a425289d5293a32f4a381046db523aa87182e0be8", + "role": "Declares AutoRegisterMeta, RegistryConfig and RegistryFamily and the __registry__ catalogue that the search command and registry discipline section rely on." }, { "path": "external/python-introspect/src/python_introspect/callable_declaration.py", @@ -3765,25 +3899,26 @@ }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "3c840a89baccc8665d415175458280fd904c1a837b7fd879e303d7dbca129a1c", + "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", "role": "Resolved step/session identity and typed plan assembly invariants." }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "5f323ec8edd4221ee30e34ed0c9c09e0f4e45ef502749c1a2d331b7a5452af99", + "sha256": "b6a7231bc2964d472070731018f6a4935e5f23cf8f238a232045c32244e6545a", "role": "RuntimeValueStore recording, replacement, query, and observation semantics." }, { "path": "tests/unit/test_configuration_mirror_deletion.py", - "sha256": "26002ddfbbf89bf06ae54888645c6b8c90154536a16e93feee1a2fae713c4963", + "sha256": "8f9da50e930925ed3ecdd57add9cc919e008a9202c9e405ff922dcd1c9a418cc", "role": "Deletion gate proving obsolete parallel configuration and browser modules remain absent." } ], "findings": [ - "The package-ownership summary omitted python-introspect's callable-declaration projection responsibility." + "The registry-discipline section pointed generic consumers at openhcs.core.registry_strategies, which commit b221f8491 deleted; the enum, nominal-type and most-derived strategy mixins are now imported from metaclass_registry.strategies, and the page names that module." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/respecting_codebase_architecture.rst in full and compared it against the changed authorities: openhcs/core/registry_strategies.py (deleted in b221f8491), openhcs/core/pipeline/compilation_session.py, openhcs/core/runtime_stores.py, tests/unit/test_configuration_mirror_deletion.py; confirmed the three strategy mixins and the registry types named in the search command exist in metaclass-registry, replaced the deleted openhcs.core.registry_strategies reference with metaclass_registry.strategies, and found the typed-plan, RuntimeValueStore and mirror-deletion claims still accurate.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", "Source search confirmed the three named strategy mixins and direct typed-plan consumers; mirror-deletion tests protect removed parallel surfaces", "On 2026-08-21, re-read the guide against callable_declaration.py and the updated python-introspect owner documentation.", @@ -3793,7 +3928,7 @@ }, { "path": "docs/source/development/runtime_system_assembly_rules.rst", - "source_sha256": "a2275aad184f08bd2966a26e44463909c92c5151b79936f1ce76f83766fc7164", + "source_sha256": "1a4cf1b603e199ed6dac4e5646530eb8e5dfb58f7939dbaf57a3f2e7b71b75ab", "audience": [ "OpenHCS compiler, adapter, and runtime maintainers" ], @@ -3802,45 +3937,48 @@ "authority": [ { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "3c840a89baccc8665d415175458280fd904c1a837b7fd879e303d7dbca129a1c", - "role": "Resolved step/session identity and typed plan assembly invariants." + "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", + "role": "Declares ResolvedPipelineDefinition and CompilationSession, the resolved declaration view and per-context compiler boundary named in the assembly order." }, { "path": "openhcs/core/compiled_execution.py", - "sha256": "941115f2dca2e53538d3141a59f3ae1f928a87cf741fcfa177f7cd64a5f41aea", - "role": "CompiledExecutionBundle and runtime-environment authority; execution-wide worker facts remain separate from per-step framework devices." + "sha256": "85972d418f0958e59f3185f07caf25a103bc9e101e481b2b63903377cc05e397", + "role": "CompiledExecutionBundle fields: runtime and transport contexts, worker assignments and runtime environment plan." }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "5f323ec8edd4221ee30e34ed0c9c09e0f4e45ef502749c1a2d331b7a5452af99", + "sha256": "b6a7231bc2964d472070731018f6a4935e5f23cf8f238a232045c32244e6545a", "role": "RuntimeValueStore recording, replacement, query, and observation semantics." }, { "path": "openhcs/core/runtime_slice_projection.py", - "sha256": "8a1cb65d3025ed8a87ee59af26deac9765a8f59b21bf2cd61b6f96d82999641d", - "role": "Nominal runtime slice projection registry and aggregation behaviour." + "sha256": "8f45a17d6aabd4fae1df55e8d3cda2efdd8df2156357c69133f06ccf1cfb2d96", + "role": "RuntimeSliceProjection boundary between owned projectable values and foreign values, with source-provenance checks on stacks." }, { "path": "openhcs/core/callable_contract.py", - "sha256": "6a39b5e171901131ecb7280a78241e8ea72409a5003eabba3db34eb4851991d7", - "role": "Callable ABI and input/output/execution framework declaration authority." + "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", + "role": "Callable ABI and input/output/execution memory-type declaration authority." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "6d46571e6f2d069b98a92b41d711b2e20bfab9f0493cf73d3f150d5be08e7f96", - "role": "Typed per-step semantic and framework-device decisions." + "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", + "role": "Typed per-step source-binding plan and framework-device decisions." }, { "path": "openhcs/core/pipeline/framework_device_assignment.py", - "sha256": "f77f3fb2d30c4af8cab6210dbe294f9fb5e246dad0a6bbf88bcaf1e2b58dead7", + "sha256": "cdfde10f996ab682952a1d4948d3de2d23d48353eae41f3124770e19fc28a193", "role": "Compiler projection from declared framework footprint to exact step bindings." } ], "findings": [ - "The assembly owner map omitted callable execution-memory roles and per-step framework-local device bindings." + "The assembly order named StepSnapshot, which no longer exists; compilation now runs from ResolvedPipelineDefinition into CompilationSession.", + "The CompiledExecutionBundle entry listed worker-safe steps and plans; the bundle now holds runtime contexts, pickle-safe transport contexts, worker assignments and the runtime environment.", + "The RuntimeSliceProjection entry described a nominal strategy registry with aggregation; the strategy family was removed, owned values declare slice behaviour through RuntimeSliceProjectableValue, and no aggregation exists there." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/runtime_system_assembly_rules.rst in full and compared it against the changed authorities: compilation_session.py, compiled_execution.py, runtime_stores.py, runtime_slice_projection.py, callable_contract.py, compiled_step_plan.py, framework_device_assignment.py; replaced the deleted StepSnapshot with ResolvedPipelineDefinition, corrected the CompiledExecutionBundle field summary and the RuntimeSliceProjection description, and confirmed CallableContract memory roles, CellProfilerModuleArtifactContracts, SourceBindingsConfig, CompiledSourceUniversePlan, ArtifactSpecRef and ArtifactKey still exist as described.", "Re-read docs/source/development/runtime_system_assembly_rules.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/pipeline/compilation_session.py; openhcs/core/compiled_execution.py; openhcs/core/runtime_stores.py; openhcs/core/runtime_slice_projection.py; openhcs/core/callable_contract.py; openhcs/core/compiled_step_plan.py; openhcs/core/pipeline/framework_device_assignment.py.", "Focused feature-lattice QA passed 159 tests covering the N-Z architecture and developer workflow authorities.", @@ -3852,7 +3990,7 @@ }, { "path": "docs/source/development/runtime_value_extension.rst", - "source_sha256": "eb0f4bdd7eac7fff1ded970acbe29db94f76e06d1bb61d8e25d564f43ea9e249", + "source_sha256": "73aaa60318f73e06b27473e7beaaad533a99f9665be7c017ae480e506063b4cb", "audience": [ "OpenHCS runtime-value extension authors" ], @@ -3861,28 +3999,37 @@ "authority": [ { "path": "openhcs/core/runtime_artifact_values.py", - "sha256": "9ac62ec4c4a6a6f7f375026ea112cc2e43f09ee6e6ca270a3d955be8ef5347cf", - "role": "RuntimeValue, ArtifactKey, location, and nominal payload wrapping." + "sha256": "930752071d37ef1a8669812e7a629e151690b3f33bf4a29dba92d1d479b9074d", + "role": "RuntimeValue and ArtifactKey construction from compiled output plans." }, { - "path": "openhcs/core/runtime_stores.py", - "sha256": "5f323ec8edd4221ee30e34ed0c9c09e0f4e45ef502749c1a2d331b7a5452af99", - "role": "RuntimeValueStore recording, replacement, query, and observation semantics." + "path": "openhcs/core/runtime_plane_projection.py", + "sha256": "3c88ed042a7f1c65c9495a1c62777e008f1e59c4742ec0563b2849d77ca87cdd", + "role": "Declares RuntimeSliceProjectableValue and its indexed, invariant and identity-stamped subclasses that payload authors join." }, { "path": "openhcs/core/runtime_slice_projection.py", - "sha256": "8a1cb65d3025ed8a87ee59af26deac9765a8f59b21bf2cd61b6f96d82999641d", - "role": "Nominal runtime slice projection registry and aggregation behaviour." + "sha256": "8f45a17d6aabd4fae1df55e8d3cda2efdd8df2156357c69133f06ccf1cfb2d96", + "role": "RuntimeSliceProjection foreign-value list and rejection of owned values without slice declarations." + }, + { + "path": "openhcs/core/runtime_stores.py", + "sha256": "b6a7231bc2964d472070731018f6a4935e5f23cf8f238a232045c32244e6545a", + "role": "RuntimeValueStore recording, replacement, query, and observation semantics." }, { "path": "tests/unit/test_runtime_value_store.py", - "sha256": "8900a7b83b59d0a40f9698dbafaaba778df982f9d7a0b628f80eab97335aeab0", + "sha256": "4ea189341b761bda993eaa2b6e24b83917568d047f4834d9f44410f7545d1452", "role": "Regression proof for typed runtime recording, replacement, queries, and failure behaviour." } ], - "findings": [], - "disposition": "keep", + "findings": [ + "Step 3 told authors to register a RuntimeSliceProjectionStrategy; that strategy family was deleted. Payloads now subclass RuntimeSliceProjectableValue (or RuntimeSliceIndexedValue / RuntimeSliceInvariantValue) and RuntimeSliceProjection handles only foreign values.", + "The page claimed projection strategies are chosen by MRO specificity and that slices are aggregated; neither holds for the current projection boundary." + ], + "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/runtime_value_extension.rst in full and compared it against the changed authorities: runtime_artifact_values.py, runtime_stores.py, runtime_slice_projection.py, tests/unit/test_runtime_value_store.py; replaced the deleted RuntimeSliceProjectionStrategy registration step with the RuntimeSliceProjectableValue base classes from runtime_plane_projection.py, removed the MRO-selection and aggregation claims, and confirmed RuntimeValue, ArtifactKey, separate RuntimeArtifactLocation and fail-closed store queries remain as described.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", "tests/unit/test_runtime_value_store.py and tests/unit/test_runtime_slice_projection.py passed", "On 2026-08-24, re-read the complete page against the changed declaration, dependency, UI state, knowledge, and test authorities named by this record; the documented behavior and ownership boundaries remain accurate." @@ -3890,7 +4037,7 @@ }, { "path": "docs/source/development/source_binding_extension.rst", - "source_sha256": "a1b709b00dca507f7183aa847d208192d6f79f2eba8cf5f452cec37b9510a63d", + "source_sha256": "65a1cd3ee98bed4974a17d20d1998e325dc9685eb044e071393189c680c8df79", "audience": [ "OpenHCS source-ingestion and compiler extension authors" ], @@ -3899,30 +4046,42 @@ "authority": [ { "path": "openhcs/core/source_bindings.py", - "sha256": "e0afbbe40cce3847bcc2314910d0083ecc579a5a0d6e4278c3a5df4c62e7eb01", - "role": "Public typed source-binding declaration vocabulary." + "sha256": "b1474cfd969bd2ab3af6caba7e6e36f300f59fba5eeaaada08fd2254aef36598", + "role": "Declares SourceFilterClause, MetadataExtractionRule, NamedSourceBinding, ImagePlaneSource, SourceBindingsConfig, StepSourceBindingsConfig and the compiled source universe and binding plans." }, { - "path": "openhcs/core/source_binding_workspace.py", - "sha256": "7dce6e45b9e846713b246561340d43de378e251e35f6ef68d5527fccc1b31ad3", - "role": "Compiled virtual source workspace and binding projection." + "path": "openhcs/core/source_binding_selection.py", + "sha256": "f97a4475aa26e465679b89ff9a1fe96690ab7631649d71650608fc7a8aa4bf5a", + "role": "SourceUniverseRequest family that selects source universes." }, { - "path": "openhcs/core/pipeline/path_planner.py", - "sha256": "0ed8bb8d843ff41382f4de25367a74c9535a39425c6fc328091ad3025d526151", - "role": "Compiled source satisfaction, artifact edge, path, grouping, and plan decisions." + "path": "openhcs/core/source_matching.py", + "sha256": "c608880341d5c40d745ebf4254d8a2931ae611c354a6fe5bb5b3d18611679c63", + "role": "SourceFilterMatcher and SourceFilterTargetResolver matching families." }, { - "path": "external/PolyStore/src/polystore/filemanager.py", - "sha256": "877a2b10731b12904b1784a5b42c9e39688e3a84f7f86018c1c2beee96ea2296", - "role": "Generic FileManager storage and backend composition." + "path": "openhcs/core/source_workspace_projection.py", + "sha256": "d833421b963550aa5fe428c68c0286216437aec97834e4204b0b88def011f98b", + "role": "VirtualWorkspaceSourceProjection mapping virtual paths to SourcePixelRef and source metadata." + }, + { + "path": "openhcs/core/source_binding_workspace.py", + "sha256": "4d28066a9f73b283fc637662e05fb2632c6fa18af164476c2899fce58cd647fe", + "role": "Compiled virtual source workspace and binding projection." + }, + { + "path": "external/PolyStore/src/polystore/virtual_workspace.py", + "sha256": "b19fcda85369d812e70bfc1a0efc5d8861ae256f69e94372e2e9515c591d36af", + "role": "PolyStore declaration of SourcePixelRef and generic virtual-workspace addressing." } ], "findings": [ - "The how-to did not require proof that a raw component selector may differ from the projected biological component identity." + "The strategy-selection section asked authors to declare an enum/type/context key on an unnamed strategy root; source families are now AutoRegisterMeta subclass families such as SourceUniverseRequest and SourceFilterMatcher, and the page names them and asks for the family's registry key.", + "The virtual-workspace paragraph said OpenHCS owns microscope metadata; the projection is built from dataset source metadata, so the page now says that." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/source_binding_extension.rst in full and compared it against the changed authorities: source_bindings.py (runtime-context and path-lookup classes removed), source_binding_workspace.py, pipeline/path_planner.py, PolyStore filemanager.py; confirmed every named declaration and compiled plan class still exists, rewrote the strategy-selection paragraph around the current SourceUniverseRequest / SourceFilterMatcher families, changed microscope metadata to dataset source metadata, and moved the PolyStore authority to virtual_workspace.py where SourcePixelRef is declared; path_planner.py and filemanager.py were dropped as the page states no fact they own.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", "tests/unit/test_source_bindings.py and tests/unit/test_source_projection.py passed; source inspection confirmed both compiled source plan types", "On 2026-08-22, re-read the complete how-to against the current source-binding and workspace owners; the focused OpenHCS documentation-drift suite passed 238 tests.", diff --git a/docs/audits/project_surface.json b/docs/audits/project_surface.json index ab29a11a7f..ea484ace6d 100644 --- a/docs/audits/project_surface.json +++ b/docs/audits/project_surface.json @@ -1,7 +1,7 @@ [ { "path": "README.md", - "source_sha256": "1a600010deb7ddc3fe141b5efe3528bbe4a4e119846157e935e56784628445ed", + "source_sha256": "6eda2fe873aaecfb28e60fa4a7c3ba4f6a33bf3f19d79961149b735d21572b7b", "audience": [ "imaging scientists evaluating OpenHCS", "pipeline authors", @@ -12,8 +12,8 @@ "authority": [ { "path": "pyproject.toml", - "sha256": "6217b5f029796edcd83aa47fb8632d37aa7552344755c578fa4f181a6b23282d", - "role": "Declares the published README, Python range, package dependencies, optional surfaces, console commands, and project links." + "sha256": "3592cc37444e2e1f5bb0284e939643039c0d513217861af8133dd8bd6e8a4325", + "role": "Declares the published README, Python range, package dependencies, optional extras, console commands, and project links." }, { "path": ".gitmodules", @@ -21,93 +21,107 @@ "role": "Declares the eight extracted package checkouts composed by the source repository." }, { - "path": "openhcs/desktop_installation.py", - "sha256": "244ade0711356f68df9f376e2517dfe1c9fb77cc960c517f7c4e8ee418b5711f", - "role": "Nominally declares the CPU-safe desktop extras and native installation policy without copying project or brand declarations." + "path": "openhcs/desktop/installation.py", + "sha256": "b2124aaa3e4ef4a310f91a51c1879de41de1818b25fc619cc7cdad888c4202b6", + "role": "Declares the CPU-safe desktop package extras selected by the graphical installers." }, { "path": "openhcs/core/pipeline_document.py", - "sha256": "0849800b3e6474b8176fe35bc4aaebc63cfa85bd77251000c50da5f4c928b1b2", + "sha256": "8c554c8275c665f308eb098955bda517aa804eeb34418b5e0f41d412650ce7b5", "role": "Owns the validated Python pipeline document shared by visual, code, and agent authoring." }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "ace8ae0c3e54bbe604b97cec7c3973a1abcf90e1dc517c2adda947dc2c35e882", - "role": "Owns compile-time resolution across selected axes and construction of the execution bundle." + "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "role": "Owns compile-time resolution across selected axes, compile progress emission, and construction of the execution bundle." + }, + { + "path": "openhcs/core/pipeline/compilation_session.py", + "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", + "role": "Declares ResolvedPipelineDefinition and CompilationSession named in the compilation diagram." + }, + { + "path": "openhcs/core/progress/types.py", + "sha256": "beabbfeb8a6ed727b4bb09f6155de60c4e8b89e71e7ed053bb85574aa1675419", + "role": "Declares set_progress_queue, which compilation requires before it emits progress events." }, { "path": "openhcs/core/artifacts.py", - "sha256": "bb958c74c98cb96c97c5ac1ea68ac997673aac6afcad57bdf0c9496065e895fa", + "sha256": "65eb0ec5c903366243ca8bd5ea07412e8d34bdb81e6015fa76b3c4c59838733f", "role": "Declares image, object, measurement, relationship, table, and file artifact semantics used by materialization." }, { - "path": "openhcs/constants/constants.py", - "sha256": "4cd6c29abd8b8e3ecc396b424d2b8b61e819a215fc1d770d3f6852b260a0691d", - "role": "Declares supported microscope source identities, processing axes, memory types, and file formats." + "path": "openhcs/core/dataset_sources/source.py", + "sha256": "097fde4d2b74031eacf53a88e79f499417e8c1c97ba0753e8a9df831b307efa6", + "role": "Declares the auto-registered DatasetSource family that the ImageXpress, Opera Phenix, Bio-Formats, OMERO and OpenHCS-format sources join." }, { "path": "openhcs/processing/backends/lib_registry/unified_registry.py", - "sha256": "70462cddcad1bfee447ce0786a9e4a85abbd297726d59a1208156af0ef3a3743", - "role": "Owns declaration-based callable discovery, execution memory contracts, dimensional execution, and registry caching." + "sha256": "b1e9253021b9c3ab14a070b7d0f10909f641ba07784c794ad95d2ce36041b1d3", + "role": "Owns declaration-based callable discovery, execution memory contracts, processing contracts, and registry caching." }, { "path": "openhcs/core/viewer_streaming_service.py", - "sha256": "46982fd46f47b962033e02d82afb6c3ce2abbafdabfbce99e591d9c6dff2fbab", + "sha256": "c96eca530f767002cfcc9aca4fe75c93d7a5848a7f42df9e5d336bf6712ed51e", "role": "Owns manual and orchestrated image and ROI streaming through registered viewer declarations." }, { "path": "openhcs/agent/capabilities.py", - "sha256": "7fe9ab91490dedd6f750a57386e91835cf1067a27d10079966510aa9821adaf0", - "role": "Declares the MCP discovery, authoring, execution, UI, and viewer-review capabilities and their surface policies." + "sha256": "eaebe33af01d057d1d21b3701f958bc96ede3ec5d0bddddca9917c1164ef0042", + "role": "Declares the MCP capability profiles, including the full profile and governed custom-function registration." }, { "path": "openhcs/mcp/client_registration.py", - "sha256": "28459b95d3b155778ce178f36302cb830958b74f2603980641ec492d81c41a9a", - "role": "Declares supported local MCP client targets and stable-launcher registration behaviour." + "sha256": "55b701bb77a3de19243fa9a6276f784fad8329e2d435fb26fcc7fdce1e060731", + "role": "Declares supported local MCP client targets such as ChatGPT desktop/Codex and Claude Desktop." }, { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "b547c838b2c86fe7cc5fe12d6cbf6a6715e83055c2d8c1a109ac22566d22c51e", - "role": "Owns lowering CellProfiler cppipe modules into ordinary OpenHCS pipeline declarations." + "sha256": "3439d96f3fbfcb104a53019dc150049021d9a8132bd27096f1ccf510eefbe37b", + "role": "Declares import_cellprofiler_pipeline and its returned FunctionStep list and PipelineConfig." }, { "path": "tests/integration/test_cellprofiler_official30_zmq.py", - "sha256": "bc5abfddf076996a050b0601c46b6039d2d812ab57cd303847b241e477176f53", + "sha256": "dbd6e113a7135901f936e9bc8278d727d678e9971a7c951fced9080adfd9d6df", "role": "Exercises all 30 declared CellProfiler comparison cases over ZMQ and validates live Napari and Fiji projections." }, { "path": "benchmark/cellprofiler_benchmark_cli.py", - "sha256": "2b92755c556c1d66d28ffa0af6e2ac61cac5af4631514382290a2af0f907c26b", - "role": "Owns the installed registry-derived benchmark execution and report command surface." + "sha256": "4025c591644e29f3df61e476ecae2143690e014336ea9f889aadeff4e692445e", + "role": "Declares the openhcs-benchmark subcommands and options, including run-measured through the OpenHCS session." }, { - "path": "benchmark/contracts/run_receipt.py", - "sha256": "1704e1ab47411d25d382b95871c96c1e60c94db14b8ffd55f4dc6c8a917758b9", - "role": "Owns typed run provenance, lifecycle status, exact rerun arguments, and receipt validation." + "path": "benchmark/mcp_extension.py", + "sha256": "056ab5834fa3c37df1a36a37d80f2a821fb4b59c39892d7737e62fa6a57cb53b", + "role": "Declares the expert benchmark MCP tool names, including the session-backed measured-run finalizer." }, { - "path": "benchmark/contracts/control.py", - "sha256": "0bce24c29ea9dfb7b50c88937b616118078505db6f497312fff87766397e5ce9", - "role": "Declares the typed benchmark inspection request and result exposed through MCP." + "path": "benchmark/contracts/run_receipt.py", + "sha256": "4cd61bd33f8f6199b3cc4130ed091dddde9c3a23975e132b1bee06c3c6a43cd7", + "role": "Owns typed run provenance, lifecycle status, exact rerun arguments, and receipt validation." }, { "path": "benchmark/control.py", - "sha256": "4706078a8fbeb516a52bc51600751ae86c39c3ef7e397a2f8cf13d49204db941", + "sha256": "83fe9016625d7d8721905b8a153a28c3dcaaf2b9e925a3c97711c634868156b7", "role": "Owns read-only inspection of receipts, observation progress, and structured artifacts." }, { "path": "tests/unit/test_benchmark_control.py", - "sha256": "ee75209799216c90cfa3a585dd9b7af99c99405f93e9fb5e4f900ea26d9d3ea0", + "sha256": "093b584eeb37cbb12089d5af082ad45acd779e9d2bdd5b3feaea08234fc22183", "role": "Proves the installed command derivation, typed receipts, MCP projection, path policy, and inspect-only import boundary." } ], "findings": [ - "The agent-workflow capability still claimed that the removed embedded UI assistant could use Ollama or remote LLM endpoints.", - "The root README is the package-index long description but was outside the editorial drift gate that covered active RTD pages.", - "The package overview mentioned Official30 evidence but did not identify the installed benchmark command, typed run receipt, or the expert MCP inspect-only boundary." + "The Python example compiled without installing a progress queue; compilation now emits typed progress events and raises ProgressError when none is installed.", + "The run-measured paragraph said the command submits through ordinary headless job control; it now runs through the OpenHCS session and mirrors the never-written source plate into an execution workspace (EVIDENCE/workspace or --workspace-root).", + "The finalizer paragraph described an ordinary headless job and server completion record; the finalizer now takes the submission and server record from the OpenHCS session.", + "The metaclass-registry row and source table named microscope handlers; sources are now the DatasetSource family.", + "Authority openhcs/desktop_installation.py was deleted (moved to openhcs/desktop/installation.py), and openhcs/constants/constants.py no longer declares microscope sources or axes.", + "The compile-time validation card and the compilation diagram named step snapshots/StepSnapshot, which were deleted; compilation resolves a ResolvedPipelineDefinition and runs per-context CompilationSession objects." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read README.md in full and compared it against the changed authorities: pyproject.toml, openhcs/desktop_installation.py (moved to openhcs/desktop/installation.py), openhcs/core/pipeline_document.py, openhcs/core/pipeline/compiler.py, openhcs/core/artifacts.py, openhcs/constants/constants.py, unified_registry.py, viewer_streaming_service.py, openhcs/agent/capabilities.py, openhcs/mcp/client_registration.py, pipeline_import.py, the Official30 integration test and the benchmark CLI/receipt/control modules; installed a progress queue in the CellProfiler example and renamed its root to dataset_root, rewrote the run-measured and finalizer paragraphs for the session-backed run, replaced microscope-handler wording with dataset sources, replaced the deleted StepSnapshot with ResolvedPipelineDefinition in the validation card and compilation diagram, and confirmed extras, console scripts, benchmark subcommands/options, MCP tool names, the full profile, client targets, and the eight submodules against current source.", "Re-read README.md in full and challenged its explanation role, audience, capability claims, installation routes, package boundaries, flow, and links against the hash-pinned declaration owners.", "Removed the obsolete embedded-assistant claim while preserving the MCP-first agent workflow.", "Passed 12 focused documentation-audit tests, including missing, changed, and Python-block-validated project README regressions.", diff --git a/docs/audits/user_docs.json b/docs/audits/user_docs.json index 7a7ea449d9..a6a279080c 100644 --- a/docs/audits/user_docs.json +++ b/docs/audits/user_docs.json @@ -1,7 +1,7 @@ [ { "path": "docs/source/api/index.rst", - "source_sha256": "68401481faeeb4745afa97a50248df8533cfdb6ca0aa09a94557bae2cc4ef57c", + "source_sha256": "0d2ac4119c086b8fde3f738f0937145feba52c6a0c8d18b1fbce7f1f51012fa2", "audience": [ "Python integrators", "application developers" @@ -11,45 +11,52 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "ef81b50ee22b8da69513fd50d6e45a929117fa4bfaa62a20ef08a4c02531c043", - "role": "Declares the supported OpenHCS configuration dataclasses, nested scopes, defaults, and field help metadata." + "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "role": "Declares PipelineConfig, GlobalPipelineConfig, ProcessingConfig and the lazy processing configuration used by the step example." }, { "path": "openhcs/core/steps/function_step.py", "sha256": "8e1243404095691030e9ebb95bb5611679f96c66c6b6819ae71e4b41249e74bc", "role": "Declares FunctionStep and its callable, chain, and dictionary composition forms." }, + { + "path": "openhcs/domains/microscopy/axes.py", + "sha256": "531695809259324032e0b88632813323b4fe6a9a176ad6f636aeef8dec332399", + "role": "Declares the microscopy axis family members such as Microscopy.Site used as variable components." + }, { "path": "openhcs/core/orchestrator/orchestrator.py", - "sha256": "843e994d7b16592f43a15ce36c33a51ead534cf22df792838ce2b800621114b7", - "role": "Owns execution orchestration, worker submission, progress handling, and cleanup of compiled bundles." + "sha256": "a61658b107f170c192eea505966c268ab6826826f03586bd64562a44800e1ccc", + "role": "Declares the PipelineOrchestrator constructor, compile_pipelines, progress_context and execute_compiled_bundle signatures." }, { - "path": "openhcs/core/compiled_execution.py", - "sha256": "941115f2dca2e53538d3141a59f3ae1f928a87cf741fcfa177f7cd64a5f41aea", - "role": "Declares the frozen compiled execution bundle consumed by workers after compilation." + "path": "openhcs/core/progress/types.py", + "sha256": "beabbfeb8a6ed727b4bb09f6155de60c4e8b89e71e7ed053bb85574aa1675419", + "role": "Declares ProgressQueue, set_progress_queue and ProgressExecutionContext, which compilation and execution emit through." }, { - "path": "tests/unit/test_compiled_execution.py", - "sha256": "a7ef4c3f011e0770d8e7a48928aa25c6ca550095c1f80146a25170d449ecc31f", - "role": "Proves compilation produces immutable step plans and execution bundles with the expected resolved values." + "path": "openhcs/core/compiled_execution.py", + "sha256": "85972d418f0958e59f3185f07caf25a103bc9e101e481b2b63903377cc05e397", + "role": "Declares the CompiledExecutionBundle returned by compilation." }, { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "b547c838b2c86fe7cc5fe12d6cbf6a6715e83055c2d8c1a109ac22566d22c51e", - "role": "Owns the supported CellProfiler pipeline import boundary demonstrated by this API orientation." + "sha256": "3439d96f3fbfcb104a53019dc150049021d9a8132bd27096f1ccf510eefbe37b", + "role": "Declares import_cellprofiler_pipeline and its returned steps and PipelineConfig." }, { "path": "tests/unit/test_cellprofiler_pipeline_import.py", - "sha256": "353f0c1d8e0bc3552d2e418d792794ab0631daa16c78a3e10191b1cc1ea07d73", + "sha256": "6b1d201ec5c4ba620fa2367a9cbef1f6fbdd25edbdda9bd7b723ebe3a3363412", "role": "Proves direct import rejects modules requiring interactive desktop input instead of omitting them." } ], "findings": [ - "The API reference did not state that CellProfiler modules requiring interactive desktop input are rejected instead of being silently omitted from the imported pipeline." + "The execution example called the removed PipelineOrchestrator.execute_compiled_plate with a hand-built progress dict keyed by plate_id; execution is now execute_compiled_bundle with a typed ProgressExecutionContext from orchestrator.progress_context().", + "Compilation emits progress events and fails without a queue installed by set_progress_queue; the example compiled before creating any queue." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/api/index.rst in full and compared it against the changed authorities: openhcs/core/config.py, openhcs/core/orchestrator/orchestrator.py, openhcs/core/compiled_execution.py, tests/unit/test_compiled_execution.py, openhcs/interop/cellprofiler/pipeline_import.py, tests/unit/test_cellprofiler_pipeline_import.py; rewrote the compile/execute example to install a progress queue, call execute_compiled_bundle with orchestrator.progress_context(), and name the constructor argument dataset_root, matching tests/unit/remote_sensing_witness_pipeline.py; confirmed the FunctionStep/ProcessingConfig example, the import_cellprofiler_pipeline signature and the interactive-module rejection against current source.", "Re-read docs/source/api/index.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/config.py; openhcs/core/steps/function_step.py; openhcs/core/orchestrator/orchestrator.py; openhcs/core/compiled_execution.py; tests/unit/test_compiled_execution.py; openhcs/interop/cellprofiler/pipeline_import.py; tests/unit/test_cellprofiler_pipeline_import.py.", "Focused cross-lattice QA passed 179 tests spanning callable contracts, framework-device scope, custom functions, catalogue discovery, installer/update policy, MCP release, and packaged OMERO deployment.", @@ -63,7 +70,7 @@ }, { "path": "docs/source/appendices/glossary.rst", - "source_sha256": "0d2911da50a8546e893ffbf0091415ae995700bef3f271170cd8eafb7abffb85", + "source_sha256": "09220b13b10a0ec98237a505e09077d6c41ae85e7fa5ad4b57a61673867008ee", "audience": [ "OpenHCS users", "contributors" @@ -73,43 +80,63 @@ "authority": [ { "path": "openhcs/core/artifacts.py", - "sha256": "bb958c74c98cb96c97c5ac1ea68ac997673aac6afcad57bdf0c9496065e895fa", + "sha256": "65eb0ec5c903366243ca8bd5ea07412e8d34bdb81e6015fa76b3c4c59838733f", "role": "Declares the nominal artefact types exchanged by compiled steps and runtime consumers." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "6d46571e6f2d069b98a92b41d711b2e20bfab9f0493cf73d3f150d5be08e7f96", - "role": "Declares typed per-step memory, conversion, framework-device assignment, and runtime decisions." + "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", + "role": "Declares the per-step compiled plan read by runtime code." + }, + { + "path": "openhcs/core/compiled_execution.py", + "sha256": "85972d418f0958e59f3185f07caf25a103bc9e101e481b2b63903377cc05e397", + "role": "Declares CompiledExecutionBundle, the worker-safe compiled contexts consumed by bundle execution." }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "3c840a89baccc8665d415175458280fd904c1a837b7fd879e303d7dbca129a1c", - "role": "Owns compilation-scoped state and resolves declarations into frozen plans without leaking runtime values." + "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", + "role": "Owns compilation-scoped source, artifact and plan state." }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "5f323ec8edd4221ee30e34ed0c9c09e0f4e45ef502749c1a2d331b7a5452af99", + "sha256": "b6a7231bc2964d472070731018f6a4935e5f23cf8f238a232045c32244e6545a", "role": "Implements scoped typed runtime-value storage and lookup for compiled execution." }, { "path": "openhcs/core/source_bindings.py", - "sha256": "e0afbbe40cce3847bcc2314910d0083ecc579a5a0d6e4278c3a5df4c62e7eb01", - "role": "Declares source bindings that map logical pipeline inputs to microscope-backed workspaces." + "sha256": "b1474cfd969bd2ab3af6caba7e6e36f300f59fba5eeaaada08fd2254aef36598", + "role": "Declares source bindings that connect callable inputs to sources or artifact plans." }, { "path": "openhcs/core/callable_contract.py", - "sha256": "6a39b5e171901131ecb7280a78241e8ea72409a5003eabba3db34eb4851991d7", - "role": "Declares the compiler-visible callable metadata and contract terms defined by the glossary." + "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", + "role": "Declares the compiler-visible callable metadata defined by the glossary." }, { - "path": "openhcs/processing/backends/lib_registry/unified_registry.py", - "sha256": "70462cddcad1bfee447ce0786a9e4a85abbd297726d59a1208156af0ef3a3743", - "role": "Declares the nominal processing-contract members defined by the glossary." + "path": "openhcs/core/processing_contracts.py", + "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", + "role": "Declares the ProcessingContract family with Pure2DContract and Pure3DContract." + }, + { + "path": "openhcs/core/axes.py", + "sha256": "8d93133639c576050f089fb8aab99937c8982aff07e742fea44f8a12e62bb7b2", + "role": "Declares the PartitionAxis role and the axis family that names the partition axis." + }, + { + "path": "openhcs/core/dataset_sources/source.py", + "sha256": "097fde4d2b74031eacf53a88e79f499417e8c1c97ba0753e8a9df831b307efa6", + "role": "Declares DatasetSource with detection, parsing, metadata, virtual-workspace and backend behaviour." } ], - "findings": [], - "disposition": "keep", + "findings": [ + "The Processing contract entry named PURE_2D/PURE_3D enum members; contracts are now classes of the ProcessingContract family (Pure2DContract, Pure3DContract).", + "The Microscope handler entry and the Source workspace projection entry named the removed microscope-handler concept; the registered owner is now DatasetSource.", + "The Execution axis entry and the Compiled step plan entry used the old runtime vocabulary; the axis family now names a PartitionAxis role, and compiled bundles are consumed by bundle execution rather than plate execution." + ], + "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/appendices/glossary.rst in full and compared it against the changed authorities: openhcs/core/artifacts.py, compiled_step_plan.py, pipeline/compilation_session.py, runtime_stores.py, source_bindings.py, callable_contract.py, openhcs/processing/backends/lib_registry/unified_registry.py, plus the G1/G4/G5/G7 owners; rewrote the Processing contract, Dataset source (was Microscope handler), Partition axis (was Execution axis), Compiled step plan, Compiled execution bundle and Source workspace projection entries, and confirmed the remaining terms (FunctionStep, CompiledExecutionBundle, SourceUniverse, RuntimeAdapterSpec, CellProfilerModuleArtifactContracts, variable_components, group_by) against current source.", "On 2026-08-27, re-read docs/source/appendices/glossary.rst in full against callable-owned form exclusions and registry-owned non-authorable controls; challenged its reference definitions for accuracy, completeness, owner boundaries, and navigation, and found no prose drift.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/artifacts.py; openhcs/core/compiled_step_plan.py; openhcs/core/pipeline/compilation_session.py; openhcs/core/runtime_stores.py; openhcs/core/source_bindings.py; openhcs/core/callable_contract.py; openhcs/processing/backends/lib_registry/unified_registry.py.", "Focused cross-lattice QA passed 179 tests spanning callable contracts, framework-device scope, custom functions, catalogue discovery, installer/update policy, MCP release, and packaged OMERO deployment.", @@ -133,19 +160,22 @@ "diataxis": "reference", "authority": [ { - "path": "openhcs/core/artifacts.py", - "sha256": "bb958c74c98cb96c97c5ac1ea68ac997673aac6afcad57bdf0c9496065e895fa", - "role": "Declares the nominal artefact types exchanged by compiled steps and runtime consumers." + "path": ".readthedocs.yaml", + "sha256": "bd5346a882747235c8202c2de4ecfc8c9902accc1cf1534a55d1d1c7a93c373c", + "role": "Declares the Read the Docs build that publishes the appendix navigation from docs/source/conf.py." }, { "path": "docs/source/conf.py", "sha256": "37c6c130d75662601d1e0fece5e0f206b7bf366ae04b596ae39aa9f691ad1c94", - "role": "Registers the active Sphinx extensions and documentation build configuration that expose the appendix navigation." + "role": "Registers the Sphinx configuration whose toctree resolution exposes the glossary entry." } ], - "findings": [], + "findings": [ + "The record cited openhcs/core/artifacts.py, which states no fact on this navigation-only page; replaced it with the Read the Docs build declaration." + ], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/appendices/index.rst in full and compared it against the changed authorities: openhcs/core/artifacts.py; the page is a single-entry toctree to the glossary and makes no artefact claim, so the text stays and the artefact module was dropped from its authorities in favour of .readthedocs.yaml.", "Reviewed docs/source/appendices/index.rst in full; the recorded findings and disposition are the resulting page-specific editorial decision.", "Compared its factual claims directly with the hash-pinned files listed in authority: openhcs/core/artifacts.py; docs/source/conf.py.", "Passed `.venv/bin/python scripts/validate_docs.py docs/source/appendices`: documentation validation passed: 3 files, 0 Python blocks.", @@ -164,42 +194,36 @@ "user_need": "Understand why unsupported impact and performance claims are not published as current product facts.", "diataxis": "explanation", "authority": [ - { - "path": "benchmark/contracts/pipeline.py", - "sha256": "54857ee11404bdc510478e53873b359d05f4a1a103d915626c27acfb4884946c", - "role": "Declares the benchmark pipeline specification used to represent reproducible workload definitions." - }, { "path": "pyproject.toml", - "sha256": "6217b5f029796edcd83aa47fb8632d37aa7552344755c578fa4f181a6b23282d", - "role": "Declares the package version, Python requirement, dependency extras, scripts, and distribution metadata." + "sha256": "3592cc37444e2e1f5bb0284e939643039c0d513217861af8133dd8bd6e8a4325", + "role": "Declares the packaged openhcs-benchmark console script entry point." }, { "path": "benchmark/cellprofiler_benchmark_cli.py", - "sha256": "2b92755c556c1d66d28ffa0af6e2ac61cac5af4631514382290a2af0f907c26b", + "sha256": "4025c591644e29f3df61e476ecae2143690e014336ea9f889aadeff4e692445e", "role": "Owns installed benchmark execution and report commands." }, { "path": "benchmark/contracts/run_receipt.py", - "sha256": "1704e1ab47411d25d382b95871c96c1e60c94db14b8ffd55f4dc6c8a917758b9", + "sha256": "4cd61bd33f8f6199b3cc4130ed091dddde9c3a23975e132b1bee06c3c6a43cd7", "role": "Owns typed comparison-run provenance and lifecycle receipts." }, { "path": "benchmark/control.py", - "sha256": "4706078a8fbeb516a52bc51600751ae86c39c3ef7e397a2f8cf13d49204db941", - "role": "Owns read-only progress and structured-artifact inspection." + "sha256": "83fe9016625d7d8721905b8a153a28c3dcaaf2b9e925a3c97711c634868156b7", + "role": "Owns read-only measured-run inspection and bounded report rendering." }, { "path": "tests/unit/test_benchmark_control.py", - "sha256": "ee75209799216c90cfa3a585dd9b7af99c99405f93e9fb5e4f900ea26d9d3ea0", + "sha256": "093b584eeb37cbb12089d5af082ad45acd779e9d2bdd5b3feaea08234fc22183", "role": "Proves receipt validation and the inspect-only MCP projection." } ], - "findings": [ - "The retired-impact notice did not distinguish reproducibility receipts and read-only run inspection from scientific comparison proof or execution authority." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/appendices/research_impact.rst in full and compared it against the changed authorities: pyproject.toml, benchmark/cellprofiler_benchmark_cli.py, benchmark/contracts/run_receipt.py, benchmark/control.py, tests/unit/test_benchmark_control.py; the CLI now runs through the OpenHCS session and receipts name a subject_id, but the openhcs-benchmark entry point, typed receipts, read-only inspect/report service and the archive path the page names all still exist, so no claim changed. Dropped benchmark/contracts/pipeline.py, which the page does not rely on.", "On 2026-08-27, re-read docs/source/appendices/research_impact.rst in full after the python-introspect dependency floor advanced to 0.1.12; challenged its explanation boundary and confirmed that retiring unsupported impact claims remains accurate.", "Compared every factual claim directly with the current hash-pinned authorities: benchmark/contracts/pipeline.py; pyproject.toml.", "Focused cross-lattice QA passed 179 tests spanning callable contracts, framework-device scope, custom functions, catalogue discovery, installer/update policy, MCP release, and packaged OMERO deployment.", @@ -228,7 +252,7 @@ }, { "path": "docs/source/concepts/building_intuition.rst", - "source_sha256": "f4eb04f78f1e52241fa1b7f79b39609142f106832a0bf38b953a170592cf8f78", + "source_sha256": "8827a6443b6df28347af61ad301226dbcf3396fbf45c35a8127c086f15185ce9", "audience": [ "pipeline authors", "new contributors" @@ -238,35 +262,42 @@ "authority": [ { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "ace8ae0c3e54bbe604b97cec7c3973a1abcf90e1dc517c2adda947dc2c35e882", + "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", "role": "Compiles pipeline declarations into validated step plans and execution bundles." }, { "path": "openhcs/core/source_bindings.py", - "sha256": "e0afbbe40cce3847bcc2314910d0083ecc579a5a0d6e4278c3a5df4c62e7eb01", - "role": "Declares source bindings that map logical pipeline inputs to microscope-backed workspaces." + "sha256": "b1474cfd969bd2ab3af6caba7e6e36f300f59fba5eeaaada08fd2254aef36598", + "role": "Source-binding declarations that name pipeline inputs from files and metadata." }, { - "path": "openhcs/core/artifacts.py", - "sha256": "bb958c74c98cb96c97c5ac1ea68ac997673aac6afcad57bdf0c9496065e895fa", - "role": "Declares the nominal artefact types exchanged by compiled steps and runtime consumers." + "path": "openhcs/core/dataset_sources/source.py", + "sha256": "097fde4d2b74031eacf53a88e79f499417e8c1c97ba0753e8a9df831b307efa6", + "role": "DatasetSource family that owns recognized dataset layouts." }, { - "path": "openhcs/processing/backends/lib_registry/unified_registry.py", - "sha256": "70462cddcad1bfee447ce0786a9e4a85abbd297726d59a1208156af0ef3a3743", - "role": "Defines canonical callable metadata and derives the unified catalogue from registry declarations." + "path": "openhcs/core/processing_contracts.py", + "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", + "role": "ProcessingContract family deciding per-plane versus whole-stack execution." + }, + { + "path": "openhcs/core/artifacts.py", + "sha256": "65eb0ec5c903366243ca8bd5ea07412e8d34bdb81e6015fa76b3c4c59838733f", + "role": "Nominal artifact types exchanged by compiled steps and runtime consumers." }, { "path": "external/arraybridge/README.md", - "sha256": "b108a144e2f797425a1734f745a73ac7a215bd5e58b3f6fe4342b18e7d16b892", - "role": "States ArrayBridge ownership of reusable array conversion, callable memory metadata, and framework device mechanics." + "sha256": "2fe1a330ca101dadb6cea5ac6f0482a5ffe8c7e0e4b9bd7bc8bc2220e106415a", + "role": "States ArrayBridge ownership of array conversion, callable memory metadata, and framework device mechanics." } ], "findings": [ - "The owner checklist previously reduced ArrayBridge to conversion and omitted its callable memory metadata and framework-local device mechanics." + "G4 replaced microscope handlers with the DatasetSource family; the owner checklist named 'microscope integration'.", + "Swapped the unified_registry.py authority (the page states nothing about the catalogue) for processing_contracts.py, which owns the ProcessingContract claim." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/concepts/building_intuition.rst in full and compared it against the changed authorities: pipeline/compiler.py, source_bindings.py, artifacts.py, lib_registry/unified_registry.py and external/arraybridge/README.md; confirmed FunctionStep, variable_components, group_by and ProcessingContract exist, that force_disk_output is absent, and renamed the source-integration owner to DatasetSource.", "On 2026-08-27, re-read docs/source/concepts/building_intuition.rst in full against the full-catalogue editor and owner-declared exclusion changes; challenged its explanation flow and ownership model and found no prose drift.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/pipeline/compiler.py; openhcs/core/source_bindings.py; openhcs/core/artifacts.py; openhcs/processing/backends/lib_registry/unified_registry.py; external/arraybridge/README.md.", "Focused cross-lattice QA passed 179 tests spanning callable contracts, framework-device scope, custom functions, catalogue discovery, installer/update policy, MCP release, and packaged OMERO deployment.", @@ -281,35 +312,43 @@ }, { "path": "docs/source/concepts/component_system_concepts.rst", - "source_sha256": "e3816c371c55821e36e5012787776092beee9650ea638f0006673808e3e81539", + "source_sha256": "7bd02c6e5e670bfc02d4e90586a136647c7f6a384226aa7dc7e6edfd872c1be1", "audience": [ "pipeline authors", "compiler contributors" ], - "user_need": "Understand how microscopy component identities differ from grouping and runtime scope.", + "user_need": "Understand how dataset axis identities differ from grouping and runtime scope.", "diataxis": "explanation", "authority": [ { - "path": "openhcs/constants/constants.py", - "sha256": "4cd6c29abd8b8e3ecc396b424d2b8b61e819a215fc1d770d3f6852b260a0691d", - "role": "Declares the nominal microscopy dimensions, grouping identities, and shared component enums used by pipelines." + "path": "openhcs/core/axes.py", + "sha256": "8d93133639c576050f089fb8aab99937c8982aff07e742fea44f8a12e62bb7b2", + "role": "Declares Axis, AxisFamily, the axis roles, Ungrouped, AxisFamily.active() and with_role()." + }, + { + "path": "openhcs/domains/microscopy/axes.py", + "sha256": "531695809259324032e0b88632813323b4fe6a9a176ad6f636aeef8dec332399", + "role": "Declares the Microscopy family with Site, Channel, ZIndex, Timepoint and the well partition axis." }, { "path": "openhcs/core/component_group_scope.py", - "sha256": "16e7f8f05223f7f9d65d1b96ac0418899f82710389768fa0e063d700e368f409", - "role": "Declares how component groups map nominal dimensions to execution scopes." + "sha256": "9fc6006db14e4533ca0b45204a529ab23bac8c69382c8f1f1f883fa87179cd3c", + "role": "Declares ComponentGroupScope and RuntimeExecutionAxisScope." }, { - "path": "tests/unit/test_configuration_mirror_deletion.py", - "sha256": "26002ddfbbf89bf06ae54888645c6b8c90154536a16e93feee1a2fae713c4963", - "role": "Proves configuration consumers use declaration-owned metadata without parallel field registries." + "path": "tests/unit/test_axis_family_guards.py", + "sha256": "a81f12f9057e2ed7393e6daeb30f1c086f5c867222289a2bb4af3cf297b59c5c", + "role": "Proves the component enums are deleted and kernel modules name no microscopy member." } ], "findings": [ - "The previous audit named an unused dynamic-interface scaffold as the component-enum authority; the actual declaration owner is openhcs/constants/constants.py." + "The authority record still named openhcs/constants/constants.py, which no longer declares any axis or component identity after G1; the owners are openhcs/core/axes.py and openhcs/domains/microscopy/axes.py.", + "\"Site declares the stack axis\" read as if Site carried the StackAxis role, which belongs to ZIndex; reworded to say Site is the axis the step assembles its stacks along.", + "tests/unit/test_configuration_mirror_deletion.py does not support this page's claims and was dropped." ], - "disposition": "keep", + "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/concepts/component_system_concepts.rst in full and compared it against the changed authorities: openhcs/constants/constants.py, openhcs/core/component_group_scope.py, tests/unit/test_configuration_mirror_deletion.py; the page had already been rewritten for the G1 axis family, so checked every named class, role, module path and the ProcessingConfig/LazyProcessingConfig.from_config example against openhcs/core/axes.py, openhcs/domains/microscopy/axes.py, openhcs/core/config.py and the ObjectState lazy factory, corrected the Site sentence, and moved the authorities to the axis owners.", "Re-read docs/source/concepts/component_system_concepts.rst in full against repository base 32ca17c17 and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared its factual claims directly with the hash-pinned files listed in authority: openhcs/constants/constants.py; openhcs/core/component_group_scope.py; tests/unit/test_configuration_mirror_deletion.py.", "On 2026-08-25, traced component-enum construction to openhcs/constants/constants.py and removed the unreferenced parallel dynamic-interface registry from the authority record.", @@ -318,7 +357,7 @@ }, { "path": "docs/source/concepts/core_model.rst", - "source_sha256": "abf8f245abf9ca3526a63840d54dbc5f9677604422f027c0d838ff2256800830", + "source_sha256": "7e81dd24da51a195a7e41c056773e84658f8c57ee4fe88f47cb88bc42b465cad", "audience": [ "imaging scientists", "integrators", @@ -329,38 +368,53 @@ "authority": [ { "path": "openhcs/core/pipeline_document.py", - "sha256": "0849800b3e6474b8176fe35bc4aaebc63cfa85bd77251000c50da5f4c928b1b2", - "role": "Declares the public pipeline document, step collection, and serialisable declaration boundary." + "sha256": "8c554c8275c665f308eb098955bda517aa804eeb34418b5e0f41d412650ce7b5", + "role": "PipelineDocument holds a PipelineConfig plus an ordered step list; no Pipeline wrapper class exists." }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "ace8ae0c3e54bbe604b97cec7c3973a1abcf90e1dc517c2adda947dc2c35e882", - "role": "Compiles pipeline declarations into validated step plans and execution bundles." + "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "role": "Creates one CompilationSession per partition-axis context and emits the compiled bundle after validation." + }, + { + "path": "openhcs/core/pipeline/compilation_session.py", + "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", + "role": "Declares CompilationSession with the ResolvedPipelineDefinition, source workspace projection and CompiledStepPlan map." }, { "path": "openhcs/core/compiled_execution.py", - "sha256": "941115f2dca2e53538d3141a59f3ae1f928a87cf741fcfa177f7cd64a5f41aea", - "role": "Declares the frozen compiled execution bundle consumed by workers after compilation." + "sha256": "85972d418f0958e59f3185f07caf25a103bc9e101e481b2b63903377cc05e397", + "role": "Declares CompiledExecutionBundle." }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "5f323ec8edd4221ee30e34ed0c9c09e0f4e45ef502749c1a2d331b7a5452af99", - "role": "Implements scoped typed runtime-value storage and lookup for compiled execution." + "sha256": "b6a7231bc2964d472070731018f6a4935e5f23cf8f238a232045c32244e6545a", + "role": "Declares RuntimeValueStore keyed by typed artifact keys and execution scope." + }, + { + "path": "openhcs/core/callable_contract.py", + "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", + "role": "Declares the CallableContract fields: memory types, required axis roles, allowed group-by roles, processing contract, execution scope, runtime adapter and artifact specs." }, { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "b547c838b2c86fe7cc5fe12d6cbf6a6715e83055c2d8c1a109ac22566d22c51e", - "role": "Imports CellProfiler modules into ordinary OpenHCS pipeline declarations through the interop boundary." + "sha256": "3439d96f3fbfcb104a53019dc150049021d9a8132bd27096f1ccf510eefbe37b", + "role": "Lowers parsed .cppipe modules into ordinary source bindings and FunctionStep declarations." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "6d46571e6f2d069b98a92b41d711b2e20bfab9f0493cf73d3f150d5be08e7f96", - "role": "Declares the typed source, artifact, memory, framework-device, materialisation, and streaming plan fields summarised by the core model." + "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", + "role": "Declares the source, artifact, memory, materialisation, streaming and worker fields of CompiledStepPlan." } ], - "findings": [], - "disposition": "keep", + "findings": [ + "The compilation section named StepSnapshot objects, which were deleted when the compiler moved to compiling from the resolved pipeline definition (commit 0adb68108); the session now carries a ResolvedPipelineDefinition.", + "\"Each execution axis receives a CompilationSession\" misstated the scope: one session is created per partition-axis value (context).", + "Callable contracts now declare required axis roles (required_axis_roles) rather than required axes, after G1." + ], + "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/concepts/core_model.rst in full and compared it against the changed authorities: openhcs/core/pipeline_document.py, openhcs/core/pipeline/compiler.py, openhcs/core/compiled_execution.py, openhcs/core/runtime_stores.py, openhcs/interop/cellprofiler/pipeline_import.py, openhcs/core/compiled_step_plan.py; grepped every named class (CompilationSession, CompiledExecutionBundle, RuntimeValueStore, CellProfilerModule, SourceBindingsConfig, ArtifactSpec, CompiledStepPlan, ProcessingContract), replaced the deleted StepSnapshot, corrected session scope and contract axis wording, and added compilation_session.py and callable_contract.py as owners.", "On 2026-08-27, re-read docs/source/concepts/core_model.rst in full against the explicit CellProfiler import-validation hook; challenged its declaration-to-runtime explanation and confirmed that its supported lowering model remains accurate.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/pipeline_document.py; openhcs/core/pipeline/compiler.py; openhcs/core/compiled_execution.py; openhcs/core/runtime_stores.py; openhcs/interop/cellprofiler/pipeline_import.py; openhcs/core/compiled_step_plan.py.", "Focused cross-lattice QA passed 179 tests spanning callable contracts, framework-device scope, custom functions, catalogue discovery, installer/update policy, MCP release, and packaged OMERO deployment.", @@ -373,7 +427,7 @@ }, { "path": "docs/source/concepts/data_dimensions.rst", - "source_sha256": "09ab2832cfb480cc532867a118850e12242f2fc9951df718f95317bd746ee564", + "source_sha256": "75be1966bb36c44edcd081e1cc2adfa23a929661d6565c04919521e9f2e58037", "audience": [ "pipeline authors", "imaging scientists" @@ -382,61 +436,54 @@ "diataxis": "explanation", "authority": [ { - "path": "openhcs/constants/constants.py", - "sha256": "4cd6c29abd8b8e3ecc396b424d2b8b61e819a215fc1d770d3f6852b260a0691d", - "role": "Declares the nominal microscopy dimensions, grouping identities, and shared component enums used by pipelines." + "path": "openhcs/core/axes.py", + "sha256": "8d93133639c576050f089fb8aab99937c8982aff07e742fea44f8a12e62bb7b2", + "role": "Declares AxisFamily, its single PartitionAxis and the variable axes a step may stack along." }, { - "path": "openhcs/core/variable_component_stack_requirement.py", - "sha256": "0f34c6ecb44cae29266e6c572b80c5393ef588a144e215fa3fe135ea29f737a2", - "role": "Declares which nominal stack components a callable requires and how those requirements are validated." + "path": "openhcs/core/processing_contracts.py", + "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", + "role": "Declares the Pure2DContract, Pure3DContract, FlexibleContract and VolumetricToSliceContract subclasses of ProcessingContract." }, { "path": "openhcs/core/component_group_scope.py", - "sha256": "16e7f8f05223f7f9d65d1b96ac0418899f82710389768fa0e063d700e368f409", - "role": "Declares how component groups map nominal dimensions to execution scopes." - }, - { - "path": "openhcs/processing/backends/lib_registry/unified_registry.py", - "sha256": "70462cddcad1bfee447ce0786a9e4a85abbd297726d59a1208156af0ef3a3743", - "role": "Defines canonical callable metadata and derives the unified catalogue from registry declarations." + "sha256": "9fc6006db14e4533ca0b45204a529ab23bac8c69382c8f1f1f883fa87179cd3c", + "role": "Declares ComponentGroupScope and RuntimeExecutionAxisScope carried by artifact keys." }, { "path": "openhcs/core/runtime_object_label_domains.py", - "sha256": "32824678741c1caf3fbabe621619d6d8d463936a28bc2d689916ed69fa7582ed", - "role": "Declares payload-scoped and plane-scoped object identity domains without inferring them from array shape." + "sha256": "cb56a2b3a391cfd5081e1487a04e2e95b88628879abc42a2ec3b69ab327ab0cf", + "role": "Declares the PAYLOAD and PLANE object-label domain scopes." }, { "path": "openhcs/processing/backends/cellprofiler/watershed.py", - "sha256": "9bd009be9881a88e295e1c5ca6fbc02b4402bc97d207cfd20848ff685fe64bde", - "role": "Declares the full-stack image payload mode used by CellProfiler-compatible volumetric Watershed." + "sha256": "41c8cb9cb139f8b4041a25e1a7a2d19aa85cf0427422fc87b9551c8141069625", + "role": "Declares Watershed with FullStackExecution so it consumes the complete payload." }, { "path": "openhcs/processing/backends/cellprofiler/primary_objects.py", - "sha256": "6710b507fcdead2954e8a1038c91e2a99572d67b6d4ded8a7d41100fb229aa29", - "role": "Declares IdentifyPrimaryObjects as a plane-local processing callable." + "sha256": "b5b8a9b98dd9166b34a8f5cebf884e8af6da85f02fe878ef1227d6656d31ae4c", + "role": "Declares IdentifyPrimaryObjects with Pure2DContract." }, { "path": "openhcs/processing/backends/cellprofiler/secondary.py", - "sha256": "9a032a11e51c3f950538f89f4e4312094496fd6408e0fdceebc7dc8b91be43de", - "role": "Declares IdentifySecondaryObjects and IdentifyTertiaryObjects as plane-local processing callables." - }, - { - "path": "tests/unit/test_callable_contract.py", - "sha256": "493f76a54f5fe024ca8f10c53b1f810b8c369626ad7141205b42ef3a78282137", - "role": "Proves callable contract derivation for dimensions, memory types, dtype policy, and outputs." + "sha256": "fc9d9b1bbc3baff7216c6abe3a7135ffddfc9f7e1212d4a391fa8534038a354c", + "role": "Declares IdentifySecondaryObjects and IdentifyTertiaryObjects with Pure2DContract." }, { "path": "tests/unit/test_documented_dimensionality.py", - "sha256": "83bef8dcad91750c9acda63b8a4293ccbd0904dd5aea7a32be197f9d51f300a7", - "role": "Binds the published dimensionality boundary to the current callable and feature declarations." + "sha256": "45762e74bf5cecbbada509f378ddbfa47c20fa6a9f5bd668d05d003e59bb38e1", + "role": "Binds the published dimensionality boundary to the current callable declarations." } ], "findings": [ - "The previous explanation did not distinguish a Z stack from a payload-scoped volumetric object domain, leaving true 3D support ambiguous." + "The processing-locality list named enum members PURE_2D, PURE_3D, FLEXIBLE and VOLUMETRIC_TO_SLICE; after G5 ProcessingContract is a class family (Pure2DContract, Pure3DContract, FlexibleContract, VolumetricToSliceContract).", + "The page called the well axis the configured multiprocessing execution axis; after G1 the active axis family declares a single partition axis.", + "openhcs/constants/constants.py no longer declares any axis and was replaced by openhcs/core/axes.py; unified_registry.py, variable_component_stack_requirement.py and tests/unit/test_callable_contract.py supported no claim the page makes and were dropped." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/concepts/data_dimensions.rst in full and compared it against the changed authorities: openhcs/constants/constants.py, openhcs/core/component_group_scope.py, openhcs/processing/backends/lib_registry/unified_registry.py, openhcs/core/runtime_object_label_domains.py, the three CellProfiler backends, tests/unit/test_callable_contract.py and tests/unit/test_documented_dimensionality.py; confirmed Watershed still declares FullStackExecution and the Identify modules Pure2DContract, the PAYLOAD/PLANE scopes are unchanged, and corrected the processing-contract names and partition-axis wording.", "On 2026-08-27, re-read docs/source/concepts/data_dimensions.rst in full against registry form exclusions, explicit secondary-backend default selection, and expanded contract tests; challenged its dimensional explanation and found no semantic drift.", "On 2026-08-27, re-read the complete data-dimensions explanation after library registries gained declaration-owned runtime-parameter exclusions; dimensional and runtime-identity claims remain unchanged.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/constants/constants.py; openhcs/core/variable_component_stack_requirement.py; openhcs/core/component_group_scope.py; openhcs/processing/backends/lib_registry/unified_registry.py; openhcs/core/runtime_object_label_domains.py; openhcs/processing/backends/cellprofiler/watershed.py; openhcs/processing/backends/cellprofiler/primary_objects.py; openhcs/processing/backends/cellprofiler/secondary.py; tests/unit/test_callable_contract.py; tests/unit/test_documented_dimensionality.py.", @@ -449,7 +496,7 @@ }, { "path": "docs/source/concepts/domain_fundamentals.rst", - "source_sha256": "9f4a6f7aa270b5814397a0e8227f44064cc6f5e4cf2219609da187ae7557341e", + "source_sha256": "741c1cca9905d47ae0170570bdc473f503d68ec526bc07f309368d6b37f0e185", "audience": [ "imaging scientists new to OpenHCS" ], @@ -457,24 +504,28 @@ "diataxis": "explanation", "authority": [ { - "path": "openhcs/microscopes/microscope_interfaces.py", - "sha256": "a603fa1c58ba005f8ba22713e46efe4e33aee0f762ba188467482ae3ae429200", - "role": "Declares the microscope source interface, plate dimensions, metadata, and acquisition access contract." + "path": "openhcs/core/dataset_sources/source.py", + "sha256": "097fde4d2b74031eacf53a88e79f499417e8c1c97ba0753e8a9df831b307efa6", + "role": "Declares the registered DatasetSource family that projects each microscope format into axis values, backends and metadata." }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "ace8ae0c3e54bbe604b97cec7c3973a1abcf90e1dc517c2adda947dc2c35e882", - "role": "Compiles pipeline declarations into validated step plans and execution bundles." + "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "role": "Resolves inputs, axes, locality, memory conversion and artifact dependencies before emitting compiled plans." }, { "path": "openhcs/core/runtime_artifact_values.py", - "sha256": "9ac62ec4c4a6a6f7f375026ea112cc2e43f09ee6e6ca270a3d955be8ef5347cf", - "role": "Declares the typed runtime values carried for nominal artefact kinds." + "sha256": "930752071d37ef1a8669812e7a629e151690b3f33bf4a29dba92d1d479b9074d", + "role": "Declares ArtifactKey and RuntimeValue, the typed runtime result carrier." } ], - "findings": [], - "disposition": "keep", + "findings": [ + "Authority openhcs/microscopes/microscope_interfaces.py was deleted by G4 (c6d9a52ce); the source interface now lives in openhcs/core/dataset_sources/source.py as the DatasetSource family.", + "The page said \"registered microscope handlers\"; reworded to registered dataset sources, matching the G4 family name." + ], + "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/concepts/domain_fundamentals.rst in full and compared it against the changed authorities: openhcs/microscopes/microscope_interfaces.py (deleted in G4), openhcs/core/pipeline/compiler.py, openhcs/core/runtime_artifact_values.py; replaced the deleted source interface with openhcs/core/dataset_sources/source.py, updated the dataset-source wording, and confirmed the compilation and typed-result claims still hold.", "Re-read docs/source/concepts/domain_fundamentals.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/microscopes/microscope_interfaces.py; openhcs/core/pipeline/compiler.py; openhcs/core/runtime_artifact_values.py.", "Focused cross-lattice QA passed 179 tests spanning callable contracts, framework-device scope, custom functions, catalogue discovery, installer/update policy, MCP release, and packaged OMERO deployment.", @@ -495,28 +546,34 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "ef81b50ee22b8da69513fd50d6e45a929117fa4bfaa62a20ef08a4c02531c043", - "role": "Declares the supported OpenHCS configuration dataclasses, nested scopes, defaults, and field help metadata." + "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "role": "Declares DtypeConfig as a DtypeConversionConfig subclass with NATIVE_OUTPUT as its default_dtype_conversion." + }, + { + "path": "openhcs/core/steps/abstract.py", + "sha256": "63734f0c5198272b08f128ae974ada725c8201cb44efd0ce32500fd280b879bf", + "role": "Declares the step-level dtype_config field separate from processing_config." }, { "path": "openhcs/core/memory/decorators.py", - "sha256": "d9f9cf0be7593dc0ebfc57f258aebed75bcf9cfa508ed1e942137366d819766a", - "role": "Exports OpenHCS memory and dtype decorators while delegating conversion semantics to ArrayBridge declarations." + "sha256": "76f234785f4865257fd19c935e0aace82d1f5586198737fec619ee243b79cbf6", + "role": "Exports the OpenHCS memory decorators that delegate dtype conversion to ArrayBridge." }, { "path": "external/arraybridge/src/arraybridge/decorators.py", - "sha256": "99823105b0ae59054df1d84013633b8b6628d0a6dc3326383199ac7c65c69bed", - "role": "Declares decorators that write independent input, output, and execution memory roles plus dtype policy." + "sha256": "28f9bf4770bd4cc1ef34045fdef28065063ede029296f62b8a6c4140cc88e40e", + "role": "Declares the DtypeConversion members and DtypeConversionConfig as a runtime-bound parameter declaration." }, { "path": "tests/unit/test_callable_contract.py", - "sha256": "493f76a54f5fe024ca8f10c53b1f810b8c369626ad7141205b42ef3a78282137", - "role": "Proves callable contract derivation for dimensions, memory types, dtype policy, and outputs." + "sha256": "cc220f53107a08b0406b6e0dd49896e3e4c3c0af6e527b0ef00d4fdaba8f0b2c", + "role": "Proves dtype_config is the overridable runtime-bound parameter on a decorated callable's contract." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/concepts/dtype_config_system.rst in full and compared it against the changed authorities: openhcs/core/config.py, openhcs/core/memory/decorators.py, external/arraybridge/src/arraybridge/decorators.py, tests/unit/test_callable_contract.py; checked the DtypeConversion members, the NATIVE_OUTPUT default, the import paths in the example, the step-level dtype_config field (now owned by openhcs/core/steps/abstract.py, added as an authority) and the runtime-bound dtype_config contract parameter; no claim changed.", "On 2026-08-27, re-read docs/source/concepts/dtype_config_system.rst in full after callable contracts began projecting runtime-owned inputs into generic form exclusions; challenged the dtype ownership boundary and found no prose drift.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/config.py; openhcs/core/memory/decorators.py; external/arraybridge/src/arraybridge/decorators.py; tests/unit/test_callable_contract.py.", "Focused cross-lattice QA passed 179 tests spanning callable contracts, framework-device scope, custom functions, catalogue discovery, installer/update policy, MCP release, and packaged OMERO deployment.", @@ -529,7 +586,7 @@ }, { "path": "docs/source/concepts/function_library.rst", - "source_sha256": "5f283828046aaa3e7c9e57ff0ef34cd77561031f110619b0d41aa9696047a769", + "source_sha256": "d1cc56563fe60736d196ac23358cc4f4f26ee57572478351bddbae1c17c624d1", "audience": [ "imaging scientists", "callable authors" @@ -538,70 +595,68 @@ "diataxis": "explanation", "authority": [ { - "path": "openhcs/processing/backends/lib_registry/unified_registry.py", - "sha256": "70462cddcad1bfee447ce0786a9e4a85abbd297726d59a1208156af0ef3a3743", - "role": "Defines canonical callable metadata and derives the unified catalogue from registry declarations." + "path": "openhcs/core/callable_contract.py", + "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", + "role": "Declares the CallableContract fields the page lists: memory types, artifact specs, runtime-bound parameters, required_axis_roles, allowed_group_by_roles, processing contract, runtime adapter, execution scope and prepare hook." }, { - "path": "openhcs/processing/backends/lib_registry/registry_service.py", - "sha256": "046d3d088f685a993ddad6b5923cf63beef84b201127b72ae32af6721dd5ad63", - "role": "Owns backend discovery, availability filtering, and the canonical process catalogue projection." + "path": "openhcs/core/processing_contracts.py", + "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", + "role": "Declares the ProcessingContract class family selecting per-slice, full-stack, flexible or volumetric-to-slice execution." }, { - "path": "openhcs/core/callable_contract.py", - "sha256": "6a39b5e171901131ecb7280a78241e8ea72409a5003eabba3db34eb4851991d7", - "role": "Derives callable input, output, execution-framework, dimensional, artifact, and runtime requirements from declarations." + "path": "openhcs/processing/backends/lib_registry/unified_registry.py", + "sha256": "b1e9253021b9c3ab14a070b7d0f10909f641ba07784c794ad95d2ce36041b1d3", + "role": "Derives catalogue metadata and form exclusions from library registry declarations." }, { - "path": "openhcs/agent/services/function_catalog_service.py", - "sha256": "93cfc4431cff9ce182190dfd3c5cb265c532a38c78d2baa39153f972c5f97cc8", - "role": "Projects callable contracts from the unified function registry for agent discovery and selection." + "path": "openhcs/processing/backends/lib_registry/registry_service.py", + "sha256": "daef8925338fddf8504a9faa066af42456a13fe4602976bf64be06547a45fcc8", + "role": "Owns backend discovery, availability filtering and the process catalogue projection." }, { - "path": "tests/unit/test_callable_contract.py", - "sha256": "493f76a54f5fe024ca8f10c53b1f810b8c369626ad7141205b42ef3a78282137", - "role": "Proves callable contract derivation for dimensions, memory types, dtype policy, and outputs." + "path": "openhcs/processing/backends/lib_registry/openhcs_registry.py", + "sha256": "3fff5c3549ef8e1073d75e6d5544b2cd75fbb7cc9b17263207760ffca45cf885", + "role": "Derives the public native callable catalogue and excludes private decorated helpers." }, { - "path": "external/arraybridge/src/arraybridge/types.py", - "sha256": "7b94760d57e54008d7adf4e4d37dca45d8a2c288a9dc8400be00c26148cf7624", - "role": "Owns framework capabilities and the canonical callable memory-metadata keys used by function declarations." + "path": "openhcs/agent/services/function_catalog_service.py", + "sha256": "211c81f3747dcbc5cad2bde36191bf7ac90fab007f44c960c729ac33c2737df1", + "role": "Serves catalogue search and callable detail to MCP and GUI authoring from the execution endpoint." }, { "path": "external/arraybridge/src/arraybridge/decorators.py", - "sha256": "99823105b0ae59054df1d84013633b8b6628d0a6dc3326383199ac7c65c69bed", - "role": "Writes the independent input, output, and execution framework declarations described by the page." + "sha256": "28f9bf4770bd4cc1ef34045fdef28065063ede029296f62b8a6c4140cc88e40e", + "role": "Writes the independent input, output and execution framework declarations." }, { - "path": "openhcs/processing/backends/lib_registry/openhcs_registry.py", - "sha256": "aa3079341cb2af8788938e14efee6f9f6c4b6c01874227e74c6340850d525884", - "role": "Derives the public native callable catalogue while retaining explicit reconstruction from each callable declaration." + "path": "external/arraybridge/src/arraybridge/types.py", + "sha256": "08f7ff4df8807bb3340e00a02c6276b1c702a0f00a2eedacd701142736c5d804", + "role": "Owns the framework capabilities and callable memory-metadata keys." }, { "path": "tests/unit/test_library_registry_discovery.py", - "sha256": "a7ee2d3a2002ea291876c27aa2dce51a889cea6be9a3055602c76f1e9cf76f1b", + "sha256": "e612d181d0053bacb38cb7bd3a73e8af8b3f7cdbe3789c1b7deb8f5f6a655170", "role": "Proves declaration-derived catalogue discovery and process policy filtering." }, { "path": "tests/unit/test_unified_registry_runtime_payloads.py", - "sha256": "e731379d3e48fd5c98709203668e09baa195bb0c9301a2b041e1c799fa8140e3", + "sha256": "76e02cc26b40f5af984d31e874eab1b18d334259fb5cd44bc9fe5945c474b139", "role": "Proves external library runtime controls stay out of authored function parameters." }, { "path": "tests/unit/pyqt_gui/test_function_catalog_form_materialization.py", "sha256": "711b69ff0b145b962f0f3518c7a7a77db38d8a545bb9f5ea7d2e259d70598667", - "role": "Proves every locally available catalogue declaration materialises through the real desktop function form." + "role": "Proves every locally available catalogue declaration materialises through the desktop function form." } ], "findings": [ - "The page did not explain that optional backend visibility requires a usable runtime and complete declared module inventory.", - "The callable-contract explanation omitted the independent execution framework and did not state that catalogue admission checks every declared framework role.", - "The function-pattern explanation did not state the nominal enum-member requirement enforced by callable keyword validation.", - "The function-library explanation did not distinguish public catalogue declarations from private decorated implementation helpers.", - "The explanation did not connect contract-owned exclusions and library-owned non-authorable controls to full-catalogue desktop form materialisation." + "The callable-contract list still said contracts declare required variable components and allowed grouping; after G1 they declare required_axis_roles and allowed_group_by_roles, and after G5 the processing contract is a ProcessingContract subclass rather than an enum member.", + "Replaced the jargon \"admitted\" with \"catalogued\" for native and custom function inclusion." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/concepts/function_library.rst in full and compared it against the changed authorities: unified_registry.py, registry_service.py, openhcs/core/callable_contract.py, function_catalog_service.py, openhcs_registry.py, the ArrayBridge types and decorators, and the three registry/contract tests; the G5 processing-contract move and catalogue-service signature/doc projection changes leave the page's catalogue, exclusion and framework claims intact, corrected the contract field names, and added openhcs/core/processing_contracts.py as an owner.", "Re-read docs/source/concepts/function_library.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/processing/backends/lib_registry/unified_registry.py; openhcs/processing/backends/lib_registry/registry_service.py; openhcs/core/callable_contract.py; openhcs/agent/services/function_catalog_service.py; tests/unit/test_callable_contract.py; external/arraybridge/src/arraybridge/types.py; external/arraybridge/src/arraybridge/decorators.py; openhcs/processing/backends/lib_registry/openhcs_registry.py; tests/unit/test_library_registry_discovery.py; tests/unit/test_unified_registry_runtime_payloads.py; tests/unit/pyqt_gui/test_function_catalog_form_materialization.py.", "Focused cross-lattice QA passed 179 tests spanning callable contracts, framework-device scope, custom functions, catalogue discovery, installer/update policy, MCP release, and packaged OMERO deployment.", @@ -620,7 +675,7 @@ }, { "path": "docs/source/concepts/function_patterns.rst", - "source_sha256": "2e60a937bc001bdcc961f378c5fb29d1829580351cbf87db4eec9eda02d77411", + "source_sha256": "5db317a1e98084d593e6b7ffc9cb514a7400a6c125d8296c9fda5ffadf361929", "audience": [ "pipeline authors" ], @@ -630,24 +685,30 @@ { "path": "openhcs/core/steps/function_step.py", "sha256": "8e1243404095691030e9ebb95bb5611679f96c66c6b6819ae71e4b41249e74bc", - "role": "Declares FunctionStep composition forms and canonical callable-pattern declaration identity." + "role": "Declares the FunctionStep func forms and callable-pattern declaration identity." + }, + { + "path": "openhcs/core/function_patterns.py", + "sha256": "95b9056c6980e4f7943f909aeae8262252c9aa175247a534befc8d4cc459fdd2", + "role": "Normalises callable, chain and dictionary patterns into CompiledFunctionPattern." }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "ace8ae0c3e54bbe604b97cec7c3973a1abcf90e1dc517c2adda947dc2c35e882", - "role": "Compiles pipeline declarations into validated step plans and execution bundles." + "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "role": "Validates per-invocation callable contracts and dictionary keys against compiled grouping." }, { "path": "tests/unit/test_function_patterns.py", - "sha256": "2b18e6f40f3f5e4a9673f954544b0113eba710675573fcda3fac6ef18d5eb725", - "role": "Proves callable, sequential-chain, and dictionary function patterns compile with their declared semantics." + "sha256": "d0160652d548748901a8b95d21abde9dca050259fd2f74d4a85cb60a50441175", + "role": "Proves callable, chain and dictionary patterns compile with their declared semantics." } ], "findings": [ - "The explanation omitted that wrappers and explicitly supplied signature defaults do not create distinct callable declarations." + "The callable-contract paragraph said invocations satisfy declared axes; after G1 contracts declare required axis roles." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/concepts/function_patterns.rst in full and compared it against the changed authorities: the page itself (example already moved to Microscopy axes), openhcs/core/pipeline/compiler.py, tests/unit/test_function_patterns.py; confirmed the example imports, CompiledFunctionPattern, and the grouping-versus-stacking claims, corrected the axis-role wording, and added openhcs/core/function_patterns.py as the pattern owner.", "Re-read docs/source/concepts/function_patterns.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/steps/function_step.py; openhcs/core/pipeline/compiler.py; tests/unit/test_function_patterns.py.", "Focused cross-lattice QA passed 179 tests spanning callable contracts, framework-device scope, custom functions, catalogue discovery, installer/update policy, MCP release, and packaged OMERO deployment.", @@ -669,23 +730,24 @@ "authority": [ { "path": "openhcs/core/pipeline_document.py", - "sha256": "0849800b3e6474b8176fe35bc4aaebc63cfa85bd77251000c50da5f4c928b1b2", - "role": "Declares the public pipeline document, step collection, and serialisable declaration boundary." + "sha256": "8c554c8275c665f308eb098955bda517aa804eeb34418b5e0f41d412650ce7b5", + "role": "Declares the pipeline document (config plus ordered steps) that the core-building-blocks pages explain." }, { "path": "openhcs/core/config.py", - "sha256": "ef81b50ee22b8da69513fd50d6e45a929117fa4bfaa62a20ef08a4c02531c043", - "role": "Declares the supported OpenHCS configuration dataclasses, nested scopes, defaults, and field help metadata." + "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "role": "Declares the processing, dtype and storage configuration the processing-and-storage pages explain." }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "ace8ae0c3e54bbe604b97cec7c3973a1abcf90e1dc517c2adda947dc2c35e882", - "role": "Compiles pipeline declarations into validated step plans and execution bundles." + "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "role": "Compiles declarations into plans, the step the core-model and dimensions pages describe." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/concepts/index.rst in full and compared it against the changed authorities: openhcs/core/pipeline_document.py, openhcs/core/config.py, openhcs/core/pipeline/compiler.py; the page is a reading-order toctree whose every entry still exists under docs/source/concepts and makes no product claim affected by the G1/G4/G5 renames; no change.", "Re-read docs/source/concepts/index.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/pipeline_document.py; openhcs/core/config.py; openhcs/core/pipeline/compiler.py.", "Focused cross-lattice QA passed 179 tests spanning callable contracts, framework-device scope, custom functions, catalogue discovery, installer/update policy, MCP release, and packaged OMERO deployment.", @@ -697,7 +759,7 @@ }, { "path": "docs/source/concepts/module_structure.rst", - "source_sha256": "6537ec140390a40a617770c4c4c7691a568133bf9d72b0be41ee3acc7e61c404", + "source_sha256": "20d7d199be94986d9ed0cbb3de76059339c8e0d5f3ee78505f7fdd78d0d612da", "audience": [ "contributors", "integrators" @@ -707,89 +769,84 @@ "authority": [ { "path": "pyproject.toml", - "sha256": "6217b5f029796edcd83aa47fb8632d37aa7552344755c578fa4f181a6b23282d", - "role": "Declares the package version, Python requirement, dependency extras, scripts, and distribution metadata." - }, - { - "path": "external/PolyStore/src/polystore/omero_address.py", - "sha256": "8a3ae8789fceba0f099eef2ee983d858a8f83b764f19b70dfa7bb88f521c0c92", - "role": "Owns canonical OMERO well, concrete plane-address, and symbolic filename-template semantics." - }, - { - "path": "external/python-introspect/README.md", - "sha256": "d861f8aff2e79d9e77fbb2286b2f353c5b568412482a57efce9a488e58d543fe", - "role": "States python-introspect ownership of callable/signature analysis and signature-derived declaration projection." + "sha256": "3592cc37444e2e1f5bb0284e939643039c0d513217861af8133dd8bd6e8a4325", + "role": "Declares the eight first-party foundation distributions as OpenHCS dependencies." }, { "path": ".gitmodules", "sha256": "b70034bf0b9661f2528f002e08c03042d6cece29de79bd96c11d2d5a943684bd", - "role": "Pins the repository paths and upstream URLs of the first-party submodules named in the ownership map." + "role": "Pins the external/ submodule paths named in the foundations list." }, { - "path": "openhcs/core/config.py", - "sha256": "ef81b50ee22b8da69513fd50d6e45a929117fa4bfaa62a20ef08a4c02531c043", - "role": "Declares the supported OpenHCS configuration dataclasses, nested scopes, defaults, and field help metadata." + "path": "openhcs/domains/__init__.py", + "sha256": "544d3e1ba5c07915ecee96e1beb57f0d58843fbdf1324bdac0e7f539a3b19c0b", + "role": "States that openhcs/domains holds the declarations instantiating the domain-blind kernel." }, { - "path": "openhcs/processing/backends/lib_registry/unified_registry.py", - "sha256": "70462cddcad1bfee447ce0786a9e4a85abbd297726d59a1208156af0ef3a3743", - "role": "Defines canonical callable metadata and derives the unified catalogue from registry declarations." + "path": "openhcs/authoring/session/__init__.py", + "sha256": "8e1768912dbb8c23bc00463c9849b99e6a0b3027370f9299c0b48ed782e2ae83", + "role": "States that the session owns datasets, pipelines, compile, run and events rendered by the GUI and MCP clients." }, { - "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "b547c838b2c86fe7cc5fe12d6cbf6a6715e83055c2d8c1a109ac22566d22c51e", - "role": "Imports CellProfiler modules into ordinary OpenHCS pipeline declarations through the interop boundary." + "path": "external/PolyStore/src/polystore/omero_address.py", + "sha256": "8a3ae8789fceba0f099eef2ee983d858a8f83b764f19b70dfa7bb88f521c0c92", + "role": "Declares OMEROWellAddress, OMEROPlaneAddress and OMEROPlaneFilenameTemplate." }, { - "path": "openhcs/runtime/viewer_protocol.py", - "sha256": "30441eab97da9ffcb5b9dc33d65c2ecc56c426e79bb2b27eedc78a68aaf13bc3", - "role": "Declares the transport-independent viewer messages, stream configuration, and server contract." + "path": "openhcs/microscopes/omero.py", + "sha256": "a3ab5a257490f729e96ba413c3f11d12f10f68dae44a728206f81b850325da31", + "role": "Declares OMEROFilenameParser projecting PolyStore OMERO addresses into the parser family." }, { - "path": "external/zmqruntime/src/zmqruntime/client.py", - "sha256": "fc8bdf68455673921d79742204bf6c88e42b52bbe407d10ef4a831e084ca86a8", - "role": "Owns exact spawned viewer processes, terminal observation, automatic child reaping, and explicit application and test teardown." + "path": "external/python-introspect/README.md", + "sha256": "059dcfaafc0b2222e8e64b7fe072d24bddfb5a347a6f25113e715ed7946e44a1", + "role": "States python-introspect ownership of callable/signature analysis and declaration projection." }, { "path": "external/arraybridge/README.md", - "sha256": "b108a144e2f797425a1734f745a73ac7a215bd5e58b3f6fe4342b18e7d16b892", - "role": "States ArrayBridge ownership of reusable array conversion, callable memory metadata, and framework device mechanics." + "sha256": "2fe1a330ca101dadb6cea5ac6f0482a5ffe8c7e0e4b9bd7bc8bc2220e106415a", + "role": "States ArrayBridge ownership of array conversion, callable memory metadata and framework device mechanics." }, { "path": "external/arraybridge/src/arraybridge/types.py", - "sha256": "7b94760d57e54008d7adf4e4d37dca45d8a2c288a9dc8400be00c26148cf7624", - "role": "Declares framework capabilities, callable memory metadata, and the child-process import environment owned by each MemoryType member." + "sha256": "08f7ff4df8807bb3340e00a02c6276b1c702a0f00a2eedacd701142736c5d804", + "role": "Declares framework capabilities and the child-process import environment per memory type." }, { "path": "external/metaclass-registry/src/metaclass_registry/discovery.py", - "sha256": "f92c94a448701a8dec278c8c6ce4b0a48c49ffc7df48620c5f2a51bc1734d24b", - "role": "Owns registry discovery imports that preserve host-owned root logging." + "sha256": "757958272d11633e7b854c7104c07f0d90845c37fee6ab0c3646fb14c2ed8daa", + "role": "Owns registry discovery imports that preserve host root logging." }, { - "path": "benchmark/cellprofiler_benchmark_cli.py", - "sha256": "2b92755c556c1d66d28ffa0af6e2ac61cac5af4631514382290a2af0f907c26b", - "role": "Owns registered benchmark command declarations and installed CLI construction." + "path": "external/zmqruntime/src/zmqruntime/client.py", + "sha256": "7d3cce31d5abaa070927f02fe87d919687619e2cf4adbc37ab82483b6b8fc212", + "role": "Owns the generic execution client and transport protocol." }, { - "path": "benchmark/contracts/run_receipt.py", - "sha256": "1704e1ab47411d25d382b95871c96c1e60c94db14b8ffd55f4dc6c8a917758b9", - "role": "Owns typed run declaration and receipt semantics inside the benchmark package." + "path": "benchmark/mcp_extension.py", + "sha256": "056ab5834fa3c37df1a36a37d80f2a821fb4b59c39892d7737e62fa6a57cb53b", + "role": "Declares the expert MCP capabilities to discover cases, finalize a measured run, and inspect or report run directories." }, { "path": "benchmark/control.py", - "sha256": "4706078a8fbeb516a52bc51600751ae86c39c3ef7e397a2f8cf13d49204db941", - "role": "Owns read-only inspection derived from benchmark contracts and artifacts." + "sha256": "83fe9016625d7d8721905b8a153a28c3dcaaf2b9e925a3c97711c634868156b7", + "role": "Implements read-only case discovery, inspection and reporting over benchmark contracts." + }, + { + "path": "benchmark/contracts/run_receipt.py", + "sha256": "4cd61bd33f8f6199b3cc4130ed091dddde9c3a23975e132b1bee06c3c6a43cd7", + "role": "Declares the typed benchmark run receipts." } ], "findings": [ - "The package ownership map reduced ArrayBridge to memory conversion and omitted its callable metadata and framework-local execution capabilities.", - "The package ownership map omitted python-introspect's callable-declaration projection responsibility.", - "The source-checkout note linked to a duplicate Markdown setup procedure instead of the canonical audited repository how-to.", - "The package ownership reference omitted ArrayBridge child environments and metaclass-registry logging-safe discovery imports.", - "The OpenHCS-owned package map omitted the newly distributed benchmark package and its execution-versus-inspection ownership boundary." + "The OpenHCS-owned area map omitted openhcs/domains (G1: domain declarations such as the microscopy axis family) and openhcs/authoring (U1: the session every client renders).", + "openhcs/core was described without the axis family, dataset sources and callable/processing contracts it now owns, and openhcs/processing was credited with callable decorators/contracts that live in openhcs/core and ArrayBridge.", + "openhcs/microscopes now holds dataset sources (G4); the description was updated.", + "Dropped config.py, unified_registry.py, pipeline_import.py, viewer_protocol.py and the benchmark CLI from the authorities; the page's claims do not rest on them." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/concepts/module_structure.rst in full and compared it against the changed authorities: pyproject.toml, external/python-introspect/README.md, openhcs/core/config.py, unified_registry.py, pipeline_import.py, openhcs/runtime/viewer_protocol.py, external/zmqruntime client, ArrayBridge README and types, metaclass-registry discovery, benchmark CLI, run_receipt.py and control.py; listed the current openhcs/ top-level packages, confirmed the OMERO address and parser classes, the eight foundation distributions and the benchmark MCP capabilities (now session-backed), and added the domains and authoring areas.", "On 2026-08-27, re-read docs/source/concepts/module_structure.rst in full against the python-introspect 0.1.12 floor, declaration-owned catalogue inputs, and CellProfiler import validation; challenged the owner map and found no prose drift.", "Compared every factual claim directly with the current hash-pinned authorities: pyproject.toml; .gitmodules; openhcs/core/config.py; openhcs/processing/backends/lib_registry/unified_registry.py; openhcs/interop/cellprofiler/pipeline_import.py; openhcs/runtime/viewer_protocol.py; external/arraybridge/README.md; external/arraybridge/src/arraybridge/types.py.", "Focused cross-lattice QA passed 179 tests spanning callable contracts, framework-device scope, custom functions, catalogue discovery, installer/update policy, MCP release, and packaged OMERO deployment.", @@ -824,7 +881,7 @@ }, { "path": "docs/source/concepts/pipelines_and_steps.rst", - "source_sha256": "ae82cf8baa401e60948956fd26590ae6d32c0a08db147df5c7a70ab09398eed6", + "source_sha256": "396b99971bf876c9176f9e351fe9ca556838f9f111d79db736cad89d2dc0628c", "audience": [ "pipeline authors" ], @@ -833,33 +890,53 @@ "authority": [ { "path": "openhcs/core/pipeline_document.py", - "sha256": "0849800b3e6474b8176fe35bc4aaebc63cfa85bd77251000c50da5f4c928b1b2", - "role": "Declares the public pipeline document, step collection, and serialisable declaration boundary." + "sha256": "8c554c8275c665f308eb098955bda517aa804eeb34418b5e0f41d412650ce7b5", + "role": "PipelineDocument holds a PipelineConfig and an ordered FunctionStep list." }, { "path": "openhcs/core/steps/function_step.py", "sha256": "8e1243404095691030e9ebb95bb5611679f96c66c6b6819ae71e4b41249e74bc", - "role": "Declares FunctionStep and its callable, chain, and dictionary composition forms." + "role": "Declares FunctionStep and its callable, tuple, chain and dictionary func forms." }, { "path": "openhcs/core/config.py", - "sha256": "ef81b50ee22b8da69513fd50d6e45a929117fa4bfaa62a20ef08a4c02531c043", - "role": "Declares the supported OpenHCS configuration dataclasses, nested scopes, defaults, and field help metadata." + "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "role": "Declares ProcessingConfig (variable_components, group_by, input_source) and PipelineConfig." }, { - "path": "openhcs/core/pipeline/compiler.py", - "sha256": "ace8ae0c3e54bbe604b97cec7c3973a1abcf90e1dc517c2adda947dc2c35e882", - "role": "Compiles pipeline declarations into validated step plans and execution bundles." + "path": "openhcs/core/orchestrator/orchestrator.py", + "sha256": "a61658b107f170c192eea505966c268ab6826826f03586bd64562a44800e1ccc", + "role": "Declares PipelineOrchestrator.compile_pipelines over selected partition values, returning CompiledExecutionBundle." + }, + { + "path": "openhcs/core/compiled_execution.py", + "sha256": "85972d418f0958e59f3185f07caf25a103bc9e101e481b2b63903377cc05e397", + "role": "Declares the CompiledExecutionBundle fields: runtime and transport contexts, worker assignments and runtime environment plan." + }, + { + "path": "openhcs/authoring/session/compile_batch.py", + "sha256": "0cbe5a6b86a0fb5e9b92c4fc45f858c979031ac93594b317b9761c28f5aaf9ab", + "role": "Compiles the selected datasets through the session before execution for GUI and MCP clients." + }, + { + "path": "openhcs/interop/cellprofiler/pipeline_import.py", + "sha256": "3439d96f3fbfcb104a53019dc150049021d9a8132bd27096f1ccf510eefbe37b", + "role": "import_cellprofiler_pipeline returns the lowered steps with a populated PipelineConfig." }, { "path": "tests/unit/test_pipeline_document.py", - "sha256": "2c18222ac2a34ee267788063a374112b17874c64693bfe542669c4183bc90c36", - "role": "Proves pipeline-document construction, step mutation, serialisation, and declaration invariants." + "sha256": "4df57a9c4f12a009222981bf1ac28554405874f3533656b61344ee94056d1bab", + "role": "Proves pipeline-document construction and declaration invariants." } ], - "findings": [], - "disposition": "keep", + "findings": [ + "The compile section described the CompiledExecutionBundle as holding worker steps and environment decisions; its fields are runtime and transport contexts, worker assignments and a runtime environment plan.", + "\"compiles every selected execution axis\" now reads as every selected partition-axis value (G1 vocabulary); required axes in contracts are now required axis roles.", + "After U1 the GUI no longer compiles through its own path: GUI and MCP clients compile the selected datasets through the OpenHCS session, which reaches the same orchestrator boundary." + ], + "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/concepts/pipelines_and_steps.rst in full and compared it against the changed authorities: the page (example already on Microscopy axes), openhcs/core/pipeline_document.py, openhcs/core/config.py, openhcs/core/pipeline/compiler.py, tests/unit/test_pipeline_document.py; checked the example imports, compile_pipelines signature, CompiledExecutionBundle fields, the session compile path and the CellProfiler import return, and corrected the compile section and axis wording.", "Re-read docs/source/concepts/pipelines_and_steps.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/pipeline_document.py; openhcs/core/steps/function_step.py; openhcs/core/config.py; openhcs/core/pipeline/compiler.py; tests/unit/test_pipeline_document.py.", "Focused cross-lattice QA passed 179 tests spanning callable contracts, framework-device scope, custom functions, catalogue discovery, installer/update policy, MCP release, and packaged OMERO deployment.", @@ -871,7 +948,7 @@ }, { "path": "docs/source/concepts/storage_system.rst", - "source_sha256": "6457c0c132bd45f195274e51b255f45713b5efda9ee7ada352de556ac4aa1dde", + "source_sha256": "7d868cabb7acf21d703b8f78c6df3b96ee45ac2e1e4d432edfe3aee75be27e96", "audience": [ "pipeline authors", "storage integrators" @@ -881,56 +958,61 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "ef81b50ee22b8da69513fd50d6e45a929117fa4bfaa62a20ef08a4c02531c043", - "role": "Declares the supported OpenHCS configuration dataclasses, nested scopes, defaults, and field help metadata." + "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "role": "Declares VFSConfig read/intermediate/materialization backends and the fieldless OpenHCS ZarrConfig and TiffConfig subtypes." }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "5f323ec8edd4221ee30e34ed0c9c09e0f4e45ef502749c1a2d331b7a5452af99", - "role": "Implements scoped typed runtime-value storage and lookup for compiled execution." + "sha256": "b6a7231bc2964d472070731018f6a4935e5f23cf8f238a232045c32244e6545a", + "role": "Declares RuntimeValueStore with explicit replace and retained observation history." }, { "path": "openhcs/core/steps/function_artifact_materialization.py", - "sha256": "18c809a2d511501c4e308f6e92c18d5bb59744d4d3f59bb2bfc2507f2a2bdd03", - "role": "Owns the rules that materialise function outputs through the configured storage backend." + "sha256": "2375f8e784d4d0a5116882ae588c0681940696289b252ec6bd4b53e503c8b20b", + "role": "Materialises planned function outputs and forms filenames without changing artifact metadata." }, { - "path": "external/PolyStore/src/polystore/base.py", - "sha256": "ae641affae21b61448816a14bd68b7f2f3fc0082f0f1fbe51079ed32520450b1", - "role": "Declares the storage backend interface and lifecycle shared by local and remote materialisation backends." + "path": "openhcs/core/pipeline/function_contracts.py", + "sha256": "13bcaf7c7a62654f512d2cc739edccd2deeff35d99970ca493f3e827afbda91a", + "role": "Declares the @artifact_outputs decorator named by the page." }, { "path": "external/PolyStore/src/polystore/config.py", - "sha256": "62c8acad3da3abed8ccf7f1166fc6a0d094be1c3e842cec6446780541df292a0", - "role": "Declares lossless disk-TIFF codec settings, defaults, and path-specific batching." + "sha256": "34313f3c12056a2a316f7f4712597444b269106b0f6786aca00c5a450d439daf", + "role": "Declares ZarrConfig, the compressor factories, and TiffConfig with NONE and DEFLATE codecs and compression_level." + }, + { + "path": "external/PolyStore/src/polystore/base.py", + "sha256": "8f2082ade15072bc8e8eba363c75a1d1cff7c67c74845e6485f69eab244634dc", + "role": "Declares the storage backend interface and lifecycle." }, { "path": "external/PolyStore/src/polystore/disk.py", - "sha256": "cf089b8c79f86099a7cea1b6cbe43ceaec6331196a44b43183f2e64b7746c219", - "role": "Applies TIFF codec settings in the disk writer without changing other formats." + "sha256": "58733bc3890a9062572fedccb5643a382ce69d31486d496cd7f6c1ff9ac991c6", + "role": "Applies TIFF codec settings in the disk writer only." }, { "path": "external/arraybridge/src/arraybridge/decorators.py", - "sha256": "99823105b0ae59054df1d84013633b8b6628d0a6dc3326383199ac7c65c69bed", - "role": "Applies declared dtype conversion to a callable result before materialisation can persist it." + "sha256": "28f9bf4770bd4cc1ef34045fdef28065063ede029296f62b8a6c4140cc88e40e", + "role": "Applies declared dtype conversion to a callable result before storage." }, { "path": "tests/unit/test_runtime_value_store.py", - "sha256": "8900a7b83b59d0a40f9698dbafaaba778df982f9d7a0b628f80eab97335aeab0", - "role": "Proves scoped typed runtime-value insertion, lookup, isolation, and cleanup." + "sha256": "4ea189341b761bda993eaa2b6e24b83917568d047f4834d9f44410f7545d1452", + "role": "Proves scoped typed runtime-value insertion, replacement and lookup." }, { "path": "tests/unit/test_tiff_output_compression.py", - "sha256": "458f576e545a033ed9e23b9dc349ab92c19d296ec7f0e269ea89884cd9b82942", - "role": "Proves configured main-flow and named-artifact disk-TIFF writes round-trip exactly." + "sha256": "2b08553609d3121987bfbb185bd3e5a332db76293f8ce7f25b170eb0b6d1b31f", + "role": "Proves main-flow and named-artifact disk-TIFF writes round-trip exactly." } ], "findings": [ - "The storage explanation did not distinguish optional disk-TIFF compression from Zarr compression or semantic materialization.", - "The compression explanation did not contrast lossless codec selection with the existing value-changing callable dtype policy." + "The @artifact_outputs sentence (rewritten when special_outputs was removed) was left as one overlong unwrapped line; rewrapped to the page's line width." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/concepts/storage_system.rst in full and compared it against the changed authorities: the page, openhcs/core/config.py, openhcs/core/runtime_stores.py, function_artifact_materialization.py, PolyStore base/config/disk, ArrayBridge decorators, and the runtime-store and TIFF compression tests; confirmed the three VFSConfig backends and AUTO default, the fieldless ZarrConfig/TiffConfig subtypes, the TIFF NONE/DEFLATE codecs, the @artifact_outputs decorator and RuntimeValueStore replace/observation behaviour, and added openhcs/core/pipeline/function_contracts.py as the decorator owner.", "Re-read docs/source/concepts/storage_system.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/config.py; openhcs/core/runtime_stores.py; openhcs/core/steps/function_artifact_materialization.py; external/PolyStore/src/polystore/base.py; tests/unit/test_runtime_value_store.py.", "Focused cross-lattice QA passed 179 tests spanning callable contracts, framework-device scope, custom functions, catalogue discovery, installer/update policy, MCP release, and packaged OMERO deployment.", @@ -947,7 +1029,7 @@ }, { "path": "docs/source/getting_started/getting_started.rst", - "source_sha256": "151b7d49018e4ef31c21bd96729c1ccd3af746a82a5d239bd27bf163474c067c", + "source_sha256": "93d8e1a92b8d9fdda4f505d1565bc4118dc4688d5ac555cdd3ec3c3a8f88c45b", "audience": [ "new desktop users", "Python installers" @@ -957,107 +1039,92 @@ "authority": [ { "path": "pyproject.toml", - "sha256": "6217b5f029796edcd83aa47fb8632d37aa7552344755c578fa4f181a6b23282d", + "sha256": "3592cc37444e2e1f5bb0284e939643039c0d513217861af8133dd8bd6e8a4325", "role": "Declares the package version, Python requirement, dependency extras, scripts, and distribution metadata." }, { "path": "openhcs/cli.py", - "sha256": "51c2570a5a49c83e826c3a2e85943a2c39593e5e30e1ce8af396ec4a4c724df1", + "sha256": "59025ea51309dbae40416d75fc203287657a109379f65072d77cff41c712e11e", "role": "Declares the installed OpenHCS command entry points and their launch routing." }, { "path": "openhcs/mcp/installed_demo.py", - "sha256": "4ea6c0730db6d3938eea1cac558dd46ea5c956ef151a5d68b8ea956e8bf57655", + "sha256": "5debdb5ba24fbe29f48f065f9b3f473022c17209724e618f2a0b36190fb82913", "role": "Implements the installed-package MCP demonstration used by the getting-started verification flow." }, { - "path": "openhcs/desktop_installation.py", - "sha256": "244ade0711356f68df9f376e2517dfe1c9fb77cc960c517f7c4e8ee418b5711f", - "role": "Nominally owns only the desktop extras, Python, binary-wheel, and uv policy shared by native installation and staged updates." + "path": "openhcs/desktop/installation.py", + "sha256": "b2124aaa3e4ef4a310f91a51c1879de41de1818b25fc619cc7cdad888c4202b6", + "role": "Desktop install profile: package extras, Python minor, binary-only packages, uv release and suppressed pip/uv package-source variables shared by native installation and staged updates." }, { "path": "scripts/render_installer_contract.py", - "sha256": "da9045e1012cb0a71ff0dfd2f2d9fc98232299f5a2f019b01228e344cf3b40e7", + "sha256": "1be39dfcc514e2f6f544e6907b3af0992485d3752877502fc761f8da760a7785", "role": "Projects installer policy together with package entry points and brand identity into an exact-version native contract." }, { "path": "packaging/installers/windows/Install-OpenHCS.ps1", - "sha256": "f94eb1c0460cab7dca770a600d889bf55ae98f1aa652b2755de0ecc906ee83d8", + "sha256": "ca368082f3d7a414f9e9136b2c5ae459bfb2a9a2a37c096d06eeb93d5c403358", "role": "Implements the Windows installer, short managed paths, package installation, and shortcut creation." }, { "path": "packaging/installers/macos/install-openhcs.sh", - "sha256": "b78b6b5849265926f83f77f392f7f4ac74ce5968f62e0b6972c6ce7976498203", + "sha256": "3a0f43c57c14bd7f8ceeb38464e174728f903e0ec524d980dbdb2b5f9bc11544", "role": "Implements the macOS application installer, managed environment creation, package installation, and launcher setup." }, { - "path": "packaging/installers/macos/README.md", - "sha256": "f7e7a45c5e0b2a80cd2360f9dced3717738c103181c148330437f951de54be3a", - "role": "Defines the exact actionable Gatekeeper recovery sequence for the unsigned bootstrap period." + "path": "tests/installer/test_macos_simple_installer.py", + "sha256": "c7127f024a17e2408842522969a3a5626c6aa070a23fc209df045996537849cf", + "role": "Checks that the user-facing Gatekeeper instructions keep the exact Open Anyway control label for the unsigned bootstrap." }, { "path": "scripts/smoke_installed_desktop.py", - "sha256": "f745f93e245e5c1b02f6e0ece3a4a0e2af086c3cab77282e3907a167f5ce0a7f", - "role": "Exercises the installed distribution, launchers, MCP/runtime path, viewers, and packaged restart worker outside the source checkout while deriving the GUI subprocess ceiling from the installed-GUI smoke timing authority." - }, - { - "path": "tests/installer/test_installed_desktop_smoke.py", - "sha256": "5ac9f0e9e0e2ffe73cbab940da132421424e355c354b7476a8a072090c5aad00", - "role": "Proves platform smoke orchestration passes the exact installed Python, install root, environment, and declaration-derived GUI operation and process-exit budgets to the installed verifiers." + "sha256": "d5ce5e2a2b3590be2101b487b2957b270f386dcf47b4c197f1ce4c2ab6aa9d65", + "role": "Exercises the installed distribution, launchers, MCP/runtime path, viewers, and packaged restart worker outside the source checkout." }, { "path": "tests/installer/test_installer_contract.py", - "sha256": "42a551a7806f043ff464402795bc493ab8ee750fa0dcfa1e1822a4b6e79f8405", + "sha256": "cc7273eec242a0374d38f8aef9c6bb38f29b1ec5ba5141e2a648606a2b3db598", "role": "Proves both native installers and staged updates consume the same declaration-owned package and binary-wheel policy." }, { - "path": "openhcs/pyqt_gui/services/desktop_update.py", - "sha256": "42910339c4f11afe8d413dfbe024910a20ac6b275c563d1c6947452f8baec908", + "path": "openhcs/desktop/update.py", + "sha256": "7bd5ab7abffdc20a9f6f85fb2e5f169ecda462e50b210b175d918453dbebfb3c", "role": "Builds a staged desktop-update plan from the active managed environment." }, { - "path": "openhcs/pyqt_gui/services/desktop_update_worker.py", - "sha256": "c8d157ea5d61b5cb52f09f3835b430fe1ef3358822b988451e4ee2a6832729bf", + "path": "openhcs/desktop/update_worker.py", + "sha256": "d59781418acbcbd863cd843c6f61ddf7612d7b962136e41168033bc45dde5ed9", "role": "Creates and verifies a sibling replacement environment before publishing it and preserves the previous environment." }, { - "path": "openhcs/desktop_deployment.py", - "sha256": "d5393386a14b693af3356484bdcb560e8a3cc0faea061352f46854c0cfead051", + "path": "openhcs/desktop/deployment.py", + "sha256": "4c6a7d63bf5ea759cc3eecd46e93fa335304113873869474f5fe6823f60c298f", "role": "Owns platform-specific managed environment paths, launchers, shortcuts, and atomic desktop publication." }, { - "path": "tests/unit/test_desktop_deployment.py", - "sha256": "258a657c79a5ad96eacc06a9d4436018ffa10d7ca6d4ae56d5630e0a81c52238", - "role": "Proves Windows launcher deferral is limited to a sharing violation against a valid live GUI launcher." - }, - { - "path": "tests/unit/pyqt_gui/test_desktop_update.py", - "sha256": "b79c4c2620f271ad3a2c08f3002ec9d2c17c8f80b0b72b8ac6b1baba9dd8815c", - "role": "Proves the running environment is not mutated while a sibling update candidate is prepared." - }, - { - "path": "tests/unit/pyqt_gui/test_desktop_update_worker.py", - "sha256": "68dfc122f38164cf6eb47e856bfe369002e03e3b0699ad6bb2fda9f45db2dc8b", + "path": "tests/unit/desktop/test_desktop_update_worker.py", + "sha256": "3a521574524aabb668bfee34c393a515c25ab98351fdf1bd46ae331e3aeda515", "role": "Proves staged installation, verification, publication, rollback, and restart behaviour." }, { "path": ".github/workflows/integration-tests.yml", - "sha256": "c1ec94eab493bc46986c55b8b1428d9a0a5c0eaf5789f2dddbc06df2c0843ff7", - "role": "Exercises Windows and macOS native installers and their staged-update transition in installed environments." + "sha256": "5017eb98536006a0cec4f34573a35c1a547c3ad89889e276e88a8fb6f992991c", + "role": "Exercises Windows and macOS native installers, staged updates, and cellprofiler-compat installation on Python 3.14." }, { "path": "tests/unit/test_installed_mcp_demo.py", - "sha256": "f5d4a719a7902b364b551a3cbc4a816ccd7f46e5d9abbc87bf2d53cb0180053b", - "role": "Proves the documented installed MCP verification flow, including progress-aware stall detection." + "sha256": "8babc443f7ec59cebebd3772633dde9ae9c49b125066cd5e7c7d527dc74c2a09", + "role": "Proves the documented installed MCP verification flow." } ], "findings": [ - "The update migration needed staged-update implementation and CI evidence; the macOS recovery sequence now preserves the actionable Open Anyway control label while linking Apple's current guidance.", - "The Windows update path now preserves a valid live stable launcher, publishes the verified sibling environment, and retries the deferred launcher cache after the process releases it.", - "Workstation pip configuration could redirect or stall native installation and staged updates; each managed mutation path now suppresses pip configuration files and inherited primary and extra index overrides, and native CI injects an unreachable index to prove that boundary." + "The manual-installation note said the cellprofiler-compat extra requires Python 3.13 or earlier because of its parity dependency; that extra declares only mahotas and matplotlib, Centrosome is a development-only dependency, and CI installs dev,cellprofiler-compat on Python 3.14 on every OS. The note was removed.", + "Seven authorities were moved by commit 73a2aece2 into the openhcs.desktop package and tests/unit/desktop; the record now names openhcs/desktop/installation.py, update.py, update_worker.py, deployment.py and their moved tests." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/getting_started/getting_started.rst in full and compared it against the changed authorities: pyproject.toml, openhcs/cli.py, openhcs/mcp/installed_demo.py, render_installer_contract.py, both native installers, smoke_installed_desktop.py, installer tests, integration-tests.yml, test_installed_mcp_demo.py, and the desktop modules moved into openhcs/desktop by 73a2aece2; confirmed the extras, entry points (openhcs, openhcs-mcp-demo), --log-level handling in pyqt_gui/launch.py, the demo's generated plate / neurite preset / nonzero Napari payload check / output_root report, the desktop profile extras and pip/uv override suppression, and sibling-environment update with a preserved previous environment; removed the false Python 3.13 limit on cellprofiler-compat and replaced the moved authorities.", "On 2026-08-27, re-read docs/source/getting_started/getting_started.rst in full after the python-introspect dependency floor advanced to 0.1.12; challenged the installation how-to flow and confirmed that the supported procedure is unchanged.", "On 2026-08-27, re-read the complete installation how-to after the shared workflow made its Python quality baseline job-owned; native installation, staged update, and package-index isolation claims remain unchanged.", "Compared every factual claim directly with the current hash-pinned authorities: pyproject.toml; openhcs/cli.py; openhcs/mcp/installed_demo.py; openhcs/desktop_installation.py; scripts/render_installer_contract.py; packaging/installers/windows/Install-OpenHCS.ps1; packaging/installers/macos/install-openhcs.sh; packaging/installers/macos/README.md; scripts/smoke_installed_desktop.py; tests/installer/test_installed_desktop_smoke.py; tests/installer/test_installer_contract.py; openhcs/pyqt_gui/services/desktop_update.py; openhcs/pyqt_gui/services/desktop_update_worker.py; openhcs/desktop_deployment.py; tests/unit/test_desktop_deployment.py; tests/unit/pyqt_gui/test_desktop_update.py; tests/unit/pyqt_gui/test_desktop_update_worker.py; .github/workflows/integration-tests.yml.", @@ -1127,7 +1194,7 @@ }, { "path": "docs/source/guide_for_biologists/basic_interface.rst", - "source_sha256": "2478a75261b5e9ab12e01f897561485fb004da645f15bf030870fda628162b41", + "source_sha256": "8e9ad410dfedb59a70fa07c8f806be298b136fb89027d5c53e9cc18cc6bd680b", "audience": [ "desktop users", "MCP-assisted operators" @@ -1137,67 +1204,62 @@ "authority": [ { "path": "openhcs/pyqt_gui/main.py", - "sha256": "a156a006a3c40f43107b9b1f6ecb97ed150879eb88ded8ef89e25214309ae33a", - "role": "Applies the same preview policy to both managers at startup and after configuration save." - }, - { - "path": "tests/unit/pyqt_gui/test_ui_config_lifecycle.py", - "sha256": "eeb7c260d3ae8bc7b5e6d94917b1e5e9a214feac9e2a0c2651552950cfe3a179", - "role": "Prevents the removed Textual package authority and local UI mirrors from returning." + "sha256": "e43085c1eaf0c836ce34e1a1dae7740488d91bdfea56844e7ddd4e6925561cf6", + "role": "Dock layout (system monitor above, Datasets and ZMQ server manager left, Pipeline Editor right), View/Tools/Help menus, and preview-policy application to both panes." }, { "path": "openhcs/pyqt_gui/widgets/plate_manager.py", - "sha256": "6fa48fbcd58e47eb7132d5d9f86a002c6a0e8ff077d9654a03a791629e135378", + "sha256": "bf77012db6c0984deebf09bc5d696d8ea8f273ff535ad964354b6ddbfe839791", "role": "Implements plate import, selection, pipeline assignment, and execution controls." }, { "path": "openhcs/pyqt_gui/widgets/pipeline_editor.py", - "sha256": "450d311712d032d2f811de28a618915bc7e8a7f389e9f23c8a12c3e6ae273206", + "sha256": "8e418631fd9ba4f8ac3d6b257f9af127f9e8711281608516ea2617d7855d6f36", "role": "Implements step editing, pipeline compilation controls, and declaration feedback in the main UI." }, { "path": "openhcs/agent/services/ui_bridge_service.py", - "sha256": "37015a88828820e7ae98bf804f42c37f1fa26009b1ad28b3c537f65a656e943a", + "sha256": "c7bdf8b5caec1c21c986cdc2224524839fadfa8764dafcd05125d90c72351899", "role": "Owns agent-to-UI declaration inspection and mutation, including code and form synchronisation." }, { "path": "tests/unit/pyqt_gui/test_ui_agent_bridge.py", - "sha256": "1feee4bb413738f04e81cbdf26244dc0d585a68921e6c7749a941a08518007a1", + "sha256": "de6758859d9743e826369162855327e4a84056cafdd7ddfdc33868d2a3093353", "role": "Proves bidirectional UI-bridge inspection and mutation of live declarations, plus truthful rejection of unsupported window-navigation targets." }, { "path": "openhcs/core/pipeline_document.py", - "sha256": "0849800b3e6474b8176fe35bc4aaebc63cfa85bd77251000c50da5f4c928b1b2", - "role": "Owns the Pipeline Editor document rule that missing pipeline_config selects PipelineConfig()." + "sha256": "8c554c8275c665f308eb098955bda517aa804eeb34418b5e0f41d412650ce7b5", + "role": "PipelineDocumentCodec rule that pipeline_steps is required and a missing pipeline_config selects PipelineConfig()." }, { "path": "tests/unit/pyqt_gui/test_pipeline_code_document_transactionality.py", - "sha256": "b7e546e904b897f13d76f0c523e751afa541a75c627349ba79a3a87313d30f4c", + "sha256": "0bb9b43d867a4e26f13b51531a2b7a847c6c3685841f8e002e209200a37366b6", "role": "Proves Pipeline Editor code validates and applies pipeline_steps without an explicit pipeline_config." }, { "path": "openhcs/pyqt_gui/widgets/shared/services/pipeline_editor_workflows.py", - "sha256": "6664126ac4daa851cacff5014e04a0c09a56bbc76f62661ff97d713e9137d447", + "sha256": "c704ef7e6c5f8a5745c741e923bc4e620a321fdce069e1cdcdf521c0176e4a4f", "role": "Applies Pipeline Editor documents and commits the reconciled editor state graph." }, { "path": "openhcs/agent/services/endpoint_function_catalog_service.py", - "sha256": "977a74003819f2547548d322f4c5f0cb148b23fec20f31be4fa462bfcfb7ea04", + "sha256": "bc232090472554ab075f7800bbcc238e27a646a2d8225c87e5e443fb3c5d9f78", "role": "Owns asynchronous preparation and reuse of the endpoint callable catalogue across desktop and local MCP authoring." }, { "path": "tests/unit/pyqt_gui/test_pipeline_editor_widget.py", - "sha256": "8f28719c5bd9eb5681e67af8e1f7aceb445bcae97c12a13348f943c8b6bafa35", + "sha256": "0ce88271aa7f5f55a8fc4daec8279d732844e03dfd437c11e6e6265ca196b4a0", "role": "Proves Pipeline Editor code apply leaves the reconciled root, step, and nested function states saved." }, { "path": "scripts/gallery_catalog.py", - "sha256": "7f493e0cb305e14ee4cf7d3e995c67df4200186a8a8aaf00eef0bc66d8d191cf", + "sha256": "7fd3fce2b8fdae1abf049bb4f8a3a310ab598a7f2c507578425be537903ccfca", "role": "Owns the documentation-enabled Function Selector capture declaration, accessibility text, capture target, and generated visual projection." }, { "path": "openhcs/pyqt_gui/services/ui_bridge_windows.py", - "sha256": "edb992542dffb88a9cc4557dbd6647b76d5b68584dca27a4c9486ef545be74e9", + "sha256": "e76298f78e3cbd949e8426a2b42cd6768cc54770a5f45c9dd6c9f788b257785e", "role": "Projects embedded manager navigation through the generic declared driver." }, { @@ -1212,7 +1274,7 @@ }, { "path": "tests/unit/pyqt_gui/test_main_config_propagation.py", - "sha256": "291dc45ca20ead6025c255a8109a2e6c55d90bd7858e80ad12484dafa733d7c5", + "sha256": "303d90b43bee7bd531a1c7664fbf9707a2e7be1696a5147614f93c40e5c7021f", "role": "Verifies live preview policy propagation through the main configuration lifecycle." }, { @@ -1244,19 +1306,30 @@ "path": "external/pyqt-reactive/tests/test_list_preview_wrapping.py", "sha256": "60bfe5b16e9f17f3fed1f88213e69644ad2bcf278d7012b446689fbc75e181a2", "role": "Native Qt regressions prove row wrapping, disclosure, refresh/reorder preservation, graphical guides, structural action parity, and delegate paint acknowledgement." + }, + { + "path": "openhcs/authoring/session/views.py", + "sha256": "8ce6cac27293ac9105f4d2ebc76d10eebeba1ef90e0e8954c0a117910e17e3b3", + "role": "DatasetListView and PipelineStepsView titles and the operations each pane binds." + }, + { + "path": "openhcs/authoring/session/operations/datasets.py", + "sha256": "208274effc2e790775d90528211fa4f640eb01f7ca3fdd9c927c3bf71e39b632", + "role": "Compile/Connect/Connecting slot, Run availability (compiled, endpoint accepting requests) and Stop while a batch runs." + }, + { + "path": "openhcs/agent/ui_bridge_identities.py", + "sha256": "66dcf838d08cac2004ab8f73db777f73270cc520697281cf4f94a5fc6af4dc64", + "role": "plate_manager, pipeline_editor and main_window widget identities and stable window titles used by MCP navigation." } ], "findings": [ - "The desktop map still described the removed Textual implementation as retained.", - "The Pipeline Editor reference previously implied an explicit PipelineConfig assignment; it now records the supported steps-only source form.", - "The interface reference omitted selected-plate code-document scope and complete pipeline replacement semantics.", - "The reference omitted the Pipeline Editor's code-apply commit semantics and the non-blocking endpoint-catalogue loading state.", - "The agent-navigation reference did not distinguish successful window focus from declaration-owned field navigation or explain unsupported targets.", - "Explained list-preview wrapping and detail settings, immediate two-panel updates, and unchanged full editor values.", - "Clarified row-arrow overrides and repeated graphical guides, distinguishing shared wrapping defaults from row-owned display choices." + "The page called the left dataset pane Plate Manager; since the session interface (U1) the pane and its View menu entry are titled Datasets (DatasetListView), while MCP still identifies the window as plate_manager. The page now uses the visible name and gives the MCP identity once.", + "Agent navigation step 6 said to poll operation/state surfaces; openhcs_ui_invoke_action now returns an event_sequence and work is followed through openhcs_ui_session_events, so the step names session events." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/basic_interface.rst in full and compared it against the changed authorities: pyqt_gui/main.py, plate_manager.py, pipeline_editor.py, ui_bridge_service.py, ui_bridge_windows.py, pipeline_document.py, pipeline_editor_workflows.py, endpoint_function_catalog_service.py, gallery_catalog.py and their tests; confirmed the dock layout, menus (Image Browser, Log Viewer, Global Configuration, Custom Functions, Analysis Consolidation), Connect/Connecting/Compile/Run/Stop rules in the session operations, selected/all dataset code documents, steps-only pipeline documents, background catalogue preparation, the ui-function-selector gallery scenario and unsupported-navigation reporting; renamed the Plate Manager pane to its current Datasets title and pointed agent step 6 at session events.", "Re-read docs/source/guide_for_biologists/basic_interface.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/pyqt_gui/main.py; tests/unit/pyqt_gui/test_ui_config_lifecycle.py; openhcs/pyqt_gui/widgets/plate_manager.py; openhcs/pyqt_gui/widgets/pipeline_editor.py; openhcs/agent/services/ui_bridge_service.py; tests/unit/pyqt_gui/test_ui_agent_bridge.py; openhcs/core/pipeline_document.py; tests/unit/pyqt_gui/test_pipeline_code_document_transactionality.py; openhcs/pyqt_gui/widgets/shared/services/pipeline_editor_workflows.py; openhcs/agent/services/endpoint_function_catalog_service.py; tests/unit/pyqt_gui/test_pipeline_editor_widget.py.", "On 2026-08-22, removed the stale retained-Textual wording after the deprecated implementation and its local UI mirrors were deleted.", @@ -1289,7 +1362,7 @@ }, { "path": "docs/source/guide_for_biologists/configuration_reference.rst", - "source_sha256": "ba3c4ef923c4f643c06accc8641c462b2f63a8e46694ef984dde96ef49a0d9c7", + "source_sha256": "023ca1f76200be284520e2f118005a7e0f131f88141d3693271be0782deafa95", "audience": [ "imaging scientists", "pipeline authors" @@ -1299,67 +1372,73 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "ef81b50ee22b8da69513fd50d6e45a929117fa4bfaa62a20ef08a4c02531c043", + "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", "role": "Declares the supported OpenHCS configuration dataclasses, nested scopes, defaults, and field help metadata." }, { "path": "external/ObjectState/src/objectstate/object_state.py", - "sha256": "735ce860ab650b281534e24e3c7354f37b64bb06832cf186d6b1f626f87108fe", + "sha256": "03f818c3045b6a07024444ab2b5552fb6be9eab32dd151bb165b6f904d2863a3", "role": "Implements ObjectState provenance, inheritance resolution, and field-change propagation used by nested configuration objects." }, { "path": "openhcs/agent/services/config_service.py", - "sha256": "c4dc3860e1720fe6b383aec8b0ddcb498eb219da74da6ccd6709547205193f22", + "sha256": "902bb12178abf758cf0d7883c9cade8c5870f5362a3b5f7cc79e5640da9fb42d", "role": "Resolves and mutates configuration through the registered ObjectState-backed configuration services." }, { "path": "external/python-introspect/src/python_introspect/annotation_types.py", - "sha256": "dc126879a8a581cea0758eec4df18ec20c47ca4b1e135abde23699b65b790bde", + "sha256": "004a3ce2e214a184aaa3230ceaa89ccaa5f188c1c3d885aacad5222726c34b7a", "role": "Owns declaration-derived enum discovery, accepted input strings, and member coercion across transparent annotation wrappers." }, { "path": "openhcs/pyqt_gui/services/ui_bridge_object_state.py", - "sha256": "747c284c37e420092ffda9abbb8275fc7deaaff7f3676cecf3f5b1f324314744", + "sha256": "e548b367507b3fb6830ce9ecb844e3b756243af96fe38e0d35de5da0b974c1fd", "role": "Projects declaration-derived accepted enum inputs through live ObjectState field help." }, { "path": "tests/unit/pyqt_gui/test_ui_agent_bridge.py", - "sha256": "1feee4bb413738f04e81cbdf26244dc0d585a68921e6c7749a941a08518007a1", + "sha256": "de6758859d9743e826369162855327e4a84056cafdd7ddfdc33868d2a3093353", "role": "Proves live field help reports the exact accepted inputs for enum-backed configuration fields." }, { "path": "tests/unit/agent/test_mcp_self_description_contracts.py", - "sha256": "67c7f8052c852beef3dd875b5e96c0c7635e001df13a585589e320955667a59d", + "sha256": "d569b936f38c66bbef3d0142518720669c3859f4a42945b42ebd7dbb153da73d", "role": "Proves MCP self-description exposes the declared configuration and capability contracts." }, { "path": "tests/unit/test_configuration_mirror_deletion.py", - "sha256": "26002ddfbbf89bf06ae54888645c6b8c90154536a16e93feee1a2fae713c4963", + "sha256": "8f9da50e930925ed3ecdd57add9cc919e008a9202c9e405ff922dcd1c9a418cc", "role": "Proves configuration consumers use declaration-owned metadata without parallel field registries." }, { "path": "openhcs/core/pipeline_document.py", - "sha256": "0849800b3e6474b8176fe35bc4aaebc63cfa85bd77251000c50da5f4c928b1b2", + "sha256": "8c554c8275c665f308eb098955bda517aa804eeb34418b5e0f41d412650ce7b5", "role": "Owns default PipelineConfig construction for source documents that provide pipeline_steps only." }, { "path": "tests/unit/test_pipeline_document.py", - "sha256": "2c18222ac2a34ee267788063a374112b17874c64693bfe542669c4183bc90c36", + "sha256": "4df57a9c4f12a009222981bf1ac28554405874f3533656b61344ee94056d1bab", "role": "Proves the default-config compatibility rule and the remaining source-document validation constraints." }, { "path": "scripts/gallery_catalog.py", - "sha256": "7f493e0cb305e14ee4cf7d3e995c67df4200186a8a8aaf00eef0bc66d8d191cf", + "sha256": "7fd3fce2b8fdae1abf049bb4f8a3a310ab598a7f2c507578425be537903ccfca", "role": "Owns the documentation-enabled global, pipeline, step, and nested function configuration capture declarations, accessibility text, capture targets, and generated visual projections." + }, + { + "path": "openhcs/domains/microscopy/config.py", + "sha256": "79b8baefdbf22b85c199807f62b8234daca6904633a574886661a1c2e3d03a9b", + "role": "Declares AnalysisConsolidationConfig, PlateMetadataConfig and the standalone ExperimentalAnalysisConfig named in the post-run section." } ], "findings": [ - "The configuration explanation now distinguishes source-document omission, which selects PipelineConfig(), from normal lazy-field inheritance.", - "The explanation did not tell agents that live field help projects accepted enum input strings, including member names for non-string values.", - "The practitioner configuration map did not name the new optional TIFF compression family or distinguish its output scope from Zarr and viewer streams." + "The mental-model diagram and inheritance paragraph ended in immutable step snapshots; StepSnapshot no longer exists and compilation runs from the resolved step declarations, so both now say resolved declarations.", + "AnalysisConsolidationConfig was described as including a well pattern; the declaration has no such field and instead carries MetaXpress-style formatting, file extensions, exclude patterns and output filenames.", + "The earlier WellFilterConfig to PartitionFilterConfig rename edited the page without review and left the inheritance diagram misaligned; the class and field names were checked against config.py and the diagram realigned." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/configuration_reference.rst in full and compared it against the changed authorities: the page's own unreviewed WellFilter-to-PartitionFilter rename, openhcs/core/config.py, ObjectState object_state.py, config_service.py, python-introspect annotation_types.py, ui_bridge_object_state.py, pipeline_document.py, gallery_catalog.py and their tests; imported GlobalPipelineConfig/PipelineConfig and AbstractStep to confirm every named config field and the step-overridable families, checked the PartitionFilterConfig inheritance tree, InputSource values, main_flow_axis_persistence_enabled, openhcs_describe_config_schema and the four gallery scenarios; replaced the step-snapshot wording, corrected the consolidation field summary and realigned the diagram.", "Re-read docs/source/guide_for_biologists/configuration_reference.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/config.py; external/ObjectState/src/objectstate/object_state.py; openhcs/agent/services/config_service.py; tests/unit/agent/test_mcp_self_description_contracts.py; tests/unit/test_configuration_mirror_deletion.py; openhcs/core/pipeline_document.py; tests/unit/test_pipeline_document.py.", "Focused cross-lattice QA passed 179 tests spanning callable contracts, framework-device scope, custom functions, catalogue discovery, installer/update policy, MCP release, and packaged OMERO deployment.", @@ -1379,7 +1458,7 @@ }, { "path": "docs/source/guide_for_biologists/domain_expert_onboarding.rst", - "source_sha256": "99af4a6bf51653532d929d7008e20f515bf4ff1303a1eccfec7e7dd79be5ca4d", + "source_sha256": "53919654c9cb3da65d16f83a2d689ca59735968f3a7101f71b60920e573e2377", "audience": [ "imaging scientists", "scientists working with an MCP agent" @@ -1389,61 +1468,71 @@ "authority": [ { "path": "openhcs/agent/capabilities.py", - "sha256": "7fe9ab91490dedd6f750a57386e91835cf1067a27d10079966510aa9821adaf0", + "sha256": "eaebe33af01d057d1d21b3701f958bc96ede3ec5d0bddddca9917c1164ef0042", "role": "Declares the agent capability catalogue and safety metadata exposed through MCP self-description." }, - { - "path": "openhcs/agent/services/architecture_projection_service.py", - "sha256": "b209098d78c5520fd34d744d17b10c1ef0375a9bc2655449f6f3f9908decce0e", - "role": "Projects maintained architecture sources into agent-readable results without copying an architecture catalogue." - }, { "path": "openhcs/agent/services/knowledge_base_service.py", - "sha256": "88e10e18c66eb3a8bc2ec5dd33493a9473510c0b0a9d355502d668042f32fdd9", + "sha256": "dfc26135b16f3827ced85b122cc1555c1325da1c5e805b421afcee9cdd7d91f3", "role": "Discovers and serves the maintained knowledge and example corpus used by MCP clients." }, { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "b547c838b2c86fe7cc5fe12d6cbf6a6715e83055c2d8c1a109ac22566d22c51e", + "sha256": "3439d96f3fbfcb104a53019dc150049021d9a8132bd27096f1ccf510eefbe37b", "role": "Imports CellProfiler modules into ordinary OpenHCS pipeline declarations through the interop boundary." }, { "path": "tests/unit/agent/test_architecture_projection_service.py", - "sha256": "e820969a0499995097af17fbde9b4694f1626c0ca6d70c75fd20bd4d2cdd7be9", + "sha256": "78c3851d3b95bfd4518b82d5c1ff20c70822dcabb10da0257e8d1c794eed1c12", "role": "Proves that architecture projections come from maintained sources and preserve requested scope." }, { "path": "tests/unit/test_cellprofiler_pipeline_import.py", - "sha256": "353f0c1d8e0bc3552d2e418d792794ab0631daa16c78a3e10191b1cc1ea07d73", + "sha256": "6b1d201ec5c4ba620fa2367a9cbef1f6fbdd25edbdda9bd7b723ebe3a3363412", "role": "Proves direct import rejects modules requiring interactive desktop input instead of omitting them." }, { "path": "benchmark/cellprofiler_benchmark_cli.py", - "sha256": "2b92755c556c1d66d28ffa0af6e2ac61cac5af4631514382290a2af0f907c26b", + "sha256": "4025c591644e29f3df61e476ecae2143690e014336ea9f889aadeff4e692445e", "role": "Owns packaged comparison execution and reporting commands." }, { "path": "benchmark/contracts/run_receipt.py", - "sha256": "1704e1ab47411d25d382b95871c96c1e60c94db14b8ffd55f4dc6c8a917758b9", + "sha256": "4cd61bd33f8f6199b3cc4130ed091dddde9c3a23975e132b1bee06c3c6a43cd7", "role": "Owns typed comparison-run provenance, lifecycle status, and rerun arguments." }, { "path": "benchmark/control.py", - "sha256": "4706078a8fbeb516a52bc51600751ae86c39c3ef7e397a2f8cf13d49204db941", + "sha256": "83fe9016625d7d8721905b8a153a28c3dcaaf2b9e925a3c97711c634868156b7", "role": "Owns read-only inspection of existing benchmark directories." }, { "path": "tests/unit/test_benchmark_control.py", - "sha256": "ee75209799216c90cfa3a585dd9b7af99c99405f93e9fb5e4f900ea26d9d3ea0", + "sha256": "093b584eeb37cbb12089d5af082ad45acd779e9d2bdd5b3feaea08234fc22183", "role": "Proves exact receipt, progress, artifact, path-policy, and MCP projection behavior." + }, + { + "path": "openhcs/agent/services/architecture_projection_service.py", + "sha256": "b209098d78c5520fd34d744d17b10c1ef0375a9bc2655449f6f3f9908decce0e", + "role": "Declares the cellprofiler_translation architecture topic served by openhcs_explain_architecture." + }, + { + "path": "openhcs/interop/cellprofiler/module_artifact_declarations.py", + "sha256": "f70b52114fb5d5e2c2fc49cd9a2b46986bde48ce02d82f1a34a67f072ca2ab20", + "role": "Raises the import error for CellProfiler modules that require interactive desktop input." + }, + { + "path": "openhcs/authoring/session/views.py", + "sha256": "8ce6cac27293ac9105f4d2ebc76d10eebeba1ef90e0e8954c0a117910e17e3b3", + "role": "DatasetListView title for the desktop pane that UI-owned work populates." } ], "findings": [ - "The onboarding how-to did not tell scientists how to handle CellProfiler modules that require interactive desktop input.", - "The Official30 guidance did not tell readers where reviewed comparison execution lives or distinguish it from read-only MCP result inspection." + "The MCP section said a headless draft does not create Plate Manager state; the desktop pane is now titled Datasets (session DatasetListView), so the sentence names the Datasets pane." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/domain_expert_onboarding.rst in full and compared it against the changed authorities: agent/capabilities.py, knowledge_base_service.py, interop/cellprofiler/pipeline_import.py, test_architecture_projection_service.py, test_cellprofiler_pipeline_import.py, benchmark/cellprofiler_benchmark_cli.py, benchmark/contracts/run_receipt.py, benchmark/control.py, test_benchmark_control.py; confirmed openhcs_explain_architecture with topic cellprofiler_translation, the 50,000-character knowledge document bound, the interactive-module import rejection, the full MCP profile, the openhcs-benchmark entry point and read-only receipt/artifact inspection in benchmark/control.py; renamed the Plate Manager reference to the Datasets pane.", "Reviewed docs/source/guide_for_biologists/domain_expert_onboarding.rst in full; the recorded findings and disposition are the resulting page-specific editorial decision.", "Compared its factual claims directly with the hash-pinned files listed in authority: openhcs/agent/capabilities.py; openhcs/agent/services/architecture_projection_service.py; openhcs/agent/services/knowledge_base_service.py; openhcs/interop/cellprofiler/pipeline_import.py; tests/unit/agent/test_architecture_projection_service.py; tests/unit/test_cellprofiler_pipeline_import.py.", "Passed `.venv/bin/python scripts/validate_docs.py docs/source/guide_for_biologists`: documentation validation passed: 10 files, 2 Python blocks.", @@ -1477,24 +1566,25 @@ "diataxis": "reference", "authority": [ { - "path": "openhcs/constants/constants.py", - "sha256": "4cd6c29abd8b8e3ecc396b424d2b8b61e819a215fc1d770d3f6852b260a0691d", - "role": "Declares the nominal microscopy dimensions, grouping identities, and shared component enums used by pipelines." + "path": "openhcs/core/config.py", + "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "role": "ProcessingConfig declares variable_components as the stack-axis authority and group_by as post-assembly partitioning." }, { - "path": "openhcs/processing/backends/lib_registry/unified_registry.py", - "sha256": "70462cddcad1bfee447ce0786a9e4a85abbd297726d59a1208156af0ef3a3743", - "role": "Defines canonical callable metadata and derives the unified catalogue from registry declarations." + "path": "openhcs/core/processing_contracts.py", + "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", + "role": "ProcessingContract family (for example Pure2DContract and WholeStackInputContract) declares whether a callable runs per plane or on the whole stack." }, { - "path": "openhcs/core/config.py", - "sha256": "ef81b50ee22b8da69513fd50d6e45a929117fa4bfaa62a20ef08a4c02531c043", - "role": "Declares the supported OpenHCS configuration dataclasses, nested scopes, defaults, and field help metadata." + "path": "openhcs/core/axes.py", + "sha256": "8d93133639c576050f089fb8aab99937c8982aff07e742fea44f8a12e62bb7b2", + "role": "AxisFamily and Axis declarations that variable_components and group_by take their values from." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/glossary.rst in full and compared it against the changed authorities: openhcs/constants/constants.py (component enums removed by the axis-family refactor), lib_registry/unified_registry.py, openhcs/core/config.py; confirmed variable_components and group_by docstrings in ProcessingConfig and that ProcessingContract still exists as the per-plane versus whole-stack family in processing_contracts.py; the three distinctions remain accurate. Replaced constants.py and unified_registry.py, which no longer own these facts, with processing_contracts.py and core/axes.py.", "On 2026-08-27, re-read docs/source/guide_for_biologists/glossary.rst in full against the declaration-owned catalogue input changes; challenged its concise reference boundary and confirmed that the three axis-processing distinctions remain accurate.", "On 2026-08-27, re-read the complete biologist glossary after library registries gained declaration-owned runtime-parameter exclusions; its three axis-processing distinctions remain unchanged.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/constants/constants.py; openhcs/processing/backends/lib_registry/unified_registry.py; openhcs/core/config.py.", @@ -1507,7 +1597,7 @@ }, { "path": "docs/source/guide_for_biologists/image_sources.rst", - "source_sha256": "59e171378f8827620c00e6cc6407ea2ec2d13c18e51afea57dfb67a0c19db418", + "source_sha256": "6ba98557f6404d739a12fbf152167b23066920504f76b497d67d1ae0527c2d1b", "audience": [ "imaging scientists", "source-integration operators" @@ -1515,54 +1605,39 @@ "user_need": "Load, name, subset, and diagnose microscopy sources without bypassing source declarations.", "diataxis": "how-to", "authority": [ - { - "path": "openhcs/microscopes/microscope_interfaces.py", - "sha256": "a603fa1c58ba005f8ba22713e46efe4e33aee0f762ba188467482ae3ae429200", - "role": "Declares the microscope source interface, plate dimensions, metadata, and acquisition access contract." - }, - { - "path": "openhcs/microscopes/source_bindings_handler.py", - "sha256": "88f7c624310ea31a4cf5de884fdae295e4675a578f8a2821a837140e4e4b07b2", - "role": "Resolves configured source bindings through the selected microscope handler." - }, { "path": "openhcs/core/source_bindings.py", - "sha256": "e0afbbe40cce3847bcc2314910d0083ecc579a5a0d6e4278c3a5df4c62e7eb01", + "sha256": "b1474cfd969bd2ab3af6caba7e6e36f300f59fba5eeaaada08fd2254aef36598", "role": "Declares source bindings that map logical pipeline inputs to microscope-backed workspaces." }, { "path": "openhcs/core/source_workspace_projection.py", - "sha256": "eeb629edcf353e46e125762b85084744723eafa98b7b2b8cd7ce925a578d3304", + "sha256": "d833421b963550aa5fe428c68c0286216437aec97834e4204b0b88def011f98b", "role": "Projects source bindings and persisted semantic metadata into the concrete workspace identities compiled for execution." }, - { - "path": "openhcs/core/pipeline/compiler.py", - "sha256": "ace8ae0c3e54bbe604b97cec7c3973a1abcf90e1dc517c2adda947dc2c35e882", - "role": "Compiles pipeline declarations into validated step plans and execution bundles." - }, { "path": "tests/unit/test_source_bindings.py", - "sha256": "c7cfdd8015abc6948f0db3b3277a75d015f19a4cad68032badda99756f257942", + "sha256": "3d2bca343054aa8aa2d66ce1136ad23d1805d2c800134ee5682fac91feabaadd", "role": "Proves source-binding validation and projection into concrete workspace identities." }, { "path": "openhcs/core/source_binding_workspace.py", - "sha256": "7dce6e45b9e846713b246561340d43de378e251e35f6ef68d5527fccc1b31ad3", + "sha256": "4d28066a9f73b283fc637662e05fb2632c6fa18af164476c2899fce58cd647fe", "role": "Selects raw candidates and projects binding-owned component identity." }, { "path": "tests/unit/test_source_binding_workspace.py", - "sha256": "223c51f0915ee8012ef39b085efe5fa43b446a11e33ba10f4844aee894957995", + "sha256": "2d3582e109bfd2541a1c217cea4209674e7603dd0443f06733644d35ed80a55a", "role": "Proves raw selector coordinates are retained as original metadata while the projected address uses semantic component identity." }, { "path": "scripts/gallery_catalog.py", - "sha256": "7f493e0cb305e14ee4cf7d3e995c67df4200186a8a8aaf00eef0bc66d8d191cf", + "sha256": "7fd3fce2b8fdae1abf049bb4f8a3a310ab598a7f2c507578425be537903ccfca", "role": "Owns the documentation-enabled source-image browser capture, caption, accessibility text, dimensions, and source evidence." }, { "path": "openhcs/microscopes/bioformats_adapter.py", - "sha256": "8ed1ae0589533e80ee4eaa2738c790c3249c4579274cb5d4d5996d474332d8d0", + "sha256": "ce3cd53ee704b9a28a9a108642f6d6e7434b7a8b53114c77fd60d734b59ebd5c", "role": "Projects native OME-Zarr metadata into source planes, validates named axes and single-resolution images, and keeps Java-backed CZI and OME-TIFF decoding separate." }, { @@ -1572,12 +1647,12 @@ }, { "path": "external/PolyStore/src/polystore/zarr.py", - "sha256": "29310bc988689fa20b0d11fe6da0fdbd474eee26881e759b513f3d1e0feb875a", + "sha256": "f6b4c7eea3db8c3bb250caf548aecb9a7d2ee1a63e93b66f53c5ce0be89a542c", "role": "Owns Zarr output format through upstream FormatV04, declared batch coordinates and chunks, and reads image arrays through their NGFF declarations." }, { "path": "external/PolyStore/src/polystore/config.py", - "sha256": "62c8acad3da3abed8ccf7f1166fc6a0d094be1c3e842cec6446780541df292a0", + "sha256": "34313f3c12056a2a316f7f4712597444b269106b0f6786aca00c5a450d439daf", "role": "Owns the nominal Zarr compressor factories, Numcodecs creation, chunk strategy, and backend configuration." }, { @@ -1587,17 +1662,17 @@ }, { "path": "tests/unit/test_plane_store_sources.py", - "sha256": "792a832934e9e25ddccf08796bab4ed5c519212f158af9915226f84cea08120c", + "sha256": "f0f795651e4e3d48da30a6352088234d3ac8dcdce3a90c93e86c420491d02c1f", "role": "Proves both physical NGFF formats retain axis cardinalities, pixels, source identities, bindings, materialisation, and reinitialisation through OpenHCS." }, { "path": "tests/integration/test_source_store_code_mode_zmq.py", - "sha256": "cd4bfc193471f6c99dfd899dc9092db49db67af12d725e18dd33648fb3f5032c", + "sha256": "2bc8c0ae1d365e6d62febab95890daf8641bb12c7a07e4634f5a3178851837fb", "role": "Exercises mixed-store code documents, source projection and the opt-in compile-then-execute ZMQ path for both NGFF formats." }, { "path": "external/PolyStore/tests/test_zarr_backend.py", - "sha256": "7896159c33ee7fefb89976291e35db507acf282e92361c1abd86eca54160e2fa", + "sha256": "9a74e35521552391e1b33ea06b985debe2efaa3429299793c784db4f28104c57", "role": "Proves format-2 output, every configured codec, array dtypes, chunks, metadata, historical precedence, batch mappings, and non-overwriting storage operations." }, { @@ -1607,31 +1682,59 @@ }, { "path": "openhcs/core/artifacts.py", - "sha256": "c80450105c2acf01cb8cf2fbb0a0de0653b0f7171a744608fe7e43f879298d53", + "sha256": "65eb0ec5c903366243ca8bd5ea07412e8d34bdb81e6015fa76b3c4c59838733f", "role": "Owns InputImageSetContextSourceRelation and its exact input-image and context-bearing target constraints." }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "d9987379e3729720d0efcc32bff113d4a583f5b97bb5bfb1a5abd45c57acb8ea", + "sha256": "b6a7231bc2964d472070731018f6a4935e5f23cf8f238a232045c32244e6545a", "role": "RuntimeArtifactInput resolves exact artifact addresses and checks execution context through the declared image-set relation; it does not select arbitrary same-shape labels." }, { "path": "openhcs/core/source_matching.py", - "sha256": "128e3b8be9ef3c24ebe357a385a1ca6685cacdc83545bd2dd362849c1dcf95b3", + "sha256": "c608880341d5c40d745ebf4254d8a2931ae611c354a6fe5bb5b3d18611679c63", "role": "SourceImageSetIdentityPolicy derives plane membership and retained sample identity from source declarations." }, { "path": "openhcs/interop/cellprofiler/module_artifact_declarations.py", - "sha256": "ae3582a5d3252827c520ce14d15b0fd860e0d9486dd103e9178f3340805ff889", + "sha256": "f70b52114fb5d5e2c2fc49cd9a2b46986bde48ce02d82f1a34a67f072ca2ab20", "role": "The existing object-measurement declaration ancestor derives label input image-set relations from its selected image specs." + }, + { + "path": "openhcs/core/dataset_sources/source.py", + "sha256": "097fde4d2b74031eacf53a88e79f499417e8c1c97ba0753e8a9df831b307efa6", + "role": "DatasetSource family with format-specific, broad structured-store and declared-file selection roles and their detection order." + }, + { + "path": "openhcs/core/dataset_sources/source_bindings_source.py", + "sha256": "fccda877f39e7b2f99d6cc95143480e5955186264e316e832ca9a69c2ae90b8c", + "role": "SourceBindingsSource, the declared-file fallback for unrecognized folders driven by source_bindings_config." + }, + { + "path": "openhcs/agent/dto/plate.py", + "sha256": "689e7e720190f7abc2e0229373b88c5c60b2059efa4f7a7de14aa2d74dea3e69", + "role": "PlatePathInspectionRequest source_format field and format_specific_handler_candidates result field." + }, + { + "path": "openhcs/domains/microscopy/axes.py", + "sha256": "531695809259324032e0b88632813323b4fe6a9a176ad6f636aeef8dec332399", + "role": "Microscopy axis family (Well, Site, Channel, ZIndex, Timepoint) used for component selectors, variable_components and group_by." + }, + { + "path": "openhcs/core/axes.py", + "sha256": "8d93133639c576050f089fb8aab99937c8982aff07e742fea44f8a12e62bb7b2", + "role": "Ungrouped grouping declaration that overlapping non-dict group_by normalizes to." } ], "findings": [ - "The source-selection how-to did not explain or demonstrate the selector-to-semantic-identity projection boundary.", - "The image-source how-to omitted the supported NGFF storage formats and unchanged output policy, and its decoder wording did not distinguish native OME-Zarr from Java-backed containers. The reviewed text now scopes NGFF inputs to single-resolution images and links generic format and codec policy to PolyStore." + "The page named SourceBindingsHandler and MicroscopeHandler, both deleted by the G4 dataset-source refactor (c6d9a52ce); the fallback is now SourceBindingsSource and ingestion owners are DatasetSource subclasses with selection roles. Handler wording was changed to dataset source throughout.", + "openhcs_inspect_plate_path no longer takes microscope_type; the request field is source_format (\"auto\", \"opera_phenix\", ...).", + "Prose used the deleted component enum spellings SITE, CHANNEL, WELL, Z_INDEX, TIMEPOINT and group_by=NONE; they now read Microscopy.Site/Channel/Well/ZIndex/Timepoint and Ungrouped, matching the axis family and the already-updated code example.", + "The page told users to add directories to the Plate Manager; the desktop pane is now titled Datasets." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/image_sources.rst in full and compared it against the changed authorities: the page's own unreviewed axis-family edit, microscope_interfaces.py and source_bindings_handler.py (deleted in c6d9a52ce), source_bindings.py, source_workspace_projection.py, pipeline/compiler.py, source_binding_workspace.py, bioformats_adapter.py, PolyStore zarr.py and config.py, artifacts.py, runtime_stores.py, source_matching.py, module_artifact_declarations.py and their tests; replaced the deleted handler classes with DatasetSource / SourceBindingsSource, microscope_type with source_format, deleted component enum spellings with Microscopy axes and Ungrouped, and Plate Manager with the Datasets pane; confirmed source_stack_components, ImportedMetadataTable, InputImageSetContextSourceRelation, SourceFilterSubject/MatchType, viewer route keys and both code blocks' imports still exist.", "Re-read docs/source/guide_for_biologists/image_sources.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/microscopes/microscope_interfaces.py; openhcs/microscopes/source_bindings_handler.py; openhcs/core/source_bindings.py; openhcs/core/source_workspace_projection.py; openhcs/core/pipeline/compiler.py; tests/unit/test_source_bindings.py.", "Focused cross-lattice QA passed 179 tests spanning callable contracts, framework-device scope, custom functions, catalogue discovery, installer/update policy, MCP release, and packaged OMERO deployment.", @@ -1659,23 +1762,24 @@ "authority": [ { "path": "openhcs/pyqt_gui/main.py", - "sha256": "a156a006a3c40f43107b9b1f6ecb97ed150879eb88ded8ef89e25214309ae33a", + "sha256": "e43085c1eaf0c836ce34e1a1dae7740488d91bdfea56844e7ddd4e6925561cf6", "role": "Composes the main desktop window, menus, task windows, and registered UI services." }, { - "path": "openhcs/core/pipeline_document.py", - "sha256": "0849800b3e6474b8176fe35bc4aaebc63cfa85bd77251000c50da5f4c928b1b2", - "role": "Declares the public pipeline document, step collection, and serialisable declaration boundary." + "path": "openhcs/pyqt_gui/windows/synthetic_plate_generator_window.py", + "sha256": "66ddba05885106acab8f5f9219665e8b611f50984b09c6a4e3408ab783685a9c", + "role": "Generates the synthetic plate and loads openhcs.demo.synthetic_plate_pipeline for the first-workflow tutorial." }, { - "path": "openhcs/microscopes/microscope_interfaces.py", - "sha256": "a603fa1c58ba005f8ba22713e46efe4e33aee0f762ba188467482ae3ae429200", - "role": "Declares the microscope source interface, plate dimensions, metadata, and acquisition access contract." + "path": "openhcs/demo/synthetic_plate_pipeline.py", + "sha256": "3709ac48de4d6937ae8bfe58a4f7bfe81fe5e198c719931da86dcf6b11da2fb0", + "role": "The included eight-step pipeline the Learn by doing entry promises." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/index.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/main.py, openhcs/core/pipeline_document.py, openhcs/microscopes/microscope_interfaces.py (deleted in c6d9a52ce); the page states no fact owned by the deleted microscope interface or by the pipeline document, so both were replaced by the synthetic-plate generator window and the eight-step synthetic_plate_pipeline.py that back its only product claim; the toctree and links remain correct.", "Reviewed docs/source/guide_for_biologists/index.rst in full; the recorded findings and disposition are the resulting page-specific editorial decision.", "Compared its factual claims directly with the hash-pinned files listed in authority: openhcs/pyqt_gui/main.py; openhcs/core/pipeline_document.py; openhcs/microscopes/microscope_interfaces.py.", "Passed `.venv/bin/python scripts/validate_docs.py docs/source/guide_for_biologists`: documentation validation passed: 10 files, 2 Python blocks.", @@ -1697,30 +1801,31 @@ "authority": [ { "path": "pyproject.toml", - "sha256": "6217b5f029796edcd83aa47fb8632d37aa7552344755c578fa4f181a6b23282d", + "sha256": "3592cc37444e2e1f5bb0284e939643039c0d513217861af8133dd8bd6e8a4325", "role": "Declares the package version, Python requirement, dependency extras, scripts, and distribution metadata." }, { "path": "openhcs/cli.py", - "sha256": "51c2570a5a49c83e826c3a2e85943a2c39593e5e30e1ce8af396ec4a4c724df1", + "sha256": "59025ea51309dbae40416d75fc203287657a109379f65072d77cff41c712e11e", "role": "Declares the installed OpenHCS command entry points and their launch routing." }, { "path": "packaging/installers/windows/Install-OpenHCS.ps1", - "sha256": "f94eb1c0460cab7dca770a600d889bf55ae98f1aa652b2755de0ecc906ee83d8", + "sha256": "ca368082f3d7a414f9e9136b2c5ae459bfb2a9a2a37c096d06eeb93d5c403358", "role": "Implements the Windows installer, short managed paths, package installation, and shortcut creation." }, { "path": "packaging/installers/macos/install-openhcs.sh", - "sha256": "b78b6b5849265926f83f77f392f7f4ac74ce5968f62e0b6972c6ce7976498203", + "sha256": "3a0f43c57c14bd7f8ceeb38464e174728f903e0ec524d980dbdb2b5f9bc11544", "role": "Implements the macOS application installer, managed environment creation, package installation, and launcher setup." } ], "findings": [ - "A second installation procedure would duplicate volatile package and installer facts; the compatibility page correctly redirects to the canonical task." + "The page is still only a redirect to getting_started; it restates no installer, package or entry-point fact, so the changed pyproject.toml, cli.py and native installer scripts do not make it wrong. A second installation procedure here would duplicate volatile installer facts." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/installation_and_setup.rst in full and compared it against the changed authorities: pyproject.toml, openhcs/cli.py, packaging/installers/windows/Install-OpenHCS.ps1, packaging/installers/macos/install-openhcs.sh; the page contains only the redirect sentence and its target getting_started.rst was re-reviewed in this batch, so the redirect disposition stands.", "On 2026-08-27, re-read docs/source/guide_for_biologists/installation_and_setup.rst in full after the python-introspect dependency floor advanced to 0.1.12; confirmed that its redirect still reaches the single maintained installation how-to.", "On 2026-08-26, re-read the complete compatibility page after native installer transcript and evidence handling changed; retaining one redirect to the canonical installation how-to remains accurate and avoids duplicated guidance.", "Compared every factual claim directly with the current hash-pinned authorities: pyproject.toml; openhcs/cli.py; packaging/installers/windows/Install-OpenHCS.ps1; packaging/installers/macos/install-openhcs.sh.", @@ -1753,7 +1858,7 @@ }, { "path": "docs/source/guide_for_biologists/intro.rst", - "source_sha256": "40fac7417c9d8920edb90f3b3dbae55b8e05f333ea58271ef2cce2514717fde4", + "source_sha256": "2bae4f9921e6da0170c52daf3fb66c62ffcdc1cdfd8a7a1fd857cfa54a45cd0f", "audience": [ "prospective imaging-scientist users" ], @@ -1762,28 +1867,36 @@ "authority": [ { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "ace8ae0c3e54bbe604b97cec7c3973a1abcf90e1dc517c2adda947dc2c35e882", + "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", "role": "Compiles pipeline declarations into validated step plans and execution bundles." }, { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "b547c838b2c86fe7cc5fe12d6cbf6a6715e83055c2d8c1a109ac22566d22c51e", + "sha256": "3439d96f3fbfcb104a53019dc150049021d9a8132bd27096f1ccf510eefbe37b", "role": "Imports CellProfiler modules into ordinary OpenHCS pipeline declarations through the interop boundary." }, { "path": "openhcs/processing/backends/lib_registry/registry_service.py", - "sha256": "046d3d088f685a993ddad6b5923cf63beef84b201127b72ae32af6721dd5ad63", + "sha256": "daef8925338fddf8504a9faa066af42456a13fe4602976bf64be06547a45fcc8", "role": "Owns backend discovery, availability filtering, and the canonical process catalogue projection." }, { "path": "openhcs/pyqt_gui/widgets/plate_manager.py", - "sha256": "6fa48fbcd58e47eb7132d5d9f86a002c6a0e8ff077d9654a03a791629e135378", - "role": "Implements plate import, selection, pipeline assignment, and execution controls." + "sha256": "bf77012db6c0984deebf09bc5d696d8ea8f273ff535ad964354b6ddbfe839791", + "role": "Implements dataset import, selection, initialization, compilation and run controls in the Datasets pane." + }, + { + "path": "openhcs/core/dataset_sources/choice.py", + "sha256": "7fb3c7ce701f0d629f288b79090044f43ac8ce1b51ca4f50937534407f226f1a", + "role": "DatasetSourceChoices projects the registered dataset sources, plus auto, as the dataset_source field's choices." } ], - "findings": [], - "disposition": "keep", + "findings": [ + "The page pointed readers to the Plate Manager as the catalogue of available microscope formats; the pane is now titled Datasets and lists datasets, while the registered formats are the dataset_source choices (DatasetSourceChoices) shown in the configuration form. The sentence now names those choices." + ], + "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/intro.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compiler.py, openhcs/interop/cellprofiler/pipeline_import.py, lib_registry/registry_service.py, pyqt_gui/widgets/plate_manager.py; the fit list, CellProfiler import claim and compile-before-run flow remain accurate (Run requires a compiled selection in the session operations); replaced the Plate Manager format-catalogue pointer with the dataset_source choices declared in openhcs/core/config.py and dataset_sources/choice.py.", "On 2026-08-27, re-read docs/source/guide_for_biologists/intro.rst in full against the explicit CellProfiler import-validation hook; challenged its explanation of supported imports and found no prose drift.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/pipeline/compiler.py; openhcs/interop/cellprofiler/pipeline_import.py; openhcs/processing/backends/lib_registry/registry_service.py; openhcs/pyqt_gui/widgets/plate_manager.py.", "Focused cross-lattice QA passed 179 tests spanning callable contracts, framework-device scope, custom functions, catalogue discovery, installer/update policy, MCP release, and packaged OMERO deployment.", @@ -1800,7 +1913,7 @@ }, { "path": "docs/source/guide_for_biologists/intro_stitching.rst", - "source_sha256": "fde83b50eb661c94d16d4d8986b86f565f6db1bdf7f57668293af7b1612e29d2", + "source_sha256": "3bcb8fa804b88f764b4d0c52e81b336815fd4a9a1a984baefca0a20f861341ba", "audience": [ "first-time desktop users" ], @@ -1814,30 +1927,37 @@ }, { "path": "openhcs/demo/synthetic_data.py", - "sha256": "7a8f4a79e716ad944d5ceb1ed14e62cab9211aee65b1c0dc571e097e0f4d3c09", + "sha256": "be80973ecd87b4b3b36b7a7283d962a5d7cafd491cfa16eafe4622e09391463a", "role": "Generates the packaged deterministic synthetic microscopy plate consumed by the introductory workflow." }, { "path": "openhcs/demo/synthetic_plate_pipeline.py", - "sha256": "3f8916ff71938ac5b53c7f55329b037653519e9592cf564257f78dde158fbd11", - "role": "Provides the packaged example pipeline used to process the generated synthetic plate." + "sha256": "3709ac48de4d6937ae8bfe58a4f7bfe81fe5e198c719931da86dcf6b11da2fb0", + "role": "Declares the eight supplied steps, from Image Enhancement Processing to Cell Counting, that the tutorial tells readers to expect." + }, + { + "path": "openhcs/authoring/session/operations/datasets.py", + "sha256": "208274effc2e790775d90528211fa4f640eb01f7ca3fdd9c927c3bf71e39b632", + "role": "Declares the Init, Compile, Run, Viewer, and Results dataset operations and their button labels named in the tutorial steps." }, { - "path": "tests/pyqt_gui/integration/test_end_to_end_workflow_foundation.py", - "sha256": "db2f98b9878565df935d78aaf8d8365cdac5fe160cd4fcee2d6246502dc355d5", - "role": "Proves synthetic plate import and the desktop configuration lifecycle, including form readiness, inheritance, persistence, reset, and teardown." + "path": "openhcs/pyqt_gui/main.py", + "sha256": "e43085c1eaf0c836ce34e1a1dae7740488d91bdfea56844e7ddd4e6925561cf6", + "role": "Places Generate Synthetic Plate in the View menu." }, { "path": "scripts/gallery_catalog.py", - "sha256": "7f493e0cb305e14ee4cf7d3e995c67df4200186a8a8aaf00eef0bc66d8d191cf", + "sha256": "7fd3fce2b8fdae1abf049bb4f8a3a310ab598a7f2c507578425be537903ccfca", "role": "Owns the documentation-enabled synthetic generator, plate metadata, and live measurement result capture declarations used by the first-workflow tutorial." } ], "findings": [ - "The previous evidence role overstated the GUI foundation as proving pipeline execution; the generator and preset declarations remain the authorities for the tutorial's compile-and-run path." + "The page sent readers to an unnamed 'metadata viewer'; after U1 the dataset action is the Viewer operation (ShowDatasetImages) with a Metadata tab, so the step now names that button and tab.", + "The end-to-end GUI foundation test no longer adds evidence for this tutorial's claims beyond the generator and session operations; it was replaced as an authority by the operation declarations and the menu owner." ], - "disposition": "keep", + "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/intro_stitching.rst in full and compared it against the changed authorities: openhcs/demo/synthetic_data.py, openhcs/demo/synthetic_plate_pipeline.py, tests/pyqt_gui/integration/test_end_to_end_workflow_foundation.py, scripts/gallery_catalog.py; confirmed the eight step names, the View menu entry, the generator dialog labels, the Init/Compile/Run/Results labels and the four gallery scenario ids, and replaced the unnamed metadata viewer with the Viewer button and Metadata tab.", "Re-read docs/source/guide_for_biologists/intro_stitching.rst in full against repository base bb4218fae and the current shared worktree; challenged its tutorial user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/pyqt_gui/windows/synthetic_plate_generator_window.py; openhcs/demo/synthetic_data.py; openhcs/demo/synthetic_plate_pipeline.py; tests/pyqt_gui/integration/test_end_to_end_workflow_foundation.py.", "Focused cross-lattice QA passed 179 tests spanning callable contracts, framework-device scope, custom functions, catalogue discovery, installer/update policy, MCP release, and packaged OMERO deployment.", @@ -1851,7 +1971,7 @@ }, { "path": "docs/source/guide_for_biologists/troubleshooting_FAQ.rst", - "source_sha256": "0c2fb0dbb43ec18efb8d4aeff2a728a35630516a6e506de677de5d070089f428", + "source_sha256": "7499a25552e6f1fed5ef0d669970b4363a80e96f676279b6218ede90a39e8191", "audience": [ "desktop operators diagnosing a failed workflow" ], @@ -1859,29 +1979,39 @@ "diataxis": "how-to", "authority": [ { - "path": "openhcs/pyqt_gui/widgets/plate_manager.py", - "sha256": "6fa48fbcd58e47eb7132d5d9f86a002c6a0e8ff077d9654a03a791629e135378", - "role": "Implements plate import, selection, pipeline assignment, and execution controls." + "path": "openhcs/authoring/session/operations/datasets.py", + "sha256": "208274effc2e790775d90528211fa4f640eb01f7ca3fdd9c927c3bf71e39b632", + "role": "Declares the Init, Compile, Run, Connect, and Code dataset operations and the submit_timeout_ms request field that the recovery steps name." + }, + { + "path": "openhcs/authoring/session/operations/application.py", + "sha256": "628fb1dd209d62d1d5830e710cf6e0ea6804f576569489da7df42f974652ffd8", + "role": "Declares the Restart OpenHCS and restore session operation and its warning that the old process's operation status does not survive the restart." }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "ace8ae0c3e54bbe604b97cec7c3973a1abcf90e1dc517c2adda947dc2c35e882", - "role": "Compiles pipeline declarations into validated step plans and execution bundles." + "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "role": "Stops before execution on unsatisfied declarations and applies materialize_runtime_artifacts to automatic terminal outputs." }, { "path": "openhcs/utils/environment.py", - "sha256": "d408e5c8bd86f9b7d26ce23595fb75200a9320d9f3ee9bfca8d29861cecb9c9d", - "role": "Detects runtime environment and optional backend availability, including CPU-only operation." + "sha256": "2d8738e593b3e92c966089772624f47d17d38bb27fb90974707af659c91e8ad8", + "role": "Reads OPENHCS_CPU_ONLY to hide GPU-backed declarations." + }, + { + "path": "openhcs/pyqt_gui/launch.py", + "sha256": "bf47d568e2d9bcb758342f34a5b3f030fd8abd9276c5e4fa2406666301e0561f", + "role": "Defines the openhcs --no-gpu launch option." }, { "path": "openhcs/runtime/viewer_protocol.py", - "sha256": "30441eab97da9ffcb5b9dc33d65c2ecc56c426e79bb2b27eedc78a68aaf13bc3", - "role": "Declares the transport-independent viewer messages, stream configuration, and server contract." + "sha256": "c4bdae4bae6dee412f0d27a9cf75a95db53b3bd95c03cee7d046be16b52fe5b8", + "role": "Declares the viewer server contract whose readiness and dependency failures the viewer section diagnoses." }, { "path": "openhcs/runtime/zmq_execution_client.py", - "sha256": "31081f2a323a81c060aede1201a4bf1a1b39a51f37928c281a419ffaceadfe60", - "role": "Implements execution-server connection, request submission, progress consumption, and connection-state reporting." + "sha256": "147fb663d3ea60b1348063440e05b77d54b40634d1ae81cf4fb4f3b5457fbc99", + "role": "Implements execution-server connection and retains the .startup.jsonl journal for a server that fails before readiness." }, { "path": "openhcs/runtime/zmq_application.py", @@ -1890,86 +2020,69 @@ }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "a156a006a3c40f43107b9b1f6ecb97ed150879eb88ded8ef89e25214309ae33a", - "role": "Wires the endpoint compatibility observation to the owning version-restart workflow." + "sha256": "e43085c1eaf0c836ce34e1a1dae7740488d91bdfea56844e7ddd4e6925561cf6", + "role": "Wires endpoint compatibility to the version-restart workflow and gates the restart operation until initialisation, compilation and execution have finished." }, { - "path": "openhcs/pyqt_gui/services/zmq_version_restart.py", - "sha256": "86d4bf273c75417b2c4f5c38ac83a66e3487bb233be88a608140a0bdb4c8bc67", - "role": "Owns the mismatch prompt, one endpoint-replacement transaction, and its success or recovery outcome." + "path": "openhcs/desktop/zmq_version_restart.py", + "sha256": "84b4f13432862bc9d61dbbcd19d4b54a0240fb243f9ddd2fb94a7b68f5808acb", + "role": "Owns the mismatch prompt, its cancel path, one endpoint-replacement transaction, and its success or recovery outcome." }, { - "path": "openhcs/pyqt_gui/services/desktop_restart.py", - "sha256": "24ca24c3afc0038a268280a73ae7615045eb107ee541a59c13fcdf07005603a2", - "role": "Preserves the current desktop session and ObjectState history across the version restart." + "path": "openhcs/desktop/restart.py", + "sha256": "53028fa6380292520ca42945b12c6844019dd91a77aef3d15e20108830a18c0b", + "role": "Preserves the current desktop session and edit history across a restart." }, { - "path": "openhcs/pyqt_gui/services/desktop_restart_worker.py", + "path": "openhcs/desktop/restart_worker.py", "sha256": "52d8a8f223d49dbd9ef42951fe507dfbea803b33ff4dfce3e198f4b1a9fe4ed7", "role": "Launches the saved restart command only after the current UI process exits." }, { - "path": "tests/unit/pyqt_gui/test_desktop_restart.py", - "sha256": "aafaa6e7a46859f50b44232e12f8db606e0dd4631340a834694a0fe795b5ff70", - "role": "Proves the mismatch dialog, saved session, and version-specific restart behavior." - }, - { - "path": "tests/unit/pyqt_gui/test_zmq_version_restart.py", - "sha256": "052699f393497ab23ec49d23fa136d969245c95c0973d6e9898aedfb860c43f9", + "path": "tests/unit/desktop/test_zmq_version_restart.py", + "sha256": "f1968ec55295c577e118ada34ba80d987f50d69df30105b8b3a65a1a774c0018", "role": "Proves expected replacement and restart failures leave the current UI usable and discard stale saved sessions." }, { - "path": "tests/unit/pyqt_gui/test_desktop_restart_worker.py", - "sha256": "a9fbda7d6c78972d6300cd9fe67c2e88497b6b802a9b48057568007bb33b0e1c", - "role": "Proves parent-exit gating, Windows handle cleanup, exact restart arguments, and fail-closed launch outcomes." - }, - { - "path": "scripts/smoke_installed_desktop.py", - "sha256": "f745f93e245e5c1b02f6e0ece3a4a0e2af086c3cab77282e3907a167f5ce0a7f", - "role": "Executes the GUI acceptance probe and packaged restart worker from the installed environment while deriving the outer process ceiling from the probe's timing authority." - }, - { - "path": "tests/installer/test_installed_desktop_smoke.py", - "sha256": "5ac9f0e9e0e2ffe73cbab940da132421424e355c354b7476a8a072090c5aad00", - "role": "Proves platform smoke orchestration passes the exact installed Python, install root, environment, and declaration-derived GUI operation and process-exit budgets to the installed verifiers." + "path": "tests/unit/desktop/test_desktop_restart.py", + "sha256": "50425f39e481a6fff384b9732bc0f2cd68f223d3ac980234a9142b6db981b48f", + "role": "Proves the mismatch dialog, saved session, and version-specific restart behavior." }, { "path": "openhcs/agent/services/endpoint_function_catalog_service.py", - "sha256": "977a74003819f2547548d322f4c5f0cb148b23fec20f31be4fa462bfcfb7ea04", + "sha256": "bc232090472554ab075f7800bbcc238e27a646a2d8225c87e5e443fb3c5d9f78", "role": "Owns asynchronous execution-endpoint catalogue loading, failure projection, revision state, and reuse." }, { - "path": "tests/unit/pyqt_gui/test_function_catalog_projection.py", - "sha256": "b16c69e8290aa7c74dd142288b16b7e0975652fb227384f0abb50fc156387a60", - "role": "Proves one shared non-blocking catalogue request, endpoint invalidation, and safe background-client lifecycle." + "path": "openhcs/mcp/dev_client.py", + "sha256": "c40f72cb9ba7f62b574d3f9e7b96470b5a78db60c87f6a4a99897449ae875714", + "role": "Defines the MCP development client's --timeout-seconds inactivity limit renewed by progress." + }, + { + "path": "openhcs/agent/ui_bridge_environment.py", + "sha256": "30f459623ed48c6d653725080453630ef9813f875dd8e63d171147f5e73caacf", + "role": "Reads OPENHCS_UI_CONFIG_CACHE_FILE to select the UI configuration shared by the desktop and MCP client." }, { "path": "openhcs/agent/services/ui_bridge_transport.py", - "sha256": "32e759baf65b34989c99223d8fa9e64800ab566cd05160df2bc125b04dea5953", + "sha256": "1c2fb99aef62969cfabb9d22ccb954e8894ee72fd2e66e625304ffdb96a41b95", "role": "Rejects UI bridge responses from an incompatible OpenHCS application or bridge protocol." }, { "path": "openhcs/pyqt_gui/services/ui_bridge_server.py", - "sha256": "667f582fffd4b489ca7274f8962a759c3d33d24b10ec8e6d1aa5cac01233b067", + "sha256": "4b0664fa62d24557294324b631ee3e39052c84e578d40ec408197bce5adcedac", "role": "Advertises the current application identity and rejects incompatible MCP requests before UI dispatch." - }, - { - "path": "tests/unit/pyqt_gui/test_ui_agent_bridge.py", - "sha256": "1feee4bb413738f04e81cbdf26244dc0d585a68921e6c7749a941a08518007a1", - "role": "Proves stale protocol and OpenHCS versions cannot reach running UI operations." } ], "findings": [ - "The server recovery guidance now explains the version mismatch decision and its state-preserving endpoint and desktop restart.", - "The guide lacked a recovery path for first-start function-catalogue loading; it now distinguishes asynchronous progress from endpoint failure.", - "The guide did not tell operators how to distinguish a known pre-dispatch MCP timeout from an execute request with an unknown acceptance outcome.", - "The guide did not distinguish execution-server replacement from recovery after an MCP/UI bridge version mismatch.", - "The documented version-restart recovery lacked direct evidence for capture, executor-shutdown, endpoint-replacement, and detached-launch failure outcomes.", - "The selector recovery section did not explain that progress renews client inactivity or that closing the server reaps the exact cold-catalogue preparation process.", - "Clarified current endpoint catalogue reads after remote registration and retained asynchronous/cache ownership." + "The desktop restart, version-restart and restart-worker modules and their tests moved from openhcs/pyqt_gui/services and tests/unit/pyqt_gui to openhcs/desktop and tests/unit/desktop (D1/U5); the record's authorities pointed at deleted paths.", + "The restart paragraph described an 'action receipt' and 'operation receipt'; the restart operation now states that it returns before the old UI exits and that the old process's operation status does not survive, so the wording follows it.", + "Plate Manager actions are now session operations (U1); Connect, Code, Init, Compile and Run keep the labels the page uses, so no step changed.", + "The isolated-session paragraph called the selected UI configuration 'canonical'; it now says 'shared'." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/troubleshooting_FAQ.rst in full and compared it against the changed authorities: plate_manager.py, compiler.py, environment.py, viewer_protocol.py, zmq_execution_client.py, pyqt_gui/main.py, the moved desktop restart modules and tests, smoke_installed_desktop.py, the catalogue service and UI bridge transport/server; confirmed every named option, environment variable, journal file, button and action label in current source, replaced the receipt wording of the restart paragraph, and repointed restart authorities to openhcs/desktop.", "Re-read docs/source/guide_for_biologists/troubleshooting_FAQ.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/pyqt_gui/widgets/plate_manager.py; openhcs/core/pipeline/compiler.py; openhcs/utils/environment.py; openhcs/runtime/viewer_protocol.py; openhcs/runtime/zmq_execution_client.py; openhcs/runtime/zmq_application.py; openhcs/pyqt_gui/main.py; openhcs/pyqt_gui/services/desktop_restart.py; tests/unit/pyqt_gui/test_desktop_restart.py; openhcs/agent/services/endpoint_function_catalog_service.py; tests/unit/pyqt_gui/test_function_catalog_projection.py.", "Focused cross-lattice QA passed 179 tests spanning callable contracts, framework-device scope, custom functions, catalogue discovery, installer/update policy, MCP release, and packaged OMERO deployment.", @@ -2005,7 +2118,7 @@ }, { "path": "docs/source/guides/complete_examples.rst", - "source_sha256": "ad07bdc258a85e22c4b67c3d4694c84550dc8e0c1e136998d174e558bc9682d7", + "source_sha256": "75215edce0d3c4318b82c500c57640e4a3e6981bc28c40b6c634609ffd825576", "audience": [ "pipeline authors", "demo operators" @@ -2015,33 +2128,58 @@ "authority": [ { "path": "openhcs/agent/services/knowledge_base_service.py", - "sha256": "88e10e18c66eb3a8bc2ec5dd33493a9473510c0b0a9d355502d668042f32fdd9", - "role": "Discovers and serves the maintained knowledge and example corpus used by MCP clients." + "sha256": "dfc26135b16f3827ced85b122cc1555c1325da1c5e805b421afcee9cdd7d91f3", + "role": "Generates the Official30 recipe sections lazily from the manifest-resolved .cppipe and serves them through KnowledgeBaseService." + }, + { + "path": "tests/unit/agent/test_official30_knowledge_examples.py", + "sha256": "e8c5b917eceaed404e4a4858ede3acfa7cf3864255e27b82f3d6b8c4799ada9e", + "role": "Proves the openhcs_official30_benchmark_recipes document id and the examplehuman-openhcs-python section reached by an 'OpenHCS Python' search." + }, + { + "path": "openhcs/core/processing_contracts.py", + "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", + "role": "Declares Pure2DContract used in the minimal declaration example." }, { "path": "openhcs/processing/presets/mfd_specs.py", - "sha256": "6f1808c7cecab58e3317bfa020db7d087947ed2e3b3d11838f11d47924e3a18c", - "role": "Declares the maintained MFD example pipeline specifications exposed through the example corpus." + "sha256": "0c1f43037c3789e502e497e6204157a1cdbfe17b1ec18954743a05ab17ee1f68", + "role": "Declares MfdPresetKey, build_mfd_preset and the relative whole-device template path." }, { "path": "openhcs/processing/presets/pipelines/loose_operaphenix_neurite_outgrowth.py", - "sha256": "46fb1ff75071febae663e5efa088590f579ccdd60e81a997476c32b07ddea491", - "role": "Declares the maintained multi-step Operetta/Opera Phenix neurite-outgrowth example pipeline." + "sha256": "1997c15cf71b8c65c8b49f238fced11b2d7a6c8bd5725ea75bdcd8cf7ab14314", + "role": "Declares the example_inputs fields, including source_voxel_spacing, and the UnifiedNeurons result of the loose Opera Phenix example." + }, + { + "path": "openhcs/microscopes/opera_phenix.py", + "sha256": "0af8f549988b0fb919d7b58121a3c260a47249e6cca6bf6d7e1a854171f2de38", + "role": "Declares OperaPhenixHandler, which requires Index.xml for native plate semantics." }, { "path": "scripts/master_multi_plate_demo.py", - "sha256": "8115619d0dd4aeedfd8a4ce1b20f7eabd8f3c1f0ffbdc20e4dc8b2231cff7ffb", - "role": "Implements the maintained end-to-end multi-plate example referenced by the example guide." + "sha256": "82f8f6e074625118ec9a96b0f67b5aa6ad39fcc95aed2b7e877d8f442770abe3", + "role": "Implements the master showcase's command-line options, base port 5900, contributor loading and output root." }, { - "path": "tests/unit/agent/test_official30_knowledge_examples.py", - "sha256": "6b81450dc01a33370fdad7a9a3b88bc87c3f85eddcd8b872373da9a89172c7a4", - "role": "Proves that the official example corpus remains discoverable and structurally valid." + "path": "scripts/mcp_assay_showcase.py", + "sha256": "a1ea7a9b6ffdec553931fbe5e9f2b755e63a98b92561ff9edee13b296d926ff6", + "role": "Declares the seven built-in scenario blueprints the master showcase composes." + }, + { + "path": "openhcs/interop/cellprofiler/pipeline_import.py", + "sha256": "3439d96f3fbfcb104a53019dc150049021d9a8132bd27096f1ccf510eefbe37b", + "role": "Defines import_cellprofiler_pipeline, which lowers a .cppipe into steps and a pipeline config." } ], - "findings": [], - "disposition": "keep", + "findings": [ + "The page told readers to use the 'Microscope.OPERAPHENIX handler'; the Microscope enum was removed with the dataset-source family (G4), so the page now names OperaPhenixHandler selected through dataset_source detection.", + "The loose Opera Phenix example gained a source_voxel_spacing input for calibration the copied TIFFs lose with Index.xml; the list of fields to edit now includes it.", + "The Official30 paragraph said 'canonical importer'; it now says 'public importer', matching the CellProfiler import section." + ], + "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guides/complete_examples.rst in full and compared it against the changed authorities: knowledge_base_service.py, mfd_specs.py, loose_operaphenix_neurite_outgrowth.py, master_multi_plate_demo.py, test_official30_knowledge_examples.py; confirmed the code block's imports, the Official30 document and section ids, the MFD owner names and template path, the master demo flags, port, blueprint count and contributor callables, replaced Microscope.OPERAPHENIX with OperaPhenixHandler and added the source_voxel_spacing input.", "Reviewed docs/source/guides/complete_examples.rst in full; the recorded findings and disposition are the resulting page-specific editorial decision.", "Compared its factual claims directly with the hash-pinned files listed in authority: openhcs/agent/services/knowledge_base_service.py; openhcs/processing/presets/mfd_specs.py; openhcs/processing/presets/pipelines/loose_operaphenix_neurite_outgrowth.py; scripts/master_multi_plate_demo.py; tests/unit/agent/test_official30_knowledge_examples.py.", "Passed `.venv/bin/python scripts/validate_docs.py docs/source/guides`: documentation validation passed: 9 files, 2 Python blocks.", @@ -2053,7 +2191,7 @@ }, { "path": "docs/source/guides/example_corpus_map.rst", - "source_sha256": "e3e4af860a98ec0b9412d66a45cef57837858e71dcf2e2c65dccd0ea43e11e6a", + "source_sha256": "d9cdad8b4acffb93f051bf435612df6550dc38eab63330d3a817319ad8baa9d3", "audience": [ "pipeline authors", "MCP agents assisting scientists" @@ -2063,28 +2201,41 @@ "authority": [ { "path": "openhcs/agent/services/knowledge_base_service.py", - "sha256": "88e10e18c66eb3a8bc2ec5dd33493a9473510c0b0a9d355502d668042f32fdd9", - "role": "Discovers and serves the maintained knowledge and example corpus used by MCP clients." + "sha256": "dfc26135b16f3827ced85b122cc1555c1325da1c5e805b421afcee9cdd7d91f3", + "role": "Serves the openhcs_official30_benchmark_recipes sections and derives the Native Example Source Index from the Python paths this page declares." }, { "path": "openhcs/processing/presets/mfd_specs.py", - "sha256": "6f1808c7cecab58e3317bfa020db7d087947ed2e3b3d11838f11d47924e3a18c", - "role": "Declares the maintained MFD example pipeline specifications exposed through the example corpus." + "sha256": "0c1f43037c3789e502e497e6204157a1cdbfe17b1ec18954743a05ab17ee1f68", + "role": "Owns MfdPresetKey, build_mfd_preset and the plate-relative whole-device template path wrapped by the four 10x_mfd modules." + }, + { + "path": "openhcs/processing/presets/pipelines/loose_operaphenix_neurite_outgrowth.py", + "sha256": "1997c15cf71b8c65c8b49f238fced11b2d7a6c8bd5725ea75bdcd8cf7ab14314", + "role": "Declares the loose Opera Phenix example's exact Hoechst, MAP2 and SMI312 sources, one-well bound and unified neuron result." + }, + { + "path": "openhcs/agent/capabilities.py", + "sha256": "eaebe33af01d057d1d21b3701f958bc96ede3ec5d0bddddca9917c1164ef0042", + "role": "Declares openhcs_get_authoring_context and the inspect-plate, query-plate-files, selected-plate-images and selected-plate-files inspection tools." }, { "path": "benchmark/datasets/manifest.py", - "sha256": "f3ab9ca4152d54bb29a7659a96ecf5762c08844b0f0caaf0f491807c2d924802", + "sha256": "9a556f7dddee400493369e88eef91c19faadd6ab967a1d256ee462280ec00325", "role": "Declares benchmark dataset identities and the manifest metadata used to obtain their inputs." }, { "path": "tests/unit/agent/test_knowledge_base_service.py", - "sha256": "7b4ab158ced3ec3355d314d04e9baf0c8a4e8b2e60ce84aa56b2723c44f563c1", - "role": "Proves discovery and retrieval of maintained knowledge-base and example resources." + "sha256": "ce4636580893a8736fd4607c6ed0f431d25c97e0c44e2b95912af3fc36b1d750", + "role": "Proves discovery and retrieval of the example corpus map and other maintained knowledge documents." } ], - "findings": [], - "disposition": "keep", + "findings": [ + "The page called the benchmark's parity evidence 'parity receipts'; changed to 'parity evidence'." + ], + "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guides/example_corpus_map.rst in full and compared it against the changed authorities: knowledge_base_service.py, mfd_specs.py, benchmark/datasets/manifest.py, test_knowledge_base_service.py; confirmed every repository path, the thirty scoped native reference rows, the four MFD wrapper modules, the cellprofiler_translation topic, the authoring-context and plate inspection tool names and the Native Example Source Index; replaced 'parity receipts' with 'parity evidence'.", "Reviewed docs/source/guides/example_corpus_map.rst in full; the recorded findings and disposition are the resulting page-specific editorial decision.", "Compared its factual claims directly with the hash-pinned files listed in authority: openhcs/agent/services/knowledge_base_service.py; openhcs/processing/presets/mfd_specs.py; benchmark/datasets/manifest.py; tests/unit/agent/test_knowledge_base_service.py.", "Passed `.venv/bin/python scripts/validate_docs.py docs/source/guides`: documentation validation passed: 9 files, 2 Python blocks.", @@ -2095,7 +2246,7 @@ }, { "path": "docs/source/guides/fiji_viewer_management.rst", - "source_sha256": "43f0e1f0effa40bf1a1efe18ac3efe8da0a184ad746da086b19f756f06178943", + "source_sha256": "0ab33853df56f236383b04988dce4497ec11a258059341604e62c38a27b28075", "audience": [ "viewer integrators", "pipeline authors" @@ -2105,28 +2256,42 @@ "authority": [ { "path": "openhcs/runtime/fiji_viewer_server.py", - "sha256": "cde38de711dced10b8af10b3b0ca69b1111c234a2877f0aade1e762ccbb6777b", - "role": "Implements Fiji stream reception, image-stack updates, ROI handling, and endpoint lifecycle." + "sha256": "ee246d32ed828bbf4665cc8a411e807da29bd43c2475012243f27e93cb550410", + "role": "Initializes Fiji through FIJI_IMAGEJ_RUNTIME, tracks debounced display-batch settlement, registers the run-macro action and delegates gateway shutdown." + }, + { + "path": "openhcs/runtime/viewer_control_actions.py", + "sha256": "1bbd370347a9e417fac7ffcb42ec42eae0589dc1a77286814b7a586c8318ce35", + "role": "Gives every viewer the shared shutdown, clear-state, process-launch and settle actions and answers unregistered control types with an unsupported-control error." }, { "path": "openhcs/runtime/viewer_protocol.py", - "sha256": "30441eab97da9ffcb5b9dc33d65c2ecc56c426e79bb2b27eedc78a68aaf13bc3", - "role": "Declares the transport-independent viewer messages, stream configuration, and server contract." + "sha256": "c4bdae4bae6dee412f0d27a9cf75a95db53b3bd95c03cee7d046be16b52fe5b8", + "role": "Declares ViewerSettleProgress and the viewer control message types named on the page." }, { "path": "external/zmqruntime/src/zmqruntime/__init__.py", - "sha256": "37f3e14f58b4c1dda3d338a3a2fb075aefea215f6f25b69f44091891341755ef", + "sha256": "8d6315480e060a02ca247179eef2ecae71727a4b30bea3496c3c32ad48bf3640", "role": "Exports the generic ZMQ runtime contracts used by viewer and execution endpoints." }, + { + "path": "pyproject.toml", + "sha256": "3592cc37444e2e1f5bb0284e939643039c0d513217861af8133dd8bd6e8a4325", + "role": "Declares the fiji and viz extras named in setup." + }, { "path": "tests/unit/test_fiji_viewer_server.py", - "sha256": "2d590f629243eec979531b0959a1006690befbd17b9778ec9f4fbefea72877f3", - "role": "Proves Fiji endpoint lifecycle, streamed image updates, and ROI message handling." + "sha256": "da575dfb02895ecc831b34d6bdf6b58112b986556459444e9701cbf43221ea22", + "role": "Proves Fiji endpoint lifecycle, streamed image updates, settlement and control handling." } ], - "findings": [], - "disposition": "keep", + "findings": [ + "The page said Fiji's display path is synchronous and SETTLE returns terminal progress immediately; Fiji tracks queued and active debounced display batches and SETTLE reports running, complete or failed progress through the shared settle action.", + "The per-viewer Fiji control plans for STATE and PAYLOADS were removed; Fiji registers no such action and the shared unknown-control action now answers them with an unsupported-control error. The page states that instead of naming dedicated error plans." + ], + "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guides/fiji_viewer_management.rst in full and compared it against the changed authorities: fiji_viewer_server.py, viewer_protocol.py, zmqruntime __init__.py, test_fiji_viewer_server.py, plus the new shared viewer_control_actions.py; rewrote the settlement section for batch-tracked settle progress and the unsupported STATE/PAYLOADS answer, and confirmed the extras, FIJI_IMAGEJ_RUNTIME initialization and shutdown delegation.", "Reviewed docs/source/guides/fiji_viewer_management.rst in full; the recorded findings and disposition are the resulting page-specific editorial decision.", "Compared its factual claims directly with the hash-pinned files listed in authority: openhcs/runtime/fiji_viewer_server.py; openhcs/runtime/viewer_protocol.py; external/zmqruntime/src/zmqruntime/__init__.py; tests/unit/test_fiji_viewer_server.py.", "Passed `.venv/bin/python scripts/validate_docs.py docs/source/guides`: documentation validation passed: 9 files, 2 Python blocks.", @@ -2149,35 +2314,39 @@ "authority": [ { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "ace8ae0c3e54bbe604b97cec7c3973a1abcf90e1dc517c2adda947dc2c35e882", + "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", "role": "Compiles pipeline declarations into validated step plans and execution bundles." }, { "path": "openhcs/core/memory/decorators.py", - "sha256": "d9f9cf0be7593dc0ebfc57f258aebed75bcf9cfa508ed1e942137366d819766a", + "sha256": "76f234785f4865257fd19c935e0aace82d1f5586198737fec619ee243b79cbf6", "role": "Exports OpenHCS memory and dtype decorators while delegating conversion semantics to ArrayBridge declarations." }, + { + "path": "external/arraybridge/src/arraybridge/types.py", + "sha256": "08f7ff4df8807bb3340e00a02c6276b1c702a0f00a2eedacd701142736c5d804", + "role": "Owns the input, output and execution memory attributes and framework runtimes named by the memory-integration route." + }, + { + "path": "openhcs/core/pipeline/framework_device_assignment.py", + "sha256": "cdfde10f996ab682952a1d4948d3de2d23d48353eae41f3124770e19fc28a193", + "role": "Plans framework-local worker device assignment named by the memory-integration route." + }, { "path": "openhcs/runtime/viewer_protocol.py", - "sha256": "30441eab97da9ffcb5b9dc33d65c2ecc56c426e79bb2b27eedc78a68aaf13bc3", + "sha256": "c4bdae4bae6dee412f0d27a9cf75a95db53b3bd95c03cee7d046be16b52fe5b8", "role": "Declares the transport-independent viewer messages, stream configuration, and server contract." }, { "path": "external/PolyStore/src/polystore/omero_local.py", "sha256": "35d39564317510234c2a3ded00bb06a09417d1b6f7359e52e74c7bd0a086f642", "role": "Implements the OMERO-backed PolyStore adapter, including connection and object materialisation behaviour." - }, - { - "path": "external/arraybridge/src/arraybridge/types.py", - "sha256": "7b94760d57e54008d7adf4e4d37dca45d8a2c288a9dc8400be00c26148cf7624", - "role": "Owns the callable memory metadata and framework capabilities named by the memory-integration route." } ], - "findings": [ - "The memory-integration route omitted ArrayBridge execution metadata and framework-local planning." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guides/index.rst in full and compared it against the changed authorities: compiler.py, core/memory/decorators.py, viewer_protocol.py, arraybridge types.py; the five toctree targets exist, ArrayBridge still declares input, output and execution memory attributes, framework device assignment still plans worker scope, and the viewer and OMERO routes name current owners; no correction was needed.", "Re-read docs/source/guides/index.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/pipeline/compiler.py; openhcs/core/memory/decorators.py; openhcs/runtime/viewer_protocol.py; external/PolyStore/src/polystore/omero_local.py; external/arraybridge/src/arraybridge/types.py.", "Focused cross-lattice QA passed 179 tests spanning callable contracts, framework-device scope, custom functions, catalogue discovery, installer/update policy, MCP release, and packaged OMERO deployment.", @@ -2192,7 +2361,7 @@ }, { "path": "docs/source/guides/memory_type_integration.rst", - "source_sha256": "7890c784379b0de4ecc3871068f1acd13de522077d16a7589436ab94a8019adb", + "source_sha256": "8861e967f496aa32bc9976946d593aa6e2af606de66ec2de75fb44416feb58e9", "audience": [ "callable authors", "compiler contributors" @@ -2202,52 +2371,42 @@ "authority": [ { "path": "external/arraybridge/src/arraybridge/types.py", - "sha256": "7b94760d57e54008d7adf4e4d37dca45d8a2c288a9dc8400be00c26148cf7624", - "role": "Declares framework capabilities, callable memory metadata, and the child-process import environment owned by each MemoryType member." + "sha256": "08f7ff4df8807bb3340e00a02c6276b1c702a0f00a2eedacd701142736c5d804", + "role": "Declares MemoryContractAttribute's input, output and execution keys, framework runtimes, and MemoryType.subprocess_environment()." }, { "path": "openhcs/constants/constants.py", - "sha256": "4cd6c29abd8b8e3ecc396b424d2b8b61e819a215fc1d770d3f6852b260a0691d", - "role": "Provides the legacy OpenHCS constants import surface as identity-preserving compatibility re-exports." + "sha256": "83190e34eaa0f1ae690e10aac3fb4cdcd3929ee337841bc8f97e2e304e8feab0", + "role": "Re-exports arraybridge.MemoryType as openhcs.constants.MemoryType without a translation." }, { "path": "openhcs/core/memory/__init__.py", - "sha256": "9d5223bda2ace85b6dfb91b0d510fafcbfe6fc19c99f6e2f9c4325dc0d295f47", - "role": "Provides the OpenHCS memory integration import surface over ArrayBridge." + "sha256": "94584daee1b57afe94a2b143901cc156cc947c44ce654fc8e52882e3ed7f464f", + "role": "Re-exports arraybridge.MemoryType as openhcs.core.memory.MemoryType." }, { "path": "openhcs/core/memory/decorators.py", - "sha256": "d9f9cf0be7593dc0ebfc57f258aebed75bcf9cfa508ed1e942137366d819766a", - "role": "Exports OpenHCS memory and dtype decorators while delegating conversion semantics to ArrayBridge declarations." + "sha256": "76f234785f4865257fd19c935e0aace82d1f5586198737fec619ee243b79cbf6", + "role": "Exports the OpenHCS memory decorators used in the example while delegating conversion semantics to ArrayBridge." }, { - "path": "openhcs/core/callable_contract.py", - "sha256": "6a39b5e171901131ecb7280a78241e8ea72409a5003eabba3db34eb4851991d7", - "role": "Derives callable input, output, execution-framework, dimensional, artifact, and runtime requirements from declarations." - }, - { - "path": "openhcs/core/compiled_step_plan.py", - "sha256": "6d46571e6f2d069b98a92b41d711b2e20bfab9f0493cf73d3f150d5be08e7f96", - "role": "Declares typed per-step memory, conversion, framework-device assignment, and runtime decisions." - }, - { - "path": "external/arraybridge/src/arraybridge/decorators.py", - "sha256": "99823105b0ae59054df1d84013633b8b6628d0a6dc3326383199ac7c65c69bed", - "role": "Declares decorators that write independent input, output, and execution memory roles plus dtype policy." + "path": "openhcs/core/processing_contracts.py", + "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", + "role": "Declares the ProcessingContract family and Pure2DContract used in the example." }, { - "path": "tests/unit/test_callable_contract.py", - "sha256": "493f76a54f5fe024ca8f10c53b1f810b8c369626ad7141205b42ef3a78282137", - "role": "Proves callable contract derivation for dimensions, memory types, dtype policy, and outputs." + "path": "openhcs/core/callable_contract.py", + "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", + "role": "Reads input, output and execution memory types through MemoryContractAttribute." }, { - "path": "tests/unit/test_memory_type_authority.py", - "sha256": "0da705f6f5468c2bc8a866f44057c6790a2a679d80648f47c9b373ce1af64632", - "role": "Proves enum and framework-set identity across the ArrayBridge and compatibility import surfaces, including legacy pickle loading." + "path": "openhcs/core/compiled_step_plan.py", + "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", + "role": "Declares FrameworkDeviceAssignment and the typed per-step conversion decisions recorded by compilation." }, { "path": "openhcs/core/pipeline/framework_device_assignment.py", - "sha256": "f77f3fb2d30c4af8cab6210dbe294f9fb5e246dad0a6bbf88bcaf1e2b58dead7", + "sha256": "cdfde10f996ab682952a1d4948d3de2d23d48353eae41f3124770e19fc28a193", "role": "Projects each compilation's declaration-derived framework footprint into exact per-step device assignments." }, { @@ -2257,31 +2416,36 @@ }, { "path": "openhcs/core/steps/function_runtime.py", - "sha256": "b1449120bebe7cf4898c055d841dc258c7d91d976d9e5ff7396aed81ad5011e7", - "role": "Scopes a callable body through its declared execution framework during worker invocation." + "sha256": "9f99f76c18669177abad9bfa4921c031875f4f9c3079f18aa3855f444b0bbe3b", + "role": "Enters the invocation's execution device scope around the function body." }, { "path": "openhcs/core/orchestrator/worker_execution.py", - "sha256": "3c952465268e4b6c44676a853bb9c99d2d04a0490c7d398d2cdaf8b3b5f966d7", + "sha256": "23a3ef2a70378dd1bb5c413a56e36bbb3fa013142741c9b45407bfc47de8c4a4", "role": "Cleans the exact compiled framework-device footprint during worker teardown." }, { - "path": "tests/unit/test_framework_device_resolver.py", - "sha256": "ba1814ae8eae05a39b594a81b4fc0d1444bf1334574bf4efe49aa829eec43a65", - "role": "Proves framework-local device resolution, scope selection, and cleanup behaviour." + "path": "openhcs/core/config.py", + "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "role": "Declares variable_components and group_by on the step processing configuration." + }, + { + "path": "tests/unit/test_memory_type_authority.py", + "sha256": "0da705f6f5468c2bc8a866f44057c6790a2a679d80648f47c9b373ce1af64632", + "role": "Proves MemoryType identity across the ArrayBridge and re-export import paths." }, { "path": "tests/unit/test_framework_execution_scope.py", - "sha256": "888a7693d387b0d5007c9868f0b0344fa3df26620f35e40b45832f0931ef7126", + "sha256": "ff92362a71f31b4b893a3816b1fdd396ed30307f45c7084270b4929c1aaa7aa4", "role": "Proves execution memory metadata selects the runtime framework device scope independently of boundary types." } ], "findings": [ - "The explanation covered only input/output memory boundaries and the removed application-owned GPU model; it now documents execution ownership, framework-local assignment, scoping, and cleanup.", - "The integration explanation omitted the declaration-derived environment used before optional frameworks are imported in fresh child processes." + "ProcessingContract is now an ABC family with subclasses such as Pure2DContract (G5), not an enum; the paragraph now names the family and its subclass rather than describing ProcessingContract as a single value." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guides/memory_type_integration.rst in full and compared it against the changed authorities: arraybridge types.py and decorators.py, constants.py, core/memory/__init__.py and decorators.py, callable_contract.py, compiled_step_plan.py, test_callable_contract.py, framework_device_assignment.py, function_runtime.py, worker_execution.py, test_framework_execution_scope.py; confirmed the MemoryType re-exports, MemoryContractAttribute reads, FrameworkDeviceAssignment, execution device scope, subprocess_environment consumers and variable_components/group_by, and reworded the processing-contract paragraph for the subclass family.", "On 2026-08-27, re-read docs/source/guides/memory_type_integration.rst in full after callable contracts began projecting runtime-owned inputs into generic form exclusions; challenged the ArrayBridge/OpenHCS ownership boundary and found no prose drift.", "Compared every factual claim directly with the current hash-pinned authorities: external/arraybridge/src/arraybridge/types.py; openhcs/constants/constants.py; openhcs/core/memory/__init__.py; openhcs/core/memory/decorators.py; openhcs/core/callable_contract.py; openhcs/core/compiled_step_plan.py; external/arraybridge/src/arraybridge/decorators.py; tests/unit/test_callable_contract.py; tests/unit/test_memory_type_authority.py; openhcs/core/pipeline/framework_device_assignment.py; openhcs/core/framework_device_resolver.py; openhcs/core/steps/function_runtime.py; openhcs/core/orchestrator/worker_execution.py; tests/unit/test_framework_device_resolver.py; tests/unit/test_framework_execution_scope.py.", "Focused cross-lattice QA passed 179 tests spanning callable contracts, framework-device scope, custom functions, catalogue discovery, installer/update policy, MCP release, and packaged OMERO deployment.", @@ -2296,7 +2460,7 @@ }, { "path": "docs/source/guides/omero_integration.rst", - "source_sha256": "b900bea91d2eaecb90d593786b213830c6fd689b6cbc09c25f9b0c7ab2a6183e", + "source_sha256": "9ce2fd0bdbac524d6db0ff923ea53b8e13758d45c2bc271f7ec59edaac62424f", "audience": [ "OMERO integrators", "deployment contributors" @@ -2307,18 +2471,23 @@ { "path": "external/PolyStore/src/polystore/omero_local.py", "sha256": "35d39564317510234c2a3ded00bb06a09417d1b6f7359e52e74c7bd0a086f642", - "role": "Implements the OMERO-backed PolyStore adapter, including connection and object materialisation behaviour." + "role": "Implements OMEROLocalBackend, its virtual images_dir projection, PurePosixPath normalization and save_batch plane resolution." }, { "path": "external/PolyStore/src/polystore/omero_address.py", "sha256": "8a3ae8789fceba0f099eef2ee983d858a8f83b764f19b70dfa7bb88f521c0c92", - "role": "Owns strict concrete OMERO plane addresses and symbolic filename templates through one component grammar." + "role": "Owns OMEROPlaneAddress, OMEROPlaneFilenameTemplate and the multi-letter OMEROWellAddress." }, { "path": "external/PolyStore/src/polystore/filemanager.py", - "sha256": "877a2b10731b12904b1784a5b42c9e39688e3a84f7f86018c1c2beee96ea2296", + "sha256": "a9e9f3a8205216efc769accad06ab64d5373e6726733b2a1f2e3881613d1880e", "role": "Delegates contextual save arguments and persistence to the selected DataSink instance." }, + { + "path": "external/PolyStore/src/polystore/base.py", + "sha256": "8f2082ade15072bc8e8eba363c75a1d1cff7c67c74845e6485f69eab244634dc", + "role": "Declares the DataSink interface whose contextual save arguments generic materialization requests." + }, { "path": "external/PolyStore/src/polystore/omero_tables.py", "sha256": "910aaa6bea75f40bae82df71dd414415278b1387b80c7f1bc11e8d681874451e", @@ -2330,29 +2499,24 @@ "role": "Declares supported text extensions, MIME types, and table parsers together." }, { - "path": "openhcs/core/orchestrator/analysis_consolidation.py", - "sha256": "3ef9c20c347c1ac724353c84b24f9d7e13513aba7b9c5c01e194171b7e72bb52", - "role": "Projects execution-ledger CSV content and writes summaries through the compiled persistent backend." + "path": "openhcs/domains/microscopy/analysis_consolidation.py", + "sha256": "292f7138a00dc22dba5b8d8af5de6140d247b955723cc807cc982ffec7a7dd2d", + "role": "Consolidates execution-ledger CSV content and writes summaries through FileManager and the compiled backend." }, { - "path": "external/PolyStore/src/polystore/base.py", - "sha256": "ae641affae21b61448816a14bd68b7f2f3fc0082f0f1fbe51079ed32520450b1", - "role": "Declares the storage backend interface and lifecycle shared by local and remote materialisation backends." - }, - { - "path": "openhcs/microscopes/microscope_interfaces.py", - "sha256": "a603fa1c58ba005f8ba22713e46efe4e33aee0f762ba188467482ae3ae429200", - "role": "Declares the microscope source interface, plate dimensions, metadata, and acquisition access contract." + "path": "openhcs/core/dataset_sources/source.py", + "sha256": "097fde4d2b74031eacf53a88e79f499417e8c1c97ba0753e8a9df831b307efa6", + "role": "Declares the DatasetSource interface the OMERO source implements for workspace and metadata access." }, { "path": "openhcs/microscopes/omero.py", - "sha256": "341733257cfb4c3bbe35dbc311c60e1666d357af6c6b7dc416399f77a92c570e", - "role": "Projects PolyStore's concrete and symbolic OMERO filename boundaries into OpenHCS components." + "sha256": "a3ab5a257490f729e96ba413c3f11d12f10f68dae44a728206f81b850325da31", + "role": "Declares OMEROHandler and OMEROFilenameParser, which map PolyStore's concrete and symbolic OMERO filenames into FilenameParseResult." }, { "path": "tests/unit/test_omero_microscope_metadata.py", - "sha256": "066e612d760bb5d0a75c5301cc6fb2d39ef460f1d7e84c67679505bec11ec7b9", - "role": "Proves microscope metadata plus concrete and symbolic OMERO pattern projection through the common interfaces." + "sha256": "55adf5ec17e7e717c8b4a70828736743c878599ee3f70b55430bbd138316ea8f", + "role": "Proves OMERO metadata plus concrete and symbolic pattern projection." }, { "path": "openhcs/omero/plugin/README.md", @@ -2367,11 +2531,11 @@ { "path": "openhcs/runtime/omero_instance_manager.py", "sha256": "af94c8b851bd4e1456e5c85b16f83ac8008944b05a71ed9d9db5574416704933", - "role": "Owns OpenHCS instance lifecycle, packaged-only table-component recovery, exact Compose invocation, and complete readiness acceptance." + "role": "Owns OMEROInstanceManager lifecycle, derived local defaults, remote overrides, bounded Docker wait and packaged table-component recovery." }, { "path": "pyproject.toml", - "sha256": "6217b5f029796edcd83aa47fb8632d37aa7552344755c578fa4f181a6b23282d", + "sha256": "3592cc37444e2e1f5bb0284e939643039c0d513217861af8133dd8bd6e8a4325", "role": "Includes the complete openhcs/omero deployment and plugin bundle in installed distributions." }, { @@ -2382,39 +2546,21 @@ { "path": "tests/unit/test_omero_instance_manager.py", "sha256": "c9341221f85069ebb465e67f28f240191a94bc690c2999d9de36212505dccda1", - "role": "Proves canonical compose resolution, pinned images, packaged table recovery, external lifecycle preservation, and fail-closed context entry." - }, - { - "path": "tests/unit/test_validate_wheel_deployment.py", - "sha256": "7909c30f7afc321f939aebf52c5c603e153cfa13e869492e0538f78dc632e145", - "role": "Proves packaged deployment completeness and rejects parallel compose authorities, nested build output, nested package metadata, or developer-home paths." - }, - { - "path": "tests/integration/test_main.py", - "sha256": "1a0c532eb631a89b77f92a4202a7b55033c86d7b872350156f106d276b061e9f", - "role": "Exercises live OMERO instance and integration behaviour through the packaged OpenHCS boundary." + "role": "Proves compose resolution, pinned images, packaged table recovery, external lifecycle preservation, and fail-closed context entry." }, { "path": "openhcs/agent/ui_bridge_identities.py", - "sha256": "5913d14b99340174af87b2a7cafa3cdd080a331553c0cf59392ade7d253f1234", - "role": "Declares the complete stable desktop-window family used to verify that the current UI exposes no OMERO management or credential surface." + "sha256": "66dcf838d08cac2004ab8f73db777f73270cc520697281cf4f94a5fc6af4dc64", + "role": "Declares the desktop-window family, which contains no OMERO management or credential window." } ], "findings": [ - "The page mirrored volatile alpha and legacy-client status from another package; it now delegates release compatibility to that owner and states only the OpenHCS declaration gate.", - "The explanation treated omero_openhcs as a repository-root deployment project; it now distinguishes the bundled web application at openhcs/omero/plugin from the installed OpenHCS deployment and instance-lifecycle owner.", - "A wheel built from a developer tree could absorb ignored OMERO build output and nested package metadata; package-data selection now names the canonical bundle, build output is reprojected from scratch, and the installed-wheel gate seeds both contamination forms.", - "The explanation did not state that runtime analysis summaries consume execution-ledger content and persist through the compiled backend instead of reopening OMERO virtual paths as local files.", - "Two compose declarations and hardcoded shell probes competed with the runtime lifecycle owner; the bundle now ships one pinned compose file and the wheel gate rejects parallel declarations.", - "The default compose stack exposed an alpha panel that cannot submit the current execution contract; the upstream OMERO.web viewer remains available while the panel stays behind its owner-package integration gate.", - "The manager mirrored local connection defaults from Compose; typed settings now project the canonical declaration and explicit values remain remote-instance overrides.", - "Generic platform-string dispatch attempted to start Docker Desktop or a host service manager; the integration boundary now requires host-owned Docker availability.", - "Pattern discovery passed symbolic fields through the strict concrete plane-address type; PolyStore now owns a separate template identity without weakening address validation.", - "A one-shot Docker daemon probe made otherwise valid cold local deployments fail transiently; the instance manager now waits boundedly without taking ownership of daemon startup.", - "A cold packaged deployment could leave Tables-0 disabled after it started before the managed repository; the explanation now distinguishes PolyStore readiness from OpenHCS-owned local recovery." + "Two authorities were deleted by G4: openhcs/microscopes/microscope_interfaces.py (now the DatasetSource family in openhcs/core/dataset_sources/source.py) and openhcs/core/orchestrator/analysis_consolidation.py (now openhcs/domains/microscopy/analysis_consolidation.py); the page's claims hold against the new owners.", + "The PolyStore paragraph called the OMERO addresses 'canonical'; the word was dropped." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guides/omero_integration.rst in full and compared it against the changed authorities: PolyStore filemanager.py and base.py, microscopes/omero.py, test_omero_microscope_metadata.py, pyproject.toml, tests/integration/test_main.py, ui_bridge_identities.py, plus the moved analysis consolidation and dataset-source owners; confirmed OMEROFilenameParser, FilenameParseResult, the PolyStore address types, DataSink contextual saves, ledger-based consolidation through FileManager and OMEROInstanceManager, and dropped 'canonical'.", "On 2026-08-27, re-read docs/source/guides/omero_integration.rst in full after the python-introspect dependency floor advanced to 0.1.12; challenged its OMERO ownership explanation and found no affected claim.", "Compared every factual claim directly with the current hash-pinned authorities: external/PolyStore/src/polystore/omero_local.py; external/PolyStore/src/polystore/base.py; external/PolyStore/src/polystore/omero_tables.py; openhcs/microscopes/microscope_interfaces.py; tests/unit/test_omero_microscope_metadata.py; openhcs/omero/plugin/README.md; openhcs/omero/docker-compose.yml; openhcs/runtime/omero_instance_manager.py; pyproject.toml; scripts/validate_wheel_deployment.py; tests/unit/test_omero_instance_manager.py; tests/unit/test_validate_wheel_deployment.py; tests/integration/test_main.py.", "Focused cross-lattice QA passed 179 tests spanning callable contracts, framework-device scope, custom functions, catalogue discovery, installer/update policy, MCP release, and packaged OMERO deployment.", @@ -2451,7 +2597,7 @@ }, { "path": "docs/source/guides/pipeline_compilation_workflow.rst", - "source_sha256": "051f6d70831eb9939375063651d825604695186de950275b512f5bd8c64a6293", + "source_sha256": "c74f1449defb5d46a2537850d5cf7f0d3183bcea11be0f85883f29fba3cbf97f", "audience": [ "pipeline integrators", "compiler contributors" @@ -2461,38 +2607,46 @@ "authority": [ { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "ace8ae0c3e54bbe604b97cec7c3973a1abcf90e1dc517c2adda947dc2c35e882", - "role": "Compiles pipeline declarations into validated step plans and execution bundles." + "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "role": "Defines compile_pipelines over an orchestrator and list[AbstractStep] and returns a CompiledExecutionBundle." + }, + { + "path": "openhcs/core/orchestrator/orchestrator.py", + "sha256": "a61658b107f170c192eea505966c268ab6826826f03586bd64562a44800e1ccc", + "role": "Supplies the initialized orchestrator, its DatasetSource workspace and selected partition values to compilation." }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "3c840a89baccc8665d415175458280fd904c1a837b7fd879e303d7dbca129a1c", - "role": "Owns compilation-scoped state and resolves declarations into frozen plans without leaking runtime values." + "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", + "role": "Declares CompilationSession, which owns cross-step compile state." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "6d46571e6f2d069b98a92b41d711b2e20bfab9f0493cf73d3f150d5be08e7f96", - "role": "Declares typed per-step memory, conversion, framework-device assignment, and runtime decisions." + "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", + "role": "Declares CompiledStepPlan and its typed per-step memory, conversion, device and runtime fields." }, { "path": "openhcs/core/compiled_execution.py", - "sha256": "941115f2dca2e53538d3141a59f3ae1f928a87cf741fcfa177f7cd64a5f41aea", - "role": "Declares the frozen compiled execution bundle consumed by workers after compilation." - }, - { - "path": "tests/unit/test_compiled_execution.py", - "sha256": "a7ef4c3f011e0770d8e7a48928aa25c6ca550095c1f80146a25170d449ecc31f", - "role": "Proves compilation produces immutable step plans and execution bundles with the expected resolved values." + "sha256": "85972d418f0958e59f3185f07caf25a103bc9e101e481b2b63903377cc05e397", + "role": "Declares the frozen CompiledExecutionBundle consumed by workers after compilation." }, { "path": "openhcs/core/pipeline/framework_device_assignment.py", - "sha256": "f77f3fb2d30c4af8cab6210dbe294f9fb5e246dad0a6bbf88bcaf1e2b58dead7", + "sha256": "cdfde10f996ab682952a1d4948d3de2d23d48353eae41f3124770e19fc28a193", "role": "Owns declaration-derived framework device assignment during compilation." + }, + { + "path": "tests/unit/test_compiled_execution.py", + "sha256": "cb6c7034e9534470daf99a471b27a9ae662b5d2828f01af8613534412e8eea81", + "role": "Proves compilation produces immutable step plans and execution bundles with the expected resolved values." } ], - "findings": [], - "disposition": "keep", + "findings": [ + "The inputs list named a 'microscope/source workspace'; after G4 the orchestrator holds a DatasetSource, so the item now says dataset-source workspace." + ], + "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guides/pipeline_compilation_workflow.rst in full and compared it against the changed authorities: compiler.py, compilation_session.py, compiled_step_plan.py, compiled_execution.py, test_compiled_execution.py, framework_device_assignment.py; confirmed CompilationSession, CompiledStepPlan, CompiledExecutionBundle and the compile_pipelines inputs, and renamed the microscope/source workspace to the dataset-source workspace.", "Re-read docs/source/guides/pipeline_compilation_workflow.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/pipeline/compiler.py; openhcs/core/pipeline/compilation_session.py; openhcs/core/compiled_step_plan.py; openhcs/core/compiled_execution.py; tests/unit/test_compiled_execution.py; openhcs/core/pipeline/framework_device_assignment.py.", "Focused cross-lattice QA passed 179 tests spanning callable contracts, framework-device scope, custom functions, catalogue discovery, installer/update policy, MCP release, and packaged OMERO deployment.", @@ -2504,7 +2658,7 @@ }, { "path": "docs/source/guides/testing_guide.rst", - "source_sha256": "82d14d7b3163bd6c7313922d7509cd80325e1021d83e80e50aa8775ac902e59b", + "source_sha256": "7e1836870d03c13b0bc04c38e699e09b4ffbc5ae485667ac76f262f6e3187241", "audience": [ "contributors", "integration-test operators" @@ -2514,7 +2668,7 @@ "authority": [ { "path": "pytest.ini", - "sha256": "abeb2cd3d7ac83c5988690e5b991c0989a96b46d62febc48ba1c7664b6a30d1f", + "sha256": "d00f34c3bddc54800f38769af84f43dc2e91497fe7a7891eb06706bb257326be", "role": "Declares the repository's pytest markers, collection rules, and default test configuration." }, { @@ -2524,12 +2678,12 @@ }, { "path": ".github/workflows/integration-tests.yml", - "sha256": "c1ec94eab493bc46986c55b8b1428d9a0a5c0eaf5789f2dddbc06df2c0843ff7", - "role": "Defines the operating-system integration matrix, dependency setup, installed-wheel live GUI/MCP operation, retained application and three-stage installer screenshot artifacts, pytest commands, and source-gate coverage artifacts used by CI." + "sha256": "5017eb98536006a0cec4f34573a35c1a547c3ad89889e276e88a8fb6f992991c", + "role": "Defines the operating-system integration matrix, the Python 3.14 core job, the scoped four-platform Official30 parity lane and its required check, installed-wheel GUI/MCP operation, installer screenshot artifacts, and source-gate coverage artifacts used by CI." }, { "path": "scripts/resolve_successful_job_baseline.py", - "sha256": "8e6541a53dac376176a63cb449e9166a425ef673954847cccd96fe444d60ad48", + "sha256": "be32836149414a8edf9ea73e1316907847376c86156b758b786b21fbd96a3283", "role": "Resolves the cumulative changed-Python boundary from the latest successful code-quality job on an ancestor workflow run." }, { @@ -2544,7 +2698,7 @@ }, { "path": "tests/installer/test_windows_simple_installer.py", - "sha256": "3ab5b37fb4deee277f9deee41dc61deab5235e245e5ff37c98f32b06fde65697", + "sha256": "c83a5de13f423f7fd83624cabe38dec2901cb4ccfdc367cbca8e7c0460cbe6e7", "role": "Proves the Windows native wizard waits for rendered controls, synchronizes window composition, and retains each declared installer evidence state." }, { @@ -2564,8 +2718,8 @@ }, { "path": ".github/workflows/website-pages.yml", - "sha256": "94fd5634ace8d7f138446a120acb49de27d0ea7230b09b1e958ef8b8fcd85b37", - "role": "Validates gallery generation against metadata-resolved source candidates before deploying the checked-in dependency-free website projection." + "sha256": "577e0747692e7738bdb22f1b579c08984d6dffa8e709e0e33ed9edf4110b3be6", + "role": "Installs the OpenHCS candidate from submodule sources with install_ci_candidate before validating gallery generation, then deploys the checked-in dependency-free website projection." }, { "path": "tests/unit/test_ci_action_pinning.py", @@ -2577,6 +2731,11 @@ "sha256": "840eb6612bb41758979104cab1165d7e2297011fc6ea907ab67b9112f349b3e2", "role": "Builds the metadata-discovered first-party source wheelhouse or installs the hash-pinned public dependency projection selected by each CI evidence lane." }, + { + "path": "scripts/run_installed_tests.py", + "sha256": "db4991c7d7fa1cb1cfd5918525f26b42fce799fb4f8136d8740001a9e8ac6b56", + "role": "Runs the checkout's tests with checkout source paths removed and fails when OpenHCS would import from the checkout instead of the installed wheel." + }, { "path": "tests/unit/test_install_ci_candidate.py", "sha256": "43b636293b30767c8040d7da77d440090089d097df75dfcf0649efc07fcddcb0", @@ -2594,22 +2753,22 @@ }, { "path": "tests/unit/test_ci_package_boundaries.py", - "sha256": "691efcdafe66255c55ae11411b441930bdb58161cd74e2580c2531a486863303", + "sha256": "df85419ab722292f54a520bf61bd1a11ecc08b7637ffa2abdf70a16f16088e04", "role": "Proves source-gate coverage fails closed, website validation uses local candidate owners, and cross-platform candidate and published GUI canaries retain their typed screenshot evidence." }, { "path": "scripts/smoke_installed_desktop.py", - "sha256": "f745f93e245e5c1b02f6e0ece3a4a0e2af086c3cab77282e3907a167f5ce0a7f", + "sha256": "d5ce5e2a2b3590be2101b487b2957b270f386dcf47b4c197f1ce4c2ab6aa9d65", "role": "Runs installed GUI readiness and screenshot evidence, launch, MCP/runtime, viewer, and packaged restart-worker acceptance outside the source checkout while deriving the outer GUI process ceiling from the probe's timing authority." }, { "path": "tests/installer/test_installed_desktop_smoke.py", - "sha256": "5ac9f0e9e0e2ffe73cbab940da132421424e355c354b7476a8a072090c5aad00", + "sha256": "bf6fa5db1ff253c4ee3df16b3d46bdc64e87d54171770287baeed3516d44bb05", "role": "Proves platform smoke orchestration passes the exact installed Python, isolated runtime, evidence directory, and declaration-derived GUI timing to the installed verifiers." }, { "path": "scripts/smoke_installed_gui.py", - "sha256": "a5af7f55eab45b7ab43772a9ff8cf758e4940c5ad58857611cc51dbe6004bd89", + "sha256": "d9bf24b9e5c631b4eb6a3de0fc7a79a5ee56a848693ed6692b60dc9c570d4efc", "role": "Owns the installed-GUI acceptance timing and single terminal completion, applies the declared logging configuration, constructs the application on declaration-allocated endpoints, proves the background function catalogue and advertised endpoint log are usable, drives the GUI through packaged MCP, validates retained window snapshots, and cleans up exact owned resources outside the checkout." }, { @@ -2624,13 +2783,13 @@ }, { "path": "openhcs/core/xdg_paths.py", - "sha256": "c0ecf7576243107dbd1a76fd74f1162ab31cb98634e458294b0210cf88774513", + "sha256": "d015a57a46b0052a717d03febe00065041200c7adf079639aa1ac887c24cf45c", "role": "Resolves OpenHCS data, cache, state, and log locations from the platform and XDG environment." }, { "path": "tests/integration/conftest.py", - "sha256": "7397b869b835c535a8707412e35f5f4b25aa92eec5b7fe9300c83dbce0bc987a", - "role": "Composes the integration parameter axes and parametrises collected integration cases." + "sha256": "d618490a1a01c26190e264e311e188b4c21ffbbaeb42b8173c9948602052d664", + "role": "Composes the integration parameter axes, parametrises collected integration cases, and allocates per-test endpoint ports." }, { "path": ".vscode/settings.json", @@ -2649,30 +2808,12 @@ } ], "findings": [ - "Failure guidance named an unsupported ~/.local/state log tree and implied Unix-only paths; it now follows the configured log owner and launch-reported paths.", - "The guide attributed integration composition to the root conftest instead of tests/integration/conftest.py and omitted the authority for the VS Code OMERO argument.", - "The Windows installer section now records the dual-lock staged-update smoke and the subsequent deferred launcher refresh.", - "The CI section omitted the maintained GUI source gate and described live Napari/Fiji validation as only local or ad hoc despite the dedicated viewer smoke.", - "The changed-Python quality gate could forget a failed file on the next push; the guide now records the cumulative successful-job ancestor boundary.", - "The combined report omitted the foundational unit/core and maintained GUI source gates, and its description implied coverage from uninstrumented jobs; both source gates now emit fail-closed artifacts and the guide names the report's exact evidence boundary.", - "The native installer section did not record that each supported Windows/macOS architecture now exercises the packaged restart worker against a real parent exit and detached command.", - "The generated coverage README rendered its update command literally instead of recording when the report was produced; the workflow now appends an ISO 8601 UTC timestamp.", - "Native installer acceptance did not exercise hostile workstation pip index configuration; Windows installation and reinstall, macOS installation, and the staged updater now run while an unreachable index is injected.", - "The release matrix proved dependency resolution and importability but did not construct the installed GUI, and no cross-platform canary re-resolved a published release after publication; both boundaries now exercise painted application readiness.", - "The OMERO failure guide did not describe the cold repository race exposed by Python 3.12 CI or the boundary that prevents OpenHCS from restarting externally managed components.", - "The installed-GUI probe reused the default execution endpoint, allowing an existing desktop server to influence an otherwise isolated package canary; it now allocates the endpoint through the selected transport declaration.", - "Installed GUI and MCP package checks terminated before either operated the other; the source-wheel, publication, native-installer, and scheduled canaries now keep the packaged GUI live while the packaged desktop MCP discovers it and invokes a declared UI action.", - "The fixed desktop candidate was operated before publication only on Linux, while Windows and macOS evidence remained behind the unpublished-dependency gate; a complementary source-candidate matrix now proves Windows and both macOS architectures alongside the Linux wheel lane.", - "The native installer source job still depended on public first-party release tags and therefore skipped the real Windows and macOS installer paths while dependency candidates were unpublished; it now builds the complete metadata-discovered source wheelhouse independently, while the PyPI installation lane remains publication-gated.", - "Cross-platform GUI canaries proved readiness and MCP actions but discarded the rendered UI, so native theming regressions had no retained evidence; candidate and published-package jobs now upload typed, hash-verified window snapshots.", - "The installed-GUI canary bypassed the production logging setup and therefore could pass while its execution server advertised an empty log; the probe now applies LoggingConfig and verifies readable endpoint output.", - "Native installer tests exercised worker mode without proving that the visible applications could start or drive installation; each supported lane now activates the shipping wizard and retains its welcome, progress, and completion states.", - "The Windows installer evidence harness captured its welcome frame when only the top-level handle existed, before child controls painted; capture now requires the real primary control and synchronizes recursive rendering with desktop composition.", - "Website validation attempted to install an unpublished first-party dependency from PyPI; it now resolves the required source candidates from the recursively recorded submodules while deployment continues to consume the dependency-free checked-in projection.", - "Integration and Documentation workflows retained superseded same-ref matrices even though only exact-head evidence can support release readiness; both now declare event-and-ref concurrency." + "The CI section did not mention the Official30 headless parity lane, which now runs on Python 3.12 Linux and Python 3.14 Linux, Windows and macOS behind a path-scope check whose skip a required job accepts only for proven-irrelevant pull requests.", + "The CI section did not say that test jobs now run the checkout's tests against the installed candidate wheel through scripts/run_installed_tests.py; one paragraph now states it." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guides/testing_guide.rst in full and compared it against the changed authorities: pytest.ini, integration-tests.yml, resolve_successful_job_baseline.py, test_windows_simple_installer.py, website-pages.yml, test_ci_package_boundaries.py, smoke_installed_desktop.py, test_installed_desktop_smoke.py, smoke_installed_gui.py, xdg_paths.py, tests/integration/conftest.py; confirmed the default discovery options, --it-* flags, artifact names, concurrency, actionlint, code-action probe and log-directory paths, and added the scoped Official30 parity lane and the installed-wheel test runner.", "Re-read docs/source/guides/testing_guide.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: pytest.ini; tests/pytest_integration_options.py; .github/workflows/integration-tests.yml; scripts/install_ci_candidate.py; .github/workflows/coverage-pages.yml; tests/unit/test_ci_package_boundaries.py; scripts/smoke_installed_desktop.py; tests/installer/test_installed_desktop_smoke.py; openhcs/pyqt_gui/config.py; openhcs/core/xdg_paths.py; tests/integration/conftest.py; .vscode/settings.json.", "Focused cross-lattice QA passed 179 tests spanning callable contracts, framework-device scope, custom functions, catalogue discovery, installer/update policy, MCP release, and packaged OMERO deployment.", @@ -2739,7 +2880,7 @@ }, { "path": "docs/source/guides/viewer_management.rst", - "source_sha256": "cf6b38318a5464e6b0d30394f7a5c3bed5b3ca8af1a336486a9d4962bc104996", + "source_sha256": "d9187f4ee21b5aa5beacd73f2cb372fe314a420d0e791a9a3546c16defdd5000", "audience": [ "viewer integrators", "pipeline authors" @@ -2747,10 +2888,15 @@ "user_need": "Understand viewer lifecycle, ROI ownership, settlement, and artefact boundaries.", "diataxis": "explanation", "authority": [ + { + "path": "openhcs/core/config.py", + "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "role": "Declares StreamingConfig, NapariStreamingConfig, FijiStreamingConfig and StepMaterializationConfig, including the partition filter inherited by streaming configuration." + }, { "path": "openhcs/runtime/napari_viewer_server.py", - "sha256": "64ccfef65a10e04878ef0806ae72d0446540efcbfc72af46c4e2ab785418a84c", - "role": "Implements Napari stream reception, layer updates, ROI navigation, and endpoint lifecycle." + "sha256": "038d7a76eeac8d69079591cf6eb7736af5415b28a7a067d6677bd15d115d11c7", + "role": "Mounts the ROI Manager, owns the OpenHCS ROI selection toolbar and queued-route settlement, and applies layer isolation." }, { "path": "openhcs/napari_roi_manager/__init__.py", @@ -2759,54 +2905,72 @@ }, { "path": "openhcs/runtime/viewer_protocol.py", - "sha256": "30441eab97da9ffcb5b9dc33d65c2ecc56c426e79bb2b27eedc78a68aaf13bc3", - "role": "Declares the transport-independent viewer messages, stream configuration, and server contract." + "sha256": "c4bdae4bae6dee412f0d27a9cf75a95db53b3bd95c03cee7d046be16b52fe5b8", + "role": "Declares the SHAPES streaming data type and the settle progress fields." + }, + { + "path": "openhcs/runtime/viewer_control_actions.py", + "sha256": "1bbd370347a9e417fac7ffcb42ec42eae0589dc1a77286814b7a586c8318ce35", + "role": "Answers settle requests on the control transport thread without waiting for Qt." }, { "path": "openhcs/runtime/viewer_controls.py", - "sha256": "3227e6bf60e9b43fd792cc4299b6b50721741f012a0f7f856268b35022fd6062", - "role": "Declares caller payload controls, response-wide shape projection policy, and one atomic layer-isolation request." + "sha256": "df0ac0aee66ba52a8b1706a1a99e03acc1300cfaf5cc74c2f93614a1c77465f6", + "role": "Declares caller payload controls, response-wide shape projection policy, and the atomic layer-isolation request." }, { "path": "openhcs/agent/services/viewer_window_service.py", - "sha256": "257718ca7da768132be2283326730f65d9b399f0ef852b2c7e1e9e48263f9eed", - "role": "Projects bounded ROI metadata and one bulk isolation response into viewer-agent results." + "sha256": "cfe222586a221cb317099fc245216778f2ce15589604e5ce03410998dec4bb1a", + "role": "Projects bounded ROI metadata under one max_rois budget and one bulk isolation response into viewer-agent results." }, { - "path": "openhcs/napari.yaml", - "sha256": "48001da52bd77a2a436b021676c7d68a994b0d2a708d7ac6647f1fe0c8d8204a", - "role": "Declares the Napari plugin contribution and dock-widget entry point installed with OpenHCS." + "path": "openhcs/agent/authoring_contexts.py", + "sha256": "a714fab8481102a5ab7800a465fcbdf7267d1ed4468cb3803b75e03f2f29ff33", + "role": "Targets the openhcs_viewer_qa knowledge document for viewer review." }, { - "path": "tests/unit/test_napari_roi_manager_integration.py", - "sha256": "de66b3d7aeb49b100ac5bb3c3acdf1c14afc19fb3681071b4356f6d7949a52dc", - "role": "Proves the packaged Napari ROI manager is discovered and integrated with the viewer contract." + "path": "packaging/codex/openhcs/skills/use-openhcs/references/viewer-qa.md", + "sha256": "6aff1f00a2cfb11a658850831c511298010339acba99c080cabed437c544e28b", + "role": "Bundles the viewer QA selective-retirement procedure the page defers to." }, { - "path": "tests/unit/test_napari_streaming_handlers.py", - "sha256": "a338d6f2ef3094fa429f3d640709c7231eb741a7bf2859318bf4bbfa196afc07", - "role": "Proves Napari streaming updates layers and carries ROI navigation through the declared protocol." + "path": "openhcs/processing/materialization/options.py", + "sha256": "271346eda98bfcabe2614696e1d2499327b84dcd8245f3ad61025cc18ddcae0f", + "role": "Declares SWCOptions and SpatialGraphROIOptions." }, { - "path": "tests/unit/test_viewer_result_element_selection.py", - "sha256": "58480f4da61b427ec1c55560a77686e74df70404fbbf77e701ba386ad9f89a03", - "role": "Proves isolation preflights every requested route and selected-route axis before changing live Napari state." + "path": "openhcs/core/artifacts.py", + "sha256": "65eb0ec5c903366243ca8bd5ea07412e8d34bdb81e6015fa76b3c4c59838733f", + "role": "Declares SpatialGraphArtifactType." + }, + { + "path": "openhcs/napari.yaml", + "sha256": "a42e9d6288fd43d0855154e415eee0b998d120934754759c480e529c30bacafe", + "role": "Declares the ROI Manager widget and SWC reader contributed under Plugins > OpenHCS." }, { "path": "openhcs/runtime/napari_streaming_handlers.py", - "sha256": "86f4ab319c7c68b626e7e2a1adef81eb3d7244f044b5b3e4dddd78bbee2aa4b6", + "sha256": "be585484075783f7717776d2b8ba163aab8a69aa24c560ca3844214485bd9eef", "role": "Owns semantic display-axis alignment for images, points, shapes, translations, and exact-slice navigation." + }, + { + "path": "tests/unit/test_napari_roi_manager_integration.py", + "sha256": "a4b7dee556f0ceccaa212751e29eb8160d3d0c08c741760715ee8372b0c0e927", + "role": "Proves the packaged Napari ROI manager is discovered and integrated with the viewer contract." + }, + { + "path": "tests/unit/test_viewer_result_element_selection.py", + "sha256": "b1dae580f0291f53be9453df6f6c6df5458046bd3ff947b550f7ddea94b29f96", + "role": "Proves isolation preflights every requested route and selected-route axis before changing live Napari state." } ], "findings": [ - "The viewer explanation did not state that every route shares the display declaration's axis slots or that reduced components occupy singleton slots.", - "The viewer explanation did not state that the selected OpenHCS route owns dimension-label presentation or that non-OpenHCS selection clears stale route state.", - "The persistence explanation treated execution-server teardown as owning persistent viewers, which contradicted their post-run reuse and inspection contract.", - "The explanation did not distinguish bounded metadata-only ROI summaries from explicit raw geometry inspection.", - "The explanation did not state that visibility, selection, and selected-route navigation are validated and applied as one viewer operation." + "The streaming-config entry said well selection belongs to streaming configuration; after G1 the field is the partition filter, so the entry now says partition (well) selection.", + "The viewer QA paragraph used 'scene/payload custody'; reworded as retained scene and payload state." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guides/viewer_management.rst in full and compared it against the changed authorities: napari_viewer_server.py, viewer_protocol.py, viewer_controls.py, viewer_window_service.py, napari.yaml, the ROI manager, streaming handler and element-selection tests, and napari_streaming_handlers.py; confirmed the streaming and materialization config classes, SHAPES boundary, ROI toolbar labels, plugin menu entries, license file, spatial-graph options, max_rois, layer isolation and transport-thread settle, renamed well selection to partition selection, and replaced 'custody'.", "Reviewed docs/source/guides/viewer_management.rst in full; the recorded findings and disposition are the resulting page-specific editorial decision.", "Compared its factual claims directly with the hash-pinned files listed in authority: openhcs/runtime/napari_viewer_server.py; openhcs/napari_roi_manager/__init__.py; openhcs/runtime/viewer_protocol.py; openhcs/napari.yaml; tests/unit/test_napari_roi_manager_integration.py; tests/unit/test_napari_streaming_handlers.py.", "Passed `.venv/bin/python scripts/validate_docs.py docs/source/guides`: documentation validation passed: 9 files, 2 Python blocks.", @@ -2833,33 +2997,34 @@ "authority": [ { "path": "openhcs/core/pipeline_document.py", - "sha256": "0849800b3e6474b8176fe35bc4aaebc63cfa85bd77251000c50da5f4c928b1b2", - "role": "Declares the public pipeline document, step collection, and serialisable declaration boundary." + "sha256": "8c554c8275c665f308eb098955bda517aa804eeb34418b5e0f41d412650ce7b5", + "role": "Declares PipelineDocument and PipelineDocumentCodec, the one pipeline config-plus-steps document shared by the GUI, Python and agents." }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "a156a006a3c40f43107b9b1f6ecb97ed150879eb88ded8ef89e25214309ae33a", + "sha256": "e43085c1eaf0c836ce34e1a1dae7740488d91bdfea56844e7ddd4e6925561cf6", "role": "Composes the main desktop window, menus, task windows, and registered UI services." }, { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "b547c838b2c86fe7cc5fe12d6cbf6a6715e83055c2d8c1a109ac22566d22c51e", + "sha256": "3439d96f3fbfcb104a53019dc150049021d9a8132bd27096f1ccf510eefbe37b", "role": "Imports CellProfiler modules into ordinary OpenHCS pipeline declarations through the interop boundary." }, { "path": "openhcs/mcp/bootstrap.py", - "sha256": "64b2ef9a4f4fe50154b4c921a7617b2e7c9dcf8bb475df7423e37e8395a8b250", + "sha256": "0e8e94cf54a32a1772e52a77d1c89a9ee02c6d2a58eb04f6e18fad2ce79d0394", "role": "Builds the installed MCP server, registers its tools and resources, and starts the selected transport." }, { "path": "scripts/gallery_catalog.py", - "sha256": "7f493e0cb305e14ee4cf7d3e995c67df4200186a8a8aaf00eef0bc66d8d191cf", - "role": "Owns the landing-page workspace scenario, representative image, caption, description, alt text, dimensions, and release evidence." + "sha256": "7fd3fce2b8fdae1abf049bb4f8a3a310ab598a7f2c507578425be537903ccfca", + "role": "Owns the multi-plate-overview landing-page scenario, representative image, caption, description, alt text, dimensions, and release evidence." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/index.rst in full and compared it against the changed authorities: pipeline_document.py (PipelineDocumentAuthority renamed PipelineDocumentCodec), pyqt_gui/main.py, cellprofiler pipeline_import.py, mcp/bootstrap.py, gallery_catalog.py; the page names none of the renamed symbols, the multi-plate-overview scenario still exists, and the routing claims and toctree remain accurate; no correction was needed.", "On 2026-08-27, re-read docs/source/index.rst in full against the explicit CellProfiler import-validation hook; challenged its Diataxis routing and supported-import overview and found no prose drift.", "Compared its factual claims directly with the hash-pinned files listed in authority: openhcs/core/pipeline_document.py; openhcs/pyqt_gui/main.py; openhcs/interop/cellprofiler/pipeline_import.py; openhcs/mcp/bootstrap.py.", "Passed `.venv/bin/python scripts/validate_docs.py docs/source/index.rst`: documentation validation passed: 1 files, 0 Python blocks.", @@ -2875,7 +3040,7 @@ }, { "path": "docs/source/installation.rst", - "source_sha256": "05f5519d47c42adea4d67ef1e70d0eebab7819f0ea4fe66b293a5c78eb34f9ca", + "source_sha256": "67df29d4cb62e1bbef72a1a91d535c83fcd0a32f7f83c808f89cc8a857c09f1c", "audience": [ "returning users following an old installation link" ], @@ -2884,30 +3049,32 @@ "authority": [ { "path": "pyproject.toml", - "sha256": "6217b5f029796edcd83aa47fb8632d37aa7552344755c578fa4f181a6b23282d", - "role": "Declares the package version, Python requirement, dependency extras, scripts, and distribution metadata." + "sha256": "3592cc37444e2e1f5bb0284e939643039c0d513217861af8133dd8bd6e8a4325", + "role": "Declares the package version, Python requirement, dependency extras, scripts, and distribution metadata the redirected guide covers." }, { "path": "openhcs/cli.py", - "sha256": "51c2570a5a49c83e826c3a2e85943a2c39593e5e30e1ce8af396ec4a4c724df1", - "role": "Declares the installed OpenHCS command entry points and their launch routing." + "sha256": "59025ea51309dbae40416d75fc203287657a109379f65072d77cff41c712e11e", + "role": "Declares the installed OpenHCS command entry points, including the skills subcommand, and their launch routing." }, { "path": "packaging/installers/windows/Install-OpenHCS.ps1", - "sha256": "f94eb1c0460cab7dca770a600d889bf55ae98f1aa652b2755de0ecc906ee83d8", - "role": "Implements the Windows installer, short managed paths, package installation, and shortcut creation." + "sha256": "ca368082f3d7a414f9e9136b2c5ae459bfb2a9a2a37c096d06eeb93d5c403358", + "role": "Implements the Windows installer, short managed paths, package installation, skill sync, and shortcut creation." }, { "path": "packaging/installers/macos/install-openhcs.sh", - "sha256": "b78b6b5849265926f83f77f392f7f4ac74ce5968f62e0b6972c6ce7976498203", - "role": "Implements the macOS application installer, managed environment creation, package installation, and launcher setup." + "sha256": "3a0f43c57c14bd7f8ceeb38464e174728f903e0ec524d980dbdb2b5f9bc11544", + "role": "Implements the macOS application installer, managed environment creation, package installation, skill sync, and launcher setup." } ], "findings": [ - "Maintaining installation instructions here would duplicate package extras and installer behaviour; the page correctly redirects to the canonical task." + "Maintaining installation instructions here would duplicate package extras and installer behaviour; the page correctly redirects to the maintained task.", + "The redirect sentence called the Getting Started instructions 'canonical'; it now says 'maintained'." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/installation.rst in full and compared it against the changed authorities: pyproject.toml (dependency floors, JAX extra), openhcs/cli.py (skills subcommand), the Windows and macOS installers (skill sync); the page copies none of those facts and still redirects to getting_started; replaced 'canonical' with 'maintained'.", "On 2026-08-27, re-read docs/source/installation.rst in full after the python-introspect dependency floor advanced to 0.1.12; confirmed that its compatibility redirect still reaches the canonical installation procedure.", "On 2026-08-26, re-read the complete compatibility page after native installer transcript and evidence handling changed; retaining one redirect to the canonical installation how-to remains accurate and avoids duplicated guidance.", "Compared every factual claim directly with the current hash-pinned authorities: pyproject.toml; openhcs/cli.py; packaging/installers/windows/Install-OpenHCS.ps1; packaging/installers/macos/install-openhcs.sh.", @@ -2951,45 +3118,49 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "ef81b50ee22b8da69513fd50d6e45a929117fa4bfaa62a20ef08a4c02531c043", - "role": "Declares the supported OpenHCS configuration dataclasses, nested scopes, defaults, and field help metadata." + "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "role": "Declares GlobalPipelineConfig, PipelineConfig and the step configuration families with their defaults and field help." }, { "path": "openhcs/pyqt_gui/config.py", "sha256": "a3eece2e5b2926fc6fc0edd197221da451ab7c0d0d6f4ffc8bb2ff865ad74bd4", - "role": "Declares UI configuration, including logging, theming, viewer, and execution-server settings." + "role": "Declares UIConfig with list previews, logging, shortcuts, progress, execution transport and agent-bridge settings." + }, + { + "path": "openhcs/agent/services/config_service.py", + "sha256": "902bb12178abf758cf0d7883c9cade8c5870f5362a3b5f7cc79e5640da9fb42d", + "role": "Resolves the global, pipeline, step and ui directive names to configuration declarations." }, { "path": "openhcs/agent/services/config_reference_service.py", "sha256": "18b740aad750284c0aeaa4c51e81a31828c8e51e609f4997b6a1a50cabfb4e32", - "role": "Builds configuration reference records from live dataclass fields, types, defaults, and help metadata." + "role": "Renders configuration reference records from live dataclass fields, types, defaults, and help metadata." + }, + { + "path": "openhcs/agent/capabilities.py", + "sha256": "eaebe33af01d057d1d21b3701f958bc96ede3ec5d0bddddca9917c1164ef0042", + "role": "Declares openhcs_describe_config_schema, the agent view of the same introspection." }, { - "path": "docs/source/_ext/openhcs_config_reference.py", - "sha256": "a45ccbd142745b0e636ea815134d6154c669f84b06a2d087fd76f993ee06aca3", - "role": "Generates the Sphinx configuration reference from the live dataclass declarations instead of a copied field table." + "path": "openhcs/agent/dto/config.py", + "sha256": "b85b56d41b09feb826daf09d12427a1a78fb2d03705c4cb1177629e1711587dd", + "role": "Declares ConfigPatch, which the page excludes from UI configuration edits." }, { "path": "tests/unit/agent/test_config_reference_service.py", - "sha256": "de18fb8ff76efe2bdbc46a3f46137622fe53fe6a2a28449d34728f4d04a32a1d", - "role": "Proves that configuration reference output is derived from live declarations and field metadata." + "sha256": "43521f7fbad821905bd59fb231f8d96c4efcfa99d7a27b5655a70e9507199e01", + "role": "Proves that configuration reference output and inherited field help are derived from live declarations." }, { "path": "tests/unit/pyqt_gui/test_user_facing_help_surfaces.py", - "sha256": "540d0fc2d796ff80543a7ef245d2450c1ecf988994e244c13472f9756cd904e1", + "sha256": "c7bfe985e0cf08cdab7de7f33aed443e697cbf7e181fbdc3ef4bb177c17ec897", "role": "Proves user-facing field help is sourced from declaration metadata rather than copied UI text." - }, - { - "path": "external/pyqt-reactive/src/pyqt_reactive/strategies/preview_formatting.py", - "sha256": "dc1a423246ca40f13633847d5d3ffde3916e53c1a8a8a92de893d3e80ad8fda7", - "role": "Owns preview detail, collection bounds, and wrapping declarations." } ], - "findings": [ - "Identified list-preview presentation in the UIConfig overview while retaining generated field reference as authority." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/reference/configuration.rst in full and compared it against the changed authorities: core/config.py, test_config_reference_service.py, test_user_facing_help_surfaces.py; the four directive names still resolve through config_service aliases, the named config classes, openhcs_describe_config_schema and ConfigPatch exist, and the UIConfig overview matches its fields; no correction was needed. The generated field tables carry the G1/G4 renames without page edits.", "Re-read docs/source/reference/configuration.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/config.py; openhcs/pyqt_gui/config.py; openhcs/agent/services/config_reference_service.py; docs/source/_ext/openhcs_config_reference.py; tests/unit/agent/test_config_reference_service.py; tests/unit/pyqt_gui/test_user_facing_help_surfaces.py.", "Focused cross-lattice QA passed 179 tests spanning callable contracts, framework-device scope, custom functions, catalogue discovery, installer/update policy, MCP release, and packaged OMERO deployment.", @@ -3013,58 +3184,58 @@ "diataxis": "reference", "authority": [ { - "path": "openhcs/core/aligned_image_payload.py", - "sha256": "086a6130abf3e0c759127aeacf4140157aee3b4bdc306c8d02067abce102acd0", - "role": "Declares the runtime image payload execution modes used to preserve complete stacks." + "path": "openhcs/core/processing_contracts.py", + "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", + "role": "Declares the ProcessingContract family: Pure2D, Pure3D, Flexible, and VolumetricToSlice contracts." }, { - "path": "openhcs/core/runtime_object_label_domains.py", - "sha256": "32824678741c1caf3fbabe621619d6d8d463936a28bc2d689916ed69fa7582ed", - "role": "Declares payload-scoped and plane-scoped object identity domains." + "path": "openhcs/core/image_payload_execution_mode.py", + "sha256": "7657995da0722cebfd0448c0cdce4cada1b00e0cff419f62e17dcc3654666ce0", + "role": "Declares the image payload execution modes, including FullStackExecution, that preserve a nested volume under an outer plane axis." }, { - "path": "openhcs/processing/backends/lib_registry/unified_registry.py", - "sha256": "70462cddcad1bfee447ce0786a9e4a85abbd297726d59a1208156af0ef3a3743", - "role": "Defines canonical callable metadata and derives the unified catalogue from registry declarations." + "path": "openhcs/core/runtime_object_label_domains.py", + "sha256": "cb56a2b3a391cfd5081e1487a04e2e95b88628879abc42a2ec3b69ab327ab0cf", + "role": "Declares the PAYLOAD and PLANE object-label domain scopes." }, { "path": "openhcs/processing/backends/cellprofiler/watershed.py", - "sha256": "9bd009be9881a88e295e1c5ca6fbc02b4402bc97d207cfd20848ff685fe64bde", + "sha256": "41c8cb9cb139f8b4041a25e1a7a2d19aa85cf0427422fc87b9551c8141069625", "role": "Declares the full-stack payload route for CellProfiler-compatible Watershed." }, { "path": "openhcs/processing/backends/cellprofiler/primary_objects.py", - "sha256": "6710b507fcdead2954e8a1038c91e2a99572d67b6d4ded8a7d41100fb229aa29", + "sha256": "b5b8a9b98dd9166b34a8f5cebf884e8af6da85f02fe878ef1227d6656d31ae4c", "role": "Declares the current plane-local IdentifyPrimaryObjects route." }, { "path": "openhcs/processing/backends/cellprofiler/secondary.py", - "sha256": "9a032a11e51c3f950538f89f4e4312094496fd6408e0fdceebc7dc8b91be43de", + "sha256": "fc9d9b1bbc3baff7216c6abe3a7135ffddfc9f7e1212d4a391fa8534038a354c", "role": "Declares the current plane-local IdentifySecondaryObjects and IdentifyTertiaryObjects routes." }, { "path": "openhcs/processing/backends/cellprofiler/intensity.py", - "sha256": "646e837759d6504e496b09fd903744c95e020ae02d0a424bbe408cb772d8b2f0", + "sha256": "43bfa771bb26453f864b633a48371c0e68cf5aae0e175ed6d7a1eabb38ec5dbe", "role": "Declares volumetric object-intensity execution and its Z measurement features." }, { "path": "openhcs/processing/backends/cellprofiler/shape.py", - "sha256": "42e4f72dba460e5f32a5d1e70e4ca792a8e648158e509274dcc37762cdd08718", + "sha256": "69e82de4faa46adc1b5ea543e352edf9a253c4b3bdedec61e0e5e3a7d95b3019", "role": "Declares full-stack object-shape input and the exact 3D feature schema." }, { "path": "openhcs/processing/backends/cellprofiler/area_occupied.py", - "sha256": "5d6112d86c5df1e068d36e4c08fd9cc59cb2e91f5226c2bc2a017e204d2d4009", + "sha256": "8316d44a5ea2f88f855aeef1c3d4fd979fb01a95a6dc9f6d1bf005629f25b18b", "role": "Declares 3D occupied-volume measurements and physical voxel-spacing input." }, { "path": "benchmark/manifests/official30_portable_axis1.json", - "sha256": "7f11549c0bc870f9ce2792afc9176f52b6a2b83236e47b33cbedb4a2cac73267", + "sha256": "1be9bb67ea05d78fd1e0b8991b994ff5635da2fff0358302b8e0df888e62888e", "role": "Declares the Official30 3D monolayer case and its compatibility evidence category." }, { "path": "tests/unit/test_documented_dimensionality.py", - "sha256": "83bef8dcad91750c9acda63b8a4293ccbd0904dd5aea7a32be197f9d51f300a7", + "sha256": "45762e74bf5cecbbada509f378ddbfa47c20fa6a9f5bd668d05d003e59bb38e1", "role": "Binds the published dimensionality claims to the current callable and feature declarations." }, { @@ -3074,25 +3245,26 @@ }, { "path": "tests/unit/test_cellprofiler_library_loading.py", - "sha256": "c756d24a1cd9469505d94de5cabe4cdb3129e67c4e1bd6c66ad02588274a8e5c", + "sha256": "316732ba044a62b80711a0466d8ce5812cb882ff91fbf0417ecae90afa3ff610", "role": "Proves object intensity measures a 3D object as one volume and emits Z features." }, { "path": "tests/unit/test_cellprofiler_module_execution.py", - "sha256": "f5be1bcae3be455df60ae80623e6cc170830b1754b9b435294f5fc5e1b75fc43", + "sha256": "f5d678e29026578687a68f6ce7abc3184eec5223a7e4e2337038fb2079a98ea2", "role": "Proves payload-scoped size-and-shape rows use the 3D volume schema." }, { "path": "tests/unit/test_cellprofiler_pipeline_import.py", - "sha256": "353f0c1d8e0bc3552d2e418d792794ab0631daa16c78a3e10191b1cc1ea07d73", + "sha256": "6b1d201ec5c4ba620fa2367a9cbef1f6fbdd25edbdda9bd7b723ebe3a3363412", "role": "Proves CellProfiler import preserves 3D image, object, and morphology variants from declarations." } ], "findings": [ - "Dimensionality and feature breadth were previously spread across implementation and conceptual pages, so prospective users could not look up the current boundary without reading architecture internals." + "The page's claims still hold after G3/G5: ProcessingContract is now a class family (Pure2DContract, Pure3DContract, FlexibleContract, VolumetricToSliceContract) and image execution modes moved to openhcs/core/image_payload_execution_mode.py. The recorded authorities aligned_image_payload.py and unified_registry.py no longer own the stated facts and were replaced by processing_contracts.py and image_payload_execution_mode.py." ], - "disposition": "revise", + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/reference/dimensionality_and_measurements.rst in full and compared it against the changed authorities: aligned_image_payload.py, runtime_object_label_domains.py, unified_registry.py, the six CellProfiler backends, the Official30 axis1 manifest, and the four tests; confirmed primary/secondary/tertiary still declare Pure2DContract, Watershed and intensity use FullStackExecution, shape and area-occupied use FullStackLabels, occupied-volume callables use Pure3DContract, PAYLOAD/PLANE scopes remain, and every listed measurement family has a backend module; no page change was needed.", "On 2026-08-27, re-read docs/source/reference/dimensionality_and_measurements.rst in full against registry form exclusions, explicit secondary-backend default selection, and expanded import tests; challenged every dimensional capability claim and found no semantic drift.", "On 2026-08-27, re-read the complete capability reference after library registries gained declaration-owned runtime-parameter exclusions; segmentation and measurement boundaries remain unchanged.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/aligned_image_payload.py; openhcs/core/runtime_object_label_domains.py; openhcs/processing/backends/lib_registry/unified_registry.py; openhcs/processing/backends/cellprofiler/watershed.py; openhcs/processing/backends/cellprofiler/primary_objects.py; openhcs/processing/backends/cellprofiler/secondary.py; openhcs/processing/backends/cellprofiler/intensity.py; openhcs/processing/backends/cellprofiler/shape.py; openhcs/processing/backends/cellprofiler/area_occupied.py; benchmark/manifests/official30_portable_axis1.json; tests/unit/test_documented_dimensionality.py; tests/unit/test_cellprofiler_watershed_3d.py; tests/unit/test_cellprofiler_library_loading.py; tests/unit/test_cellprofiler_module_execution.py; tests/unit/test_cellprofiler_pipeline_import.py.", @@ -3108,7 +3280,7 @@ }, { "path": "docs/source/reference/experimental_config_syntax.rst", - "source_sha256": "730244294388f85e5e49fc46d9850cf8f38912fb4039ac45ab89da88d7825051", + "source_sha256": "b40850f9bdef7085e23271bead36af7ba1488a8f3c8afa0374990c09aaf1f225", "audience": [ "experimental-analysis operators", "analysis integrators" @@ -3117,48 +3289,48 @@ "diataxis": "reference", "authority": [ { - "path": "openhcs/formats/experimental_layout_rows.py", + "path": "openhcs/processing/backends/experimental_analysis/layout_rows.py", "sha256": "69e1428d0aefe683f1ee1cf741f83915fbaa1318d9d3f19b8bf8120583d883d2", - "role": "Declares workbook row roles and parses controls, exclusions, doses, well groups, and replicate identities." + "role": "Declares workbook row roles (replicate count, Wells/WellsN) and the EDDU_CX5 and EDDU_metaxpress scope values." }, { - "path": "openhcs/formats/experimental_analysis.py", - "sha256": "1169fed585163a9e26376909b06675e35218ac459a11bbbe2e56e82014856a01", - "role": "Parses experimental workbooks, assigns plate metadata, and prepares inputs for the analysis engine." + "path": "openhcs/processing/backends/experimental_analysis/analysis.py", + "sha256": "17a701e0740eac3e11319fe73f10302bbcab603218349bfba4e07014aabc27bb", + "role": "Parses the drug_curve_map and plate_groups sheets, applies positional dose/well/plate-group mapping, controls, exclusions, and Per Well Datapoints." }, { - "path": "openhcs/formats/experimental_result_formats.py", - "sha256": "ce46b895e4caafec7d026112cacfdfc496de0cb4cd5978cceb94db02b98fe288", - "role": "Owns the CX5 workbook and MetaXpress CSV result contracts selected by the workbook scope." + "path": "openhcs/processing/backends/experimental_analysis/result_formats.py", + "sha256": "6951f41a2ff2bdac1945c7e56584a578c111c078cac1d684030c00f95a23f528", + "role": "Owns the CX5 workbook and MetaXpress CSV result readers selected by the workbook scope." }, { - "path": "openhcs/core/config.py", - "sha256": "ef81b50ee22b8da69513fd50d6e45a929117fa4bfaa62a20ef08a4c02531c043", - "role": "Declares the supported OpenHCS configuration dataclasses, nested scopes, defaults, and field help metadata." + "path": "openhcs/domains/microscopy/config.py", + "sha256": "79b8baefdbf22b85c199807f62b8234daca6904633a574886661a1c2e3d03a9b", + "role": "Declares ExperimentalAnalysisConfig with its five filenames, sheet names, and normalization method defaults." }, { "path": "openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py", - "sha256": "316be268b761bfa74ed1df137f2ba014a1f1c273aa9abd598a8a2e10388afb79", - "role": "Runs the unified experimental analysis pipeline and emits the documented summary outputs." + "sha256": "2705d459204557c4f7092df3fd21ed687dcf56449ed361485382b1c0ce196444", + "role": "Exports ExperimentalAnalysisEngine, whose run_directory projects config filenames and returns the documented result dictionary." }, { "path": "tests/unit/test_experimental_layout_rows.py", - "sha256": "d71b1e77aeb31a543e9da104a9dca2169aa7ea6c6e690a5c19302b26e76ae31b", + "sha256": "c3b06c37f8f2a9d960d73238c6b38018de1de824d3e683b06dce5367e95a0adc", "role": "Proves workbook row parsing, positional mapping, multi-digit groups, result-format leaves, normalization semantics, and conditional-format plate grids." }, { "path": "tests/unit/test_experimental_analysis_engine.py", - "sha256": "23b5d73def871a4c1f1e60625faf56d4664a315f90ddbddff7fa1ceab2117c17", + "sha256": "641fadecf8c940cc2316a06771c74db5f3f2db9dafdfbfebdc535760d6d776ec", "role": "Exercises declaration-owned paths and a complete MetaXpress directory workflow with all three outputs." } ], "findings": [ - "The page used Markdown headings in RST, mixed task advice into reference, documented unsupported validation, and showed a deprecated entry point; it was rewritten from the current parser and engine.", - "Source verification exposed Cartesian dose/well expansion and single-digit replicate parsing; the owning declarations were corrected to positional mapping with cardinality validation and multi-digit identities before the reference was finalised.", - "Result-format verification exposed three mirrored dispatch systems and filename-based format inference. The workbook scope now selects one nominal result-format leaf, and the reference distinguishes the MetaXpress desktop default from configured CX5 runs." + "The programmatic example imported ExperimentalAnalysisConfig from openhcs.core.config; G2/G4 moved it to openhcs.domains.microscopy.config. The import was corrected.", + "The workbook parser, row roles, and result formats moved from openhcs/formats/ into openhcs/processing/backends/experimental_analysis/ (layout_rows.py, analysis.py, result_formats.py) without behaviour change; the authorities were repointed and the workbook syntax claims still hold." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/reference/experimental_config_syntax.rst in full and compared it against the changed authorities: the deleted openhcs/formats/experimental_{layout_rows,analysis,result_formats}.py (moved to processing/backends/experimental_analysis/), openhcs/core/config.py, unified_analysis_engine.py, and both experimental-analysis tests; checked the row roles, scope values, engine export and run_directory signature against the moved modules, and corrected the ExperimentalAnalysisConfig import to openhcs.domains.microscopy.config.", "Re-read docs/source/reference/experimental_config_syntax.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/formats/experimental_layout_rows.py; openhcs/formats/experimental_analysis.py; openhcs/formats/experimental_result_formats.py; openhcs/core/config.py; openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py; tests/unit/test_experimental_layout_rows.py; tests/unit/test_experimental_analysis_engine.py.", "Focused cross-lattice QA passed 179 tests spanning callable contracts, framework-device scope, custom functions, catalogue discovery, installer/update policy, MCP release, and packaged OMERO deployment.", @@ -3178,28 +3350,31 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "ef81b50ee22b8da69513fd50d6e45a929117fa4bfaa62a20ef08a4c02531c043", - "role": "Declares the supported OpenHCS configuration dataclasses, nested scopes, defaults, and field help metadata." + "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "role": "Declares the global, pipeline, step, and desktop configuration dataclasses that the generated configuration reference lists." }, { - "path": "openhcs/formats/experimental_analysis.py", - "sha256": "1169fed585163a9e26376909b06675e35218ac459a11bbbe2e56e82014856a01", - "role": "Parses experimental workbooks, assigns plate metadata, and prepares inputs for the analysis engine." + "path": "openhcs/processing/backends/experimental_analysis/analysis.py", + "sha256": "17a701e0740eac3e11319fe73f10302bbcab603218349bfba4e07014aabc27bb", + "role": "Parses the Excel experimental-design workbook that the syntax reference documents." }, { "path": "openhcs/agent/services/knowledge_base_service.py", - "sha256": "88e10e18c66eb3a8bc2ec5dd33493a9473510c0b0a9d355502d668042f32fdd9", - "role": "Discovers and serves the maintained knowledge and example corpus used by MCP clients." + "sha256": "dfc26135b16f3827ced85b122cc1555c1325da1c5e805b421afcee9cdd7d91f3", + "role": "Serves the example corpus map and its native example source projection." }, { "path": "openhcs/core/artifacts.py", - "sha256": "bb958c74c98cb96c97c5ac1ea68ac997673aac6afcad57bdf0c9496065e895fa", - "role": "Declares the nominal artefact types exchanged by compiled steps and runtime consumers." + "sha256": "65eb0ec5c903366243ca8bd5ea07412e8d34bdb81e6015fa76b3c4c59838733f", + "role": "Declares the artifact types named by the shared terminology the index points to." } ], - "findings": [], + "findings": [ + "The page's navigation and descriptions remain accurate; its recorded authority openhcs/formats/experimental_analysis.py was deleted (moved to openhcs/processing/backends/experimental_analysis/analysis.py) and was repointed." + ], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/reference/index.rst in full and compared it against the changed authorities: openhcs/core/config.py, the deleted openhcs/formats/experimental_analysis.py, openhcs/agent/services/knowledge_base_service.py, and openhcs/core/artifacts.py; checked that every toctree target exists and each one-line description still matches its page and owner; no page change was needed.", "Re-read docs/source/reference/index.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/config.py; openhcs/formats/experimental_analysis.py; openhcs/agent/services/knowledge_base_service.py; openhcs/core/artifacts.py.", "Focused cross-lattice QA passed 179 tests spanning callable contracts, framework-device scope, custom functions, catalogue discovery, installer/update policy, MCP release, and packaged OMERO deployment.", @@ -3225,22 +3400,27 @@ "authority": [ { "path": "openhcs/agent/ui_bridge_identities.py", - "sha256": "5913d14b99340174af87b2a7cafa3cdd080a331553c0cf59392ade7d253f1234", - "role": "Declares the complete registered family of stable public UI window identities and their presentation titles." + "sha256": "66dcf838d08cac2004ab8f73db777f73270cc520697281cf4f94a5fc6af4dc64", + "role": "Declares the registered family of stable UI window identities; plate-manager and pipeline-editor identities take their titles from the session view they render." + }, + { + "path": "openhcs/authoring/session/views.py", + "sha256": "8ce6cac27293ac9105f4d2ebc76d10eebeba1ef90e0e8954c0a117910e17e3b3", + "role": "Owns the session view titles that the plate-manager and pipeline-editor window identities present." }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "a156a006a3c40f43107b9b1f6ecb97ed150879eb88ded8ef89e25214309ae33a", + "sha256": "e43085c1eaf0c836ce34e1a1dae7740488d91bdfea56844e7ddd4e6925561cf6", "role": "Composes the main desktop window and its stable docked surfaces from the registered identities." }, { "path": "openhcs/pyqt_gui/services/main_window_workflows.py", - "sha256": "74d3e4fa2dda4a8223c8d878aad55de9df78a71ff359c0b83346be991d14a594", + "sha256": "883fc3ddd1c406719ba75439ef9f3330bbbab1ff85e3476f6e343111a8f3c8fc", "role": "Projects registered window identities into WindowManager specifications for stable task windows." }, { "path": "scripts/gallery_catalog.py", - "sha256": "7f493e0cb305e14ee4cf7d3e995c67df4200186a8a8aaf00eef0bc66d8d191cf", + "sha256": "7fd3fce2b8fdae1abf049bb4f8a3a310ab598a7f2c507578425be537903ccfca", "role": "Projects every registered stable-window identity into its documentation scenario, capture target, filename, and accessibility text." }, { @@ -3250,18 +3430,19 @@ }, { "path": "tests/unit/test_build_website.py", - "sha256": "4a8952ba0f51590ffd9f1d787986653802506807079a7e0d118343264252e0e0", + "sha256": "e88fc8ae73c5610dfb0a9948271a016e666451228ecade92aa1b2892da106059", "role": "Proves stable-window scenario identities, titles, filenames, and capture targets derive one-to-one from the identity registry." }, { "path": "tests/unit/test_capture_media_gallery.py", - "sha256": "aabfe3defb657b72150caaa3737f06352056e8580060633999a4296818ee8dcd", + "sha256": "c1ee8fe70c63a4308c8c4b38b2733e73c4e9c7719695c305b723e7892c8dc04b", "role": "Proves the stable-window capture batch processes every projected scenario and records its source evidence." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/reference/ui_windows.rst in full and compared it against the changed authorities: openhcs/agent/ui_bridge_identities.py, openhcs/pyqt_gui/main.py, openhcs/pyqt_gui/services/main_window_workflows.py, scripts/gallery_catalog.py, tests/unit/test_build_website.py, and tests/unit/test_capture_media_gallery.py; the U1 session refactor moved two window titles onto session views but the inventory, titles, and capture targets are still projected from the registered identity declarations, no OMERO window is registered, and all linked pages exist; added openhcs/authoring/session/views.py as the title owner; no page change was needed.", "On 2026-08-28, re-read docs/source/reference/ui_windows.rst in full and challenged its reference boundary, completeness, duplication, navigation, and current desktop claims against the registered identity and projection owners.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/agent/ui_bridge_identities.py; openhcs/pyqt_gui/main.py; openhcs/pyqt_gui/services/main_window_workflows.py; scripts/gallery_catalog.py; docs/source/_ext/openhcs_gallery.py; tests/unit/test_build_website.py; tests/unit/test_capture_media_gallery.py.", "A current-source projection resolved 10 registered identities one-to-one to 10 scenarios, titles, declaration-derived filenames, and checked-in WebP figures; the registry contains no OMERO management window.", @@ -3274,7 +3455,7 @@ }, { "path": "docs/source/user_guide/analysis_consolidation.rst", - "source_sha256": "ddb564adfdece91f7ea7223bbd8407b78492175b83fa500c2e4449ae82cfc50a", + "source_sha256": "d277f21a16284bdcd021959e98b2e7833661e7639db3a73247cd3acc43af2825", "audience": [ "desktop operators with materialised analysis results" ], @@ -3283,40 +3464,37 @@ "authority": [ { "path": "openhcs/pyqt_gui/main.py", - "sha256": "a156a006a3c40f43107b9b1f6ecb97ed150879eb88ded8ef89e25214309ae33a", - "role": "Composes the main desktop window, menus, task windows, and registered UI services." + "sha256": "e43085c1eaf0c836ce34e1a1dae7740488d91bdfea56844e7ddd4e6925561cf6", + "role": "Declares the four Tools > Analysis Consolidation actions and the experimental-analysis directory check." }, { - "path": "openhcs/core/config.py", - "sha256": "ef81b50ee22b8da69513fd50d6e45a929117fa4bfaa62a20ef08a4c02531c043", - "role": "Declares the supported OpenHCS configuration dataclasses, nested scopes, defaults, and field help metadata." - }, - { - "path": "openhcs/formats/experimental_analysis.py", - "sha256": "1169fed585163a9e26376909b06675e35218ac459a11bbbe2e56e82014856a01", - "role": "Parses experimental workbooks, assigns plate metadata, and prepares inputs for the analysis engine." + "path": "openhcs/domains/microscopy/config.py", + "sha256": "79b8baefdbf22b85c199807f62b8234daca6904633a574886661a1c2e3d03a9b", + "role": "Declares AnalysisConsolidationConfig fields (enabled, metaxpress_style, file_extensions, exclude_patterns, output names) and ExperimentalAnalysisConfig filenames." }, { "path": "openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py", - "sha256": "316be268b761bfa74ed1df137f2ba014a1f1c273aa9abd598a8a2e10388afb79", + "sha256": "2705d459204557c4f7092df3fd21ed687dcf56449ed361485382b1c0ce196444", "role": "Projects the five configured filenames and writes normalized, raw, and heatmap outputs." }, { "path": "tests/unit/test_experimental_analysis_engine.py", - "sha256": "23b5d73def871a4c1f1e60625faf56d4664a315f90ddbddff7fa1ceab2117c17", + "sha256": "641fadecf8c940cc2316a06771c74db5f3f2db9dafdfbfebdc535760d6d776ec", "role": "Proves the documented directory contract and three output files end to end." }, { - "path": "openhcs/core/orchestrator/analysis_consolidation.py", - "sha256": "3ef9c20c347c1ac724353c84b24f9d7e13513aba7b9c5c01e194171b7e72bb52", - "role": "Owns automatic per-directory consolidation and the merged summary beneath the compiled output plate root." + "path": "openhcs/domains/microscopy/analysis_consolidation.py", + "sha256": "292f7138a00dc22dba5b8d8af5de6140d247b955723cc807cc982ffec7a7dd2d", + "role": "Owns the post-execute hook that consolidates execution-saved tables per directory and writes the merged summary beneath the compiled output plate root." } ], "findings": [ - "The task omitted the desktop action's required filenames and produced outputs; those source-owned inputs and outputs are now explicit and link to reference syntax." + "The page said AnalysisConsolidationConfig selects well extraction; the config declares no well field (the well comes from the partition axis value), so the sentence now lists the declared choices: MetaXpress-style layout, file extensions, exclusion patterns, and output names. It also placed the config 'on the pipeline'; it is a global pipeline configuration section.", + "Recorded authorities named the deleted openhcs/formats/experimental_analysis.py and openhcs/core/orchestrator/analysis_consolidation.py; consolidation moved to openhcs/domains/microscopy/analysis_consolidation.py and both configs to openhcs/domains/microscopy/config.py." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/user_guide/analysis_consolidation.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/main.py, openhcs/core/config.py, the deleted openhcs/formats/experimental_analysis.py and openhcs/core/orchestrator/analysis_consolidation.py (now openhcs/domains/microscopy/analysis_consolidation.py), unified_analysis_engine.py, and the engine test; checked the menu actions, directory filenames, consolidation config fields, and per-directory plus merged summary behaviour; corrected the config description.", "Re-read docs/source/user_guide/analysis_consolidation.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/pyqt_gui/main.py; openhcs/core/config.py; openhcs/formats/experimental_analysis.py; openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py; tests/unit/test_experimental_analysis_engine.py.", "Focused cross-lattice QA passed 179 tests spanning callable contracts, framework-device scope, custom functions, catalogue discovery, installer/update policy, MCP release, and packaged OMERO deployment.", @@ -3332,7 +3510,7 @@ }, { "path": "docs/source/user_guide/code_ui_editing.rst", - "source_sha256": "7617d0594ee253c7a19ad940eee142a931b094171b76428ac83bb74861ad2c17", + "source_sha256": "cbc76b557c64d0c183b351c4e9c465777e465856e3e2c4954f4909bc4f2fd26e", "audience": [ "desktop pipeline authors", "MCP-assisted editors" @@ -3342,13 +3520,33 @@ "authority": [ { "path": "openhcs/core/pipeline_document.py", - "sha256": "0849800b3e6474b8176fe35bc4aaebc63cfa85bd77251000c50da5f4c928b1b2", - "role": "Declares the public pipeline document, step collection, and serialisable declaration boundary." + "sha256": "8c554c8275c665f308eb098955bda517aa804eeb34418b5e0f41d412650ce7b5", + "role": "PipelineDocumentCodec validates, parses, and renders the pipeline_steps and optional pipeline_config document." + }, + { + "path": "openhcs/authoring/session/dataset_document.py", + "sha256": "305da3c0bf7bb15a0f251a101696b721f9151d8620450a6b3e12413b79218574", + "role": "Owns the selected and all-datasets document scopes, selected-scope ID enforcement, and the global-draft and per-dataset change checks on apply." + }, + { + "path": "openhcs/authoring/session/pipelines.py", + "sha256": "245bed768c49ae17b26252d41a1040e59c46d59e0894b553309cd5ec8a9a1793", + "role": "PipelineObjectStateBinding updates and commits a dataset's step ObjectState graph after a complete pipeline replacement." + }, + { + "path": "openhcs/authoring/session/views.py", + "sha256": "8ce6cac27293ac9105f4d2ebc76d10eebeba1ef90e0e8954c0a117910e17e3b3", + "role": "Declares the Datasets and Pipeline Editor view titles shown for the two code-document owners." + }, + { + "path": "openhcs/pyqt_gui/widgets/pipeline_editor.py", + "sha256": "8e418631fd9ba4f8ac3d6b257f9af127f9e8711281608516ea2617d7855d6f36", + "role": "Makes the Pipeline Editor document writable only while a dataset is selected." }, { "path": "openhcs/pyqt_gui/services/config_window_code_document.py", "sha256": "bdbf33b254451945239bd1c1f256dedbed40d2a8c0e598c4426bcba939ab5d2d", - "role": "Owns typed configuration source projection and applies parsed updates through ConfigDocumentAuthority." + "role": "Owns typed configuration source projection and applies parsed updates." }, { "path": "external/pyqt-reactive/src/pyqt_reactive/services/window_code_document.py", @@ -3356,103 +3554,93 @@ "role": "Owns generic window-document writability and declaration-normalised source comparison." }, { - "path": "openhcs/pyqt_gui/services/ui_bridge_object_state.py", - "sha256": "747c284c37e420092ffda9abbb8275fc7deaaff7f3676cecf3f5b1f324314744", - "role": "Projects driver capabilities and enforces them before applying a document to live ObjectState." + "path": "external/pyqt-reactive/src/pyqt_reactive/services/scope_token_service.py", + "sha256": "f84056a6b542d1d9a7df9e05c2ec344d6acb7cafd3301054158e89f04eb32b17", + "role": "Reconciles submitted step occurrences with existing scopes, giving ambiguous duplicates new scopes." }, { - "path": "openhcs/agent/services/ui_bridge_service.py", - "sha256": "37015a88828820e7ae98bf804f42c37f1fa26009b1ad28b3c537f65a656e943a", - "role": "Owns agent-to-UI declaration inspection and mutation, including code and form synchronisation." - }, - { - "path": "external/pycodify/src/pycodify/__init__.py", - "sha256": "b839bc2a7cd2afdfd8756eb39f22b791cb4f7aa55c0fc9c11099b7219c9c663b", - "role": "Exports the PyCodify source-generation and reconstruction API used for declaration round-tripping." + "path": "openhcs/pyqt_gui/services/ui_bridge_object_state.py", + "sha256": "e548b367507b3fb6830ce9ecb844e3b756243af96fe38e0d35de5da0b974c1fd", + "role": "Enforces driver writability, returns unchanged for equivalent documents, and applies documents against revision tokens." }, { - "path": "tests/unit/pyqt_gui/test_ui_agent_bridge.py", - "sha256": "1feee4bb413738f04e81cbdf26244dc0d585a68921e6c7749a941a08518007a1", - "role": "Proves bidirectional UI-bridge inspection, capability enforcement, semantic no-op handling, and mutation of live declarations." + "path": "openhcs/pyqt_gui/services/ui_agent_bridge.py", + "sha256": "ce650c52475764657bbc0221109879594f9cf1482de8245d6c141f00474d5d23", + "role": "Rejects wildcard imports and resolves catalogue-listed external functions in submitted documents." }, { - "path": "openhcs/pyqt_gui/widgets/shared/services/plate_manager_workflows.py", - "sha256": "d640bf4b2710009dc3f92719ec97cf09fe3d93ed7eb1bebaf2b02913e8fafbfd", - "role": "Owns proof-bearing selected and all-plates code-document application." + "path": "openhcs/agent/services/ui_bridge_service.py", + "sha256": "c7bdf8b5caec1c21c986cdc2224524839fadfa8764dafcd05125d90c72351899", + "role": "Owns agent-to-UI document inspection and mutation and returns mutation receipts." }, { - "path": "openhcs/pyqt_gui/services/pipeline_object_state_binding.py", - "sha256": "afb190ac9ac38e800c8fe04497bbb78a2f1b6381a21eb14890e3061c61ae5b0b", - "role": "Owns fresh identity graphs for complete pipeline replacement and stable occurrence identities for incremental edits." + "path": "external/pycodify/src/pycodify/__init__.py", + "sha256": "5869747d68ca64e6497f55e09b53cbbaa87741d39ae922edb035997c53f19c49", + "role": "Exports the PyCodify source-generation and reconstruction API used for declaration round-tripping." }, { - "path": "openhcs/core/steps/function_step.py", - "sha256": "8e1243404095691030e9ebb95bb5611679f96c66c6b6819ae71e4b41249e74bc", - "role": "Owns complete FunctionStep declaration equality through canonical callable-pattern projection." + "path": "openhcs/serialization/pycodify_formatters.py", + "sha256": "fe06f3ab873b537b7b2c0b13599ffd11030ae6a8ac504caea9e3dcfa1d4695b4", + "role": "Supplies OpenHCS semantic equality, function-reference import rendering, and clean/full choices to parameter projection." }, { - "path": "tests/unit/pyqt_gui/test_plate_manager_widget.py", - "sha256": "b02e81c4a73213577df02a1effbd6843935f5c92f459d6c61ab2998bd489ad58", - "role": "Proves selected code-document application preserves unselected plate graphs." + "path": "external/python-introspect/src/python_introspect/callable_declaration.py", + "sha256": "3bb8b474624fe6b2964a5c1f9a6aa540b2703500bfa124e7808995cb789d6951", + "role": "Orders supplied parameters by signature with optional default omission and caller-owned equality." }, { - "path": "tests/unit/test_pipeline_document.py", - "sha256": "2c18222ac2a34ee267788063a374112b17874c64693bfe542669c4183bc90c36", - "role": "Proves omitted pipeline_config becomes PipelineConfig() and canonical rendering restores the explicit assignment." + "path": "external/pyqt-reactive/src/pyqt_reactive/services/function_pattern_code_document.py", + "sha256": "bdf41b58bff009890e6e696cfc4f18313adfb1669566b2566f9e7b93d863c071", + "role": "Retains a child parameter state only while its function and editable surface match." }, { - "path": "tests/unit/pyqt_gui/test_pipeline_code_document_transactionality.py", - "sha256": "b7e546e904b897f13d76f0c523e751afa541a75c627349ba79a3a87313d30f4c", - "role": "Proves Pipeline Editor validates and applies the abbreviated source through its normal transactional workflow." + "path": "external/pyqt-reactive/src/pyqt_reactive/protocols/widget_adapters.py", + "sha256": "1ae6653bf3f2786eafa9f53f246e145ebfa34b81c190aaa8f41c19a1463a42f9", + "role": "Owns locale-aware compact numeric text without reducing configured precision." }, { "path": "openhcs/pyqt_gui/widgets/shared/services/pipeline_editor_workflows.py", - "sha256": "6664126ac4daa851cacff5014e04a0c09a56bbc76f62661ff97d713e9137d447", + "sha256": "c704ef7e6c5f8a5745c741e923bc4e620a321fdce069e1cdcdf521c0176e4a4f", "role": "Applies a complete Pipeline Editor document and commits the reconciled live state graph." }, - { - "path": "tests/unit/pyqt_gui/test_pipeline_editor_widget.py", - "sha256": "8f28719c5bd9eb5681e67af8e1f7aceb445bcae97c12a13348f943c8b6bafa35", - "role": "Proves selected-plate writability and that code apply advances saved baselines for the editor, step, and nested function states." - }, { "path": "scripts/gallery_catalog.py", - "sha256": "7f493e0cb305e14ee4cf7d3e995c67df4200186a8a8aaf00eef0bc66d8d191cf", - "role": "Owns the lazy-inheritance scenario, representative poster, caption, description, alt text, dimensions, and release evidence." + "sha256": "7fd3fce2b8fdae1abf049bb4f8a3a310ab598a7f2c507578425be537903ccfca", + "role": "Owns the lazy-inheritance gallery scenario, poster, caption, and alt text." }, { - "path": "openhcs/pyqt_gui/widgets/step_parameter_editor.py", - "sha256": "54e0bdd74f8d781c2bfdf1da559bbe7e2393f19ef74826d374a2281e8eb96258", - "role": "Applies edited function patterns through generic reconciliation before rebuilding parameter panes." + "path": "tests/unit/pyqt_gui/test_code_document_catalog_imports.py", + "sha256": "7391f26f23c396c974fb0f7eada5ae5a0646a5ec2beb5515388fe7367efe954c", + "role": "Proves catalogue imports such as skimage.filters.edges.sobel keep the registry wrapper and wildcard or unlisted imports are rejected." }, { - "path": "openhcs/serialization/pycodify_formatters.py", - "sha256": "316ff58e34dcb87816082ead03347cd280fe72b4e3ab8ea31affc1627e548c23", - "role": "Supplies OpenHCS semantic equality and clean/full choices to shared parameter projection." + "path": "tests/unit/pyqt_gui/test_ui_agent_bridge.py", + "sha256": "de6758859d9743e826369162855327e4a84056cafdd7ddfdc33868d2a3093353", + "role": "Proves bidirectional UI-bridge inspection, capability enforcement, semantic no-op handling, and mutation of live declarations." }, { - "path": "external/python-introspect/src/python_introspect/callable_declaration.py", - "sha256": "3bb8b474624fe6b2964a5c1f9a6aa540b2703500bfa124e7808995cb789d6951", - "role": "Orders supplied parameters by signature with optional default omission and caller-owned equality." + "path": "tests/unit/pyqt_gui/test_plate_manager_widget.py", + "sha256": "4daa91c0de965dc2dd9cd77ea4969f4fa0aff90fc5b69cbcadaba3bd2a1cfc6b", + "role": "Proves selected code-document application preserves unselected dataset graphs." }, { - "path": "external/pyqt-reactive/src/pyqt_reactive/services/function_pattern_code_document.py", - "sha256": "bdf41b58bff009890e6e696cfc4f18313adfb1669566b2566f9e7b93d863c071", - "role": "Projects child parameter states into function-pattern code documents, retaining a child only while its function authority and editable surface match." + "path": "tests/unit/test_pipeline_document.py", + "sha256": "4df57a9c4f12a009222981bf1ac28554405874f3533656b61344ee94056d1bab", + "role": "Proves omitted pipeline_config becomes PipelineConfig() and rendering restores the explicit assignment." }, { - "path": "external/pyqt-reactive/src/pyqt_reactive/protocols/widget_adapters.py", - "sha256": "1ae6653bf3f2786eafa9f53f246e145ebfa34b81c190aaa8f41c19a1463a42f9", - "role": "Owns locale-aware compact numeric text without reducing configured precision." + "path": "tests/unit/pyqt_gui/test_pipeline_editor_widget.py", + "sha256": "0ce88271aa7f5f55a8fc4daec8279d732844e03dfd437c11e6e6265ca196b4a0", + "role": "Proves selected-dataset writability and that code apply advances saved baselines for the editor, step, and nested function states." }, { "path": "tests/unit/pyqt_gui/test_step_code_widget_roundtrip.py", "sha256": "a67876053a5381e1767060d0ee5d54c79d96dfbaec8ef0e267bd767993a821cd", - "role": "Verifies real step-code and widget round trips, including retained and hidden child states." + "role": "Verifies real step-code and widget round trips, including retained and rebuilt child states." }, { "path": "tests/unit/test_pycodify_formatters.py", - "sha256": "445479f81fd70ce7f288d5b7537b9bbb85d1b5dd724d27baabf8e4c4e508128c", + "sha256": "2f0c6683a5255d01442ae33f676f601d5e3f1e3e04d1b2c2fc945b3c564796ce", "role": "Verifies clean/full parameter order and semantic default handling." }, { @@ -3462,16 +3650,13 @@ } ], "findings": [ - "The code-mode procedure now records the steps-only compatibility form and canonical re-rendering of the default PipelineConfig assignment.", - "The guide omitted Plate Manager selection authority and the identity semantics of complete pipeline replacement.", - "The guide incorrectly implied a second save action after Pipeline Editor code apply; it now records the exact saved-baseline boundary.", - "The guide omitted selected-plate writability and declaration-normalised no-op behaviour.", - "Callable wrapper and explicit-default differences no longer create false occurrence replacements during code round trips.", - "Explained signature order, compact decimal presentation, and retained parameter states after step-code application.", - "The apply procedure previously implied every retained function occurrence kept its child ObjectState even when explicit keyword additions or removals changed the editable surface." + "The page called the multi-dataset window Plate Manager; after U1 the window renders the session's DatasetListView titled Datasets, so the four references now name the Datasets window and Datasets document.", + "Recorded authority openhcs/pyqt_gui/services/pipeline_object_state_binding.py was deleted in U1; pipeline replacement now goes through openhcs/authoring/session/pipelines.py and dataset document scopes through openhcs/authoring/session/dataset_document.py. The selected/all scope, global-draft protection, writability, and reconciliation claims were rechecked against them and still hold.", + "Other pages still name the window Plate Manager (24 pages under docs/source); reported to the lead rather than edited here." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/user_guide/code_ui_editing.rst in full and compared it against the changed authorities: pipeline_document.py, ui_bridge_object_state.py, ui_bridge_service.py, plate_manager_workflows.py and pipeline_editor_workflows.py, the deleted pipeline_object_state_binding.py (now authoring/session/pipelines.py), step_parameter_editor.py, pycodify_formatters.py, gallery_catalog.py, and the changed tests; checked selected/all document scopes, selected-dataset writability, global-draft protection while work runs, catalogue imports and wildcard rejection, unchanged no-op results, and saved-baseline commit; renamed Plate Manager to the Datasets window.", "Re-read docs/source/user_guide/code_ui_editing.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/pipeline_document.py; openhcs/pyqt_gui/services/pycodified_window_code_document.py; openhcs/agent/services/ui_bridge_service.py; external/pycodify/src/pycodify/__init__.py; tests/unit/pyqt_gui/test_ui_agent_bridge.py; openhcs/pyqt_gui/widgets/shared/services/plate_manager_workflows.py; openhcs/pyqt_gui/services/pipeline_object_state_binding.py; tests/unit/pyqt_gui/test_plate_manager_widget.py; tests/unit/test_pipeline_document.py; tests/unit/pyqt_gui/test_pipeline_code_document_transactionality.py; openhcs/pyqt_gui/widgets/shared/services/pipeline_editor_workflows.py; tests/unit/pyqt_gui/test_pipeline_editor_widget.py.", "Focused cross-lattice QA passed 179 tests spanning callable contracts, framework-device scope, custom functions, catalogue discovery, installer/update policy, MCP release, and packaged OMERO deployment.", @@ -3502,28 +3687,34 @@ "authority": [ { "path": "openhcs/utils/environment.py", - "sha256": "d408e5c8bd86f9b7d26ce23595fb75200a9320d9f3ee9bfca8d29861cecb9c9d", - "role": "Detects runtime environment and optional backend availability, including CPU-only operation." + "sha256": "2d8738e593b3e92c966089772624f47d17d38bb27fb90974707af659c91e8ad8", + "role": "Declares OPENHCS_CPU_ONLY and projects the CPU-only import policy (including JAX_PLATFORMS=cpu) before dependencies import their runtimes." + }, + { + "path": "openhcs/pyqt_gui/launch.py", + "sha256": "bf47d568e2d9bcb758342f34a5b3f030fd8abd9276c5e4fa2406666301e0561f", + "role": "Declares the desktop --no-gpu flag and enables CPU-only mode before Qt and backend startup." }, { "path": "openhcs/processing/backends/lib_registry/registry_service.py", - "sha256": "046d3d088f685a993ddad6b5923cf63beef84b201127b72ae32af6721dd5ad63", - "role": "Owns backend discovery, availability filtering, and the canonical process catalogue projection." + "sha256": "daef8925338fddf8504a9faa066af42456a13fe4602976bf64be06547a45fcc8", + "role": "Skips registries that do not support CPU-only mode during function discovery." }, { "path": "tests/unit/test_library_registry_discovery.py", - "sha256": "a7ee2d3a2002ea291876c27aa2dce51a889cea6be9a3055602c76f1e9cf76f1b", + "sha256": "e612d181d0053bacb38cb7bd3a73e8af8b3f7cdbe3789c1b7deb8f5f6a655170", "role": "Proves backend discovery and availability filtering produce the expected callable registry." }, { "path": "pyproject.toml", - "sha256": "6217b5f029796edcd83aa47fb8632d37aa7552344755c578fa4f181a6b23282d", - "role": "Declares the package version, Python requirement, dependency extras, scripts, and distribution metadata." + "sha256": "3592cc37444e2e1f5bb0284e939643039c0d513217861af8133dd8bd6e8a4325", + "role": "Declares the optional gpu extra and the openhcs console entry point." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/user_guide/cpu_only_mode.rst in full and compared it against the changed authorities: openhcs/utils/environment.py, registry_service.py, test_library_registry_discovery.py, and pyproject.toml; confirmed --no-gpu and OPENHCS_CPU_ONLY still select the same mode, discovery still filters registries by CPU-only support, and the gpu extra still exists; the new import-policy defaults (JAX platform, NumPy huge pages, worker profiling) do not contradict the page; no page change was needed.", "On 2026-08-27, re-read docs/source/user_guide/cpu_only_mode.rst in full against declaration-owned catalogue inputs and the python-introspect 0.1.12 floor; challenged the CPU-only workflow and found no behaviour change.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/utils/environment.py; openhcs/processing/backends/lib_registry/registry_service.py; tests/unit/test_library_registry_discovery.py; pyproject.toml.", "Focused cross-lattice QA passed 179 tests spanning callable contracts, framework-device scope, custom functions, catalogue discovery, installer/update policy, MCP release, and packaged OMERO deployment.", @@ -3551,7 +3742,7 @@ }, { "path": "docs/source/user_guide/custom_function_management.rst", - "source_sha256": "a543658a18b61b83e0e516d59f13e543090213d904b7503c89ef479eb9a55f87", + "source_sha256": "949fb40f353aaeca3dab0272ec4e021166ab70f984e4acff6fe83ef73ea948ec", "audience": [ "desktop callable authors" ], @@ -3560,13 +3751,18 @@ "authority": [ { "path": "openhcs/processing/custom_functions/manager.py", - "sha256": "aa5aa5a7032cd38c170d29d800175022883f04df7ce2d3f2fb4f6decad43f97d", - "role": "Owns persisted custom-function source and coordinates validated lifecycle transactions with the runtime projection owner." + "sha256": "f87d622c4e03db3b6d7f977daa86187b892f59133be264ffa381472785247e56", + "role": "Owns persisted source under the data directory, requires one decorated processing function per source, rejects collisions, and coordinates create, load, replace, and delete." + }, + { + "path": "openhcs/processing/custom_functions/runtime_registry.py", + "sha256": "160fb2fbad1b4d2f1ffaf784bacc5d7355d78c2720e718083cfc8c05748384df", + "role": "Atomically publishes process-local callable metadata and module exports, retaining ephemeral declarations when persisted source reloads." }, { "path": "openhcs/processing/custom_functions/events.py", "sha256": "ed62fe7aef90d56ef4c3421d61257a272e9a54f7ec2005cc865ff0e605d8ea35", - "role": "Publishes process-local custom-function changes through weak identity-preserving subscriptions without importing a GUI runtime." + "role": "Publishes process-local custom-function changes through weak subscriptions without importing a GUI runtime." }, { "path": "openhcs/processing/custom_functions/signals.py", @@ -3574,80 +3770,68 @@ "role": "Adapts the headless domain event into Qt signal delivery for desktop consumers." }, { - "path": "tests/unit/test_custom_function_events.py", - "sha256": "8cc545a424a3c5235dbca2c123ce7f1e266aaaa007b85082154cbf06ed78122c", - "role": "Proves weak subscriber lifetimes, exact callback identity, duplicate bound-method suppression, and unsubscribe on Qt destruction even while its Python wrapper survives." + "path": "openhcs/pyqt_gui/main.py", + "sha256": "e43085c1eaf0c836ce34e1a1dae7740488d91bdfea56844e7ddd4e6925561cf6", + "role": "Declares the Tools > Custom Functions > Manage Functions action." }, { "path": "openhcs/pyqt_gui/dialogs/custom_function_manager_dialog.py", - "sha256": "31b0073aa4e61966b3bb1020d8fe455f087563490dff0d78a47724fe0d677767", + "sha256": "fc67b871212b512661ebbf82641814194472bdc94febc67af85aa3f1e7eb2b4e", "role": "Implements the desktop controls for inspecting, editing, reloading, and removing custom functions." }, - { - "path": "openhcs/pyqt_gui/main.py", - "sha256": "a156a006a3c40f43107b9b1f6ecb97ed150879eb88ded8ef89e25214309ae33a", - "role": "Composes the main desktop window, menus, task windows, and registered UI services." - }, - { - "path": "tests/unit/test_function_catalog_zmq.py", - "sha256": "f146a79c06ad1bec0db105ce88322a3a7158e0a6c60c8bac8e8608784fa7983c", - "role": "Proves persisted custom functions propagate through registry discovery and the execution-server catalogue." - }, { "path": "openhcs/agent/services/endpoint_function_catalog_service.py", - "sha256": "977a74003819f2547548d322f4c5f0cb148b23fec20f31be4fa462bfcfb7ea04", - "role": "Owns endpoint registration, asynchronous invalidation, and refresh of the shared catalogue projection." + "sha256": "bc232090472554ab075f7800bbcc238e27a646a2d8225c87e5e443fb3c5d9f78", + "role": "Owns endpoint catalogue preparation, invalidation, cancellation, and custom-function registration on the execution endpoint." }, { "path": "openhcs/pyqt_gui/dialogs/function_selector_dialog.py", "sha256": "e11c456a5322038408edd42df7d2674cda42e1992f7f6c6e567e7923bf3526c3", - "role": "Projects catalogue loading and failure state without blocking the Qt thread." + "role": "Invalidates and reloads the endpoint catalogue without blocking the Qt thread." }, { - "path": "openhcs/processing/custom_functions/runtime_registry.py", - "sha256": "ee935ccc09f3a967d84c9ebd08511ad729d53e73d5bceb52074b39a1cd6ead9c", - "role": "Atomically publishes process-local custom callable metadata and public module exports from validated source declarations." + "path": "openhcs/mcp/context.py", + "sha256": "1292c9bb7173e6afe5aa6237b861c5f39bc2e854f364fd722c75a3c6b76f534e", + "role": "Supplies the local MCP tools with the session's endpoint function catalogue." }, { - "path": "tests/unit/test_custom_function_lifecycle.py", - "sha256": "918326f0f530d833172b30f25a3e91c4f3fc402ea6645a515bd666a83816994d", - "role": "Proves transactional create, load, replace, delete, collision, and source-revision behaviour." + "path": "external/pyqt-reactive/src/pyqt_reactive/core/future_completion.py", + "sha256": "7ba6ef203bae81319004ccc4ddfe4226e0230b177df663cfb0d02c364e1dbc47", + "role": "Owns future completion relay lifetime and Qt receiver disconnection." }, { "path": "scripts/gallery_catalog.py", - "sha256": "7f493e0cb305e14ee4cf7d3e995c67df4200186a8a8aaf00eef0bc66d8d191cf", - "role": "Owns the documentation-enabled Custom Function Manager capture declaration, accessibility text, capture target, and generated visual projection." + "sha256": "7fd3fce2b8fdae1abf049bb4f8a3a310ab598a7f2c507578425be537903ccfca", + "role": "Owns the Custom Function Manager capture declaration, accessibility text, and capture target." }, { - "path": "external/pyqt-reactive/src/pyqt_reactive/core/future_completion.py", - "sha256": "7ba6ef203bae81319004ccc4ddfe4226e0230b177df663cfb0d02c364e1dbc47", - "role": "Owns future completion relay lifetime and Qt receiver disconnection." + "path": "tests/unit/test_custom_function_events.py", + "sha256": "8cc545a424a3c5235dbca2c123ce7f1e266aaaa007b85082154cbf06ed78122c", + "role": "Proves weak subscriber lifetimes, exact callback identity, and unsubscribe on Qt destruction." }, { - "path": "external/pyqt-reactive/tests/test_future_completion.py", - "sha256": "fddee2a4c4b210e43c137a24f5730c0e8303a38ecc67afec7a4ac5f39d2ad860", - "role": "Verifies native receiver destruction, delivery thread, garbage collection and cancellation." + "path": "tests/unit/test_custom_function_lifecycle.py", + "sha256": "70c87f0dbefd2c1a204d060c2211ec57f4f84023f7c6931d2cb7d81eb82e9252", + "role": "Proves transactional create, load, replace, delete, collision, and source-revision behaviour." }, { - "path": "openhcs/mcp/context.py", - "sha256": "7bb92e1357e02a0676243370179e5ff471d4e15c4c88b9a9f1d69b79f505c7cf", - "role": "Projects the local MCP catalogue endpoint from the canonical UI configuration." + "path": "tests/unit/test_function_catalog_zmq.py", + "sha256": "7d10d9c80c101d97f7502cb8195cbb38f1742ac86579b3e4587f3b8a2048a1a6", + "role": "Proves persisted custom functions propagate through registry discovery and the execution-server catalogue." }, { - "path": "openhcs/pyqt_gui/config.py", - "sha256": "a3eece2e5b2926fc6fc0edd197221da451ab7c0d0d6f4ffc8bb2ff865ad74bd4", - "role": "Owns the explicit canonical UI configuration file selector." + "path": "external/pyqt-reactive/tests/test_future_completion.py", + "sha256": "fddee2a4c4b210e43c137a24f5730c0e8303a38ecc67afec7a4ac5f39d2ad860", + "role": "Verifies native receiver destruction, delivery thread, garbage collection and cancellation." } ], "findings": [ - "The task guide did not explain that custom-function changes invalidate and asynchronously refresh the endpoint-owned catalogue projection.", - "The guide assigned both persistence and runtime publication to CustomFunctionManager instead of distinguishing persisted source from the atomic process projection.", - "The custom-function manager imported a global Qt signal as part of its headless lifecycle; change publication is now a domain event with a separate desktop Qt adapter.", - "The domain event retained temporary Qt adapters after their C++ objects were destroyed, allowing a later custom-function update to invoke an invalid callback and crash the process.", - "Clarified current endpoint catalogue reads after remote registration and retained asynchronous/cache ownership." + "The page said the manager requires a supported memory decorator; it now requires exactly one function with a declared callable contract per source (the contract may name an execution scope rather than a memory type), so the sentence now says one decorated processing function per source.", + "openhcs/pyqt_gui/config.py was dropped as an authority; the page makes no UI-configuration claim and the MCP catalogue now comes from the session's endpoint catalogue in openhcs/mcp/context.py." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/user_guide/custom_function_management.rst in full and compared it against the changed authorities: custom_functions/manager.py and runtime_registry.py, custom_function_manager_dialog.py, openhcs/pyqt_gui/main.py, endpoint_function_catalog_service.py, openhcs/mcp/context.py, gallery_catalog.py, and the catalogue and lifecycle tests; checked the menu path, register_from_code signature and package export, source validation rules, ephemeral retention on reload, and catalogue invalidation; corrected the decorator requirement.", "Re-read docs/source/user_guide/custom_function_management.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/processing/custom_functions/manager.py; openhcs/processing/custom_functions/events.py; openhcs/processing/custom_functions/signals.py; tests/unit/test_custom_function_events.py; openhcs/pyqt_gui/dialogs/custom_function_manager_dialog.py; openhcs/pyqt_gui/main.py; tests/unit/test_function_catalog_zmq.py; openhcs/agent/services/endpoint_function_catalog_service.py; openhcs/pyqt_gui/dialogs/function_selector_dialog.py; openhcs/processing/custom_functions/runtime_registry.py; tests/unit/test_custom_function_lifecycle.py.", "Focused cross-lattice QA passed 179 tests spanning callable contracts, framework-device scope, custom functions, catalogue discovery, installer/update policy, MCP release, and packaged OMERO deployment.", @@ -3672,7 +3856,7 @@ }, { "path": "docs/source/user_guide/custom_functions.rst", - "source_sha256": "6dec340f9cb1eb45e984513bb0ad3cdae229c546fc846df128564a0df0f61398", + "source_sha256": "b3116c2edcbfbfbe84ebe0874e453b74a50ae5a47cb5348dd2bfc6a442a6c174", "audience": [ "Python-capable pipeline authors" ], @@ -3681,45 +3865,57 @@ "authority": [ { "path": "openhcs/core/memory/decorators.py", - "sha256": "d9f9cf0be7593dc0ebfc57f258aebed75bcf9cfa508ed1e942137366d819766a", - "role": "Exports OpenHCS memory and dtype decorators while delegating conversion semantics to ArrayBridge declarations." + "sha256": "76f234785f4865257fd19c935e0aace82d1f5586198737fec619ee243b79cbf6", + "role": "Exports one OpenHCS memory decorator per MemoryType, including numpy, accepting a contract argument." }, { - "path": "openhcs/processing/backends/lib_registry/unified_registry.py", - "sha256": "70462cddcad1bfee447ce0786a9e4a85abbd297726d59a1208156af0ef3a3743", - "role": "Defines canonical callable metadata and derives the unified catalogue from registry declarations." + "path": "openhcs/core/processing_contracts.py", + "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", + "role": "Declares the ProcessingContract classes Pure2DContract, Pure3DContract, VolumetricToSliceContract, and FlexibleContract." }, { - "path": "openhcs/processing/custom_functions/manager.py", - "sha256": "aa5aa5a7032cd38c170d29d800175022883f04df7ce2d3f2fb4f6decad43f97d", - "role": "Owns persisted custom-function source and coordinates validated lifecycle transactions with the runtime projection owner." + "path": "external/arraybridge/src/arraybridge/decorators.py", + "sha256": "28f9bf4770bd4cc1ef34045fdef28065063ede029296f62b8a6c4140cc88e40e", + "role": "Declares the input, output, and execution framework metadata written by supported memory decorators." }, { - "path": "tests/unit/test_function_catalog_zmq.py", - "sha256": "f146a79c06ad1bec0db105ce88322a3a7158e0a6c60c8bac8e8608784fa7983c", - "role": "Proves persisted custom functions propagate through registry discovery and the execution-server catalogue." + "path": "openhcs/pyqt_gui/main.py", + "sha256": "e43085c1eaf0c836ce34e1a1dae7740488d91bdfea56844e7ddd4e6925561cf6", + "role": "Declares the Tools > Custom Functions > Create New Function action." }, { - "path": "external/arraybridge/src/arraybridge/decorators.py", - "sha256": "99823105b0ae59054df1d84013633b8b6628d0a6dc3326383199ac7c65c69bed", - "role": "Declares the input, output, and execution framework metadata written by supported memory decorators." + "path": "openhcs/processing/custom_functions/templates.py", + "sha256": "e114252a60ce5e57111519b98b44954f08733216335fab64413b643f5af61e50", + "role": "Supplies the starting template shown by the custom-function editor." + }, + { + "path": "openhcs/processing/custom_functions/manager.py", + "sha256": "f87d622c4e03db3b6d7f977daa86187b892f59133be264ffa381472785247e56", + "role": "Validates and registers saved custom-function source into the function catalogue." }, { "path": "openhcs/processing/custom_functions/runtime_registry.py", - "sha256": "ee935ccc09f3a967d84c9ebd08511ad729d53e73d5bceb52074b39a1cd6ead9c", + "sha256": "160fb2fbad1b4d2f1ffaf784bacc5d7355d78c2720e718083cfc8c05748384df", "role": "Projects a validated custom declaration into the process-local callable catalogue." }, { "path": "tests/unit/test_custom_function_lifecycle.py", - "sha256": "918326f0f530d833172b30f25a3e91c4f3fc402ea6645a515bd666a83816994d", + "sha256": "70c87f0dbefd2c1a204d060c2211ec57f4f84023f7c6931d2cb7d81eb82e9252", "role": "Proves validated custom declarations publish atomically without mirrored catalogue state." + }, + { + "path": "tests/unit/test_function_catalog_zmq.py", + "sha256": "7d10d9c80c101d97f7502cb8195cbb38f1742ac86579b3e4587f3b8a2048a1a6", + "role": "Proves persisted custom functions propagate through registry discovery and the execution-server catalogue." } ], "findings": [ - "The authoring guide described one undifferentiated memory declaration and omitted the decorator's independent input, output, and execution framework roles." + "After G5 made processing contracts a class family, the code example already imported Pure2DContract, but the contract list still named enum members PURE_2D, PURE_3D, VOLUMETRIC_TO_SLICE, and FLEXIBLE that no longer exist; the list now names Pure2DContract, Pure3DContract, VolumetricToSliceContract, and FlexibleContract.", + "unified_registry.py no longer owns the processing contract and was replaced as an authority by openhcs/core/processing_contracts.py." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/user_guide/custom_functions.rst in full and compared it against the changed authorities: the page's own earlier contract-import edit, openhcs/core/memory/decorators.py, unified_registry.py, custom_functions/manager.py and runtime_registry.py, external/arraybridge decorators.py, and the catalogue and lifecycle tests; checked the menu path, the decorator export, and the four contract classes in openhcs/core/processing_contracts.py; corrected the contract names in the list.", "On 2026-08-27, re-read docs/source/user_guide/custom_functions.rst in full against callable- and registry-owned form exclusions; challenged the authoring steps and confirmed that custom runtime inputs remain declaration-owned.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/memory/decorators.py; openhcs/processing/backends/lib_registry/unified_registry.py; openhcs/processing/custom_functions/manager.py; tests/unit/test_function_catalog_zmq.py; external/arraybridge/src/arraybridge/decorators.py; openhcs/processing/custom_functions/runtime_registry.py; tests/unit/test_custom_function_lifecycle.py.", "Focused cross-lattice QA passed 179 tests spanning callable contracts, framework-device scope, custom functions, catalogue discovery, installer/update policy, MCP release, and packaged OMERO deployment.", @@ -3743,51 +3939,49 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "ef81b50ee22b8da69513fd50d6e45a929117fa4bfaa62a20ef08a4c02531c043", - "role": "Declares the supported OpenHCS configuration dataclasses, nested scopes, defaults, and field help metadata." + "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "role": "Declares DtypeConfig with default_dtype_conversion (NATIVE_OUTPUT default) and StepMaterializationConfig, from which the lazy step forms are generated." }, { "path": "openhcs/core/memory/decorators.py", - "sha256": "d9f9cf0be7593dc0ebfc57f258aebed75bcf9cfa508ed1e942137366d819766a", - "role": "Exports OpenHCS memory and dtype decorators while delegating conversion semantics to ArrayBridge declarations." + "sha256": "76f234785f4865257fd19c935e0aace82d1f5586198737fec619ee243b79cbf6", + "role": "Exports OpenHCS memory decorators while delegating conversion semantics to ArrayBridge declarations." }, { "path": "external/arraybridge/src/arraybridge/decorators.py", - "sha256": "99823105b0ae59054df1d84013633b8b6628d0a6dc3326383199ac7c65c69bed", - "role": "Declares decorators that write independent input, output, and execution memory roles plus dtype policy." + "sha256": "28f9bf4770bd4cc1ef34045fdef28065063ede029296f62b8a6c4140cc88e40e", + "role": "Declares DtypeConversion members and applies the conversion to a decorated callable's main result." }, { "path": "external/arraybridge/src/arraybridge/array_operations.py", - "sha256": "7e9ebda3d058702cdf8662dc5ab1faf1254649f3cbde76fa0adcbe02677fa103", + "sha256": "ea8c4e21ffbef5b5d502f9e5b7779bd5a341cec4408337d16ba89613f4305636", "role": "Defines NumPy float-to-integer scaling and integer-to-integer casting at the callable dtype boundary." }, { "path": "openhcs/processing/backends/processors/numpy_processor.py", - "sha256": "311634ec55ec8ddefe05118329d208e4b33df86f81d54ca4e258e7f4d2dbc980", - "role": "Declares the decorated projection function used in the 8-bit materialisation example." + "sha256": "58df7297d1021decca547d9f5959b8d8e563544b4644ac4977452030ed8b30da", + "role": "Declares the decorated create_projection function used in the 8-bit materialisation example." }, { "path": "external/PolyStore/src/polystore/disk.py", - "sha256": "cf089b8c79f86099a7cea1b6cbe43ceaec6331196a44b43183f2e64b7746c219", + "sha256": "58733bc3890a9062572fedccb5643a382ce69d31486d496cd7f6c1ff9ac991c6", "role": "Writes materialised TIFF pixels at the dtype supplied by the processing result." }, { "path": "tests/unit/test_callable_contract.py", - "sha256": "493f76a54f5fe024ca8f10c53b1f810b8c369626ad7141205b42ef3a78282137", + "sha256": "cc220f53107a08b0406b6e0dd49896e3e4c3c0af6e527b0ef00d4fdaba8f0b2c", "role": "Proves callable contract derivation for dimensions, memory types, dtype policy, and outputs." }, { "path": "openhcs/core/steps/abstract.py", - "sha256": "5b58b6af32470c0d3d3b80c33c80102de16db07d4050aa59d82c81571917f36a", - "role": "Declares the step-level lazy dtype configuration override used by the task example." + "sha256": "63734f0c5198272b08f128ae974ada725c8201cb44efd0ce32500fd280b879bf", + "role": "Declares the step-level dtype_config and step_materialization_config lazy overrides used by the examples." } ], - "findings": [ - "The task guide told users to change a desktop or step setting without naming the control or showing an executable override.", - "The callable override lacked a concrete 8-bit materialisation example and warnings about NumPy scaling, integer wrapping, downstream values, and auxiliary outputs." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/user_guide/dtype_conversion.rst in full and compared it against the changed authorities: openhcs/core/config.py, openhcs/core/memory/decorators.py, external/arraybridge decorators.py and array_operations.py, numpy_processor.py, external/PolyStore disk.py, test_callable_contract.py, and openhcs/core/steps/abstract.py; confirmed DtypeConfig.default_dtype_conversion and its NATIVE_OUTPUT default, the PRESERVE_INPUT and UINT8 members, LazyDtypeConfig and LazyStepMaterializationConfig imports from openhcs.core.config, create_projection, and step dtype_config/step_materialization_config parameters; no page change was needed.", "On 2026-08-27, re-read docs/source/user_guide/dtype_conversion.rst in full after callable contracts began projecting runtime-owned inputs into generic form exclusions; challenged the dtype task flow and found no behaviour change.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/config.py; openhcs/core/memory/decorators.py; external/arraybridge/src/arraybridge/decorators.py; tests/unit/test_callable_contract.py; openhcs/core/steps/abstract.py.", "Focused cross-lattice QA passed 179 tests spanning callable contracts, framework-device scope, custom functions, catalogue discovery, installer/update policy, MCP release, and packaged OMERO deployment.", @@ -3810,40 +4004,41 @@ "authority": [ { "path": "openhcs/pyqt_gui/main.py", - "sha256": "a156a006a3c40f43107b9b1f6ecb97ed150879eb88ded8ef89e25214309ae33a", - "role": "Composes the main desktop window, menus, task windows, and registered UI services." + "sha256": "e43085c1eaf0c836ce34e1a1dae7740488d91bdfea56844e7ddd4e6925561cf6", + "role": "Declares the Tools > Analysis Consolidation > Run Experimental Analysis action and its directory check for the config and results files." }, { - "path": "openhcs/formats/experimental_analysis.py", - "sha256": "1169fed585163a9e26376909b06675e35218ac459a11bbbe2e56e82014856a01", - "role": "Parses experimental workbooks, assigns plate metadata, and prepares inputs for the analysis engine." + "path": "openhcs/processing/backends/experimental_analysis/analysis.py", + "sha256": "17a701e0740eac3e11319fe73f10302bbcab603218349bfba4e07014aabc27bb", + "role": "Parses the drug_curve_map and plate_groups sheets and applies replicate-local normalization and heatmap export." }, { - "path": "openhcs/core/config.py", - "sha256": "ef81b50ee22b8da69513fd50d6e45a929117fa4bfaa62a20ef08a4c02531c043", - "role": "Declares the default directory filenames and replicate-local normalization method." + "path": "openhcs/domains/microscopy/config.py", + "sha256": "79b8baefdbf22b85c199807f62b8234daca6904633a574886661a1c2e3d03a9b", + "role": "Declares ExperimentalAnalysisConfig's default directory filenames and the normalization_method choices." }, { "path": "openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py", - "sha256": "316be268b761bfa74ed1df137f2ba014a1f1c273aa9abd598a8a2e10388afb79", - "role": "Runs the unified experimental analysis pipeline and emits the documented summary outputs." + "sha256": "2705d459204557c4f7092df3fd21ed687dcf56449ed361485382b1c0ce196444", + "role": "Runs the directory workflow and writes the normalized, raw, and heatmap workbooks." }, { "path": "tests/unit/test_experimental_layout_rows.py", - "sha256": "d71b1e77aeb31a543e9da104a9dca2169aa7ea6c6e690a5c19302b26e76ae31b", + "sha256": "c3b06c37f8f2a9d960d73238c6b38018de1de824d3e683b06dce5367e95a0adc", "role": "Proves workbook row parsing, positional mapping, multi-digit groups, exclusions, normalization, and heatmap output." }, { "path": "tests/unit/test_experimental_analysis_engine.py", - "sha256": "23b5d73def871a4c1f1e60625faf56d4664a315f90ddbddff7fa1ceab2117c17", + "sha256": "641fadecf8c940cc2316a06771c74db5f3f2db9dafdfbfebdc535760d6d776ec", "role": "Runs the documented MetaXpress directory task end to end." } ], "findings": [ - "The page explained layout concepts but did not carry the operator through the actual file contract, menu action, or outputs; it now provides one complete desktop task flow." + "The page's claims still hold, but its recorded authorities named the deleted openhcs/formats/experimental_analysis.py and openhcs/core/config.py, which no longer declares ExperimentalAnalysisConfig; they were repointed to processing/backends/experimental_analysis/analysis.py and openhcs/domains/microscopy/config.py." ], - "disposition": "revise", + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/user_guide/experimental_layouts.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/main.py, the deleted openhcs/formats/experimental_analysis.py (moved to processing/backends/experimental_analysis/analysis.py), openhcs/core/config.py, unified_analysis_engine.py, and both experimental-analysis tests; checked the menu path, the five default filenames, normalization_method, and that variable_components and group_by remain step-config fields; no page change was needed.", "Re-read docs/source/user_guide/experimental_layouts.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/pyqt_gui/main.py; openhcs/formats/experimental_analysis.py; openhcs/core/config.py; openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py; tests/unit/test_experimental_layout_rows.py; tests/unit/test_experimental_analysis_engine.py.", "Focused cross-lattice QA passed 179 tests spanning callable contracts, framework-device scope, custom functions, catalogue discovery, installer/update policy, MCP release, and packaged OMERO deployment.", @@ -3868,30 +4063,34 @@ "authority": [ { "path": "openhcs/pyqt_gui/main.py", - "sha256": "a156a006a3c40f43107b9b1f6ecb97ed150879eb88ded8ef89e25214309ae33a", - "role": "Composes the main desktop window, menus, task windows, and registered UI services." + "sha256": "e43085c1eaf0c836ce34e1a1dae7740488d91bdfea56844e7ddd4e6925561cf6", + "role": "Composes the PyQt desktop main window that the openhcs command launches." }, { "path": "tests/unit/pyqt_gui/test_ui_config_lifecycle.py", - "sha256": "eeb7c260d3ae8bc7b5e6d94917b1e5e9a214feac9e2a0c2651552950cfe3a179", - "role": "Prevents the removed Textual package authority and local UI mirrors from returning." + "sha256": "5fd4194e70de5210a6b51283693a1d25c6a6a1278b8a869c7deebcbd583503e0", + "role": "Prevents the removed Textual package and local UI mirrors from returning." }, { "path": "openhcs/core/pipeline_document.py", - "sha256": "0849800b3e6474b8176fe35bc4aaebc63cfa85bd77251000c50da5f4c928b1b2", - "role": "Declares the public pipeline document, step collection, and serialisable declaration boundary." + "sha256": "8c554c8275c665f308eb098955bda517aa804eeb34418b5e0f41d412650ce7b5", + "role": "Declares the pipeline document as pipeline_steps plus PipelineConfig." + }, + { + "path": "openhcs/core/config.py", + "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "role": "Declares the nested processing, materialization, and streaming configuration dataclasses on PipelineConfig and steps." }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "ace8ae0c3e54bbe604b97cec7c3973a1abcf90e1dc517c2adda947dc2c35e882", - "role": "Compiles pipeline declarations into validated step plans and execution bundles." + "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "role": "Compiles pipeline declarations into typed step plans and a CompiledExecutionBundle before execution." } ], - "findings": [ - "The guide index still described the removed Textual interface as deprecated rather than removed." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/user_guide/index.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/main.py, test_ui_config_lifecycle.py, pipeline_document.py, and openhcs/core/pipeline/compiler.py; confirmed the Textual interface is still absent, the declaration is pipeline_steps plus PipelineConfig, nested step configuration still owns processing, input source, materialization and streaming options, and compilation still returns a CompiledExecutionBundle; all toctree targets exist; added openhcs/core/config.py for the nested-configuration claim; no page change was needed.", "Re-read docs/source/user_guide/index.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/pyqt_gui/main.py; tests/unit/pyqt_gui/test_ui_config_lifecycle.py; openhcs/core/pipeline_document.py; openhcs/core/pipeline/compiler.py.", "On 2026-08-22, corrected the interface status after the deprecated Textual implementation and its local UI mirrors were removed.", @@ -3919,7 +4118,7 @@ { "path": "external/pyqt-reactive/src/pyqt_reactive/widgets/log_viewer.py", "sha256": "e68eda879c7cd3f5360dd1b4a08a481298cabf85cd72bdfda66b0efbd54d56f9", - "role": "Implements log discovery, live-process filtering, record filtering, and viewer selection behaviour." + "role": "Implements log discovery, live-process filtering, level and source filtering, and log selection." }, { "path": "external/pyqt-reactive/src/pyqt_reactive/services/coalesced_scan_requests.py", @@ -3928,30 +4127,29 @@ }, { "path": "openhcs/pyqt_gui/services/reactor_providers.py", - "sha256": "e937c310e6dd7f06bf8e2cfc227ce442310f6d249fc0abed4ed0e588b4121fe8", + "sha256": "aecf1c77636095408486cab53a95cb0fc044fcf2ead0d7b72ac674842cf69abe", "role": "Registers the log, process, status, and execution reactors consumed by desktop widgets." }, { "path": "openhcs/pyqt_gui/config.py", "sha256": "a3eece2e5b2926fc6fc0edd197221da451ab7c0d0d6f4ffc8bb2ff865ad74bd4", - "role": "Declares UI configuration, including logging, theming, viewer, and execution-server settings." + "role": "Declares UI configuration, including logging and execution-server settings." }, { "path": "openhcs/runtime/zmq_execution_client.py", - "sha256": "31081f2a323a81c060aede1201a4bf1a1b39a51f37928c281a419ffaceadfe60", - "role": "Implements execution-server connection, request submission, progress consumption, and connection-state reporting." + "sha256": "147fb663d3ea60b1348063440e05b77d54b40634d1ae81cf4fb4f3b5457fbc99", + "role": "Spawns the execution server with its own per-port log file and startup status file and reports connection state." }, { "path": "scripts/gallery_catalog.py", - "sha256": "7f493e0cb305e14ee4cf7d3e995c67df4200186a8a8aaf00eef0bc66d8d191cf", - "role": "Owns the documentation-enabled execution-log capture, caption, accessibility text, dimensions, and source evidence." + "sha256": "7fd3fce2b8fdae1abf049bb4f8a3a310ab598a7f2c507578425be537903ccfca", + "role": "Owns the execution-log-viewer capture, caption, accessibility text, and dimensions." } ], - "findings": [ - "The selector scanned only during construction, so an execution server that started later could never expose its dedicated live log; visible non-overlapping discovery now updates the catalogue only when it changes." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/user_guide/log_viewer.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/services/reactor_providers.py, openhcs/runtime/zmq_execution_client.py, and scripts/gallery_catalog.py; the reactor changes (axis-family grouping, session scope) and the execution server's per-port log and startup-status files are consistent with the page's claims that subprocesses write dedicated logs reported by the launching process and that readiness is recorded separately; no page change was needed.", "Reviewed docs/source/user_guide/log_viewer.rst in full; the recorded findings and disposition are the resulting page-specific editorial decision.", "Compared its factual claims directly with the hash-pinned files listed in authority: external/pyqt-reactive/src/pyqt_reactive/widgets/log_viewer.py; external/pyqt-reactive/src/pyqt_reactive/services/coalesced_scan_requests.py; openhcs/pyqt_gui/services/reactor_providers.py; openhcs/pyqt_gui/config.py; openhcs/runtime/zmq_execution_client.py.", "Passed `.venv/bin/python scripts/validate_docs.py docs/source/user_guide`: documentation validation passed: 12 files, 2 Python blocks.", @@ -3970,7 +4168,7 @@ }, { "path": "docs/source/user_guide/mcp_clients.rst", - "source_sha256": "f47e4de0a9e8653155f47898627953ad7635c2b72e05805483a72bb6c85f0e26", + "source_sha256": "2c38bbbf07310596c6cc95a102909162dbcb13f53f5b045e7fb2d025e31da46e", "audience": [ "desktop users connecting local agent clients", "MCP administrators" @@ -3980,13 +4178,23 @@ "authority": [ { "path": "openhcs/mcp/bootstrap.py", - "sha256": "64b2ef9a4f4fe50154b4c921a7617b2e7c9dcf8bb475df7423e37e8395a8b250", - "role": "Builds the installed MCP server, registers its tools and resources, and starts the selected transport." + "sha256": "0e8e94cf54a32a1772e52a77d1c89a9ee02c6d2a58eb04f6e18fad2ce79d0394", + "role": "Declares the openhcs-mcp --surface option with the desktop default and starts the stdio transport." + }, + { + "path": "openhcs/agent/capabilities.py", + "sha256": "eaebe33af01d057d1d21b3701f958bc96ede3ec5d0bddddca9917c1164ef0042", + "role": "Declares the full, desktop, authoring, and core surface profiles and derives the openhcs_ session tools from headless session operations." + }, + { + "path": "openhcs/authoring/session/operations/datasets.py", + "sha256": "208274effc2e790775d90528211fa4f640eb01f7ca3fdd9c927c3bf71e39b632", + "role": "Declares the add, set-pipeline, initialize, compile, run, and stop dataset operations behind the session tools." }, { "path": "openhcs/mcp/client_registration.py", - "sha256": "28459b95d3b155778ce178f36302cb830958b74f2603980641ec492d81c41a9a", - "role": "Owns discovery and registration of the OpenHCS MCP server in supported desktop clients." + "sha256": "55b701bb77a3de19243fa9a6276f784fad8329e2d435fb26fcc7fdce1e060731", + "role": "Owns detection and registration of the stable launcher (and skill sync) in supported desktop clients." }, { "path": "openhcs/mcp/lifecycle.py", @@ -3994,73 +4202,74 @@ "role": "Owns MCP process lifecycle, endpoint readiness, and shutdown coordination." }, { - "path": "openhcs/agent/capabilities.py", - "sha256": "7fe9ab91490dedd6f750a57386e91835cf1067a27d10079966510aa9821adaf0", - "role": "Declares the agent capability catalogue and safety metadata exposed through MCP self-description." + "path": "openhcs/agent/path_policy.py", + "sha256": "73fb2d2f729487a926861bd3919ae26c92da79229db52d9c61e94ca88c7f392d", + "role": "Owns OPENHCS_AGENT_READ_ROOTS and OPENHCS_AGENT_WRITE_ROOTS and rejection of paths outside them." }, { "path": "packaging/mcpb/openhcs/manifest.json", - "sha256": "a51d5f32ad56d1ea22cf338b5b4eed1e343b7bbb62d9a1acde0ce5cf1fab20e5", - "role": "Declares the packaged MCP server command, arguments, compatibility, and client-visible metadata." - }, - { - "path": "tests/unit/agent/test_mcp_client_registration.py", - "sha256": "0763752fc04c32d2812f509845886f10c366e1a67ec290d50e433096a8d4debe", - "role": "Proves supported-client discovery, registration commands, and resulting MCP configuration." - }, - { - "path": "tests/unit/agent/test_mcp_server.py", - "sha256": "b99b6ca27be471f95ddf0852f8f94ce02797a459bf98bf6e0fa4851c0bb8c698", - "role": "Proves MCP server tool registration, request handling, and lifecycle behaviour." + "sha256": "c4899e221db575ada4dc291c5e40c6b2c7ab52913177186b9546b061e56c6759", + "role": "Declares the packaged Claude Desktop server command, path-policy inputs, and client-visible metadata." }, { - "path": "scripts/gallery_catalog.py", - "sha256": "7f493e0cb305e14ee4cf7d3e995c67df4200186a8a8aaf00eef0bc66d8d191cf", - "role": "Owns the local-MCP endpoint-startup scenario, representative poster, caption, description, alt text, dimensions, and release evidence." + "path": "packaging/codex/openhcs/.codex-plugin/plugin.json", + "sha256": "56e6f69d55609d1f2c4e92d2ba758e71d37c99181d19e7b51d7abaa5c8168fd4", + "role": "Declares the Codex release plugin that ships the use-openhcs skill." }, { "path": "benchmark/cellprofiler_benchmark_cli.py", - "sha256": "2b92755c556c1d66d28ffa0af6e2ac61cac5af4631514382290a2af0f907c26b", - "role": "Owns the installed benchmark execution and report command surface." + "sha256": "4025c591644e29f3df61e476ecae2143690e014336ea9f889aadeff4e692445e", + "role": "Declares the list-cases, inspect-run, inspect-measured, and run-measured commands, including the required --wait-timeout-ms." }, { "path": "benchmark/contracts/run_receipt.py", - "sha256": "1704e1ab47411d25d382b95871c96c1e60c94db14b8ffd55f4dc6c8a917758b9", + "sha256": "4cd61bd33f8f6199b3cc4130ed091dddde9c3a23975e132b1bee06c3c6a43cd7", "role": "Owns typed lifecycle status, rerun provenance, and receipt validation." }, { "path": "benchmark/contracts/control.py", - "sha256": "0bce24c29ea9dfb7b50c88937b616118078505db6f497312fff87766397e5ce9", - "role": "Declares the expert MCP benchmark inspection request and result." + "sha256": "ce9c4a1146723e445abc7c9b54eff9e8808e115413cd2842efc742fd1a6b300e", + "role": "Declares the expert MCP benchmark inspection and measured-run finalization requests and results." }, { "path": "benchmark/control.py", - "sha256": "4706078a8fbeb516a52bc51600751ae86c39c3ef7e397a2f8cf13d49204db941", + "sha256": "83fe9016625d7d8721905b8a153a28c3dcaaf2b9e925a3c97711c634868156b7", "role": "Reads authorized existing run directories without invoking benchmark execution." }, { "path": "benchmark/control_service.py", - "sha256": "bf2b39696a49d8c87cf74d39770d63013972b9c8cf884863c467ccdf46eb12d3", - "role": "Requires manifest and every declared benchmark source location to satisfy the agent read policy." + "sha256": "a1256b349ffdc8d04cab927326d97821462b61f57b29e564b61cc627302be49c", + "role": "Requires the manifest and every declared benchmark source location to satisfy the agent read policy." }, { - "path": "openhcs/agent/path_policy.py", - "sha256": "6ee0dc362b52d147586e9e61ea89499f5b35355ff697a2bfe5bbcc33febc45ba", - "role": "Owns explicit MCP readable roots and rejection of paths outside them." + "path": "scripts/gallery_catalog.py", + "sha256": "7fd3fce2b8fdae1abf049bb4f8a3a310ab598a7f2c507578425be537903ccfca", + "role": "Owns the zmq-startup-compile gallery scenario, poster, caption, and alt text." + }, + { + "path": "tests/unit/agent/test_mcp_client_registration.py", + "sha256": "0763752fc04c32d2812f509845886f10c366e1a67ec290d50e433096a8d4debe", + "role": "Proves supported-client discovery, registration commands, and resulting MCP configuration." + }, + { + "path": "tests/unit/agent/test_mcp_server.py", + "sha256": "4e6f6b1761b2c0316c99984baad6ea2c97e5c76c3f155111b8e498d7358e3175", + "role": "Proves MCP server tool registration, request handling, and lifecycle behaviour." }, { "path": "tests/unit/test_benchmark_control.py", - "sha256": "ee75209799216c90cfa3a585dd9b7af99c99405f93e9fb5e4f900ea26d9d3ea0", - "role": "Proves full-surface capability binding, enum projection, path policy, progress, artifacts, and lightweight imports." + "sha256": "093b584eeb37cbb12089d5af082ad45acd779e9d2bdd5b3feaea08234fc22183", + "role": "Proves benchmark capability binding, path policy, progress, artifacts, and lightweight imports." } ], "findings": [ - "The desktop MCP profile description excluded headless execution; it now documents the declaration-owned headless group and distinguishes independent sessions from UI-owned state.", - "The client how-to did not explain that benchmark inspection is expert-only and read-only, or direct users to the separately installed benchmark execution command.", - "The measured-run inspection instructions now distinguish retained_evidence_valid from separate image and table output validation." + "The run-measured paragraph had dropped the required --wait-timeout-ms argument; benchmark/cellprofiler_benchmark_cli.py still requires it and the session stops a run that exceeds it. The requirement was restored.", + "The page named the desktop multi-dataset window Plate Manager; after U1 it is titled Datasets, so the headless-session sentence now says Datasets window state.", + "Code issue, not edited: the openhcs-mcp --surface help text in openhcs/mcp/bootstrap.py says the default desktop surface hides headless tools, but DesktopLocalCapabilitySurfaceProfile includes CapabilityWorkflowGroup.HEADLESS_EXECUTION; the page's statement that the desktop surface includes headless-execution tools matches the declaration." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/user_guide/mcp_clients.rst in full and compared it against the changed authorities: the page's own session-tool edit, openhcs/mcp/bootstrap.py, client_registration.py, openhcs/agent/capabilities.py, the mcpb manifest, test_mcp_server.py, gallery_catalog.py, the benchmark CLI, contracts, control and control_service, openhcs/agent/path_policy.py, and test_benchmark_control.py; confirmed every named session tool derives from a declared headless operation, the four surface names and desktop default, the benchmark subcommands and flags, the path-policy variables, and the packaging paths; restored the --wait-timeout-ms requirement and renamed the window.", "Reviewed docs/source/user_guide/mcp_clients.rst in full; the recorded findings and disposition are the resulting page-specific editorial decision.", "Compared its factual claims directly with the hash-pinned files listed in authority: openhcs/mcp/bootstrap.py; openhcs/mcp/client_registration.py; openhcs/mcp/lifecycle.py; openhcs/agent/capabilities.py; packaging/mcpb/openhcs/manifest.json; tests/unit/agent/test_mcp_client_registration.py; tests/unit/agent/test_mcp_server.py.", "Passed `.venv/bin/python scripts/validate_docs.py docs/source/user_guide`: documentation validation passed: 12 files, 2 Python blocks.", @@ -4104,7 +4313,7 @@ }, { "path": "docs/source/user_guide/real_time_visualization.rst", - "source_sha256": "c269dd225183533dbd30237661eb2f8926c9f42b46082023e457a6c9232f7e3d", + "source_sha256": "df6ba5fcf3861d895b456d2cfa47b7fe6670e2696f3c60e0c5d8fb49e5ba01d5", "audience": [ "pipeline authors reviewing live scientific outputs" ], @@ -4113,67 +4322,87 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "ef81b50ee22b8da69513fd50d6e45a929117fa4bfaa62a20ef08a4c02531c043", - "role": "Declares the supported OpenHCS configuration dataclasses, nested scopes, defaults, and field help metadata." + "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "role": "Declares StreamingDefaults (persistent, host, transport_mode, enabled), partition_filter, and the Napari and Fiji streaming configs with their ports." + }, + { + "path": "openhcs/core/steps/abstract.py", + "sha256": "63734f0c5198272b08f128ae974ada725c8201cb44efd0ce32500fd280b879bf", + "role": "Declares the per-step napari_streaming_config, fiji_streaming_config, and step_materialization_config fields." + }, + { + "path": "openhcs/runtime/viewer_display.py", + "sha256": "c8979a3ef72676124270bfca21459b7a071b956333c72a4f070433b1dc2cba8b", + "role": "Declares the Napari and Fiji display settings selectable on a streaming config." + }, + { + "path": "pyproject.toml", + "sha256": "3592cc37444e2e1f5bb0284e939643039c0d513217861af8133dd8bd6e8a4325", + "role": "Declares the gui, napari, fiji, and viz extras." }, { "path": "openhcs/runtime/viewer_protocol.py", - "sha256": "30441eab97da9ffcb5b9dc33d65c2ecc56c426e79bb2b27eedc78a68aaf13bc3", - "role": "Declares the transport-independent viewer messages, stream configuration, and server contract." + "sha256": "c4bdae4bae6dee412f0d27a9cf75a95db53b3bd95c03cee7d046be16b52fe5b8", + "role": "Declares the transport-independent viewer messages and incremental settlement-progress fields." }, { "path": "openhcs/runtime/viewer_controls.py", - "sha256": "3227e6bf60e9b43fd792cc4299b6b50721741f012a0f7f856268b35022fd6062", - "role": "Declares the bounded payload and atomic layer-isolation controls used by focused agent review." + "sha256": "df0ac0aee66ba52a8b1706a1a99e03acc1300cfaf5cc74c2f93614a1c77465f6", + "role": "Declares bounded viewer-state payload limits and atomic layer-isolation controls." + }, + { + "path": "openhcs/agent/dto/viewer.py", + "sha256": "24435279b0d31bfd62ea8c29c07cbadc564a3427fd2b06c5e4abf20992f27f12", + "role": "Declares the ROI-summary request with its max_rois bound and the geometry-free ROI summary records." }, { "path": "openhcs/agent/services/viewer_window_service.py", - "sha256": "257718ca7da768132be2283326730f65d9b399f0ef852b2c7e1e9e48263f9eed", + "sha256": "cfe222586a221cb317099fc245216778f2ce15589604e5ce03410998dec4bb1a", "role": "Projects viewer responses into bounded ROI summaries and one-operation isolation results." }, { "path": "openhcs/runtime/napari_viewer_server.py", - "sha256": "64ccfef65a10e04878ef0806ae72d0446540efcbfc72af46c4e2ab785418a84c", - "role": "Implements Napari stream reception, layer updates, ROI navigation, and endpoint lifecycle." + "sha256": "038d7a76eeac8d69079591cf6eb7736af5415b28a7a067d6677bd15d115d11c7", + "role": "Implements Napari stream reception, layer updates, selected-layer routing, and the dimension-label overlay." + }, + { + "path": "openhcs/runtime/napari_streaming_handlers.py", + "sha256": "be585484075783f7717776d2b8ba163aab8a69aa24c560ca3844214485bd9eef", + "role": "Owns shared semantic axis slots, singleton alignment, and per-route forward-progress records for Napari layers." }, { "path": "openhcs/runtime/fiji_viewer_server.py", - "sha256": "cde38de711dced10b8af10b3b0ca69b1111c234a2877f0aade1e762ccbb6777b", + "sha256": "ee246d32ed828bbf4665cc8a411e807da29bd43c2475012243f27e93cb550410", "role": "Implements Fiji stream reception, image-stack updates, ROI handling, and endpoint lifecycle." }, { "path": "tests/unit/test_napari_streaming_handlers.py", - "sha256": "a338d6f2ef3094fa429f3d640709c7231eb741a7bf2859318bf4bbfa196afc07", + "sha256": "93244d4d09c5cd6d7a2645fca2482578ad52646f3bb9e6de5ebe505b157fbfea", "role": "Proves Napari streaming updates layers and carries ROI navigation through the declared protocol." }, { "path": "tests/unit/test_viewer_result_element_selection.py", - "sha256": "58480f4da61b427ec1c55560a77686e74df70404fbbf77e701ba386ad9f89a03", + "sha256": "b1dae580f0291f53be9453df6f6c6df5458046bd3ff947b550f7ddea94b29f96", "role": "Proves focused layer isolation is rejected before mutation when any route or axis is invalid." }, { "path": "tests/unit/test_fiji_viewer_server.py", - "sha256": "2d590f629243eec979531b0959a1006690befbd17b9778ec9f4fbefea72877f3", + "sha256": "da575dfb02895ecc831b34d6bdf6b58112b986556459444e9701cbf43221ea22", "role": "Proves Fiji endpoint lifecycle, streamed image updates, and ROI message handling." }, - { - "path": "openhcs/runtime/napari_streaming_handlers.py", - "sha256": "86f4ab319c7c68b626e7e2a1adef81eb3d7244f044b5b3e4dddd78bbee2aa4b6", - "role": "Owns shared semantic axis slots and route-local singleton alignment for Napari layers." - }, { "path": "scripts/gallery_catalog.py", - "sha256": "7f493e0cb305e14ee4cf7d3e995c67df4200186a8a8aaf00eef0bc66d8d191cf", - "role": "Owns the Napari ROI-navigation and Fiji ROI-review scenarios, representative images, captions, descriptions, alt text, dimensions, and release evidence." + "sha256": "7fd3fce2b8fdae1abf049bb4f8a3a310ab598a7f2c507578425be537903ccfca", + "role": "Owns the Napari ROI-navigation and Fiji ROI-review scenarios, captions, and alt text." } ], "findings": [ - "The task guide did not explain how Napari preserves declaration-owned axis sizes across full and component-reduced step outputs.", - "The task guide did not explain the selected-route dimension-label invariant or how to recognise stale overlay state.", - "The task guide did not identify bounded ROI metadata summaries and one-operation layer isolation as the focused agent-review path." + "Step 3 listed batching and well filters among viewer options; streaming configs declare no batching option, and the filter field is partition_filter (G1), so the step now lists persistence, partition filters, transport, and display options.", + "max_rois is declared on openhcs/agent/dto/viewer.py, not on the recorded viewer authorities, so that file was added as an authority." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/user_guide/real_time_visualization.rst in full and compared it against the changed authorities: the page's own partition_filter edit, openhcs/core/config.py, viewer_protocol.py, viewer_controls.py, viewer_window_service.py, napari_viewer_server.py, fiji_viewer_server.py, napari_streaming_handlers.py, the three viewer tests, and gallery_catalog.py; confirmed the extras, the per-step streaming fields, persistent and partition_filter, max_rois, the selected-layer overlay controller, and forward-progress settlement; corrected the step-3 option list.", "Re-read docs/source/user_guide/real_time_visualization.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/config.py; openhcs/runtime/viewer_protocol.py; openhcs/runtime/napari_viewer_server.py; openhcs/runtime/fiji_viewer_server.py; tests/unit/test_napari_streaming_handlers.py; tests/unit/test_fiji_viewer_server.py; openhcs/runtime/napari_streaming_handlers.py.", "Focused cross-lattice QA passed 179 tests spanning callable contracts, framework-device scope, custom functions, catalogue discovery, installer/update policy, MCP release, and packaged OMERO deployment.", diff --git a/docs/source/api/index.rst b/docs/source/api/index.rst index 392e57e4db..7b7ab3b22d 100644 --- a/docs/source/api/index.rst +++ b/docs/source/api/index.rst @@ -66,37 +66,36 @@ Compilation and execution .. code-block:: python + import queue from pathlib import Path from objectstate import ensure_global_config_context from openhcs.core.config import GlobalPipelineConfig from openhcs.core.orchestrator.orchestrator import PipelineOrchestrator + from openhcs.core.progress import set_progress_queue - plate_path = Path("/data/plate").resolve() + dataset_root = Path("/data/plate").resolve() ensure_global_config_context(GlobalPipelineConfig, GlobalPipelineConfig()) orchestrator = PipelineOrchestrator( - plate_path, + dataset_root, pipeline_config=pipeline_config, ).initialize() - execution_bundle = orchestrator.compile_pipelines(pipeline_steps) - - progress_context = { - "execution_id": f"local::{plate_path}", - "plate_id": str(plate_path), - "axis_id": "", - } - progress_queue = ( - execution_bundle.runtime_environment.worker_start - .multiprocessing_context() - .Queue() - ) - - results = orchestrator.execute_compiled_plate( - execution_bundle=execution_bundle, - progress_queue=progress_queue, - progress_context=progress_context, - ) + progress_queue = queue.Queue() + set_progress_queue(progress_queue) + try: + execution_bundle = orchestrator.compile_pipelines(pipeline_steps) + results = orchestrator.execute_compiled_bundle( + execution_bundle=execution_bundle, + progress_queue=progress_queue, + progress_context=orchestrator.progress_context(), + ) + finally: + set_progress_queue(None) + +Compilation and execution emit typed progress events into the queue installed +with ``set_progress_queue``; ``orchestrator.progress_context()`` names the +execution those events report on. Applications should normally let the GUI or an execution service own progress queue lifecycle, cancellation, and result presentation. The explicit call above diff --git a/docs/source/appendices/glossary.rst b/docs/source/appendices/glossary.rst index c55eb509b4..3a3296482c 100644 --- a/docs/source/appendices/glossary.rst +++ b/docs/source/appendices/glossary.rst @@ -24,19 +24,19 @@ Glossary Compiled execution bundle The validated worker-safe steps, frozen processing contexts, and - environment decisions consumed by plate execution. + environment decisions consumed by bundle execution. Compiled step plan - The typed compiler-owned plan for one step on one execution axis. Runtime - code reads it but does not repeat its inference. + The typed compiler-owned plan for one step in one processing context. + Runtime code reads it but does not repeat its inference. Compilation session The cross-step authority for source, artifact, and plan state while a pipeline is compiled. - Execution axis - The component that partitions independent processing contexts, normally - well. + Partition axis + The axis role whose values partition independent processing contexts and + worker lanes; for microscopy, the well. Function pattern A callable, a callable plus keyword arguments, a chain, or a dictionary @@ -58,9 +58,10 @@ Glossary A typed quantitative result associated with images, objects, relationships, wells, or a plate. - Microscope handler - A registered source-format declaration that owns detection, parsing, - metadata projection, virtual-workspace behavior, and compatible backends. + Dataset source + A registered way to open a dataset root, such as a microscope format. It + owns detection, filename parsing, metadata, virtual-workspace behavior, + and compatible backends. Module artifact contract The artifact semantics declared by a module family, including a lowered @@ -79,8 +80,9 @@ Glossary ``FunctionStep`` objects. Processing contract - The semantic locality of a callable, such as plane-local ``PURE_2D`` or - whole-stack ``PURE_3D``. It is independent of array memory type. + The semantic locality of a callable, a member of the + ``ProcessingContract`` family such as plane-local ``Pure2DContract`` or + whole-stack ``Pure3DContract``. It is independent of array memory type. Runtime adapter A declared boundary that converts typed compiler/runtime values into the @@ -102,7 +104,7 @@ Glossary to satisfy a step. Source workspace projection - The virtual source view exposed by a microscope handler and PolyStore to + The virtual source view exposed by a dataset source and PolyStore to initialization and compilation. Variable components diff --git a/docs/source/architecture/abstraction_lattices.rst b/docs/source/architecture/abstraction_lattices.rst index 87bbdc21a0..a3af6cbdfb 100644 --- a/docs/source/architecture/abstraction_lattices.rst +++ b/docs/source/architecture/abstraction_lattices.rst @@ -21,7 +21,7 @@ Start with the semantic question an abstraction answers. Examples include: - Which artifact does this callable produce? - Which CellProfiler module owns this parsed module name? - Which runtime value projection applies to this nominal value type? -- Which microscope handler recognises and projects this source layout? +- Which dataset source recognises and projects this source layout? A valid owner contains facts that cannot be obtained from another declaration. A mirror derives or copies its answer from an existing owner and then exposes @@ -68,17 +68,18 @@ The following boundaries are verified by current declarations and tests: nominal leaves. ``CompiledStepPlan`` adds execution consequences without becoming a second declaration graph. -``MicroscopeHandler`` and ``SourceBindingsConfig`` - These are distinct owners, not duplicates. A handler recognises and projects - a physical or virtual source layout; source bindings select, name, and assign - component identity to sources. ``SourceBindingsHandler`` composes both for - arbitrary declared files. +``DatasetSource`` and ``SourceBindingsConfig`` + These are distinct owners, not duplicates. A dataset source recognises and + projects a physical or virtual source layout; source bindings select, name, + and assign component identity to sources. ``SourceBindingsSource`` composes + both for arbitrary declared files. Strategy selection mixins ``EnumKeyedStrategyMixin``, ``NominalTypeKeyedStrategyMixin``, and - ``MostDerivedContextStrategyMixin`` centralise selection by enum member, - nearest nominal type, or most-derived matching context. Leaves own - specialisation; consumers make one call through the root. + ``MostDerivedContextStrategyMixin``, provided by ``metaclass-registry``, + centralise selection by enum member, nearest nominal type, or most-derived + matching context. Leaves own specialisation; consumers make one call through + the root. ``AgentCapabilityDeclaration`` MCP tool identity, contracts, effects, security facts, and surface @@ -119,11 +120,11 @@ declarations. Source layouts and source bindings ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -A microscope handler and source bindings sometimes mention the same files, but -they answer different questions. The handler determines how a layout becomes a -workspace and which storage backends it supports. Source bindings determine -which declared files satisfy named semantic inputs. Combining them in -``SourceBindingsHandler`` is therefore composition of two authorities, not a +A dataset source and source bindings sometimes mention the same files, but +they answer different questions. The dataset source determines how a layout +becomes a workspace and which storage backends it supports. Source bindings +determine which declared files satisfy named semantic inputs. Combining them in +``SourceBindingsSource`` is therefore composition of two authorities, not a mirror. Collapsing every source concern into either owner would lose information. A diff --git a/docs/source/architecture/analysis_consolidation_system.rst b/docs/source/architecture/analysis_consolidation_system.rst index 4aca5a6f57..abf9836d25 100644 --- a/docs/source/architecture/analysis_consolidation_system.rst +++ b/docs/source/architecture/analysis_consolidation_system.rst @@ -5,20 +5,17 @@ Analysis consolidation is a post-plate operation over materialised tabular artifacts. It is separate from callable execution and does not reopen an output path to rediscover content already recorded by the runtime. -``AnalysisConsolidationConfig`` is resolved into each processing context. -After a compiled plate finishes successfully, -``consolidate_analysis_outputs`` reads the configuration from the compiled -contexts, projects the exact CSV content recorded in that execution's runtime -observations, groups it by materialisation directory, and delegates to the -analysis consolidation functions. Each directory retains its compiled persistent -destination. Consolidation therefore neither scans a -results directory nor treats a virtual backend address as a host file. - -Output selection is passed into runtime materialisation. A writer with a declared -runtime-path projection can skip rendering when none of its outputs match the -consolidation selection. ROI paths derive from the same target request used by -the ROI writer, so consolidating tables does not regenerate unrelated contours. -Writers without that projection filter their actual rendered outputs. +Consolidation is a microscopy post-execute hook. ``AnalysisConsolidationHook`` +is a ``PostExecuteHook`` bound from the global ``AnalysisConsolidationConfig`` +and ``PlateMetadataConfig`` and carried by each compiled processing context. +When a step saves CSV outputs, the hook records the exact text the writer saved, +keeping only files the consolidation selection includes; explicitly reused debug +outputs are read back through their compiled backend. These observations travel +in each context's runtime observation. After every partition succeeds, they are +combined, grouped by materialisation directory, and passed to the analysis +consolidation functions. Each directory retains its compiled persistent +destination. Consolidation therefore neither renders outputs again, scans a +results directory, nor treats a virtual backend address as a host file. The generic consolidation layer reads ``AnalysisTableSource`` values and writes through an ``AnalysisSummaryWriter`` ABC. Local desktop consolidation uses a @@ -41,7 +38,7 @@ Ownership rendering, and summary filenames. Consolidation never infers missing artifacts from step names and does not mutate -compiled plans. A source or destination failure is reported as a plate execution +compiled plans. A source or destination failure is raised as an execution failure rather than logged and discarded. Manual desktop consolidation uses the same analysis functions on an explicitly selected results directory. diff --git a/docs/source/architecture/artifact_contract_system.rst b/docs/source/architecture/artifact_contract_system.rst index f2f71b050d..57295bbd39 100644 --- a/docs/source/architecture/artifact_contract_system.rst +++ b/docs/source/architecture/artifact_contract_system.rst @@ -105,8 +105,9 @@ not present an ROI-shaped base path as though it were also the CSV output. Callable ABI versus semantic artifacts -------------------------------------- -The ``special_inputs`` decorator names callable ABI slots. They do not by themselves declare artifact types, producers, -materialisation, or runtime-store identity. +The ``special_inputs`` decorator names callable ABI slots. Those names do not +by themselves declare artifact types, producers, materialisation, or +runtime-store identity. Semantic ownership comes from ``artifact_inputs`` and ``artifact_outputs`` on the callable. A CellProfiler module derives those declarations through its @@ -182,8 +183,8 @@ records: - the exact declared ``ArtifactSpec`` - an optional prior-producer ``ArtifactInputPlan`` and runtime projection -- whether the occurrence consumes ordinary main flow -- which ``MainFlowInputProjection`` applies +- the ``MainFlowInputProjection`` that applies when the occurrence consumes + ordinary main flow This is why two callable parameters can consume the same semantic artifact without collapsing into one edge. ``CompiledStepPlan.artifact_inputs`` remains @@ -193,8 +194,8 @@ on the compiled invocation edges. Implicit main-flow projection ----------------------------- -An invocation edge with ``consumes_main_flow=True`` cannot also carry a storage -plan. Its projection is one of two explicit modes: +An invocation edge that consumes main flow carries a ``main_flow_projection`` +and no storage plan. Its projection is one of two explicit modes: ``MainFlowInputProjection.COMPLETE_PAYLOAD`` The invocation has one main-flow reference and consumes the current payload @@ -207,9 +208,9 @@ plan. Its projection is one of two explicit modes: The end-to-end regression in ``tests/unit/test_implicit_main_flow_compilation.py`` compiles grouped native ``percentile_normalize`` output into a CellProfiler ``Threshold`` consumer. It -asserts the step dependency, channels ``1``/``2``/``4`` group scope, -``consumes_main_flow=True``, absence of a storage plan, and axis execution -scope. +asserts the step dependency, channels ``1``/``2``/``4`` group scope, a +main-flow projection on the input edge, absence of a storage plan, and axis +execution scope. Compiled plans -------------- diff --git a/docs/source/architecture/cellprofiler_interop.rst b/docs/source/architecture/cellprofiler_interop.rst index 1354a60a4f..c8d73ec3c5 100644 --- a/docs/source/architecture/cellprofiler_interop.rst +++ b/docs/source/architecture/cellprofiler_interop.rst @@ -148,8 +148,9 @@ runtime retry, or silent substitution with an unrelated backend. Exact compile-time reconstruction --------------------------------- -``CellProfilerInvocationContractProviderFactory`` walks the resolved -``StepSnapshot`` sequence and one forward ``ArtifactDeclarationStepContext``: +``CellProfilerInvocationContractProviderFactory`` walks the steps of the +``ResolvedPipelineDefinition`` with one forward +``ArtifactDeclarationStepContext``: .. code-block:: text diff --git a/docs/source/architecture/compilation_system_detailed.rst b/docs/source/architecture/compilation_system_detailed.rst index 7e8f4904d9..e441f129b5 100644 --- a/docs/source/architecture/compilation_system_detailed.rst +++ b/docs/source/architecture/compilation_system_detailed.rst @@ -3,5 +3,5 @@ Detailed compiler transition The former fixed-phase and string-keyed compiler narrative is retained only in the documentation archive. See :doc:`pipeline_compilation_system` for the -current ``StepSnapshot``, ``CompilationSession``, typed-plan, and -``CompiledExecutionBundle`` architecture. +current ``ResolvedPipelineDefinition``, ``CompilationSession``, typed-plan, +and ``CompiledExecutionBundle`` architecture. diff --git a/docs/source/architecture/component_system_integration.rst b/docs/source/architecture/component_system_integration.rst index 62493f7a5e..d6c327d246 100644 --- a/docs/source/architecture/component_system_integration.rst +++ b/docs/source/architecture/component_system_integration.rst @@ -5,7 +5,7 @@ The microscope/source model discovers component values for a plate. The compiler carries those values into source universes, execution-axis contexts, stack construction, grouping, and filtering. -The same component declarations have distinct jobs: +The step, pipeline, and family declarations have distinct jobs: ``variable_components`` Determines what each position on the transported third array axis means. @@ -16,8 +16,9 @@ The same component declarations have distinct jobs: ``sequential_components`` Chooses dimensions processed in bounded sequential slices. -``multiprocessing_axis`` - Partitions independent compiled contexts, normally by well. +The partition axis + The family's single ``PartitionAxis``; it partitions independent compiled + contexts, by well in the microscopy family. Metadata caches and source projections are the authority for values available on a particular plate. UI selectors, compiler stages, and runtime filters query diff --git a/docs/source/architecture/component_validation_system.rst b/docs/source/architecture/component_validation_system.rst index 2905286bb7..8089c6502b 100644 --- a/docs/source/architecture/component_validation_system.rst +++ b/docs/source/architecture/component_validation_system.rst @@ -1,22 +1,26 @@ Component validation ==================== -Component declarations describe semantic microscope dimensions such as well, -site, channel, Z, and time point. Each is an ``Axis`` class declared by the -active ``AxisFamily``; subsets such as the variable axes are family queries, -and a step's absent grouping is the explicit ``Ungrouped`` declaration. +Component declarations describe the data's dimensions; the microscopy family +declares well, site, channel, Z, and time point. Each is an ``Axis`` class +declared by the active ``AxisFamily``; subsets such as the variable axes are +family queries, and a step's absent grouping is the explicit ``Ungrouped`` +declaration. Validation decodes external names through ``AxisFamily.named`` and rejects unknown or semantically invalid combinations. It checks, among other things, that: -- configured variable components exist in the source universe; +- a step's variable and grouping axes are variable axes of the family (every + axis but the partition axis), and the grouping axis is not also a variable + axis; +- dictionary-pattern keys exist among the dataset's values for the grouping + axis; - grouping is allowed by every callable contract in the pattern; -- required callable axes are present; -- sequential and multiprocessing axes do not create an invalid execution - topology. +- the step's variable axes cover every axis role a callable requires; +- no pipeline sequential axis is also a step's variable axis. -Consumers should use the shared conversion/validation helpers and nominal enum -members. They must not maintain copied component-name sets. See +Consumers should use the shared validation helpers and the axis classes +themselves. They must not maintain copied component-name sets. See :doc:`../concepts/data_dimensions` and :doc:`processing_semantics`. diff --git a/docs/source/architecture/concurrency_model.rst b/docs/source/architecture/concurrency_model.rst index 491f8e843f..a48f743858 100644 --- a/docs/source/architecture/concurrency_model.rst +++ b/docs/source/architecture/concurrency_model.rst @@ -3,33 +3,36 @@ Concurrency model OpenHCS compiles the selected execution axes before starting workers. A ``CompiledExecutionBundle`` carries the worker-safe steps, frozen -``ProcessingContext`` instances, and environment decisions for one plate. +``ProcessingContext`` instances, and environment decisions for one dataset. Execution units --------------- -The configured multiprocessing axis normally yields one context per well. -Source discovery determines the available axis values; compiler stages create -the corresponding typed step plans. Workers receive one compiled context at a -time and never share mutable compiler state. +The active family's partition axis yields one context per value; in the +microscopy family that is one context per well. Source discovery determines the +available axis values; compiler stages create the corresponding typed step +plans. Each worker executes one lane, the compiled contexts of the partition +values it owns, one context at a time, and never shares mutable compiler state. Each frozen ``ProcessingContext`` owns its multiprocessing ``axis_id``. Mapping keys distinguish runtime contexts, including sequential combinations, but do not encode an axis contract. Bundle projections and worker-lane grouping read the context-owned identity instead of parsing those keys. -``compiled_plate_execution`` owns the plate-level execution lifecycle. +``bundle_execution`` owns the execution lifecycle of one compiled bundle. ``worker_execution`` owns executor construction, bundle installation, task -submission, result collection, cancellation, and cleanup. ``worker_lanes`` -provides the process-local bundle authority used by worker code. +submission, result collection, cancellation, and cleanup; process workers serve +a typed ``WorkerCommand`` family over their connection to the parent. +``worker_lanes`` provides the process-local bundle authority used by worker +code. At execution, each context binds its ``WorkerLaneExecutionContext`` directly. The lane identity references the execution's worker-assignment mapping, without carrying other workers' processing contexts. Progress projects the current -lane's owned axes; streaming projects the complete assigned axis domain so that +lane's owned partition values; streaming projects the complete assigned axis domain so that every worker declares the same viewer coordinates. ``ProcessingContext`` does -not retain separate copies of the lane's execution, plate, worker, or ownership -fields. +not retain separate copies of the lane's execution, dataset, worker, or +ownership fields. Start-method policy ------------------- @@ -42,7 +45,8 @@ default to ``spawn`` unless an explicit unsafe override is allowed. Progress and cancellation ------------------------- -Workers emit typed progress events to the execution's queue. The parent and ZMQ +Workers emit typed progress events to the execution's progress queue; process +workers forward them over their parent connection. The parent and ZMQ runtime project those events for UI consumers. ZMQRuntime owns the terminal execution status and applies cancellation only to the requested execution. OpenHCS owns an execution-scoped cancellation authority: inline and threaded @@ -52,8 +56,9 @@ boundary before stopping. Cancellation never becomes success or failure when late worker completion or an exception reaches the server, and every path still runs cleanup. Worker ``AXIS_COMPLETED`` events close only the axis-scoped portion. Terminal -plate steps, output consolidation, viewer settlement, and final state projection -remain parent-owned; :doc:`progress_runtime_projection_system` defines the +dataset-scoped steps, post-execute hooks such as the microscopy domain's output +consolidation, viewer settlement, and final state projection remain +parent-owned; :doc:`progress_runtime_projection_system` defines the separate final ``SUCCESS`` boundary. Invariants diff --git a/docs/source/architecture/declarative_window_system.rst b/docs/source/architecture/declarative_window_system.rst index a5bf624c76..4c4872757d 100644 --- a/docs/source/architecture/declarative_window_system.rst +++ b/docs/source/architecture/declarative_window_system.rst @@ -10,6 +10,8 @@ See :external+pyqt-reactive:doc:`window manager usage `. OpenHCS retains stable product window identities, domain factories, ObjectState -scope selection, and pipeline/execution actions. Their cross-package composition +scope selection, and the session views and operations behind the dataset, +pipeline, and execution windows; a session-view window takes its title from the +view it renders. Their cross-package composition is documented in :doc:`ui_services_architecture`; package ownership and migration policy are in :doc:`external_foundations`. diff --git a/docs/source/architecture/experimental_analysis_system.rst b/docs/source/architecture/experimental_analysis_system.rst index 9b42a4324f..c5fe0fb095 100644 --- a/docs/source/architecture/experimental_analysis_system.rst +++ b/docs/source/architecture/experimental_analysis_system.rst @@ -10,13 +10,13 @@ compilation or execution. Current ownership ----------------- -``ExperimentalAnalysisConfig`` owns the standalone engine options and all five -filenames used by the directory workflow. It is deliberately not a global or -pipeline configuration. ``ExperimentalAnalysisEngine`` owns format selection +``ExperimentalAnalysisConfig``, declared by the microscopy domain, owns the +standalone engine options and all five filenames used by the directory +workflow. It is deliberately not a global or pipeline configuration section. ``ExperimentalAnalysisEngine`` owns format selection and orchestration; its ``run_directory`` projection derives every input and output path from that configuration. The desktop action constructs the default configuration and calls the same engine boundary rather than restating those -filenames or using the deprecated compatibility wrapper. +filenames. ``NormalizationMethod`` owns the fold-change, control z-score, and percentage-of-control leaf calculations. The shared analysis module traverses diff --git a/docs/source/architecture/external_foundations.rst b/docs/source/architecture/external_foundations.rst index 9a3951b29e..e7c319eeb2 100644 --- a/docs/source/architecture/external_foundations.rst +++ b/docs/source/architecture/external_foundations.rst @@ -18,8 +18,9 @@ ObjectState ArrayBridge Owns memory-type taxonomy, callable memory metadata and decorators (including input, output, and execution memory), device-neutral array geometry, array - conversion, stack utilities, device streams, OOM behaviour, and the child- - process environment required to import each framework consistently. OpenHCS + conversion, per-memory-type native array operations, stack utilities, device + streams, OOM behaviour, and the child-process environment required to import + each framework consistently. OpenHCS projects those declarations into callable contracts and process launches, and owns semantic image shape roles, compiled conversion and framework-device plans, worker resource lifecycle, and resource policy. See @@ -27,8 +28,9 @@ ArrayBridge metaclass-registry Owns ``AutoRegisterMeta``, ``RegistryConfig``, ``RegistryFamily``, discovery, - generic caching, and discovery imports that preserve host-owned root logging. - OpenHCS owns its nominal roots, declarations, and shared selection mixins. See + discovery imports that preserve host-owned root logging, the shared strategy + and selection mixins, and the bounded and process-local cache family. + OpenHCS owns its nominal roots and declarations. See :external+metaclass-registry:doc:`metaclass-registry documentation `. PolyStore @@ -52,15 +54,18 @@ pyqt-reactive python-introspect Owns callable/signature analysis, wrapped-target resolution, parameter - metadata, callable-declaration projection, and analyzer extensions. OpenHCS + metadata, callable-declaration projection, analyzer extensions, the + ``to_jsonable`` encoder paired with the ``dataclass_from_mapping`` decoder, + and declared public-API and lazy-export helpers. OpenHCS owns domain exclusions, callable contracts, and step-editor consumption. See the `python-introspect repository `_. ZMQRuntime Owns generic process lifecycle, exact spawned-process ownership groups, - automatic child reaping, request/status/progress protocols, typed connection - and operation cancellation, and viewer-control transport. + automatic child reaping, the background-process launch policy, + request/status/progress protocols, typed connection and operation + cancellation, and viewer-control transport. OpenHCS owns mapping compiled execution, source axes, runtime observations, and application services onto those protocols. See the `ZMQRuntime repository `_. diff --git a/docs/source/architecture/external_integrations_overview.rst b/docs/source/architecture/external_integrations_overview.rst index 5acf8428c2..03817fcc86 100644 --- a/docs/source/architecture/external_integrations_overview.rst +++ b/docs/source/architecture/external_integrations_overview.rst @@ -8,7 +8,8 @@ parallel pipeline, source, artifact, or execution model. CellProfiler ------------ -The ``.cppipe`` importer parses CellProfiler modules and lowers them to +The ``.cppipe`` importer, registered from interop as a member of the kernel's +``PipelineImporter`` family, parses CellProfiler modules and lowers them to ``PipelineConfig`` plus ``FunctionStep`` declarations. Module declarations own their callable, artifact, and headless-import semantics; interactive-only modules fail explicitly. See :doc:`cellprofiler_interop`. @@ -16,8 +17,9 @@ modules fail explicitly. See :doc:`cellprofiler_interop`. Microscope and storage sources ------------------------------ -Registered ``MicroscopeHandler`` implementations connect filename/metadata -semantics to PolyStore source references and virtual workspaces. OpenHCS owns +Registered ``DatasetSource`` implementations, such as the microscope handlers, +connect filename/metadata semantics to PolyStore source references and virtual +workspaces. OpenHCS owns source selection and compiler bindings; PolyStore owns generic storage. See :doc:`microscope_handler_integration` and :doc:`source_model`. diff --git a/docs/source/architecture/gpu_resource_management.rst b/docs/source/architecture/gpu_resource_management.rst index d3cfa8ab18..ec1a1052e2 100644 --- a/docs/source/architecture/gpu_resource_management.rst +++ b/docs/source/architecture/gpu_resource_management.rst @@ -21,8 +21,8 @@ instead of preparing the complete catalogue. Fresh catalogue-preparation and execution-server processes receive ``MemoryType.subprocess_environment()``. That projection combines each member's declared import defaults and member-owned native-library paths before the child -imports any framework, so parent and child admission do not depend on which GPU -library happened to be imported first in the parent. +imports any framework, so parent and child framework imports do not depend on +which GPU library happened to be imported first in the parent. The desktop ``--no-gpu`` option and ``OPENHCS_CPU_ONLY=true`` reject a GPU footprint before optional frameworks are imported. diff --git a/docs/source/architecture/list_item_preview_system.rst b/docs/source/architecture/list_item_preview_system.rst index 686c9fc248..d74378ea24 100644 --- a/docs/source/architecture/list_item_preview_system.rst +++ b/docs/source/architecture/list_item_preview_system.rst @@ -12,7 +12,7 @@ OpenHCS integration OpenHCS registers one formatter for the ``PartitionFilterConfig`` declaration family. Config classes own their preview labels and whether an enabled member -remains visible without an explicit well selection. ``PartitionFilterMode`` members +remains visible without an explicit partition filter. ``PartitionFilterMode`` members own the compact include or exclude operator. The formatter resolves those declarations through pyqt-reactive's preview API; it does not scan unrelated registries or reproduce subtype tables. diff --git a/docs/source/architecture/mcp_distribution.rst b/docs/source/architecture/mcp_distribution.rst index a57c681288..2c885e892f 100644 --- a/docs/source/architecture/mcp_distribution.rst +++ b/docs/source/architecture/mcp_distribution.rst @@ -43,7 +43,10 @@ Local process lane ------------------ The local MCP server runs over stdio and is owned by the client process. It -does not import or host the PyQt UI. GUI, viewer, and local runtime capabilities +hosts no GUI: the transport keeps a Qt core application on the process main +thread for main-thread-owned ObjectState work and runs the MCP SDK loop on a +worker thread, so protocol notifications continue while main-thread work runs. +GUI, viewer, and local runtime capabilities remain local-only and attach through their authenticated process bridges. UI-bridge and viewer control budgets remain facts of their respective connection declarations. MCP timeout policies bound caller overrides from @@ -57,8 +60,9 @@ does not import every processing library to rebuild a second catalogue. At MCP process startup, the composition root snapshots the persisted desktop ZMQ configuration so a non-default host, port, or transport remains the same endpoint identity for both authoring surfaces. That snapshot is projected from -the unique ``OpenHCSZMQConfig`` field in the canonical ``UIConfig`` Python -document. The projection derives the field from the dataclass schema and does +the unique ``OpenHCSZMQConfig`` field in the saved ``UIConfig`` Python +document. The headless session that MCP session tools act on connects through +that same endpoint configuration. The projection derives the field from the dataclass schema and does not construct unrelated UI configuration or import Qt. The stdio transport reserves process stdin and stdout before constructing the @@ -106,8 +110,8 @@ measured-pipeline receipt is produced only after an ordinary pipeline's selected value or outcome export passes its validation. Outcome-only export retains per-axis status without claiming value equivalence. The expert-only -``openhcs_finalize_measured_pipeline_run`` capability consumes the exact -completed ordinary job, validates the selected export, and retains the submitted +``openhcs_finalize_measured_pipeline_run`` capability consumes the OpenHCS +session's exact finished execution, validates the selected export, and retains the submitted source, server result and timing receipt. It does not create another execution or status authority. ``openhcs_inspect_measured_pipeline_run`` checks that receipt, submitted-source digests, and retained observation/summary digests; @@ -190,16 +194,16 @@ The shared registered JSON projection owns Cursor, Gemini CLI, and Windsurf user configuration; Visual Studio Code is registered through its documented command-line interface rather than a guessed profile path. -``DesktopDeploymentAuthority`` owns the native launcher, shortcut, and -application-icon projection. Its registered Windows and macOS leaves read the -installed distribution entry points and packaged brand assets. Both initial -setup and the in-application updater call this authority; the PowerShell and +``DesktopDeployment`` in ``openhcs.desktop.deployment`` owns the native +launcher, shortcut, and application-icon projection. Its registered Windows and +macOS leaves read the installed distribution entry points and packaged brand +assets. Both initial setup and the in-application updater call it; the PowerShell and shell bootstrap adapters do not carry parallel launcher or app-bundle templates. On Windows, the stable GUI launcher is a forwarding cache over the separately published current-environment pointer. If another launch still holds that -executable without delete sharing during an update, the Windows authority +executable without delete sharing during an update, the Windows deployment retains the valid launcher, publishes the verified environment, and reports the deferred launcher and fingerprint paths. A later installer or desktop refresh retries those projections after the executable is released. Other publication @@ -209,9 +213,12 @@ The projection owns only the ``openhcs`` server entry and preserves every other client setting. Its command targets the stable launcher, never a version-stamped private environment, so a verified installer update can switch the environment without rewriting client configuration. Registration does not -copy capability lists, path-policy roots, client credentials, or agent +copy capability lists, path-policy roots, client credentials, or server instructions. Those remain owned by the MCP initialisation handshake, -``AgentPathPolicy``, and the client itself. +``AgentPathPolicy``, and the client itself. A client target that declares a +documented skill directory (currently Codex) also receives the installed +OpenHCS skill bundle; the registration report records that synchronisation +separately from the client's MCP connection result. The launcher projects two lifecycle values into the MCP process: one JSON argv for reconnecting through that same stable adapter and one installer-owned @@ -232,8 +239,9 @@ they never contain MCP tool-name lists. The installed server defaults to ``desktop`` while the development client explicitly uses ``full`` for complete surface audits. The desktop profile includes both UI-owned workflows and independent headless execution. Those routes share pipeline declarations but -retain separate state owners: a headless session does not create Plate Manager -rows or ObjectState history. Capability discovery and bound tools/resources are +retain separate state owners: the headless session keeps its datasets and +pipelines in the MCP process's own ObjectState, so it does not create Plate +Manager rows or ObjectState history in a running desktop application. Capability discovery and bound tools/resources are filtered by the same ``AgentCapabilitySurfaceSelection`` instance. The development client resolves each requested tool to its capability @@ -326,20 +334,22 @@ invocation and writes a delayed stack diagnostic to stderr. This exposes a blocked worker even when the event loop continues emitting liveness updates. The diagnostic does not alter the operation's result or execution context. -Source-backed orchestrator-session creation remains on the MCP process's main -thread because its import and compiler-facing setup is thread-sensitive. The -binder emits the standard ``started`` progress event before entering that -synchronous section, and the service logs its source-parse start and -completion. The event loop cannot emit periodic progress while the synchronous -main-thread operation is blocked. None of those signals is scientific percent -completion or success; the client must still wait for the actual typed result. - -Session creation and submitted execution are separate phases. The development -``execute-source`` command first creates the source-backed session, then submits -that exact session. Automation that must remain observable over a long run -submits without an opaque aggregate wait and polls the typed execution-status -capability until a terminal state. The initial progress event and the -start/completion diagnostics expose the pre-submit phase; execution status, +Capabilities whose work touches ObjectState or other thread-sensitive setup +declare that they are not worker-thread safe. The binder emits the standard +``started`` progress event and then runs them on the process main thread +through the main-thread dispatcher; because the SDK loop runs on its own +thread, periodic liveness updates continue meanwhile. None of those signals is +scientific percent completion or success; the client must still wait for the +actual typed result. + +Each headless session operation is one MCP tool on the OpenHCS session: +adding datasets, setting a dataset's pipeline source, and initializing, +compiling, or running it. An operation returns the event sequence it started +at, and ``openhcs_session_events`` returns the events published after that +sequence. The development ``execute-source`` command chains those operations +and follows each stage through session events; ``--no-wait`` returns once the +run is accepted. Automation that must remain observable over a long run follows +session events instead of an opaque aggregate wait. Execution status, cancellation, results, and viewer settlement remain runtime-service responsibilities. @@ -350,7 +360,7 @@ Distribution metadata ``scripts/sync_mcp_release_metadata.py`` projects it into the Codex plugin, Claude MCPB, and ``server.json``. Its dependency-free phase validates only directly readable package/version structure. After the built wheel is installed, -``--capability-requirements`` imports the canonical selected capability registry +``--capability-requirements`` imports the selected capability registry and verifies that all declaration-owned required extras are projected. These checked artifacts are install-surface metadata only; they do not own OpenHCS behaviour. @@ -368,7 +378,7 @@ manager. The same profile projects its binary-wheel policy into both paths, so platform compatibility lines remain one package declaration and both native consumers require the reviewed wheel-backed releases. -Canonical MCP knowledge documents remain in the documentation tree. The wheel +MCP knowledge documents remain in the documentation tree. The wheel build deterministically projects the manifest-declared sources into package resources, so installed servers do not depend on a source checkout and there is no manually maintained second document corpus. diff --git a/docs/source/architecture/measurement_equivalence_system.rst b/docs/source/architecture/measurement_equivalence_system.rst index 2f4166221b..c13e359bbd 100644 --- a/docs/source/architecture/measurement_equivalence_system.rst +++ b/docs/source/architecture/measurement_equivalence_system.rst @@ -25,9 +25,12 @@ Measurement identity -------------------- ``RuntimeMeasurementFeatureKey`` separates subject, feature, source -qualification, and aggregate identity. ``RuntimeMeasurementDialect`` declares -how a producer encodes source names, qualifiers, aliases, and row layout. This -lets two outputs be normalised without erasing meaningful distinctions. +qualification, and aggregate identity. A registered ``MeasurementDialect`` +declares how a producer names sample and object fields, scopes, feature +prefixes, aliases, and row qualifiers; ``PlainMeasurementDialect`` is the +kernel's own spelling and CellProfiler declares its dialect in its interop +package. This lets two outputs be normalised without erasing meaningful +distinctions. Row projection derives stable identities for image-, object-, and relationship-scoped measurements. Wide and long-form tables project into the @@ -65,12 +68,19 @@ Output counts, measurement features/content, tables, and images. An empty difference tuple means the compared outputs are semantically equivalent. +The product package ``openhcs.core.equivalence`` keeps only the measurement +semantics that production code uses: feature keys, semantic profiles, rows, +relationships, and the policy. Output snapshots, comparison, and the report +live in ``benchmark/equivalence`` beside the benchmark code that consumes them; +no product module imports the benchmark package. + Official30 evidence boundary ---------------------------- The portable acceptance definition is -``benchmark/manifests/official30_portable_axis1.json`` plus the two integration -tests in ``tests/integration/test_cellprofiler_official30_zmq.py``. The headless +``benchmark/manifests/official30_portable_axis1.json`` plus the headless and +Napari integration tests in +``tests/integration/test_cellprofiler_official30_zmq.py``. The headless test loads exactly 30 cases, requires native references, runs every case with ``continue_on_error=True``, and requires every native/OpenHCS execution to succeed and every observation to be equivalent. The Napari test exercises the @@ -93,7 +103,7 @@ developer account: Use the sibling ``test_official30_nonpersistent_napari_isolated_per_case`` target for the Napari route. Different ZMQ/viewer ports avoid endpoint conflict, but overlapping runs -remain diagnostic only and must not be reported as canonical acceptance timing. +remain diagnostic only and must not be reported as acceptance timing. Compared modalities and policy ------------------------------ @@ -110,9 +120,8 @@ exported files; the outcome receipt alone does not establish it. Current exports also retain the compiler's exact axis membership: a missing compiled axis or an outcome for an uncompiled axis invalidates the run even if every retained outcome reports success. The measured-run finaliser refuses an -empty execution. Archived outcome-only exports that predate this membership -field remain readable, but their observed axes alone cannot prove complete -compiled coverage and cannot be finalised as a new success receipt. +empty execution. Outcome and observation exports written under an earlier +schema version are rejected when read. Repeated measurements can share one connected ordinary execution server while each run compiles its own artifact and retains a distinct observation and receipt. The benchmark does not create a second execution path to keep that @@ -120,8 +129,7 @@ server warm. Full-value observation expectations retain the compiled axis that owns each artifact kind. A plate-scoped export is checked on its one owning axis rather -than required independently on every image axis. Archived observation exports -keep their earlier all-axis expectation when read. +than required independently on every image axis. A typed completed-run receipt derives execution identity, source/configuration digests, runtime-observation and execution-summary digests, and output references @@ -134,8 +142,8 @@ The CellProfiler adapter prepares one such submission and then applies its optional native-reference equivalence policy. It does not own a second execution engine. An agent can finalise the same receipt after a normal headless job completes: -the execution service supplies the exact submission, endpoint handshake and -server completion record, while the benchmark extension validates and retains +the OpenHCS session's finished execution supplies the exact request, endpoint +handshake and server completion record, while the benchmark extension validates and retains the observation. Both ordinary export scopes carry the producing server execution ID. The shared finaliser rejects an export from a different job before writing a success receipt; the runtime export writer also refuses to @@ -151,7 +159,10 @@ server job, not progress events, supplies the execution-only duration. A missing progress event leaves that diagnostic interval absent; client wait time is not used to guess it. Server, progress, and client wait intervals overlap, so additive phase totals count the client submit/wait phases but not their nested -server or progress observations. +server or progress observations. Benchmark validation phases (runtime +validation, output snapshots, and equivalence comparison) are also excluded +from the additive total because they check a run rather than operate either +tool. If finalisation stops between evidence files, inspection derives the ``unreceipted_artifacts`` list from the measured-artifact declarations. Their diff --git a/docs/source/architecture/microscope_handler_integration.rst b/docs/source/architecture/microscope_handler_integration.rst index 7164dec998..413d2473bb 100644 --- a/docs/source/architecture/microscope_handler_integration.rst +++ b/docs/source/architecture/microscope_handler_integration.rst @@ -1,75 +1,84 @@ -Microscope handler integration -============================== +Dataset source integration +========================== -``MicroscopeHandler`` is the nominal root for microscope/source-specific -behaviour. Concrete handlers self-register through ``AutoRegisterMeta`` and are -loaded by package discovery. ``create_microscope_handler`` selects an explicit -handler or performs ordered detection through the registered family. +``DatasetSource`` is the nominal root for format-specific dataset behaviour. +Concrete sources self-register through ``AutoRegisterMeta`` under their +``source_name``. The registry discovers the kernel's own members in +``openhcs.core.dataset_sources`` and then imports the modules the active +``AxisFamily`` lists in ``extension_modules``; for microscopy these include +``openhcs.microscopes``. The kernel never imports a domain module by name. Owned declarations ------------------ -A handler owns: +A source owns: -- its microscope identity and detection behaviour; +- its ``source_name`` identity, selection role, and detection behaviour; - filename parsing and metadata-handler type; - virtual-workspace root and source projection; - compatible storage backends; -- viewer metadata required for component-aware display. +- viewer metadata required for axis-aware display. -The handler registry is the source of truth. Generic discovery, compiler, and -UI code iterate ``MicroscopeHandler.__registry__.values()`` or the registry -service. They must not maintain a copied microscope list or import concrete -handler modules to inspect names. +The registry is the source of truth. Generic discovery, compiler, and UI code +iterate ``DatasetSource.__registry__`` or ``DatasetSourceChoice.choices()``. +They must not maintain a copied source list or import concrete source modules +to inspect names. -The configurable ``Microscope`` members, excluding ``AUTO``, match the -registered handler keys exactly. This keeps user-facing choices, agent -discovery, and runtime construction on the same declaration set without hidden -handlers or orphaned enum values. +Selection +--------- -The factory binds the root-declared ``plate_folder`` context uniformly after a -registered leaf is created. Leaf-specific initialisation belongs on that -leaf's ``create`` hook; the factory does not dispatch on concrete handler -names. +``PipelineConfig.dataset_source`` holds a ``DatasetSourceChoice`` class: +``AutoDetectedSource`` (spelled ``auto``) or one registered source. Boundaries +spell the choice by ``source_name``. Automatic detection walks +``DatasetSource.detection_order()``: prepared OpenHCS workspaces first, then +sources ranked by their selection-role mixin. Selection roles such as +``FormatSpecificSource``, ``DeclaredFileSource``, and +``PreparedWorkspaceSource`` are capability mixins on the source class. + +``DatasetSourceChoice.open`` resolves the source class, lets declared source +bindings select the declared-file source when the role allows it, constructs +the leaf through its ``create`` hook, and binds ``dataset_root`` uniformly. +Leaf-specific initialisation belongs on that leaf's ``create`` hook; the opener +does not dispatch on concrete source names. Source and compiler boundary ---------------------------- -After initialisation, the handler supplies plate metadata and source files to +After initialisation, the source supplies dataset metadata and source files to the source-workspace projection. The compiler uses that projection to construct source universes, execution-axis values, main-flow dependencies, and named source bindings. PolyStore owns the underlying references, virtual paths, and backend I/O. -A handler may use a decoder-owned metadata handler while discovering exact -source planes. Once it persists the canonical virtual workspace, its active +A source may use a decoder-owned metadata handler while discovering exact +source planes. Once it persists the virtual workspace, its active ``metadata_handler`` becomes the workspace metadata owner. Consumers therefore -read one post-initialisation authority instead of conditionally combining raw -store coordinates with projected workspace coordinates. +read one post-initialisation metadata handler instead of conditionally +combining raw store coordinates with projected workspace coordinates. ``MetadataHandler.component_value_set()`` returns one complete ``MetadataComponentValueSet`` keyed by the axes of the active ``AxisFamily``. -Handlers implement that single projection; generic consumers -iterate the family instead of calling one handler method per concrete -axis. Viewer and serialized mappings receive string keys only at their explicit -boundary. +Metadata handlers implement that single projection; generic consumers iterate +the family instead of calling one method per concrete axis. Viewer and +serialized mappings receive string keys only at their explicit boundary. -The persisted workspace records the handler declaration that produced its -backing references. When the workspace is reopened, that registered handler -class re-registers its decoder backends together with the plate-specific -virtual-workspace backend. Replay therefore follows the original declaration +The persisted workspace records the ``source_name`` of the source that produced +its backing references. When the workspace is reopened, that registered source +class re-registers its decoder backends together with the dataset-specific +virtual-workspace backend. Replay therefore follows the recorded declaration without reconstructing a configuration-dependent ingestion instance or -requiring the original ``SourceBindingsConfig``. +requiring the ``SourceBindingsConfig`` used at ingestion. Extension boundary ------------------ -A new microscope integration is a registered handler leaf with its parser, -metadata handler, identity, detection policy, and backend compatibility at the -owning module. Dataset-specific ordinary-file layouts whose selection and axes -can be declared with ``SourceBindingsConfig`` use the source-binding handler; -the dataset name is not registered as a microscope type. Generic discovery and -compilation consume the resulting declaration. +A new format integration is a registered ``DatasetSource`` leaf, in a module +listed by the active family's ``extension_modules``, with its parser, metadata +handler, identity, selection role, and backend compatibility at the owning +module. Dataset-specific ordinary-file layouts whose selection and axes can be +declared with ``SourceBindingsConfig`` use the source-binding source +(``source_bindings``); the dataset name is not registered as a source. +Generic discovery and compilation consume the resulting declaration. The task-oriented procedure and required tests are in :doc:`../development/source_binding_extension`; the ownership rule is expanded in :doc:`nominal_ownership`. diff --git a/docs/source/architecture/multiprocessing_coordination_system.rst b/docs/source/architecture/multiprocessing_coordination_system.rst index aa1f084c50..18e01cadd0 100644 --- a/docs/source/architecture/multiprocessing_coordination_system.rst +++ b/docs/source/architecture/multiprocessing_coordination_system.rst @@ -4,22 +4,25 @@ Multiprocessing coordination Multiprocessing is coordinated from compiled execution facts rather than from a second task-planning model. -1. The microscope/source model discovers execution-axis values. +1. The dataset source discovers the values of the active axis family's + partition axis. 2. ``PipelineCompiler`` produces a frozen ``ProcessingContext`` for each value. -3. Bundle assembly validates the contexts and worker-safe steps. +3. Bundle execution validates the contexts and worker-safe steps. 4. The worker-start policy chooses one multiprocessing context for the entire execution. -5. The executor submits one task per compiled axis and collects typed - ``ExecutionResult`` values. +5. The lane planner assigns the compiled axis contexts to deterministic worker + lanes. The executor submits one task per active lane and collects a typed + ``ExecutionResult`` for each axis. -Worker processes install the ``CompiledExecutionBundle`` through the worker-lane -authority. Runtime stores and adapters are process-local; cross-process state is -limited to serialized bundle data, progress channels, and result messages. -Cancellation remains parent-owned: process workers are interrupted through the -execution server's worker ownership boundary, whereas inline and threaded lanes -observe the execution-scoped cooperative cancellation signal directly. +Worker processes install the ``CompiledExecutionBundle`` through +``ForkInheritedWorkerExecutionState``, either inherited at fork or unpickled +from the bundle shipped to a prepared worker. Runtime stores and adapters are +process-local; cross-process state is limited to serialized bundle data, +progress channels, and result messages. Cancellation remains parent-owned: +process workers are terminated by the executor resources that own them, whereas +inline and threaded lanes observe the execution-scoped cooperative cancellation +signal directly. -Generic code must not ask the orchestrator for hard-coded component names. -Component identities come from the source model and nominal component -declarations. See :doc:`component_system_integration` and -:doc:`concurrency_model`. +Generic code must not ask the orchestrator for hard-coded axis names. Axis +identities come from the active ``AxisFamily`` declaration and the dataset +source. See :doc:`component_system_integration` and :doc:`concurrency_model`. diff --git a/docs/source/architecture/nominal_ownership.rst b/docs/source/architecture/nominal_ownership.rst index 05ed756126..ef0d66c8bc 100644 --- a/docs/source/architecture/nominal_ownership.rst +++ b/docs/source/architecture/nominal_ownership.rst @@ -34,7 +34,8 @@ An ``AutoRegisterMeta`` family has a nominal root and an authoritative use a shared selection mixin. ``RegistryConfig`` and ``RegistryFamily`` describe generic registration mechanics and are owned by metaclass-registry. -OpenHCS uses three shared selection models: +OpenHCS uses three shared selection models, imported from +``metaclass_registry.strategies``: ``EnumKeyedStrategyMixin`` A concrete strategy declares the enum member it owns. The mixin derives the @@ -72,7 +73,9 @@ Examples in the current architecture ------------------------------------ - ``ArtifactType`` and module artifact partitions own artifact roles. -- ``RuntimeSliceProjectionStrategy`` selects behavior by nominal runtime type. +- owned runtime values subclass ``RuntimeSliceProjectableValue`` and implement + their own runtime-slice projection; ``RuntimeSliceProjection`` handles only + foreign values such as arrays, containers, and primitives. - runtime equivalence feature profiles select the most-derived semantic owner for a feature context. - ``ProcessingContract`` subclasses own how a callable executes over its diff --git a/docs/source/architecture/orchestrator_cleanup_guarantees.rst b/docs/source/architecture/orchestrator_cleanup_guarantees.rst index 55d2a0d345..54cbb01d7c 100644 --- a/docs/source/architecture/orchestrator_cleanup_guarantees.rst +++ b/docs/source/architecture/orchestrator_cleanup_guarantees.rst @@ -1,7 +1,7 @@ Execution cleanup guarantees ============================ -The compiled plate execution boundary makes cleanup converge across success, +The compiled-bundle execution boundary makes cleanup converge across success, failure, cancellation, and a broken process pool. Viewer acquisition is part of that execution transaction. If one required diff --git a/docs/source/architecture/orchestrator_configuration_management.rst b/docs/source/architecture/orchestrator_configuration_management.rst index 507ccb5dbd..4713b27ae5 100644 --- a/docs/source/architecture/orchestrator_configuration_management.rst +++ b/docs/source/architecture/orchestrator_configuration_management.rst @@ -18,12 +18,13 @@ through its declared ObjectState boundary. The desktop forms and code projection edit the same state. Assignment or restoration changes the declaration; it does not mutate already compiled contexts. -A local plate path is declaration identity before it is a runtime input. Session -restoration can therefore reconstruct and expose a plate whose directory is -temporarily unavailable. The selected microscope handler's source role owns its -availability contract, which ``PipelineOrchestrator.initialize`` enforces. A -failed check leaves that orchestrator uninitialized, so the declaration remains -editable without pretending that source discovery succeeded. +A local dataset root is declaration identity before it is a runtime input. +Session restoration can therefore reconstruct and expose a plate whose directory +is temporarily unavailable. The selected ``DatasetSource`` class's +``SourceSelectionRole`` owns its availability contract, which +``PipelineOrchestrator.initialize`` enforces. A failed check leaves that +orchestrator uninitialized, so the declaration remains editable without +pretending that source discovery succeeded. Ownership --------- diff --git a/docs/source/architecture/pipeline_compilation_system.rst b/docs/source/architecture/pipeline_compilation_system.rst index 821741d2e7..f7c1b2c9b0 100644 --- a/docs/source/architecture/pipeline_compilation_system.rst +++ b/docs/source/architecture/pipeline_compilation_system.rst @@ -19,15 +19,10 @@ Output Resolve once ------------ -The normal compiler path materializes the pipeline and each step from ObjectState -once. ``ResolvedPipelineDefinition`` groups the resolved steps, their ObjectState -map, and a ``StepSnapshot`` tuple shared by every axis compilation. - -A ``StepSnapshot`` contains: - -- stable step index -- ObjectState scope identity -- the already-resolved step object +The normal compiler path resolves the pipeline and each saved step through +ObjectState once, without registering editor states. ``ResolvedPipelineDefinition`` +holds the enabled resolved steps with each step's ObjectState scope identity and +saved provenance, keyed by step index, and is shared by every axis compilation. All step semantics remain on that resolved step and its authoritative callable or module declarations. Later stages must not call ObjectState again to recover @@ -49,16 +44,18 @@ catalogue module that exposes them. Catalogue discovery projects those nominal claims, rejects duplicate canonical names, and does not maintain a parallel module-to-function list. -Compilation owns the deferred warmup. Callable leaves become typed -``FunctionReference`` and ``CallableContract`` values. Resolving those -references initializes the function catalog, and -``prepare_compiled_context_callables()`` resolves and caches every compiled -invocation and runs its declaration-owned module or callable preparation hook -before the execution bundle is emitted. A first compilation may therefore pay -catalog and backend preparation costs; function-step orchestration does not own -another preparation phase in its hot path. A spawn worker may still resolve a -reference and rebuild a process-local callable cache because parent-process -caches cannot cross that process boundary. +An execution endpoint prepares the registered catalogue before it reports +readiness: ``RegistryService.prepare_in_current_process()`` populates kernel +caches, runs each callable's declaration-owned preparation hook, and publishes +prepared signatures. Compilation then binds the selected callables. Callable +leaves become typed ``FunctionReference`` and ``CallableContract`` values; +``CallableContract.from_prepared_callable()`` prepares an authored target that +the catalogue did not already prepare, and each ``CompiledFunctionInvocation`` +resolves its runtime callable through the process-local contract cache when it +is built, before the execution bundle is emitted. Function-step orchestration +does not own another preparation phase in its hot path. A spawn worker may still +resolve a reference and rebuild a process-local callable cache because +parent-process caches cannot cross that process boundary. This ordering is an ownership boundary, not merely a startup optimization. Source inspection must stay safe and bounded, while compilation must fail if a @@ -82,14 +79,14 @@ new ``ProcessingContext`` before constructing the narrower session boundary. Each axis receives a ``CompilationSession`` joining: - the axis ``ProcessingContext`` -- resolved steps and snapshots -- the step ObjectState map -- global and orchestrator configuration +- the shared ``ResolvedPipelineDefinition`` +- the resolved global configuration and the orchestrator - the virtual source-workspace projection - the mutable ``CompiledStepPlan`` map -- plate and transport execution facts +- dataset scope, path resolution, and transport execution facts -The session validates one-to-one step, state, snapshot, and plan identity. A +``ResolvedPipelineDefinition`` rejects a step without scope and provenance +facts, and the session reports a missing compiled plan for any step index. A compiler stage accepts the session or a narrower typed request derived from it; it does not pass loosely related dictionaries between phases. @@ -100,12 +97,12 @@ Typed step plans single plan authority for a step. Important fields include: - input/output paths and storage backends -- source binding, source universe, and source load plans +- source binding and source universe plans - main-flow dependency - artifact input/output plans and execution group scope - compiled function pattern and callable contracts - variable components, grouping, and sequential filters -- input, output, and execution memory roles plus framework-local device bindings +- input and output memory types plus framework-local device bindings - materialization plans and enabled streaming declarations Stages mutate these fields directly. Code should not recreate diff --git a/docs/source/architecture/plate_manager_services.rst b/docs/source/architecture/plate_manager_services.rst index 72b628f62c..8aae1322bc 100644 --- a/docs/source/architecture/plate_manager_services.rst +++ b/docs/source/architecture/plate_manager_services.rst @@ -13,7 +13,7 @@ Ownership map ``session.py`` ``Session``: dataset add/delete/split, initialisation, compile and run, stop, - debug runs, finished-execution records, admission checks, and the ports a + debug runs, finished-execution records, work and mutation checks, and the ports a client supplies (``MainThread``, ``DatasetAccess``, ``Renderer``). ``operations/`` diff --git a/docs/source/architecture/processing_semantics.rst b/docs/source/architecture/processing_semantics.rst index 6378d413ad..8a55ef7b27 100644 --- a/docs/source/architecture/processing_semantics.rst +++ b/docs/source/architecture/processing_semantics.rst @@ -1,9 +1,11 @@ Axes, grouping, and processing semantics ======================================== -OpenHCS transports image data as 3D arrays, including logical single-plane -inputs. Array rank alone does not determine what the leading/third axis means or -whether a callable has per-plane or whole-stack semantics. +OpenHCS transports image data as tensor payloads whose dimensions are declared +(``PayloadAxes``): a leading runtime plane axis, spatial dimensions named by the +payload's spatial domain, and other declared axes such as colour samples. Array +rank alone does not determine what the leading axis means or whether a callable +has per-plane or whole-stack semantics. Five questions, five owners --------------------------- @@ -58,21 +60,25 @@ Related execution declarations Processing contracts -------------------- -``PURE_2D`` +Each contract is a ``ProcessingContract`` subclass that owns its execution +algorithm; a callable declares one by class, for example +``@numpy(contract=Pure2DContract)``. + +``Pure2DContract`` The result for one plane is semantically independent of every other plane. - OpenHCS may still transport or batch a 3D array, but the contract executes and + OpenHCS may still transport or batch a stack, but the contract executes and aggregates through nominal per-slice strategies. -``PURE_3D`` +``Pure3DContract`` The callable depends on the full variable-component stack and executes once with whole-stack semantics. -``FLEXIBLE`` +``FlexibleContract`` The callable supports both modes through an explicit semantic-control - parameter. The contract consumes that control and delegates to ``PURE_2D`` or - ``PURE_3D``; callers do not infer the mode from shape. + parameter. The contract runs per plane when its slice-by-slice control is on + and on the whole stack otherwise; callers do not infer the mode from shape. -``VOLUMETRIC_TO_SLICE`` +``VolumetricToSliceContract`` The callable consumes a real stack and produces a collapsed plane domain. The contract updates payload provenance to reflect the consumed leading axis. @@ -100,8 +106,10 @@ the resolved facts, including selected framework-local device bindings, on Runtime projection ------------------ -When a contract requires per-slice execution, ``RuntimeSliceProjection`` selects -a registered strategy for each nominal value family. Image payloads, object +When a contract requires per-slice execution, each owned runtime value family +projects itself: it subclasses ``RuntimeSliceProjectableValue`` and declares its +slice count and per-slice value. ``RuntimeSliceProjection`` handles only foreign +values such as arrays, containers, and primitives. Image payloads, object labels, measurements, relationships, tables, and aligned collections retain their semantic identity during projection and aggregation. A missing projection declaration is an error; OpenHCS does not guess from an object's shape or fields. @@ -112,7 +120,7 @@ Common mistakes - Treating ``group_by`` as the definition of the stack axis. - Treating a named source binding as a third ``InputSource`` enum value. - Treating a value's presence in the runtime store as a persistence request. -- Treating ``PURE_2D`` as “the callable only accepts rank-2 arrays.” +- Treating ``Pure2DContract`` as “the callable only accepts rank-2 arrays.” - Passing ``variable_components`` or ``group_by`` directly to ``FunctionStep`` instead of its processing configuration. - Adding backend-specific dispatch to core rather than extending the callable or diff --git a/docs/source/architecture/progress_runtime_projection_system.rst b/docs/source/architecture/progress_runtime_projection_system.rst index e72a4aa7f7..e7a98f18c7 100644 --- a/docs/source/architecture/progress_runtime_projection_system.rst +++ b/docs/source/architecture/progress_runtime_projection_system.rst @@ -33,30 +33,33 @@ Consumption path: 2. Route events through explicit queue wiring via ``set_progress_queue(...)``. 3. Register events in ``ProgressRegistry`` with semantic channel keying. 4. Reconcile typed server-heartbeat running/queued entries by execution and - plate identity in ``ExecutionRuntimeProjection``. + subject identity in ``ExecutionRuntimeProjection``. 5. Build runtime and execution-tree projections for status derivation. -6. Coalesce Plate Manager registry mutations through its explicitly - GUI-thread-owned timer. -7. Render the same projection in Plate Manager and ZMQ server browser services. +6. Coalesce registry mutations in the session's ``ExecutionProgress``: each + mutation marks the projection dirty, and the session's main thread rebuilds + it at most once per interval and publishes it as session events. +7. Render the same projection in the Plate Manager (through session events) and + the ZMQ server browser. OpenHCS Invariants ------------------ -- Canonical execution tree path is ``plate -> worker -> well -> step``. -- ``PATTERN_GROUP`` is step-detail only and does not set well pipeline percent. -- Plate/server status is derived from projection snapshots, not mutable cache. +- The execution tree path is ``subject -> worker -> partition -> step``. +- ``PATTERN_GROUP`` is step-detail only and does not set partition pipeline + percent. +- Subject/server status is derived from projection snapshots, not mutable cache. - Event retention is monotonic per semantic key. Stale or duplicate timestamps are rejected before listeners and typed side-effect consumers run. - Topology is reconstructed from the retained event snapshot. A late subscriber may render a shallow lifecycle node when the initialization event is absent. -- ``PlateRuntimeStateDeclarationBase`` owns state labels, count labels, +- Each ``SubjectRuntimeState`` subclass owns its status label, count label, terminal policy, and live-server transition policy. ``state_counts`` is the only retained count projection; per-state count fields are forbidden. -- Runtime identity is ``(execution_id, plate_id)``. A new queued execution may - replace an older visible execution for the same plate without conflating their - retained histories. -- A background progress callback may only dirty the retained projection. Qt - timers and row rendering are application-thread lifecycle responsibilities. +- Runtime identity is ``SubjectRuntimeIdentity(execution_id, subject_id)``. A + new queued execution may replace an older visible execution for the same + subject without conflating their retained histories. +- A background progress callback may only dirty the retained projection. + Rebuilding and rendering run on the client's main thread. Debugger ownership ------------------ @@ -135,7 +138,8 @@ plate-scoped step. It is a successful terminal event for that worker/axis channel, but it does not assert that the plate execution has finished. After every worker result succeeds, the parent merges runtime records and calls -``execute_plate_scoped_steps()`` once for each terminal plate step. The parent +``execute_dataset_scoped_steps()``, which runs each terminal plate-scoped step +once. The parent then materializes those outputs, runs analysis consolidation, settles viewer state, and projects the orchestrator result. Only after those stages succeed does it emit the pipeline ``SUCCESS`` event at 100 percent. Failure, diff --git a/docs/source/architecture/quick_start.rst b/docs/source/architecture/quick_start.rst index 288729e4de..b854f38c06 100644 --- a/docs/source/architecture/quick_start.rst +++ b/docs/source/architecture/quick_start.rst @@ -39,7 +39,7 @@ Every supported authoring route produces the same public declaration shape: PipelineConfig + ordered list[FunctionStep] -> ObjectState resolves lazy configuration - -> StepSnapshot + CompilationSession + -> ResolvedPipelineDefinition + CompilationSession -> typed CompiledStepPlan objects -> CompiledExecutionBundle -> runtime values, artifacts, and materialized outputs @@ -205,15 +205,16 @@ Keep these independent concepts separate: function patterns. It does not define the stack axis. ``ProcessingContract`` - Declares callable locality. ``PURE_2D`` retains per-plane semantics even when - planes travel in a batch; ``PURE_3D`` means the result depends on the stack. + Declares callable locality. ``Pure2DContract`` retains per-plane semantics + even when planes travel in a batch; ``Pure3DContract`` means the result + depends on the stack. Unsupported microscope folders ------------------------------- ``SourceBindingsConfig`` is the supported generic input boundary when a folder -does not match a registered microscope handler. A non-empty source-binding -declaration makes auto-detection select ``SourceBindingsHandler``. The agent or +does not match a registered dataset source. A non-empty source-binding +declaration makes auto-detection select ``SourceBindingsSource``. The agent or user describes the files; processing callables remain independent of physical filenames. @@ -221,6 +222,7 @@ For files such as ``A01_s1_DNA.tif`` and ``A01_s1_GFP.tif``: .. code-block:: python + from openhcs.constants.input_source import InputSource from openhcs.core.config import ( LazyProcessingConfig, LazySourceBindingsConfig, @@ -322,8 +324,8 @@ materialization, memory conversion, worker requirements, and execution scope. Runtime workers consume the resulting ``CompiledExecutionBundle`` and typed runtime values; they do not rediscover those contracts from strings or files. -The desktop application and MCP execution services own progress, cancellation, -and result presentation. For the explicit lower-level Python boundary, see +The OpenHCS ``Session`` that both the desktop application and MCP operate owns +progress, cancellation, and result presentation. For the explicit lower-level Python boundary, see :doc:`../api/index`. Current example authorities diff --git a/docs/source/architecture/roi_system.rst b/docs/source/architecture/roi_system.rst index dd15c0b5a5..74c74c35a8 100644 --- a/docs/source/architecture/roi_system.rst +++ b/docs/source/architecture/roi_system.rst @@ -38,16 +38,16 @@ type, producer, execution scope, source relations, materialization options, and object-subject relations. Runtime materialization gives the registered writer that complete plan. -Object-label artifacts use their nominal materialization strategy to project -label members into ROI shapes. Spatial graphs have two independent registered -projections: - Projected materialization keeps semantic source identity separate from filename identity. A semantic identity may intentionally omit an axis that was collapsed by processing. The complete record-source identity may fill only the missing parser coordinates needed to construct a filename; it does not restore that axis to the artifact metadata. +Object-label artifacts use their nominal materialization strategy to project +label members into ROI shapes. Spatial graphs have two independent registered +projections: + ``SpatialGraphROIOptions`` Writes one feature-bearing polyline ROI per graph edge. It retains edge identity and scalar branch features. ImageJ ROI geometry is two-dimensional, diff --git a/docs/source/architecture/runtime_value_system.rst b/docs/source/architecture/runtime_value_system.rst index dbf7cf8665..6abaeee633 100644 --- a/docs/source/architecture/runtime_value_system.rst +++ b/docs/source/architecture/runtime_value_system.rst @@ -13,8 +13,8 @@ Value families Nominal array protocol with explicit NumPy interoperation. image payloads - Image data plus mask, plane-axis declaration, source metadata, spatial domain, - and source-plane provenance. + ``ImagePayload``: image data plus mask, declared payload axes, plane-axis + declaration, source metadata, spatial domain, and source-plane provenance. object-label values Dense or sparse labeled-object domains, grouped sets, and storage/building @@ -38,8 +38,8 @@ semantic identity. Persistence location is represented separately by ``RuntimeArtifactLocation``. Invocation-selected output plans remain keyed by ``ArtifactSpecRef`` when they -enter ``RuntimeAdapterRequest``. ``RuntimeReturnedOutputMatcher`` binds returned -positions to the exact plan/spec refs, and adapter lookup accepts the +enter ``RuntimeAdapterRequest``. ``CallableContract.resolve_returned_plan_values`` +binds returned positions to the exact plan/spec refs, and adapter lookup accepts the declaration-derived ref whose key agrees with its plan. Two outputs may therefore share a name when their artifact types differ: selection, matching, and request lookup preserve both identities rather than collapsing to the name. @@ -71,26 +71,29 @@ part of artifact identity rather than encoded into ad hoc path strings. Runtime slice projection ------------------------ -``RuntimeSliceProjectionStrategy`` is a nominal-type registry family. A strategy -declares the runtime value type it owns. Selection follows the value's MRO, and -projection uses explicit plane-axis and source-identity declarations. +Each owned runtime value family subclasses ``RuntimeSliceProjectableValue`` and +implements its own slice behaviour: slice count, declared plane axis, per-slice +values, and alignment slices. Projection uses explicit plane-axis and +source-identity declarations. -Registered strategies cover image payloads, aligned image collections, object -labels, measurement tables, columnar rows, sparse labels, relationships, and -identity-projectable semantic values. Primitive/pass-through types are declared -explicitly. If a new value family can carry a runtime slice, its projection and -aggregation behavior must be registered with the value type. +Image payloads, aligned value sets, object labels, columnar rows (including +measurement tables and sparse labels), relationships, and identity-projectable +semantic values implement it. ``RuntimeSliceProjection`` handles only foreign +values: it lists the array, mapping and primitive types that carry no +runtime-slice axis, and projects tuples and lists item by item. Any other value +without the ABC is an error. If a new value family can carry a runtime slice, it +must subclass ``RuntimeSliceProjectableValue``. Function-output contextualization --------------------------------- A callable return does not inherit context from whichever runtime value happens -to be nearby. ``FunctionOutputContextStrategy`` first selects behavior from the -exact compiled output artifact type. For an image output, -``ImageOutputSourceContextStrategy`` then selects behavior from the nominal type -of the source chosen by the output's compiled ``source_context_source`` -relation. This is a two-stage owner dispatch, not a table of callable names or -payload shapes. +to be nearby. The exact compiled output artifact type first selects behaviour +through its ``ArtifactType.contextualize_output`` method. For an image output, +``ImageArtifactType`` then calls ``contextualize_image_output`` on the source +value chosen by the output's compiled ``source_context_source`` relation, so the +source's nominal type decides. This is a two-stage owner dispatch, not a table +of callable names or payload shapes. An image result that already carries context may bypass derivation only when its source identity is complete and, for a declared artifact output, its represented @@ -103,7 +106,7 @@ Image results that select or reorder source planes implement ``SourceProjectedImageOutput``. Their leaf implementation validates the declared selection and projects source metadata before generic image contextualization. ``SelectedPlaneImageOutput`` carries a cropped array and its ordered source -indices, preserving channel identity even when only one channel remains. +indices, preserving source-plane identity even when only one plane remains. Object-label sources have a specialized image-output rule. A rendered label volume with no selected plane must acquire the invocation's declared plane axis, @@ -118,11 +121,10 @@ the artifact name and type, and the runtime store owns the resulting artifact key. A contextualizer must not rename a value, select a same-named plan, or coerce one artifact family into another. -Artifact families without contextual image semantics use the root -``UnchangedFunctionOutputContextStrategy``. It returns their values unchanged -and does not invoke the image plane projector. Image, measurements, spatial -graph, and object-label declarations select more-derived strategies through the -artifact-type hierarchy. The artifact declaration therefore determines +Artifact families without contextual image semantics inherit the root +``ArtifactType`` methods. They return their values unchanged and do not invoke +the image plane projector. Image, measurements, spatial grid, spatial graph, +and object-label artifact types override them. The artifact declaration therefore determines projection behaviour; individual backend result classes do not need a copied exemption registry. diff --git a/docs/source/architecture/source_model.rst b/docs/source/architecture/source_model.rst index d86cc30b59..b76573e06e 100644 --- a/docs/source/architecture/source_model.rst +++ b/docs/source/architecture/source_model.rst @@ -10,11 +10,11 @@ Generic folders and unsupported microscopes ------------------------------------------- Source bindings are the nominal microscope-independent ingestion path for -arbitrary folders. When microscope auto-detection fails and a non-empty -``SourceBindingsConfig`` is present, the handler factory selects -``SourceBindingsHandler``. That handler recursively inventories the folder and -projects only the ordinary files selected by the declarations into a virtual -workspace. +arbitrary folders. When dataset-source auto-detection fails and a non-empty +``SourceBindingsConfig`` is present, ``AutoDetectedSource`` selects the +registered declared-file source, ``SourceBindingsSource``. That source +recursively inventories the folder and projects only the ordinary files selected +by the declarations into a virtual workspace. This fallback is distinct from semantic source selection on a recognized store. Native HCS handlers retain ownership of recognized layouts; CZI, OME-TIFF, and diff --git a/docs/source/architecture/special_io_system.rst b/docs/source/architecture/special_io_system.rst index 312c842292..a24808863d 100644 --- a/docs/source/architecture/special_io_system.rst +++ b/docs/source/architecture/special_io_system.rst @@ -7,8 +7,9 @@ Special I/O terminology (transition) generated-pickle-path architecture has been superseded by typed callable contracts and compiled artifact plans. Read :doc:`artifact_contract_system`. -``special_inputs`` is callable-ABI metadata. It names Python input parameters; they do not own artifact type, producer identity, materialization, or -runtime storage. +``special_inputs`` is callable-ABI metadata. It names Python input parameters +and does not own artifact type, producer identity, materialization, or runtime +storage. ``special_outputs`` no longer exists. Current ownership is split across: diff --git a/docs/source/architecture/streaming_boundary_and_wrappers.rst b/docs/source/architecture/streaming_boundary_and_wrappers.rst index 2c0d2240c9..0d78a5b1e7 100644 --- a/docs/source/architecture/streaming_boundary_and_wrappers.rst +++ b/docs/source/architecture/streaming_boundary_and_wrappers.rst @@ -55,8 +55,9 @@ This path has two independent identities: producer identity Identifies the compiled step and exact output surface that emitted the value. - ``FunctionStepOutputProducerIdentityAuthority`` derives it from the compiled - plan and the returned-output or artifact context. + ``CompiledStepPlan.producer_identity_for_main_flow`` and + ``producer_identity_for_artifact`` derive it from the compiled step plan and + the main-flow output context or artifact output plan. source identity Identifies the source image plane or source domain represented by the value. @@ -85,9 +86,9 @@ branch on a CellProfiler module name or infer labels from an ndarray dtype. When a callable deliberately renders an image from labels, the compiled ``source_context_source`` relation selects the label artifact as the output's -context source. Runtime contextualization then uses the registered -``ObjectLabelImageOutputSourceContextStrategy``. For an unselected volumetric -invocation, including a depth-one volume, that strategy requires the declared +context source. Runtime contextualization then calls +``ObjectLabelValue.contextualize_image_output`` on that label source. For an +unselected volumetric invocation, including a depth-one volume, it requires the declared plane axis, requires source-plane provenance cardinality to match it, and validates the rendered output shape. A true scalar 2D label image remains scalar. Array rank alone never creates source identity. @@ -175,7 +176,7 @@ terminal progress for its synchronous update path. Every recorded route must succeed before settlement completes, so an exception crosses the asynchronous Qt boundary instead of becoming a silent log message or an unrelated timeout. -Compiled plate execution follows this order on success: +Compiled bundle execution follows this order on success: 1. settle every execution viewer; 2. read typed state from non-persistent viewers while their endpoints are live; diff --git a/docs/source/architecture/system_overview.rst b/docs/source/architecture/system_overview.rst index c5db47ad4c..7695fa2aec 100644 --- a/docs/source/architecture/system_overview.rst +++ b/docs/source/architecture/system_overview.rst @@ -17,7 +17,7 @@ End-to-end flow | ObjectState resolves once v ResolvedPipelineDefinition - + StepSnapshot tuple + + step scope ids and field provenance | | one CompilationSession per execution axis v @@ -55,18 +55,18 @@ Resolution boundary ObjectState owns generic lazy configuration and live editing. Before compilation continues, OpenHCS materializes one resolved step object for every declaration. -``StepSnapshot`` binds compiler identity and ObjectState scope identity to that -already-resolved step. Downstream compiler stages read semantics from the -resolved step and its callable/module declarations; they do not repeatedly ask -ObjectState to reinterpret fields. +``ResolvedPipelineDefinition`` holds those resolved steps with each step's +ObjectState scope id and per-field provenance. Downstream compiler stages read +semantics from the resolved step and its callable/module declarations; they do +not repeatedly ask ObjectState to reinterpret fields. Compilation boundary -------------------- -``CompilationSession`` ties together one ``ProcessingContext``, the resolved -steps, snapshots, ObjectState map, source-workspace projection, and mutable typed -plans for one execution axis. Its invariants prevent phases from silently -mixing steps, scopes, or contexts. +``CompilationSession`` ties together one ``ProcessingContext``, the +``ResolvedPipelineDefinition``, source-workspace projection, and mutable typed +plans for one execution axis. Phases read steps and plans from that one session +by step index, and a missing plan is an error rather than a default. Each ``CompiledStepPlan`` is the source of truth for the compiled step. It has typed fields for source bindings, source universe, artifact inputs and outputs, @@ -90,8 +90,8 @@ resolving declaration configuration again. Runtime data is represented by nominal value families and stored under typed artifact keys. ``RuntimeValueStore`` is the authority for validated values, -locations, replacement, queries, and observation history. Projection strategies -adapt values to declared slice or grouping scopes without guessing from raw +locations, replacement, queries, and observation history. Each value family +projects itself onto declared slice or grouping scopes without guessing from raw array shape. Owners and adapters @@ -107,7 +107,8 @@ extracted packages own their generic mechanisms: - PolyStore: storage, formats, ROI, and virtual-workspace primitives - metaclass-registry: generic nominal registration and logging-safe discovery - pyqt-reactive: generic reactive forms and widgets -- python-introspect: callable/signature analysis and declaration projection +- python-introspect: callable/signature analysis, declaration projection, and + the JSON codec (``to_jsonable``/``dataclass_from_mapping``) - ZMQRuntime: generic process and transport protocols - pycodify: Python-source serialization diff --git a/docs/source/architecture/ui_services_architecture.rst b/docs/source/architecture/ui_services_architecture.rst index dc8f0214fc..5df3ea327e 100644 --- a/docs/source/architecture/ui_services_architecture.rst +++ b/docs/source/architecture/ui_services_architecture.rst @@ -45,11 +45,13 @@ generic registration, focus/reuse, parentage, navigation, and close cleanup. Reopening a scoped editor therefore focuses the existing window; it does not create a second domain state or a second window registry. -``MainWindowAction`` members own their enablement, invocation, side effects and -recovery warnings. The UI bridge projects those members directly. Session restart -uses the same ``DesktopRestartSession`` capture and history restoration boundary -as updates and version-mismatch recovery; its accepted receipt precedes old-UI -shutdown and requires discovery of the new bridge afterwards. +Main-window actions such as ``CheckForUpdates``, ``RestartApplication`` and +``ExitApplication`` are ``SessionOperation`` classes that own their availability, +invocation, side effects and warnings. The UI bridge projects those classes +directly through ``SessionOperationActionProvider``. Session restart uses the +same ``DesktopRestartSession`` capture and history restoration as updates and +version-mismatch recovery; the action returns before the old UI exits, so the +caller must rediscover the new bridge afterwards. The process configuration ObjectState scopes share one stable configuration window. OpenHCS registers those declared scopes as routes to that window, while @@ -129,7 +131,8 @@ Process configuration topology ``UIConfig`` is the process-global desktop declaration. Its ``zmq`` field is an OpenHCS-owned ``OpenHCSZMQConfig`` subtype of ZMQRuntime's generic -``ZMQConfig``; it is not mirrored as a widget settings table. +``ExecutionTransportConfig`` that only sets OpenHCS's application names; it is +not mirrored as a widget settings table. ``PyQtGuiRuntimeContext`` carries the current ``UIConfig`` beside ``GlobalPipelineConfig``. The global configuration window edits their registered ObjectState scopes. The declaration contains only @@ -145,9 +148,8 @@ duplicated as editable UIConfig fields. The update-check preference is itself a first reconciles every live consumer and only then publishes the runtime-context value and ``ui_config_changed`` signal. The system monitor applies ``performance_monitor`` through its idempotent update owner. Plate Manager -updates its execution connection declaration and the already-materialized -progress timer; a not-yet-materialized progress service receives the same value -when constructed. The main-window shortcut lifecycle validates the complete key +passes the execution transport declaration and progress-update interval to the +session it renders. The main-window shortcut lifecycle validates the complete key set before rebinding its concrete QActions and time-travel event filter. ZMQ Manager rebuilds its scan inputs from the current transport declaration and the bridge's actual running binding, never from an unstarted configured port. @@ -242,9 +244,11 @@ renderer, or fallback help system is retained. Results and selected-plate ownership ------------------------------------ -Plate Manager owns the visible plate selection and delegates pipeline editing, -compile/run actions, Results, and viewer launch to the orchestrator registered -for that row. Output rows resolve through the declared source/output relation so +Plate Manager renders the ``Session`` dataset list and holds no session state: +rows, selection, pending work, compiled artifacts and execution state are read +from the session, and its buttons are session operations. Pipeline editing, +compile/run actions, Results, and viewer launch act on the orchestrator +registered for the selected row. Output rows resolve through the declared source/output relation so Results can use the producing orchestrator rather than treating the output plate as an unrelated session. diff --git a/docs/source/architecture/zmq_execution_service_extracted.rst b/docs/source/architecture/zmq_execution_service_extracted.rst index a1919517ee..d15f9072aa 100644 --- a/docs/source/architecture/zmq_execution_service_extracted.rst +++ b/docs/source/architecture/zmq_execution_service_extracted.rst @@ -38,59 +38,57 @@ inline and threaded work then stops at the next cooperative boundary. OpenHCS execution submission reuses ZMQRuntime's monotonic ``OperationDeadline`` across endpoint startup, progress registration, task -serialisation, and the execute request. ``OpenHCSZMQConfig`` owns this dedicated -submission budget separately from the shorter budget for status, stop, and -other control requests. Startup activity may refresh the endpoint inactivity +serialisation, and the execute request. ZMQRuntime's +``ExecutionTransportConfig``, which ``OpenHCSZMQConfig`` extends with OpenHCS +application names, declares this dedicated submission budget separately from +the shorter budget for status, stop, and other control requests. Startup activity may refresh the endpoint inactivity deadline, but it cannot extend the caller's total submit budget. OpenHCS reports separately whether preparation expired before the execute request or the request was sent without a reply, so callers do not invent an execution identifier or retry an unknown outcome blindly. -An accepted headless job retains the exact client that submitted it. Status -polling reuses that client, projects its ZMQRuntime-owned latest progress -observation, and disconnects it after caching a terminal response. It does not -recreate clients for polling or mirror transport progress in an OpenHCS-owned -registry. The job also retains the exact typed submission and accepting endpoint -handshake. A successful completion projects those facts with the server's typed -execution record, including its results and timing boundaries, so evidence +Desktop and headless MCP clients submit through the same ``Session``, which +owns one execution client. ``ExecutionSubmission`` records each accepted +submission with its typed request and accepting endpoint handshake, and follows +its status through that client until it is terminal. The session then keeps the +submission, endpoint and the server's typed ``ExecutionRecord``, including its +results and timing boundaries, together as one finished execution, so evidence writers need not reconstruct the pipeline or parse a separate status source. The ordinary ZMQ client checks the endpoint's OpenHCS application compatibility before sending any compile or execution request. Benchmark callers use that same admission method; they do not maintain a separate version rule. -The headless cancellation capability addresses that retained client's exact -job identifier. Its service delegates the bounded request to the ordinary -execution client and returns the server's applied flag with the observed job -status; a client timeout alone is not treated as successful cancellation. -Optional runtime-observation export follows the same submission path: the -agent path policy validates the destination, and the client/server shared -auxiliary declaration carries it and its typed value-or-outcome scope to the -existing execution server. Outcome-only export does not require runtime array +Session stop shuts down the execution endpoint, gracefully or by force, and +marks the active datasets cancelled. The ordinary execution client also keeps a +bounded ``cancel_execution`` control request for one execution identifier. +Optional runtime-observation export follows the ordinary submission path: the +client/server shared ``ZMQAuxiliaryExecutionParams`` declaration carries the +destination and its typed value-or-outcome scope to the existing execution +server. Outcome-only export does not require runtime array values to return to the parent when the compiled plan does not need them. It retains file-export paths from the same compiled output contexts as a full observation, without collecting intermediate runtime records. The benchmark layer need not submit a second kind of job to collect that evidence. Both export scopes carry the producing server execution ID, and the ordinary export writer will not replace an existing file. The benchmark finaliser -checks that ID against the completed ordinary job before retaining its receipt. +checks that ID against the completed ordinary job before retaining its evidence. The current value and outcome exports also compare execution-result membership with the compiled axis set; a successful subset cannot become a completed measurement merely because its reported axes succeeded. The server-owned observation also carries a startup snapshot of its Python -interpreter and installed distribution versions. Benchmark receipts project +interpreter and installed distribution versions. Benchmark evidence projects that observation without treating the client environment as the server's. The snapshot does not establish the environment of a remote worker. The ordinary compiled-run result also projects the server's completion summary and typed output-plate metadata; benchmark adapters consume those projections instead of decoding transport response keys themselves. -A source-backed execution session may retain an original ``plate_path`` while -declaring a different prepared ``execution_plate_path``. The shared ZMQ -execution identity selects the prepared path when present, independently of a -selected external pipeline file. The pycodified ``pipeline_source`` remains -the pipeline authority; a second selected-pipeline path is rejected for that -session. Both paths pass the ordinary agent read-path policy before submission. +An execution request names its ``subject_id`` and may declare a different +prepared ``execution_subject_id``. The shared ZMQ execution identity carries +both, and the server's orchestrator environment opens the prepared subject when +present, independently of a selected external pipeline file. The pycodified +pipeline document remains the pipeline authority. The execution server owns one ``FunctionCatalogPreparation``. On a cold cache, ``RegistryService`` launches the launcher's dedicated diff --git a/docs/source/architecture/zmq_server_browser_system.rst b/docs/source/architecture/zmq_server_browser_system.rst index 1d1fba6805..76ebd389a9 100644 --- a/docs/source/architecture/zmq_server_browser_system.rst +++ b/docs/source/architecture/zmq_server_browser_system.rst @@ -38,7 +38,7 @@ OpenHCS Browser Components canonicalises responsive heartbeats and in-progress startup observations by port, then derives endpoint status without persistent connection flags. - ``RuntimeExecutionTopology``: - derives worker/well ownership and step names from each retained event + derives worker/partition ownership and step names from each retained event snapshot and validates worker claims without persistent browser state. - ``RuntimeTreeProjectionBuilder``: builds typed core runtime trees with recursive aggregation policies. @@ -65,17 +65,17 @@ Canonical Tree Path Execution subtree path is invariant: -``plate -> worker -> well -> step`` +``subject -> worker -> partition -> step`` -Compilation path is plate -> compilation entries until execution events begin. +Compilation path is subject -> compilation entries until execution events begin. Aggregation Semantics --------------------- Tree percentages are recursive and policy-driven: -- ``mean`` for parent aggregates (plate, worker) -- ``explicit`` for leaf/detail nodes (well, step, compilation) +- ``mean`` for parent aggregates (subject, worker) +- ``explicit`` for leaf/detail nodes (partition, step, compilation) Policies are enforced per registered node type in ``RuntimeTreeProjectionBuilder``. @@ -83,8 +83,8 @@ Policies are enforced per registered node type in Refresh/State Preservation -------------------------- -The Plate Manager's execution client is the sole subscriber that registers -progress events. The browser observes accepted registry mutations, marshals the +The ``Session`` execution client (``ZMQClientService``) is the sole subscriber +that registers progress events. The browser observes accepted registry mutations, marshals the notification through a queued Qt signal, and coalesces redraws at the interval declared by ``ProgressUIConfig``. Server scans independently supply the current typed running/queued heartbeat entries. Expansion/selection preservation is diff --git a/docs/source/concepts/building_intuition.rst b/docs/source/concepts/building_intuition.rst index 917b7f0fa8..037e65d715 100644 --- a/docs/source/concepts/building_intuition.rst +++ b/docs/source/concepts/building_intuition.rst @@ -55,7 +55,7 @@ selection. Choose the owning layer ----------------------- -- source/file semantics: source declarations or microscope integration +- source/file semantics: source declarations or a ``DatasetSource`` integration - callable behavior: callable contract or owning module declaration - storage mechanism: PolyStore - array conversion, framework-local device mechanics, and generic callable diff --git a/docs/source/concepts/component_system_concepts.rst b/docs/source/concepts/component_system_concepts.rst index 7b694af41a..8413f38ff4 100644 --- a/docs/source/concepts/component_system_concepts.rst +++ b/docs/source/concepts/component_system_concepts.rst @@ -50,7 +50,7 @@ Stack and grouping use processing_config=LazyProcessingConfig.from_config(processing), ) -Site declares the stack axis. Channel partitions those site stacks for +Site is the axis the step assembles its stacks along. Channel partitions those site stacks for dictionary routing. ``ProcessingContract`` separately declares whether each callable has per-plane or whole-stack semantics. diff --git a/docs/source/concepts/core_model.rst b/docs/source/concepts/core_model.rst index 70b63d8d98..f258517df3 100644 --- a/docs/source/concepts/core_model.rst +++ b/docs/source/concepts/core_model.rst @@ -70,7 +70,7 @@ the resolved declarations rather than repeatedly resolving individual fields. Contracts and artifacts ----------------------- -Callable contracts declare memory types, required axes, allowed grouping, +Callable contracts declare memory types, required axis roles, allowed grouping, processing contract, execution scope, runtime adapter, and artifact inputs and outputs. CellProfiler module leaves derive invocation-specific ``ArtifactSpec`` inputs and outputs into that same callable contract. Compiler source plans, @@ -84,8 +84,8 @@ addresses and materialization targets. Compilation ----------- -Each execution axis receives a ``CompilationSession`` with resolved steps, -``StepSnapshot`` objects, source projection, and typed ``CompiledStepPlan`` +Each value of the partition axis receives a ``CompilationSession`` with the +resolved pipeline definition, source projection, and typed ``CompiledStepPlan`` objects. Plans contain source, artifact, function, memory, materialization, streaming, GPU, and worker facts. Compilation emits a ``CompiledExecutionBundle`` only after these facts satisfy runtime invariants. diff --git a/docs/source/concepts/data_dimensions.rst b/docs/source/concepts/data_dimensions.rst index baedf54635..fd4f69e87f 100644 --- a/docs/source/concepts/data_dimensions.rst +++ b/docs/source/concepts/data_dimensions.rst @@ -9,8 +9,9 @@ Plate and execution axis ------------------------ The orchestrator discovers component keys through the microscope/source model. -One configured component—normally well—is the multiprocessing execution axis. -Compilation creates one context and typed plan set for each selected axis value. +The active axis family declares one partition axis (well, for microscopy) that +splits work across processes. Compilation creates one context and typed plan +set for each selected value on that axis. Image stack axis ---------------- @@ -34,18 +35,19 @@ An array that contains several Z planes is not automatically treated as one volumetric object domain. Component axes describe what the data contains; callable declarations describe how a function consumes it. -``ProcessingContract`` controls how a callable handles the assembled runtime -stack independently of the component identities: +A ``ProcessingContract`` subclass controls how a callable handles the +assembled runtime stack independently of the component identities: -- ``PURE_2D``: each plane is semantically independent; -- ``PURE_3D``: the callable depends on the full stack; -- ``FLEXIBLE``: an explicit control selects either semantic mode; -- ``VOLUMETRIC_TO_SLICE``: a stack is consumed into a collapsed plane domain. +- ``Pure2DContract``: each plane is semantically independent; +- ``Pure3DContract``: the callable depends on the full stack; +- ``FlexibleContract``: an explicit control selects either semantic mode; +- ``VolumetricToSliceContract``: a stack is consumed into a collapsed plane + domain. Other callable declarations can preserve a complete image payload or a full-stack object-label input. The effective behaviour therefore comes from -the callable's complete contract, not from array shape or one enum in -isolation. For example, the CellProfiler-compatible Watershed route consumes a +the callable's complete contract, not from array shape or one processing +contract in isolation. For example, the CellProfiler-compatible Watershed route consumes a complete 3D payload, whereas ``IdentifyPrimaryObjects``, ``IdentifySecondaryObjects``, and ``IdentifyTertiaryObjects`` currently operate plane by plane. diff --git a/docs/source/concepts/domain_fundamentals.rst b/docs/source/concepts/domain_fundamentals.rst index c7f8dba20b..083ce29978 100644 --- a/docs/source/concepts/domain_fundamentals.rst +++ b/docs/source/concepts/domain_fundamentals.rst @@ -11,10 +11,10 @@ Why the data model matters -------------------------- Microscope vendors differ in filename layout, metadata, and storage. OpenHCS -uses registered microscope handlers and PolyStore source references to project -those formats into a common source model. The original files remain the source; -the projection supplies typed component identities and provenance to the -compiler. +uses registered dataset sources (one per microscope format) and PolyStore +source references to project those formats into a common source model. The +acquired files remain the source; the projection supplies typed component +identities and provenance to the compiler. Why compilation matters ----------------------- diff --git a/docs/source/concepts/function_library.rst b/docs/source/concepts/function_library.rst index abd010deeb..fdb9f12bd2 100644 --- a/docs/source/concepts/function_library.rst +++ b/docs/source/concepts/function_library.rst @@ -14,8 +14,8 @@ declare: - input, output, and execution memory types; - artifact inputs and outputs; - runtime-bound parameters; -- required variable components and allowed grouping; -- ``ProcessingContract``; +- required axis roles and allowed group-by roles; +- a ``ProcessingContract`` subclass; - runtime adapter and image execution mode; - axis- or plate-level execution scope; - compiler preparation hooks. @@ -79,7 +79,7 @@ The desktop form surface materialises every locally available catalogue declaration after applying its owner-declared exclusions; authorable parameters become editable controls while runtime-owned inputs remain hidden. -OpenHCS-native and custom functions are admitted only when every declared input, +OpenHCS-native and custom functions are catalogued only when every declared input, output, and execution framework is installed and allowed by the current process policy. The browsable native catalogue contains public module declarations; private decorated helpers remain internal unless an explicit pipeline transports diff --git a/docs/source/concepts/function_patterns.rst b/docs/source/concepts/function_patterns.rst index 6c90e2d698..8649da5038 100644 --- a/docs/source/concepts/function_patterns.rst +++ b/docs/source/concepts/function_patterns.rst @@ -68,7 +68,7 @@ Callable contracts ------------------ Every normalized invocation has a ``CallableContract``. All invocations in a -step must agree on execution scope and satisfy declared axes, grouping, runtime +step must agree on execution scope and satisfy declared axis roles, grouping, runtime parameters, image mode, and artifacts. Compilation turns the public pattern into a typed compiled pattern; workers consume that compiled form. diff --git a/docs/source/concepts/module_structure.rst b/docs/source/concepts/module_structure.rst index b67b87fc3a..9dcc734147 100644 --- a/docs/source/concepts/module_structure.rst +++ b/docs/source/concepts/module_structure.rst @@ -9,19 +9,29 @@ OpenHCS-owned areas ------------------- ``openhcs/core`` - Pipeline declarations, source/artifact semantics, compiler plans, runtime - values, contexts, orchestrator, progress, and equivalence. + The domain-blind kernel: axis families, dataset sources, pipeline + declarations, callable and processing contracts, source/artifact semantics, + compiler plans, runtime values, contexts, orchestrator, progress, and + equivalence. + +``openhcs/domains`` + Domain declarations that instantiate the kernel; microscopy declares its axis + family and domain configuration here. ``openhcs/processing`` - Processing-library discovery, callable decorators/contracts, native and - CellProfiler-compatible processing implementations. + Processing-library discovery, native and CellProfiler-compatible processing + implementations, presets, and custom functions. ``openhcs/interop`` Translation boundaries such as CellProfiler parsing, module declarations, settings binding, and public lowering. ``openhcs/microscopes`` - Microscope-specific plate layout and metadata integration. + Microscope-specific dataset sources: plate layout and metadata integration. + +``openhcs/authoring`` + The session (datasets, pipelines, compile, run, and their events) that the + Qt GUI and MCP clients render and invoke. ``openhcs/pyqt_gui`` OpenHCS screens, workflow services, editors, Plate Manager, and adapters over diff --git a/docs/source/concepts/pipelines_and_steps.rst b/docs/source/concepts/pipelines_and_steps.rst index 0f49040652..8d6cb75e1b 100644 --- a/docs/source/concepts/pipelines_and_steps.rst +++ b/docs/source/concepts/pipelines_and_steps.rst @@ -44,7 +44,7 @@ The ``func`` field accepts: - a dictionary mapping compiled group keys to either form. Callable contracts—not examples or UI labels—are the authority for accepted -parameters, runtime-injected parameters, required axes, processing behavior, +parameters, runtime-injected parameters, required axis roles, processing behavior, memory types, and artifacts. Data flow @@ -70,10 +70,11 @@ Compile before execution ------------------------ ``PipelineOrchestrator.compile_pipelines(pipeline_steps)`` resolves the pipeline -once and compiles every selected execution axis. The returned -``CompiledExecutionBundle`` contains runtime contexts, worker steps, and -environment decisions. The GUI uses the same boundary and compiles the complete -selected set before executing it. +once and compiles every selected value of the partition axis. The returned +``CompiledExecutionBundle`` contains runtime contexts, worker assignments, and +the runtime environment plan. The GUI and MCP clients reach the same boundary +through the OpenHCS session, which compiles the complete selected set of +datasets before executing it. Related pages ------------- diff --git a/docs/source/concepts/storage_system.rst b/docs/source/concepts/storage_system.rst index c2108e8589..53228ee44b 100644 --- a/docs/source/concepts/storage_system.rst +++ b/docs/source/concepts/storage_system.rst @@ -50,7 +50,8 @@ Callable and module contracts declare typed artifact inputs and outputs. The artifact graph resolves producers and dependencies. Path planning then adds backend addresses and materialization targets to typed plans. -A Python output-slot name or ``@artifact_outputs`` declaration does not make a value persistent. Materialization is an explicit configuration and +A Python output-slot name or ``@artifact_outputs`` declaration does not make a +value persistent. Materialization is an explicit configuration and artifact-plan decision. Runtime storage diff --git a/docs/source/development/ast_refactoring_workflow.rst b/docs/source/development/ast_refactoring_workflow.rst index 89d08aa349..ff645c0f19 100644 --- a/docs/source/development/ast_refactoring_workflow.rst +++ b/docs/source/development/ast_refactoring_workflow.rst @@ -15,7 +15,7 @@ Workflow changes. 5. Review every change that alters configuration nesting, ownership, or runtime semantics. -6. run ``scripts/validate_docs.py`` and Sphinx with warnings as errors. +6. Run ``scripts/validate_docs.py`` and Sphinx with warnings as errors. The AST pass is useful for import moves, renamed call sites, and detecting old constructor keywords. It must not invent compatibility aliases, mirror diff --git a/docs/source/development/callable_artifact_authoring.rst b/docs/source/development/callable_artifact_authoring.rst index 364c9da1ee..8b06c44fc9 100644 --- a/docs/source/development/callable_artifact_authoring.rst +++ b/docs/source/development/callable_artifact_authoring.rst @@ -9,8 +9,8 @@ Callable contract ----------------- The contract can carry input, output, and execution memory types, artifact -inputs and outputs, runtime-bound parameters, required variable components, -allowed grouping, processing contract, execution scope, runtime adapter, +inputs and outputs, runtime-bound parameters, required axis roles, allowed +group-by roles, processing contract, execution scope, runtime adapter, preparation hook, and a callable request binding. The execution memory role is the framework whose device scope must be active while the callable runs; it may differ from an input or output conversion boundary. Use the existing decorators @@ -63,8 +63,8 @@ main-flow publication, and the active output subset are then represented by compiled source, edge, and artifact plans; do not copy them into declaration partitions. -``special_inputs`` describes ABI parameters only. They are not a substitute for artifact types, producer edges, -or materialization declarations. +``special_inputs`` describes ABI parameters only. They are not a substitute +for artifact types, producer edges, or materialization declarations. ``special_inputs("labels")`` alone does not bind a required ``labels`` parameter. Compilation rejects that incomplete contract, even if authored kwargs contain a @@ -209,7 +209,7 @@ still determine their return positions and materialisation. An output image reference is not an input-qualified group-scope source. ``MaterializationSourceIdentityRelation`` is supported for image targets, not labels or measurements, and is therefore not attached to those outputs here. -``PURE_2D`` asks the runtime to dispatch planes independently; +``Pure2DContract`` asks the runtime to dispatch planes independently; the runtime projects the local ``slice_index=0`` rows onto the assembled stack's plane axis. It does not create volumetric object identities. @@ -252,7 +252,7 @@ where you saved it. For MCP custom registration, submit the complete declaration block as ``source_code`` to the discovered ``openhcs_register_custom_function`` capability. Follow the ``openhcs_custom_function_workflow`` guide for explicit endpoint, intended storage, preparation and uncertainty handling. Retain the -registration receipt, then describe the deliberately authored public function +registration result, then describe the deliberately authored public function identifier to verify its ``function_id``, ``import_path`` and artifact contract rather than assuming registration returns a function catalogue. Custom callables are projected under ``openhcs.processing.custom_functions``. A local @@ -337,7 +337,7 @@ On the existing centre-producing callable, declare ``@artifact_outputs(CENTRE_IMAGE, CENTRE_LABELS, CENTRE_ROWS)`` and return ``image, labels, DataclassMeasurementColumnarRows(rows, row_type=CentreRow)``. Use the existing memory/processing decorators, as in the executable reference; -``PURE_3D`` is appropriate only for an actual volumetric calculation. Retain +``Pure3DContract`` is appropriate only for an actual volumetric calculation. Retain the complete source bindings and compiled ZYX grouping: the decorator alone does not establish axis order. The row's representative rule (peak, body centre or another task-defined location) remains the analysis contract, not the writer. @@ -458,7 +458,7 @@ the compatible producer, retaining the complete pipeline configuration/source bindings. ``np.asarray(objects)`` is a local pixel view for this calculation, not a replacement for the nominal input or its provenance. If the task needs object/plane identity, use the value's domain and plane APIs rather than -reconstructing them from dense pixels. ``PURE_2D`` here describes locality; +reconstructing them from dense pixels. ``Pure2DContract`` here describes locality; it does not select the pipeline's variable axis. Repair the earliest failed declaration before trying execution. If compilation diff --git a/docs/source/development/compiler_extension.rst b/docs/source/development/compiler_extension.rst index 719e470049..05c69a604b 100644 --- a/docs/source/development/compiler_extension.rst +++ b/docs/source/development/compiler_extension.rst @@ -8,10 +8,12 @@ parallel dictionary plan. Invariants ---------- -``StepSnapshot`` binds one resolved step, compiler index, and ObjectState scope -identity. ``CompilationSession`` validates that steps, snapshots, states, -context, plans, source projection, and plate scope describe the same execution -axis. +``ResolvedPipelineDefinition`` captures the enabled steps with their +ObjectState scope identities and provenance, and admits their artifact +contracts and forward topology once before axis fan-out. ``CompilationSession`` +binds that definition to one ``ProcessingContext`` (one axis or sequential +combination), its compiled-plan map, source projection, and dataset scope; +``plan(index)`` fails when a step has no compiled plan. Adding a decision ----------------- diff --git a/docs/source/development/extension_workflows.rst b/docs/source/development/extension_workflows.rst index 53226a780a..69ee1d18f0 100644 --- a/docs/source/development/extension_workflows.rst +++ b/docs/source/development/extension_workflows.rst @@ -70,7 +70,7 @@ Compiler/runtime split ---------------------- Mutable ObjectState is resolved once. Compiler phases consume the resolved -step through ``StepSnapshot`` and ``CompilationSession``. Workers consume +steps through ``ResolvedPipelineDefinition`` and ``CompilationSession``. Workers consume ``CompiledExecutionBundle`` and must not reinterpret UI state or callable attributes. @@ -92,11 +92,13 @@ Serialization boundaries Python source generation and JSON transport are different contracts. pycodify owns executable Python source, imports, and formatter composition. -OpenHCS ``to_jsonable`` owns the JSON-native projection used by agent -capabilities and UI-bridge transport. It uses ``singledispatch`` for mappings, -sequences, callables, registered nominal types, enums, paths, and scalars. +python-introspect ``to_jsonable`` owns the JSON-native projection used by +agent capabilities and UI-bridge transport, paired with the +``dataclass_from_mapping`` decoder. It uses ``singledispatch`` for dataclasses, +mappings, sequences, callables, registered nominal types, enums, paths, and +scalars. -Add a ``to_jsonable.register`` implementation at the OpenHCS value type only +Add a ``to_jsonable.register`` implementation beside the OpenHCS value type only when that type has a stable public transport representation. Do not make pycodify formatters or generic JSON consumers guess domain identity, and never use either projection to reconstruct compiler semantics that belong in typed diff --git a/docs/source/development/git_worktree_testing.rst b/docs/source/development/git_worktree_testing.rst index 5739c814af..333d018f2b 100644 --- a/docs/source/development/git_worktree_testing.rst +++ b/docs/source/development/git_worktree_testing.rst @@ -80,14 +80,14 @@ Run the smallest suite that exercises the change first: .. code-block:: console - OPENHCS_CPU_ONLY=1 python -m pytest tests/unit/path/to/test_owner.py -q + OPENHCS_CPU_ONLY=true python -m pytest tests/unit/path/to/test_owner.py -q When the change crosses compilation or execution boundaries, run a bounded integration configuration before widening the matrix: .. code-block:: console - OPENHCS_CPU_ONLY=1 python -m pytest tests/integration/test_main.py \ + OPENHCS_CPU_ONLY=true python -m pytest tests/integration/test_main.py \ --it-backends disk \ --it-microscopes ImageXpress \ --it-dims 3d \ @@ -111,7 +111,7 @@ commit and recursive submodule revisions beside each result: git rev-parse HEAD git submodule status --recursive - OPENHCS_CPU_ONLY=1 python -m pytest tests/unit -q > feature-results.txt 2>&1 + OPENHCS_CPU_ONLY=true python -m pytest tests/unit -q > feature-results.txt 2>&1 Repeat from the comparison worktree, then compare the result files with ``diff``. Tracked fixtures are checked out independently in each worktree. diff --git a/docs/source/development/mcp_knowledge_base_manifest.json b/docs/source/development/mcp_knowledge_base_manifest.json index d79d1b0224..56012988d5 100644 --- a/docs/source/development/mcp_knowledge_base_manifest.json +++ b/docs/source/development/mcp_knowledge_base_manifest.json @@ -362,7 +362,7 @@ "document_id": "openhcs_pipeline_compilation_system", "section_count": 0, "source_path": "docs/source/architecture/pipeline_compilation_system.rst", - "summary": "One-time ObjectState resolution, StepSnapshot and CompilationSession ownership, typed CompiledStepPlan fields, artifact/source planning, validation, and runtime bundle construction.", + "summary": "One-time ObjectState resolution, ResolvedPipelineDefinition and CompilationSession ownership, typed CompiledStepPlan fields, artifact/source planning, validation, and runtime bundle construction.", "tags": [ "compiler", "runtime", diff --git a/docs/source/development/mcp_release.rst b/docs/source/development/mcp_release.rst index cbea5463b6..dfb43cf60c 100644 --- a/docs/source/development/mcp_release.rst +++ b/docs/source/development/mcp_release.rst @@ -78,7 +78,7 @@ a complete, revisioned function catalogue, resolves one returned function reference, and verifies that the ready endpoint's advertised log is readable and contains output. A packaged desktop MCP session then verifies health and version identity, discovers the live main and Plate Manager windows, and invokes -the declared Plate Manager code action. The probe requires the resulting window +the Plate Manager's ``show_dataset_code`` session operation. The probe requires the resulting window to appear, then captures the main window and opened editor through the production MCP snapshot capability. It verifies each PNG's path, dimensions, byte size, and SHA-256 before closing the GUI, removing the bridge descriptor, @@ -242,8 +242,11 @@ annotated ``v`` tag through the configured upstream remote. Do not create a second MCP-specific or installer-specific tag. That tag starts ``.github/workflows/publish.yml``. The workflow builds and -validates the Windows and macOS installer assets first, then builds and smoke -tests the OpenHCS wheel outside the checkout. After publishing the wheel and +validates the Windows and macOS installer assets and, with ``cibuildwheel``, +the stable-ABI OpenHCS wheels for Linux (x86_64 and aarch64), Windows, and +macOS (x86_64 and arm64), smoke-testing each wheel's native extension. It then +builds the source distribution, validates every wheel, and smoke tests the +Linux x86_64 wheel outside the checkout. After publishing the wheels and source distribution to PyPI, it creates one GitHub Release containing those Python artifacts plus the directly runnable ``OpenHCS-Windows-Installer.exe`` and the single-file @@ -273,8 +276,8 @@ complete release from that tag: --field publish_python_package=true \ --field publish_desktop_installers=true -This path validates and smoke-tests the built wheel, publishes the wheel and -source distribution to PyPI, assembles the GitHub Release with both native +This path builds and validates the native wheels, smoke-tests the Linux wheel, +publishes the wheels and source distribution to PyPI, assembles the GitHub Release with both native installers, then publishes the matching official MCP Registry entry. It reuses the existing tag. The upload remains fail-closed when that PyPI version already exists so independently rebuilt files cannot be mistaken for the published @@ -291,7 +294,7 @@ again: --field publish_python_package=false \ --field publish_desktop_installers=true -This path downloads the exact wheel and source distribution already owned by +This path downloads the exact wheels and source distribution already owned by PyPI and verifies their declared SHA-256 digests. It builds both native installers, attaches the complete asset set to the existing tag, and continues to MCP Registry publication. The Python files on PyPI and GitHub are therefore diff --git a/docs/source/development/respecting_codebase_architecture.rst b/docs/source/development/respecting_codebase_architecture.rst index 592be234c9..a42f2c173e 100644 --- a/docs/source/development/respecting_codebase_architecture.rst +++ b/docs/source/development/respecting_codebase_architecture.rst @@ -30,7 +30,7 @@ Registry discipline For a nominal family, the root registry is the catalog. Generic consumers normally iterate ``Root.__registry__.values()`` or use the helpers in -``openhcs.core.registry_strategies``. +``metaclass_registry.strategies``. Do not add a second dictionary, set, ordered list, priority table, copied field list, compatibility alias, or fallback chain for the same fact. Such mirrors diff --git a/docs/source/development/runtime_system_assembly_rules.rst b/docs/source/development/runtime_system_assembly_rules.rst index 3eb6e19fb6..e0c667c865 100644 --- a/docs/source/development/runtime_system_assembly_rules.rst +++ b/docs/source/development/runtime_system_assembly_rules.rst @@ -11,7 +11,7 @@ Assembly order PipelineConfig + FunctionStep declarations -> one ObjectState resolution - -> StepSnapshot + CompilationSession + -> ResolvedPipelineDefinition + CompilationSession -> typed CompiledStepPlan objects -> CompiledExecutionBundle -> workers + RuntimeValueStore @@ -43,14 +43,17 @@ Owner map including the exact framework-local device bindings selected for each step. ``CompiledExecutionBundle`` - Contexts, worker-safe steps, plans, and the resolved runtime environment. + Runtime contexts, pickle-safe transport contexts, worker assignments, and the + resolved runtime environment. ``RuntimeValueStore`` Validated runtime values, artifact keys, locations, current bindings, and observation history. ``RuntimeSliceProjection`` - Nominal, provenance-aware slice projection and aggregation. + Slice projection of runtime values. Owned value types declare their slice + behaviour through ``RuntimeSliceProjectableValue``; stack values are checked + against their source provenance. Source and artifacts -------------------- diff --git a/docs/source/development/runtime_value_extension.rst b/docs/source/development/runtime_value_extension.rst index 904403092a..b379d1354f 100644 --- a/docs/source/development/runtime_value_extension.rst +++ b/docs/source/development/runtime_value_extension.rst @@ -9,22 +9,25 @@ Required decisions 1. Define the payload and its artifact type/spec boundary. 2. Decide how it is validated and wrapped in ``RuntimeValue``. -3. Register ``RuntimeSliceProjectionStrategy`` behavior if it can be projected - or aggregated across runtime slices. +3. Subclass ``RuntimeSliceProjectableValue`` (or a narrower base such as + ``RuntimeSliceIndexedValue`` or ``RuntimeSliceInvariantValue``) if the + payload can be projected across runtime slices. 4. Add serialization, worker transport, materialization, equivalence, or viewer strategies only where the value participates in those systems. 5. Keep persistence location separate from its ``ArtifactKey``. -Projection strategies are selected by nominal type and MRO specificity. A value -that preserves a runtime slice must carry explicit plane-axis and source -identity declarations; raw shape is insufficient. +Projection behaviour lives on the payload class itself. +``RuntimeSliceProjection`` handles only foreign values such as arrays, +mappings and primitives, and rejects other values that declare no slice +behaviour. A value that preserves a runtime slice must carry explicit +plane-axis and source identity declarations; raw shape is insufficient. ``RuntimeValueStore`` records validated values under an artifact key and explicit backend location. Consumers resolve typed queries derived from compiled artifact edges. Missing or ambiguous results must fail rather than searching paths or fallback names. -Verify store record/replace/query behavior, slice projection and aggregation, +Verify store record/replace/query behavior, slice projection, transport round trips, and materialization for every supported backend. See :doc:`../architecture/runtime_value_system`. diff --git a/docs/source/development/source_binding_extension.rst b/docs/source/development/source_binding_extension.rst index 558eec463e..52e0573d28 100644 --- a/docs/source/development/source_binding_extension.rst +++ b/docs/source/development/source_binding_extension.rst @@ -23,15 +23,16 @@ Strategy selection ------------------ Existing families select source universes, matching behavior, component -projection, and workspace projection. Add a concrete strategy to the correct -root and declare its enum/type/context key. Generic consumers should continue -to use the root registry or its strategy mixin. +projection, and workspace projection, for example ``SourceUniverseRequest`` +and ``SourceFilterMatcher``. Add a concrete subclass to the correct family root +and declare its registry key. Generic consumers should continue to use the +root registry or its strategy mixin. Virtual workspaces ------------------ PolyStore owns ``SourcePixelRef`` and generic backend addressing. OpenHCS owns -microscope metadata and ``VirtualWorkspaceSourceProjection``, which maps +dataset source metadata and ``VirtualWorkspaceSourceProjection``, which maps virtual paths to source references and provenance for one compilation session. Verify declaration validation, strategy selection, physical and virtual path @@ -41,7 +42,7 @@ folder metadata rules project the declared biological address without changing the backend-owned source address, and that reopening the persisted workspace reports the projected component values and labels. When a component selector and component identity use different values, verify that the selector matches -the original candidate while the projected address, compiler group keys, and +the emitted candidate while the projected address, compiler group keys, and runtime provenance use the declared component identity. See :doc:`../architecture/source_model`. diff --git a/docs/source/getting_started/getting_started.rst b/docs/source/getting_started/getting_started.rst index ff2a9c5deb..31d7b31719 100644 --- a/docs/source/getting_started/getting_started.rst +++ b/docs/source/getting_started/getting_started.rst @@ -56,12 +56,6 @@ Manual Python installation Create a virtual environment, then install the same CPU-safe capability set as the desktop installer: -.. note:: - - The complete CellProfiler compatibility extra currently requires Python 3.13 - or earlier because its optional parity dependency does not publish Python - 3.14 wheels. Python 3.14 supports the base package and core-service extras. - .. code-block:: bash python -m pip install --upgrade pip diff --git a/docs/source/guide_for_biologists/basic_interface.rst b/docs/source/guide_for_biologists/basic_interface.rst index abb2c249ae..51624b6d3e 100644 --- a/docs/source/guide_for_biologists/basic_interface.rst +++ b/docs/source/guide_for_biologists/basic_interface.rst @@ -13,7 +13,7 @@ The desktop is a set of ObjectState-backed workflow windows, not one giant form: .. code-block:: text Main window - |- Plate Manager datasets, selection, init/compile/run + |- Datasets datasets, selection, init/compile/run |- Pipeline Editor PipelineConfig + ordered FunctionStep list | `- Step editors callable pattern + step-local config |- Global Configuration application/session defaults @@ -21,19 +21,20 @@ The desktop is a set of ObjectState-backed workflow windows, not one giant form: |- Log/progress/debug operation status and diagnostics `- Analysis/tools consolidation, layouts, custom functions -The Plate Manager is the operational hub. Other windows edit or inspect the +The Datasets pane is the operational hub. Other windows edit or inspect the selected plate, pipeline, step, source projection, or application config. A Napari or Fiji viewer is a separate runtime window launched only when requested. -In the supported PyQt desktop shell, Plate Manager and the ZMQ process manager +In the supported PyQt desktop shell, the Datasets pane and the ZMQ server manager occupy the left workspace, Pipeline Editor occupies the right workspace, and the system monitor is docked above them. Managed editors, configuration windows, logs, and tools open or focus around that embedded workspace. The PyQt desktop shell is the supported MCP-attached UI. -Plate Manager +Datasets pane ------------- -Plate Manager is the starting point for a dataset. Add a local or configured +The **Datasets** pane (window identity ``plate_manager`` for MCP agents) is the +starting point for a dataset. Add a local or configured remote plate, initialize it, and select the plate before editing or running its pipeline. Initialization discovers the microscope/source format and component metadata. Compile and run actions are disabled until their prerequisites are @@ -49,7 +50,7 @@ It is disabled during startup or disconnection, even if a compiled plate remains selected. During execution, the same control provides **Stop**; stopping controls remain available if the endpoint becomes unavailable. -Plate Manager code mode records whether its document represents all plates or +Datasets code mode records whether its document represents all plates or only the current selection. Applying a selected document changes only those plates and preserves every unselected plate. Read a new document if you need to change which plates the edit covers. @@ -92,7 +93,7 @@ reference to the authoring process; it does not rebuild the catalogue there. List previews ------------- -The ``list_previews`` section of UI configuration controls the Plate Manager +The ``list_previews`` section of UI configuration controls the Datasets and Pipeline Editor summaries. It sets the default wrapping behaviour, includes or hides detail lines and modified settings, and shows collections compactly or in detail. Saving applies the same display policy to both panels immediately. @@ -136,16 +137,17 @@ Agent navigation An attached MCP agent should navigate semantically: 1. list running windows and ObjectState scopes; -2. read the Plate Manager state surface to identify selected/source/output rows; +2. read the Datasets (``plate_manager``) state surface to identify selected/source/output rows; 3. open or focus the relevant window/scope; 4. list its code documents or fields rather than guessing widget labels; 5. validate code, then apply with the current revision token; -6. dispatch semantic init/compile/run actions and poll operation/state surfaces. +6. dispatch semantic init/compile/run actions and follow the work they start + through the session events and state surfaces. To select a manager item through MCP, use that manager's window identity and the item identity from its state surface, then check the current selection before invoking an action. Opening a plate's configuration window does not -select its Plate Manager row. +select its Datasets row. Window focus and field navigation are reported separately. A configuration field request selects the tab whose declared ObjectState scope owns that path. @@ -154,7 +156,7 @@ request reports that the target was unsupported instead of claiming it was navigated. The stable ``main_window`` snapshot captures the complete workspace and status -bar. Pane identities capture a focused Plate Manager, Pipeline Editor, server +bar. Pane identities capture a focused Datasets pane, Pipeline Editor, server browser, or system-monitor view, while transient dialogs are listed when they are open. Window snapshots and the widget tree are useful for orientation or diagnosing a missing semantic surface. They are not configuration authorities. diff --git a/docs/source/guide_for_biologists/configuration_reference.rst b/docs/source/guide_for_biologists/configuration_reference.rst index 5152c26bd3..220a8c3b61 100644 --- a/docs/source/guide_for_biologists/configuration_reference.rst +++ b/docs/source/guide_for_biologists/configuration_reference.rst @@ -27,14 +27,14 @@ There are three declaration scopes, followed by one resolution boundary: ObjectState resolution at compile time | v - immutable step snapshots and typed compiled plans + resolved step declarations and typed compiled plans .. openhcs-gallery:: ui-pipeline-configuration ``None`` in a lazy field normally means *inherit*, not *disable*. Enableable configs have a separate ``enabled`` field; resetting a lazy field returns it to inheritance. Compilation resolves the hierarchy once. Runtime workers consume -the resolved snapshots and plans and never reinterpret the live forms. +the resolved declarations and plans and never reinterpret the live forms. Scope precedence is only one axis. Config dataclass inheritance is the other: it determines which related policies share a default. The important well-filter @@ -43,7 +43,7 @@ relationship is: .. code-block:: text PartitionFilterConfig pipeline execution domain - |-- PathPlanningConfig automatic output-plate persistence + |-- PathPlanningConfig automatic output-plate persistence `-- StepPartitionFilterConfig step-policy default |-- StepMaterializationConfig `-- StreamingDefaults @@ -181,8 +181,8 @@ Post-run analysis and metadata ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ``AnalysisConsolidationConfig`` - Optional post-plate consolidation, including well pattern, included - extensions, exclusions, and output filenames. + Optional post-plate consolidation, including MetaXpress-style formatting, + included extensions, exclusions, and output filenames. ``PlateMetadataConfig`` Plate identity and acquisition metadata used by compatible consolidated diff --git a/docs/source/guide_for_biologists/domain_expert_onboarding.rst b/docs/source/guide_for_biologists/domain_expert_onboarding.rst index 636008e6f5..afd1d60843 100644 --- a/docs/source/guide_for_biologists/domain_expert_onboarding.rst +++ b/docs/source/guide_for_biologists/domain_expert_onboarding.rst @@ -87,8 +87,8 @@ to execution and result review. Choose UI-visible or headless ownership before creating a pipeline. When the scientist should see or continue editing the work in the desktop, use the -UI-owned route; a headless draft or execution does not create Plate Manager -state. Discover the capabilities exposed by the current profile before +UI-owned route; a headless draft or execution does not create a row in the +desktop Datasets pane. Discover the capabilities exposed by the current profile before following a guide, and validate a representative bounded run before scaling up. CellProfiler Mental Model diff --git a/docs/source/guide_for_biologists/image_sources.rst b/docs/source/guide_for_biologists/image_sources.rst index 36e6577440..5c1fccffd8 100644 --- a/docs/source/guide_for_biologists/image_sources.rst +++ b/docs/source/guide_for_biologists/image_sources.rst @@ -4,22 +4,22 @@ Loading and naming image sources An OpenHCS plate can contain one image store or several stores together. Two separate decisions are involved: -1. a microscope or store handler owns **ingestion** and publishes addressable - planes; +1. a microscope or store dataset source owns **ingestion** and publishes + addressable planes; 2. source bindings optionally **select and name** those planes for pipeline inputs. .. openhcs-gallery:: source-image-browser -For a recognized HCS layout, OpenHCS keeps the native microscope handler. CZI, +For a recognized HCS layout, OpenHCS keeps the native microscope source. CZI, OME-TIFF, and other supported rich containers are decoded by the Bio-Formats -store handler using their embedded metadata. Add the containing directory to -the Plate Manager and initialize with auto-detection; a source binding is not +store source using their embedded metadata. Add the containing directory to +the Datasets pane and initialize with auto-detection; a source binding is not required merely to open or browse the store. -``SourceBindingsHandler`` has a different role: it is the ingestion fallback for +``SourceBindingsSource`` has a different role: it is the ingestion fallback for an arbitrary image folder whose layout is not recognized. A non-empty -``PipelineConfig.source_bindings_config`` tells that handler which ordinary +``PipelineConfig.source_bindings_config`` tells that source which ordinary TIFF, PNG, JPEG, or other registered files form the source universe and how filename metadata should be interpreted. @@ -30,7 +30,7 @@ their original file provenance and gives them a stable name such as ``DNA`` or internals and do not select a storage backend. This means ``SourceBindingsConfig`` can still name channels or samples from a -CZI after Bio-Formats has decoded it. In that case the Bio-Formats handler remains +CZI after Bio-Formats has decoded it. In that case the Bio-Formats source remains the ingestion owner; the binding is semantic selection, not a replacement CZI loader. @@ -40,7 +40,7 @@ OME-Zarr input and output OpenHCS reads OME-Zarr single-resolution images and plates stored as Zarr 2 / NGFF 0.4 or Zarr 3 / NGFF 0.5. Discovery uses the declared image hierarchy and axes, including time, channel and Z dimensions. Add the directory containing the -store to the Plate Manager as for other rich image containers. +store to the Datasets pane as for other rich image containers. Selecting Zarr for pipeline output continues to produce Zarr 2 / NGFF 0.4 stores. Updating the installed Zarr library does not change that output format. @@ -53,11 +53,11 @@ Native layout versus complete metadata ---------------------------------------- Filename/layout recognition and automatic detection are related but distinct. -Run ``openhcs_inspect_plate_path`` with ``microscope_type="auto"`` first. Its +Run ``openhcs_inspect_plate_path`` with ``source_format="auto"`` first. Its ``format_specific_handler_candidates`` are derived live from the registered -``MicroscopeHandler`` parsers and detection contracts. If a broad store handler +``DatasetSource`` parsers and detection contracts. If a broad store source was selected but a format-specific candidate recognizes every tested physical -filename under its declared ``root_dir``, do not assume the broad handler is the +filename under its declared ``root_dir``, do not assume the broad source is the better semantic owner merely because native metadata is missing. Opera Phenix is a representative case. A complete plate export contains the @@ -70,17 +70,17 @@ owner requires its ``Index.xml`` detection contract; matching filenames alone do not make the subset a valid native plate. If those TIFFs are intentionally being treated as loose ordinary images, select them with ``SourceBindingsConfig`` and declare their Well/Site/Channel/Z/Time identities instead of explicitly selecting -``microscope_type="opera_phenix"`` or accepting a broad decoder's inferred sample +``source_format="opera_phenix"`` or accepting a broad decoder's inferred sample layout. -The folder-onboarding authoring context lists every currently registered handler -and its handler-owned selection role. Use the format-specific owner for recognized +The folder-onboarding authoring context lists every currently registered +dataset source and its selection role. Use the format-specific owner for recognized vendor layouts, the broad structured-store owner for supported rich containers without a stronger native match, and the declared-file fallback for arbitrary ordinary files whose schema you provide. ``source_stack_components`` describes axes physically present inside one selected -file or store payload. Do not declare ``SITE`` or another source-stack axis when +file or store payload. Do not declare ``Microscopy.Site`` or another source-stack axis when each site is a separate ordinary 2-D file. Give each file its component identity, then use ``processing_config.variable_components`` to assemble those selected files along the callable's stack axis. @@ -88,10 +88,10 @@ along the callable's stack axis. Coordinates and source identity ------------------------------- -Every resolved plane has one sample identity represented by ``WELL``. This is the +Every resolved plane has one sample identity represented by ``Microscopy.Well``. This is the embedded plate well when the image metadata declares one; otherwise it is the -exact source-container identity. ``SITE``, ``CHANNEL``, ``Z_INDEX``, and -``TIMEPOINT`` identify the plane within that sample. If a store declares an axis +exact source-container identity. ``Microscopy.Site``, ``Microscopy.Channel``, +``Microscopy.ZIndex``, and ``Microscopy.Timepoint`` identify the plane within that sample. If a store declares an axis absent or singleton, its coordinate is ``"1"``. Embedded coordinates are never replaced by filename guesses, and conflicting plane identities stop initialization with an explicit error. @@ -132,7 +132,7 @@ Keep three layers separate: pipeline source universe ``PipelineConfig.source_bindings_config`` declares the full named physical - source universe and the inputs consumed by the nominal handler projection. It + source universe and the inputs consumed by the dataset source projection. It may correctly contain Hoechst channel 1, MAP2 channel 2, and SMI312 channel 4 even when one step needs only two of them. Seeing all three in the source workspace inventory is valid. @@ -145,7 +145,7 @@ per-step binding plan callable stack positions ``processing_config.variable_components`` owns the meaning of the assembled array axis. A callable ``channel_index`` is a zero-based position on that - invocation's assembled axis, not a physical microscope ``CHANNEL`` value. + invocation's assembled axis, not a physical ``Microscopy.Channel`` value. When viewer routes carry physical component identities from the first layer, they retain the declared values; they do not renumber them to the third layer's @@ -157,7 +157,7 @@ declaration choices: simplest inherited full stack Set ``input_source=PIPELINE_START`` and - ``variable_components=[CHANNEL]``, but omit ``FunctionStep.source_bindings``. + ``variable_components=[Microscopy.Channel]``, but omit ``FunctionStep.source_bindings``. The step inherits the complete pipeline order, so use ``nuclear_stain.channel_index=0``, ``cell_body.channel_index=1``, and ``neurite_channel_index=2``. Source provenance retains physical channels 1, @@ -172,11 +172,11 @@ legacy shared-signal subset not equivalent to MAP2-seeded analysis: retain MAP2 and use the inherited three-channel form whenever MAP2 owns the neuronal bodies. -In either option, ``variable_components=[CHANNEL]`` assembles the channel stack. -Do not use ``group_by=CHANNEL`` for assembly. ``group_by`` partitions an already +In either option, ``variable_components=[Microscopy.Channel]`` assembles the channel stack. +Do not use ``group_by=Microscopy.Channel`` for assembly. ``group_by`` partitions an already assembled value and selects branches only for a dictionary function pattern. For this non-dictionary MetaXpress callable, an overlapping -``group_by=CHANNEL`` is redundant and the compiler normalizes it to +``group_by=Microscopy.Channel`` is redundant and the compiler normalizes it to ``Ungrouped``. The explicit-subset regression is intentionally generic rather than a @@ -225,8 +225,8 @@ authoring a multi-image measurement, separate its current image input from its typed artifact inputs. ``input_source=PIPELINE_START`` returns to acquisition images; it does not remove the step's inherited variable-component scope. When the invocation needs an assembled channel stack, declare -``variable_components=[CHANNEL]`` with the intended step bindings/order. A -``SITE`` stack or ``group_by=NONE`` alone does not assemble channels. Inspect the +``variable_components=[Microscopy.Channel]`` with the intended step bindings/order. A +``Microscopy.Site`` stack or ``group_by=Ungrouped`` alone does not assemble channels. Inspect the resolved step configuration and artifact plan, not just the pipeline universe. Object-label inputs also retain their producer's source context. Equal shape, @@ -282,10 +282,10 @@ The following block constructs a complete Pipeline Editor code document for a directory that mixes stores. The intermediate config values deliberately show both source-binding roles: -- ``tiff_png_config`` can drive ``SourceBindingsHandler`` when the directory is +- ``tiff_png_config`` can drive ``SourceBindingsSource`` when the directory is an otherwise unrecognized collection of ordinary files; - the CZI, OME-TIFF, and OME-Zarr configs name/select planes after their store - handlers decode them. + sources decode them. Choose one as ``pipeline_config`` and keep the ``pipeline_steps`` assignment in the same document. Filenames are exact examples; replace them with names present @@ -390,14 +390,14 @@ detection. OpenHCS first chooses the native, Bio-Formats/store, or arbitrary- folder ingestion owner, then resolves every binding over the planes that owner published. CZI and OME-TIFF inputs use the Bio-Formats runtime included by an OpenHCS installation that enables those formats; OME-Zarr uses PolyStore -directly. Do not force a rich image container through ``SourceBindingsHandler`` +directly. Do not force a rich image container through ``SourceBindingsSource`` when its structured decoder is missing or unhealthy; repair that decoder instead. Reopening OpenHCS outputs ------------------------- -Add a generated output plate to the Plate Manager and initialize it normally. +Add a generated output plate to the Datasets pane and initialize it normally. OpenHCS reloads the semantic axes and source contributors saved with each processed image. Do not add source bindings merely to recreate an axis that a processing step collapsed: a coordinate retained in the filename identifies the @@ -415,8 +415,8 @@ coordinate in a format-specific path rule. Use it as ``pipeline_config`` in the complete document above; source-backed MCP execution and Pipeline Editor code mode must also receive ``pipeline_steps``. -``selector.components`` matches the coordinates published by the source -handler. ``component_identity`` is optional and assigns the semantic coordinate +``selector.components`` matches the coordinates published by the dataset +source. ``component_identity`` is optional and assigns the semantic coordinate used after selection. Here the selector matches physical channel ``2`` while ``component_identity`` assigns the projected channel value ``DNA``. The original channel remains in original source metadata; compilation, grouping, diff --git a/docs/source/guide_for_biologists/intro.rst b/docs/source/guide_for_biologists/intro.rst index 8548e3b02d..036e644365 100644 --- a/docs/source/guide_for_biologists/intro.rst +++ b/docs/source/guide_for_biologists/intro.rst @@ -22,8 +22,8 @@ OpenHCS is useful when you need to: The available microscope formats and function libraries depend on the installed version and optional dependencies. The application discovers them from their -registered declarations; use the Plate Manager and function browser as the -current capability catalog. +registered declarations; use the ``dataset_source`` choices in the +configuration form and the function browser as the current capability catalog. How a run works --------------- diff --git a/docs/source/guide_for_biologists/intro_stitching.rst b/docs/source/guide_for_biologists/intro_stitching.rst index acb74cb4d3..92fbd2030b 100644 --- a/docs/source/guide_for_biologists/intro_stitching.rst +++ b/docs/source/guide_for_biologists/intro_stitching.rst @@ -37,9 +37,9 @@ Initialise the source data 2. Select **Init** in Plate Manager. 3. Wait for initialisation to finish. -When initialisation succeeds, **Compile** becomes available. Open the metadata -viewer if you want to confirm the discovered wells, sites, channels, and Z -planes before continuing. +When initialisation succeeds, **Compile** becomes available. Select **Viewer** +and open its **Metadata** tab if you want to confirm the discovered wells, +sites, channels, and Z planes before continuing. .. openhcs-gallery:: ui-plate-metadata diff --git a/docs/source/guide_for_biologists/troubleshooting_FAQ.rst b/docs/source/guide_for_biologists/troubleshooting_FAQ.rst index 4e3897435d..507a936249 100644 --- a/docs/source/guide_for_biologists/troubleshooting_FAQ.rst +++ b/docs/source/guide_for_biologists/troubleshooting_FAQ.rst @@ -59,9 +59,9 @@ the running UI. An attached MCP client can request the main window's **Restart OpenHCS and restore session** action once initialisation, compilation and execution have finished. It saves the plate declarations and edit history before relaunching. -The accepted action receipt precedes shutdown: rediscover the new UI bridge and -check the restored session after restart. The old process's operation receipt -does not persist across that boundary. +The action returns before the old UI exits: rediscover the new UI bridge and +check the restored session after restart. The old process's operation status +does not survive the restart. If a restored session refers to a local plate directory that is unavailable, OpenHCS restores the plate and edit history but leaves that plate uninitialized. @@ -100,7 +100,7 @@ configuration. If a function registered through MCP is missing, confirm that the MCP client and desktop use the same execution endpoint, then reopen the selector. Each opening requests current server membership. For isolated development sessions, -``OPENHCS_UI_CONFIG_CACHE_FILE`` selects the canonical UI configuration for +``OPENHCS_UI_CONFIG_CACHE_FILE`` selects the shared UI configuration for both processes; selecting a UI bridge descriptor alone does not select the execution endpoint. diff --git a/docs/source/guides/complete_examples.rst b/docs/source/guides/complete_examples.rst index 2b8b491d1b..9efddbdcf3 100644 --- a/docs/source/guides/complete_examples.rst +++ b/docs/source/guides/complete_examples.rst @@ -14,7 +14,7 @@ the returned section from ``openhcs_official30_benchmark_recipes`` with ``max_chars=50000``. For example, section ``examplehuman-openhcs-python`` defines both ``pipeline_config`` and ``pipeline_steps`` using the public API. Sections are generated lazily from the -manifest-resolved ``.cppipe`` through the canonical importer; they are not a +manifest-resolved ``.cppipe`` through the public importer; they are not a second checked-in set of generated scripts. The same source is available to Python clients through @@ -74,7 +74,8 @@ Loose Opera Phenix neurite outgrowth is a complete, parameterized CellProfiler-backed example for selected Opera Phenix TIFFs copied without ``Index.xml``. Edit its ``example_inputs`` boundary for the plate path, exact Hoechst/MAP2/SMI312 filenames, well/site/Z/time -identities, output root, and viewer port. +identities, output root, and viewer port. Set its ``source_voxel_spacing`` when +the copied TIFFs need a calibration; they lose it with ``Index.xml``. The example uses MAP2 objects as neuronal seeds, enhances and skeletonizes SMI312 neurites, measures topology per seed, and propagates seed identities into @@ -91,8 +92,9 @@ It composes the registered CellProfiler-compatible leaves behind one MetaXpress-style step while retaining typed measurements and unified neuron labels. -Use the native ``Microscope.OPERAPHENIX`` handler when the complete plate and -``Index.xml`` are available. Source bindings are appropriate here because the +Use the native Opera Phenix source (``OperaPhenixHandler``, selected by +``dataset_source`` detection) when the complete plate and ``Index.xml`` are +available. Source bindings are appropriate here because the loose files no longer carry the plate-level metadata needed by that handler. Master multi-plate lab-meeting showcase diff --git a/docs/source/guides/example_corpus_map.rst b/docs/source/guides/example_corpus_map.rst index cce8aef510..34423f3d20 100644 --- a/docs/source/guides/example_corpus_map.rst +++ b/docs/source/guides/example_corpus_map.rst @@ -120,7 +120,7 @@ within that benchmark's source and measurement scope. It does not establish biological acceptance for a new assay: review its raw pixels and output overlay at matched coordinates, then preserve that new validation and any failures before promoting an adapted recipe. The portable acceptance boundary and -required parity receipts are described in +required parity evidence are described in :doc:`../architecture/measurement_equivalence_system` and ``benchmark/manifests/README.md``; do not infer a particular case passed from its presence in the recipe catalogue alone. diff --git a/docs/source/guides/fiji_viewer_management.rst b/docs/source/guides/fiji_viewer_management.rst index 2dee62ad6d..522b2e774b 100644 --- a/docs/source/guides/fiji_viewer_management.rst +++ b/docs/source/guides/fiji_viewer_management.rst @@ -38,19 +38,21 @@ the bound process is compatible. Settlement and inspection boundary ---------------------------------- -Fiji participates in the same typed settlement control contract as Napari, but -its current display path is synchronous. A ``SETTLE`` request therefore returns -terminal ``ViewerSettleProgress`` immediately; Fiji does not expose Napari's -queue of deferred Qt layer-route updates. This still provides one common -execution/lifecycle protocol without pretending the two viewer implementations -have the same internal update model. +Fiji answers the same typed ``SETTLE`` control as Napari through the shared +viewer control actions. Its display path batches streamed items through +PolyStore's debounced batch engine, and ``SETTLE`` reports +``ViewerSettleProgress`` for the queued and active batches: running until every +queued item has rendered, complete afterwards, or failed with the retained +native rendering error. Fiji has no queue of deferred Qt layer-route updates; +the two viewers share the settlement contract, not an internal update model. Fiji does **not** currently provide Napari-style live viewer-state or payload -projection. ``STATE`` and ``PAYLOADS`` control requests return explicit errors -until a Fiji state projector exists. Inspection clients therefore cannot query -Fiji layers, axes, labels, payload arrays, or payload summaries through those -controls. This is a declared capability boundary, not a port/readiness failure; -do not retry it as transport recovery or infer viewer state from window titles. +projection. It registers no ``STATE`` or ``PAYLOADS`` control action, so those +requests return an explicit unsupported-control error. Inspection clients +therefore cannot query Fiji layers, axes, labels, payload arrays, or payload +summaries through those controls. This is a declared capability boundary, not a +port/readiness failure; do not retry it as transport recovery or infer viewer +state from window titles. Fiji macro execution, display construction, readiness, clear-state control, and process lifecycle remain separate supported surfaces. Use Napari when automated diff --git a/docs/source/guides/memory_type_integration.rst b/docs/source/guides/memory_type_integration.rst index fc332f29ed..bf57c24909 100644 --- a/docs/source/guides/memory_type_integration.rst +++ b/docs/source/guides/memory_type_integration.rst @@ -55,8 +55,9 @@ projection. OpenHCS launchers consume the combined result; they do not mirror framework package layouts or rely on a parent process importing frameworks in a particular order. -``ProcessingContract`` is independent of memory type. It says whether a -callable is semantically plane-local or stack-dependent. Likewise, +The ``ProcessingContract`` family, such as ``Pure2DContract`` above, is +independent of memory type. Its subclass says whether a callable is +semantically plane-local or stack-dependent. Likewise, ``variable_components`` defines the transported stack axis and ``group_by`` defines post-stack grouping. See :doc:`../architecture/processing_semantics`. diff --git a/docs/source/guides/omero_integration.rst b/docs/source/guides/omero_integration.rst index b332189f03..2f19ba1ecf 100644 --- a/docs/source/guides/omero_integration.rst +++ b/docs/source/guides/omero_integration.rst @@ -12,7 +12,7 @@ PolyStore Owns generic storage backends, virtual paths, source references, and ROI persistence primitives. Its OMERO declarations own text formats and MIME types, table parsing and service readiness, image-plane batching, and the - canonical OMERO well and plane addresses. + OMERO well and plane addresses. OpenHCS Owns microscope/source selection, source bindings, compilation, processing, diff --git a/docs/source/guides/pipeline_compilation_workflow.rst b/docs/source/guides/pipeline_compilation_workflow.rst index 7a1d9e0251..41fdce8248 100644 --- a/docs/source/guides/pipeline_compilation_workflow.rst +++ b/docs/source/guides/pipeline_compilation_workflow.rst @@ -12,7 +12,7 @@ The compiler accepts an initialized ``PipelineOrchestrator`` and an ordered ``list[AbstractStep]``. Together they provide: - a ``PipelineConfig`` and ObjectState scope topology; -- the microscope/source workspace and selected execution axes; +- the dataset-source workspace and selected execution axes; - callable, module, source, and artifact declarations; - per-step processing, materialization, filtering, and streaming configuration. diff --git a/docs/source/guides/testing_guide.rst b/docs/source/guides/testing_guide.rst index 088654dd7c..b4eca62aaa 100644 --- a/docs/source/guides/testing_guide.rst +++ b/docs/source/guides/testing_guide.rst @@ -71,6 +71,19 @@ An uncaptured native Qt/OpenGL preflight proves that context before pytest, so missing system libraries produce their own diagnostics rather than a captured application-startup abort. +The Official30 headless parity lane compares all thirty workflows against the +committed native CellProfiler outputs on Python 3.12 Linux and on Python 3.14 +Linux, Windows, and macOS. A pull request that changes none of the processing, +core, format, CellProfiler interop, submodule, benchmark, integration-test, or +packaging paths skips that lane; +the required parity check accepts the skip only when that scope check proved +the change irrelevant. + +The test jobs install candidate wheels with ``scripts/install_ci_candidate.py`` +and run the checkout's tests through ``scripts/run_installed_tests.py``, which +removes checkout source paths from the import path so OpenHCS is imported from +the installed wheel. + The Python/OS boundary matrix also opens fresh MCP stdio sessions before importing NumPy and SciPy inside a tool. These tests exercise cold native-library loading while the protocol reader is active, plus protocol-channel isolation diff --git a/docs/source/guides/viewer_management.rst b/docs/source/guides/viewer_management.rst index fb485f7151..78a3759d0a 100644 --- a/docs/source/guides/viewer_management.rst +++ b/docs/source/guides/viewer_management.rst @@ -29,9 +29,10 @@ session's exact process group and are stopped when that session is cleaned up. For resource-bound candidate review, retrieve the ``openhcs_viewer_qa`` knowledge document, also bundled in the ``use-openhcs`` skill. Its conditional selective- retirement procedure follows the installed typed capability and distinguishes -scene/payload custody from visibility, durable evidence and measured memory. -That is the operating procedure; this lifecycle overview does not maintain a -second retirement recipe or assume that every installed viewer supports it. +retained scene and payload state from visibility, durable evidence and measured +memory. That is the operating procedure; this lifecycle overview does not +maintain a second retirement recipe or assume that every installed viewer +supports it. Display-axis ownership ---------------------- @@ -176,7 +177,8 @@ These mechanisms answer different questions: ``NapariStreamingConfig`` / ``FijiStreamingConfig`` Show eligible outputs while the selected step executes. Display axes, - batching, transport, viewer persistence, and well selection belong here. + batching, transport, viewer persistence, and partition (well) selection belong + here. ``StepMaterializationConfig`` Save the step's ordinary main-flow result as a persistent checkpoint. Set it diff --git a/docs/source/installation.rst b/docs/source/installation.rst index b5107b8746..67aa1306fa 100644 --- a/docs/source/installation.rst +++ b/docs/source/installation.rst @@ -1,7 +1,7 @@ Installation ============ -The canonical installation instructions live in the Getting Started guide. +The maintained installation instructions live in the Getting Started guide. See :doc:`getting_started/getting_started` for desktop installers, manual Python installation, optional capabilities, launch verification, and updates. diff --git a/docs/source/reference/experimental_config_syntax.rst b/docs/source/reference/experimental_config_syntax.rst index 5bb8076a7d..d551023883 100644 --- a/docs/source/reference/experimental_config_syntax.rst +++ b/docs/source/reference/experimental_config_syntax.rst @@ -176,7 +176,7 @@ pipeline or step configuration. .. code-block:: python - from openhcs.core.config import ExperimentalAnalysisConfig + from openhcs.domains.microscopy.config import ExperimentalAnalysisConfig from openhcs.processing.backends.experimental_analysis import ( ExperimentalAnalysisEngine, ) diff --git a/docs/source/user_guide/analysis_consolidation.rst b/docs/source/user_guide/analysis_consolidation.rst index dc251a0ae4..56137d3e7a 100644 --- a/docs/source/user_guide/analysis_consolidation.rst +++ b/docs/source/user_guide/analysis_consolidation.rst @@ -20,8 +20,9 @@ workbook contract is documented in :doc:`../reference/experimental_config_syntax`. Automatic post-plate consolidation is controlled by -``AnalysisConsolidationConfig`` on the pipeline. It selects file extensions, -well extraction, exclusions, and output names. Consolidation consumes +``AnalysisConsolidationConfig`` in the global pipeline configuration. It +selects the MetaXpress-style layout, file extensions, exclusion patterns, and +output names. Consolidation consumes materialized analysis artifacts; it is not an image-processing ``FunctionStep`` and does not reconstruct compiler state. diff --git a/docs/source/user_guide/code_ui_editing.rst b/docs/source/user_guide/code_ui_editing.rst index b0fd50a011..e8cd77ea8a 100644 --- a/docs/source/user_guide/code_ui_editing.rst +++ b/docs/source/user_guide/code_ui_editing.rst @@ -16,7 +16,7 @@ trip is: Choose the document that owns the edited object. Pipeline Editor projects one complete ``PipelineDocument``; a step editor projects one ``FunctionStep``; a -configuration window projects its typed config; Plate Manager projects the +configuration window projects its typed config; the Datasets window projects the multi-plate aggregate. The available Code controls and MCP document catalog are discovered from those nominal owners rather than a copied button list. @@ -24,7 +24,7 @@ Select a plate before applying a Pipeline Editor document. Without a selected plate, the document remains readable for inspection but reports ``writable=False`` and rejects apply requests without changing UI state. -A Plate Manager document also carries its read-time selection scope. An +A Datasets document also carries its read-time selection scope. An all-plates document represents the complete visible collection, so applying it synchronises that collection. A selected document may replace only the named plates and leaves every unselected plate unchanged. Its payload must contain the @@ -43,8 +43,8 @@ generated code includes the explicit assignment again. Provide ``pipeline_config`` when the pipeline needs non-default source, processing, viewer, or materialisation settings. -Reuse the exact imports shown in generated code. Plate Manager accepts external -processing functions listed in the execution endpoint's catalogue, including +Reuse the exact imports shown in generated code. The Datasets document accepts +external processing functions listed in the execution endpoint's catalogue, including ``skimage.filters.edges.sobel``. Applying the document resolves the selected catalogue wrapper, preserving its processing contract and runtime controls. Wildcard imports and functions absent from that catalogue are rejected. @@ -59,7 +59,7 @@ declared parameters in their supplied order. Decimal fields use compact, locale-aware text, so a value such as ``99.8`` remains readable without changing the field's configured numeric precision. -Applying a complete Pipeline Editor or Plate Manager code document reconciles +Applying a complete Pipeline Editor or Datasets code document reconciles the submitted declaration with the live ObjectState graph. Unchanged and unambiguously edited occurrences retain their history across reordering; added occurrences receive new scopes and omitted occurrences are removed. Ambiguous diff --git a/docs/source/user_guide/custom_function_management.rst b/docs/source/user_guide/custom_function_management.rst index b5bfef6ca3..8259d684cf 100644 --- a/docs/source/user_guide/custom_function_management.rst +++ b/docs/source/user_guide/custom_function_management.rst @@ -18,8 +18,8 @@ shared request. ``CustomFunctionManager`` owns persisted source and coordinates create, load, replace, and delete transactions. It stores source under the platform-specific -OpenHCS data directory, validates code before execution, requires a supported -memory decorator, and rejects name collisions. The process-local +OpenHCS data directory, validates code before execution, requires exactly one +decorated processing function per source, and rejects name collisions. The process-local ``CustomFunctionRuntimeRegistry`` atomically publishes the derived callable metadata, source lifetime, and public module exports, so readers see either the previous complete catalogue or the validated replacement. Reloading persisted diff --git a/docs/source/user_guide/custom_functions.rst b/docs/source/user_guide/custom_functions.rst index ea2299b488..ec6c66daa6 100644 --- a/docs/source/user_guide/custom_functions.rst +++ b/docs/source/user_guide/custom_functions.rst @@ -20,10 +20,11 @@ function in the OpenHCS function catalog. Choose the processing contract deliberately: -- ``PURE_2D`` means each plane is semantically independent; -- ``PURE_3D`` means the result depends on the whole stack; -- ``VOLUMETRIC_TO_SLICE`` consumes a volume and returns plane-local output; -- ``FLEXIBLE`` is for callables that explicitly support either form. +- ``Pure2DContract`` means each plane is semantically independent; +- ``Pure3DContract`` means the result depends on the whole stack; +- ``VolumetricToSliceContract`` consumes a volume and returns plane-local + output; +- ``FlexibleContract`` is for callables that explicitly support either form. Do not attach compiler fields manually or inspect array framework names inside the function. Declare memory, locality, required axes, runtime parameters, and diff --git a/docs/source/user_guide/mcp_clients.rst b/docs/source/user_guide/mcp_clients.rst index 7a968d4b95..4b6197c79b 100644 --- a/docs/source/user_guide/mcp_clients.rst +++ b/docs/source/user_guide/mcp_clients.rst @@ -40,7 +40,7 @@ open PyQt windows automatically. The installed stdio server defaults to the ``desktop`` capability surface. It includes normal UI, selected-plate, viewer, plate-data, knowledge, function-authoring, pipeline-authoring, and independent headless-execution -tools. Headless sessions do not create or mirror desktop Plate Manager state. +tools. Headless sessions do not create or mirror desktop Datasets window state. The desktop profile hides runtime-server, fallback-widget, and expert-only tools. The surface is projected from capability declaration metadata, not a copied tool list. Advanced users can select a narrower or development-oriented @@ -241,7 +241,8 @@ directory for a new run. These inspection routes do not submit a pipeline; use the session tools above to run one. The ``openhcs-benchmark run-measured`` CLI command accepts a normal Python pipeline source file, plate and empty evidence directory, runs it on the -ordinary execution runtime and retains the measured receipt; +ordinary execution runtime and retains the measured receipt. It requires an +explicit ``--wait-timeout-ms``; the session stops a run that exceeds it. ``--execution-plate`` can identify a prepared input while preserving the original plate identity. For large pipelines, ``--observation-scope outcomes`` retains per-axis status, output roots, and diff --git a/docs/source/user_guide/real_time_visualization.rst b/docs/source/user_guide/real_time_visualization.rst index 8a65439154..db5327d83b 100644 --- a/docs/source/user_guide/real_time_visualization.rst +++ b/docs/source/user_guide/real_time_visualization.rst @@ -8,8 +8,8 @@ not artifact materialization, a special pipeline class, or a direct viewer call. 1. Install ``openhcs[gui,napari]``, ``openhcs[gui,fiji]``, or ``openhcs[gui,viz]``. 2. Open a step in the Pipeline Editor. -3. Enable its viewer configuration and choose persistence, batching, well - filters, and display options. +3. Enable its viewer configuration and choose persistence, partition + filters, transport, and display options. 4. Compile. OpenHCS validates source metadata, transport, and viewer requirements independently of artifact materialization. 5. Run the compiled plate and inspect the detached viewer. From f3eb57aa901c3aa11038ba23d0f8556921d460d6 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 03:42:01 -0400 Subject: [PATCH 10/84] B6: pin zmqruntime docs without the removed viewer-mode members; manifest names ResolvedPipelineDefinition Lockstep with OpenHCSDev/ZMQRuntime#29. Co-Authored-By: Claude Opus 5.5 --- external/zmqruntime | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/external/zmqruntime b/external/zmqruntime index a1db4a5439..f6f81ef975 160000 --- a/external/zmqruntime +++ b/external/zmqruntime @@ -1 +1 @@ -Subproject commit a1db4a5439caceaddbc5d89fa43300dec09aad35 +Subproject commit f6f81ef975627afdd547cfb03b1a854dc3772f8d From 47724ccd4ed7c7b9b2a8c2822bb36e7aa592cef9 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 03:25:57 -0400 Subject: [PATCH 11/84] B6: granularity native accepts uint32 queues exported as 'L' on Windows NumPy exports uint32 buffers with format 'L' on LLP64 Windows and 'I' on LP64 platforms; reconstruct_f32 compared the format string exactly and rejected the queues on Windows. Compare unsigned codes by width. Co-Authored-By: Claude Opus 5.5 --- .../cellprofiler/_granularity_native.cpp | 20 ++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/openhcs/processing/backends/cellprofiler/_granularity_native.cpp b/openhcs/processing/backends/cellprofiler/_granularity_native.cpp index a49ac9118d..b875d7d539 100644 --- a/openhcs/processing/backends/cellprofiler/_granularity_native.cpp +++ b/openhcs/processing/backends/cellprofiler/_granularity_native.cpp @@ -110,9 +110,27 @@ class OwnedArrayBuffer { return acquired_; } const Py_buffer& view() const { return view_; } + // One unsigned C width has several buffer codes: NumPy exports uint32 as + // 'I' on LP64 platforms and as 'L' on LLP64 Windows. Compare by width. + static Py_ssize_t unsigned_width(const char* format) { + if (format[0] == '\0' || format[1] != '\0') return 0; + switch (format[0]) { + case 'B': return sizeof(unsigned char); + case 'H': return sizeof(unsigned short); + case 'I': return sizeof(unsigned int); + case 'L': return sizeof(unsigned long); + case 'Q': return sizeof(unsigned long long); + default: return 0; + } + } + bool has_format(const char* format) const { + if (std::strcmp(view_.format, format) == 0) return true; + const Py_ssize_t width = unsigned_width(format); + return width != 0 && unsigned_width(view_.format) == width && view_.itemsize == width; + } bool matches(const char* name, const char* format, int ndim, bool writable) const { if (view_.ndim != ndim || view_.format == nullptr || format == nullptr || - std::strcmp(view_.format, format) != 0 || + !has_format(format) || !PyBuffer_IsContiguous(&view_, 'C') || (writable && view_.readonly)) { PyErr_Format(PyExc_ValueError, "%s must be a C-contiguous %d-D array with format %s%s", From d5172f39857549fbb73a179b018d095690369030 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 03:25:57 -0400 Subject: [PATCH 12/84] B6: persist custom-function sources as the exact bytes their revision hashes _write_temporary_source wrote in text mode, so on Windows newline translation stored CRLF bytes while the revision digest was computed from the LF source; require_source then reported every unchanged source as changed. Write and read the source bytes without newline translation. Co-Authored-By: Claude Opus 5.5 --- openhcs/processing/custom_functions/manager.py | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/openhcs/processing/custom_functions/manager.py b/openhcs/processing/custom_functions/manager.py index e4934e2a11..eec75f2639 100644 --- a/openhcs/processing/custom_functions/manager.py +++ b/openhcs/processing/custom_functions/manager.py @@ -366,7 +366,7 @@ def list_custom_functions(self) -> list[CustomFunctionInfo]: for py_file in sorted(self.storage_dir.glob("*.py")): try: - metadata = self._prepare_source(py_file.read_text(encoding="utf-8")) + metadata = self._prepare_source(py_file.read_bytes().decode("utf-8")) contract = CallableContract.from_callable(metadata.func) functions.append( CustomFunctionInfo( @@ -400,7 +400,7 @@ def get_function_code(self, func_name: str) -> str: if not file_path.exists(): raise ValueError(f"Custom function '{func_name}' not found") - return file_path.read_text(encoding="utf-8") + return file_path.read_bytes().decode("utf-8") def update_custom_function(self, old_name: str, new_code: str) -> str: """ @@ -661,8 +661,10 @@ def _write_temporary_source(self, code: str) -> Path: dir=self.storage_dir, ) try: - with os.fdopen(file_descriptor, "w", encoding="utf-8") as stream: - stream.write(code) + # Persist the exact bytes the revision digest was computed from; + # text-mode newline translation would change them on Windows. + with os.fdopen(file_descriptor, "wb") as stream: + stream.write(code.encode("utf-8")) stream.flush() os.fsync(stream.fileno()) except BaseException: From f59da9b8fd11e906383e0a6af57bc99192b00f62 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 03:31:40 -0400 Subject: [PATCH 13/84] B6: GUI tests drive the session-backed plate manager Stale after U1/U5: the end-to-end workflow added plates through the deleted add_plate_callback and string button keys; it now invokes AddDatasets on the widget's session, keys buttons by operation_id and waits for the session to report initialization. The image-browser inventory stub now subclasses MetadataHandler so it inherits the handler defaults (source_workspace_root). The managed-window stubs follow the widget protocol (_get_current_orchestrator, session-taking PlateManagerWidget; the deleted PlateManagerWindow._setup_connections is no longer patched) and also assert the session is passed through. Co-Authored-By: Claude Opus 5.5 --- .../test_end_to_end_workflow_foundation.py | 33 ++++++++++++++----- .../test_image_browser_filter_controller.py | 19 ++++++++--- tests/pyqt_gui/test_managed_windows.py | 8 +++-- 3 files changed, 44 insertions(+), 16 deletions(-) diff --git a/tests/pyqt_gui/integration/test_end_to_end_workflow_foundation.py b/tests/pyqt_gui/integration/test_end_to_end_workflow_foundation.py index 6007bc0d21..a8ed448dc0 100644 --- a/tests/pyqt_gui/integration/test_end_to_end_workflow_foundation.py +++ b/tests/pyqt_gui/integration/test_end_to_end_workflow_foundation.py @@ -37,6 +37,11 @@ ) from pyqt_reactive.forms.parameter_form_manager import ParameterFormManager +from openhcs.authoring.session.operations.datasets import ( + AddDatasets, + EditDatasetConfig, + InitializeDatasets, +) from openhcs.core.config import ( GlobalPipelineConfig, LazyPathPlanningConfig, @@ -630,8 +635,12 @@ def _access_plate_manager(context: WorkflowContext) -> WorkflowContext: def _add_and_select_plate(context: WorkflowContext) -> WorkflowContext: - """Add synthetic plate and select it in plate manager.""" - context.plate_manager_widget.add_plate_callback([context.synthetic_plate_dir]) + """Add synthetic plate through the session and select it in plate manager.""" + result = context.plate_manager_widget.session.invoke( + AddDatasets, + AddDatasets.request(roots=(str(context.synthetic_plate_dir),)), + ) + assert result.accepted, result.errors _wait_for_gui(TIMING.ACTION_DELAY) item_list = context.plate_manager_widget.item_list @@ -647,14 +656,21 @@ def _add_and_select_plate(context: WorkflowContext) -> WorkflowContext: def _initialize_plate(context: WorkflowContext) -> WorkflowContext: """Initialize plate using Init button.""" - init_button = context.plate_manager_widget.buttons["init_plate"] + plate_manager = context.plate_manager_widget + init_button = plate_manager.buttons[InitializeDatasets.operation_id] if not init_button.isEnabled(): raise AssertionError( "Init button is disabled - plate may not be properly added" ) init_button.click() - _wait_for_gui(TIMING.SAVE_DELAY) + scope_id = plate_manager.selected_plate_path + session = plate_manager.session + deadline = time.monotonic() + 60.0 + while not session.is_initialized(scope_id) or scope_id in session.init_pending: + if time.monotonic() >= deadline: + raise AssertionError(f"Dataset {scope_id!r} did not finish initializing") + _wait_for_gui(0.05) return context @@ -665,9 +681,8 @@ def _apply_orchestrator_config(context: WorkflowContext) -> WorkflowContext: "Test scenario must be provided for parameterized orchestrator configuration" ) - orchestrator = context.plate_manager_widget.get_selected_orchestrator() - if orchestrator is None: - raise AssertionError("The selected plate has no initialized orchestrator") + plate_manager = context.plate_manager_widget + orchestrator = plate_manager.session.orchestrator(plate_manager.selected_plate_path) orchestrator.apply_pipeline_config(context.test_scenario.pipeline_config) _wait_for_gui(TIMING.ACTION_DELAY) @@ -709,7 +724,7 @@ def _wait_for_form_build( ) def _open_config_window(context: WorkflowContext) -> WorkflowContext: """Open plate-specific configuration window.""" - edit_button = context.plate_manager_widget.buttons["edit_config"] + edit_button = context.plate_manager_widget.buttons[EditDatasetConfig.operation_id] if not edit_button.isEnabled(): raise AssertionError( "Edit button is disabled - plate may not be properly initialized" @@ -891,7 +906,7 @@ def _reopen_config_window(context: WorkflowContext) -> WorkflowContext: context = _close_config_window(context) # Validate edit button state - edit_button = context.plate_manager_widget.buttons["edit_config"] + edit_button = context.plate_manager_widget.buttons[EditDatasetConfig.operation_id] if not edit_button.isEnabled(): raise AssertionError( "LAZY CONFIG BUG: Edit button disabled after closing config window. " diff --git a/tests/pyqt_gui/test_image_browser_filter_controller.py b/tests/pyqt_gui/test_image_browser_filter_controller.py index fb5d82e26b..0ba8845dab 100644 --- a/tests/pyqt_gui/test_image_browser_filter_controller.py +++ b/tests/pyqt_gui/test_image_browser_filter_controller.py @@ -20,6 +20,7 @@ from openhcs.core.dataset_sources.interfaces import ( AnalysisResultDirectory, FilenameParserCapability, + MetadataHandler, ) from openhcs.pyqt_gui.widgets.image_browser import ( ImageBrowserFilterController, @@ -126,7 +127,9 @@ def parse_filename(self, filename: str): ) -class _InventoryMetadataHandler: +class _InventoryMetadataHandler(MetadataHandler): + """Inherits the handler defaults (workspace root, no virtual workspace).""" + def __init__(self, handler_result_dir: Path) -> None: self._handler_result_dir = handler_result_dir @@ -134,11 +137,17 @@ def get_image_files(self, _plate_path: Path, *, all_subdirs: bool): assert all_subdirs is True return ("images/A01_w1.tif",) - def source_workspace_metadata_document(self, _plate_path: Path): - return None + def find_metadata_file(self, _plate_path: Path) -> Path: + raise AssertionError("Inventory does not read the metadata file.") - def source_dataset(self, _plate_path: Path): - return None + def get_grid_dimensions(self, _plate_path: Path): + raise AssertionError("Inventory does not read grid dimensions.") + + def get_pixel_size(self, _plate_path: Path): + raise AssertionError("Inventory does not read pixel size.") + + def component_value_set(self, _plate_path: Path): + raise AssertionError("Inventory does not read the component value set.") def analysis_result_directories(self, _plate_path: Path): return ( diff --git a/tests/pyqt_gui/test_managed_windows.py b/tests/pyqt_gui/test_managed_windows.py index b82f64d0a9..e167e980d3 100644 --- a/tests/pyqt_gui/test_managed_windows.py +++ b/tests/pyqt_gui/test_managed_windows.py @@ -33,7 +33,7 @@ def __init__(self, orchestrator: _OrchestratorStub) -> None: self._orchestrator = orchestrator self.plate_selected = _Signal() - def get_selected_orchestrator(self) -> _OrchestratorStub: + def _get_current_orchestrator(self) -> _OrchestratorStub: return self._orchestrator def select(self, orchestrator: _OrchestratorStub) -> None: @@ -141,12 +141,14 @@ class PlateManagerStub(QWidget): def __init__( self, service_adapter, + session, color_scheme, *, gui_config, ) -> None: super().__init__() del service_adapter, color_scheme + captured["session"] = session captured["gui_config"] = gui_config def cleanup(self) -> None: @@ -159,11 +161,13 @@ def cleanup(self) -> None: "PlateManagerWidget", PlateManagerStub, ) - monkeypatch.setattr(PlateManagerWindow, "_setup_connections", lambda self: None) + session = object() main_window = QWidget() + main_window.session = session window = PlateManagerWindow(main_window, ServiceAdapterStub()) qtbot.addWidget(main_window) qtbot.addWidget(window) assert captured["gui_config"] is resolved_ui_config + assert captured["session"] is session From 03ca14cc90f644e426b461f8793b58eff4389221 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 03:39:20 -0400 Subject: [PATCH 14/84] B6: GUI smoke reads the catalog only after the GUI's endpoint is up The function catalog attaches only to an existing endpoint, but the smoke requested it as soon as the window painted, while the GUI was still starting its endpoint in _prepare_execution_services; the attach failed (CI) or blocked on the startup lock past its 5 s control deadline (locally). The main window now keeps the Future of its execution-services preparation and the smoke waits on it before reading the catalog. Co-Authored-By: Claude Opus 5.5 --- openhcs/pyqt_gui/main.py | 7 +++++- scripts/smoke_installed_gui.py | 33 +++++++++++++++++++------- tests/unit/test_smoke_installed_gui.py | 29 ++++++++++++++++++++++ 3 files changed, 60 insertions(+), 9 deletions(-) diff --git a/openhcs/pyqt_gui/main.py b/openhcs/pyqt_gui/main.py index 1c69fa6a76..24b602a83a 100644 --- a/openhcs/pyqt_gui/main.py +++ b/openhcs/pyqt_gui/main.py @@ -5,6 +5,7 @@ """ import logging +from concurrent.futures import Future from collections.abc import Callable from functools import singledispatchmethod from pathlib import Path @@ -219,6 +220,8 @@ def __init__( # Core configuration self.runtime_context = runtime_context self.function_catalog_service = function_catalog_service + # Completes once the shared endpoint is attached and its catalog prewarmed. + self.execution_services_prepared: Future[None] | None = None # Create shared components self.storage_registry = storage_registry @@ -366,7 +369,9 @@ def deferred_initialization(self): def start_background_services(self) -> None: """Start non-visual services after the initialized window has painted.""" - self.service_adapter.execute_async_operation(self._prepare_execution_services) + self.execution_services_prepared = self.service_adapter.execute_async_operation( + self._prepare_execution_services + ) self._start_ui_bridge_if_enabled() self._check_for_updates_on_startup() diff --git a/scripts/smoke_installed_gui.py b/scripts/smoke_installed_gui.py index e2010984c5..e9d1837b5d 100644 --- a/scripts/smoke_installed_gui.py +++ b/scripts/smoke_installed_gui.py @@ -34,6 +34,7 @@ JsonValue = str | int | bool | None | list["JsonValue"] | dict[str, "JsonValue"] AgentDtoT = TypeVar("AgentDtoT") +ResultT = TypeVar("ResultT") @dataclass(frozen=True, slots=True) @@ -496,20 +497,31 @@ def _capture_installed_gui_snapshot( ) -def verify_installed_function_catalog( - service: ZMQFunctionCatalogService, - *, - deadline: OperationDeadline, -) -> InstalledFunctionCatalogSmokeResult: - """Require the GUI's background endpoint catalogue and one usable reference.""" +def _result_within(future: Future[ResultT], deadline: OperationDeadline) -> ResultT: + """Wait inside the smoke deadline; an owner's own timeout stays its own.""" - future = service.prepare() try: - page = future.result(timeout=deadline.remaining_seconds()) + return future.result(timeout=deadline.remaining_seconds()) except TimeoutError as error: if not future.cancelled() and future.done() and future.exception() is error: raise raise deadline.timeout_error() from error + + +def verify_installed_function_catalog( + service: ZMQFunctionCatalogService, + *, + execution_services_prepared: Future[None], + deadline: OperationDeadline, +) -> InstalledFunctionCatalogSmokeResult: + """Require the GUI's background endpoint catalogue and one usable reference. + + The catalog attaches only to an existing endpoint, so first wait for the + GUI's own endpoint start and catalog prewarm to finish. + """ + + _result_within(execution_services_prepared, deadline) + page = _result_within(service.prepare(), deadline) if not page.revision.strip(): raise AssertionError("Installed function catalog omitted its revision.") if page.total <= 0 or not page.items: @@ -541,6 +553,7 @@ def run_installed_live_mcp_smoke( evidence_directory: Path, forbidden_root: Path, function_catalog_service: ZMQFunctionCatalogService, + execution_services_prepared: Future[None], execution_config: OpenHCSZMQConfig, deadline: OperationDeadline, journal: InstalledGuiSmokeEvidenceJournal, @@ -575,6 +588,7 @@ def run_installed_live_mcp_smoke( journal.record(InstalledGuiSmokePhase.WAITING_FOR_FUNCTION_CATALOG) function_catalog = verify_installed_function_catalog( function_catalog_service, + execution_services_prepared=execution_services_prepared, deadline=deadline, ) journal.record( @@ -868,6 +882,9 @@ def run_live_mcp() -> None: evidence_directory=evidence_directory, forbidden_root=forbidden_root, function_catalog_service=application.function_catalog_service, + execution_services_prepared=( + main_window.execution_services_prepared + ), execution_config=ui_config.zmq, deadline=deadline, journal=journal, diff --git a/tests/unit/test_smoke_installed_gui.py b/tests/unit/test_smoke_installed_gui.py index 5a378c4850..359a1dc2e0 100644 --- a/tests/unit/test_smoke_installed_gui.py +++ b/tests/unit/test_smoke_installed_gui.py @@ -164,12 +164,39 @@ def test_gui_smoke_allows_wheel_venv_inside_checkout(tmp_path: Path) -> None: ) +def _prepared_execution_services() -> Future[None]: + future: Future[None] = Future() + future.set_result(None) + return future + + +def test_gui_smoke_waits_for_gui_endpoint_before_reading_catalog() -> None: + class UnreachableFunctionCatalogService: + @staticmethod + def prepare() -> Future[FunctionCatalogPage]: + raise AssertionError("The catalog attaches only after the GUI endpoint starts.") + + with pytest.raises( + OperationTimeoutError, + match="Timed out waiting for test installed function catalog", + ): + verify_installed_function_catalog( + UnreachableFunctionCatalogService(), + execution_services_prepared=Future(), + deadline=OperationDeadline.after_milliseconds( + 1, + operation="test installed function catalog", + ), + ) + + def test_gui_smoke_requires_prepared_catalog_and_resolvable_reference() -> None: entry = _catalog_entry() service = _FunctionCatalogService(_catalog_page(entry)) result = verify_installed_function_catalog( service, + execution_services_prepared=_prepared_execution_services(), deadline=OperationDeadline.after_milliseconds( 2_000, operation="test installed function catalog", @@ -195,6 +222,7 @@ def prepare() -> Future[FunctionCatalogPage]: ): verify_installed_function_catalog( PendingFunctionCatalogService(), + execution_services_prepared=_prepared_execution_services(), deadline=OperationDeadline.after_milliseconds( 1, operation="test installed function catalog", @@ -215,6 +243,7 @@ def prepare() -> Future[FunctionCatalogPage]: with pytest.raises(TimeoutError, match="catalog owner timed out") as caught: verify_installed_function_catalog( FailedFunctionCatalogService(), + execution_services_prepared=_prepared_execution_services(), deadline=OperationDeadline.after_milliseconds( 2_000, operation="test installed function catalog", From c3ab99415e8a37bd3e20fb354ea74a5af4c9894d Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 03:39:20 -0400 Subject: [PATCH 15/84] B6: requestless UI-bridge capabilities pass the connection by keyword UiBridgeOperationExecution unpacked (*request, connection) and called invoke(operation, *request, connection) positionally; with no request the connection landed in the request slot, so ui status/list_windows over MCP failed with "does not accept a request". Co-Authored-By: Claude Opus 5.5 --- openhcs/agent/capabilities.py | 4 +++- tests/unit/agent/test_capabilities.py | 22 ++++++++++++++++++++++ 2 files changed, 25 insertions(+), 1 deletion(-) diff --git a/openhcs/agent/capabilities.py b/openhcs/agent/capabilities.py index 27a03c7018..18482e2405 100644 --- a/openhcs/agent/capabilities.py +++ b/openhcs/agent/capabilities.py @@ -1659,7 +1659,9 @@ class UiBridgeOperationExecution(AgentCapabilityInvocation): def execute(self, context: object, *arguments: object) -> object: *request, connection = arguments - return context.ui_bridge_service.invoke(self.operation, *request, connection) + return context.ui_bridge_service.invoke( + self.operation, *request, connection=connection + ) @dataclass(frozen=True, slots=True) diff --git a/tests/unit/agent/test_capabilities.py b/tests/unit/agent/test_capabilities.py index eacfe31e20..b6db329076 100644 --- a/tests/unit/agent/test_capabilities.py +++ b/tests/unit/agent/test_capabilities.py @@ -528,3 +528,25 @@ def test_connection_shape_is_declared_by_the_invocation_family(): ): assert issubclass(capability, RuntimeServerCapability) assert isinstance(capability.invocation, AgentFromFieldsServiceInvocation) + + +def test_requestless_ui_bridge_capability_passes_connection_as_connection(): + """A requestless operation's only argument is the connection, never a request.""" + + from types import SimpleNamespace + + from openhcs.agent.services.ui_bridge_service import UiBridgeService + + seen = [] + + class RecordingUiBridgeService(UiBridgeService): + def invoke(self, operation, request=None, connection=None): + seen.append((operation, request, connection)) + return "served" + + connection = object() + invocation = agent_capabilities.ui_bridge_status.invocation + context = SimpleNamespace(ui_bridge_service=RecordingUiBridgeService(gateway=object())) + + assert invocation.execute(context, connection) == "served" + assert seen == [(invocation.operation, None, connection)] From 87e4df3ffdd6ea73b5646dc63363899c0a8687d6 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 03:41:38 -0400 Subject: [PATCH 16/84] B6: MCP smoke names synthetic partitions with partition_values G2 renamed SyntheticPlateGenerationRequest.wells to partition_values; the installed MCP smoke still sent wells, so the tool rejected the arguments (extra_forbidden) and returned FastMCP's plain-text error, which the smoke could not decode as JSON. Co-Authored-By: Claude Opus 5.5 --- scripts/smoke_installed_mcp.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/smoke_installed_mcp.py b/scripts/smoke_installed_mcp.py index 5f3bd995f0..74ef95ebe5 100644 --- a/scripts/smoke_installed_mcp.py +++ b/scripts/smoke_installed_mcp.py @@ -280,7 +280,7 @@ async def _run_measured_execution_protocol_smoke( "wavelengths": 1, "z_stack_levels": 1, "num_cells": 2, - "wells": ["A01"], + "partition_values": ["A01"], "format": "ImageXpress", "random_seed": 7, }, From 333d3e0ab1e55e0406881c9eebdb85ab651b62d1 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 03:42:41 -0400 Subject: [PATCH 17/84] B6: pin zmqruntime 0.6.1 (startup owner read through the held lock) On Windows portalocker locks are mandatory, so startup_owner's second handle on the lock file raised PermissionError in every Windows ZMQ test. Lockstep with OpenHCSDev/ZMQRuntime#29; pypi-dependency-readiness needs v0.6.1 tagged. Co-Authored-By: Claude Opus 5.5 --- external/zmqruntime | 2 +- pyproject.toml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/external/zmqruntime b/external/zmqruntime index f6f81ef975..fd2d0b8c5a 160000 --- a/external/zmqruntime +++ b/external/zmqruntime @@ -1 +1 @@ -Subproject commit f6f81ef975627afdd547cfb03b1a854dc3772f8d +Subproject commit fd2d0b8c5af07182955ba4a258f0c8709e54ae01 diff --git a/pyproject.toml b/pyproject.toml index b71c07fc48..cc824c4856 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -99,7 +99,7 @@ dependencies = [ "psutil>=5.9.0", # External dependencies (PyPI versions for production builds) - "zmqruntime>=0.6,<0.7", + "zmqruntime>=0.6.1,<0.7", "pycodify>=0.1.4,<0.2", "objectstate>=1.3.1,<2", "python-introspect>=0.2.5,<0.3", From c1013dd51f51d9d3ce1f981ed25afb564229e81d Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 03:50:45 -0400 Subject: [PATCH 18/84] B6: catalog excludes modules whose distribution the package markers exclude here Intel macOS installs omit openhcs-basicpy by its pyproject marker, yet the execution server's catalog preparation imported basic_processor_jax and died with ModuleNotFoundError: basicpy (regressed in 45437a8a9). A catalog module now names the distribution it needs in a literal REQUIRED_DISTRIBUTIONS tuple. OpenHCSRegistry.get_modules_to_scan reads it statically and skips the module only when no OpenHCS Requires-Dist entry for that distribution applies to this environment (markers, extras) and it is not installed. A distribution required here still fails loudly on import, and a declaration naming a distribution OpenHCS never requires is rejected. The excluded set joins the catalog cache identity. Co-Authored-By: Claude Opus 5.5 --- .../backends/enhance/basic_processor_jax.py | 4 + .../backends/lib_registry/openhcs_registry.py | 138 ++++++++++++++++-- tests/integration/test_basicpy_real_fit.py | 7 + tests/unit/test_library_registry_discovery.py | 100 +++++++++++++ 4 files changed, 239 insertions(+), 10 deletions(-) diff --git a/openhcs/processing/backends/enhance/basic_processor_jax.py b/openhcs/processing/backends/enhance/basic_processor_jax.py index 6a747069ba..148d5fd86e 100644 --- a/openhcs/processing/backends/enhance/basic_processor_jax.py +++ b/openhcs/processing/backends/enhance/basic_processor_jax.py @@ -32,6 +32,10 @@ MaterializedFilenameIdentity, ) +# The OpenHCS requirement markers decide where this distribution is installed; +# the function catalog leaves this module out wherever they exclude it. +REQUIRED_DISTRIBUTIONS = ("openhcs-basicpy",) + def _fitted_field_output(name: str) -> ArtifactSpec: """Project group lineage; runtime field owns aggregate contributor context.""" diff --git a/openhcs/processing/backends/lib_registry/openhcs_registry.py b/openhcs/processing/backends/lib_registry/openhcs_registry.py index e89c959654..86803ca427 100644 --- a/openhcs/processing/backends/lib_registry/openhcs_registry.py +++ b/openhcs/processing/backends/lib_registry/openhcs_registry.py @@ -9,6 +9,7 @@ import ast import importlib import importlib.abc +import importlib.metadata import inspect import logging from abc import ABC, abstractmethod @@ -22,6 +23,8 @@ import numpy as np from metaclass_registry import LazyDiscoveryDict, import_module_preserving_root_logging +from packaging.requirements import Requirement +from packaging.utils import canonicalize_name from openhcs.constants import VALID_MEMORY_TYPES, MemoryType from openhcs.core.callable_contract import ( @@ -293,6 +296,18 @@ def _catalog_memory_types(func: Callable) -> frozenset[MemoryType] | None: return declared +def _module_source(module_name: str) -> tuple[str, str] | None: + """Return loader-decoded module source and origin, or None without source.""" + + spec = importlib.util.find_spec(module_name) + if spec is None or not isinstance(spec.loader, importlib.abc.InspectLoader): + return None + source = spec.loader.get_source(module_name) + if source is None: + return None + return source, spec.origin or module_name + + def _module_declares_allowed_memory_type( module_name: str, allowed_memory_types: frozenset[str], @@ -300,15 +315,95 @@ def _module_declares_allowed_memory_type( spec = importlib.util.find_spec(module_name) if spec is not None and spec.submodule_search_locations is not None: return True - if spec is None or not isinstance(spec.loader, importlib.abc.InspectLoader): + module_source = _module_source(module_name) + if module_source is None: return False - source = spec.loader.get_source(module_name) - if source is None: + return _source_declares_allowed_memory_type(*module_source, allowed_memory_types) + + +_REQUIRED_DISTRIBUTIONS_NAME = "REQUIRED_DISTRIBUTIONS" + + +def _openhcs_requirements() -> tuple[Requirement, ...]: + """Return the installed OpenHCS distribution's declared requirements.""" + + return tuple( + Requirement(value) for value in importlib.metadata.requires("openhcs") or () + ) + + +def _distribution_excluded_by_package_metadata(name: str) -> bool: + """Return whether OpenHCS's own requirement markers leave ``name`` out here. + + A distribution is legitimately absent only when no OpenHCS requirement for + it applies to this environment (markers, extras) and it is not installed. + A distribution OpenHCS requires here is never excluded, so its absence + surfaces as the module's import failure. + """ + + canonical = canonicalize_name(name) + requirements = tuple( + requirement + for requirement in _openhcs_requirements() + if canonicalize_name(requirement.name) == canonical + ) + if not requirements: + raise ValueError( + f"Distribution {name!r} is not declared by the OpenHCS package " + "requirements; a module cannot depend on an undeclared distribution." + ) + if any( + requirement.marker is None or requirement.marker.evaluate({"extra": ""}) + for requirement in requirements + ): return False - return _source_declares_allowed_memory_type( - source, - spec.origin or module_name, - allowed_memory_types, + try: + importlib.metadata.distribution(name) + except importlib.metadata.PackageNotFoundError: + return True + return False + + +@lru_cache(maxsize=1024) +def _source_required_distributions(source: str, origin: str) -> tuple[str, ...]: + """Read a module's literal ``REQUIRED_DISTRIBUTIONS`` declaration.""" + + for statement in ast.parse(source, filename=origin).body: + if isinstance(statement, ast.Assign): + targets = statement.targets + elif isinstance(statement, ast.AnnAssign): + targets = (statement.target,) + else: + continue + if not any( + isinstance(target, ast.Name) and target.id == _REQUIRED_DISTRIBUTIONS_NAME + for target in targets + ): + continue + value = ast.literal_eval(statement.value) + if not isinstance(value, tuple) or not all( + isinstance(item, str) and item for item in value + ): + raise ValueError( + f"{origin}: {_REQUIRED_DISTRIBUTIONS_NAME} must be a literal tuple " + "of distribution names." + ) + return value + return () + + +def _module_distributions_excluded_by_package_metadata( + module_name: str, +) -> tuple[str, ...]: + """Return the module's declared distributions this installation excludes.""" + + module_source = _module_source(module_name) + if module_source is None: + return () + return tuple( + name + for name in _source_required_distributions(*module_source) + if _distribution_excluded_by_package_metadata(name) ) @@ -489,8 +584,11 @@ def _get_openhcs_modules(self) -> List[str]: def get_modules_to_scan(self) -> List[Tuple[str, Any]]: """Import every admitted native declaration before catalog publication. - Memory-type eligibility owns exclusions. An eligible OpenHCS module - that cannot import is a broken installation, not an optional function + Memory-type eligibility and the OpenHCS package requirement markers + own exclusions: a module whose ``REQUIRED_DISTRIBUTIONS`` names a + distribution those markers leave out of this environment is not part + of this installation's catalog. Any other eligible OpenHCS module that + cannot import is a broken installation, not an optional function inventory; its failure must prevent apparent catalog readiness. """ self._ensure_module_inventory() @@ -507,6 +605,17 @@ def get_modules_to_scan(self) -> List[Tuple[str, Any]]: module_name, ) continue + excluded = _module_distributions_excluded_by_package_metadata( + module_name + ) + if excluded: + logger.info( + "Skipping OpenHCS module %s - package requirements exclude %s " + "from this environment", + module_name, + ", ".join(excluded), + ) + continue module = import_module_preserving_root_logging(module_name) modules.append((module_name, module)) return modules @@ -540,7 +649,16 @@ def cache_source_mtimes(self) -> Dict[str, float]: def cache_discovery_context(self) -> dict[str, Any]: """Project the declaration-import policy into catalogue cache identity.""" - return {"allowed_memory_types": sorted(_allowed_openhcs_memory_types())} + return { + "allowed_memory_types": sorted(_allowed_openhcs_memory_types()), + "distributions_excluded_by_package_metadata": sorted( + { + canonicalize_name(requirement.name) + for requirement in _openhcs_requirements() + if _distribution_excluded_by_package_metadata(requirement.name) + } + ), + } def is_library_available(self) -> bool: """OpenHCS is always available.""" diff --git a/tests/integration/test_basicpy_real_fit.py b/tests/integration/test_basicpy_real_fit.py index 8c271094f9..4025252c49 100644 --- a/tests/integration/test_basicpy_real_fit.py +++ b/tests/integration/test_basicpy_real_fit.py @@ -17,6 +17,7 @@ def test_real_algorithm_uses_numpy_transport_and_is_cpu_catalog_eligible(monkeyp from openhcs.processing.backends.lib_registry.openhcs_registry import ( _catalog_memory_types, _module_declares_allowed_memory_type, + _module_distributions_excluded_by_package_metadata, ) monkeypatch.setenv("OPENHCS_CPU_ONLY", "true") @@ -28,6 +29,12 @@ def test_real_algorithm_uses_numpy_transport_and_is_cpu_catalog_eligible(monkeyp assert _module_declares_allowed_memory_type( basic_flatfield_correction_jax.__module__, frozenset({MemoryType.NUMPY.value}) ) + assert ( + _module_distributions_excluded_by_package_metadata( + basic_flatfield_correction_jax.__module__ + ) + == () + ) def run_fit(observations): diff --git a/tests/unit/test_library_registry_discovery.py b/tests/unit/test_library_registry_discovery.py index f364215d7f..a0f3403be1 100644 --- a/tests/unit/test_library_registry_discovery.py +++ b/tests/unit/test_library_registry_discovery.py @@ -50,6 +50,106 @@ def test_native_catalog_import_failures_follow_declared_memory_admission( assert registry.get_modules_to_scan() == [] +@pytest.mark.parametrize( + "requirement, outcome", + [ + # The package's own marker leaves the distribution out of this environment. + ("absent-catalog-probe>=1; platform_system == 'NoSuchSystem'", "excluded"), + # Declared only for an extra this installation did not select. + ("absent-catalog-probe>=1; extra == 'probe'", "excluded"), + # Required here, yet absent: a broken installation fails loudly. + ("absent-catalog-probe>=1", "broken"), + # A module cannot claim a distribution the package never declares. + ("unrelated-catalog-probe>=1", "undeclared"), + ], +) +def test_native_catalog_excludes_modules_whose_distribution_markers_exclude_here( + tmp_path, monkeypatch, requirement, outcome +) -> None: + from packaging.requirements import Requirement + + from openhcs.processing.backends.lib_registry import openhcs_registry + from openhcs.processing.backends.lib_registry.openhcs_registry import ( + OpenHCSRegistry, + ) + + package_name = "native_catalog_distribution_probe" + package = tmp_path / package_name + package.mkdir() + (package / "__init__.py").write_text("") + (package / "processing.py").write_text( + "from openhcs.core.memory.decorators import numpy\n" + "import absent_catalog_probe\n" + "REQUIRED_DISTRIBUTIONS = ('absent-catalog-probe',)\n" + "@numpy\n" + "def process(image):\n return image\n" + ) + monkeypatch.syspath_prepend(str(tmp_path)) + monkeypatch.setenv("OPENHCS_CPU_ONLY", "1") + monkeypatch.setattr( + openhcs_registry, + "_openhcs_requirements", + lambda: (Requirement(requirement),), + ) + registry = OpenHCSRegistry() + registry.MODULES_TO_SCAN = [f"{package_name}.processing"] + + if outcome == "excluded": + assert registry.get_modules_to_scan() == [] + assert registry.cache_discovery_context()[ + "distributions_excluded_by_package_metadata" + ] == ["absent-catalog-probe"] + elif outcome == "broken": + with pytest.raises(ModuleNotFoundError, match="absent_catalog_probe"): + registry.get_modules_to_scan() + else: + with pytest.raises(ValueError, match="absent-catalog-probe"): + registry.get_modules_to_scan() + + +@pytest.mark.parametrize( + "platform_system, platform_machine, excluded", + [("Darwin", "x86_64", True), ("Darwin", "arm64", False), ("Linux", "x86_64", False)], +) +def test_basicpy_catalog_module_follows_the_package_basicpy_marker( + monkeypatch, platform_system, platform_machine, excluded +) -> None: + """Intel macOS installs omit openhcs-basicpy; its catalog module follows.""" + + import importlib.metadata + + import packaging.markers + + from openhcs.processing.backends.lib_registry.openhcs_registry import ( + _module_distributions_excluded_by_package_metadata, + ) + + environment = packaging.markers.default_environment() + monkeypatch.setattr( + packaging.markers, + "default_environment", + lambda: { + **environment, + "platform_system": platform_system, + "platform_machine": platform_machine, + }, + ) + installed_distribution = importlib.metadata.distribution + + def distribution_without_basicpy(name: str): + if name == "openhcs-basicpy": + raise importlib.metadata.PackageNotFoundError(name) + return installed_distribution(name) + + monkeypatch.setattr( + importlib.metadata, "distribution", distribution_without_basicpy + ) + + assert _module_distributions_excluded_by_package_metadata( + "openhcs.processing.backends.enhance.basic_processor_jax" + ) == (("openhcs-basicpy",) if excluded else ()) + + def test_concurrent_catalog_requests_share_one_metadata_load(monkeypatch) -> None: from openhcs.processing.backends.lib_registry.registry_service import ( RegistryService, From c00a652d49a294a854245b07ad26c7dcb8f8e7be Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 03:44:05 -0400 Subject: [PATCH 19/84] B6: BaSiC observation check reads bare arrays through ImagePayload.of validate_observation_domain is typed RuntimeArrayData (payload or bare array); G3 replaced image_payload_metadata(source) with source.metadata, which a bare ndarray lacks. ImagePayload.of is the declared entry that wraps a bare array. Co-Authored-By: Claude Opus 5.5 --- openhcs/processing/backends/enhance/flatfield.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/openhcs/processing/backends/enhance/flatfield.py b/openhcs/processing/backends/enhance/flatfield.py index 4f049bb880..aaced4b44c 100644 --- a/openhcs/processing/backends/enhance/flatfield.py +++ b/openhcs/processing/backends/enhance/flatfield.py @@ -6,6 +6,7 @@ from openhcs.core.projected_image_output import SourceProjectedImageOutput from openhcs.core.runtime_array_values import RuntimeArrayData +from openhcs.core.runtime_image_values import ImagePayload from openhcs.core.runtime_plane_projection import RuntimePlaneAxisValueProjection from openhcs.core.axes import AxisFamily, AxisRole, TileAxis @@ -34,7 +35,7 @@ def __post_init__(self) -> None: @classmethod def validate_observation_domain(cls, source: RuntimeArrayData) -> None: """Reject mislabeled metadata-backed ensembles before fitting.""" - metadata = source.metadata + metadata = ImagePayload.of(source).metadata family = AxisFamily.active() retained_axes = tuple( family.named(name) for name in metadata.retained_plane_component_values() From 4b1dec1e2977d6ce55d76f2bde5f29316f2e3a0f Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 03:44:05 -0400 Subject: [PATCH 20/84] B6: STEP_INPUT selectors match the stored source paths of current planes A source-bindings workspace names planes by virtual paths; the stored paths a plane was admitted from travel in OpenHCSSourceFilterPaths. Candidate matching already prefers them; provenance matching compared FILE filters to the virtual path and selected nothing. SourceImageIdentity.filter_paths owns the rule. Co-Authored-By: Claude Opus 5.5 --- openhcs/core/source_bindings.py | 2 +- openhcs/core/source_image_provenance.py | 12 ++++++++++++ 2 files changed, 13 insertions(+), 1 deletion(-) diff --git a/openhcs/core/source_bindings.py b/openhcs/core/source_bindings.py index 8ff126a170..e688e378b0 100644 --- a/openhcs/core/source_bindings.py +++ b/openhcs/core/source_bindings.py @@ -724,7 +724,7 @@ def matches_provenance(self, provenance: SourceImageProvenance) -> bool: identities = provenance.represented_source_identities return ( (not self.filters or self.path_filters_match( - tuple(identity.path for identity in identities if identity.path is not None) + tuple(path for identity in identities for path in identity.filter_paths) )) and self.metadata_candidates_match( tuple(identity.component_metadata or {} for identity in identities) diff --git a/openhcs/core/source_image_provenance.py b/openhcs/core/source_image_provenance.py index 7a4d6a499c..3db63f50f2 100644 --- a/openhcs/core/source_image_provenance.py +++ b/openhcs/core/source_image_provenance.py @@ -126,6 +126,18 @@ def from_mapping(cls, values: Mapping[str, object]) -> "SourceImageIdentity": def addressable(self) -> bool: return self.path is not None or self.component_metadata is not None + @property + def filter_paths(self) -> tuple[str, ...]: + """Store-owned paths that source filters match. + + A virtual workspace path names the plane; the stored source paths it + was admitted from are what a FILE filter selects. + """ + stored = SourceMetadataFields.source_filter_paths(self.component_metadata or {}) + if stored: + return stored + return () if self.path is None else (self.path,) + @property def filename_extension(self) -> str | None: """Project the declared source filename fact, never a format default.""" From 2eb48fb9d6397fb96e5fe16d92b946b4645cda26 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 03:44:16 -0400 Subject: [PATCH 21/84] B6: Official30 viewer variants select viewer families, not ViewerType members StreamingConfig.__registry__ is keyed by ViewerFamily classes; the test selected ViewerType members and read .family on registry keys. Co-Authored-By: Claude Opus 5.5 --- .../test_cellprofiler_official30_zmq.py | 75 ++++++++++--------- 1 file changed, 40 insertions(+), 35 deletions(-) diff --git a/tests/integration/test_cellprofiler_official30_zmq.py b/tests/integration/test_cellprofiler_official30_zmq.py index a298d1d30d..7f07bf67a5 100644 --- a/tests/integration/test_cellprofiler_official30_zmq.py +++ b/tests/integration/test_cellprofiler_official30_zmq.py @@ -51,7 +51,12 @@ runtime_artifact_viewer_component_identity, ) from openhcs.core.source_spatial_domain import SourceSpatialDomain -from openhcs.core.streaming_config_declarations import ViewerType +from openhcs.core.streaming_config_declarations import ( + FijiViewer, + NapariViewer, + ViewerFamily, + ViewerType, +) from openhcs.runtime.fiji_macro_runtime import FijiMacroExecutionRequest from openhcs.runtime.viewer_protocol import ( ViewerControlMessageRequest, @@ -75,20 +80,20 @@ for case_index, case in enumerate(_OFFICIAL30_CASES) ) _OFFICIAL30_FIJI_VIEWER_VARIANTS = ( - (ViewerType.FIJI,), - (ViewerType.FIJI, ViewerType.NAPARI), + (FijiViewer,), + (FijiViewer, NapariViewer), ) _OFFICIAL30_FIJI_VIEWER_PARAMS = tuple( pytest.param( - viewer_types, + viewers, case_index, case, id=( - "+".join(viewer_type.value for viewer_type in viewer_types) + "+".join(viewer.wire_value for viewer in viewers) + f"-{case_index:02d}-{case.name}" ), ) - for viewer_types in _OFFICIAL30_FIJI_VIEWER_VARIANTS + for viewers in _OFFICIAL30_FIJI_VIEWER_VARIANTS for case_index, case in enumerate(_OFFICIAL30_CASES) ) _FIJI_FRESH_PROCESS_IMAGE_PROBE = r"""#@ String Directory @@ -236,13 +241,13 @@ def _free_zmq_port_pair( def _registered_streaming_config_kwargs( - viewer_types: Sequence[ViewerType], - ports_by_viewer: Mapping[ViewerType, int], + viewers: Sequence[type[ViewerFamily]], + ports_by_viewer: Mapping[type[ViewerFamily], int], ) -> dict[str, StreamingConfig]: """Project one viewer selection through registered config owners.""" - selected_viewers = set(viewer_types) - if len(selected_viewers) != len(viewer_types): + selected_viewers = set(viewers) + if len(selected_viewers) != len(viewers): raise ValueError("Official30 viewer variants require unique declarations.") if set(ports_by_viewer) != selected_viewers: raise ValueError( @@ -250,8 +255,8 @@ def _registered_streaming_config_kwargs( ) config_kwargs: dict[str, StreamingConfig] = {} - for viewer_type, config_type in StreamingConfig.__registry__.items(): - enabled = viewer_type in selected_viewers + for viewer, config_type in StreamingConfig.__registry__.items(): + enabled = viewer in selected_viewers init_kwargs: dict[str, object] = { "enabled": enabled, "persistent": enabled, @@ -259,10 +264,10 @@ def _registered_streaming_config_kwargs( if enabled: init_kwargs.update( host="127.0.0.1", - port=ports_by_viewer[viewer_type], + port=ports_by_viewer[viewer], transport_mode=TransportMode.TCP, ) - config_kwargs[viewer_type.family.config_key] = config_type(**init_kwargs) + config_kwargs[viewer.config_key] = config_type(**init_kwargs) if selected_viewers - set(StreamingConfig.__registry__): raise ValueError("Official30 viewer variant selected an unregistered config.") @@ -563,13 +568,13 @@ def test_official30_fiji_variants_project_registered_viewer_configs() -> None: partition_filter_config=PartitionFilterConfig(partition_filter=1) ) - for viewer_types in _OFFICIAL30_FIJI_VIEWER_VARIANTS: + for viewers in _OFFICIAL30_FIJI_VIEWER_VARIANTS: ports_by_viewer = { - viewer_type: 20_000 + index * 100 - for index, viewer_type in enumerate(viewer_types) + viewer: 20_000 + index * 100 + for index, viewer in enumerate(viewers) } configs = _registered_streaming_config_kwargs( - viewer_types, + viewers, ports_by_viewer, ) variant_config = GlobalPipelineConfig( @@ -585,11 +590,11 @@ def test_official30_fiji_variants_project_registered_viewer_configs() -> None: if field.name not in registered_keys ) assert { - config.viewer_family.viewer_type() for config in configs.values() if config.enabled - } == set(viewer_types) + config.viewer_family for config in configs.values() if config.enabled + } == set(viewers) assert all(config.persistent is config.enabled for config in configs.values()) assert { - config.viewer_family.viewer_type(): config.port + config.viewer_family: config.port for config in configs.values() if config.enabled } == ports_by_viewer @@ -742,12 +747,12 @@ def test_official30_nonpersistent_napari_isolated_per_case( @pytest.mark.integration @pytest.mark.parametrize( - ("viewer_types", "case_index", "case"), + ("viewers", "case_index", "case"), _OFFICIAL30_FIJI_VIEWER_PARAMS, ) def test_official30_persistent_fiji_variants_isolated_per_case( tmp_path: Path, - viewer_types: tuple[ViewerType, ...], + viewers: tuple[type[ViewerFamily], ...], case_index: int, case: CellProfilerComparisonCase, ) -> None: @@ -760,14 +765,14 @@ def test_official30_persistent_fiji_variants_isolated_per_case( transport_mode=OPENHCS_ZMQ_CONFIG.transport_mode, ) ports_by_viewer = { - viewer_type: _free_zmq_port_pair( + viewer: _free_zmq_port_pair( excluded_ports, transport_mode=TransportMode.TCP, ) - for viewer_type in viewer_types + for viewer in viewers } streaming_configs = _registered_streaming_config_kwargs( - viewer_types, + viewers, ports_by_viewer, ) global_config = GlobalPipelineConfig( @@ -775,15 +780,15 @@ def test_official30_persistent_fiji_variants_isolated_per_case( **streaming_configs, ) endpoints = { - viewer_type.value: ViewerRuntimeEndpoint( - streaming_configs[viewer_type.family.config_key] + viewer.wire_value: ViewerRuntimeEndpoint( + streaming_configs[viewer.config_key] .viewer_runtime_config() .transport_endpoint, OPENHCS_ZMQ_CONFIG, ) - for viewer_type in viewer_types + for viewer in viewers } - variant_id = "+".join(viewer_type.value for viewer_type in viewer_types) + variant_id = "+".join(viewer.wire_value for viewer in viewers) case_output_root = tmp_path / variant_id / f"case_{case_index:02d}" macro_path = tmp_path / f"{variant_id}_fresh_process_probe.ijm" macro_path.write_text(_FIJI_FRESH_PROCESS_IMAGE_PROBE, encoding="utf-8") @@ -821,18 +826,18 @@ def test_official30_persistent_fiji_variants_isolated_per_case( assert (case_output_root / "summary.csv").is_file() runtime_observation = _runtime_observation(observations[0]) - for viewer_type in viewer_types: - config = streaming_configs[viewer_type.family.config_key] - if viewer_type is ViewerType.NAPARI: + for viewer in viewers: + config = streaming_configs[viewer.config_key] + if viewer is NapariViewer: _assert_live_napari_state(config, runtime_observation) - elif viewer_type is ViewerType.FIJI: + elif viewer is FijiViewer: _assert_fiji_fresh_process_image( config=config, macro_path=macro_path, ) else: raise AssertionError( - f"Unhandled registered Official30 viewer {viewer_type.name!r}." + f"Unhandled registered Official30 viewer {viewer.display_name!r}." ) except Exception as error: failures.append( From dbae3e0b464f130e8b0c54053ef3811a7e261870 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 03:44:16 -0400 Subject: [PATCH 22/84] B6: GUI session journey runs in the GUI job; orchestrator exposes dataset_root The Qt journey created a QApplication inside backend/OMERO integration jobs, which install no GUI extra and set no Qt platform; on Linux it aborted the whole pytest process. It now lives under tests/pyqt_gui, run by gui-tests with the gui extras and QT_QPA_PLATFORM=offscreen, without importorskip. The read-only journey reads PipelineOrchestrator.dataset_root (G7 rename). Co-Authored-By: Claude Opus 5.5 --- tests/integration/test_session_journey.py | 96 ++--------------- tests/pyqt_gui/test_session_gui_journey.py | 115 +++++++++++++++++++++ 2 files changed, 121 insertions(+), 90 deletions(-) create mode 100644 tests/pyqt_gui/test_session_gui_journey.py diff --git a/tests/integration/test_session_journey.py b/tests/integration/test_session_journey.py index fe7966d343..f99d4f83ec 100644 --- a/tests/integration/test_session_journey.py +++ b/tests/integration/test_session_journey.py @@ -1,8 +1,9 @@ -"""One dataset through the session, headless over MCP and in the Qt GUI. +"""One dataset through the session, headless over MCP. -Both journeys add a synthetic dataset, give it a pipeline, initialize, compile -and run it on a real execution server, following the session's events. They -use the same SessionOperation classes; only the client differs. +The journey adds a synthetic dataset, gives it a pipeline, initializes, +compiles and runs it on a real execution server, following the session's +events. tests/pyqt_gui/test_session_gui_journey.py runs the same +SessionOperation classes through the Qt widgets. """ from __future__ import annotations @@ -18,12 +19,9 @@ from openhcs.agent.path_policy import AgentPathPolicy from openhcs.authoring.session.operations.datasets import ( AddDatasets, - CompileDatasets, InitializeDatasets, - RunDatasets, ) from openhcs.authoring.session.session import CallerThread -from openhcs.authoring.session.views import DatasetListView from openhcs.core.config import PipelineConfig from openhcs.core.pipeline_document import PipelineDocumentCodec from openhcs.core.steps import FunctionStep @@ -152,91 +150,9 @@ def test_headless_mcp_journey_adds_compiles_and_runs_a_dataset(tmp_path: Path) - context.session.close() -def test_gui_journey_runs_the_same_operations_through_the_widgets( - tmp_path: Path, -) -> None: - pytest.importorskip("PyQt6") - from PyQt6.QtWidgets import QApplication - from pyqt_reactive.services.ui_thread_dispatch import UiThreadDispatcher - - from openhcs.authoring.session.session import DispatcherThread, Session - from openhcs.core.config import GlobalPipelineConfig - from openhcs.pyqt_gui.config import get_default_ui_config - from openhcs.pyqt_gui.widgets.plate_manager import PlateManagerWidget - from tests.unit.pyqt_gui.session_harness import GuiServiceStub, release_widgets - - application = QApplication.instance() or QApplication([]) - root, source = _synthetic_dataset(tmp_path) - dispatcher = UiThreadDispatcher() - session = Session( - transport_config=OpenHCSZMQConfig(default_port=_port(1), persistent=False), - global_config=GlobalPipelineConfig(), - main_thread=DispatcherThread(dispatcher), - ) - services = GuiServiceStub() - manager = PlateManagerWidget( - services, session, gui_config=get_default_ui_config() - ) - try: - assert session.invoke(AddDatasets, AddDatasets.request(roots=(str(root),))).accepted - (scope_id,) = session.dataset_scope_ids() - session.set_pipeline_source(scope_id, source) - application.processEvents() - assert [row.scope_id for row in manager.plates] == [scope_id] - manager.item_list.setCurrentRow(0) - application.processEvents() - assert manager.selection_scope_ids() == (scope_id,) - - def wait_until(condition, label) -> None: - deadline = time.monotonic() + STAGE_TIMEOUT_SECONDS - while not condition(): - application.processEvents() - assert time.monotonic() < deadline, label - time.sleep(0.05) - - for operation, finished in ( - (InitializeDatasets, lambda row: row.initialized and not row.init_pending), - (CompileDatasets, lambda row: row.compiled and not row.compile_pending), - ( - RunDatasets, - lambda row: row.terminal_status is not None and not row.execution_active, - ), - ): - if operation is CompileDatasets: - # Disconnected, the compile button is Connect; connect first. - assert manager.buttons[operation.operation_id].text() == "Connect" - manager.handle_button_action(operation.operation_id) - wait_until( - lambda: CompileDatasets.resolved(session) is CompileDatasets, - "connect", - ) - manager.update_button_states() - assert manager.buttons[operation.operation_id].isEnabled(), operation - manager.handle_button_action(operation.operation_id) - deadline = time.monotonic() + STAGE_TIMEOUT_SECONDS - while True: - application.processEvents() - (row,) = DatasetListView.state_of(session).rows - if finished(row): - break - assert time.monotonic() < deadline, (operation, row) - time.sleep(0.05) - assert row.terminal_status == "complete", row - wait_until(lambda: not session.execution_state.busy, "batch finished") - application.processEvents() - assert manager.buttons[RunDatasets.operation_id].text() == "Run" - finally: - manager.cleanup() - manager.close() - release_widgets(application, manager) - session.close() - dispatcher.close() - - def test_read_only_dataset_initializes_in_a_mirrored_workspace(tmp_path: Path) -> None: """Initialization never writes into a source it may not write.""" - from openhcs.authoring.session.operations.datasets import AddDatasets from openhcs.authoring.session.session import Session from openhcs.core.config import GlobalPipelineConfig @@ -272,7 +188,7 @@ def test_read_only_dataset_initializes_in_a_mirrored_workspace(tmp_path: Path) - assert not failures, failures assert time.monotonic() < deadline time.sleep(0.05) - assert session.orchestrator(scope_id).plate_path.is_relative_to(workspaces) + assert session.orchestrator(scope_id).dataset_root.is_relative_to(workspaces) assert _source_state(root) == before finally: session.close() diff --git a/tests/pyqt_gui/test_session_gui_journey.py b/tests/pyqt_gui/test_session_gui_journey.py new file mode 100644 index 0000000000..e5d4ce61a4 --- /dev/null +++ b/tests/pyqt_gui/test_session_gui_journey.py @@ -0,0 +1,115 @@ +"""One dataset through the session, driven through the Qt widgets. + +The same SessionOperation classes as the headless MCP journey in +tests/integration/test_session_journey.py run here through the plate manager +on a real execution server; only the client differs. +""" + +from __future__ import annotations + +import time +from pathlib import Path + +import pytest +from objectstate.object_state import ObjectStateRegistry +from PyQt6.QtWidgets import QApplication +from pyqt_reactive.services.ui_thread_dispatch import UiThreadDispatcher + +from openhcs.authoring.session.operations.datasets import ( + AddDatasets, + CompileDatasets, + InitializeDatasets, + RunDatasets, +) +from openhcs.authoring.session.session import DispatcherThread, Session +from openhcs.authoring.session.views import DatasetListView +from openhcs.core.config import GlobalPipelineConfig +from openhcs.pyqt_gui.config import get_default_ui_config +from openhcs.pyqt_gui.widgets.plate_manager import PlateManagerWidget +from openhcs.runtime.zmq_config import OpenHCSZMQConfig +from tests.integration.test_session_journey import ( + STAGE_TIMEOUT_SECONDS, + _port, + _synthetic_dataset, +) +from tests.unit.pyqt_gui.session_harness import GuiServiceStub, release_widgets + + +@pytest.fixture(autouse=True) +def fresh_object_state_registry(): + """The session starts from an empty dataset list.""" + + ObjectStateRegistry.clear() + yield + ObjectStateRegistry.clear() + + +def test_gui_journey_runs_the_same_operations_through_the_widgets( + tmp_path: Path, +) -> None: + application = QApplication.instance() or QApplication([]) + root, source = _synthetic_dataset(tmp_path) + dispatcher = UiThreadDispatcher() + session = Session( + transport_config=OpenHCSZMQConfig(default_port=_port(1), persistent=False), + global_config=GlobalPipelineConfig(), + main_thread=DispatcherThread(dispatcher), + ) + services = GuiServiceStub() + manager = PlateManagerWidget( + services, session, gui_config=get_default_ui_config() + ) + try: + assert session.invoke(AddDatasets, AddDatasets.request(roots=(str(root),))).accepted + (scope_id,) = session.dataset_scope_ids() + session.set_pipeline_source(scope_id, source) + application.processEvents() + assert [row.scope_id for row in manager.plates] == [scope_id] + manager.item_list.setCurrentRow(0) + application.processEvents() + assert manager.selection_scope_ids() == (scope_id,) + + def wait_until(condition, label) -> None: + deadline = time.monotonic() + STAGE_TIMEOUT_SECONDS + while not condition(): + application.processEvents() + assert time.monotonic() < deadline, label + time.sleep(0.05) + + for operation, finished in ( + (InitializeDatasets, lambda row: row.initialized and not row.init_pending), + (CompileDatasets, lambda row: row.compiled and not row.compile_pending), + ( + RunDatasets, + lambda row: row.terminal_status is not None and not row.execution_active, + ), + ): + if operation is CompileDatasets: + # Disconnected, the compile button is Connect; connect first. + assert manager.buttons[operation.operation_id].text() == "Connect" + manager.handle_button_action(operation.operation_id) + wait_until( + lambda: CompileDatasets.resolved(session) is CompileDatasets, + "connect", + ) + manager.update_button_states() + assert manager.buttons[operation.operation_id].isEnabled(), operation + manager.handle_button_action(operation.operation_id) + deadline = time.monotonic() + STAGE_TIMEOUT_SECONDS + while True: + application.processEvents() + (row,) = DatasetListView.state_of(session).rows + if finished(row): + break + assert time.monotonic() < deadline, (operation, row) + time.sleep(0.05) + assert row.terminal_status == "complete", row + wait_until(lambda: not session.execution_state.busy, "batch finished") + application.processEvents() + assert manager.buttons[RunDatasets.operation_id].text() == "Run" + finally: + manager.cleanup() + manager.close() + release_widgets(application, manager) + session.close() + dispatcher.close() From 1bca1b139b04869d3574c0eeeee7d094f2efaf19 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 03:44:16 -0400 Subject: [PATCH 23/84] B6: custom sources persist their exact bytes; transport test emits typed progress The source digest is computed over the code's UTF-8 bytes and checked against the file's bytes; text-mode writing translated newlines on Windows, so every persisted source read back as changed. Write with newline="". The spawn test used the deleted progress emit() and dict events; it now emits through ProgressExecutionContext and reads ProgressEvent fields. Co-Authored-By: Claude Opus 5.5 --- .../test_custom_source_transport.py | 25 +++++++++++-------- 1 file changed, 15 insertions(+), 10 deletions(-) diff --git a/tests/integration/test_custom_source_transport.py b/tests/integration/test_custom_source_transport.py index 1c3e3db2af..3d6f005222 100644 --- a/tests/integration/test_custom_source_transport.py +++ b/tests/integration/test_custom_source_transport.py @@ -403,7 +403,11 @@ def _require_stale_rejection(operation): def _spawned_custom_task(payload, execution_id, subject_id): """Run actual registered declarations and emit through the initialized queue.""" import numpy as np - from openhcs.core.progress import emit, ProgressPhase, ProgressStatus + from openhcs.core.progress import ( + ProgressExecutionContext, + ProgressPhase, + ProgressStatus, + ) _assert_payload(payload, "reference") values = np.arange(6, dtype=np.uint16).reshape(2, 3) @@ -412,9 +416,10 @@ def _spawned_custom_task(payload, execution_id, subject_id): np.testing.assert_array_equal(reference.resolve()(values), values) owner = reference.metadata.artifact_outputs[0].measurement_feature_owner results.append(owner.make_row(2.5)) - emit(execution_id=execution_id, subject_id=subject_id, axis_id="synthetic", - step_name="custom", phase=ProgressPhase.STEP_COMPLETED, - status=ProgressStatus.SUCCESS, percent=100) + ProgressExecutionContext(execution_id, subject_id).emit( + axis_id="synthetic", step_name="custom", phase=ProgressPhase.STEP_COMPLETED, + status=ProgressStatus.SUCCESS, percent=100, completed=1, total=1, + ) return os.getpid(), tuple(results) @@ -493,7 +498,7 @@ def _spawn(path): from openhcs.core.config import MultiprocessingStartMethod from openhcs.core.orchestrator.cancellation import ExecutionCancellationSignal from openhcs.core.orchestrator.worker_execution import WorkerExecutorFactory - from openhcs.core.progress import ProgressExecutionContext + from openhcs.core.progress import ProgressExecutionContext, ProgressStatus # A future derived context can carry nontransportable local runtime state; # no generic worker-bootstrap consumer should need to learn its fields. @@ -539,11 +544,11 @@ class RichProgressContext(ProgressExecutionContext): assert type(row) is helpers[4] assert row == helpers[5] progress = queue.get(timeout=5) - assert progress["execution_id"] == context.execution_id - assert progress["subject_id"] == context.subject_id - assert progress["pid"] == pid - assert progress["status"] == "success" - assert progress["percent"] == 100 + assert progress.execution_id == context.execution_id + assert progress.subject_id == context.subject_id + assert progress.pid == pid + assert progress.status is ProgressStatus.SUCCESS + assert progress.percent == 100 print(f"spawned custom revision/progress/rows: worker {pid}", flush=True) finally: queue.close() From 42f51b7a71f941449cd16646d6b3107c24f2cbfc Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 03:44:16 -0400 Subject: [PATCH 24/84] B6: measured CLI child imports the installation under test PYTHONPATH= made the child import openhcs from the source checkout, which in CI has no built native extensions, instead of the installed wheel. Co-Authored-By: Claude Opus 5.5 --- tests/integration/test_measured_cli_session.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/tests/integration/test_measured_cli_session.py b/tests/integration/test_measured_cli_session.py index 7eb363e046..b81290c951 100644 --- a/tests/integration/test_measured_cli_session.py +++ b/tests/integration/test_measured_cli_session.py @@ -114,7 +114,9 @@ def _command( def _environment(tmp_path: Path) -> dict[str, str]: - environment = {**os.environ, "PYTHONPATH": str(REPOSITORY_ROOT)} + # The CLI child imports benchmark and openhcs from the same installation as + # this test process; a checkout path would shadow an installed candidate. + environment = dict(os.environ) environment.setdefault("XDG_DATA_HOME", str(tmp_path / "xdg")) environment.pop("DISPLAY", None) return environment From 5e519401fe778ad2173f487e27c9374cc3a7e370 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 03:44:16 -0400 Subject: [PATCH 25/84] B6: mahotas is a core runtime requirement Every execution server prepares the registered catalog, including the CellProfiler 4 watershed entries whose paths import mahotas; the registry treats an eligible module that cannot run as a broken installation. A core install (python-314-core-tests: dev,mcp) could not start a server. mahotas has wheels for every supported platform and Python 3.14. Co-Authored-By: Claude Opus 5.5 --- pyproject.toml | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/pyproject.toml b/pyproject.toml index cc824c4856..6033a7c971 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -63,6 +63,9 @@ dependencies = [ "numba>=0.61.0,<0.63; platform_system == 'Darwin' and platform_machine == 'x86_64'", "numba>=0.61.0; platform_system != 'Darwin' or platform_machine != 'x86_64'", "skan>=0.13.1", + # Every execution server prepares the CellProfiler watershed catalog entries, + # whose CellProfiler 4 paths run mahotas; it is a runtime requirement. + "mahotas>=1.4.18", "threadpoolctl>=3.6.0", # Image I/O and formats @@ -132,7 +135,6 @@ dev-gui = [ # Production CellProfiler compatibility dependencies cellprofiler-compat = [ - "mahotas>=1.4.18", "matplotlib>=3.10.0", ] From f8d6cdcdca9db93dd9c9b0fe0b24f401dcd8bb4a Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 03:47:46 -0400 Subject: [PATCH 26/84] B6: U1 surface doc names where the GUI journey lives Co-Authored-By: Claude Opus 5.5 --- docs/refactor/consolidation-20261009/U1-session-interface.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/refactor/consolidation-20261009/U1-session-interface.md b/docs/refactor/consolidation-20261009/U1-session-interface.md index 1f2929f8b5..1380f5346e 100644 --- a/docs/refactor/consolidation-20261009/U1-session-interface.md +++ b/docs/refactor/consolidation-20261009/U1-session-interface.md @@ -88,7 +88,7 @@ Measured against `origin/main` (`89b8fdbff`, Python plus the MCP contract fixtur Corrections to the plan above: - **MCP tools are derived, not subclassed.** `SessionOperation` does not subclass `AgentCapabilityDeclaration` (that import cycles through the DTOs); `capabilities.py` derives one `openhcs_` capability per `HeadlessOperation`. The guard checks the derivation is total. -- **The headless journey runs a real execution server**, not a fake client; it and the GUI journey live in `tests/integration/test_session_journey.py`. +- **The headless journey runs a real execution server**, not a fake client; it lives in `tests/integration/test_session_journey.py`; the GUI journey, which needs the GUI extras and a Qt platform, lives in `tests/pyqt_gui/test_session_gui_journey.py`. - **The pipeline editor keeps `displayed_steps`**, a render projection of `PipelineStepsView`, never written back; the guard allows it. - **500 ms polling advice remains on the UI bridge** (`recommended_poll_interval_ms`): bridge events belong to U2. The session's own event stream is push (`openhcs_session_events`). - **The session package loads PyQt6 transitively** through `pyqt_reactive.protocols`; the guard checks direct imports. Owner: U4 (library moves). From 54e87d5b41c265aa639d8cc9491a2bc410c8c7f0 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 03:58:37 -0400 Subject: [PATCH 27/84] B6: pin PolyStore 0.6.1 (OMERO backend answers is_dir) Every OMERO execution failed at the pattern engine's is_dir check with 'OMEROLocalBackend object has no attribute is_dir'. Lockstep with OpenHCSDev/PolyStore#38; release-readiness needs v0.6.1 tagged. Co-Authored-By: Claude Opus 5.5 --- external/PolyStore | 2 +- pyproject.toml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/external/PolyStore b/external/PolyStore index 43cfd8714f..25ee78d94a 160000 --- a/external/PolyStore +++ b/external/PolyStore @@ -1 +1 @@ -Subproject commit 43cfd8714fff0369114982feb88960dcb7d7b682 +Subproject commit 25ee78d94aa31f3e12693c14be3de0d10e104086 diff --git a/pyproject.toml b/pyproject.toml index 6033a7c971..40b2d20292 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -108,7 +108,7 @@ dependencies = [ "python-introspect>=0.2.5,<0.3", "metaclass-registry>=0.3.0,<0.4", "arraybridge>=0.4.1,<0.5", - "polystore>=0.6,<0.7", + "polystore>=0.6.1,<0.7", "pyqt-reactive>=0.4,<0.5", ] From df75920fab79b9c82281651406c068865d6fe61c Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 04:04:50 -0400 Subject: [PATCH 28/84] B6: MCP transport releases the QCoreApplication it created An in-process MCP transport left its QCoreApplication behind, so a later QWidget in the same process found a non-GUI application and Qt aborted the unit-test run (test_viewer_render_snapshot under xvfb, exit 134). Co-Authored-By: Claude Opus 5.5 --- openhcs/mcp/execution.py | 13 +++++++++++-- .../agent/test_mcp_process_resource_lifetime.py | 16 ++++++++++++++++ 2 files changed, 27 insertions(+), 2 deletions(-) diff --git a/openhcs/mcp/execution.py b/openhcs/mcp/execution.py index 39b179072a..aff5e5d721 100644 --- a/openhcs/mcp/execution.py +++ b/openhcs/mcp/execution.py @@ -9,6 +9,7 @@ import threading from typing import Any, ParamSpec, TypeVar +from PyQt6 import sip from PyQt6.QtCore import QCoreApplication, QEventLoop, QThread from polystore import cleanup_backend_connections from pyqt_reactive.core.future_completion import FutureCompletion @@ -85,7 +86,11 @@ class McpTransportExecutor(AsyncOperationExecutor): def __init__(self) -> None: if threading.current_thread() is not threading.main_thread(): raise RuntimeError("MCP transport must be started on the process main thread.") - self._application = QCoreApplication.instance() or QCoreApplication([]) + existing_application = QCoreApplication.instance() + # A process holds one Qt application; one this transport created is + # its own and goes with it, so a later GUI can create a QApplication. + self._created_application = existing_application is None + self._application = existing_application or QCoreApplication([]) if QThread.currentThread() != self._application.thread(): raise RuntimeError("MCP transport requires the Qt application main thread.") super().__init__(max_workers=1) @@ -128,4 +133,8 @@ def close(self) -> None: try: super().close() finally: - cleanup_backend_connections(include_process_resources=True) + try: + cleanup_backend_connections(include_process_resources=True) + finally: + if self._created_application: + sip.delete(self._application) diff --git a/tests/unit/agent/test_mcp_process_resource_lifetime.py b/tests/unit/agent/test_mcp_process_resource_lifetime.py index 1c0079f189..775d4723bc 100644 --- a/tests/unit/agent/test_mcp_process_resource_lifetime.py +++ b/tests/unit/agent/test_mcp_process_resource_lifetime.py @@ -252,3 +252,19 @@ async def journey(): assert calls == ["process resource closed"] transport.execution.close() assert calls == ["process resource closed"] + + +def test_transport_releases_only_the_qt_application_it_created(): + from PyQt6.QtCore import QCoreApplication + + existing = QCoreApplication.instance() + executor = McpTransportExecutor() + application = QCoreApplication.instance() + executor.close() + + if existing is None: + # A later GUI in this process must be able to create its QApplication. + assert QCoreApplication.instance() is None + else: + assert application is existing + assert QCoreApplication.instance() is existing From 89269b9a85c8e21d554410da8c6bb4b29d9014b4 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 03:26:13 -0400 Subject: [PATCH 29/84] B6 cpproc: passthrough outputs keep their producer's filename qualifier A declared passthrough chain (e.g. SaveImages after CorrectIlluminationCalculate) inherits each input's producer context, but its output filename dropped the qualifier the producer published it under, so sibling surfaces (IllumDAPI, IllumDAPIAvg) collided on one path. Co-Authored-By: Claude Opus 5.5 --- openhcs/core/steps/function_runtime.py | 88 ++++++++++++++++++-------- 1 file changed, 61 insertions(+), 27 deletions(-) diff --git a/openhcs/core/steps/function_runtime.py b/openhcs/core/steps/function_runtime.py index 154150e7fc..cc2c741d64 100644 --- a/openhcs/core/steps/function_runtime.py +++ b/openhcs/core/steps/function_runtime.py @@ -543,6 +543,21 @@ def _producer_output_contexts( self.context.microscope_handler.parser, ) + def _producer_filename_qualifiers( + self, + matching_files: Sequence[str], + ) -> tuple[str | None, ...]: + """Resolve the filename qualifier each producer gave the loaded paths.""" + + records = step_output_manifest(self.context).producer_output_records_for_paths( + self.execution_plan, + matching_files, + self.context.microscope_handler.parser, + ) + if records is None: + return (None,) * len(matching_files) + return tuple(record.filename_qualifier for record in records) + def load_input_stack( self, ) -> tuple[list[str], RuntimeArrayData]: @@ -999,6 +1014,24 @@ def _apply_source_binding_payload( ) return binding.apply_loaded_payload(payload, source_context) + def _passes_producer_outputs_through( + self, + processed_stack: ImagePayload, + output_count: int, + input_count: int, + ) -> bool: + """Whether each published slice is the producer's aligned output, passed on. + + A declared passthrough chain names no output of its own, so its slices + keep the exact producer declaration (context and filename qualifier) + of the input file they are aligned to. + """ + if processed_stack.owns_output_surfaces: + return False + if self._unwrapped_main_flow_output_context() is not None: + return False + return output_count == input_count + def _project_output_slices( self, processed_stack: ImagePayload, @@ -1007,21 +1040,13 @@ def _project_output_slices( """Split the function's main output into its published slices.""" if processed_stack.declares_whole_image_output: output_context = self._unwrapped_main_flow_output_context() - contexts = ( - (output_context,) - if output_context is not None - else ( - self._producer_output_contexts(matching_files) - if len(matching_files) == 1 - else () - ) - ) - context = ( - contexts[0] - if contexts - else AlignedImageSliceContext.anonymous_main_flow() - ) - return ((processed_stack, context),) + if output_context is None and self._passes_producer_outputs_through( + processed_stack, 1, len(matching_files) + ): + output_context = self._producer_output_contexts(matching_files)[0] + if output_context is None: + output_context = AlignedImageSliceContext.anonymous_main_flow() + return ((processed_stack, output_context),) unstack_started_at = time.perf_counter() projected = processed_stack.main_output_slices( lambda data: self._unstack_main_output(data, len(matching_files)) @@ -1037,19 +1062,15 @@ def _project_output_slices( if processed_stack.owns_output_surfaces: return projected output_payloads = tuple(payload for payload, _context in projected) - output_context = self._unwrapped_main_flow_output_context() - slice_contexts = ( - (output_context,) * len(output_payloads) - if output_context is not None - else () - ) - if not slice_contexts and len(output_payloads) == len(matching_files): + if self._passes_producer_outputs_through( + processed_stack, len(output_payloads), len(matching_files) + ): slice_contexts = self._producer_output_contexts(matching_files) - if not slice_contexts: - slice_contexts = tuple( - AlignedImageSliceContext.anonymous_main_flow() - for _payload in output_payloads - ) + else: + output_context = self._unwrapped_main_flow_output_context() + if output_context is None: + output_context = AlignedImageSliceContext.anonymous_main_flow() + slice_contexts = (output_context,) * len(output_payloads) return tuple(zip(output_payloads, slice_contexts, strict=True)) def _unstack_main_output( @@ -1123,6 +1144,13 @@ def _save_outputs( output_slices = tuple(payload for payload, _context in projected_outputs) num_outputs = len(output_slices) num_inputs = len(matching_files) + passed_through_qualifiers = ( + self._producer_filename_qualifiers(matching_files) + if self._passes_producer_outputs_through( + processed_stack, num_outputs, num_inputs + ) + else (None,) * num_outputs + ) if num_outputs < num_inputs: logger.debug( @@ -1183,6 +1211,12 @@ def _save_outputs( output_identity = output_identity.with_filename_qualifier( output_context.output_key ) + elif passed_through_qualifiers[i] is not None: + # A passed-through surface keeps the qualifier its producer + # published it under; dropping it would merge sibling surfaces. + output_identity = output_identity.with_filename_qualifier( + passed_through_qualifiers[i] + ) output_path = output_identity.path_for_request(output_path_request) output_path_text = str(output_path) img_slice = output_context.contextualize_image_payload(img_slice) From 34556a1acf2db2b945ca8e2ce8770958782d1075 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 03:33:06 -0400 Subject: [PATCH 30/84] B6 cpproc: synthetic-plate integration tests follow current mechanisms - Known images overwrite the generator's acquisition files instead of adding differently spelled duplicates that collide in the virtual workspace. - count_cells: ImageXpressHandler import had been moved into the persisted custom-function source string; import it at module level. - neurite: checkpoint names derive from the callable's declared output family; stream requests carry the ack return route; runtime images carry the acquisition calibration (the injected pixel_size). Co-Authored-By: Claude Opus 5.5 --- .../test_count_cells_simple_dual_channel.py | 10 ++--- .../test_neurite_outgrowth_synthetic_plate.py | 39 +++++++++---------- 2 files changed, 24 insertions(+), 25 deletions(-) diff --git a/tests/integration/test_count_cells_simple_dual_channel.py b/tests/integration/test_count_cells_simple_dual_channel.py index bc9bc2ada2..cd24f2b7a9 100644 --- a/tests/integration/test_count_cells_simple_dual_channel.py +++ b/tests/integration/test_count_cells_simple_dual_channel.py @@ -55,6 +55,7 @@ SyntheticMicroscopyGenerator, ) from openhcs.domains.microscopy.axes import Microscopy +from openhcs.microscopes.imagexpress import ImageXpressHandler def _write_known_dual_channel_images(plate_dir): @@ -74,19 +75,19 @@ def _write_known_dual_channel_images(plate_dir): image_dir = plate_dir / "TimePoint_1" for well in ("A01", "B01"): tifffile.imwrite( - image_dir / f"{well}_s001_w1_z001_t001.tif", + image_dir / f"{well}_s001_w1.tif", channel_1, ) tifffile.imwrite( - image_dir / f"{well}_s001_w2_z001_t001.tif", + image_dir / f"{well}_s001_w2.tif", channel_2, ) tifffile.imwrite( - image_dir / f"{well}_s002_w1_z001_t001.tif", + image_dir / f"{well}_s002_w1.tif", channel_1, ) tifffile.imwrite( - image_dir / f"{well}_s002_w2_z001_t001.tif", + image_dir / f"{well}_s002_w2.tif", channel_2, ) @@ -186,7 +187,6 @@ def test_dual_channel_count_runs_on_synthetic_plate_with_channel_stack( from openhcs.processing.materialization import CsvOptions, MaterializationSpec, ROIOptions import numpy as np -from openhcs.microscopes.imagexpress import ImageXpressHandler @numpy diff --git a/tests/integration/test_neurite_outgrowth_synthetic_plate.py b/tests/integration/test_neurite_outgrowth_synthetic_plate.py index f96301ea0a..14e16ffa01 100644 --- a/tests/integration/test_neurite_outgrowth_synthetic_plate.py +++ b/tests/integration/test_neurite_outgrowth_synthetic_plate.py @@ -67,6 +67,7 @@ ViewerStreamBackendCallKwargs, ) from openhcs.processing.backends.analysis.neurite_outgrowth import ( + CellProfilerNeuriteEngineProfile, MetaXpressCellBodySettings, MetaXpressNuclearSettings, MetaXpressOutgrowthSettings, @@ -94,13 +95,13 @@ def _write_known_neurite_images(plate_dir): image_dir = plate_dir / "TimePoint_1" for site in (1, 2): - tifffile.imwrite(image_dir / f"A01_s{site:03d}_w1_z001_t001.tif", neurites) - tifffile.imwrite(image_dir / f"A01_s{site:03d}_w2_z001_t001.tif", nuclei) + tifffile.imwrite(image_dir / f"A01_s{site:03d}_w1.tif", neurites) + tifffile.imwrite(image_dir / f"A01_s{site:03d}_w2.tif", nuclei) return np.stack((neurites, nuclei)) def test_neurite_outgrowth_runs_on_synthetic_plate_as_2d_channel_stack( - tmp_path, caplog, monkeypatch + tmp_path, caplog, monkeypatch, viewer_ack_return_route ): caplog.set_level(logging.CRITICAL) plate_dir = tmp_path / "synthetic_neurite_plate" @@ -311,19 +312,13 @@ def observe_dense_output(output, components): np.testing.assert_array_equal(retained, expected_labels) checkpoint_paths = [] + # Checkpoints are the declared outputs between the label artifacts and + # the trailing morphology table; the callable's output family owns them. + checkpoint_names = tuple( + spec.name for spec in CellProfilerNeuriteEngineProfile.artifact_outputs()[6:-1] + ) for artifact_name, expected_checkpoint in zip( - ( - "neurite_candidate_mask", - "neurite_unrooted_residual", - "neurite_secondary_ownership", - "neurite_topology_dropped_trace", - "neurite_topology_added_trace", - "neurite_enhanced_response", - "neurite_threshold_support", - "neurite_retained_support", - "neurite_local_response", - "neurite_local_support", - ), + checkpoint_names, expected[7:-1], strict=True, ): @@ -336,7 +331,7 @@ def observe_dense_output(output, components): retained, np.asarray(expected_checkpoint)[0], ) - assert len({path.name for path in checkpoint_paths}) == 20 + assert len({path.name for path in checkpoint_paths}) == 2 * len(checkpoint_names) assert compiled_plan.output_dataset_root is not None output_dataset_root = Path(compiled_plan.output_dataset_root) @@ -368,7 +363,8 @@ def observe_dense_output(output, components): # Exercise the real stream projection of the compiled writer outputs. # Artifact storage axes are legitimately empty; source image planes are not. - assert len(dense_outputs) == 18 + # Both sites publish four label images and every declared checkpoint. + assert len(dense_outputs) == 2 * (4 + len(checkpoint_names)) stream_kwargs = ViewerStreamBackendCallKwargs( ViewerStreamBackendKwargs( ViewerStreamRequest( @@ -424,13 +420,16 @@ def observe_dense_output(output, components): request ), display_payload_extra=backend.display_payload_extra(request), + return_route=viewer_ack_return_route, ), ) for output, item in zip(outputs, batch.batch_images, strict=True): assert output.variable_components == () - # This fixture injects physical pixel_size into the callable, - # but does not declare voxel spacing on its runtime images. - assert output.metadata.source_voxel_spacing == SourceVoxelSpacing() + # The acquisition's calibration is the same physical pixel + # size the compiler injected into the callable. + assert output.metadata.source_voxel_spacing == SourceVoxelSpacing( + (compiled_invocation.kwargs_dict["pixel_size"],) * 2 + ) if checkpoint_batch: assert "_w1_" in Path(output.path).name else: From 0f0ad9f7c679f9de10e4d8d1a8a00dcacc9b08d8 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 03:49:58 -0400 Subject: [PATCH 31/84] B6 cpproc: benchmark adapter keeps imported config inheriting the global config dataclasses.replace on the imported lazy PipelineConfig froze every unset field at its class default, so the submitted pipeline config overrode the adapter's materialize_runtime_artifacts=False with True. Use replace_raw. That exposed the adapter's absolute materialization_results_path, which named a plate root () other than the compiled one (the source workspace's); delete it so the default relative 'results' resolves inside the compiled output root. Certificate: derived (restated the default relation with the wrong root); output_dataset_root was used only for it and an empty mkdir. The explicit-exports test now asserts the compiler's declared policy (only declared exports persist; retained labels do not), as test_compiler_auto_materialization_policy already states. Co-Authored-By: Claude Opus 5.5 --- benchmark/adapters/openhcs.py | 13 +++++-------- .../test_benchmark_openhcs_adapter_cppipe.py | 5 +++-- 2 files changed, 8 insertions(+), 10 deletions(-) diff --git a/benchmark/adapters/openhcs.py b/benchmark/adapters/openhcs.py index 6448fa115f..71dbcfd692 100644 --- a/benchmark/adapters/openhcs.py +++ b/benchmark/adapters/openhcs.py @@ -18,6 +18,8 @@ from pathlib import Path from typing import Any +from objectstate import replace_raw + from benchmark.adapters.cppipe_source import ( CPPipeSourceRequest, CPPipeSourceResolution, @@ -307,7 +309,6 @@ def _execution_global_config( request: OpenHCSRunRequest, *, output_suffix: str, - output_dataset_root: Path, compilation_debug_config: CompilationDebugConfig, ) -> GlobalPipelineConfig: """Select benchmark output paths without replacing declared path policy.""" @@ -329,7 +330,6 @@ def _execution_global_config( ), compilation_debug_config=compilation_debug_config, materialize_runtime_artifacts=request.materialize_runtime_artifacts, - materialization_results_path=output_dataset_root / "results", ) def _run_converted_cppipe_pipeline( @@ -360,9 +360,6 @@ def _run_converted_cppipe_pipeline( ) from exc output_suffix = f"_{request.pipeline_name}_converted_cppipe" - output_dataset_root = ( - request.output_dir / f"{request.dataset_path.name}{output_suffix}" - ) generated_source_path = request.output_dir / f"{cppipe_path.stem}_openhcs.py" source_workspace_path = ( request.output_dir @@ -416,7 +413,9 @@ def _run_converted_cppipe_pipeline( "UI/ZMQ FunctionStep source before execution." ) if compilation_debug_config.enabled: - pipeline_config = replace( + # replace_raw keeps the imported config's unset fields inheriting + # from the global config; dataclasses.replace would freeze them. + pipeline_config = replace_raw( pipeline_config, compilation_debug_config=LazyCompilationDebugConfig( enabled=compilation_debug_config.enabled, @@ -429,7 +428,6 @@ def _run_converted_cppipe_pipeline( global_config = self._execution_global_config( request, output_suffix=output_suffix, - output_dataset_root=output_dataset_root, compilation_debug_config=compilation_debug_config, ) ensure_global_config_context(GlobalPipelineConfig, global_config) @@ -542,7 +540,6 @@ def _run_converted_cppipe_pipeline( raise ToolExecutionError(equivalence_failure_message) metric_results = self._metric_results(request.metrics) - output_dataset_root.mkdir(parents=True, exist_ok=True) execution_output_root.mkdir(parents=True, exist_ok=True) provenance = { diff --git a/tests/integration/test_benchmark_openhcs_adapter_cppipe.py b/tests/integration/test_benchmark_openhcs_adapter_cppipe.py index 42621085c3..d2fb082d19 100644 --- a/tests/integration/test_benchmark_openhcs_adapter_cppipe.py +++ b/tests/integration/test_benchmark_openhcs_adapter_cppipe.py @@ -105,8 +105,9 @@ def test_openhcs_adapter_retains_declared_path_policy_and_explicit_exports( for path in result.output_path.rglob("*") if path.is_file() and path.suffix.lower() in {".tif", ".tiff"} ) - assert tiff_outputs - assert all(path.name.endswith(".labels.tif") for path in tiff_outputs) + # With automatic materialization disabled only declared exports persist: + # the spreadsheet. Retained labels and QA checkpoints stay in memory. + assert tiff_outputs == () def test_openhcs_adapter_resolves_dataset_reference_cppipe( From 0d83e1e168b3fc8ab20e76d4b4471fdff23cc665 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 03:52:26 -0400 Subject: [PATCH 32/84] B6 cpproc: benchmark watchdog measures silence from run submission The progress watchdog wraps the client's server launch, and the observer's clock started at construction, so a cold execution server (CI: >120 s with coverage, before any progress) tripped 'no server progress ... none observed'. Startup belongs to the client's readiness timeout; silence now counts from begin_run. Also follow the adapter's deleted output_dataset_root parameter in the unit test. Co-Authored-By: Claude Opus 5.5 --- benchmark/openhcs_measured_run.py | 9 ++++++++- tests/unit/test_openhcs_adapter.py | 16 +++++++++++++++- 2 files changed, 23 insertions(+), 2 deletions(-) diff --git a/benchmark/openhcs_measured_run.py b/benchmark/openhcs_measured_run.py index 2a40080799..50181faf35 100644 --- a/benchmark/openhcs_measured_run.py +++ b/benchmark/openhcs_measured_run.py @@ -134,7 +134,12 @@ class _ZMQProgressTimingObserver: compile_completed_at: float | None = None execution_started_at: float | None = None execution_completed_at: float | None = None - last_progress_monotonic: float = field(default_factory=time.monotonic) + last_progress_monotonic: float | None = None + """Monotonic time of the latest progress; None until a run is submitted. + + Starting the execution server is not execution: its client owns the + startup readiness timeout, so silence is measured from begin_run. + """ last_progress_phase: str = "" last_progress_status: str = "" on_event: Callable[[Mapping[str, Any]], None] | None = None @@ -217,6 +222,8 @@ def _duration(started_at: float | None, ended_at: float | None) -> float | None: def inactivity_seconds(self, *, observed_at: float | None = None) -> float: """Return elapsed monotonic time since the latest server progress event.""" + if self.last_progress_monotonic is None: + return 0.0 current = time.monotonic() if observed_at is None else observed_at return max(0.0, current - self.last_progress_monotonic) diff --git a/tests/unit/test_openhcs_adapter.py b/tests/unit/test_openhcs_adapter.py index 81d555b7e6..c2f94a805c 100644 --- a/tests/unit/test_openhcs_adapter.py +++ b/tests/unit/test_openhcs_adapter.py @@ -153,7 +153,6 @@ def test_openhcs_adapter_preserves_declared_main_flow_output_policy( configured = adapter._execution_global_config( request, output_suffix="_candidate", - output_dataset_root=tmp_path / "candidate-plate", compilation_debug_config=CompilationDebugConfig(), ) @@ -168,6 +167,8 @@ def test_openhcs_adapter_preserves_declared_main_flow_output_policy( assert ( configured.analysis_consolidation_config.output_filename == "custom-summary.csv" ) + # Analysis results resolve inside the compiled output root by default. + assert configured.materialization_results_path == Path("results") assert base_config.path_planning_config.global_output_folder is None assert base_config.analysis_consolidation_config.enabled is True @@ -486,6 +487,19 @@ def test_openhcs_progress_observer_tracks_every_server_event( assert observer.progress_description() == "artifact_transfer/running" +def test_openhcs_progress_silence_is_measured_from_run_submission( + monkeypatch: pytest.MonkeyPatch, +) -> None: + """Server startup belongs to the client's readiness timeout, not the watchdog.""" + observer = _ZMQProgressTimingObserver() + assert observer.inactivity_seconds(observed_at=500.0) == 0.0 + + monkeypatch.setattr("benchmark.openhcs_measured_run.time.monotonic", lambda: 200.0) + observer.begin_run() + + assert observer.inactivity_seconds(observed_at=321.0) == 121.0 + + def test_openhcs_watchdog_allows_execution_without_interval_timer( monkeypatch: pytest.MonkeyPatch, ) -> None: From 35abb00d68739cb793ff5fe6b6be42e12f02e965 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 03:53:14 -0400 Subject: [PATCH 33/84] B6 cpproc: BaSiC accepts plain NumPy observations again validate_observation_domain read source.metadata, which a bare ndarray (the callable's documented plain input, 'no claimed acquisition domain') does not have since the ImagePayload family (G3). Project through ImagePayload.of. Co-Authored-By: Claude Opus 5.5 --- openhcs/processing/backends/enhance/flatfield.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/openhcs/processing/backends/enhance/flatfield.py b/openhcs/processing/backends/enhance/flatfield.py index aaced4b44c..1b95dce858 100644 --- a/openhcs/processing/backends/enhance/flatfield.py +++ b/openhcs/processing/backends/enhance/flatfield.py @@ -68,7 +68,7 @@ def resolve_source_context( if source.shape != (self.observation_count, *self.data.shape): raise ValueError("A fitted field must retain the source spatial grid.") self.validate_observation_domain(source) - metadata = source.metadata.collapse_leading_plane_axis() + metadata = ImagePayload.of(source).metadata.collapse_leading_plane_axis() # Model parameters are not unit-interval input pixels. Contributor source # dtypes/identities remain in provenance, not on the fitted field's range. metadata = metadata.replace_fields( From 7bb55dc3a155e12cd77c1134f6c7d5afd91df4c3 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 04:04:48 -0400 Subject: [PATCH 34/84] B6 cpproc: drop the dataclasses.field import the watchdog change left unused Co-Authored-By: Claude Opus 5.5 --- benchmark/openhcs_measured_run.py | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/benchmark/openhcs_measured_run.py b/benchmark/openhcs_measured_run.py index 50181faf35..d13287a6d7 100644 --- a/benchmark/openhcs_measured_run.py +++ b/benchmark/openhcs_measured_run.py @@ -8,10 +8,7 @@ import sys import time from collections.abc import Callable, Mapping -from dataclasses import ( - dataclass, - field, -) +from dataclasses import dataclass from pathlib import Path from typing import Any From fac0519e59e728c8f00f6bd122fc1ba8e2eae19c Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 04:05:51 -0400 Subject: [PATCH 35/84] B6: pin ObjectState 1.3.2 (own annotations on Python 3.14) Official30 parity on 3.14 failed because ObjectState rebuilt a config class whose redeclared field it misread as inherited. Lockstep with OpenHCSDev/ObjectState#26; release-readiness needs v1.3.2 tagged. Co-Authored-By: Claude Opus 5.5 --- external/ObjectState | 2 +- pyproject.toml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/external/ObjectState b/external/ObjectState index c75a457182..1ac5777004 160000 --- a/external/ObjectState +++ b/external/ObjectState @@ -1 +1 @@ -Subproject commit c75a457182e6b158ce68c759cb1ef5c86c849afa +Subproject commit 1ac577700485e6ccee7f0aa24b8828c5f7f4cad2 diff --git a/pyproject.toml b/pyproject.toml index 40b2d20292..ee0e6396a0 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -104,7 +104,7 @@ dependencies = [ # External dependencies (PyPI versions for production builds) "zmqruntime>=0.6.1,<0.7", "pycodify>=0.1.4,<0.2", - "objectstate>=1.3.1,<2", + "objectstate>=1.3.2,<2", "python-introspect>=0.2.5,<0.3", "metaclass-registry>=0.3.0,<0.4", "arraybridge>=0.4.1,<0.5", From cf0b8eb8c6a2aceb3493a0aa9159745cddecc567 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 04:07:15 -0400 Subject: [PATCH 36/84] B6: black and the 3.11 syntax check over every Python file changed since code-quality last passed main's code-quality job checks the files changed since its last successful head (5c4bcd2ba); 412 of them were unformatted and one test used 3.12-only f-string quote reuse, a syntax error on supported Python 3.11. Formatting only, plus that quote. Co-Authored-By: Claude Opus 5.5 --- benchmark/adapters/cellprofiler.py | 7 +- .../agent_validation/mcp_attempt_recorder.py | 4 +- benchmark/cellprofiler_benchmark_cli.py | 8 +- benchmark/control_service.py | 4 +- .../equivalence/object_label_measurements.py | 6 +- benchmark/equivalence/runtime.py | 2 +- benchmark/equivalence/table_snapshots.py | 8 +- benchmark/openhcs_measured_run.py | 4 +- benchmark/session_measured_run.py | 8 +- benchmark/well_throughput_scaling.py | 21 +- openhcs/agent/capabilities.py | 37 +- openhcs/agent/dto/__init__.py | 8 +- openhcs/agent/dto/functions.py | 118 ++- openhcs/agent/dto/plate.py | 17 +- openhcs/agent/dto/ui_bridge.py | 4 - .../artifact_plan_inspection_service.py | 3 +- .../services/function_catalog_service.py | 83 +- .../services/plate_inspection_service.py | 30 +- .../agent/services/plate_streaming_service.py | 28 +- .../agent/services/selected_plate_service.py | 3 +- openhcs/agent/services/ui_bridge_service.py | 9 +- openhcs/authoring/session/compile_batch.py | 20 +- openhcs/authoring/session/dataset_document.py | 8 +- openhcs/authoring/session/debug_runs.py | 4 +- openhcs/authoring/session/events.py | 4 +- .../authoring/session/operations/datasets.py | 21 +- .../authoring/session/operations/pipelines.py | 4 +- openhcs/authoring/session/progress.py | 5 +- openhcs/authoring/session/session.py | 95 +- openhcs/authoring/session/submission.py | 9 +- openhcs/authoring/session/views.py | 18 +- openhcs/core/aligned_image_payload.py | 323 ++++-- openhcs/core/artifacts.py | 12 +- openhcs/core/axes.py | 33 +- openhcs/core/callable_contract.py | 198 +++- openhcs/core/compiled_step_plan.py | 29 +- openhcs/core/component_filters.py | 10 +- openhcs/core/config.py | 34 +- openhcs/core/dataset_sources/choice.py | 5 +- openhcs/core/dataset_sources/interfaces.py | 9 +- .../core/dataset_sources/openhcs_format.py | 109 +- openhcs/core/dataset_sources/source.py | 7 +- .../dataset_sources/source_bindings_source.py | 7 +- openhcs/core/dataset_sources/source_schema.py | 1 - openhcs/core/debug.py | 6 +- openhcs/core/equivalence/keys.py | 4 +- openhcs/core/equivalence/measurement_facts.py | 7 +- .../core/equivalence/measurement_features.py | 8 +- openhcs/core/equivalence/measurement_rows.py | 5 +- openhcs/core/execution_state.py | 4 +- openhcs/core/function_patterns.py | 89 +- openhcs/core/function_reference.py | 12 +- openhcs/core/image_file_serialization.py | 8 +- openhcs/core/image_payload_execution_mode.py | 15 +- openhcs/core/measurement_dialect.py | 34 +- openhcs/core/measurement_feature_queries.py | 42 +- openhcs/core/measurement_image_alignment.py | 6 +- .../core/measurement_row_materialization.py | 1 - openhcs/core/memory/decorators.py | 12 +- openhcs/core/metadata_cache.py | 1 - openhcs/core/orchestrator/bundle_execution.py | 1 + openhcs/core/orchestrator/execution_result.py | 28 +- openhcs/core/orchestrator/orchestrator.py | 12 +- openhcs/core/orchestrator/worker_execution.py | 21 +- openhcs/core/orchestrator/worker_profiling.py | 7 +- openhcs/core/partition_filter.py | 23 +- openhcs/core/pipeline/compilation_session.py | 15 +- openhcs/core/pipeline/compiler.py | 82 +- .../pipeline/funcstep_contract_validator.py | 1 - openhcs/core/pipeline/function_contracts.py | 17 +- openhcs/core/pipeline/path_planner.py | 124 ++- openhcs/core/post_execute.py | 4 +- openhcs/core/processing_contracts.py | 122 ++- openhcs/core/progress/projection.py | 19 +- openhcs/core/progress/runtime_tree.py | 28 +- openhcs/core/progress/types.py | 20 +- openhcs/core/runtime_adapters.py | 3 +- openhcs/core/runtime_batch_contracts.py | 37 +- openhcs/core/runtime_exports.py | 4 +- openhcs/core/runtime_measurements.py | 11 +- .../core/runtime_object_label_aggregation.py | 8 +- openhcs/core/runtime_object_label_building.py | 3 +- openhcs/core/runtime_object_labels.py | 202 ++-- openhcs/core/runtime_relationships.py | 4 +- openhcs/core/runtime_stores.py | 86 +- openhcs/core/source_binding_context.py | 4 +- openhcs/core/source_binding_selection.py | 105 +- openhcs/core/source_binding_workspace.py | 98 +- openhcs/core/source_bindings.py | 87 +- openhcs/core/source_image_provenance.py | 33 +- openhcs/core/source_metadata.py | 41 +- openhcs/core/source_workspace_projection.py | 195 ++-- .../function_artifact_materialization.py | 65 +- openhcs/core/steps/function_execution.py | 10 +- openhcs/core/steps/function_io.py | 12 +- .../core/steps/function_output_identity.py | 7 +- openhcs/core/steps/function_outputs.py | 54 +- openhcs/core/steps/function_runtime.py | 53 +- .../core/steps/stream_component_semantics.py | 9 +- openhcs/core/viewer_streaming_service.py | 27 +- openhcs/core/virtual_workspace_metadata.py | 56 +- openhcs/demo/basic_pipeline.py | 12 +- .../microscopy/analysis_consolidation.py | 28 +- openhcs/domains/microscopy/axes.py | 8 +- openhcs/formats/pattern/pattern_discovery.py | 16 +- .../interop/cellprofiler/analyst_export.py | 25 +- .../cellprofiler/database_column_dialect.py | 4 +- .../cellprofiler/image_set_numbering.py | 12 +- .../cellprofiler/measurement_dialect.py | 4 +- .../cellprofiler/measurement_lookup.py | 1 - .../interop/cellprofiler/measurement_scope.py | 1 - .../cellprofiler/module_artifact_contracts.py | 1 - .../module_artifact_declarations.py | 37 +- .../cellprofiler/module_declarations.py | 20 +- .../interop/cellprofiler/pipeline_import.py | 10 +- .../runtime/function_contract_execution.py | 8 +- .../cellprofiler/runtime/invocation.py | 14 +- .../runtime/measurement_recording.py | 17 +- .../cellprofiler/runtime/module_execution.py | 67 +- .../interop/cellprofiler/workspace_export.py | 5 +- openhcs/mcp/context.py | 10 +- openhcs/mcp/dev_client_commands/ui.py | 12 +- openhcs/mcp/dev_client_core.py | 15 +- openhcs/mcp/dev_client_renderers/pipeline.py | 8 +- openhcs/mcp/dev_client_renderers/plate.py | 16 +- openhcs/mcp/dev_client_renderers/ui_bridge.py | 56 +- openhcs/mcp/dev_client_session.py | 6 +- openhcs/mcp/execution.py | 8 +- openhcs/mcp/server.py | 16 +- openhcs/microscopes/bioformats.py | 6 +- openhcs/microscopes/imagexpress.py | 10 +- openhcs/microscopes/omero.py | 8 +- openhcs/microscopes/opera_phenix.py | 2 - openhcs/microscopes/vendor_layout.py | 9 +- openhcs/omero/plugin/omero_openhcs/views.py | 8 +- .../backends/analysis/neurite_outgrowth.py | 397 ++++--- .../backends/cellprofiler/alignment.py | 5 +- .../backends/cellprofiler/area_occupied.py | 39 +- .../cellprofiler/calculate_statistics.py | 5 +- .../backends/cellprofiler/classification.py | 117 ++- .../backends/cellprofiler/colocalization.py | 31 +- .../processing/backends/cellprofiler/color.py | 65 +- .../processing/backends/cellprofiler/crop.py | 5 +- .../backends/cellprofiler/display_modules.py | 6 +- .../processing/backends/cellprofiler/edge.py | 10 +- .../cellprofiler/feature_enhancement.py | 9 +- .../backends/cellprofiler/granularity.py | 1 - .../backends/cellprofiler/illumination.py | 12 +- .../backends/cellprofiler/image_geometry.py | 15 +- .../backends/cellprofiler/image_math.py | 4 +- .../backends/cellprofiler/image_quality.py | 1 - .../backends/cellprofiler/intensity.py | 19 +- .../cellprofiler/intensity_distribution.py | 26 +- .../backends/cellprofiler/median_filter.py | 7 +- .../backends/cellprofiler/morphology.py | 51 +- .../backends/cellprofiler/neighbors.py | 13 +- .../backends/cellprofiler/object_images.py | 11 +- .../backends/cellprofiler/object_overlap.py | 5 +- .../backends/cellprofiler/outlines.py | 25 +- .../primary_object_diagnostics.py | 16 +- .../backends/cellprofiler/primary_objects.py | 16 +- .../backends/cellprofiler/relationships.py | 5 +- .../backends/cellprofiler/secondary.py | 2 - .../processing/backends/cellprofiler/shape.py | 24 +- .../backends/cellprofiler/skeleton.py | 6 +- .../backends/cellprofiler/smoothing.py | 21 +- .../cellprofiler/spreadsheet_export.py | 55 +- .../backends/cellprofiler/texture.py | 2 - .../backends/cellprofiler/thresholding.py | 14 +- .../backends/cellprofiler/tracking.py | 39 +- .../backends/cellprofiler/watershed.py | 9 +- .../processing/backends/cellprofiler/worms.py | 19 +- .../backends/cellprofiler/zernike.py | 1 - .../backends/enhance/basic_processor_jax.py | 1 - .../experimental_analysis/analysis.py | 4 +- .../experimental_analysis/result_formats.py | 4 +- .../unified_analysis_engine.py | 36 +- .../backends/lib_registry/openhcs_registry.py | 10 +- .../backends/lib_registry/registry_service.py | 28 +- .../backends/lib_registry/unified_registry.py | 12 +- .../backends/processors/cupy_processor.py | 4 +- .../backends/processors/jax_processor.py | 4 +- .../backends/processors/numpy_processor.py | 25 +- .../processors/pyclesperanto_processor.py | 4 +- .../processors/tensorflow_processor.py | 4 +- .../backends/processors/torch_processor.py | 4 +- .../processing/custom_functions/manager.py | 25 +- .../processing/custom_functions/validation.py | 20 +- openhcs/processing/materialization/core.py | 127 ++- .../pipelines/czi_brain_axon_cellbody.py | 7 +- .../loose_operaphenix_neurite_outgrowth.py | 4 +- ...uroncyto_ii_crossover_neurite_outgrowth.py | 8 +- openhcs/pyqt_gui/main.py | 4 +- .../pyqt_gui/services/reactor_providers.py | 5 +- openhcs/pyqt_gui/services/ui_agent_bridge.py | 9 +- .../services/ui_bridge_composition.py | 1 - .../services/ui_bridge_object_state.py | 4 +- .../services/ui_bridge_plate_manager.py | 1 + .../services/ui_bridge_session_actions.py | 10 +- .../pyqt_gui/services/ui_bridge_windows.py | 1 - openhcs/pyqt_gui/session_rendering.py | 4 +- openhcs/pyqt_gui/widgets/image_browser.py | 9 +- openhcs/pyqt_gui/widgets/pipeline_editor.py | 28 +- .../services/pipeline_editor_workflows.py | 16 +- openhcs/runtime/fiji_viewer_server.py | 25 +- openhcs/runtime/viewer_protocol.py | 152 ++- openhcs/runtime/zmq_compilation.py | 4 +- openhcs/runtime/zmq_control.py | 17 +- openhcs/runtime/zmq_execution_client.py | 15 +- openhcs/runtime/zmq_execution_server.py | 17 +- openhcs/runtime/zmq_execution_signature.py | 4 +- .../runtime/zmq_orchestrator_environment.py | 4 +- openhcs/runtime/zmq_server_hooks.py | 4 +- ...notate_wells_with_experimental_metadata.py | 5 +- scripts/capture_media_gallery.py | 36 +- scripts/check_basicpy_source.py | 17 +- scripts/figures/generate_figure_overlays.py | 5 +- scripts/master_multi_plate_demo.py | 4 +- scripts/mcp_assay_showcase.py | 4 +- scripts/mcp_memory_diagnostic.py | 40 +- scripts/mcp_thesis_demo_live.py | 47 +- scripts/prototype_flatten_object_state.py | 7 +- scripts/smoke_installed_mcp.py | 9 +- scripts/test_real_global_config.py | 9 +- .../diagnostics/check_owned_bootstrap_live.py | 630 ++++++++---- .../diagnostics/volume_projection_fixture.py | 43 +- .../test_agent_observation_export_zmq.py | 23 +- .../test_cellprofiler_generated_pipeline.py | 4 +- .../test_cellprofiler_official30_zmq.py | 3 +- .../test_chained_callable_lineage_journey.py | 72 +- .../test_custom_source_transport.py | 84 +- .../test_image_math_publication_620.py | 162 ++- .../integration/test_measured_cli_session.py | 8 +- .../test_measurement_declaration_journey.py | 297 ++++-- .../test_neurite_outgrowth_synthetic_plate.py | 11 +- ...rimary_segmentation_diagnostics_journey.py | 10 +- tests/integration/test_session_journey.py | 10 +- ...est_source_schema_aggregate_publication.py | 86 +- .../test_end_to_end_workflow_foundation.py | 8 +- tests/pyqt_gui/test_session_gui_journey.py | 11 +- tests/unit/agent/test_agent_services.py | 53 +- .../test_authoring_roundtrip_boundaries.py | 4 +- .../agent/test_callable_artifact_reference.py | 188 +++- tests/unit/agent/test_capabilities.py | 57 +- .../agent/test_cold_inspection_feedback.py | 89 +- ...st_compile_inspection_declared_callable.py | 57 +- .../agent/test_compile_selector_authoring.py | 4 +- .../test_declared_callable_documentation.py | 64 +- .../agent/test_function_declaration_help.py | 16 +- .../agent/test_imagexpress_readonly_paths.py | 15 +- .../unit/agent/test_knowledge_base_service.py | 3 +- .../test_mcp_affine_inspection_progress.py | 153 ++- .../agent/test_mcp_affine_real_compile.py | 48 +- .../test_mcp_dev_client_config_renderer.py | 34 +- .../test_mcp_dev_client_pipeline_results.py | 134 ++- .../test_mcp_dev_client_plate_renderers.py | 202 +++- .../agent/test_mcp_dev_client_renderers.py | 24 +- .../test_mcp_dev_client_sampling_results.py | 289 ++++-- .../test_mcp_dev_client_ui_action_results.py | 269 +++-- .../agent/test_mcp_dev_client_ui_renderers.py | 164 ++- .../test_mcp_dev_client_viewer_renderers.py | 777 ++++++++++---- .../agent/test_mcp_plate_stream_progress.py | 148 ++- .../test_mcp_process_resource_lifetime.py | 54 +- tests/unit/agent/test_mcp_server.py | 551 ++++++---- .../test_physical_source_inspection_445.py | 50 +- .../test_plate_inspection_cold_runtime.py | 6 +- .../agent/test_plate_inspection_service.py | 16 +- .../agent/test_plate_streaming_service.py | 78 +- .../test_progressive_authoring_context.py | 21 +- .../test_public_callable_enum_ingress.py | 66 +- .../test_selected_plane_reopen_stream.py | 5 +- tests/unit/agent/test_u1_g8_guards.py | 21 +- .../agent/test_ui_bridge_operation_family.py | 3 +- tests/unit/agent/test_ui_bridge_service.py | 32 +- .../agent/test_user_facing_help_projection.py | 4 +- .../test_agent_validation_attempt_recorder.py | 29 +- .../unit/cellprofiler_runtime_test_support.py | 2 +- tests/unit/desktop/test_desktop_restart.py | 4 +- tests/unit/progress/test_debug_projection.py | 4 +- tests/unit/pyqt_gui/session_harness.py | 4 +- ...test_concurrent_initialization_progress.py | 4 +- .../test_config_preview_formatters.py | 10 +- tests/unit/pyqt_gui/test_debug_toolbar.py | 8 +- .../pyqt_gui/test_main_config_propagation.py | 10 +- tests/unit/pyqt_gui/test_main_status_bar.py | 5 +- .../pyqt_gui/test_main_window_file_actions.py | 10 +- .../pyqt_gui/test_one_persistence_format.py | 10 +- ...est_parameter_form_manager_cross_window.py | 28 +- ...pipeline_code_document_transactionality.py | 8 +- .../pyqt_gui/test_pipeline_editor_widget.py | 23 +- .../test_plate_compile_connection_action.py | 6 +- .../pyqt_gui/test_plate_manager_widget.py | 65 +- .../pyqt_gui/test_session_batch_workflows.py | 60 +- .../test_synthetic_plate_pipeline_document.py | 12 +- tests/unit/pyqt_gui/test_ui_agent_bridge.py | 73 +- .../test_ui_bridge_live_measurements.py | 4 +- .../pyqt_gui/test_ui_window_flash_snapshot.py | 73 +- .../unit/pyqt_gui/test_zmq_client_service.py | 5 +- tests/unit/remote_sensing_witness_pipeline.py | 3 +- tests/unit/test_ack_return_route_journey.py | 68 +- ...cquisition_calibration_pipeline_journey.py | 4 +- tests/unit/test_aggregate_source_scope.py | 211 +++- .../unit/test_artifact_publication_journey.py | 121 ++- ...utomatic_image_role_materialization_435.py | 555 +++++++--- tests/unit/test_axis_family_guards.py | 63 +- tests/unit/test_axis_family_witness.py | 24 +- tests/unit/test_benchmark_control.py | 5 +- .../test_bioformats_microscope_handler.py | 8 +- .../test_calibrated_gaussian_spatial_axes.py | 87 +- tests/unit/test_callable_abi_preparation.py | 325 ++++-- tests/unit/test_callable_contract.py | 100 +- .../test_canonical_custom_source_lookup.py | 145 ++- tests/unit/test_catalog_current_readiness.py | 154 ++- tests/unit/test_cellprofiler_adapter.py | 3 +- ...rofiler_additional_prepared_payload_abi.py | 100 +- .../unit/test_cellprofiler_analyst_export.py | 205 +++- ...test_cellprofiler_artifact_declarations.py | 29 +- .../test_cellprofiler_backend_public_api.py | 2 +- ..._display_manual_projection_declarations.py | 4 +- .../test_cellprofiler_export_equivalence.py | 32 +- ...cellprofiler_full_stack_materialization.py | 101 +- ...ellprofiler_function_contract_execution.py | 248 +++-- ...lprofiler_geometry_prepared_payload_abi.py | 34 +- ..._identify_secondary_replacement_primary.py | 9 +- tests/unit/test_cellprofiler_image_quality.py | 8 +- .../unit/test_cellprofiler_library_loading.py | 56 +- .../test_cellprofiler_measurement_rows.py | 27 +- .../test_cellprofiler_module_execution.py | 582 +++++++---- ...cellprofiler_module_execution_semantics.py | 7 +- ...ellprofiler_morphology_public_contracts.py | 5 +- ...t_cellprofiler_output_record_occurrence.py | 87 +- .../test_cellprofiler_prepared_payload_abi.py | 58 +- .../unit/test_cellprofiler_runtime_adapter.py | 192 ++-- ...profiler_runtime_callable_introspection.py | 4 +- tests/unit/test_cellprofiler_save_images.py | 48 +- .../unit/test_cellprofiler_source_bindings.py | 24 +- ..._cellprofiler_source_bindings_ingestion.py | 9 +- ...st_cellprofiler_special_io_declarations.py | 13 +- .../test_cellprofiler_spreadsheet_export.py | 263 +++-- .../test_cellprofiler_watershed_provider.py | 54 +- .../test_cellprofiler_workspace_export.py | 21 +- ...est_classification_two_vector_authoring.py | 75 +- ...test_collapsed_persisted_occurrence_435.py | 4 +- ...colocalization_metric_setting_authority.py | 2 +- ...st_colocalization_preparation_readiness.py | 12 +- tests/unit/test_compilation_paths.py | 33 +- tests/unit/test_compilation_session.py | 40 +- tests/unit/test_compiled_execution.py | 20 +- ..._completed_output_publication_lifecycle.py | 152 ++- tests/unit/test_custom_function_lifecycle.py | 127 ++- .../unit/test_czi_brain_axon_cellbody_demo.py | 26 +- tests/unit/test_dataset_source_family.py | 25 +- tests/unit/test_dataset_source_guards.py | 8 +- tests/unit/test_debug_runtime.py | 22 +- ...test_declared_singleton_composition_437.py | 48 +- .../unit/test_derived_role_publication_435.py | 434 +++++--- tests/unit/test_documented_dimensionality.py | 4 +- .../unit/test_experimental_analysis_engine.py | 8 +- .../test_exported_relationship_equivalence.py | 89 +- .../unit/test_exported_table_axis_snapshot.py | 22 +- tests/unit/test_fast_speckle_opening.py | 57 +- tests/unit/test_fiji_viewer_server.py | 22 +- .../unit/test_funcstep_contract_validator.py | 7 +- .../test_function_artifact_materialization.py | 386 +++++-- tests/unit/test_function_artifact_outputs.py | 137 ++- ...st_function_execution_lifecycle_anchors.py | 3 +- tests/unit/test_function_io.py | 7 +- tests/unit/test_function_outputs.py | 164 +-- .../unit/test_function_reference_identity.py | 10 +- ...test_function_runtime_source_projection.py | 197 ++-- .../unit/test_function_step_execution_plan.py | 17 +- .../test_function_step_execution_scope.py | 72 +- tests/unit/test_function_step_transport.py | 98 +- .../test_g5_processing_contract_guards.py | 4 +- tests/unit/test_g7_runtime_protocol_guards.py | 16 +- .../test_gray_to_color_binding_axis_432.py | 159 ++- ..._gray_to_color_payload_axis_compile_432.py | 230 +++-- tests/unit/test_grid_addressed_axis.py | 5 +- tests/unit/test_image_plane_contracts.py | 45 +- .../test_implicit_main_flow_compilation.py | 2 +- tests/unit/test_installed_mcp_demo.py | 30 +- .../unit/test_invocation_contract_provider.py | 4 +- ...nvocation_input_source_context_identity.py | 21 +- .../test_l3_zmqruntime_vocabulary_guards.py | 10 +- tests/unit/test_library_registry_discovery.py | 6 +- .../test_logical_image_output_snapshot.py | 18 +- tests/unit/test_matched_cellprofiler_batch.py | 9 +- .../test_materialization_batch_outcomes.py | 20 +- .../unit/test_materialization_flag_planner.py | 4 +- tests/unit/test_mcp_release_workflow.py | 4 +- tests/unit/test_measurement_dialect.py | 20 +- ...test_measureobjectintensitydistribution.py | 4 +- .../unit/test_named_checkpoint_publication.py | 105 +- .../test_napari_accepted_work_settlement.py | 200 ++-- .../test_napari_roi_manager_integration.py | 40 +- tests/unit/test_napari_streaming_handlers.py | 396 ++++--- tests/unit/test_native_viewer_presentation.py | 25 +- tests/unit/test_neurite_outgrowth.py | 356 +++++-- tests/unit/test_neuroncyto_ii_neurite_demo.py | 24 +- tests/unit/test_nlm_slice_contract.py | 4 +- tests/unit/test_openhcs_adapter.py | 6 +- tests/unit/test_openhcs_measured_run.py | 10 +- tests/unit/test_orchestrator_lane_planning.py | 8 +- .../test_path_planner_axis_source_scope.py | 4 +- .../unit/test_path_planner_materialization.py | 169 +-- tests/unit/test_payload_axis_compile_gate.py | 48 +- tests/unit/test_persisted_image_metadata.py | 69 +- .../test_persisted_roi_only_publication.py | 106 +- .../unit/test_persisted_semantic_metadata.py | 28 +- tests/unit/test_plane_nlm_declaration.py | 67 +- tests/unit/test_plane_store_sources.py | 8 +- tests/unit/test_plate_image_inventory.py | 4 +- tests/unit/test_primary_object_diagnostics.py | 169 ++- tests/unit/test_process_local_cache.py | 5 +- .../test_processor_operations_written_once.py | 19 +- tests/unit/test_pycodify_formatters.py | 24 +- ...gistered_configuration_source_readiness.py | 26 +- .../test_rescale_stream_plane_declaration.py | 192 +++- tests/unit/test_returned_output_abi.py | 6 +- .../test_runner_cellprofiler_compatibility.py | 4 +- .../test_runtime_adapter_source_artifacts.py | 65 +- .../unit/test_runtime_context_declaration.py | 41 +- tests/unit/test_runtime_equivalence.py | 10 +- .../unit/test_runtime_equivalence_package.py | 9 +- .../unit/test_runtime_execution_validation.py | 3 +- .../test_runtime_image_singleton_alignment.py | 13 +- tests/unit/test_runtime_slice_projection.py | 59 +- tests/unit/test_runtime_value_store.py | 430 +++++--- tests/unit/test_runtime_values.py | 144 +-- .../test_saved_source_label_plane_domain.py | 194 +++- ...t_selected_plane_stream_materialization.py | 61 +- ...test_singleton_rgb_label_invocation_477.py | 147 ++- tests/unit/test_smoke_installed_gui.py | 4 +- tests/unit/test_snapshot_deadline.py | 24 +- tests/unit/test_source_binding_workspace.py | 299 ++++-- tests/unit/test_source_bindings.py | 110 +- ..._source_bound_matched_image_set_anchors.py | 36 +- ..._source_provenance_constructor_defaults.py | 5 +- tests/unit/test_source_spatial_domain.py | 7 +- tests/unit/test_source_tile_geometry.py | 41 +- tests/unit/test_stream_component_semantics.py | 26 +- tests/unit/test_streaming_service.py | 60 +- tests/unit/test_u1_session_guards.py | 39 +- .../test_unified_registry_runtime_payloads.py | 41 +- tests/unit/test_viewer_receipt_replay.py | 203 ++-- .../test_viewer_result_element_selection.py | 32 +- tests/unit/test_viewer_spatial_sample.py | 4 +- tests/unit/test_well_throughput_scaling.py | 20 +- ...test_zarr_input_conversion_subdirectory.py | 10 +- tests/unit/test_zmq_compiled_pipeline_run.py | 26 +- tests/unit/test_zmq_execution_server.py | 62 +- tests/unit/test_zmq_progress.py | 25 +- tests/unit/test_zmq_submission_timeout.py | 16 +- validation/public_two_vector_388.py | 967 +++++++++++++----- 454 files changed, 16436 insertions(+), 7379 deletions(-) diff --git a/benchmark/adapters/cellprofiler.py b/benchmark/adapters/cellprofiler.py index fc3b77a141..178b19021f 100644 --- a/benchmark/adapters/cellprofiler.py +++ b/benchmark/adapters/cellprofiler.py @@ -827,8 +827,8 @@ def prepare( selected_input_dir = selected_input_path.input_dir() file_list_path = None if requires_flat_input_dir: - selected_source_universe = selected_source_universe.materialize_flat_input_dir( - selected_input_dir + selected_source_universe = ( + selected_source_universe.materialize_flat_input_dir(selected_input_dir) ) else: file_list_path = self._write_file_list( @@ -1285,7 +1285,8 @@ def native_cellprofiler_partition_filter_scope_slug( filter_slug = _scope_slug_fragment(partition_filter) else: filter_slug = ( - "_".join(_scope_slug_fragment(value) for value in partition_filter) or "empty" + "_".join(_scope_slug_fragment(value) for value in partition_filter) + or "empty" ) return f"wells_{partition_filter_config.partition_filter_mode.value}_{filter_slug}" diff --git a/benchmark/agent_validation/mcp_attempt_recorder.py b/benchmark/agent_validation/mcp_attempt_recorder.py index f8f5efc98d..c607550efc 100644 --- a/benchmark/agent_validation/mcp_attempt_recorder.py +++ b/benchmark/agent_validation/mcp_attempt_recorder.py @@ -253,9 +253,7 @@ def _derive_evidence( receipt_paths.append(receipt.payload_path) successful_results.extend(command_results) tool_names = tuple(result.tool for result in successful_results) - declarations = tuple( - get_agent_capability(tool_name) for tool_name in tool_names - ) + declarations = tuple(get_agent_capability(tool_name) for tool_name in tool_names) requirements: set[DslRequirement] = set() explanations: dict[DslRequirement, str] = {} diff --git a/benchmark/cellprofiler_benchmark_cli.py b/benchmark/cellprofiler_benchmark_cli.py index 7a149cf0d0..22d3832e21 100644 --- a/benchmark/cellprofiler_benchmark_cli.py +++ b/benchmark/cellprofiler_benchmark_cli.py @@ -321,7 +321,9 @@ def run(self, args: argparse.Namespace) -> int: ) return 130 except MeasuredRunEnded as ended: - print(json.dumps(to_jsonable(ended.row), sort_keys=True), file=sys.stderr) + print( + json.dumps(to_jsonable(ended.row), sort_keys=True), file=sys.stderr + ) raise RuntimeError( f"Ordinary pipeline run did not complete: {ended}" ) from ended @@ -824,7 +826,9 @@ def configure( required=True, help="MODE_LABEL=qualified_summary.csv; repeat for measured well/worker modes.", ) - parser.add_argument("--scope", choices=("execution", "total", "amortization"), required=True) + parser.add_argument( + "--scope", choices=("execution", "total", "amortization"), required=True + ) parser.add_argument("--output-dir", type=Path, required=True) return parser diff --git a/benchmark/control_service.py b/benchmark/control_service.py index 9c5783fe0b..2461723171 100644 --- a/benchmark/control_service.py +++ b/benchmark/control_service.py @@ -48,7 +48,9 @@ def finalize_measured_run( """Write benchmark evidence from the ordinary job's exact completion.""" if self._session is None: - raise RuntimeError("Measured run finalization requires the OpenHCS session.") + raise RuntimeError( + "Measured run finalization requires the OpenHCS session." + ) from benchmark.openhcs_measured_run import ( measured_endpoint_provenance, retain_measured_openhcs_completion, diff --git a/benchmark/equivalence/object_label_measurements.py b/benchmark/equivalence/object_label_measurements.py index bbe8e8a5e2..c6386a5a2f 100644 --- a/benchmark/equivalence/object_label_measurements.py +++ b/benchmark/equivalence/object_label_measurements.py @@ -63,6 +63,7 @@ RuntimeMeasurementFeatureSemanticMarker, ) from openhcs.core.runtime_stores import StoredRuntimeValue + if TYPE_CHECKING: pass from types import MappingProxyType @@ -77,7 +78,6 @@ ObjectLabelInstanceDomains, ) - RuntimeObjectValuesByLabel = dict[ RuntimeMeasurementFeatureKey, dict[ObjectInstanceKey, float], @@ -572,10 +572,6 @@ def _object_location_mean_feature_fact( return ((key, runtime_cell_signature(str(float(np.mean(finite_values))), policy)),) - - - - @dataclass(frozen=True, slots=True) class RuntimeObjectLabelMeasurementAuthority: """Nominal object-label ownership before primary-row fallback is allowed.""" diff --git a/benchmark/equivalence/runtime.py b/benchmark/equivalence/runtime.py index 421f931fbf..a1cdf60fb1 100644 --- a/benchmark/equivalence/runtime.py +++ b/benchmark/equivalence/runtime.py @@ -1655,7 +1655,7 @@ def from_output_snapshots( for value in table.measurement_image_number_values( policy.measurement_dialect ) - ) + ), ) identity = tuple( table.measurement_input_key(policy, image_offset) diff --git a/benchmark/equivalence/table_snapshots.py b/benchmark/equivalence/table_snapshots.py index 419c4e2d9f..6058b5fdb5 100644 --- a/benchmark/equivalence/table_snapshots.py +++ b/benchmark/equivalence/table_snapshots.py @@ -460,10 +460,10 @@ def _subject_for_columns( normalized_name = normalize_runtime_identifier(self.path.stem) if normalized_name == _scope_token(dialect, MeasurementScope.RUN): return MeasurementSubject(MeasurementScope.RUN) - if ( - normalized_name != _scope_token(dialect, MeasurementScope.SAMPLE) - and normalized_header - & frozenset(MeasurementRowAxisField.object_id_field_names()) + if normalized_name != _scope_token( + dialect, MeasurementScope.SAMPLE + ) and normalized_header & frozenset( + MeasurementRowAxisField.object_id_field_names() ): return self._object_subject( self.path.stem, diff --git a/benchmark/openhcs_measured_run.py b/benchmark/openhcs_measured_run.py index d13287a6d7..c472c90ded 100644 --- a/benchmark/openhcs_measured_run.py +++ b/benchmark/openhcs_measured_run.py @@ -370,9 +370,7 @@ def retain_measured_openhcs_completion( if expected_axis_count is not None and expected_axis_count < 1: raise ValueError("Expected axis count must be positive when declared.") - auxiliary_params = ZMQAuxiliaryExecutionParams.from_transport( - request.config_params - ) + auxiliary_params = ZMQAuxiliaryExecutionParams.from_transport(request.config_params) observation_export_path = auxiliary_params.runtime_observation_export_path if observation_export_path is None: raise ValueError("Measured OpenHCS runs require runtime observation export.") diff --git a/benchmark/session_measured_run.py b/benchmark/session_measured_run.py index fd44e217d0..4953822d56 100644 --- a/benchmark/session_measured_run.py +++ b/benchmark/session_measured_run.py @@ -70,7 +70,9 @@ def add(self, plate: Path, execution_plate: Path | None) -> str: AddDatasets, DatasetRootsRequest( roots=(str(plate),), - execution_root=None if execution_plate is None else str(execution_plate), + execution_root=( + None if execution_plate is None else str(execution_plate) + ), ), ) (self.scope_id,) = result.target_scope_ids @@ -79,7 +81,9 @@ def add(self, plate: Path, execution_plate: Path | None) -> str: def set_pipeline_source(self, source: str) -> None: self._require( SetDatasetPipeline, - DatasetPipelineSourceRequest(scope_id=self._scope_id, pipeline_source=source), + DatasetPipelineSourceRequest( + scope_id=self._scope_id, pipeline_source=source + ), ) def initialize(self) -> DatasetRowState: diff --git a/benchmark/well_throughput_scaling.py b/benchmark/well_throughput_scaling.py index 3f1d117bfa..a3dca4b5a4 100644 --- a/benchmark/well_throughput_scaling.py +++ b/benchmark/well_throughput_scaling.py @@ -706,7 +706,9 @@ def generate_core_scaling_figures( for row in metric_rows ), filename_stem="03_core_scaling_average_with_pipeline_points_speedup", - methods=methods, output_dir=self.output_dir, output_formats=self.output_formats, + methods=methods, + output_dir=self.output_dir, + output_formats=self.output_formats, title="Average execution speedup by core count", ylabel="Execution speedup vs CellProfiler (x)", value_key="speedup", @@ -737,7 +739,9 @@ def generate_core_scaling_figures( outputs.extend( FIGURE_STYLE.generate_average_point_figures( metric_rows, - methods=methods, output_dir=self.output_dir, output_formats=self.output_formats, + methods=methods, + output_dir=self.output_dir, + output_formats=self.output_formats, filename_stem="04_core_scaling_average_with_pipeline_points_ram", title="Average RAM usage by core count", ylabel="Peak process-tree RSS (MB)", @@ -2111,7 +2115,9 @@ def run_case_well_throughput( use_threading=None, multiprocessing_start_method=None, materialize_runtime_artifacts=False, - partition_filter_config=LazyPartitionFilterConfig(partition_filter=list(well_ids)), + partition_filter_config=LazyPartitionFilterConfig( + partition_filter=list(well_ids) + ), path_planning_config=LazyPathPlanningConfig( partition_filter=0, global_output_folder=output_root, @@ -3144,7 +3150,10 @@ def _replicate_source_binding_workspace_wells( if projection.address is not None ) ) - if source_partition_filter is None or source_partition_filter.partition_filter is None: + if ( + source_partition_filter is None + or source_partition_filter.partition_filter is None + ): selected_source_wells = available_source_wells else: selected_source_wells = tuple( @@ -3242,7 +3251,9 @@ def _replicate_source_binding_workspace_wells( ) ) main_metadata[FIELDS.IMAGE_FILES] = expanded_image_files - main_metadata[Microscopy.Well.metadata_collection_field] = {well_id: None for well_id in target_wells} + main_metadata[Microscopy.Well.metadata_collection_field] = { + well_id: None for well_id in target_wells + } AtomicMetadataWriter().replace_subdirectory_metadata( metadata_path, FIELDS.DEFAULT_SUBDIRECTORY, main_metadata ) diff --git a/openhcs/agent/capabilities.py b/openhcs/agent/capabilities.py index 18482e2405..5cf4ac9586 100644 --- a/openhcs/agent/capabilities.py +++ b/openhcs/agent/capabilities.py @@ -461,7 +461,8 @@ class NonExpertCapabilitySurfaceMixin: def includes(self, capability: type["AgentCapabilityDeclaration"]) -> bool: return ( capability.exposition.visibility is not CapabilityVisibility.EXPERT - and capability.exposition.role not in (CapabilityRole.EXPERT, CapabilityRole.FALLBACK) + and capability.exposition.role + not in (CapabilityRole.EXPERT, CapabilityRole.FALLBACK) and super().includes(capability) ) @@ -472,8 +473,9 @@ class WorkflowGroupCapabilitySurfaceMixin: workflow_groups: ClassVar[frozenset[CapabilityWorkflowGroup]] def includes(self, capability: type["AgentCapabilityDeclaration"]) -> bool: - return capability.exposition.workflow_group in self.workflow_groups and super().includes( - capability + return ( + capability.exposition.workflow_group in self.workflow_groups + and super().includes(capability) ) @@ -654,7 +656,9 @@ def require_agent_type_contract(contract: AgentContract | None) -> type: def _keyword_parameter(name: str, annotation: object, default: object) -> Parameter: - return Parameter(name, Parameter.KEYWORD_ONLY, default=default, annotation=annotation) + return Parameter( + name, Parameter.KEYWORD_ONLY, default=default, annotation=annotation + ) class AgentCapabilityInvocation(ABC): @@ -786,7 +790,6 @@ class AgentResourceInvocationMixin: kind: ClassVar[CapabilityKind] = CapabilityKind.RESOURCE - def bind_mcp( self, declaration: type[AgentCapabilityDeclaration], @@ -1036,8 +1039,12 @@ def tool_parameters(self, declaration, binder): ) def invoke(self, declaration, binder, arguments): - return super().invoke(declaration, binder, arguments).as_jsonable( - compact_actions=bool(arguments["compact_actions"]), + return ( + super() + .invoke(declaration, binder, arguments) + .as_jsonable( + compact_actions=bool(arguments["compact_actions"]), + ) ) @@ -1152,8 +1159,6 @@ class AgentConfigPatchServiceInvocation( __slots__ = () - - class AgentConnectionRequestServiceInvocation( AgentUiConnectionMixin, AgentFromFieldsInputMixin, @@ -1162,7 +1167,6 @@ class AgentConnectionRequestServiceInvocation( __slots__ = () - class AgentViewerWindowConnectionServiceInvocation( AgentViewerWindowConnectionMixin, AgentServiceInvocation, @@ -1195,6 +1199,7 @@ def invoke(self, declaration, binder, arguments): *self.call_arguments(declaration, binder, arguments), ) + @dataclass(frozen=True, slots=True) class AgentCapabilitySearchRequest: """Bounded registry query over declaration-owned capability metadata.""" @@ -2162,7 +2167,9 @@ class ObserveCustomFunctionRegistrationCapability(FunctionCatalogCapability): output_contract = CustomFunctionRegistrationObservation invocation = AgentDataclassRequestServiceInvocation( service=lambda context: context.function_catalog, - method=lambda service, request: service.observe_custom_function_registration(request), + method=lambda service, request: service.observe_custom_function_registration( + request + ), ) @@ -2381,7 +2388,9 @@ class SamplePlateImageCapability(MainThreadProgressCapability, PlatePathCapabili ) -class StreamPlateFilesToViewerCapability(MainThreadProgressCapability, PlatePathCapability): +class StreamPlateFilesToViewerCapability( + MainThreadProgressCapability, PlatePathCapability +): name = "openhcs_stream_plate_files_to_viewer" cli_command = "stream-plate-files" title = "Stream plate files to viewer" @@ -2510,7 +2519,9 @@ class UiSampleSelectedPlateImageCapability(UiSelectedPlateCapability): ) -class UiStreamSelectedPlateFilesToViewerCapability(MainThreadProgressCapability, UiSelectedPlateCapability): +class UiStreamSelectedPlateFilesToViewerCapability( + MainThreadProgressCapability, UiSelectedPlateCapability +): name = "openhcs_ui_stream_selected_plate_files_to_viewer" cli_command = "selected-plate-stream" title = "Stream selected plate files to viewer" diff --git a/openhcs/agent/dto/__init__.py b/openhcs/agent/dto/__init__.py index 144a967fce..a5e35e90c1 100644 --- a/openhcs/agent/dto/__init__.py +++ b/openhcs/agent/dto/__init__.py @@ -16,9 +16,7 @@ "AgentWarning", "RenderedSource", ), - "openhcs.agent.dto.mcp": ( - "McpServerHealthResult", - ), + "openhcs.agent.dto.mcp": ("McpServerHealthResult",), "pyqt_reactive.services.window_snapshot": ( "WindowSnapshotCaptureScope", "WindowSnapshotCaptureSpec", @@ -93,9 +91,7 @@ "SourceWorkspaceSummary", "ViewerStreamingPlanSummary", ), - "openhcs.agent.dto.execution_connection": ( - "ExecutionConnectionSpec", - ), + "openhcs.agent.dto.execution_connection": ("ExecutionConnectionSpec",), "openhcs.agent.dto.pipeline": ( "CreatePipelineRequest", "FunctionSpecRef", diff --git a/openhcs/agent/dto/functions.py b/openhcs/agent/dto/functions.py index 6528f22aee..5045a176b5 100644 --- a/openhcs/agent/dto/functions.py +++ b/openhcs/agent/dto/functions.py @@ -141,21 +141,29 @@ class CustomFunctionRegistrationHandle(FunctionCatalogOperationHandle): @classmethod def from_request(cls, request: CustomFunctionRegistrationRequest) -> Self: return cls( - connection=request.connection, server_identity=request.require_selected_server_identity(), - content_sha256=CustomFunctionSource.content_digest(request.source_code.encode("utf-8")), - function_name=request.function_name, persist=request.persist, + connection=request.connection, + server_identity=request.require_selected_server_identity(), + content_sha256=CustomFunctionSource.content_digest( + request.source_code.encode("utf-8") + ), + function_name=request.function_name, + persist=request.persist, storage_dir=request.storage_dir, ) def require_named_source(self) -> CustomFunctionSource: if self.function_name is None: - raise ValueError("Persisted registration observation requires its admitted name.") + raise ValueError( + "Persisted registration observation requires its admitted name." + ) return CustomFunctionSource(self.function_name, self.content_sha256) def require_storage_dir(self) -> Path: """Decode this handle's admitted source store once for native/path owners.""" if self.storage_dir is None: - raise ValueError("Persisted registration observation requires its admitted store.") + raise ValueError( + "Persisted registration observation requires its admitted store." + ) return Path(self.storage_dir) @@ -182,10 +190,15 @@ class CustomFunctionRegistrationObservation(AgentResultEnvelope): outcome: CustomFunctionRegistrationObservationOutcome = field(init=False) def __post_init__(self) -> None: - object.__setattr__(self, "outcome", CustomFunctionRegistrationObservationOutcome.from_proofs( - published=bool(self.published_sources), persisted=self.persisted_source is not None, - persist=self.handle.persist, - )) + object.__setattr__( + self, + "outcome", + CustomFunctionRegistrationObservationOutcome.from_proofs( + published=bool(self.published_sources), + persisted=self.persisted_source is not None, + persist=self.handle.persist, + ), + ) class FunctionCatalogPreparationOutcome(str, Enum): @@ -274,25 +287,39 @@ class CustomFunctionRegistrationRequest(FunctionCatalogControlRequestABC): connection: ExecutionConnectionSpec = field(default_factory=ExecutionConnectionSpec) function_name: str | None = None storage_dir: str | None = None - admission_policy: AgentPathPolicy | None = field(default=None, repr=False, compare=False) + admission_policy: AgentPathPolicy | None = field( + default=None, repr=False, compare=False + ) server_identity: ProcessIdentity | None = None message_type = FunctionCatalogControlMessageType.REGISTER_CUSTOM @classmethod def from_fields( - cls, *, source_code: str, persist: bool = True, - compact_signature: bool = True, function_name: str | None = None, - storage_dir: str | None = None, host: str = ExecutionConnectionSpec().host, - port: int | None = None, transport_mode: TransportMode | None = None, + cls, + *, + source_code: str, + persist: bool = True, + compact_signature: bool = True, + function_name: str | None = None, + storage_dir: str | None = None, + host: str = ExecutionConnectionSpec().host, + port: int | None = None, + transport_mode: TransportMode | None = None, persistent: bool = ExecutionConnectionSpec().persistent, ) -> Self: """Public boundary: callers cannot supply their own write authority.""" return cls( - source_code=source_code, persist=persist, compact_signature=compact_signature, - function_name=function_name, storage_dir=storage_dir, + source_code=source_code, + persist=persist, + compact_signature=compact_signature, + function_name=function_name, + storage_dir=storage_dir, connection=ExecutionConnectionSpec( - host=host, port=port, transport_mode=transport_mode, persistent=persistent, + host=host, + port=port, + transport_mode=transport_mode, + persistent=persistent, ), ) @@ -303,12 +330,16 @@ def admitted(self, policy: AgentPathPolicy) -> Self: self.connection.require_port("Custom function registration") if self.persist: if self.function_name is None or self.storage_dir is None: - raise ValueError("Persisted registration requires function_name and the endpoint's storage_dir.") + raise ValueError( + "Persisted registration requires function_name and the endpoint's storage_dir." + ) root = Path(self.storage_dir).expanduser() if not root.is_absolute(): raise ValueError("Custom function storage_dir must be absolute.") policy.assert_writable(root) - policy.assert_writable(CustomFunctionManager.source_path_for_name(root, self.function_name)) + policy.assert_writable( + CustomFunctionManager.source_path_for_name(root, self.function_name) + ) return replace(self, admission_policy=policy) def require_selected_server_identity(self) -> ProcessIdentity: @@ -361,12 +392,15 @@ def require_request(self, request: CustomFunctionRegistrationRequest) -> None: expected_root = Path(request.storage_dir).expanduser() expected_file = CustomFunctionManager.source_path_for_name( - expected_root, request.function_name, + expected_root, + request.function_name, ) if ( - Path(self.storage_dir).resolve(strict=False) != expected_root.resolve(strict=False) + Path(self.storage_dir).resolve(strict=False) + != expected_root.resolve(strict=False) or self.source_file_path is None - or Path(self.source_file_path).resolve(strict=False) != expected_file.resolve(strict=False) + or Path(self.source_file_path).resolve(strict=False) + != expected_file.resolve(strict=False) ): raise ValueError( f"Selected endpoint owns custom storage {self.storage_dir}, " @@ -375,11 +409,15 @@ def require_request(self, request: CustomFunctionRegistrationRequest) -> None: ) def require_result( - self, request: CustomFunctionRegistrationRequest, + self, + request: CustomFunctionRegistrationRequest, result: CustomFunctionRegistrationResult, ) -> None: """Require the returned mutation receipt to identify the admitted owner.""" - if result.server_identity != self.server_identity or result.connection != request.connection: + if ( + result.server_identity != self.server_identity + or result.connection != request.connection + ): raise ValueError("Registration returned a different execution owner.") if result.persisted != request.persist: raise ValueError("Registration returned a different persistence policy.") @@ -387,7 +425,9 @@ def require_result( result.storage_dir != self.storage_dir or result.source_file_paths != (self.source_file_path,) ): - raise ValueError("Registration returned a different persistence destination.") + raise ValueError( + "Registration returned a different persistence destination." + ) class FunctionParameterSource(str, Enum): @@ -747,20 +787,32 @@ class CustomFunctionRegistrationResult(AgentResultEnvelope): @classmethod def uncertain( - cls, request: CustomFunctionRegistrationRequest, error: Exception, + cls, + request: CustomFunctionRegistrationRequest, + error: Exception, ) -> Self: """Preserve the actual cause and same-owner read-only recovery identity.""" handle = CustomFunctionRegistrationHandle.from_request(request) - cause = error.to_agent_error() if isinstance(error, AgentFacingErrorMixin) else AgentError.from_exception( - "custom_function_registration_uncertain", error, - hint="Preserve the original request. Observe this handle; do not replay registration. Absence is not proof of no mutation.", + cause = ( + error.to_agent_error() + if isinstance(error, AgentFacingErrorMixin) + else AgentError.from_exception( + "custom_function_registration_uncertain", + error, + hint="Preserve the original request. Observe this handle; do not replay registration. Absence is not proof of no mutation.", + ) ) return cls( - schema_version=SCHEMA_VERSION, persisted=request.persist, - storage_dir=request.storage_dir, connection=request.connection, - server_identity=request.server_identity, observation_handle=handle, + schema_version=SCHEMA_VERSION, + persisted=request.persist, + storage_dir=request.storage_dir, + connection=request.connection, + server_identity=request.server_identity, + observation_handle=handle, errors=(cause,), - next_steps=("Observe this observation_handle with openhcs_get_custom_function_registration_status; do not replay source.",), + next_steps=( + "Observe this observation_handle with openhcs_get_custom_function_registration_status; do not replay source.", + ), ) diff --git a/openhcs/agent/dto/plate.py b/openhcs/agent/dto/plate.py index 3341c0459e..44a1af39c4 100644 --- a/openhcs/agent/dto/plate.py +++ b/openhcs/agent/dto/plate.py @@ -30,7 +30,6 @@ SyntheticPlateFormat, ) - COMPONENT_FILTERS_DESCRIPTION = ( "Accepted values per axis name of the active axis family, for example " '{"well": ["A01", "B02"]}; a file matches when every named axis holds one ' @@ -178,7 +177,9 @@ class SelectedPlateFileFilterOptions(SelectedPlateTargetOptions): path_contains: str | None = None component_filters: ComponentFilters = field( default=ComponentFilters(), - metadata={AGENT_PARAMETER_DESCRIPTION_METADATA_KEY: COMPONENT_FILTERS_DESCRIPTION}, + metadata={ + AGENT_PARAMETER_DESCRIPTION_METADATA_KEY: COMPONENT_FILTERS_DESCRIPTION + }, ) limit: int = 1 @@ -349,7 +350,9 @@ class PlateFileQueryRequest(DatasetTarget): path_contains: str | None = None component_filters: ComponentFilters = field( default=ComponentFilters(), - metadata={AGENT_PARAMETER_DESCRIPTION_METADATA_KEY: COMPONENT_FILTERS_DESCRIPTION}, + metadata={ + AGENT_PARAMETER_DESCRIPTION_METADATA_KEY: COMPONENT_FILTERS_DESCRIPTION + }, ) offset: int = 0 limit: int = 50 @@ -419,7 +422,9 @@ class PlateFileStreamRequest(DatasetTarget): path_contains: str | None = None component_filters: ComponentFilters = field( default=ComponentFilters(), - metadata={AGENT_PARAMETER_DESCRIPTION_METADATA_KEY: COMPONENT_FILTERS_DESCRIPTION}, + metadata={ + AGENT_PARAMETER_DESCRIPTION_METADATA_KEY: COMPONENT_FILTERS_DESCRIPTION + }, ) limit: int = 1 viewer_config_key: str = NapariViewer.config_key @@ -831,7 +836,9 @@ class SyntheticPlateGenerationRequest: shared_cell_fraction: float = ( SYNTHETIC_PLATE_GENERATION_PROFILE.default_request.shared_cell_fraction ) - partition_values: tuple[str, ...] = SYNTHETIC_PLATE_GENERATION_PROFILE.default_request.partition_values + partition_values: tuple[str, ...] = ( + SYNTHETIC_PLATE_GENERATION_PROFILE.default_request.partition_values + ) format: SyntheticPlateFormat = ( SYNTHETIC_PLATE_GENERATION_PROFILE.default_request.format ) diff --git a/openhcs/agent/dto/ui_bridge.py b/openhcs/agent/dto/ui_bridge.py index 638bba5c3a..736689ecfc 100644 --- a/openhcs/agent/dto/ui_bridge.py +++ b/openhcs/agent/dto/ui_bridge.py @@ -744,10 +744,6 @@ class UiCodeDocument( errors: tuple[AgentError, ...] = () - - - - @dataclass(frozen=True, kw_only=True) class UiStateSurfaceEnvelope(AgentResultEnvelope): """Shared envelope for typed UI state-surface payloads.""" diff --git a/openhcs/agent/services/artifact_plan_inspection_service.py b/openhcs/agent/services/artifact_plan_inspection_service.py index ee16315ac3..6f08a06355 100644 --- a/openhcs/agent/services/artifact_plan_inspection_service.py +++ b/openhcs/agent/services/artifact_plan_inspection_service.py @@ -93,7 +93,8 @@ def compile(self, request: CompileInspectionInput) -> CompileInspectionResult: result = self._compile(request) except Exception: EndpointStartupStatus( - EndpointStartupPhase.FAILED, "Source inspection compiler failed", + EndpointStartupPhase.FAILED, + "Source inspection compiler failed", ).publish() raise EndpointStartupStatus( diff --git a/openhcs/agent/services/function_catalog_service.py b/openhcs/agent/services/function_catalog_service.py index f406b1429e..f442844996 100644 --- a/openhcs/agent/services/function_catalog_service.py +++ b/openhcs/agent/services/function_catalog_service.py @@ -235,7 +235,8 @@ def search_match( (CatalogSearchRank.SUMMARY_CONTAINS, entry.summary or ""), ( CatalogSearchRank.DOC_CONTAINS, - PARAMETER_DOCUMENTATION_POLICY.detail_doc(metadata.func, metadata.doc) or "", + PARAMETER_DOCUMENTATION_POLICY.detail_doc(metadata.func, metadata.doc) + or "", ), ): score = self._text_score(value) @@ -486,7 +487,9 @@ def agent_parameter_names( ) compile_only = tuple( name - for name in PipelineInvocationContractProviderAuthority.compile_time_parameter_names(contract) + for name in PipelineInvocationContractProviderAuthority.compile_time_parameter_names( + contract + ) if name not in runtime_names ) return tuple(dict.fromkeys((*ordinary, *compile_only))) @@ -606,7 +609,8 @@ def register_custom_function( @abstractmethod def observe_custom_function_registration( - self, handle: CustomFunctionRegistrationHandle, + self, + handle: CustomFunctionRegistrationHandle, ) -> CustomFunctionRegistrationObservation: """Observe exact existing publication/persistence without evaluating source.""" @@ -710,17 +714,21 @@ def register_custom_function( request.admitted(self._path_policy) server_identity = request.require_server_identity() manager = custom_function_manager.CustomFunctionManager(create_storage=False) - if request.persist and manager.storage_dir.resolve(strict=False) != Path(request.storage_dir).resolve(strict=False): + if request.persist and manager.storage_dir.resolve(strict=False) != Path( + request.storage_dir + ).resolve(strict=False): raise ValueError( f"Selected endpoint owns custom storage {manager.storage_dir}, " f"not requested {request.storage_dir}; no source was evaluated." ) + def admit_write(path: Path) -> Path: self._path_policy.assert_writable(path) return request.admission_policy.assert_writable(path) registered_functions = manager.register_from_code( - request.source_code, persist=request.persist, + request.source_code, + persist=request.persist, expected_function_name=request.function_name, write_admission=admit_write, ) @@ -764,33 +772,53 @@ def admit_write(path: Path) -> Path: ) def observe_custom_function_registration( - self, handle: CustomFunctionRegistrationHandle, + self, + handle: CustomFunctionRegistrationHandle, ) -> CustomFunctionRegistrationObservation: from openhcs.agent.dto.common import AgentWarning - from openhcs.processing.custom_functions.runtime_registry import CustomFunctionRuntimeRegistry + from openhcs.processing.custom_functions.runtime_registry import ( + CustomFunctionRuntimeRegistry, + ) handle.require_current_owner() published = CustomFunctionRuntimeRegistry.published_sources_for_content( - handle.content_sha256, function_name=handle.function_name, + handle.content_sha256, + function_name=handle.function_name, ) persisted = None warnings = () if handle.persist: - manager = custom_function_manager.CustomFunctionManager(create_storage=False) - if handle.require_storage_dir().resolve(strict=False) != manager.storage_dir.resolve(strict=False): - raise ValueError("Registration observation reached a different native source store.") + manager = custom_function_manager.CustomFunctionManager( + create_storage=False + ) + if handle.require_storage_dir().resolve( + strict=False + ) != manager.storage_dir.resolve(strict=False): + raise ValueError( + "Registration observation reached a different native source store." + ) source = handle.require_named_source() - self._path_policy.assert_readable_location(manager.source_path_for_name(manager.storage_dir, source.function_name)) + self._path_policy.assert_readable_location( + manager.source_path_for_name(manager.storage_dir, source.function_name) + ) try: manager.require_source(source) except (OSError, RuntimeError) as error: - warnings = (AgentWarning("registration_source_not_observed", str(error), - "Missing/changed current source is not proof of no original mutation; do not replay."),) + warnings = ( + AgentWarning( + "registration_source_not_observed", + str(error), + "Missing/changed current source is not proof of no original mutation; do not replay.", + ), + ) else: persisted = source return CustomFunctionRegistrationObservation( - schema_version=SCHEMA_VERSION, handle=handle, published_sources=published, - persisted_source=persisted, warnings=warnings, + schema_version=SCHEMA_VERSION, + handle=handle, + published_sources=published, + persisted_source=persisted, + warnings=warnings, ) def custom_function_registration_destination( @@ -802,8 +830,13 @@ def custom_function_registration_destination( return CustomFunctionRegistrationDestination( storage_dir=str(storage_dir), source_file_path=( - str(manager_type.source_path_for_name(storage_dir, request.function_name)) - if request.function_name is not None else None + str( + manager_type.source_path_for_name( + storage_dir, request.function_name + ) + ) + if request.function_name is not None + else None ), ) @@ -934,8 +967,12 @@ def _search_projections( metadata.func, contract ) entry = self._entry( - function_id, metadata, SignatureView.FULL, SummaryView.FULL, - contract=contract, visible_signature=signature, + function_id, + metadata, + SignatureView.FULL, + SummaryView.FULL, + contract=contract, + visible_signature=signature, ) projections.append( CatalogSearchProjection( @@ -951,7 +988,8 @@ def _search_projections( signature, entry.name, SignatureView.COMPACT ), summary=( - None if entry.summary is None + None + if entry.summary is None else _bounded_summary(entry.summary, SummaryView.COMPACT) ), ), @@ -1252,7 +1290,8 @@ def _runtime_contract_summary( return FunctionRuntimeContractSummary( callable_kind=callable_kind, processing_contract=( - None if contract.processing_contract is None + None + if contract.processing_contract is None else contract.processing_contract.key ), runtime_bound_parameters=contract.runtime_bound_parameters, diff --git a/openhcs/agent/services/plate_inspection_service.py b/openhcs/agent/services/plate_inspection_service.py index df564102cf..e417adcb47 100644 --- a/openhcs/agent/services/plate_inspection_service.py +++ b/openhcs/agent/services/plate_inspection_service.py @@ -63,6 +63,7 @@ ) from openhcs.core.axes import Axis, AxisFamily from openhcs.core.dataset_sources.choice import AutoDetectedSource, DatasetSourceChoice + if TYPE_CHECKING: from openhcs.core.components.parser_metaprogramming import ( FilenameParseResult, @@ -168,10 +169,7 @@ def candidates( if DatasetSourceChoice.named(requested_source_format) is not AutoDetectedSource: return () - if ( - type(selected_handler).source_selection_role() - is not BroadStoreSource - ): + if type(selected_handler).source_selection_role() is not BroadStoreSource: return () try: @@ -188,10 +186,7 @@ def candidates( candidates: list[PlateInspectionHandlerCandidate] = [] for handler_type in DatasetSource.__registry__.values(): - if ( - handler_type.source_selection_role() - is not FormatSpecificSource - ): + if handler_type.source_selection_role() is not FormatSpecificSource: continue candidate = cls._candidate( handler_type=handler_type, @@ -477,7 +472,9 @@ class PlateInspectionMutableParsedComponents: """Mutable accumulator before parsed component values are frozen.""" values: dict[type[Axis], set[str]] = field( - default_factory=lambda: {component: set() for component in AxisFamily.active().axes} + default_factory=lambda: { + component: set() for component in AxisFamily.active().axes + } ) def add_parsed_filename_values(self, parsed: "FilenameParseResult") -> None: @@ -873,7 +870,8 @@ def for_handler( selection_message = type(handler).source_selection_guidance() initialization_message = ( f"initialize with explicit source_format={ingestion_owner!r}" - if DatasetSourceChoice.named(requested_source_format) is not AutoDetectedSource + if DatasetSourceChoice.named(requested_source_format) + is not AutoDetectedSource else "initialize with auto-detection" ) message = ( @@ -1207,7 +1205,8 @@ def _query_files( parser = self._parser( handler, warnings, - warn=query_kind is not PlateFileKind.RESULT or bool(request.component_filters), + warn=query_kind is not PlateFileKind.RESULT + or bool(request.component_filters), ) file_inventory = self._plate_file_inventory_for_query( handler, @@ -1401,7 +1400,9 @@ def _sample_image( record, record.sample( filemanager, - plate_path=handler.metadata_handler.source_workspace_root(plate_path), + plate_path=handler.metadata_handler.source_workspace_root( + plate_path + ), request=ImageSamplingRequest( origin_yx=(request.y, request.x), shape_yx=(request.height, request.width), @@ -1573,10 +1574,7 @@ def _inspect( ) filemanager = self._filemanager_factory.create() - if ( - request.detects_source_format - and request.pattern_format is None - ): + if request.detects_source_format and request.pattern_format is None: result_only = self._result_only_inspection( request=request, available_types=available_types, diff --git a/openhcs/agent/services/plate_streaming_service.py b/openhcs/agent/services/plate_streaming_service.py index db62faf29c..139856757d 100644 --- a/openhcs/agent/services/plate_streaming_service.py +++ b/openhcs/agent/services/plate_streaming_service.py @@ -148,7 +148,9 @@ def stream_files( requested_source_format=request.source_format, detected_source_format=context.source_format, handler_class=type(context.handler).__name__, - parser_class=None if context.parser is None else type(context.parser).__name__, + parser_class=( + None if context.parser is None else type(context.parser).__name__ + ), viewer_config_key=request.viewer_config_key, connection=connection, requested_paths=request.file_paths, @@ -164,9 +166,15 @@ def stream_files( transport_mode=config.transport_mode, persistent=config.persistent, ) - result = replace(result, viewer_type=config.viewer_family.viewer_type(), connection=connection) + result = replace( + result, + viewer_type=config.viewer_family.viewer_type(), + connection=connection, + ) if request.result_directory is not None: - if request.kind is not PlateFileKind.RESULT or bool(request.component_filters): + if request.kind is not PlateFileKind.RESULT or bool( + request.component_filters + ): raise ValueError( "Explicit result-directory streaming requires kind='result' " "and no acquisition-component filter." @@ -304,11 +312,15 @@ def stream_files( require_source_metadata=request.result_directory is not None, ) ) - self._report_progress("Selected artifacts published and viewer settlement completed") + self._report_progress( + "Selected artifacts published and viewer settlement completed" + ) except Exception as exc: return replace( result, - viewer_type=None if config is None else config.viewer_family.viewer_type(), + viewer_type=( + None if config is None else config.viewer_family.viewer_type() + ), connection=connection, errors=(self._stream_error(exc, plate_path=request.plate_path),), ) @@ -327,7 +339,8 @@ def stream_files( def _report_progress(message: str) -> None: """Use the original request-local relay, without a viewer observer/store.""" EndpointStartupStatus( - EndpointStartupPhase.PREPARING_CAPABILITIES, message, + EndpointStartupPhase.PREPARING_CAPABILITIES, + message, ).publish() def _record_stream_status(self, messages: list[str], message: str) -> None: @@ -408,7 +421,8 @@ def _receipt_source_projection( producers.append(producer) plane_domain = ViewerComponentValueDomainPayload.from_ordered_wire_mapping( record.summary.require_plane_components(), - declared_axes=_active_declared_axes(), context="image receipt plane coordinates", + declared_axes=_active_declared_axes(), + context="image receipt plane coordinates", ) planes = SourceImageProvenancePlanes.from_component_domain( path=image_path, diff --git a/openhcs/agent/services/selected_plate_service.py b/openhcs/agent/services/selected_plate_service.py index ffdce1016e..e9521e9091 100644 --- a/openhcs/agent/services/selected_plate_service.py +++ b/openhcs/agent/services/selected_plate_service.py @@ -392,8 +392,7 @@ def selected_plate_row_from_state_surface( matched_rows = tuple( row for row in rows_value - if isinstance(row, Mapping) - and row.get("scope_id") == selected_scope_id + if isinstance(row, Mapping) and row.get("scope_id") == selected_scope_id ) if len(matched_rows) != 1: return None, AgentError( diff --git a/openhcs/agent/services/ui_bridge_service.py b/openhcs/agent/services/ui_bridge_service.py index 4e659db252..5ebc57883b 100644 --- a/openhcs/agent/services/ui_bridge_service.py +++ b/openhcs/agent/services/ui_bridge_service.py @@ -400,7 +400,9 @@ def as_served(cls, result: UiBridgeStatus, binding: UiBridgeConnectionSpec): @classmethod def failed(cls, request, errors): del request - return UiBridgeStatus(schema_version=SCHEMA_VERSION, reachable=False, errors=errors) + return UiBridgeStatus( + schema_version=SCHEMA_VERSION, reachable=False, errors=errors + ) @staticmethod def _unreachable(resolution: "UiBridgeConnectionResolution") -> UiBridgeStatus: @@ -1067,7 +1069,6 @@ def invoke( raise NotImplementedError - class UiBridgeGatewayErrorABC(ABC): """Gateway-originated bridge failure that knows its agent-facing errors.""" @@ -1789,7 +1790,9 @@ def invoke( """Run one operation; failures come back as the operation's own result.""" return operation.respond(self, connection, request) - def resolve(self, connection: UiBridgeConnectionSpec) -> UiBridgeConnectionResolution: + def resolve( + self, connection: UiBridgeConnectionSpec + ) -> UiBridgeConnectionResolution: return self._descriptor_resolver.resolve(connection) def connection_from_args( diff --git a/openhcs/authoring/session/compile_batch.py b/openhcs/authoring/session/compile_batch.py index b0594a3d0d..dcf7aa80e9 100644 --- a/openhcs/authoring/session/compile_batch.py +++ b/openhcs/authoring/session/compile_batch.py @@ -50,7 +50,9 @@ async def compile_datasets(self, scope_ids: tuple[str, ...]) -> None: try: client = await session.connect_client() session.publish( - StatusReported(f"Queueing compilation for {len(scope_ids)} dataset(s)...") + StatusReported( + f"Queueing compilation for {len(scope_ids)} dataset(s)..." + ) ) requests: list[DatasetPipelineRequest] = [] for scope_id in scope_ids: @@ -129,7 +131,9 @@ def on_wait_start(_request, _index, _total) -> None: session.publish(DatasetsChanged()) def on_wait_success(request, _execution_id, index, total) -> None: - session.publish(StatusReported(f"Compiled {index}/{total}: {request.scope_id}")) + session.publish( + StatusReported(f"Compiled {index}/{total}: {request.scope_id}") + ) session.publish(DatasetsChanged()) def on_failure(request, error, _index, _total) -> None: @@ -139,8 +143,12 @@ def on_failure(request, error, _index, _total) -> None: error, exc_info=True, ) - session.batch.mark_terminal(request.scope_id, TerminalExecutionStatus.FAILED) - session.publish(ErrorReported(f"Compile failed for {request.scope_id}: {error}")) + session.batch.mark_terminal( + request.scope_id, TerminalExecutionStatus.FAILED + ) + session.publish( + ErrorReported(f"Compile failed for {request.scope_id}: {error}") + ) session.publish(DatasetsChanged()) return await self._engine.run( @@ -174,7 +182,9 @@ async def _submit(self, client, request: DatasetPipelineRequest) -> str: self._session.set_compiled(request.scope_id, None) return await submit_compile(client, request) - async def _wait(self, client, execution_id: str, request: DatasetPipelineRequest) -> None: + async def _wait( + self, client, execution_id: str, request: DatasetPipelineRequest + ) -> None: try: inspection = await wait_for_compile( client, diff --git a/openhcs/authoring/session/dataset_document.py b/openhcs/authoring/session/dataset_document.py index e1de6c4d07..2b1f68f945 100644 --- a/openhcs/authoring/session/dataset_document.py +++ b/openhcs/authoring/session/dataset_document.py @@ -46,7 +46,9 @@ def from_carrier( default: SelectedAllSelectionMode = SelectedAllSelectionMode.SELECTED, ) -> "DatasetDocumentScope": mode = SelectedAllSelectionMode(carrier.resolved_selection_mode(default)) - return cls.__registry__[mode](selected_scope_ids=tuple(carrier.selected_scope_ids)) + return cls.__registry__[mode]( + selected_scope_ids=tuple(carrier.selected_scope_ids) + ) def require_payload_scope(self, scope_ids: tuple[str, ...]) -> None: if self.selected_scope_ids and frozenset(scope_ids) != frozenset( @@ -114,7 +116,9 @@ def authored_pipeline_config(scope_id: str) -> PipelineConfig: state = ObjectStateRegistry.get_by_scope(scope_id) if state is None or ObjectStateRegistry.get_object(scope_id) is None: - raise ValueError(f"Dataset {scope_id!r} has no registered orchestrator ObjectState.") + raise ValueError( + f"Dataset {scope_id!r} has no registered orchestrator ObjectState." + ) config = state.to_object(update_delegate=False) if not isinstance(config, PipelineConfig): raise TypeError("A dataset ObjectState must rebuild a PipelineConfig.") diff --git a/openhcs/authoring/session/debug_runs.py b/openhcs/authoring/session/debug_runs.py index 2c5622f5a9..891e4aa94a 100644 --- a/openhcs/authoring/session/debug_runs.py +++ b/openhcs/authoring/session/debug_runs.py @@ -134,7 +134,9 @@ async def inspect_runtime(self, *, debug_session_id: str): ) ) self._session.publish( - StatusReported(f"Loaded runtime inspection for session {debug_session_id[:8]}") + StatusReported( + f"Loaded runtime inspection for session {debug_session_id[:8]}" + ) ) return view_model diff --git a/openhcs/authoring/session/events.py b/openhcs/authoring/session/events.py index 1687356af7..8ba45c8760 100644 --- a/openhcs/authoring/session/events.py +++ b/openhcs/authoring/session/events.py @@ -324,4 +324,6 @@ def after( lambda: self._next_sequence - 1 > sequence, timeout=timeout_seconds, ) - return tuple(record for record in self._records if record.sequence > sequence) + return tuple( + record for record in self._records if record.sequence > sequence + ) diff --git a/openhcs/authoring/session/operations/datasets.py b/openhcs/authoring/session/operations/datasets.py index d9030c452f..74fce20b05 100644 --- a/openhcs/authoring/session/operations/datasets.py +++ b/openhcs/authoring/session/operations/datasets.py @@ -148,7 +148,9 @@ class DeleteDatasets(IdleTargetsOperation, HeadlessOperation): operation_id = "delete_datasets" label = "Del" tooltip = "Delete selected datasets" - description = "Removes datasets and their configuration and pipeline from the session." + description = ( + "Removes datasets and their configuration and pipeline from the session." + ) side_effects = ("mutates_dataset_list",) @classmethod @@ -170,7 +172,9 @@ def available(cls, session, request) -> AgentError | None: error = super().available(session, request) if error is not None: return error - pending = [s for s in request.scope_ids if session.has_pending_definition_work(s)] + pending = [ + s for s in request.scope_ids if session.has_pending_definition_work(s) + ] if pending: return AgentError( code="dataset_definition_pending", @@ -179,7 +183,9 @@ def available(cls, session, request) -> AgentError | None: return None -class InitializeDatasets(IdleTargetsOperation, DatasetWorkflowOperation, HeadlessOperation): +class InitializeDatasets( + IdleTargetsOperation, DatasetWorkflowOperation, HeadlessOperation +): operation_id = "initialize_datasets" workflow_name = "INIT" label = "Init" @@ -330,7 +336,9 @@ def available(cls, session, request) -> AgentError | None: code="execution_batch_active", message="An execution batch is already running.", ) - pending = [s for s in request.scope_ids if session.has_pending_definition_work(s)] + pending = [ + s for s in request.scope_ids if session.has_pending_definition_work(s) + ] if pending: return AgentError( code="dataset_definition_pending", @@ -388,7 +396,9 @@ def button_label(cls, session) -> str: def available(cls, session, request) -> AgentError | None: state = session.execution_state if not state.busy: - return AgentError(code="no_execution_running", message="Nothing is running.") + return AgentError( + code="no_execution_running", message="Nothing is running." + ) if not state.run_button_enabled(True): return AgentError( code="execution_stopping", @@ -461,4 +471,3 @@ class ShowDatasetImages(InitializedTargetsOperation, RendererOperation): tooltip = "Browse dataset images and metadata" description = "Opens the image and metadata browser of the selected datasets." side_effects = ("opens_dataset_viewer_window",) - diff --git a/openhcs/authoring/session/operations/pipelines.py b/openhcs/authoring/session/operations/pipelines.py index 66dd730864..8027197a17 100644 --- a/openhcs/authoring/session/operations/pipelines.py +++ b/openhcs/authoring/session/operations/pipelines.py @@ -87,7 +87,9 @@ def available(cls, session, request) -> AgentError | None: return None -class AddPipelineStep(CurrentDatasetRequestMixin, EditablePipelineOperation, RendererOperation): +class AddPipelineStep( + CurrentDatasetRequestMixin, EditablePipelineOperation, RendererOperation +): operation_id = "add_step" label = "Add" tooltip = "Add new pipeline step" diff --git a/openhcs/authoring/session/progress.py b/openhcs/authoring/session/progress.py index 360bd19e82..62f5d35e9a 100644 --- a/openhcs/authoring/session/progress.py +++ b/openhcs/authoring/session/progress.py @@ -193,7 +193,9 @@ def on_progress(self, message: dict) -> None: session.live_measurements.add_notification(live) session.publish(LiveMeasurementAvailable(live)) try: - artifact = RuntimeArtifactAvailableNotification.from_progress_event(event) + artifact = RuntimeArtifactAvailableNotification.from_progress_event( + event + ) except RuntimeArtifactPayloadError as error: logger.warning( "Malformed runtime artifact progress context for " @@ -229,4 +231,3 @@ def _fetch_server_info_snapshot(self) -> ExecutionServerInfo: f"Expected ExecutionServerInfo, got {type(parsed).__name__}" ) return parsed - diff --git a/openhcs/authoring/session/session.py b/openhcs/authoring/session/session.py index 01671cacae..12d38523aa 100644 --- a/openhcs/authoring/session/session.py +++ b/openhcs/authoring/session/session.py @@ -310,7 +310,9 @@ def __init__( self.global_config = global_config self.client = ZMQClientService( config=transport_config, - status_callback=lambda status: self.publish(ServerConnectionChanged(status)), + status_callback=lambda status: self.publish( + ServerConnectionChanged(status) + ), compatibility_callback=lambda compatibility: self.publish( ServerCompatibilityObserved(compatibility) ), @@ -345,7 +347,9 @@ def __init__( self.submission = ExecutionSubmission(self) self.execution_control = ExecutionControl(self) self.debug_runs = DebugRuns(self) - self.progress = ExecutionProgress(self, interval_seconds=progress_interval_seconds) + self.progress = ExecutionProgress( + self, interval_seconds=progress_interval_seconds + ) self._work = AsyncOperationExecutor() self._closed = False @@ -357,7 +361,9 @@ def __init__( def publish(self, event: SessionEvent) -> SessionEventRecord: return self.event_log.publish(event) - def subscribe(self, listener: Callable[[SessionEventRecord], None]) -> Callable[[], None]: + def subscribe( + self, listener: Callable[[SessionEventRecord], None] + ) -> Callable[[], None]: return self.event_log.subscribe(listener) def events_after( @@ -507,7 +513,9 @@ def add_dataset_roots( on that separately prepared root. """ - roots = tuple(self.dataset_access.require_readable(Path(root)) for root in roots) + roots = tuple( + self.dataset_access.require_readable(Path(root)) for root in roots + ) if execution_root is not None: if len(roots) != 1: raise ValueError("A prepared execution root needs exactly one root.") @@ -522,7 +530,9 @@ def add_dataset_roots( ) else: offers = tuple( - offer for root in roots for offer in DatasetScopeKind.offers_for_root(root) + offer + for root in roots + for offer in DatasetScopeKind.offers_for_root(root) ) added = self._add_scopes( tuple(offer.scope for offer in offers), @@ -636,7 +646,9 @@ def _split_dataset_scopes(self) -> None: def delete_datasets(self, scope_ids: Iterable[str]) -> None: doomed = {str(scope_id) for scope_id in scope_ids} - active = sorted(scope_id for scope_id in doomed if self.has_active_work(scope_id)) + active = sorted( + scope_id for scope_id in doomed if self.has_active_work(scope_id) + ) if active: raise RuntimeError( "Cannot delete a dataset with active initialization, compilation, " @@ -704,7 +716,9 @@ def set_pipeline_document(self, scope_id: str, document: PipelineDocument) -> No PipelineObjectStateBinding.commit_plate_state(scope_id) self.pipeline_changed(scope_id) self.publish( - StatusReported(f"Pipeline updated with {len(document.pipeline_steps)} steps") + StatusReported( + f"Pipeline updated with {len(document.pipeline_steps)} steps" + ) ) def set_pipeline_source(self, scope_id: str, source: str) -> None: @@ -748,7 +762,9 @@ def replace_step( edited_step: FunctionStep, ) -> None: self.require_definition_mutation_allowed(scope_id) - PipelineObjectStateBinding.replace_plate_step(scope_id, current_step, edited_step) + PipelineObjectStateBinding.replace_plate_step( + scope_id, current_step, edited_step + ) self.pipeline_changed(scope_id) self.publish(StatusReported(f"Updated step: {edited_step.name}")) @@ -780,7 +796,9 @@ def delete_steps(self, scope_id: str, step_scope_ids: Iterable[str]) -> None: doomed = set(step_scope_ids) self.require_definition_mutation_allowed(scope_id) steps = self.pipeline_steps(scope_id) - kept = [step for step in steps if self.step_scope_id(scope_id, step) not in doomed] + kept = [ + step for step in steps if self.step_scope_id(scope_id, step) not in doomed + ] names = [step.name for step in steps if step not in kept] with ObjectStateRegistry.atomic(f"delete steps {', '.join(names)}"): for step_scope_id in doomed: @@ -883,7 +901,9 @@ def require_definition_mutation_allowed(self, scope_id: str | None = None) -> No "active initialization or compilation. Other datasets remain editable." ) - def require_definition_mutation_allowed_for_object_scope(self, scope_id: str) -> None: + def require_definition_mutation_allowed_for_object_scope( + self, scope_id: str + ) -> None: """Authorize an ObjectState mutation that may belong to a dataset.""" if scope_id == "": @@ -1024,10 +1044,13 @@ async def initialize(scope_id: str) -> None: self.publish(DatasetsChanged()) def run() -> InputWorkspacePreparationResult | None: - ensure_global_config_context(GlobalPipelineConfig, self.global_config) + ensure_global_config_context( + GlobalPipelineConfig, self.global_config + ) workspace = scope.prepare_input_workspace() - if workspace is None and not self.dataset_access.initializes_in_place( - scope.root + if ( + workspace is None + and not self.dataset_access.initializes_in_place(scope.root) ): workspace = self.mirrored_workspace(scope.root) if workspace is not None: @@ -1055,9 +1078,7 @@ def run() -> InputWorkspacePreparationResult | None: ) self.set_dataset_state(scope_id, OrchestratorState.READY) if scope_id == self.current_scope_id or not self.current_scope_id: - self.main_thread.call( - lambda: self._reselect(scope_id) - ) + self.main_thread.call(lambda: self._reselect(scope_id)) done += 1 self.publish(ProgressAdvanced(done)) @@ -1122,7 +1143,9 @@ def _arm_run_deadline( timer = threading.Timer( wait_timeout_ms / 1000, lambda: self.main_thread.post( - lambda: self._run_deadline_passed(batch_number, scope_ids, wait_timeout_ms) + lambda: self._run_deadline_passed( + batch_number, scope_ids, wait_timeout_ms + ) ), ) timer.daemon = True @@ -1207,7 +1230,9 @@ def observation_export_params( return None path = self.dataset_access.require_writable(Path(export_path)) if path.exists(): - raise FileExistsError(f"Runtime observation export path already exists: {path}") + raise FileExistsError( + f"Runtime observation export path already exists: {path}" + ) return ZMQAuxiliaryExecutionParams( runtime_observation_export_path=path, runtime_observation_export_scope=scope, @@ -1241,7 +1266,9 @@ def finish_dataset_execution( ) -> None: """Record one dataset's terminal status (from any thread).""" - self.main_thread.post(lambda: self._finish_dataset_execution(completion, scope_id)) + self.main_thread.post( + lambda: self._finish_dataset_execution(completion, scope_id) + ) def _finish_dataset_execution( self, @@ -1273,7 +1300,9 @@ def _finish_dataset_execution( if status.emit_failure: detail = completion.traceback_text or completion.message separator = ":\n\n" if completion.traceback_text else ": " - self.publish(ErrorReported(f"Execution failed for {scope_id}{separator}{detail}")) + self.publish( + ErrorReported(f"Execution failed for {scope_id}{separator}{detail}") + ) def _add_output_dataset( self, @@ -1309,12 +1338,17 @@ def _add_output_dataset( def finish_batch(self, completed_count: int, failed_count: int) -> None: self.execution_state = ManagerExecutionState.IDLE summary = f"All done: {completed_count} completed, {failed_count} failed" - if completed_count > 1 and self.global_config.analysis_consolidation_config.enabled: + if ( + completed_count > 1 + and self.global_config.analysis_consolidation_config.enabled + ): try: consolidate_batch_results(self) summary += ". Global summary created." except Exception as error: - logger.error("Failed to create global summary: %s", error, exc_info=True) + logger.error( + "Failed to create global summary: %s", error, exc_info=True + ) summary += ". Global summary failed." self.publish(StatusReported(summary)) self.publish(BatchFinished(completed_count, failed_count)) @@ -1347,7 +1381,9 @@ async def run_debug( return self.require_execution_allowed((scope_id,)) - debug_session = DebugSession.create(subject_id=scope_id, command_type=command_type) + debug_session = DebugSession.create( + subject_id=scope_id, command_type=command_type + ) self.debug_sessions[scope_id] = debug_session root = DatasetScope.parse(scope_id).root snapshot_root = (root if root.is_dir() else root.parent) / ".openhcs_debug" @@ -1368,7 +1404,9 @@ async def run_debug( replay_mode=DebugReplayMode.PERSISTENT_PAUSED_WORKER, ) request = dataset_pipeline_request(scope_id, self.global_config) - artifact_id = await self.debug_runs.compile_artifact_id(request, debug_request) + artifact_id = await self.debug_runs.compile_artifact_id( + request, debug_request + ) self.publish(StatusReported(f"Submitting debug run for {scope_id}...")) await self.debug_runs.submit( request, @@ -1430,7 +1468,9 @@ def inspect_debug_snapshot( snapshot_store_backend=context.snapshot_store_backend, axis_id=event.axis_id, ).with_cursor(context.cursor) - elif summary is not None and summary.debug_session_id == context.debug_session_id: + elif ( + summary is not None and summary.debug_session_id == context.debug_session_id + ): self.inspected_debug_sessions.pop(scope_id, None) if snapshot is not None: self.debug_terminal_summaries[scope_id] = summary.with_snapshot( @@ -1480,7 +1520,9 @@ def debug_context(self, scope_id: str) -> DebugSessionProjectionContext: return DebugSessionProjectionContext( target=DebugSessionTargetState( current_plate_scope_id=scope_id, - pipeline_scope_id=PipelineScopeIdentity.from_plate_scope(scope_id).scope_id, + pipeline_scope_id=PipelineScopeIdentity.from_plate_scope( + scope_id + ).scope_id, initialized=self.is_initialized(scope_id), compiled=scope_id in self.compiled, terminal_status=( @@ -1506,4 +1548,3 @@ def current_debug_context(self) -> DebugSessionProjectionContext | None: if not self.current_scope_id: return None return self.debug_context(self.current_scope_id) - diff --git a/openhcs/authoring/session/submission.py b/openhcs/authoring/session/submission.py index da4e2f9b39..d882213f2e 100644 --- a/openhcs/authoring/session/submission.py +++ b/openhcs/authoring/session/submission.py @@ -44,6 +44,7 @@ class SubmittedExecution: request: ZMQExecutionRequestBuilder endpoint: "PongResponse | None" + logger = logging.getLogger(__name__) @@ -74,7 +75,9 @@ async def submit_execution( execution_id = await self._submit( request, compile_artifact_id=compile_artifact_id, - send=lambda: client.submit_prepared_pipeline(prepared, timeout_ms=timeout_ms), + send=lambda: client.submit_prepared_pipeline( + prepared, timeout_ms=timeout_ms + ), label="execution", ) if execution_id is not None: @@ -132,7 +135,9 @@ async def _submit( error_text = response.require_failure_text(f"{label} submission") logger.error("%s submission failed for %s: %s", label, scope_id, error_text) - session.publish(ErrorReported(f"Submission failed for {scope_id}: {error_text}")) + session.publish( + ErrorReported(f"Submission failed for {scope_id}: {error_text}") + ) session.batch.mark_terminal(scope_id, TerminalExecutionStatus.FAILED) session.set_dataset_state(scope_id, OrchestratorState.EXEC_FAILED) return None diff --git a/openhcs/authoring/session/views.py b/openhcs/authoring/session/views.py index 4f52d57c13..6132a3b535 100644 --- a/openhcs/authoring/session/views.py +++ b/openhcs/authoring/session/views.py @@ -174,7 +174,11 @@ def orchestrator_state(self): if terminal is not None: return terminal.orchestrator_state orchestrator = ObjectStateRegistry.get_object(self.scope_id) - return orchestrator.state if isinstance(orchestrator, PipelineOrchestrator) else None + return ( + orchestrator.state + if isinstance(orchestrator, PipelineOrchestrator) + else None + ) @property def execution_id(self) -> str | None: @@ -255,7 +259,9 @@ def state_of(cls, session: "Session") -> DatasetListState: selected = set(session.selected_scope_ids) return DatasetListState( rows=tuple( - cls.row_state(session, row, relations[row.scope_id], row.scope_id in selected) + cls.row_state( + session, row, relations[row.scope_id], row.scope_id in selected + ) for row in rows ), selected_scope_ids=session.selected_scope_ids, @@ -328,7 +334,9 @@ def row_state( def _function_entries(function_spec) -> tuple: if function_spec is None: return () - entries = tuple(function_spec) if isinstance(function_spec, list) else (function_spec,) + entries = ( + tuple(function_spec) if isinstance(function_spec, list) else (function_spec,) + ) functions = [] for entry in entries: function = entry[0] if isinstance(entry, tuple) else entry @@ -393,7 +401,9 @@ def state_of( selected=step_scope_id in selected, dirty=bool(state.dirty_fields) if state is not None else False, default_diff=( - bool(state.signature_diff_fields) if state is not None else False + bool(state.signature_diff_fields) + if state is not None + else False ), description=step.description, debug_pause=step.debug_pause, diff --git a/openhcs/core/aligned_image_payload.py b/openhcs/core/aligned_image_payload.py index c7dd71391d..acf9a7d909 100644 --- a/openhcs/core/aligned_image_payload.py +++ b/openhcs/core/aligned_image_payload.py @@ -83,7 +83,9 @@ def metadata(self) -> ImagePayloadMetadata: return self.value.metadata @classmethod - def for_payload(cls, value: ImagePayload) -> "ImagePayloadSourceSpatialDomainAdapter": + def for_payload( + cls, value: ImagePayload + ) -> "ImagePayloadSourceSpatialDomainAdapter": """Place an image payload in the source domain its metadata declares.""" return cls( value, @@ -226,7 +228,8 @@ def value_in_payload_domain( ), ) materialized_mask = materialized.mask - return materialized.with_pixels(target.extract_source_array( + return materialized.with_pixels( + target.extract_source_array( materialized.data, spatial_axes_yx=self.spatial_axes_yx, ), @@ -238,12 +241,11 @@ def value_in_payload_domain( spatial_axes_yx=self.spatial_axes_yx, ) ), - metadata=metadata,) + metadata=metadata, + ) -class BareArraySourceSpatialDomainAdapter( - ImagePayloadSourceSpatialDomainAdapter -): +class BareArraySourceSpatialDomainAdapter(ImagePayloadSourceSpatialDomainAdapter): """Source-domain adapter for bare pixels, which declare no placement.""" @classmethod @@ -388,9 +390,10 @@ def validate_main_flow_cohort( producer_records: Sequence[ProducedOutputSemantics] | None, ) -> None: """One input cohort has one declared whole-image composition domain.""" - if producer_records and len( - {record.main_flow_plane_axis for record in producer_records} - ) != 1: + if ( + producer_records + and len({record.main_flow_plane_axis for record in producer_records}) != 1 + ): raise ValueError( "One main-flow cohort cannot combine different declared image axes." ) @@ -414,7 +417,9 @@ def from_loaded_images( ): main_data_stack = payloads[0].copied( memory_type=execution_plan.input_memory_type, - device_id=execution_plan.device_id_for(execution_plan.input_memory_type), + device_id=execution_plan.device_id_for( + execution_plan.input_memory_type + ), ) else: metadata_mode = ImagePayloadMetadataCompositionMode.STACK @@ -425,9 +430,7 @@ def from_loaded_images( metadata_mode = ( ImagePayloadMetadataCompositionMode.BUNDLE if declared_axis is None - and len( - {record.output_context for record in producer_records} - ) > 1 + and len({record.output_context for record in producer_records}) > 1 else ImagePayloadMetadataCompositionMode.for_plane_axis( declared_axis or RuntimePlaneAxis.RUNTIME_SLICE ) @@ -438,13 +441,17 @@ def from_loaded_images( ) if metadata_mode is ImagePayloadMetadataCompositionMode.STACK: main_data_stack = stack_image_payloads( - payloads, metadata_mode=metadata_mode, + payloads, + metadata_mode=metadata_mode, memory_type=execution_plan.input_memory_type, - device_id=execution_plan.device_id_for(execution_plan.input_memory_type), + device_id=execution_plan.device_id_for( + execution_plan.input_memory_type + ), ) elif metadata_mode is ImagePayloadMetadataCompositionMode.BUNDLE: main_data_stack = ImagePayloadBundleContext.from_payloads( - tuple(payloads), metadata_mode=metadata_mode, + tuple(payloads), + metadata_mode=metadata_mode, ).compose() return main_data_stack @@ -464,8 +471,13 @@ def with_saved_output_context( and single_output_plane_axis is metadata[0].plane_axis and np.shape(data) == np.shape(payloads[0].data) ): - member = metadata[0].with_current_intensity_from(current_intensity).payload_with( - data, stack_payload.mask, + member = ( + metadata[0] + .with_current_intensity_from(current_intensity) + .payload_with( + data, + stack_payload.mask, + ) ) if member.keeps_whole_as_single_member( declared_plane_axis=single_output_plane_axis, @@ -473,7 +485,8 @@ def with_saved_output_context( return member # Cache the cohort the next step composes from this one saved member. return stack_image_payloads( - (member,), metadata_mode=ImagePayloadMetadataCompositionMode.STACK, + (member,), + metadata_mode=ImagePayloadMetadataCompositionMode.STACK, ) if np.shape(data)[:1] != (len(payloads),): raise ValueError( @@ -487,17 +500,21 @@ def with_saved_output_context( else ImagePayloadMetadataCompositionMode.for_plane_axis(stack_plane_axis) ) output_metadata = ImagePayloadMetadata.compose( - tuple(payloads), mode=mode, source_metadata=tuple( + tuple(payloads), + mode=mode, + source_metadata=tuple( record.with_current_intensity_from(current_intensity, plane_index=index) for index, record in enumerate(metadata) ), ) return output_metadata.payload_with( - data, _stack_image_payload_mask(tuple(payloads), data), + data, + _stack_image_payload_mask(tuple(payloads), data), ) def composition_source_metadata( - self, payloads: Sequence[Any] | None = None, + self, + payloads: Sequence[Any] | None = None, ) -> tuple[ImagePayloadMetadata, ...]: return tuple( self.composition_payload_metadata(payload.metadata) @@ -512,8 +529,10 @@ def composition_payload_metadata( @staticmethod def composition_memory_domain( - payloads: tuple[RuntimeArrayData, ...], *, - memory_type: str | None = None, device_id: int | None = None, + payloads: tuple[RuntimeArrayData, ...], + *, + memory_type: str | None = None, + device_id: int | None = None, ) -> tuple[str, int | None]: """Resolve where a composition is placed: explicitly, else beside its first payload.""" if memory_type is not None: @@ -522,17 +541,24 @@ def composition_memory_domain( return memory_type, MemoryType(memory_type).device_id_of(payloads[0]) def compose( - self, *, memory_type: str | None = None, device_id: int | None = None, + self, + *, + memory_type: str | None = None, + device_id: int | None = None, ) -> Any: # Resolve the destination from the input payloads before reconciling intensity. memory_type, device_id = self.composition_memory_domain( tuple(payload.data for payload in self.composition_payloads), - memory_type=memory_type, device_id=device_id, + memory_type=memory_type, + device_id=device_id, + ) + payloads = ImagePayloadMetadata.intensity_coherent_payloads( + self.composition_payloads ) - payloads = ImagePayloadMetadata.intensity_coherent_payloads(self.composition_payloads) composed = self.compose_unmasked( tuple(payload.data for payload in payloads), - memory_type=memory_type, device_id=device_id, + memory_type=memory_type, + device_id=device_id, ) metadata = ImagePayloadMetadata.compose( payloads, @@ -542,14 +568,21 @@ def compose( return metadata.payload_with(composed, self.compose_mask(composed, metadata)) def compose_unmasked( - self, payloads: tuple[RuntimeArrayData, ...], *, - memory_type: str | None = None, device_id: int | None = None, + self, + payloads: tuple[RuntimeArrayData, ...], + *, + memory_type: str | None = None, + device_id: int | None = None, ) -> RuntimeArrayData: memory_type, device_id = self.composition_memory_domain( - payloads, memory_type=memory_type, device_id=device_id, + payloads, + memory_type=memory_type, + device_id=device_id, ) return stack_runtime_slices( - payloads, memory_type, device_id, + payloads, + memory_type, + device_id, ) def compose_mask(self, composed: Any, metadata: ImagePayloadMetadata) -> Any | None: @@ -592,7 +625,8 @@ def stack_image_payloads( """Stack image payloads in their declared memory domain with full context.""" return ImagePayloadStackContext(image_payloads, metadata_mode).compose( - memory_type=memory_type, device_id=device_id, + memory_type=memory_type, + device_id=device_id, ) @@ -629,9 +663,7 @@ def _stack_image_payload_mask( stacked_mask_shape ): resolved_masks = tuple( - payload.metadata - .mask_domain(slice_domain) - .broadcast_to_data(mask) + payload.metadata.mask_domain(slice_domain).broadcast_to_data(mask) for payload, slice_domain, mask in zip( payloads, output_slice_domains, resolved_masks, strict=True ) @@ -657,10 +689,14 @@ def _complete_image_payload_mask( f"{tuple(np.shape(payload_data))!r}." ) return project_image_mask_to_data_domain( - mask, payload_data, metadata=payload.metadata, + mask, + payload_data, + metadata=payload.metadata, ) return MemoryType(detect_memory_type(payload_data)).ones_like( - payload_data, shape=mask_domain.default_mask_shape(), dtype=bool, + payload_data, + shape=mask_domain.default_mask_shape(), + dtype=bool, ) @@ -825,7 +861,9 @@ def combined_mask(self, composed: Any) -> RuntimeArrayData | None: device_id = memory_type.device_id_of(data) prepared = tuple( memory_type.astype( - MemoryType(detect_memory_type(mask)).convert_to(mask, memory_type, device_id), + MemoryType(detect_memory_type(mask)).convert_to( + mask, memory_type, device_id + ), bool, ) for mask in masks @@ -841,11 +879,15 @@ def combined_mask(self, composed: Any) -> RuntimeArrayData | None: def compose_unmasked( self, payloads: tuple[RuntimeArrayData, ...], - *, memory_type: str | None = None, device_id: int | None = None, + *, + memory_type: str | None = None, + device_id: int | None = None, ) -> RuntimeArrayData: """Compose image payload arrays without mask/metadata wrapping.""" memory_type, device_id = self.composition_memory_domain( - payloads, memory_type=memory_type, device_id=device_id, + payloads, + memory_type=memory_type, + device_id=device_id, ) channel_axes = tuple( metadata.axis_index(ColourAxis, payload) @@ -858,7 +900,9 @@ def compose_unmasked( declared_channel_count = sum(axis is not None for axis in channel_axes) if declared_channel_count in {0, len(payloads)}: return super(ImagePayloadBundleContext, self).compose_unmasked( - payloads, memory_type=memory_type, device_id=device_id, + payloads, + memory_type=memory_type, + device_id=device_id, ) return self.compose_mixed_channel_payloads( payloads, @@ -928,9 +972,17 @@ def compose_mixed_channel_payloads( else target.broadcast_to( target.reshape( payload, - (*payload.shape[:channel_axis], 1, *payload.shape[channel_axis:]), + ( + *payload.shape[:channel_axis], + 1, + *payload.shape[channel_axis:], + ), + ), + ( + *payload.shape[:channel_axis], + channel_count, + *payload.shape[channel_axis:], ), - (*payload.shape[:channel_axis], channel_count, *payload.shape[channel_axis:]), ) ) for payload, axis in zip(prepared_payloads, channel_axes, strict=True) @@ -1094,6 +1146,7 @@ class ImagePayloadSliceStack(ImagePayloadStackComposition, ImagePayload): slices: tuple[Any, ...] slice_contexts: tuple[AlignedImageSliceContext, ...] = () + @classmethod def from_output_slices( cls, @@ -1104,7 +1157,9 @@ def from_output_slices( ) -> "ProducedImageStack": """Wrap produced image slices as a stack without allocating its pixels.""" return ProducedImageStack( - tuple(slices), memory_type=memory_type, plane_axis=plane_axis, + tuple(slices), + memory_type=memory_type, + plane_axis=plane_axis, ) @property @@ -1162,7 +1217,9 @@ def composition_payloads(self) -> tuple[Any, ...]: return self.slices @property - def projected_output_composition_mode(self) -> ImagePayloadMetadataCompositionMode | None: + def projected_output_composition_mode( + self, + ) -> ImagePayloadMetadataCompositionMode | None: """Declare the outer runtime axis retained by projected output members.""" return self.composition_metadata_mode @@ -1173,7 +1230,9 @@ def alignment_slices(self) -> tuple[Any, ...]: return self.slices def map_slices(self, transform: Any) -> "ImagePayloadSliceStack": - return self.with_slices(tuple(transform(image_slice) for image_slice in self.slices)) + return self.with_slices( + tuple(transform(image_slice) for image_slice in self.slices) + ) def output_slices(self) -> tuple[Any, ...]: return tuple(output for output, _context in self.projected_output_slices()) @@ -1195,13 +1254,16 @@ def value_for_slice(self, context: RuntimePlaneAxisValueProjection) -> Any: ) if all( projected is image_slice - for projected, image_slice in zip(projected_slices, self.slices, strict=True) + for projected, image_slice in zip( + projected_slices, self.slices, strict=True + ) ): return self return self.with_slices(projected_slices) def plane_axis_for_output_context( - self, context: AlignedImageSliceContext, + self, + context: AlignedImageSliceContext, ) -> RuntimePlaneAxis | None: """An explicitly aligned stack declares its outer runtime-slice domain.""" return RuntimePlaneAxis.RUNTIME_SLICE @@ -1212,7 +1274,8 @@ def owns_output_surfaces(self) -> bool: return True def main_output_slices( - self, unstack: Any, + self, + unstack: Any, ) -> tuple[tuple[Any, AlignedImageSliceContext | None], ...]: del unstack return tuple(self.projected_output_slices()) @@ -1225,7 +1288,9 @@ def output_stack_copy( device_id: int | None, ) -> RuntimeArrayData | None: return self.copy_projected_output_stack( - projected_outputs, memory_type=memory_type, device_id=device_id, + projected_outputs, + memory_type=memory_type, + device_id=device_id, ) def __post_init__(self) -> None: @@ -1266,7 +1331,10 @@ def with_slices(self, slices: Sequence[Any]) -> "ImagePayloadSliceStack": return type(self)(tuple(slices), self.slice_contexts) def copied( - self, *, memory_type: str, device_id: int | None, + self, + *, + memory_type: str, + device_id: int | None, ) -> "ImagePayloadSliceStack": """Retain aligned member domains while copying them into independent buffers.""" return self.with_slices( @@ -1305,15 +1373,21 @@ def copy_projected_output_stack( self.plane_axis_for_output_context(context) for _payload, context in projected_outputs } - if len(declared_axes) != 1 or len({tuple(np.shape(item)) for item in data}) != 1: + if ( + len(declared_axes) != 1 + or len({tuple(np.shape(item)) for item in data}) != 1 + ): return None if metadata_mode is ImagePayloadMetadataCompositionMode.BUNDLE: return ImagePayloadBundleContext.from_payloads( - payloads, metadata_mode=metadata_mode, + payloads, + metadata_mode=metadata_mode, ).compose() return stack_image_payloads( - payloads, metadata_mode=metadata_mode, - memory_type=memory_type, device_id=device_id, + payloads, + metadata_mode=metadata_mode, + memory_type=memory_type, + device_id=device_id, ) def output_values_for_artifact_specs( @@ -1453,9 +1527,14 @@ class ProducedImageStack(ImagePayloadSliceStack): plane_axis: RuntimePlaneAxis source_metadata: InitVar[ImagePayloadMetadata | None] = None _metadata: ImagePayloadMetadata = field(init=False, repr=False) - _composed_payload: RuntimeArrayData | None = field(default=None, init=False, repr=False) + _composed_payload: RuntimeArrayData | None = field( + default=None, init=False, repr=False + ) _realization_lock: LockType = field( - default_factory=Lock, init=False, repr=False, compare=False, + default_factory=Lock, + init=False, + repr=False, + compare=False, ) def __post_init__(self, source_metadata: ImagePayloadMetadata | None) -> None: @@ -1467,22 +1546,32 @@ def __post_init__(self, source_metadata: ImagePayloadMetadata | None) -> None: masks = tuple(payload.mask for payload in self.slices) present_masks = tuple(mask for mask in masks if mask is not None) if present_masks and len(present_masks) != len(masks): - raise ValueError("Cannot aggregate a mix of masked and unmasked image payloads.") + raise ValueError( + "Cannot aggregate a mix of masked and unmasked image payloads." + ) self._metadata = ( ImagePayloadMetadata.compose( self.slices, - mode=ImagePayloadMetadataCompositionMode.for_plane_axis(self.plane_axis), + mode=ImagePayloadMetadataCompositionMode.for_plane_axis( + self.plane_axis + ), ) - if source_metadata is None else source_metadata + if source_metadata is None + else source_metadata ) if self._metadata.plane_axis is not self.plane_axis: - raise ValueError("Produced image metadata must retain its declared plane axis.") + raise ValueError( + "Produced image metadata must retain its declared plane axis." + ) shared_mask = ( - self._shared_image_mask(present_masks, data_geometry) if present_masks else None + self._shared_image_mask(present_masks, data_geometry) + if present_masks + else None ) mask_geometry = ( array_geometry(shared_mask) - if shared_mask is not None else ( + if shared_mask is not None + else ( runtime_slice_stack_geometry(present_masks) if present_masks else None ) ) @@ -1506,14 +1595,16 @@ def composition_metadata_mode(self) -> ImagePayloadMetadataCompositionMode: def dtype(self) -> Any: return np.result_type( *( - MemoryType(detect_memory_type(payload.data)) - .canonical_dtype_name(payload.data.dtype) + MemoryType(detect_memory_type(payload.data)).canonical_dtype_name( + payload.data.dtype + ) for payload in self.slices ) ) def plane_axis_for_output_context( - self, context: AlignedImageSliceContext, + self, + context: AlignedImageSliceContext, ) -> RuntimePlaneAxis: return self.plane_axis @@ -1534,26 +1625,37 @@ def with_metadata(self, metadata: ImagePayloadMetadata) -> Any: return super(ProducedImageStack, self).with_metadata(metadata) slices = tuple( metadata.for_leading_source_plane(index).payload_with( - payload.data, payload.mask, + payload.data, + payload.mask, ) for index, payload in enumerate(self.slices) ) return type(self)( - slices, self.slice_contexts, memory_type=self.memory_type, - plane_axis=self.plane_axis, source_metadata=metadata, + slices, + self.slice_contexts, + memory_type=self.memory_type, + plane_axis=self.plane_axis, + source_metadata=metadata, ) def normalize_intensity_payload( - self, *, dtype: Any = None, channel_index: int = 0, + self, + *, + dtype: Any = None, + channel_index: int = 0, ) -> Any: if self._metadata.normalization_dtype(self.dtype, dtype) is None: return self return self._metadata.normalize_intensity_payload( - self.compose(), dtype=dtype, channel_index=channel_index, + self.compose(), + dtype=dtype, + channel_index=channel_index, ) def _shared_image_mask( - self, masks: tuple[Any, ...], data_geometry: ArrayGeometry, + self, + masks: tuple[Any, ...], + data_geometry: ArrayGeometry, ) -> Any | None: first = masks[0] if all(mask is first for mask in masks) and self._metadata.mask_domain( @@ -1577,8 +1679,14 @@ def _materialized_image_mask(self) -> Any | None: if masks[0] is None: return None shared = self._shared_image_mask(masks, self.geometry) - return shared if shared is not None else stack_runtime_slices( - masks, self.memory_type, 0, + return ( + shared + if shared is not None + else stack_runtime_slices( + masks, + self.memory_type, + 0, + ) ) def __getstate__(self) -> dict[str, Any]: @@ -1611,12 +1719,17 @@ def _retain_composed_slices(self) -> None: def with_slices(self, slices: Sequence[Any]) -> "ProducedImageStack": return type(self)( - tuple(slices), self.slice_contexts, - memory_type=self.memory_type, plane_axis=self.plane_axis, + tuple(slices), + self.slice_contexts, + memory_type=self.memory_type, + plane_axis=self.plane_axis, ) def copied( - self, *, memory_type: str, device_id: int | None, + self, + *, + memory_type: str, + device_id: int | None, ) -> "ProducedImageStack": """Snapshot produced pixels directly into one dense buffer. @@ -1626,7 +1739,8 @@ def copied( """ data = stack_runtime_slices( tuple(payload.data for payload in self.slices), - memory_type, device_id, + memory_type, + device_id, ) masks = tuple(payload.mask for payload in self.slices) mask = None @@ -1634,30 +1748,41 @@ def copied( shared = self._shared_image_mask(masks, self.geometry) mask = ( stack_runtime_slices(masks, memory_type, device_id) - if shared is None else stack_runtime_slices( - (shared,), memory_type, device_id, + if shared is None + else stack_runtime_slices( + (shared,), + memory_type, + device_id, )[0] ) metadata = self._metadata.replace_fields() copied = type(self)( - self.slices, self.slice_contexts, memory_type=memory_type, - plane_axis=self.plane_axis, source_metadata=metadata, + self.slices, + self.slice_contexts, + memory_type=memory_type, + plane_axis=self.plane_axis, + source_metadata=metadata, ) copied._composed_payload = metadata.payload_with(data, mask) copied._retain_composed_slices() return copied def compose( - self, *, memory_type: str | None = None, device_id: int | None = None, + self, + *, + memory_type: str | None = None, + device_id: int | None = None, ) -> Any: with self._realization_lock: if self._composed_payload is None: data = stack_runtime_slices( tuple(payload.data for payload in self.slices), - self.memory_type, 0, + self.memory_type, + 0, ) self._composed_payload = self._metadata.payload_with( - data, self._materialized_image_mask(), + data, + self._materialized_image_mask(), ) self._retain_composed_slices() if memory_type is None: @@ -1670,8 +1795,14 @@ def compose( mask = self._composed_payload.mask return self._metadata.payload_with( source.convert_to(data, target, device_id), - None if mask is None else MemoryType(detect_memory_type(mask)).convert_to( - mask, target, device_id, + ( + None + if mask is None + else MemoryType(detect_memory_type(mask)).convert_to( + mask, + target, + device_id, + ) ), ) @@ -1692,7 +1823,9 @@ def composition_metadata_mode(self) -> ImagePayloadMetadataCompositionMode: return ImagePayloadMetadataCompositionMode.BUNDLE @property - def projected_output_composition_mode(self) -> ImagePayloadMetadataCompositionMode | None: + def projected_output_composition_mode( + self, + ) -> ImagePayloadMetadataCompositionMode | None: """Flatten declared inner runtime planes, retaining other named image domains.""" if any( payload.metadata.plane_axis is RuntimePlaneAxis.RUNTIME_SLICE @@ -1704,11 +1837,15 @@ def projected_output_composition_mode(self) -> ImagePayloadMetadataCompositionMo return self.composition_metadata_mode def plane_axis_for_output_context( - self, context: AlignedImageSliceContext, + self, + context: AlignedImageSliceContext, ) -> RuntimePlaneAxis | None: """Resolve a named output's own inner plane axis before slice projection.""" payloads = tuple( - payload for payload, declared_context in zip(self.slices, self.slice_contexts, strict=True) + payload + for payload, declared_context in zip( + self.slices, self.slice_contexts, strict=True + ) if declared_context == context ) if len(payloads) != 1: @@ -1884,9 +2021,7 @@ def compose_aligned_image_payload( if len(image_payloads) == 1: return ImagePayloadComposition( payload=aligned_payloads[0], - execution_mode=( - AlignedStackExecution - ), + execution_mode=(AlignedStackExecution), ) declared_contexts = tuple( payload.slice_contexts diff --git a/openhcs/core/artifacts.py b/openhcs/core/artifacts.py index 3310dc79a0..fbe89bde93 100644 --- a/openhcs/core/artifacts.py +++ b/openhcs/core/artifacts.py @@ -658,7 +658,8 @@ def contextualize_output( tuple( cls.contextualize_output( RuntimeSliceProjection.value_for_slice( - source_payload, plane_projection.selected_plane(slice_index), + source_payload, + plane_projection.selected_plane(slice_index), ), item.data if isinstance(item, RuntimeValue) else item, output_plan, @@ -684,7 +685,9 @@ def contextualize_output( output_metadata.plane_axis is None and output_metadata.source_provenance.source_plane_count == plane_projection.axis_size - and plane_projection.dense_shape_carries_axis(output_value.geometry.shape) + and plane_projection.dense_shape_carries_axis( + output_value.geometry.shape + ) ): output_value = output_metadata.replace_fields( plane_axis=plane_projection.axis, @@ -831,7 +834,9 @@ def compose_runtime_values( from openhcs.core.aligned_image_payload import stack_image_payloads from openhcs.core.memory import detect_memory_type - from openhcs.core.runtime_image_values import ImagePayloadMetadataCompositionMode + from openhcs.core.runtime_image_values import ( + ImagePayloadMetadataCompositionMode, + ) from openhcs.core.runtime_plane_projection import RuntimePlaneAxis components = {value.key.scope.component for value in values} @@ -3650,7 +3655,6 @@ def materialization_metadata( ) -> "ImagePayloadMetadata": """Derive filename metadata from the declared naming source.""" - payload_metadata = self.artifact_type.materialization_image_metadata(value) materialization_source = self.materialization_source() if ( diff --git a/openhcs/core/axes.py b/openhcs/core/axes.py index d79cccb8dc..0adc4aa06e 100644 --- a/openhcs/core/axes.py +++ b/openhcs/core/axes.py @@ -351,8 +351,17 @@ def _validate_declaration(cls) -> None: if issubclass(cls, GridAddressed): undeclared = [ member - for member in ("default_grid", "grid_coordinates", "grid_position", "row_label") - if not any(member in base.__dict__ for base in cls.__mro__ if base is not GridAddressed) + for member in ( + "default_grid", + "grid_coordinates", + "grid_position", + "row_label", + ) + if not any( + member in base.__dict__ + for base in cls.__mro__ + if base is not GridAddressed + ) ] if undeclared: raise TypeError( @@ -396,7 +405,6 @@ def filename_token(cls, value: object) -> str: return f"{cls.filename_prefix}{text}" - class Ungrouped(GroupingDeclaration): """Explicit absent-grouping declaration: the step does not partition its value.""" @@ -498,21 +506,28 @@ def _validate_declaration(cls) -> None: f"Axis family {cls.__qualname__} must declare exactly one PartitionAxis." ) spatial_rank = cls.__dict__.get("payload_spatial_rank") - if not isinstance(spatial_rank, int) or isinstance(spatial_rank, bool) or spatial_rank < 0: + if ( + not isinstance(spatial_rank, int) + or isinstance(spatial_rank, bool) + or spatial_rank < 0 + ): raise TypeError( f"Axis family {cls.__qualname__} must declare payload_spatial_rank " "as a nonnegative int." ) for axis in declared: if "family" in axis.__dict__: - raise TypeError(f"Axis {axis.__qualname__} already belongs to a family.") + raise TypeError( + f"Axis {axis.__qualname__} already belongs to a family." + ) axis.family = cls cls.axes = declared cls.family_name = f"{cls.__module__}:{cls.__qualname__}" - def __new__(cls, *args: object, **kwargs: object): - raise TypeError(f"{cls.__qualname__} is a family declaration, not a value type.") + raise TypeError( + f"{cls.__qualname__} is a family declaration, not a value type." + ) # -- activation --------------------------------------------------------- @@ -539,7 +554,9 @@ def activate_entry_point(cls) -> None: """ inherited = os.environ.get(ACTIVE_FAMILY_VARIABLE) - (cls if inherited is None else AxisFamily.from_family_name(inherited)).activate() + ( + cls if inherited is None else AxisFamily.from_family_name(inherited) + ).activate() @staticmethod def from_family_name(family_name: str) -> type[AxisFamily]: diff --git a/openhcs/core/callable_contract.py b/openhcs/core/callable_contract.py index baf16cd4c8..d1fdca8dc2 100644 --- a/openhcs/core/callable_contract.py +++ b/openhcs/core/callable_contract.py @@ -274,7 +274,9 @@ def validate_source_metadata( ) -> None: """Validate one exact source after source-binding transformations.""" - if not load_as_monochrome and metadata.pixel_semantics.axes.with_role(self.role): + if not load_as_monochrome and metadata.pixel_semantics.axes.with_role( + self.role + ): return raise ValueError( f"Primary image requires a declared {self.role.__name__} payload axis, " @@ -334,8 +336,7 @@ def _validate_optional_instance( if value is not None and not isinstance(value, kind): raise TypeError( - f"{owner} must be {kind.__name__} or None, got " - f"{type(value).__name__}." + f"{owner} must be {kind.__name__} or None, got " f"{type(value).__name__}." ) @@ -349,7 +350,7 @@ def __init__( parameters=None, *, return_annotation=inspect.Signature.empty, - __validate_parameters__=True + __validate_parameters__=True, ): from openhcs.core.vfs_protocol import DatasetPathDeclaration @@ -417,14 +418,18 @@ class CallableMetadata: payload_axis_transition: PayloadAxisTransition | None = None @staticmethod - def prepared_callable_signature(func: Callable[..., object]) -> inspect.Signature | None: + def prepared_callable_signature( + func: Callable[..., object], + ) -> inspect.Signature | None: """Admit an owned snapshot only for its exact semantic callable boundary.""" raw = getattr(func, FunctionContractAttribute.raw_processing_function, None) if raw is not None and raw is not func: return None signature = getattr(func, FunctionContractAttribute.canonical_signature, None) if signature is not None and not isinstance(signature, inspect.Signature): - raise TypeError("Prepared callable signature must be an inspect.Signature or None.") + raise TypeError( + "Prepared callable signature must be an inspect.Signature or None." + ) return signature @classmethod @@ -467,7 +472,9 @@ def is_request_boundary(candidate: Callable[..., object]) -> bool: ) return resolved - def resolve_signature(self, raw_callable: Callable[..., object]) -> inspect.Signature: + def resolve_signature( + self, raw_callable: Callable[..., object] + ) -> inspect.Signature: """Resolve the semantic ABI once after declaration preparation.""" signature = inspect.signature(raw_callable) annotations = ( @@ -477,7 +484,9 @@ def resolve_signature(self, raw_callable: Callable[..., object]) -> inspect.Sign ) return signature.replace( parameters=tuple( - parameter.replace(annotation=annotations.get(name, parameter.annotation)) + parameter.replace( + annotation=annotations.get(name, parameter.annotation) + ) for name, parameter in signature.parameters.items() ), return_annotation=annotations.get("return", signature.return_annotation), @@ -498,7 +507,8 @@ def _nominal_argument_types(cls, annotation: object) -> tuple[type, ...]: annotation = resolve_annotated(annotation) if is_union_type(annotation): return tuple( - argument_type for member in get_args(annotation) + argument_type + for member in get_args(annotation) for argument_type in cls._nominal_argument_types(member) ) return ( @@ -507,7 +517,9 @@ def _nominal_argument_types(cls, annotation: object) -> tuple[type, ...]: else () ) - def raw_main_flow_call_argument(self, source_payload: Any, signature: inspect.Signature) -> Any: + def raw_main_flow_call_argument( + self, source_payload: Any, signature: inspect.Signature + ) -> Any: """Pass the main-flow value as the primary parameter declares it. A bare array enters the payload family here; a parameter that does not @@ -517,17 +529,26 @@ def raw_main_flow_call_argument(self, source_payload: Any, signature: inspect.Si from openhcs.core.runtime_image_values import ImagePayload, owned_runtime_value primary_name = self.primary_input_name(signature.parameters) - annotation = None if primary_name is None else signature.parameters[primary_name].annotation + annotation = ( + None + if primary_name is None + else signature.parameters[primary_name].annotation + ) if ( annotation is inspect.Parameter.empty and self.image_payload_consumption is ImagePayloadConsumption.COMPOSED ): return source_payload source_payload = owned_runtime_value(source_payload) - return source_payload if any( - isinstance(source_payload, argument_type) - for argument_type in self._nominal_argument_types(annotation) - ) or not isinstance(source_payload, ImagePayload) else source_payload.data + return ( + source_payload + if any( + isinstance(source_payload, argument_type) + for argument_type in self._nominal_argument_types(annotation) + ) + or not isinstance(source_payload, ImagePayload) + else source_payload.data + ) def canonical_signature_for(self, func: Any) -> inspect.Signature: """Use the prepared semantic ABI; unprepared authoring remains live.""" @@ -551,14 +572,21 @@ def raw_runtime_signature_for(self, func: Any) -> inspect.Signature: return self.canonical_signature return self.resolve_signature(self.resolve_raw_runtime_callable(func)) - def with_prepared_signatures(self, canonical: Callable, runtime: Callable) -> "CallableMetadata": + def with_prepared_signatures( + self, canonical: Callable, runtime: Callable + ) -> "CallableMetadata": """Snapshot each actual target independently after declaration preparation.""" return dataclasses.replace( - self, canonical_signature=self.resolve_signature(canonical), - raw_runtime_signature=None if runtime is canonical else self.resolve_signature(runtime), + self, + canonical_signature=self.resolve_signature(canonical), + raw_runtime_signature=( + None if runtime is canonical else self.resolve_signature(runtime) + ), ) - def with_missing_signatures(self, canonical: Callable[[], Callable], runtime: Callable[[], Callable]) -> "CallableMetadata": + def with_missing_signatures( + self, canonical: Callable[[], Callable], runtime: Callable[[], Callable] + ) -> "CallableMetadata": """Reuse library readiness or capture the newly prepared authored targets.""" if self.canonical_signature is not None: return self @@ -566,10 +594,14 @@ def with_missing_signatures(self, canonical: Callable[[], Callable], runtime: Ca def publish_signatures(self, namespace: MutableMapping[str, Any]) -> None: """Replace the namespace pair as one library preparation epoch.""" - namespace[FunctionContractAttribute.canonical_signature] = self.canonical_signature + namespace[FunctionContractAttribute.canonical_signature] = ( + self.canonical_signature + ) namespace.pop(FunctionContractAttribute.raw_runtime_signature, None) if self.raw_runtime_signature is not None: - namespace[FunctionContractAttribute.raw_runtime_signature] = self.raw_runtime_signature + namespace[FunctionContractAttribute.raw_runtime_signature] = ( + self.raw_runtime_signature + ) @property def artifact_output_policy(self) -> type[ArtifactOutputPolicy]: @@ -584,7 +616,8 @@ def __post_init__(self) -> None: if self.canonical_signature is not None: object.__setattr__( - self, "canonical_signature", + self, + "canonical_signature", CallableSignature.from_signature(self.canonical_signature), ) @@ -665,13 +698,19 @@ def from_projection( if runtime_adapter is not None and callable(projection.func): runtime_adapter.validate_callable_signature(projection.func) declared_artifact_inputs = _artifact_specs_from_namespace( - namespace, projection.name, FunctionContractAttribute.artifact_inputs, + namespace, + projection.name, + FunctionContractAttribute.artifact_inputs, ) signature = ( - cls.callable_signature(projection.func) if callable(projection.func) else None + cls.callable_signature(projection.func) + if callable(projection.func) + else None ) artifact_inputs = _artifact_input_specs_from_projection( - projection, declared_artifact_inputs, signature, + projection, + declared_artifact_inputs, + signature, ) return cls( input_memory_type=reader.optional_string( @@ -766,9 +805,13 @@ def as_namespace(self) -> dict[str, object]: """Project typed metadata into callable declaration keys.""" namespace: dict[str, object] = {} if self.canonical_signature is not None: - namespace[FunctionContractAttribute.canonical_signature] = self.canonical_signature + namespace[FunctionContractAttribute.canonical_signature] = ( + self.canonical_signature + ) if self.raw_runtime_signature is not None: - namespace[FunctionContractAttribute.raw_runtime_signature] = self.raw_runtime_signature + namespace[FunctionContractAttribute.raw_runtime_signature] = ( + self.raw_runtime_signature + ) if self.input_memory_type is not None: MemoryContractAttribute.INPUT.write(namespace, self.input_memory_type) if self.output_memory_type is not None: @@ -1235,7 +1278,10 @@ def lifecycle_anchor_owner_inputs(self) -> ArtifactSpecCollection: undeclared artifact inputs are context for that anchor. A declared group-lineage relation still names its owning inputs. """ - if self.main_flow_supplies_primary_image and not self.output_group_scope_sources: + if ( + self.main_flow_supplies_primary_image + and not self.output_group_scope_sources + ): return ArtifactSpecCollection(()) return self.group_scope_inputs @@ -1358,7 +1404,9 @@ def declared_path_values( signature = self.canonical_signature values: dict[str, tuple["DatasetPathDeclaration", object]] = {} for parameter_name, declaration in self.declared_path_parameters.items(): - value = kwargs.get(parameter_name, signature.parameters[parameter_name].default) + value = kwargs.get( + parameter_name, signature.parameters[parameter_name].default + ) if parameter_name not in kwargs and value is inspect.Parameter.empty: continue if value is not None: @@ -1433,12 +1481,15 @@ def runtime_main_flow_call_argument(self, source_payload: Any) -> Any: def raw_main_flow_call_argument(self, source_payload: Any) -> Any: """Project the metadata-owned raw carrier declaration for this contract.""" - return self.metadata.raw_main_flow_call_argument(source_payload, self.canonical_signature) + return self.metadata.raw_main_flow_call_argument( + source_payload, self.canonical_signature + ) def main_flow_call_argument(self, source_payload: Any) -> Any: """Let the processing declaration retain context needed before raw calls.""" return self.require_processing_contract().main_flow_call_argument( - self, source_payload, + self, + source_payload, ) @property @@ -1605,9 +1656,13 @@ def canonical_signature(self) -> inspect.Signature: def with_prepared_signature(self) -> "CallableContract": """Freeze the prepared declaration into its existing compiled contract.""" - return dataclasses.replace(self, metadata=self.metadata.with_prepared_signatures( - self.resolve_canonical_raw_callable(), self.resolve_raw_runtime_callable(), - )) + return dataclasses.replace( + self, + metadata=self.metadata.with_prepared_signatures( + self.resolve_canonical_raw_callable(), + self.resolve_raw_runtime_callable(), + ), + ) @property def raw_runtime_signature(self) -> inspect.Signature: @@ -1620,9 +1675,13 @@ def from_prepared_callable(cls, func: Callable[..., object]) -> "CallableContrac projection = CallableProjection.from_callable(func) prepared = cls.from_callable(projection.prepared_callable()) return dataclasses.replace( - prepared, func=func, function_name=projection.name, module_name=projection.module_name, + prepared, + func=func, + function_name=projection.name, + module_name=projection.module_name, metadata=prepared.metadata.with_missing_signatures( - prepared.resolve_canonical_raw_callable, prepared.resolve_raw_runtime_callable, + prepared.resolve_canonical_raw_callable, + prepared.resolve_raw_runtime_callable, ), ) @@ -1631,12 +1690,19 @@ def decode_public_kwargs(self, kwargs: Mapping[str, object]) -> dict[str, object decoded = dict(kwargs) for name, annotation in self.canonical_parameter_annotations.items(): - if name not in kwargs or enum_member_type(resolve_annotated(annotation)) is None: + if ( + name not in kwargs + or enum_member_type(resolve_annotated(annotation)) is None + ): continue value = kwargs[name] - decoded[name] = value if value is None else coerce_enum_member(annotation, value) + decoded[name] = ( + value if value is None else coerce_enum_member(annotation, value) + ) validate_annotation_value( - annotation, decoded[name], path=f"{self.function_name}.{name}", + annotation, + decoded[name], + path=f"{self.function_name}.{name}", ) return decoded @@ -1661,7 +1727,10 @@ def bind_runtime_owned(parameter_name: str) -> None: parameter = signature.parameters.get(parameter_name) if parameter is None: return - if parameter_name in call_kwargs and call_kwargs[parameter_name] is not runtime_owned_value: + if ( + parameter_name in call_kwargs + and call_kwargs[parameter_name] is not runtime_owned_value + ): raise TypeError( f"Callable {self.function_name!r} public kwargs cannot set " f"runtime-owned parameter {parameter_name!r}." @@ -1675,13 +1744,22 @@ def bind_runtime_owned(parameter_name: str) -> None: overridable_parameters = self.overridable_runtime_parameter_names for parameter_name in ( - *(name for name in self.artifact_input_parameter_names - if name not in call_kwargs or name in runtime_loaded_parameters), + *( + name + for name in self.artifact_input_parameter_names + if name not in call_kwargs or name in runtime_loaded_parameters + ), self.runtime_context_parameter, - self.runtime_adapter.require_parameter_name() if self.runtime_adapter is not None else None, - *(parameter_type.require_parameter_name() - for parameter_type in self.runtime_bound_parameter_types - if parameter_type.require_parameter_name() not in overridable_parameters), + ( + self.runtime_adapter.require_parameter_name() + if self.runtime_adapter is not None + else None + ), + *( + parameter_type.require_parameter_name() + for parameter_type in self.runtime_bound_parameter_types + if parameter_type.require_parameter_name() not in overridable_parameters + ), *self.config_bound_parameter_names, ): if parameter_name is not None: @@ -2012,8 +2090,12 @@ def attach_callable_contract_metadata( payload_axis_transition: PayloadAxisTransition | None = None, ) -> None: """Attach OpenHCS callable metadata used by compiler/runtime phases.""" - _mutable_callable_namespace(func).pop(FunctionContractAttribute.canonical_signature, None) - _mutable_callable_namespace(func).pop(FunctionContractAttribute.raw_runtime_signature, None) + _mutable_callable_namespace(func).pop( + FunctionContractAttribute.canonical_signature, None + ) + _mutable_callable_namespace(func).pop( + FunctionContractAttribute.raw_runtime_signature, None + ) if processing_contract is not None: if not ( isinstance(processing_contract, type) @@ -2370,7 +2452,11 @@ def _runtime_context_parameter( from openhcs.core.context.processing_context import ProcessingContext parameter_name = ProcessingContext.require_parameter_name() - return parameter_name if signature is not None and parameter_name in signature.parameters else None + return ( + parameter_name + if signature is not None and parameter_name in signature.parameters + else None + ) @dataclass(frozen=True, slots=True) @@ -2382,12 +2468,17 @@ class CallableMetadataReader: def has_prepared_signature(self) -> bool: """Validate the namespace-owned readiness declaration before phase selection.""" - return self.optional_signature(FunctionContractAttribute.canonical_signature) is not None + return ( + self.optional_signature(FunctionContractAttribute.canonical_signature) + is not None + ) def optional_signature(self, field_name: str) -> inspect.Signature | None: value = self.namespace.get(field_name) if value is not None and not isinstance(value, inspect.Signature): - raise TypeError(f"{self.function_name!r}.{field_name} must be an inspect.Signature or None.") + raise TypeError( + f"{self.function_name!r}.{field_name} must be an inspect.Signature or None." + ) return value def optional_string(self, field_name: str) -> str | None: @@ -2480,8 +2571,9 @@ def optional_variable_component_stack_requirement( ) return value - - def optional_subclass(self, field_name: str, family: type[_ValueT]) -> type[_ValueT] | None: + def optional_subclass( + self, field_name: str, family: type[_ValueT] + ) -> type[_ValueT] | None: """Return an optional declared class from one nominal family.""" value = self.namespace.get(field_name) if value is None: diff --git a/openhcs/core/compiled_step_plan.py b/openhcs/core/compiled_step_plan.py index b53c3096ef..f6286708f0 100644 --- a/openhcs/core/compiled_step_plan.py +++ b/openhcs/core/compiled_step_plan.py @@ -318,20 +318,27 @@ class CompiledStepPlan: error: str | None = None def stored_primary_input_edges_for_group( - self, group: CompiledFunctionGroup, component_key: str | None, + self, + group: CompiledFunctionGroup, + component_key: str | None, ) -> tuple[InvocationArtifactInputEdgePlan, ...] | None: """Admit exact producer transport only without path-based transforms.""" if self.input_conversion is not None or self.sequential_filter_plan.enabled: return None edges = group.stored_primary_input_edges_for_component( - self.execution_group_scope, component_key, + self.execution_group_scope, + component_key, source_bindings=self.source_binding_plan, ) if edges and any( - edge.storage_plan.source_step_scope_id == self.step_scope_id - if edge.storage_plan.source_step_scope_id is not None - else edge.storage_plan.source_step_id in ( - self.step_index, self.step_scope_id, + ( + edge.storage_plan.source_step_scope_id == self.step_scope_id + if edge.storage_plan.source_step_scope_id is not None + else edge.storage_plan.source_step_id + in ( + self.step_index, + self.step_scope_id, + ) ) for edge in edges ): @@ -339,7 +346,8 @@ def stored_primary_input_edges_for_group( return edges def requires_main_flow_checkpoint( - self, plans: Mapping[int, "CompiledStepPlan"], + self, + plans: Mapping[int, "CompiledStepPlan"], ) -> bool: """Follow this value's consumers through preserved input lifetimes.""" if not plans or self.step_index not in plans: @@ -371,8 +379,10 @@ def requires_main_flow_checkpoint( if pattern is None or any( (group := pattern.group_for_component(component_key)) is None or consumer.stored_primary_input_edges_for_group( - group, component_key, - ) is None + group, + component_key, + ) + is None for component_key in consumer.execution_group_scope.keys ): return True @@ -552,7 +562,6 @@ def memory_device_scope( return self.device_assignment.scope_for(declaration) - @property def execution_group_value(self) -> str | None: component = self.execution_group_scope.component diff --git a/openhcs/core/component_filters.py b/openhcs/core/component_filters.py index 7a24821549..39301693aa 100644 --- a/openhcs/core/component_filters.py +++ b/openhcs/core/component_filters.py @@ -25,7 +25,9 @@ def __post_init__(self) -> None: if not self.by_axis: return family = AxisFamily.active() - unknown = sorted(name for name, _values in self.by_axis if name not in family.names()) + unknown = sorted( + name for name, _values in self.by_axis if name not in family.names() + ) if unknown: raise ValueError( f"component_filters name unknown axes {unknown}; the " @@ -45,7 +47,11 @@ def from_mapping(cls, mapping: ComponentFilterMapping | None) -> ComponentFilter by_axis=tuple( ( str(name), - (values,) if isinstance(values, str) else tuple(str(value) for value in values), + ( + (values,) + if isinstance(values, str) + else tuple(str(value) for value in values) + ), ) for name, values in mapping.items() ) diff --git a/openhcs/core/config.py b/openhcs/core/config.py index dc7cc5c551..80b39f4fb7 100644 --- a/openhcs/core/config.py +++ b/openhcs/core/config.py @@ -257,7 +257,9 @@ def slot_for_axis(self, axis: type[Axis]) -> str: def component_modes(self) -> dict[str, str]: """The slot of every axis of the active family, by axis name.""" - return {axis.name: self.slot_for_axis(axis) for axis in AxisFamily.active().axes} + return { + axis.name: self.slot_for_axis(axis) for axis in AxisFamily.active().axes + } def display_payload_extra(self) -> dict[str, ViewerWireValue]: """Stream the axis declarations and this viewer's settings.""" @@ -308,7 +310,9 @@ class NapariDisplayConfig(AxisRoleModeDisplayConfig, NapariDisplaySettings): # Apply the global pipeline config decorator with ui_hidden=True # This config is only inherited by NapariStreamingConfig, so hide it from UI -NapariDisplayConfig = global_pipeline_config(ui_hidden=True, inherit_as_none=False)(NapariDisplayConfig) +NapariDisplayConfig = global_pipeline_config(ui_hidden=True, inherit_as_none=False)( + NapariDisplayConfig +) FijiDimensionMode = FijiSlots.choice_enum("FijiDimensionMode", __name__) @@ -337,7 +341,9 @@ class FijiDisplayConfig(AxisRoleModeDisplayConfig, FijiDisplaySettings): # Apply the global pipeline config decorator with ui_hidden=True # This config is only inherited by FijiStreamingConfig, so hide it from UI -FijiDisplayConfig = global_pipeline_config(ui_hidden=True, inherit_as_none=False)(FijiDisplayConfig) +FijiDisplayConfig = global_pipeline_config(ui_hidden=True, inherit_as_none=False)( + FijiDisplayConfig +) @abbreviation("pfc") @@ -353,9 +359,9 @@ class PartitionFilterConfig(AnnotatedDataclassValidationMixin): show_preview_without_partition_filter: ClassVar[bool] = False - partition_filter: Annotated[Optional[Union[List[str], str, int]], abbreviation("")] = ( - None - ) + partition_filter: Annotated[ + Optional[Union[List[str], str, int]], abbreviation("") + ] = None """Selection matched against the dataset's ordered partition values. Use a list for exact IDs; a string for one ID, comma-separated IDs, @@ -365,9 +371,9 @@ class PartitionFilterConfig(AnnotatedDataclassValidationMixin): mode selects all. """ - partition_filter_mode: Annotated[PartitionFilterMode, abbreviation("filter_mode")] = ( - PartitionFilterMode.INCLUDE - ) + partition_filter_mode: Annotated[ + PartitionFilterMode, abbreviation("filter_mode") + ] = PartitionFilterMode.INCLUDE """Apply ``partition_filter`` as the values to include or the values to exclude. Include mode rejects unknown explicit values. Exclude mode ignores unknown @@ -622,7 +628,9 @@ class StepPartitionFilterConfig(PartitionFilterConfig): @abbreviation("mat") @global_pipeline_config(preview_label="MAT", always_viewable_fields=["sub_dir"]) @dataclass(frozen=True) -class StepMaterializationConfig(Enableable, StepPartitionFilterConfig, PathPlanningConfig): +class StepMaterializationConfig( + Enableable, StepPartitionFilterConfig, PathPlanningConfig +): """ Configuration for persistent copies of a step's ordinary main-flow result. @@ -737,11 +745,6 @@ def port(self) -> TcpPort: """Port for streaming communication. Each streamer type has its own default.""" pass - - - - - @classmethod @abstractmethod def port_from_config(cls, config) -> Optional[int]: @@ -889,7 +892,6 @@ class CompilationDebugConfig(AnnotatedDataclassValidationMixin, Enableable): """ - # Inject all accumulated fields at the end of module loading. # Use the ObjectState owner that registered the pending field declarations. from objectstate.lazy_factory import _inject_all_pending_fields diff --git a/openhcs/core/dataset_sources/choice.py b/openhcs/core/dataset_sources/choice.py index fc772fa8ba..36af173540 100644 --- a/openhcs/core/dataset_sources/choice.py +++ b/openhcs/core/dataset_sources/choice.py @@ -127,7 +127,10 @@ def source_type_for( filemanager: "FileManager", source_bindings: "SourceBindingsConfig | None", ) -> "type[DatasetSource]": - from openhcs.core.dataset_sources.source import DatasetSource, DeclaredFileSource + from openhcs.core.dataset_sources.source import ( + DatasetSource, + DeclaredFileSource, + ) detected = DatasetSource.detect_source_type(root, filemanager, source_bindings) if detected is not None: diff --git a/openhcs/core/dataset_sources/interfaces.py b/openhcs/core/dataset_sources/interfaces.py index 3c75d6a6a9..3d368d79c3 100644 --- a/openhcs/core/dataset_sources/interfaces.py +++ b/openhcs/core/dataset_sources/interfaces.py @@ -29,7 +29,6 @@ from polystore.virtual_workspace import SourcePixelRef from openhcs.core.axes import Axis, AxisFamily - if TYPE_CHECKING: from openhcs.core.source_projection import SourcePlaneDataset, SourceProjection from openhcs.core.source_workspace_projection import ( @@ -408,9 +407,7 @@ def parse_image_path(self, relative_path: str) -> FilenameParseResult | None: else parsed.with_values(self.image_path_components(path)) ) - def image_path_components( - self, path: Path - ) -> tuple[tuple[type[Axis], int], ...]: + def image_path_components(self, path: Path) -> tuple[tuple[type[Axis], int], ...]: """Identity terminus for cooperative acquisition-coordinate capabilities.""" return () @@ -577,7 +574,9 @@ def get_metadata_pixel_size(self, dataset_root: Union[str, Path]) -> float: """ return self.get_pixel_size(dataset_root) - def source_voxel_spacing(self, dataset_root: Union[str, Path]) -> SourceVoxelSpacing: + def source_voxel_spacing( + self, dataset_root: Union[str, Path] + ) -> SourceVoxelSpacing: """Project this acquisition's physical calibration into source coordinates.""" pixel_size = self.get_pixel_size(dataset_root) diff --git a/openhcs/core/dataset_sources/openhcs_format.py b/openhcs/core/dataset_sources/openhcs_format.py index 6e564a493c..4dcafe8eed 100644 --- a/openhcs/core/dataset_sources/openhcs_format.py +++ b/openhcs/core/dataset_sources/openhcs_format.py @@ -103,7 +103,11 @@ def invalidate_metadata_cache(self) -> None: self._metadata_dict_dataset_root_cache: Optional[Path] = None def _metadata_field( - self, dataset_root: Union[str, Path], field: str, *, merge: bool = False, + self, + dataset_root: Union[str, Path], + field: str, + *, + merge: bool = False, ) -> Any: """Resolve only the requested fact within the declared projection scope. @@ -112,14 +116,16 @@ def _metadata_field( silently choose one projection's geometry or execution input. """ subdirectories = self._metadata_subdirectories( - self.load_metadata_document(dataset_root), dataset_root, + self.load_metadata_document(dataset_root), + dataset_root, ) try: selected = self._main_subdirectory_name(subdirectories, dataset_root) except MetadataNotFoundError: values = {name: data.get(field) for name, data in subdirectories.items()} resolver = ( - self._merge_subdirectory_mapping if merge + self._merge_subdirectory_mapping + if merge else self._consistent_subdirectory_value ) return resolver(values, dataset_root, field) @@ -302,7 +308,9 @@ def get_pixel_size(self, dataset_root: Union[str, Path]) -> float: self._source_voxel_spacings(dataset_root) ) - def source_voxel_spacing(self, dataset_root: Union[str, Path]) -> SourceVoxelSpacing: + def source_voxel_spacing( + self, dataset_root: Union[str, Path] + ) -> SourceVoxelSpacing: """Preserve the stored source frame, including unknown/relative units.""" return SourceVoxelSpacing.common(self._source_voxel_spacings(dataset_root)) @@ -313,7 +321,8 @@ def _source_voxel_spacings( return tuple( SourceVoxelSpacing.from_source_metadata(source) for source in ( - self._metadata_field(dataset_root, FIELDS.SOURCE_METADATA, merge=True) or {} + self._metadata_field(dataset_root, FIELDS.SOURCE_METADATA, merge=True) + or {} ).values() ) @@ -328,7 +337,9 @@ def get_metadata_pixel_size(self, dataset_root: Union[str, Path]) -> float: def get_source_filename_parser_name(self, dataset_root: Union[str, Path]) -> str: """Get source filename parser name from OpenHCS metadata.""" - parser_name = self._metadata_field(dataset_root, FIELDS.SOURCE_FILENAME_PARSER_NAME) + parser_name = self._metadata_field( + dataset_root, FIELDS.SOURCE_FILENAME_PARSER_NAME + ) if not (isinstance(parser_name, str) and parser_name): raise ValueError( f"'{FIELDS.SOURCE_FILENAME_PARSER_NAME}' must be a non-empty string in {self.METADATA_FILENAME}" @@ -397,7 +408,9 @@ def reconciliation_directories( ) document = OpenHCSMetadataSubdirectories.from_path(metadata_path) accepted_entries = { - name: VirtualWorkspaceSourceProjectionEntries.from_subdirectory(subdirectory) + name: VirtualWorkspaceSourceProjectionEntries.from_subdirectory( + subdirectory + ) for name, subdirectory in document.items() } return self.reconciliation_directories_from_document( @@ -428,7 +441,9 @@ def reconciliation_directories_from_document( is not None ) subdirectories = self._metadata_subdirectories( - document.metadata, dataset_root, workspace_root=dataset_root, + document.metadata, + dataset_root, + workspace_root=dataset_root, ) source_projection = None if document.has_workspace_mapping(): @@ -629,13 +644,13 @@ def _metadata_subdirectories( root = ( self.source_workspace_root(dataset_root) - if workspace_root is None else workspace_root + if workspace_root is None + else workspace_root ).absolute() requested = Path(dataset_root).absolute() if requested != root: matches = tuple( - name for name in subdirectories - if requested.is_relative_to(root / name) + name for name in subdirectories if requested.is_relative_to(root / name) ) if not matches: raise MetadataNotFoundError( @@ -732,7 +747,9 @@ def get_plate_acquisition_datetime( self, dataset_root: Union[str, Path] ) -> Optional[str]: """Get plate acquisition datetime if available.""" - return self._get_optional_metadata_str(dataset_root, FIELDS.ACQUISITION_DATETIME) + return self._get_optional_metadata_str( + dataset_root, FIELDS.ACQUISITION_DATETIME + ) def get_plate_name(self, dataset_root: Union[str, Path]) -> Optional[str]: """Get plate name if available.""" @@ -761,9 +778,14 @@ def backend_availability(self, input_dir: Union[str, Path]) -> Dict[str, bool]: Raises: MetadataNotFoundError: If metadata file cannot be found or parsed """ - available_backends = self._metadata_field( - input_dir, FIELDS.AVAILABLE_BACKENDS, merge=True, - ) or {} + available_backends = ( + self._metadata_field( + input_dir, + FIELDS.AVAILABLE_BACKENDS, + merge=True, + ) + or {} + ) if not isinstance(available_backends, dict): logger.warning( @@ -860,7 +882,9 @@ class OpenHCSMetadata: def collection_fields() -> tuple[str, ...]: """Persisted value-label keys, one per declared axis.""" - return tuple(axis.metadata_collection_field for axis in AxisFamily.active().axes) + return tuple( + axis.metadata_collection_field for axis in AxisFamily.active().axes + ) @staticmethod def labels_by_field( @@ -912,7 +936,10 @@ def serialized_values( pixel_size=pixel_size, image_files=image_files, axis_value_labels=cls.labels_by_field( - {component: serialized_values(component) for component in AxisFamily.active().axes} + { + component: serialized_values(component) + for component in AxisFamily.active().axes + } ), available_backends=available_backends, source_diagnostics=source_diagnostics, @@ -936,7 +963,10 @@ def _openhcs_metadata_from_subdirectory( ) -> OpenHCSMetadata: missing_fields = tuple( field - for field in (*_OPENHCS_METADATA_REQUIRED_FIELDS, *OpenHCSMetadata.collection_fields()) + for field in ( + *_OPENHCS_METADATA_REQUIRED_FIELDS, + *OpenHCSMetadata.collection_fields(), + ) if field not in subdirectory_data ) if missing_fields: @@ -1307,18 +1337,23 @@ class OpenHCSDatasetSource(PreparedWorkspaceSource, DatasetSource): @classmethod def create( - cls, *, filemanager: FileManager, pattern_format: Optional[str] = None, + cls, + *, + filemanager: FileManager, + pattern_format: Optional[str] = None, source_bindings_config=None, ) -> "OpenHCSDatasetSource": """Keep the prepared source while applying the declared source bindings.""" from openhcs.core.source_bindings import source_bindings_defaults_to_base handler = super().create( - filemanager=filemanager, pattern_format=pattern_format, + filemanager=filemanager, + pattern_format=pattern_format, source_bindings_config=source_bindings_config, ) handler._source_bindings_config = ( - None if source_bindings_config is None + None + if source_bindings_config is None else source_bindings_defaults_to_base(source_bindings_config) ) return handler @@ -1327,7 +1362,6 @@ def source_bindings_still_required(self): """Expose the original prepared-workspace declaration to runtime readers.""" return self._source_bindings_config - @classmethod def source_selection_guidance(cls) -> str: """Explain when OpenHCS workspace metadata describes the dataset.""" @@ -1455,8 +1489,6 @@ def root_dir(self) -> str: # Return empty string as placeholder (not used for virtual workspace) return "" - - @property def compatible_backends(self) -> List[Backend]: """ @@ -1549,7 +1581,9 @@ def get_primary_backend( # 3. Fall back to first available backend (usually disk) return next(iter(available_backends_dict.keys())) - def initialize_workspace(self, dataset_root: Path, filemanager: FileManager) -> Path: + def initialize_workspace( + self, dataset_root: Path, filemanager: FileManager + ) -> Path: """ OpenHCS format doesn't need workspace - determines the correct input subdirectory from metadata. @@ -1570,7 +1604,9 @@ def initialize_workspace(self, dataset_root: Path, filemanager: FileManager) -> # The caller's declared projection selects facts; storage stays root-relative. self.dataset_root = dataset_root if self._source_bindings_config is not None: - from openhcs.core.source_workspace_projection import WorkspaceSourceProjections + from openhcs.core.source_workspace_projection import ( + WorkspaceSourceProjections, + ) projection = WorkspaceSourceProjections.from_dataset_metadata( dataset_root=dataset_root, @@ -1578,8 +1614,13 @@ def initialize_workspace(self, dataset_root: Path, filemanager: FileManager) -> filemanager=filemanager, source_bindings=self._source_bindings_config, ).projection_if_available() - if projection is None and self._source_bindings_config.source_filter_declarations: - raise ValueError("Prepared source filtering requires a typed workspace projection.") + if ( + projection is None + and self._source_bindings_config.source_filter_declarations + ): + raise ValueError( + "Prepared source filtering requires a typed workspace projection." + ) logger.debug("OpenHCSHandler: dataset_root set to %s", self.dataset_root) # Determine the main subdirectory from metadata - fail-loud on errors @@ -1613,7 +1654,9 @@ def _main_subdirectory_metadata( self, dataset_root: Path, ) -> Mapping[str, object]: - metadata = self.metadata_handler.source_workspace_metadata_document(dataset_root) + metadata = self.metadata_handler.source_workspace_metadata_document( + dataset_root + ) if not isinstance(metadata, Mapping): raise ValueError("OpenHCS source-workspace metadata must be a mapping.") subdirectories = metadata.get(FIELDS.SUBDIRECTORIES) @@ -1649,10 +1692,14 @@ def _register_declared_workspace_backends( @classmethod def register_workspace_backends( - cls, dataset_root: Union[str, Path], filemanager: FileManager, + cls, + dataset_root: Union[str, Path], + filemanager: FileManager, ) -> None: """Register storage at its metadata root, not the selected projection child.""" - workspace_root = OpenHCSMetadataHandler(filemanager).source_workspace_root(dataset_root) + workspace_root = OpenHCSMetadataHandler(filemanager).source_workspace_root( + dataset_root + ) super().register_workspace_backends(workspace_root, filemanager) def post_workspace( diff --git a/openhcs/core/dataset_sources/source.py b/openhcs/core/dataset_sources/source.py index a988d67125..5e5be7665a 100644 --- a/openhcs/core/dataset_sources/source.py +++ b/openhcs/core/dataset_sources/source.py @@ -189,8 +189,7 @@ def source_selection_role(cls) -> type[SourceSelectionRole]: return next( base for base in cls.__mro__ - if isinstance(base, type) - and SourceSelectionRole in base.__bases__ + if isinstance(base, type) and SourceSelectionRole in base.__bases__ ) @classmethod @@ -312,7 +311,9 @@ def compatible_backends(self) -> List[Backend]: """Storage backends this source can read, in priority order.""" @abstractmethod - def initialize_workspace(self, dataset_root: Path, filemanager: FileManager) -> Path: + def initialize_workspace( + self, dataset_root: Path, filemanager: FileManager + ) -> Path: """Prepare the dataset for reading and return its image directory.""" def get_required_backend(self) -> Optional["MaterializationBackend"]: diff --git a/openhcs/core/dataset_sources/source_bindings_source.py b/openhcs/core/dataset_sources/source_bindings_source.py index ec0d4ceb05..c2639242b5 100644 --- a/openhcs/core/dataset_sources/source_bindings_source.py +++ b/openhcs/core/dataset_sources/source_bindings_source.py @@ -31,7 +31,6 @@ def projects_declared_source_bindings(cls) -> bool: return True - @classmethod def source_selection_guidance(cls) -> str: """Explain when declarations, rather than a vendor/store owner, ingest files.""" @@ -96,8 +95,6 @@ def __init__( def root_dir(self) -> str: return FIELDS.DEFAULT_SUBDIRECTORY - - @property def compatible_backends(self) -> list[Backend]: return [Backend.DISK] @@ -133,7 +130,9 @@ def _replay_persisted_workspace( VirtualWorkspaceSourceProjection, ) - metadata = self.metadata_handler.source_workspace_metadata_document(dataset_root) + metadata = self.metadata_handler.source_workspace_metadata_document( + dataset_root + ) workspace_metadata = self.metadata_handler.workspace_mapping_metadata( dataset_root ) diff --git a/openhcs/core/dataset_sources/source_schema.py b/openhcs/core/dataset_sources/source_schema.py index 5725c18708..0c31e7bb8c 100644 --- a/openhcs/core/dataset_sources/source_schema.py +++ b/openhcs/core/dataset_sources/source_schema.py @@ -45,4 +45,3 @@ def construct_filename(self, components: FilenameParseResult) -> str: for component, _value in components.declared_values() ) return address.filename(extension=components.extension) - diff --git a/openhcs/core/debug.py b/openhcs/core/debug.py index 2bbaa8deee..7dc9b1844a 100644 --- a/openhcs/core/debug.py +++ b/openhcs/core/debug.py @@ -275,11 +275,7 @@ def to_dict(self) -> dict[str, Any]: return { "debug_session_id": self.debug_session_id, "state": self.state.value, - "cursor": ( - None - if self.cursor is None - else to_jsonable(self.cursor) - ), + "cursor": (None if self.cursor is None else to_jsonable(self.cursor)), } @classmethod diff --git a/openhcs/core/equivalence/keys.py b/openhcs/core/equivalence/keys.py index e0e83f2ef5..b0ae80583c 100644 --- a/openhcs/core/equivalence/keys.py +++ b/openhcs/core/equivalence/keys.py @@ -674,9 +674,7 @@ def project_uncached(self) -> tuple[str, str | None]: return aggregate_feature parts_projection = parts_projection.strip_category_prefix_for_core() - direct_alias = self.dialect.feature_part_aliases().get( - parts_projection.parts - ) + direct_alias = self.dialect.feature_part_aliases().get(parts_projection.parts) if direct_alias is not None: return "_".join(direct_alias), None diff --git a/openhcs/core/equivalence/measurement_facts.py b/openhcs/core/equivalence/measurement_facts.py index e589998224..c1ab76c874 100644 --- a/openhcs/core/equivalence/measurement_facts.py +++ b/openhcs/core/equivalence/measurement_facts.py @@ -80,9 +80,7 @@ class RuntimeDirectionalPairMeasurementDerivationContract: @property def regression_slope_feature(self) -> str | None: """Return the dialect-declared pair regression-slope family.""" - return ( - self.policy.measurement_dialect.pair_regression_slope_feature_name() - ) + return self.policy.measurement_dialect.pair_regression_slope_feature_name() @property def correlation_feature(self) -> str | None: @@ -369,8 +367,7 @@ def dedupe_observed_alias_records( ) -> RuntimeMeasurementFacts: """Filter unobserved padding groups and collapse same-row aliases.""" return cls.dedupe_alias_facts( - (record.key, record.value) - for record in cls.observed_records(records) + (record.key, record.value) for record in cls.observed_records(records) ) @classmethod diff --git a/openhcs/core/equivalence/measurement_features.py b/openhcs/core/equivalence/measurement_features.py index 1ab162b52e..edb4c1b79e 100644 --- a/openhcs/core/equivalence/measurement_features.py +++ b/openhcs/core/equivalence/measurement_features.py @@ -91,9 +91,7 @@ def _for_feature_key_payload( key = RuntimeMeasurementFeatureKey.from_cache_payload(key_payload) context = RuntimeMeasurementFeatureSemanticContext( key, - RuntimeEquivalencePolicy( - measurement_dialect=dialect - ), + RuntimeEquivalencePolicy(measurement_dialect=dialect), ) strategy = cls.for_context( context, @@ -489,9 +487,7 @@ def _object_measurement_feature_matches_marker_cached( dialect: MeasurementDialect, ) -> bool: key = RuntimeMeasurementFeatureKey.from_cache_payload(key_payload) - policy = RuntimeEquivalencePolicy( - measurement_dialect=dialect - ) + policy = RuntimeEquivalencePolicy(measurement_dialect=dialect) provider_marker_types = policy.measurement_dialect.measurement_feature_marker_types( key ) diff --git a/openhcs/core/equivalence/measurement_rows.py b/openhcs/core/equivalence/measurement_rows.py index 73d0d9dfb5..8f5a8b023c 100644 --- a/openhcs/core/equivalence/measurement_rows.py +++ b/openhcs/core/equivalence/measurement_rows.py @@ -72,6 +72,7 @@ ObjectInstanceKey, ) + @lru_cache(maxsize=64) def measurement_identity_fields( contract: RuntimeMeasurementRowIdentityContract, @@ -162,7 +163,9 @@ def measurement_row_sample_identity_key( for field_name, value in row.items() if value is not None and str(value).strip() ) - selected_fields = contract.selected_sample_identity_fields(normalized_present_fields) + selected_fields = contract.selected_sample_identity_fields( + normalized_present_fields + ) identity_values: list[tuple[str, object]] = [] for field_name, value in row.items(): normalized_field_name = normalize_runtime_identifier(field_name) diff --git a/openhcs/core/execution_state.py b/openhcs/core/execution_state.py index e5824c9b63..783891af3b 100644 --- a/openhcs/core/execution_state.py +++ b/openhcs/core/execution_state.py @@ -31,9 +31,7 @@ def __post_init__(self) -> None: self.auto_add_output_dataset, bool, ): - raise TypeError( - "auto_add_output_dataset must be a bool or None" - ) + raise TypeError("auto_add_output_dataset must be a bool or None") @classmethod def from_results_summary( diff --git a/openhcs/core/function_patterns.py b/openhcs/core/function_patterns.py index 373464139e..6529849019 100644 --- a/openhcs/core/function_patterns.py +++ b/openhcs/core/function_patterns.py @@ -255,7 +255,10 @@ def source_projection( ) -> MainFlowInputProjection | None: """Derive the primary source role independently of its pixel producer.""" main_flow_refs = main_flow_artifacts.ref_set() - if spec.ref() not in main_flow_refs or spec.ref() not in invocation_sources.ref_set(): + if ( + spec.ref() not in main_flow_refs + or spec.ref() not in invocation_sources.ref_set() + ): return None return ( MainFlowInputProjection.COMPLETE_PAYLOAD @@ -277,11 +280,15 @@ def from_source_declarations( """Project one unstored source occurrence from its exact declarations.""" main_flow_projection = cls.source_projection( - spec, main_flow_artifacts, invocation_sources, + spec, + main_flow_artifacts, + invocation_sources, ) edge_type = ( CompiledMetadataArtifactInputEdgePlan - if metadata_available and main_flow_projection is None and not source_binding_available + if metadata_available + and main_flow_projection is None + and not source_binding_available else cls ) return edge_type( @@ -297,7 +304,8 @@ def uses_runtime_storage(self) -> bool: return self.storage_plan is not None def requires_current_image_carrier( - self, source_bindings: CompiledSourceBindingPlan, + self, + source_bindings: CompiledSourceBindingPlan, ) -> bool: """Whether this unstored input consumes declared current source pixels.""" if self.uses_runtime_storage(): @@ -321,13 +329,16 @@ def resolve_unstored_payload( """Admit pre-call input pixels through their exact compiled origin.""" request = ( scope.runtime_adapter_request(primary_source_payload) - if request is None else request + if request is None + else request ) loaded_payload = scope.loaded_source_artifact_payload(request, self) if loaded_payload is not None: return loaded_payload return scope.declared_source_payload( - self.spec.ref(), primary_source_payload, loaded_artifact_payloads={}, + self.spec.ref(), + primary_source_payload, + loaded_artifact_payloads={}, request=request, ) @@ -394,9 +405,13 @@ def artifact_parameter_value(self, spec: ArtifactSpec) -> object: for parameter_name, value in self.kwargs: if parameter_name == spec.parameter_name: if value is None and spec.required: - raise ValueError(f"Required metadata artifact {spec.ref()!r} has no value.") + raise ValueError( + f"Required metadata artifact {spec.ref()!r} has no value." + ) return value - raise ValueError(f"Metadata artifact {spec.ref()!r} has no compiled parameter value.") + raise ValueError( + f"Metadata artifact {spec.ref()!r} has no compiled parameter value." + ) @dataclass(frozen=True, slots=True) @@ -691,13 +706,18 @@ def convert_input(self, payload: object, source_memory_type: str) -> object: source = MemoryType(source_memory_type) if source is self._input_memory_type and not source.is_gpu: mask = payload.mask - if project_image_mask_to_data_domain( - mask, payload, metadata=payload.metadata - ) is mask: + if ( + project_image_mask_to_data_domain( + mask, payload, metadata=payload.metadata + ) + is mask + ): return payload - return payload.with_pixels(source.convert_to( + return payload.with_pixels( + source.convert_to( payload.data, self._input_memory_type, self.input_device_id - ),) + ), + ) def main_flow_call_argument(self, source_payload: object) -> object: """Apply the captured processing declaration to current request pixels.""" @@ -853,7 +873,9 @@ def select_outputs( else compiled_output_plans ) if any(not isinstance(plan, ArtifactOutputPlan) for plan in active_outputs): - raise TypeError("Compiled artifact outputs must be ArtifactOutputPlan values.") + raise TypeError( + "Compiled artifact outputs must be ArtifactOutputPlan values." + ) selected: dict[ArtifactSpecRef, ArtifactOutputPlan] = {} for compiled_plan in active_outputs: runtime_plan = output_plans.get(compiled_plan.ref()) @@ -933,7 +955,8 @@ def from_normalized_item( artifact_graph: "ArtifactGraph | None" = None, ) -> "CompiledFunctionInvocation": contract_plan = ( - invocation_contract_provider(item, step_context) if artifact_graph is None + invocation_contract_provider(item, step_context) + if artifact_graph is None else artifact_graph.invocation_contract_plans[item.key] ) if contract_plan is None: @@ -945,7 +968,8 @@ def from_normalized_item( ) item = replace(item, contract=contract_plan.contract) artifact_selector = ( - declaration_provider(item, step_context) if artifact_graph is None + declaration_provider(item, step_context) + if artifact_graph is None else artifact_graph.invocation_declarations[item.key] ) if artifact_graph is None: @@ -983,7 +1007,9 @@ def from_normalized_item( f"paths {relative_parameters!r} but compilation supplied no " "CompilationPathResolver." ) - runtime_loaded_input_refs = frozenset(plan.ref() for plan in artifact_input_plans) + runtime_loaded_input_refs = frozenset( + plan.ref() for plan in artifact_input_plans + ) validated_kwargs = item.contract.validate_public_kwargs( public_kwargs, runtime_loaded_artifact_parameter_names=( @@ -1018,7 +1044,9 @@ def runtime_domain(self) -> RuntimeInvocationDomain: return RuntimeInvocationDomain.from_invocations(self.invocations) def main_flow_input_refs( - self, *, source_bindings: CompiledSourceBindingPlan, + self, + *, + source_bindings: CompiledSourceBindingPlan, ) -> tuple[ArtifactSpecRef, ...] | None: """Return exact main-flow refs, or None for an implicit image argument.""" @@ -1026,14 +1054,16 @@ def main_flow_input_refs( tuple( (invocation, invocation.artifact_input_edges) for invocation in self.invocations - ), source_bindings=source_bindings, + ), + source_bindings=source_bindings, ) def main_flow_input_refs_for_component( self, execution_scope: ComponentGroupScope, component_key: str | None, - *, source_bindings: CompiledSourceBindingPlan, + *, + source_bindings: CompiledSourceBindingPlan, ) -> tuple[ArtifactSpecRef, ...] | None: """Return main-flow refs active for one compiled component execution.""" @@ -1049,7 +1079,8 @@ def main_flow_input_refs_for_component( is not None ) return self._main_flow_input_refs( - invocations_and_edges, source_bindings=source_bindings, + invocations_and_edges, + source_bindings=source_bindings, ) @staticmethod @@ -1060,7 +1091,8 @@ def _main_flow_input_refs( Sequence[InvocationArtifactInputEdgePlan], ] ], - *, source_bindings: CompiledSourceBindingPlan, + *, + source_bindings: CompiledSourceBindingPlan, ) -> tuple[ArtifactSpecRef, ...] | None: """Return exact main-flow refs for the supplied active invocations.""" @@ -1099,9 +1131,11 @@ def stored_primary_input_edges_for_component( for invocation in self.invocations if ( active_outputs := invocation.output_plans_for_component( - execution_scope, component_key, + execution_scope, + component_key, ) - ) is not None + ) + is not None ) if not active or any( invocation.contract.accepts_implicit_main_flow_input @@ -1130,10 +1164,11 @@ def stored_primary_input_edges_for_component( primary_edges = tuple( edge for invocation, edges in active - for domain_refs in (invocation.contract.invocation_domain_inputs.ref_set(),) + for domain_refs in ( + invocation.contract.invocation_domain_inputs.ref_set(), + ) for edge in edges - if edge.spec.ref() - in domain_refs + if edge.spec.ref() in domain_refs and edge.spec.artifact_type.carries_source_image_context ) if not primary_edges or any( diff --git a/openhcs/core/function_reference.py b/openhcs/core/function_reference.py index 6fb96799c2..49d01179f4 100644 --- a/openhcs/core/function_reference.py +++ b/openhcs/core/function_reference.py @@ -79,11 +79,13 @@ def source_literal(self) -> str: return self.source_expression(self.source_import_identity.function_name) def source_literal_with_names( - self, name_mappings: Mapping[tuple[str, str], str], + self, + name_mappings: Mapping[tuple[str, str], str], ) -> str: identity = self.source_import_identity imported_name = name_mappings.get( - (identity.module_name, identity.function_name), identity.function_name, + (identity.module_name, identity.function_name), + identity.function_name, ) return self.source_expression(imported_name) @@ -176,7 +178,8 @@ def resolve(self) -> Callable: @dataclass(frozen=True, kw_only=True) class ModuleExportRegistryFunctionReference( - ModuleExportFunctionSource, RegistryFunctionReference, + ModuleExportFunctionSource, + RegistryFunctionReference, ): """Direct source import with registry-owned transport and wrapper identity.""" @@ -412,7 +415,8 @@ def _registry_reference( ) identity = function_metadata.import_identity exported = FunctionReferenceTransportAuthority.importable_function( - identity.module_name, identity.function_name, + identity.module_name, + identity.function_name, ) reference_type = ( ModuleExportRegistryFunctionReference diff --git a/openhcs/core/image_file_serialization.py b/openhcs/core/image_file_serialization.py index 17a897c95b..41074a2c27 100644 --- a/openhcs/core/image_file_serialization.py +++ b/openhcs/core/image_file_serialization.py @@ -158,7 +158,9 @@ def require_image_geometry(self) -> tuple[Any, int, int]: """Require complete format-declared dtype and physical image dimensions.""" if self.image_shape_yx is None or self.source_dtype is None: - raise ValueError("Image-file header does not declare image dtype and YX geometry.") + raise ValueError( + "Image-file header does not declare image dtype and YX geometry." + ) height, width = self.image_shape_yx return self.source_dtype, height, width @@ -415,9 +417,7 @@ def persisted_metadata(self, path: Path, payload: Any) -> ImagePayloadMetadata: payload = ImagePayload.of(payload) return header.project_image_metadata( payload.metadata, - values_preserved=self.preserves_pixel_values( - payload.data.dtype - ), + values_preserved=self.preserves_pixel_values(payload.data.dtype), ) def requires_plane_store_decoder(self, path: Path) -> bool: diff --git a/openhcs/core/image_payload_execution_mode.py b/openhcs/core/image_payload_execution_mode.py index 859666e5ff..62666f1ac5 100644 --- a/openhcs/core/image_payload_execution_mode.py +++ b/openhcs/core/image_payload_execution_mode.py @@ -43,7 +43,8 @@ def execute( @classmethod def batch_request( - cls, request: "RuntimeBatchInvocationRequest", + cls, + request: "RuntimeBatchInvocationRequest", ) -> "RuntimeBatchInvocationRequest | None": """Project a request into a batch executor's image domain, or ``None``.""" return request @@ -57,7 +58,8 @@ def executable_payload(cls, payload: Any) -> Any: @classmethod def for_payload_scoped_labels( - cls, runtime_slice_count: int | None, + cls, + runtime_slice_count: int | None, ) -> type[ImagePayloadExecutionMode]: """Return the mode for labels whose domain is the whole payload.""" del runtime_slice_count @@ -65,7 +67,8 @@ def for_payload_scoped_labels( @classmethod def for_plane_scoped_labels( - cls, projection: "RuntimePlaneAxisValueProjection | None", + cls, + projection: "RuntimePlaneAxisValueProjection | None", ) -> type[ImagePayloadExecutionMode]: """Return the mode for labels that still carry an unprojected plane axis.""" del projection @@ -237,7 +240,11 @@ def execute(cls, contract, call, image, kwargs): ) def execute_aligned_stack_slice( - slice_func, slice_payload, slice_kwargs, slice_index, slice_count, + slice_func, + slice_payload, + slice_kwargs, + slice_index, + slice_count, ): return call.invoke( slice_payload, diff --git a/openhcs/core/measurement_dialect.py b/openhcs/core/measurement_dialect.py index 3ec0b427cf..77b65c45c4 100644 --- a/openhcs/core/measurement_dialect.py +++ b/openhcs/core/measurement_dialect.py @@ -71,8 +71,6 @@ def __post_init__(self) -> None: ) - - class RuntimeMeasurementQualifierValueMode(str, Enum): """How a row qualifier value is rendered into a feature suffix.""" @@ -81,8 +79,6 @@ class RuntimeMeasurementQualifierValueMode(str, Enum): FRACTION_OF_COUNT = "fraction_of_count" - - @dataclass(frozen=True, slots=True) class RuntimeMeasurementRowQualifier: """Declarative row fields that qualify measurement feature names.""" @@ -260,7 +256,6 @@ def __post_init__(self) -> None: ) - def _parts(prefix: Iterable[str]) -> FeatureParts: return tuple(part for part in prefix if part) @@ -314,7 +309,9 @@ class MeasurementDialect(ABC, metaclass=AutoRegisterMeta): source_qualifier_suffix_tokens: ClassVar[frozenset[str]] = frozenset() _declaration_generation: ClassVar[int] = 0 - _shared_instances: ClassVar[dict[type["MeasurementDialect"], "MeasurementDialect"]] = {} + _shared_instances: ClassVar[ + dict[type["MeasurementDialect"], "MeasurementDialect"] + ] = {} def __init__(self) -> None: self._cached_values: dict[str, object] = {} @@ -357,7 +354,9 @@ def registered_dialects(cls) -> tuple["MeasurementDialect", ...]: @classmethod def unqualified_sample_names(cls) -> frozenset[str]: """Source names that mean "the sample itself" in any registered dialect.""" - return _unqualified_sample_names(tuple(MeasurementDialect.__registry__.values())) + return _unqualified_sample_names( + tuple(MeasurementDialect.__registry__.values()) + ) @classmethod def for_active_family(cls) -> "MeasurementDialect": @@ -451,7 +450,9 @@ def measurement_feature_marker_declarations( del key return () - def indexed_descriptor_suffix_width(self, feature_parts: FeatureParts) -> int | None: + def indexed_descriptor_suffix_width( + self, feature_parts: FeatureParts + ) -> int | None: """Trailing descriptor-index token width of one feature, if indexed.""" del feature_parts return None @@ -706,14 +707,21 @@ def feature_parts(self, parts: FeatureParts) -> FeatureParts: """Return dialect-normalized feature parts for one lookup token.""" resolved_parts = parts for prefix in self.category_prefixes(): - if len(resolved_parts) > len(prefix) and resolved_parts[: len(prefix)] == prefix: + if ( + len(resolved_parts) > len(prefix) + and resolved_parts[: len(prefix)] == prefix + ): resolved_parts = resolved_parts[len(prefix) :] break return self.feature_part_aliases().get(resolved_parts, resolved_parts) - def alternative_feature_parts(self, parts: FeatureParts) -> tuple[FeatureParts, ...]: + def alternative_feature_parts( + self, parts: FeatureParts + ) -> tuple[FeatureParts, ...]: """Return ordered semantic fallback fields for one lookup token.""" - return self.alternative_feature_part_aliases().get(self.feature_parts(parts), ()) + return self.alternative_feature_part_aliases().get( + self.feature_parts(parts), () + ) def feature_lookup(self, feature_name: str) -> "RuntimeMeasurementFeatureLookup": """Return the lookup identity for one external feature name.""" @@ -928,8 +936,7 @@ def qualifier_sequence_identifies_feature_suffix( return True base_tokens = feature_tokens[: len(feature_tokens) - len(suffix_tokens)] return any( - base_tokens == prefix - for prefix in self.scale_qualified_feature_prefixes() + base_tokens == prefix for prefix in self.scale_qualified_feature_prefixes() ) def feature_suffix_source_insertion_index( @@ -992,7 +999,6 @@ def source_qualified_feature_family( return None - @lru_cache(maxsize=8) def _unqualified_sample_names( dialect_types: tuple[type[MeasurementDialect], ...], diff --git a/openhcs/core/measurement_feature_queries.py b/openhcs/core/measurement_feature_queries.py index 00d1d7225d..9b672deefc 100644 --- a/openhcs/core/measurement_feature_queries.py +++ b/openhcs/core/measurement_feature_queries.py @@ -635,8 +635,8 @@ def _project_feature_indexes( row_indices[effective_mask], object_values, strict=True ): if row_index not in object_labels_by_row: - object_labels_by_row[row_index] = measurement_object_label_value( - object_ids[row_index] + object_labels_by_row[row_index] = ( + measurement_object_label_value(object_ids[row_index]) ) object_label = object_labels_by_row[row_index] if object_label is not None: @@ -693,9 +693,7 @@ class MeasurementFeatureQuery: feature_name: str object_name: str | None = None - dialect: MeasurementDialectLike = ( - EXECUTING_MEASUREMENT_DIALECT - ) + dialect: MeasurementDialectLike = EXECUTING_MEASUREMENT_DIALECT def __post_init__(self) -> None: if not self.feature_name: @@ -907,9 +905,7 @@ class MeasurementObjectFeatureVectorBatchQuery: feature_name: str object_names: tuple[str, ...] - dialect: MeasurementDialectLike = ( - EXECUTING_MEASUREMENT_DIALECT - ) + dialect: MeasurementDialectLike = EXECUTING_MEASUREMENT_DIALECT def __post_init__(self) -> None: if not self.feature_name: @@ -932,9 +928,9 @@ def value_indexes( ) -> MeasurementValueIndexesByObject: """Return current object-feature indexes from one correlated table pass.""" object_names = self.normalized_object_names - lookup = resolve_measurement_dialect( - self.dialect - ).feature_lookup(self.feature_name) + lookup = resolve_measurement_dialect(self.dialect).feature_lookup( + self.feature_name + ) query_objects_by_requested_object = { object_name: lookup.query_object_name(object_name) for object_name in object_names @@ -990,9 +986,9 @@ def value_indexes_by_axis( ) -> dict[int, MeasurementValueIndexesByObject] | None: """Return current feature indexes preserving complete object/axis rows.""" object_names = self.normalized_object_names - lookup = resolve_measurement_dialect( - self.dialect - ).feature_lookup(self.feature_name) + lookup = resolve_measurement_dialect(self.dialect).feature_lookup( + self.feature_name + ) query_objects_by_requested_object = { object_name: lookup.query_object_name(object_name) for object_name in object_names @@ -1091,9 +1087,9 @@ def table_object_names( semantics = ColumnarMeasurementTableSchema.from_table(table) if not semantics.object_names(table): return requested_object_names - feature_lookup = resolve_measurement_dialect( - self.dialect - ).feature_lookup(self.feature_name) + feature_lookup = resolve_measurement_dialect(self.dialect).feature_lookup( + self.feature_name + ) return tuple( object_name for object_name in requested_object_names @@ -1388,15 +1384,11 @@ def normalize_measurement_token(value: object) -> str: def ordered_measurement_feature_candidates( feature_name: str, *, - dialect: MeasurementDialectLike = ( - EXECUTING_MEASUREMENT_DIALECT - ), + dialect: MeasurementDialectLike = (EXECUTING_MEASUREMENT_DIALECT), ) -> tuple[str, ...]: """Return schema-safe feature-field aliases from most specific to least specific.""" return ( - resolve_measurement_dialect(dialect) - .feature_lookup(feature_name) - .field_aliases + resolve_measurement_dialect(dialect).feature_lookup(feature_name).field_aliases ) @@ -1457,9 +1449,7 @@ def measurement_values_for_feature( object_count: int, object_ids: Sequence[int] | None = None, object_name: str | None = None, - dialect: MeasurementDialectLike = ( - EXECUTING_MEASUREMENT_DIALECT - ), + dialect: MeasurementDialectLike = (EXECUTING_MEASUREMENT_DIALECT), ) -> Any: """Return object-indexed measurement values for one feature.""" query = MeasurementFeatureQuery( diff --git a/openhcs/core/measurement_image_alignment.py b/openhcs/core/measurement_image_alignment.py index 1528fc95ea..5ea13be7ff 100644 --- a/openhcs/core/measurement_image_alignment.py +++ b/openhcs/core/measurement_image_alignment.py @@ -403,10 +403,12 @@ def object_label_reference_image(self) -> RuntimeArrayData | AlignedImageStack: "Object-label reference alignment requires an image source domain " "adapter." ) - return request.image.with_pixels(label_domain_adapter.extract_source_array( + return request.image.with_pixels( + label_domain_adapter.extract_source_array( request.image_array, spatial_axes_yx=image_domain_adapter.spatial_axes_yx, - ),) + ), + ) @classmethod def validate_alignment( diff --git a/openhcs/core/measurement_row_materialization.py b/openhcs/core/measurement_row_materialization.py index 8e1ba7b116..6b9a900e7a 100644 --- a/openhcs/core/measurement_row_materialization.py +++ b/openhcs/core/measurement_row_materialization.py @@ -2419,7 +2419,6 @@ def metadata_for_row(row_index: int): if not all(is_structural_missing_measurement_cell(value) for value in values): coordinate_columns[component.name] = values - def source_names_for_row(row_index: int) -> tuple[str, ...]: if slice_values is None: return source_provenance.represented_source_image_names diff --git a/openhcs/core/memory/decorators.py b/openhcs/core/memory/decorators.py index e72b96f42d..f4141be461 100644 --- a/openhcs/core/memory/decorators.py +++ b/openhcs/core/memory/decorators.py @@ -97,7 +97,9 @@ def _admits_plain_image_payload(annotation: Any) -> bool: annotation = resolve_annotated(annotation) if is_union_type(annotation): - return any(_admits_plain_image_payload(member) for member in get_args(annotation)) + return any( + _admits_plain_image_payload(member) for member in get_args(annotation) + ) return annotation is ImagePayload @@ -160,7 +162,9 @@ def image_payload_callable(*args: Any, **kwargs: Any) -> Any: result = wrapped(*args, **kwargs) # Bare pixels in, bare pixels out: a caller that passed an array (an # array library, slice-by-slice stacking) gets arrays back. - if primary in positions.values() and not isinstance(primary_value, ImagePayload): + if primary in positions.values() and not isinstance( + primary_value, ImagePayload + ): return as_bare(result) return result @@ -231,7 +235,9 @@ def _signature_with_resolved_raw_annotations(wrapped: Any) -> inspect.Signature: def _strip_unowned_semantic_controls(wrapped: Any, contract: Any) -> None: - allowed_semantic_control_names = contract.injected_semantic_control_parameter_names() + allowed_semantic_control_names = ( + contract.injected_semantic_control_parameter_names() + ) semantic_control_names = { parameter_type.require_parameter_name() for parameter_type in contract.semantic_control_parameter_types() diff --git a/openhcs/core/metadata_cache.py b/openhcs/core/metadata_cache.py index c5058551c9..2e77372c5d 100644 --- a/openhcs/core/metadata_cache.py +++ b/openhcs/core/metadata_cache.py @@ -106,4 +106,3 @@ def _is_cache_valid(self) -> bool: ): return False return True - diff --git a/openhcs/core/orchestrator/bundle_execution.py b/openhcs/core/orchestrator/bundle_execution.py index cedcef5927..99f041da39 100644 --- a/openhcs/core/orchestrator/bundle_execution.py +++ b/openhcs/core/orchestrator/bundle_execution.py @@ -1055,6 +1055,7 @@ def wait_until_visualizers_ready( def not_ready_ports() -> list[int]: running_ports = {v.port for v in visualizers if v.is_running} return [v.port for v in visualizers if v.port not in running_ports] + start_time = time.time() while time.time() - start_time < max_wait: diff --git a/openhcs/core/orchestrator/execution_result.py b/openhcs/core/orchestrator/execution_result.py index 157ca6db78..39459d46a7 100644 --- a/openhcs/core/orchestrator/execution_result.py +++ b/openhcs/core/orchestrator/execution_result.py @@ -60,7 +60,9 @@ class RuntimeContextObservation: context_key: str records: tuple[StoredRuntimeValue, ...] - outputs: StepExecutionObservation = field(default_factory=StepExecutionObservation.empty) + outputs: StepExecutionObservation = field( + default_factory=StepExecutionObservation.empty + ) @classmethod def from_context( @@ -91,15 +93,17 @@ def from_completed_outputs( cls, contexts: Mapping[str, ProcessingContext] ) -> "RuntimeExecutionObservation": """Project saved facts independently of retained runtime pixel records.""" - return cls(contexts=tuple( - RuntimeContextObservation( - context_key=context_key, - records=(), - outputs=context.completed_step_outputs, + return cls( + contexts=tuple( + RuntimeContextObservation( + context_key=context_key, + records=(), + outputs=context.completed_step_outputs, + ) + for context_key, context in contexts.items() + if not context.completed_step_outputs.is_empty ) - for context_key, context in contexts.items() - if not context.completed_step_outputs.is_empty - )) + ) def merge_into(self, execution_contexts: Mapping[str, ProcessingContext]) -> None: """Merge returned runtime records into parent-owned compiled contexts.""" @@ -249,7 +253,8 @@ def error( error_message=error_message, runtime_observation=( RuntimeExecutionObservation() - if runtime_observation is None else runtime_observation + if runtime_observation is None + else runtime_observation ), ) @@ -268,6 +273,7 @@ def cancelled( error_message=error_message, runtime_observation=( RuntimeExecutionObservation() - if runtime_observation is None else runtime_observation + if runtime_observation is None + else runtime_observation ), ) diff --git a/openhcs/core/orchestrator/orchestrator.py b/openhcs/core/orchestrator/orchestrator.py index c46b482582..ac847cfbbe 100644 --- a/openhcs/core/orchestrator/orchestrator.py +++ b/openhcs/core/orchestrator/orchestrator.py @@ -575,7 +575,9 @@ def _ensure_openhcs_metadata(self, resolved_config: GlobalPipelineConfig) -> Non source_role = self.microscope_handler.source_selection_role() if not source_role.requires_local_directory: - logger.debug("Skipping local metadata creation for %s", source_role.role_name) + logger.debug( + "Skipping local metadata creation for %s", source_role.role_name + ) return # For datasets with a virtual workspace, metadata is already created by _build_virtual_mapping() @@ -666,9 +668,7 @@ def create_context( # Extract cached metadata from service and convert to dict format expected by OpenHCSMetadataGenerator metadata_dict = {} for component in AxisFamily.active().axes: - cached_metadata = self.metadata_cache.get_cached_metadata( - component - ) + cached_metadata = self.metadata_cache.get_cached_metadata(component) if cached_metadata: metadata_dict[component] = cached_metadata context.metadata_cache = metadata_dict @@ -903,9 +903,7 @@ def cache_component_keys( if components is None: components = list(AxisFamily.active().axes) - logger.info( - f"Caching component keys for: {[comp.name for comp in components]}" - ) + logger.info(f"Caching component keys for: {[comp.name for comp in components]}") try: axis_values = self.microscope_handler.axis_values( diff --git a/openhcs/core/orchestrator/worker_execution.py b/openhcs/core/orchestrator/worker_execution.py index 0528dbdb70..b30f98c40e 100644 --- a/openhcs/core/orchestrator/worker_execution.py +++ b/openhcs/core/orchestrator/worker_execution.py @@ -771,7 +771,9 @@ class WorkerProcess: def send(self, command: WorkerCommand) -> None: self.connection.send(command) - def receive(self, progress_queue: ProgressQueue | None = None) -> WorkerResult | None: + def receive( + self, progress_queue: ProgressQueue | None = None + ) -> WorkerResult | None: """Read one reply; return the result it completes, if any.""" try: @@ -980,7 +982,9 @@ def release(self, workers: list[WorkerProcess], *, retire: bool) -> None: worker.process.join() worker.connection.close() self.workers = [ - owned for owned in self.workers if owned.process is not worker.process + owned + for owned in self.workers + if owned.process is not worker.process ] def close(self) -> None: @@ -1631,12 +1635,13 @@ def execute_worker_lane( ) -> Dict[str, ExecutionResult]: """Execute a deterministic worker lane: its partitions in order, in one slot.""" - with CProfileWorkerProfilingPolicy.from_environment().profile( - lane_context - ), RuntimeProfileLogger.run( - execution_id=lane_context.execution_id, - worker_slot=lane_context.worker_slot, - owned_partition_values=tuple(lane_context.owned_partition_values), + with ( + CProfileWorkerProfilingPolicy.from_environment().profile(lane_context), + RuntimeProfileLogger.run( + execution_id=lane_context.execution_id, + worker_slot=lane_context.worker_slot, + owned_partition_values=tuple(lane_context.owned_partition_values), + ), ): lane_results: Dict[str, ExecutionResult] = {} for axis_id, axis_contexts in lane_axis_contexts: diff --git a/openhcs/core/orchestrator/worker_profiling.py b/openhcs/core/orchestrator/worker_profiling.py index e9c7b4dc65..9f37b01769 100644 --- a/openhcs/core/orchestrator/worker_profiling.py +++ b/openhcs/core/orchestrator/worker_profiling.py @@ -72,12 +72,7 @@ def profile( yield finally: profiler.disable() - profiler.dump_stats( - str( - self.output_dir - / self.profile_filename(lane) - ) - ) + profiler.dump_stats(str(self.output_dir / self.profile_filename(lane))) @abstractmethod def configure_profile_event_scope(self) -> None: diff --git a/openhcs/core/partition_filter.py b/openhcs/core/partition_filter.py index 1512e69b17..a535f7247e 100644 --- a/openhcs/core/partition_filter.py +++ b/openhcs/core/partition_filter.py @@ -66,7 +66,9 @@ def from_value(cls, value: list) -> "PartitionValueList": def matches(self, available_values: Sequence[str], *, strict: bool) -> Set[str]: available_keys = {_identity_key(value) for value in available_values} - unknown = [value for value in self.values if _identity_key(value) not in available_keys] + unknown = [ + value for value in self.values if _identity_key(value) not in available_keys + ] if unknown and strict: raise ValueError( f"Invalid partition values specified: {unknown}. " @@ -75,7 +77,9 @@ def matches(self, available_values: Sequence[str], *, strict: bool) -> Set[str]: if unknown: logger.warning(f"Ignoring unknown partition values in filter: {unknown}") requested = {_identity_key(value) for value in self.values} - return {value for value in available_values if _identity_key(value) in requested} + return { + value for value in available_values if _identity_key(value) in requested + } @dataclass(frozen=True) @@ -136,7 +140,9 @@ def matches(self, available_values: Sequence[str], *, strict: bool) -> Set[str]: if pattern.startswith(self.ROW_PREFIX): return self._row(pattern[len(self.ROW_PREFIX) :].strip(), available_values) if pattern.startswith(self.COL_PREFIX): - return self._column(pattern[len(self.COL_PREFIX) :].strip(), available_values) + return self._column( + pattern[len(self.COL_PREFIX) :].strip(), available_values + ) if self.RANGE_SEPARATOR in pattern: return self._range(pattern, available_values) return {pattern} @@ -148,7 +154,9 @@ def _row(row: str, available_values: Sequence[str]) -> Set[str]: # Opera Phenix format fallback (A → R01C*, B → R02C*) if not result and len(row) == 1 and row.isalpha(): row_pattern = f"R{ord(row.upper()) - ord('A') + 1:02d}C" - result = {value for value in available_values if value.startswith(row_pattern)} + result = { + value for value in available_values if value.startswith(row_pattern) + } return result @staticmethod @@ -164,7 +172,9 @@ def numeric_suffix(value: str) -> int: digits = "".join(char for char in reversed(value) if char.isdigit()) return int(digits[::-1]) if digits else 0 - result = {value for value in available_values if numeric_suffix(value) in col_range} + result = { + value for value in available_values if numeric_suffix(value) in col_range + } # Opera Phenix format fallback (C01, C02, etc.) if not result: patterns = {f"C{col:02d}" for col in col_range} @@ -213,7 +223,8 @@ def resolve_filter_with_mode( result = [ value for value in available_values - if (_identity_key(value) in matched_keys) is partition_filter_mode.keeps_matches + if (_identity_key(value) in matched_keys) + is partition_filter_mode.keeps_matches ] logger.debug( "Partition filter %r (%s) over %s selects %s", diff --git a/openhcs/core/pipeline/compilation_session.py b/openhcs/core/pipeline/compilation_session.py index d3f7296d9b..01d5840b6b 100644 --- a/openhcs/core/pipeline/compilation_session.py +++ b/openhcs/core/pipeline/compilation_session.py @@ -250,9 +250,9 @@ def _admit_artifact_graphs(self) -> None: step, index, execution_scope=FunctionStepExecutionScope.require_uniform( - () if graph.pattern is None else ( - item.contract for item in graph.pattern.iter_items() - ) + () + if graph.pattern is None + else (item.contract for item in graph.pattern.iter_items()) ), source_bindings=step.source_bindings, context=context, @@ -339,9 +339,12 @@ def validate_source_group_domains( or context.group_by is Ungrouped ): continue - if FunctionStepExecutionScope.require_uniform( - item.contract for item in pattern.iter_items() - ) is FunctionStepExecutionScope.PLATE: + if ( + FunctionStepExecutionScope.require_uniform( + item.contract for item in pattern.iter_items() + ) + is FunctionStepExecutionScope.PLATE + ): continue result = validator.validate_dict_pattern_keys( pattern, diff --git a/openhcs/core/pipeline/compiler.py b/openhcs/core/pipeline/compiler.py index 67f7a5e04c..dd06a27294 100644 --- a/openhcs/core/pipeline/compiler.py +++ b/openhcs/core/pipeline/compiler.py @@ -283,9 +283,7 @@ def _configure_input_conversion_if_needed( if vfs_config.materialization_backend != MaterializationBackend.ZARR: return - available_backends = context.microscope_handler.available_backends( - dataset_root - ) + available_backends = context.microscope_handler.available_backends(dataset_root) if Backend.ZARR in available_backends: return @@ -507,7 +505,8 @@ def declare_zarr_stores(session: CompilationSession) -> None: last_step_index = session.step_count - 1 session.plan(last_step_index).zarr_config = { "all_wells": session.orchestrator.get_component_keys( - AxisFamily.active().partition_axis(), resolved_config=session.global_config + AxisFamily.active().partition_axis(), + resolved_config=session.global_config, ), "needs_initialization": True, } @@ -560,7 +559,8 @@ def plan_materialization_flags( f"Missing required keys: {missing_keys}." ) PipelineCompiler._compile_runtime_artifact_materialization_plans( - session, materialization_planner, + session, + materialization_planner, ) @staticmethod @@ -582,10 +582,8 @@ def _compile_runtime_artifact_materialization_plans( persistent_step_indexes.add(step_index) if persistent_step_indexes: - persistent_backend = ( - materialization_planner.resolve_backend( - session.global_config.vfs_config.materialization_backend, - ) + persistent_backend = materialization_planner.resolve_backend( + session.global_config.vfs_config.materialization_backend, ) for step_index, step_plan in session.plans.items(): persistent_enabled = step_index in persistent_step_indexes @@ -679,7 +677,6 @@ def analyze_pipeline_sequential_mode( seq_comp, resolved_config=global_config ) - if not component_values: logger.warning(f"No {seq_comp} values found in orchestrator cache") component_values_lists.append([]) @@ -892,7 +889,10 @@ def _axis_values_to_process( partition_filter_config = ( effective_config.partition_filter_config if effective_config else None ) - if partition_filter_config and partition_filter_config.partition_filter is not None: + if ( + partition_filter_config + and partition_filter_config.partition_filter is not None + ): available_values = orchestrator.get_component_keys( AxisFamily.active().partition_axis(), resolved_config=effective_config ) @@ -1088,7 +1088,8 @@ def _compile_sequential_axis_contexts( ) PipelineCompiler.declare_zarr_stores(session) PipelineCompiler.plan_materialization_flags( - session, request.materialization_planner, + session, + request.materialization_planner, ) PipelineCompiler._run_post_plan_compile_stages( session, @@ -1161,11 +1162,9 @@ def validate_payload_axis_requirements( f"{group.group_key!r}, invocation " f"{invocation.contract.function_name!r}" ) - prefix_proof = ( - group.payload_axis_proof( - requirement, - stop_before=invocation_index, - ) + prefix_proof = group.payload_axis_proof( + requirement, + stop_before=invocation_index, ) prefix_proof.validate_obligation( failure_message=lambda unproved: ( @@ -1249,8 +1248,11 @@ def _validate_payload_axis_source_anchor( and source_binding_plan.primary_plane_bindings ): PipelineCompiler._validate_pipeline_start_payload_axis_requirement( - session=session, source_binding_plan=source_binding_plan, - owner=owner, requirement=requirement, failures=failures, + session=session, + source_binding_plan=source_binding_plan, + owner=owner, + requirement=requirement, + failures=failures, ) return True if plan.requires_terminal_source_projection: @@ -1273,9 +1275,13 @@ def _validate_inherited_payload_axis_requirement( """Validate inheritance until a declaration creates the required payload axis.""" if PipelineCompiler._validate_payload_axis_source_anchor( - plan, group, session=session, owner=owner, + plan, + group, + session=session, + owner=owner, grouped_pattern=plan.compiled_function_pattern.is_grouped, - requirement=requirement, failures=failures, + requirement=requirement, + failures=failures, ): return True try: @@ -1317,9 +1323,7 @@ def _validate_inherited_payload_axis_requirement( f"empty producer group at step {producer.step_index}." ) return True - proof = producer_group.payload_axis_proof( - requirement - ) + proof = producer_group.payload_axis_proof(requirement) if proof.validate_obligation( failure_message=lambda unproved: ( f"{owner}: requirement {requirement.label} is not " @@ -1327,9 +1331,13 @@ def _validate_inherited_payload_axis_requirement( f"{unproved.contract.function_name!r}." ), source_validation=lambda: PipelineCompiler._validate_payload_axis_source_anchor( - producer, producer_group, session=session, owner=owner, + producer, + producer_group, + session=session, + owner=owner, grouped_pattern=producer.compiled_function_pattern.is_grouped, - requirement=requirement, failures=failures, + requirement=requirement, + failures=failures, ), failures=failures, ): @@ -1600,7 +1608,9 @@ def compile_pipelines( ), ) path_resolver = CompilationPathResolver( - dataset_scope=CompilationDatasetScope.from_path(orchestrator.dataset_root), + dataset_scope=CompilationDatasetScope.from_path( + orchestrator.dataset_root + ), filemanager=orchestrator.filemanager, backend=Backend.DISK.value, ) @@ -1627,10 +1637,12 @@ def compile_pipelines( orchestrator, effective_config.vfs_config, ) - global_step_axis_filters = PipelineCompiler._resolve_global_step_axis_filters( - orchestrator, - pipeline_inputs, - effective_config, + global_step_axis_filters = ( + PipelineCompiler._resolve_global_step_axis_filters( + orchestrator, + pipeline_inputs, + effective_config, + ) ) axis_request = AxisCompilationRequest( orchestrator=orchestrator, @@ -1644,7 +1656,8 @@ def compile_pipelines( filemanager=orchestrator.filemanager, input_dir=orchestrator.input_dir, available_axis_values=orchestrator.get_component_keys( - AxisFamily.active().partition_axis(), resolved_config=effective_config + AxisFamily.active().partition_axis(), + resolved_config=effective_config, ), ), source_projections_by_axis=DEFAULT_SOURCE_PROJECTION_CACHE.partition_by_axes( @@ -1753,7 +1766,10 @@ def _resolve_step_axis_filters( for step_index, step in enumerate(pipeline.steps): step_filters: dict[type[PartitionFilterConfig], StepAxisFilterResolution] = {} for field_name, config in vars(step).items(): - if not isinstance(config, PartitionFilterConfig) or config.partition_filter is None: + if ( + not isinstance(config, PartitionFilterConfig) + or config.partition_filter is None + ): continue provenance = pipeline.step_provenance[step_index].get( f"{field_name}.partition_filter" diff --git a/openhcs/core/pipeline/funcstep_contract_validator.py b/openhcs/core/pipeline/funcstep_contract_validator.py index e9b97c74a6..6198e8ba23 100644 --- a/openhcs/core/pipeline/funcstep_contract_validator.py +++ b/openhcs/core/pipeline/funcstep_contract_validator.py @@ -1247,4 +1247,3 @@ def validate_allowed_group_by( f"group_by axes with role {allowed_names}; resolved " f"{resolved_group_by.name}." ) - diff --git a/openhcs/core/pipeline/function_contracts.py b/openhcs/core/pipeline/function_contracts.py index c79d486e06..6a0f6efc29 100644 --- a/openhcs/core/pipeline/function_contracts.py +++ b/openhcs/core/pipeline/function_contracts.py @@ -80,9 +80,16 @@ def resolved_callable_type_hints(func: Callable) -> dict[str, Any]: from typing import _strip_annotations return { - **{name: _strip_annotations(parameter.annotation) for name, parameter in signature.parameters.items() - if parameter.annotation is not inspect.Parameter.empty}, - **({} if signature.return_annotation is inspect.Signature.empty else {"return": _strip_annotations(signature.return_annotation)}), + **{ + name: _strip_annotations(parameter.annotation) + for name, parameter in signature.parameters.items() + if parameter.annotation is not inspect.Parameter.empty + }, + **( + {} + if signature.return_annotation is inspect.Signature.empty + else {"return": _strip_annotations(signature.return_annotation)} + ), } @@ -182,9 +189,7 @@ def invocation_kwargs( @classmethod def semantic_label_payload(cls, source_projected_payload, completion_payload): """Return the label payload that represents this measurement domain.""" - raise TypeError( - f"{cls.__name__} declares no object-measurement label domain." - ) + raise TypeError(f"{cls.__name__} declares no object-measurement label domain.") @classmethod def image_execution_mode( diff --git a/openhcs/core/pipeline/path_planner.py b/openhcs/core/pipeline/path_planner.py index f81d6a71cc..2e65c8b3b0 100644 --- a/openhcs/core/pipeline/path_planner.py +++ b/openhcs/core/pipeline/path_planner.py @@ -286,7 +286,9 @@ def context_producer_scope( output_plan, source_component ) if producer.producer_step_index is not None: - scope = self.planner.plans[producer.producer_step_index].execution_group_scope + scope = self.planner.plans[ + producer.producer_step_index + ].execution_group_scope return PathPlannerGroupScope.from_raw(scope.keys, component=scope.component) if producer.spec.ref() != PIPELINE_INPUT_ARTIFACT.ref(): return PathPlannerGroupScope.from_raw( @@ -301,7 +303,8 @@ def context_producer_scope( ) return ( PathPlannerGroupScope.from_raw(source_keys, component=source_component) - if source_keys else PathPlannerGroupScope.dynamic(source_component) + if source_keys + else PathPlannerGroupScope.dynamic(source_component) ) def get_execution_groups( @@ -431,7 +434,8 @@ def artifact_owned_execution_scope( continue has_source_lineage = ( - contract_source_bindings.binding_for_artifact_ref(spec.ref()) is not None + contract_source_bindings.binding_for_artifact_ref(spec.ref()) + is not None or available_artifacts.by_name_and_artifact_type( spec.name, spec.artifact_type, @@ -505,7 +509,9 @@ def dynamic_execution_scope_for_group_by( group_by: type[GroupingDeclaration] | None, ) -> PathPlannerGroupScope: """Return a typed runtime-discovered scope for a concrete group axis.""" - group_by_component = PathPlannerComponentScopes.component_from_group_by(group_by) + group_by_component = PathPlannerComponentScopes.component_from_group_by( + group_by + ) if group_by_component is None: return PathPlannerGroupScope.ungrouped() source_keys = ( @@ -535,7 +541,9 @@ def source_binding_scope_for_group_by( source_bindings: SourceBindingDeclarationsMixin | None = None, ) -> PathPlannerGroupScope: """Derive execution groups declared by source-binding component identity.""" - group_by_component = PathPlannerComponentScopes.component_from_group_by(group_by) + group_by_component = PathPlannerComponentScopes.component_from_group_by( + group_by + ) if group_by_component is None: return PathPlannerGroupScope.ungrouped() @@ -568,7 +576,6 @@ def execution_component_for_dict_pattern( ) return grouping_axes[0] - @staticmethod def normalized_group_by(step: AbstractStep) -> type[GroupingDeclaration]: """Use the same group_by normalization as compiled execution plans.""" @@ -603,9 +610,10 @@ def inject_metadata( ) -> FunctionPatternSyntax: """Inject values once; compiled metadata edges reference these kwargs.""" for spec in inputs.values(): - if ( - spec.ref().for_plan_type(ArtifactOutputPlan) not in self.planner.declared - and self.metadata_artifact_available(spec.name) + if spec.ref().for_plan_type( + ArtifactOutputPlan + ) not in self.planner.declared and self.metadata_artifact_available( + spec.name ): value = self.planner.ctx.microscope_handler.resolve_metadata_artifact( spec.name, @@ -641,11 +649,13 @@ def source_binding_component_domains( PathPlannerGroupScope.from_raw(values, component=component) for component in AxisFamily.active().axes for values in ( - tuple(dict.fromkeys( - value - for domains in binding_domains - for value in domains.get(component, ()) - )), + tuple( + dict.fromkeys( + value + for domains in binding_domains + for value in domains.get(component, ()) + ) + ), ) if values ) @@ -657,7 +667,9 @@ def compile_plan_maps( declarations: ArtifactGraph, group_scope: PathPlannerGroupScope, execution_scope: FunctionStepExecutionScope = FunctionStepExecutionScope.AXIS, - source_bindings: StepSourceBindingsConfig | CompiledSourceBindingPlan = EMPTY_SOURCE_BINDINGS, + source_bindings: ( + StepSourceBindingsConfig | CompiledSourceBindingPlan + ) = EMPTY_SOURCE_BINDINGS, ) -> ArtifactPlanMaps: """Compile artifact declarations into runtime I/O maps.""" step_name = step.name @@ -1084,7 +1096,9 @@ def compile_invocation_input_edges( ], execution_group_scope: PathPlannerGroupScope, consumer_variable_components: ComponentSet, - source_bindings: StepSourceBindingsConfig | CompiledSourceBindingPlan = EMPTY_SOURCE_BINDINGS, + source_bindings: ( + StepSourceBindingsConfig | CompiledSourceBindingPlan + ) = EMPTY_SOURCE_BINDINGS, available_artifacts: ArtifactSpecCollection = ArtifactSpecCollection(()), main_flow_artifacts: ArtifactSpecCollection = ArtifactSpecCollection(()), metadata_artifact_available: Callable[[str], bool] = lambda _name: False, @@ -1131,8 +1145,9 @@ def compile_invocation_input_edges( adapter is not None and adapter.manages_artifact_inputs ) raw_main_flow_refs = ( - main_flow_artifacts.ref_set() - .intersection(invocation.contract.output_group_scope_sources) + main_flow_artifacts.ref_set().intersection( + invocation.contract.output_group_scope_sources + ) if not owns_artifact_inputs else frozenset() ) @@ -1143,7 +1158,8 @@ def compile_invocation_input_edges( strict=True, ): storage_plan = ( - None if input_spec.ref() in raw_main_flow_refs + None + if input_spec.ref() in raw_main_flow_refs else selected_plans_by_ref.get(input_spec.ref()) ) if storage_plan is None: @@ -1153,8 +1169,12 @@ def compile_invocation_input_edges( spec=input_spec, main_flow_artifacts=main_flow_artifacts, invocation_sources=invocation.contract.group_scope_inputs, - metadata_available=metadata_artifact_available(input_spec.name), - source_binding_available=source_bindings.declares_artifact_ref(input_spec.ref()), + metadata_available=metadata_artifact_available( + input_spec.name + ), + source_binding_available=source_bindings.declares_artifact_ref( + input_spec.ref() + ), ) ) continue @@ -1169,11 +1189,15 @@ def compile_invocation_input_edges( consumer_variable_components=consumer_variable_components, source_bindings=source_bindings, available_artifacts=available_artifacts, - main_flow_projection=InvocationArtifactInputEdgePlan.source_projection( - input_spec, - main_flow_artifacts, - invocation.contract.group_scope_inputs, - ) if owns_artifact_inputs else None, + main_flow_projection=( + InvocationArtifactInputEdgePlan.source_projection( + input_spec, + main_flow_artifacts, + invocation.contract.group_scope_inputs, + ) + if owns_artifact_inputs + else None + ), ) ) invocations.append(invocation.with_artifact_input_edges(tuple(edges))) @@ -1258,13 +1282,15 @@ def invocation_input_edge( invocation=invocation, artifact_ref=artifact_ref, ) - producer_selection_scope = ArtifactInputProjectionPlan.producer_selection_for_invocation( - input_spec=input_spec, - storage_plan=storage_plan, - component_scopes=component_scopes, - consumer_variable_components=consumer_variable_components, - invocation_key=invocation.key, - execution_scope=invocation.contract.execution_scope, + producer_selection_scope = ( + ArtifactInputProjectionPlan.producer_selection_for_invocation( + input_spec=input_spec, + storage_plan=storage_plan, + component_scopes=component_scopes, + consumer_variable_components=consumer_variable_components, + invocation_key=invocation.key, + execution_scope=invocation.contract.execution_scope, + ) ) projection = ArtifactInputProjectionPlan( @@ -1769,7 +1795,9 @@ def materialized_output_dir_for_step( step_index, StepAxisFilterSet.empty(), ) - if not step_axis_filters.allows(materialization_config, self.planner.ctx.axis_id): + if not step_axis_filters.allows( + materialization_config, self.planner.ctx.axis_id + ): logger.debug( "Skipping materialization for step %s, axis %s (filtered out)", step.name, @@ -1856,7 +1884,9 @@ def validate(self) -> None: ): continue if dependency.kind is not StepInputDependencyKind.STEP_OUTPUT: - raise ValueError(f"Step {curr.name} has unresolved main input dependency.") + raise ValueError( + f"Step {curr.name} has unresolved main input dependency." + ) source_step_index = dependency.source_step_index if source_step_index is None: raise ValueError( @@ -1889,7 +1919,8 @@ def validate_materialization_paths(self) -> None: self.planner.paths.build_output_path(step.step_materialization_config), ) for i, step in enumerate(self.planner.session.pipeline.steps) - if step.step_materialization_config and step.step_materialization_config.enabled + if step.step_materialization_config + and step.step_materialization_config.enabled ] path_groups = defaultdict(list) @@ -2083,7 +2114,8 @@ def plan_step(self, step: AbstractStep, step_index: int) -> None: declarations = pipeline.artifact_graphs[step_index] func_pattern = declarations.pattern contracts = ( - () if func_pattern is None + () + if func_pattern is None else tuple(item.contract for item in func_pattern.iter_items()) ) execution_scope = FunctionStepExecutionScope.require_uniform(contracts) @@ -2147,7 +2179,9 @@ def plan_step(self, step: AbstractStep, step_index: int) -> None: self.planner.materialization.apply_materialization_plan( step, step_index, - self.planner.materialization.materialized_output_dir_for_step(step, step_index), + self.planner.materialization.materialized_output_dir_for_step( + step, step_index + ), ) self.planner.materialization.apply_input_conversion_plan( step_index, @@ -2207,7 +2241,9 @@ def update_core_step_plan( step_plan.main_input_dependency = main_input_dependency step_plan.artifact_inputs = artifact_maps.inputs step_plan.artifact_outputs = artifact_maps.outputs - step_plan.future_artifact_inputs = self.planner.future_artifact_inputs[step_index] + step_plan.future_artifact_inputs = self.planner.future_artifact_inputs[ + step_index + ] step_plan.execution_group_scope = artifact_maps.group_scope step_plan.compiled_function_pattern = compiled_function_pattern step_plan.source_binding_plan = artifact_maps.source_binding_plan @@ -2220,11 +2256,16 @@ def main_input_dependency( *, declarations: ArtifactGraph = ArtifactGraph(), execution_scope: FunctionStepExecutionScope = FunctionStepExecutionScope.AXIS, - source_bindings: StepSourceBindingsConfig | CompiledSourceBindingPlan = EMPTY_SOURCE_BINDINGS, + source_bindings: ( + StepSourceBindingsConfig | CompiledSourceBindingPlan + ) = EMPTY_SOURCE_BINDINGS, ) -> StepInputDependency: """Resolve the explicit main-input edge for one step.""" existing_plan = self.planner.plans.get(step_index) - if existing_plan is not None and existing_plan.main_input_dependency.is_resolved: + if ( + existing_plan is not None + and existing_plan.main_input_dependency.is_resolved + ): return existing_plan.main_input_dependency previous = self.planner.plans.get(step_index - 1) @@ -2389,7 +2430,6 @@ def _build_axis_filename( return _cached_axis_filename(axis_id, key, extension, step_index) - @lru_cache(maxsize=65536) def _cached_output_dataset_root( dataset_root: str, diff --git a/openhcs/core/post_execute.py b/openhcs/core/post_execute.py index 340025b598..bbd4594be8 100644 --- a/openhcs/core/post_execute.py +++ b/openhcs/core/post_execute.py @@ -93,7 +93,9 @@ def run( # -- the family as the kernel uses it ------------------------------------- @staticmethod - def bind_all(global_config: "GlobalPipelineConfig") -> tuple["PostExecuteHook", ...]: + def bind_all( + global_config: "GlobalPipelineConfig", + ) -> tuple["PostExecuteHook", ...]: return tuple( hook_type.bind(global_config) for hook_type in PostExecuteHook.__registry__.values() diff --git a/openhcs/core/processing_contracts.py b/openhcs/core/processing_contracts.py index afcdf2eb35..4c01053e89 100644 --- a/openhcs/core/processing_contracts.py +++ b/openhcs/core/processing_contracts.py @@ -150,7 +150,9 @@ def accepted_kwargs( from openhcs.core.callable_contract import CallableMetadata parameters = ( - CallableMetadata.callable_signature(func) if signature is None else signature + CallableMetadata.callable_signature(func) + if signature is None + else signature ).parameters if any( parameter.kind is inspect.Parameter.VAR_KEYWORD @@ -278,8 +280,7 @@ def contextualize_main_image_output(source_image: Any, result: Any) -> Any: return result source_image = owned_runtime_value(source_image) if not isinstance(source_image, ImagePayload) or ( - source_image.mask is None - and not source_image.metadata.has_values + source_image.mask is None and not source_image.metadata.has_values ): return result return source_image.with_pixels(result) @@ -444,7 +445,9 @@ class ProducedImageStackPure2DInputSlicer(ImagePayloadPure2DInputSlicer): value_type = ProducedImageStack - def slice_value(self, value: ProducedImageStack, memory_type: str) -> tuple[Any, ...]: + def slice_value( + self, value: ProducedImageStack, memory_type: str + ) -> tuple[Any, ...]: del memory_type return tuple( RuntimeSliceProjection.value_for_slice( @@ -697,7 +700,9 @@ def aggregate_values( plane_axis: RuntimePlaneAxis, ) -> Any: return ImagePayloadSliceStack.from_output_slices( - values, memory_type=memory_type, plane_axis=plane_axis, + values, + memory_type=memory_type, + plane_axis=plane_axis, ) @@ -763,7 +768,9 @@ def aggregate_values( plane_axis: RuntimePlaneAxis, ) -> Any: return ImagePayloadSliceStack.from_output_slices( - values, memory_type=memory_type, plane_axis=plane_axis, + values, + memory_type=memory_type, + plane_axis=plane_axis, ) @@ -845,8 +852,6 @@ class TuplePure2DAuxiliaryOutputAggregator(FlatSequencePure2DAuxiliaryOutputAggr value_type = tuple - - # --------------------------------------------------------------------------- # The contract family # --------------------------------------------------------------------------- @@ -909,13 +914,17 @@ def invoke_unbound(self, value: Any, kwargs: Mapping[str, Any]) -> Any: """Call once with the value as the stack holds it.""" def full_stack_inputs( - self, image: Any, kwargs: Mapping[str, Any], + self, + image: Any, + kwargs: Mapping[str, Any], ) -> tuple[Any, Mapping[str, Any]]: """Return the image and kwargs a whole-stack call receives.""" return image, kwargs def validate_full_stack_kwargs( - self, contract: type["ProcessingContract"], kwargs: Mapping[str, Any], + self, + contract: type["ProcessingContract"], + kwargs: Mapping[str, Any], ) -> None: """Reject whole-stack kwargs this environment cannot hand ``contract``.""" @@ -947,7 +956,9 @@ def slice_kwargs( @abstractmethod def aggregate_slices( - self, split: PlaneSplit, batch: "Pure2DSliceResultBatch", + self, + split: PlaneSplit, + batch: "Pure2DSliceResultBatch", ) -> Any: """Restack per-plane results.""" @@ -989,7 +1000,8 @@ def for_key(cls, key: str) -> type["ProcessingContract"]: def for_probe(cls, probe: ContractProbe) -> type["ProcessingContract"] | None: """Return the contract whose declaration explains a probe outcome.""" explaining = tuple( - contract for contract in cls.__registry__.values() + contract + for contract in cls.__registry__.values() if contract.explains_probe(probe) ) if len(explaining) > 1: @@ -1019,7 +1031,8 @@ def explains_probe(cls, probe: ContractProbe) -> bool: @classmethod def supports_measurement_image_batch( - cls, request: RuntimeBatchInvocationRequest, + cls, + request: RuntimeBatchInvocationRequest, ) -> bool: """Whether the request already occupies this callable's image domain.""" return True @@ -1063,7 +1076,9 @@ def main_flow_output_source_payload(cls, source_payload: Any) -> Any: @classmethod def main_flow_call_argument( - cls, callable_contract: "CallableContract", source_payload: Any, + cls, + callable_contract: "CallableContract", + source_payload: Any, ) -> Any: """Expose the raw ABI when this contract needs no earlier plane slicing.""" return callable_contract.runtime_main_flow_call_argument(source_payload) @@ -1081,7 +1096,8 @@ def consume_semantic_controls( from openhcs.core.callable_contract import CallableMetadata parameters = ( - {} if func is None + {} + if func is None else CallableMetadata.callable_signature(func).parameters ) values: dict[str, Any] = {} @@ -1112,13 +1128,19 @@ def variable_component_stack_requirement( @classmethod @abstractmethod def execute( - cls, call: ContractCall, image: Any, kwargs: dict[str, Any], + cls, + call: ContractCall, + image: Any, + kwargs: dict[str, Any], ) -> Any: """Execute ``call.func`` on ``image`` under this contract.""" @classmethod def execute_whole_stack( - cls, call: ContractCall, image: Any, kwargs: dict[str, Any], + cls, + call: ContractCall, + image: Any, + kwargs: dict[str, Any], ) -> Any: """Call once on the whole stack.""" projected_image, projected_kwargs = call.full_stack_inputs(image, kwargs) @@ -1162,7 +1184,8 @@ def explains_probe(cls, probe: ContractProbe) -> bool: @classmethod def supports_measurement_image_batch( - cls, request: RuntimeBatchInvocationRequest, + cls, + request: RuntimeBatchInvocationRequest, ) -> bool: """Leave preserved per-plane domains to the plane slicer.""" return ( @@ -1173,7 +1196,9 @@ def supports_measurement_image_batch( @classmethod def main_flow_call_argument( - cls, callable_contract: "CallableContract", source_payload: Any, + cls, + callable_contract: "CallableContract", + source_payload: Any, ) -> Any: """Retain plane context only for the declared contract execution wrapper.""" if callable_contract.raw_processing_function is not None: @@ -1186,7 +1211,10 @@ def execute(cls, call: ContractCall, image: Any, kwargs: dict[str, Any]) -> Any: @classmethod def execute_per_plane( - cls, call: ContractCall, image: Any, kwargs: dict[str, Any], + cls, + call: ContractCall, + image: Any, + kwargs: dict[str, Any], ) -> Any: """Call once per plane and restack the results.""" split = call.plane_split(image, kwargs) @@ -1218,7 +1246,9 @@ def execute_slice( signature=call.signature, ) ) - return call.aggregate_slices(split, Pure2DSliceResultBatch.from_results(results)) + return call.aggregate_slices( + split, Pure2DSliceResultBatch.from_results(results) + ) class FlexibleContract(Pure2DContract, Pure3DContract): @@ -1237,13 +1267,15 @@ def explains_probe(cls, probe: ContractProbe) -> bool: @classmethod def supports_measurement_image_batch( - cls, request: RuntimeBatchInvocationRequest, + cls, + request: RuntimeBatchInvocationRequest, ) -> bool: - controls = cls.consume_semantic_controls(dict(request.kwargs), func=request.func) - return ( - not any(bool(value) for value in controls.values()) - or super().supports_measurement_image_batch(request) + controls = cls.consume_semantic_controls( + dict(request.kwargs), func=request.func ) + return not any( + bool(value) for value in controls.values() + ) or super().supports_measurement_image_batch(request) @classmethod def runtime_parameter_types( @@ -1346,16 +1378,29 @@ def invoke( *, func: Callable[..., Any] | None = None, ) -> Any: - return RuntimeCallablePolicy().invocation( - self.func if func is None else func, - (self.callable_contract.runtime_main_flow_call_argument(value), *self.args), - kwargs, - ).call() + return ( + RuntimeCallablePolicy() + .invocation( + self.func if func is None else func, + ( + self.callable_contract.runtime_main_flow_call_argument(value), + *self.args, + ), + kwargs, + ) + .call() + ) def invoke_unbound(self, value: Any, kwargs: Mapping[str, Any]) -> Any: - return RuntimeCallablePolicy().invocation( - self.func, (value, *self.args), kwargs, - ).call() + return ( + RuntimeCallablePolicy() + .invocation( + self.func, + (value, *self.args), + kwargs, + ) + .call() + ) def whole_output(self, image: Any, result: Any) -> Any: return contextualize_main_image_output(image, result) @@ -1383,7 +1428,8 @@ def _consumes_full_label_stack(self, kwargs: Mapping[str, Any]) -> bool: return False positional = ( dict(zip((p.name for p in self._positional_parameters()), self.args)) - if self.args else {} + if self.args + else {} ) return {**positional, **kwargs}.get("labels") is not None @@ -1418,7 +1464,7 @@ def plane_split(self, image: Any, kwargs: Mapping[str, Any]) -> PlaneSplit | Non ) for parameter, value in zip(positional_parameters, self.args): kwargs.setdefault(parameter.name, value) - self.args = self.args[len(positional_parameters):] + self.args = self.args[len(positional_parameters) :] return PlaneSplit( slices=tuple(slicer.slice_value(image, input_memory_type)), plane_axis=plane_axis, @@ -1450,7 +1496,9 @@ def slice_kwargs( ) def aggregate_slices( - self, split: PlaneSplit, batch: "Pure2DSliceResultBatch", + self, + split: PlaneSplit, + batch: "Pure2DSliceResultBatch", ) -> Any: stacked_main_output = Pure2DAuxiliaryOutputAggregator.aggregate( batch.main_outputs, diff --git a/openhcs/core/progress/projection.py b/openhcs/core/progress/projection.py index c63366af2b..867f0b9b18 100644 --- a/openhcs/core/progress/projection.py +++ b/openhcs/core/progress/projection.py @@ -184,9 +184,7 @@ def accepts_server_lifecycle_state(cls, state: type[SubjectRuntimeState]) -> boo return state is ExecutingSubjectRuntimeState -class CompleteSubjectRuntimeState( - TerminalSubjectRuntimeState, SubjectRuntimeState -): +class CompleteSubjectRuntimeState(TerminalSubjectRuntimeState, SubjectRuntimeState): state_name = "complete" default_channel = ProgressChannel.PIPELINE status_label = "✅ Complete" @@ -194,9 +192,7 @@ class CompleteSubjectRuntimeState( count_sort_order = 40 -class FailedSubjectRuntimeState( - TerminalSubjectRuntimeState, SubjectRuntimeState -): +class FailedSubjectRuntimeState(TerminalSubjectRuntimeState, SubjectRuntimeState): state_name = "failed" status_label = "❌ Failed" count_label = "❌ {count} failed" @@ -386,9 +382,7 @@ def count_for_state(self, state: type[SubjectRuntimeState]) -> int: def count_status_labels(self) -> Tuple[str, ...]: labels: List[str] = [] for declaration in SubjectRuntimeState.counted_declarations(): - label = declaration.count_status_label( - self.count_for_state(declaration) - ) + label = declaration.count_status_label(self.count_for_state(declaration)) if label: labels.append(label) return tuple(labels) @@ -418,7 +412,9 @@ def get_subject( self, subject_id: str, execution_id: Optional[str] = None ) -> Optional[SubjectRuntimeProjection]: if execution_id is not None: - return self.by_identity.get(SubjectRuntimeIdentity(execution_id, subject_id)) + return self.by_identity.get( + SubjectRuntimeIdentity(execution_id, subject_id) + ) return self.by_subject_latest.get(subject_id) def reconcile_server_executions( @@ -486,8 +482,7 @@ def reconcile_server_executions( def recalculate_summary(self) -> None: self.state_counts = { - declaration: 0 - for declaration in SubjectRuntimeState.__registry__.values() + declaration: 0 for declaration in SubjectRuntimeState.__registry__.values() } for subject in self.by_subject_latest.values(): self.state_counts[subject.state] += 1 diff --git a/openhcs/core/progress/runtime_tree.py b/openhcs/core/progress/runtime_tree.py index efb472feb1..0125a0da4c 100644 --- a/openhcs/core/progress/runtime_tree.py +++ b/openhcs/core/progress/runtime_tree.py @@ -122,7 +122,9 @@ def aggregate_percent( return node_percent -class SubjectProgressTreeNode(MeanPercentRuntimeTreeNode, RuntimeTreeNodeDeclarationBase): +class SubjectProgressTreeNode( + MeanPercentRuntimeTreeNode, RuntimeTreeNodeDeclarationBase +): node_kind = "subject" sort_order = 0 @@ -271,7 +273,9 @@ def from_events( dict[str, tuple[str, ...]], ] = {} step_name_sequences: dict[tuple[str, str], tuple[str, ...]] = {} - events_by_subject: dict[tuple[str, str], list[ProgressEvent]] = defaultdict(list) + events_by_subject: dict[tuple[str, str], list[ProgressEvent]] = defaultdict( + list + ) for execution_id, events in executions.items(): for event in sorted(events, key=lambda candidate: candidate.timestamp): @@ -636,7 +640,10 @@ def finalize_subject_node( elif self.all_leaves_queued(node): node.status = "⏳ Queued" self.apply_node_info_text(node) - if subject_projection is not None and subject_projection.queue_position is not None: + if ( + subject_projection is not None + and subject_projection.queue_position is not None + ): node.info = f"{node.percent:.1f}% (q#{subject_projection.queue_position})" def aggregate_percent_recursive(self, node: RuntimeTreeNode) -> float: @@ -784,7 +791,9 @@ def build_progress_tree( topology: RuntimeExecutionTopology, get_subject_name: Callable[[str, str | None], str], ) -> List[RuntimeTreeNode]: - events_by_subject: Dict[tuple[str, str], List[ProgressEvent]] = defaultdict(list) + events_by_subject: Dict[tuple[str, str], List[ProgressEvent]] = defaultdict( + list + ) for exec_id, events_list in executions.items(): for event in events_list: events_by_subject[(exec_id, event.subject_id)].append(event) @@ -802,7 +811,8 @@ def build_progress_tree( worker_assignments=topology.worker_assignments, ) missing_execution_topology = ( - is_executing and (exec_id, subject_id) not in topology.worker_assignments + is_executing + and (exec_id, subject_id) not in topology.worker_assignments ) if missing_execution_topology: children = [] @@ -852,7 +862,9 @@ def build_progress_tree( subject_node, subject_projection=subject_projection, ) - nodes_by_identity[SubjectRuntimeIdentity(exec_id, subject_id)] = subject_node + nodes_by_identity[SubjectRuntimeIdentity(exec_id, subject_id)] = ( + subject_node + ) visible_nodes: list[RuntimeTreeNode] = [] for subject_projection in runtime_projection.by_subject_latest.values(): @@ -960,7 +972,9 @@ def _build_compilation_children( for event in channels[ProgressChannel.COMPILE] if event.axis_id } - known_axis_ids = list(known_partition_values.get((execution_id, subject_id), [])) + known_axis_ids = list( + known_partition_values.get((execution_id, subject_id), []) + ) axis_ids = known_axis_ids if known_axis_ids else sorted(compile_by_axis.keys()) extra_axis_ids = [ axis_id for axis_id in compile_by_axis if axis_id not in axis_ids diff --git a/openhcs/core/progress/types.py b/openhcs/core/progress/types.py index e84fa4f8a2..8abd3ceae8 100644 --- a/openhcs/core/progress/types.py +++ b/openhcs/core/progress/types.py @@ -224,25 +224,33 @@ def is_success_terminal_for_status( class InitChannelProgressPhase: """Trait for progress phases carried on the init channel.""" - progress_channel: ClassVar[type[ProgressChannelDeclarationBase]] = InitProgressChannel + progress_channel: ClassVar[type[ProgressChannelDeclarationBase]] = ( + InitProgressChannel + ) class CompileChannelProgressPhase: """Trait for progress phases carried on the compile channel.""" - progress_channel: ClassVar[type[ProgressChannelDeclarationBase]] = CompileProgressChannel + progress_channel: ClassVar[type[ProgressChannelDeclarationBase]] = ( + CompileProgressChannel + ) class PipelineChannelProgressPhase: """Trait for progress phases carried on the pipeline execution channel.""" - progress_channel: ClassVar[type[ProgressChannelDeclarationBase]] = PipelineProgressChannel + progress_channel: ClassVar[type[ProgressChannelDeclarationBase]] = ( + PipelineProgressChannel + ) class StepChannelProgressPhase: """Trait for progress phases carried on the step execution channel.""" - progress_channel: ClassVar[type[ProgressChannelDeclarationBase]] = StepProgressChannel + progress_channel: ClassVar[type[ProgressChannelDeclarationBase]] = ( + StepProgressChannel + ) class InitProgressPhase(InitChannelProgressPhase, ProgressPhaseDeclarationBase): @@ -404,7 +412,9 @@ class QueuedProgressStatus(ProgressStatusDeclarationBase): def phase_channel(phase: ProgressPhase) -> ProgressChannel: """Classify phase to semantic channel.""" - return ProgressPhaseDeclarationBase.for_phase(phase).progress_channel.require_channel() + return ProgressPhaseDeclarationBase.for_phase( + phase + ).progress_channel.require_channel() def progress_channel_role(channel: ProgressChannel) -> ProgressChannelRole: diff --git a/openhcs/core/runtime_adapters.py b/openhcs/core/runtime_adapters.py index 5954af7d00..1999040b4c 100644 --- a/openhcs/core/runtime_adapters.py +++ b/openhcs/core/runtime_adapters.py @@ -262,7 +262,8 @@ class RuntimeAdapterSpec: runtime_callable_factory: RuntimeCallableFactory | None = None def invocation_domain_inputs( - self, contract: "CallableContract", + self, + contract: "CallableContract", ) -> ArtifactSpecCollection: """Use declared context inputs when the callable has no raw image ABI.""" if contract.accepts_implicit_main_flow_input: diff --git a/openhcs/core/runtime_batch_contracts.py b/openhcs/core/runtime_batch_contracts.py index 2dbd7d8cfa..6f8b2136d9 100644 --- a/openhcs/core/runtime_batch_contracts.py +++ b/openhcs/core/runtime_batch_contracts.py @@ -12,7 +12,11 @@ from metaclass_registry import AutoRegisterMeta -from openhcs.core.callable_contract import CallableContract, CallableMetadata, KeywordRuntimeParameter +from openhcs.core.callable_contract import ( + CallableContract, + CallableMetadata, + KeywordRuntimeParameter, +) from openhcs.core.function_reference import FunctionReference from openhcs.core.runtime_adapters import RuntimeImageExecutionContext from openhcs.core.runtime_plane_projection import ( @@ -48,11 +52,17 @@ class RuntimePlaneAxisValueProjectionParameter(KeywordRuntimeParameter): def runtime_callable_defaults( - func: Callable[..., object], *, signature: inspect.Signature | None = None, + func: Callable[..., object], + *, + signature: inspect.Signature | None = None, ) -> Mapping[str, object]: """Return callable defaults visible to runtime batch executors.""" try: - callable_signature = CallableMetadata.callable_signature(func) if signature is None else signature + callable_signature = ( + CallableMetadata.callable_signature(func) + if signature is None + else signature + ) except (TypeError, ValueError) as exc: raise TypeError( f"Runtime batch function {func!r} must expose an inspectable signature." @@ -95,7 +105,9 @@ def __post_init__(self) -> None: ) def batch_executor_request( - self, *, processing_contract: "type[ProcessingContract]", + self, + *, + processing_contract: "type[ProcessingContract]", ) -> "RuntimeBatchInvocationRequest | None": """Return a request projected into the batch executor's image domain. @@ -228,7 +240,9 @@ def execute_one(self, slice_index: int) -> RuntimeSliceResultT: def execute_each(self) -> list[RuntimeSliceResultT]: """Execute every slice of this batch, one invocation per slice.""" - return [self.execute_one(slice_index) for slice_index in range(self.slice_count)] + return [ + self.execute_one(slice_index) for slice_index in range(self.slice_count) + ] def execute_one_with_kwargs( self, @@ -297,9 +311,7 @@ def executors(self) -> Mapping[RuntimeBatchExecutionDomain, Callable]: if isinstance(self.raw_processing_function, FunctionReference) else self.raw_processing_function ) - inherited = runtime_batch_executors_from_callable( - raw_callable - ) + inherited = runtime_batch_executors_from_callable(raw_callable) for domain, executor in inherited.items(): if domain not in batch_executors: batch_executors[domain] = executor @@ -311,10 +323,15 @@ class Pure2DSliceBatchExecutor(RuntimeBatchExecutor): @classmethod def from_executors( - cls, executors: Mapping[RuntimeBatchExecutionDomain, Callable] | None, + cls, + executors: Mapping[RuntimeBatchExecutionDomain, Callable] | None, ) -> Callable: """Honor a declared pure-2D executor before the nominal serial default.""" - declared = None if executors is None else executors.get(RuntimeBatchExecutionDomain.PURE_2D_SLICES) + declared = ( + None + if executors is None + else executors.get(RuntimeBatchExecutionDomain.PURE_2D_SLICES) + ) return declared if callable(declared) else cls.default_executor() @classmethod diff --git a/openhcs/core/runtime_exports.py b/openhcs/core/runtime_exports.py index c1c88183e5..75bc60d8f7 100644 --- a/openhcs/core/runtime_exports.py +++ b/openhcs/core/runtime_exports.py @@ -112,7 +112,9 @@ class RuntimeExportObservation: table_headers_by_path: Mapping[Path, tuple[str, ...]] table_row_counts_by_path: Mapping[Path, int] output_files: tuple[Path, ...] = () - outputs: StepExecutionObservation = field(default_factory=StepExecutionObservation.empty) + outputs: StepExecutionObservation = field( + default_factory=StepExecutionObservation.empty + ) @classmethod def from_runtime_observations( diff --git a/openhcs/core/runtime_measurements.py b/openhcs/core/runtime_measurements.py index 0597660053..166ab5d24a 100644 --- a/openhcs/core/runtime_measurements.py +++ b/openhcs/core/runtime_measurements.py @@ -1098,6 +1098,8 @@ def object_location_coordinate_arrays( ) coordinates.append((feature.value, values)) return tuple(coordinates) + + @dataclass(frozen=True, slots=True) class ObjectMeasurementValueRow: """Nominal long-form object measurement row.""" @@ -1761,7 +1763,8 @@ def rows(self) -> list[dict[str, float | int]]: value_index = value_indexes.get(object_id) row[feature_name] = ( missing_values[object_id].scalar - if value_index is None else python_values[value_index] + if value_index is None + else python_values[value_index] ) self.complete_row(row) rows.append(row) @@ -1838,7 +1841,9 @@ def complete_row(self, row: dict[str, float | int]) -> None: del row def feature_missing_values_for_objects( - self, feature_name: str, object_ids: tuple[int, ...], + self, + feature_name: str, + object_ids: tuple[int, ...], ) -> Mapping[int, ObjectFeatureMissingValue]: """Project one column's missing-value policy onto its absent objects.""" missing_value = self.feature_missing_values.get( @@ -1914,5 +1919,3 @@ def identity_token(self) -> str: self.id_field or "", ) ) - - diff --git a/openhcs/core/runtime_object_label_aggregation.py b/openhcs/core/runtime_object_label_aggregation.py index e3ed2e107d..b21ac16d24 100644 --- a/openhcs/core/runtime_object_label_aggregation.py +++ b/openhcs/core/runtime_object_label_aggregation.py @@ -182,9 +182,13 @@ def aggregate_values(self) -> ObjectLabelValue: and MemoryType(self.memory_type) is MemoryType.NUMPY ): declared = self.declared_variants() - final_planes = tuple(self.slice_labels(value, FinalLabels) for value in self.values) + final_planes = tuple( + self.slice_labels(value, FinalLabels) for value in self.values + ) variant_planes = { - variant: tuple(self.slice_labels(value, variant) for value in self.values) + variant: tuple( + self.slice_labels(value, variant) for value in self.values + ) for variant in declared } return self.output_value( diff --git a/openhcs/core/runtime_object_label_building.py b/openhcs/core/runtime_object_label_building.py index 1095e7fcab..125793b807 100644 --- a/openhcs/core/runtime_object_label_building.py +++ b/openhcs/core/runtime_object_label_building.py @@ -150,7 +150,8 @@ def plane_semantics(self) -> RuntimePlaneAxisValueProjection | None: return None metadata = self.metadata projection.validate_source_declaration( - metadata.plane_axis, metadata.source_provenance, + metadata.plane_axis, + metadata.source_provenance, ) label_array = np.asarray(self.labels) projection.validate_shape( diff --git a/openhcs/core/runtime_object_labels.py b/openhcs/core/runtime_object_labels.py index 89f953d0f3..4720f0357f 100644 --- a/openhcs/core/runtime_object_labels.py +++ b/openhcs/core/runtime_object_labels.py @@ -136,7 +136,10 @@ def ordered_label_variants( if variant is FinalLabels: raise ValueError("Final labels are carried as labels, not as a variant.") return MappingProxyType( - {variant: variants[variant] for variant in sorted(variants, key=lambda v: v.name)} + { + variant: variants[variant] + for variant in sorted(variants, key=lambda v: v.name) + } ) @@ -152,7 +155,9 @@ def __post_init__(self) -> None: @property def shape(self) -> tuple[int, ...] | None: - return ObjectLabelStorageStrategy.for_value(self.labels).label_shape(self.labels) + return ObjectLabelStorageStrategy.for_value(self.labels).label_shape( + self.labels + ) @property def dtype(self) -> Any: @@ -160,7 +165,9 @@ def dtype(self) -> Any: def validate_plane_count(self, plane_count: int, context: str) -> None: object_label_validate_plane_count( - self.labels, plane_count=plane_count, context=context, + self.labels, + plane_count=plane_count, + context=context, ) @classmethod @@ -325,12 +332,17 @@ def project_plane( ) def project_planes( - self, plane_indices: tuple[int, ...], *, plane_count: int, + self, + plane_indices: tuple[int, ...], + *, + plane_count: int, ) -> "ObjectLabelVariantData": """Project each variant through the same ordered plane selection.""" return self.project( lambda labels: object_label_project_planes( - labels, plane_indices, plane_count=plane_count, + labels, + plane_indices, + plane_count=plane_count, ) ) @@ -356,7 +368,9 @@ class PlaneStackObjectLabelVariantData(ObjectLabelVariantData): __slots__ = ("_planes", "_dense_variants", "_memory_type", "_shape", "_lock") def __init__( - self, variants: Sequence[ObjectLabelVariantData], memory_type: str, + self, + variants: Sequence[ObjectLabelVariantData], + memory_type: str, ) -> None: from openhcs.core.memory import MemoryType, runtime_slice_stack_geometry @@ -396,9 +410,13 @@ def __repr__(self) -> str: def _dense_variant(self, variant: type[ObjectLabelVariant]) -> ObjectLabelData: with self._lock: if variant not in self._dense_variants: - dense = object_label_stack_planes(self._planes[variant], self._memory_type) + dense = object_label_stack_planes( + self._planes[variant], self._memory_type + ) self._dense_variants[variant] = dense - self._planes[variant] = tuple(dense[index] for index in range(len(dense))) + self._planes[variant] = tuple( + dense[index] for index in range(len(dense)) + ) return self._dense_variants[variant] @property @@ -408,11 +426,15 @@ def labels(self) -> ObjectLabelData: @property def variants(self) -> ObjectLabelVariants: return MappingProxyType( - {variant: self._dense_variant(variant) for variant in self.present_variants[1:]} + { + variant: self._dense_variant(variant) + for variant in self.present_variants[1:] + } ) def variant_labels( - self, variant: type[ObjectLabelVariant], + self, + variant: type[ObjectLabelVariant], ) -> ObjectLabelData | None: if variant not in self.present_variants: return None @@ -438,22 +460,35 @@ def validate_plane_count(self, plane_count: int, context: str) -> None: ) def validate_representation( - self, *, representation: ObjectLabelRepresentation, value_label: str, + self, + *, + representation: ObjectLabelRepresentation, + value_label: str, ) -> None: for planes in self._planes.values(): for plane in planes: ObjectLabelStorageStrategy.for_value(plane).validate_representation( - plane, representation=representation, value_label=value_label, + plane, + representation=representation, + value_label=value_label, ) - def in_representation(self, representation: ObjectLabelRepresentation) -> ObjectLabelVariantData: + def in_representation( + self, representation: ObjectLabelRepresentation + ) -> ObjectLabelVariantData: if representation is ObjectLabelRepresentation.DENSE_LABELS: - self.validate_representation(representation=representation, value_label=type(self).__name__) + self.validate_representation( + representation=representation, value_label=type(self).__name__ + ) return self return super().in_representation(representation) def project_plane( - self, *, plane_axis: RuntimePlaneAxis, plane_index: int, plane_count: int, + self, + *, + plane_axis: RuntimePlaneAxis, + plane_index: int, + plane_count: int, ) -> ObjectLabelVariantData: self.validate_plane_count(plane_count, "Object-label data plane projection") if plane_index < 0 or plane_index >= plane_count: @@ -461,7 +496,10 @@ def project_plane( return ObjectLabelPlaneVariantData(self, plane_index) def project_planes( - self, plane_indices: tuple[int, ...], *, plane_count: int, + self, + plane_indices: tuple[int, ...], + *, + plane_count: int, ) -> ObjectLabelVariantData: self.validate_plane_count(plane_count, "Object-label data plane projection") if any(index < 0 or index >= plane_count for index in plane_indices): @@ -472,8 +510,11 @@ def __getstate__(self) -> dict[str, Any]: with self._lock: return { "variants": { - variant: self._dense_variants[variant] - if variant in self._dense_variants else planes + variant: ( + self._dense_variants[variant] + if variant in self._dense_variants + else planes + ) for variant, planes in self._planes.items() }, "memory_type": self._memory_type, @@ -481,15 +522,27 @@ def __getstate__(self) -> dict[str, Any]: } def __setstate__(self, state: dict[str, Any]) -> None: - object.__setattr__(self, "_dense_variants", { - variant: data for variant, data in state["variants"].items() - if isinstance(data, np.ndarray) - }) - object.__setattr__(self, "_planes", { - variant: tuple(data[index] for index in range(len(data))) - if isinstance(data, np.ndarray) else data - for variant, data in state["variants"].items() - }) + object.__setattr__( + self, + "_dense_variants", + { + variant: data + for variant, data in state["variants"].items() + if isinstance(data, np.ndarray) + }, + ) + object.__setattr__( + self, + "_planes", + { + variant: ( + tuple(data[index] for index in range(len(data))) + if isinstance(data, np.ndarray) + else data + ) + for variant, data in state["variants"].items() + }, + ) object.__setattr__(self, "_memory_type", state["memory_type"]) object.__setattr__(self, "_shape", state["shape"]) object.__setattr__(self, "_lock", Lock()) @@ -501,7 +554,9 @@ class ProjectedObjectLabelVariantData(PlaneStackObjectLabelVariantData): __slots__ = ("_source", "_indices") def __init__( - self, source: PlaneStackObjectLabelVariantData, indices: tuple[int, ...], + self, + source: PlaneStackObjectLabelVariantData, + indices: tuple[int, ...], ) -> None: object.__setattr__(self, "_source", source) object.__setattr__(self, "_indices", indices) @@ -526,14 +581,20 @@ def _dense_variant(self, variant: type[ObjectLabelVariant]) -> ObjectLabelData: def _project_variant(self, source: ObjectLabelData) -> ObjectLabelData: return object_label_project_planes( - source, self._indices, plane_count=self._source.shape[0], + source, + self._indices, + plane_count=self._source.shape[0], ) def validate_representation( - self, *, representation: ObjectLabelRepresentation, value_label: str, + self, + *, + representation: ObjectLabelRepresentation, + value_label: str, ) -> None: self._source.validate_representation( - representation=representation, value_label=value_label, + representation=representation, + value_label=value_label, ) def __reduce__(self): @@ -552,7 +613,9 @@ def __init__(self, source: PlaneStackObjectLabelVariantData, index: int) -> None def _project_variant(self, source: ObjectLabelData) -> ObjectLabelData: return object_label_project_plane( - source, self._indices[0], plane_count=self._source.shape[0], + source, + self._indices[0], + plane_count=self._source.shape[0], ) @@ -628,7 +691,8 @@ def __len__(self) -> int: return len(self.labels) def variant_labels( - self, variant: type[ObjectLabelVariant], + self, + variant: type[ObjectLabelVariant], ) -> ObjectLabelData | None: """Return one declared label variant, or None when absent.""" return self.variant_data.variant_labels(variant) @@ -761,7 +825,8 @@ def spatial_adapter( ) return ObjectLabelPayloadSourceSpatialDomainAdapter( - self, source_shape_override_yx=source_shape_override_yx, + self, + source_shape_override_yx=source_shape_override_yx, ) # -- derived image outputs ------------------------------------------------ @@ -792,7 +857,9 @@ def contextualize_image_output( return source_metadata.derive_payload(self, output, plane_projection=None) if plane_projection is None or plane_projection.plane_index is not None: return source_metadata.derive_payload( - self, output, plane_projection=plane_projection, + self, + output, + plane_projection=plane_projection, ) plane_count = source_metadata.source_provenance.source_plane_count if plane_count != plane_projection.axis_size: @@ -809,7 +876,9 @@ def contextualize_image_output( plane_axis=plane_projection.axis, ).attach_to(output) return source_metadata.derive_payload( - self, contextualized_output, plane_projection=plane_projection, + self, + contextualized_output, + plane_projection=plane_projection, ) def object_label_output( @@ -826,7 +895,8 @@ def alignment_slices(self) -> tuple[Any, ...]: if slice_count is None: return (self,) projection = RuntimePlaneAxisValueProjection.preserve( - axis=RuntimePlaneAxis.RUNTIME_SLICE, axis_size=slice_count, + axis=RuntimePlaneAxis.RUNTIME_SLICE, + axis_size=slice_count, ) return tuple( self.value_for_slice(projection.selected_plane(index)) @@ -996,9 +1066,7 @@ def with_measurement_labels(self, labels: ObjectLabelData) -> Self: plane_count=plane_count, context="Object-label measurement replacement", ) - if ( - variants.same_arrays_as(self.variant_data) - ): + if variants.same_arrays_as(self.variant_data): return self return self.with_variants(variants) @@ -1057,7 +1125,8 @@ def with_runtime_slice_projection( ) if label_plane_indices is not None else source_variants.project_runtime_slice( - slice_index=slice_index, slice_count=slice_count, + slice_index=slice_index, + slice_count=slice_count, ) ) projected_domain = self.runtime_slice_domain( @@ -1107,11 +1176,10 @@ def with_plane_projection( ) if normalized_indices == tuple(range(plane_count)): return self - metadata = self.metadata.for_source_planes( - normalized_indices - ) + metadata = self.metadata.for_source_planes(normalized_indices) variants = self.variant_data.project_planes( - normalized_indices, plane_count=plane_count, + normalized_indices, + plane_count=plane_count, ) return self.with_variants( variants, @@ -1415,7 +1483,9 @@ def from_payload( if isinstance(payload, ObjectLabelSet): payload.validate_artifact_name() if payload.source_image_name == "": - raise ValueError("ObjectLabelSet.source_image_name cannot be empty.") + raise ValueError( + "ObjectLabelSet.source_image_name cannot be empty." + ) variants.validate_representation( representation=ObjectLabelRepresentation(payload.representation), value_label=type(payload).__name__, @@ -1514,13 +1584,17 @@ def for_value(cls, labels: object) -> "ObjectLabelStorageStrategy": ) @classmethod - def for_planes(cls, labels: Sequence[ObjectLabelData]) -> "ObjectLabelStorageStrategy": + def for_planes( + cls, labels: Sequence[ObjectLabelData] + ) -> "ObjectLabelStorageStrategy": """Admit one homogeneous semantic plane family before stacking.""" if not labels: raise ValueError("Object-label plane stacking requires values.") authority = cls.for_value(labels[0]) if any(type(value) is not type(labels[0]) for value in labels[1:]): - raise TypeError("Object-label plane stacking requires one nominal storage type.") + raise TypeError( + "Object-label plane stacking requires one nominal storage type." + ) return authority @abstractmethod @@ -1540,10 +1614,15 @@ def dense_data( """Materialize dense labels from this storage.""" def rendering_layers( - self, labels: object, *, source_spatial_shape_yx: tuple[int, int] | None, + self, + labels: object, + *, + source_spatial_shape_yx: tuple[int, int] | None, ) -> tuple[np.ndarray, ...]: """Return nonoverlapping dense layers for image rendering consumers.""" - return (self.dense_data(labels, source_spatial_shape_yx=source_spatial_shape_yx),) + return ( + self.dense_data(labels, source_spatial_shape_yx=source_spatial_shape_yx), + ) @abstractmethod def sparse_ijv_rows(self, labels: object) -> SparseIJVLabelRows: @@ -1719,7 +1798,8 @@ def axis_centers( return super().axis_centers(labels, domain=domain) coordinate_columns = tuple(range(3 - array.ndim, 3)) sums, pixel_counts = _dense_label_coordinate_moments_numba( - coordinate_planes, coordinate_domain.max_label, + coordinate_planes, + coordinate_domain.max_label, ) del coordinate_domain # Like sparse conversion, the moments snapshot precedes domain callbacks. @@ -1822,7 +1902,10 @@ def dense_data( ) def rendering_layers( - self, labels: object, *, source_spatial_shape_yx: tuple[int, int] | None, + self, + labels: object, + *, + source_spatial_shape_yx: tuple[int, int] | None, ) -> tuple[np.ndarray, ...]: return cast(SparseIJVLabelRows, labels).nonoverlapping_dense_layers( source_spatial_shape_yx=source_spatial_shape_yx, @@ -1944,13 +2027,17 @@ def dense_data( ) def rendering_layers( - self, labels: object, *, source_spatial_shape_yx: tuple[int, int] | None, + self, + labels: object, + *, + source_spatial_shape_yx: tuple[int, int] | None, ) -> tuple[np.ndarray, ...]: del source_spatial_shape_yx value = cast(ObjectLabelValue, labels) data = self.label_data(value) return ObjectLabelStorageStrategy.for_value(data).rendering_layers( - data, source_spatial_shape_yx=value.source_spatial_shape_yx, + data, + source_spatial_shape_yx=value.source_spatial_shape_yx, ) def sparse_ijv_rows(self, labels: object) -> SparseIJVLabelRows: @@ -2113,9 +2200,7 @@ def _dense_label_coordinate_moments_numba( @njit(cache=True) -def dense_label_centers_2d_numba( - labels: np.ndarray, label_count: int -) -> np.ndarray: +def dense_label_centers_2d_numba(labels: np.ndarray, label_count: int) -> np.ndarray: """Validate compiled two-dimensional geometry and return its y/x centers.""" height, width = labels.shape sums, counts = _dense_label_coordinate_moments_numba( @@ -2125,7 +2210,8 @@ def dense_label_centers_2d_numba( for label_id in range(label_count + 1): for axis in range(2): centers[label_id, axis] = ( - np.nan if counts[label_id] == 0 + np.nan + if counts[label_id] == 0 else sums[label_id, axis + 1] / counts[label_id] ) return centers diff --git a/openhcs/core/runtime_relationships.py b/openhcs/core/runtime_relationships.py index 2d5004b8a2..f366fe63eb 100644 --- a/openhcs/core/runtime_relationships.py +++ b/openhcs/core/runtime_relationships.py @@ -893,7 +893,9 @@ def __post_init__(self) -> None: def required_slice_index(self) -> int: """Admit this object identity for a comparison requiring an explicit plane.""" if self.slice_index is None: - raise ValueError("Object instance requires an explicit image or slice identity.") + raise ValueError( + "Object instance requires an explicit image or slice identity." + ) return self.slice_index @classmethod diff --git a/openhcs/core/runtime_stores.py b/openhcs/core/runtime_stores.py index e044250c3e..3a0f7c0bda 100644 --- a/openhcs/core/runtime_stores.py +++ b/openhcs/core/runtime_stores.py @@ -152,9 +152,7 @@ def __post_init__(self) -> None: raise ValueError( "RuntimeArtifactDynamicComponentTarget.backend cannot be empty." ) - object.__setattr__( - self, "input_plan", self.input_plan.runtime_query_snapshot() - ) + object.__setattr__(self, "input_plan", self.input_plan.runtime_query_snapshot()) def matches(self, record: "StoredRuntimeValue") -> bool: scope = record.key.scope @@ -366,7 +364,9 @@ def selects_complete_producer(self) -> bool: producer_scope = storage_plan.producer_group_scope() selection_scope = projection.producer_selection_scope return ( - projection.selects_declared_complete_producer(self.edge_plan.spec, storage_plan) + projection.selects_declared_complete_producer( + self.edge_plan.spec, storage_plan + ) or storage_plan.composes_producer_groups( ComponentSet.of(projection.consumer_variable_components) ) @@ -399,7 +399,9 @@ def candidate_execution_scopes( identity_policy = self._source_context_identity_policy() candidates = tuple( record - for group_key in ((None,) if producer_scope.is_dynamic else producer_scope.keys) + for group_key in ( + (None,) if producer_scope.is_dynamic else producer_scope.keys + ) for record in store.find_matching( RuntimeArtifactQuery.from_input_plan( storage_plan, @@ -414,8 +416,9 @@ def candidate_execution_scopes( record_scope = record.key.scope if record_scope.component is not producer_scope.component: continue - if not selects_complete_producer and not selection_scope.contains_runtime_key( - record_scope.value_text + if ( + not selects_complete_producer + and not selection_scope.contains_runtime_key(record_scope.value_text) ): continue record_coordinates = dict(record_scope.source_component_values) @@ -423,10 +426,13 @@ def candidate_execution_scopes( record_coordinates.pop(producer_scope.component, None) if projected_components: provenance = record.data.metadata.source_provenance - plane_metadata = provenance.source_image_provenance_planes.runtime_component_metadata + plane_metadata = ( + provenance.source_image_provenance_planes.runtime_component_metadata + ) metadata_rows = plane_metadata or ( (provenance.source_component_metadata,) - if provenance.source_component_metadata is not None else () + if provenance.source_component_metadata is not None + else () ) else: metadata_rows = (None,) @@ -438,7 +444,8 @@ def candidate_execution_scopes( component_scope = projection.component_scope(component) if component_scope.is_dynamic: component_value = semantic_source_metadata_value( - metadata, component.name, + metadata, + component.name, ) if component_value is None: break @@ -446,12 +453,19 @@ def candidate_execution_scopes( else: component_value = component_scope.select_runtime_key(None) if not SourceAxisMetadataScope.constraint_matches_metadata( - metadata, component.name, component_value, + metadata, + component.name, + component_value, ): break existing_value = coordinates.get(component) - if existing_value is not None and not SourceAxisMetadataScope.constraint_matches_metadata( - metadata, component.name, existing_value, + if ( + existing_value is not None + and not SourceAxisMetadataScope.constraint_matches_metadata( + metadata, + component.name, + existing_value, + ) ): break coordinates.setdefault(component, component_value) @@ -469,7 +483,10 @@ def candidate_execution_scopes( ): continue fixed_group_value = coordinates.get(execution_scope.component) - if fixed_group_value is not None and fixed_group_value != component_key: + if ( + fixed_group_value is not None + and fixed_group_value != component_key + ): continue fixed_values = tuple( (component, value) @@ -754,8 +771,7 @@ def _matches_execution_scope( ) identity_policy = self._source_context_identity_policy() context_components = ( - context_components - .intersection(record_scope.source_components) + context_components.intersection(record_scope.source_components) .intersection(self.axis_scope.source_components) .intersection(ComponentSet(identity_policy.identity_components())) ) @@ -766,10 +782,12 @@ def _matches_execution_scope( return True return SourceImageSetIdentityCompatibility( record_scope.source_image_set_identity( - identity_policy, components=context_components, + identity_policy, + components=context_components, ), self.axis_scope.source_image_set_identity( - identity_policy, components=context_components, + identity_policy, + components=context_components, ), ).matches() @@ -988,8 +1006,9 @@ def from_dict(cls, data: Mapping[str, Any]) -> "RuntimeArtifactAddress": artifact_type=ArtifactType.coerce(str(key["artifact_type"])), scope=RuntimeExecutionAxisScope.from_raw( str(scope["axis_id"]), - component=None if component is None else family.named(str(component)), - + component=( + None if component is None else family.named(str(component)) + ), value=None if value is None else str(value), fixed_component_values=tuple(fixed_values), ), @@ -1257,7 +1276,9 @@ def values(self) -> tuple[StoredRuntimeValue, ...]: @property def observed_values(self) -> tuple[StoredRuntimeValue, ...]: """Return retained runtime artifact writes in their original order.""" - return tuple(record for record in self._observation_records if record is not None) + return tuple( + record for record in self._observation_records if record is not None + ) def observation_cursor(self) -> RuntimeStoreObservationCursor: """Return a cursor for future observation-delta queries.""" @@ -1278,7 +1299,8 @@ def observed_values_after( f"{len(self._observation_records)}." ) return tuple( - record for record in self._observation_records[cursor.index :] + record + for record in self._observation_records[cursor.index :] if record is not None ) @@ -1296,7 +1318,9 @@ def release_unconsumed( their observation history. Empty stream slots mark retired writes, not fabricated runtime values. """ - retained_addresses = {(record.key, record.location) for record in retained_records} + retained_addresses = { + (record.key, record.location) for record in retained_records + } retained_inputs = { (ref.artifact_type, ref.name) for ref in retained_refs @@ -1304,10 +1328,10 @@ def release_unconsumed( } def retained(record: StoredRuntimeValue) -> bool: - return ( - (record.key.artifact_type, record.key.name) in retained_inputs - or (record.key, record.location) in retained_addresses - ) + return (record.key.artifact_type, record.key.name) in retained_inputs or ( + record.key, + record.location, + ) in retained_addresses retired = tuple( record for record in self.observed_values if not retained(record) @@ -1315,11 +1339,15 @@ def retained(record: StoredRuntimeValue) -> bool: if not retired: return live_locations = { - record.location for record in self._records_by_location.values() + record.location + for record in self._records_by_location.values() if retained(record) } for location in dict.fromkeys(record.location for record in retired): - if location not in live_locations and location.backend == Backend.MEMORY.value: + if ( + location not in live_locations + and location.backend == Backend.MEMORY.value + ): if filemanager.exists(location.path, location.backend): filemanager.delete(location.path, location.backend) for address, record in tuple(self._records_by_location.items()): diff --git a/openhcs/core/source_binding_context.py b/openhcs/core/source_binding_context.py index b0ea1f1a06..8e468fca91 100644 --- a/openhcs/core/source_binding_context.py +++ b/openhcs/core/source_binding_context.py @@ -26,7 +26,9 @@ class SourceBindingContext: source_backend: str def __post_init__(self) -> None: - object.__setattr__(self, "display_dataset_root", Path(self.display_dataset_root)) + object.__setattr__( + self, "display_dataset_root", Path(self.display_dataset_root) + ) object.__setattr__( self, "execution_dataset_root", Path(self.execution_dataset_root) ) diff --git a/openhcs/core/source_binding_selection.py b/openhcs/core/source_binding_selection.py index aa74db615a..273f770c88 100644 --- a/openhcs/core/source_binding_selection.py +++ b/openhcs/core/source_binding_selection.py @@ -69,7 +69,9 @@ if TYPE_CHECKING: from openhcs.core.runtime_adapters import RuntimeAdapterRequest - from openhcs.core.runtime_source_binding_cache import RuntimeSourceResolutionSnapshot + from openhcs.core.runtime_source_binding_cache import ( + RuntimeSourceResolutionSnapshot, + ) from polystore.filemanager import FileManager from openhcs.core.context.processing_context import ProcessingContext from openhcs.core.dataset_sources.interfaces import FilenameParser @@ -288,7 +290,8 @@ def _candidate_path_resolution( for path, declared in self.source_projections_by_virtual_path.items() if declared is projection ) - if projection is not None else (exact_virtual_path,) + if projection is not None + else (exact_virtual_path,) ) else: virtual_matches = tuple( @@ -383,7 +386,8 @@ def declared_positions_for_candidates( projection = self.source_projections_by_virtual_path.get(position) positions.append( projection_positions[id(projection)] - if projection is not None else position + if projection is not None + else position ) return tuple(dict.fromkeys(positions)) @@ -558,7 +562,9 @@ def matches( if not selector.components and not selector.metadata: return True - return selector.metadata_candidates_match(source_context.candidate_metadata(candidate)) + return selector.metadata_candidates_match( + source_context.candidate_metadata(candidate) + ) @classmethod def compatible_candidates( @@ -1129,7 +1135,8 @@ def matching_candidates_for_source_identities( if self.resolution_snapshot is not None: return self.resolution_snapshot.matching_candidates_for_source_identities( - identities, candidates, + identities, + candidates, ) candidates = self.declared_positions_for_candidates(candidates) @@ -1232,7 +1239,9 @@ def expand( if not resolution_bindings: # No alias selects files: the candidate matcher keeps every anchor. return SourceBindingCandidateMatcher.compatible_candidates( - anchors, bindings=(), source_context=self, + anchors, + bindings=(), + source_context=self, ) return tuple( dict.fromkeys( @@ -1549,43 +1558,60 @@ def load_images( if self.backend is Backend.MEMORY: return [ ImagePayload.of(value) - for value in filemanager.load_batch(list(self.files), self.backend.value) + for value in filemanager.load_batch( + list(self.files), self.backend.value + ) ] - missing = tuple(dict.fromkeys( - path for path in self.files - if not filemanager.exists(path, Backend.MEMORY.value) - )) + missing = tuple( + dict.fromkeys( + path + for path in self.files + if not filemanager.exists(path, Backend.MEMORY.value) + ) + ) loaded_by_path = {} if missing: pixels = filemanager.load_batch( - list(missing), self.backend.value, - **({"zarr_config": zarr_config} if self.backend is Backend.ZARR else {}), + list(missing), + self.backend.value, + **( + {"zarr_config": zarr_config} if self.backend is Backend.ZARR else {} + ), ) loaded_by_path.update( - (path, ImagePayloadSourceMetadataContext( - SourceImageIdentity(path), - read_backend=self.backend.value, - filemanager=filemanager, - ).payload(image)) + ( + path, + ImagePayloadSourceMetadataContext( + SourceImageIdentity(path), + read_backend=self.backend.value, + filemanager=filemanager, + ).payload(image), + ) for path, image in zip(missing, pixels, strict=True) ) for parent in dict.fromkeys(str(Path(path).parent) for path in missing): filemanager.ensure_directory(parent, Backend.MEMORY.value) filemanager.save_batch( - list(loaded_by_path.values()), list(loaded_by_path), Backend.MEMORY.value, + list(loaded_by_path.values()), + list(loaded_by_path), + Backend.MEMORY.value, ) - retained = tuple(dict.fromkeys( - path for path in self.files if path not in loaded_by_path - )) + retained = tuple( + dict.fromkeys(path for path in self.files if path not in loaded_by_path) + ) if retained: - loaded_by_path.update(zip( - retained, - ( - ImagePayload.of(value) - for value in filemanager.load_batch(list(retained), Backend.MEMORY.value) - ), - strict=True, - )) + loaded_by_path.update( + zip( + retained, + ( + ImagePayload.of(value) + for value in filemanager.load_batch( + list(retained), Backend.MEMORY.value + ) + ), + strict=True, + ) + ) return [loaded_by_path[path] for path in self.files] @@ -1666,7 +1692,9 @@ def from_context( @classmethod def source_artifact_payload( - cls, request: RuntimeAdapterRequest, binding: NamedSourceBinding, + cls, + request: RuntimeAdapterRequest, + binding: NamedSourceBinding, ) -> object: """Resolve original source pixels in this origin's workspace universe.""" @@ -1728,7 +1756,9 @@ def source_artifact_payload( ) projected_payloads = projection.load_binding_payloads( - members, binding=binding, filemanager=request.context.filemanager, + members, + binding=binding, + filemanager=request.context.filemanager, ) payload = ( projected_payloads[0] @@ -1774,7 +1804,8 @@ def admit_source_artifact_cohort( ) metadata = payload.metadata domain = source_binding_plan.source_spatial_domain.with_source_cohort( - metadata.source_spatial_domain, depth=member_count, + metadata.source_spatial_domain, + depth=member_count, ) return metadata.replace_fields(source_spatial_domain=domain).attach_to(payload) @@ -1931,13 +1962,17 @@ class StepInputSourceUniverseRequest(SourceUniverseRequest): @classmethod def source_artifact_payload( - cls, request: RuntimeAdapterRequest, binding: NamedSourceBinding, + cls, + request: RuntimeAdapterRequest, + binding: NamedSourceBinding, ) -> object: """Resolve primary planes from current pixels; companions from source.""" if not binding.requires_current_pixels: return SourceUniverseRequest.source_artifact_payload(request, binding) if request.source_payload is None: - raise ValueError(f"STEP_INPUT binding {binding.alias!r} requires current pixels.") + raise ValueError( + f"STEP_INPUT binding {binding.alias!r} requires current pixels." + ) return binding.project_step_input_payload(request.source_payload) def source_universe(self) -> SourceFileUniverse: diff --git a/openhcs/core/source_binding_workspace.py b/openhcs/core/source_binding_workspace.py index 794963d1bc..2feaf8c66e 100644 --- a/openhcs/core/source_binding_workspace.py +++ b/openhcs/core/source_binding_workspace.py @@ -67,7 +67,9 @@ from openhcs.core.axes import AxisFamily, ColourAxis, PartitionAxis if TYPE_CHECKING: - from openhcs.core.source_workspace_projection import VirtualWorkspaceSourceProjection + from openhcs.core.source_workspace_projection import ( + VirtualWorkspaceSourceProjection, + ) @dataclass(frozen=True, slots=True) @@ -647,26 +649,39 @@ def accept_prepared_projection( VirtualWorkspacePathLookup.from_paths(path, path) ) filter_paths = ( - () if metadata is None else SourceMetadataFields.source_filter_paths(metadata) + () + if metadata is None + else SourceMetadataFields.source_filter_paths(metadata) ) or (ref.backend_address, path) if self.source_bindings.source_path_filters_match(filter_paths): selected_paths.add(path) selected_metadata_paths.update((path, ref.backend_address)) if not selected_paths: - raise ValueError("Declared source filters matched no prepared workspace sources.") + raise ValueError( + "Declared source filters matched no prepared workspace sources." + ) return VirtualWorkspaceSourceProjection( - source_refs_by_virtual_path=MappingProxyType({ - path: ref for path, ref in projection.source_refs_by_virtual_path.items() - if path in selected_paths - }), - source_metadata_by_path=MappingProxyType({ - path: metadata for path, metadata in projection.source_metadata_by_path.items() - if path in selected_metadata_paths - }), - source_projections_by_virtual_path=MappingProxyType({ - path: source for path, source in projection.source_projections_by_virtual_path.items() - if path in selected_paths - }), + source_refs_by_virtual_path=MappingProxyType( + { + path: ref + for path, ref in projection.source_refs_by_virtual_path.items() + if path in selected_paths + } + ), + source_metadata_by_path=MappingProxyType( + { + path: metadata + for path, metadata in projection.source_metadata_by_path.items() + if path in selected_metadata_paths + } + ), + source_projections_by_virtual_path=MappingProxyType( + { + path: source + for path, source in projection.source_projections_by_virtual_path.items() + if path in selected_paths + } + ), workspace_root=projection.workspace_root, ) @@ -1023,21 +1038,14 @@ def candidate_matches_binding( ): return False selector = binding.selector - return ( - all( - ( - value := semantic_source_metadata_value( - candidate.metadata, item.field - ) - ) - is not None - and source_metadata_values_equal(value, item.value) - for item in selector.metadata - ) - and all( - self._component_is_compatible(candidate.metadata, item) - for item in selector.components - ) + return all( + (value := semantic_source_metadata_value(candidate.metadata, item.field)) + is not None + and source_metadata_values_equal(value, item.value) + for item in selector.metadata + ) and all( + self._component_is_compatible(candidate.metadata, item) + for item in selector.components ) @staticmethod @@ -1204,14 +1212,16 @@ def _source_set_projections( address = OpenHCSPlaneAddress( ( axis, - well - if issubclass(axis, PartitionAxis) - else SourceAxisProjection.project_bound( - axis, - set_metadata=source_set.metadata, - set_index=source_set.index, - candidate_metadata=candidate.metadata, - candidate_index=projection_index, + ( + well + if issubclass(axis, PartitionAxis) + else SourceAxisProjection.project_bound( + axis, + set_metadata=source_set.metadata, + set_index=source_set.index, + candidate_metadata=candidate.metadata, + candidate_index=projection_index, + ) ), ) for axis in AxisFamily.active().axes @@ -1258,12 +1268,12 @@ def _source_set_well(self, source_set: _SourceSet) -> str: partition_axis = AxisFamily.active().partition_axis() fields_are_partition_identity = all( - source_metadata_component(field) - is partition_axis - for field in fields + source_metadata_component(field) is partition_axis for field in fields ) if fields_are_partition_identity: - partition_value = SourceMetadataFields.component_value(group_metadata, partition_axis) + partition_value = SourceMetadataFields.component_value( + group_metadata, partition_axis + ) if partition_value is None: raise ValueError( f"Source grouping fields declared {partition_axis.name} identity " @@ -1427,7 +1437,9 @@ def _projected_candidate_components( value for candidate in candidates if ( - value := SourceMetadataFields.component_value(candidate.metadata, component) + value := SourceMetadataFields.component_value( + candidate.metadata, component + ) ) is not None } diff --git a/openhcs/core/source_bindings.py b/openhcs/core/source_bindings.py index e688e378b0..b07a08bf36 100644 --- a/openhcs/core/source_bindings.py +++ b/openhcs/core/source_bindings.py @@ -644,7 +644,9 @@ class SourceSelector: This filters candidates; it does not assign the output identity. """ - metadata: Annotated[tuple[MetadataSelector, ...], FieldLabel("Select Metadata")] = () + metadata: Annotated[ + tuple[MetadataSelector, ...], FieldLabel("Select Metadata") + ] = () """Exact metadata field/value pairs required on selected sources, e.g. Well=A01. This filters candidates; pairing sources into sets uses the match plan. @@ -707,12 +709,15 @@ def metadata_candidates_match( any( source_metadata_values_equal(value, selector.value) for metadata in candidates - for value in source_component_metadata_values(metadata, selector.component) + for value in source_component_metadata_values( + metadata, selector.component + ) ) for selector in self.components ) and all( any( - value is not None and source_metadata_values_equal(value, selector.value) + value is not None + and source_metadata_values_equal(value, selector.value) for metadata in candidates for value in (semantic_source_metadata_value(metadata, selector.field),) ) @@ -723,12 +728,12 @@ def matches_provenance(self, provenance: SourceImageProvenance) -> bool: """Apply the same selectors to identities carried by current pixels.""" identities = provenance.represented_source_identities return ( - (not self.filters or self.path_filters_match( + not self.filters + or self.path_filters_match( tuple(path for identity in identities for path in identity.filter_paths) - )) - and self.metadata_candidates_match( - tuple(identity.component_metadata or {} for identity in identities) ) + ) and self.metadata_candidates_match( + tuple(identity.component_metadata or {} for identity in identities) ) @@ -1142,24 +1147,32 @@ def project_step_input_payload(self, payload: RuntimeArrayData) -> RuntimeArrayD metadata = payload.metadata provenance = metadata.source_provenance if not provenance.has_values: - raise ValueError(f"STEP_INPUT binding {self.alias!r} requires source provenance.") + raise ValueError( + f"STEP_INPUT binding {self.alias!r} requires source provenance." + ) if not provenance.source_plane_count: if not self.selector.matches_provenance(provenance): - raise ValueError(f"STEP_INPUT binding {self.alias!r} selects no current planes.") + raise ValueError( + f"STEP_INPUT binding {self.alias!r} selects no current planes." + ) selected = payload else: named_planes = provenance.source_plane_selection(self.alias) candidate_planes = ( range(provenance.source_plane_count) - if named_planes is None else named_planes + if named_planes is None + else named_planes ) selection = tuple( - index for index in candidate_planes + index + for index in candidate_planes if self.selector.matches_provenance(provenance.for_source_plane(index)) ) if not selection: - raise ValueError(f"STEP_INPUT binding {self.alias!r} selects no current planes.") + raise ValueError( + f"STEP_INPUT binding {self.alias!r} selects no current planes." + ) selected = metadata.project_source_planes(payload, selection) return self.apply_loaded_payload(selected, source_context=None) @@ -1180,11 +1193,10 @@ def _monochrome_source_data( """Convert declared RGB channels using the source intensity domain.""" import numpy as np - - normalized = metadata.payload_with(data).normalize_intensity_payload(dtype=np.float32,) - rgb = np.moveaxis( - np.asarray(normalized.data), channel_axis, -1 - )[..., :3] + normalized = metadata.payload_with(data).normalize_intensity_payload( + dtype=np.float32, + ) + rgb = np.moveaxis(np.asarray(normalized.data), channel_axis, -1)[..., :3] if np.all(rgb == rgb[..., :1]): return np.ascontiguousarray(rgb[..., 0]) @@ -1216,9 +1228,7 @@ def component_values( values.extend(source_component_metadata_values(metadata, component)) return tuple(dict.fromkeys(values)) - def _declared_component_values( - self, component: type[Axis] - ) -> tuple[str, ...]: + def _declared_component_values(self, component: type[Axis]) -> tuple[str, ...]: """Resolve exact identity values before selector-derived coordinates.""" for declarations in (self.component_identity, self.selector.components): values = tuple( @@ -1242,7 +1252,9 @@ def component_domains( for values in (self._declared_component_values(component),) if values } - if realized_source_metadata is None or len(domains) == len(AxisFamily.active().axes): + if realized_source_metadata is None or len(domains) == len( + AxisFamily.active().axes + ): return domains realized: dict[type[Axis], list[str]] = {} for metadata in realized_source_metadata: @@ -2318,7 +2330,8 @@ def from_config( return cls( bindings=( config.binding_declarations - if selected_bindings is None else selected_bindings + if selected_bindings is None + else selected_bindings ), metadata_rules=config.metadata_rule_declarations, match_plan=config.match_plan, @@ -2345,18 +2358,23 @@ def from_contracts( and any(contract.accepts_implicit_main_flow_input for contract in contracts) else () ) - source_specs = tuple(dict.fromkeys(( - *implicit_specs, - *( - spec - for contract in contracts - for spec in contract.artifact_inputs - if config.declares_artifact_ref(spec.ref()) - or available_artifacts.by_name_and_artifact_type( - spec.name, spec.artifact_type - ) is not None - ), - ))) + source_specs = tuple( + dict.fromkeys( + ( + *implicit_specs, + *( + spec + for contract in contracts + for spec in contract.artifact_inputs + if config.declares_artifact_ref(spec.ref()) + or available_artifacts.by_name_and_artifact_type( + spec.name, spec.artifact_type + ) + is not None + ), + ) + ) + ) if not source_specs: return cls.empty() bindings = config.bindings_for_artifact_specs(source_specs, available_artifacts) @@ -2531,7 +2549,6 @@ def _coerce_component(value: Any, field_name: str) -> type[Axis]: raise TypeError(f"{field_name}: {error}") from error - def _require_name(value: str, field_name: str) -> None: if not value: raise ValueError(f"{field_name} cannot be empty.") diff --git a/openhcs/core/source_image_provenance.py b/openhcs/core/source_image_provenance.py index 3db63f50f2..f063cf90fa 100644 --- a/openhcs/core/source_image_provenance.py +++ b/openhcs/core/source_image_provenance.py @@ -250,7 +250,11 @@ def from_mapping( raise ValueError(f"Unknown provenance plane fields: {sorted(unknown)!r}.") kind = values.get( "identity_kind", - RuntimeSourceImageProvenancePlane.identity_kind if identities is None else None, + ( + RuntimeSourceImageProvenancePlane.identity_kind + if identities is None + else None + ), ) if not isinstance(kind, str) or kind not in cls.__registry__: raise ValueError(f"Unknown source-image provenance identity kind {kind!r}.") @@ -265,7 +269,9 @@ def from_mapping( raise TypeError("Source provenance contributors must be a sequence.") if identities is None: facts = { - key: values[key] for key in ("path", "component_metadata") if key in values + key: values[key] + for key in ("path", "component_metadata") + if key in values } else: index = values.get("identity") @@ -548,11 +554,14 @@ def from_mapping(cls, values: object) -> "SourceImageProvenancePlanes": else: identities = None planes = values - if not isinstance(planes, Sequence) or isinstance(planes, (str, bytes, bytearray)): + if not isinstance(planes, Sequence) or isinstance( + planes, (str, bytes, bytearray) + ): raise TypeError("Source provenance planes must be a sequence.") return cls( tuple( - SourceImageProvenancePlane.from_mapping(item, identities) for item in planes + SourceImageProvenancePlane.from_mapping(item, identities) + for item in planes ) ) @@ -1079,11 +1088,15 @@ def required_scalar_components( ): continue values = frozenset( - value for values in contributor_components for value in values[component] + value + for values in contributor_components + for value in values[component] ) if len(values) > 1: reduced.add(component.name) - return tuple(component for component in component_order if component not in reduced) + return tuple( + component for component in component_order if component not in reduced + ) def image_set_identities( self, @@ -1108,11 +1121,9 @@ def image_set_plane_identities( ) -> tuple[frozenset[SourceImageSetIdentity], ...]: """Return one image-set identity entry for every runtime source plane.""" return tuple( - self.source_image_provenance_planes.plane( - index - ).source_identity.with_missing_from( - self.source_identity - ).image_set_identities(policy) + self.source_image_provenance_planes.plane(index) + .source_identity.with_missing_from(self.source_identity) + .image_set_identities(policy) for index in range(self.source_plane_count) ) diff --git a/openhcs/core/source_metadata.py b/openhcs/core/source_metadata.py index 27eb7b08a4..91518e78fa 100644 --- a/openhcs/core/source_metadata.py +++ b/openhcs/core/source_metadata.py @@ -218,7 +218,9 @@ def provenance_identity_items( return cls._view( metadata, "provenance_identity_items", - lambda: tuple((key, repr(value)) for key, value in cls.identity_items(metadata)), + lambda: tuple( + (key, repr(value)) for key, value in cls.identity_items(metadata) + ), ) @staticmethod @@ -256,10 +258,12 @@ def literal_field_types( value_types = types_by_name.setdefault(name, set()) if value is not None: value_types.add(type(value)) - return MappingProxyType({ - name: next(iter(value_types)) if len(value_types) == 1 else None - for name, value_types in types_by_name.items() - }) + return MappingProxyType( + { + name: next(iter(value_types)) if len(value_types) == 1 else None + for name, value_types in types_by_name.items() + } + ) @classmethod def component_value( @@ -298,11 +302,13 @@ def component_values( cls, metadata: SourceMetadataMapping, component: type[Axis] ) -> tuple[str, ...]: """Read only the requested component from the current field admission.""" - return tuple(dict.fromkeys( - str(value) - for owner, value in cls._ordered_component_fields(metadata) - if owner is component - )) + return tuple( + dict.fromkeys( + str(value) + for owner, value in cls._ordered_component_fields(metadata) + if owner is component + ) + ) @classmethod def component_domains( @@ -538,7 +544,8 @@ def _derived_view( @property def _accepted_components_complete(self) -> bool: return self._cacheable and all( - self._views.get(component) is not None for component in AxisFamily.active().axes + self._views.get(component) is not None + for component in AxisFamily.active().axes ) def _literal_value(self, key: str) -> SourceMetadataScalar: @@ -856,7 +863,9 @@ def __new__( name: str, physical_projection: Callable[["SourceVoxelSpacing"], float | None], native_unit: str, - physical_coordinates: Callable[["SourceVoxelSpacing"], tuple[float, ...] | None], + physical_coordinates: Callable[ + ["SourceVoxelSpacing"], tuple[float, ...] | None + ], ): member = str.__new__(cls, name) member._value_ = name @@ -868,7 +877,9 @@ def __new__( def physical_pixel_size(self, spacing: "SourceVoxelSpacing") -> float | None: return self._physical_projection(spacing) - def physical_coordinates(self, spacing: "SourceVoxelSpacing") -> tuple[float, ...] | None: + def physical_coordinates( + self, spacing: "SourceVoxelSpacing" + ) -> tuple[float, ...] | None: """Project coordinates only when this declaration establishes physical units.""" return self._physical_coordinates(spacing) @@ -1120,9 +1131,7 @@ def source_metadata_component(field: str) -> type[Axis] | None: @lru_cache(maxsize=1024) -def _axis_for_metadata_field( - family: type[AxisFamily], field: str -) -> type[Axis] | None: +def _axis_for_metadata_field(family: type[AxisFamily], field: str) -> type[Axis] | None: normalized = source_metadata_field_identity(field) owners = tuple( axis diff --git a/openhcs/core/source_workspace_projection.py b/openhcs/core/source_workspace_projection.py index b56d16fcd2..fcc19366bf 100644 --- a/openhcs/core/source_workspace_projection.py +++ b/openhcs/core/source_workspace_projection.py @@ -33,7 +33,10 @@ source_metadata_values_equal, ) from openhcs.core.source_path_identity import source_path_identity_key -from openhcs.core.source_projection import SourceProjection, SourceProjectionMetadataSerializer +from openhcs.core.source_projection import ( + SourceProjection, + SourceProjectionMetadataSerializer, +) from openhcs.core.virtual_workspace_metadata import ( OpenHCSMetadataPayload, OpenHCSMetadataSubdirectories, @@ -440,17 +443,20 @@ def load_binding_payloads( f"Workspace projection for {path!r} does not match compiled " f"source artifact {binding.input_spec().ref()!r}." ) - projected_payloads.append(binding.apply_loaded_payload( - self.project_payload(lookup, payload), - ImagePayloadSourceMetadataContext( - SourceImageIdentity( - self.logical_path_for(lookup), self.source_metadata_for(lookup), + projected_payloads.append( + binding.apply_loaded_payload( + self.project_payload(lookup, payload), + ImagePayloadSourceMetadataContext( + SourceImageIdentity( + self.logical_path_for(lookup), + self.source_metadata_for(lookup), + ), + projection.ref.backend, + filemanager, + projection.ref.backend_address, ), - projection.ref.backend, - filemanager, - projection.ref.backend_address, - ), - )) + ) + ) return tuple(projected_payloads) def validate_runtime_metadata_projection( @@ -525,15 +531,21 @@ def relative_virtual_paths(self) -> tuple[str, ...]: def component_values(self, component) -> tuple[str, ...]: """Derive the execution component domain from admitted source metadata.""" - return tuple(dict.fromkeys( - value - for path in self.relative_virtual_paths() - for value in source_component_metadata_values( - self.source_metadata_for(VirtualWorkspacePathLookup.from_paths( - path, self._loadable_virtual_path(path) - )) or {}, component, + return tuple( + dict.fromkeys( + value + for path in self.relative_virtual_paths() + for value in source_component_metadata_values( + self.source_metadata_for( + VirtualWorkspacePathLookup.from_paths( + path, self._loadable_virtual_path(path) + ) + ) + or {}, + component, + ) ) - )) + ) def filtered_by_axis( self, @@ -562,14 +574,16 @@ def partition_by_axes( ) ) values = ( - () if metadata is None + () + if metadata is None else source_component_metadata_values( metadata, AxisFamily.active().partition_axis() ) ) selected_axes = ( tuple(dict.fromkeys(value for value in values if value in source_refs)) - if values else tuple(source_refs) + if values + else tuple(source_refs) ) if not selected_axes: continue @@ -589,17 +603,19 @@ def partition_by_axes( source_projections[axis_id][virtual_path] = projection source_metadata[axis_id].update(metadata_records) - return MappingProxyType({ - axis_id: VirtualWorkspaceSourceProjection( - source_refs_by_virtual_path=MappingProxyType(refs), - source_metadata_by_path=MappingProxyType(source_metadata[axis_id]), - source_projections_by_virtual_path=MappingProxyType( - source_projections[axis_id] - ), - workspace_root=self.workspace_root, - ) - for axis_id, refs in source_refs.items() - }) + return MappingProxyType( + { + axis_id: VirtualWorkspaceSourceProjection( + source_refs_by_virtual_path=MappingProxyType(refs), + source_metadata_by_path=MappingProxyType(source_metadata[axis_id]), + source_projections_by_virtual_path=MappingProxyType( + source_projections[axis_id] + ), + workspace_root=self.workspace_root, + ) + for axis_id, refs in source_refs.items() + } + ) def _path_belongs_to_axis( self, @@ -668,9 +684,7 @@ def apply(self, payload: RuntimeArrayData) -> RuntimeArrayData: metadata = metadata.replace_fields( source_spatial_domain=metadata.source_spatial_domain.with_native_image_context( current_metadata.source_spatial_domain, - image_shape_yx=current_metadata.spatial_shape_yx( - payload.data - ), + image_shape_yx=current_metadata.spatial_shape_yx(payload.data), ) ) if source_metadata is not None and self.persisted_metadata is None: @@ -704,14 +718,18 @@ class VirtualWorkspaceSourceProjectionCacheEntry: metadata: OpenHCSMetadataPayload projection: VirtualWorkspaceSourceProjection | None axis_filtered_projections: dict[str, VirtualWorkspaceSourceProjection] = field( - default_factory=dict, compare=False, repr=False, + default_factory=dict, + compare=False, + repr=False, ) source_accepted_entries: dict[ - tuple[SourceFilterClause, ...], VirtualWorkspaceSourceProjectionCacheEntry, + tuple[SourceFilterClause, ...], + VirtualWorkspaceSourceProjectionCacheEntry, ] = field(default_factory=dict, compare=False, repr=False) def accepted_for( - self, source_bindings: SourceBindingsConfig | None, + self, + source_bindings: SourceBindingsConfig | None, ) -> VirtualWorkspaceSourceProjectionCacheEntry: """Bind prepared-source admission to its document and declarations.""" if self.projection is None or source_bindings is None: @@ -721,46 +739,57 @@ def accepted_for( return self admitted = self.source_accepted_entries.get(declarations) if admitted is None: - from openhcs.core.source_binding_workspace import SourceBindingWorkspaceProjector + from openhcs.core.source_binding_workspace import ( + SourceBindingWorkspaceProjector, + ) admitted = VirtualWorkspaceSourceProjectionCacheEntry( self.metadata, - SourceBindingWorkspaceProjector(source_bindings).accept_prepared_projection( - self.projection - ), + SourceBindingWorkspaceProjector( + source_bindings + ).accept_prepared_projection(self.projection), ) self.source_accepted_entries[declarations] = admitted return admitted def entry_for_projection( - self, projection: VirtualWorkspaceSourceProjection, + self, + projection: VirtualWorkspaceSourceProjection, ) -> VirtualWorkspaceSourceProjectionCacheEntry | None: """Recognize only projections retained by this admitted document.""" if self.projection is projection: return self - return next(( - entry for entry in self.source_accepted_entries.values() - if entry.projection is projection - ), None) + return next( + ( + entry + for entry in self.source_accepted_entries.values() + if entry.projection is projection + ), + None, + ) def partition_by_axes( - self, axis_ids: Sequence[str], + self, + axis_ids: Sequence[str], ) -> Mapping[str, VirtualWorkspaceSourceProjection]: """Derive missing axis views together from this retained document.""" if self.projection is None: - raise ValueError("Axis views require an admitted source workspace projection.") + raise ValueError( + "Axis views require an admitted source workspace projection." + ) selected = tuple(dict.fromkeys(axis_ids)) missing = tuple( - axis_id for axis_id in selected + axis_id + for axis_id in selected if axis_id not in self.axis_filtered_projections ) if missing: self.axis_filtered_projections.update( self.projection.partition_by_axes(missing) ) - return MappingProxyType({ - axis_id: self.axis_filtered_projections[axis_id] for axis_id in selected - }) + return MappingProxyType( + {axis_id: self.axis_filtered_projections[axis_id] for axis_id in selected} + ) @dataclass(slots=True) @@ -782,9 +811,11 @@ def projection_for( dataset_key = str(dataset_root) cached = self.projections_by_dataset_root.get(dataset_key) if cached is None or cached.metadata is not metadata: - projection = VirtualWorkspaceSourceProjection.from_openhcs_metadata_if_available( - dataset_root, - metadata, + projection = ( + VirtualWorkspaceSourceProjection.from_openhcs_metadata_if_available( + dataset_root, + metadata, + ) ) cached = VirtualWorkspaceSourceProjectionCacheEntry(metadata, projection) self.projections_by_dataset_root[dataset_key] = cached @@ -815,7 +846,8 @@ def partition_by_axes( """Share admitted axis views between compilation and runtime queries.""" document_entry = self.projections_by_dataset_root.get(projection.workspace_root) cached = ( - None if document_entry is None + None + if document_entry is None else document_entry.entry_for_projection(projection) ) if cached is None: @@ -840,7 +872,10 @@ class WorkspaceSourceProjections: cache: VirtualWorkspaceSourceProjectionCache | None = None source_bindings: SourceBindingsConfig | None = None _workspace_metadata_handler: "OpenHCSMetadataHandler | None" = field( - default=None, init=False, compare=False, repr=False, + default=None, + init=False, + compare=False, + repr=False, ) @classmethod @@ -887,7 +922,10 @@ def is_bound_to_context(self, context: "ProcessingContext") -> bool: and self.filemanager is context.filemanager ): return False - return self.source_bindings == context.microscope_handler.source_bindings_still_required() + return ( + self.source_bindings + == context.microscope_handler.source_bindings_still_required() + ) def metadata_handlers(self) -> tuple["MetadataHandler", ...]: """Observe workspace eligibility live while retaining admitted providers.""" @@ -932,24 +970,32 @@ def _projection_for_axis( return self.cache.filtered_by_axis(projection, axis_id=axis_id) def projection_if_available( - self, *, axis_id: str | None = None, + self, + *, + axis_id: str | None = None, ) -> VirtualWorkspaceSourceProjection | None: workspace_root = self.metadata_handler.source_workspace_root(self.dataset_root) for metadata in self.metadata_documents(): if self.cache is None: - projection = VirtualWorkspaceSourceProjection.from_openhcs_metadata_if_available( - workspace_root, - metadata, + projection = ( + VirtualWorkspaceSourceProjection.from_openhcs_metadata_if_available( + workspace_root, + metadata, + ) ) if projection is not None and self.source_bindings is not None: - from openhcs.core.source_binding_workspace import SourceBindingWorkspaceProjector + from openhcs.core.source_binding_workspace import ( + SourceBindingWorkspaceProjector, + ) projection = SourceBindingWorkspaceProjector( self.source_bindings ).accept_prepared_projection(projection) else: projection = self.cache.projection_for( - workspace_root, metadata, source_bindings=self.source_bindings, + workspace_root, + metadata, + source_bindings=self.source_bindings, ) if projection is None: continue @@ -957,7 +1003,9 @@ def projection_if_available( return None def projection_or_empty( - self, *, axis_id: str | None = None, + self, + *, + axis_id: str | None = None, ) -> VirtualWorkspaceSourceProjection: projection = self.projection_if_available(axis_id=axis_id) if projection is not None: @@ -966,9 +1014,7 @@ def projection_or_empty( @dataclass(frozen=True, slots=True) -class RuntimeWorkspaceSourceProjections( - WorkspaceSourceProjections -): +class RuntimeWorkspaceSourceProjections(WorkspaceSourceProjections): """Read completed same-plate outputs from their execution observation owner.""" context: "ProcessingContext" = field(kw_only=True, compare=False, repr=False) @@ -979,7 +1025,9 @@ def is_bound_to_context(self, context: "ProcessingContext") -> bool: ).is_bound_to_context(context) def projection_if_available( - self, *, axis_id: str | None = None, + self, + *, + axis_id: str | None = None, ) -> VirtualWorkspaceSourceProjection | None: projection = super( RuntimeWorkspaceSourceProjections, self @@ -995,7 +1043,8 @@ def projection_if_available( ) if not produced_entries: return ( - None if projection is None + None + if projection is None else self._projection_for_axis(projection, axis_id=axis_id) ) builder = VirtualWorkspaceSourceProjectionBuilder(self.dataset_root) @@ -1009,7 +1058,9 @@ def projection_if_available( ) ) builder.ingest_source_metadata( - VirtualWorkspaceSourceMetadataEntries(projection.source_metadata_by_path) + VirtualWorkspaceSourceMetadataEntries( + projection.source_metadata_by_path + ) ) for entries in produced_entries: fields = SourceProjectionMetadataSerializer.workspace_fields( diff --git a/openhcs/core/steps/function_artifact_materialization.py b/openhcs/core/steps/function_artifact_materialization.py index 1dd7a13372..f53da132ba 100644 --- a/openhcs/core/steps/function_artifact_materialization.py +++ b/openhcs/core/steps/function_artifact_materialization.py @@ -177,27 +177,38 @@ def materialize_outputs( pipeline_position=plan.pipeline_position, output_plan=materialization.output_plan, ) - batch = replace(batch, context=replace( - batch.context, - backend_kwargs=BackendKwargsAuthority.normalize(self.backend_kwargs( - materialization=materialization, - outputs=batch.outputs, - persistent_backend_kwargs=persistent_backend_kwargs, - streaming_viewer_surfaces=streaming_viewer_surfaces, - fallback_source_identity=( - materialization.filename_source_identity - if materialization.spec.uses_filename_source_identity(data) - else materialization.source_identity - ), - producer_identity=( - plan.producer_identity_for_artifact(materialization.output_plan) + batch = replace( + batch, + context=replace( + batch.context, + backend_kwargs=BackendKwargsAuthority.normalize( + self.backend_kwargs( + materialization=materialization, + outputs=batch.outputs, + persistent_backend_kwargs=persistent_backend_kwargs, + streaming_viewer_surfaces=streaming_viewer_surfaces, + fallback_source_identity=( + materialization.filename_source_identity + if materialization.spec.uses_filename_source_identity( + data + ) + else materialization.source_identity + ), + producer_identity=( + plan.producer_identity_for_artifact( + materialization.output_plan + ) + ), + context=context, + filemanager=filemanager, + images_dir=images_dir, + stream_output_paths=tuple( + output.path for output in batch.outputs + ), + ) ), - context=context, - filemanager=filemanager, - images_dir=images_dir, - stream_output_paths=tuple(output.path for output in batch.outputs), - )), - )) + ), + ) saved_materializations.append( MaterializedRuntimeArtifact( outputs_by_backend=MappingProxyType( @@ -223,7 +234,9 @@ def streaming_viewer_surfaces( ) return ( { - config.viewer_family.backend.value: config.streaming_viewer_surface(context) + config.viewer_family.backend.value: config.streaming_viewer_surface( + context + ) for config in plan.streaming_configs.values() if step_axis_allows_config( context.step_axis_filters, @@ -307,8 +320,11 @@ def backend_kwargs( for backend, viewer_surface in streamable_viewer_surfaces.items(): backend_instance = filemanager._get_backend(backend) source_metadata_items = materialization.stream_source_metadata_items( - tuple(output for output in outputs - if backend_instance.accepts_payload(output.content, output.path)), + tuple( + output + for output in outputs + if backend_instance.accepts_payload(output.content, output.path) + ), fallback_source_identity, ) stream_backend_kwargs = StreamComponentMessageExtraAuthority.from_context( @@ -1236,8 +1252,7 @@ def observation( return StepExecutionObservation.empty() address = RuntimeArtifactAddress.from_record(self.materialization.record) locations = tuple( - RuntimeArtifactLocation.from_output(output, backend) - for output in outputs + RuntimeArtifactLocation.from_output(output, backend) for output in outputs ) paths = ( tuple(Path(output.path) for output in outputs) diff --git a/openhcs/core/steps/function_execution.py b/openhcs/core/steps/function_execution.py index 652463832b..23c416408f 100644 --- a/openhcs/core/steps/function_execution.py +++ b/openhcs/core/steps/function_execution.py @@ -523,8 +523,10 @@ def apply( def source_pattern_context(self) -> SourcePatternResolutionContext: """Return source-path context used to filter source-bound anchors.""" - projection = self.context.runtime_source_workspace_projections.projection_or_empty( - axis_id=self.plan.axis_id, + projection = ( + self.context.runtime_source_workspace_projections.projection_or_empty( + axis_id=self.plan.axis_id, + ) ) return self.context.runtime_source_binding_context_cache.source_pattern_context( parser=self.context.microscope_handler.parser, @@ -804,7 +806,9 @@ def _detect_patterns(self) -> dict[str, DiscoveredPatternCollection]: plan = self.plan source_files = step_output_manifest(self.context).producer_paths_for(plan) if source_files is None: - source_projection = self.context.runtime_source_workspace_projections.projection_if_available() + source_projection = ( + self.context.runtime_source_workspace_projections.projection_if_available() + ) if ( plan.main_input_dependency.kind is StepInputDependencyKind.PIPELINE_START diff --git a/openhcs/core/steps/function_io.py b/openhcs/core/steps/function_io.py index 32e0212ee1..3b2b2b5f1e 100644 --- a/openhcs/core/steps/function_io.py +++ b/openhcs/core/steps/function_io.py @@ -183,12 +183,12 @@ def project_item_values( "Parsed output identities disagree on component " f"{component.name!r}; missing from item indices {missing_indices!r}" ) - return tuple(cls.item_value(identity, component) for identity in item_identities) + return tuple( + cls.item_value(identity, component) for identity in item_identities + ) @classmethod - def item_value( - cls, identity: ZarrBatchItemIdentity, component: type[Axis] - ) -> str: + def item_value(cls, identity: ZarrBatchItemIdentity, component: type[Axis]) -> str: value = identity.component_values.value_for(component) if value is None: raise ValueError( @@ -216,9 +216,7 @@ class ColourZarrAxisProjection(ZarrComponentAxisProjection): axis_name = "c" @classmethod - def item_value( - cls, identity: ZarrBatchItemIdentity, component: type[Axis] - ) -> str: + def item_value(cls, identity: ZarrBatchItemIdentity, component: type[Axis]) -> str: value = super().item_value(identity, component) qualifier = identity.filename_qualifier return value if qualifier is None else f"{value}:{qualifier}" diff --git a/openhcs/core/steps/function_output_identity.py b/openhcs/core/steps/function_output_identity.py index b0ffc46718..2f21e02e95 100644 --- a/openhcs/core/steps/function_output_identity.py +++ b/openhcs/core/steps/function_output_identity.py @@ -712,7 +712,8 @@ def _complete_identity_from_paths( identity_cache: FunctionOutputIdentityCache, ) -> FunctionOutputIdentity: if identity.extension is not None and all( - component.name in identity.component_values for component in AxisFamily.active().axes + component.name in identity.component_values + for component in AxisFamily.active().axes ): return identity for path, source in candidates: @@ -929,7 +930,9 @@ def _source_stack_semantic_component_keys( - identity_component_values ) ordered_keys = tuple( - component.name for component in AxisFamily.active().axes if component.name in keys + component.name + for component in AxisFamily.active().axes + if component.name in keys ) extra_keys = tuple(sorted(keys - frozenset(ordered_keys))) return (*ordered_keys, *extra_keys) diff --git a/openhcs/core/steps/function_outputs.py b/openhcs/core/steps/function_outputs.py index 7361f8a384..90480d3505 100644 --- a/openhcs/core/steps/function_outputs.py +++ b/openhcs/core/steps/function_outputs.py @@ -18,7 +18,10 @@ extract_key_from_class_name, ) from polystore.streaming.identity import StreamProducerIdentity -from polystore.streaming.viewer_transport import ViewerStreamProducer, ViewerStreamSourceMetadata +from polystore.streaming.viewer_transport import ( + ViewerStreamProducer, + ViewerStreamSourceMetadata, +) from polystore.virtual_workspace import SourcePixelRef from openhcs.constants.constants import Backend @@ -513,9 +516,7 @@ def partition_by_item_fields( def project_item( request: RuntimeProjectionSourceIdentityRequest, ) -> tuple[RuntimeProjectedPayloadItem, ...]: - return ( - RequiredSourceComponentMetadata - ).project_payload_items(request) + return (RequiredSourceComponentMetadata).project_payload_items(request) @staticmethod def stream_path_for_projected_item( @@ -693,7 +694,8 @@ def for_execution( for declaration in cls.__registry__.values() if (owner := declaration.from_execution(context, plan)) is not None for target in replace( - owner, artifact_materializations=artifact_materializations, + owner, + artifact_materializations=artifact_materializations, create_openhcs_metadata=plan.create_openhcs_metadata, ).production_targets(context, plan) ) @@ -711,7 +713,9 @@ def reconciliation_targets( context: ProcessingContext, *, document: OpenHCSMetadataSubdirectories | None = None, - accepted_entries: Mapping[str, VirtualWorkspaceSourceProjectionEntries] | None = None, + accepted_entries: ( + Mapping[str, VirtualWorkspaceSourceProjectionEntries] | None + ) = None, ) -> tuple[OpenHCSMetadataTarget, ...]: """Resolve destinations after runtime values have been released.""" return (self,) @@ -783,7 +787,9 @@ def write( str(self.output_dir), self.backend ) ) - return (metadata_writer or AtomicMetadataWriter()).publish_source_projection_metadata( + return ( + metadata_writer or AtomicMetadataWriter() + ).publish_source_projection_metadata( METADATA_CONFIG.metadata_path(self.dataset_root), self.sub_dir, projection_entries, @@ -817,7 +823,8 @@ def produced_projection_entries( raise ValueError("OpenHCS metadata requires a file manager.") target = type(self).from_plan(plan) if target is None or self not in replace( - target, artifact_materializations=self.artifact_materializations, + target, + artifact_materializations=self.artifact_materializations, create_openhcs_metadata=self.create_openhcs_metadata, ).production_targets(context, plan): raise ValueError("Produced metadata plan does not own this output target.") @@ -1064,11 +1071,19 @@ def finalize_completed_dataset( for target in cls.for_plan(plan): target_contexts.setdefault(target, context) - produced_by_document: dict[Path, dict[OpenHCSMetadataTarget, list[VirtualWorkspaceSourceProjectionEntries]]] = {} + produced_by_document: dict[ + Path, + dict[OpenHCSMetadataTarget, list[VirtualWorkspaceSourceProjectionEntries]], + ] = {} for observation in runtime_observations: for context_observation in observation.contexts: context = compiled_contexts[context_observation.context_key] - for target, entries in context_observation.outputs.source_projection_entries_by_target.items(): + for ( + target, + entries, + ) in ( + context_observation.outputs.source_projection_entries_by_target.items() + ): produced_by_document.setdefault( METADATA_CONFIG.metadata_path(target.dataset_root), {} ).setdefault(target, []).append(entries) @@ -1082,13 +1097,17 @@ def finalize_completed_dataset( targets_by_document.setdefault( METADATA_CONFIG.metadata_path(owner.dataset_root), {} )[owner] = context - for metadata_path in dict.fromkeys((*targets_by_document, *produced_by_document)): + for metadata_path in dict.fromkeys( + (*targets_by_document, *produced_by_document) + ): AtomicMetadataWriter().reconcile_completed_dataset( metadata_path, targets_by_document.get(metadata_path, {}), produced_entries_by_target={ target: VirtualWorkspaceSourceProjectionEntries.combine(entries) - for target, entries in produced_by_document.get(metadata_path, {}).items() + for target, entries in produced_by_document.get( + metadata_path, {} + ).items() }, ) @@ -1193,17 +1212,22 @@ def reconciliation_targets( context: ProcessingContext, *, document: OpenHCSMetadataSubdirectories | None = None, - accepted_entries: Mapping[str, VirtualWorkspaceSourceProjectionEntries] | None = None, + accepted_entries: ( + Mapping[str, VirtualWorkspaceSourceProjectionEntries] | None + ) = None, ) -> tuple[RuntimeArtifactMetadataTarget, ...]: """Use durable typed projections, without reloading cleaned artifact values.""" from openhcs.core.dataset_sources.openhcs_format import OpenHCSMetadataHandler if (document is None) != (accepted_entries is None): - raise ValueError("Reconciliation requires one document and its admitted entries.") + raise ValueError( + "Reconciliation requires one document and its admitted entries." + ) handler = OpenHCSMetadataHandler(context.filemanager) directories = ( handler.reconciliation_directories(self.dataset_root, self.backend) - if document is None else handler.reconciliation_directories_from_document( + if document is None + else handler.reconciliation_directories_from_document( self.dataset_root, self.backend, document, accepted_entries ) ) diff --git a/openhcs/core/steps/function_runtime.py b/openhcs/core/steps/function_runtime.py index cc2c741d64..176e27a46b 100644 --- a/openhcs/core/steps/function_runtime.py +++ b/openhcs/core/steps/function_runtime.py @@ -264,7 +264,10 @@ def source_binding_plan(self) -> CompiledSourceBindingPlan: source_refs = self.invocation_source_artifact_refs if source_refs: return declared_plan.for_artifact_refs(source_refs) - if self.compiled_group.runtime_domain is RuntimeInvocationDomain.ARTIFACT_MANAGED: + if ( + self.compiled_group.runtime_domain + is RuntimeInvocationDomain.ARTIFACT_MANAGED + ): # The compiler has already traced stored artifact lineage to its # source bindings. A produced name is not itself a raw alias. return declared_plan @@ -344,7 +347,9 @@ def loaded_fixed_component_values( payload: RuntimeArrayData, ) -> RuntimeFixedComponentValues: """Capture the source-selected cohort's common fixed coordinates.""" - source_provenance = payload.metadata.source_provenance.with_common_scalar_identity_from_planes() + source_provenance = ( + payload.metadata.source_provenance.with_common_scalar_identity_from_planes() + ) common_source_metadata = source_provenance.source_component_metadata or {} variable_components = ComponentSet.of( self.execution_plan.variable_components or () @@ -413,7 +418,9 @@ def _input_relative_path(cls, input_dir: Path, matched_path: str) -> Path: def run(self) -> None: start_time = time.time() if logger.isEnabledFor(logging.DEBUG) else None plan = self.execution_plan - logger.debug("Processing pattern %s for axis %s", self.pattern_repr, plan.axis_id) + logger.debug( + "Processing pattern %s for axis %s", self.pattern_repr, plan.axis_id + ) try: load_started_at = time.perf_counter() @@ -501,7 +508,9 @@ def run(self) -> None: ) except Exception as e: logger.error( - "Error processing pattern group %s: %s", self.pattern_repr, e, + "Error processing pattern group %s: %s", + self.pattern_repr, + e, exc_info=True, ) raise ValueError( @@ -774,7 +783,6 @@ def _filter_matching_files_for_group( parser, matching_files, group_component, - component_value, ) if not filtered: @@ -1074,7 +1082,9 @@ def _project_output_slices( return tuple(zip(output_payloads, slice_contexts, strict=True)) def _unstack_main_output( - self, processed_data: Any, expected_count: int, + self, + processed_data: Any, + expected_count: int, ) -> tuple[Any, ...]: """Split undeclared main-output pixels along their leading runtime-slice axis.""" try: @@ -1235,7 +1245,9 @@ def _save_outputs( output_identity, output_context=output_context, image_metadata=output_metadata, - main_flow_plane_axis=processed_stack.plane_axis_for_output_context(output_context), + main_flow_plane_axis=processed_stack.plane_axis_for_output_context( + output_context + ), ) if output_directory_exists and context.filemanager.exists( @@ -1595,10 +1607,6 @@ def _save_artifact_value( return runtime_value.data - - - - @dataclass(frozen=True, slots=True) class FunctionCoreExecutor: """Execute one scoped callable invocation and route declared artifact I/O.""" @@ -1721,21 +1729,24 @@ def loaded_source_artifact_payload( or request.artifact_inputs.get(edge.key) is not edge or edge.storage_plan is not None or edge.main_flow_projection is None - or self.group_data.main_flow_source_binding_plan.binding_for_artifact_ref(ref) + or self.group_data.main_flow_source_binding_plan.binding_for_artifact_ref( + ref + ) != binding ): return None payload = self.invocation.convert_input( - self.group_data.main_data_stack, self.source_memory_type, + self.group_data.main_data_stack, + self.source_memory_type, ) if edge.main_flow_projection is MainFlowInputProjection.DECLARED_SOURCE_IMAGE: payload = payload.project_declared_source(ref.name) elif edge.main_flow_projection is not MainFlowInputProjection.COMPLETE_PAYLOAD: return None metadata = payload.metadata - if ( - not metadata.persists_whole_image() - and metadata.plane_axis not in (None, RuntimePlaneAxis.RUNTIME_SLICE) + if not metadata.persists_whole_image() and metadata.plane_axis not in ( + None, + RuntimePlaneAxis.RUNTIME_SLICE, ): # A remaining binding axis does not prove this source's ordered # physical members. Resolve it through its declared origin owner. @@ -1747,7 +1758,9 @@ def loaded_source_artifact_payload( member_count=member_count if member_count is not None else 1, ) projection = RuntimePlaneAxisValueProjection.from_projector( - request.plane_projection, RuntimePlaneAxis.RUNTIME_SLICE, (), + request.plane_projection, + RuntimePlaneAxis.RUNTIME_SLICE, + (), ) if projection is not None and projection.plane_index is not None: payload = RuntimeSliceProjection.value_for_slice(payload, projection) @@ -1770,7 +1783,8 @@ def declared_source_payload( stored_payload = loaded_artifact_payloads.get(source_ref) source_binding_plan = ( self.group_data.source_binding_plan - if request is None else request.source_binding_plan + if request is None + else request.source_binding_plan ) source_binding = source_binding_plan.binding_for_artifact_ref(source_ref) main_flow_edges = tuple( @@ -1801,7 +1815,8 @@ def declared_source_payload( RuntimePayload, ( self.runtime_adapter_request(primary_source_payload) - if request is None else request + if request is None + else request ).source_artifact_payload(source_ref), ) if len(main_flow_edges) != 1: diff --git a/openhcs/core/steps/stream_component_semantics.py b/openhcs/core/steps/stream_component_semantics.py index 8df6ace26f..2d4e5713a6 100644 --- a/openhcs/core/steps/stream_component_semantics.py +++ b/openhcs/core/steps/stream_component_semantics.py @@ -98,9 +98,7 @@ def payload_items_for_display( axis=metadata.plane_axis, axis_size=metadata.source_provenance.source_plane_count, ) - source_identity = ( - RequiredSourceComponentMetadata - ) + source_identity = RequiredSourceComponentMetadata return source_identity.project_payload_items( RuntimeProjectionSourceIdentityRequest( value=metadata.attach_to(payload), @@ -184,7 +182,9 @@ def _item_fields( ) colour_axis_position = metadata.axis_position(ColourAxis) if colour_axis_position is not None: - item_fields[ViewerWireField.SOURCE_CHANNEL_AXIS.value] = colour_axis_position + item_fields[ViewerWireField.SOURCE_CHANNEL_AXIS.value] = ( + colour_axis_position + ) if metadata.plane_axis is None: return item_fields @@ -730,7 +730,6 @@ def build_for_component( def supports(cls, component: str) -> bool: return component == AxisFamily.active().partition_axis().name - def domain_metadata_items(self) -> StreamComponentDomainMetadataItems: return tuple({self.component: value} for value in self.execution_axis_values) diff --git a/openhcs/core/viewer_streaming_service.py b/openhcs/core/viewer_streaming_service.py index 8ea8da76f6..572763cf04 100644 --- a/openhcs/core/viewer_streaming_service.py +++ b/openhcs/core/viewer_streaming_service.py @@ -412,9 +412,7 @@ def load_image( source_address=source_address, ).metadata(image) metadata = self.calibrated_metadata(metadata) - return metadata.payload_with( - image.data, image.mask - ) + return metadata.payload_with(image.data, image.mask) def component_metadata_by_path( self, @@ -604,7 +602,9 @@ def _wait_for_viewer_ready( f"{config.viewer_family.display_name} viewer on port {viewer.port} failed to become ready" ) - logger.info(f"{config.viewer_family.display_name} viewer on port {viewer.port} is ready") + logger.info( + f"{config.viewer_family.display_name} viewer on port {viewer.port} is ready" + ) @staticmethod def _require_viewer_settled(request: ViewerStreamingContext) -> None: @@ -843,7 +843,9 @@ def stream_rois( ) archive_metadata.append(metadata) if metadata is not None: - geometric_domain = ROIArchiveSourceMetadata.source_component_domain(rois, metadata) + geometric_domain = ROIArchiveSourceMetadata.source_component_domain( + rois, metadata + ) if geometric_domain is not None: geometric_domains[filename] = geometric_domain data_list.append(rois) @@ -922,7 +924,9 @@ def stream_rois( source_metadata_items = StreamSourceComponentMetadataItems.from_values( component_metadata for path in paths - for component_metadata in geometric_domains.get(path, (metadata_by_path[path],)) + for component_metadata in geometric_domains.get( + path, (metadata_by_path[path],) + ) ) message_authority = StreamComponentMessageExtraAuthority.from_viewer_surface( viewer_surface, @@ -944,12 +948,17 @@ def stream_rois( component_order = message_authority.layout.component_order item_fields_by_archive = tuple( ROIArchiveSourceMetadata.stream_item_fields( - rois, metadata, - StreamImagePayloadMetadataProjector.item_fields(metadata, component_order), + rois, + metadata, + StreamImagePayloadMetadataProjector.item_fields( + metadata, component_order + ), ) for rois, metadata in zip(data_list, payload_metadata, strict=True) ) - for indices in StreamImagePayloadMetadataProjector.partition_item_fields(item_fields_by_archive): + for indices in StreamImagePayloadMetadataProjector.partition_item_fields( + item_fields_by_archive + ): item_fields = item_fields_by_archive[indices[0]] stream_backend_kwargs = message_authority.viewer_backend_kwargs( producer=producer.for_indices( diff --git a/openhcs/core/virtual_workspace_metadata.py b/openhcs/core/virtual_workspace_metadata.py index 21cd8b65cd..6b1a4fc007 100644 --- a/openhcs/core/virtual_workspace_metadata.py +++ b/openhcs/core/virtual_workspace_metadata.py @@ -168,7 +168,8 @@ def update(data): ) published_entries = ( VirtualWorkspaceSourceProjectionEntries(MappingProxyType({})) - if projection_entries is None else projection_entries + if projection_entries is None + else projection_entries ).publish_into_subdirectory( subdirectory, saved_image_paths=saved_paths, @@ -224,16 +225,22 @@ def reconcile_completed_dataset( ] = MappingProxyType({}), ) -> None: """Admit and reconcile all targets from one locked plate document.""" - if not produced_entries_by_target and not Path(metadata_path).is_file() and not any( - target.contains_outputs(context) - for target, context in target_contexts.items() + if ( + not produced_entries_by_target + and not Path(metadata_path).is_file() + and not any( + target.contains_outputs(context) + for target, context in target_contexts.items() + ) ): return def update(data): data = self._ensure_subdirectories_structure(data) subdirectories = data[METADATA_CONFIG.SUBDIRECTORIES_KEY] - updates_by_directory: dict[str, list[VirtualWorkspaceSourceProjectionEntries]] = {} + updates_by_directory: dict[ + str, list[VirtualWorkspaceSourceProjectionEntries] + ] = {} for target, entries in produced_entries_by_target.items(): updates_by_directory.setdefault(target.sub_dir, []).append(entries) admitted = {} @@ -245,7 +252,11 @@ def update(data): document = OpenHCSMetadataSubdirectories(data) for name, subdirectory in document.items(): if name not in admitted: - admitted[name] = VirtualWorkspaceSourceProjectionEntries.from_subdirectory(subdirectory) + admitted[name] = ( + VirtualWorkspaceSourceProjectionEntries.from_subdirectory( + subdirectory + ) + ) for name in updates_by_directory: self._update_projection_geometry( subdirectories[name], admitted[name].entries.values() @@ -264,7 +275,9 @@ def update(data): metadata_document=data, accepted_entries=admitted.get( target.sub_dir, - VirtualWorkspaceSourceProjectionEntries(MappingProxyType({})), + VirtualWorkspaceSourceProjectionEntries( + MappingProxyType({}) + ), ), ) return data @@ -318,8 +331,6 @@ def get_metadata_path(dataset_root: str | Path) -> Path: return METADATA_CONFIG.metadata_path(dataset_root) - - @dataclass(frozen=True) class OpenHCSMetadataFields: """Field identities declared by the OpenHCS metadata contract. @@ -483,9 +494,7 @@ def from_projection_paths( projection_paths: Sequence[tuple[SourceProjection, str]], ) -> "VirtualWorkspaceSourceProjectionEntries": """Admit a producer's typed updates in persisted path order.""" - SourceProjectionSet( - tuple(projection for projection, _path in projection_paths) - ) + SourceProjectionSet(tuple(projection for projection, _path in projection_paths)) return cls( MappingProxyType( {path: projection for projection, path in projection_paths} @@ -522,7 +531,8 @@ def _accept_retained_records( for path, record in records.items(): virtual_path, projection = ( (path, self.entries[path]) - if path in self.entries else self._projection_record(record) + if path in self.entries + else self._projection_record(record) ) if virtual_path in entries: raise RuntimeError( @@ -577,7 +587,8 @@ def publish_into_subdirectory( records = self._records_by_path(subdirectory) admitted = ( self._accept_retained_records(records) - if accepted_entries is None else accepted_entries + if accepted_entries is None + else accepted_entries ) saved_set = frozenset(saved_image_paths) missing = saved_set.difference(admitted.entries) @@ -608,7 +619,8 @@ def publish_into_subdirectory( normalized_path = self._required_text(record, "virtual_path") if normalized_path in retained.entries: retained_records[normalized_path] = ( - record if path == normalized_path + record + if path == normalized_path else {**record, "virtual_path": normalized_path} ) retained_records.update( @@ -689,12 +701,17 @@ def _projection_record( source_metadata = cls._optional_metadata(record, "source_metadata") component_labels = cls._optional_component_labels(record) source_alias = cls._optional_text(record, "source_alias") - image_metadata_value = record.get(SourcePlaneProjection.image_metadata_wire_field()) - if image_metadata_value is not None and not isinstance(image_metadata_value, Mapping): + image_metadata_value = record.get( + SourcePlaneProjection.image_metadata_wire_field() + ) + if image_metadata_value is not None and not isinstance( + image_metadata_value, Mapping + ): raise TypeError("Persisted image metadata must be a mapping.") image_metadata = ( - None if image_metadata_value is None else - ImagePayloadMetadata.from_mapping(image_metadata_value) + None + if image_metadata_value is None + else ImagePayloadMetadata.from_mapping(image_metadata_value) ) if projection_role is SourceProjectionRole.PRIMARY_PLANE: projection: SourceProjection = SourcePlaneProjection( @@ -892,6 +909,5 @@ def from_subdirectory( entries.update({str(key): str(value) for key, value in labels.items()}) return cls(MappingProxyType(entries)) - def label_for(self, channel_value: SourceMetadataScalar) -> str | None: return self.entries.get(str(channel_value)) diff --git a/openhcs/demo/basic_pipeline.py b/openhcs/demo/basic_pipeline.py index e7857a4c93..7264e3a100 100644 --- a/openhcs/demo/basic_pipeline.py +++ b/openhcs/demo/basic_pipeline.py @@ -45,9 +45,7 @@ step_2 = FunctionStep( func=create_composite, name="create_composite", - processing_config=LazyProcessingConfig( - variable_components=[Microscopy.Channel] - ), + processing_config=LazyProcessingConfig(variable_components=[Microscopy.Channel]), ) pipeline_steps.append(step_2) @@ -55,9 +53,7 @@ step_3 = FunctionStep( func=create_projection, name="Z-Stack Flattening", - processing_config=LazyProcessingConfig( - variable_components=[Microscopy.ZIndex] - ), + processing_config=LazyProcessingConfig(variable_components=[Microscopy.ZIndex]), step_materialization_config=LazyStepMaterializationConfig(), ) pipeline_steps.append(step_3) @@ -84,9 +80,7 @@ step_7 = FunctionStep( func=create_projection, name="Z-Stack Flattening", - processing_config=LazyProcessingConfig( - variable_components=[Microscopy.ZIndex] - ), + processing_config=LazyProcessingConfig(variable_components=[Microscopy.ZIndex]), napari_streaming_config=LazyNapariStreamingConfig(), fiji_streaming_config=LazyFijiStreamingConfig(), ) diff --git a/openhcs/domains/microscopy/analysis_consolidation.py b/openhcs/domains/microscopy/analysis_consolidation.py index 4d4a5c4e0a..7906e2c9b8 100644 --- a/openhcs/domains/microscopy/analysis_consolidation.py +++ b/openhcs/domains/microscopy/analysis_consolidation.py @@ -78,7 +78,9 @@ def from_saved_outputs( return None backend = plan.runtime_artifact_materialization.require_persistent_backend() return cls._from_table_contents( - config, plan, saved.materialization, + config, + plan, + saved.materialization, ( (Path(output.path), output.require_text_content()) for output in saved.outputs_for_backend(backend) @@ -103,7 +105,9 @@ def from_reused_outputs( return None backend = plan.runtime_artifact_materialization.require_persistent_backend() return cls._from_table_contents( - config, plan, materialization, + config, + plan, + materialization, ( (Path(output.path), context.filemanager.load_text(output.path, backend)) for output in outputs @@ -130,10 +134,12 @@ def _from_table_contents( return None backend = plan.runtime_artifact_materialization.require_persistent_backend() destination = RuntimeAnalysisSummaryDestination( - backend=backend, images_dir=plan.artifact_images_dir, + backend=backend, + images_dir=plan.artifact_images_dir, ) output_groups: dict[ - tuple[Path, RuntimeAnalysisSummaryDestination], list[RuntimeAnalysisTableOutput] + tuple[Path, RuntimeAnalysisSummaryDestination], + list[RuntimeAnalysisTableOutput], ] = {} for output_path, content in contents: output_groups.setdefault((output_path.parent, destination), []).append( @@ -156,7 +162,8 @@ def combine( ) -> RuntimeAnalysisConsolidationInputs | None: """Combine actual rendered projections without revisiting their payloads.""" output_groups: dict[ - tuple[Path, RuntimeAnalysisSummaryDestination], list[RuntimeAnalysisTableOutput] + tuple[Path, RuntimeAnalysisSummaryDestination], + list[RuntimeAnalysisTableOutput], ] = {} destinations: set[RuntimeAnalysisSummaryDestination] = set() seen_paths: set[tuple[str, Path]] = set() @@ -170,15 +177,17 @@ def combine( if identity in seen_paths: continue seen_paths.add(identity) - output_groups.setdefault((directory, group.destination), []).append(output) + output_groups.setdefault((directory, group.destination), []).append( + output + ) return cls._from_groups(output_groups, destinations) - @classmethod def _from_groups( cls, output_groups: Mapping[ - tuple[Path, RuntimeAnalysisSummaryDestination], list[RuntimeAnalysisTableOutput] + tuple[Path, RuntimeAnalysisSummaryDestination], + list[RuntimeAnalysisTableOutput], ], destinations: set[RuntimeAnalysisSummaryDestination], ) -> RuntimeAnalysisConsolidationInputs | None: @@ -193,7 +202,8 @@ def _from_groups( return cls( groups={ directory: RuntimeAnalysisDirectoryInputs( - outputs=tuple(outputs), destination=destination, + outputs=tuple(outputs), + destination=destination, ) for (directory, destination), outputs in output_groups.items() }, diff --git a/openhcs/domains/microscopy/axes.py b/openhcs/domains/microscopy/axes.py index 1b9c571173..697ef68fa4 100644 --- a/openhcs/domains/microscopy/axes.py +++ b/openhcs/domains/microscopy/axes.py @@ -86,7 +86,9 @@ def grid_coordinates(cls, value: object) -> tuple[str, str]: match = re.match(r"^([A-Za-z]+)([0-9]+)$", str(value)) if match is None: - raise ValueError(f"{value!r} is not a well spelled .") + raise ValueError( + f"{value!r} is not a well spelled ." + ) return match.group(1), match.group(2) @classmethod @@ -94,7 +96,9 @@ def grid_position(cls, row_label: str, column_label: str) -> tuple[int, int]: """Rows count A=1 … Z=26, AA=27; columns are the column number.""" if not row_label.isalpha() or not column_label.isdecimal(): - raise ValueError(f"Not a well row and column: {row_label!r}, {column_label!r}.") + raise ValueError( + f"Not a well row and column: {row_label!r}, {column_label!r}." + ) row = 0 for letter in row_label.upper(): row = row * 26 + (ord(letter) - ord("A") + 1) diff --git a/openhcs/formats/pattern/pattern_discovery.py b/openhcs/formats/pattern/pattern_discovery.py index 1f4884ba8e..419c70d333 100644 --- a/openhcs/formats/pattern/pattern_discovery.py +++ b/openhcs/formats/pattern/pattern_discovery.py @@ -97,7 +97,9 @@ def path_list_from_pattern( logger.debug("Using pattern template: %s", pattern_str) # Parse pattern template to get expected structure - pattern_metadata = self.pattern_cache.metadata_for_filename(self.parser, pattern_str) + pattern_metadata = self.pattern_cache.metadata_for_filename( + self.parser, pattern_str + ) if not pattern_metadata: logger.error("Failed to parse pattern template: %s", pattern_str) return [] @@ -117,7 +119,9 @@ def path_list_from_pattern( continue # Parse the actual filename - file_metadata = self.pattern_cache.metadata_for_filename(self.parser, filename) + file_metadata = self.pattern_cache.metadata_for_filename( + self.parser, filename + ) if not file_metadata: continue @@ -185,7 +189,9 @@ def group_patterns_by_component( # The has_placeholders() check is only relevant when using patterns as concrete filenames # For pattern discovery and grouping, we WANT patterns with placeholders - metadata = self.pattern_cache.metadata_for_filename(self.parser, pattern_str) + metadata = self.pattern_cache.metadata_for_filename( + self.parser, pattern_str + ) if metadata is None or metadata.value_for(component_declaration) is None: raise ValueError( @@ -208,7 +214,9 @@ def subdivide_patterns_by_components( subdivided = defaultdict(list) component_declarations = tuple(components) for pattern in patterns: - metadata = self.pattern_cache.metadata_for_filename(self.parser, str(pattern)) + metadata = self.pattern_cache.metadata_for_filename( + self.parser, str(pattern) + ) if not metadata: raise ValueError(f"Failed to parse pattern: {pattern}") key = tuple( diff --git a/openhcs/interop/cellprofiler/analyst_export.py b/openhcs/interop/cellprofiler/analyst_export.py index e81ec26b33..6c71520d89 100644 --- a/openhcs/interop/cellprofiler/analyst_export.py +++ b/openhcs/interop/cellprofiler/analyst_export.py @@ -737,7 +737,10 @@ def _measurement_subject_rows( scope=scope, table=table, ) - default_subject = table.subject.name or CELLPROFILER_MEASUREMENT_DIALECT.scope_name(table.subject.scope) + default_subject = ( + table.subject.name + or CELLPROFILER_MEASUREMENT_DIALECT.scope_name(table.subject.scope) + ) if ( table.subject.scope is MeasurementScope.RUN and measurement_table_row_layout_from_fields(table.rows.fields) @@ -1177,7 +1180,9 @@ def collect_source_bound_image_provenance( "CPA source-bound thumbnails require a runtime context." ) return - workspace = self.context.runtime_source_workspace_projections.projection_if_available() + workspace = ( + self.context.runtime_source_workspace_projections.projection_if_available() + ) if workspace is None: if settings.write_image_thumbnails and any( binding.alias in settings.thumbnail_image_names @@ -1322,7 +1327,9 @@ def _resolved_source_path(self, source_path: str | None) -> Path | None: raise RuntimeError( "Virtual workspace source resolution requires a runtime FileManager." ) - projection = self.context.runtime_source_workspace_projections.projection_if_available() + projection = ( + self.context.runtime_source_workspace_projections.projection_if_available() + ) if projection is None: return Path(source_path).resolve(strict=False) return Path( @@ -1416,7 +1423,10 @@ def build( ), self.context, ) - image_subject = MeasurementSubject(MeasurementScope.SAMPLE, CELLPROFILER_MEASUREMENT_DIALECT.scope_name(MeasurementScope.SAMPLE)) + image_subject = MeasurementSubject( + MeasurementScope.SAMPLE, + CELLPROFILER_MEASUREMENT_DIALECT.scope_name(MeasurementScope.SAMPLE), + ) image_rows_by_number: dict[ int, dict[str, Any], @@ -1757,7 +1767,12 @@ def project_measurement_partition( table_name=projection.dialect.image_table_name(), rows=tuple(image_rows.values()), columns=image_columns, - subject=MeasurementSubject(MeasurementScope.SAMPLE, CELLPROFILER_MEASUREMENT_DIALECT.scope_name(MeasurementScope.SAMPLE)), + subject=MeasurementSubject( + MeasurementScope.SAMPLE, + CELLPROFILER_MEASUREMENT_DIALECT.scope_name( + MeasurementScope.SAMPLE + ), + ), ), experiment_table=CellProfilerProjectedTable( table_name=projection.dialect.object_table_name("Experiment"), diff --git a/openhcs/interop/cellprofiler/database_column_dialect.py b/openhcs/interop/cellprofiler/database_column_dialect.py index 774ea969cb..5e17c1c3f6 100644 --- a/openhcs/interop/cellprofiler/database_column_dialect.py +++ b/openhcs/interop/cellprofiler/database_column_dialect.py @@ -513,7 +513,9 @@ def image_subject( ) -> MeasurementSubject: """Invert an exact CPA image-table declaration into its subject.""" - subject = MeasurementSubject(MeasurementScope.SAMPLE, CELLPROFILER_SCOPE_NAMES[MeasurementScope.SAMPLE]) + subject = MeasurementSubject( + MeasurementScope.SAMPLE, CELLPROFILER_SCOPE_NAMES[MeasurementScope.SAMPLE] + ) expected_table = self.image_table_name() expected_image_id = self.image_id_field().name if ( diff --git a/openhcs/interop/cellprofiler/image_set_numbering.py b/openhcs/interop/cellprofiler/image_set_numbering.py index dd78bcb137..4f4a2aa185 100644 --- a/openhcs/interop/cellprofiler/image_set_numbering.py +++ b/openhcs/interop/cellprofiler/image_set_numbering.py @@ -270,8 +270,12 @@ def _project_reference_columns( reference_indices.append(index) reference_numbers: dict[int, int] = {} for name in table.rows.columns: - wide_reference = CellProfilerMeasurementDialect.row_identity_contract.is_sample_number_reference(name) - if CellProfilerMeasurementDialect.row_identity_contract.is_aggregate_sample_number_reference(name): + wide_reference = CellProfilerMeasurementDialect.row_identity_contract.is_sample_number_reference( + name + ) + if CellProfilerMeasurementDialect.row_identity_contract.is_aggregate_sample_number_reference( + name + ): continue if not wide_reference and ( name not in value_fields or not reference_indices @@ -323,8 +327,8 @@ def source_slices_for_measurement_table( ) if not projection.has_axisless_rows(MeasurementRowAxisField.SLICE_INDEX): return indices - source_indices = ( - tuple(range(table.source_provenance.source_plane_count)) or (0,) + source_indices = tuple(range(table.source_provenance.source_plane_count)) or ( + 0, ) return tuple(dict.fromkeys((*indices, *source_indices))) diff --git a/openhcs/interop/cellprofiler/measurement_dialect.py b/openhcs/interop/cellprofiler/measurement_dialect.py index d5ade66b94..04fe9059a5 100644 --- a/openhcs/interop/cellprofiler/measurement_dialect.py +++ b/openhcs/interop/cellprofiler/measurement_dialect.py @@ -223,7 +223,9 @@ def parent_reference_feature_name(self, parent_object_name: str) -> str: ) def parent_reference_object_name(self, feature_name: str) -> str | None: - identity = DirectParentReferenceFeatureDeclaration.from_feature_name(feature_name) + identity = DirectParentReferenceFeatureDeclaration.from_feature_name( + feature_name + ) return None if identity is None else identity.parent_object_name def child_count_feature_name(self, child_object_name: str) -> str: diff --git a/openhcs/interop/cellprofiler/measurement_lookup.py b/openhcs/interop/cellprofiler/measurement_lookup.py index d7eb4f6a63..ffdea014ea 100644 --- a/openhcs/interop/cellprofiler/measurement_lookup.py +++ b/openhcs/interop/cellprofiler/measurement_lookup.py @@ -81,7 +81,6 @@ def feature_name(cls, identity: str) -> str: return f"{cls.prefix}{name}{cls.suffix}" - class CellProfilerMeasurementFeatureKind(Enum): """CellProfiler measurement feature families with structured semantics.""" diff --git a/openhcs/interop/cellprofiler/measurement_scope.py b/openhcs/interop/cellprofiler/measurement_scope.py index 851b5456c7..f0ce9f25c9 100644 --- a/openhcs/interop/cellprofiler/measurement_scope.py +++ b/openhcs/interop/cellprofiler/measurement_scope.py @@ -10,7 +10,6 @@ MeasurementScopeSelection, ) - CELLPROFILER_SCOPE_NAMES = MappingProxyType( { MeasurementScope.SAMPLE: "Image", diff --git a/openhcs/interop/cellprofiler/module_artifact_contracts.py b/openhcs/interop/cellprofiler/module_artifact_contracts.py index 804bfe194c..53879158dd 100644 --- a/openhcs/interop/cellprofiler/module_artifact_contracts.py +++ b/openhcs/interop/cellprofiler/module_artifact_contracts.py @@ -222,7 +222,6 @@ def _artifact_context_for_group( ): raise ValueError( f"No source binding declares {grouped_component.name} " - f"group {group_key!r}." ) source_bindings = replace( diff --git a/openhcs/interop/cellprofiler/module_artifact_declarations.py b/openhcs/interop/cellprofiler/module_artifact_declarations.py index 920638029f..8f60f5cfe0 100644 --- a/openhcs/interop/cellprofiler/module_artifact_declarations.py +++ b/openhcs/interop/cellprofiler/module_artifact_declarations.py @@ -653,12 +653,15 @@ def prior_measurement_artifact_inputs( ) group_lineage_refs = source_refs or object_refs for producer in feature_matches: - object_subjects = tuple(dict.fromkeys( - InputObjectMeasurementSourceRelation(relation.source) - for relation in producer.spec.relations - if relation.source.artifact_type is ObjectLabelsArtifactType - and relation.source.for_plan_type(ArtifactInputPlan) in object_refs - )) + object_subjects = tuple( + dict.fromkeys( + InputObjectMeasurementSourceRelation(relation.source) + for relation in producer.spec.relations + if relation.source.artifact_type is ObjectLabelsArtifactType + and relation.source.for_plan_type(ArtifactInputPlan) + in object_refs + ) + ) if object_subjects: measurement_input = object_subjects[0].input_spec_for_output( producer.spec @@ -1232,9 +1235,11 @@ def finalize_artifact_contract_inputs( for spec in inputs.of_artifact_type(ImageArtifactType) ) return tuple( - replace(spec, relations=(*spec.relations, *image_contexts)) - if spec.artifact_type is ObjectLabelsArtifactType and image_contexts - else spec + ( + replace(spec, relations=(*spec.relations, *image_contexts)) + if spec.artifact_type is ObjectLabelsArtifactType and image_contexts + else spec + ) for spec in inputs ) @@ -1252,8 +1257,11 @@ def input_source_for_contract( ) if any( spec.ref() not in produced_refs - and step_context.source_bindings.binding_for_artifact_ref(spec.ref()) is not None - for spec in callable_contract.artifact_inputs.of_artifact_type(ImageArtifactType) + and step_context.source_bindings.binding_for_artifact_ref(spec.ref()) + is not None + for spec in callable_contract.artifact_inputs.of_artifact_type( + ImageArtifactType + ) ): return InputSource.PIPELINE_START return super().input_source_for_contract( @@ -1372,7 +1380,9 @@ def finalize_module_blocks_for_invocation( """Require selection intent before reconstructing multiple object subjects.""" blocks = super().finalize_module_blocks_for_invocation( - blocks, invocation=invocation, step_context=step_context, + blocks, + invocation=invocation, + step_context=step_context, ) binding = cls.object_measurement_binding selector = binding.require_parameter_name() @@ -1390,7 +1400,8 @@ def processing_group_scope_inputs( ) -> tuple[ArtifactSpec, ...]: """Keep all declared measurement subjects in one logical module batch.""" source_refs = frozenset( - spec.ref() for spec in ( + spec.ref() + for spec in ( *super().processing_group_scope_inputs(callable_contract), *callable_contract.group_scope_inputs, ) diff --git a/openhcs/interop/cellprofiler/module_declarations.py b/openhcs/interop/cellprofiler/module_declarations.py index 4699376d3e..ffa8eb176b 100644 --- a/openhcs/interop/cellprofiler/module_declarations.py +++ b/openhcs/interop/cellprofiler/module_declarations.py @@ -68,7 +68,6 @@ Ungrouped, ) - _CELLPROFILER_BACKEND_PACKAGE = "openhcs.processing.backends.cellprofiler" _CELLPROFILER_MODULE_REGISTRY = LazyDiscoveryDict(enable_cache=False) if TYPE_CHECKING: @@ -438,8 +437,12 @@ def source_bindings_for_modules( ) modules = tuple(modules) cls.discover_source_declarations( - frozenset(_module_lookup_key(module.name) for module in modules if module.enabled), - ("module_name", "aliases"), _module_lookup_key, _declared_lookup_keys, + frozenset( + _module_lookup_key(module.name) for module in modules if module.enabled + ), + ("module_name", "aliases"), + _module_lookup_key, + _declared_lookup_keys, ) for module in modules: if not module.enabled: @@ -594,7 +597,8 @@ def require_callable( implementation = cls._require_declared_callable(function_name) cls.prepare_declared_callable(implementation) cls._validate_callable_parameter_exclusions( - implementation, CallableContract.from_callable(implementation), + implementation, + CallableContract.from_callable(implementation), ) return implementation @@ -657,7 +661,9 @@ def for_module(cls, module_name: str) -> type["CellProfilerModule"] | None: """Return the registered module class for a canonical name or alias.""" lookup_key = _module_lookup_key(module_name) matches = cls.discover_source_declarations( - frozenset((lookup_key,)), ("module_name", "aliases"), _module_lookup_key, + frozenset((lookup_key,)), + ("module_name", "aliases"), + _module_lookup_key, _declared_lookup_keys, ) return matches[0] if matches else None @@ -949,7 +955,8 @@ def input_source_for_contract( ): return InputSource.PREVIOUS_STEP if any( - step_context.source_bindings.binding_for_artifact_ref(spec.ref()) is not None + step_context.source_bindings.binding_for_artifact_ref(spec.ref()) + is not None for spec in callable_contract.artifact_inputs ): return InputSource.PIPELINE_START @@ -998,7 +1005,6 @@ def processing_config( if grouping_axes and step_context is not None: (grouping_component,) = grouping_axes source_anchor_group_keys = step_context.source_bindings.component_group_keys_for_artifact_specs( - grouping_component, cls.processing_group_scope_inputs(callable_contract), step_context.available_artifacts, diff --git a/openhcs/interop/cellprofiler/pipeline_import.py b/openhcs/interop/cellprofiler/pipeline_import.py index 5d30c844d9..340b63ba6e 100644 --- a/openhcs/interop/cellprofiler/pipeline_import.py +++ b/openhcs/interop/cellprofiler/pipeline_import.py @@ -1061,9 +1061,7 @@ def projected_leaf( if group_key in grouped_items and not preserves_input_main_flow: return None grouped_items.setdefault(group_key, []).append(leaf) - target_units_by_group_key.setdefault(group_key, []).append( - unit - ) + target_units_by_group_key.setdefault(group_key, []).append(unit) public_pattern = { group_key: items[0] if len(items) == 1 else items for group_key, items in grouped_items.items() @@ -1104,7 +1102,8 @@ def target_units_for_invocation( return None invocation_target_contract = ( invocation_target_units[0].contract - if len(invocation_target_units) == 1 else target_contract + if len(invocation_target_units) == 1 + else target_contract ) # Projection may omit a selector only in its original context. # Reuse the binding-owned occurrence relation after earlier calls @@ -1114,7 +1113,8 @@ def target_units_for_invocation( module_type, invocation_target_units, candidate_group_keys=( - () if invocation.key.group_key == DEFAULT_GROUP_KEY + () + if invocation.key.group_key == DEFAULT_GROUP_KEY else (invocation.key.group_key,) ), step_context=group_context, diff --git a/openhcs/interop/cellprofiler/runtime/function_contract_execution.py b/openhcs/interop/cellprofiler/runtime/function_contract_execution.py index f1a405b973..1890135ef5 100644 --- a/openhcs/interop/cellprofiler/runtime/function_contract_execution.py +++ b/openhcs/interop/cellprofiler/runtime/function_contract_execution.py @@ -129,7 +129,9 @@ def validate_full_stack_kwargs(self, contract, kwargs): def stack_reduction_output(self, result): result_2d = ImagePayload.of(result) stacked = stack_slices( - [result_2d.data], detect_memory_type(result_2d.data), 0, + [result_2d.data], + detect_memory_type(result_2d.data), + 0, ) return result_2d.metadata.payload_with(stacked, mask=result_2d.mask) @@ -290,9 +292,7 @@ def execute( plane_projection, ) execute_started_at = time.perf_counter() - with executing_measurement_dialect( - CELLPROFILER_MEASUREMENT_DIALECT - ): + with executing_measurement_dialect(CELLPROFILER_MEASUREMENT_DIALECT): result = execution_mode.execute( processing_contract, call, image, dict(kwargs) ) diff --git a/openhcs/interop/cellprofiler/runtime/invocation.py b/openhcs/interop/cellprofiler/runtime/invocation.py index 25d0030abf..3542c72b6d 100644 --- a/openhcs/interop/cellprofiler/runtime/invocation.py +++ b/openhcs/interop/cellprofiler/runtime/invocation.py @@ -9,7 +9,7 @@ import numpy as np from openhcs.core.alias_property import AliasProperty -from openhcs.core.aligned_image_payload import (AlignedImageStack, ImageOutputBundle) +from openhcs.core.aligned_image_payload import AlignedImageStack, ImageOutputBundle from openhcs.core.equivalence.keys import RuntimeMeasurementSourcePair from openhcs.core.measurement_image_alignment import ( MeasurementImageAlignmentSource, @@ -199,7 +199,9 @@ def aligned_source_metadata( """ if not sources: raise ValueError("Aligned measurement metadata requires source images.") - source_axes = tuple(cls.source_provenances(source.payload) for source in sources) + source_axes = tuple( + cls.source_provenances(source.payload) for source in sources + ) planes = tuple( SourceImageProvenance.bundle( tuple(source.with_runtime_planes_as_contributors() for source in plane) @@ -228,7 +230,9 @@ def source_provenances( cls.scalar_source_provenance(value) for value in payload.slices ) if isinstance(payload, RuntimeSliceAlignedValueSet): - return tuple(cls.scalar_source_provenance(value) for value in payload.values) + return tuple( + cls.scalar_source_provenance(value) for value in payload.values + ) count = RuntimeSliceProjection.slice_count_from_values((payload,)) metadata = image_metadata_of(payload) provenance = metadata.source_provenance @@ -307,7 +311,9 @@ def __post_init__(self) -> None: if self.primary_input_payloads and ( len(self.primary_input_payloads) != len(self.primary_input_edges) ): - raise ValueError("Primary input contexts must preserve exact binding order.") + raise ValueError( + "Primary input contexts must preserve exact binding order." + ) def consumed_input_value( self, diff --git a/openhcs/interop/cellprofiler/runtime/measurement_recording.py b/openhcs/interop/cellprofiler/runtime/measurement_recording.py index 0af88eb3d8..866a703a93 100644 --- a/openhcs/interop/cellprofiler/runtime/measurement_recording.py +++ b/openhcs/interop/cellprofiler/runtime/measurement_recording.py @@ -149,7 +149,8 @@ def build_measurement_table( if object_name is not None else MeasurementSubject( MeasurementScope.SAMPLE, - source_image_name or CELLPROFILER_SCOPE_NAMES[MeasurementScope.SAMPLE].casefold(), + source_image_name + or CELLPROFILER_SCOPE_NAMES[MeasurementScope.SAMPLE].casefold(), ) ) return MeasurementTable( @@ -712,9 +713,11 @@ def measurement_record_source_metadata( ) -> ImagePayloadMetadata: del rows source_spec = cls.primary_image_output_spec(request) - return (request.source.payload + return ( + request.source.payload if source_spec is None - else request.artifact_output_value(source_spec)).metadata + else request.artifact_output_value(source_spec) + ).metadata class ProducedImagePayloadMeasurementRecordMixin( @@ -759,9 +762,7 @@ def measurement_record_source_metadata( request.adapter.request.source_binding_plan ) identity_axes = tuple( - payload.metadata.source_provenance.image_set_axis( - identity_policy - ) + payload.metadata.source_provenance.image_set_axis(identity_policy) for payload in source_payloads ) missing_identity_specs = tuple( @@ -786,9 +787,7 @@ def measurement_record_source_metadata( ) source_metadata = ImagePayloadMetadata.compose( source_payloads, - source_metadata=tuple( - payload.metadata for payload in source_payloads - ), + source_metadata=tuple(payload.metadata for payload in source_payloads), ).collapse_leading_plane_axis() return source_metadata diff --git a/openhcs/interop/cellprofiler/runtime/module_execution.py b/openhcs/interop/cellprofiler/runtime/module_execution.py index d7fa4dae61..f23a8b9e03 100644 --- a/openhcs/interop/cellprofiler/runtime/module_execution.py +++ b/openhcs/interop/cellprofiler/runtime/module_execution.py @@ -11,7 +11,11 @@ ) from typing import cast -from openhcs.core.aligned_image_payload import (AlignedImageSliceContext, AlignedImageStack, ImageOutputBundle) +from openhcs.core.aligned_image_payload import ( + AlignedImageSliceContext, + AlignedImageStack, + ImageOutputBundle, +) from openhcs.core.image_payload_execution_mode import ImagePayloadExecutionMode from openhcs.core.artifacts import ( ArtifactOutputPlan, @@ -1289,8 +1293,7 @@ def _image_request( if ( runtime_projection is not None and runtime_projection.plane_index is not None - and current_image_payload.metadata.plane_axis - is runtime_projection.axis + and current_image_payload.metadata.plane_axis is runtime_projection.axis ): current_image_payload = cast( RuntimeCallableArgument, @@ -1319,9 +1322,7 @@ def _image_request( image_input_specs = ArtifactSpecCollection(image_inputs) payloads = [] source_names: list[str | None] = [] - image_strategy = CellProfilerOutputRecorder.for_artifact_type( - ImageArtifactType - ) + image_strategy = CellProfilerOutputRecorder.for_artifact_type(ImageArtifactType) input_binding = replace(input_binding, current_image=current_runtime_payload) primary_input_edges = [] for spec in image_inputs: @@ -1330,20 +1331,23 @@ def _image_request( if ( scope is not None and edge.storage_plan is None - and adapter.request.source_binding_plan.declares_artifact_ref(spec.ref()) + and adapter.request.source_binding_plan.declares_artifact_ref( + spec.ref() + ) ): value = edge.resolve_unstored_payload( - scope, current_runtime_payload, - request=replace(adapter.request, source_payload=current_runtime_payload), + scope, + current_runtime_payload, + request=replace( + adapter.request, source_payload=current_runtime_payload + ), ) else: value = input_binding.artifact_value(edge) primary_input_edges.append(edge) payload = image_strategy.runtime_input_value(spec, value) payloads.append(payload) - source_names.append( - image_strategy.source_image_name_from_value(payload) - ) + source_names.append(image_strategy.source_image_name_from_value(payload)) parameter_image_inputs = input_binding.image_inputs broadcast_sources = tuple( sources[0] @@ -1354,24 +1358,26 @@ def _image_request( align_primary_images = len(image_inputs) > 1 and broadcast_sources == tuple( spec.ref() for spec in image_inputs ) - composition = self.callable_contract.image_payload_consumption.compose_image_payload( - f"{module_type.require_module_name()} image inputs " - f"{tuple(spec.name for spec in image_inputs)!r}", - tuple(payloads), - slice_contexts=( - tuple( - AlignedImageSliceContext.main_flow( - output_key=spec.name, - artifact_kind=spec.artifact_type.value, + composition = ( + self.callable_contract.image_payload_consumption.compose_image_payload( + f"{module_type.require_module_name()} image inputs " + f"{tuple(spec.name for spec in image_inputs)!r}", + tuple(payloads), + slice_contexts=( + tuple( + AlignedImageSliceContext.main_flow( + output_key=spec.name, + artifact_kind=spec.artifact_type.value, + ) + for spec in image_inputs ) - for spec in image_inputs - ) - if align_primary_images - else () - ), - stack_broadcast_source_indices=( - image_input_specs.stack_broadcast_source_indices() - ), + if align_primary_images + else () + ), + stack_broadcast_source_indices=( + image_input_specs.stack_broadcast_source_indices() + ), + ) ) return CellProfilerImageRequest( input_binding_request=binding_request, @@ -1520,7 +1526,8 @@ def _invocation_request( invocation_kwargs = object_label_input_execution_mode_from_callable( self.raw_func ).invocation_kwargs( - runtime_kwargs, execution_mode=execution_mode, + runtime_kwargs, + execution_mode=execution_mode, image_projection=image_request.plane_projection, runtime_projection=runtime_projection, semantic_controls=_execution_mode_semantic_control_kwargs( diff --git a/openhcs/interop/cellprofiler/workspace_export.py b/openhcs/interop/cellprofiler/workspace_export.py index b1976fe70e..08911537bc 100644 --- a/openhcs/interop/cellprofiler/workspace_export.py +++ b/openhcs/interop/cellprofiler/workspace_export.py @@ -70,7 +70,10 @@ class CPAImageWorkspaceAxis(CPAWorkspaceAxis): def field_name(self, dialect): return dialect.measurement_field( - MeasurementSubject(MeasurementScope.SAMPLE, CELLPROFILER_SCOPE_NAMES[MeasurementScope.SAMPLE]), + MeasurementSubject( + MeasurementScope.SAMPLE, + CELLPROFILER_SCOPE_NAMES[MeasurementScope.SAMPLE], + ), FieldSpec(self.measurement, None), ).name diff --git a/openhcs/mcp/context.py b/openhcs/mcp/context.py index c47a6938aa..a7f442ffac 100644 --- a/openhcs/mcp/context.py +++ b/openhcs/mcp/context.py @@ -144,7 +144,8 @@ def endpoint_function_catalog(self) -> "EndpointFunctionCatalogServiceABC": endpoint = load_cached_ui_execution_endpoint_sync() self._endpoint_function_catalog = ZMQFunctionCatalogService( - lambda: endpoint, path_policy=self.path_policy, + lambda: endpoint, + path_policy=self.path_policy, ) return self._endpoint_function_catalog @@ -281,7 +282,9 @@ def runtime_server_service(self) -> "RuntimeServerService": RuntimeServerService, ) - self._runtime_server_service = RuntimeServerService(path_policy=self.path_policy) + self._runtime_server_service = RuntimeServerService( + path_policy=self.path_policy + ) return self._runtime_server_service @property @@ -339,7 +342,8 @@ def create_agent_context() -> OpenHCSAgentContext: endpoint_config = load_cached_ui_execution_endpoint_sync() path_policy = AgentPathPolicy.from_environment() catalog = ZMQFunctionCatalogService( - lambda: endpoint_config, path_policy=path_policy, + lambda: endpoint_config, + path_policy=path_policy, ) return OpenHCSAgentContext( path_policy=path_policy, diff --git a/openhcs/mcp/dev_client_commands/ui.py b/openhcs/mcp/dev_client_commands/ui.py index 368505fb91..ab394934b2 100644 --- a/openhcs/mcp/dev_client_commands/ui.py +++ b/openhcs/mcp/dev_client_commands/ui.py @@ -369,7 +369,9 @@ async def _poll_workflow_state( poll_deadline = loop.time() + args.poll_timeout_seconds async def await_events(after_sequence: int) -> int: - connection_seconds = (args.timeout_ms or DEFAULT_UI_BRIDGE_TIMEOUT_MS) / 1000 + connection_seconds = ( + args.timeout_ms or DEFAULT_UI_BRIDGE_TIMEOUT_MS + ) / 1000 wait_seconds = min( max(poll_deadline - loop.time(), 0.0), 0.8 * connection_seconds ) @@ -462,7 +464,9 @@ def render_compact(self, response, args: argparse.Namespace) -> str: lines.append(f"Targets: {', '.join(summary.target_scope_ids)}") if summary.skip_reason is not None: lines.append(f"Skip reason: {summary.skip_reason.value}") - lines.extend(McpDevOutputRenderer.diagnostic_lines(response.diagnostic_errors())) + lines.extend( + McpDevOutputRenderer.diagnostic_lines(response.diagnostic_errors()) + ) final_rows = self._final_state_rows(response) if final_rows: lines.append("Rows:") @@ -538,7 +542,9 @@ def calls_from_args( ) -class ValidateCodeDocumentCommandSpec(StdinSourceCommandSpec, CapabilityBackedCommandSpec): +class ValidateCodeDocumentCommandSpec( + StdinSourceCommandSpec, CapabilityBackedCommandSpec +): capability = agent_capabilities.ui_validate_code_document def configure_parser(self, parser: argparse.ArgumentParser) -> None: diff --git a/openhcs/mcp/dev_client_core.py b/openhcs/mcp/dev_client_core.py index e975db7572..8d85248149 100644 --- a/openhcs/mcp/dev_client_core.py +++ b/openhcs/mcp/dev_client_core.py @@ -16,7 +16,16 @@ from enum import Enum from functools import singledispatch from pathlib import Path -from typing import TYPE_CHECKING, ClassVar, Self, TextIO, TypeVar, cast, get_args, get_type_hints +from typing import ( + TYPE_CHECKING, + ClassVar, + Self, + TextIO, + TypeVar, + cast, + get_args, + get_type_hints, +) from metaclass_registry import AutoRegisterMeta from python_introspect import ( @@ -1156,7 +1165,9 @@ def _agent_errors(value: object) -> tuple[AgentError, ...]: return tuple( error for declaration in fields(value) - if not isinstance(field_value := getattr(value, declaration.name), AgentError) + if not isinstance( + field_value := getattr(value, declaration.name), AgentError + ) for error in _agent_errors(field_value) ) return () diff --git a/openhcs/mcp/dev_client_renderers/pipeline.py b/openhcs/mcp/dev_client_renderers/pipeline.py index cb3f07ba00..9b6b33ea92 100644 --- a/openhcs/mcp/dev_client_renderers/pipeline.py +++ b/openhcs/mcp/dev_client_renderers/pipeline.py @@ -132,9 +132,7 @@ def render(cls, response, *, max_source_chars: int = 2_000) -> str: lines.append(f"Ref: uri={created.uri}") if validated is not None and validated.errors: lines.append("Validate errors:") - lines.extend( - diagnostic_lines(decoded.diagnostic_errors()) - ) + lines.extend(diagnostic_lines(decoded.diagnostic_errors())) if validated is not None: if validated.warnings: lines.append("Validate warnings:") @@ -321,9 +319,7 @@ def _materialization_lines( mode = ( "disabled" if materialization.disabled - else "runtime-resolved" - if materialization.runtime_resolved - else "explicit" + else "runtime-resolved" if materialization.runtime_resolved else "explicit" ) parts = [mode, f"persistent={materialization.persistent_enabled}"] parts.extend( diff --git a/openhcs/mcp/dev_client_renderers/plate.py b/openhcs/mcp/dev_client_renderers/plate.py index e4a05b8010..de82572746 100644 --- a/openhcs/mcp/dev_client_renderers/plate.py +++ b/openhcs/mcp/dev_client_renderers/plate.py @@ -188,7 +188,9 @@ def _roi_area_text(preview: PlateInspectionResultFilePreview) -> str: ): return "" mean = ( - "" if preview.roi_area_mean is None else f"{preview.roi_area_mean:.3f}" + "" + if preview.roi_area_mean is None + else f"{preview.roi_area_mean:.3f}" ) return ( f"min={McpDevOutputRenderer.text(preview.roi_area_min)}," @@ -208,7 +210,9 @@ def _csv_lines(cls, preview: PlateInspectionResultFilePreview) -> list[str]: return [] lines = [f" csv columns: {cls._csv_columns_text(columns)}"] visible_rows = rows[: cls.MAX_CSV_PREVIEW_ROWS] - lines.extend(f" csv row: {cls._csv_row_text(columns, row)}" for row in visible_rows) + lines.extend( + f" csv row: {cls._csv_row_text(columns, row)}" for row in visible_rows + ) omitted_row_count = len(rows) - len(visible_rows) if omitted_row_count > 0: lines.append( @@ -588,9 +592,11 @@ def _axis_summary_line( ( f"unknown-{axis.name}" if axis.name not in counts - else f"multi-{axis.name}" - if counts[axis.name] > 1 - else f"single-{axis.name}" + else ( + f"multi-{axis.name}" + if counts[axis.name] > 1 + else f"single-{axis.name}" + ) ) for axis in AxisFamily.active().variable_axes() ) diff --git a/openhcs/mcp/dev_client_renderers/ui_bridge.py b/openhcs/mcp/dev_client_renderers/ui_bridge.py index 782e3b6994..fead3eb5fa 100644 --- a/openhcs/mcp/dev_client_renderers/ui_bridge.py +++ b/openhcs/mcp/dev_client_renderers/ui_bridge.py @@ -157,20 +157,28 @@ def render(cls, response) -> str: return "\n".join( ( f"UI smoke: results={len(decoded.results)} mcp_errors={mcp_errors}", - "Health: missing" - if health is None - else f"Health: status={health.status} " - f"restart_required={health.restart_required} " - f"stale_paths={len(health.stale_source_paths)}", - UiBridgeStatusRenderer.unavailable_summary - if status is None - else McpDevOutputRenderer.render_payload_value(status, options), - "Bridges: missing" - if bridges is None - else f"Bridges: live={len(bridges.bridges)} errors={len(bridges.errors)}", - UiWindowCatalogRenderer.unavailable_summary - if windows is None - else McpDevOutputRenderer.render_payload_value(windows, options), + ( + "Health: missing" + if health is None + else f"Health: status={health.status} " + f"restart_required={health.restart_required} " + f"stale_paths={len(health.stale_source_paths)}" + ), + ( + UiBridgeStatusRenderer.unavailable_summary + if status is None + else McpDevOutputRenderer.render_payload_value(status, options) + ), + ( + "Bridges: missing" + if bridges is None + else f"Bridges: live={len(bridges.bridges)} errors={len(bridges.errors)}" + ), + ( + UiWindowCatalogRenderer.unavailable_summary + if windows is None + else McpDevOutputRenderer.render_payload_value(windows, options) + ), *McpDevOutputRenderer.diagnostic_lines(decoded.diagnostic_errors()), ) ) @@ -220,7 +228,9 @@ class UiStateSurfaceStateRenderer(McpDevOutputRenderer): surface_identity: ClassVar[type[UiStateSurfaceIdentityDeclarationBase]] @classmethod - def for_surface_id(cls, surface_id: str) -> type["UiStateSurfaceStateRenderer"] | None: + def for_surface_id( + cls, surface_id: str + ) -> type["UiStateSurfaceStateRenderer"] | None: return next( ( renderer @@ -308,7 +318,9 @@ def _section_lines(cls, section: UiLiveOverviewSection) -> list[str]: ("surface", item.source_surface_id), ("window", item.source_window_id), ): - parts.extend(cls.optional_lines(value, lambda text: (f"{label}={text}",))) + parts.extend( + cls.optional_lines(value, lambda text: (f"{label}={text}",)) + ) lines.append(f"- {item.label}: " + " ".join(parts)) return lines @@ -353,7 +365,9 @@ def row_lines(cls, rows: tuple[DatasetRowState, ...]) -> list[str]: ("output", row.output_root), ("source", row.source_root), ): - parts.extend(cls.optional_lines(value, lambda text: (f"{label}={text}",))) + parts.extend( + cls.optional_lines(value, lambda text: (f"{label}={text}",)) + ) lines.append(f"- {row.name}: " + ", ".join(parts)) return lines @@ -484,7 +498,9 @@ class WidgetTreeOutlineRenderer(McpDevOutputRenderer): output_contract = UiWidgetTreeResult render_options_type = WidgetTreeRenderOptions MAX_LABEL_CHARS = 96 - TECHNICAL_WIDGET_CLASSES = frozenset({"QHeaderView", "QScrollBar", "QSplitterHandle"}) + TECHNICAL_WIDGET_CLASSES = frozenset( + {"QHeaderView", "QScrollBar", "QSplitterHandle"} + ) TECHNICAL_OBJECT_NAME_PREFIX = "qt_scrollarea_" @classmethod @@ -575,9 +591,7 @@ def _node_label( parts = [part for part in (node.class_name,) if part] if node.object_name: parts.append(f"#{node.object_name}") - addressable = node.actionable or ( - node.visible and any(node.action_kinds) - ) + addressable = node.actionable or (node.visible and any(node.action_kinds)) path_id = node.path_id if addressable else "" action = actions[path_id] if path_id in actions else None semantic_parts = [] if action is None else cls._semantic_parts(action) diff --git a/openhcs/mcp/dev_client_session.py b/openhcs/mcp/dev_client_session.py index 2fae6f25bf..3417f13417 100644 --- a/openhcs/mcp/dev_client_session.py +++ b/openhcs/mcp/dev_client_session.py @@ -90,7 +90,9 @@ class SessionJourney: timeout_seconds: float results: list[McpDevToolResult] - async def call(self, capability, arguments: dict[str, JsonValue]) -> McpDevToolResult: + async def call( + self, capability, arguments: dict[str, JsonValue] + ) -> McpDevToolResult: result = await call_mcp_tool( self.session, McpDevToolCall(capability.name, McpToolArguments.from_payload(arguments)), @@ -163,7 +165,7 @@ async def run_source( added = await self.call(agent_capabilities.add_datasets, {"roots": [root]}) if _rejected(added): return - (scope_id, *_rest) = added.first_decoded_payload().target_scope_ids + scope_id, *_rest = added.first_decoded_payload().target_scope_ids piped = await self.call( agent_capabilities.set_dataset_pipeline, {"scope_id": scope_id, "pipeline_source": pipeline_source}, diff --git a/openhcs/mcp/execution.py b/openhcs/mcp/execution.py index aff5e5d721..a90b3a6428 100644 --- a/openhcs/mcp/execution.py +++ b/openhcs/mcp/execution.py @@ -85,7 +85,9 @@ class McpTransportExecutor(AsyncOperationExecutor): def __init__(self) -> None: if threading.current_thread() is not threading.main_thread(): - raise RuntimeError("MCP transport must be started on the process main thread.") + raise RuntimeError( + "MCP transport must be started on the process main thread." + ) existing_application = QCoreApplication.instance() # A process holds one Qt application; one this transport created is # its own and goes with it, so a later GUI can create a QApplication. @@ -124,7 +126,9 @@ def close(self) -> None: leaving it to interpreter finalization is not an MCP lifecycle. """ if threading.current_thread() is not threading.main_thread(): - raise RuntimeError("MCP transport must be closed on the process main thread.") + raise RuntimeError( + "MCP transport must be closed on the process main thread." + ) if self._closed: return try: diff --git a/openhcs/mcp/server.py b/openhcs/mcp/server.py index 36c3850441..f93f4e8473 100644 --- a/openhcs/mcp/server.py +++ b/openhcs/mcp/server.py @@ -434,7 +434,9 @@ async def _report_progress_if_available( @contextmanager -def _verbose_blocking_operation_diagnostics(capability: type[AgentCapabilityDeclaration]): +def _verbose_blocking_operation_diagnostics( + capability: type[AgentCapabilityDeclaration], +): """Dump a blocked invocation without affecting normal MCP output.""" heartbeat_seconds = capability.progress_heartbeat_seconds @@ -475,7 +477,9 @@ def _mcp_server_health() -> McpServerHealthResult: server_import_mtime_ns=MCP_SERVER_IMPORT_MTIME_NS, server_current_mtime_ns=current_source_mtime_ns, server_source_changed_since_import=bool(stale_source_paths), - stale_source_paths=tuple(str(source_path) for source_path in stale_source_paths), + stale_source_paths=tuple( + str(source_path) for source_path in stale_source_paths + ), recovery_reason=recovery.reason.value, installation_pointer_path=recovery.installation_pointer_path, installation_pointer_changed_since_import=( @@ -613,14 +617,18 @@ def ui_connection_parameter() -> Parameter: annotation=McpUiBridgeConnectionRequest | None, ) - def ui_connection(self, arguments: Mapping[str, JsonValue]) -> UiBridgeConnectionSpec: + def ui_connection( + self, arguments: Mapping[str, JsonValue] + ) -> UiBridgeConnectionSpec: return UiBridgeConnectionToolArgs.from_request(arguments["connection"]).resolve( self.context ) @staticmethod def viewer_connection_parameters() -> tuple[Parameter, ...]: - return McpViewerConnectionToolFields.signature_parameters(McpViewerTimeoutPolicy) + return McpViewerConnectionToolFields.signature_parameters( + McpViewerTimeoutPolicy + ) @staticmethod def viewer_control( diff --git a/openhcs/microscopes/bioformats.py b/openhcs/microscopes/bioformats.py index 067ac01a23..c2628a5f43 100644 --- a/openhcs/microscopes/bioformats.py +++ b/openhcs/microscopes/bioformats.py @@ -134,7 +134,9 @@ def get_metadata_pixel_size(self, dataset_root: Union[str, Path]) -> float: """Retain the dataset's numeric view without asserting physical units.""" return self.source_dataset(dataset_root).pixel_size - def source_voxel_spacing(self, dataset_root: Union[str, Path]) -> SourceVoxelSpacing: + def source_voxel_spacing( + self, dataset_root: Union[str, Path] + ) -> SourceVoxelSpacing: return SourceVoxelSpacing.common(self._source_voxel_spacings(dataset_root)) def _source_voxel_spacings( @@ -236,8 +238,6 @@ def __init__( def root_dir(self) -> str: return FIELDS.DEFAULT_SUBDIRECTORY - - @property def compatible_backends(self) -> List[Backend]: return [Backend.BIOFORMATS] diff --git a/openhcs/microscopes/imagexpress.py b/openhcs/microscopes/imagexpress.py index caa6019d30..b74b88999e 100644 --- a/openhcs/microscopes/imagexpress.py +++ b/openhcs/microscopes/imagexpress.py @@ -35,9 +35,7 @@ class ImageXpressTimePointPaths(FilenameParserCapability): _timepoint_folder_pattern = re.compile(r"TimePoint[_-]?(\d+)", re.IGNORECASE) - def image_path_components( - self, path: Path - ) -> tuple[tuple[type[Axis], int], ...]: + def image_path_components(self, path: Path) -> tuple[tuple[type[Axis], int], ...]: return ( *super().image_path_components(path), *self.indexed_folder_components( @@ -53,9 +51,7 @@ class ImageXpressZStepPaths(FilenameParserCapability): _zstep_folder_pattern = re.compile(r"ZStep[_-]?(\d+)", re.IGNORECASE) - def image_path_components( - self, path: Path - ) -> tuple[tuple[type[Axis], int], ...]: + def image_path_components(self, path: Path) -> tuple[tuple[type[Axis], int], ...]: return ( *super().image_path_components(path), *self.indexed_folder_components( @@ -462,8 +458,6 @@ def root_dir(self) -> str: """ return "." - - @property def compatible_backends(self) -> List[Backend]: """ diff --git a/openhcs/microscopes/omero.py b/openhcs/microscopes/omero.py index 7c2ff295d5..5e5ea70516 100644 --- a/openhcs/microscopes/omero.py +++ b/openhcs/microscopes/omero.py @@ -278,7 +278,9 @@ def get_image_files( all_subdirs: Unused for OMERO (no subdirectories), kept for interface compatibility """ plate_id = ( - dataset_root if isinstance(dataset_root, int) else int(Path(dataset_root).name) + dataset_root + if isinstance(dataset_root, int) + else int(Path(dataset_root).name) ) # Get OMERO backend from registry @@ -343,7 +345,6 @@ def construct_filename(self, components: FilenameParseResult) -> str: extension=components.extension, ).filename() - def extract_component_coordinates(self, component_value: str) -> tuple[str, str]: """Extract coordinates from well identifier (e.g., 'A01' → ('A', '01')).""" @@ -357,7 +358,6 @@ class OMEROHandler(RemoteServiceSource, DatasetSource): source_name = "omero" metadata_handler_class = OMEROMetadataHandler - @classmethod def source_selection_guidance(cls) -> str: """Explain when the OMERO virtual source owner should be selected.""" @@ -391,8 +391,6 @@ def root_dir(self) -> str: """ return "Images" - - @property def compatible_backends(self) -> list[Backend]: """OMERO is only compatible with OMERO_LOCAL backend.""" diff --git a/openhcs/microscopes/opera_phenix.py b/openhcs/microscopes/opera_phenix.py index 9d5f30eb30..979e33d15d 100644 --- a/openhcs/microscopes/opera_phenix.py +++ b/openhcs/microscopes/opera_phenix.py @@ -506,8 +506,6 @@ def root_dir(self) -> str: """ return "Images" - - @property def compatible_backends(self) -> List[Backend]: """ diff --git a/openhcs/microscopes/vendor_layout.py b/openhcs/microscopes/vendor_layout.py index 6b7544afa2..c802a0ed20 100644 --- a/openhcs/microscopes/vendor_layout.py +++ b/openhcs/microscopes/vendor_layout.py @@ -26,10 +26,14 @@ class VirtualMappingSource(DatasetSource): """ @abstractmethod - def _build_virtual_mapping(self, dataset_root: Path, filemanager: FileManager) -> Path: + def _build_virtual_mapping( + self, dataset_root: Path, filemanager: FileManager + ) -> Path: """Write this layout's mapping to metadata and return the image directory.""" - def initialize_workspace(self, dataset_root: Path, filemanager: FileManager) -> Path: + def initialize_workspace( + self, dataset_root: Path, filemanager: FileManager + ) -> Path: dataset_root = Path(dataset_root) self.dataset_root = dataset_root self._build_virtual_mapping(dataset_root, filemanager) @@ -101,4 +105,3 @@ def _normalize_mapped_filenames( except Exception as e: logger.error("Error renaming %s to %s: %s", listed_path, new_path, e) return image_dir - diff --git a/openhcs/omero/plugin/omero_openhcs/views.py b/openhcs/omero/plugin/omero_openhcs/views.py index 98211eac7b..ed76e6b7f4 100644 --- a/openhcs/omero/plugin/omero_openhcs/views.py +++ b/openhcs/omero/plugin/omero_openhcs/views.py @@ -70,11 +70,15 @@ def ping(self): def get_server_info(self): """Get detailed server info including workers.""" - return self._send_request(ControlRequestHeader(ControlMessageType.PING).to_dict()) + return self._send_request( + ControlRequestHeader(ControlMessageType.PING).to_dict() + ) def get_status(self, execution_id=None): """Get execution status.""" - return self._send_request(StatusRequest(execution_id=execution_id or None).to_dict()) + return self._send_request( + StatusRequest(execution_id=execution_id or None).to_dict() + ) def execute_pipeline(self, plate_id, pipeline_code, config_code): """Execute pipeline on server.""" diff --git a/openhcs/processing/backends/analysis/neurite_outgrowth.py b/openhcs/processing/backends/analysis/neurite_outgrowth.py index 7f8e638841..fd22c8d424 100644 --- a/openhcs/processing/backends/analysis/neurite_outgrowth.py +++ b/openhcs/processing/backends/analysis/neurite_outgrowth.py @@ -277,7 +277,9 @@ def analyze( if use_nuclear_stain: nuclear_stain.validate("nuclear_stain") if not 0 <= nuclear_stain.channel_index < image_array.shape[0]: - raise ValueError("nuclear_stain.channel_index is outside the input stack") + raise ValueError( + "nuclear_stain.channel_index is outside the input stack" + ) if nuclear_stain.channel_index == neurite_channel_index: raise ValueError( "nuclear_stain.channel_index must differ from neurite_channel_index" @@ -454,14 +456,18 @@ def analyze( physically_soma_rooted_trace = _physically_soma_rooted_owner_mask( pre_topology_owner_skeleton, cell_body_labels, - maximum_root_distance=_TopologyResult.soma_attachment_radius(outgrowth_width_px), + maximum_root_distance=_TopologyResult.soma_attachment_radius( + outgrowth_width_px + ), ) topology_dropped_trace = ( (pre_topology_owner_skeleton > 0) & (neurite_skeleton == 0) & ~crossing_core_mask ) - topology_added_trace = (neurite_skeleton > 0) & (pre_topology_owner_skeleton == 0) + topology_added_trace = (neurite_skeleton > 0) & ( + pre_topology_owner_skeleton == 0 + ) final_topology_owned_trace_pixels = int(np.count_nonzero(neurite_skeleton)) # A physical crossing core supports two logical paths, but an object-label # raster can store only one identity per pixel. The topology above remains @@ -501,7 +507,9 @@ def analyze( ) secondary_owned_unrooted_trace_pixels = int( np.count_nonzero( - candidate_trace_mask & ~rooted_trace_mask & (secondary_owner_regions > 0) + candidate_trace_mask + & ~rooted_trace_mask + & (secondary_owner_regions > 0) ) ) candidate_mask_pixels = int(np.count_nonzero(candidate_neurite_mask)) @@ -509,7 +517,9 @@ def analyze( np.count_nonzero(candidate_neurite_mask & rooted_mask) ) unrooted_residual_pixels = int(np.count_nonzero(unrooted_residual)) - secondary_owned_residual_pixels = int(np.count_nonzero(secondary_owned_residual)) + secondary_owned_residual_pixels = int( + np.count_nonzero(secondary_owned_residual) + ) cell_results = _build_cell_results( cell_body_labels, @@ -574,10 +584,14 @@ def analyze( final_topology_dropped_unrepresented_trace_pixels=int( np.count_nonzero(topology_dropped_trace & ~topology_path_mask) ), - final_topology_added_trace_pixels=int(np.count_nonzero(topology_added_trace)), + final_topology_added_trace_pixels=int( + np.count_nonzero(topology_added_trace) + ), final_topology_owned_trace_pixels=final_topology_owned_trace_pixels, published_owned_trace_pixels=int(np.count_nonzero(rooted_trace_mask)), - secondary_owned_unrooted_trace_pixels=(secondary_owned_unrooted_trace_pixels), + secondary_owned_unrooted_trace_pixels=( + secondary_owned_unrooted_trace_pixels + ), secondary_unowned_unrooted_trace_pixels=( candidate_trace_pixels - rooted_candidate_trace_pixels @@ -651,13 +665,14 @@ def analyze( ), *admission.planes.selected_outputs(neurite_channel_index), *NeuriteOwnershipPlanes( - initial_owned_trace, adopted_owned_trace, pre_topology_owner_skeleton, + initial_owned_trace, + adopted_owned_trace, + pre_topology_owner_skeleton, ).selected_outputs(neurite_channel_index), neurite_morphology, ) - class PixelMetricCoordinates: """Independent pixel-analysis/export capability composed with the CP recipe.""" @@ -764,7 +779,8 @@ def minimum_area_px(self, coordinate_scale: float) -> float: @staticmethod def _inscribed_disc_support( - radius: np.ndarray, medial_support: np.ndarray, + radius: np.ndarray, + medial_support: np.ndarray, ) -> np.ndarray: """Project original medial discs without adding a new geometric rule.""" support = np.zeros(radius.shape, dtype=bool) @@ -772,10 +788,12 @@ def _inscribed_disc_support( disc_radius = float(radius[row, column]) extent = int(np.ceil(disc_radius)) row_start, row_stop = ( - max(0, row - extent), min(radius.shape[0], row + extent + 1), + max(0, row - extent), + min(radius.shape[0], row + extent + 1), ) column_start, column_stop = ( - max(0, column - extent), min(radius.shape[1], column + extent + 1), + max(0, column - extent), + min(radius.shape[1], column + extent + 1), ) rows, columns = np.ogrid[row_start:row_stop, column_start:column_stop] disc = (rows - row) ** 2 + (columns - column) ** 2 < disc_radius**2 @@ -806,9 +824,12 @@ def separate_terminal_shafts( """ body = np.asarray(body, dtype=bool) radius = ndi.distance_transform_edt(np.pad(body, 1))[1:-1, 1:-1] - skeleton = _raw_processing_leaf(medialaxis)( - np.pad(body, 1).astype(np.float32, copy=False) - )[1:-1, 1:-1] > 0 + skeleton = ( + _raw_processing_leaf(medialaxis)( + np.pad(body, 1).astype(np.float32, copy=False) + )[1:-1, 1:-1] + > 0 + ) distance_to_nucleus = ndi.distance_transform_edt(~nuclear_seed) broad_medial = skeleton & ( (2.0 * radius - 1.0 > maximum_shaft_width_px) @@ -820,9 +841,12 @@ def separate_terminal_shafts( nuclear_regions = np.unique(broad_regions[broad_support & nuclear_seed]) soma_support = broad_support & np.isin(broad_regions, nuclear_regions) protected = skeleton & soma_support - degrees = ndi.convolve( - skeleton.astype(np.int32), connectivity, mode="constant", cval=0 - ) - skeleton + degrees = ( + ndi.convolve( + skeleton.astype(np.int32), connectivity, mode="constant", cval=0 + ) + - skeleton + ) thin_components, count = ndi.label(skeleton & ~protected, connectivity) removed_medial = np.zeros(body.shape, dtype=bool) for component_id in range(1, count + 1): @@ -840,7 +864,8 @@ def separate_terminal_shafts( return body retained_support = self._inscribed_disc_support( - radius, skeleton & ~removed_medial, + radius, + skeleton & ~removed_medial, ) removed_support = self._inscribed_disc_support(radius, removed_medial) retained = body & (~removed_support | retained_support | nuclear_seed) @@ -1229,7 +1254,8 @@ def selected_outputs( ) -> tuple[SelectedDiagnosticPlaneImageOutput, ...]: return tuple( SelectedDiagnosticPlaneImageOutput( - getattr(self, field.name)[None], (source_index,), + getattr(self, field.name)[None], + (source_index,), ) for field in fields(self) ) @@ -1253,7 +1279,6 @@ class NeuriteAdmissionPlanes(NeuriteDiagnosticPlanes): soma_attachment_response: np.ndarray - @dataclass(frozen=True) class NeuriteOwnershipPlanes(NeuriteDiagnosticPlanes): """Original owner labels at each graph stage, before final topology. @@ -1281,7 +1306,9 @@ def response(self) -> np.ndarray: return self.planes.local_response def soma_attachment_labels( - self, cell_body_labels: np.ndarray, minimum_response: float, + self, + cell_body_labels: np.ndarray, + minimum_response: float, ) -> np.ndarray: """Qualify the declared root neighborhood at the soma signal scale. @@ -1290,11 +1317,13 @@ def soma_attachment_labels( neighborhood and nearest accepted soma; it cannot admit remote tracks. """ _, neighborhood = _TopologyResult.soma_neighborhood( - cell_body_labels, self.outgrowth_width_px, + cell_body_labels, + self.outgrowth_width_px, ) return np.where( self.planes.soma_attachment_response >= minimum_response, - neighborhood, 0, + neighborhood, + 0, ).astype(np.int32, copy=False) def skeleton_for(self, cell_body_labels: np.ndarray) -> np.ndarray: @@ -1316,15 +1345,31 @@ def skeleton_for(self, cell_body_labels: np.ndarray) -> np.ndarray: # Both callable ABIs derive diagnostic slots from the same stage declaration. -NeuriteOutgrowthRuntimeTuple = Tuple[( - np.ndarray, DataclassMeasurementColumnarRows, DataclassMeasurementColumnarRows, - np.ndarray, np.ndarray, np.ndarray, np.ndarray, - SelectedPlaneImageOutput, SelectedPlaneImageOutput, SelectedPlaneImageOutput, - SelectedPlaneImageOutput, SelectedPlaneImageOutput, - *(SelectedDiagnosticPlaneImageOutput for _ in get_type_hints(NeuriteAdmissionPlanes)), - *(SelectedDiagnosticPlaneImageOutput for _ in get_type_hints(NeuriteOwnershipPlanes)), - SpatialGraph, -)] +NeuriteOutgrowthRuntimeTuple = Tuple[ + ( + np.ndarray, + DataclassMeasurementColumnarRows, + DataclassMeasurementColumnarRows, + np.ndarray, + np.ndarray, + np.ndarray, + np.ndarray, + SelectedPlaneImageOutput, + SelectedPlaneImageOutput, + SelectedPlaneImageOutput, + SelectedPlaneImageOutput, + SelectedPlaneImageOutput, + *( + SelectedDiagnosticPlaneImageOutput + for _ in get_type_hints(NeuriteAdmissionPlanes) + ), + *( + SelectedDiagnosticPlaneImageOutput + for _ in get_type_hints(NeuriteOwnershipPlanes) + ), + SpatialGraph, + ) +] @dataclass(frozen=True) @@ -1339,9 +1384,12 @@ def supports_owner(self, owner: int, path_owners: np.ndarray) -> bool: return any(int(path_owners[path]) == owner for path in self.arm_paths) def crossover_paths(self, path_owners: np.ndarray) -> tuple[int, ...]: - owners = Counter(int(path_owners[path]) for path in self.arm_paths - if path_owners[path] > 0) - return tuple(path for path in self.arm_paths if owners[int(path_owners[path])] == 2) + owners = Counter( + int(path_owners[path]) for path in self.arm_paths if path_owners[path] > 0 + ) + return tuple( + path for path in self.arm_paths if owners[int(path_owners[path])] == 2 + ) def core_coordinates(self, path_coordinates: tuple[np.ndarray, ...]) -> np.ndarray: if not self.core_paths: @@ -1349,7 +1397,9 @@ def core_coordinates(self, path_coordinates: tuple[np.ndarray, ...]) -> np.ndarr return np.concatenate([path_coordinates[path] for path in self.core_paths]) def regional_core_coordinates( - self, path_coordinates: tuple[np.ndarray, ...], shape: tuple[int, int], + self, + path_coordinates: tuple[np.ndarray, ...], + shape: tuple[int, int], origin: tuple[int, int], ) -> np.ndarray: coordinates = self.core_coordinates(path_coordinates) - origin @@ -1378,20 +1428,26 @@ def soma_attachment_radius(outgrowth_width_px: float) -> int: @classmethod def soma_neighborhood( - cls, cell_body_labels: np.ndarray, outgrowth_width_px: float, + cls, + cell_body_labels: np.ndarray, + outgrowth_width_px: float, ) -> tuple[np.ndarray, np.ndarray]: distance, nearest = ndi.distance_transform_edt( - cell_body_labels == 0, return_indices=True, + cell_body_labels == 0, + return_indices=True, ) labels = np.where( distance <= cls.soma_attachment_radius(outgrowth_width_px), - cell_body_labels[tuple(nearest)], 0, + cell_body_labels[tuple(nearest)], + 0, ) return distance, labels @classmethod def soma_ownership_seeds( - cls, cell_body_labels: np.ndarray, outgrowth_width_px: float, + cls, + cell_body_labels: np.ndarray, + outgrowth_width_px: float, path_coordinates: Sequence[np.ndarray], ) -> dict[int, tuple[int, ...]]: """Near-body evidence seeds ownership, not biological process count.""" @@ -1401,12 +1457,16 @@ def soma_ownership_seeds( labels = neighborhood[tuple(coordinates.T)] labels = labels[labels > 0] if labels.size: - roots[path] = tuple(label for label, _ in Counter(map(int, labels)).most_common()) + roots[path] = tuple( + label for label, _ in Counter(map(int, labels)).most_common() + ) return roots @classmethod def soma_process_roots( - cls, cell_body_labels: np.ndarray, outgrowth_width_px: float, + cls, + cell_body_labels: np.ndarray, + outgrowth_width_px: float, path_coordinates: Sequence[np.ndarray], transitions: Mapping[int, Iterable[int]], path_owners: np.ndarray, @@ -1419,7 +1479,8 @@ def soma_process_roots( distinct exits separated by a more distal junction remain separate. """ distance, neighborhood = cls.soma_neighborhood( - cell_body_labels, outgrowth_width_px, + cell_body_labels, + outgrowth_width_px, ) proximity_by_owner: dict[int, dict[int, float]] = defaultdict(dict) for path, coordinates in enumerate(path_coordinates): @@ -1443,7 +1504,8 @@ def soma_process_roots( while frontier: path = frontier.pop() adjacent = { - neighbor for neighbor in transitions[path] + neighbor + for neighbor in transitions[path] if neighbor in remaining and proximity[neighbor] == proximity[seed] } @@ -1451,9 +1513,9 @@ def soma_process_roots( plateau.update(adjacent) frontier.extend(sorted(adjacent)) if any( - neighbor in proximity - and proximity[neighbor] < proximity[seed] - for path in plateau for neighbor in transitions[path] + neighbor in proximity and proximity[neighbor] < proximity[seed] + for path in plateau + for neighbor in transitions[path] ): continue roots[owner].append(min(plateau)) @@ -1499,12 +1561,15 @@ def classify_owned_endpoints( branch_groups: dict[int, list[int]] = defaultdict(list) for owner, owner_paths in owned_paths.items(): candidates = { - group: paths for (cell, group), paths in endpoint_paths.items() + group: paths + for (cell, group), paths in endpoint_paths.items() if cell == owner and len(paths) >= 3 } if not candidates: continue - coordinates = np.concatenate([self.path_coordinates[path] for path in owner_paths]) + coordinates = np.concatenate( + [self.path_coordinates[path] for path in owner_paths] + ) lower, upper = coordinates.min(axis=0), coordinates.max(axis=0) + 1 body_slice = body_slices[owner - 1] if owner <= len(body_slices) else None if body_slice is not None: @@ -1513,7 +1578,9 @@ def classify_owned_endpoints( margin = self.soma_attachment_radius(outgrowth_width_px) lower = np.maximum(0, lower - margin) upper = np.minimum(cell_body_labels.shape, upper + margin) - region = tuple(slice(int(start), int(stop)) for start, stop in zip(lower, upper)) + region = tuple( + slice(int(start), int(stop)) for start, stop in zip(lower, upper) + ) local_bodies = cell_body_labels[region] own_trace = np.zeros(local_bodies.shape, dtype=np.int32) own_trace[tuple((coordinates - lower).T)] = owner @@ -1526,24 +1593,54 @@ def classify_owned_endpoints( foreground_radius = ndi.distance_transform_edt(foreground) for group, paths in candidates.items(): # Sample physical endpoints, not fractional exported centroids. - centers = np.unique(np.asarray([ - self.path_coordinates[path][0 if endpoint == 0 else -1] - lower - for path in paths for endpoint in (0, 1) - if self.path_endpoint_groups[path][endpoint] == group - ]), axis=0) + centers = np.unique( + np.asarray( + [ + self.path_coordinates[path][0 if endpoint == 0 else -1] + - lower + for path in paths + for endpoint in (0, 1) + if self.path_endpoint_groups[path][endpoint] == group + ] + ), + axis=0, + ) radii = foreground_radius[tuple(centers.T)] neighborhood_radii = radii + outgrowth_width_px - ring_lower = np.maximum(0, np.floor((centers - neighborhood_radii[:, None]).min(axis=0)).astype(int) - 2) - ring_upper = np.minimum(local_bodies.shape, np.ceil((centers + neighborhood_radii[:, None]).max(axis=0)).astype(int) + 3) - ring_slice = tuple(slice(int(start), int(stop)) for start, stop in zip(ring_lower, ring_upper)) - grid = np.moveaxis(np.indices(tuple(ring_upper - ring_lower)), 0, -1) + ring_lower + ring_lower = np.maximum( + 0, + np.floor( + (centers - neighborhood_radii[:, None]).min(axis=0) + ).astype(int) + - 2, + ) + ring_upper = np.minimum( + local_bodies.shape, + np.ceil((centers + neighborhood_radii[:, None]).max(axis=0)).astype( + int + ) + + 3, + ) + ring_slice = tuple( + slice(int(start), int(stop)) + for start, stop in zip(ring_lower, ring_upper) + ) + grid = ( + np.moveaxis(np.indices(tuple(ring_upper - ring_lower)), 0, -1) + + ring_lower + ) distances = np.linalg.norm(grid[:, :, None] - centers, axis=3) for expansion in range(int(np.floor(outgrowth_width_px)) + 1): perimeter_radii = radii + expansion core = np.any(distances <= perimeter_radii, axis=2) - ring = ndi.binary_dilation(core, structure=np.ones((3, 3), dtype=bool)) & ~core + ring = ( + ndi.binary_dilation(core, structure=np.ones((3, 3), dtype=bool)) + & ~core + ) ring &= foreground[ring_slice] & (local_bodies[ring_slice] == 0) - corridors, _ = ndi.label(ring, structure=np.ones((3, 3), dtype=bool)) + corridors, _ = ndi.label( + ring, structure=np.ones((3, 3), dtype=bool) + ) resolved_corridors: set[int] = set() proximal_arm = False for path in paths: @@ -1551,26 +1648,44 @@ def classify_owned_endpoints( for endpoint in (0, 1): if groups[endpoint] != group: continue - arm = self.path_coordinates[path][::1 if endpoint == 0 else -1] - lower - inside_core = np.any(np.linalg.norm(arm[:, None] - centers, axis=2) <= perimeter_radii, axis=1) + arm = ( + self.path_coordinates[path][ + :: 1 if endpoint == 0 else -1 + ] + - lower + ) + inside_core = np.any( + np.linalg.norm(arm[:, None] - centers, axis=2) + <= perimeter_radii, + axis=1, + ) departures = np.flatnonzero(~inside_core) if departures.size: first_exit = arm[departures[0]] - ring_lower - if np.all((first_exit >= 0) & (first_exit < corridors.shape)): + if np.all( + (first_exit >= 0) & (first_exit < corridors.shape) + ): corridor = int(corridors[tuple(first_exit)]) if corridor: resolved_corridors.add(corridor) elif groups[0] != groups[1]: # Only the opposite endpoint can be a short root. neighbors = arm[-1] + body_neighbor_offsets - inside = np.all((neighbors >= 0) & (neighbors < local_bodies.shape), axis=1) - if np.any(local_bodies[tuple(neighbors[inside].T)] == owner): + inside = np.all( + (neighbors >= 0) & (neighbors < local_bodies.shape), + axis=1, + ) + if np.any( + local_bodies[tuple(neighbors[inside].T)] == owner + ): proximal_arm = True if len(resolved_corridors) + int(proximal_arm) >= 3: branch_groups[owner].append(group) break qualified_groups = { - (owner, group) for owner, groups in branch_groups.items() for group in groups + (owner, group) + for owner, groups in branch_groups.items() + for group in groups } branch_types = np.zeros(len(self.path_owners), dtype=np.int32) for path_index, groups in enumerate(self.path_endpoint_groups): @@ -1578,9 +1693,9 @@ def classify_owned_endpoints( if owner <= 0 or path_index in crossing_core_paths: continue branch_types[path_index] = ( - 3 if groups[0] == groups[1] else sum( - (owner, group) in qualified_groups for group in groups - ) + 3 + if groups[0] == groups[1] + else sum((owner, group) in qualified_groups for group in groups) ) coordinates = { group: np.asarray(coordinate) @@ -1597,35 +1712,50 @@ def crossing_nodes(self) -> frozenset[int]: @property def crossing_paths(self) -> frozenset[int]: - return frozenset(path for crossing in self.resolved_crossings - for path in crossing.crossover_paths(self.path_owners)) + return frozenset( + path + for crossing in self.resolved_crossings + for path in crossing.crossover_paths(self.path_owners) + ) @property def crossing_core_paths(self) -> frozenset[int]: - return frozenset(path for crossing in self.resolved_crossings - for path in crossing.core_paths) + return frozenset( + path for crossing in self.resolved_crossings for path in crossing.core_paths + ) def crossing_core_coordinates(self, *, owner: int) -> np.ndarray: - coordinates = [crossing.core_coordinates(self.path_coordinates) - for crossing in self.resolved_crossings - if crossing.supports_owner(owner, self.path_owners)] - return np.concatenate(coordinates) if coordinates else np.empty((0, 2), dtype=int) + coordinates = [ + crossing.core_coordinates(self.path_coordinates) + for crossing in self.resolved_crossings + if crossing.supports_owner(owner, self.path_owners) + ] + return ( + np.concatenate(coordinates) if coordinates else np.empty((0, 2), dtype=int) + ) def crossing_core_mask( - self, shape: tuple[int, int], *, + self, + shape: tuple[int, int], + *, origin: tuple[int, int] = (0, 0), ) -> np.ndarray: """Project the physical cores reserved by resolved logical crossings.""" mask = np.zeros(shape, dtype=bool) for crossing in self.resolved_crossings: coordinates = crossing.regional_core_coordinates( - self.path_coordinates, shape, origin, + self.path_coordinates, + shape, + origin, ) mask[tuple(coordinates.T)] = True return mask def crossing_support_mask( - self, occupied_owners: np.ndarray, *, owner: int, + self, + occupied_owners: np.ndarray, + *, + owner: int, origin: tuple[int, int], ) -> np.ndarray: """Project one logical owner's core without overwriting foreign input.""" @@ -1634,7 +1764,9 @@ def crossing_support_mask( if not crossing.supports_owner(owner, self.path_owners): continue coordinates = crossing.regional_core_coordinates( - self.path_coordinates, occupied_owners.shape, origin, + self.path_coordinates, + occupied_owners.shape, + origin, ) current = occupied_owners[tuple(coordinates.T)] arm_owners = self.path_owners[list(crossing.arm_paths)] @@ -2067,8 +2199,7 @@ def _cell_body_contract_candidates( ) if ( region.area >= minimum_area_px - and maximum_inscribed_diameter_px - >= minimum_inscribed_diameter_px + and maximum_inscribed_diameter_px >= minimum_inscribed_diameter_px and region.axis_minor_length <= maximum_width_px and region_response.size and float(np.mean(region_response)) >= intensity_threshold @@ -2176,14 +2307,20 @@ def _identify_neurites_cellprofiler( ) local_support = response >= settings.intensity_above_local_background soma_attachment_response = local_background_response( - image, object_width_px=body_width_px, bright_objects=bright_objects, + image, + object_width_px=body_width_px, + bright_objects=bright_objects, ) outgrowth_mask = cp_mask & local_support return NeuriteAdmissionResult( outgrowth_mask, NeuriteAdmissionPlanes( - np.asarray(enhanced), threshold_support, cp_mask, - response, local_support, soma_attachment_response, + np.asarray(enhanced), + threshold_support, + cp_mask, + response, + local_support, + soma_attachment_response, ), outgrowth_width_px, ) @@ -2462,7 +2599,13 @@ def _repair_signal_supported_skeleton( shaft_support = np.asarray(admission.mask, dtype=bool) regions = np.asarray(owner_regions, dtype=np.int32) bodies = np.asarray(cell_body_labels, dtype=np.int32) - if not repaired.shape == response.shape == shaft_support.shape == regions.shape == bodies.shape: + if ( + not repaired.shape + == response.shape + == shaft_support.shape + == regions.shape + == bodies.shape + ): raise ValueError( "labels, admission, owner_regions, and cell_body_labels must " "have the same shape" @@ -2508,14 +2651,18 @@ def _repair_signal_supported_skeleton( original_owner = (local_repaired == owner) & ~body_mask origin = (owner_slice[0].start, owner_slice[1].start) local_core = crossing_topology.crossing_core_mask( - local_repaired.shape, origin=origin, + local_repaired.shape, + origin=origin, ) local_shared_core = crossing_topology.crossing_support_mask( - labels[owner_slice], owner=owner, origin=origin, + labels[owner_slice], + owner=owner, + origin=origin, ) original_core_values = local_repaired[local_shared_core].copy() shared_signal_support = ( - local_shared_core & (local_response >= minimum_response) + local_shared_core + & (local_response >= minimum_response) & ((local_bodies == 0) | body_mask) ) local_repaired[(local_repaired == owner) & body_mask] = 0 @@ -2543,9 +2690,16 @@ def _repair_signal_supported_skeleton( signal_support = local_shaft_support & (local_regions == owner) root_support = soma_attachments[owner_slice] == owner allowed = ( - signal_support | original_owner | body_mask | shared_signal_support | root_support - ) & ~occupied_by_other_owner & ~foreign_body & ( - ~local_core | shared_signal_support | original_owner | body_mask + ( + signal_support + | original_owner + | body_mask + | shared_signal_support + | root_support + ) + & ~occupied_by_other_owner + & ~foreign_body + & (~local_core | shared_signal_support | original_owner | body_mask) ) while True: @@ -2833,7 +2987,9 @@ def _analyze_topology( transitions[second].add(first) root_labels_by_path = _TopologyResult.soma_ownership_seeds( - cell_body_labels, outgrowth_width_px, path_coordinates, + cell_body_labels, + outgrowth_width_px, + path_coordinates, ) path_owners, path_distances = _propagate_path_owners( @@ -2893,11 +3049,16 @@ def _analyze_topology( roots_by_cell, ) process_roots = _TopologyResult.soma_process_roots( - cell_body_labels, outgrowth_width_px, path_coordinates, transitions, path_owners, + cell_body_labels, + outgrowth_width_px, + path_coordinates, + transitions, + path_owners, ) resolved_crossings = tuple( - _ResolvedCrossing(node, crossing_paths_by_node[node], - crossing_core_paths_by_node[node]) + _ResolvedCrossing( + node, crossing_paths_by_node[node], crossing_core_paths_by_node[node] + ) for node in sorted(crossing_nodes) ) topology = _TopologyResult( @@ -2915,15 +3076,22 @@ def _analyze_topology( root_paths_by_cell=process_roots, branch_nodes_by_cell={}, resolved_crossings=tuple( - crossing for crossing in resolved_crossings + crossing + for crossing in resolved_crossings if crossing.crossover_paths(path_owners) ), ) path_branch_types, branch_nodes_by_cell = topology.classify_owned_endpoints( - branch_support=branch_support, cell_body_labels=cell_body_labels, - outgrowth_width_px=outgrowth_width_px, merge_radius=max(1.0, outgrowth_width_px), + branch_support=branch_support, + cell_body_labels=cell_body_labels, + outgrowth_width_px=outgrowth_width_px, + merge_radius=max(1.0, outgrowth_width_px), + ) + return replace( + topology, + path_branch_types=path_branch_types, + branch_nodes_by_cell=branch_nodes_by_cell, ) - return replace(topology, path_branch_types=path_branch_types, branch_nodes_by_cell=branch_nodes_by_cell) def _analyze_owned_topology( @@ -2989,7 +3157,8 @@ def _analyze_owned_topology( ) local_owner_mask = owned[owner_slice] == owner local_owner_mask |= crossing_topology.crossing_support_mask( - owned[owner_slice], owner=owner, + owned[owner_slice], + owner=owner, origin=(owner_slice[0].start, owner_slice[1].start), ) local_owned = np.where(local_owner_mask, owner, 0).astype( @@ -3056,9 +3225,11 @@ def _analyze_owned_topology( } ) resolved_crossings.extend( - _ResolvedCrossing(crossing.node + node_offset, - tuple(path + path_offset for path in crossing.arm_paths), - tuple(path + path_offset for path in crossing.core_paths)) + _ResolvedCrossing( + crossing.node + node_offset, + tuple(path + path_offset for path in crossing.arm_paths), + tuple(path + path_offset for path in crossing.core_paths), + ) for crossing in local.resolved_crossings ) @@ -3417,9 +3588,7 @@ def _build_neurite_morphology_graph( root_node = SpatialGraphNode.from_features( node_id=next_node_id, coordinates=root_coordinate, - radius=( - soma_radius if root_role == "soma_root" else process_radius - ), + radius=(soma_radius if root_role == "soma_root" else process_radius), features={ "label": owner, "neuron_label": owner, @@ -3507,9 +3676,7 @@ def enqueue_from(group_id: int) -> None: target_node = nodes_by_group[target_group] coordinates[-1] = target_node.coordinates branch_distance = float(topology.path_lengths[path_index]) - euclidean_distance = float( - topology.path_euclidean_lengths[path_index] - ) + euclidean_distance = float(topology.path_euclidean_lengths[path_index]) graph_edges.append( SpatialGraphEdge.from_features( edge_id=next_edge_id, @@ -3883,9 +4050,7 @@ def _build_summary( number_of_cells=cell_count, coordinate_unit=coordinate_unit, total_outgrowth=total_outgrowth, - mean_outgrowth_per_cell=( - total_outgrowth / cell_count if cell_count else 0.0 - ), + mean_outgrowth_per_cell=(total_outgrowth / cell_count if cell_count else 0.0), total_processes=total_processes, mean_processes_per_cell=(total_processes / cell_count if cell_count else 0.0), total_branches=total_branches, diff --git a/openhcs/processing/backends/cellprofiler/alignment.py b/openhcs/processing/backends/cellprofiler/alignment.py index 071ce95ca6..16cea63739 100644 --- a/openhcs/processing/backends/cellprofiler/alignment.py +++ b/openhcs/processing/backends/cellprofiler/alignment.py @@ -500,7 +500,6 @@ class AlignmentBackendStrategy( ): """Alignment operations keyed by OpenHCS memory type/provider.""" - @abstractmethod def mutual_information_offset( self, @@ -783,9 +782,7 @@ def execute( ) for index in range(plane_count) ) - images = tuple( - np.asarray(payload.data) for payload in image_payloads - ) + images = tuple(np.asarray(payload.data) for payload in image_payloads) first_image, second_image = images[:2] metadata = tuple(payload.metadata for payload in image_payloads) masks = tuple( diff --git a/openhcs/processing/backends/cellprofiler/area_occupied.py b/openhcs/processing/backends/cellprofiler/area_occupied.py index dfbd07989f..d6c7114097 100644 --- a/openhcs/processing/backends/cellprofiler/area_occupied.py +++ b/openhcs/processing/backends/cellprofiler/area_occupied.py @@ -27,7 +27,13 @@ MeasurementSparseColumnarRows, ) from openhcs.core.memory.decorators import numpy -from openhcs.core.pipeline.function_contracts import (composed_image_payload, object_label_input_execution_mode, resolved_callable_parameter, runtime_bound_parameters, special_inputs) +from openhcs.core.pipeline.function_contracts import ( + composed_image_payload, + object_label_input_execution_mode, + resolved_callable_parameter, + runtime_bound_parameters, + special_inputs, +) from openhcs.core.runtime_array_values import RuntimeArrayData from openhcs.core.runtime_batch_contracts import SliceIndexRuntimeParameter from openhcs.core.runtime_measurements import ( @@ -168,12 +174,15 @@ def primary_image_domain_input_binding(cls) -> SettingToKeywordBinding: @classmethod def primary_image_inputs( - cls, contract: CallableContract, declared_inputs: tuple[ArtifactSpec, ...], + cls, + contract: CallableContract, + declared_inputs: tuple[ArtifactSpec, ...], ) -> tuple[ArtifactSpec, ...]: """Binary operands retain their declared main-image domain.""" del cls, contract return tuple( - spec for spec in declared_inputs + spec + for spec in declared_inputs if spec.artifact_type is ImageArtifactType and spec.parameter_name is None and not spec.stack_broadcast_sources() @@ -181,7 +190,8 @@ def primary_image_inputs( @classmethod def primary_image_domain_specs( - cls, domain_inputs: tuple[ArtifactSpec, ...], + cls, + domain_inputs: tuple[ArtifactSpec, ...], ) -> tuple[ArtifactSpec, ...]: """Each object operand owns its independent source-qualified row.""" if not domain_inputs: @@ -190,21 +200,28 @@ def primary_image_domain_specs( @classmethod def invocation_domain_inputs( - cls, contract: CallableContract, declared_inputs: tuple[ArtifactSpec, ...], + cls, + contract: CallableContract, + declared_inputs: tuple[ArtifactSpec, ...], ) -> tuple[ArtifactSpec, ...]: image_inputs = cls.primary_image_inputs(contract, declared_inputs) - return image_inputs or super().invocation_domain_inputs(contract, declared_inputs) + return image_inputs or super().invocation_domain_inputs( + contract, declared_inputs + ) @classmethod def project_invocation_image_request( - cls, *, image_request: CellProfilerImageRequest, + cls, + *, + image_request: CellProfilerImageRequest, runtime_kwargs: RuntimeCallableKwargs, ) -> CellProfilerImageRequest: rows = runtime_kwargs[AreaOccupiedRowsRuntimeParameter.require_parameter_name()] if any(row.operand is OperandChoice.BINARY_IMAGE for row in rows): return image_request return super().project_invocation_image_request( - image_request=image_request, runtime_kwargs=runtime_kwargs, + image_request=image_request, + runtime_kwargs=runtime_kwargs, ) @classmethod @@ -892,7 +909,8 @@ def _binary_images_from_payload( ) return tuple( np.asarray( - (RuntimeSliceProjection.value_for_slice( + ( + RuntimeSliceProjection.value_for_slice( image, RuntimePlaneAxisValueProjection.from_selected_plane( axis=RuntimePlaneAxis.SOURCE_BINDING, @@ -900,7 +918,8 @@ def _binary_images_from_payload( axis_size=projection.axis_size, source_aliases=projection.source_aliases, ), - )).data + ) + ).data ) for index in range(projection.axis_size) ) diff --git a/openhcs/processing/backends/cellprofiler/calculate_statistics.py b/openhcs/processing/backends/cellprofiler/calculate_statistics.py index 16b74ec1da..62489963e3 100644 --- a/openhcs/processing/backends/cellprofiler/calculate_statistics.py +++ b/openhcs/processing/backends/cellprofiler/calculate_statistics.py @@ -419,7 +419,10 @@ def _statistics_feature_series( """Enumerate numeric image and averaged-object features from typed tables.""" series_by_identity: dict[tuple[str, str], dict[int, float]] = {} for table in measurement_tables: - if table.subject.scope not in {MeasurementScope.SAMPLE, MeasurementScope.OBJECT}: + if table.subject.scope not in { + MeasurementScope.SAMPLE, + MeasurementScope.OBJECT, + }: continue semantics = ColumnarMeasurementTableSchema.from_table(table) object_names = ( diff --git a/openhcs/processing/backends/cellprofiler/classification.py b/openhcs/processing/backends/cellprofiler/classification.py index a1ac8b4ed9..9d98ac0b56 100644 --- a/openhcs/processing/backends/cellprofiler/classification.py +++ b/openhcs/processing/backends/cellprofiler/classification.py @@ -111,7 +111,9 @@ class _ClassificationRuntimeInputParameter(KeywordRuntimeParameter, ABC): @classmethod def bind_runtime_inputs( - cls, request: RuntimeInputBindingRequest, object_spec: ArtifactSpec, + cls, + request: RuntimeInputBindingRequest, + object_spec: ArtifactSpec, labels: ObjectLabelValue, ) -> dict[str, RuntimeCallableArgument]: if not cls.runtime_binding_active(request): @@ -125,7 +127,9 @@ def runtime_binding_active(cls, request: RuntimeInputBindingRequest) -> bool: @classmethod @abstractmethod def runtime_input_values( - cls, request: RuntimeInputBindingRequest, object_spec: ArtifactSpec, + cls, + request: RuntimeInputBindingRequest, + object_spec: ArtifactSpec, labels: ObjectLabelValue, ) -> dict[str, RuntimeCallableArgument]: """Bind this declaration's contribution, cooperatively where composed.""" @@ -149,7 +153,9 @@ class _ClassifiedImageRuleIndicesRuntimeParameter(_ClassificationRuntimeInputPar @classmethod def runtime_input_values( - cls, request: RuntimeInputBindingRequest, object_spec: ArtifactSpec, + cls, + request: RuntimeInputBindingRequest, + object_spec: ArtifactSpec, labels: ObjectLabelValue, ) -> dict[str, RuntimeCallableArgument]: image_outputs = request.adapter.request.require_callable_contract().artifact_outputs.of_artifact_type( @@ -172,7 +178,9 @@ def runtime_input_values( return {cls.require_parameter_name(): tuple(indices)} -class _ClassificationMeasurementVectorRuntimeParameter(_ClassificationRuntimeInputParameter): +class _ClassificationMeasurementVectorRuntimeParameter( + _ClassificationRuntimeInputParameter +): """Bind a declared feature through the original object-vector authority.""" annotation_type = np.ndarray | None @@ -248,7 +256,9 @@ def classification_method(cls) -> ClassificationMethod: @classmethod def runtime_binding_active(cls, request: RuntimeInputBindingRequest) -> bool: - return not _ClassificationRuleValuesRuntimeParameter.runtime_binding_active(request) + return not _ClassificationRuleValuesRuntimeParameter.runtime_binding_active( + request + ) @classmethod def measurement_feature_binding(cls) -> MeasurementFeatureSettingBinding: @@ -315,7 +325,9 @@ def runtime_binding_active(cls, request: RuntimeInputBindingRequest) -> bool: @classmethod def runtime_input_values( - cls, request: RuntimeInputBindingRequest, object_spec: ArtifactSpec, + cls, + request: RuntimeInputBindingRequest, + object_spec: ArtifactSpec, labels: ObjectLabelValue, ) -> dict[str, RuntimeCallableArgument]: return { @@ -324,10 +336,13 @@ def runtime_input_values( request, object_ref=object_spec, feature_name=_classification_rule_measurement_feature( - rule, request.adapter.request.require_callable_contract().module_name + rule, + request.adapter.request.require_callable_contract().module_name, ), labels=labels, - ).vector().runtime_value + ) + .vector() + .runtime_value for rule in cls.rules_for_request(request) ) } @@ -641,7 +656,9 @@ def object_class_rows( def _classification_kwargs( cls, module: "ModuleBlock", binder: "SettingsBinder" ) -> "RuntimeCallableKwargs": - return ClassificationMethod.from_module(cls, module).bind_settings(cls, module, binder) + return ClassificationMethod.from_module(cls, module).bind_settings( + cls, module, binder + ) @classmethod def bind_settings(cls, module, *, binder): @@ -722,9 +739,9 @@ def finalize_module_blocks_for_invocation( invocation=invocation, step_context=step_context, ) - return ClassificationMethod.from_callable_contract(invocation.contract).finalize_module_blocks( - cls, blocks, invocation - ) + return ClassificationMethod.from_callable_contract( + invocation.contract + ).finalize_module_blocks(cls, blocks, invocation) @classmethod def _block_with_scalar_classification_output( @@ -1124,7 +1141,9 @@ def classified_image_outputs( ) -> tuple[ClassifiedImageOutput, ...]: """Return every active repeated classified-image row in rule order.""" - return ClassificationMethod.from_module(cls, module).classified_image_outputs(cls, module) + return ClassificationMethod.from_module(cls, module).classified_image_outputs( + cls, module + ) @classmethod def _single_measurement_classified_image_outputs( @@ -1311,21 +1330,27 @@ class _ClassificationMethodBehavior(ABC): @abstractmethod def bind_settings( - self, module_type: type[ClassifyObjectsSingleMeasurementModule], - module: ModuleBlock, binder: SettingsBinder, + self, + module_type: type[ClassifyObjectsSingleMeasurementModule], + module: ModuleBlock, + binder: SettingsBinder, ) -> RuntimeCallableKwargs: """Bind this mode through the concrete module's existing setting hook.""" @abstractmethod def classified_image_outputs( - self, module_type: type[ClassifyObjectsSingleMeasurementModule], module: ModuleBlock, + self, + module_type: type[ClassifyObjectsSingleMeasurementModule], + module: ModuleBlock, ) -> tuple[ClassifiedImageOutput, ...]: """Project this mode's concrete module image-output hook.""" @abstractmethod def finalize_mode_blocks( - self, module_type: type[ClassifyObjectsSingleMeasurementModule], - blocks: tuple[ModuleBlock, ...], invocation: NormalizedFunctionItem, + self, + module_type: type[ClassifyObjectsSingleMeasurementModule], + blocks: tuple[ModuleBlock, ...], + invocation: NormalizedFunctionItem, ) -> tuple[ModuleBlock, ...]: """Complete this mode after the shared template emits its literal.""" @@ -1343,7 +1368,8 @@ def finalize_mode_blocks(self, module_type, blocks, invocation): rules = invocation.kwargs_dict.get("classification_rules") if rules is not None: if not isinstance(rules, tuple) or any( - not isinstance(rule, SingleMeasurementClassificationRule) for rule in rules + not isinstance(rule, SingleMeasurementClassificationRule) + for rule in rules ): raise TypeError( "ClassifyObjects classification_rules must be a tuple of " @@ -1401,7 +1427,9 @@ class ClassificationMethod(Enum): cellprofiler_literals: tuple[str, ...] def __new__( - cls, absorbed_value: str, behavior: _ClassificationMethodBehavior, + cls, + absorbed_value: str, + behavior: _ClassificationMethodBehavior, *cellprofiler_literals: str, ) -> "ClassificationMethod": member = enum_member_with_payload( @@ -1427,7 +1455,9 @@ def __new__( @classmethod def from_module( - cls, module_type: type[ClassifyObjectsSingleMeasurementModule], module: ModuleBlock, + cls, + module_type: type[ClassifyObjectsSingleMeasurementModule], + module: ModuleBlock, ) -> ClassificationMethod: return coerce_cellprofiler_enum( cls, @@ -1452,14 +1482,18 @@ def from_callable_contract(cls, contract: CallableContract) -> ClassificationMet return next(iter(methods)) def require_callable( - self, module_type: type[ClassifyObjectsSingleMeasurementModule], + self, + module_type: type[ClassifyObjectsSingleMeasurementModule], ) -> Callable[..., object]: matches = tuple( func for name in module_type.declared_function_names() if self.from_callable_contract( - CallableContract.from_callable(func := module_type.require_callable(name)) - ) is self + CallableContract.from_callable( + func := module_type.require_callable(name) + ) + ) + is self ) if len(matches) != 1: raise ValueError( @@ -1468,19 +1502,25 @@ def require_callable( return matches[0] def bind_settings( - self, module_type: type[ClassifyObjectsSingleMeasurementModule], - module: ModuleBlock, binder: SettingsBinder, + self, + module_type: type[ClassifyObjectsSingleMeasurementModule], + module: ModuleBlock, + binder: SettingsBinder, ) -> RuntimeCallableKwargs: return self._behavior.bind_settings(module_type, module, binder) def classified_image_outputs( - self, module_type: type[ClassifyObjectsSingleMeasurementModule], module: ModuleBlock, + self, + module_type: type[ClassifyObjectsSingleMeasurementModule], + module: ModuleBlock, ) -> tuple[ClassifiedImageOutput, ...]: return self._behavior.classified_image_outputs(module_type, module) def finalize_module_blocks( - self, module_type: type[ClassifyObjectsSingleMeasurementModule], - blocks: tuple[ModuleBlock, ...], invocation: NormalizedFunctionItem, + self, + module_type: type[ClassifyObjectsSingleMeasurementModule], + blocks: tuple[ModuleBlock, ...], + invocation: NormalizedFunctionItem, ) -> tuple[ModuleBlock, ...]: blocks = tuple( replace( @@ -1491,9 +1531,11 @@ def finalize_module_blocks( cellprofiler_setting_literal(self.cellprofiler_literals[1]), ), *( - record for record in block.iter_settings() + record + for record in block.iter_settings() if not setting_name_matches( - record.name, module_type.classification_decision_count_setting + record.name, + module_type.classification_decision_count_setting, ) ), ], @@ -1949,7 +1991,6 @@ class ObjectClassificationBackendStrategy( ): """Object classification primitives keyed by memory type/provider.""" - @abstractmethod def positive_label_ids(self, labels: np.ndarray) -> np.ndarray: """Return positive label ids present in ``labels``.""" @@ -2106,8 +2147,10 @@ def classify_objects_single_measurement( measurement_values=rule_values, ).classify(image, labels, backend) classified_images.append( - image.with_pixels(classification_rgb_image(classified_labels), - metadata=classified_image_metadata,) + image.with_pixels( + classification_rgb_image(classified_labels), + metadata=classified_image_metadata, + ) ) results.append(result) configured_output_indices = tuple( @@ -2154,8 +2197,10 @@ def classify_objects_single_measurement( f"{configured_output_indices!r}." ) output = ( - image.with_pixels(classification_rgb_image(classified_labels), - metadata=classified_image_metadata,) + image.with_pixels( + classification_rgb_image(classified_labels), + metadata=classified_image_metadata, + ) if classified_image_rule_indices else labels ) diff --git a/openhcs/processing/backends/cellprofiler/colocalization.py b/openhcs/processing/backends/cellprofiler/colocalization.py index ad162a63f1..17fd94d66a 100644 --- a/openhcs/processing/backends/cellprofiler/colocalization.py +++ b/openhcs/processing/backends/cellprofiler/colocalization.py @@ -17,7 +17,10 @@ from openhcs.core.axes import ColourAxis, TileAxis from openhcs.constants.constants import MemoryType -from openhcs.core.aligned_image_payload import (AlignedImageStack, pack_aligned_image_outputs) +from openhcs.core.aligned_image_payload import ( + AlignedImageStack, + pack_aligned_image_outputs, +) from openhcs.core.artifacts import ( ArtifactSpecCollection, ArtifactSpecRelation, @@ -36,7 +39,14 @@ MeasurementProjectedColumnarRows, ObjectMeasurementColumnarRows, ) -from openhcs.core.pipeline.function_contracts import (composed_image_payload, object_label_input_execution_mode, required_axis_roles, resolved_callable_parameter, runtime_bound_parameters, special_inputs) +from openhcs.core.pipeline.function_contracts import ( + composed_image_payload, + object_label_input_execution_mode, + required_axis_roles, + resolved_callable_parameter, + runtime_bound_parameters, + special_inputs, +) from python_introspect import public_names_from_objects from openhcs.core.runtime_batch_contracts import ( RuntimeBatchInvocationRequest, @@ -333,7 +343,11 @@ def complete_table_measurement_rows( func = request.callable_contract.resolve_canonical_raw_callable() source_names = request.source.source_aliases channel_indexes = tuple( - int(request.kwargs.get(name, resolved_callable_parameter(func, name).default)) + int( + request.kwargs.get( + name, resolved_callable_parameter(func, name).default + ) + ) for name in ("channel_1", "channel_2") ) first_index, second_index = channel_indexes @@ -1288,7 +1302,6 @@ class ColocalizationCostesBackendStrategy( ): """Costes thresholding primitives keyed by OpenHCS memory/provider.""" - @abstractmethod def linear_costes( self, @@ -2619,9 +2632,7 @@ def measure_colocalization( time.perf_counter() - phase_started_at, function="measure_colocalization", scale_max=options.scale_max, - source_plane_intensity_scales=( - image.metadata.source_plane_intensity_scales - ), + source_plane_intensity_scales=(image.metadata.source_plane_intensity_scales), intensity_scale=image.metadata.intensity_scale, payload_type=type(image).__name__, ) @@ -3004,8 +3015,10 @@ def _colocalization_source_plane_output( "MeasureColocalization source channels require the declared " f"source-binding plane axis, got {metadata.plane_axis.value!r}." ) - source_plane = image.slice_payload(image_data[source_channel_index], - source_channel_index,) + source_plane = image.slice_payload( + image_data[source_channel_index], + source_channel_index, + ) return source_plane.with_pixels(output_data) diff --git a/openhcs/processing/backends/cellprofiler/color.py b/openhcs/processing/backends/cellprofiler/color.py index a265279158..cfd08f4497 100644 --- a/openhcs/processing/backends/cellprofiler/color.py +++ b/openhcs/processing/backends/cellprofiler/color.py @@ -110,8 +110,10 @@ def channel_output( self, image: RuntimeArrayData, *, channel_index: int, output: RuntimeArrayData ) -> RuntimeArrayData: del channel_index - return image.with_pixels(output, - metadata=image.metadata.without_axis(ColourAxis),) + return image.with_pixels( + output, + metadata=image.metadata.without_axis(ColourAxis), + ) class ImageChannelType(Enum): @@ -382,7 +384,8 @@ def _canonical_output_names( return tuple(f"{base_name}Channel{index + 1}" for index in channel_indices) declaration = cls.fixed_image_type(image_type) if any( - index < 0 or index >= len(declaration.fixed_channels) for index in channel_indices + index < 0 or index >= len(declaration.fixed_channels) + for index in channel_indices ): raise ValueError( f"ColorToGray channel indices must address {image_type.value} " @@ -872,7 +875,8 @@ def authoring_default_kwargs(cls, module, *, authored_kwargs): from openhcs.interop.cellprofiler.settings_binder import SettingsBinder inherited = super().authoring_default_kwargs( - module, authored_kwargs=authored_kwargs, + module, + authored_kwargs=authored_kwargs, ) if not any( binding.require_parameter_name() in authored_kwargs @@ -1651,13 +1655,17 @@ def color_to_gray( """ if mode is ColorToGrayMode.COMBINE: output = combine_color_to_gray(image, channel_indices, contributions) - return image.with_pixels(output, - metadata=color_to_gray_combine_output_metadata(image),) + return image.with_pixels( + output, + metadata=color_to_gray_combine_output_metadata(image), + ) outputs = split_color_to_gray(image, image_type, channel_indices) return pack_aligned_image_outputs( tuple( image_type.scalar_output_projection.channel_output( - image, channel_index=channel_index, output=output, + image, + channel_index=channel_index, + output=output, ) for channel_index, output in zip(channel_indices, outputs, strict=True) ) @@ -1736,8 +1744,10 @@ def _invert_for_printing_result( ) if output_mode is OutputMode.COLOR: outputs = ( - image.with_pixels(np.stack(inverted_channels, axis=-1), - metadata=image.metadata.with_axis(ColourSampleAxisSpec(), -1),), + image.with_pixels( + np.stack(inverted_channels, axis=-1), + metadata=image.metadata.with_axis(ColourSampleAxisSpec(), -1), + ), ) output_names = (color_output_name,) else: @@ -1753,8 +1763,10 @@ def _invert_for_printing_result( ) output_metadata = image.metadata.without_axis(ColourAxis) outputs = tuple( - image.with_pixels(output, - metadata=output_metadata,) + image.with_pixels( + output, + metadata=output_metadata, + ) for enabled, output in zip( enabled_outputs, inverted_channels, @@ -1909,22 +1921,25 @@ def combine_color_to_gray( channels = np.asarray(channel_indices, dtype=int) weights = np.asarray(contributions, dtype=float) / float(sum(contributions)) result = _indexed_weighted_projection( - color_stack, channels, weights, axis=-1, + color_stack, + channels, + weights, + axis=-1, ) return restore_color_to_gray_shape(image, result) def color_to_gray_combine_output_metadata(image: RuntimeArrayData): """Return metadata for a color-to-grayscale semantic collapse.""" - return ( - image.metadata - .without_unit_interval_intensity_scale() - .without_axis(ColourAxis) + return image.metadata.without_unit_interval_intensity_scale().without_axis( + ColourAxis ) def split_color_to_gray( - image: RuntimeArrayData, image_type: ImageChannelType, channel_indices: tuple[int, ...] + image: RuntimeArrayData, + image_type: ImageChannelType, + channel_indices: tuple[int, ...], ) -> tuple[np.ndarray, ...]: color_stack = nhwc_color_stack(image).astype(np.float32) source_stack = ( @@ -1981,7 +1996,9 @@ def nhwc_color_stack(image: RuntimeArrayData) -> np.ndarray: return channel_last -def restore_color_to_gray_shape(original: RuntimeArrayData, stack: np.ndarray) -> np.ndarray: +def restore_color_to_gray_shape( + original: RuntimeArrayData, stack: np.ndarray +) -> np.ndarray: metadata = original.metadata if metadata.declares_colour_samples_plane(original): if stack.shape[0] != 1: @@ -2084,8 +2101,10 @@ def unmix_colors( if stain_names: return pack_aligned_image_outputs( tuple( - image.with_pixels(output, - metadata=output_metadata,) + image.with_pixels( + output, + metadata=output_metadata, + ) for output in _unmix_stain_outputs( rgb_image, _stain_definitions(stain_names, custom_absorbances), @@ -2119,8 +2138,10 @@ def unmix_colors( raise ValueError( f"output_stain_index must be in [0, {len(outputs) - 1}], got {output_stain_index}." ) - return image.with_pixels(outputs[output_stain_index], - metadata=output_metadata,) + return image.with_pixels( + outputs[output_stain_index], + metadata=output_metadata, + ) def _stain_definitions( diff --git a/openhcs/processing/backends/cellprofiler/crop.py b/openhcs/processing/backends/cellprofiler/crop.py index 85312c48df..f3fc9ea606 100644 --- a/openhcs/processing/backends/cellprofiler/crop.py +++ b/openhcs/processing/backends/cellprofiler/crop.py @@ -375,10 +375,7 @@ def finalize_artifact_contract_inputs( ImageArtifactType, ) (topology_input,) = topology_inputs - if ( - shape is cls.Shape.CROPPING - and topology_input.sidecar_role is not CropMask - ): + if shape is cls.Shape.CROPPING and topology_input.sidecar_role is not CropMask: raise ValueError( f"Crop({module.module_num}) requires one exact crop-mask sidecar, " f"got {topology_input!r}." diff --git a/openhcs/processing/backends/cellprofiler/display_modules.py b/openhcs/processing/backends/cellprofiler/display_modules.py index f2b1e1399b..bcc1a2654b 100644 --- a/openhcs/processing/backends/cellprofiler/display_modules.py +++ b/openhcs/processing/backends/cellprofiler/display_modules.py @@ -562,8 +562,10 @@ def display_data_on_image( ) output = DisplayDataOnImageRenderer(request).render_slice() - return image.with_pixels(output, - metadata=image.metadata.with_axis(ColourSampleAxisSpec(), -1),) + return image.with_pixels( + output, + metadata=image.metadata.with_axis(ColourSampleAxisSpec(), -1), + ) class DensityPlotScaleType(Enum): diff --git a/openhcs/processing/backends/cellprofiler/edge.py b/openhcs/processing/backends/cellprofiler/edge.py index ef984bddc5..6a2fb75a6b 100644 --- a/openhcs/processing/backends/cellprofiler/edge.py +++ b/openhcs/processing/backends/cellprofiler/edge.py @@ -116,9 +116,7 @@ class EdgeEnhancementStrategyKey: @property def label(self) -> str: - return ( - f"{self.backend_provider.selection_name}:{self.method.value}:{self.direction.value}" - ) + return f"{self.backend_provider.selection_name}:{self.method.value}:{self.direction.value}" @dataclass(frozen=True, slots=True) @@ -553,9 +551,11 @@ def enhance_edges( output = ( EdgeEnhancementStrategy.for_request(request).enhance(request).astype(np.float32) ) - return image.with_pixels(output, + return image.with_pixels( + output, mask=operation_mask if payload_mask is not None else None, - metadata=image.metadata.without_unit_interval_intensity_scale(),) + metadata=image.metadata.without_unit_interval_intensity_scale(), + ) @njit(cache=True) diff --git a/openhcs/processing/backends/cellprofiler/feature_enhancement.py b/openhcs/processing/backends/cellprofiler/feature_enhancement.py index f7285288ba..5938165125 100644 --- a/openhcs/processing/backends/cellprofiler/feature_enhancement.py +++ b/openhcs/processing/backends/cellprofiler/feature_enhancement.py @@ -111,10 +111,8 @@ def enhance_or_suppress_features( dic_decay=dic_decay, ) ) - return ( - image.metadata - .without_unit_interval_intensity_scale() - .payload_with(np.asarray(result, dtype=np.float32), image.mask) + return image.metadata.without_unit_interval_intensity_scale().payload_with( + np.asarray(result, dtype=np.float32), image.mask ) @@ -312,7 +310,8 @@ def apply(self, request: FeatureEnhancementRequest) -> np.ndarray: enhance_or_suppress_features, backend_provider=OpencvBackendProvider, ).grayscale_opening( - masked, footprint, + masked, + footprint, ) result = masked - opened else: diff --git a/openhcs/processing/backends/cellprofiler/granularity.py b/openhcs/processing/backends/cellprofiler/granularity.py index 08fd8dbe60..6d4c0b9a21 100644 --- a/openhcs/processing/backends/cellprofiler/granularity.py +++ b/openhcs/processing/backends/cellprofiler/granularity.py @@ -712,7 +712,6 @@ class GranularityReconstructionBackendStrategy( ): """Exact grayscale reconstruction backend used by MeasureGranularity.""" - @abstractmethod def reconstruct_radius_one( self, diff --git a/openhcs/processing/backends/cellprofiler/illumination.py b/openhcs/processing/backends/cellprofiler/illumination.py index 14c6725019..2b2f16f562 100644 --- a/openhcs/processing/backends/cellprofiler/illumination.py +++ b/openhcs/processing/backends/cellprofiler/illumination.py @@ -15,7 +15,7 @@ from numba import njit from openhcs.constants.constants import MemoryType -from openhcs.core.aligned_image_payload import (AlignedImageStack) +from openhcs.core.aligned_image_payload import AlignedImageStack from openhcs.core.image_payload_execution_mode import ImagePayloadExecutionMode from openhcs.core.artifacts import ( ArtifactSpec, @@ -1086,12 +1086,8 @@ def correct_illumination_apply( if truncate_high: np.minimum(output_pixels, 1.0, out=output_pixels) mask = image.mask - return ( - image.metadata - .without_unit_interval_intensity_scale() - .payload_with( - output_pixels, None if mask is None else np.asarray(mask, dtype=bool) - ) + return image.metadata.without_unit_interval_intensity_scale().payload_with( + output_pixels, None if mask is None else np.asarray(mask, dtype=bool) ) @@ -1409,7 +1405,6 @@ class ConvexHullSmoothingBackendStrategy( ): """Convex-hull illumination smoothing keyed by OpenHCS memory/provider.""" - @abstractmethod def smooth_background_plane( self, @@ -1427,7 +1422,6 @@ class RankMedianSmoothingBackendStrategy( ): """Rank-median illumination smoothing keyed by OpenHCS memory/provider.""" - @staticmethod def disk_rows(footprint: np.ndarray) -> tuple[np.ndarray, np.ndarray]: """Collapse a dense disk footprint into per-row horizontal radii.""" diff --git a/openhcs/processing/backends/cellprofiler/image_geometry.py b/openhcs/processing/backends/cellprofiler/image_geometry.py index 8ea5a1a9d3..3a8b68163f 100644 --- a/openhcs/processing/backends/cellprofiler/image_geometry.py +++ b/openhcs/processing/backends/cellprofiler/image_geometry.py @@ -1120,9 +1120,9 @@ def mask_image( output_mask = restore_image_mask_planes( image.data, tuple((result[1] for result in masked_plane_results)) ) - return replace( - image.metadata, mask_defines_border=True - ).payload_with(masked_data, output_mask) + return replace(image.metadata, mask_defines_border=True).payload_with( + masked_data, output_mask + ) def masked_image_plane( @@ -1141,7 +1141,8 @@ def masked_image_plane( existing_mask = image.mask if existing_mask is not None: existing_projected_mask = bare_image.mask_for_data( - image_data, mask=existing_mask, + image_data, + mask=existing_mask, ) if existing_projected_mask is not None: binary_mask = binary_mask & np.asarray(existing_projected_mask, dtype=bool) @@ -1396,9 +1397,9 @@ def flip_and_rotate( max_j = max_area_idx[1] + 1 pixel_data = pixel_data[min_i:max_i, min_j:max_j] return ( - image.metadata - .with_spatial_resize(pixel_data.shape[:2]) - .payload_with(pixel_data.astype(np.float32), None), + image.metadata.with_spatial_resize(pixel_data.shape[:2]).payload_with( + pixel_data.astype(np.float32), None + ), DataclassMeasurementColumnarRows( (RotationResult(slice_index=0, rotation_angle=angle),), row_type=RotationResult, diff --git a/openhcs/processing/backends/cellprofiler/image_math.py b/openhcs/processing/backends/cellprofiler/image_math.py index 2502a28cf9..5036ad721c 100644 --- a/openhcs/processing/backends/cellprofiler/image_math.py +++ b/openhcs/processing/backends/cellprofiler/image_math.py @@ -813,9 +813,7 @@ def from_inputs( f"ImageMath {operation_strategy.operation.value!r} requires exactly " f"one declared image operand, got {len(source_payloads)}." ) - operand_pixels = tuple( - np.asarray(payload.data) for payload in source_payloads - ) + operand_pixels = tuple(np.asarray(payload.data) for payload in source_payloads) return cls( source_image=image, source_payloads=source_payloads, diff --git a/openhcs/processing/backends/cellprofiler/image_quality.py b/openhcs/processing/backends/cellprofiler/image_quality.py index c7cca642a4..d8a208d7b5 100644 --- a/openhcs/processing/backends/cellprofiler/image_quality.py +++ b/openhcs/processing/backends/cellprofiler/image_quality.py @@ -772,7 +772,6 @@ class ImageQualityBackendStrategy( ): """Image-quality primitives keyed by OpenHCS memory type/provider.""" - @abstractmethod def haralick_h3(self, image: np.ndarray, *, scale: int) -> float: """Return CP-style Haralick H3 correlation for one image plane.""" diff --git a/openhcs/processing/backends/cellprofiler/intensity.py b/openhcs/processing/backends/cellprofiler/intensity.py index 194642350a..4162fbdf10 100644 --- a/openhcs/processing/backends/cellprofiler/intensity.py +++ b/openhcs/processing/backends/cellprofiler/intensity.py @@ -276,7 +276,11 @@ def active_artifact_bindings( from openhcs.constants.constants import MemoryType from openhcs.core.callable_contract import runtime_image_execution_mode from openhcs.core.memory import numpy as numpy_decorator -from openhcs.core.pipeline.function_contracts import (object_label_input_execution_mode, runtime_bound_parameters, special_inputs) +from openhcs.core.pipeline.function_contracts import ( + object_label_input_execution_mode, + runtime_bound_parameters, + special_inputs, +) from openhcs.core.runtime_batch_contracts import ( RuntimePure2DSliceBatchRequest, SliceIndexRuntimeParameter, @@ -415,9 +419,7 @@ class RescaleIntensityModule(CellProfilerModule): confidence = 1.0 setting_bindings = ( SettingToKeywordBinding.input("Select the input image", ImageArtifactType), - ModuleOnlySettingBinding( - "Select image to match in maximum intensity" - ), + ModuleOnlySettingBinding("Select image to match in maximum intensity"), SettingToKeywordBinding.output("Name the output image", ImageArtifactType), SettingToKeywordBinding( "Rescaling method", @@ -905,9 +907,7 @@ def from_function_arguments( ) -> "ObjectIntensityMeasurementContext": return cls( labels=labels, - backend_provider=CellProfilerBackendSelection.from_input( - backend_provider - ), + backend_provider=CellProfilerBackendSelection.from_input(backend_provider), slice_index=int(slice_index), object_domain=object_domain, object_row_identity=object_row_identity, @@ -1026,7 +1026,6 @@ class ObjectIntensityBackendStrategy( ): """Object-intensity operations keyed by OpenHCS memory type/provider.""" - @abstractmethod def measure( self, image: np.ndarray, labels: ObjectIntensityLabelInput @@ -1456,9 +1455,7 @@ def measure_object_intensity_measurement_image_batch( first_request ).backend_provider ) - images = tuple( - (np.asarray(request.image.data) for _index, request in group) - ) + images = tuple((np.asarray(request.image.data) for _index, request in group)) batch_started_at = time.perf_counter() measurement_batches = backend.measure_prepared_batch(images, prepared_labels) RuntimeProfileLogger.log( diff --git a/openhcs/processing/backends/cellprofiler/intensity_distribution.py b/openhcs/processing/backends/cellprofiler/intensity_distribution.py index 423237e5b0..2e5f741df7 100644 --- a/openhcs/processing/backends/cellprofiler/intensity_distribution.py +++ b/openhcs/processing/backends/cellprofiler/intensity_distribution.py @@ -37,10 +37,16 @@ ObjectMeasurementColumnarRows, ) from openhcs.core.memory.decorators import numpy -from openhcs.core.pipeline.function_contracts import (object_label_input_execution_mode, runtime_bound_parameters, special_inputs) +from openhcs.core.pipeline.function_contracts import ( + object_label_input_execution_mode, + runtime_bound_parameters, + special_inputs, +) from python_introspect import public_names_from_objects from openhcs.core.runtime_identifier import normalize_runtime_identifier -from openhcs.core.measurement_dialect import RuntimeMeasurementQualifierSuffixMatchStrategy +from openhcs.core.measurement_dialect import ( + RuntimeMeasurementQualifierSuffixMatchStrategy, +) from metaclass_registry.strategies import enum_member_with_payload from openhcs.core.runtime_array_values import RuntimeArrayData from openhcs.core.runtime_batch_contracts import SliceIndexRuntimeParameter @@ -959,9 +965,7 @@ def _runtime_heatmap_outputs( @dataclass class RadialLabelGeometryCache( - ProcessLocalBoundedCache[ - "RadialLabelGeometryCacheKey", "RadialLabelGeometry" - ] + ProcessLocalBoundedCache["RadialLabelGeometryCacheKey", "RadialLabelGeometry"] ): """Process-local numerical geometry with shared bounded storage.""" @@ -2632,8 +2636,10 @@ def _intensity_distribution_heatmap( heatmap[rows, columns] = values colormap_name = group.colormap if colormap_name == "gray": - return image.with_pixels(heatmap, - metadata=image.metadata.without_axis(ColourAxis),) + return image.with_pixels( + heatmap, + metadata=image.metadata.without_axis(ColourAxis), + ) if colormap_name == "Default": colormap_name = "viridis" import matplotlib @@ -2643,8 +2649,10 @@ def _intensity_distribution_heatmap( :, :, :3 ] rgb[labels == 0] = 0 - return image.with_pixels(rgb, - metadata=image.metadata.with_axis(ColourSampleAxisSpec(), -1),) + return image.with_pixels( + rgb, + metadata=image.metadata.with_axis(ColourSampleAxisSpec(), -1), + ) def intensity_distribution_object_domain(labels: object) -> tuple[int, ...]: diff --git a/openhcs/processing/backends/cellprofiler/median_filter.py b/openhcs/processing/backends/cellprofiler/median_filter.py index 390768256b..f160506ef0 100644 --- a/openhcs/processing/backends/cellprofiler/median_filter.py +++ b/openhcs/processing/backends/cellprofiler/median_filter.py @@ -129,7 +129,6 @@ class MedianFilterBackendStrategy( ): """Median filtering operations keyed by OpenHCS memory type/provider.""" - @abstractmethod def filter( self, @@ -292,8 +291,10 @@ def selection_network_filter( return None radius = int(window_size) // 2 padded = np.pad( - image, ((radius, radius), (radius, radius), (radius, radius + 7)), - mode="constant", constant_values=0, + image, + ((radius, radius), (radius, radius), (radius, radius + 7)), + mode="constant", + constant_values=0, ) depth, height, width = image.shape output_width = ((width + 7) // 8) * 8 diff --git a/openhcs/processing/backends/cellprofiler/morphology.py b/openhcs/processing/backends/cellprofiler/morphology.py index c83fc7c575..b346e30d1a 100644 --- a/openhcs/processing/backends/cellprofiler/morphology.py +++ b/openhcs/processing/backends/cellprofiler/morphology.py @@ -683,7 +683,10 @@ def artifact_output_relations( from openhcs.core.measurement_row_materialization import ( DataclassMeasurementColumnarRows, ) -from openhcs.core.pipeline.function_contracts import (object_label_input_execution_mode, special_inputs) +from openhcs.core.pipeline.function_contracts import ( + object_label_input_execution_mode, + special_inputs, +) from python_introspect import public_names_from_objects from metaclass_registry.strategies import EnumKeyedStrategyMixin from openhcs.processing.backends.cellprofiler.morphology_connected_components_numba import ( @@ -1517,8 +1520,10 @@ def _morph_image_payload( image = ImagePayload.of(image) pixel_data = image.data result = _morph_image_pixels(pixel_data, structuring_element, size, operation) - return image.with_pixels(result.astype(pixel_data.dtype, copy=False), - metadata=image.metadata.without_unit_interval_intensity_scale(),) + return image.with_pixels( + result.astype(pixel_data.dtype, copy=False), + metadata=image.metadata.without_unit_interval_intensity_scale(), + ) @runtime_image_execution_mode(FullStackExecution) @@ -1880,7 +1885,6 @@ class MorphologyBackendStrategy( ): """Nominal morphology operations keyed by OpenHCS memory type.""" - @classmethod def for_memory_type( cls, @@ -2113,7 +2117,9 @@ def grayscale_closing(self, image: np.ndarray, footprint: np.ndarray) -> np.ndar return self._grayscale_morphology(image, footprint, first_pass_is_dilation=True) def grayscale_opening(self, image: np.ndarray, footprint: np.ndarray) -> np.ndarray: - return self._grayscale_morphology(image, footprint, first_pass_is_dilation=False) + return self._grayscale_morphology( + image, footprint, first_pass_is_dilation=False + ) def _grayscale_morphology( self, @@ -2346,7 +2352,8 @@ def prepare_backend(self) -> None: for writable_mask in (False, True): smoothing_mask = ( np.ones(mask.shape, dtype=np.bool_) - if full_mask else mask.copy() + if full_mask + else mask.copy() ) smoothing_mask.flags.writeable = writable_mask self.smooth_image_for_declumping(image, smoothing_mask, 1.0) @@ -3070,10 +3077,7 @@ def _scipy_connected_components( @njit(cache=True, inline="always") def _labeled_hole_node_at_numba(source, background, offset, y, x): background_node = background[y, x] - return ( - source[y, x] - if background_node == 0 else background_node + offset + 1 - ) + return source[y, x] if background_node == 0 else background_node + offset + 1 @njit(cache=True) @@ -3089,11 +3093,15 @@ def _labeled_hole_graph_numba(source, background, offset, node_count): if node != 0 and (y == 0 or y + 1 == height or x == 0 or x + 1 == width): boundary_nodes.add(node) if y + 1 < height: - other = _labeled_hole_node_at_numba(source, background, offset, y + 1, x) + other = _labeled_hole_node_at_numba( + source, background, offset, y + 1, x + ) if node != other: edges.add(min(node, other) * node_count + max(node, other)) if x + 1 < width: - other = _labeled_hole_node_at_numba(source, background, offset, y, x + 1) + other = _labeled_hole_node_at_numba( + source, background, offset, y, x + 1 + ) if node != other: edges.add(min(node, other) * node_count + max(node, other)) encoded = np.empty(len(edges), dtype=np.int64) @@ -3237,11 +3245,16 @@ def _cellprofiler_fill_labeled_holes_2d( np.empty(captured_source.shape)[mask_array] output_mask = ( np.empty((0, 0), dtype=np.bool_) - if mask_array is None else np.ascontiguousarray(mask_array) + if mask_array is None + else np.ascontiguousarray(mask_array) ) output = _write_labeled_holes_numba( - captured_source, background_labels, foreground_label_count, lookup, - output_mask, mask_array is not None, + captured_source, + background_labels, + foreground_label_count, + lookup, + output_mask, + mask_array is not None, ) return output.astype(source.dtype, copy=False) @@ -3658,9 +3671,7 @@ def horizontal_spans(self) -> tuple[tuple[int, int, int], ...]: spans.append((y, x, x)) return tuple(spans) - def grayscale_extremum( - self, image: np.ndarray, *, maximum: bool - ) -> np.ndarray: + def grayscale_extremum(self, image: np.ndarray, *, maximum: bool) -> np.ndarray: """Reduce the exact union of spans, sharing each horizontal window.""" from scipy.ndimage import maximum_filter1d, minimum_filter1d @@ -6610,9 +6621,7 @@ def erode_objects( runtime_profiler.log( "erode_objects_lineage", time.perf_counter() - phase_started_at ) - runtime_profiler.log( - "erode_objects_total", time.perf_counter() - total_started_at - ) + runtime_profiler.log("erode_objects_total", time.perf_counter() - total_started_at) return ( image, DataclassMeasurementColumnarRows((stats,), row_type=ErosionStats), diff --git a/openhcs/processing/backends/cellprofiler/neighbors.py b/openhcs/processing/backends/cellprofiler/neighbors.py index b73c9a65dd..069c7b57fb 100644 --- a/openhcs/processing/backends/cellprofiler/neighbors.py +++ b/openhcs/processing/backends/cellprofiler/neighbors.py @@ -322,17 +322,21 @@ def retained_images( output_metadata = bare_image.metadata.with_axis(ColourSampleAxisSpec(), -1) if self.retain_neighbor_count_image: retained.append( - bare_image.with_pixels(self.colored_metric_image( + bare_image.with_pixels( + self.colored_metric_image( neighbor_count_image, self.neighbor_count_colormap ), - metadata=output_metadata,) + metadata=output_metadata, + ) ) if self.retain_percent_touching_image: retained.append( - bare_image.with_pixels(self.colored_metric_image( + bare_image.with_pixels( + self.colored_metric_image( percent_touching_image, self.percent_touching_colormap ), - metadata=output_metadata,) + metadata=output_metadata, + ) ) return tuple(retained) @@ -444,7 +448,6 @@ class NeighborTopologyBackendStrategy( ): """Neighbor topology operations keyed by OpenHCS memory type/provider.""" - @abstractmethod def measure_topology( self, diff --git a/openhcs/processing/backends/cellprofiler/object_images.py b/openhcs/processing/backends/cellprofiler/object_images.py index 251654920e..640abba3c4 100644 --- a/openhcs/processing/backends/cellprofiler/object_images.py +++ b/openhcs/processing/backends/cellprofiler/object_images.py @@ -18,7 +18,10 @@ from openhcs.core.measurement_row_materialization import ( DataclassMeasurementColumnarRows, ) -from openhcs.core.pipeline.function_contracts import (object_label_input_execution_mode, special_inputs) +from openhcs.core.pipeline.function_contracts import ( + object_label_input_execution_mode, + special_inputs, +) from python_introspect import public_names_from_objects from openhcs.core.processing_preparation import PersistentNumbaKernelPreparation from metaclass_registry.strategies import EnumKeyedStrategyMixin @@ -382,8 +385,10 @@ def convert_objects_to_image( if image_mode is ImageMode.UINT16 else label_metadata ) - return labels.with_pixels(rendered, - metadata=output_metadata,) + return labels.with_pixels( + rendered, + metadata=output_metadata, + ) class ConvertImageToObjectsModule( diff --git a/openhcs/processing/backends/cellprofiler/object_overlap.py b/openhcs/processing/backends/cellprofiler/object_overlap.py index 3a22b935e1..667ad0bffd 100644 --- a/openhcs/processing/backends/cellprofiler/object_overlap.py +++ b/openhcs/processing/backends/cellprofiler/object_overlap.py @@ -15,7 +15,10 @@ DataclassMeasurementColumnarRows, MeasurementProjectedColumnarRows, ) -from openhcs.core.pipeline.function_contracts import (object_label_input_execution_mode, special_inputs) +from openhcs.core.pipeline.function_contracts import ( + object_label_input_execution_mode, + special_inputs, +) from python_introspect import public_names_from_objects from openhcs.core.runtime_tabular_values import ( FieldSpec, diff --git a/openhcs/processing/backends/cellprofiler/outlines.py b/openhcs/processing/backends/cellprofiler/outlines.py index bb1429870b..f2694178ed 100644 --- a/openhcs/processing/backends/cellprofiler/outlines.py +++ b/openhcs/processing/backends/cellprofiler/outlines.py @@ -37,7 +37,10 @@ from openhcs.core.runtime_array_values import RuntimeArrayData from openhcs.core.steps.function_runtime import RuntimeCallableKwargs from openhcs.core.memory.decorators import numpy -from openhcs.core.pipeline.function_contracts import (object_label_input_execution_mode, special_inputs) +from openhcs.core.pipeline.function_contracts import ( + object_label_input_execution_mode, + special_inputs, +) from python_introspect import public_names_from_objects from openhcs.core.runtime_object_labels import ( ObjectLabelValue, @@ -690,7 +693,6 @@ class ObjectOutlineBackendStrategy( ): """Object outline operations keyed by OpenHCS memory type/provider.""" - @abstractmethod def outline(self, labels: np.ndarray) -> np.ndarray: """Return a labeled inner outline image.""" @@ -904,13 +906,15 @@ def overlay_outlines( image, blank_image=context.blank_image, image_row_count=context.image_row_count ) output = context.render(image_sources) - return image.with_pixels(output, + return image.with_pixels( + output, metadata=replace( image.metadata, axes=PayloadAxes.colour_samples( -1 if context.display_mode is OutlineDisplayMode.COLOR else None ), - ),) + ), + ) @numpy(contract=FlexibleContract) @@ -955,8 +959,10 @@ def overlay_objects( seed=seed, colormap_value=colormap, ) - return image.with_pixels(overlay, - metadata=image.metadata.with_axis(ColourSampleAxisSpec(), -1),) + return image.with_pixels( + overlay, + metadata=image.metadata.with_axis(ColourSampleAxisSpec(), -1), + ) def _runtime_rows( @@ -1060,7 +1066,8 @@ def _draw_object_labels( else: outline_color = outline_intensity for label_plane in ObjectLabelStorageStrategy.for_value(labels).rendering_layers( - labels, source_spatial_shape_yx=labels.source_spatial_shape_yx, + labels, + source_spatial_shape_yx=labels.source_spatial_shape_yx, ): if label_plane.ndim != 2: raise ValueError( @@ -1103,9 +1110,7 @@ def _draw_outline_image( def _outline_image_mask(outline_image: RuntimeArrayData) -> np.ndarray: outline_image = ImagePayload.of(outline_image) mask = np.asarray(outline_image.data) > 0 - channel_axis = outline_image.metadata.axis_index(ColourAxis, - outline_image - ) + channel_axis = outline_image.metadata.axis_index(ColourAxis, outline_image) if channel_axis is not None: return np.any(mask, axis=channel_axis) return mask diff --git a/openhcs/processing/backends/cellprofiler/primary_object_diagnostics.py b/openhcs/processing/backends/cellprofiler/primary_object_diagnostics.py index 8c59ecf5b3..54babcdad1 100644 --- a/openhcs/processing/backends/cellprofiler/primary_object_diagnostics.py +++ b/openhcs/processing/backends/cellprofiler/primary_object_diagnostics.py @@ -19,7 +19,9 @@ ImageArtifactType, SourceStackLineageSourceRelation, ) -from openhcs.core.measurement_row_materialization import DataclassMeasurementColumnarRows +from openhcs.core.measurement_row_materialization import ( + DataclassMeasurementColumnarRows, +) from openhcs.core.projected_image_output import SelectedPlaneImageOutput from openhcs.core.runtime_image_values import ( ImagePayloadMetadata, @@ -36,6 +38,7 @@ MaterializedFilenameIdentity, ) + @dataclass(frozen=True, slots=True) class DiagnosticPlaneSource: """Decode source identity and validity once, sharing one mask across stages.""" @@ -48,8 +51,11 @@ def from_image(cls, image: object) -> "DiagnosticPlaneSource": source_mask = image.mask return cls( image.metadata, - np.ones(np.asarray(image.data).shape, dtype=bool) - if source_mask is None else np.asarray(source_mask, dtype=bool), + ( + np.ones(np.asarray(image.data).shape, dtype=bool) + if source_mask is None + else np.asarray(source_mask, dtype=bool) + ), ) def plane( @@ -138,7 +144,9 @@ def planes( shape = source.validity_mask.shape invalid = np.zeros(shape, dtype=bool) return ( - source.plane(np.full(shape, np.nan, dtype=np.float32), validity_mask=invalid), + source.plane( + np.full(shape, np.nan, dtype=np.float32), validity_mask=invalid + ), source.plane(np.zeros(shape, dtype=bool), validity_mask=invalid), source.plane(np.zeros(shape, dtype=np.int32), validity_mask=invalid), ) diff --git a/openhcs/processing/backends/cellprofiler/primary_objects.py b/openhcs/processing/backends/cellprofiler/primary_objects.py index b5db53409d..a9b13ed721 100644 --- a/openhcs/processing/backends/cellprofiler/primary_objects.py +++ b/openhcs/processing/backends/cellprofiler/primary_objects.py @@ -261,9 +261,7 @@ def identify_primary_objects( else np.asarray(input_mask_payload, dtype=bool) ) input_metadata = ( - image.metadata - if respect_source_border_metadata - else ImagePayloadMetadata() + image.metadata if respect_source_border_metadata else ImagePayloadMetadata() ) raw_image_data = np.asarray(image.data) proven_unit_interval_scale = unit_interval_scale_for_threshold_selection( @@ -666,10 +664,16 @@ def finalize_artifact_contract_outputs( artifact_inputs.specs, ) if len(source) != 1: - raise ValueError("IdentifyPrimaryObjects diagnostics require one source image.") - objects = ArtifactSpecCollection(outputs).of_artifact_type(ObjectLabelsArtifactType) + raise ValueError( + "IdentifyPrimaryObjects diagnostics require one source image." + ) + objects = ArtifactSpecCollection(outputs).of_artifact_type( + ObjectLabelsArtifactType + ) if len(objects) != 1: - raise ValueError("IdentifyPrimaryObjects diagnostics require one object output.") + raise ValueError( + "IdentifyPrimaryObjects diagnostics require one object output." + ) return ( *outputs, *PrimaryObjectDiagnosticPlanes.artifact_specs( diff --git a/openhcs/processing/backends/cellprofiler/relationships.py b/openhcs/processing/backends/cellprofiler/relationships.py index 85df875ef8..99a379ab58 100644 --- a/openhcs/processing/backends/cellprofiler/relationships.py +++ b/openhcs/processing/backends/cellprofiler/relationships.py @@ -881,7 +881,6 @@ class ObjectRelationshipBackendStrategy( ): """Object relationship operations keyed by OpenHCS memory type/provider.""" - @abstractmethod def relate_children_to_parents( self, parent_labels: np.ndarray, child_labels: np.ndarray, child_count: int @@ -1622,7 +1621,9 @@ def upstream_child_feature_indexes( identity_policy = ( self.request.adapter.request.context.source_image_set_identity_policy ) - child_axis = child_labels.metadata.source_provenance.image_set_axis(identity_policy) + child_axis = child_labels.metadata.source_provenance.image_set_axis( + identity_policy + ) if not child_axis: raise ValueError( f"RelateObjects child measurements for {child_spec.name!r} require " diff --git a/openhcs/processing/backends/cellprofiler/secondary.py b/openhcs/processing/backends/cellprofiler/secondary.py index 7db1c7723e..a445b3f059 100644 --- a/openhcs/processing/backends/cellprofiler/secondary.py +++ b/openhcs/processing/backends/cellprofiler/secondary.py @@ -535,7 +535,6 @@ class SecondaryDistanceTransformBackendStrategy( ): """Distance transform operations used by secondary segmentation.""" - @abstractmethod def distance_to_foreground(self, labels: np.ndarray) -> np.ndarray: """Return Euclidean distance to the nearest positive label.""" @@ -616,7 +615,6 @@ class SecondaryPropagationBackendStrategy( ): """Regularized label propagation backend for secondary segmentation.""" - @abstractmethod def propagate_result( self, diff --git a/openhcs/processing/backends/cellprofiler/shape.py b/openhcs/processing/backends/cellprofiler/shape.py index 419691aafb..c7b963bf06 100644 --- a/openhcs/processing/backends/cellprofiler/shape.py +++ b/openhcs/processing/backends/cellprofiler/shape.py @@ -457,7 +457,10 @@ def postprocess_bound_settings( from numba import njit from openhcs.constants.constants import MemoryType from openhcs.core.memory import numpy as numpy_decorator -from openhcs.core.pipeline.function_contracts import (object_label_input_execution_mode, special_inputs) +from openhcs.core.pipeline.function_contracts import ( + object_label_input_execution_mode, + special_inputs, +) from openhcs.core.runtime_object_label_domains import ( ObjectLabelDomain, dense_object_label_measurement_row_domain, @@ -525,20 +528,20 @@ def feature_array_domain(self, feature_name: str) -> ObjectFeatureArrayDomain: f"{type(self).__name__} feature {feature_name!r} has no declared " "feature-array domain." ) - if ( - ShapeObjectZernikeDescriptorDeclaration.from_feature_name(feature_name) - is not None - or any( - feature.value == feature_name - and LabelIndexedShapeFeature.matches_feature(feature) - for feature in MeasureObjectSizeShapeModule.MeasurementFeature - ) + if ShapeObjectZernikeDescriptorDeclaration.from_feature_name( + feature_name + ) is not None or any( + feature.value == feature_name + and LabelIndexedShapeFeature.matches_feature(feature) + for feature in MeasureObjectSizeShapeModule.MeasurementFeature ): return ObjectFeatureArrayDomain.MEASURED_OBJECT_ID return ObjectFeatureArrayDomain.ROW_ORDINAL def feature_missing_values_for_objects( - self, feature_name: str, object_ids: tuple[int, ...], + self, + feature_name: str, + object_ids: tuple[int, ...], ) -> Mapping[int, ObjectFeatureMissingValue]: """Native radius/Feret vectors contain zeros within their material extent.""" array_domain = self.feature_array_domain(feature_name) @@ -1261,7 +1264,6 @@ class ShapeMeasurementBackendStrategy( ): """Shape-measurement operations keyed by OpenHCS memory type/provider.""" - @abstractmethod def form_factor_values( self, labels: np.ndarray, label_ids: np.ndarray diff --git a/openhcs/processing/backends/cellprofiler/skeleton.py b/openhcs/processing/backends/cellprofiler/skeleton.py index 8f081e5e37..0a60c676c6 100644 --- a/openhcs/processing/backends/cellprofiler/skeleton.py +++ b/openhcs/processing/backends/cellprofiler/skeleton.py @@ -378,8 +378,10 @@ def measure_object_skeleton_with_branchpoint_image( fill_small_holes=fill_small_holes, maximum_hole_size=maximum_hole_size, ) - branchpoint_payload = image.with_pixels(result.branchpoint_image, - metadata=image.metadata.with_axis(ColourSampleAxisSpec(), -1),) + branchpoint_payload = image.with_pixels( + result.branchpoint_image, + metadata=image.metadata.with_axis(ColourSampleAxisSpec(), -1), + ) branchpoint_image = pack_aligned_image_outputs( (branchpoint_payload,), slice_contexts=( diff --git a/openhcs/processing/backends/cellprofiler/smoothing.py b/openhcs/processing/backends/cellprofiler/smoothing.py index ab7d4811c9..6e2530b82f 100644 --- a/openhcs/processing/backends/cellprofiler/smoothing.py +++ b/openhcs/processing/backends/cellprofiler/smoothing.py @@ -396,7 +396,9 @@ def _normalize_full_support( if width <= kernel.size: np.divide(filtered, weights, out=filtered) else: - np.divide(filtered[:, :radius], weights[:, :radius], out=filtered[:, :radius]) + np.divide( + filtered[:, :radius], weights[:, :radius], out=filtered[:, :radius] + ) np.divide( filtered[:, radius : width - radius], weights[:, radius : radius + 1], @@ -871,10 +873,8 @@ def smooth_image( clip_polynomial=bool(clip_polynomial), ) output = SmoothingStrategy.for_request(request).smooth(request) - return ( - image.metadata - .without_unit_interval_intensity_scale() - .payload_with(np.asarray(output, dtype=np.float32), mask) + return image.metadata.without_unit_interval_intensity_scale().payload_with( + np.asarray(output, dtype=np.float32), mask ) @@ -931,10 +931,7 @@ def smooth_batch(request: RuntimePure2DSliceBatchRequest) -> list[Any]: smoothing_method = kwargs.get("smoothing_method", SmoothingMethod.GAUSSIAN_FILTER) pixel_stack = np.ascontiguousarray( np.stack( - [ - np.asarray(slice_2d.data, dtype=np.float32) - for slice_2d in slices_2d - ], + [np.asarray(slice_2d.data, dtype=np.float32) for slice_2d in slices_2d], axis=0, ) ) @@ -974,9 +971,9 @@ def smooth_batch(request: RuntimePure2DSliceBatchRequest) -> list[Any]: pixel_stack, mask_stack, float(selection_request.sigma) ).astype(np.float32, copy=False) return [ - slice_2d.metadata - .without_unit_interval_intensity_scale() - .payload_with(output_stack[slice_index], masks[slice_index]) + slice_2d.metadata.without_unit_interval_intensity_scale().payload_with( + output_stack[slice_index], masks[slice_index] + ) for slice_index, slice_2d in enumerate(slices_2d) ] diff --git a/openhcs/processing/backends/cellprofiler/spreadsheet_export.py b/openhcs/processing/backends/cellprofiler/spreadsheet_export.py index 2d6dbbf15b..53692766a7 100644 --- a/openhcs/processing/backends/cellprofiler/spreadsheet_export.py +++ b/openhcs/processing/backends/cellprofiler/spreadsheet_export.py @@ -109,7 +109,10 @@ WriteMode, ) -from openhcs.processing.materialization.core import ColumnarCsvOutput, RenderedColumnarCsvOutput +from openhcs.processing.materialization.core import ( + ColumnarCsvOutput, + RenderedColumnarCsvOutput, +) if TYPE_CHECKING: from openhcs.core.function_patterns import FunctionInvocationKey @@ -226,7 +229,9 @@ class SpreadsheetFileSelection: file_name: str relationship_subject_name: ClassVar[str] = "Object relationships" - experiment_subject_name: ClassVar[str] = CELLPROFILER_MEASUREMENT_DIALECT.scope_name(MeasurementScope.RUN) + experiment_subject_name: ClassVar[str] = ( + CELLPROFILER_MEASUREMENT_DIALECT.scope_name(MeasurementScope.RUN) + ) @classmethod def for_subject( @@ -285,7 +290,9 @@ def reference_mean_subjects( references = tuple( name for name in (() if image_rows is None else image_rows.columns) - if CellProfilerMeasurementDialect.row_identity_contract.is_aggregate_sample_number_reference(name) + if CellProfilerMeasurementDialect.row_identity_contract.is_aggregate_sample_number_reference( + name + ) ) return frozenset( subject @@ -407,7 +414,10 @@ def combined_rows( ) def header_rows( - self, rows: ColumnarRows, *, active_subjects: tuple[str, ...], + self, + rows: ColumnarRows, + *, + active_subjects: tuple[str, ...], ) -> tuple[tuple[str, ...], ...]: """Derive native headers in their existing first-present field order.""" @@ -492,7 +502,9 @@ def render_csv( ) -> str: """Realized view of this selection's canonical typed CSV output.""" output = self.prepare_csv( - rows, active_subjects=active_subjects, delimiter=delimiter, + rows, + active_subjects=active_subjects, + delimiter=delimiter, nan_representation=nan_representation, ) return output.options.render(output.content) @@ -508,7 +520,8 @@ class CellProfilerSpreadsheetCsvOptions(CsvOptions): nan_representation: SpreadsheetNanRepresentation def csv_schema( - self, rows: ColumnarRows, + self, + rows: ColumnarRows, ) -> tuple[tuple[str, ...], tuple[tuple[str, ...], ...]]: columns, headers = self.selection.csv_schema( rows, @@ -731,7 +744,8 @@ def prepare_spreadsheet_bundle( def render_spreadsheet_bundle( - artifact_batch: RuntimeArtifactBatch, **kwargs: object, + artifact_batch: RuntimeArtifactBatch, + **kwargs: object, ) -> dict[str, str | bytes]: """Realized view of the canonical schema-bearing spreadsheet bundle.""" return { @@ -987,7 +1001,8 @@ def _source_metadata_measurement_rows( ] = [] dialect = CellProfilerDatabaseColumnDialect() image_subject = MeasurementSubject( - MeasurementScope.SAMPLE, CELLPROFILER_MEASUREMENT_DIALECT.scope_name(MeasurementScope.SAMPLE) + MeasurementScope.SAMPLE, + CELLPROFILER_MEASUREMENT_DIALECT.scope_name(MeasurementScope.SAMPLE), ) for slice_index in image_numbers_by_slice: provenance = table.source_provenance.for_source_plane(slice_index) @@ -1003,8 +1018,11 @@ def _source_metadata_measurement_rows( acquisition.update( dialect.source_metadata_values( None, - Path(provenance.source_path) - if provenance.source_path is not None else None, + ( + Path(provenance.source_path) + if provenance.source_path is not None + else None + ), ) ) file_values: dict[str, str] = {} @@ -1028,7 +1046,12 @@ def _source_metadata_measurement_rows( ) if metadata or acquisition or file_values: rows.append( - (image_numbers_by_slice[slice_index], metadata, acquisition, file_values) + ( + image_numbers_by_slice[slice_index], + metadata, + acquisition, + file_values, + ) ) return tuple(rows) @@ -1175,7 +1198,9 @@ def _with_requested_aggregates( reference_means = frozenset( name for name in (() if source_image is None else source_image.columns) - if CellProfilerMeasurementDialect.row_identity_contract.is_aggregate_sample_number_reference(name) + if CellProfilerMeasurementDialect.row_identity_contract.is_aggregate_sample_number_reference( + name + ) ) if not (mean or median or standard_deviation or reference_means): return tables @@ -1558,8 +1583,10 @@ def _prepare_spreadsheet_partition( ) for path, output in bundle.items() } - return {path: output.realized_for_composition(partition_fields=request.partition_fields) - for path, output in bundle.items()} + return { + path: output.realized_for_composition(partition_fields=request.partition_fields) + for path, output in bundle.items() + } def _partitioned_spreadsheet_export( diff --git a/openhcs/processing/backends/cellprofiler/texture.py b/openhcs/processing/backends/cellprofiler/texture.py index a7e95831a0..aafa7df2d5 100644 --- a/openhcs/processing/backends/cellprofiler/texture.py +++ b/openhcs/processing/backends/cellprofiler/texture.py @@ -230,7 +230,6 @@ class ObjectTextureCropBackendStrategy( ): """Extract masked object intensity crops for texture measurement.""" - @abstractmethod def object_intensity_crops( self, image: np.ndarray, labels: np.ndarray @@ -243,7 +242,6 @@ class HaralickTextureBackendStrategy( ): """Compute CP-compatible 2-D Haralick feature matrices.""" - @abstractmethod def haralick_features( self, pixel_data: np.ndarray, *, scale: int, ignore_zeros: bool diff --git a/openhcs/processing/backends/cellprofiler/thresholding.py b/openhcs/processing/backends/cellprofiler/thresholding.py index 2fc2d83fff..f5e41cfad2 100644 --- a/openhcs/processing/backends/cellprofiler/thresholding.py +++ b/openhcs/processing/backends/cellprofiler/thresholding.py @@ -555,7 +555,10 @@ class GlobalThresholdSourceSelection: def normalize_cellprofiler_image(image: RuntimeArrayData) -> np.ndarray: """Return an image in CellProfiler's normalized pixel-data convention.""" - return normalize_cellprofiler_image_payload( image, dtype=np.float32, ).data + return normalize_cellprofiler_image_payload( + image, + dtype=np.float32, + ).data def unit_interval_scale_for_threshold_selection( @@ -755,7 +758,6 @@ class ThresholdSmoothingBackendStrategy( ): """Memory-backend-specific threshold smoothing.""" - @abstractmethod def smooth_threshold_image( self, @@ -864,7 +866,6 @@ class ThresholdDiagnosticsBackendStrategy( ): """Memory-backend-specific threshold diagnostic measurements.""" - def diagnostics( self, image: np.ndarray, @@ -1272,7 +1273,6 @@ class ThresholdPrimitiveBackendStrategy( ): """Small threshold helper primitives supplied by an explicit provider.""" - @abstractmethod def log_transform(self, values: np.ndarray) -> tuple[np.ndarray, object]: """Return CP-compatible log-transformed values and conversion state.""" @@ -2637,9 +2637,9 @@ def threshold( proven_unit_interval_scale=proven_unit_interval_scale, ).calculate() output_image = ( - source_payload.metadata - .without_unit_interval_intensity_scale() - .payload_with(threshold_result.mask.astype(np.float32), mask) + source_payload.metadata.without_unit_interval_intensity_scale().payload_with( + threshold_result.mask.astype(np.float32), mask + ) ) return ( output_image, diff --git a/openhcs/processing/backends/cellprofiler/tracking.py b/openhcs/processing/backends/cellprofiler/tracking.py index 14cc20c861..20a41c9ddc 100644 --- a/openhcs/processing/backends/cellprofiler/tracking.py +++ b/openhcs/processing/backends/cellprofiler/tracking.py @@ -38,7 +38,11 @@ measurement_row_has_object_identity, ) from openhcs.core.memory.decorators import numpy -from openhcs.core.pipeline.function_contracts import (object_label_input_execution_mode, required_axis_roles, special_inputs) +from openhcs.core.pipeline.function_contracts import ( + object_label_input_execution_mode, + required_axis_roles, + special_inputs, +) from python_introspect import public_names_from_objects from metaclass_registry.strategies import EnumKeyedStrategyMixin from openhcs.core.runtime_image_values import RuntimeArrayData @@ -296,7 +300,9 @@ def rows(self) -> MeasurementSparseColumnarRows: return row_policy.annotate_record_rows( rows, object_name=self.object_name, - source_image_name=CELLPROFILER_SCOPE_NAMES[MeasurementScope.SAMPLE].casefold(), + source_image_name=CELLPROFILER_SCOPE_NAMES[ + MeasurementScope.SAMPLE + ].casefold(), ) @classmethod @@ -649,8 +655,10 @@ class CellProfilerRendererAgg(RendererAgg): def _text_bitmap(self, text, properties): pixels, facts = rasterize( - font_manager.findfont(properties), properties.get_size_in_points(), - self.dpi, text, + font_manager.findfont(properties), + properties.get_size_in_points(), + self.dpi, + text, ) return np.frombuffer(pixels, dtype=np.uint8).reshape(facts[1], facts[0]), facts @@ -696,11 +704,15 @@ def render_frame( """Draw one label plane with CellProfiler's stable ID palette.""" if labels.ndim != 2: raise ValueError("Tracked-image rendering requires a 2-D label plane.") - if len(object_numbers) != len(centers[0]) or len(object_numbers) != len(centers[1]): - raise ValueError("Tracked-image IDs and centers must share one label domain.") + if len(object_numbers) != len(centers[0]) or len(object_numbers) != len( + centers[1] + ): + raise ValueError( + "Tracked-image IDs and centers must share one label domain." + ) indexer = np.zeros(len(object_numbers) + 1, dtype=int) indexer[1:] = object_numbers - powers = 2 ** np.mgrid[0:8, 0:len(indexer)][0] + powers = 2 ** np.mgrid[0:8, 0 : len(indexer)][0] bits = (indexer & powers).astype(bool) indexer = np.sum(bits.transpose() * (2 ** np.arange(7, -1, -1)), axis=1) figure = Figure() @@ -914,7 +926,9 @@ def from_frames( ) ) normalized = normalize_cellprofiler_image_payload(rendered) - metadata = normalized.metadata.with_axis(ColourSampleAxisSpec(), -1).with_source_context_from(labels.metadata) + metadata = normalized.metadata.with_axis( + ColourSampleAxisSpec(), -1 + ).with_source_context_from(labels.metadata) output_image = metadata.payload_with(normalized.data, None) return cls( output_image=output_image, @@ -974,7 +988,6 @@ class ObjectTrackingBackendStrategy( ): """TrackObjects primitives keyed by OpenHCS memory type/provider.""" - @abstractmethod def label_centers(self, labels: np.ndarray) -> tuple[np.ndarray, np.ndarray]: """Return y/x centers for dense positive labels.""" @@ -1029,7 +1042,9 @@ def label_centers(self, labels: np.ndarray) -> tuple[np.ndarray, np.ndarray]: label_count = int(labels_array.max()) if labels_array.size else 0 if label_count == 0: return (np.array([], dtype=np.float64), np.array([], dtype=np.float64)) - centers = dense_label_centers_2d_numba(np.ascontiguousarray(labels_array), label_count) + centers = dense_label_centers_2d_numba( + np.ascontiguousarray(labels_array), label_count + ) return (centers[1:, 0], centers[1:, 1]) def track_by_overlap( @@ -1682,7 +1697,9 @@ def _track_by_overlap_numba( def _track_by_distance_numba( frame: TrackingKernelFrame, pixel_radius: int ) -> tuple[np.ndarray, np.ndarray, np.ndarray, int]: - current_centers = dense_label_centers_2d_numba(frame.current_labels, frame.current_count) + current_centers = dense_label_centers_2d_numba( + frame.current_labels, frame.current_count + ) old_centers = dense_label_centers_2d_numba(frame.old_labels, frame.old_count) new_labels = np.zeros(frame.current_count, dtype=np.int64) parent_object_numbers = np.zeros(frame.current_count, dtype=np.int64) diff --git a/openhcs/processing/backends/cellprofiler/watershed.py b/openhcs/processing/backends/cellprofiler/watershed.py index ff017169bf..6502fe6ae3 100644 --- a/openhcs/processing/backends/cellprofiler/watershed.py +++ b/openhcs/processing/backends/cellprofiler/watershed.py @@ -12,7 +12,9 @@ ObjectLabelsArtifactType, SourceStackLineageSourceRelation, ) -from openhcs.processing.backends.cellprofiler.morphology import MorphologyBackendStrategy +from openhcs.processing.backends.cellprofiler.morphology import ( + MorphologyBackendStrategy, +) from metaclass_registry.strategies import ( EnumKeyedStrategyMixin, ) @@ -1625,7 +1627,9 @@ class NumpyLegacyWatershedBackendStrategy(LegacyWatershedBackendStrategy): prefer_fast = False -class CentrosomeNumpyLegacyWatershedBackendStrategy(NumpyLegacyWatershedBackendStrategy): +class CentrosomeNumpyLegacyWatershedBackendStrategy( + NumpyLegacyWatershedBackendStrategy +): """Centrosome provider using the absorbed CellProfiler reference semantics.""" memory_type = MemoryType.NUMPY @@ -1655,7 +1659,6 @@ class CellProfiler4DistanceMarkerBackendStrategy( ): """Build CellProfiler 4 distance-watershed markers through a typed backend.""" - @abstractmethod def distance_markers( self, diff --git a/openhcs/processing/backends/cellprofiler/worms.py b/openhcs/processing/backends/cellprofiler/worms.py index 6961321277..8aae3e57ee 100644 --- a/openhcs/processing/backends/cellprofiler/worms.py +++ b/openhcs/processing/backends/cellprofiler/worms.py @@ -104,7 +104,12 @@ FlexibleContract, Pure2DContract, ) -from openhcs.core.pipeline.function_contracts import (object_label_input_execution_mode, required_axis_roles, runtime_bound_parameters, special_inputs) +from openhcs.core.pipeline.function_contracts import ( + object_label_input_execution_mode, + required_axis_roles, + runtime_bound_parameters, + special_inputs, +) from metaclass_registry.strategies import EnumKeyedStrategyMixin from openhcs.interop.cellprofiler.settings_binder import coerce_cellprofiler_enum from openhcs.interop.cellprofiler.module_settings import ( @@ -3000,8 +3005,10 @@ def _overlapping_worm_outline( local_mask[label_y - y_start, label_x - x_start] = True local_outline = find_boundaries(local_mask, mode="inner") output[y_start:y_stop, x_start:x_stop][local_outline] = colors[label_id] - return source_image.with_pixels(output, - metadata=source_image.metadata.with_axis(ColourSampleAxisSpec(), -1),) + return source_image.with_pixels( + output, + metadata=source_image.metadata.with_axis(ColourSampleAxisSpec(), -1), + ) def _nonoverlapping_worm_outline( @@ -3016,8 +3023,10 @@ def _nonoverlapping_worm_outline( object_label_dense_array(labels, dtype=np.int32), mode="inner", ) - return source_image.with_pixels(outline, - metadata=source_image.metadata.without_axis(ColourAxis),) + return source_image.with_pixels( + outline, + metadata=source_image.metadata.without_axis(ColourAxis), + ) def _reconstructed_worm_pixels( diff --git a/openhcs/processing/backends/cellprofiler/zernike.py b/openhcs/processing/backends/cellprofiler/zernike.py index a81eb11e36..efd349cc07 100644 --- a/openhcs/processing/backends/cellprofiler/zernike.py +++ b/openhcs/processing/backends/cellprofiler/zernike.py @@ -785,7 +785,6 @@ class ShapeZernikeBackendStrategy( ): """Shape Zernike moment backends keyed by OpenHCS memory type.""" - @abstractmethod def shape_zernike_moments( self, diff --git a/openhcs/processing/backends/enhance/basic_processor_jax.py b/openhcs/processing/backends/enhance/basic_processor_jax.py index 148d5fd86e..0640f7ec21 100644 --- a/openhcs/processing/backends/enhance/basic_processor_jax.py +++ b/openhcs/processing/backends/enhance/basic_processor_jax.py @@ -61,7 +61,6 @@ def _fitted_field_output(name: str) -> ArtifactSpec: @numpy_func(contract=Pure3DContract, dtype_config_default=DtypeConfig()) @allowed_group_by_roles(ColourAxis) @required_axis_roles(FittedIlluminationFieldOutput.observation_role) - @artifact_outputs(CORRECTED_OUTPUT, FLATFIELD_OUTPUT, DARKFIELD_OUTPUT) def basic_flatfield_correction_jax( image: np.ndarray, diff --git a/openhcs/processing/backends/experimental_analysis/analysis.py b/openhcs/processing/backends/experimental_analysis/analysis.py index ee871bb570..ac4ed4b98a 100755 --- a/openhcs/processing/backends/experimental_analysis/analysis.py +++ b/openhcs/processing/backends/experimental_analysis/analysis.py @@ -26,7 +26,9 @@ ExperimentalAnalysisScope, ExperimentalLayoutRowRole, ) -from openhcs.processing.backends.experimental_analysis.result_formats import ExperimentalResultFormatStrategy +from openhcs.processing.backends.experimental_analysis.result_formats import ( + ExperimentalResultFormatStrategy, +) @dataclass(frozen=True, slots=True) diff --git a/openhcs/processing/backends/experimental_analysis/result_formats.py b/openhcs/processing/backends/experimental_analysis/result_formats.py index d3039f5574..884868e75d 100644 --- a/openhcs/processing/backends/experimental_analysis/result_formats.py +++ b/openhcs/processing/backends/experimental_analysis/result_formats.py @@ -11,7 +11,9 @@ import pandas as pd from metaclass_registry.strategies import EnumKeyedStrategyMixin -from openhcs.processing.backends.experimental_analysis.layout_rows import ExperimentalAnalysisScope +from openhcs.processing.backends.experimental_analysis.layout_rows import ( + ExperimentalAnalysisScope, +) PlateDictionary = dict[str, dict[str, dict[str, Any]]] diff --git a/openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py b/openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py index b1b2487cf7..bfc9ac163c 100644 --- a/openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py +++ b/openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py @@ -4,7 +4,9 @@ from typing import Dict, List, Any, Optional, Tuple from openhcs.domains.microscopy.config import ExperimentalAnalysisConfig -from openhcs.processing.backends.experimental_analysis.layout_rows import ExperimentalAnalysisScope +from openhcs.processing.backends.experimental_analysis.layout_rows import ( + ExperimentalAnalysisScope, +) from openhcs.processing.backends.experimental_analysis.result_formats import ( ExperimentalResultFormatStrategy, ) @@ -17,7 +19,9 @@ class DataProcessingError(RuntimeError): class ExperimentalAnalysisEngine: """Coordinate analysis through the result scope's nominal strategy.""" - def __init__(self, config: ExperimentalAnalysisConfig = ExperimentalAnalysisConfig()): + def __init__( + self, config: ExperimentalAnalysisConfig = ExperimentalAnalysisConfig() + ): """Initialize the engine with a configuration (default: the declared defaults).""" self.config = config @@ -218,7 +222,9 @@ def _read_plate_layout( Returns: Tuple of (scope, plate_layout, conditions, ctrl_positions, excluded_positions, per_well_datapoints) """ - from openhcs.processing.backends.experimental_analysis.analysis import read_plate_layout + from openhcs.processing.backends.experimental_analysis.analysis import ( + read_plate_layout, + ) return read_plate_layout( config_path, @@ -235,7 +241,9 @@ def _load_plate_groups(self, config_path: str) -> Dict: Returns: Plate groups dictionary """ - from openhcs.processing.backends.experimental_analysis.analysis import load_plate_groups + from openhcs.processing.backends.experimental_analysis.analysis import ( + load_plate_groups, + ) return load_plate_groups( config_path, @@ -246,7 +254,9 @@ def _make_experiment_dict_locations( self, plate_groups: Dict, plate_layout: Dict, conditions: List ) -> Dict: """Create experiment location mapping.""" - from openhcs.processing.backends.experimental_analysis.analysis import make_experiment_dict_locations + from openhcs.processing.backends.experimental_analysis.analysis import ( + make_experiment_dict_locations, + ) return make_experiment_dict_locations(plate_groups, plate_layout, conditions) @@ -259,7 +269,9 @@ def _make_experiment_dict_values( per_well_datapoints: bool = False, ) -> Dict: """Map experimental design to measured values.""" - from openhcs.processing.backends.experimental_analysis.analysis import make_experiment_dict_values + from openhcs.processing.backends.experimental_analysis.analysis import ( + make_experiment_dict_values, + ) return make_experiment_dict_values( plates_dict, @@ -299,7 +311,9 @@ def _normalize_experiment( plate_groups: Dict, ) -> Dict: """Apply normalization using control wells.""" - from openhcs.processing.backends.experimental_analysis.analysis import normalize_experiment + from openhcs.processing.backends.experimental_analysis.analysis import ( + normalize_experiment, + ) return normalize_experiment( experiment_dict_values, @@ -317,7 +331,9 @@ def _create_all_feature_tables( per_well_datapoints: bool = False, ) -> Dict: """Create feature tables for export.""" - from openhcs.processing.backends.experimental_analysis.analysis import create_all_feature_tables + from openhcs.processing.backends.experimental_analysis.analysis import ( + create_all_feature_tables, + ) return create_all_feature_tables( experiment_dict_values, features, per_well_datapoints @@ -325,7 +341,9 @@ def _create_all_feature_tables( def _export_results(self, feature_tables: Dict, output_path: str): """Export results to Excel file.""" - from openhcs.processing.backends.experimental_analysis.analysis import feature_tables_to_excel + from openhcs.processing.backends.experimental_analysis.analysis import ( + feature_tables_to_excel, + ) feature_tables_to_excel(feature_tables, output_path) diff --git a/openhcs/processing/backends/lib_registry/openhcs_registry.py b/openhcs/processing/backends/lib_registry/openhcs_registry.py index 86803ca427..9edb85602f 100644 --- a/openhcs/processing/backends/lib_registry/openhcs_registry.py +++ b/openhcs/processing/backends/lib_registry/openhcs_registry.py @@ -605,9 +605,7 @@ def get_modules_to_scan(self) -> List[Tuple[str, Any]]: module_name, ) continue - excluded = _module_distributions_excluded_by_package_metadata( - module_name - ) + excluded = _module_distributions_excluded_by_package_metadata(module_name) if excluded: logger.info( "Skipping OpenHCS module %s - package requirements exclude %s " @@ -944,9 +942,11 @@ def reconstruct_cached_callable( """Reconstruct cached metadata through the declaration's one wrapper.""" return _registered_wrapper( - self, func, lambda: super(OpenHCSRegistry, self).reconstruct_cached_callable( + self, + func, + lambda: super(OpenHCSRegistry, self).reconstruct_cached_callable( func, contract - ) + ), ) def _processing_contract_for_function( diff --git a/openhcs/processing/backends/lib_registry/registry_service.py b/openhcs/processing/backends/lib_registry/registry_service.py index b706eff7f5..dbe13006f0 100644 --- a/openhcs/processing/backends/lib_registry/registry_service.py +++ b/openhcs/processing/backends/lib_registry/registry_service.py @@ -109,9 +109,13 @@ def metadata_for_canonical_key(cls, function_id: str) -> FunctionMetadata: """Ask the original registry declaration to resolve one public identity.""" registry_name, separator, name = function_id.partition(":") - registry_type = LIBRARY_REGISTRIES.get(registry_name) if separator and name else None - claims = () if registry_type is None else tuple( - registry_type._canonical_metadata_claims(function_id) + registry_type = ( + LIBRARY_REGISTRIES.get(registry_name) if separator and name else None + ) + claims = ( + () + if registry_type is None + else tuple(registry_type._canonical_metadata_claims(function_id)) ) for metadata in claims: if metadata.composite_key != function_id: @@ -128,7 +132,9 @@ def metadata_for_canonical_key(cls, function_id: str) -> FunctionMetadata: or inspect.unwrap(metadata.func) is not inspect.unwrap(first.func) for metadata in claims[1:] ): - raise ValueError(f"Canonical function {function_id!r} has ambiguous owners.") + raise ValueError( + f"Canonical function {function_id!r} has ambiguous owners." + ) return first @classmethod @@ -187,11 +193,15 @@ def prepare_in_current_process( cls._metadata_cache = cls._metadata_from_instances( cls._available_registry_instances() ) - callables = tuple(dict.fromkeys( - target - for metadata in cls._metadata_cache.values() - for target in CallableProjection.from_callable(metadata.func).prepare_targets() - )) + callables = tuple( + dict.fromkeys( + target + for metadata in cls._metadata_cache.values() + for target in CallableProjection.from_callable( + metadata.func + ).prepare_targets() + ) + ) emit_status("Preparing registered kernel caches") PreparationCacheBatch.from_callables(callables).populate_child_caches( max_workers=os.cpu_count() or 1, diff --git a/openhcs/processing/backends/lib_registry/unified_registry.py b/openhcs/processing/backends/lib_registry/unified_registry.py index 0e5efdf371..c678271fb1 100644 --- a/openhcs/processing/backends/lib_registry/unified_registry.py +++ b/openhcs/processing/backends/lib_registry/unified_registry.py @@ -311,7 +311,10 @@ def metadata_for_declared_callable( @classmethod def _canonical_metadata_claims( - cls, function_id: str, *, prepare_catalog: bool = True, + cls, + function_id: str, + *, + prepare_catalog: bool = True, ) -> Iterator[FunctionMetadata]: """Supply catalog-owned claims; independent capabilities compose via super.""" @@ -319,7 +322,8 @@ def _canonical_metadata_claims( catalog = ( RegistryService.get_all_functions_with_metadata() - if prepare_catalog else RegistryService.cached_metadata_snapshot() + if prepare_catalog + else RegistryService.cached_metadata_snapshot() ) metadata = catalog.get(function_id) if metadata is not None: @@ -550,7 +554,9 @@ def wrapper(image, *args, **kwargs): } return contract.execute( - LibraryContractCall(func, args), image, filtered_kwargs, + LibraryContractCall(func, args), + image, + filtered_kwargs, ) wrapper.__signature__ = public_sig.replace(parameters=new_params) diff --git a/openhcs/processing/backends/processors/cupy_processor.py b/openhcs/processing/backends/processors/cupy_processor.py index 7126155a57..7003761895 100644 --- a/openhcs/processing/backends/processors/cupy_processor.py +++ b/openhcs/processing/backends/processors/cupy_processor.py @@ -229,7 +229,9 @@ def create_weight_mask( Returns: 2D CuPy weight mask of shape (Y, X) """ - return create_linear_weight_mask(*operations.weight_mask_extent(shape), margin_ratio) + return create_linear_weight_mask( + *operations.weight_mask_extent(shape), margin_ratio + ) @cupy_func diff --git a/openhcs/processing/backends/processors/jax_processor.py b/openhcs/processing/backends/processors/jax_processor.py index a5cbf5f481..76045a8f91 100644 --- a/openhcs/processing/backends/processors/jax_processor.py +++ b/openhcs/processing/backends/processors/jax_processor.py @@ -347,7 +347,9 @@ def create_weight_mask( Returns: 2D JAX weight mask of shape (Y, X) """ - return create_linear_weight_mask(*operations.weight_mask_extent(shape), margin_ratio) + return create_linear_weight_mask( + *operations.weight_mask_extent(shape), margin_ratio + ) @jax_func diff --git a/openhcs/processing/backends/processors/numpy_processor.py b/openhcs/processing/backends/processors/numpy_processor.py index 5b41c859de..a62d8da5e8 100644 --- a/openhcs/processing/backends/processors/numpy_processor.py +++ b/openhcs/processing/backends/processors/numpy_processor.py @@ -329,14 +329,16 @@ def _indexed_weighted_projection( preserve_numpy_sum_order and not values.flags.c_contiguous ) if ( - not supported or pairwise + not supported + or pairwise or dtype not in (np.dtype(np.float32), np.dtype(np.float64)) ): if preserve_numpy_sum_order: converted = members.astype(dtype) factors = weights.reshape((len(weights),) + (1,) * (members.ndim - 1)) selected_members = ( - converted if np.array_equal(selected, np.arange(count)) + converted + if np.array_equal(selected, np.arange(count)) else converted[selected] ) return np.sum(selected_members * factors, axis=0) @@ -367,12 +369,16 @@ def execute(self) -> None: for shape in ((3, 2, 2), (3, 2, 2, 2)): values = np.ones(shape, dtype=source_dtype) strided = np.moveaxis( - np.moveaxis(values, 0, -1).copy(), -1, 0, + np.moveaxis(values, 0, -1).copy(), + -1, + 0, ) for members in (values, np.asfortranarray(values), strided): for writable in (True, False): members.flags.writeable = writable - _indexed_weighted_projection(members, indices, weights, axis=0) + _indexed_weighted_projection( + members, indices, weights, axis=0 + ) @numpy_func(contract=VolumetricToSliceContract) @@ -399,8 +405,11 @@ def create_composite( weights_array = np.array(normalized_weights, dtype=np.float32) composite_slice = _indexed_weighted_projection( - stack, np.arange(n_slices, dtype=np.int64), weights_array, - axis=0, preserve_numpy_sum_order=True, + stack, + np.arange(n_slices, dtype=np.int64), + weights_array, + axis=0, + preserve_numpy_sum_order=True, ) # Convert back to original dtype @@ -439,7 +448,9 @@ def create_weight_mask(shape: Tuple[int, int], margin_ratio: float = 0.1) -> np. Returns: 2D NumPy weight mask of shape (Y, X) """ - return create_linear_weight_mask(*operations.weight_mask_extent(shape), margin_ratio) + return create_linear_weight_mask( + *operations.weight_mask_extent(shape), margin_ratio + ) @numpy_func diff --git a/openhcs/processing/backends/processors/pyclesperanto_processor.py b/openhcs/processing/backends/processors/pyclesperanto_processor.py index deab9b2b81..3e3bf132d1 100644 --- a/openhcs/processing/backends/processors/pyclesperanto_processor.py +++ b/openhcs/processing/backends/processors/pyclesperanto_processor.py @@ -642,4 +642,6 @@ def create_weight_mask(shape: tuple, margin_ratio: float = 0.1) -> "cle.Array": Returns: 2D pyclesperanto Array of shape (Y, X) """ - return create_linear_weight_mask(*operations.weight_mask_extent(shape), margin_ratio) + return create_linear_weight_mask( + *operations.weight_mask_extent(shape), margin_ratio + ) diff --git a/openhcs/processing/backends/processors/tensorflow_processor.py b/openhcs/processing/backends/processors/tensorflow_processor.py index f0b88191b3..1ab6342612 100644 --- a/openhcs/processing/backends/processors/tensorflow_processor.py +++ b/openhcs/processing/backends/processors/tensorflow_processor.py @@ -367,7 +367,9 @@ def create_weight_mask( Returns: 2D TensorFlow weight mask of shape (Y, X) """ - return create_linear_weight_mask(*operations.weight_mask_extent(shape), margin_ratio) + return create_linear_weight_mask( + *operations.weight_mask_extent(shape), margin_ratio + ) @tensorflow_func diff --git a/openhcs/processing/backends/processors/torch_processor.py b/openhcs/processing/backends/processors/torch_processor.py index 82b9b63273..be0c90321e 100644 --- a/openhcs/processing/backends/processors/torch_processor.py +++ b/openhcs/processing/backends/processors/torch_processor.py @@ -406,7 +406,9 @@ def create_weight_mask( Returns: 2D PyTorch weight mask of shape (Y, X) """ - return create_linear_weight_mask(*operations.weight_mask_extent(shape), margin_ratio) + return create_linear_weight_mask( + *operations.weight_mask_extent(shape), margin_ratio + ) @torch_func diff --git a/openhcs/processing/custom_functions/manager.py b/openhcs/processing/custom_functions/manager.py index eec75f2639..c607ca8c36 100644 --- a/openhcs/processing/custom_functions/manager.py +++ b/openhcs/processing/custom_functions/manager.py @@ -113,7 +113,9 @@ def default_storage_directory(cls) -> Path: def source_path_for_name(storage_dir: Path, function_name: str) -> Path: """Own source naming for both admission and actual persistence.""" if not function_name.isidentifier() or function_name.startswith("_"): - raise ValidationError("Custom function name must be a public Python identifier.") + raise ValidationError( + "Custom function name must be a public Python identifier." + ) return storage_dir / f"{function_name}.py" def __init__(self, *, create_storage: bool = True): @@ -154,11 +156,18 @@ def register_from_code( """ if persist and write_admission is not None: if expected_function_name is None: - raise ValueError("Write admission requires an explicit custom function name.") + raise ValueError( + "Write admission requires an explicit custom function name." + ) write_admission(self.storage_dir) - write_admission(self.source_path_for_name(self.storage_dir, expected_function_name)) + write_admission( + self.source_path_for_name(self.storage_dir, expected_function_name) + ) metadata = self._prepare_source(code) - if expected_function_name is not None and metadata.original_name != expected_function_name: + if ( + expected_function_name is not None + and metadata.original_name != expected_function_name + ): raise ValidationError( f"Prepared declaration {metadata.original_name!r} does not match " f"admitted function {expected_function_name!r}." @@ -207,7 +216,9 @@ def source_revision(self) -> CustomFunctionSourceRevision: sources = tuple( CustomFunctionSource( function_name=source_path.stem, - content_sha256=CustomFunctionSource.content_digest(source_path.read_bytes()), + content_sha256=CustomFunctionSource.content_digest( + source_path.read_bytes() + ), ) for source_path in sorted(self.storage_dir.glob("*.py")) ) @@ -489,7 +500,9 @@ def _prepare_source( ) -> "FunctionMetadata": """Validate and project one source without mutating runtime or disk state.""" - from openhcs.processing.backends.lib_registry.openhcs_registry import OpenHCSRegistry + from openhcs.processing.backends.lib_registry.openhcs_registry import ( + OpenHCSRegistry, + ) validation_result = validate_code(code) if not validation_result.is_valid: diff --git a/openhcs/processing/custom_functions/validation.py b/openhcs/processing/custom_functions/validation.py index 25368d9cbe..679722c21b 100644 --- a/openhcs/processing/custom_functions/validation.py +++ b/openhcs/processing/custom_functions/validation.py @@ -188,18 +188,25 @@ def validate_code(code: str) -> ValidationResult: def validate_function(func: Callable) -> ValidationResult: """Validate the selected declaration using its original scope contract.""" - from openhcs.processing.backends.lib_registry.openhcs_registry import OpenHCSRegistry + from openhcs.processing.backends.lib_registry.openhcs_registry import ( + OpenHCSRegistry, + ) errors: list[str] = [] try: contract = OpenHCSRegistry.declared_callable_contract(func) if contract is None: - raise ValueError(f"Function {func.__name__!r} is not an admitted OpenHCS declaration.") + raise ValueError( + f"Function {func.__name__!r} is not an admitted OpenHCS declaration." + ) if contract.execution_scope is FunctionStepExecutionScope.PLATE: - from openhcs.core.pipeline.funcstep_contract_validator import FuncStepContractValidator + from openhcs.core.pipeline.funcstep_contract_validator import ( + FuncStepContractValidator, + ) FuncStepContractValidator.validate_dataset_callable_contracts( - (contract,), func.__name__, + (contract,), + func.__name__, ) else: params = tuple(inspect.signature(func).parameters) @@ -210,5 +217,8 @@ def validate_function(func: Callable) -> ValidationResult: except (TypeError, ValueError) as exc: errors.append(str(exc)) return ValidationResult( - is_valid=not errors, errors=errors, warnings=[], function_names=[func.__name__], + is_valid=not errors, + errors=errors, + warnings=[], + function_names=[func.__name__], ) diff --git a/openhcs/processing/materialization/core.py b/openhcs/processing/materialization/core.py index 80a45f9d4b..bb2a47c33a 100644 --- a/openhcs/processing/materialization/core.py +++ b/openhcs/processing/materialization/core.py @@ -15,7 +15,13 @@ import string from abc import ABC, abstractmethod from collections.abc import Callable, Mapping, Sequence, Sized -from dataclasses import dataclass, field, fields as dataclass_fields, is_dataclass, replace +from dataclasses import ( + dataclass, + field, + fields as dataclass_fields, + is_dataclass, + replace, +) from functools import lru_cache, singledispatch from pathlib import Path, PurePosixPath, PureWindowsPath from types import MappingProxyType @@ -341,21 +347,29 @@ def viewer_source_identity(self) -> SourceImageIdentity | None: return self.source_identity def stream_source_metadata_items( - self, fallback_source_identity: SourceImageIdentity | None, + self, + fallback_source_identity: SourceImageIdentity | None, ) -> StreamSourceComponentMetadataItems: """Observe sources from the actual rendered output's declared planes.""" if self.metadata is not None: return StreamSourceComponentMetadataItems.from_image_metadata( - self.metadata, fallback_source_identity=fallback_source_identity, + self.metadata, + fallback_source_identity=fallback_source_identity, ) return StreamSourceComponentMetadataItems.from_values( - (fallback_source_identity.component_metadata - if fallback_source_identity is not None else None,) + ( + ( + fallback_source_identity.component_metadata + if fallback_source_identity is not None + else None + ), + ) ) def viewer_item_fields(self) -> dict[str, ViewerWireValue]: return StreamImagePayloadMetadataProjector.item_fields_for_plane_components( - self.metadata, self.variable_components, + self.metadata, + self.variable_components, ) @property @@ -422,7 +436,9 @@ def source_component_domain(self) -> tuple[SourceComponentMetadata, ...] | None: if self.metadata is None: return None - return ROIArchiveSourceMetadata.source_component_domain(self.content, self.metadata) + return ROIArchiveSourceMetadata.source_component_domain( + self.content, self.metadata + ) @property def viewer_source_identity(self) -> SourceImageIdentity | None: @@ -436,7 +452,8 @@ def viewer_source_identity(self) -> SourceImageIdentity | None: ) def stream_source_metadata_items( - self, fallback_source_identity: SourceImageIdentity | None, + self, + fallback_source_identity: SourceImageIdentity | None, ) -> StreamSourceComponentMetadataItems: domain = self.source_component_domain if domain is None: @@ -447,7 +464,9 @@ def viewer_item_fields(self) -> dict[str, ViewerWireValue]: from openhcs.core.roi_source_metadata import ROIArchiveSourceMetadata return ROIArchiveSourceMetadata.stream_item_fields( - self.content, self.metadata, super().viewer_item_fields(), + self.content, + self.metadata, + super().viewer_item_fields(), ) @@ -466,6 +485,7 @@ def source_component_domain(self) -> tuple[SourceComponentMetadata, ...]: raise ValueError("Point ROI output requires fractional-Z metadata.") return domain + @dataclass(frozen=True) class TextOutput(Output): """Materialization output whose canonical payload is text.""" @@ -485,7 +505,9 @@ class Utf8TextOutput(Output): """Text output stored as UTF-8 bytes for file-bundle compatibility.""" content: bytes - table_shape_info: tuple[tuple[str, ...], int] | None = field(default=None, compare=False) + table_shape_info: tuple[tuple[str, ...], int] | None = field( + default=None, compare=False + ) def table_shape(self) -> tuple[tuple[str, ...], int] | None: return self.table_shape_info @@ -511,6 +533,7 @@ def viewer_stream_requires_source_metadata(self) -> bool: def require_text_content(self) -> str: return self.content.decode("utf-8") + @dataclass(frozen=True) class ColumnarCsvOutput(Output): """CSV whose correlated row domain survives until file materialization. @@ -1964,9 +1987,7 @@ def from_runtime_slice_projected_value( runtime_plane_metadata=item.runtime_plane_metadata, ) for source_payload in source_payloads - for item in ( - OptionalSourceIdentity - ).project_payload_items( + for item in (OptionalSourceIdentity).project_payload_items( RuntimeProjectionSourceIdentityRequest( value=source_payload, source_description="materialization payload", @@ -2010,9 +2031,7 @@ def from_image_value( source_description=item.source_description, runtime_plane_metadata=item.runtime_plane_metadata, ) - for item in ( - OptionalSourceIdentity - ).project_payload_items( + for item in (OptionalSourceIdentity).project_payload_items( RuntimeProjectionSourceIdentityRequest( value=image.value, source_description=image.source_description, @@ -2264,10 +2283,14 @@ def to_filemanager_kwargs( if output_fields is None: return self.values.to_kwargs() component_metadata, item_fields, display_config = output_fields - return self._stream_kwargs(display_config, item_fields).with_single_item_source( - component_metadata, - item_fields, - ).to_kwargs() + return ( + self._stream_kwargs(display_config, item_fields) + .with_single_item_source( + component_metadata, + item_fields, + ) + .to_kwargs() + ) def _stream_kwargs( self, @@ -2276,7 +2299,9 @@ def _stream_kwargs( ) -> ViewerStreamBackendKwargs: """Apply one source-owned display scope to the original typed request.""" request = self.values.with_item_fields(item_fields).stream_request - return ViewerStreamBackendKwargs(replace(request, display_config=display_config)) + return ViewerStreamBackendKwargs( + replace(request, display_config=display_config) + ) def filemanager_batches( self, @@ -2298,8 +2323,11 @@ def filemanager_batches( continue component_metadata, item_fields, display_config = output_fields group = next( - (candidate for candidate in grouped - if candidate[0] == item_fields and candidate[1] == display_config), + ( + candidate + for candidate in grouped + if candidate[0] == item_fields and candidate[1] == display_config + ), None, ) if group is None: @@ -2323,7 +2351,9 @@ def filemanager_batches( raise ValueError( "Viewer materialization batch requires unique output paths." ) - stream_request = self._stream_kwargs(display_config, item_fields).stream_request + stream_request = self._stream_kwargs( + display_config, item_fields + ).stream_request stream_request = replace( stream_request, source=replace( @@ -2368,9 +2398,13 @@ def projected_outputs(self, outputs: Sequence[Output]) -> tuple[Output, ...]: fallback_identity_path=output.path, ) if identity is None: - raise ValueError("Projected viewer output requires a filename identity.") + raise ValueError( + "Projected viewer output requires a filename identity." + ) identity = replace(identity, extension=output_path.suffix) - filename = identity.with_filename_qualifier(output_path.stem).filename(parser) + filename = identity.with_filename_qualifier(output_path.stem).filename( + parser + ) result.append( replace( output, @@ -2384,11 +2418,14 @@ def projected_outputs(self, outputs: Sequence[Output]) -> tuple[Output, ...]: def _output_fields( self, output: Output, - ) -> tuple[ - dict[str, MaterializationValue], - dict[str, ViewerWireValue], - ViewerDisplayConfigABC, - ] | None: + ) -> ( + tuple[ + dict[str, MaterializationValue], + dict[str, ViewerWireValue], + ViewerDisplayConfigABC, + ] + | None + ): source_identity = output.viewer_source_identity if source_identity is not None: source_identity = source_identity.with_parsed_path_components( @@ -2518,7 +2555,9 @@ class MaterializationContext: materialization_spec: MaterializationSpec | None = None def named_source_filename( - self, metadata: ImagePayloadMetadata, extension: str, + self, + metadata: ImagePayloadMetadata, + extension: str, ) -> str | None: """Name a retained image from the actual rendering purpose and role.""" if self.materialization_spec is None: @@ -2526,12 +2565,17 @@ def named_source_filename( qualifier = self.materialization_spec.filename_qualifier(self.output_plan) if qualifier is None: return None - parser = SourceStemAuthoritySelection.from_processing_context(self.context).required_parser() + parser = SourceStemAuthoritySelection.from_processing_context( + self.context + ).required_parser() identity = FunctionOutputIdentity.from_filename_metadata(parser, metadata) if identity is None: - raise ValueError("Retained image output has no addressable source filename identity.") + raise ValueError( + "Retained image output has no addressable source filename identity." + ) identity = self.materialization_spec.filename_identity_for_output( - replace(identity, extension=extension), self.output_plan, + replace(identity, extension=extension), + self.output_plan, ) return identity.filename(parser) @@ -3316,16 +3360,22 @@ def write_image_file( for sequence_index, item in enumerate(projected_items, start=1): if options.relative_path_template is not None: path = _image_relative_output_path( - item.value, options, context, sequence_index=sequence_index, + item.value, + options, + context, + sequence_index=sequence_index, ) elif preserves_planned_path: path = paths.primary_output_path(options) else: - filename = context.named_source_filename(item.metadata, options.primary_output_suffix) + filename = context.named_source_filename( + item.metadata, options.primary_output_suffix + ) if filename is None: filename = ( source_stem_authority.required_source_stem( - item.metadata, context.artifact_filename_identity, + item.metadata, + context.artifact_filename_identity, ) + options.primary_output_suffix ) @@ -4656,7 +4706,6 @@ def metadata_for_slices( component for component in request.variable_components if component.name in selected_variable_components - ), request.artifact_source_identity, ) diff --git a/openhcs/processing/presets/pipelines/czi_brain_axon_cellbody.py b/openhcs/processing/presets/pipelines/czi_brain_axon_cellbody.py index c8710ae585..6a72cd9cbe 100644 --- a/openhcs/processing/presets/pipelines/czi_brain_axon_cellbody.py +++ b/openhcs/processing/presets/pipelines/czi_brain_axon_cellbody.py @@ -287,8 +287,7 @@ def _declared_crop_candidates( for candidate in dataset.candidates if candidate.component_labels.get(Microscopy.Site.name) == selection.site_label - and candidate.component_labels.get(Microscopy.Channel.name) - == channel_label + and candidate.component_labels.get(Microscopy.Channel.name) == channel_label and candidate.declared_address is not None and candidate.declared_address.value_for(Microscopy.ZIndex) == "1" and candidate.declared_address.value_for(Microscopy.Timepoint) == "1" @@ -476,9 +475,7 @@ def artifact_presentation( output_spec: ArtifactSpec, ) -> StreamProducerIdentity: return StreamProducerIdentity.pipeline_output( - output_kind=( - CompiledStepPlan.ARTIFACT_OUTPUT_KIND - ), + output_kind=(CompiledStepPlan.ARTIFACT_OUTPUT_KIND), output_key=output_spec.name, projection_key=output_spec.name, step_name=step.name, diff --git a/openhcs/processing/presets/pipelines/loose_operaphenix_neurite_outgrowth.py b/openhcs/processing/presets/pipelines/loose_operaphenix_neurite_outgrowth.py index f019801cae..341a3437f7 100644 --- a/openhcs/processing/presets/pipelines/loose_operaphenix_neurite_outgrowth.py +++ b/openhcs/processing/presets/pipelines/loose_operaphenix_neurite_outgrowth.py @@ -441,9 +441,7 @@ def prepare() -> None: pipeline_config=pipeline_config, pipeline_steps=tuple(pipeline_steps), presentation_identity=StreamProducerIdentity.pipeline_output( - output_kind=( - CompiledStepPlan.ARTIFACT_OUTPUT_KIND - ), + output_kind=(CompiledStepPlan.ARTIFACT_OUTPUT_KIND), output_key=secondary_output_name, projection_key=secondary_output_name, step_name=analysis_step.name, diff --git a/openhcs/processing/presets/pipelines/neuroncyto_ii_crossover_neurite_outgrowth.py b/openhcs/processing/presets/pipelines/neuroncyto_ii_crossover_neurite_outgrowth.py index 05e91f68f0..1af800129e 100644 --- a/openhcs/processing/presets/pipelines/neuroncyto_ii_crossover_neurite_outgrowth.py +++ b/openhcs/processing/presets/pipelines/neuroncyto_ii_crossover_neurite_outgrowth.py @@ -127,7 +127,9 @@ def build_neuroncyto_ii_crossover_demo( ) pipeline_config = PipelineConfig( dataset_source=SourceBindingsSource, - partition_filter_config=LazyPartitionFilterConfig(partition_filter=inputs.image_id), + partition_filter_config=LazyPartitionFilterConfig( + partition_filter=inputs.image_id + ), path_planning_config=LazyPathPlanningConfig( partition_filter=0, global_output_folder=output_root, @@ -295,9 +297,7 @@ def prepare() -> None: pipeline_config=pipeline_config, pipeline_steps=tuple(pipeline_steps), presentation_identity=StreamProducerIdentity.pipeline_output( - output_kind=( - CompiledStepPlan.ARTIFACT_OUTPUT_KIND - ), + output_kind=(CompiledStepPlan.ARTIFACT_OUTPUT_KIND), output_key=NEURITE_MORPHOLOGY_OUTPUT.name, projection_key=NEURITE_MORPHOLOGY_OUTPUT.name, step_name=analysis_step.name, diff --git a/openhcs/pyqt_gui/main.py b/openhcs/pyqt_gui/main.py index 24b602a83a..4c56133572 100644 --- a/openhcs/pyqt_gui/main.py +++ b/openhcs/pyqt_gui/main.py @@ -1311,7 +1311,9 @@ def _on_synthetic_plate_generated(self, output_dir: str, pipeline_path: str): self.show_pipeline_editor() for scope_id in self.session.add_dataset_roots((output_dir,)): self.session.load_pipeline_file(scope_id, pipeline_path) - logger.info("Added synthetic dataset and loaded its test pipeline: %s", output_dir) + logger.info( + "Added synthetic dataset and loaded its test pipeline: %s", output_dir + ) def _on_consolidate_results(self): """Open file dialog to select results directory and consolidate analysis results.""" diff --git a/openhcs/pyqt_gui/services/reactor_providers.py b/openhcs/pyqt_gui/services/reactor_providers.py index adc685e206..5743d0cbfb 100644 --- a/openhcs/pyqt_gui/services/reactor_providers.py +++ b/openhcs/pyqt_gui/services/reactor_providers.py @@ -231,7 +231,9 @@ def _get_current_orchestrator(self) -> PipelineOrchestrator | None: from openhcs.constants.constants import OrchestratorState from openhcs.core.orchestrator import PipelineOrchestrator - orchestrator = ObjectStateRegistry.get_object(plate_manager.session.current_scope_id) + orchestrator = ObjectStateRegistry.get_object( + plate_manager.session.current_scope_id + ) if not isinstance(orchestrator, PipelineOrchestrator): return None if orchestrator.state is OrchestratorState.CREATED: @@ -279,7 +281,6 @@ def select_components( group_by=group_by, group_label=self.grouping_label(group_by), metadata_lookup=self.get_component_display_name, - parent=parent, ) diff --git a/openhcs/pyqt_gui/services/ui_agent_bridge.py b/openhcs/pyqt_gui/services/ui_agent_bridge.py index 913d91547c..f017860c34 100644 --- a/openhcs/pyqt_gui/services/ui_agent_bridge.py +++ b/openhcs/pyqt_gui/services/ui_agent_bridge.py @@ -184,6 +184,7 @@ "for the current template." ) + @dataclass(frozen=True, slots=True) class CodeDocumentExecutionResult(PlateManagerOrchestratorCodePayload): """Parsed and normalized payload from one declarative UI code document.""" @@ -787,9 +788,7 @@ def complete_result( ) -> UiBridgeOperationRef: """Project one typed completion through its registered declaration owner.""" operation = self.get(operation_id) - contract = UiBridgeOperation.for_name( - operation.identity.operation_name - ) + contract = UiBridgeOperation.for_name(operation.identity.operation_name) if not isinstance(result, contract.result_type): raise TypeError( f"{contract.require_name()} requires {contract.result_type.__qualname__}." @@ -811,9 +810,7 @@ def fail_exception( self, operation_id: str, error: Exception ) -> UiBridgeOperationRef: operation = self.get(operation_id) - contract = UiBridgeOperation.for_name( - operation.identity.operation_name - ) + contract = UiBridgeOperation.for_name(operation.identity.operation_name) return self.complete( operation_id, status=UiBridgeOperationStatus.FAILED, diff --git a/openhcs/pyqt_gui/services/ui_bridge_composition.py b/openhcs/pyqt_gui/services/ui_bridge_composition.py index 3ebbac7430..9cf6763066 100644 --- a/openhcs/pyqt_gui/services/ui_bridge_composition.py +++ b/openhcs/pyqt_gui/services/ui_bridge_composition.py @@ -17,7 +17,6 @@ UiBridgeSurfaceRegistry, ) - if TYPE_CHECKING: from openhcs.authoring.session.session import Session diff --git a/openhcs/pyqt_gui/services/ui_bridge_object_state.py b/openhcs/pyqt_gui/services/ui_bridge_object_state.py index 74221355d2..c7d117dbcb 100644 --- a/openhcs/pyqt_gui/services/ui_bridge_object_state.py +++ b/openhcs/pyqt_gui/services/ui_bridge_object_state.py @@ -688,9 +688,7 @@ def mutate( self, request: UiObjectStateFieldMutationRequest, ) -> UiObjectStateFieldMutationResult: - state = object_state_for_agent_scope_id( - request.object_state_scope_id - ) + state = object_state_for_agent_scope_id(request.object_state_scope_id) if state is None: return self._error_result( request, diff --git a/openhcs/pyqt_gui/services/ui_bridge_plate_manager.py b/openhcs/pyqt_gui/services/ui_bridge_plate_manager.py index ba59cd15a6..61bea973f3 100644 --- a/openhcs/pyqt_gui/services/ui_bridge_plate_manager.py +++ b/openhcs/pyqt_gui/services/ui_bridge_plate_manager.py @@ -116,6 +116,7 @@ PlateManagerOrchestratorCodeDocumentIdentity.require_value() ) + class PlateManagerBridgeProviderSet(UiBridgeProviderSetABC): """Register all PlateManager surfaces with a UI bridge registry.""" diff --git a/openhcs/pyqt_gui/services/ui_bridge_session_actions.py b/openhcs/pyqt_gui/services/ui_bridge_session_actions.py index d73f4e997b..5a90d8d5ee 100644 --- a/openhcs/pyqt_gui/services/ui_bridge_session_actions.py +++ b/openhcs/pyqt_gui/services/ui_bridge_session_actions.py @@ -68,7 +68,9 @@ def summary(self, action_id: str) -> UiActionSummary: ) return UiActionSummary( schema_version=SCHEMA_VERSION, - identity=UiActionIdentity(widget_id=self.identity.widget_id, action_id=action_id), + identity=UiActionIdentity( + widget_id=self.identity.widget_id, action_id=action_id + ), title=operation.button_label(self._session), enabled=error is None, disabled_error=error, @@ -92,7 +94,11 @@ def dispatch(self, request: UiActionInvokeRequest) -> UiActionDispatch: ) if session_request is None: return UiActionDispatch( - errors=(AgentError(code="ui_action_cancelled", message="The user cancelled."),) + errors=( + AgentError( + code="ui_action_cancelled", message="The user cancelled." + ), + ) ) result = self._session.invoke(operation, session_request) return UiActionDispatch( diff --git a/openhcs/pyqt_gui/services/ui_bridge_windows.py b/openhcs/pyqt_gui/services/ui_bridge_windows.py index 5b4c0b55cd..1afbc7eeb6 100644 --- a/openhcs/pyqt_gui/services/ui_bridge_windows.py +++ b/openhcs/pyqt_gui/services/ui_bridge_windows.py @@ -2885,7 +2885,6 @@ def _window_supports_action( return action.is_supported(window.managed_window_action_capabilities()) - @dataclass(frozen=True, slots=True) class MainWindowBridgeProviderSet(UiBridgeProviderSetABC): """Provider set for main-window generic UI projections.""" diff --git a/openhcs/pyqt_gui/session_rendering.py b/openhcs/pyqt_gui/session_rendering.py index 76b7be501c..3ce59d0fcc 100644 --- a/openhcs/pyqt_gui/session_rendering.py +++ b/openhcs/pyqt_gui/session_rendering.py @@ -68,9 +68,7 @@ def selection_scope_ids(self) -> tuple[str, ...]: def request_for(self, operation: type[SessionOperation]) -> object | None: if issubclass(operation, PromptedOperation): return self.session.renderer.prompt(operation) - return operation.request_for_selection( - self.session, self.selection_scope_ids() - ) + return operation.request_for_selection(self.session, self.selection_scope_ids()) def handle_button_action(self, operation_id: str) -> None: commit_focused_widget_edits() diff --git a/openhcs/pyqt_gui/widgets/image_browser.py b/openhcs/pyqt_gui/widgets/image_browser.py index c705ee166f..469ccb5b67 100644 --- a/openhcs/pyqt_gui/widgets/image_browser.py +++ b/openhcs/pyqt_gui/widgets/image_browser.py @@ -58,7 +58,6 @@ logger = logging.getLogger(__name__) - def _streaming_config_field_names() -> tuple[str, ...]: """Registered streaming-config field names used by image-browser controls.""" return StreamingConfig.supported_config_keys() @@ -1305,7 +1304,9 @@ def _stream_images_to_viewer(self, filenames: list, config_key: str): read_backend=read_backend, ) ) - logger.info(f"Streaming {len(filenames)} images to {config.viewer_family.display_name}...") + logger.info( + f"Streaming {len(filenames)} images to {config.viewer_family.display_name}..." + ) def _show_streaming_error(self, viewer_name: str, error_msg: str): """Show streaming error in UI thread.""" @@ -1512,7 +1513,9 @@ def _update_plate_view(self): grid_axis = require_grid_axis() parser = self.orchestrator.microscope_handler.parser coord_to_well = { - grid_axis.grid_position(*parser.extract_component_coordinates(well_id)): well_id + grid_axis.grid_position( + *parser.extract_component_coordinates(well_id) + ): well_id for well_id in well_ids } plate_dimensions = ( diff --git a/openhcs/pyqt_gui/widgets/pipeline_editor.py b/openhcs/pyqt_gui/widgets/pipeline_editor.py index 9d482ebc6b..4d04674f89 100644 --- a/openhcs/pyqt_gui/widgets/pipeline_editor.py +++ b/openhcs/pyqt_gui/widgets/pipeline_editor.py @@ -468,7 +468,9 @@ def code_document_source(self, clean: bool = True) -> str: pipeline_config=( authored_pipeline_config(scope_id) if scope_id else PipelineConfig() ), - pipeline_steps=self.session.pipeline_steps(scope_id) if scope_id else [], + pipeline_steps=( + self.session.pipeline_steps(scope_id) if scope_id else [] + ), ), clean_mode=clean, ) @@ -523,7 +525,9 @@ def selected_step_scope_ids(self) -> tuple[str, ...]: return () return tuple(self.step_scope_id(step) for step in self.get_selected_items()) - def get_item_insert_index(self, item: FunctionStep, scope_key: str) -> Optional[int]: + def get_item_insert_index( + self, item: FunctionStep, scope_key: str + ) -> Optional[int]: del item token = scope_key.rsplit("::", 1)[-1] parts = token.rsplit("_", 1) @@ -540,7 +544,9 @@ def _get_item_scope_id(self, item: FunctionStep, index: int) -> str: return self.step_scope_id(item) @override - def _format_item_content(self, item: FunctionStep, index: int, context: None) -> str: + def _format_item_content( + self, item: FunctionStep, index: int, context: None + ) -> str: display_text, _ = self.format_item_for_display(item, context, step_index=index) return display_text @@ -579,7 +585,9 @@ def _get_list_placeholder(self) -> tuple[str, None] | None: @override def prepare_list_update(self) -> None: if self.current_plate: - ScopeTokenService.seed_from_objects(self.current_plate, self.displayed_steps) + ScopeTokenService.seed_from_objects( + self.current_plate, self.displayed_steps + ) return None @override @@ -611,7 +619,9 @@ def on_time_travel_complete(self, dirty_states, triggering_scope): == PipelineScopeIdentity.from_plate_scope(self.current_plate).scope_id for scope_id, _state in (dirty_states or ()) ): - GuiEventBusBroadcaster(self.event_bus).pipeline_changed(self.displayed_steps) + GuiEventBusBroadcaster(self.event_bus).pipeline_changed( + self.displayed_steps + ) # -- context for step editors --------------------------------------------- @@ -673,7 +683,9 @@ def open_step_editor( function_invocation_badge_provider=( None if step_index is None - else self.function_presentation.badge_provider(step, step_index=step_index) + else self.function_presentation.badge_provider( + step, step_index=step_index + ) ), compiled_artifact_inspection_provider=self.session.compiled_inspection, before_mutation=( @@ -724,7 +736,9 @@ def _action_paste_steps(self): self.status_message.emit("No dataset selected") return selected_rows = [index.row() for index in self.item_list.selectedIndexes()] - insert_after = max(selected_rows) if selected_rows else len(self.displayed_steps) - 1 + insert_after = ( + max(selected_rows) if selected_rows else len(self.displayed_steps) - 1 + ) pasted = [copy.deepcopy(step) for step in self._clipboard_steps] self.session.insert_steps(self.current_plate, insert_after + 1, pasted) self.status_message.emit( diff --git a/openhcs/pyqt_gui/widgets/shared/services/pipeline_editor_workflows.py b/openhcs/pyqt_gui/widgets/shared/services/pipeline_editor_workflows.py index e519650835..932bd9053b 100644 --- a/openhcs/pyqt_gui/widgets/shared/services/pipeline_editor_workflows.py +++ b/openhcs/pyqt_gui/widgets/shared/services/pipeline_editor_workflows.py @@ -221,11 +221,15 @@ def run_command( ) -> None: scope_id = self.editor.current_plate if not scope_id: - self.editor.status_message.emit("Select a dataset before running debug mode.") + self.editor.status_message.emit( + "Select a dataset before running debug mode." + ) return session = self.editor.session command_label = command_type.value.replace("_", " ") - self.editor.status_message.emit(f"Submitting debug {command_label} for {scope_id}") + self.editor.status_message.emit( + f"Submitting debug {command_label} for {scope_id}" + ) cursor = self._replay_cursor(command_type) session.debug_terminal_summaries.pop(scope_id, None) session.start( @@ -234,7 +238,9 @@ def run_command( command_type=command_type, pause_step_indices=self.pause_step_indices(), start_step_index=0 if cursor is None else cursor.step_index, - start_after_invocation_key=None if cursor is None else cursor.invocation_key, + start_after_invocation_key=( + None if cursor is None else cursor.invocation_key + ), ) def pause_step_indices(self) -> tuple[int, ...]: @@ -278,7 +284,9 @@ async def inspect() -> None: view_model = await session.debug_runs.inspect_runtime( debug_session_id=active.debug_session_id, ) - session.main_thread.post(lambda: self._render_runtime_inspection(view_model)) + session.main_thread.post( + lambda: self._render_runtime_inspection(view_model) + ) session.start(inspect) diff --git a/openhcs/runtime/fiji_viewer_server.py b/openhcs/runtime/fiji_viewer_server.py index 3658337c5e..abf30f88e2 100644 --- a/openhcs/runtime/fiji_viewer_server.py +++ b/openhcs/runtime/fiji_viewer_server.py @@ -462,7 +462,9 @@ def _require_consistent_scalar_metadata( ) -> None: if component not in item.metadata: return - normalized = ViewerComponentMetadataNormalizer(self.declared_axes).normalize_value( + normalized = ViewerComponentMetadataNormalizer( + self.declared_axes + ).normalize_value( component, item.metadata[component], ) @@ -1002,8 +1004,7 @@ def expanded_dimensions(self) -> bool: return False return ( len(self.coordinates.c.values) > len(self.stored.c) - or len(self.coordinates.z.values) - > len(self.stored.z) + or len(self.coordinates.z.values) > len(self.stored.z) or len(self.coordinates.t.values) > len(self.stored.t) ) @@ -1250,7 +1251,9 @@ def batch_message(self) -> "FijiBatchMessage": display_config_payload = fields.required_mapping( ViewerBatchWireField.DISPLAY_CONFIG ) - display_config = FijiDisplaySettings.from_display_payload(display_config_payload) + display_config = FijiDisplaySettings.from_display_payload( + display_config_payload + ) component_axis_semantics = fields.component_axis_semantics( ViewerMappingDisplayConfigInput(display_config_payload), context="Fiji component value domain", @@ -2501,9 +2504,7 @@ def _set_dimension_labels( logger.info( f"🏷️ FIJI SERVER: Setting labels for {len(coordinates.c.values)} channels" ) - for idx, channel_tuple in enumerate( - coordinates.c.values, start=1 - ): + for idx, channel_tuple in enumerate(coordinates.c.values, start=1): label = coordinates.c.label_for_value( channel_tuple, component_names_metadata, @@ -2611,11 +2612,7 @@ def _build_new_hyperstack( logger.info(f"🏷️ FIJI SERVER: Setting dimension labels for {window_key}") # For single-channel images, add channel name to window title since no slider appears - if ( - nChannels == 1 - and coordinates.c.components - and coordinates.c.values - ): + if nChannels == 1 and coordinates.c.components and coordinates.c.values: first_comp = coordinates.c.components[0] first_value_tuple = coordinates.c.values[0] channel_name = component_names_metadata.display_name( @@ -2821,9 +2818,7 @@ def handle(self, request: FijiPayloadHandlerRequest) -> None: logger.info(f"🔬 FIJI SERVER: ROI metadata: {metadata}") logger.info(f"🔬 FIJI SERVER: Channel axis: {request.coordinates.c}") - logger.info( - f"🔬 FIJI SERVER: Slice axis: {request.coordinates.z}" - ) + logger.info(f"🔬 FIJI SERVER: Slice axis: {request.coordinates.z}") logger.info(f"🔬 FIJI SERVER: Frame axis: {request.coordinates.t}") c_value, z_value, t_value = request.coordinates.imagej_position(metadata) diff --git a/openhcs/runtime/viewer_protocol.py b/openhcs/runtime/viewer_protocol.py index fddf72edee..f8c06477f0 100644 --- a/openhcs/runtime/viewer_protocol.py +++ b/openhcs/runtime/viewer_protocol.py @@ -195,7 +195,8 @@ def descend(value): return value return core_schema.no_info_before_validator_function( - descend, super().__get_pydantic_core_schema__(source_type, handler), + descend, + super().__get_pydantic_core_schema__(source_type, handler), ) def __post_init__(self) -> None: @@ -211,7 +212,8 @@ def from_wire_mapping(cls, payload: Mapping[str, object]) -> Self: def to_wire_mapping(self) -> JsonObject: return { - name: value for name, value in super().to_wire_mapping().items() + name: value + for name, value in super().to_wire_mapping().items() if value is not VIEWER_FIELD_ABSENT } | self.wire_overrides() @@ -288,9 +290,15 @@ class ViewerArrayStatistics(ViewerProjectionRecord): nonzero_count: StrictInt | ViewerFieldAbsent = VIEWER_FIELD_ABSENT min: JsonScalar | ViewerFieldAbsent = VIEWER_FIELD_ABSENT max: JsonScalar | ViewerFieldAbsent = VIEWER_FIELD_ABSENT - nonzero_min_coordinate: tuple[StrictInt, ...] | ViewerFieldAbsent = VIEWER_FIELD_ABSENT - nonzero_max_coordinate: tuple[StrictInt, ...] | ViewerFieldAbsent = VIEWER_FIELD_ABSENT - nonzero_example_coordinates: tuple[tuple[StrictInt, ...], ...] | ViewerFieldAbsent = VIEWER_FIELD_ABSENT + nonzero_min_coordinate: tuple[StrictInt, ...] | ViewerFieldAbsent = ( + VIEWER_FIELD_ABSENT + ) + nonzero_max_coordinate: tuple[StrictInt, ...] | ViewerFieldAbsent = ( + VIEWER_FIELD_ABSENT + ) + nonzero_example_coordinates: ( + tuple[tuple[StrictInt, ...], ...] | ViewerFieldAbsent + ) = VIEWER_FIELD_ABSENT nonzero_coordinate_omission_reason: str | ViewerFieldAbsent = VIEWER_FIELD_ABSENT @property @@ -320,28 +328,45 @@ class ViewerShapeCoordinateBounds(ViewerProjectionRecord): class ViewerShapeStatistics(ViewerProjectionRecord): item_count: StrictInt | ViewerFieldAbsent = VIEWER_FIELD_ABSENT shape_payload_count: StrictInt | ViewerFieldAbsent = VIEWER_FIELD_ABSENT - missing_source_spatial_shape_count: StrictInt | ViewerFieldAbsent = VIEWER_FIELD_ABSENT + missing_source_spatial_shape_count: StrictInt | ViewerFieldAbsent = ( + VIEWER_FIELD_ABSENT + ) shape_coordinate_count: StrictInt | ViewerFieldAbsent = VIEWER_FIELD_ABSENT - shape_out_of_source_bounds_count: StrictInt | ViewerFieldAbsent = VIEWER_FIELD_ABSENT - shape_coordinate_bounds_yx: ViewerShapeCoordinateBounds | ViewerFieldAbsent = VIEWER_FIELD_ABSENT + shape_out_of_source_bounds_count: StrictInt | ViewerFieldAbsent = ( + VIEWER_FIELD_ABSENT + ) + shape_coordinate_bounds_yx: ViewerShapeCoordinateBounds | ViewerFieldAbsent = ( + VIEWER_FIELD_ABSENT + ) def returned_member_count(self, returned_count: int) -> int: - return (returned_count if self.shape_payload_count is VIEWER_FIELD_ABSENT - else self.shape_payload_count) + return ( + returned_count + if self.shape_payload_count is VIEWER_FIELD_ABSENT + else self.shape_payload_count + ) def wire_overrides(self) -> JsonObject: overrides = super().wire_overrides() if self.shape_coordinate_bounds_yx is not VIEWER_FIELD_ABSENT: - overrides["shape_coordinate_bounds_yx"] = self.shape_coordinate_bounds_yx.to_wire_mapping() + overrides["shape_coordinate_bounds_yx"] = ( + self.shape_coordinate_bounds_yx.to_wire_mapping() + ) return overrides @dataclass(frozen=True, kw_only=True) class ViewerSourceSpatialSummary(ViewerProjectionRecord): - source_voxel_spacing: SourceVoxelSpacing | None | ViewerFieldAbsent = VIEWER_FIELD_ABSENT + source_voxel_spacing: SourceVoxelSpacing | None | ViewerFieldAbsent = ( + VIEWER_FIELD_ABSENT + ) - spatial_origin_yx: tuple[StrictInt, StrictInt] | None | ViewerFieldAbsent = VIEWER_FIELD_ABSENT - source_spatial_shape_yx: tuple[StrictInt, StrictInt] | None | ViewerFieldAbsent = VIEWER_FIELD_ABSENT + spatial_origin_yx: tuple[StrictInt, StrictInt] | None | ViewerFieldAbsent = ( + VIEWER_FIELD_ABSENT + ) + source_spatial_shape_yx: tuple[StrictInt, StrictInt] | None | ViewerFieldAbsent = ( + VIEWER_FIELD_ABSENT + ) @property def voxel_spacing(self) -> SourceVoxelSpacing: @@ -358,12 +383,17 @@ def source_domain(self) -> ViewerSourceSpatialDomainPayload: def require_uncropped_source_shape(self) -> tuple[int, int]: domain = self.source_domain if domain.origin_yx != (0, 0): - raise ValueError("Image record requires explicit uncropped pixel placement.") + raise ValueError( + "Image record requires explicit uncropped pixel placement." + ) return domain.required_source_shape_yx(source_label="image record") + @dataclass(frozen=True, kw_only=True) class ViewerAggregateComponentSummary(ViewerProjectionRecord): - aggregate_component_values: Mapping[str, tuple[ViewerComponentValue, ...]] | None | ViewerFieldAbsent = VIEWER_FIELD_ABSENT + aggregate_component_values: ( + Mapping[str, tuple[ViewerComponentValue, ...]] | None | ViewerFieldAbsent + ) = VIEWER_FIELD_ABSENT @property def aggregate_values(self) -> Mapping[str, tuple[ViewerComponentValue, ...]]: @@ -374,16 +404,25 @@ def validate_record(self) -> None: if any(not values for values in self.aggregate_values.values()): raise ValueError("Viewer aggregate component domains must not be empty.") - def require_plane_components(self) -> Mapping[str, tuple[ViewerComponentValue, ...]]: - if self.aggregate_component_values is VIEWER_FIELD_ABSENT or self.aggregate_component_values is None: - raise ValueError("Image record requires its explicit plane component domain.") + def require_plane_components( + self, + ) -> Mapping[str, tuple[ViewerComponentValue, ...]]: + if ( + self.aggregate_component_values is VIEWER_FIELD_ABSENT + or self.aggregate_component_values is None + ): + raise ValueError( + "Image record requires its explicit plane component domain." + ) return self.aggregate_component_values @dataclass(frozen=True, kw_only=True) class ViewerPayloadSummary( - ViewerAggregateComponentSummary, ViewerSourceSpatialSummary, - ViewerShapeStatistics, ViewerArrayStatistics, + ViewerAggregateComponentSummary, + ViewerSourceSpatialSummary, + ViewerShapeStatistics, + ViewerArrayStatistics, ): """One original native summary; no agent-side mirror or raw-map carrier.""" @@ -393,12 +432,15 @@ class ViewerPayloadSummary( payload_type: str | ViewerFieldAbsent = VIEWER_FIELD_ABSENT def coordinate_components(self) -> Mapping[str, ViewerComponentValue]: - from openhcs.runtime.viewer_component_system import ViewerComponentMetadataPayload + from openhcs.runtime.viewer_component_system import ( + ViewerComponentMetadataPayload, + ) if self.components is VIEWER_FIELD_ABSENT: raise ValueError("Viewer payload summary missing components.") return ViewerComponentMetadataPayload.component_map( - self.components, context="viewer payload summary", + self.components, + context="viewer payload summary", ) @property @@ -410,9 +452,13 @@ def require_full_image_window(self) -> None: shape = self.require_positive_shape(rank=3) source_shape = self.require_uncropped_source_shape() except ValueError as error: - raise ValueError("Image record requires an explicit full three-axis image window.") from error + raise ValueError( + "Image record requires an explicit full three-axis image window." + ) from error if source_shape != shape[-2:]: - raise ValueError("Image record requires an explicit full three-axis image window.") + raise ValueError( + "Image record requires an explicit full three-axis image window." + ) @property def full_image_plane_count(self) -> int: @@ -428,8 +474,12 @@ class ViewerArrayValueSummary(ViewerArrayStatistics): included: bool | ViewerFieldAbsent = VIEWER_FIELD_ABSENT omitted_reason: str | ViewerFieldAbsent = VIEWER_FIELD_ABSENT max_array_elements: StrictInt | ViewerFieldAbsent = VIEWER_FIELD_ABSENT - slice_ranges: tuple[tuple[StrictInt, StrictInt], ...] | ViewerFieldAbsent = VIEWER_FIELD_ABSENT - requested_slice_ranges: tuple[tuple[StrictInt, StrictInt], ...] | ViewerFieldAbsent = VIEWER_FIELD_ABSENT + slice_ranges: tuple[tuple[StrictInt, StrictInt], ...] | ViewerFieldAbsent = ( + VIEWER_FIELD_ABSENT + ) + requested_slice_ranges: ( + tuple[tuple[StrictInt, StrictInt], ...] | ViewerFieldAbsent + ) = VIEWER_FIELD_ABSENT @property def protocol_supported(self) -> bool: @@ -449,10 +499,15 @@ def shape_element_count(self) -> int | None: count *= value return count - def presentation_omission_reason(self, *, include_requested: bool | None) -> str | None: + def presentation_omission_reason( + self, *, include_requested: bool | None + ) -> str | None: reason = self.optional(self.omitted_reason) - if (include_requested is False and reason == "max_array_elements_exceeded" - and self.max_array_elements == 0): + if ( + include_requested is False + and reason == "max_array_elements_exceeded" + and self.max_array_elements == 0 + ): return "array_values_not_requested" return reason @@ -468,7 +523,9 @@ class ViewerPayloadContent(ViewerProjectionRecord): aggregate_axis_indices: tuple[StrictInt, ...] = () summary: ViewerPayloadSummary = field(default_factory=ViewerPayloadSummary) array_values: tuple[JsonValue, ...] = () - array_value_summary: ViewerArrayValueSummary = field(default_factory=ViewerArrayValueSummary) + array_value_summary: ViewerArrayValueSummary = field( + default_factory=ViewerArrayValueSummary + ) @property def is_image(self) -> bool: @@ -780,9 +837,9 @@ def from_response( payload[ViewerSettleField.ACTIVE_ROUTE_WORK_UNIT_COUNT.value] ), active_route_work_unit_active=active_work_unit_value, - processed_intake_item_count=cast(int, payload[ - ViewerSettleField.PROCESSED_INTAKE_ITEM_COUNT.value - ]), + processed_intake_item_count=cast( + int, payload[ViewerSettleField.PROCESSED_INTAKE_ITEM_COUNT.value] + ), intake_item_active=payload[ViewerSettleField.INTAKE_ITEM_ACTIVE.value], ) @@ -1671,7 +1728,9 @@ def __post_init__(self) -> None: if self.timeout <= 0: raise ValueError("Viewer control timeout must be positive.") - def to_wire_mapping(self, *, deadline: OperationDeadline | None = None) -> dict[str, object]: + def to_wire_mapping( + self, *, deadline: OperationDeadline | None = None + ) -> dict[str, object]: """Project this typed request to primitive wire fields at send.""" request: dict[str, object] = { @@ -1687,8 +1746,13 @@ def to_wire_mapping(self, *, deadline: OperationDeadline | None = None) -> dict[ if isinstance(payload, ViewerWindowControlRequest): payload = replace(payload, operation_deadline=None) request[ViewerControlResponseField.PAYLOAD.value] = payload - deadline = deadline or self.operation_deadline or OperationDeadline.after_milliseconds( - max(1, int(self.timeout * 1000)), operation="viewer control request", + deadline = ( + deadline + or self.operation_deadline + or OperationDeadline.after_milliseconds( + max(1, int(self.timeout * 1000)), + operation="viewer control request", + ) ) return ControlRequestHeader.with_observation_budget(request, deadline) @@ -1701,7 +1765,8 @@ def send(self) -> ViewerControlResponse: socket = None try: deadline = self.operation_deadline or OperationDeadline.after_milliseconds( - max(1, int(self.timeout * 1000)), operation="viewer control request", + max(1, int(self.timeout * 1000)), + operation="viewer control request", ) context = zmq.Context() socket = context.socket(zmq.REQ) @@ -1828,7 +1893,9 @@ def start_viewer_now(self) -> None: port = self.required_port self.prepare_fresh_viewer_start() if self.lifecycle_state.is_active: - logger.warning("%s viewer is already running.", self.viewer_process_label) + logger.warning( + "%s viewer is already running.", self.viewer_process_label + ) return logger.info( "🔬 VISUALIZER: Starting %s %s viewer (detached) on port %s", @@ -1962,12 +2029,15 @@ def matches_requested_process_launch( """ owns_process = self.owned_viewer_process_is_alive() try: - active = self.process_launch if owns_process else self.active_process_launch() + active = ( + self.process_launch if owns_process else self.active_process_launch() + ) return requested.matches_existing_viewer(active, owns_process=owns_process) except (RuntimeError, TypeError, ValueError, KeyError, zmq.ZMQError) as error: logging.getLogger(type(self).__module__).warning( "%s viewer process-launch check failed: %s", - self.viewer_process_label, error, + self.viewer_process_label, + error, ) return False diff --git a/openhcs/runtime/zmq_compilation.py b/openhcs/runtime/zmq_compilation.py index cf761b8c9e..514d17b8c8 100644 --- a/openhcs/runtime/zmq_compilation.py +++ b/openhcs/runtime/zmq_compilation.py @@ -169,7 +169,9 @@ def compile_fresh(self) -> ZMQCompilationResult: output_dataset_root=( None if output_dataset_root is None else str(output_dataset_root) ), - auto_add_output_dataset=bool(self.resolved_config.auto_add_output_dataset), + auto_add_output_dataset=bool( + self.resolved_config.auto_add_output_dataset + ), ), ) self.progress_emitter.compiled_init_started( diff --git a/openhcs/runtime/zmq_control.py b/openhcs/runtime/zmq_control.py index ca36348501..6e359838ae 100644 --- a/openhcs/runtime/zmq_control.py +++ b/openhcs/runtime/zmq_control.py @@ -442,16 +442,21 @@ def handle(self, message: dict, context: ZMQControlRequestContext) -> dict: CustomFunctionRegistrationResult, FunctionCatalogPreparationHandle, ) + try: request = self.request_type.from_control_payload(message) except Exception as error: return self.error_response(error) try: context.require_function_catalog_preparation().observe( - FunctionCatalogPreparationHandle(request.connection, request.require_server_identity()) + FunctionCatalogPreparationHandle( + request.connection, request.require_server_identity() + ) ).require_ready() return CustomFunctionRegistrationControlResponse( - value=context.require_function_catalog().register_custom_function(request), + value=context.require_function_catalog().register_custom_function( + request + ), ).to_control_response() except Exception as error: return CustomFunctionRegistrationControlResponse( @@ -468,12 +473,16 @@ class CustomFunctionRegistrationObservationMessageStrategy(ZMQControlMessageStra registry_key = request_type.message_type.value def handle(self, message: dict, context: ZMQControlRequestContext) -> dict: - from openhcs.agent.dto.functions import CustomFunctionRegistrationObservationControlResponse + from openhcs.agent.dto.functions import ( + CustomFunctionRegistrationObservationControlResponse, + ) try: request = self.request_type.from_control_payload(message) return CustomFunctionRegistrationObservationControlResponse( - value=context.require_function_catalog().observe_custom_function_registration(request.handle), + value=context.require_function_catalog().observe_custom_function_registration( + request.handle + ), ).to_control_response() except Exception as error: return self.error_response(error) diff --git a/openhcs/runtime/zmq_execution_client.py b/openhcs/runtime/zmq_execution_client.py index 9207902662..0a9092dbf2 100644 --- a/openhcs/runtime/zmq_execution_client.py +++ b/openhcs/runtime/zmq_execution_client.py @@ -777,6 +777,7 @@ def custom_function_registration_destination( from openhcs.agent.dto.functions import ( CustomFunctionRegistrationDestinationControlResponse, ) + response = self._send_function_catalog_exchange( request, operation_deadline=operation_deadline, @@ -788,17 +789,23 @@ def custom_function_registration_destination( ) def observe_custom_function_registration( - self, handle: CustomFunctionRegistrationHandle, + self, + handle: CustomFunctionRegistrationHandle, ) -> CustomFunctionRegistrationObservation: """One read-only exchange; never register/load the requested source.""" from openhcs.agent.dto.functions import ( CustomFunctionRegistrationObservationRequest, CustomFunctionRegistrationObservationControlResponse, ) + response = self._send_function_catalog_exchange( CustomFunctionRegistrationObservationRequest(handle), ) - return CustomFunctionRegistrationObservationControlResponse.from_control_response(response).value + return ( + CustomFunctionRegistrationObservationControlResponse.from_control_response( + response + ).value + ) def function_catalog_preparation( self, @@ -1012,7 +1019,9 @@ def serialize_task( task: OpenHCSExecutionSubmission, config=None, ) -> ExecuteRequest: - return self.serialize_prepared_request(ZMQExecutionRequestBuilder.from_task(task)) + return self.serialize_prepared_request( + ZMQExecutionRequestBuilder.from_task(task) + ) def serialize_prepared_request( self, request_builder: ZMQExecutionRequestBuilder diff --git a/openhcs/runtime/zmq_execution_server.py b/openhcs/runtime/zmq_execution_server.py index 84ee87cfae..445ad5bf2c 100644 --- a/openhcs/runtime/zmq_execution_server.py +++ b/openhcs/runtime/zmq_execution_server.py @@ -302,7 +302,10 @@ def _forward_worker_progress(self, worker_queue) -> None: # by a worker lane. They carry topology but never a worker claim. if event.axis_id: self._require_worker_claim(event, assignments) - elif event.worker_slot is not None or event.owned_partition_values is not None: + elif ( + event.worker_slot is not None + or event.owned_partition_values is not None + ): raise ValueError( f"Execution-level progress cannot carry a worker claim: {event}" ) @@ -396,7 +399,9 @@ def _execute_pipeline( execution_id: str, request_payload: ZMQExecutionRequestPayload, ): - logger.info("[%s] Starting subject %s", execution_id, request_payload.subject_id) + logger.info( + "[%s] Starting subject %s", execution_id, request_payload.subject_id + ) request_payload.compile_control.validate() self._cleanup_compiled_artifacts() @@ -645,7 +650,9 @@ def _initialize_orchestrator( if execution_bundle is None: if resolved_config is None: - raise ValueError("Fresh initialization requires resolved configuration.") + raise ValueError( + "Fresh initialization requires resolved configuration." + ) orchestrator = PipelineOrchestrator( dataset_root=Path(dataset_root), pipeline_config=pipeline_config, @@ -696,9 +703,7 @@ def _partition_values_for_execution( execution_bundle.axis_ids if execution_bundle is not None else tuple( - orchestrator.get_component_keys( - AxisFamily.active().partition_axis() - ) + orchestrator.get_component_keys(AxisFamily.active().partition_axis()) ) ) return debug_execution_policy.axis_filter_for_available(available_axis_ids) diff --git a/openhcs/runtime/zmq_execution_signature.py b/openhcs/runtime/zmq_execution_signature.py index 52ea245922..d86f1c4491 100644 --- a/openhcs/runtime/zmq_execution_signature.py +++ b/openhcs/runtime/zmq_execution_signature.py @@ -84,7 +84,9 @@ def to_transport(self) -> dict[str, Any]: params: dict[str, Any] = {} if self.partition_values is not None: - params[ZMQAuxiliaryParamField.PARTITION_VALUES.value] = list(self.partition_values) + params[ZMQAuxiliaryParamField.PARTITION_VALUES.value] = list( + self.partition_values + ) if self.debug_execution_config is not None: params.update(self.debug_execution_config.to_config_params()) if self.runtime_observation_export_path is not None: diff --git a/openhcs/runtime/zmq_orchestrator_environment.py b/openhcs/runtime/zmq_orchestrator_environment.py index 4f2cb40033..76a8695d98 100644 --- a/openhcs/runtime/zmq_orchestrator_environment.py +++ b/openhcs/runtime/zmq_orchestrator_environment.py @@ -48,7 +48,9 @@ def prepared_dataset_root(self, storage_registry) -> str: from openhcs.core.dataset_sources.dataset_roots import DatasetRootRule dataset_root = str( - self.subject_id if self.execution_subject_id is None else self.execution_subject_id + self.subject_id + if self.execution_subject_id is None + else self.execution_subject_id ) return DatasetRootRule.for_dataset(dataset_root).prepare_storage( dataset_root, storage_registry diff --git a/openhcs/runtime/zmq_server_hooks.py b/openhcs/runtime/zmq_server_hooks.py index 831d552785..1811e5079c 100644 --- a/openhcs/runtime/zmq_server_hooks.py +++ b/openhcs/runtime/zmq_server_hooks.py @@ -43,9 +43,7 @@ def attach( summary = {} record.results_summary = summary - output_summary = record.get_extra( - ExecutionOutputSummary.EXECUTION_RECORD_KEY - ) + output_summary = record.get_extra(ExecutionOutputSummary.EXECUTION_RECORD_KEY) if output_summary is None: output_summary = ExecutionOutputSummary() elif not isinstance(output_summary, ExecutionOutputSummary): diff --git a/scripts/annotate_wells_with_experimental_metadata.py b/scripts/annotate_wells_with_experimental_metadata.py index 102c30f959..3c7b63b0de 100644 --- a/scripts/annotate_wells_with_experimental_metadata.py +++ b/scripts/annotate_wells_with_experimental_metadata.py @@ -23,7 +23,10 @@ # Add openhcs to path sys.path.insert(0, str(Path(__file__).parent.parent)) -from openhcs.processing.backends.experimental_analysis.analysis import read_plate_layout, load_plate_groups +from openhcs.processing.backends.experimental_analysis.analysis import ( + read_plate_layout, + load_plate_groups, +) from polystore.metadata_writer import AtomicMetadataWriter, get_metadata_path diff --git a/scripts/capture_media_gallery.py b/scripts/capture_media_gallery.py index 444196de2b..2629aec0a8 100755 --- a/scripts/capture_media_gallery.py +++ b/scripts/capture_media_gallery.py @@ -847,7 +847,9 @@ def __init__(self, request: GallerySourceCaptureRequest) -> None: ) def windows(self) -> UiWindowCatalog: - result = self._service.invoke(UiBridgeListWindowsOperation, connection=self.connection) + result = self._service.invoke( + UiBridgeListWindowsOperation, connection=self.connection + ) if result.errors: raise MediaGalleryError(f"UI window catalogue failed: {result.errors!r}") return result @@ -859,7 +861,9 @@ def visible_window_ids(self) -> frozenset[str]: def object_state_scopes(self) -> UiObjectStateScopeCatalog: request = UiObjectStateScopeListRequest.from_fields() - result = self._service.invoke(UiBridgeListObjectStateScopesOperation, request, self.connection) + result = self._service.invoke( + UiBridgeListObjectStateScopesOperation, request, self.connection + ) if result.errors: raise MediaGalleryError( f"ObjectState scope catalogue failed: {result.errors!r}" @@ -876,7 +880,9 @@ def navigate_scope( field_path=role.navigation_field_path, create_if_missing=True, ) - result = self._service.invoke(UiBridgeNavigateWindowOperation, request, self.connection) + result = self._service.invoke( + UiBridgeNavigateWindowOperation, request, self.connection + ) if ( result.errors or not result.focused @@ -888,7 +894,9 @@ def navigate_scope( return result def action_catalog(self) -> UiActionCatalog: - result = self._service.invoke(UiBridgeListActionsOperation, connection=self.connection) + result = self._service.invoke( + UiBridgeListActionsOperation, connection=self.connection + ) if result.errors: raise MediaGalleryError(f"UI action catalogue failed: {result.errors!r}") return result @@ -917,7 +925,9 @@ def invoke_plate_manager_action( observed_selection_revision_token=summary.selection_revision_token, require_confirmation=False, ) - result = self._service.invoke(UiBridgeInvokeActionOperation, request, self.connection) + result = self._service.invoke( + UiBridgeInvokeActionOperation, request, self.connection + ) if result.errors or not result.receipt.accepted: raise MediaGalleryError( f"UI action {target.action.operation_id!r} was rejected: {result!r}" @@ -938,7 +948,9 @@ def widget_tree( max_depth=None, max_nodes=2000, ) - result = self._service.invoke(UiBridgeWidgetTreeOperation, request, self.connection) + result = self._service.invoke( + UiBridgeWidgetTreeOperation, request, self.connection + ) if result.errors or not result.projected: raise MediaGalleryError( f"Widget-tree projection failed for {window_id!r}: {result!r}" @@ -1003,7 +1015,9 @@ def invoke_widget_action( action_kind=action_kind.value, target_index=target_index, ) - result = self._service.invoke(UiBridgeInvokeWidgetActionOperation, request, self.connection) + result = self._service.invoke( + UiBridgeInvokeWidgetActionOperation, request, self.connection + ) if result.errors or not result.receipt.accepted: raise MediaGalleryError( f"Widget action {summary.path_id!r} was rejected: {result!r}" @@ -1119,7 +1133,9 @@ def _poll_until(self, condition: Callable[[], bool], description: str) -> None: def close_window(self, window_id: str) -> None: request = UiWindowCloseRequest.from_fields(window_id=window_id) - result = self._service.invoke(UiBridgeCloseWindowOperation, request, self.connection) + result = self._service.invoke( + UiBridgeCloseWindowOperation, request, self.connection + ) if result.errors or not result.closed: raise MediaGalleryError( f"Could not close UI window {window_id!r}: {result!r}" @@ -1142,7 +1158,9 @@ def capture_window( output_dir_path=str(target_path.parent), create_if_missing=create_if_missing, ) - response = self._service.invoke(UiBridgeSnapshotWindowOperation, snapshot_request, self.connection) + response = self._service.invoke( + UiBridgeSnapshotWindowOperation, snapshot_request, self.connection + ) return retain_native_snapshot_source(self.request, response) diff --git a/scripts/check_basicpy_source.py b/scripts/check_basicpy_source.py index 6af9dfc94c..7eadd4a76c 100644 --- a/scripts/check_basicpy_source.py +++ b/scripts/check_basicpy_source.py @@ -153,21 +153,24 @@ def test_fields_extend_existing_projection_and_metadata_owners(self): for node in ast.walk(owner) if isinstance(node, ast.Call) ] - self.assertIn( - "source.metadata.collapse_leading_plane_axis", calls - ) + self.assertIn("source.metadata.collapse_leading_plane_axis", calls) self.assertIn("source.metadata.require_independent_observation_axis", calls) - metadata_tree = ast.parse((ROOT / "openhcs/core/runtime_image_values.py").read_text()) + metadata_tree = ast.parse( + (ROOT / "openhcs/core/runtime_image_values.py").read_text() + ) metadata_owner = next( - node for node in metadata_tree.body + node + for node in metadata_tree.body if isinstance(node, ast.ClassDef) and node.name == "ImagePayloadMetadata" ) metadata_calls = { - ast.unparse(node.func) for node in ast.walk(metadata_owner) + ast.unparse(node.func) + for node in ast.walk(metadata_owner) if isinstance(node, ast.Call) } declared_method_names = [ - node.name for node in metadata_owner.body + node.name + for node in metadata_owner.body if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)) ] self.assertEqual(len(declared_method_names), len(set(declared_method_names))) diff --git a/scripts/figures/generate_figure_overlays.py b/scripts/figures/generate_figure_overlays.py index 012cb7e89c..64f6e20a1f 100644 --- a/scripts/figures/generate_figure_overlays.py +++ b/scripts/figures/generate_figure_overlays.py @@ -34,7 +34,10 @@ sys.path.insert(0, str(Path(__file__).parent.parent)) from openhcs.core.roi import load_rois_from_zip, PolygonShape -from openhcs.processing.backends.experimental_analysis.analysis import read_plate_layout, load_plate_groups +from openhcs.processing.backends.experimental_analysis.analysis import ( + read_plate_layout, + load_plate_groups, +) logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s") logger = logging.getLogger(__name__) diff --git a/scripts/master_multi_plate_demo.py b/scripts/master_multi_plate_demo.py index 723b86db33..38ded7030e 100644 --- a/scripts/master_multi_plate_demo.py +++ b/scripts/master_multi_plate_demo.py @@ -152,9 +152,7 @@ def resolve_identity(target: StreamProducerIdentity) -> StreamProducerIdentity: "Declared visual step has no compiled viewer stream." ) - artifact_output_kind = ( - CompiledStepPlan.ARTIFACT_OUTPUT_KIND - ) + artifact_output_kind = CompiledStepPlan.ARTIFACT_OUTPUT_KIND target_outputs = [ output for output in (target_step.get("artifact_outputs") or ()) diff --git a/scripts/mcp_assay_showcase.py b/scripts/mcp_assay_showcase.py index f534cc61b2..afc1a743de 100644 --- a/scripts/mcp_assay_showcase.py +++ b/scripts/mcp_assay_showcase.py @@ -534,9 +534,7 @@ def _primary_object_source(plate_path: Path, output_path: Path) -> str: header = _common_source_header( plate_path, output_path, - processing_imports=( - "from openhcs.domains.microscopy.axes import Microscopy" - ), + processing_imports=("from openhcs.domains.microscopy.axes import Microscopy"), extra_imports=( """from openhcs.processing.backends.analysis.cell_counting_cpu import ( DetectionMethod, diff --git a/scripts/mcp_memory_diagnostic.py b/scripts/mcp_memory_diagnostic.py index a2359c9657..5551b834f9 100644 --- a/scripts/mcp_memory_diagnostic.py +++ b/scripts/mcp_memory_diagnostic.py @@ -171,7 +171,8 @@ def capture(cls, *, collect: bool) -> ProcessMemoryReceipt: for name, value in (line.split(":", 1),) } full_pressure = next( - line for line in Path("/proc/pressure/memory").read_text().splitlines() + line + for line in Path("/proc/pressure/memory").read_text().splitlines() if line.startswith("full ") ) pressure = dict(part.split("=", 1) for part in full_pressure.split()[1:]) @@ -211,11 +212,11 @@ def capture(cls, *, collect: bool) -> ProcessMemoryReceipt: if memory_type.loaded_module() is not None ), catalog_entries=catalog_entries, - custom_declarations=len( - custom.CustomFunctionRuntimeRegistry.metadata_by_name() - ) - if custom - else 0, + custom_declarations=( + len(custom.CustomFunctionRuntimeRegistry.metadata_by_name()) + if custom + else 0 + ), history_snapshots=len(state.ObjectStateRegistry._snapshots) if state else 0, imported_source_paths=tuple( modules[name].__file__ @@ -270,7 +271,10 @@ class MemoryDiagnosticReport: def __post_init__(self) -> None: if self.rounds < 2: raise ValueError("A retention slope requires at least two repeated rounds.") - if not math.isfinite(self.round_interval_seconds) or self.round_interval_seconds < 0: + if ( + not math.isfinite(self.round_interval_seconds) + or self.round_interval_seconds < 0 + ): raise ValueError("Round interval must be finite and nonnegative.") def write(self, output: Path) -> None: @@ -395,8 +399,10 @@ async def diagnose( ) report = MemoryDiagnosticReport( - rounds=rounds, request_sequence=sequence, - round_interval_seconds=round_interval_seconds, gc_at_end=gc_at_end, + rounds=rounds, + request_sequence=sequence, + round_interval_seconds=round_interval_seconds, + gc_at_end=gc_at_end, ) output.parent.mkdir(parents=True, exist_ok=True) @@ -515,14 +521,14 @@ def main() -> None: return report = asyncio.run( diagnose( - args.rounds, - args.timeout, - args.output, - surface_profile=surface_profile, - scratch_root=args.scratch_root, - sequence_path=args.sequence_json, - round_interval_seconds=args.round_interval_seconds, - gc_at_end=args.gc_at_end, + args.rounds, + args.timeout, + args.output, + surface_profile=surface_profile, + scratch_root=args.scratch_root, + sequence_path=args.sequence_json, + round_interval_seconds=args.round_interval_seconds, + gc_at_end=args.gc_at_end, ) ) print( diff --git a/scripts/mcp_thesis_demo_live.py b/scripts/mcp_thesis_demo_live.py index 80714f7ef3..7a4dd28e85 100755 --- a/scripts/mcp_thesis_demo_live.py +++ b/scripts/mcp_thesis_demo_live.py @@ -7,7 +7,10 @@ from __future__ import annotations from openhcs.authoring.session.operations import SessionOperation -from openhcs.authoring.session.operations.datasets import ShowDatasetImages, ShowLiveResults +from openhcs.authoring.session.operations.datasets import ( + ShowDatasetImages, + ShowLiveResults, +) import argparse import fcntl @@ -968,9 +971,7 @@ def managed_window_action_target( actions = first_payload(response, UiListActionsCapability.name).get("actions") if not isinstance(actions, list): raise RehearsalFailure("Managed-window action catalog is unavailable.") - summaries = ( - dataclass_from_mapping(UiActionSummary, row) for row in actions - ) + summaries = (dataclass_from_mapping(UiActionSummary, row) for row in actions) matches = [ row for row in summaries @@ -1101,8 +1102,7 @@ def exact_config_document_source( if not isinstance(documents, list): raise RehearsalFailure("Code-document catalog is unavailable.") summaries = ( - dataclass_from_mapping(UiCodeDocumentSummary, row) - for row in documents + dataclass_from_mapping(UiCodeDocumentSummary, row) for row in documents ) matches = [ document @@ -2745,8 +2745,7 @@ def visible_window_ids(ctx: RunContext, *, label: str) -> frozenset[str]: def window_ids_from_catalog(windows: list[Any]) -> frozenset[str]: return frozenset( - dataclass_from_mapping(UiWindowSummary, window).window_id - for window in windows + dataclass_from_mapping(UiWindowSummary, window).window_id for window in windows ) @@ -2766,9 +2765,7 @@ def plate_action_summary( actions = first_payload(response, UiListActionsCapability.name).get("actions") if not isinstance(actions, list): raise RehearsalFailure("PlateManager action catalog is unavailable.") - summaries = ( - dataclass_from_mapping(UiActionSummary, row) for row in actions - ) + summaries = (dataclass_from_mapping(UiActionSummary, row) for row in actions) matches = [ row for row in summaries @@ -2780,11 +2777,15 @@ def plate_action_summary( f"Expected one exact PlateManager action {action.operation_id!r}, got {len(matches)}." ) if not matches[0].enabled: - raise RehearsalFailure(f"PlateManager action {action.operation_id!r} is disabled.") + raise RehearsalFailure( + f"PlateManager action {action.operation_id!r} is disabled." + ) return matches[0] -def invoke_plate_action(ctx: RunContext, action: type[SessionOperation]) -> dict[str, Any]: +def invoke_plate_action( + ctx: RunContext, action: type[SessionOperation] +) -> dict[str, Any]: summary = plate_action_summary(ctx, action) response = command_json( ctx, @@ -2862,9 +2863,13 @@ def invoke_action_created_window( *, phase: str, ) -> str: - before = visible_window_ids(ctx, label=f"{phase}_{action.operation_id}_windows_before") + before = visible_window_ids( + ctx, label=f"{phase}_{action.operation_id}_windows_before" + ) invoke_plate_action(ctx, action) - after = visible_window_ids(ctx, label=f"{phase}_{action.operation_id}_windows_after") + after = visible_window_ids( + ctx, label=f"{phase}_{action.operation_id}_windows_after" + ) created = after - before if len(created) != 1: raise RehearsalFailure( @@ -4333,7 +4338,9 @@ def run_one( ctx, BASELINE_SOURCE_BINDING, ) - baseline_init = selected_workflow(ctx, "initialize_datasets", args.workflow_timeout) + baseline_init = selected_workflow( + ctx, "initialize_datasets", args.workflow_timeout + ) baseline_metadata = canonical_component_metadata( ctx, phase="baseline", @@ -4344,7 +4351,9 @@ def run_one( ctx, EDITED_SOURCE_BINDING, ) - edited_init = selected_workflow(ctx, "initialize_datasets", args.workflow_timeout) + edited_init = selected_workflow( + ctx, "initialize_datasets", args.workflow_timeout + ) edited_metadata = canonical_component_metadata( ctx, phase="edited", @@ -4360,7 +4369,9 @@ def run_one( ctx, REVERTED_SOURCE_BINDING, ) - reverted_init = selected_workflow(ctx, "initialize_datasets", args.workflow_timeout) + reverted_init = selected_workflow( + ctx, "initialize_datasets", args.workflow_timeout + ) reverted_metadata = canonical_component_metadata( ctx, phase="reverted", diff --git a/scripts/prototype_flatten_object_state.py b/scripts/prototype_flatten_object_state.py index 5c13c6ad92..322466904a 100644 --- a/scripts/prototype_flatten_object_state.py +++ b/scripts/prototype_flatten_object_state.py @@ -320,7 +320,8 @@ def test_reconstruction(): == original.partition_filter_config.partition_filter ) assert ( - reconstructed.partition_filter_config.enabled == original.partition_filter_config.enabled + reconstructed.partition_filter_config.enabled + == original.partition_filter_config.enabled ) assert ( reconstructed.napari_streaming_config.window_size @@ -465,7 +466,9 @@ def test_none_handling(): print( f" partition_filter_config.partition_filter: {reconstructed.partition_filter_config.partition_filter}" ) - print(f" partition_filter_config.enabled: {reconstructed.partition_filter_config.enabled}") + print( + f" partition_filter_config.enabled: {reconstructed.partition_filter_config.enabled}" + ) # The reconstructed object should have None for partition_filter (not set in filtered_fields) assert reconstructed.partition_filter_config.partition_filter is None diff --git a/scripts/smoke_installed_mcp.py b/scripts/smoke_installed_mcp.py index 74ef95ebe5..06c48c5286 100644 --- a/scripts/smoke_installed_mcp.py +++ b/scripts/smoke_installed_mcp.py @@ -302,6 +302,7 @@ async def _run_measured_execution_protocol_smoke( pipeline_source = PipelineDocumentCodec.render(document) source_file = output_dir / "pipeline.py" source_file.write_text(pipeline_source, encoding="utf-8") + async def call(name: str, arguments: dict, timeout: float = 90) -> dict: return _tool_payload( await asyncio.wait_for(session.call_tool(name, arguments), timeout=timeout) @@ -330,7 +331,9 @@ async def call(name: str, arguments: dict, timeout: float = 90) -> dict: "openhcs_run_datasets", { "runtime_observation_export_path": str( - MeasuredPipelineRunArtifact.RUNTIME_OBSERVATION.path_in(evidence_dir) + MeasuredPipelineRunArtifact.RUNTIME_OBSERVATION.path_in( + evidence_dir + ) ), "runtime_observation_export_scope": "outcomes", }, @@ -340,7 +343,9 @@ async def call(name: str, arguments: dict, timeout: float = 90) -> dict: ): started = await call(tool, {"scope_ids": [scope_id], **arguments}) if started.get("status") != "accepted": - raise AssertionError(f"Installed MCP rejected {tool}: {started} {connected}") + raise AssertionError( + f"Installed MCP rejected {tool}: {started} {connected}" + ) sequence = started["event_sequence"] deadline = time.monotonic() + 180 while True: diff --git a/scripts/test_real_global_config.py b/scripts/test_real_global_config.py index 172a17fadb..dc2fbda580 100644 --- a/scripts/test_real_global_config.py +++ b/scripts/test_real_global_config.py @@ -73,8 +73,13 @@ def test_real_global_config(): print(f"Reconstructed successfully: {type(reconstructed) == type(config)}") # Verify a few nested configs are correct - if hasattr(config, "partition_filter_config") and config.partition_filter_config is not None: - assert type(reconstructed.partition_filter_config) == type(config.partition_filter_config) + if ( + hasattr(config, "partition_filter_config") + and config.partition_filter_config is not None + ): + assert type(reconstructed.partition_filter_config) == type( + config.partition_filter_config + ) print(f"✓ partition_filter_config type matches") if ( diff --git a/tests/diagnostics/check_owned_bootstrap_live.py b/tests/diagnostics/check_owned_bootstrap_live.py index 6e8005b256..bc3304a13f 100644 --- a/tests/diagnostics/check_owned_bootstrap_live.py +++ b/tests/diagnostics/check_owned_bootstrap_live.py @@ -22,8 +22,8 @@ from openhcs.microscopes.bioformats import BioFormatsHandler SOURCE = Path(__file__).resolve().parents[2] -PYTHON = Path('/home/ts/code/projects/openhcs/.venv/bin/python') -LOCK = Path('/home/ts/wt/openhcs-issue-batch-20260929/validation.lock') +PYTHON = Path("/home/ts/code/projects/openhcs/.venv/bin/python") +LOCK = Path("/home/ts/wt/openhcs-issue-batch-20260929/validation.lock") def fixture_registration_sources(): @@ -35,85 +35,131 @@ def fixture_registration_sources(): from tests.diagnostics import volume_projection_fixture as fixture dependencies = ( - (fixture.select_volume_fixture_planes_v2, - 'ArrayPayload, numpy, Pure3DContract, artifact_outputs, SELECTED_VOLUME, ' - 'SelectedPlaneImageOutput, np'), - (fixture.inspect_volume_fixture_v2, - 'ArrayPayload, numpy, Pure3DContract, artifact_outputs, VOLUME_IMAGE, ' - 'VOLUME_LABELS, VOLUME_ROWS, np, ' - 'DataclassMeasurementColumnarRows, VolumeProjectionFixtureRow'), + ( + fixture.select_volume_fixture_planes_v2, + "ArrayPayload, numpy, Pure3DContract, artifact_outputs, SELECTED_VOLUME, " + "SelectedPlaneImageOutput, np", + ), + ( + fixture.inspect_volume_fixture_v2, + "ArrayPayload, numpy, Pure3DContract, artifact_outputs, VOLUME_IMAGE, " + "VOLUME_LABELS, VOLUME_ROWS, np, " + "DataclassMeasurementColumnarRows, VolumeProjectionFixtureRow", + ), ) return tuple( - (function.__name__, - f'from tests.diagnostics.volume_projection_fixture import ({imports})\n\n' - + inspect.getsource(function)) + ( + function.__name__, + f"from tests.diagnostics.volume_projection_fixture import ({imports})\n\n" + + inspect.getsource(function), + ) for function, imports in dependencies ) def guard(*, projected_disk_gib: float = 0.5, disk_reserve_gib: float = 2) -> dict: result = subprocess.run( - ['/home/ts/bin/agent-resource-check', '--assert-headroom'], - capture_output=True, text=True, check=False, + ["/home/ts/bin/agent-resource-check", "--assert-headroom"], + capture_output=True, + text=True, + check=False, ) resources = json.loads(result.stdout) # Warning text is not an admission policy: in particular, 20 GiB free # is a host notification threshold, not the footprint of this bounded run. - if (resources['level'] == 'critical' or resources['ram_available_gib'] < 8 - or resources['free_gib']['/home'] < projected_disk_gib + disk_reserve_gib): - raise RuntimeError(f'Resource gate closed: {resources}') + if ( + resources["level"] == "critical" + or resources["ram_available_gib"] < 8 + or resources["free_gib"]["/home"] < projected_disk_gib + disk_reserve_gib + ): + raise RuntimeError(f"Resource gate closed: {resources}") return resources def run(args) -> None: assert Path(sys.executable).resolve() == PYTHON.resolve() sha = subprocess.check_output( - ['git', 'rev-parse', 'HEAD'], cwd=SOURCE, text=True, + ["git", "rev-parse", "HEAD"], + cwd=SOURCE, + text=True, ).strip() assert sha == args.expected_sha root = args.receipts.resolve() assert root.is_relative_to(SOURCE) and not root.exists() - with LOCK.open('a+') as lock: + with LOCK.open("a+") as lock: fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB) resources = guard() root.mkdir(parents=True) - owned = root / 'owned' + owned = root / "owned" owned.mkdir() - for key in ('OMP_NUM_THREADS', 'OPENBLAS_NUM_THREADS', 'MKL_NUM_THREADS', - 'NUMEXPR_NUM_THREADS', 'NUMBA_NUM_THREADS', 'BLIS_NUM_THREADS'): - os.environ[key] = '1' + for key in ( + "OMP_NUM_THREADS", + "OPENBLAS_NUM_THREADS", + "MKL_NUM_THREADS", + "NUMEXPR_NUM_THREADS", + "NUMBA_NUM_THREADS", + "BLIS_NUM_THREADS", + ): + os.environ[key] = "1" os.environ.update( - OPENHCS_CPU_ONLY='true', OPENHCS_HEADLESS='true', - OPENHCS_USE_THREADING='true', OPENHCS_SUBPROCESS_NO_GPU='1', - POLYSTORE_SUBPROCESS_NO_GPU='1', CUDA_VISIBLE_DEVICES='', - QT_QPA_PLATFORM='offscreen', PYTHONDONTWRITEBYTECODE='1', - POLYSTORE_IMAGEJ_ALLOW_DOWNLOAD='false', - POLYSTORE_IMAGEJ_CACHE_ROOT='/home/ts/.cache/polystore/imagej', - XDG_DATA_HOME=str(owned / 'data'), XDG_CACHE_HOME=str(owned / 'cache'), - XDG_CONFIG_HOME=str(owned / 'config'), XDG_STATE_HOME=str(owned / 'state'), - XDG_RUNTIME_DIR=str(owned / 'runtime'), - NUMBA_CACHE_DIR=str(owned / 'numba'), MPLCONFIGDIR=str(owned / 'matplotlib'), - OPENHCS_UI_CONFIG_CACHE_FILE=str(owned / 'ui_config.config'), + OPENHCS_CPU_ONLY="true", + OPENHCS_HEADLESS="true", + OPENHCS_USE_THREADING="true", + OPENHCS_SUBPROCESS_NO_GPU="1", + POLYSTORE_SUBPROCESS_NO_GPU="1", + CUDA_VISIBLE_DEVICES="", + QT_QPA_PLATFORM="offscreen", + PYTHONDONTWRITEBYTECODE="1", + POLYSTORE_IMAGEJ_ALLOW_DOWNLOAD="false", + POLYSTORE_IMAGEJ_CACHE_ROOT="/home/ts/.cache/polystore/imagej", + XDG_DATA_HOME=str(owned / "data"), + XDG_CACHE_HOME=str(owned / "cache"), + XDG_CONFIG_HOME=str(owned / "config"), + XDG_STATE_HOME=str(owned / "state"), + XDG_RUNTIME_DIR=str(owned / "runtime"), + NUMBA_CACHE_DIR=str(owned / "numba"), + MPLCONFIGDIR=str(owned / "matplotlib"), + OPENHCS_UI_CONFIG_CACHE_FILE=str(owned / "ui_config.config"), + ) + receipt = dict( + accepted=False, + source_sha=sha, + driver_pid=os.getpid(), + source_live_not_installed=True, + resources=resources, + calls=[], + uncertain_dispatch=None, + runtime_terminal=False, ) - receipt = dict(accepted=False, source_sha=sha, driver_pid=os.getpid(), - source_live_not_installed=True, resources=resources, calls=[], - uncertain_dispatch=None, runtime_terminal=False) def save(): - (root / 'journey.json').write_text(json.dumps(receipt, indent=2) + '\n') + (root / "journey.json").write_text(json.dumps(receipt, indent=2) + "\n") save() # Existing declaration-owned setup builds both recorded native modules. # No package installation, download or shared source mutation. tick = time.monotonic() - with (root / 'build.txt').open('w') as log: + with (root / "build.txt").open("w") as log: build = subprocess.run( - [str(PYTHON), '-B', 'setup.py', 'build_ext', '--inplace', '-j', '1', - '--build-temp', str(root / 'build-temp'), - '--build-lib', str(root / 'build-lib')], - cwd=SOURCE, stdout=log, stderr=subprocess.STDOUT, timeout=60, + [ + str(PYTHON), + "-B", + "setup.py", + "build_ext", + "--inplace", + "-j", + "1", + "--build-temp", + str(root / "build-temp"), + "--build-lib", + str(root / "build-lib"), + ], + cwd=SOURCE, + stdout=log, + stderr=subprocess.STDOUT, + timeout=60, ) - receipt['build'] = dict(exit=build.returncode, seconds=time.monotonic()-tick) + receipt["build"] = dict(exit=build.returncode, seconds=time.monotonic() - tick) save() assert build.returncode == 0 @@ -129,11 +175,15 @@ def save(): from zmqruntime.transport import TransportEndpoint from openhcs.agent.capabilities import agent_capabilities from openhcs.agent.dto.execution import ( - RuntimeBootstrapState, RuntimeBootstrapCloseResult, - ExecutionJobRef, ExecutionJobStatus, OrchestratorSessionRef, + RuntimeBootstrapState, + RuntimeBootstrapCloseResult, + ExecutionJobRef, + ExecutionJobStatus, + OrchestratorSessionRef, ) from openhcs.agent.dto.functions import ( - FunctionCatalogPreparationState, CustomFunctionRegistrationResult, + FunctionCatalogPreparationState, + CustomFunctionRegistrationResult, FunctionCatalogPage, ) from openhcs.mcp.dev_client import McpDevClient @@ -143,81 +193,130 @@ def save(): from openhcs.runtime.zmq_execution_client import ZMQExecutionClient from python_introspect import to_jsonable - modules = (openhcs, polystore, zmqruntime, metaclass_registry, - importlib.import_module('openhcs.core._tabular_native'), - importlib.import_module('openhcs.processing.backends.cellprofiler._granularity_native')) - receipt['imports'] = {module.__name__: module.__file__ for module in modules} - assert all(Path(module.__file__).resolve().is_relative_to(SOURCE) for module in modules) - receipt['submodules'] = subprocess.check_output( - ['git', 'submodule', 'status'], cwd=SOURCE, text=True, + modules = ( + openhcs, + polystore, + zmqruntime, + metaclass_registry, + importlib.import_module("openhcs.core._tabular_native"), + importlib.import_module( + "openhcs.processing.backends.cellprofiler._granularity_native" + ), + ) + receipt["imports"] = {module.__name__: module.__file__ for module in modules} + assert all( + Path(module.__file__).resolve().is_relative_to(SOURCE) for module in modules + ) + receipt["submodules"] = subprocess.check_output( + ["git", "submodule", "status"], + cwd=SOURCE, + text=True, ).splitlines() - assert not any(row.startswith(('-', '+', 'U')) for row in receipt['submodules']) - config = replace(OPENHCS_ZMQ_CONFIG, transport_mode=TransportMode.TCP, - client_host='127.0.0.1', server_host='127.0.0.1', - default_port=args.port) - endpoint = TransportEndpoint('127.0.0.1', args.port, TransportMode.TCP) + assert not any(row.startswith(("-", "+", "U")) for row in receipt["submodules"]) + config = replace( + OPENHCS_ZMQ_CONFIG, + transport_mode=TransportMode.TCP, + client_host="127.0.0.1", + server_host="127.0.0.1", + default_port=args.port, + ) + endpoint = TransportEndpoint("127.0.0.1", args.port, TransportMode.TCP) assert args.port not in (7777, 7790, 5991) assert not endpoint.occupied_ports(config) - launch_plan = ZMQExecutionClient(port=args.port, host='127.0.0.1', - transport_mode=TransportMode.TCP, - config=config).runtime_launch_plan() + launch_plan = ZMQExecutionClient( + port=args.port, + host="127.0.0.1", + transport_mode=TransportMode.TCP, + config=config, + ).runtime_launch_plan() lock_paths = launch_plan.transport_write_paths - assert all(not path.exists() for path in lock_paths), 'Preserve existing reservations' - fixture_root = Path('/home/ts/wt/openhcs-issue-batch-20260929/issue257-synthetic-ome-inputs-20260930') - fixture_source = SOURCE / 'tests/diagnostics/volume_projection_fixture.py' - os.environ['OPENHCS_AGENT_READ_ROOTS'] = os.pathsep.join((str(SOURCE), str(fixture_root), *(str(p) for p in lock_paths))) - os.environ['OPENHCS_AGENT_WRITE_ROOTS'] = os.pathsep.join((str(owned), *(str(p) for p in lock_paths))) + assert all( + not path.exists() for path in lock_paths + ), "Preserve existing reservations" + fixture_root = Path( + "/home/ts/wt/openhcs-issue-batch-20260929/issue257-synthetic-ome-inputs-20260930" + ) + fixture_source = SOURCE / "tests/diagnostics/volume_projection_fixture.py" + os.environ["OPENHCS_AGENT_READ_ROOTS"] = os.pathsep.join( + (str(SOURCE), str(fixture_root), *(str(p) for p in lock_paths)) + ) + os.environ["OPENHCS_AGENT_WRITE_ROOTS"] = os.pathsep.join( + (str(owned), *(str(p) for p in lock_paths)) + ) assert save_ui_config_sync(UIConfig(zmq=config)) - receipt['declared_endpoint_pair'] = list(endpoint.port_pair(config).ports) - receipt['transport_locks'] = [str(p) for p in lock_paths] + receipt["declared_endpoint_pair"] = list(endpoint.port_pair(config).ports) + receipt["transport_locks"] = [str(p) for p in lock_paths] from polystore.zmq_config import POLYSTORE_ZMQ_CONFIG - receipt['ack_boundary'] = dict( + + receipt["ack_boundary"] = dict( native_config_port=config.shared_ack_port, polystore_config_port=POLYSTORE_ZMQ_CONFIG.shared_ack_port, - canonical_owner='zmqruntime.ack_listener.GlobalAckListener', - streaming_caller='polystore.streaming._streaming_backend.StreamingBackend', - bind_host='*', independent_of_execution_pair=True, - sockets_before=subprocess.check_output(['ss', '-ltnp'], text=True), - foreign_ack_cleanup='never requested', + canonical_owner="zmqruntime.ack_listener.GlobalAckListener", + streaming_caller="polystore.streaming._streaming_backend.StreamingBackend", + bind_host="*", + independent_of_execution_pair=True, + sockets_before=subprocess.check_output(["ss", "-ltnp"], text=True), + foreign_ack_cleanup="never requested", ) - original_image = fixture_root / 'A01/image.ome.tif' - plate = owned / 'plate' + original_image = fixture_root / "A01/image.ome.tif" + plate = owned / "plate" plate.mkdir() image_path = plate / original_image.name import shutil + shutil.copyfile(original_image, image_path) import tifffile + input_pixels = tifffile.imread(image_path) assert input_pixels.shape == (3, 8, 9) input_hash = hashlib.sha256(image_path.read_bytes()).hexdigest() - assert input_hash == '5fc4e6baa8015be4b5ca09356575a2083ed70834f1ff447353e6c116fec248bb' + assert ( + input_hash + == "5fc4e6baa8015be4b5ca09356575a2083ed70834f1ff447353e6c116fec248bb" + ) assert image_path.read_bytes() == original_image.read_bytes() fixture_hash = hashlib.sha256(fixture_source.read_bytes()).hexdigest() - assert fixture_hash == '9b737ed5e669c6f2325cc4af9ebb0b39b706121478149315474f464e3cc30c80' + assert ( + fixture_hash + == "9b737ed5e669c6f2325cc4af9ebb0b39b706121478149315474f464e3cc30c80" + ) registration_sources = fixture_registration_sources() from openhcs.processing.custom_functions.manager import CustomFunctionManager + manager = CustomFunctionManager(create_storage=False) for name, code in registration_sources: prepared_source = manager._prepare_source(code) assert prepared_source.original_name == name - (root/f'{name}.py').write_text(code) - receipt['fixture'] = dict(source=str(fixture_source), input=str(image_path), - original_input=str(original_image), - input_sha256=input_hash, byte_identical_copy=True, - source_sha256=fixture_hash, - plane_selections=[[], [2, 0, 1], [0, 1], [1]], - expected_source_planes=[[0, 1, 2], [2, 0, 1], [2, 0], [0]], - registration_source_hashes={name: hashlib.sha256(code.encode()).hexdigest() - for name, code in registration_sources}, - omitted_controls=[]) + (root / f"{name}.py").write_text(code) + receipt["fixture"] = dict( + source=str(fixture_source), + input=str(image_path), + original_input=str(original_image), + input_sha256=input_hash, + byte_identical_copy=True, + source_sha256=fixture_hash, + plane_selections=[[], [2, 0, 1], [0, 1], [1]], + expected_source_planes=[[0, 1, 2], [2, 0, 1], [2, 0], [0]], + registration_source_hashes={ + name: hashlib.sha256(code.encode()).hexdigest() + for name, code in registration_sources + }, + omitted_controls=[], + ) class SourceSpec(McpDevServerSpec): - mcp_environment_keys = (*McpDevServerSpec.mcp_environment_keys, - 'PYTHONPATH', 'MPLCONFIGDIR') + mcp_environment_keys = ( + *McpDevServerSpec.mcp_environment_keys, + "PYTHONPATH", + "MPLCONFIGDIR", + ) - client = McpDevClient(str(PYTHON), use_resident_server=False, - server_stderr=(root / 'mcp-stderr.txt').open('w'), - initialize_timeout_seconds=10) + client = McpDevClient( + str(PYTHON), + use_resident_server=False, + server_stderr=(root / "mcp-stderr.txt").open("w"), + initialize_timeout_seconds=10, + ) client.server_spec = SourceSpec(str(PYTHON)) original_handle = None began = time.monotonic() @@ -225,41 +324,65 @@ class SourceSpec(McpDevServerSpec): def call(name, arguments, result_type=None, *, lifecycle=False): if not lifecycle: current = guard() - size = sum(p.stat().st_size for p in root.rglob('*') if p.is_file()) - assert size < 80 * 1024 * 1024, '80MiB artifact bound' - assert time.monotonic()-began < 240, 'Original finite journey bound' - receipt['last_resources'] = current - row = dict(tool=name, arguments=arguments, started_unix=time.time(), status='dispatched') - receipt['calls'].append(row) - receipt['uncertain_dispatch'] = len(receipt['calls']) + size = sum(p.stat().st_size for p in root.rglob("*") if p.is_file()) + assert size < 80 * 1024 * 1024, "80MiB artifact bound" + assert time.monotonic() - began < 240, "Original finite journey bound" + receipt["last_resources"] = current + row = dict( + tool=name, + arguments=arguments, + started_unix=time.time(), + status="dispatched", + ) + receipt["calls"].append(row) + receipt["uncertain_dispatch"] = len(receipt["calls"]) save() tick = time.monotonic() result = client.execute( - ['--allow-error-payloads', 'call', name, '--arguments', - json.dumps(arguments), '--json'], timeout_seconds=10, + [ + "--allow-error-payloads", + "call", + name, + "--arguments", + json.dumps(arguments), + "--json", + ], + timeout_seconds=10, + ) + row.update( + seconds=time.monotonic() - tick, + response=result.payload, + status="returned", ) - row.update(seconds=time.monotonic()-tick, response=result.payload, status='returned') save() - assert not result.payload['errors'], result.payload - response = result.payload['results'][0] - assert not response['mcp_error'], response - payload = response['payloads'][0] - if payload.get('errors') and payload['errors'][0]['code'] == 'agent_path_policy_rejected': + assert not result.payload["errors"], result.payload + response = result.payload["results"][0] + assert not response["mcp_error"], response + payload = response["payloads"][0] + if ( + payload.get("errors") + and payload["errors"][0]["code"] == "agent_path_policy_rejected" + ): # This specific original path-policy owner rejects before dispatch. # Other errors, especially post-dispatch uncertainty, retain handles. - receipt['uncertain_dispatch'] = None + receipt["uncertain_dispatch"] = None save() - assert not payload.get('errors'), payload - value = dataclass_from_mapping(result_type, payload) if result_type else payload - receipt['uncertain_dispatch'] = None + assert not payload.get("errors"), payload + value = ( + dataclass_from_mapping(result_type, payload) if result_type else payload + ) + receipt["uncertain_dispatch"] = None save() print(f'{name}: {row["seconds"]:.3f}s', flush=True) return value def finish(ref): while True: - state = call(agent_capabilities.get_execution_status.name, - dict(job_id=ref.job_id, timeout_ms=1000), ExecutionJobStatus) + state = call( + agent_capabilities.get_execution_status.name, + dict(job_id=ref.job_id, timeout_ms=1000), + ExecutionJobStatus, + ) if state.is_terminal: assert state.status == ExecutionStatus.COMPLETE.value, state return state @@ -267,147 +390,258 @@ def finish(ref): try: client.start() - health = call('openhcs_health_check', {}) - assert Path(health['server_source_path']).resolve() == SOURCE/'openhcs/mcp/server.py' - assert not health['restart_required'] - receipt['mcp_pid'] = health['server_process_id'] - call('openhcs_get_authoring_context', dict(kind='first_use')) - call('openhcs_search_capabilities', dict(query='owned runtime', limit=10)) - startup = call(agent_capabilities.start_owned_runtime.name, - dict(port=args.port, host='127.0.0.1', transport_mode='tcp', - timeout_ms=500), RuntimeBootstrapState) + health = call("openhcs_health_check", {}) + assert ( + Path(health["server_source_path"]).resolve() + == SOURCE / "openhcs/mcp/server.py" + ) + assert not health["restart_required"] + receipt["mcp_pid"] = health["server_process_id"] + call("openhcs_get_authoring_context", dict(kind="first_use")) + call("openhcs_search_capabilities", dict(query="owned runtime", limit=10)) + startup = call( + agent_capabilities.start_owned_runtime.name, + dict( + port=args.port, + host="127.0.0.1", + transport_mode="tcp", + timeout_ms=500, + ), + RuntimeBootstrapState, + ) original_handle = startup.handle - receipt['bootstrap_handle'] = to_jsonable(original_handle) + receipt["bootstrap_handle"] = to_jsonable(original_handle) save() while not startup.ready: assert startup.process_alive is not False, startup time.sleep(0.5) - startup = call(agent_capabilities.observe_owned_runtime.name, - dict(handle=to_jsonable(original_handle)), RuntimeBootstrapState) + startup = call( + agent_capabilities.observe_owned_runtime.name, + dict(handle=to_jsonable(original_handle)), + RuntimeBootstrapState, + ) assert startup.handle == original_handle - receipt['bootstrap_ready_seconds'] = time.monotonic()-began + receipt["bootstrap_ready_seconds"] = time.monotonic() - began prepare_started = time.monotonic() - prepared = call(agent_capabilities.start_function_catalog_preparation.name, - dict(port=args.port, host='127.0.0.1', transport_mode='tcp'), - FunctionCatalogPreparationState) + prepared = call( + agent_capabilities.start_function_catalog_preparation.name, + dict(port=args.port, host="127.0.0.1", transport_mode="tcp"), + FunctionCatalogPreparationState, + ) preparation_handle = prepared.handle - receipt['preparation_handle'] = to_jsonable(preparation_handle) + receipt["preparation_handle"] = to_jsonable(preparation_handle) while not prepared.outcome.ready: assert not prepared.outcome.terminal, prepared time.sleep(1) - prepared = call(agent_capabilities.get_function_catalog_preparation_status.name, - to_jsonable(preparation_handle), FunctionCatalogPreparationState) + prepared = call( + agent_capabilities.get_function_catalog_preparation_status.name, + to_jsonable(preparation_handle), + FunctionCatalogPreparationState, + ) assert prepared.handle == preparation_handle - receipt['preparation_seconds'] = time.monotonic()-prepare_started - receipt['registrations'] = [] + receipt["preparation_seconds"] = time.monotonic() - prepare_started + receipt["registrations"] = [] for name, code in registration_sources: - registered = call('openhcs_register_custom_function', dict( - source_code=code, function_name=name, - storage_dir=str(original_handle.launch_plan.storage_dir), persist=True, - port=args.port, host='127.0.0.1', transport_mode='tcp', - ), CustomFunctionRegistrationResult) + registered = call( + "openhcs_register_custom_function", + dict( + source_code=code, + function_name=name, + storage_dir=str(original_handle.launch_plan.storage_dir), + persist=True, + port=args.port, + host="127.0.0.1", + transport_mode="tcp", + ), + CustomFunctionRegistrationResult, + ) assert registered.registered_count == 1 assert registered.server_identity == original_handle.process_identity [persisted] = registered.source_file_paths assert Path(persisted).read_text() == code - receipt['registrations'].append(to_jsonable(registered)) - page = call('openhcs_search_functions', dict(query=name, limit=5), FunctionCatalogPage) - assert registered.functions[0].function_id in {entry.function_id for entry in page.items} + receipt["registrations"].append(to_jsonable(registered)) + page = call( + "openhcs_search_functions", + dict(query=name, limit=5), + FunctionCatalogPage, + ) + assert registered.functions[0].function_id in { + entry.function_id for entry in page.items + } save() - from openhcs.core.config import (PipelineConfig, LazyPathPlanningConfig, - LazyVFSConfig, MaterializationBackend, - LazyStepMaterializationConfig, LazyProcessingConfig) + from openhcs.core.config import ( + PipelineConfig, + LazyPathPlanningConfig, + LazyVFSConfig, + MaterializationBackend, + LazyStepMaterializationConfig, + LazyProcessingConfig, + ) from openhcs.core.pipeline_document import PipelineDocumentCodec from openhcs.core.steps.function_step import FunctionStep from openhcs.processing.custom_functions import ( - select_volume_fixture_planes_v2, inspect_volume_fixture_v2, + select_volume_fixture_planes_v2, + inspect_volume_fixture_v2, ) + steps = [] for case, indices in enumerate(((), (2, 0, 1), (0, 1), (1,))): - for phase, function in enumerate((select_volume_fixture_planes_v2, - inspect_volume_fixture_v2, inspect_volume_fixture_v2)): - steps.append(FunctionStep( - func=((function, dict(plane_indices=indices)) if phase == 0 else function), - name=f'ProjectionCase{case}Step{phase}', - processing_config=LazyProcessingConfig(variable_components=[Microscopy.ZIndex]), - step_materialization_config=LazyStepMaterializationConfig(enabled=True))) + for phase, function in enumerate( + ( + select_volume_fixture_planes_v2, + inspect_volume_fixture_v2, + inspect_volume_fixture_v2, + ) + ): + steps.append( + FunctionStep( + func=( + (function, dict(plane_indices=indices)) + if phase == 0 + else function + ), + name=f"ProjectionCase{case}Step{phase}", + processing_config=LazyProcessingConfig( + variable_components=[Microscopy.ZIndex] + ), + step_materialization_config=LazyStepMaterializationConfig( + enabled=True + ), + ) + ) document = PipelineDocumentCodec.from_values( - pipeline_config=PipelineConfig(num_workers=1, use_threading=True, + pipeline_config=PipelineConfig( + num_workers=1, + use_threading=True, dataset_source=BioFormatsHandler, - path_planning_config=LazyPathPlanningConfig(global_output_folder=owned/'outputs'), - vfs_config=LazyVFSConfig(materialization_backend=MaterializationBackend.DISK)), + path_planning_config=LazyPathPlanningConfig( + global_output_folder=owned / "outputs" + ), + vfs_config=LazyVFSConfig( + materialization_backend=MaterializationBackend.DISK + ), + ), pipeline_steps=steps, ) source = PipelineDocumentCodec.render(document) - (root/'pipeline.py').write_text(source) + (root / "pipeline.py").write_text(source) from openhcs.agent.dto.execution import ArtifactPlanInspection - inspected = call('openhcs_inspect_pipeline_source_artifact_plan', - dict(plate_path=str(image_path.parent), pipeline_source=source), ArtifactPlanInspection) - receipt['artifact_plan'] = to_jsonable(inspected) + + inspected = call( + "openhcs_inspect_pipeline_source_artifact_plan", + dict(plate_path=str(image_path.parent), pipeline_source=source), + ArtifactPlanInspection, + ) + receipt["artifact_plan"] = to_jsonable(inspected) assert inspected.step_count == 12 and inspected.axis_count == 1 - session = call('openhcs_create_orchestrator_session_from_pipeline_source', - dict(plate_path=str(image_path.parent), pipeline_source=source, - port=args.port, host='127.0.0.1', transport_mode='tcp'), - OrchestratorSessionRef) - for tool, stage in (('openhcs_submit_compile', 'compile'), - ('openhcs_submit_pipeline_execution', 'execution')): + session = call( + "openhcs_create_orchestrator_session_from_pipeline_source", + dict( + plate_path=str(image_path.parent), + pipeline_source=source, + port=args.port, + host="127.0.0.1", + transport_mode="tcp", + ), + OrchestratorSessionRef, + ) + for tool, stage in ( + ("openhcs_submit_compile", "compile"), + ("openhcs_submit_pipeline_execution", "execution"), + ): tick = time.monotonic() arguments = dict(session_id=session.session_id, wait=False) - if stage == 'execution': - arguments['runtime_observation_export_path'] = str(owned/'observation.pkl.gz') + if stage == "execution": + arguments["runtime_observation_export_path"] = str( + owned / "observation.pkl.gz" + ) job = call(tool, arguments, ExecutionJobRef) receipt[stage] = to_jsonable(finish(job)) - receipt[stage+'_seconds'] = time.monotonic()-tick + receipt[stage + "_seconds"] = time.monotonic() - tick save() - from tests.diagnostics.owned_bootstrap_readback import verify_volume_publication - receipt['volume_publication'] = verify_volume_publication( - owned, image_path, input_pixels, inspected, + from tests.diagnostics.owned_bootstrap_readback import ( + verify_volume_publication, + ) + + receipt["volume_publication"] = verify_volume_publication( + owned, + image_path, + input_pixels, + inspected, ) assert hashlib.sha256(image_path.read_bytes()).hexdigest() == input_hash assert hashlib.sha256(original_image.read_bytes()).hexdigest() == input_hash - receipt.update(accepted=True, unchanged_input_sha256=input_hash, - registration_mcp_calls=len(registration_sources), - registrations_per_declaration=1, no_mutation_replay=True) + receipt.update( + accepted=True, + unchanged_input_sha256=input_hash, + registration_mcp_calls=len(registration_sources), + registrations_per_declaration=1, + no_mutation_replay=True, + ) save() except BaseException as error: - receipt['failure'] = dict(type=type(error).__name__, message=str(error), - traceback=traceback.format_exc()) + receipt["failure"] = dict( + type=type(error).__name__, + message=str(error), + traceback=traceback.format_exc(), + ) save() - print('STOPPED: same handle retained; read-only CLI commands or close-owned only.', flush=True) + print( + "STOPPED: same handle retained; read-only CLI commands or close-owned only.", + flush=True, + ) import shlex + for line in sys.stdin: - if line.strip() == 'close-owned': - receipt['operator_disposition'] = 'Close exact owned handle; no mutation replay.' + if line.strip() == "close-owned": + receipt["operator_disposition"] = ( + "Close exact owned handle; no mutation replay." + ) break observed = client.execute(shlex.split(line), timeout_seconds=10) - receipt.setdefault('same_handle_observations', []).append(observed.payload) + receipt.setdefault("same_handle_observations", []).append( + observed.payload + ) save() print(observed.rendered_output, flush=True) else: - raise RuntimeError('No disposition; retain original handles') from error + raise RuntimeError("No disposition; retain original handles") from error # Cleanup is a distinct authorized lifecycle request, never a source replay. if original_handle is not None: - closed = call(agent_capabilities.close_owned_runtime.name, - dict(handle=to_jsonable(original_handle), mode='force', timeout_ms=5000), - RuntimeBootstrapCloseResult, lifecycle=True) + closed = call( + agent_capabilities.close_owned_runtime.name, + dict( + handle=to_jsonable(original_handle), mode="force", timeout_ms=5000 + ), + RuntimeBootstrapCloseResult, + lifecycle=True, + ) assert closed.handle == original_handle assert closed.outcome.succeeded and closed.outcome.endpoint_terminated assert original_handle.process_identity.is_alive() is False assert not endpoint.occupied_ports(config) - receipt['close'] = to_jsonable(closed) + receipt["close"] = to_jsonable(closed) client.close() - receipt['client_closed'] = True - receipt['runtime_terminal'] = True - receipt['sockets_after'] = subprocess.check_output(['ss', '-ltnp'], text=True) - receipt['total_seconds'] = time.monotonic()-began + receipt["client_closed"] = True + receipt["runtime_terminal"] = True + receipt["sockets_after"] = subprocess.check_output(["ss", "-ltnp"], text=True) + receipt["total_seconds"] = time.monotonic() - began save() - print(json.dumps(dict(accepted=receipt['accepted'], runtime_terminal=True, - receipt=str(root/'journey.json'))), flush=True) + print( + json.dumps( + dict( + accepted=receipt["accepted"], + runtime_terminal=True, + receipt=str(root / "journey.json"), + ) + ), + flush=True, + ) -if __name__ == '__main__': +if __name__ == "__main__": parser = argparse.ArgumentParser() - parser.add_argument('--receipts', type=Path, required=True) - parser.add_argument('--expected-sha', required=True) - parser.add_argument('--port', type=int, default=5963) + parser.add_argument("--receipts", type=Path, required=True) + parser.add_argument("--expected-sha", required=True) + parser.add_argument("--port", type=int, default=5963) run(parser.parse_args()) diff --git a/tests/diagnostics/volume_projection_fixture.py b/tests/diagnostics/volume_projection_fixture.py index 9a318a25a7..1be8dbf54d 100644 --- a/tests/diagnostics/volume_projection_fixture.py +++ b/tests/diagnostics/volume_projection_fixture.py @@ -18,15 +18,24 @@ ObjectLabelsArtifactType, ObjectMeasurementSubjectRelation, ) -from openhcs.core.measurement_row_materialization import DataclassMeasurementColumnarRows +from openhcs.core.measurement_row_materialization import ( + DataclassMeasurementColumnarRows, +) from openhcs.core.memory import numpy from openhcs.core.pipeline.function_contracts import artifact_outputs from openhcs.core.projected_image_output import SelectedPlaneImageOutput -from openhcs.core.runtime_measurements import RuntimeMeasurementFeature, RuntimeMeasurementFeatureOwner +from openhcs.core.runtime_measurements import ( + RuntimeMeasurementFeature, + RuntimeMeasurementFeatureOwner, +) from openhcs.core.processing_contracts import ( Pure3DContract, ) -from openhcs.processing.materialization import CsvOptions, MaterializationSpec, ROIOptions +from openhcs.processing.materialization import ( + CsvOptions, + MaterializationSpec, + ROIOptions, +) class VolumeProjectionFixtureFeature(RuntimeMeasurementFeature): @@ -36,7 +45,10 @@ class VolumeProjectionFixtureFeature(RuntimeMeasurementFeature): class VolumeProjectionFixtureFeatureOwner(RuntimeMeasurementFeatureOwner): @classmethod def owns_measurement_feature_name(cls, feature_name: str) -> bool: - return any(feature.feature_name == feature_name for feature in VolumeProjectionFixtureFeature) + return any( + feature.feature_name == feature_name + for feature in VolumeProjectionFixtureFeature + ) @classmethod def owns_primary_measurement_feature_name(cls, feature_name: str) -> bool: @@ -51,17 +63,26 @@ class VolumeProjectionFixtureRow: SELECTED_VOLUME = MainFlowPlaneProjectionOutputSpec.output( - "selected_volume_fixture_v2", ImageArtifactType, + "selected_volume_fixture_v2", + ImageArtifactType, +) +VOLUME_IMAGE = MainFlowStackOutputSpec.output( + "volume_fixture_image_v2", ImageArtifactType ) -VOLUME_IMAGE = MainFlowStackOutputSpec.output("volume_fixture_image_v2", ImageArtifactType) VOLUME_LABELS = MainFlowStackOutputSpec.output( - "volume_fixture_labels_v2", ObjectLabelsArtifactType, + "volume_fixture_labels_v2", + ObjectLabelsArtifactType, materialization=MaterializationSpec(ROIOptions(min_area=0)), ) VOLUME_ROWS = MainFlowStackOutputSpec.output( - "volume_fixture_rows_v2", MeasurementsArtifactType, + "volume_fixture_rows_v2", + MeasurementsArtifactType, measurement_feature_owner=VolumeProjectionFixtureFeatureOwner, - relations=(ObjectMeasurementSubjectRelation(source=VOLUME_LABELS.ref(), id_field="object_label"),), + relations=( + ObjectMeasurementSubjectRelation( + source=VOLUME_LABELS.ref(), id_field="object_label" + ), + ), materialization=MaterializationSpec(CsvOptions()), ) @@ -94,7 +115,9 @@ def inspect_volume_fixture_v2( metadata = image.metadata labels = pixels.astype(np.int32, copy=True) rows = tuple( - VolumeProjectionFixtureRow(index, int(label), int(np.count_nonzero(plane == label))) + VolumeProjectionFixtureRow( + index, int(label), int(np.count_nonzero(plane == label)) + ) for index, plane in enumerate(labels) for label in np.unique(plane) if label != 0 diff --git a/tests/integration/test_agent_observation_export_zmq.py b/tests/integration/test_agent_observation_export_zmq.py index 9fc672778f..0d2280ce6b 100644 --- a/tests/integration/test_agent_observation_export_zmq.py +++ b/tests/integration/test_agent_observation_export_zmq.py @@ -113,14 +113,21 @@ def _start_run(built, plate: Path, pipeline, export_path: Path) -> tuple[str, in _call( built, "openhcs_set_dataset_pipeline", - {"scope_id": scope_id, "pipeline_source": PipelineDocumentCodec.render(pipeline)}, - ) - _wait_for_row(built, _call( - built, "openhcs_initialize_datasets", {"scope_ids": [scope_id]} - ), lambda row: row["initialized"] and not row["init_pending"]) - _wait_for_row(built, _call( - built, "openhcs_compile_datasets", {"scope_ids": [scope_id]} - ), lambda row: row["compiled"] and not row["compile_pending"]) + { + "scope_id": scope_id, + "pipeline_source": PipelineDocumentCodec.render(pipeline), + }, + ) + _wait_for_row( + built, + _call(built, "openhcs_initialize_datasets", {"scope_ids": [scope_id]}), + lambda row: row["initialized"] and not row["init_pending"], + ) + _wait_for_row( + built, + _call(built, "openhcs_compile_datasets", {"scope_ids": [scope_id]}), + lambda row: row["compiled"] and not row["compile_pending"], + ) started = _call( built, "openhcs_run_datasets", diff --git a/tests/integration/test_cellprofiler_generated_pipeline.py b/tests/integration/test_cellprofiler_generated_pipeline.py index c24a36d5b4..f6f05b5cee 100644 --- a/tests/integration/test_cellprofiler_generated_pipeline.py +++ b/tests/integration/test_cellprofiler_generated_pipeline.py @@ -184,7 +184,9 @@ def _execute_imported_cppipe_via_zmq( if partition_filter is None else LazyPartitionFilterConfig( partition_filter=( - partition_filter if isinstance(partition_filter, int) else list(partition_filter) + partition_filter + if isinstance(partition_filter, int) + else list(partition_filter) ) ) ), diff --git a/tests/integration/test_cellprofiler_official30_zmq.py b/tests/integration/test_cellprofiler_official30_zmq.py index 7f07bf67a5..2b9d024417 100644 --- a/tests/integration/test_cellprofiler_official30_zmq.py +++ b/tests/integration/test_cellprofiler_official30_zmq.py @@ -570,8 +570,7 @@ def test_official30_fiji_variants_project_registered_viewer_configs() -> None: for viewers in _OFFICIAL30_FIJI_VIEWER_VARIANTS: ports_by_viewer = { - viewer: 20_000 + index * 100 - for index, viewer in enumerate(viewers) + viewer: 20_000 + index * 100 for index, viewer in enumerate(viewers) } configs = _registered_streaming_config_kwargs( viewers, diff --git a/tests/integration/test_chained_callable_lineage_journey.py b/tests/integration/test_chained_callable_lineage_journey.py index b57f976444..28bc1b2b9d 100644 --- a/tests/integration/test_chained_callable_lineage_journey.py +++ b/tests/integration/test_chained_callable_lineage_journey.py @@ -43,7 +43,9 @@ from polystore.roi import load_rois_from_zip # Reuse the public reference's authoritative code block, not a copied callable. -from tests.unit.agent.test_callable_artifact_reference import reference_namespace # noqa: F401 +from tests.unit.agent.test_callable_artifact_reference import ( + reference_namespace, +) # noqa: F401 from openhcs.domains.microscopy.axes import Microscopy from openhcs.core.dataset_sources.source_bindings_source import SourceBindingsSource @@ -61,7 +63,9 @@ def registered_reference_callable(reference_namespace): def test_chained_public_callable_uses_declared_main_flow_not_storage_argument( - tmp_path, reference_namespace, registered_reference_callable, + tmp_path, + reference_namespace, + registered_reference_callable, ): assert reference_namespace["inspect_label_fixture"] is registered_reference_callable plate = tmp_path / "plate" @@ -71,13 +75,15 @@ def test_chained_public_callable_uses_declared_main_flow_not_storage_argument( tifffile.imwrite(plate / "A01_s001_label-fixture.tif", fixture) source = NamedSourceBinding( alias="label-fixture", - selector=SourceSelector(filters=( - SourceFilterClause( - subject=SourceFilterSubject.FILE, - match_type=SourceFilterMatchType.CONTAINS, - value="label-fixture", - ), - )), + selector=SourceSelector( + filters=( + SourceFilterClause( + subject=SourceFilterSubject.FILE, + match_type=SourceFilterMatchType.CONTAINS, + value="label-fixture", + ), + ) + ), component_identity=(ComponentSelector(Microscopy.Channel, "1"),), ) document = PipelineDocumentCodec.from_values( @@ -99,24 +105,29 @@ def test_chained_public_callable_uses_declared_main_flow_not_storage_argument( for index in range(2) ], ) - document = PipelineDocumentCodec.from_source( - PipelineDocumentCodec.render(document) - ) - bundle = InProcessCompileInspectionGateway().compile( - CompileInspectionInput( - plate=plate, - pipeline_document=document, - axis_filter=("A01",), - global_pipeline_config=GlobalPipelineConfig(num_workers=1, use_threading=True), - progress_queue=AgentProgressQueue(), + document = PipelineDocumentCodec.from_source(PipelineDocumentCodec.render(document)) + bundle = ( + InProcessCompileInspectionGateway() + .compile( + CompileInspectionInput( + plate=plate, + pipeline_document=document, + axis_filter=("A01",), + global_pipeline_config=GlobalPipelineConfig( + num_workers=1, use_threading=True + ), + progress_queue=AgentProgressQueue(), + ) ) - ).execution_bundle + .execution_bundle + ) context = bundle.runtime_contexts["A01"] first, second = (context.step_plans[index] for index in range(2)) first_invocation = next(first.compiled_function_pattern.iter_invocations()) second_invocation = next(second.compiled_function_pattern.iter_invocations()) [first_image] = ( - plan for plan in first_invocation.artifact_output_plans + plan + for plan in first_invocation.artifact_output_plans if plan.artifact_type is ImageArtifactType ) incoming_ref = first_image.ref().for_plan_type(ArtifactInputPlan) @@ -132,7 +143,8 @@ def test_chained_public_callable_uses_declared_main_flow_not_storage_argument( assert edge.projection is None orchestrator = PipelineOrchestrator( - plate, pipeline_config=document.pipeline_config, + plate, + pipeline_config=document.pipeline_config, ).initialize() results = orchestrator.execute_compiled_bundle( execution_bundle=bundle, @@ -152,7 +164,9 @@ def test_chained_public_callable_uses_declared_main_flow_not_storage_argument( for plan in (image_plan, labels_plan, rows_plan): [record] = context.runtime_value_store.find_matching( RuntimeArtifactQuery.from_output_plan( - plan, axis_id="A01", backend="memory", + plan, + axis_id="A01", + backend="memory", group_key=plan.group_scope().select_runtime_key( invocation.key.group_key ), @@ -161,7 +175,9 @@ def test_chained_public_callable_uses_declared_main_flow_not_storage_argument( records.append(record) image, labels, rows = (record.data for record in records) np.testing.assert_array_equal(np.squeeze(image.data), fixture) - np.testing.assert_array_equal(np.squeeze(object_label_dense_array(labels)), fixture) + np.testing.assert_array_equal( + np.squeeze(object_label_dense_array(labels)), fixture + ) assert rows.subject.object_name == labels_plan.name assert rows.subject.id_field == "object_label" assert tuple(rows.rows.column_values("object_label")) == (1,) @@ -175,9 +191,13 @@ def test_chained_public_callable_uses_declared_main_flow_not_storage_argument( [persisted] = DictReader(stream) assert persisted["object_label"] == "1" assert persisted["pixel_count"] == "16" - [roi_path] = analysis.glob(f"*_{labels_plan.name}_step{step_plan.step_index}*.zip") + [roi_path] = analysis.glob( + f"*_{labels_plan.name}_step{step_plan.step_index}*.zip" + ) assert len(load_rois_from_zip(roi_path)) == 1 saved_images = tuple(step_plan.materialized_output.output_dir.glob("*.tif")) assert saved_images for image_path in saved_images: - np.testing.assert_array_equal(np.squeeze(tifffile.imread(image_path)), fixture) + np.testing.assert_array_equal( + np.squeeze(tifffile.imread(image_path)), fixture + ) diff --git a/tests/integration/test_custom_source_transport.py b/tests/integration/test_custom_source_transport.py index 3d6f005222..fdf36c42d5 100644 --- a/tests/integration/test_custom_source_transport.py +++ b/tests/integration/test_custom_source_transport.py @@ -30,6 +30,7 @@ if dependency_root: sys.path.insert(0, str(WORKTREE)) from openhcs._source_dependencies import ensure_source_checkout_external_paths + ensure_source_checkout_external_paths(Path(dependency_root)) @@ -84,9 +85,9 @@ def _run_child(env, *arguments): while True: remaining = deadline - time.monotonic() assert remaining > 0, f"Child exceeded 30s: {arguments!r}" - assert os.fstat(output_file.fileno()).st_size < OUTPUT_LIMIT, ( - f"Child output exceeded {OUTPUT_LIMIT} bytes: {arguments!r}" - ) + assert ( + os.fstat(output_file.fileno()).st_size < OUTPUT_LIMIT + ), f"Child output exceeded {OUTPUT_LIMIT} bytes: {arguments!r}" try: returncode = process.wait(timeout=min(remaining, 0.1)) except subprocess.TimeoutExpired: @@ -94,9 +95,9 @@ def _run_child(env, *arguments): break output_file.seek(0) captured.extend(output_file.read(OUTPUT_LIMIT)) - assert len(captured) < OUTPUT_LIMIT, ( - f"Child output exceeded {OUTPUT_LIMIT} bytes: {arguments!r}" - ) + assert ( + len(captured) < OUTPUT_LIMIT + ), f"Child output exceeded {OUTPUT_LIMIT} bytes: {arguments!r}" finally: if process.poll() is None: process.kill() @@ -417,8 +418,13 @@ def _spawned_custom_task(payload, execution_id, subject_id): owner = reference.metadata.artifact_outputs[0].measurement_feature_owner results.append(owner.make_row(2.5)) ProgressExecutionContext(execution_id, subject_id).emit( - axis_id="synthetic", step_name="custom", phase=ProgressPhase.STEP_COMPLETED, - status=ProgressStatus.SUCCESS, percent=100, completed=1, total=1, + axis_id="synthetic", + step_name="custom", + phase=ProgressPhase.STEP_COMPLETED, + status=ProgressStatus.SUCCESS, + percent=100, + completed=1, + total=1, ) return os.getpid(), tuple(results) @@ -448,7 +454,8 @@ def _compiled_custom_contexts(payload, runtime): from openhcs.core.compiled_step_plan import CompiledStepPlan from openhcs.core.context.processing_context import ProcessingContext from openhcs.core.function_patterns import ( - compile_function_pattern, normalize_function_pattern, + compile_function_pattern, + normalize_function_pattern, ) from openhcs.core.function_step_transport import FunctionStepTransportAuthority @@ -464,11 +471,14 @@ def _compiled_custom_contexts(payload, runtime): assert callable(invocation.contract.raw_processing_function) prepared = invocation.contract.resolve_runtime_callable() plan = CompiledStepPlan( - step_index=0, step_name="Synthetic custom", - axis_id=axis_id, compiled_function_pattern=pattern, + step_index=0, + step_name="Synthetic custom", + axis_id=axis_id, + compiled_function_pattern=pattern, ) context = ProcessingContext( - axis_id=axis_id, step_plans={0: plan}, + axis_id=axis_id, + step_plans={0: plan}, filemanager=FileManager({Backend.MEMORY.value: MemoryStorageBackend()}), ) context.freeze() @@ -480,11 +490,15 @@ def _compiled_custom_contexts(payload, runtime): assert transported.contract is not invocation.contract assert invocation.contract.resolve_runtime_callable() is prepared assert callable(invocation.contract.raw_processing_function) - assert FunctionStepTransportAuthority.normalize_callable_contract( - transported.contract - ) is transported.contract + assert ( + FunctionStepTransportAuthority.normalize_callable_contract( + transported.contract + ) + is transported.contract + ) return CompiledExecutionBundle.from_runtime_contexts( - pipeline_definition=(), runtime_contexts=contexts, + pipeline_definition=(), + runtime_contexts=contexts, worker_assignments={f"worker_{i}": [axis] for i, axis in enumerate(contexts)}, runtime_environment=runtime, ).transport_contexts @@ -493,7 +507,8 @@ def _compiled_custom_contexts(payload, runtime): def _spawn(path): import multiprocessing from openhcs.core.compiled_execution import ( - CompiledRuntimeEnvironmentPlan, CompiledWorkerStartPlan, + CompiledRuntimeEnvironmentPlan, + CompiledWorkerStartPlan, ) from openhcs.core.config import MultiprocessingStartMethod from openhcs.core.orchestrator.cancellation import ExecutionCancellationSignal @@ -511,19 +526,25 @@ class RichProgressContext(ProgressExecutionContext): payload = _load(path) method = MultiprocessingStartMethod.SPAWN runtime = CompiledRuntimeEnvironmentPlan( - worker_start=CompiledWorkerStartPlan(method, method, "source fixture", False, False), - use_threading=False, configured_num_workers=2, + worker_start=CompiledWorkerStartPlan( + method, method, "source fixture", False, False + ), + use_threading=False, + configured_num_workers=2, ) queue = multiprocessing.get_context(method.value).Queue() resources = WorkerExecutorFactory( - log_file_base=None, progress_queue=queue, + log_file_base=None, + progress_queue=queue, cancellation=ExecutionCancellationSignal(), ).create(runtime_environment=runtime, actual_max_workers=2) contexts = _compiled_custom_contexts(payload, runtime) try: with resources.execution_context(): tasks = tuple( - resources.executor.submit(_spawned_compiled_custom_task, lane_context, helpers) + resources.executor.submit( + _spawned_compiled_custom_task, lane_context, helpers + ) for lane_context, (_reference, helpers) in zip( contexts.values(), payload, strict=True ) @@ -537,7 +558,10 @@ class RichProgressContext(ProgressExecutionContext): assert values == [[0, 1, 2], [3, 4, 5]] assert type(row) is helpers[4] and row == helpers[5] pid, rows = resources.executor.submit( - _spawned_custom_task, payload, context.execution_id, context.subject_id, + _spawned_custom_task, + payload, + context.execution_id, + context.subject_id, ).result(timeout=15) assert pid != os.getpid() for row, (_reference, helpers) in zip(rows, payload, strict=True): @@ -623,10 +647,20 @@ def test_persisted_payload_preserves_nominal_helpers_in_independent_consumer( def test_persisted_custom_revision_executes_in_original_spawn_factory( - tmp_path, transport_environment, + tmp_path, + transport_environment, ): - print(_run_child(transport_environment, "spawn", str(tmp_path / "spawn.pkl"), - "reference", "cold", "unchanged"), end="") + print( + _run_child( + transport_environment, + "spawn", + str(tmp_path / "spawn.pkl"), + "reference", + "cold", + "unchanged", + ), + end="", + ) @pytest.mark.parametrize("mutation", ["changed", "deleted", "renamed"]) diff --git a/tests/integration/test_image_math_publication_620.py b/tests/integration/test_image_math_publication_620.py index b849e8e233..21b86f0924 100644 --- a/tests/integration/test_image_math_publication_620.py +++ b/tests/integration/test_image_math_publication_620.py @@ -1,4 +1,5 @@ """Default automatic/named publication through actual registered ImageMath.""" + from openhcs.core.progress import ProgressExecutionContext from dataclasses import replace from hashlib import sha256 @@ -11,10 +12,14 @@ import tifffile from openhcs.agent.services.artifact_plan_inspection_service import ( - AgentProgressQueue, CompileInspectionInput, InProcessCompileInspectionGateway, + AgentProgressQueue, + CompileInspectionInput, + InProcessCompileInspectionGateway, ) from openhcs.core.config import ( - GlobalPipelineConfig, LazyNapariStreamingConfig, LazyPathPlanningConfig, + GlobalPipelineConfig, + LazyNapariStreamingConfig, + LazyPathPlanningConfig, LazyProcessingConfig, ) from openhcs.core.orchestrator.execution_result import RuntimeObservationMode @@ -24,7 +29,8 @@ from openhcs.core.source_metadata import SourceVoxelSpacing from openhcs.core.source_workspace_projection import VirtualWorkspaceSourceProjection from openhcs.core.virtual_workspace_metadata import ( - OpenHCSMetadataSubdirectories, VirtualWorkspaceSourceProjectionEntries, + OpenHCSMetadataSubdirectories, + VirtualWorkspaceSourceProjectionEntries, ) from openhcs.interop.cellprofiler.pipeline_import import import_cellprofiler_pipeline from openhcs.domains.microscopy.axes import Microscopy @@ -32,70 +38,132 @@ @pytest.mark.parametrize("rgb", (False, True)) -def test_registered_mixed_image_math_publishes_once_in_both_operand_orders(tmp_path, rgb): +def test_registered_mixed_image_math_publishes_once_in_both_operand_orders( + tmp_path, rgb +): scalar = np.arange(180, dtype=np.uint8).reshape(12, 15) color = ( np.stack((scalar, np.full_like(scalar, 32), np.full_like(scalar, 240)), axis=-1) - if rgb else 255 - scalar + if rgb + else 255 - scalar ) - source = tmp_path / 'input' + source = tmp_path / "input" source.mkdir() - for name, channel, pixels in (('Gray', 1, scalar), ('Color', 2, color)): - tifffile.imwrite(source / f'A01_s1_w{channel}_z1_t1_{name}.tif', pixels) - originals = {image: sha256(image.read_bytes()).hexdigest() for image in source.glob('*.tif')} - fixture = Path(__file__).resolve().parents[1] / 'fixtures/pipelines/image_math_mixed_publication.cppipe' + for name, channel, pixels in (("Gray", 1, scalar), ("Color", 2, color)): + tifffile.imwrite(source / f"A01_s1_w{channel}_z1_t1_{name}.tif", pixels) + originals = { + image: sha256(image.read_bytes()).hexdigest() for image in source.glob("*.tif") + } + fixture = ( + Path(__file__).resolve().parents[1] + / "fixtures/pipelines/image_math_mixed_publication.cppipe" + ) steps, imported = import_cellprofiler_pipeline(fixture) config = replace( - imported, dataset_source=SourceBindingsSource, + imported, + dataset_source=SourceBindingsSource, processing_config=LazyProcessingConfig( - variable_components=[Microscopy.ZIndex], group_by=Microscopy.Channel), - path_planning_config=LazyPathPlanningConfig(global_output_folder=tmp_path / 'results'), - source_bindings_config=replace(imported.source_bindings_config, - metadata_rules=(MetadataExtractionRule(source=MetadataSource.FILE_NAME, - pattern=r'(?P[A-H]\d{2})_s(?P\d+)_w(?P\d+)_z(?P\d+)_t(?P\d+)_(?:Gray|Color)\.tif'),), - source_stack_components=(), source_voxel_spacing=SourceVoxelSpacing((0.5, 0.5))), + variable_components=[Microscopy.ZIndex], group_by=Microscopy.Channel + ), + path_planning_config=LazyPathPlanningConfig( + global_output_folder=tmp_path / "results" + ), + source_bindings_config=replace( + imported.source_bindings_config, + metadata_rules=( + MetadataExtractionRule( + source=MetadataSource.FILE_NAME, + pattern=r"(?P[A-H]\d{2})_s(?P\d+)_w(?P\d+)_z(?P\d+)_t(?P\d+)_(?:Gray|Color)\.tif", + ), + ), + source_stack_components=(), + source_voxel_spacing=SourceVoxelSpacing((0.5, 0.5)), + ), napari_streaming_config=LazyNapariStreamingConfig(enabled=False), ) - document = PipelineDocumentCodec.from_values(pipeline_config=config, pipeline_steps=steps) - bundle = InProcessCompileInspectionGateway().compile(CompileInspectionInput( - plate=source, pipeline_document=document, axis_filter=('A01',), - global_pipeline_config=GlobalPipelineConfig(num_workers=1, use_threading=True), - progress_queue=AgentProgressQueue(), - )).execution_bundle - outcomes = PipelineOrchestrator(source, pipeline_config=config).initialize().execute_compiled_bundle( - execution_bundle=bundle, max_workers=1, - runtime_observation_mode=RuntimeObservationMode.MERGE_INTO_PARENT, - progress_queue=AgentProgressQueue(), - progress_context=ProgressExecutionContext( - execution_id='engineering620-registered', - subject_id=str(source), - ), + document = PipelineDocumentCodec.from_values( + pipeline_config=config, pipeline_steps=steps ) - assert outcomes['A01'].is_success(), outcomes['A01'].error_message + bundle = ( + InProcessCompileInspectionGateway() + .compile( + CompileInspectionInput( + plate=source, + pipeline_document=document, + axis_filter=("A01",), + global_pipeline_config=GlobalPipelineConfig( + num_workers=1, use_threading=True + ), + progress_queue=AgentProgressQueue(), + ) + ) + .execution_bundle + ) + outcomes = ( + PipelineOrchestrator(source, pipeline_config=config) + .initialize() + .execute_compiled_bundle( + execution_bundle=bundle, + max_workers=1, + runtime_observation_mode=RuntimeObservationMode.MERGE_INTO_PARENT, + progress_queue=AgentProgressQueue(), + progress_context=ProgressExecutionContext( + execution_id="engineering620-registered", + subject_id=str(source), + ), + ) + ) + assert outcomes["A01"].is_success(), outcomes["A01"].error_message gray_expected = scalar.astype(np.float32) / 255 - color_expected = rgb2gray(color.astype(np.float32) / 255) if rgb else color.astype(np.float32) / 255 - expected = {'GrayIndependent': gray_expected, 'ColorIndependent': color_expected, - 'Forward': (gray_expected + color_expected) / 2, - 'Reverse': (gray_expected + color_expected) / 2} - plate = tmp_path / 'results/input_openhcs' - saved = tuple((plate / 'images').glob('*.tif')) + color_expected = ( + rgb2gray(color.astype(np.float32) / 255) + if rgb + else color.astype(np.float32) / 255 + ) + expected = { + "GrayIndependent": gray_expected, + "ColorIndependent": color_expected, + "Forward": (gray_expected + color_expected) / 2, + "Reverse": (gray_expected + color_expected) / 2, + } + plate = tmp_path / "results/input_openhcs" + saved = tuple((plate / "images").glob("*.tif")) assert len(saved) == 4 - persisted = json.loads((plate / 'openhcs_metadata.json').read_text()) - projection = VirtualWorkspaceSourceProjection.from_openhcs_metadata(plate, persisted) + persisted = json.loads((plate / "openhcs_metadata.json").read_text()) + projection = VirtualWorkspaceSourceProjection.from_openhcs_metadata( + plate, persisted + ) # The lookup intentionally indexes relative and absolute aliases; count the # original persisted declarations, not the derived lookup's keys. - declared = tuple(entry for directory in OpenHCSMetadataSubdirectories(persisted).values() - for entry in VirtualWorkspaceSourceProjectionEntries.from_subdirectory(directory).entries.values()) + declared = tuple( + entry + for directory in OpenHCSMetadataSubdirectories(persisted).values() + for entry in VirtualWorkspaceSourceProjectionEntries.from_subdirectory( + directory + ).entries.values() + ) assert len(declared) == 4 for name, pixels in expected.items(): - image, = (image for image in saved if image.name.endswith('_' + name + '.tif')) + (image,) = ( + image for image in saved if image.name.endswith("_" + name + ".tif") + ) np.testing.assert_array_equal(tifffile.imread(image), pixels) relative = str(image.relative_to(plate)) - metadata = projection.source_projections_by_virtual_path[relative].persisted_image_metadata() + metadata = projection.source_projections_by_virtual_path[ + relative + ].persisted_image_metadata() assert metadata.source_voxel_spacing == SourceVoxelSpacing((0.5, 0.5)) - assert metadata.source_component_metadata['well'] == 'A01' - assert metadata.source_component_metadata['channel'] == ('2' if name in ('ColorIndependent', 'Reverse') else '1') + assert metadata.source_component_metadata["well"] == "A01" + assert metadata.source_component_metadata["channel"] == ( + "2" if name in ("ColorIndependent", "Reverse") else "1" + ) assert metadata.source_image_names == (name,) for image, digest in originals.items(): assert sha256(image.read_bytes()).hexdigest() == digest - print('REGISTERED_DEFAULT_PUBLICATION', rgb, len(saved), 'exact elements', sum(pixels.size for pixels in expected.values())) + print( + "REGISTERED_DEFAULT_PUBLICATION", + rgb, + len(saved), + "exact elements", + sum(pixels.size for pixels in expected.values()), + ) diff --git a/tests/integration/test_measured_cli_session.py b/tests/integration/test_measured_cli_session.py index b81290c951..230ff057b5 100644 --- a/tests/integration/test_measured_cli_session.py +++ b/tests/integration/test_measured_cli_session.py @@ -213,7 +213,13 @@ def test_read_only_source_plate_completes_and_stays_byte_identical( port = _port(2) try: completed = subprocess.run( - _command(plate, source, output_dir, port, wait_ms=int(RUN_DEADLINE_SECONDS * 1000)), + _command( + plate, + source, + output_dir, + port, + wait_ms=int(RUN_DEADLINE_SECONDS * 1000), + ), cwd=REPOSITORY_ROOT, env=_environment(tmp_path), capture_output=True, diff --git a/tests/integration/test_measurement_declaration_journey.py b/tests/integration/test_measurement_declaration_journey.py index 0a1538757d..91f4458067 100644 --- a/tests/integration/test_measurement_declaration_journey.py +++ b/tests/integration/test_measurement_declaration_journey.py @@ -40,7 +40,9 @@ ) from openhcs.core.function_step_document import FunctionStepDocumentCodec from openhcs.core.measurement_feature_queries import measurement_values_for_feature -from openhcs.core.measurement_row_materialization import DataclassMeasurementColumnarRows +from openhcs.core.measurement_row_materialization import ( + DataclassMeasurementColumnarRows, +) from openhcs.core.memory import numpy from openhcs.core.orchestrator.execution_result import RuntimeObservationMode from openhcs.core.orchestrator.orchestrator import PipelineOrchestrator @@ -121,7 +123,8 @@ def owns_primary_measurement_feature_name(cls, feature_name): COUNT_IMAGE = MainFlowStackOutputSpec.output("CountedImage", ImageArtifactType) MISSING_SUBJECT_ROWS = MainFlowStackOutputSpec.output( - "PixelCounts", MeasurementsArtifactType, + "PixelCounts", + MeasurementsArtifactType, materialization=MaterializationSpec(CsvOptions()), measurement_feature_owner=CountFeatureOwner, ) @@ -216,8 +219,14 @@ def _document(*, selected=True, two_producers=True, same_source=False): identify_primary_objects, "Primary", { - _binding_name(IdentifyPrimaryObjectsModule, ArtifactInputPlan, ImageArtifactType): "DNA", - _binding_name(IdentifyPrimaryObjectsModule, ArtifactOutputPlan, ObjectLabelsArtifactType): "Nuclei", + _binding_name( + IdentifyPrimaryObjectsModule, ArtifactInputPlan, ImageArtifactType + ): "DNA", + _binding_name( + IdentifyPrimaryObjectsModule, + ArtifactOutputPlan, + ObjectLabelsArtifactType, + ): "Nuclei", "exclude_size": False, "exclude_border_objects": False, "unclump_method": UnclumpMethod.NONE, @@ -231,15 +240,23 @@ def _document(*, selected=True, two_producers=True, same_source=False): identify_secondary_objects, "Secondary", { - _binding_name(IdentifySecondaryObjectsModule, ArtifactInputPlan, ImageArtifactType): measurement_image, - _binding_name(IdentifySecondaryObjectsModule, ArtifactInputPlan, ObjectLabelsArtifactType): "Nuclei", + _binding_name( + IdentifySecondaryObjectsModule, ArtifactInputPlan, ImageArtifactType + ): measurement_image, + _binding_name( + IdentifySecondaryObjectsModule, + ArtifactInputPlan, + ObjectLabelsArtifactType, + ): "Nuclei", IdentifySecondaryObjectsModule.secondary_output_binding.require_parameter_name(): "Cells", "method": SecondaryMethod.DISTANCE_N, "distance_to_dilate": 2, }, ) kwargs = { - MeasureObjectIntensityModule.image_measurement_binding.require_parameter_name(): (measurement_image,), + MeasureObjectIntensityModule.image_measurement_binding.require_parameter_name(): ( + measurement_image, + ), } if selected: kwargs[ @@ -250,12 +267,19 @@ def _document(*, selected=True, two_producers=True, same_source=False): pipeline_config=PipelineConfig( dataset_source=SourceBindingsSource, source_bindings_config=LazySourceBindingsConfig( - bindings=(_source("DNA", "1"),) if same_source else ( - _source("DNA", "1"), _source("Actin", "2"), + bindings=( + (_source("DNA", "1"),) + if same_source + else ( + _source("DNA", "1"), + _source("Actin", "2"), + ) ), ), ), - pipeline_steps=[primary, secondary, measure] if two_producers else [primary, measure], + pipeline_steps=( + [primary, secondary, measure] if two_producers else [primary, measure] + ), ) @@ -268,15 +292,19 @@ def _write_plate(path, *, same_source=False): def _compile(path, document, global_config): - return InProcessCompileInspectionGateway().compile( - CompileInspectionInput( - plate=path, - pipeline_document=document, - axis_filter=("A01",), - global_pipeline_config=global_config, - progress_queue=AgentProgressQueue(), + return ( + InProcessCompileInspectionGateway() + .compile( + CompileInspectionInput( + plate=path, + pipeline_document=document, + axis_filter=("A01",), + global_pipeline_config=global_config, + progress_queue=AgentProgressQueue(), + ) ) - ).execution_bundle + .execution_bundle + ) def _execute(path, document, bundle): @@ -295,16 +323,30 @@ def _execute(path, document, bundle): ) -@pytest.mark.parametrize("image_names,current_scale", [ - (("DNA",), 1), (("DNA", "Actin"), 1), (("Actin", "DNA"), 1), - (("DNAHalf",), 1), (("DNA", "DNAHalf"), 1), (("DNAHalf", "DNA"), 1), - (("DNA", "Actin"), 0.5), -]) +@pytest.mark.parametrize( + "image_names,current_scale", + [ + (("DNA",), 1), + (("DNA", "Actin"), 1), + (("Actin", "DNA"), 1), + (("DNAHalf",), 1), + (("DNA", "DNAHalf"), 1), + (("DNAHalf", "DNA"), 1), + (("DNA", "Actin"), 0.5), + ], +) def test_photometry_carrier_preserves_raw_aliases_and_produced_pixels( - tmp_path, image_names, current_scale, registered_current_transform, + tmp_path, + image_names, + current_scale, + registered_current_transform, ): """A stored label cohort must not substitute label IDs for measured images.""" - from openhcs.processing.backends.cellprofiler.intensity import rescale_intensity, RescaleMethod + from openhcs.processing.backends.cellprofiler.intensity import ( + rescale_intensity, + RescaleMethod, + ) + dna = np.zeros((16, 16), dtype=np.float32) dna[6:8, 6:8] = ((0.6, 0.7), (0.8, 0.9)) actin = (0.1 + np.arange(256).reshape(16, 16) * 0.001).astype(np.float32) @@ -312,65 +354,111 @@ def test_photometry_carrier_preserves_raw_aliases_and_produced_pixels( tifffile.imwrite(tmp_path / "A01_s001_w2_z001_t001.tif", actin) document = _document() if "DNAHalf" in image_names: - document.pipeline_steps.insert(0, _step(rescale_intensity, "Produced half intensity", { - "select_the_input_image": "DNA", "name_the_output_image": "DNAHalf", - "rescale_method": RescaleMethod.DIVIDE_BY_VALUE, "divisor_value": 2.0, - })) - document.pipeline_steps[-1] = _step(measure_object_intensity, "Actual photometry", { - "select_images_to_measure": image_names, "select_object_sets_to_measure": ("Cells",), - }) + document.pipeline_steps.insert( + 0, + _step( + rescale_intensity, + "Produced half intensity", + { + "select_the_input_image": "DNA", + "name_the_output_image": "DNAHalf", + "rescale_method": RescaleMethod.DIVIDE_BY_VALUE, + "divisor_value": 2.0, + }, + ), + ) + document.pipeline_steps[-1] = _step( + measure_object_intensity, + "Actual photometry", + { + "select_images_to_measure": image_names, + "select_object_sets_to_measure": ("Cells",), + }, + ) if current_scale != 1: - document.pipeline_steps.insert(-1, _step( - registered_current_transform, "Transform current image carrier", {}, - )) + document.pipeline_steps.insert( + -1, + _step( + registered_current_transform, + "Transform current image carrier", + {}, + ), + ) measurement = document.pipeline_steps[-1] measurement.processing_config = replace( - measurement.processing_config, input_source=InputSource.PREVIOUS_STEP, + measurement.processing_config, + input_source=InputSource.PREVIOUS_STEP, ) document = PipelineDocumentCodec.from_source(PipelineDocumentCodec.render(document)) - bundle = _compile(tmp_path, document, GlobalPipelineConfig(num_workers=1, use_threading=True)) + bundle = _compile( + tmp_path, document, GlobalPipelineConfig(num_workers=1, use_threading=True) + ) context = bundle.runtime_contexts["A01"] - plan = context.step_plans[len(document.pipeline_steps)-1] + plan = context.step_plans[len(document.pipeline_steps) - 1] group = plan.compiled_function_pattern.default_group selected = plan.stored_primary_input_edges_for_group(group, None) if any(name in ("DNA", "Actin") for name in image_names): - assert selected is None # Raw STEP_INPUT aliases need the current image carrier. + assert ( + selected is None + ) # Raw STEP_INPUT aliases need the current image carrier. results = _execute(tmp_path, document, bundle) - assert results['A01'].is_success(), results['A01'].error_message + assert results["A01"].is_success(), results["A01"].error_message (labels,) = context.runtime_value_store.find(name="Cells", axis_id="A01") foreground = object_label_dense_array(labels.data).squeeze() == 1 (invocation,) = group.invocations (output,) = invocation.artifact_output_plans (measurement,) = context.runtime_value_store.find( - name=output.name, artifact_type=MeasurementsArtifactType, axis_id="A01", + name=output.name, + artifact_type=MeasurementsArtifactType, + axis_id="A01", ) expected_images = { - "DNA": dna * current_scale, "Actin": actin * current_scale, "DNAHalf": dna / 2, + "DNA": dna * current_scale, + "Actin": actin * current_scale, + "DNAHalf": dna / 2, } for image_name in image_names: values = expected_images[image_name][foreground] assert values.size and values.std() > 0 - for feature, expected in (("MeanIntensity", values.mean()), ("MinIntensity", values.min()), - ("MaxIntensity", values.max()), ("StdIntensity", values.std())): + for feature, expected in ( + ("MeanIntensity", values.mean()), + ("MinIntensity", values.min()), + ("MaxIntensity", values.max()), + ("StdIntensity", values.std()), + ): actual = measurement_values_for_feature( - (measurement.data,), f"Intensity_{feature}_{image_name}", - object_count=1, object_name="Cells", dialect=CELLPROFILER_MEASUREMENT_DIALECT, + (measurement.data,), + f"Intensity_{feature}_{image_name}", + object_count=1, + object_name="Cells", + dialect=CELLPROFILER_MEASUREMENT_DIALECT, ) assert tuple(actual) == pytest.approx((expected,), abs=1e-7) def test_label_only_measurement_keeps_stored_cohort(tmp_path): from openhcs.processing.backends.cellprofiler.shape import ( - MeasureObjectSizeShapeModule, measure_object_size_shape, + MeasureObjectSizeShapeModule, + measure_object_size_shape, ) + _write_plate(tmp_path) document = _document() - document.pipeline_steps[-1] = _step(measure_object_size_shape, "Label-only area", { - MeasureObjectSizeShapeModule.object_measurement_binding.require_parameter_name(): ("Cells",), - "calculate_advanced": False, "calculate_zernikes": False, - }) + document.pipeline_steps[-1] = _step( + measure_object_size_shape, + "Label-only area", + { + MeasureObjectSizeShapeModule.object_measurement_binding.require_parameter_name(): ( + "Cells", + ), + "calculate_advanced": False, + "calculate_zernikes": False, + }, + ) document = PipelineDocumentCodec.from_source(PipelineDocumentCodec.render(document)) - bundle = _compile(tmp_path, document, GlobalPipelineConfig(num_workers=1, use_threading=True)) + bundle = _compile( + tmp_path, document, GlobalPipelineConfig(num_workers=1, use_threading=True) + ) context = bundle.runtime_contexts["A01"] plan = context.step_plans[len(document.pipeline_steps) - 1] group = plan.compiled_function_pattern.default_group @@ -383,20 +471,31 @@ def test_label_only_measurement_keeps_stored_cohort(tmp_path): (invocation,) = group.invocations (output,) = invocation.artifact_output_plans (measurement,) = context.runtime_value_store.find( - name=output.name, artifact_type=MeasurementsArtifactType, axis_id="A01", + name=output.name, + artifact_type=MeasurementsArtifactType, + axis_id="A01", ) actual = measurement_values_for_feature( - (measurement.data,), "AreaShape_Area", object_count=1, object_name="Cells", + (measurement.data,), + "AreaShape_Area", + object_count=1, + object_name="Cells", dialect=CELLPROFILER_MEASUREMENT_DIALECT, ) assert tuple(actual) == (np.count_nonzero(object_label_dense_array(labels.data)),) -@pytest.mark.parametrize("same_source", [True, False], ids=["single-source", "paired-channels"]) -def test_exact_secondary_selector_survives_authoring_compile_and_execution(tmp_path, same_source): +@pytest.mark.parametrize( + "same_source", [True, False], ids=["single-source", "paired-channels"] +) +def test_exact_secondary_selector_survives_authoring_compile_and_execution( + tmp_path, same_source +): _write_plate(tmp_path, same_source=same_source) document = _document(same_source=same_source) - selector = MeasureObjectIntensityModule.object_measurement_binding.require_parameter_name() + selector = ( + MeasureObjectIntensityModule.object_measurement_binding.require_parameter_name() + ) source = PipelineDocumentCodec.render(document) reconstructed = PipelineDocumentCodec.from_source(source) measurement = reconstructed.pipeline_steps[-1] @@ -423,14 +522,16 @@ def test_exact_secondary_selector_survives_authoring_compile_and_execution(tmp_p plan = context.step_plans[2] invocation = next(plan.compiled_function_pattern.iter_invocations()) (edge,) = ( - edge for edge in invocation.artifact_input_edges + edge + for edge in invocation.artifact_input_edges if edge.spec.artifact_type is ObjectLabelsArtifactType ) assert edge.spec.name == "Cells" assert edge.spec.parameter_name == "labels" assert edge.storage_plan.source_step_id == 1 producer_output = next( - output for output in secondary_invocation.artifact_output_plans + output + for output in secondary_invocation.artifact_output_plans if output.ref() == edge.spec.ref().for_plan_type(ArtifactOutputPlan) ) assert edge.storage_plan.relations == producer_output.relations @@ -450,7 +551,9 @@ def test_exact_secondary_selector_survives_authoring_compile_and_execution(tmp_p secondary_area = np.count_nonzero(object_label_dense_array(secondary.data)) assert secondary_area > primary_area > 0 [output] = invocation.artifact_output_plans - [measurement] = store.find(name=output.name, artifact_type=MeasurementsArtifactType, axis_id="A01") + [measurement] = store.find( + name=output.name, artifact_type=MeasurementsArtifactType, axis_id="A01" + ) assert measurement.data.subject.object_name == "Cells" image_name = "DNA" if same_source else "Actin" values = measurement_values_for_feature( @@ -462,7 +565,9 @@ def test_exact_secondary_selector_survives_authoring_compile_and_execution(tmp_p ) # The DNA seed is four bright pixels; the grown cell adds dim background. # Measuring primary labels instead would return exactly 4.0. - expected = 4.0 + 0.25 * (secondary_area - 4) if same_source else float(secondary_area) + expected = ( + 4.0 + 0.25 * (secondary_area - 4) if same_source else float(secondary_area) + ) assert tuple(values) == (expected,) @@ -478,7 +583,8 @@ def test_omitted_secondary_selector_still_fails_closed(tmp_path): @pytest.mark.parametrize("producer_group_by", [Ungrouped, Microscopy.Channel]) def test_explicit_measurement_rosters_preserve_compiled_source_groups( - tmp_path, producer_group_by, + tmp_path, + producer_group_by, ): from openhcs.core.steps.function_execution import FunctionStepExecutor @@ -486,14 +592,21 @@ def test_explicit_measurement_rosters_preserve_compiled_source_groups( original = _document() for producer in original.pipeline_steps[:-1]: producer.processing_config = replace( - producer.processing_config, group_by=producer_group_by, + producer.processing_config, + group_by=producer_group_by, ) original.pipeline_steps[-1] = _step( measure_object_intensity, "Measure both source images and object sets", { - MeasureObjectIntensityModule.image_measurement_binding.require_parameter_name(): ("DNA", "Actin"), - MeasureObjectIntensityModule.object_measurement_binding.require_parameter_name(): ("Nuclei", "Cells"), + MeasureObjectIntensityModule.image_measurement_binding.require_parameter_name(): ( + "DNA", + "Actin", + ), + MeasureObjectIntensityModule.object_measurement_binding.require_parameter_name(): ( + "Nuclei", + "Cells", + ), }, ) document = PipelineDocumentCodec.from_values( @@ -501,13 +614,17 @@ def test_explicit_measurement_rosters_preserve_compiled_source_groups( original.pipeline_config, source_bindings_config=LazySourceBindingsConfig( bindings=(_source("DNA", "1"), _source("Actin", "2")), - match_plan=SourceBindingMatchPlan(method=SourceBindingMatchMethod.ORDER), + match_plan=SourceBindingMatchPlan( + method=SourceBindingMatchMethod.ORDER + ), ), ), pipeline_steps=original.pipeline_steps, ) document = PipelineDocumentCodec.from_source(PipelineDocumentCodec.render(document)) - bundle = _compile(tmp_path, document, GlobalPipelineConfig(num_workers=1, use_threading=True)) + bundle = _compile( + tmp_path, document, GlobalPipelineConfig(num_workers=1, use_threading=True) + ) context = bundle.runtime_contexts["A01"] executor = FunctionStepExecutor(context, 2) prepared = executor._prepare_groups(executor._detect_patterns()) @@ -517,16 +634,23 @@ def test_explicit_measurement_rosters_preserve_compiled_source_groups( assert all(len(patterns) == 1 for patterns in prepared.values()) invocation = next(executor.plan.compiled_function_pattern.iter_invocations()) assert tuple( - spec.name for spec in invocation.contract.artifact_inputs.of_artifact_type(ImageArtifactType) + spec.name + for spec in invocation.contract.artifact_inputs.of_artifact_type( + ImageArtifactType + ) ) == ("DNA", "Actin") results = _execute(tmp_path, document, bundle) assert results["A01"].is_success(), results["A01"].error_message (output,) = invocation.artifact_output_plans measurements = context.runtime_value_store.find( - name=output.name, artifact_type=MeasurementsArtifactType, axis_id="A01", + name=output.name, + artifact_type=MeasurementsArtifactType, + axis_id="A01", ) assert len(measurements) == len(expected_keys) - assert tuple(record.key.scope.value_text for record in measurements) == expected_keys + assert ( + tuple(record.key.scope.value_text for record in measurements) == expected_keys + ) for object_name in ("Nuclei", "Cells"): (labels,) = context.runtime_value_store.find(name=object_name, axis_id="A01") area = np.count_nonzero(object_label_dense_array(labels.data)) @@ -556,10 +680,13 @@ def test_omitted_selector_remains_valid_for_one_label_producer(tmp_path): GlobalPipelineConfig(num_workers=1, use_threading=True), ) invocation = next( - bundle.runtime_contexts["A01"].step_plans[1].compiled_function_pattern.iter_invocations() + bundle.runtime_contexts["A01"] + .step_plans[1] + .compiled_function_pattern.iter_invocations() ) (edge,) = ( - edge for edge in invocation.artifact_input_edges + edge + for edge in invocation.artifact_input_edges if edge.spec.artifact_type is ObjectLabelsArtifactType ) assert edge.spec.name == "Nuclei" @@ -567,7 +694,9 @@ def test_omitted_selector_remains_valid_for_one_label_producer(tmp_path): @pytest.mark.parametrize("valid", [False, True]) def test_headless_entrypoint_requires_subject_and_executes_corrected_rows( - tmp_path, valid, registered_count_callable, + tmp_path, + valid, + registered_count_callable, ): _write_plate(tmp_path) document = PipelineDocumentCodec.from_values( @@ -577,19 +706,21 @@ def test_headless_entrypoint_requires_subject_and_executes_corrected_rows( bindings=(_source("DNA", "1"),), ), ), - pipeline_steps=[_step( - registered_count_callable, - "Count pixels", {}, - )], - ) - document = PipelineDocumentCodec.from_source( - PipelineDocumentCodec.render(document) + pipeline_steps=[ + _step( + registered_count_callable, + "Count pixels", + {}, + ) + ], ) + document = PipelineDocumentCodec.from_source(PipelineDocumentCodec.render(document)) invocation = next( normalize_function_pattern(document.pipeline_steps[0].func).iter_items() ) [rows] = ( - spec for spec in invocation.contract.artifact_specs.specs + spec + for spec in invocation.contract.artifact_specs.specs if spec.artifact_type is MeasurementsArtifactType ) assert rows.measurement_feature_owner is CountFeatureOwner @@ -611,7 +742,9 @@ def test_headless_entrypoint_requires_subject_and_executes_corrected_rows( ) is valid global_config = GlobalPipelineConfig(num_workers=1, use_threading=True) if not valid: - with pytest.raises(ValueError, match="PixelCounts.*no declared measurement subject") as exc: + with pytest.raises( + ValueError, match="PixelCounts.*no declared measurement subject" + ) as exc: _compile(tmp_path, document, global_config) assert "ImageMeasurementSubjectRelation" in str(exc.value) return diff --git a/tests/integration/test_neurite_outgrowth_synthetic_plate.py b/tests/integration/test_neurite_outgrowth_synthetic_plate.py index 14e16ffa01..bddccebbff 100644 --- a/tests/integration/test_neurite_outgrowth_synthetic_plate.py +++ b/tests/integration/test_neurite_outgrowth_synthetic_plate.py @@ -221,9 +221,7 @@ def observe_dense_output(output, components): progress_events = [] while True: try: - progress_events.append( - progress_queue.get_nowait() - ) + progress_events.append(progress_queue.get_nowait()) except queue.Empty: break live_payloads = tuple( @@ -315,7 +313,8 @@ def observe_dense_output(output, components): # Checkpoints are the declared outputs between the label artifacts and # the trailing morphology table; the callable's output family owns them. checkpoint_names = tuple( - spec.name for spec in CellProfilerNeuriteEngineProfile.artifact_outputs()[6:-1] + spec.name + for spec in CellProfilerNeuriteEngineProfile.artifact_outputs()[6:-1] ) for artifact_name, expected_checkpoint in zip( checkpoint_names, @@ -331,7 +330,9 @@ def observe_dense_output(output, components): retained, np.asarray(expected_checkpoint)[0], ) - assert len({path.name for path in checkpoint_paths}) == 2 * len(checkpoint_names) + assert len({path.name for path in checkpoint_paths}) == 2 * len( + checkpoint_names + ) assert compiled_plan.output_dataset_root is not None output_dataset_root = Path(compiled_plan.output_dataset_root) diff --git a/tests/integration/test_primary_segmentation_diagnostics_journey.py b/tests/integration/test_primary_segmentation_diagnostics_journey.py index 22755da427..9d45b7ff51 100644 --- a/tests/integration/test_primary_segmentation_diagnostics_journey.py +++ b/tests/integration/test_primary_segmentation_diagnostics_journey.py @@ -83,7 +83,8 @@ def test_real_registered_ipo_returns_same_run_stage_pixels(mode): diagnostics.unedited_objects.data, objects.variant_labels(UneditedLabels) ) np.testing.assert_array_equal( - diagnostics.small_removed_objects.data, objects.variant_labels(SmallRemovedLabels) + diagnostics.small_removed_objects.data, + objects.variant_labels(SmallRemovedLabels), ) assert measurements.row_count() > 0 if mode in ("empty", "disabled", "watershed-disabled"): @@ -251,9 +252,7 @@ def step(func, name, kwargs): ), pipeline_steps=[primary, secondary], ) - document = PipelineDocumentCodec.from_source( - PipelineDocumentCodec.render(document) - ) + document = PipelineDocumentCodec.from_source(PipelineDocumentCodec.render(document)) bundle = ( InProcessCompileInspectionGateway() .compile( @@ -354,7 +353,8 @@ def step(func, name, kwargs): persisted.append(record.data) diagnostics = PrimaryObjectDiagnosticPlanes(*persisted) np.testing.assert_array_equal( - diagnostics.unedited_objects.data, primary_record.data.variant_labels(UneditedLabels) + diagnostics.unedited_objects.data, + primary_record.data.variant_labels(UneditedLabels), ) np.testing.assert_array_equal( diagnostics.small_removed_objects.data, diff --git a/tests/integration/test_session_journey.py b/tests/integration/test_session_journey.py index f99d4f83ec..f480cb2fcd 100644 --- a/tests/integration/test_session_journey.py +++ b/tests/integration/test_session_journey.py @@ -145,7 +145,11 @@ def test_headless_mcp_journey_adds_compiles_and_runs_a_dataset(tmp_path: Path) - {"after_sequence": 0, "timeout_seconds": 0.0}, )["events"] } - assert {"DatasetsChanged", "CompiledStateChanged", "DatasetExecutionFinished"} <= kinds + assert { + "DatasetsChanged", + "CompiledStateChanged", + "DatasetExecutionFinished", + } <= kinds finally: context.session.close() @@ -172,7 +176,9 @@ def test_read_only_dataset_initializes_in_a_mirrored_workspace(tmp_path: Path) - workspace_root=workspaces, ) try: - assert session.invoke(AddDatasets, AddDatasets.request(roots=(str(root),))).accepted + assert session.invoke( + AddDatasets, AddDatasets.request(roots=(str(root),)) + ).accepted (scope_id,) = session.dataset_scope_ids() started = session.invoke( InitializeDatasets, InitializeDatasets.request(scope_ids=(scope_id,)) diff --git a/tests/integration/test_source_schema_aggregate_publication.py b/tests/integration/test_source_schema_aggregate_publication.py index 37c4614558..5ea1410efd 100644 --- a/tests/integration/test_source_schema_aggregate_publication.py +++ b/tests/integration/test_source_schema_aggregate_publication.py @@ -20,7 +20,11 @@ InProcessCompileInspectionGateway, ) from openhcs.constants.input_source import InputSource -from openhcs.core.artifacts import ImageArtifactType, MainFlowStackOutputSpec, SpecialArtifactType +from openhcs.core.artifacts import ( + ImageArtifactType, + MainFlowStackOutputSpec, + SpecialArtifactType, +) from openhcs.core.config import ( GlobalPipelineConfig, LazyProcessingConfig, @@ -57,7 +61,8 @@ @artifact_outputs( MainFlowStackOutputSpec.output("PassThrough", ImageArtifactType), MainFlowStackOutputSpec.output( - "VolumeScalar609", SpecialArtifactType, + "VolumeScalar609", + SpecialArtifactType, materialization=MaterializationSpec(CsvOptions()), ), ) @@ -67,7 +72,9 @@ def volume_scalar_609(image: np.ndarray): @pytest.mark.parametrize("group_by", [Ungrouped, Microscopy.Channel]) -def test_real_z_aggregation_persists_scalar_and_unchanged_source_planes(tmp_path, group_by): +def test_real_z_aggregation_persists_scalar_and_unchanged_source_planes( + tmp_path, group_by +): source = tmp_path / "input" source.mkdir() pixels = np.arange(140, dtype=np.uint16).reshape(4, 5, 7) @@ -83,35 +90,58 @@ def test_real_z_aggregation_persists_scalar_and_unchanged_source_planes(tmp_path pipeline_config=PipelineConfig( dataset_source=SourceBindingsSource, source_bindings_config=LazySourceBindingsConfig( - metadata_rules=(MetadataExtractionRule( - source=MetadataSource.FILE_NAME, - pattern=r"(?PA[0-9]+)_s(?P[0-9]+)_w(?P[0-9]+)_z(?P[0-9]+)_t(?P[0-9]+)", - ),), - bindings=(NamedSourceBinding( - alias="Fixture", selector=SourceSelector(), - component_identity=(ComponentSelector(Microscopy.Channel, "1"),), - ),), + metadata_rules=( + MetadataExtractionRule( + source=MetadataSource.FILE_NAME, + pattern=r"(?PA[0-9]+)_s(?P[0-9]+)_w(?P[0-9]+)_z(?P[0-9]+)_t(?P[0-9]+)", + ), + ), + bindings=( + NamedSourceBinding( + alias="Fixture", + selector=SourceSelector(), + component_identity=( + ComponentSelector(Microscopy.Channel, "1"), + ), + ), + ), ), ), - pipeline_steps=[FunctionStep( - func=registered, name="Actual volume scalar", - processing_config=LazyProcessingConfig( - variable_components=[Microscopy.ZIndex], - group_by=group_by, - input_source=InputSource.PIPELINE_START, - ), - )], + pipeline_steps=[ + FunctionStep( + func=registered, + name="Actual volume scalar", + processing_config=LazyProcessingConfig( + variable_components=[Microscopy.ZIndex], + group_by=group_by, + input_source=InputSource.PIPELINE_START, + ), + ) + ], + ) + bundle = ( + InProcessCompileInspectionGateway() + .compile( + CompileInspectionInput( + plate=source, + pipeline_document=document, + axis_filter=("A01",), + global_pipeline_config=GlobalPipelineConfig( + num_workers=1, use_threading=True + ), + progress_queue=AgentProgressQueue(), + ) + ) + .execution_bundle ) - bundle = InProcessCompileInspectionGateway().compile(CompileInspectionInput( - plate=source, pipeline_document=document, axis_filter=("A01",), - global_pipeline_config=GlobalPipelineConfig(num_workers=1, use_threading=True), - progress_queue=AgentProgressQueue(), - )).execution_bundle context = bundle.runtime_contexts["A01"] assert tuple(context.step_plans[0].variable_components) == (Microscopy.ZIndex,) - orchestrator = PipelineOrchestrator(source, pipeline_config=document.pipeline_config).initialize() + orchestrator = PipelineOrchestrator( + source, pipeline_config=document.pipeline_config + ).initialize() outcomes = orchestrator.execute_compiled_bundle( - execution_bundle=bundle, max_workers=1, + execution_bundle=bundle, + max_workers=1, runtime_observation_mode=RuntimeObservationMode.MERGE_INTO_PARENT, progress_queue=AgentProgressQueue(), progress_context=ProgressExecutionContext( @@ -126,7 +156,9 @@ def test_real_z_aggregation_persists_scalar_and_unchanged_source_planes(tmp_path assert int(row["volume_sum"]) == int(pixels.sum()) assert int(row["voxel_count"]) == pixels.size assert "_z" not in csv_path.name - saved = tuple(Path(context.step_plans[0].artifact_images_dir).glob("*PassThrough.tif")) + saved = tuple( + Path(context.step_plans[0].artifact_images_dir).glob("*PassThrough.tif") + ) assert len(saved) == 4 for image in saved: z_index = int(image.name.split("_z", 1)[1].split("_", 1)[0]) diff --git a/tests/pyqt_gui/integration/test_end_to_end_workflow_foundation.py b/tests/pyqt_gui/integration/test_end_to_end_workflow_foundation.py index a8ed448dc0..d504043025 100644 --- a/tests/pyqt_gui/integration/test_end_to_end_workflow_foundation.py +++ b/tests/pyqt_gui/integration/test_end_to_end_workflow_foundation.py @@ -232,7 +232,9 @@ def materialization_config( ), ), setup_modifications=( - FieldModificationSpec(("step_partition_filter_config", "partition_filter"), 5), + FieldModificationSpec( + ("step_partition_filter_config", "partition_filter"), 5 + ), ), field_to_test=FieldModificationSpec( ("path_planning_config", "partition_filter"), 99 @@ -245,7 +247,9 @@ def materialization_config( ("napari_streaming_config", "partition_filter"), ("fiji_streaming_config", "partition_filter"), ), - excluded_source_paths=(("path_planning_config", "partition_filter"),), + excluded_source_paths=( + ("path_planning_config", "partition_filter"), + ), ), ), ), diff --git a/tests/pyqt_gui/test_session_gui_journey.py b/tests/pyqt_gui/test_session_gui_journey.py index e5d4ce61a4..76c6b25a92 100644 --- a/tests/pyqt_gui/test_session_gui_journey.py +++ b/tests/pyqt_gui/test_session_gui_journey.py @@ -56,11 +56,11 @@ def test_gui_journey_runs_the_same_operations_through_the_widgets( main_thread=DispatcherThread(dispatcher), ) services = GuiServiceStub() - manager = PlateManagerWidget( - services, session, gui_config=get_default_ui_config() - ) + manager = PlateManagerWidget(services, session, gui_config=get_default_ui_config()) try: - assert session.invoke(AddDatasets, AddDatasets.request(roots=(str(root),))).accepted + assert session.invoke( + AddDatasets, AddDatasets.request(roots=(str(root),)) + ).accepted (scope_id,) = session.dataset_scope_ids() session.set_pipeline_source(scope_id, source) application.processEvents() @@ -81,7 +81,8 @@ def wait_until(condition, label) -> None: (CompileDatasets, lambda row: row.compiled and not row.compile_pending), ( RunDatasets, - lambda row: row.terminal_status is not None and not row.execution_active, + lambda row: row.terminal_status is not None + and not row.execution_active, ), ): if operation is CompileDatasets: diff --git a/tests/unit/agent/test_agent_services.py b/tests/unit/agent/test_agent_services.py index 36b94e72c5..43901a6e19 100644 --- a/tests/unit/agent/test_agent_services.py +++ b/tests/unit/agent/test_agent_services.py @@ -51,7 +51,9 @@ ) from openhcs.agent.path_policy import AgentPathPolicy, AgentPathPolicyError from openhcs.agent.services import function_catalog_service as function_catalog_module -from openhcs.agent.services import artifact_plan_inspection_service as artifact_plan_module +from openhcs.agent.services import ( + artifact_plan_inspection_service as artifact_plan_module, +) from openhcs.agent.services import viewer_window_service as viewer_window_service_module from openhcs.agent.services.config_service import ConfigService from openhcs.agent.services.artifact_plan_inspection_service import ( @@ -102,7 +104,10 @@ from openhcs.core.pipeline.function_contracts import artifact_inputs, artifact_outputs from openhcs.core.pipeline_document import PipelineDocumentCodec from openhcs.core.progress import ( - ProgressExecutionContext, ProgressIdentity, ProgressPhase, ProgressStatus, + ProgressExecutionContext, + ProgressIdentity, + ProgressPhase, + ProgressStatus, ) from openhcs.core.source_bindings import ( LazySourceBindingsConfig, @@ -312,9 +317,13 @@ def __init__(self) -> None: def emit_progress(request): request.progress_queue.put( ProgressExecutionContext("inspection", str(request.plate)).event( - axis_id="A01", step_name="compilation", - phase=ProgressPhase.COMPILE, status=ProgressStatus.RUNNING, - percent=0.0, completed=0, total=1, + axis_id="A01", + step_name="compilation", + phase=ProgressPhase.COMPILE, + status=ProgressStatus.RUNNING, + percent=0.0, + completed=0, + total=1, ) ) @@ -1284,7 +1293,8 @@ def test_function_catalog_resolves_detail_by_callable_import_path(monkeypatch): selected = _Metadata(tags=[]) unrelated = _Metadata.from_function(sample_large_signature_function, "Unrelated") monkeypatch.setattr( - catalog, "_all_metadata", + catalog, + "_all_metadata", lambda **_kwargs: { "test:aaa_unrelated": unrelated, "test:sample_processing_function": selected, @@ -1609,7 +1619,10 @@ def __repr__(self): == repeated.items[0].signature == "sample_processing_function(sigma=1.0)" ) - assert catalog.search(query="sample", compact_signatures=True).items[0] is first.items[0] + assert ( + catalog.search(query="sample", compact_signatures=True).items[0] + is first.items[0] + ) assert ( catalog.get("test:sample_processing_function").entry.signature == "sample_processing_function(sigma=2.0)" @@ -2912,9 +2925,8 @@ def test_artifact_plan_inspection_inspects_pipeline_source_artifact_plan( plate_path=str(tmp_path), pipeline_source=_pipeline_document_source(), global_config_id=None, - axis_filter=("A01", - ) - ), + axis_filter=("A01",), + ), ) assert compile_gateway.requests[0].plate == tmp_path.resolve() @@ -2929,7 +2941,10 @@ def test_artifact_plan_inspection_inspects_pipeline_source_artifact_plan( assert events[0].phase is ProgressPhase.COMPILE assert events[0].status is ProgressStatus.RUNNING assert events[0].identity == ProgressIdentity( - "inspection", str(tmp_path.resolve()), "A01", "compilation", + "inspection", + str(tmp_path.resolve()), + "A01", + "compilation", ) assert inspection.errors == () assert inspection.axis_count == 1 @@ -2949,9 +2964,9 @@ def test_artifact_plan_inspection_inspects_pipeline_source_artifact_plan( ) assert inspection.steps[0].viewer_streaming[0].viewer_type is ViewerType.NAPARI assert inspection.steps[0].viewer_streaming[0].effective_config["enabled"] is True - assert inspection.steps[0].viewer_streaming[0].effective_config["partition_filter"] == [ - "A01" - ] + assert inspection.steps[0].viewer_streaming[0].effective_config[ + "partition_filter" + ] == ["A01"] assert inspection.steps[0].artifact_inputs[0].name == "positions" assert inspection.steps[0].artifact_inputs[0].kind == "special" assert inspection.steps[0].artifact_inputs[0].source_step_id == 0 @@ -3049,12 +3064,12 @@ def test_compile_inspection_rejects_read_only_plate_before_initialization( with pytest.raises(AgentPathPolicyError, match="Writable path is outside"): service.inspect( - PipelineSourceArtifactPlanInspectionRequest( - plate_path=str(plate), - pipeline_source=_pipeline_document_source(), - global_config_id=None, + PipelineSourceArtifactPlanInspectionRequest( + plate_path=str(plate), + pipeline_source=_pipeline_document_source(), + global_config_id=None, + ) ) - ) assert gateway.requests == [] assert {path.name: path.read_bytes() for path in plate.iterdir()} == before diff --git a/tests/unit/agent/test_authoring_roundtrip_boundaries.py b/tests/unit/agent/test_authoring_roundtrip_boundaries.py index 6a548fe6ad..97aa71635d 100644 --- a/tests/unit/agent/test_authoring_roundtrip_boundaries.py +++ b/tests/unit/agent/test_authoring_roundtrip_boundaries.py @@ -115,9 +115,7 @@ def test_recursive_json_alias_keeps_declaration_namespace_in_new_descendant(): @pytest.fixture def gamma_metadata(monkeypatch): registry = SkimageRegistry() - wrapped = registry.reconstruct_cached_callable( - adjust_gamma, FlexibleContract - ) + wrapped = registry.reconstruct_cached_callable(adjust_gamma, FlexibleContract) metadata = FunctionMetadata( name="exposure.adjust_gamma", func=wrapped, diff --git a/tests/unit/agent/test_callable_artifact_reference.py b/tests/unit/agent/test_callable_artifact_reference.py index 668d99e5d9..6ae19883a4 100644 --- a/tests/unit/agent/test_callable_artifact_reference.py +++ b/tests/unit/agent/test_callable_artifact_reference.py @@ -86,7 +86,9 @@ def reference_namespace(): sys.modules[module.__name__] = module try: for block in REFERENCE_BLOCKS: - exec(compile(_reference_block(block), DOCUMENT_PATH, "exec"), module.__dict__) + exec( + compile(_reference_block(block), DOCUMENT_PATH, "exec"), module.__dict__ + ) yield module.__dict__ finally: sys.modules.pop(module.__name__, None) @@ -186,15 +188,18 @@ def test_diagnostic_reference_uses_one_canonical_slot_and_exact_trailing_artifac returned = function(pixels) assert len(returned) == 3 assert contract.canonical_return_output_specs.names() == ( - "fixture_image", "fixture_diagnostic", + "fixture_image", + "fixture_diagnostic", ) assert contract.trailing_return_output_specs.names() == ( - "fixture_labels", "fixture_object_rows", + "fixture_labels", + "fixture_object_rows", ) matched = contract.resolve_returned_output(returned) np.testing.assert_array_equal(matched[namespace["FIXTURE_IMAGE"].ref()], pixels) np.testing.assert_array_equal( - matched[namespace["FIXTURE_DIAGNOSTIC"].ref()], pixels > 0, + matched[namespace["FIXTURE_DIAGNOSTIC"].ref()], + pixels > 0, ) assert matched[namespace["FIXTURE_ROWS"].ref()].row_mappings() == ( {"slice_index": 0, "object_label": 7, "pixel_count": 16}, @@ -204,7 +209,8 @@ def test_diagnostic_reference_uses_one_canonical_slot_and_exact_trailing_artifac def test_points_declaration_materializes_fractional_geometry_and_full_source_domain( - reference_namespace, tmp_path, + reference_namespace, + tmp_path, ): from openhcs.core.roi_point_metadata import ROIFractionalZ from openhcs.core.roi_source_metadata import ROIArchiveSourceMetadata @@ -216,18 +222,23 @@ def test_points_declaration_materializes_fractional_geometry_and_full_source_dom spec = namespace["CENTRE_ROWS"] labels = namespace["CENTRE_LABELS"] plan = ArtifactOutputPlan( - name=spec.name, path="/synthetic/centre_rows", artifact_type=spec.artifact_type, + name=spec.name, + path="/synthetic/centre_rows", + artifact_type=spec.artifact_type, relations=spec.relations, ) assert plan.object_subject_binding().source == labels.ref() row = namespace["CentreRow"](7, 1.5, 1.25, 2.5) carrier = namespace["DataclassMeasurementColumnarRows"]( - (row,), row_type=namespace["CentreRow"], + (row,), + row_type=namespace["CentreRow"], ) # Direct materialization supplies the same nominal context normally owned # by runtime contextualization, without inventing an archive/CSV writer. table = MeasurementTable( - name=spec.name, rows=carrier, subject=plan.measurement_subject(), + name=spec.name, + rows=carrier, + subject=plan.measurement_subject(), measurement_feature_owner=namespace["CentreFeatureOwner"], source_path="/synthetic/volume.tif", source_image_provenance_planes=SourceImageProvenancePlanes.from_components( @@ -239,16 +250,23 @@ def test_points_declaration_materializes_fractional_geometry_and_full_source_dom ), ) materialize( - spec.materialization, data=table, path=str(tmp_path / "centres"), + spec.materialization, + data=table, + path=str(tmp_path / "centres"), filemanager=FileManager({"disk": DiskStorageBackend()}), - backends=["disk"], backend_kwargs={}, + backends=["disk"], + backend_kwargs={}, ) csv_options = next( - option for option in spec.materialization.outputs + option + for option in spec.materialization.outputs if isinstance(option, namespace["CsvOptions"]) ) - assert (tmp_path / ("centres" + csv_options.filename_suffix)).read_text().splitlines() == [ - "object_label,center_z,center_y,center_x", "7,1.5,1.25,2.5", + assert ( + tmp_path / ("centres" + csv_options.filename_suffix) + ).read_text().splitlines() == [ + "object_label,center_z,center_y,center_x", + "7,1.5,1.25,2.5", ] rois = load_rois_from_zip(tmp_path / "centres_points.roi.zip") assert rois[0].shapes == [PointShape(y=1.25, x=2.5)] @@ -271,7 +289,8 @@ def test_input_reference_compiles_nominal_binding_and_repairs_wrong_annotation( assert contract.artifact_input_parameter_names == ("objects",) contract.validate_artifact_input_parameter_bindings() plan = ArtifactInputPlan( - declaration.name, "/synthetic/fixture-labels.pkl", + declaration.name, + "/synthetic/fixture-labels.pkl", artifact_type=declaration.artifact_type, ) compiled = compile_function_pattern(function, {plan.ref(): plan}, {}) @@ -289,7 +308,9 @@ def test_input_reference_compiles_nominal_binding_and_repairs_wrong_annotation( original_annotation = raw.__annotations__["objects"] try: raw.__annotations__["objects"] = np.ndarray - with pytest.raises(TypeError, match="does not accept object_labels artifact payloads"): + with pytest.raises( + TypeError, match="does not accept object_labels artifact payloads" + ): compile_function_pattern(function, {plan.ref(): plan}, {}) finally: raw.__annotations__["objects"] = original_annotation @@ -328,16 +349,21 @@ def fixture_storage(_name, *, create): def test_input_reference_prepares_in_real_custom_namespace(tmp_path, monkeypatch): monkeypatch.setattr( - custom_manager, "get_data_file_path", + custom_manager, + "get_data_file_path", lambda _name, *, create: tmp_path / "custom_functions", ) manager = custom_manager.CustomFunctionManager(create_storage=False) - metadata = manager._prepare_source(_reference_block("callable-artifact-input-reference")) + metadata = manager._prepare_source( + _reference_block("callable-artifact-input-reference") + ) contract = CallableContract.from_callable(metadata.func) contract.validate_artifact_input_parameter_bindings() assert contract.artifact_inputs.specs[0].parameter_name == "objects" labels = ObjectLabelPayload( - variant_data=ObjectLabelVariantData(labels=np.asarray([[0, 7]], dtype=np.int32)), + variant_data=ObjectLabelVariantData( + labels=np.asarray([[0, 7]], dtype=np.int32) + ), ) image = np.asarray([[3, 5]], dtype=np.uint16) np.testing.assert_array_equal(metadata.func(image, objects=labels), [[0, 5]]) @@ -345,29 +371,42 @@ def test_input_reference_prepares_in_real_custom_namespace(tmp_path, monkeypatch def test_plate_reference_prepares_compiles_and_runs_in_original_parent( - reference_namespace, tmp_path, monkeypatch, + reference_namespace, + tmp_path, + monkeypatch, ): from openhcs.core.callable_contract import FunctionStepExecutionScope - from openhcs.core.pipeline.funcstep_contract_validator import FuncStepContractValidator + from openhcs.core.pipeline.funcstep_contract_validator import ( + FuncStepContractValidator, + ) from openhcs.core.artifacts import SpecialArtifactType - from openhcs.core.measurement_row_materialization import DataclassMeasurementColumnarRows + from openhcs.core.measurement_row_materialization import ( + DataclassMeasurementColumnarRows, + ) from openhcs.core.runtime_measurements import MeasurementTable from openhcs.core.runtime_stores import RuntimeArtifactBatch monkeypatch.setattr( - custom_manager, "get_data_file_path", + custom_manager, + "get_data_file_path", lambda _name, *, create: tmp_path / "custom_functions", ) manager = custom_manager.CustomFunctionManager(create_storage=False) - metadata = manager._prepare_source(_reference_block("callable-artifact-plate-reference")) + metadata = manager._prepare_source( + _reference_block("callable-artifact-plate-reference") + ) contract = CallableContract.from_callable(metadata.func) - FuncStepContractValidator.validate_dataset_callable_contracts((contract,), metadata.name) + FuncStepContractValidator.validate_dataset_callable_contracts( + (contract,), metadata.name + ) assert contract.execution_scope is FunctionStepExecutionScope.PLATE assert contract.runtime_bound_parameter_types == (RuntimeArtifactBatch,) assert contract.input_memory_type is contract.processing_contract is None assert contract.artifact_input_parameter_names == () # Reuse the existing plate execution fixtures, not a replacement dispatcher. - plate = runpy.run_path(str(ROOT / "tests/unit/test_function_step_execution_scope.py")) + plate = runpy.run_path( + str(ROOT / "tests/unit/test_function_step_execution_scope.py") + ) source = contract.artifact_inputs.specs[0] output = contract.artifact_outputs.specs[0] contexts = {} @@ -377,13 +416,21 @@ def test_plate_reference_prepares_compiles_and_runs_in_original_parent( ): path = f"/memory/{axis_id}/rows" plan = plate["_plate_step_plan"]( - axis_id=axis_id, step_index=0, func=metadata.func, - artifact_inputs=(ArtifactInputPlan( - source.name, path, artifact_type=source.artifact_type, - ),), + axis_id=axis_id, + step_index=0, + func=metadata.func, + artifact_inputs=( + ArtifactInputPlan( + source.name, + path, + artifact_type=source.artifact_type, + ), + ), artifact_output=ArtifactOutputPlan( - output.name, "/memory/plate/summary", - artifact_type=output.artifact_type, relations=output.relations, + output.name, + "/memory/plate/summary", + artifact_type=output.artifact_type, + relations=output.relations, ), metadata_writer=axis_id == "axis-a", ) @@ -393,12 +440,16 @@ def test_plate_reference_prepares_compiles_and_runs_in_original_parent( assert isinstance(record.data, MeasurementTable) record.data.rows = reference_namespace["inspect_label_fixture"](fixture)[2] plate["_record_measurements"]( - context, name="unrelated_rows", path=f"/memory/{axis_id}/unrelated", count=99, + context, + name="unrelated_rows", + path=f"/memory/{axis_id}/unrelated", + count=99, ) contexts[axis_id] = context plate["_execute_plate_steps"](contexts) summaries = tuple( - record for context in contexts.values() + record + for context in contexts.values() for record in context.runtime_value_store.values() if record.key.name == output.name ) @@ -411,22 +462,30 @@ def test_plate_reference_prepares_compiles_and_runs_in_original_parent( {"axis_id": "axis-b", "record_count": 1, "measurement_row_count": 0}, ) csv_path = materialize( - output.materialization, data=table, path=str(tmp_path / "summary"), + output.materialization, + data=table, + path=str(tmp_path / "summary"), filemanager=FileManager({"disk": DiskStorageBackend()}), - backends=["disk"], backend_kwargs={}, + backends=["disk"], + backend_kwargs={}, ) assert Path(csv_path).read_text().splitlines() == [ - "axis_id,record_count,measurement_row_count", "axis-a,1,2", "axis-b,1,0", + "axis_id,record_count,measurement_row_count", + "axis-a,1,2", + "axis-b,1,0", ] assert not manager.storage_dir.exists() @pytest.mark.parametrize("plane_count", (None, 1, 2)) def test_reference_retains_plane_domain_until_raw_numpy_invocation( - tmp_path, monkeypatch, plane_count, + tmp_path, + monkeypatch, + plane_count, ): monkeypatch.setattr( - custom_manager, "get_data_file_path", + custom_manager, + "get_data_file_path", lambda _name, *, create: tmp_path / "custom_functions", ) manager = custom_manager.CustomFunctionManager(create_storage=False) @@ -452,12 +511,18 @@ def test_reference_retains_plane_domain_until_raw_numpy_invocation( for plane in range(1 if plane_count is None else plane_count) ) projection = ( - None if plane_count is None else RuntimePlaneAxisValueProjection( - axis=RuntimePlaneAxis.RUNTIME_SLICE, source_aliases=(), - plane_index=None, axis_size=plane_count, + None + if plane_count is None + else RuntimePlaneAxisValueProjection( + axis=RuntimePlaneAxis.RUNTIME_SLICE, + source_aliases=(), + plane_index=None, + axis_size=plane_count, ) ) - label_payload = ObjectLabelsArtifactType.contextualize_output(stack, labels, None, projection) + label_payload = ObjectLabelsArtifactType.contextualize_output( + stack, labels, None, projection + ) assert isinstance(label_payload, ObjectLabelPayload) np.testing.assert_array_equal(label_payload.labels, stack.data) assert label_payload.plane_axis is ( @@ -467,7 +532,8 @@ def test_reference_retains_plane_domain_until_raw_numpy_invocation( assert label_payload.source_spatial_domain.source_shape_yx == (8, 8) assert label_payload.domain.scope is ( ObjectLabelDomainScope.PAYLOAD - if plane_count is None else ObjectLabelDomainScope.PLANE + if plane_count is None + else ObjectLabelDomainScope.PLANE ) if plane_count is not None: assert label_payload.domain.declared_object_id_domains == ((1,),) * plane_count @@ -475,7 +541,9 @@ def test_reference_retains_plane_domain_until_raw_numpy_invocation( plane_axis=RuntimePlaneAxis.RUNTIME_SLICE, ).payload_with(stack.data[:, :-1, :]) with pytest.raises(ValueError, match="Object-label spatial shape"): - ObjectLabelsArtifactType.contextualize_output(stack, wrong_spatial_shape, None, projection) + ObjectLabelsArtifactType.contextualize_output( + stack, wrong_spatial_shape, None, projection + ) assert not manager.storage_dir.exists() @@ -487,7 +555,8 @@ def test_reference_retains_plane_domain_until_raw_numpy_invocation( ), ) def test_reference_binds_all_output_lineage_to_current_image( - reference_namespace, source, + reference_namespace, + source, ): invocation = next( normalize_function_pattern( @@ -508,11 +577,10 @@ def test_reference_binds_all_output_lineage_to_current_image( assert output.source_stack_scope_sources() == (source.ref(),) assert contract.canonical_return_output_specs.names() == ("fixture_image",) assert contract.trailing_return_output_specs.names() == ( - "fixture_labels", "fixture_object_rows", - ) - rows = contract.artifact_outputs.by_ref( - reference_namespace["FIXTURE_ROWS"].ref() + "fixture_labels", + "fixture_object_rows", ) + rows = contract.artifact_outputs.by_ref(reference_namespace["FIXTURE_ROWS"].ref()) assert rows is not None subject = ArtifactOutputPlan( name=rows.name, @@ -574,7 +642,10 @@ def test_packaged_knowledge_projection_retains_executable_reference(tmp_path): ) assert "def inspect_label_fixture(" in document.content assert "from openhcs.core.memory import numpy" in document.content - for query in ("PointROIOptions fractional centres", "aligned diagnostic image returns"): + for query in ( + "PointROIOptions fractional centres", + "aligned diagnostic image returns", + ): hits = service.search(KnowledgeBaseSearchRequest(query=query, limit=10)) assert DOCUMENT_ID in {hit.document.document_id for hit in hits.hits} for block in REFERENCE_BLOCKS: @@ -583,7 +654,8 @@ def test_packaged_knowledge_projection_retains_executable_reference(tmp_path): assert line in document.content route = service.get_document( KnowledgeBaseDocumentRequest.from_fields( - document_id="openhcs_custom_function_workflow", max_chars=30_000, + document_id="openhcs_custom_function_workflow", + max_chars=30_000, ) ) source_path = "packaging/codex/openhcs/skills/use-openhcs/references/custom-function-authoring.md" @@ -593,14 +665,20 @@ def test_packaged_knowledge_projection_retains_executable_reference(tmp_path): "materialize-typed-3d-centres-as-feature-bearing-points", "return-diagnostic-images-without-flattening-the-abi", ): - selected = service.get_document(KnowledgeBaseDocumentRequest.from_fields( - document_id=DOCUMENT_ID, section_id=section_id, max_chars=10_000, - )) + selected = service.get_document( + KnowledgeBaseDocumentRequest.from_fields( + document_id=DOCUMENT_ID, + section_id=section_id, + max_chars=10_000, + ) + ) assert not selected.errors and not selected.truncated assert selected.selected_section_id == section_id original = KnowledgeBaseService(repo_root=ROOT).get_document( KnowledgeBaseDocumentRequest.from_fields( - document_id=DOCUMENT_ID, section_id=section_id, max_chars=10_000, + document_id=DOCUMENT_ID, + section_id=section_id, + max_chars=10_000, ) ) assert selected.content == original.content diff --git a/tests/unit/agent/test_capabilities.py b/tests/unit/agent/test_capabilities.py index b6db329076..6cdc70a27a 100644 --- a/tests/unit/agent/test_capabilities.py +++ b/tests/unit/agent/test_capabilities.py @@ -121,8 +121,7 @@ def test_capability_transport_defaults_to_local_stdio(): def test_capability_exposition_owns_its_json_projection(): capability = next( - capability - for capability in get_capability_registry().capabilities + capability for capability in get_capability_registry().capabilities ) exposition = capability.exposition @@ -294,9 +293,15 @@ def test_custom_function_registration_is_a_desktop_authoring_mutation(): } registration = full_capabilities["openhcs_register_custom_function"] - assert registration.exposition.workflow_group is CapabilityWorkflowGroup.FUNCTION_AUTHORING + assert ( + registration.exposition.workflow_group + is CapabilityWorkflowGroup.FUNCTION_AUTHORING + ) assert registration.exposition.workflow_stage is CapabilityWorkflowStage.AUTHORING - assert registration.exposition.target_context is CapabilityTargetContext.FUNCTION_REGISTRY + assert ( + registration.exposition.target_context + is CapabilityTargetContext.FUNCTION_REGISTRY + ) assert registration.exposition.visibility is CapabilityVisibility.STANDARD assert registration.exposition.role is CapabilityRole.PRIMARY assert registration.mutating is True @@ -344,7 +349,12 @@ def _declaration_error(**attributes) -> Exception: type( "RejectedDeclaration", (UiBridgeCapability,), - {"name": name, "title": "Rejected", "description": "", "service": "test"} + { + "name": name, + "title": "Rejected", + "description": "", + "service": "test", + } | attributes, ) finally: @@ -407,9 +417,9 @@ def test_similar_mcp_tool_names_are_disambiguated_by_target_context_and_role(): assert capabilities["openhcs_sample_plate_image"].exposition.target_context is ( CapabilityTargetContext.PLATE_PATH ) - assert capabilities["openhcs_ui_sample_selected_plate_image"].exposition.target_context is ( - CapabilityTargetContext.UI_SELECTED_PLATE - ) + assert capabilities[ + "openhcs_ui_sample_selected_plate_image" + ].exposition.target_context is (CapabilityTargetContext.UI_SELECTED_PLATE) assert capabilities["openhcs_ui_sample_selected_plate_image"].exposition.role is ( CapabilityRole.MODE_VARIANT ) @@ -420,20 +430,25 @@ def test_similar_mcp_tool_names_are_disambiguated_by_target_context_and_role(): CapabilityTargetContext.HEADLESS_SESSION ) assert ( - capabilities["openhcs_get_runtime_server_execution_status"].exposition.target_context + capabilities[ + "openhcs_get_runtime_server_execution_status" + ].exposition.target_context is CapabilityTargetContext.RUNTIME_SERVER ) - assert capabilities["openhcs_ui_get_operation_status"].exposition.target_context is ( - CapabilityTargetContext.UI_BRIDGE - ) - assert capabilities["openhcs_ui_wait_for_operation_receipt"].exposition.target_context is ( - CapabilityTargetContext.UI_BRIDGE - ) + assert capabilities[ + "openhcs_ui_get_operation_status" + ].exposition.target_context is (CapabilityTargetContext.UI_BRIDGE) + assert capabilities[ + "openhcs_ui_wait_for_operation_receipt" + ].exposition.target_context is (CapabilityTargetContext.UI_BRIDGE) assert ( capabilities["openhcs_ui_wait_for_operation_receipt"].exposition.role is CapabilityRole.PRIMARY ) - assert capabilities["openhcs_ui_invoke_action"].exposition.role is CapabilityRole.PRIMARY + assert ( + capabilities["openhcs_ui_invoke_action"].exposition.role + is CapabilityRole.PRIMARY + ) assert capabilities["openhcs_ui_invoke_widget_action"].exposition.role is ( CapabilityRole.FALLBACK ) @@ -446,9 +461,9 @@ def test_similar_mcp_tool_names_are_disambiguated_by_target_context_and_role(): assert capabilities["openhcs_validate_pipeline"].exposition.workflow_stage is ( CapabilityWorkflowStage.VALIDATION ) - assert capabilities["openhcs_ui_mutate_object_state_field"].exposition.visibility is ( - CapabilityVisibility.EXPERT - ) + assert capabilities[ + "openhcs_ui_mutate_object_state_field" + ].exposition.visibility is (CapabilityVisibility.EXPERT) def test_ui_bridge_capabilities_declare_runtime_security_and_data_exposure(): @@ -546,7 +561,9 @@ def invoke(self, operation, request=None, connection=None): connection = object() invocation = agent_capabilities.ui_bridge_status.invocation - context = SimpleNamespace(ui_bridge_service=RecordingUiBridgeService(gateway=object())) + context = SimpleNamespace( + ui_bridge_service=RecordingUiBridgeService(gateway=object()) + ) assert invocation.execute(context, connection) == "served" assert seen == [(invocation.operation, None, connection)] diff --git a/tests/unit/agent/test_cold_inspection_feedback.py b/tests/unit/agent/test_cold_inspection_feedback.py index 86e0b113b5..6c70b9fbfc 100644 --- a/tests/unit/agent/test_cold_inspection_feedback.py +++ b/tests/unit/agent/test_cold_inspection_feedback.py @@ -8,17 +8,24 @@ from zmqruntime.startup import EndpointStartupStatus from openhcs.agent.capabilities import ( - MainThreadProgressCapability, QueryPlateFilesCapability, SamplePlateImageCapability, + MainThreadProgressCapability, + QueryPlateFilesCapability, + SamplePlateImageCapability, InspectPipelineSourceArtifactPlanCapability, ) from openhcs.agent.path_policy import AgentPathPolicy from openhcs.agent.services.config_service import ConfigService from openhcs.agent.services.artifact_plan_inspection_service import ( - AgentProgressQueue, CompileInspectionGatewayABC, ArtifactPlanInspectionService, + AgentProgressQueue, + CompileInspectionGatewayABC, + ArtifactPlanInspectionService, ) from openhcs.agent.services.plate_inspection_service import PlateInspectionService from openhcs.core.progress import ( - ProgressEvent, ProgressExecutionContext, ProgressPhase, ProgressStatus, + ProgressEvent, + ProgressExecutionContext, + ProgressPhase, + ProgressStatus, ) from openhcs.mcp.execution import McpTransportExecutor from openhcs.mcp.server import build_server @@ -27,7 +34,9 @@ from tests.unit.agent.test_plate_inspection_service import ImageXpressPlateFixture -@pytest.mark.parametrize("declaration", (QueryPlateFilesCapability, SamplePlateImageCapability)) +@pytest.mark.parametrize( + "declaration", (QueryPlateFilesCapability, SamplePlateImageCapability) +) def test_physical_reader_declarations_inherit_original_affine_progress(declaration): assert issubclass(declaration, MainThreadProgressCapability) assert declaration.progress_heartbeat_seconds == 1.0 @@ -38,9 +47,13 @@ def test_physical_reader_declarations_inherit_original_affine_progress(declarati def test_original_progress_codec_is_the_only_event_store_and_rejects_bad_enum(): queue = AgentProgressQueue() event = ProgressExecutionContext("inspection", "/source").event( - axis_id="A01", step_name="new-declared-step", - phase=ProgressPhase.COMPILE, status=ProgressStatus.RUNNING, percent=25, - completed=1, total=4, + axis_id="A01", + step_name="new-declared-step", + phase=ProgressPhase.COMPILE, + status=ProgressStatus.RUNNING, + percent=25, + completed=1, + total=4, ) statuses = [] with EndpointStartupStatus.callback_scope(statuses.append): @@ -57,7 +70,9 @@ def test_original_progress_codec_is_the_only_event_store_and_rejects_bad_enum(): @pytest.mark.parametrize("fails", (False, True)) -def test_new_gateway_hook_cooperates_in_both_mros_before_terminal_result(tmp_path, fails): +def test_new_gateway_hook_cooperates_in_both_mros_before_terminal_result( + tmp_path, fails +): """Independent audit capability needs no edits to service or MCP consumers.""" identity = ContextVar("cold-feedback-request", default="outside") calls, messages = [], [] @@ -77,7 +92,9 @@ def _compile(self, request): assert threading.current_thread() is threading.main_thread() compile_entered.set() self.emit_progress(request) - assert release.wait(3), "Actual stage relay was not delivered during held work" + assert release.wait( + 3 + ), "Actual stage relay was not delivered during held work" if fails: raise error return super()._compile(request) @@ -107,30 +124,43 @@ async def report_progress(self, progress, total=None, message=None): gateway = gateway_type() service = ArtifactPlanInspectionService( path_policy=AgentPathPolicy.with_roots( - readable_roots=(tmp_path,), writable_roots=(tmp_path,), + readable_roots=(tmp_path,), + writable_roots=(tmp_path,), ), - config_service=config, compile_inspection_gateway=gateway, + config_service=config, + compile_inspection_gateway=gateway, + ) + built = build_server( + SimpleNamespace(artifact_plan_service=service), + main_thread_dispatcher=executor.dispatcher, + ) + tool = built._tool_manager.get_tool( + InspectPipelineSourceArtifactPlanCapability.name ) - built = build_server(SimpleNamespace(artifact_plan_service=service), - main_thread_dispatcher=executor.dispatcher) - tool = built._tool_manager.get_tool(InspectPipelineSourceArtifactPlanCapability.name) async def exercise(): token = identity.set("original-request") try: - return await tool.fn(mcp_context=StageContext(), plate_path=str(tmp_path), - pipeline_source="pipeline_steps = []") + return await tool.fn( + mcp_context=StageContext(), + plate_path=str(tmp_path), + pipeline_source="pipeline_steps = []", + ) finally: identity.reset(token) before = len(messages) result = executor.run(exercise) observed = messages[before:] - assert any("Preparing source inspection compiler" in item for item in observed) + assert any( + "Preparing source inspection compiler" in item for item in observed + ) if fails: assert result["errors"][0]["exception_type"] == "ValueError" assert result["errors"][0]["message"] == str(error) - assert any("Source inspection compiler failed" in item for item in observed) + assert any( + "Source inspection compiler failed" in item for item in observed + ) assert not any("projection completed" in item for item in observed) else: assert result["errors"] == [] @@ -145,9 +175,12 @@ async def exercise(): def test_real_inventory_binding_relays_shared_reader_preparation(tmp_path): plate = ImageXpressPlateFixture.write(tmp_path) - service = PlateInspectionService(AgentPathPolicy.with_roots( - readable_roots=(tmp_path,), writable_roots=(), - )) + service = PlateInspectionService( + AgentPathPolicy.with_roots( + readable_roots=(tmp_path,), + writable_roots=(), + ) + ) messages = [] class StageContext: @@ -158,13 +191,19 @@ async def report_progress(self, progress, total=None, message=None): messages.append(message) executor = McpTransportExecutor() - built = build_server(SimpleNamespace(plate_inspection_service=service), - main_thread_dispatcher=executor.dispatcher) + built = build_server( + SimpleNamespace(plate_inspection_service=service), + main_thread_dispatcher=executor.dispatcher, + ) tool = built._tool_manager.get_tool(QueryPlateFilesCapability.name) async def exercise(): - return await tool.fn(mcp_context=StageContext(), plate_path=str(plate), - source_format="imagexpress", include_previews=False) + return await tool.fn( + mcp_context=StageContext(), + plate_path=str(plate), + source_format="imagexpress", + include_previews=False, + ) try: result = executor.run(exercise) diff --git a/tests/unit/agent/test_compile_inspection_declared_callable.py b/tests/unit/agent/test_compile_inspection_declared_callable.py index 24c2847bfd..bd6da4ab18 100644 --- a/tests/unit/agent/test_compile_inspection_declared_callable.py +++ b/tests/unit/agent/test_compile_inspection_declared_callable.py @@ -16,34 +16,61 @@ from openhcs.processing.backends.lib_registry.registry_service import RegistryService -def test_real_declared_callable_compiles_without_global_catalog(monkeypatch, tmp_path: Path): +def test_real_declared_callable_compiles_without_global_catalog( + monkeypatch, tmp_path: Path +): def forbidden_catalog(*args, **kwargs): raise AssertionError("A declared callable must not prepare unrelated catalogs") - monkeypatch.setattr(RegistryService, "get_all_functions_with_metadata", forbidden_catalog) + monkeypatch.setattr( + RegistryService, "get_all_functions_with_metadata", forbidden_catalog + ) plate = tmp_path / "plate" SyntheticMicroscopyGenerator( - output_dir=str(plate), grid_size=(1, 1), tile_size=(64, 64), - wavelengths=1, z_stack_levels=1, wells=["A01"], num_cells=3, - random_seed=123, include_all_components=True, + output_dir=str(plate), + grid_size=(1, 1), + tile_size=(64, 64), + wavelengths=1, + z_stack_levels=1, + wells=["A01"], + num_cells=3, + random_seed=123, + include_all_components=True, ).generate_dataset() document = PipelineDocumentCodec.from_values( pipeline_config=PipelineConfig(num_workers=1), - pipeline_steps=[FunctionStep(func=(reducenoise, { - "patch_size": 3, "patch_distance": 3, "cutoff_distance": 0.1, - }), name="DeclaredNlm")], + pipeline_steps=[ + FunctionStep( + func=( + reducenoise, + { + "patch_size": 3, + "patch_distance": 3, + "cutoff_distance": 0.1, + }, + ), + name="DeclaredNlm", + ) + ], + ) + compiled = InProcessCompileInspectionGateway().compile( + CompileInspectionInput( + plate=plate, + pipeline_document=document, + axis_filter=(), + global_pipeline_config=GlobalPipelineConfig(num_workers=1), + progress_queue=AgentProgressQueue(), + ) ) - compiled = InProcessCompileInspectionGateway().compile(CompileInspectionInput( - plate=plate, pipeline_document=document, axis_filter=(), - global_pipeline_config=GlobalPipelineConfig(num_workers=1), - progress_queue=AgentProgressQueue(), - )) assert tuple(compiled.execution_bundle.runtime_contexts) == ("A01",) context = compiled.execution_bundle.runtime_contexts["A01"] assert len(context.step_plans) == 1 assert next(iter(context.step_plans.values())).step_name == "DeclaredNlm" inspection = artifact_plan_inspection_from_compilation( - plate_path=str(plate), axis_filter=(), compilation=compiled, - progress_event_count=0, warnings=(), + plate_path=str(plate), + axis_filter=(), + compilation=compiled, + progress_event_count=0, + warnings=(), ) assert inspection.source_workspace.file_count == 1 diff --git a/tests/unit/agent/test_compile_selector_authoring.py b/tests/unit/agent/test_compile_selector_authoring.py index a937944aa3..54ce7a017c 100644 --- a/tests/unit/agent/test_compile_selector_authoring.py +++ b/tests/unit/agent/test_compile_selector_authoring.py @@ -302,9 +302,7 @@ class NewAuthoringModule(CellProfilerModule, ExtraOutputSettings): assert len(kwargs) == 2 service, ref = draft(new_authoring_callable, kwargs) assert service.validate(ref).valid - restored = PipelineDocumentCodec.from_source( - service.render_source(ref).source - ) + restored = PipelineDocumentCodec.from_source(service.render_source(ref).source) invocation = next( normalize_function_pattern(restored.pipeline_steps[0].func).iter_items() ) diff --git a/tests/unit/agent/test_declared_callable_documentation.py b/tests/unit/agent/test_declared_callable_documentation.py index e21dd0afc5..f246e9ee5b 100644 --- a/tests/unit/agent/test_declared_callable_documentation.py +++ b/tests/unit/agent/test_declared_callable_documentation.py @@ -57,7 +57,9 @@ def test_installed_witness_declaration_doc_and_parameter_contract_agree(monkeypa @pytest.mark.parametrize("contract", (Pure2DContract, Pure3DContract)) -def test_new_declaration_preserves_authored_prose_without_consumer_edits(monkeypatch, contract): +def test_new_declaration_preserves_authored_prose_without_consumer_edits( + monkeypatch, contract +): @numpy_contract(contract=contract) def independent_doc_probe(image: np.ndarray, *, offset: float = 0.75) -> np.ndarray: """Authored plane/volume operation. @@ -73,35 +75,50 @@ def independent_doc_probe(image: np.ndarray, *, offset: float = 0.75) -> np.ndar """ return image + offset - catalog, metadata = _catalog(monkeypatch, project_custom_function(independent_doc_probe)) + catalog, metadata = _catalog( + monkeypatch, project_custom_function(independent_doc_probe) + ) detail = catalog.get(metadata.composite_key, max_doc_chars=None) assert detail.doc == inspect.getdoc(inspect.unwrap(independent_doc_probe)) assert "Additional Parameters is an authored heading" in detail.doc assert "discussion of slice_by_slice" in detail.doc assert "slice_by_slice" not in {parameter.name for parameter in detail.parameters} - offset = next(parameter for parameter in detail.parameters if parameter.name == "offset") + offset = next( + parameter for parameter in detail.parameters if parameter.name == "offset" + ) assert offset.default_repr == "0.75" assert offset.supplied_by is FunctionParameterSource.AGENT assert offset.description == "Declared intensity offset in original image units." assert catalog.search(query="original image units").total == 1 -def test_flexible_controls_remain_typed_and_existing_mro_behavior_is_unchanged(monkeypatch): +def test_flexible_controls_remain_typed_and_existing_mro_behavior_is_unchanged( + monkeypatch, +): seen = [] @numpy_contract - def independent_flexible_doc_probe(image: np.ndarray, *, offset: float = 0.5) -> np.ndarray: + def independent_flexible_doc_probe( + image: np.ndarray, *, offset: float = 0.5 + ) -> np.ndarray: """Apply an offset with declaration-owned dimensional selection.""" seen.append(image.shape) return image + offset - catalog, metadata = _catalog(monkeypatch, project_custom_function(independent_flexible_doc_probe)) + catalog, metadata = _catalog( + monkeypatch, project_custom_function(independent_flexible_doc_probe) + ) detail = catalog.get(metadata.composite_key, max_doc_chars=None) assert detail.doc == inspect.getdoc(inspect.unwrap(independent_flexible_doc_probe)) parameters = {parameter.name: parameter for parameter in detail.parameters} assert parameters["slice_by_slice"].default_repr == "False" - assert "Process 3D arrays slice-by-slice" in parameters["slice_by_slice"].description - assert parameters["dtype_config"].supplied_by is FunctionParameterSource.RUNTIME_PARAMETER + assert ( + "Process 3D arrays slice-by-slice" in parameters["slice_by_slice"].description + ) + assert ( + parameters["dtype_config"].supplied_by + is FunctionParameterSource.RUNTIME_PARAMETER + ) pixels = np.zeros((2, 4, 6)) result = metadata.func(pixels, slice_by_slice=True) assert seen == [(4, 6), (4, 6)] @@ -118,7 +135,9 @@ class IndependentDocRequest: offset: float = 0.5 -def test_nominal_request_boundary_is_not_unwrapped_to_private_implementation(monkeypatch): +def test_nominal_request_boundary_is_not_unwrapped_to_private_implementation( + monkeypatch, +): @callable_request(IndependentDocRequest) def implementation(request: IndependentDocRequest): """Private implementation prose.""" @@ -140,14 +159,23 @@ def no_doc(image): policy = PARAMETER_DOCUMENTATION_POLICY assert policy.detail_doc(no_doc, None) is None assert policy.detail_doc(no_doc, " ") is None - assert policy.detail_doc(no_doc, "External metadata documentation.") == "External metadata documentation." + assert ( + policy.detail_doc(no_doc, "External metadata documentation.") + == "External metadata documentation." + ) -def test_cellprofiler_setting_help_remains_on_final_typed_parameter_projection(monkeypatch): +def test_cellprofiler_setting_help_remains_on_final_typed_parameter_projection( + monkeypatch, +): from openhcs.interop.cellprofiler.module_declarations import CellProfilerModule - from openhcs.processing.backends.cellprofiler.classification import classify_objects_single_measurement + from openhcs.processing.backends.cellprofiler.classification import ( + classify_objects_single_measurement, + ) - module = CellProfilerModule.for_callable_contract(CallableContract.from_callable(classify_objects_single_measurement)) + module = CellProfilerModule.for_callable_contract( + CallableContract.from_callable(classify_objects_single_measurement) + ) assert module is not None implementation = module.require_callable() catalog, metadata = _catalog( @@ -167,6 +195,12 @@ def test_cellprofiler_setting_help_remains_on_final_typed_parameter_projection(m def test_doc_projection_has_no_leaf_control_dispatch_or_text_surgery(): tree = ast.parse(inspect.getsource(type(PARAMETER_DOCUMENTATION_POLICY))) - literals = {node.value for node in ast.walk(tree) if isinstance(node, ast.Constant) and isinstance(node.value, str)} + literals = { + node.value + for node in ast.walk(tree) + if isinstance(node, ast.Constant) and isinstance(node.value, str) + } assert {"slice_by_slice", "PURE_2D", "PURE_3D", "FLEXIBLE"}.isdisjoint(literals) - assert "resolve_raw_runtime_callable" in inspect.getsource(type(PARAMETER_DOCUMENTATION_POLICY).detail_doc) + assert "resolve_raw_runtime_callable" in inspect.getsource( + type(PARAMETER_DOCUMENTATION_POLICY).detail_doc + ) diff --git a/tests/unit/agent/test_function_declaration_help.py b/tests/unit/agent/test_function_declaration_help.py index 9d749e2188..3150267600 100644 --- a/tests/unit/agent/test_function_declaration_help.py +++ b/tests/unit/agent/test_function_declaration_help.py @@ -249,8 +249,7 @@ def denied_preparation(*args, **kwargs): ) assert isinstance(metadata, FunctionMetadata) assert ( - metadata.composite_key - == f"openhcs:analysis_neurite_outgrowth_{func.__name__}" + metadata.composite_key == f"openhcs:analysis_neurite_outgrowth_{func.__name__}" ) assert metadata.func is not func monkeypatch.setattr( @@ -276,11 +275,14 @@ def denied_preparation(*args, **kwargs): docstring_info_for_target(analyzed[parameter.name].param_type) ) assert document.content in parameter.description - assert next( - value["description"] - for value in to_jsonable(detail)["parameters"] - if value["name"] == parameter.name - ) == parameter.description + assert ( + next( + value["description"] + for value in to_jsonable(detail)["parameters"] + if value["name"] == parameter.name + ) + == parameter.description + ) assert not detail.doc_truncated import json diff --git a/tests/unit/agent/test_imagexpress_readonly_paths.py b/tests/unit/agent/test_imagexpress_readonly_paths.py index 381d36b9d5..3f552e9099 100644 --- a/tests/unit/agent/test_imagexpress_readonly_paths.py +++ b/tests/unit/agent/test_imagexpress_readonly_paths.py @@ -67,7 +67,9 @@ def inspect_plate(plate: Path, source_format="imagexpress", **bounds): def inventory_for(plate: Path): ensure_storage_registry() filemanager = FileManager(dict(storage_registry)) - handler = DatasetSourceChoice.named("imagexpress").open(plate, filemanager=filemanager) + handler = DatasetSourceChoice.named("imagexpress").open( + plate, filemanager=filemanager + ) return ( handler, filemanager, @@ -81,9 +83,10 @@ def inventory_for(plate: Path): def identity(record): - return frozenset( - (c, record.metadata[c.name]) for c in AxisFamily.active().axes - ), record.source_path + return ( + frozenset((c, record.metadata[c.name]) for c in AxisFamily.active().axes), + record.source_path, + ) def test_raw_inspection_retains_folder_z_and_never_initializes(tmp_path: Path): @@ -268,7 +271,9 @@ def image_path_components(self, path): assert result.image_files.count == result.parse_summary.parsed_file_count == 4 summaries = {item.component: item for item in result.components} assert tuple(v.key for v in summaries[Microscopy.Site.name].values) == ("7",) - assert tuple(v.key for v in summaries[Microscopy.Timepoint.name].values) == ("3",) + assert tuple(v.key for v in summaries[Microscopy.Timepoint.name].values) == ( + "3", + ) assert summaries[Microscopy.ZIndex.name].count == 2 query = service_for(plate).query_files( PlateFileQueryRequest.from_fields( diff --git a/tests/unit/agent/test_knowledge_base_service.py b/tests/unit/agent/test_knowledge_base_service.py index 0274bb487c..a9a6a3acd7 100644 --- a/tests/unit/agent/test_knowledge_base_service.py +++ b/tests/unit/agent/test_knowledge_base_service.py @@ -128,8 +128,7 @@ def test_search_prefers_declared_document_matches_over_equal_body_matches(tmp_pa (docs / "notes.md").write_text( "# Setup notes\n\n" + "\n".join( - f"## Note {index}\n\nThis body mentions a recipe.\n" - for index in range(8) + f"## Note {index}\n\nThis body mentions a recipe.\n" for index in range(8) ), encoding="utf-8", ) diff --git a/tests/unit/agent/test_mcp_affine_inspection_progress.py b/tests/unit/agent/test_mcp_affine_inspection_progress.py index a1e9a9bf61..50ac581510 100644 --- a/tests/unit/agent/test_mcp_affine_inspection_progress.py +++ b/tests/unit/agent/test_mcp_affine_inspection_progress.py @@ -19,19 +19,26 @@ from openhcs.agent.dto.common import AgentError, SCHEMA_VERSION from openhcs.agent.dto.execution import ArtifactPlanInspection from openhcs.mcp.execution import ( - McpMainThreadDispatcher, McpQueuedCallNotStartedError, McpTransportExecutor, + McpMainThreadDispatcher, + McpQueuedCallNotStartedError, + McpTransportExecutor, ) from pyqt_reactive.services.ui_thread_dispatch import ( - UiThreadDispatchError, UiThreadDispatcherClosedError, UiThreadDispatchTimeoutError, + UiThreadDispatchError, + UiThreadDispatcherClosedError, + UiThreadDispatchTimeoutError, ) from openhcs.mcp.server import build_server from python_introspect import to_jsonable -@pytest.mark.parametrize("declaration", ( - InspectPipelineSourceArtifactPlanCapability, - GetKnowledgeDocumentCapability, -)) +@pytest.mark.parametrize( + "declaration", + ( + InspectPipelineSourceArtifactPlanCapability, + GetKnowledgeDocumentCapability, + ), +) def test_source_leaf_composes_original_progress_and_affinity(declaration): assert issubclass(declaration, MainThreadProgressCapability) spec = declaration @@ -58,7 +65,9 @@ def inspect(self, request): EndpointStartupPhase.PREPARING_CAPABILITIES, "Source inspection on original main thread", ).publish() - assert self.wait_for_completion(), "SDK heartbeat was starved by main-affine work" + assert ( + self.wait_for_completion() + ), "SDK heartbeat was starved by main-affine work" if self.error is not None: raise self.error return ArtifactPlanInspection( @@ -83,7 +92,9 @@ async def report_progress(self, progress, total=None, message=None): self.service.release.set() -@pytest.mark.parametrize("error", (None, ValueError("original"), asyncio.CancelledError("original"))) +@pytest.mark.parametrize( + "error", (None, ValueError("original"), asyncio.CancelledError("original")) +) def test_generated_inspection_main_thread_context_progress_and_terminal(error): identity = ContextVar("affine-inspection-identity", default="outside") service = AffineInspectionService(identity, error=error) @@ -92,7 +103,9 @@ def test_generated_inspection_main_thread_context_progress_and_terminal(error): SimpleNamespace(artifact_plan_service=service), main_thread_dispatcher=executor.dispatcher, ) - tool = built._tool_manager.get_tool(InspectPipelineSourceArtifactPlanCapability.name) + tool = built._tool_manager.get_tool( + InspectPipelineSourceArtifactPlanCapability.name + ) context = InspectionProgressContext(service) async def exercise(): @@ -100,10 +113,18 @@ async def exercise(): try: if isinstance(error, asyncio.CancelledError): with pytest.raises(asyncio.CancelledError) as caught: - await tool.fn(mcp_context=context, plate_path="/synthetic", pipeline_source="pipeline_steps = []") + await tool.fn( + mcp_context=context, + plate_path="/synthetic", + pipeline_source="pipeline_steps = []", + ) assert caught.value is error return None - return await tool.fn(mcp_context=context, plate_path="/synthetic", pipeline_source="pipeline_steps = []") + return await tool.fn( + mcp_context=context, + plate_path="/synthetic", + pipeline_source="pipeline_steps = []", + ) finally: identity.reset(token) @@ -114,16 +135,24 @@ async def exercise(): assert len(service.calls) == 1 assert service.calls[0][1] == "original-request" assert identity.get() == "outside" - assert any("Source inspection on original main thread" in message for message in context.messages) + assert any( + "Source inspection on original main thread" in message + for message in context.messages + ) assert any("still running" in message for message in context.messages) if error is None: assert result["plate_path"] == "/synthetic" assert result["errors"] == [] elif isinstance(error, ValueError): - assert result["errors"] == [to_jsonable(AgentError.from_exception( - "mcp_tool_failed", error, - hint="The MCP server caught this exception at the tool boundary.", - ))] + assert result["errors"] == [ + to_jsonable( + AgentError.from_exception( + "mcp_tool_failed", + error, + hint="The MCP server caught this exception at the tool boundary.", + ) + ) + ] def test_independent_new_leaf_cooperative_hooks_need_no_consumer_edits(): @@ -164,20 +193,31 @@ class NewAfter(InspectPipelineSourceArtifactPlanCapability): identity = ContextVar("new-affine-declaration", default="new-case") service = AffineInspectionService(identity) executor = McpTransportExecutor() - built = build_server(SimpleNamespace(artifact_plan_service=service), main_thread_dispatcher=executor.dispatcher) + built = build_server( + SimpleNamespace(artifact_plan_service=service), + main_thread_dispatcher=executor.dispatcher, + ) async def exercise(): for declaration in (NewBefore, NewAfter): service.release.clear() context = InspectionProgressContext(service) tool = built._tool_manager.get_tool(declaration.name) - result = await tool.fn(mcp_context=context, plate_path="/new-case", pipeline_source="pipeline_steps = []") + result = await tool.fn( + mcp_context=context, + plate_path="/new-case", + pipeline_source="pipeline_steps = []", + ) assert result["plate_path"] == "/new-case" assert any("still running" in message for message in context.messages) try: executor.run(exercise) - assert calls == [(stage, declaration.name) for declaration in (NewBefore, NewAfter) for stage in ("audit-enter", "audit-exit")] + assert calls == [ + (stage, declaration.name) + for declaration in (NewBefore, NewAfter) + for stage in ("audit-enter", "audit-exit") + ] assert len(service.calls) == 2 finally: executor.close() @@ -219,7 +259,11 @@ async def exercise(): assert await asyncio.to_thread(entered.wait, 1) try: with pytest.raises(McpQueuedCallNotStartedError) as cancelled: - await asyncio.to_thread(executor.dispatcher.call, lambda: calls.append("forbidden"), timeout_ms=1) + await asyncio.to_thread( + executor.dispatcher.call, + lambda: calls.append("forbidden"), + timeout_ms=1, + ) assert cancelled.value.to_agent_error().code == "mcp_call_not_started" finally: second_finished.set() @@ -340,7 +384,11 @@ def wait_for_completion(self): return True def inspect(self, request): - self.error = ValueError("Original controlled source error") if request.pipeline_source == "failed" else None + self.error = ( + ValueError("Original controlled source error") + if request.pipeline_source == "failed" + else None + ) return super().inspect(request) @@ -349,18 +397,30 @@ def serve_stdio_inspection_fixture(*, work_seconds=2.4): identity = ContextVar("wire-source-identity", default="wire") with McpStdioTransport.reserve_process_stdio() as transport: - transport.run(build_server( - SimpleNamespace(artifact_plan_service=WireInspectionService(identity, work_seconds=work_seconds)), - main_thread_dispatcher=transport.execution.dispatcher, - )) + transport.run( + build_server( + SimpleNamespace( + artifact_plan_service=WireInspectionService( + identity, work_seconds=work_seconds + ) + ), + main_thread_dispatcher=transport.execution.dispatcher, + ) + ) -@pytest.mark.parametrize("work_seconds,idle_seconds,connection_count", ( - pytest.param(2.4, 1.6, 2, id="original-continuous"), - pytest.param(12.0, 10.0, 1, id="ordinary-ten-second-idle"), -)) +@pytest.mark.parametrize( + "work_seconds,idle_seconds,connection_count", + ( + pytest.param(2.4, 1.6, 2, id="original-continuous"), + pytest.param(12.0, 10.0, 1, id="ordinary-ten-second-idle"), + ), +) def test_resident_continuous_connections_keep_affinity_and_wire_progress( - unix_socket_dir, work_seconds, idle_seconds, connection_count, + unix_socket_dir, + work_seconds, + idle_seconds, + connection_count, ): from pyqt_reactive.services.async_operation_executor import AsyncOperationExecutor from openhcs.mcp.dev_client_core import McpDevServerSpec, McpDevSocketSession @@ -370,9 +430,13 @@ def test_resident_continuous_connections_keep_affinity_and_wire_progress( transport = McpSocketTransport(unix_socket_dir / "affine.sock") service = WireInspectionService( - ContextVar("resident-source-identity", default="resident"), work_seconds=work_seconds, + ContextVar("resident-source-identity", default="resident"), + work_seconds=work_seconds, + ) + built = build_server( + SimpleNamespace(artifact_plan_service=service), + main_thread_dispatcher=transport.execution.dispatcher, ) - built = build_server(SimpleNamespace(artifact_plan_service=service), main_thread_dispatcher=transport.execution.dispatcher) clients = AsyncOperationExecutor(max_workers=1) diagnostics = io.StringIO() @@ -384,10 +448,14 @@ def record_progress_notification(self, notification): progress = [] async def exercise(): - assert await asyncio.to_thread(wait_for_socket, transport.socket_path, timeout_seconds=10) + assert await asyncio.to_thread( + wait_for_socket, transport.socket_path, timeout_seconds=10 + ) try: for connection in range(connection_count): - async with ObservedSession(McpDevServerSpec(sys.executable), diagnostics, transport.socket_path) as session: + async with ObservedSession( + McpDevServerSpec(sys.executable), diagnostics, transport.socket_path + ) as session: await session.initialize(timeout_seconds=10) for source in ("cold", "failed"): started = time.monotonic() @@ -401,13 +469,24 @@ async def exercise(): elapsed = time.monotonic() - started assert elapsed >= work_seconds > idle_seconds events = progress[before:] - assert events and all(token == request_id for _, token in events) + assert events and all( + token == request_id for _, token in events + ) assert events[0][0] - started < 1 assert events[-1][0] - started > idle_seconds - print("RESIDENT_AFFINE_TERMINAL", source, elapsed, idle_seconds, flush=True) + print( + "RESIDENT_AFFINE_TERMINAL", + source, + elapsed, + idle_seconds, + flush=True, + ) payload = result["structuredContent"] if source == "failed": - assert payload["errors"][0]["message"] == "Original controlled source error" + assert ( + payload["errors"][0]["message"] + == "Original controlled source error" + ) else: assert payload["plate_path"] == "/synthetic" assert payload["errors"] == [] diff --git a/tests/unit/agent/test_mcp_affine_real_compile.py b/tests/unit/agent/test_mcp_affine_real_compile.py index aed75c5e99..403b225e73 100644 --- a/tests/unit/agent/test_mcp_affine_real_compile.py +++ b/tests/unit/agent/test_mcp_affine_real_compile.py @@ -11,7 +11,8 @@ from openhcs.agent.path_policy import AgentPathPolicy from openhcs.agent.services.config_service import ConfigService from openhcs.agent.services.artifact_plan_inspection_service import ( - ArtifactPlanInspectionService, InProcessCompileInspectionGateway, + ArtifactPlanInspectionService, + InProcessCompileInspectionGateway, ) from openhcs.demo.synthetic_data import SyntheticMicroscopyGenerator from openhcs.mcp.execution import McpTransportExecutor @@ -19,16 +20,26 @@ from openhcs.processing.backends.lib_registry.registry_service import RegistryService -def test_real_generated_source_inspection_keeps_original_compiler_on_main(monkeypatch, tmp_path: Path): +def test_real_generated_source_inspection_keeps_original_compiler_on_main( + monkeypatch, tmp_path: Path +): def forbidden_catalog(*args, **kwargs): raise AssertionError("Declared compilation must not prepare unrelated catalogs") - monkeypatch.setattr(RegistryService, "get_all_functions_with_metadata", forbidden_catalog) + monkeypatch.setattr( + RegistryService, "get_all_functions_with_metadata", forbidden_catalog + ) plate = tmp_path / "staged-source" SyntheticMicroscopyGenerator( - output_dir=str(plate), grid_size=(1, 1), tile_size=(64, 64), - wavelengths=1, z_stack_levels=1, wells=["A01"], num_cells=3, - random_seed=123, include_all_components=True, + output_dir=str(plate), + grid_size=(1, 1), + tile_size=(64, 64), + wavelengths=1, + z_stack_levels=1, + wells=["A01"], + num_cells=3, + random_seed=123, + include_all_components=True, ).generate_dataset() # This is a declared-module source fixture, not a catalog-cold/render proof. # The installed cold canonical journey remains parent-owned. @@ -62,18 +73,29 @@ async def report_progress(self, *args, **kwargs): config = ConfigService() service = ArtifactPlanInspectionService( - path_policy=AgentPathPolicy.with_roots(readable_roots=(tmp_path,), writable_roots=(tmp_path,)), + path_policy=AgentPathPolicy.with_roots( + readable_roots=(tmp_path,), writable_roots=(tmp_path,) + ), config_service=config, compile_inspection_gateway=AffineCompileGateway(), ) executor = McpTransportExecutor() - built = build_server(SimpleNamespace(artifact_plan_service=service), main_thread_dispatcher=executor.dispatcher) - tool = built._tool_manager.get_tool(InspectPipelineSourceArtifactPlanCapability.name) + built = build_server( + SimpleNamespace(artifact_plan_service=service), + main_thread_dispatcher=executor.dispatcher, + ) + tool = built._tool_manager.get_tool( + InspectPipelineSourceArtifactPlanCapability.name + ) async def exercise(): token = identity.set("exact-source-request") try: - return await tool.fn(mcp_context=ProgressContext(), plate_path=str(plate), pipeline_source=source) + return await tool.fn( + mcp_context=ProgressContext(), + plate_path=str(plate), + pipeline_source=source, + ) finally: identity.reset(token) @@ -96,8 +118,10 @@ async def exercise(): "Projecting compiled source workspace", "Source inspection compilation and source projection completed", ) - positions = [next(index for index, message in enumerate(messages) if stage in message) - for stage in stages] + positions = [ + next(index for index, message in enumerate(messages) if stage in message) + for stage in stages + ] assert positions == sorted(positions), messages assert len(calls[0].progress_queue.events) == 1 assert calls[0].progress_queue.events[0].axis_id == "A01" diff --git a/tests/unit/agent/test_mcp_dev_client_config_renderer.py b/tests/unit/agent/test_mcp_dev_client_config_renderer.py index 320611f1ad..405490e95f 100644 --- a/tests/unit/agent/test_mcp_dev_client_config_renderer.py +++ b/tests/unit/agent/test_mcp_dev_client_config_renderer.py @@ -67,25 +67,35 @@ def _schema() -> ConfigSchema: enum_values=("channel", "z_index"), ), ), - registries=(ConfigRegistrySchema(owner_type="PipelineConfig", registered_types=()),), - types=(ConfigTypeSchema(type_repr="LazyPartitionFilterConfig", description="Wells."),), + registries=( + ConfigRegistrySchema(owner_type="PipelineConfig", registered_types=()), + ), + types=( + ConfigTypeSchema( + type_repr="LazyPartitionFilterConfig", description="Wells." + ), + ), ) -def test_config_schema_renderer_is_registered_for_the_contract_and_its_subtypes() -> None: +def test_config_schema_renderer_is_registered_for_the_contract_and_its_subtypes() -> ( + None +): @dataclass(frozen=True, slots=True) class AnnotatedConfigSchema(ConfigSchema): annotation: str = "new declaration" - assert McpDevOutputRenderer.for_output_contract(ConfigSchema) is ConfigSchemaRenderer + assert ( + McpDevOutputRenderer.for_output_contract(ConfigSchema) is ConfigSchemaRenderer + ) assert ( McpDevOutputRenderer.for_output_contract(AnnotatedConfigSchema) is ConfigSchemaRenderer ) assert ConfigSchemaRenderer.render_options_type is CatalogRenderOptions - annotated = AnnotatedConfigSchema(**{ - name: getattr(_schema(), name) for name in ConfigSchema.__dataclass_fields__ - }) + annotated = AnnotatedConfigSchema( + **{name: getattr(_schema(), name) for name in ConfigSchema.__dataclass_fields__} + ) assert "flags=optional,lazy,inheritable" in ConfigSchemaRenderer.render( _batch(annotated) ) @@ -182,8 +192,14 @@ def test_config_schema_command_reads_render_options_from_its_renderer() -> None: parser = _build_parser() args = parser.parse_args( ( - "config-schema", "pipeline", "--path-prefix", "fiji_streaming_config", - "--contains", "lazy", "--limit", "1", + "config-schema", + "pipeline", + "--path-prefix", + "fiji_streaming_config", + "--contains", + "lazy", + "--limit", + "1", ) ) call = _calls_from_args(args)[0] diff --git a/tests/unit/agent/test_mcp_dev_client_pipeline_results.py b/tests/unit/agent/test_mcp_dev_client_pipeline_results.py index bb4bb7f9e2..776c547b53 100644 --- a/tests/unit/agent/test_mcp_dev_client_pipeline_results.py +++ b/tests/unit/agent/test_mcp_dev_client_pipeline_results.py @@ -85,25 +85,48 @@ def batch(capability, value): ) -@pytest.mark.parametrize("registered_count, code", ( - (0, "custom_function_registration_uncertain"), - (1, "custom_function_local_projection_failed"), -)) -def test_registration_renderer_retains_typed_error_receipt_and_handle(registered_count, code): +@pytest.mark.parametrize( + "registered_count, code", + ( + (0, "custom_function_registration_uncertain"), + (1, "custom_function_local_projection_failed"), + ), +) +def test_registration_renderer_retains_typed_error_receipt_and_handle( + registered_count, code +): from openhcs.agent.dto.execution_connection import ExecutionConnectionSpec - from openhcs.agent.dto.functions import CustomFunctionRegistrationHandle, CustomFunctionRegistrationResult - from openhcs.mcp.dev_client_renderers.knowledge import CustomFunctionRegistrationRenderer + from openhcs.agent.dto.functions import ( + CustomFunctionRegistrationHandle, + CustomFunctionRegistrationResult, + ) + from openhcs.mcp.dev_client_renderers.knowledge import ( + CustomFunctionRegistrationRenderer, + ) from zmqruntime.messages import ProcessIdentity handle = CustomFunctionRegistrationHandle( - ExecutionConnectionSpec(port=15993), ProcessIdentity.current(), - "a" * 64, "engineering_registration", False, None, + ExecutionConnectionSpec(port=15993), + ProcessIdentity.current(), + "a" * 64, + "engineering_registration", + False, + None, ) original = CustomFunctionRegistrationResult( - schema_version=SCHEMA_VERSION, registered_count=registered_count, - connection=handle.connection, server_identity=handle.server_identity, + schema_version=SCHEMA_VERSION, + registered_count=registered_count, + connection=handle.connection, + server_identity=handle.server_identity, observation_handle=handle, - errors=(AgentError(code=code, message="original cause", hint="Do not replay.", exception_type="ValueError"),), + errors=( + AgentError( + code=code, + message="original cause", + hint="Do not replay.", + exception_type="ValueError", + ), + ), ) response = batch(agent_capabilities.register_custom_function, original) decoded = McpDevToolBatchResponse.for_rendering(to_jsonable(response)) @@ -115,24 +138,44 @@ def test_registration_renderer_retains_typed_error_receipt_and_handle(registered assert text.count(f"{code}: original cause") == 1 assert "Do not replay." in text marker = "Read-only observation handle: " - rendered_handle = next(line.removeprefix(marker) for line in text.splitlines() if line.startswith(marker)) + rendered_handle = next( + line.removeprefix(marker) + for line in text.splitlines() + if line.startswith(marker) + ) assert json.loads(rendered_handle) == to_jsonable(handle) def test_registration_renderer_uses_declared_success_entries_without_mapping_reads(): - from openhcs.agent.dto.functions import CustomFunctionRegistrationResult, FunctionCatalogEntry - from openhcs.mcp.dev_client_renderers.knowledge import CustomFunctionRegistrationRenderer + from openhcs.agent.dto.functions import ( + CustomFunctionRegistrationResult, + FunctionCatalogEntry, + ) + from openhcs.mcp.dev_client_renderers.knowledge import ( + CustomFunctionRegistrationRenderer, + ) entry = FunctionCatalogEntry( - function_id="openhcs:engineering_registration", import_path="openhcs.processing.custom_functions.engineering_registration", - name="engineering_registration", module="openhcs.processing.custom_functions", library="openhcs", - signature="engineering_registration(image)", summary="Original declared summary", backend_tags=("numpy",), + function_id="openhcs:engineering_registration", + import_path="openhcs.processing.custom_functions.engineering_registration", + name="engineering_registration", + module="openhcs.processing.custom_functions", + library="openhcs", + signature="engineering_registration(image)", + summary="Original declared summary", + backend_tags=("numpy",), ) original = CustomFunctionRegistrationResult( - schema_version=SCHEMA_VERSION, registered_count=1, persisted=False, - storage_dir="/engineering", source_file_paths=("/engineering/original.py",), functions=(entry,), + schema_version=SCHEMA_VERSION, + registered_count=1, + persisted=False, + storage_dir="/engineering", + source_file_paths=("/engineering/original.py",), + functions=(entry,), + ) + text = CustomFunctionRegistrationRenderer.render( + to_jsonable(batch(agent_capabilities.register_custom_function, original)) ) - text = CustomFunctionRegistrationRenderer.render(to_jsonable(batch(agent_capabilities.register_custom_function, original))) assert "registered=1 persisted=False storage=/engineering" in text assert "Files: /engineering/original.py" in text assert f"{entry.function_id}: {entry.signature} tags=numpy" in text @@ -335,29 +378,47 @@ def test_nested_artifact_contract_and_receipt_survive_compact_limits(): def _dataset_row(**changes) -> DatasetRowState: values = dict( - scope_id="/plate", name="plate", root="/plate", pipeline_path=None, - selected=True, initialized=False, compiled=False, init_pending=False, - compile_pending=False, execution_active=False, status_prefix="", - orchestrator_state=None, execution_id=None, terminal_status=None, - runtime_state=None, runtime_percent=None, queue_position=None, + scope_id="/plate", + name="plate", + root="/plate", + pipeline_path=None, + selected=True, + initialized=False, + compiled=False, + init_pending=False, + compile_pending=False, + execution_active=False, + status_prefix="", + orchestrator_state=None, + execution_id=None, + terminal_status=None, + runtime_state=None, + runtime_percent=None, + queue_position=None, ) values.update(changes) return DatasetRowState(**values) def _datasets(row: DatasetRowState) -> DatasetListState: - return DatasetListState(rows=(row,), selected_scope_ids=(row.scope_id,), execution_state="idle") + return DatasetListState( + rows=(row,), selected_scope_ids=(row.scope_id,), execution_state="idle" + ) @pytest.mark.parametrize("waited", [True, False]) def test_execute_journey_follows_session_stages_and_renders_each_result(waited): def result(operation_id, status, **changes): - return SessionOperationResult(operation_id=operation_id, status=status, **changes) + return SessionOperationResult( + operation_id=operation_id, status=status, **changes + ) initialized = _dataset_row(initialized=True) compiled = _dataset_row(initialized=True, compiled=True) failed = _dataset_row( - initialized=True, compiled=True, terminal_status="failed", + initialized=True, + compiled=True, + terminal_status="failed", orchestrator_state="exec_failed", ) receipts = [ @@ -392,11 +453,15 @@ def result(operation_id, status, **changes): agent_capabilities.initialize_datasets.name, ] assert peer.calls[0][1]["port"] == 15993 - assert peer.calls[2][1] == {"scope_id": "/plate", "pipeline_source": "pipeline_steps = []"} + assert peer.calls[2][1] == { + "scope_id": "/plate", + "pipeline_source": "pipeline_steps = []", + } assert all( arguments == {"scope_ids": ["/plate"]} for name, arguments, _timeout in peer.calls - if name in ( + if name + in ( agent_capabilities.initialize_datasets.name, agent_capabilities.compile_datasets.name, agent_capabilities.run_datasets.name, @@ -503,7 +568,8 @@ class ExtendedInspection(ArtifactPlanInspection): value = McpDevToolResult._decode_payload( to_jsonable( ExtendedInspection(schema_version=SCHEMA_VERSION, plate_path="extended") - ), (ExtendedInspection,) + ), + (ExtendedInspection,), ) assert type(value) is ExtendedInspection and value.extension_fact == "retained" assert "plate=extended" in renderer.render_payload_value( @@ -713,7 +779,9 @@ async def controlled_entry(args): == 0 ) rendered = capsys.readouterr().out - assert "run_datasets" in rendered and "accepted" in rendered and "scope-1" in rendered + assert ( + "run_datasets" in rendered and "accepted" in rendered and "scope-1" in rendered + ) assert json.loads(rendered) == to_jsonable(response) diff --git a/tests/unit/agent/test_mcp_dev_client_plate_renderers.py b/tests/unit/agent/test_mcp_dev_client_plate_renderers.py index 39db8df27b..0ef1ffe392 100644 --- a/tests/unit/agent/test_mcp_dev_client_plate_renderers.py +++ b/tests/unit/agent/test_mcp_dev_client_plate_renderers.py @@ -71,11 +71,22 @@ def render(value) -> str: def row(**changes) -> DatasetRowState: return replace( DatasetRowState( - scope_id="/plates/a", name="plate-a", root="/plates/a", - pipeline_path=None, selected=True, initialized=True, compiled=False, - init_pending=False, compile_pending=False, execution_active=False, - status_prefix="", orchestrator_state=None, execution_id=None, - terminal_status=None, runtime_state=None, runtime_percent=None, + scope_id="/plates/a", + name="plate-a", + root="/plates/a", + pipeline_path=None, + selected=True, + initialized=True, + compiled=False, + init_pending=False, + compile_pending=False, + execution_active=False, + status_prefix="", + orchestrator_state=None, + execution_id=None, + terminal_status=None, + runtime_state=None, + runtime_percent=None, queue_position=None, ), **changes, @@ -96,11 +107,15 @@ def inspection(**changes) -> PlatePathInspectionResult: count=2, sampled_records=( PlateInspectionImageRecordSummary( - "A01_w1.tif", "/plates/a/A01_w1.tif", "/src/A01_w1.tif", + "A01_w1.tif", + "/plates/a/A01_w1.tif", + "/src/A01_w1.tif", {"modified": "2026-01-01", "size": "1 KB"}, ), PlateInspectionImageRecordSummary( - "A01_w2.tif", "/plates/a/A01_w2.tif", "A01_w2.tif", + "A01_w2.tif", + "/plates/a/A01_w2.tif", + "A01_w2.tif", {"modified": "2026-01-02"}, ), ), @@ -110,7 +125,9 @@ def inspection(**changes) -> PlatePathInspectionResult: scanned_file_count=1, sampled_records=( PlateInspectionResultFileRecordSummary( - "results/counts.csv", "/plates/a/results/counts.csv", "CSV", + "results/counts.csv", + "/plates/a/results/counts.csv", + "CSV", preview=PlateInspectionResultFilePreview( csv_columns=("well", "count"), csv_rows=({"well": "A01", "count": "11"},), @@ -120,19 +137,32 @@ def inspection(**changes) -> PlatePathInspectionResult: ), components=( PlateInspectionComponentSummary( - "well", PlateInspectionValueSource.METADATA, 1, + "well", + PlateInspectionValueSource.METADATA, + 1, (PlateInspectionComponentValue("A01"),), ), PlateInspectionComponentSummary( - "channel", PlateInspectionValueSource.PARSED_FILENAMES, 2, - (PlateInspectionComponentValue("1", "DAPI"), - PlateInspectionComponentValue("2")), + "channel", + PlateInspectionValueSource.PARSED_FILENAMES, + 2, + ( + PlateInspectionComponentValue("1", "DAPI"), + PlateInspectionComponentValue("2"), + ), ), ), format_specific_handler_candidates=( PlateInspectionHandlerCandidate( - "opera_phenix", "OperaPhenixHandler", "OperaPhenixParser", - "Images", 3, 2, False, True, False, + "opera_phenix", + "OperaPhenixHandler", + "OperaPhenixParser", + "Images", + 3, + 2, + False, + True, + False, ), ), source_diagnostics=({"diagnostic_type": "scene", "message": "two scenes"},), @@ -149,14 +179,22 @@ def query(**changes) -> PlateFileQueryResult: plate_path="/plates/a", requested_source_format="auto", detected_source_format="imagexpress", - total_count=3, returned_count=2, offset=0, limit=2, truncated_count=1, + total_count=3, + returned_count=2, + offset=0, + limit=2, + truncated_count=1, records=( PlateFileQueryRecordSummary( - PlateFileKind.IMAGE, "A01_w1.tif", {"modified": "2026-02-01"}, + PlateFileKind.IMAGE, + "A01_w1.tif", + {"modified": "2026-02-01"}, source_path="/src/A01_w1.tif", ), PlateFileQueryRecordSummary( - PlateFileKind.RESULT, "results/summary.csv", {"modified": "2026-01-01"}, + PlateFileKind.RESULT, + "results/summary.csv", + {"modified": "2026-01-01"}, relative_path="results/summary.csv", full_path="/plates/a_out/results/summary.csv", file_format="CSV", @@ -182,11 +220,15 @@ def stream(**changes) -> PlateFileStreamResult: requested_source_format="auto", viewer_type=ViewerType.NAPARI, viewer_config_key="napari_streaming_config", - connection=ExecutionConnectionSpec(port=5555, transport_mode=TransportMode.IPC), + connection=ExecutionConnectionSpec( + port=5555, transport_mode=TransportMode.IPC + ), requested_paths=("a.tif", "a.roi.zip"), streamed_image_paths=tuple(f"/src/{index}.tif" for index in range(10)), streamed_roi_paths=("/plates/a_out/a.roi.zip",), - skipped_records=(PlateFileQueryRecordSummary(PlateFileKind.IMAGE, "missing.tif"),), + skipped_records=( + PlateFileQueryRecordSummary(PlateFileKind.IMAGE, "missing.tif"), + ), status_messages=("streamed 11 files",), ), **changes, @@ -197,25 +239,34 @@ def test_every_plate_renderer_presents_its_declared_dto_and_failure(): """One family check: each renderer is registered for its DTO and none reads a dict.""" fixtures = ( SyntheticPlateGenerationResult( - schema_version=SCHEMA_VERSION, output_dir="/plates/synthetic", + schema_version=SCHEMA_VERSION, + output_dir="/plates/synthetic", requested_format=SyntheticPlateFormat.IMAGE_XPRESS, - grid_size=(2, 2), partition_values=("A01", "B02"), sampled_image_files=("a.tif",), + grid_size=(2, 2), + partition_values=("A01", "B02"), + sampled_image_files=("a.tif",), ), inspection(), query(), stream(), SelectedPlateImageInspectionResult( - schema_version=SCHEMA_VERSION, selected_plate=to_jsonable(row()), + schema_version=SCHEMA_VERSION, + selected_plate=to_jsonable(row()), inspection=inspection(), ), SelectedPlateFileQueryResult( - schema_version=SCHEMA_VERSION, selected_plate=to_jsonable(row()), query=query(), + schema_version=SCHEMA_VERSION, + selected_plate=to_jsonable(row()), + query=query(), ), SelectedPlateImageSampleResult( - schema_version=SCHEMA_VERSION, selected_plate=to_jsonable(row()), + schema_version=SCHEMA_VERSION, + selected_plate=to_jsonable(row()), ), SelectedPlateFileStreamResult( - schema_version=SCHEMA_VERSION, selected_plate=to_jsonable(row()), stream=stream(), + schema_version=SCHEMA_VERSION, + selected_plate=to_jsonable(row()), + stream=stream(), ), ) for value in fixtures: @@ -230,7 +281,10 @@ def test_every_plate_renderer_presents_its_declared_dto_and_failure(): McpDevServerSpec(sys.executable), (McpDevToolResult(CAPABILITY[type(value)].name, True, ()),), ) - assert renderer.render(tool_failure).splitlines()[0] == renderer.unavailable_summary + assert ( + renderer.render(tool_failure).splitlines()[0] + == renderer.unavailable_summary + ) def test_inspection_presents_axes_records_previews_and_next_sample(): @@ -238,12 +292,22 @@ def test_inspection_presents_axes_records_previews_and_next_sample(): assert "Status: ok confidence=none microscope=imagexpress" in text assert "Geometry: grid=2x3 pixel_size=" in text assert "Axis sizes: well=1 site= channel=2" in text - assert "profile=unknown-site,multi-channel,unknown-z_index,unknown-timepoint" in text + assert ( + "profile=unknown-site,multi-channel,unknown-z_index,unknown-timepoint" in text + ) assert "Metadata sources: well=metadata, channel=parsed_filenames" in text - assert "- channel: count=2 source=parsed_filenames values=1 (DAPI), 2 truncated=0" in text - assert " - opera_phenix parser=OperaPhenixParser recognized=2/3 root=Images" in text + assert ( + "- channel: count=2 source=parsed_filenames values=1 (DAPI), 2 truncated=0" + in text + ) + assert ( + " - opera_phenix parser=OperaPhenixParser recognized=2/3 root=Images" in text + ) assert "- scene: two scenes" in text - assert "Sampled artifacts modified: mixed latest=2026-01-02 earliest=2026-01-01" in text + assert ( + "Sampled artifacts modified: mixed latest=2026-01-02 earliest=2026-01-01" + in text + ) assert "- A01_w1.tif -> /src/A01_w1.tif modified=2026-01-01 size=1 KB" in text assert "- A01_w2.tif modified=2026-01-02" in text # The most recently modified record is the one suggested for sampling. @@ -251,7 +315,10 @@ def test_inspection_presents_axes_records_previews_and_next_sample(): "Next: sample-plate-image /plates/a A01_w2.tif " "--height 8 --width 8 --no-array-values" ) in text - assert "- results/counts.csv type=CSV\n csv columns: well, count\n csv row: well=A01, count=11" in text + assert ( + "- results/counts.csv type=CSV\n csv columns: well, count\n csv row: well=A01, count=11" + in text + ) assert text.count("plate_parse_failures") == 1 assert text.index("Warnings:") > text.index("Result records:") @@ -260,7 +327,10 @@ def test_query_presents_previews_pages_staleness_roots_and_result_hint(): text = render(query()) assert "Result: returned=2 total=3 offset=0 limit=2 truncated=1" in text assert "- image A01_w1.tif -> /src/A01_w1.tif modified=2026-02-01" in text - assert "- result results/summary.csv type=CSV -> /plates/a_out/results/summary.csv" in text + assert ( + "- result results/summary.csv type=CSV -> /plates/a_out/results/summary.csv" + in text + ) assert " csv row: well=A00; omitted wide cells: notes" in text assert " csv preview: showing 3/5 rows; 2 more in payload" in text assert "Next page: rerun with --offset 2 --limit 2" in text @@ -272,7 +342,9 @@ def test_query_presents_previews_pages_staleness_roots_and_result_hint(): records=(), truncated_count=0, warnings=( - AgentWarning(PlateInspectionIssueCode.RESULT_FILES_AVAILABLE.value, "results"), + AgentWarning( + PlateInspectionIssueCode.RESULT_FILES_AVAILABLE.value, "results" + ), ), ) ) @@ -286,7 +358,10 @@ def test_query_presents_previews_pages_staleness_roots_and_result_hint(): def test_stream_bounds_paths_and_suggests_viewer_commands(): text = render(stream()) - assert "Viewer: napari config=napari_streaming_config host=localhost port=5555 transport=ipc persistent=True" in text + assert ( + "Viewer: napari config=napari_streaming_config host=localhost port=5555 transport=ipc persistent=True" + in text + ) assert "Files: requested=2 resolved=0 images=10 rois=1 skipped=1" in text assert "- ... 2 more" in text assert "Skipped:\n- image missing.tif" in text @@ -302,18 +377,28 @@ def test_stream_bounds_paths_and_suggests_viewer_commands(): @pytest.mark.parametrize( "target,command", [ - (SelectedPlateFileQueryTarget.SELECTED, "Next: selected-plate-sample A01_w2.tif"), - (SelectedPlateFileQueryTarget.OUTPUT, "Next: selected-plate-sample --target output A01_w2.tif"), + ( + SelectedPlateFileQueryTarget.SELECTED, + "Next: selected-plate-sample A01_w2.tif", + ), + ( + SelectedPlateFileQueryTarget.OUTPUT, + "Next: selected-plate-sample --target output A01_w2.tif", + ), ], ) def test_selected_plate_images_render_the_nested_inspection(target, command): text = render( SelectedPlateImageInspectionResult( - schema_version=SCHEMA_VERSION, selected_plate=to_jsonable(row()), - target=target, inspection=inspection(), + schema_version=SCHEMA_VERSION, + selected_plate=to_jsonable(row()), + target=target, + inspection=inspection(), ) ) - assert text.startswith(f"Selected plate: plate-a root=/plates/a target={target.value}\nPlate: /plates/a") + assert text.startswith( + f"Selected plate: plate-a root=/plates/a target={target.value}\nPlate: /plates/a" + ) assert command in text assert "sample-plate-image" not in text assert text.count("plate_parse_failures") == 1 @@ -324,18 +409,27 @@ def test_selected_plate_files_hint_related_output_only_for_an_empty_selected_que empty = query(total_count=0, returned_count=0, truncated_count=0, records=()) text = render( SelectedPlateFileQueryResult( - schema_version=SCHEMA_VERSION, selected_plate=selected_row, query=empty, + schema_version=SCHEMA_VERSION, + selected_plate=selected_row, + query=empty, ) ) assert "Plate file query: /plates/a" in text - assert "Related output: /plates/a_out\nNext: selected-plate-files --target output --kind result" in text + assert ( + "Related output: /plates/a_out\nNext: selected-plate-files --target output --kind result" + in text + ) assert "Related output" not in render( SelectedPlateFileQueryResult( - schema_version=SCHEMA_VERSION, selected_plate=selected_row, query=query(), + schema_version=SCHEMA_VERSION, + selected_plate=selected_row, + query=query(), ) ) assert "Query: " in render( - SelectedPlateFileQueryResult(schema_version=SCHEMA_VERSION, selected_plate=selected_row) + SelectedPlateFileQueryResult( + schema_version=SCHEMA_VERSION, selected_plate=selected_row + ) ) @@ -343,8 +437,11 @@ def test_selected_plate_sample_and_stream_keep_target_context_on_failure(): error = AgentError("ui_selected_plate_count_not_one", "two plates selected") sample_text = render( SelectedPlateImageSampleResult( - schema_version=SCHEMA_VERSION, selected_plate=to_jsonable(row()), - target=SelectedPlateFileQueryTarget.OUTPUT, image_path="a.tif", errors=(error,), + schema_version=SCHEMA_VERSION, + selected_plate=to_jsonable(row()), + target=SelectedPlateFileQueryTarget.OUTPUT, + image_path="a.tif", + errors=(error,), ) ) assert sample_text.startswith( @@ -361,7 +458,8 @@ def test_selected_plate_sample_and_stream_keep_target_context_on_failure(): assert stream_text.count("two plates selected") == 1 nested = render( SelectedPlateFileStreamResult( - schema_version=SCHEMA_VERSION, selected_plate=to_jsonable(row()), + schema_version=SCHEMA_VERSION, + selected_plate=to_jsonable(row()), target=SelectedPlateFileQueryTarget.OUTPUT, stream=stream(warnings=(AgentWarning("viewer_slow", "slow start"),)), ) @@ -373,10 +471,16 @@ def test_selected_plate_sample_and_stream_keep_target_context_on_failure(): def test_synthetic_generation_suggests_inspection_and_query(): text = render( SyntheticPlateGenerationResult( - schema_version=SCHEMA_VERSION, output_dir="/plates/synthetic", + schema_version=SCHEMA_VERSION, + output_dir="/plates/synthetic", requested_format=SyntheticPlateFormat.IMAGE_XPRESS, - grid_size=(2, 2), tile_size=(64, 64), overlap_percent=10, partition_values=("A01", "B02"), - wavelengths=2, image_count=8, sampled_image_files=tuple(f"{i}.tif" for i in range(14)), + grid_size=(2, 2), + tile_size=(64, 64), + overlap_percent=10, + partition_values=("A01", "B02"), + wavelengths=2, + image_count=8, + sampled_image_files=tuple(f"{i}.tif" for i in range(14)), truncated_image_count=0, ) ) diff --git a/tests/unit/agent/test_mcp_dev_client_renderers.py b/tests/unit/agent/test_mcp_dev_client_renderers.py index d45e9a5930..bd715f63d2 100644 --- a/tests/unit/agent/test_mcp_dev_client_renderers.py +++ b/tests/unit/agent/test_mcp_dev_client_renderers.py @@ -53,7 +53,11 @@ def sample_value(hint, depth: int): return tuple(sample_value(args[0], depth + 1) for _ in range(2)) return tuple(sample_value(arg, depth + 1) for arg in args) if origin in (list, typing.get_origin(typing.Sequence[int])): - return [] if depth >= MAX_DEPTH else [sample_value(typing.get_args(hint)[0], depth + 1)] + return ( + [] + if depth >= MAX_DEPTH + else [sample_value(typing.get_args(hint)[0], depth + 1)] + ) if origin in (dict, typing.get_origin(typing.Mapping[str, int])): key_type, value_type = typing.get_args(hint) or (str, str) if depth >= MAX_DEPTH: @@ -72,7 +76,9 @@ def sample_value(hint, depth: int): return "sample" if is_dataclass(hint): return sample_dto(hint, depth + 1) - if isinstance(hint, typing.TypeAliasType if hasattr(typing, "TypeAliasType") else ()): + if isinstance( + hint, typing.TypeAliasType if hasattr(typing, "TypeAliasType") else () + ): return sample_value(hint.__value__, depth) # Dynamic JSON that no class models: an empty object. return {} @@ -123,7 +129,11 @@ def sample_dto(dto_type: type, depth: int = 0): def framed(payload) -> McpDevToolBatchResponse: return McpDevToolBatchResponse( server=SERVER, - results=(McpDevToolResult(tool="openhcs_test_tool", mcp_error=False, payloads=(payload,)),), + results=( + McpDevToolResult( + tool="openhcs_test_tool", mcp_error=False, payloads=(payload,) + ), + ), ) @@ -156,7 +166,9 @@ def test_every_renderer_presents_its_declared_dto(renderer): def test_every_renderer_reports_a_missing_payload_as_unavailable(renderer): failed = McpDevToolBatchResponse( server=SERVER, - results=(McpDevToolResult(tool="openhcs_test_tool", mcp_error=True, payloads=()),), + results=( + McpDevToolResult(tool="openhcs_test_tool", mcp_error=True, payloads=()), + ), ) rendered = renderer.render(failed) @@ -176,7 +188,9 @@ def test_base_appends_payload_warnings_and_declared_errors_once(): schema_version="v1", valid=False, errors=(AgentError(code="bad_source", message="Source is invalid."),), - warnings=(AgentWarning(code="slow", message="Validation was slow.", hint="wait"),), + warnings=( + AgentWarning(code="slow", message="Validation was slow.", hint="wait"), + ), ) rendered = McpDevOutputRenderer.for_output_contract( diff --git a/tests/unit/agent/test_mcp_dev_client_sampling_results.py b/tests/unit/agent/test_mcp_dev_client_sampling_results.py index 169aca3cec..49b08ecc47 100644 --- a/tests/unit/agent/test_mcp_dev_client_sampling_results.py +++ b/tests/unit/agent/test_mcp_dev_client_sampling_results.py @@ -14,16 +14,28 @@ from openhcs.agent.capabilities import SamplePlateImageCapability, agent_capabilities from openhcs.agent.dto.common import AgentError, SCHEMA_VERSION -from openhcs.agent.dto.plate import PlateImageSampleResult, SelectedPlateImageSampleResult -from openhcs.agent.dto.viewer import ViewerWindowImageSampleRecord, ViewerWindowImageSampleResult +from openhcs.agent.dto.plate import ( + PlateImageSampleResult, + SelectedPlateImageSampleResult, +) +from openhcs.agent.dto.viewer import ( + ViewerWindowImageSampleRecord, + ViewerWindowImageSampleResult, +) from openhcs.mcp import dev_client from openhcs.mcp.dev_client_commanding import CapabilityBackedCommandSpec from openhcs.mcp.dev_client_core import ( - McpDevClientPhase, McpDevPayloadFailure, McpDevServerSpec, - McpDevToolBatchResponse, McpDevToolResult, + McpDevClientPhase, + McpDevPayloadFailure, + McpDevServerSpec, + McpDevToolBatchResponse, + McpDevToolResult, ) from openhcs.mcp.dev_client_rendering import McpDevOutputRenderer -from openhcs.mcp.dev_client_renderers.plate import PlateImageSampleRenderer, SelectedPlateSampleRenderer +from openhcs.mcp.dev_client_renderers.plate import ( + PlateImageSampleRenderer, + SelectedPlateSampleRenderer, +) from openhcs.mcp.dev_client_renderers.viewer import ViewerImageSampleRenderer from openhcs.runtime.viewer_protocol import ViewerArrayValueSummary from python_introspect import to_jsonable @@ -33,29 +45,57 @@ def forbid_native_or_provider(monkeypatch): def forbidden(*args, **kwargs): raise AssertionError("Source sampling controls must not spawn a process") + monkeypatch.setattr(subprocess, "Popen", forbidden) monkeypatch.setattr(asyncio, "create_subprocess_exec", forbidden) monkeypatch.setattr(asyncio, "create_subprocess_shell", forbidden) def sample(**kwargs): - return replace(PlateImageSampleResult( - schema_version=SCHEMA_VERSION, plate_path="source-plate", requested_image_path="image.tif", - virtual_path="images/image.tif", source_path="physical/image.tif", - shape=(1, 2, 2), resolution_shape=(1, 2, 2), dtype="uint16", - minimum=0, maximum=4, mean=2.5, selected_resolution_index=0, - resolution_count=1, downsample_yx=(1.0, 1.0), statistics_scope="source_resolution", - sample_shape=(1, 2, 2), sample_included=True, sample_values=[[[0, 2], [3, 4]]], - ), **kwargs) + return replace( + PlateImageSampleResult( + schema_version=SCHEMA_VERSION, + plate_path="source-plate", + requested_image_path="image.tif", + virtual_path="images/image.tif", + source_path="physical/image.tif", + shape=(1, 2, 2), + resolution_shape=(1, 2, 2), + dtype="uint16", + minimum=0, + maximum=4, + mean=2.5, + selected_resolution_index=0, + resolution_count=1, + downsample_yx=(1.0, 1.0), + statistics_scope="source_resolution", + sample_shape=(1, 2, 2), + sample_included=True, + sample_values=[[[0, 2], [3, 4]]], + ), + **kwargs, + ) def selected_row(): return DatasetRowState( - scope_id="selected-root", name="selection", root="selected-root", - pipeline_path=None, selected=True, initialized=True, compiled=False, - init_pending=False, compile_pending=False, execution_active=False, - status_prefix="", orchestrator_state=None, execution_id=None, - terminal_status=None, runtime_state=None, runtime_percent=None, queue_position=None, + scope_id="selected-root", + name="selection", + root="selected-root", + pipeline_path=None, + selected=True, + initialized=True, + compiled=False, + init_pending=False, + compile_pending=False, + execution_active=False, + status_prefix="", + orchestrator_state=None, + execution_id=None, + terminal_status=None, + runtime_state=None, + runtime_percent=None, + queue_position=None, ) @@ -63,38 +103,53 @@ def selected(value): return SelectedPlateImageSampleResult( schema_version=SCHEMA_VERSION, selected_plate=to_jsonable(selected_row()), - image_path="image.tif", auto_selected_image_path=False, sample=value, + image_path="image.tif", + auto_selected_image_path=False, + sample=value, ) def batch(value, capability): receipt = {"isError": False, "structuredContent": to_jsonable(value), "content": []} return McpDevToolBatchResponse.from_results( - McpDevServerSpec(sys.executable), (McpDevToolResult.from_payload(capability.name, receipt),), + McpDevServerSpec(sys.executable), + (McpDevToolResult.from_payload(capability.name, receipt),), ) def command_render(response, argv): args = dev_client._build_parser().parse_args(argv) - return dev_client.McpDevCommandSpec.for_name(args.command).render_result(response, args) + return dev_client.McpDevCommandSpec.for_name(args.command).render_result( + response, args + ) @pytest.mark.parametrize("structured", [False, True]) @pytest.mark.parametrize("selected_case", [False, True]) -def test_original_session_framing_to_generated_sampling_command(structured, selected_case, monkeypatch): - from tests.unit.agent.test_mcp_dev_client_pipeline_results import ControlledWireSession +def test_original_session_framing_to_generated_sampling_command( + structured, selected_case, monkeypatch +): + from tests.unit.agent.test_mcp_dev_client_pipeline_results import ( + ControlledWireSession, + ) import openhcs.mcp.dev_client_core as ingress value = selected(sample()) if selected_case else sample() - argv = ("selected-plate-sample",) if selected_case else ("sample-plate-image", "source-plate", "image.tif") + argv = ( + ("selected-plate-sample",) + if selected_case + else ("sample-plate-image", "source-plate", "image.tif") + ) args = dev_client._build_parser().parse_args(argv) command = dev_client.McpDevCommandSpec.for_name(args.command) peer = ControlledWireSession((value,), structured=structured) decoded = [] original_decode = ingress.dataclass_from_mapping + def observe(contract, receipt): decoded.append(contract) return original_decode(contract, receipt) + monkeypatch.setattr(ingress, "dataclass_from_mapping", observe) response = asyncio.run(command.run_session(peer, args)) assert decoded == [type(value)] @@ -107,8 +162,14 @@ def observe(contract, receipt): @pytest.mark.parametrize("selected_case", [False, True]) @pytest.mark.parametrize("generic_call", [False, True]) -def test_generated_commands_retain_decoded_sampling_facts(monkeypatch, selected_case, generic_call): - capability = agent_capabilities.ui_sample_selected_plate_image if selected_case else agent_capabilities.sample_plate_image +def test_generated_commands_retain_decoded_sampling_facts( + monkeypatch, selected_case, generic_call +): + capability = ( + agent_capabilities.ui_sample_selected_plate_image + if selected_case + else agent_capabilities.sample_plate_image + ) value = selected(sample()) if selected_case else sample() response = batch(value, capability) decoded = response.results[0].first_decoded_payload() @@ -119,10 +180,17 @@ def test_generated_commands_retain_decoded_sampling_facts(monkeypatch, selected_ def forbidden(value): raise AssertionError("Typed sampling must not flatten an owned result to JSON") + monkeypatch.setattr(dev_client, "to_jsonable", forbidden) monkeypatch.setattr("python_introspect.jsonable.to_jsonable", forbidden) - argv = ("call", capability.name, "--arguments", "{}") if generic_call else ( - ("selected-plate-sample",) if selected_case else ("sample-plate-image", "source-plate", "image.tif") + argv = ( + ("call", capability.name, "--arguments", "{}") + if generic_call + else ( + ("selected-plate-sample",) + if selected_case + else ("sample-plate-image", "source-plate", "image.tif") + ) ) rendered = command_render(response, argv) assert "Image: images/image.tif" in rendered @@ -131,23 +199,51 @@ def forbidden(value): assert "mean=2.500" in rendered assert "Sample values:" in rendered if selected_case: - assert "Selected plate: selection root=selected-root target=selected" in rendered + assert ( + "Selected plate: selection root=selected-root target=selected" in rendered + ) assert "Selected image: image.tif auto=False" in rendered @pytest.mark.parametrize("selected_case", [False, True]) -@pytest.mark.parametrize("reason,shape,hint", [ - ("max_array_elements_exceeded", (1, 8, 8), "--max-array-elements 64 or smaller --width/--height"), - ("sample has 256 elements, above max_array_elements=40", (1, 16, 16), "--max-array-elements 256"), - ("array_values_not_requested", (1, 2, 2), "--include-array-values --max-array-elements 4"), - ("empty", (), "Sample values omitted: empty"), - ("max_array_elements_exceeded", (1, 0, 2), "--max-array-elements 0"), -]) +@pytest.mark.parametrize( + "reason,shape,hint", + [ + ( + "max_array_elements_exceeded", + (1, 8, 8), + "--max-array-elements 64 or smaller --width/--height", + ), + ( + "sample has 256 elements, above max_array_elements=40", + (1, 16, 16), + "--max-array-elements 256", + ), + ( + "array_values_not_requested", + (1, 2, 2), + "--include-array-values --max-array-elements 4", + ), + ("empty", (), "Sample values omitted: empty"), + ("max_array_elements_exceeded", (1, 0, 2), "--max-array-elements 0"), + ], +) def test_omission_and_budget_advice_preserved(selected_case, reason, shape, hint): - value = sample(sample_included=False, sample_values=(), sample_shape=shape, sample_omitted_reason=reason) - capability = agent_capabilities.ui_sample_selected_plate_image if selected_case else agent_capabilities.sample_plate_image + value = sample( + sample_included=False, + sample_values=(), + sample_shape=shape, + sample_omitted_reason=reason, + ) + capability = ( + agent_capabilities.ui_sample_selected_plate_image + if selected_case + else agent_capabilities.sample_plate_image + ) response = batch(selected(value) if selected_case else value, capability) - renderer = SelectedPlateSampleRenderer if selected_case else PlateImageSampleRenderer + renderer = ( + SelectedPlateSampleRenderer if selected_case else PlateImageSampleRenderer + ) assert hint in renderer.render(response) @@ -156,28 +252,54 @@ def test_legitimate_absent_sample_and_false_zero_empty_are_distinct(): rendered = SelectedPlateSampleRenderer.render(response) assert "Sample: " in rendered assert "auto=False" in rendered - zero = sample(virtual_path="", dtype="", mean=0.0, minimum=False, maximum=0, - sample_shape=(), sample_values=[], sample_included=True) - rendered = PlateImageSampleRenderer.render(batch(zero, agent_capabilities.sample_plate_image)) + zero = sample( + virtual_path="", + dtype="", + mean=0.0, + minimum=False, + maximum=0, + sample_shape=(), + sample_values=[], + sample_included=True, + ) + rendered = PlateImageSampleRenderer.render( + batch(zero, agent_capabilities.sample_plate_image) + ) assert "Image: \n" in rendered assert "dtype= min=False max=0 mean=0.000" in rendered assert "shape= included=True" in rendered assert "Sample values:\n[]" in rendered -@pytest.mark.parametrize("values,count", [([False, 0, "", None], 3), (list(range(65)), 65), ({"a": [0, False], "b": ""}, 3)]) +@pytest.mark.parametrize( + "values,count", + [([False, 0, "", None], 3), (list(range(65)), 65), ({"a": [0, False], "b": ""}, 3)], +) def test_shared_preview_policy_and_richer_viewer_owner(values, count): assert McpDevOutputRenderer.json_value_count(values) == count - response = batch(sample(sample_values=values), agent_capabilities.sample_plate_image) + response = batch( + sample(sample_values=values), agent_capabilities.sample_plate_image + ) rendered = PlateImageSampleRenderer.render(response) assert ("65 elements; pass --json" in rendered) == (count > 64) record = ViewerWindowImageSampleRecord( - layer_route_key="layer", layer_title="image", payload_route_key="payload", - data_type="image", path="source", components={}, array_values=(values,), - array_value_summary=ViewerArrayValueSummary(requested=True, included=True, shape=(1, count)), + layer_route_key="layer", + layer_title="image", + payload_route_key="payload", + data_type="image", + path="source", + components={}, + array_values=(values,), + array_value_summary=ViewerArrayValueSummary( + requested=True, included=True, shape=(1, count) + ), + ) + viewer = ViewerWindowImageSampleResult( + schema_version=SCHEMA_VERSION, observed=True, records=(record,) + ) + text = ViewerImageSampleRenderer.render( + batch(viewer, agent_capabilities.sample_viewer_window_image) ) - viewer = ViewerWindowImageSampleResult(schema_version=SCHEMA_VERSION, observed=True, records=(record,)) - text = ViewerImageSampleRenderer.render(batch(viewer, agent_capabilities.sample_viewer_window_image)) assert ("sample values:" in text) == (count <= 64) assert record.array_value_summary.shape_element_count == count @@ -186,19 +308,44 @@ def test_shared_preview_policy_and_richer_viewer_owner(values, count): def test_original_typed_diagnostics_retained_once(selected_case): error = AgentError("sample_failure", "retained cause", "retained hint") value = sample(errors=(error,)) - capability = agent_capabilities.ui_sample_selected_plate_image if selected_case else agent_capabilities.sample_plate_image - response = batch(replace(selected(value), errors=(error,)) if selected_case else value, capability) - text = (SelectedPlateSampleRenderer if selected_case else PlateImageSampleRenderer).render(response) + capability = ( + agent_capabilities.ui_sample_selected_plate_image + if selected_case + else agent_capabilities.sample_plate_image + ) + response = batch( + replace(selected(value), errors=(error,)) if selected_case else value, + capability, + ) + text = ( + SelectedPlateSampleRenderer if selected_case else PlateImageSampleRenderer + ).render(response) assert "failed" in text assert text.count("sample_failure") == 1 assert "retained cause" in text and "retained hint" in text assert response.has_errors() -@pytest.mark.parametrize("receipt", [{"alien": 1}, {"schema_version": SCHEMA_VERSION}, {"schema_version": SCHEMA_VERSION, "plate_path": "p", "requested_image_path": "i", "sample_included": "false"}]) +@pytest.mark.parametrize( + "receipt", + [ + {"alien": 1}, + {"schema_version": SCHEMA_VERSION}, + { + "schema_version": SCHEMA_VERSION, + "plate_path": "p", + "requested_image_path": "i", + "sample_included": "false", + }, + ], +) def test_malformed_receipt_never_becomes_successful_empty_sample(receipt): - result = McpDevToolResult.from_payload(agent_capabilities.sample_plate_image.name, {"structuredContent": receipt}) - response = McpDevToolBatchResponse.from_results(McpDevServerSpec(sys.executable), (result,)) + result = McpDevToolResult.from_payload( + agent_capabilities.sample_plate_image.name, {"structuredContent": receipt} + ) + response = McpDevToolBatchResponse.from_results( + McpDevServerSpec(sys.executable), (result,) + ) assert isinstance(result.payloads[0], McpDevPayloadFailure) assert result.payloads[0].payload == receipt rejection = to_jsonable(response)["results"][0]["payloads"][0] @@ -211,12 +358,19 @@ def test_malformed_receipt_never_becomes_successful_empty_sample(receipt): def test_transport_missing_and_unknown_receipts_keep_original_diagnostics(): server = McpDevServerSpec(sys.executable) - missing = McpDevToolBatchResponse.from_results(server, (McpDevToolResult(agent_capabilities.sample_plate_image.name, False, ()),)) + missing = McpDevToolBatchResponse.from_results( + server, + (McpDevToolResult(agent_capabilities.sample_plate_image.name, False, ()),), + ) assert "mcp_payload_missing" in PlateImageSampleRenderer.render(missing) - response = McpDevToolBatchResponse.from_transport_failure(server, McpDevClientPhase.CALL_TOOL, RuntimeError("wire lost")) + response = McpDevToolBatchResponse.from_transport_failure( + server, McpDevClientPhase.CALL_TOOL, RuntimeError("wire lost") + ) assert "wire lost" in PlateImageSampleRenderer.render(response) receipt = {"opaque": {"false": False, "zero": 0, "empty": ""}} - unknown = McpDevToolResult.from_payload("unregistered-external-tool", {"structuredContent": receipt}) + unknown = McpDevToolResult.from_payload( + "unregistered-external-tool", {"structuredContent": receipt} + ) assert unknown.payloads == (receipt,) assert to_jsonable(unknown)["payloads"] == [receipt] @@ -228,11 +382,16 @@ class NewSample(PlateImageSampleResult): independent_fact: str = "owned" events = [] + class SourceFact(PlateImageSampleRenderer): @classmethod def render_payload(cls, payload, options): events.append("source") - return super().render_payload(payload, options) + "\n" + payload.independent_fact + return ( + super().render_payload(payload, options) + + "\n" + + payload.independent_fact + ) class AuditFact(PlateImageSampleRenderer): @classmethod @@ -242,12 +401,17 @@ def render_payload(cls, payload, options): bases = (AuditFact, SourceFact) if reverse_order else (SourceFact, AuditFact) NewRenderer = type("NewSampleRenderer", bases, {"output_contract": NewSample}) + class NewCapability(SamplePlateImageCapability): name = f"openhcs_s1_new_sample_{int(reverse_order)}" cli_command = f"s1-new-sample-{int(reverse_order)}" output_contract = NewSample - value = NewSample(schema_version=SCHEMA_VERSION, plate_path="new-plate", requested_image_path="new-image") + value = NewSample( + schema_version=SCHEMA_VERSION, + plate_path="new-plate", + requested_image_path="new-image", + ) response = batch(value, NewCapability) assert type(response.results[0].first_decoded_payload()) is NewSample command = CapabilityBackedCommandSpec.for_capability_name(NewCapability.name) @@ -261,4 +425,7 @@ class NewCapability(SamplePlateImageCapability): def test_sampling_renderers_share_the_json_preview_count(): assert "_json_value_count" not in vars(PlateImageSampleRenderer) assert "_json_value_count" not in vars(ViewerImageSampleRenderer) - assert PlateImageSampleRenderer.json_value_count.__func__ is ViewerImageSampleRenderer.json_value_count.__func__ + assert ( + PlateImageSampleRenderer.json_value_count.__func__ + is ViewerImageSampleRenderer.json_value_count.__func__ + ) diff --git a/tests/unit/agent/test_mcp_dev_client_ui_action_results.py b/tests/unit/agent/test_mcp_dev_client_ui_action_results.py index f7b4c6d5bc..ec3f37be10 100644 --- a/tests/unit/agent/test_mcp_dev_client_ui_action_results.py +++ b/tests/unit/agent/test_mcp_dev_client_ui_action_results.py @@ -1,4 +1,5 @@ from openhcs.agent.dto.session import DatasetRowState + """Real CLI/codec action-family checks without a server or UI mutation.""" import sys import json @@ -28,12 +29,16 @@ ) from openhcs.mcp import dev_client from openhcs.mcp.dev_client_core import ( - McpDevServerSpec, McpDevToolBatchResponse, McpDevToolResult, - workflow_result_action_status, workflow_result_target_scope_ids, + McpDevServerSpec, + McpDevToolBatchResponse, + McpDevToolResult, + workflow_result_action_status, + workflow_result_target_scope_ids, workflow_poll_summary_result, ) from openhcs.mcp.dev_client_rendering import ( - McpDevOutputRenderer, McpDevOutputRenderOptions, + McpDevOutputRenderer, + McpDevOutputRenderOptions, ) from openhcs.mcp.dev_client_renderers.ui_bridge import UiActionInvokeRenderer from python_introspect import to_jsonable @@ -50,55 +55,92 @@ def action_fixture(): def action_result(capability, value): - return McpDevToolResult.from_payload(capability.name, { - "isError": False, "structuredContent": to_jsonable(value), "content": [], - }) + return McpDevToolResult.from_payload( + capability.name, + { + "isError": False, + "structuredContent": to_jsonable(value), + "content": [], + }, + ) def state_fixture(compiled, revision): summary = UiStateSurfaceSummary( - SCHEMA_VERSION, UiStateSurfaceIdentity(surface_id="plate_manager.state"), - "Plate Manager", True, widget_id="plate_manager", + SCHEMA_VERSION, + UiStateSurfaceIdentity(surface_id="plate_manager.state"), + "Plate Manager", + True, + widget_id="plate_manager", ) row = DatasetRowState( - scope_id="scope-1", name="plate-one", root="/plate-one", - pipeline_path=None, selected=True, initialized=True, compiled=compiled, - init_pending=False, compile_pending=False, execution_active=False, + scope_id="scope-1", + name="plate-one", + root="/plate-one", + pipeline_path=None, + selected=True, + initialized=True, + compiled=compiled, + init_pending=False, + compile_pending=False, + execution_active=False, status_prefix="Compiled" if compiled else "Created", orchestrator_state="compiled" if compiled else "created", - execution_id=None, terminal_status=None, - runtime_state=None, runtime_percent=None, queue_position=None, + execution_id=None, + terminal_status=None, + runtime_state=None, + runtime_percent=None, + queue_position=None, ) state = UiPlateManagerState( - schema_version=SCHEMA_VERSION, summary=summary, object_state_token=revision, - manager_execution_state="idle", rows=(row,), + schema_version=SCHEMA_VERSION, + summary=summary, + object_state_token=revision, + manager_execution_state="idle", + rows=(row,), current_revision_token=f"rev-{revision}", ) return UiStateSurfaceDocument( - schema_version=SCHEMA_VERSION, summary=summary, - payload_schema="openhcs.ui.plate_manager_state.v1", payload=to_jsonable(state), + schema_version=SCHEMA_VERSION, + summary=summary, + payload_schema="openhcs.ui.plate_manager_state.v1", + payload=to_jsonable(state), current_revision_token=state.current_revision_token, ) @pytest.mark.parametrize("json_output", (False, True)) def test_named_cli_wait_preserves_action_receipt_poll_and_final_rows( - monkeypatch, capsys, json_output, + monkeypatch, + capsys, + json_output, ): """Actual CLI/controller/codec chain, with only the MCP wire controlled.""" native = UiSelectedPlateWorkflowResult( - SCHEMA_VERSION, UiSelectedPlateWorkflowKind("compile_datasets"), - replace(action_fixture(), - identity=UiActionIdentity(widget_id="plate_manager", action_id="compile_datasets"), - receipt=UiMutationReceipt(UiMutationRequestToken(), "operation-one", True)), + SCHEMA_VERSION, + UiSelectedPlateWorkflowKind("compile_datasets"), + replace( + action_fixture(), + identity=UiActionIdentity( + widget_id="plate_manager", action_id="compile_datasets" + ), + receipt=UiMutationReceipt(UiMutationRequestToken(), "operation-one", True), + ), ) receipt = UiBridgeOperationRef( - schema_version=SCHEMA_VERSION, status="completed", started_at_unix=1.0, - completed_at_unix=2.0, outcome="completed", - identity=UiBridgeOperationIdentity("operation-one", UiBridgeOperationRoute("invoke_action")), + schema_version=SCHEMA_VERSION, + status="completed", + started_at_unix=1.0, + completed_at_unix=2.0, + outcome="completed", + identity=UiBridgeOperationIdentity( + "operation-one", UiBridgeOperationRoute("invoke_action") + ), ) calls = [] - wire_values = iter((state_fixture(False, 1), native, receipt, state_fixture(True, 2))) + wire_values = iter( + (state_fixture(False, 1), native, receipt, state_fixture(True, 2)) + ) wire_payloads = tuple(to_jsonable(value) for value in wire_values) class ControlledWireSession: @@ -106,27 +148,43 @@ class ControlledWireSession: async def call_tool(self, name, arguments, *, timeout_seconds): calls.append(name) - return {"isError": False, "structuredContent": wire_payloads[len(calls)-1], "content": []} + return { + "isError": False, + "structuredContent": wire_payloads[len(calls) - 1], + "content": [], + } async def controlled_session(args): return await dev_client.McpDevCommandSpec.for_name(args.command).run_session( - ControlledWireSession(), args, + ControlledWireSession(), + args, ) import python_introspect.jsonable as serialization + monkeypatch.setattr(dev_client, "_run_async", controlled_session) if not json_output: + def forbidden_serialization(value): raise AssertionError("Compact polling must retain its typed summary") + monkeypatch.setattr(serialization, "to_jsonable", forbidden_serialization) - argv = ["selected-workflow", "compile_datasets", "--wait", "--wait-interval-seconds", "0"] + argv = [ + "selected-workflow", + "compile_datasets", + "--wait", + "--wait-interval-seconds", + "0", + ] if json_output: argv.append("--json") assert dev_client.main(argv) == 0 - assert calls == [agent_capabilities.ui_get_state_surface.name, - agent_capabilities.ui_selected_plate_workflow.name, - agent_capabilities.ui_wait_for_operation_receipt.name, - agent_capabilities.ui_get_state_surface.name] + assert calls == [ + agent_capabilities.ui_get_state_surface.name, + agent_capabilities.ui_selected_plate_workflow.name, + agent_capabilities.ui_wait_for_operation_receipt.name, + agent_capabilities.ui_get_state_surface.name, + ] rendered = capsys.readouterr().out if json_output: response = json.loads(rendered) @@ -143,13 +201,17 @@ def forbidden_serialization(value): assert "terminal=, selected=True, root=/plate-one" in rendered -def test_malformed_action_is_nonzero_and_preserves_invalid_wire_payload(monkeypatch, capsys): +def test_malformed_action_is_nonzero_and_preserves_invalid_wire_payload( + monkeypatch, capsys +): malformed = {"workflow": "run_datasets", "action_result": {"status": "accepted"}} result = McpDevToolResult.from_payload( agent_capabilities.ui_selected_plate_workflow.name, {"isError": False, "structuredContent": malformed, "content": []}, ) - response = McpDevToolBatchResponse.from_results(McpDevServerSpec(sys.executable), (result,)) + response = McpDevToolBatchResponse.from_results( + McpDevServerSpec(sys.executable), (result,) + ) async def controlled_wire(args): return response @@ -168,23 +230,38 @@ def test_poll_presentation_rejects_row_missing_declared_state_field(): del state.payload["rows"][0]["initialized"] response = McpDevToolBatchResponse.from_results( McpDevServerSpec(sys.executable), - (action_result(agent_capabilities.ui_get_state_surface, state), - workflow_poll_summary_result( - workflow="compile_datasets", poll_requested=True, - poll_completed=True, poll_count=1, action_status="accepted", - )), + ( + action_result(agent_capabilities.ui_get_state_surface, state), + workflow_poll_summary_result( + workflow="compile_datasets", + poll_requested=True, + poll_completed=True, + poll_count=1, + action_status="accepted", + ), + ), + ) + args = dev_client._build_parser().parse_args( + ["selected-workflow", "compile_datasets", "--wait"] ) - args = dev_client._build_parser().parse_args(["selected-workflow", "compile_datasets", "--wait"]) with pytest.raises(ValueError, match="missing required field.*initialized"): - dev_client.McpDevCommandSpec.for_name("selected-workflow").render_result(response, args) + dev_client.McpDevCommandSpec.for_name("selected-workflow").render_result( + response, args + ) -def test_actual_generic_cli_retains_nested_action_without_json_roundtrip(monkeypatch, capsys): +def test_actual_generic_cli_retains_nested_action_without_json_roundtrip( + monkeypatch, capsys +): native = UiSelectedPlateWorkflowResult( - SCHEMA_VERSION, UiSelectedPlateWorkflowKind("run_datasets"), action_fixture(), + SCHEMA_VERSION, + UiSelectedPlateWorkflowKind("run_datasets"), + action_fixture(), ) result = action_result(agent_capabilities.ui_selected_plate_workflow, native) - response = McpDevToolBatchResponse.from_results(McpDevServerSpec(sys.executable), (result,)) + response = McpDevToolBatchResponse.from_results( + McpDevServerSpec(sys.executable), (result,) + ) async def controlled_wire(args): return response @@ -193,12 +270,20 @@ def forbidden_serialization(value): raise AssertionError("Compact presentation must retain the decoded result") import python_introspect.jsonable as serialization + monkeypatch.setattr(dev_client, "_run_async", controlled_wire) monkeypatch.setattr(serialization, "to_jsonable", forbidden_serialization) - assert dev_client.main([ - "call", agent_capabilities.ui_selected_plate_workflow.name, - "--arguments", '{"workflow":"run_datasets"}', - ]) == 0 + assert ( + dev_client.main( + [ + "call", + agent_capabilities.ui_selected_plate_workflow.name, + "--arguments", + '{"workflow":"run_datasets"}', + ] + ) + == 0 + ) rendered = capsys.readouterr().out assert "action=plate_manager/run_datasets status=accepted" in rendered assert "accepted=True" in rendered and "targets=scope-1" in rendered @@ -206,20 +291,32 @@ def forbidden_serialization(value): assert workflow_result_target_scope_ids(result) == ("scope-1",) -@pytest.mark.parametrize("presentation", ((), ("--output", "outline"), - ("--output", "json"), ("--json",))) +@pytest.mark.parametrize( + "presentation", ((), ("--output", "outline"), ("--output", "json"), ("--json",)) +) def test_actual_widget_tree_cli_uses_its_declared_output_format( - monkeypatch, capsys, presentation, + monkeypatch, + capsys, + presentation, ): """The real CLI must not require a second undeclared args.json flag.""" native = UiWidgetTreeResult( - schema_version=SCHEMA_VERSION, window_id="main_window", projected=True, - summary=UiWindowSummary(SCHEMA_VERSION, UiWindowIdentity(window_id="main_window"), - "OpenHCS regression window", "qt_top_level", True, True), + schema_version=SCHEMA_VERSION, + window_id="main_window", + projected=True, + summary=UiWindowSummary( + SCHEMA_VERSION, + UiWindowIdentity(window_id="main_window"), + "OpenHCS regression window", + "qt_top_level", + True, + True, + ), ) result = action_result(agent_capabilities.ui_get_widget_tree, native) response = McpDevToolBatchResponse.from_results( - McpDevServerSpec(sys.executable), (result,), + McpDevServerSpec(sys.executable), + (result,), ) async def controlled_wire(args): @@ -239,21 +336,39 @@ async def controlled_wire(args): def test_generic_widget_tree_call_uses_the_same_output_declaration(monkeypatch, capsys): native = UiWidgetTreeResult( - schema_version=SCHEMA_VERSION, window_id="main_window", projected=True, - summary=UiWindowSummary(SCHEMA_VERSION, UiWindowIdentity(window_id="main_window"), - "OpenHCS regression window", "qt_top_level", True, True), + schema_version=SCHEMA_VERSION, + window_id="main_window", + projected=True, + summary=UiWindowSummary( + SCHEMA_VERSION, + UiWindowIdentity(window_id="main_window"), + "OpenHCS regression window", + "qt_top_level", + True, + True, + ), ) result = action_result(agent_capabilities.ui_get_widget_tree, native) response = McpDevToolBatchResponse.from_results( - McpDevServerSpec(sys.executable), (result,), + McpDevServerSpec(sys.executable), + (result,), ) async def controlled_wire(args): return response monkeypatch.setattr(dev_client, "_run_async", controlled_wire) - assert dev_client.main(["call", agent_capabilities.ui_get_widget_tree.name, - "--arguments", '{"window_id":"main_window"}']) == 0 + assert ( + dev_client.main( + [ + "call", + agent_capabilities.ui_get_widget_tree.name, + "--arguments", + '{"window_id":"main_window"}', + ] + ) + == 0 + ) assert "Window: OpenHCS regression window" in capsys.readouterr().out @@ -277,21 +392,35 @@ def introduction_lines(cls, payload): class ExtendedAction(UiActionInvokeResult): annotation: str = "new declared fact" - bases = ((SecondCapability, FirstCapability) if reverse else - (FirstCapability, SecondCapability)) - extension = type("ExtendedActionRenderer", (*bases, UiActionInvokeRenderer), - {"output_contract": ExtendedAction}) + bases = ( + (SecondCapability, FirstCapability) + if reverse + else (FirstCapability, SecondCapability) + ) + extension = type( + "ExtendedActionRenderer", + (*bases, UiActionInvokeRenderer), + {"output_contract": ExtendedAction}, + ) try: assert McpDevOutputRenderer.for_output_contract(ExtendedAction) is extension value = ExtendedAction( - SCHEMA_VERSION, UiActionIdentity(widget_id="plate_manager", action_id="run_datasets"), - "accepted", UiMutationReceipt(UiMutationRequestToken(), accepted=True), + SCHEMA_VERSION, + UiActionIdentity(widget_id="plate_manager", action_id="run_datasets"), + "accepted", + UiMutationReceipt(UiMutationRequestToken(), accepted=True), + ) + decoded = McpDevToolResult._decode_payload( + to_jsonable(value), (ExtendedAction,) ) - decoded = McpDevToolResult._decode_payload(to_jsonable(value), (ExtendedAction,)) assert decoded.annotation == "new declared fact" rendered = extension.render_payload_value(decoded, McpDevOutputRenderOptions()) assert events == (["second", "first"] if reverse else ["first", "second"]) - assert rendered.count("first capability") == rendered.count("second capability") == 1 + assert ( + rendered.count("first capability") + == rendered.count("second capability") + == 1 + ) assert rendered.count("UI action invoke:") == 1 finally: del McpDevOutputRenderer.__registry__[ExtendedAction] diff --git a/tests/unit/agent/test_mcp_dev_client_ui_renderers.py b/tests/unit/agent/test_mcp_dev_client_ui_renderers.py index f7dd2edd0e..24353fde25 100644 --- a/tests/unit/agent/test_mcp_dev_client_ui_renderers.py +++ b/tests/unit/agent/test_mcp_dev_client_ui_renderers.py @@ -71,7 +71,10 @@ def render(payload, options=None) -> str: def snapshot(index: int, label: str = "edit") -> UiSnapshotRef: return replace( - sample_dto(UiSnapshotRef), index=index, label=label, branch="main", + sample_dto(UiSnapshotRef), + index=index, + label=label, + branch="main", snapshot_id=f"snapshot-{index}", ) @@ -112,9 +115,11 @@ def test_bridge_status_and_unavailable_bridge(): server=SERVER, results=(McpDevToolResult(tool="t", mcp_error=True, payloads=()),), ) - assert McpDevOutputRenderer.for_output_contract(UiBridgeStatus).render( - failed - ).startswith("UI bridge: unavailable\nErrors:") + assert ( + McpDevOutputRenderer.for_output_contract(UiBridgeStatus) + .render(failed) + .startswith("UI bridge: unavailable\nErrors:") + ) def window(window_id: str, title: str, kind: str, visible: bool) -> UiWindowSummary: @@ -147,21 +152,28 @@ def test_window_catalog_flags_only_visible_stray_top_level_windows(): 'Attention: 1 visible top-level window(s): qt_top_level:1 title="Error"' in rendered ) - assert '- plate_manager [embedded] visible=True' in rendered + assert "- plate_manager [embedded] visible=True" in rendered def test_ui_smoke_composes_each_tool_payload(): from openhcs.agent.capabilities import agent_capabilities def result(capability, payload): - return McpDevToolResult(tool=capability.name, mcp_error=False, payloads=(payload,)) + return McpDevToolResult( + tool=capability.name, mcp_error=False, payloads=(payload,) + ) health = replace( sample_dto(McpServerHealthResult), - status="ok", restart_required=False, stale_source_paths=(), warnings=(), + status="ok", + restart_required=False, + stale_source_paths=(), + warnings=(), ) bridges = replace(sample_dto(UiBridgeCatalog), warnings=()) - status = replace(sample_dto(UiBridgeStatus), reachable=True, descriptor_status="ok", warnings=()) + status = replace( + sample_dto(UiBridgeStatus), reachable=True, descriptor_status="ok", warnings=() + ) response = McpDevToolBatchResponse( server=SERVER, results=( @@ -211,9 +223,15 @@ def document(document_id): def plate_row(name: str, selected: bool, **values) -> DatasetRowState: return replace( sample_dto(DatasetRowState), - name=name, selected=selected, orchestrator_state="compiled", - status_prefix="", terminal_status=None, root=f"/{name}", - output_root=None, source_root=None, **values, + name=name, + selected=selected, + orchestrator_state="compiled", + status_prefix="", + terminal_status=None, + root=f"/{name}", + output_root=None, + source_root=None, + **values, ) @@ -245,15 +263,21 @@ def test_state_surface_documents_render_through_their_surface_state(): sample_dto(UiPipelineEditorState), summary=surface_summary( PipelineEditorStateSurfaceIdentityDeclaration.require_value(), - current_selection_count=1, total_scope_count=2, + current_selection_count=1, + total_scope_count=2, ), current_snapshot=snapshot(4, "edit pipeline"), selected_scope_ids=("/tmp/plate-a::functionstep_1",), steps=( replace( sample_dto(PipelineStepState), - index=0, name="normalize", dirty=True, default_diff=False, - function_names=("normalize",), function_ids=(), debug_pause=False, + index=0, + name="normalize", + dirty=True, + default_diff=False, + function_names=("normalize",), + function_ids=(), + debug_pause=False, step_scope_id=None, ), ), @@ -262,7 +286,9 @@ def test_state_surface_documents_render_through_their_surface_state(): rendered = render(document_for(editor)) assert 'Snapshot: 4 "edit pipeline"' in rendered assert "Selected scopes: /tmp/plate-a::functionstep_1" in rendered - assert "- 0. [*] normalize: enabled=True, selected=True, funcs=normalize" in rendered + assert ( + "- 0. [*] normalize: enabled=True, selected=True, funcs=normalize" in rendered + ) assert "ids=" not in rendered @@ -288,7 +314,9 @@ def test_unregistered_or_unreadable_surface_points_to_the_catalog(): summary=surface_summary("plate_manager", readable=False), payload={}, warnings=(), - errors=(AgentError(code="ui_state_surface_unreadable", message="Not readable."),), + errors=( + AgentError(code="ui_state_surface_unreadable", message="Not readable."), + ), ) rendered = render(document) assert "Surface: plate_manager" in rendered @@ -299,38 +327,68 @@ def test_unregistered_or_unreadable_surface_points_to_the_catalog(): def tree_node(class_name: str, *children, **values) -> UiWidgetTreeNode: defaults = dict( - object_name="", text=None, title=None, current_text=None, visible=True, - enabled=True, clickable=True, actionable=False, action_kinds=(), + object_name="", + text=None, + title=None, + current_text=None, + visible=True, + enabled=True, + clickable=True, + actionable=False, + action_kinds=(), ) return replace( sample_dto(UiWidgetTreeNode), - class_name=class_name, children=children, **(defaults | values), + class_name=class_name, + children=children, + **(defaults | values), ) def test_widget_tree_outline_shows_addressable_paths_and_hides_technical_nodes(): root = tree_node( "PipelineEditorWidget", - tree_node("QPushButton", text="Edit", path_id="1.0.2", action_kinds=("button",), - enabled=False, clickable=False), - tree_node("QPushButton", text="Generate", path_id="1.0.3", - action_kinds=("button",), visible=False), + tree_node( + "QPushButton", + text="Edit", + path_id="1.0.2", + action_kinds=("button",), + enabled=False, + clickable=False, + ), + tree_node( + "QPushButton", + text="Generate", + path_id="1.0.3", + action_kinds=("button",), + visible=False, + ), tree_node("QScrollBar"), tree_node("QLineEdit", path_id="1.0.4", actionable=True, text="Gamma"), tree_node("QLabel", current_text="long " * 40), ) action = replace( sample_dto(UiWidgetActionSummary), - path_id="1.0.4", object_state_scope_id="/plate::step", field_path="gamma", + path_id="1.0.4", + object_state_scope_id="/plate::step", + field_path="gamma", semantic_markers=("*",), ) result = replace( sample_dto(UiWidgetTreeResult), - root=root, actionable_widgets=(action,), tree_truncated=False, - actionable_widgets_truncated=False, warnings=(), - summary=replace(sample_dto(UiWindowSummary), title="Pipeline Editor", - dirty=False, dirty_field_count=0, signature_diff=True, - semantic_markers=()), + root=root, + actionable_widgets=(action,), + tree_truncated=False, + actionable_widgets_truncated=False, + warnings=(), + summary=replace( + sample_dto(UiWindowSummary), + title="Pipeline Editor", + dirty=False, + dirty_field_count=0, + signature_diff=True, + semantic_markers=(), + ), ) rendered = render(result) assert "Window: Pipeline Editor" in rendered @@ -357,22 +415,29 @@ def test_widget_tree_outline_shows_addressable_paths_and_hides_technical_nodes() def test_code_document_bounds_its_source(): document = replace( - sample_dto(UiCodeDocument), source="pattern = (some_func," + "x" * 50, - current_snapshot=snapshot(10), warnings=(), + sample_dto(UiCodeDocument), + source="pattern = (some_func," + "x" * 50, + current_snapshot=snapshot(10), + warnings=(), ) rendered = render(document, CodeDocumentRenderOptions(max_source_chars=20)) assert "Revision: token=" in rendered and "snapshot=main@10 head=" in rendered assert "Source:\npattern = (some_func" in rendered assert "..." in rendered - assert "Source:" not in render(document, CodeDocumentRenderOptions(include_source=False)) + assert "Source:" not in render( + document, CodeDocumentRenderOptions(include_source=False) + ) def test_code_document_apply_shows_revisions_and_snapshots(): applied = replace( sample_dto(UiCodeDocumentApplyResult), - base_revision_token="rev-123", current_revision_token="rev-456", - new_revision_token="rev-456", current_snapshot=snapshot(10), - undo_snapshot=None, warnings=(), + base_revision_token="rev-123", + current_revision_token="rev-456", + new_revision_token="rev-456", + current_snapshot=snapshot(10), + undo_snapshot=None, + warnings=(), ) rendered = render(applied) assert "Revision: base=rev-123 current=rev-456 new=rev-456" in rendered @@ -393,13 +458,20 @@ def test_action_catalog_filters_actions_and_warnings_by_widget(): sample_dto(UiActionCatalog), actions=( ui_action( - "plate_manager", "compile_datasets", enabled=False, - disabled_error=AgentError(code="no_plate", message="No plate.", hint="Add a plate."), + "plate_manager", + "compile_datasets", + enabled=False, + disabled_error=AgentError( + code="no_plate", message="No plate.", hint="Add a plate." + ), ), ui_action("image_browser", "open_file"), ), warnings=( - AgentWarning(code="plate_path_setup_uses_code_document", message="Use plate_manager code."), + AgentWarning( + code="plate_path_setup_uses_code_document", + message="Use plate_manager code.", + ), AgentWarning(code="unrelated", message="Something else."), ), ) @@ -427,7 +499,9 @@ def test_action_catalog_filters_actions_and_warnings_by_widget(): def test_disabled_actions_do_not_make_the_call_fail(): catalog = replace( sample_dto(UiActionCatalog), - actions=(ui_action("w", "a", disabled_error=AgentError(code="off", message="Off.")),), + actions=( + ui_action("w", "a", disabled_error=AgentError(code="off", message="Off.")), + ), ) result = McpDevToolResult(tool="t", mcp_error=False, payloads=(catalog,)) assert not result.has_errors() @@ -438,8 +512,12 @@ def test_tool_list_filters_groups_and_flattens(): server=SERVER, tool_count=3, tools=( - McpDevToolMetadata("openhcs_get_viewer_window_state", "Read bounded viewer state.", {}), - McpDevToolMetadata("openhcs_validate_viewer_window_state", "Validate viewer.", {}), + McpDevToolMetadata( + "openhcs_get_viewer_window_state", "Read bounded viewer state.", {} + ), + McpDevToolMetadata( + "openhcs_validate_viewer_window_state", "Validate viewer.", {} + ), McpDevToolMetadata("openhcs_health_check", "Health.", {}), ), ) @@ -453,4 +531,6 @@ def test_tool_list_filters_groups_and_flattens(): assert "[Viewer Review]" not in flat projected = ToolListRenderer.project_response(tools, contains="viewer", limit=1) assert projected["matched_tool_count"] == 2 - assert [tool["name"] for tool in projected["tools"]] == ["openhcs_get_viewer_window_state"] + assert [tool["name"] for tool in projected["tools"]] == [ + "openhcs_get_viewer_window_state" + ] diff --git a/tests/unit/agent/test_mcp_dev_client_viewer_renderers.py b/tests/unit/agent/test_mcp_dev_client_viewer_renderers.py index 7b8469ba12..70520a7af9 100644 --- a/tests/unit/agent/test_mcp_dev_client_viewer_renderers.py +++ b/tests/unit/agent/test_mcp_dev_client_viewer_renderers.py @@ -50,40 +50,57 @@ def batch(capability, *payloads) -> McpDevToolBatchResponse: def typed(contract, **values): - return dataclass_from_mapping(contract, {"schema_version": SCHEMA_VERSION, **values}) + return dataclass_from_mapping( + contract, {"schema_version": SCHEMA_VERSION, **values} + ) def render(capability, payload, *command_args: str) -> str: """Render through the capability's command, as the CLI does.""" command = capability.cli_command args = _build_parser().parse_args((command, *command_args)) - return McpDevCommandSpec.for_name(command).render_result(batch(capability, payload), args) + return McpDevCommandSpec.for_name(command).render_result( + batch(capability, payload), args + ) def test_every_viewer_output_contract_has_its_own_typed_renderer(): contracts = ( - RuntimeServerScanResult, RuntimeServerInfo, RuntimeExecutionStatus, - ViewerWindowSnapshotResult, ViewerWindowStateResult, ViewerWindowPayloadResult, - ViewerWindowImageSampleResult, ViewerWindowRoiSummaryResult, - ViewerWindowNavigationResult, ViewerWindowLayerIsolationResult, - ViewerWindowProbeResult, ViewerWindowValidationSummaryResult, + RuntimeServerScanResult, + RuntimeServerInfo, + RuntimeExecutionStatus, + ViewerWindowSnapshotResult, + ViewerWindowStateResult, + ViewerWindowPayloadResult, + ViewerWindowImageSampleResult, + ViewerWindowRoiSummaryResult, + ViewerWindowNavigationResult, + ViewerWindowLayerIsolationResult, + ViewerWindowProbeResult, + ViewerWindowValidationSummaryResult, UiWindowSnapshotResult, ) - renderers = [McpDevOutputRenderer.for_output_contract(contract) for contract in contracts] + renderers = [ + McpDevOutputRenderer.for_output_contract(contract) for contract in contracts + ] assert all(renderer is not None for renderer in renderers) - assert all(renderer.output_contract is contract - for renderer, contract in zip(renderers, contracts)) + assert all( + renderer.output_contract is contract + for renderer, contract in zip(renderers, contracts) + ) assert len(set(renderers)) == len(contracts) def test_missing_payload_renders_unavailable_summary_and_diagnostic_once(): empty = McpDevToolBatchResponse( server=McpDevServerIdentity(command="python", module="openhcs.mcp.server"), - results=(McpDevToolResult( - agent_capabilities.navigate_viewer_window.name, False, ()),), + results=( + McpDevToolResult(agent_capabilities.navigate_viewer_window.name, False, ()), + ), ) rendered = McpDevOutputRenderer.for_output_contract( - ViewerWindowNavigationResult).render(empty) + ViewerWindowNavigationResult + ).render(empty) assert rendered.startswith("Viewer navigation: failed\nErrors:\n") assert "mcp_payload_missing" in rendered @@ -92,11 +109,25 @@ def test_validation_tool_boundary_error_is_the_whole_compact_output(): capability = agent_capabilities.validate_viewer_window_state raw = McpDevToolBatchResponse( server=McpDevServerIdentity(command="python", module="openhcs.mcp.server"), - results=(McpDevToolResult(capability.name, False, ({ - "schema_version": SCHEMA_VERSION, "ok": False, "tool": capability.name, - "errors": [{"code": "mcp_tool_failed", - "message": "Viewer MCP timeout must not exceed 2000ms."}], - },)),), + results=( + McpDevToolResult( + capability.name, + False, + ( + { + "schema_version": SCHEMA_VERSION, + "ok": False, + "tool": capability.name, + "errors": [ + { + "code": "mcp_tool_failed", + "message": "Viewer MCP timeout must not exceed 2000ms.", + } + ], + }, + ), + ), + ), ) args = _build_parser().parse_args(("validate-viewer", "5555")) assert McpDevCommandSpec.for_name("validate-viewer").render_result(raw, args) == ( @@ -108,60 +139,105 @@ def test_validation_tool_boundary_error_is_the_whole_compact_output(): def test_validation_reports_policy_layers_and_axis_component_hint(): payload = typed( ViewerWindowValidationSummaryResult, - valid=False, observed=True, layer_count=1, mounted_layer_count=1, - payload_count=1, nonzero_payload_count=1, - validation_policy={"expected_layer_count": 2, - "required_axis_labels": ["site", "well", "channel"]}, + valid=False, + observed=True, + layer_count=1, + mounted_layer_count=1, + payload_count=1, + nonzero_payload_count=1, + validation_policy={ + "expected_layer_count": 2, + "required_axis_labels": ["site", "well", "channel"], + }, connection={"host": "localhost", "port": 5555, "transport_mode": "ipc"}, - layer_summaries=[{ - "route_key": "image-layer", "title": "Image", "valid": False, - "mounted": True, "item_count": 1, "axis_labels": ["channel", "y", "x"], - "component_labels": ["channel", "site", "well"], - "missing_required_axis_labels": ["site", "well"], - "axis_labels_present_as_components": ["site", "well"], - }], - warnings=[{ - "code": "viewer_required_axis_labels_missing", - "message": "Viewer layer 'Image' is missing required axis labels: site, well.", - "hint": "These labels are present as component metadata rather than mounted axes: site, well.", - }], + layer_summaries=[ + { + "route_key": "image-layer", + "title": "Image", + "valid": False, + "mounted": True, + "item_count": 1, + "axis_labels": ["channel", "y", "x"], + "component_labels": ["channel", "site", "well"], + "missing_required_axis_labels": ["site", "well"], + "axis_labels_present_as_components": ["site", "well"], + } + ], + warnings=[ + { + "code": "viewer_required_axis_labels_missing", + "message": "Viewer layer 'Image' is missing required axis labels: site, well.", + "hint": "These labels are present as component metadata rather than mounted axes: site, well.", + } + ], ) rendered = render(agent_capabilities.validate_viewer_window_state, payload, "5555") assert "Viewer validation: valid=False observed=True layers=1 mounted=1" in rendered - assert "Policy: expected_layers=2 required_axes=site,well,channel required_components=" in rendered + assert ( + "Policy: expected_layers=2 required_axes=site,well,channel required_components=" + in rendered + ) assert "Connection: localhost:5555 transport=ipc" in rendered - assert "- image-layer: valid=False mounted=True items=1 axes=channel,y,x" in rendered - assert "missing_axes=site,well components=channel,site,well missing_components=" in rendered + assert ( + "- image-layer: valid=False mounted=True items=1 axes=channel,y,x" in rendered + ) + assert ( + "missing_axes=site,well components=channel,site,well missing_components=" + in rendered + ) assert "axis_as_components=site,well" in rendered - assert ('Warnings:\n- viewer_required_axis_labels_missing: ' - "Viewer layer 'Image' is missing required axis labels: site, well. " - 'hint="These labels are present as component metadata rather than ' - 'mounted axes: site, well."') in rendered + assert ( + "Warnings:\n- viewer_required_axis_labels_missing: " + "Viewer layer 'Image' is missing required axis labels: site, well. " + 'hint="These labels are present as component metadata rather than ' + 'mounted axes: site, well."' + ) in rendered def test_state_merges_component_values_and_bounds_payload_summaries(): payload = typed( ViewerWindowStateResult, - **PORT, observed=True, + **PORT, + observed=True, viewer={"viewer_type": "napari", "title": "OpenHCS Napari Visualization"}, - layer_count=1, viewer_ndim=3, axis_labels=["channel", "y", "x"], - current_step=[0, 47, 47], active_dimension_label_route="image-layer", - component_group_count=1, component_item_count=2, - layers=[{ - "route_key": "image-layer", "title": "1. Agent invert", "mounted": True, - "visible": True, "selected": True, "item_count": 2, - "data_types": ["image"], "axis_labels": ["channel", "y", "x"], - "component_values": [ - {"well": "A01", "channel": 1, "site": 1}, - {"well": "A01", "channel": 2, "site": 1}, - ], - "axis_component_values": {"channel": [1, 2]}, - }], + layer_count=1, + viewer_ndim=3, + axis_labels=["channel", "y", "x"], + current_step=[0, 47, 47], + active_dimension_label_route="image-layer", + component_group_count=1, + component_item_count=2, + layers=[ + { + "route_key": "image-layer", + "title": "1. Agent invert", + "mounted": True, + "visible": True, + "selected": True, + "item_count": 2, + "data_types": ["image"], + "axis_labels": ["channel", "y", "x"], + "component_values": [ + {"well": "A01", "channel": 1, "site": 1}, + {"well": "A01", "channel": 2, "site": 1}, + ], + "axis_component_values": {"channel": [1, 2]}, + } + ], ) rendered = render(agent_capabilities.get_viewer_window_state, payload, "5555") - assert 'Viewer state: observed=True type=napari title="OpenHCS Napari Visualization"' in rendered - assert "Window: layers=1 ndim=3 axes=channel,y,x current_step=[0, 47, 47] active_route=image-layer" in rendered - assert '- image-layer: title="1. Agent invert" visible=True selected=True items=2 types=image' in rendered + assert ( + 'Viewer state: observed=True type=napari title="OpenHCS Napari Visualization"' + in rendered + ) + assert ( + "Window: layers=1 ndim=3 axes=channel,y,x current_step=[0, 47, 47] active_route=image-layer" + in rendered + ) + assert ( + '- image-layer: title="1. Agent invert" visible=True selected=True items=2 types=image' + in rendered + ) assert " components: channel=1,2, site=1, well=A01" in rendered assert " axis values: channel=1,2" in rendered @@ -170,35 +246,74 @@ def test_payloads_mark_streamed_paths_and_shape_payload_counts(tmp_path): streamed_path = tmp_path / "streamed_A01_w1.tif" payload = typed( ViewerWindowPayloadResult, - **PORT, observed=True, layer_count=2, + **PORT, + observed=True, + layer_count=2, layers=[ - {"route_key": "image-layer", "title": "Images", "mounted": True, - "item_count": 2, "axis_labels": ["channel", "y", "x"], - "stack_axes": ["channel"], - "payloads": [{ - "route_key": "image-layer:0", "data_type": "image", "axis_indices": [0], - "components": {"well": "A01", "channel": 1}, - "path": str(streamed_path), - "summary": {"shape": [96, 96], "dtype": "uint16", "nonzero_count": 9216}, - "array_value_summary": {"included": False, "shape": [96, 96], - "omitted_reason": "max_array_elements_exceeded"}, - }]}, - {"route_key": "roi-layer", "title": "ROIs", "mounted": True, - "item_count": 1, "axis_labels": ["y", "x"], - "payloads": [{ - "route_key": "roi-layer:0", "data_type": "shapes", "components": {"channel": 2}, - "path": "relative/A01_w2.roi.zip", - "summary": {"shape_payload_count": 7, "nonzero_count": 7}, - "shape_payloads": [{"metadata": {"label": 1}}, {"metadata": {"label": 2}}], - }]}, + { + "route_key": "image-layer", + "title": "Images", + "mounted": True, + "item_count": 2, + "axis_labels": ["channel", "y", "x"], + "stack_axes": ["channel"], + "payloads": [ + { + "route_key": "image-layer:0", + "data_type": "image", + "axis_indices": [0], + "components": {"well": "A01", "channel": 1}, + "path": str(streamed_path), + "summary": { + "shape": [96, 96], + "dtype": "uint16", + "nonzero_count": 9216, + }, + "array_value_summary": { + "included": False, + "shape": [96, 96], + "omitted_reason": "max_array_elements_exceeded", + }, + } + ], + }, + { + "route_key": "roi-layer", + "title": "ROIs", + "mounted": True, + "item_count": 1, + "axis_labels": ["y", "x"], + "payloads": [ + { + "route_key": "roi-layer:0", + "data_type": "shapes", + "components": {"channel": 2}, + "path": "relative/A01_w2.roi.zip", + "summary": {"shape_payload_count": 7, "nonzero_count": 7}, + "shape_payloads": [ + {"metadata": {"label": 1}}, + {"metadata": {"label": 2}}, + ], + } + ], + }, ], ) rendered = render(agent_capabilities.get_viewer_window_payloads, payload, "5555") assert "Viewer payloads: observed=True layers=2" in rendered - assert '- image-layer: title="Images" mounted=True items=2 axes=channel,y,x stack=channel payloads=1' in rendered - assert "payload type=image axis=0 aggregate_axis= components=channel=1, well=A01" in rendered + assert ( + '- image-layer: title="Images" mounted=True items=2 axes=channel,y,x stack=channel payloads=1' + in rendered + ) + assert ( + "payload type=image axis=0 aggregate_axis= components=channel=1, well=A01" + in rendered + ) assert "shape=[96, 96] dtype=uint16 nonzero=9216" in rendered - assert "array=included=False sample_shape=[96, 96] reason=max_array_elements_exceeded" in rendered + assert ( + "array=included=False sample_shape=[96, 96] reason=max_array_elements_exceeded" + in rendered + ) assert f"path={streamed_path} (streamed/non-materialized)" in rendered assert "payload type=shapes axis=" in rendered assert "shape_members=7 returned_shapes=2 semantic_rois=use-viewer-rois" in rendered @@ -207,15 +322,25 @@ def test_payloads_mark_streamed_paths_and_shape_payload_counts(tmp_path): ROI_PAYLOAD = { - "layer_route_key": "roi-layer", "layer_title": "ROIs", - "payload_route_key": "roi-layer:0", "path": "/tmp/A01.roi.zip", - "components": {"well": "A01", "channel": 1}, "axis_indices": [0, 1], - "roi_count": 3, "returned_roi_count": 3, "roi_count_exact": False, - "roi_member_count": 7, "returned_roi_member_count": 3, - "roi_duplicate_member_count": 0, "roi_payloads_truncated": True, + "layer_route_key": "roi-layer", + "layer_title": "ROIs", + "payload_route_key": "roi-layer:0", + "path": "/tmp/A01.roi.zip", + "components": {"well": "A01", "channel": 1}, + "axis_indices": [0, 1], + "roi_count": 3, + "returned_roi_count": 3, + "roi_count_exact": False, + "roi_member_count": 7, + "returned_roi_member_count": 3, + "roi_duplicate_member_count": 0, + "roi_payloads_truncated": True, "area": {"min": 10.0, "median": 42.0, "mean": 40.0, "max": 80.0}, - "perimeter": None, "bounds_yx": None, "coordinate_count": 128, - "spatial_origin_yx": [0, 0], "source_spatial_shape_yx": [96, 96], + "perimeter": None, + "bounds_yx": None, + "coordinate_count": 128, + "spatial_origin_yx": [0, 0], + "source_spatial_shape_yx": [96, 96], "out_of_source_bounds_count": 0, "example_rois": [{"label": "cell-1", "area": 42, "centroid_yx": [5, 6]}], } @@ -224,21 +349,37 @@ def test_payloads_mark_streamed_paths_and_shape_payload_counts(tmp_path): def test_rois_render_payload_statistics_and_examples(): payload = typed( ViewerWindowRoiSummaryResult, - observed=True, route_key="roi-layer", axis_indices=[0, 1], layer_count=2, - payload_record_count=1, payload_type_counts={"shapes": 1}, roi_payload_count=1, - total_roi_count=3, returned_roi_count=3, roi_count_exact=False, - total_roi_member_count=7, returned_roi_member_count=3, - roi_payloads_truncated=True, payloads=[ROI_PAYLOAD], + observed=True, + route_key="roi-layer", + axis_indices=[0, 1], + layer_count=2, + payload_record_count=1, + payload_type_counts={"shapes": 1}, + roi_payload_count=1, + total_roi_count=3, + returned_roi_count=3, + roi_count_exact=False, + total_roi_member_count=7, + returned_roi_member_count=3, + roi_payloads_truncated=True, + payloads=[ROI_PAYLOAD], + ) + rendered = render( + agent_capabilities.summarize_viewer_window_rois, payload, "5555", "roi-layer" ) - rendered = render(agent_capabilities.summarize_viewer_window_rois, payload, "5555", "roi-layer") assert "Viewer ROIs: observed=True route=roi-layer axis=0,1" in rendered assert "ROIs: total=3 returned=3 exact=False members=7/3 truncated=True" in rendered assert "Payload types: shapes=1" in rendered - assert ('- title="ROIs" layer_route=roi-layer payload_route=roi-layer:0 axis=0,1 ' - "components=channel=1, well=A01 roi_count=3 returned=3 exact=False " - "members=7/3 duplicate_members=0") in rendered + assert ( + '- title="ROIs" layer_route=roi-layer payload_route=roi-layer:0 axis=0,1 ' + "components=channel=1, well=A01 roi_count=3 returned=3 exact=False " + "members=7/3 duplicate_members=0" + ) in rendered assert "area=min=10.0,median=42.0,mean=40.0,max=80.0 perimeter=" in rendered - assert "coords=128 source_origin=[0, 0] source_shape=[96, 96] out_of_bounds=0" in rendered + assert ( + "coords=128 source_origin=[0, 0] source_shape=[96, 96] out_of_bounds=0" + in rendered + ) assert "example label=cell-1 area=42 centroid=[5, 6]" in rendered @@ -246,14 +387,23 @@ def test_rois_render_payload_statistics_and_examples(): def test_rois_explain_missing_roi_payloads_for_the_requested_scope(route_key): payload = typed( ViewerWindowRoiSummaryResult, - observed=True, route_key=route_key, layer_count=7, - payload_record_count=32, payload_type_counts={"image": 32}, + observed=True, + route_key=route_key, + layer_count=7, + payload_record_count=32, + payload_type_counts={"image": 32}, ) rendered = render(agent_capabilities.summarize_viewer_window_rois, payload, "5555") - assert f"Viewer ROIs: observed=True route={route_key or ''} axis= layers=7 records=32 payloads=0" in rendered + assert ( + f"Viewer ROIs: observed=True route={route_key or ''} axis= layers=7 records=32 payloads=0" + in rendered + ) assert "Payload types: image=32" in rendered suffix = " route." if route_key else "." - assert f"Interpretation: no ROI/shapes payloads were found for the requested viewer{suffix}" in rendered + assert ( + f"Interpretation: no ROI/shapes payloads were found for the requested viewer{suffix}" + in rendered + ) rerun = "- If `viewer-state` shows a shapes layer, rerun `viewer-rois ` for that route." assert (rerun in rendered) is (route_key is None) assert "- If payload types are image-only" in rendered @@ -261,7 +411,8 @@ def test_rois_explain_missing_roi_payloads_for_the_requested_scope(route_key): def test_rois_with_errors_do_not_explain_absence(): payload = typed( - ViewerWindowRoiSummaryResult, observed=False, + ViewerWindowRoiSummaryResult, + observed=False, errors=[{"code": "viewer_window_rois_failed", "message": "timed out"}], ) rendered = render(agent_capabilities.summarize_viewer_window_rois, payload, "5555") @@ -272,34 +423,69 @@ def test_rois_with_errors_do_not_explain_absence(): def image_sample(max_array_elements: int, *, included: bool = False, values=()): return typed( ViewerWindowImageSampleResult, - observed=True, route_key="image-layer", axis_indices={"channel": 1}, - array_slices=[[0, 8], [0, 8]], record_count=1, returned_record_count=1, - raw_image_record_count=1, total_payload_record_count=2, + observed=True, + route_key="image-layer", + axis_indices={"channel": 1}, + array_slices=[[0, 8], [0, 8]], + record_count=1, + returned_record_count=1, + raw_image_record_count=1, + total_payload_record_count=2, sample_protocol_supported=True, - sample_included_count=int(included), sample_omitted_count=int(not included), - records=[{ - "layer_title": "image-layer", "data_type": "image", "components": {}, - "payload_route_key": "image-layer:0", "layer_route_key": "image-layer", - "axis_indices": [1], "path": "virtual/image.tif", - "summary": {"shape": [96, 96], "dtype": "uint16", "min": 23, - "max": 10755, "nonzero_count": 9216}, - "array_value_summary": { - "included": included, "shape": [8, 8], - **({} if included else {"omitted_reason": "max_array_elements_exceeded"}), - "max_array_elements": max_array_elements, - }, - "array_values": list(values), - }], + sample_included_count=int(included), + sample_omitted_count=int(not included), + records=[ + { + "layer_title": "image-layer", + "data_type": "image", + "components": {}, + "payload_route_key": "image-layer:0", + "layer_route_key": "image-layer", + "axis_indices": [1], + "path": "virtual/image.tif", + "summary": { + "shape": [96, 96], + "dtype": "uint16", + "min": 23, + "max": 10755, + "nonzero_count": 9216, + }, + "array_value_summary": { + "included": included, + "shape": [8, 8], + **( + {} + if included + else {"omitted_reason": "max_array_elements_exceeded"} + ), + "max_array_elements": max_array_elements, + }, + "array_values": list(values), + } + ], ) def test_image_sample_includes_small_sample_values(): sample = image_sample(64, included=True, values=([1, 2], [3, 4])) - rendered = render(agent_capabilities.sample_viewer_window_image, sample, - "5555", "image-layer", "--include-array-values") - assert "Viewer image sample: observed=True route=image-layer axis=channel=1 slices=[[0, 8], [0, 8]]" in rendered - assert "Records: matched=1 returned=1 truncated=0 image=1 total_payloads=2 sample_supported=True" in rendered - assert "- image-layer:0: layer=image-layer axis=1 path=virtual/image.tif" in rendered + rendered = render( + agent_capabilities.sample_viewer_window_image, + sample, + "5555", + "image-layer", + "--include-array-values", + ) + assert ( + "Viewer image sample: observed=True route=image-layer axis=channel=1 slices=[[0, 8], [0, 8]]" + in rendered + ) + assert ( + "Records: matched=1 returned=1 truncated=0 image=1 total_payloads=2 sample_supported=True" + in rendered + ) + assert ( + "- image-layer:0: layer=image-layer axis=1 path=virtual/image.tif" in rendered + ) assert "shape=[96, 96] dtype=uint16 min=23 max=10755 nonzero=9216" in rendered assert "included=True sample_shape=[8, 8]" in rendered assert "sample values: [[1, 2], [3, 4]]" in rendered @@ -308,15 +494,29 @@ def test_image_sample_includes_small_sample_values(): @pytest.mark.parametrize( "command_args,budget,expected", ( - ((), 0, "reason=array_values_not_requested max_elements=0 " - "rerun_with=--include-array-values --max-array-elements 64"), - (("--include-array-values", "--max-array-elements", "40"), 40, - "reason=max_array_elements_exceeded max_elements=40 rerun_max_elements=64"), + ( + (), + 0, + "reason=array_values_not_requested max_elements=0 " + "rerun_with=--include-array-values --max-array-elements 64", + ), + ( + ("--include-array-values", "--max-array-elements", "40"), + 40, + "reason=max_array_elements_exceeded max_elements=40 rerun_max_elements=64", + ), ), ) -def test_image_sample_omission_names_the_rerun_that_includes_values(command_args, budget, expected): - rendered = render(agent_capabilities.sample_viewer_window_image, image_sample(budget), - "5555", "image-layer", *command_args) +def test_image_sample_omission_names_the_rerun_that_includes_values( + command_args, budget, expected +): + rendered = render( + agent_capabilities.sample_viewer_window_image, + image_sample(budget), + "5555", + "image-layer", + *command_args, + ) assert "axis={'channel': 1}" not in rendered assert "included=False sample_shape=[8, 8]" in rendered assert expected in rendered @@ -325,146 +525,292 @@ def test_image_sample_omission_names_the_rerun_that_includes_values(command_args def test_navigation_and_isolation_report_position_layers_and_grouped_errors(): navigation = typed( ViewerWindowNavigationResult, - **PORT, observed=True, route_key="roi-layer", visible=True, selected=True, - axis_labels=["channel", "y", "x"], current_step=[1, 0, 0], + **PORT, + observed=True, + route_key="roi-layer", + visible=True, + selected=True, + axis_labels=["channel", "y", "x"], + current_step=[1, 0, 0], active_dimension_label_route="roi-layer", available_layers=[ - {"route_key": "image-layer", "title": "Image", "visible": True, "selected": False}, - {"route_key": "roi-layer", "title": "ROIs", "visible": True, "selected": True}, + { + "route_key": "image-layer", + "title": "Image", + "visible": True, + "selected": False, + }, + { + "route_key": "roi-layer", + "title": "ROIs", + "visible": True, + "selected": True, + }, + ], + warnings=[ + {"code": "viewer_axis_missing", "message": "z_index is not mounted."} ], - warnings=[{"code": "viewer_axis_missing", "message": "z_index is not mounted."}], ) - rendered = render(agent_capabilities.navigate_viewer_window, navigation, "5555", "roi-layer") - assert "Viewer navigation: observed=True route=roi-layer visible=True selected=True" in rendered - assert "Position: axes=channel,y,x current_step=[1, 0, 0] active_route=roi-layer" in rendered - assert ('Available layers:\n- image-layer: visible=True selected=False title="Image"\n' - '- roi-layer: visible=True selected=True title="ROIs"') in rendered + rendered = render( + agent_capabilities.navigate_viewer_window, navigation, "5555", "roi-layer" + ) + assert ( + "Viewer navigation: observed=True route=roi-layer visible=True selected=True" + in rendered + ) + assert ( + "Position: axes=channel,y,x current_step=[1, 0, 0] active_route=roi-layer" + in rendered + ) + assert ( + 'Available layers:\n- image-layer: visible=True selected=False title="Image"\n' + '- roi-layer: visible=True selected=True title="ROIs"' + ) in rendered assert "Warnings:\n- viewer_axis_missing: z_index is not mounted." in rendered isolation = typed( ViewerWindowLayerIsolationResult, - **PORT, observed=True, applied=True, selected_route_key="roi-layer", - changed_route_count=2, layer_count=2, axis_labels=["channel", "y", "x"], - current_step=[1, 0, 0], visible_route_keys=["roi-layer"], - hidden_route_keys=["image-layer"], missing_route_keys=["gone"], - visible_layers=[{"route_key": "roi-layer", "title": "ROIs", - "visible": True, "selected": True}], - ) - rendered = render(agent_capabilities.isolate_viewer_window_layers, isolation, "5555", "roi-layer") - assert "Viewer isolation: observed=True applied=True selected=roi-layer changed=2 layers=2" in rendered + **PORT, + observed=True, + applied=True, + selected_route_key="roi-layer", + changed_route_count=2, + layer_count=2, + axis_labels=["channel", "y", "x"], + current_step=[1, 0, 0], + visible_route_keys=["roi-layer"], + hidden_route_keys=["image-layer"], + missing_route_keys=["gone"], + visible_layers=[ + { + "route_key": "roi-layer", + "title": "ROIs", + "visible": True, + "selected": True, + } + ], + ) + rendered = render( + agent_capabilities.isolate_viewer_window_layers, isolation, "5555", "roi-layer" + ) + assert ( + "Viewer isolation: observed=True applied=True selected=roi-layer changed=2 layers=2" + in rendered + ) assert "Visible: roi-layer\nHidden: image-layer\nMissing routes: gone" in rendered - assert 'Visible layers:\n- roi-layer: visible=True selected=True title="ROIs"' in rendered + assert ( + 'Visible layers:\n- roi-layer: visible=True selected=True title="ROIs"' + in rendered + ) failed = typed( - ViewerWindowLayerIsolationResult, **PORT, observed=False, + ViewerWindowLayerIsolationResult, + **PORT, + observed=False, selected_route_key="roi-layer", errors=[ - {"code": "viewer_window_navigation_failed", "message": "Viewer control request timed out."}, - {"code": "viewer_window_navigation_failed", "message": "Viewer control request timed out."}, - {"code": "viewer_window_state_failed", "message": "Viewer control request timed out."}, + { + "code": "viewer_window_navigation_failed", + "message": "Viewer control request timed out.", + }, + { + "code": "viewer_window_navigation_failed", + "message": "Viewer control request timed out.", + }, + { + "code": "viewer_window_state_failed", + "message": "Viewer control request timed out.", + }, ], ) - rendered = render(agent_capabilities.isolate_viewer_window_layers, failed, "5555", "roi-layer") - assert ("- viewer_window_navigation_failed, viewer_window_state_failed: " - "Viewer control request timed out.") in rendered + rendered = render( + agent_capabilities.isolate_viewer_window_layers, failed, "5555", "roi-layer" + ) + assert ( + "- viewer_window_navigation_failed, viewer_window_state_failed: " + "Viewer control request timed out." + ) in rendered assert rendered.count("Viewer control request timed out.") == 1 def test_snapshots_render_image_and_resource_facts(): - resource = {"path": "/tmp/snapshots/viewer.png", "uri": "file:///tmp/snapshots/viewer.png", - "title": "viewer.png", "mime_type": "image/png", "size_bytes": 12345, - "sha256": "abc123"} + resource = { + "path": "/tmp/snapshots/viewer.png", + "uri": "file:///tmp/snapshots/viewer.png", + "title": "viewer.png", + "mime_type": "image/png", + "size_bytes": 12345, + "sha256": "abc123", + } viewer = typed( - ViewerWindowSnapshotResult, **PORT, output_dir_path="/tmp/snapshots", - captured=True, capture_scope="widget", + ViewerWindowSnapshotResult, + **PORT, + output_dir_path="/tmp/snapshots", + captured=True, + capture_scope="widget", viewer={"viewer_type": "napari", "title": "Viewer"}, - width=640, height=480, resource=resource, + width=640, + height=480, + resource=resource, ) rendered = render(agent_capabilities.viewer_snapshot_window, viewer, "5555") - assert 'Viewer snapshot: captured=True type=napari title="Viewer" scope=widget' in rendered + assert ( + 'Viewer snapshot: captured=True type=napari title="Viewer" scope=widget' + in rendered + ) assert "Image: size=640x480 bytes=12345 mime=image/png" in rendered - assert ("Resource: path=/tmp/snapshots/viewer.png " - "uri=file:///tmp/snapshots/viewer.png sha256=abc123") in rendered + assert ( + "Resource: path=/tmp/snapshots/viewer.png " + "uri=file:///tmp/snapshots/viewer.png sha256=abc123" + ) in rendered window = typed( - UiWindowSnapshotResult, output_dir_path="/tmp/snapshots", - captured=True, window_id="global_config", capture_scope="window", - width=550, height=600, resource=resource, + UiWindowSnapshotResult, + output_dir_path="/tmp/snapshots", + captured=True, + window_id="global_config", + capture_scope="window", + width=550, + height=600, + resource=resource, summary={ - "schema_version": SCHEMA_VERSION, "identity": {"window_id": "global_config"}, - "title": "Configuration - GlobalPipelineConfig", "window_kind": "scope", - "visible": True, "focusable": True, "signature_diff": True, - "signature_diff_field_count": 3, "semantic_markers": ["_"], + "schema_version": SCHEMA_VERSION, + "identity": {"window_id": "global_config"}, + "title": "Configuration - GlobalPipelineConfig", + "window_kind": "scope", + "visible": True, + "focusable": True, + "signature_diff": True, + "signature_diff_field_count": 3, + "semantic_markers": ["_"], "object_state_scope_id": "global_config", "managed_action_ids": ["save_and_close", "save_without_close"], }, ) capability = agent_capabilities.ui_snapshot_window rendered = render(capability, window, "global_config") - assert ("Window snapshot: captured=True window=global_config " - 'title="Configuration - GlobalPipelineConfig" kind=scope scope=window') in rendered - assert ("Status: visible=True dirty=False dirty_fields=0 default_diff=True " - "default_diff_fields=3 markers=_") in rendered + assert ( + "Window snapshot: captured=True window=global_config " + 'title="Configuration - GlobalPipelineConfig" kind=scope scope=window' + ) in rendered + assert ( + "Status: visible=True dirty=False dirty_fields=0 default_diff=True " + "default_diff_fields=3 markers=_" + ) in rendered assert "Image: size=550x600 bytes=12345 mime=image/png" in rendered assert "ObjectState: scope=global_config" in rendered assert "Actions: save_and_close,save_without_close" in rendered call_args = _build_parser().parse_args( - ("call", capability.name, "--arguments", '{"window_id":"global_config"}')) - call_rendered = McpDevCommandSpec.for_name("call").render_result(batch(capability, window), call_args) + ("call", capability.name, "--arguments", '{"window_id":"global_config"}') + ) + call_rendered = McpDevCommandSpec.for_name("call").render_result( + batch(capability, window), call_args + ) assert call_rendered == rendered assert '"payloads"' not in call_rendered def test_runtime_scan_info_and_status_render_server_facts(): - napari = dict(schema_version=SCHEMA_VERSION, connection={"port": 5555}, - server="NapariViewer", reachable=True, ready=True, control_port=6555, - log_file_path="/tmp/napari.log") - execution = dict(schema_version=SCHEMA_VERSION, connection={"port": 7777}, - server="ZMQExecutionServer", reachable=True, ready=True, - control_port=8777, active_executions=1, uptime=12.34, - log_file_path="/tmp/exec.log") - scan = typed(RuntimeServerScanResult, ports=[5555, 7777], timeout_ms=300, - servers=[napari, execution]) + napari = dict( + schema_version=SCHEMA_VERSION, + connection={"port": 5555}, + server="NapariViewer", + reachable=True, + ready=True, + control_port=6555, + log_file_path="/tmp/napari.log", + ) + execution = dict( + schema_version=SCHEMA_VERSION, + connection={"port": 7777}, + server="ZMQExecutionServer", + reachable=True, + ready=True, + control_port=8777, + active_executions=1, + uptime=12.34, + log_file_path="/tmp/exec.log", + ) + scan = typed( + RuntimeServerScanResult, + ports=[5555, 7777], + timeout_ms=300, + servers=[napari, execution], + ) rendered = render(agent_capabilities.scan_runtime_servers, scan, "5555", "7777") assert "Runtime scan: ports=5555,7777 timeout_ms=300 servers=2" in rendered - assert "port=5555 server=NapariViewer reachable=True ready=True control=6555" in rendered - assert ("port=7777 server=ZMQExecutionServer reachable=True ready=True control=8777 " - "active=1 running=0 queued=0 workers=0 uptime=12.3s log=/tmp/exec.log") in rendered + assert ( + "port=5555 server=NapariViewer reachable=True ready=True control=6555" + in rendered + ) + assert ( + "port=7777 server=ZMQExecutionServer reachable=True ready=True control=8777 " + "active=1 running=0 queued=0 workers=0 uptime=12.3s log=/tmp/exec.log" + ) in rendered unreachable = RuntimeServerInfo( - schema_version=SCHEMA_VERSION, connection=ExecutionConnectionSpec(port=7777), - reachable=False, errors=(AgentError(code="runtime_unreachable", message="no pong"),), + schema_version=SCHEMA_VERSION, + connection=ExecutionConnectionSpec(port=7777), + reachable=False, + errors=(AgentError(code="runtime_unreachable", message="no pong"),), ) rendered = McpDevOutputRenderer.for_output_contract(RuntimeServerInfo).render( - batch(agent_capabilities.get_runtime_server_info, unreachable)) - assert rendered.startswith("Runtime server:\n- port=7777 server= reachable=False") + batch(agent_capabilities.get_runtime_server_info, unreachable) + ) + assert rendered.startswith( + "Runtime server:\n- port=7777 server= reachable=False" + ) assert rendered.endswith("Errors:\n- runtime_unreachable: no pong") status = RuntimeExecutionStatus( - schema_version=SCHEMA_VERSION, connection=ExecutionConnectionSpec(port=7777), - execution_id="run-1", status="ok", - response={"status": "ok", "active_executions": 1, "uptime": 12.34, - "executions": ["run-1", "run-2"], "queued_executions": [], - "running_executions": [{"execution_id": "run-1", "subject_id": "p", - "start_time": 1.0, "elapsed": 2.0}]}, + schema_version=SCHEMA_VERSION, + connection=ExecutionConnectionSpec(port=7777), + execution_id="run-1", + status="ok", + response={ + "status": "ok", + "active_executions": 1, + "uptime": 12.34, + "executions": ["run-1", "run-2"], + "queued_executions": [], + "running_executions": [ + { + "execution_id": "run-1", + "subject_id": "p", + "start_time": 1.0, + "elapsed": 2.0, + } + ], + }, ) rendered = RuntimeExecutionStatusRenderer.render( - batch(agent_capabilities.get_runtime_server_execution_status, status)) - assert "Runtime execution status: status=ok execution_id=run-1 port=7777" in rendered + batch(agent_capabilities.get_runtime_server_execution_status, status) + ) + assert ( + "Runtime execution status: status=ok execution_id=run-1 port=7777" in rendered + ) assert "Executions: known=2 active=1 running=1 queued=0 uptime=12.3s" in rendered def test_probe_presents_false_zero_and_empty_facts(): probe = ViewerWindowProbeResult( - schema_version=SCHEMA_VERSION, connection=ExecutionConnectionSpec(port=5555), - reachable=True, observed=True, layer_count=2, component_group_count=2, + schema_version=SCHEMA_VERSION, + connection=ExecutionConnectionSpec(port=5555), + reachable=True, + observed=True, + layer_count=2, + component_group_count=2, component_item_count=4, - viewer=ViewerWindowDescriptor(ViewerType.NAPARI, "OpenHCS Napari Visualization"), + viewer=ViewerWindowDescriptor( + ViewerType.NAPARI, "OpenHCS Napari Visualization" + ), warnings=(AgentWarning(code="notice", message="shown"),), ) rendered = render(agent_capabilities.probe_viewer_window, probe, "5555") - assert ('Viewer probe: reachable=True observed=True type=napari ' - 'title="OpenHCS Napari Visualization"') in rendered + assert ( + "Viewer probe: reachable=True observed=True type=napari " + 'title="OpenHCS Napari Visualization"' + ) in rendered assert "Window: port=5555 layers=2 component_groups=2 component_items=4" in rendered assert rendered.endswith("Warnings:\n- notice: shown") @@ -476,7 +822,12 @@ def test_viewer_renderers_read_no_record_by_key(): import openhcs.mcp.dev_client_renderers.viewer as module tree = ast.parse(inspect.getsource(module)) - calls = [node for node in ast.walk(tree) if isinstance(node, ast.Call) - and isinstance(node.func, ast.Attribute) and node.func.attr == "get"] + calls = [ + node + for node in ast.walk(tree) + if isinstance(node, ast.Call) + and isinstance(node.func, ast.Attribute) + and node.func.attr == "get" + ] assert calls == [] assert issubclass(module.ViewerProbeRenderer, ViewerResultRenderer) diff --git a/tests/unit/agent/test_mcp_plate_stream_progress.py b/tests/unit/agent/test_mcp_plate_stream_progress.py index 9c71292a29..fb954b6a45 100644 --- a/tests/unit/agent/test_mcp_plate_stream_progress.py +++ b/tests/unit/agent/test_mcp_plate_stream_progress.py @@ -18,8 +18,10 @@ from polystore.streaming.viewer_transport import ViewerStreamKwarg from openhcs.agent.capabilities import ( - AgentCapabilityDeclaration, MainThreadProgressCapability, - StreamPlateFilesToViewerCapability, UiStreamSelectedPlateFilesToViewerCapability, + AgentCapabilityDeclaration, + MainThreadProgressCapability, + StreamPlateFilesToViewerCapability, + UiStreamSelectedPlateFilesToViewerCapability, ) from openhcs.agent.services.plate_streaming_service import PlateStreamingService from openhcs.core.roi_source_metadata import ROIArchiveSourceMetadata @@ -30,7 +32,10 @@ from openhcs.mcp.server import build_server from openhcs.mcp.socket import McpSocketTransport, wait_for_socket -from tests.unit.agent.test_persisted_output_reopen import NoRuntimeBridge, declared_output +from tests.unit.agent.test_persisted_output_reopen import ( + NoRuntimeBridge, + declared_output, +) from tests.unit.test_streaming_service import FakeViewer @@ -44,19 +49,26 @@ def saved_roi_stream(tmp_path, monkeypatch): archive = result_root / "not-an-acquisition.roi.zip" geometry = [ROI([PointShape(12.5, 8.25)], {"label": 7})] DiskStorageBackend().save( - ROIArchiveSourceMetadata.bind(geometry, projection.image_metadata), archive, + ROIArchiveSourceMetadata.bind(geometry, projection.image_metadata), + archive, ) persisted = ROIArchiveSourceMetadata.decode(DiskStorageBackend().load(archive)) assert persisted.source_provenance == projection.image_metadata.source_provenance - assert persisted.source_voxel_spacing == projection.image_metadata.source_voxel_spacing + assert ( + persisted.source_voxel_spacing == projection.image_metadata.source_voxel_spacing + ) unsigned = result_root / "missing-source.roi.zip" DiskStorageBackend().save(geometry, unsigned) # Reuse the original path authority, explicitly adding this synthetic root. from openhcs.agent.path_policy import AgentPathPolicy from openhcs.agent.services.plate_inspection_service import PlateInspectionService - inspection = PlateInspectionService(AgentPathPolicy.with_roots( - readable_roots=(tmp_path,), writable_roots=(), - )) + + inspection = PlateInspectionService( + AgentPathPolicy.with_roots( + readable_roots=(tmp_path,), + writable_roots=(), + ) + ) viewer = FakeViewer() received = [] timings = [] @@ -75,12 +87,21 @@ def receive(_manager, data, paths, backend, **fields): restored = ImagePayloadMetadata.from_viewer_image_metadata( request.source.item_fields["image_metadata"] ) - assert restored.to_viewer_image_metadata() == projection.image_metadata.to_viewer_image_metadata() + assert ( + restored.to_viewer_image_metadata() + == projection.image_metadata.to_viewer_image_metadata() + ) assert request.source.metadata.metadata_by_path[str(archive)] == dict( persisted.source_provenance.scalar_source_identity.component_metadata, ) - assert restored.source_voxel_spacing == projection.image_metadata.source_voxel_spacing - assert restored.source_spatial_domain == projection.image_metadata.source_spatial_domain + assert ( + restored.source_voxel_spacing + == projection.image_metadata.source_voxel_spacing + ) + assert ( + restored.source_spatial_domain + == projection.image_metadata.source_spatial_domain + ) # The archive travels as saved: its geometry plus the bound source # declaration, which the viewer reads and keeps out of the feature table. assert [ROIArchiveSourceMetadata.geometry(rois) for rois in data] == [geometry] @@ -98,22 +119,30 @@ def settle(): return True monkeypatch.setattr( - "openhcs.agent.services.plate_streaming_service.StreamingViewerLifecycle.get_or_create_visualizer", acquire, + "openhcs.agent.services.plate_streaming_service.StreamingViewerLifecycle.get_or_create_visualizer", + acquire, ) monkeypatch.setattr(FileManager, "save_batch", receive) monkeypatch.setattr(viewer, "settle_viewer_state", settle) service = PlateStreamingService(inspection, NoRuntimeBridge()) arguments = { "plate_path": str(source_path.parent.parent), - "result_directory": str(result_root), "kind": "result", "limit": 1, - "file_paths": [archive.name], "fresh_viewer": False, + "result_directory": str(result_root), + "kind": "result", + "limit": 1, + "file_paths": [archive.name], + "fresh_viewer": False, } return service, arguments, identity, received, timings, unsigned -@pytest.mark.parametrize("declaration", ( - StreamPlateFilesToViewerCapability, UiStreamSelectedPlateFilesToViewerCapability, -)) +@pytest.mark.parametrize( + "declaration", + ( + StreamPlateFilesToViewerCapability, + UiStreamSelectedPlateFilesToViewerCapability, + ), +) def test_stream_declaration_composes_original_progress_and_main_affinity(declaration): assert issubclass(declaration, MainThreadProgressCapability) spec = declaration @@ -124,12 +153,15 @@ def test_stream_declaration_composes_original_progress_and_main_affinity(declara def test_saved_roi_continuous_sdk_receipt_error_reuse_beyond_ordinary_idle( - unix_socket_dir, saved_roi_stream, + unix_socket_dir, + saved_roi_stream, ): service, arguments, identity, received, timings, unsigned = saved_roi_stream transport = McpSocketTransport(unix_socket_dir / "stream.sock") - built = build_server(OpenHCSAgentContext(plate_streaming_service=service), - main_thread_dispatcher=transport.execution.dispatcher) + built = build_server( + OpenHCSAgentContext(plate_streaming_service=service), + main_thread_dispatcher=transport.execution.dispatcher, + ) clients = AsyncOperationExecutor(max_workers=1) diagnostics = io.StringIO() events = [] @@ -138,37 +170,72 @@ class ObservedSession(McpDevSocketSession): def record_progress_notification(self, notification): super().record_progress_notification(notification) assert threading.current_thread() is not threading.main_thread() - events.append((time.monotonic(), notification.params.progressToken, notification.params.message)) + events.append( + ( + time.monotonic(), + notification.params.progressToken, + notification.params.message, + ) + ) async def exercise(): - assert await asyncio.to_thread(wait_for_socket, transport.socket_path, timeout_seconds=10) + assert await asyncio.to_thread( + wait_for_socket, transport.socket_path, timeout_seconds=10 + ) try: - async with ObservedSession(McpDevServerSpec(sys.executable), diagnostics, transport.socket_path) as session: + async with ObservedSession( + McpDevServerSpec(sys.executable), diagnostics, transport.socket_path + ) as session: await session.initialize(timeout_seconds=10) - for index, paths in enumerate((arguments["file_paths"], [unsigned.name], arguments["file_paths"])): + for index, paths in enumerate( + (arguments["file_paths"], [unsigned.name], arguments["file_paths"]) + ): before = len(events) request_id = session.request_id + 1 started = time.monotonic() - result = await session.call_tool(StreamPlateFilesToViewerCapability.name, - {**arguments, "file_paths": paths}, timeout_seconds=10) + result = await session.call_tool( + StreamPlateFilesToViewerCapability.name, + {**arguments, "file_paths": paths}, + timeout_seconds=10, + ) elapsed = time.monotonic() - started progress = events[before:] assert progress and progress[0][0] - started < 1 assert all(event[1] == request_id for event in progress) payload = result["structuredContent"] if index == 1: - assert payload["errors"][0]["code"] == "plate_file_stream_failed" - assert "Native ROI source metadata is required" in payload["errors"][0]["message"] + assert ( + payload["errors"][0]["code"] == "plate_file_stream_failed" + ) + assert ( + "Native ROI source metadata is required" + in payload["errors"][0]["message"] + ) assert len(received) == 1 else: assert payload["errors"] == [] - assert payload["streamed_roi_paths"] == [str(Path(arguments["result_directory"]) / paths[0])] + assert payload["streamed_roi_paths"] == [ + str(Path(arguments["result_directory"]) / paths[0]) + ] assert payload["streamed_image_paths"] == [] if index == 0: assert elapsed >= 12 - assert any(event[0] - started > 10 and "still running" in event[2] for event in progress) - assert any("Streaming 1 ROI file(s)" in event[2] for event in progress) - print("ROI_STREAM_TERMINAL", index, elapsed, "idle10", "token", request_id, flush=True) + assert any( + event[0] - started > 10 and "still running" in event[2] + for event in progress + ) + assert any( + "Streaming 1 ROI file(s)" in event[2] for event in progress + ) + print( + "ROI_STREAM_TERMINAL", + index, + elapsed, + "idle10", + "token", + request_id, + flush=True, + ) finally: transport._stop.set() if transport._listener is not None: @@ -185,7 +252,9 @@ async def exercise(): clients.close() -def test_independent_stream_leaf_cooperative_hook_uses_generated_consumer(saved_roi_stream): +def test_independent_stream_leaf_cooperative_hook_uses_generated_consumer( + saved_roi_stream, +): service, arguments, identity, received, timings, _unsigned = saved_roi_stream timings.append(0.0) # This declaration test does not repeat the slow experiment. hooks = [] @@ -219,15 +288,18 @@ async def report_progress(self, progress, total=None, message=None): hooks.append(message) executor = McpTransportExecutor() - built = build_server(OpenHCSAgentContext(plate_streaming_service=service), - main_thread_dispatcher=executor.dispatcher) + built = build_server( + OpenHCSAgentContext(plate_streaming_service=service), + main_thread_dispatcher=executor.dispatcher, + ) async def exercise(): token = identity.set("stream-request") try: for declaration in (IndependentStream, IndependentStreamAfter): result = await built._tool_manager.get_tool(declaration.name).fn( - mcp_context=ProgressContext(), **arguments) + mcp_context=ProgressContext(), **arguments + ) assert result["errors"] == [] assert received[-1][3] == "stream-request" finally: @@ -238,7 +310,9 @@ async def exercise(): assert len(received) == 2 assert hooks.count("enter") == hooks.count("exit") == 2 assert hooks.index("enter") < hooks.index("exit") - assert any("published and viewer settlement completed" in message for message in hooks) + assert any( + "published and viewer settlement completed" in message for message in hooks + ) finally: executor.close() for declaration in (IndependentStream, IndependentStreamAfter): diff --git a/tests/unit/agent/test_mcp_process_resource_lifetime.py b/tests/unit/agent/test_mcp_process_resource_lifetime.py index 775d4723bc..d351041c20 100644 --- a/tests/unit/agent/test_mcp_process_resource_lifetime.py +++ b/tests/unit/agent/test_mcp_process_resource_lifetime.py @@ -35,7 +35,9 @@ def shutdown_jvm(self): context.ij = Gateway() resource_patch.setattr(BioFormatsJavaContext, "_instance", context) resource_patch.setattr(backend_registry, "_cleanup_callbacks", []) - backend_registry.register_cleanup_callback(BioFormatsJavaContext.shutdown_instance) + backend_registry.register_cleanup_callback( + BioFormatsJavaContext.shutdown_instance + ) class AuditCloseCapability: def close(self): @@ -68,7 +70,10 @@ def independent_resource(): assert BioFormatsJavaContext._instance is None assert context.ij is None assert calls[:4] == [ - "cooperative enter", "gateway disposed", "JVM stopped", "independent resource" + "cooperative enter", + "gateway disposed", + "JVM stopped", + "independent resource", ] assert calls.count("cooperative exit") == int(failure is None) executor.close() @@ -79,7 +84,9 @@ def independent_resource(): def test_off_main_close_rejected_before_resource_release(monkeypatch): calls = [] - monkeypatch.setattr(backend_registry, "_cleanup_callbacks", [lambda: calls.append("closed")]) + monkeypatch.setattr( + backend_registry, "_cleanup_callbacks", [lambda: calls.append("closed")] + ) executor = McpTransportExecutor() failures = [] @@ -121,7 +128,8 @@ def test_stdio_eof_retains_exact_normal_child_exit(tmp_path, exit_code): async def journey(): session = McpDevStdioSession( - ScriptServerSpec(python_executable=sys.executable, script=script), io.StringIO() + ScriptServerSpec(python_executable=sys.executable, script=script), + io.StringIO(), ) with (tmp_path / "diagnostic.log").open("w+") as diagnostic: session.server_stderr = diagnostic @@ -145,7 +153,8 @@ def test_stdio_stuck_child_uses_unchanged_forced_reap_budget(tmp_path): async def journey(): with (tmp_path / "diagnostic.log").open("w+") as diagnostic: session = McpDevStdioSession( - ScriptServerSpec(python_executable=sys.executable, script=script), diagnostic + ScriptServerSpec(python_executable=sys.executable, script=script), + diagnostic, ) assert session.teardown_timeout_seconds == 2.0 async with session: @@ -157,7 +166,9 @@ async def journey(): @pytest.mark.parametrize("fail_cleanup", (False, True)) -def test_real_sdk_stdio_declared_resource_closes_before_child_exit(tmp_path, fail_cleanup): +def test_real_sdk_stdio_declared_resource_closes_before_child_exit( + tmp_path, fail_cleanup +): from openhcs.runtime.import_authority import OpenHCSRuntimeImportAuthority source_root = OpenHCSRuntimeImportAuthority.current().import_root @@ -175,7 +186,11 @@ def test_real_sdk_stdio_declared_resource_closes_before_child_exit(tmp_path, fai "def close_resource():\n" " assert threading.current_thread() is threading.main_thread()\n" f" Path({str(closed)!r}).write_text('main-thread resource closed')\n" - + (" raise ValueError('original declared cleanup error')\n" if fail_cleanup else "") + + ( + " raise ValueError('original declared cleanup error')\n" + if fail_cleanup + else "" + ) + "register_cleanup_callback(close_resource)\n" "with McpStdioTransport.reserve_process_stdio() as transport:\n" " transport.run(server)\n" @@ -184,21 +199,32 @@ def test_real_sdk_stdio_declared_resource_closes_before_child_exit(tmp_path, fai async def journey(): with (tmp_path / "sdk-diagnostic.log").open("w+") as diagnostic: session = McpDevStdioSession( - ScriptServerSpec(python_executable=sys.executable, script=script), diagnostic + ScriptServerSpec(python_executable=sys.executable, script=script), + diagnostic, ) async with session: process = session.require_process() await session.initialize(timeout_seconds=10) first = await session.call_tool("health", {}, timeout_seconds=10) second = await session.call_tool("health", {}, timeout_seconds=10) - assert first["structuredContent"] == second["structuredContent"] == {"ok": True} + assert ( + first["structuredContent"] + == second["structuredContent"] + == {"ok": True} + ) assert process.returncode == int(fail_cleanup) assert closed.read_text() == "main-thread resource closed" diagnostic.seek(0) text = diagnostic.read() assert "Fatal error" not in text assert ("original declared cleanup error" in text) is fail_cleanup - print("SDK_STDIO_DECLARED_RESOURCE", process.pid, process.returncode, fail_cleanup, flush=True) + print( + "SDK_STDIO_DECLARED_RESOURCE", + process.pid, + process.returncode, + fail_cleanup, + flush=True, + ) asyncio.run(journey()) @@ -228,11 +254,15 @@ def release_resource(): clients = AsyncOperationExecutor(max_workers=1) async def journey(): - assert await asyncio.to_thread(wait_for_socket, transport.socket_path, timeout_seconds=10) + assert await asyncio.to_thread( + wait_for_socket, transport.socket_path, timeout_seconds=10 + ) try: for connection in range(2): async with McpDevSocketSession( - McpDevServerSpec(sys.executable), io.StringIO(), transport.socket_path + McpDevServerSpec(sys.executable), + io.StringIO(), + transport.socket_path, ) as session: await session.initialize(timeout_seconds=10) result = await session.call_tool("health", {}, timeout_seconds=10) diff --git a/tests/unit/agent/test_mcp_server.py b/tests/unit/agent/test_mcp_server.py index 6cf70f5288..d85e82d787 100644 --- a/tests/unit/agent/test_mcp_server.py +++ b/tests/unit/agent/test_mcp_server.py @@ -110,23 +110,29 @@ from openhcs.core.streaming_config_declarations import ViewerType from openhcs.mcp.context import OpenHCSAgentContext from openhcs.runtime.import_authority import OpenHCSRuntimeImportAuthority -from openhcs.runtime.viewer_protocol import ViewerPayloadSummary, ViewerArrayValueSummary +from openhcs.runtime.viewer_protocol import ( + ViewerPayloadSummary, + ViewerArrayValueSummary, +) from openhcs.runtime.zmq_config import OPENHCS_ZMQ_CONFIG - class _OperationMethodsBridgeService: """Fake bridge service answering each operation from the method named for it.""" def invoke(self, operation, request=None, connection=None): - name = operation.name or re.sub( - r"(? str: """Return first text block from FastMCP structured or legacy direct calls.""" if hasattr(result, "content"): @@ -1222,10 +1228,7 @@ def test_mcp_ui_catalog_invocations_preserve_declared_identity( ) assert payload == to_jsonable(catalog) - assert ( - dataclass_from_mapping(capability.output_contract, payload) - == catalog - ) + assert dataclass_from_mapping(capability.output_contract, payload) == catalog def test_mcp_widget_tree_projection_can_return_full_action_fields(): @@ -2180,7 +2183,9 @@ def list_object_state_scopes(self, request, connection): field_summary("napari_streaming_config.enabled"), field_summary("napari_streaming_config.port"), field_summary("partition_filter_config.partition_filter"), - field_summary("partition_filter_config.partition_filter_mode"), + field_summary( + "partition_filter_config.partition_filter_mode" + ), ), ), ), @@ -5618,7 +5623,9 @@ def test_mcp_dev_client_object_state_set_command_projects_tool_arguments(): assert reset_call.name == "openhcs_ui_mutate_object_state_field" assert reset_call.arguments["object_state_scope_id"] == "global_config" - assert reset_call.arguments["field_path"] == "partition_filter_config.partition_filter" + assert ( + reset_call.arguments["field_path"] == "partition_filter_config.partition_filter" + ) assert reset_call.arguments["value"] is None assert reset_call.arguments["reset"] is True assert reset_call.arguments["include_field_values"] is False @@ -5873,15 +5880,29 @@ def test_mcp_dev_client_workflow_poll_terminal_state_policy(): from dataclasses import replace import openhcs.mcp.dev_client as dev_client - + idle = DatasetRowState( - scope_id="scope", name="plate", root="/plate", pipeline_path=None, - selected=True, initialized=False, compiled=False, init_pending=False, - compile_pending=False, execution_active=False, status_prefix="", - orchestrator_state=None, execution_id=None, terminal_status=None, - runtime_state=None, runtime_percent=None, queue_position=None, + scope_id="scope", + name="plate", + root="/plate", + pipeline_path=None, + selected=True, + initialized=False, + compiled=False, + init_pending=False, + compile_pending=False, + execution_active=False, + status_prefix="", + orchestrator_state=None, + execution_id=None, + terminal_status=None, + runtime_state=None, + runtime_percent=None, + queue_position=None, + ) + init_policy = dev_client.WorkflowStatePollPolicy.from_workflow_text( + "initialize_datasets" ) - init_policy = dev_client.WorkflowStatePollPolicy.from_workflow_text("initialize_datasets") compile_policy = dev_client.WorkflowStatePollPolicy.from_workflow_text( "compile_datasets" ) @@ -5904,20 +5925,42 @@ def _complete_plate_manager_state(sparse): from openhcs.agent.dto.ui_bridge import UiPlateManagerState summary = UiStateSurfaceSummary( - SCHEMA_VERSION, UiStateSurfaceIdentity(surface_id="plate_manager.state"), - "Plate Manager", True, widget_id="plate_manager", - ) - row = to_jsonable(DatasetRowState( - scope_id="scope", name="plate", root="/plate", pipeline_path=None, - selected=False, initialized=False, compiled=False, init_pending=False, - compile_pending=False, execution_active=False, status_prefix="", - orchestrator_state=None, execution_id=None, terminal_status=None, - runtime_state=None, runtime_percent=None, queue_position=None, - )) - state = to_jsonable(UiPlateManagerState( - schema_version=SCHEMA_VERSION, summary=summary, object_state_token=0, - manager_execution_state="idle", rows=(), - )) + SCHEMA_VERSION, + UiStateSurfaceIdentity(surface_id="plate_manager.state"), + "Plate Manager", + True, + widget_id="plate_manager", + ) + row = to_jsonable( + DatasetRowState( + scope_id="scope", + name="plate", + root="/plate", + pipeline_path=None, + selected=False, + initialized=False, + compiled=False, + init_pending=False, + compile_pending=False, + execution_active=False, + status_prefix="", + orchestrator_state=None, + execution_id=None, + terminal_status=None, + runtime_state=None, + runtime_percent=None, + queue_position=None, + ) + ) + state = to_jsonable( + UiPlateManagerState( + schema_version=SCHEMA_VERSION, + summary=summary, + object_state_token=0, + manager_execution_state="idle", + rows=(), + ) + ) return { **state, **sparse, @@ -5927,14 +5970,20 @@ def _complete_plate_manager_state(sparse): def _state_surface_dev_wire(dev_client): """Declare the complete wire envelope; each poll test supplies its body.""" - return to_jsonable(UiStateSurfaceDocument( - schema_version=SCHEMA_VERSION, - summary=UiStateSurfaceSummary( - SCHEMA_VERSION, UiStateSurfaceIdentity(surface_id="plate_manager.state"), - "Plate Manager", True, widget_id="plate_manager", - ), - payload_schema="openhcs.ui.plate_manager_state.v1", payload={}, - )) + return to_jsonable( + UiStateSurfaceDocument( + schema_version=SCHEMA_VERSION, + summary=UiStateSurfaceSummary( + SCHEMA_VERSION, + UiStateSurfaceIdentity(surface_id="plate_manager.state"), + "Plate Manager", + True, + widget_id="plate_manager", + ), + payload_schema="openhcs.ui.plate_manager_state.v1", + payload={}, + ) + ) def test_mcp_dev_client_workflow_poll_filters_target_scope_ids(): @@ -5946,21 +5995,23 @@ def test_mcp_dev_client_workflow_poll_filters_target_scope_ids(): payloads=( { **_state_surface_dev_wire(dev_client), - "payload": _complete_plate_manager_state({ - "manager_execution_state": "idle", - "rows": [ - { - "scope_id": "scope-a", - "compile_pending": False, - "compiled": True, - }, - { - "scope_id": "scope-b", - "compile_pending": True, - "compiled": False, - }, - ], - }) + "payload": _complete_plate_manager_state( + { + "manager_execution_state": "idle", + "rows": [ + { + "scope_id": "scope-a", + "compile_pending": False, + "compiled": True, + }, + { + "scope_id": "scope-b", + "compile_pending": True, + "compiled": False, + }, + ], + } + ), }, ), ) @@ -5992,16 +6043,18 @@ def test_mcp_dev_client_workflow_poll_waits_for_manager_finalization(): payloads=( { **_state_surface_dev_wire(dev_client), - "payload": _complete_plate_manager_state({ - "manager_execution_state": "running", - "rows": [ - { - "scope_id": "scope-a", - "compile_pending": False, - "compiled": True, - } - ], - }) + "payload": _complete_plate_manager_state( + { + "manager_execution_state": "running", + "rows": [ + { + "scope_id": "scope-a", + "compile_pending": False, + "compiled": True, + } + ], + } + ), }, ), ) @@ -6035,17 +6088,19 @@ def test_mcp_dev_client_workflow_poll_reports_failed_terminal_rows(): payloads=( { **_state_surface_dev_wire(dev_client), - "payload": _complete_plate_manager_state({ - "manager_execution_state": "idle", - "rows": [ - { - "scope_id": "scope-a", - "execution_active": False, - "queue_position": None, - "terminal_status": "failed", - } - ], - }) + "payload": _complete_plate_manager_state( + { + "manager_execution_state": "idle", + "rows": [ + { + "scope_id": "scope-a", + "execution_active": False, + "queue_position": None, + "terminal_status": "failed", + } + ], + } + ), }, ), ) @@ -6055,17 +6110,19 @@ def test_mcp_dev_client_workflow_poll_reports_failed_terminal_rows(): payloads=( { **_state_surface_dev_wire(dev_client), - "payload": _complete_plate_manager_state({ - "manager_execution_state": "idle", - "rows": [ - { - "scope_id": "scope-a", - "compile_pending": False, - "compiled": False, - "orchestrator_state": "compile_failed", - } - ], - }) + "payload": _complete_plate_manager_state( + { + "manager_execution_state": "idle", + "rows": [ + { + "scope_id": "scope-a", + "compile_pending": False, + "compiled": False, + "orchestrator_state": "compile_failed", + } + ], + } + ), }, ), ) @@ -6122,23 +6179,29 @@ async def fake_call_tool(session, call, timeout_seconds): { **_state_surface_dev_wire(dev_client), "current_revision_token": f"rev-{state_call_count}", - "payload": _complete_plate_manager_state({ - "manager_execution_state": ("idle" if compiled else "running"), - "object_state_token": state_call_count, - "rows": [ - { - "scope_id": "scope-a", - "compile_pending": False, - "compiled": compiled, - } - ], - }), + "payload": _complete_plate_manager_state( + { + "manager_execution_state": ( + "idle" if compiled else "running" + ), + "object_state_token": state_call_count, + "rows": [ + { + "scope_id": "scope-a", + "compile_pending": False, + "compiled": compiled, + } + ], + } + ), }, ), ) monkeypatch.setattr( - ui_commands, "call_mcp_tool", _answering_session_events(dev_client, fake_call_tool) + ui_commands, + "call_mcp_tool", + _answering_session_events(dev_client, fake_call_tool), ) parser = dev_client._build_parser() @@ -6251,13 +6314,17 @@ async def fake_call_tool(session, call, timeout_seconds): { **_state_surface_dev_wire(dev_client), "current_revision_token": "baseline", - "payload": _complete_plate_manager_state({"object_state_token": 1, "rows": []}), + "payload": _complete_plate_manager_state( + {"object_state_token": 1, "rows": []} + ), }, ), ) monkeypatch.setattr( - ui_commands, "call_mcp_tool", _answering_session_events(dev_client, fake_call_tool) + ui_commands, + "call_mcp_tool", + _answering_session_events(dev_client, fake_call_tool), ) args = dev_client._build_parser().parse_args( @@ -6331,13 +6398,17 @@ async def fake_call_tool(session, call, timeout_seconds): { **_state_surface_dev_wire(dev_client), "current_revision_token": "baseline", - "payload": _complete_plate_manager_state({"object_state_token": 1, "rows": []}), + "payload": _complete_plate_manager_state( + {"object_state_token": 1, "rows": []} + ), }, ), ) monkeypatch.setattr( - ui_commands, "call_mcp_tool", _answering_session_events(dev_client, fake_call_tool) + ui_commands, + "call_mcp_tool", + _answering_session_events(dev_client, fake_call_tool), ) args = dev_client._build_parser().parse_args( ("selected-workflow", "compile_datasets", "--wait") @@ -6355,9 +6426,16 @@ async def fake_call_tool(session, call, timeout_seconds): "openhcs_ui_selected_plate_workflow", "openhcs_ui_wait_for_operation_receipt", ] - assert response.results[-1].payloads[0].status is dev_client.WorkflowPollSummaryStatus.FAILED + assert ( + response.results[-1].payloads[0].status + is dev_client.WorkflowPollSummaryStatus.FAILED + ) from openhcs.mcp.dev_client_core import WorkflowPollSkipReason - assert response.results[-1].payloads[0].skip_reason is WorkflowPollSkipReason.OPERATION_RECEIPT_FAILED + + assert ( + response.results[-1].payloads[0].skip_reason + is WorkflowPollSkipReason.OPERATION_RECEIPT_FAILED + ) def test_mcp_dev_client_selected_workflow_wait_rejects_stale_terminal_state( @@ -6377,17 +6455,19 @@ def test_mcp_dev_client_selected_workflow_wait_rejects_stale_terminal_state( { **_state_surface_dev_wire(dev_client), "current_revision_token": "prior-completed-run", - "payload": _complete_plate_manager_state({ - "manager_execution_state": "idle", - "object_state_token": 1, - "rows": [ - { - "scope_id": "scope-a", - "compile_pending": False, - "compiled": True, - } - ], - }), + "payload": _complete_plate_manager_state( + { + "manager_execution_state": "idle", + "object_state_token": 1, + "rows": [ + { + "scope_id": "scope-a", + "compile_pending": False, + "compiled": True, + } + ], + } + ), }, ), ) @@ -6404,7 +6484,9 @@ async def fake_call_tool(session, call, timeout_seconds): return stale_state monkeypatch.setattr( - ui_commands, "call_mcp_tool", _answering_session_events(dev_client, fake_call_tool) + ui_commands, + "call_mcp_tool", + _answering_session_events(dev_client, fake_call_tool), ) args = dev_client._build_parser().parse_args( @@ -6454,17 +6536,19 @@ def test_mcp_dev_client_selected_workflow_accepts_idempotent_init_terminal_state { **_state_surface_dev_wire(dev_client), "current_revision_token": "already-initialized", - "payload": _complete_plate_manager_state({ - "manager_execution_state": "idle", - "object_state_token": 1, - "rows": [ - { - "scope_id": "scope-a", - "init_pending": False, - "initialized": True, - } - ], - }), + "payload": _complete_plate_manager_state( + { + "manager_execution_state": "idle", + "object_state_token": 1, + "rows": [ + { + "scope_id": "scope-a", + "init_pending": False, + "initialized": True, + } + ], + } + ), }, ), ) @@ -6485,7 +6569,9 @@ async def fake_call_tool(session, call, timeout_seconds): return initialized_state monkeypatch.setattr( - ui_commands, "call_mcp_tool", _answering_session_events(dev_client, fake_call_tool) + ui_commands, + "call_mcp_tool", + _answering_session_events(dev_client, fake_call_tool), ) args = dev_client._build_parser().parse_args( @@ -6568,23 +6654,29 @@ async def fake_call_tool(session, call, timeout_seconds): { **_state_surface_dev_wire(dev_client), "current_revision_token": f"rev-{state_call_count}", - "payload": _complete_plate_manager_state({ - "manager_execution_state": ("idle" if compiled else "running"), - "object_state_token": state_call_count, - "rows": [ - { - "scope_id": "scope-a", - "compile_pending": False, - "compiled": compiled, - } - ], - }), + "payload": _complete_plate_manager_state( + { + "manager_execution_state": ( + "idle" if compiled else "running" + ), + "object_state_token": state_call_count, + "rows": [ + { + "scope_id": "scope-a", + "compile_pending": False, + "compiled": compiled, + } + ], + } + ), }, ), ) monkeypatch.setattr( - ui_commands, "call_mcp_tool", _answering_session_events(dev_client, fake_call_tool) + ui_commands, + "call_mcp_tool", + _answering_session_events(dev_client, fake_call_tool), ) parser = dev_client._build_parser() @@ -6678,23 +6770,27 @@ async def fake_call_tool(session, call, timeout_seconds): { **_state_surface_dev_wire(dev_client), "current_revision_token": "terminal", - "payload": _complete_plate_manager_state({ - "manager_execution_state": "idle", - "object_state_token": 2, - "rows": [ - { - "scope_id": "scope-a", - "compile_pending": False, - "compiled": True, - } - ], - }), + "payload": _complete_plate_manager_state( + { + "manager_execution_state": "idle", + "object_state_token": 2, + "rows": [ + { + "scope_id": "scope-a", + "compile_pending": False, + "compiled": True, + } + ], + } + ), }, ), ) monkeypatch.setattr( - ui_commands, "call_mcp_tool", _answering_session_events(dev_client, fake_call_tool) + ui_commands, + "call_mcp_tool", + _answering_session_events(dev_client, fake_call_tool), ) parser = dev_client._build_parser() @@ -6772,7 +6868,9 @@ async def fake_call_tool(session, call, timeout_seconds): { **_state_surface_dev_wire(dev_client), "current_revision_token": "baseline", - "payload": _complete_plate_manager_state({"object_state_token": 1, "rows": []}), + "payload": _complete_plate_manager_state( + {"object_state_token": 1, "rows": []} + ), }, ), ) @@ -6792,7 +6890,9 @@ async def fake_call_tool(session, call, timeout_seconds): ) monkeypatch.setattr( - ui_commands, "call_mcp_tool", _answering_session_events(dev_client, fake_call_tool) + ui_commands, + "call_mcp_tool", + _answering_session_events(dev_client, fake_call_tool), ) parser = dev_client._build_parser() @@ -6865,27 +6965,33 @@ async def fake_call_tool(session, call, timeout_seconds): { **_state_surface_dev_wire(dev_client), "current_revision_token": f"rev-{state_call_count}", - "payload": _complete_plate_manager_state({ - "manager_execution_state": ("idle" if failed else "running"), - "object_state_token": state_call_count, - "rows": [ - { - "scope_id": "scope-a", - "execution_active": False, - "queue_position": None, - "terminal_status": ("failed" if failed else None), - "orchestrator_state": ( - "exec_failed" if failed else "compiled" - ), - } - ], - }), + "payload": _complete_plate_manager_state( + { + "manager_execution_state": ( + "idle" if failed else "running" + ), + "object_state_token": state_call_count, + "rows": [ + { + "scope_id": "scope-a", + "execution_active": False, + "queue_position": None, + "terminal_status": ("failed" if failed else None), + "orchestrator_state": ( + "exec_failed" if failed else "compiled" + ), + } + ], + } + ), }, ), ) monkeypatch.setattr( - ui_commands, "call_mcp_tool", _answering_session_events(dev_client, fake_call_tool) + ui_commands, + "call_mcp_tool", + _answering_session_events(dev_client, fake_call_tool), ) parser = dev_client._build_parser() @@ -6955,10 +7061,12 @@ async def fake_call_tool(session, call, timeout_seconds): { **_state_surface_dev_wire(dev_client), "current_revision_token": "rev-1", - "payload": _complete_plate_manager_state({ - "object_state_token": 1, - "rows": [], - }), + "payload": _complete_plate_manager_state( + { + "object_state_token": 1, + "rows": [], + } + ), }, ), ) @@ -6978,7 +7086,9 @@ async def fake_call_tool(session, call, timeout_seconds): ) monkeypatch.setattr( - ui_commands, "call_mcp_tool", _answering_session_events(dev_client, fake_call_tool) + ui_commands, + "call_mcp_tool", + _answering_session_events(dev_client, fake_call_tool), ) parser = dev_client._build_parser() @@ -7026,22 +7136,33 @@ def test_mcp_dev_client_selected_workflow_poll_summarizes_rejection( import openhcs.mcp.dev_client as dev_client import openhcs.mcp.dev_client_commands.ui as ui_commands from openhcs.agent.dto.ui_bridge import ( - UiPlateManagerState, UiStateSurfaceDocument, - UiStateSurfaceIdentity, UiStateSurfaceSummary, + UiPlateManagerState, + UiStateSurfaceDocument, + UiStateSurfaceIdentity, + UiStateSurfaceSummary, ) surface_summary = UiStateSurfaceSummary( - "test", UiStateSurfaceIdentity(surface_id="plate_manager.state"), - "Plate Manager", True, widget_id="plate_manager", + "test", + UiStateSurfaceIdentity(surface_id="plate_manager.state"), + "Plate Manager", + True, + widget_id="plate_manager", ) baseline = UiPlateManagerState( - schema_version="test", summary=surface_summary, object_state_token=1, - manager_execution_state="idle", rows=(), current_revision_token="rev-1", + schema_version="test", + summary=surface_summary, + object_state_token=1, + manager_execution_state="idle", + rows=(), + current_revision_token="rev-1", ) baseline_document = UiStateSurfaceDocument( - schema_version="test", summary=surface_summary, + schema_version="test", + summary=surface_summary, payload_schema="openhcs.ui.plate_manager_state.v1", - payload=to_jsonable(baseline), current_revision_token="rev-1", + payload=to_jsonable(baseline), + current_revision_token="rev-1", ) calls: list[dev_client.McpDevToolCall] = [] @@ -7060,7 +7181,9 @@ async def fake_call_tool(session, call, timeout_seconds): ) monkeypatch.setattr( - ui_commands, "call_mcp_tool", _answering_session_events(dev_client, fake_call_tool) + ui_commands, + "call_mcp_tool", + _answering_session_events(dev_client, fake_call_tool), ) parser = dev_client._build_parser() @@ -7099,9 +7222,9 @@ async def fake_call_tool(session, call, timeout_seconds): "action_status": "rejected", } - rendered = dev_client.McpDevCommandSpec.for_name( - "selected-workflow" - ).render_result(response, args) + rendered = dev_client.McpDevCommandSpec.for_name("selected-workflow").render_result( + response, args + ) assert "Workflow: compile_datasets" in rendered assert "Skip reason: workflow_not_accepted" in rendered @@ -7947,7 +8070,8 @@ def window_payloads(self, request): path="/tmp/A01_w2_rois.roi.zip", components={"well": "A01", "channel": 2}, summary=ViewerPayloadSummary( - shape_payload_count=3, shape_coordinate_count=12, + shape_payload_count=3, + shape_coordinate_count=12, ), shape_payloads=( { @@ -8045,10 +8169,15 @@ def window_payloads(self, request): components={"well": "A01", "channel": 1}, axis_indices=(index,), summary=ViewerPayloadSummary( - shape=(96, 96), dtype="uint16", min=1, max=4, + shape=(96, 96), + dtype="uint16", + min=1, + max=4, nonzero_count=10, ), - array_value_summary=ViewerArrayValueSummary(requested=True, included=False), + array_value_summary=ViewerArrayValueSummary( + requested=True, included=False + ), ) for index in range(5) ), @@ -8123,7 +8252,9 @@ def window_payloads(self, request): components={"well": "A01"}, axis_indices=(0,), summary=ViewerPayloadSummary(shape=(8, 8)), - array_value_summary=ViewerArrayValueSummary(requested=True, included=False), + array_value_summary=ViewerArrayValueSummary( + requested=True, included=False + ), ), ), ), @@ -8140,7 +8271,9 @@ def window_payloads(self, request): components={"well": "A01"}, axis_indices=(0,), summary=ViewerPayloadSummary(shape=(8, 8)), - array_value_summary=ViewerArrayValueSummary(requested=True, included=False), + array_value_summary=ViewerArrayValueSummary( + requested=True, included=False + ), ), ), ), @@ -8204,7 +8337,9 @@ def window_payloads(self, request): components={"well": "A01"}, axis_indices=(0,), summary=ViewerPayloadSummary(shape=(8, 8)), - array_value_summary=ViewerArrayValueSummary(requested=True, included=False), + array_value_summary=ViewerArrayValueSummary( + requested=True, included=False + ), ), ), ), @@ -8221,7 +8356,9 @@ def window_payloads(self, request): components={"well": "A01"}, axis_indices=(0,), summary=ViewerPayloadSummary(shape=(8, 8)), - array_value_summary=ViewerArrayValueSummary(requested=True, included=False), + array_value_summary=ViewerArrayValueSummary( + requested=True, included=False + ), ), ), ), @@ -8720,7 +8857,10 @@ def test_mcp_bootstrap_wraps_server_run_failure(monkeypatch): class FailingRunServer: def __init__(self, *args, **kwargs): - assert kwargs["main_thread_dispatcher"] is recording_transport.execution.dispatcher + assert ( + kwargs["main_thread_dispatcher"] + is recording_transport.execution.dispatcher + ) def run(self, *, transport: str) -> None: transports.append(transport) @@ -8837,9 +8977,14 @@ def test_mcp_server_exposes_session_tools_and_no_job_tools(): tool_names = {tool.name for tool in tools} for operation_id in ( - "add_datasets", "connect_server", "set_dataset_pipeline", - "initialize_datasets", "compile_datasets", "run_datasets", - "stop_execution", "delete_datasets", + "add_datasets", + "connect_server", + "set_dataset_pipeline", + "initialize_datasets", + "compile_datasets", + "run_datasets", + "stop_execution", + "delete_datasets", ): assert f"openhcs_{operation_id}" in tool_names for state in ("datasets", "pipeline", "events"): @@ -9001,9 +9146,7 @@ def test_mcp_run_datasets_projects_observation_export_request(): tools = ( asyncio.run(listed_tools) if inspect.isawaitable(listed_tools) else listed_tools ) - schema = {tool.name: tool.inputSchema for tool in tools}[ - "openhcs_run_datasets" - ] + schema = {tool.name: tool.inputSchema for tool in tools}["openhcs_run_datasets"] assert "runtime_observation_export_path" in schema["properties"] assert "runtime_observation_export_path" not in schema["required"] diff --git a/tests/unit/agent/test_physical_source_inspection_445.py b/tests/unit/agent/test_physical_source_inspection_445.py index 7617b3e176..e9173034df 100644 --- a/tests/unit/agent/test_physical_source_inspection_445.py +++ b/tests/unit/agent/test_physical_source_inspection_445.py @@ -49,7 +49,10 @@ def source_dataset(self, plate_path): return replace( dataset, candidates=tuple( - replace(candidate, metadata={**candidate.metadata, "inspection_witness": "declared"}) + replace( + candidate, + metadata={**candidate.metadata, "inspection_witness": "declared"}, + ) for candidate in dataset.candidates ), ) @@ -62,7 +65,9 @@ def _fixture(root: Path, channels: int): manifest_path = root / "bioformats_spw.json" manifest = json.loads(manifest_path.read_text()) image = manifest["images"][0] - image["channel_names"] = [f"engineering-{channel}" for channel in range(1, channels + 1)] + image["channel_names"] = [ + f"engineering-{channel}" for channel in range(1, channels + 1) + ] image["pixels"]["size_c"] = channels image["pixels"]["planes"] = [ {"c": channel, "z": 1, "t": 1, "index": channel - 1} @@ -106,11 +111,15 @@ def _sample(service, root, image_path, *, source_format="bioformats"): @pytest.mark.parametrize("channels", (3, 4)) -def test_original_physical_candidates_are_sampleable_before_projection(tmp_path, channels): +def test_original_physical_candidates_are_sampleable_before_projection( + tmp_path, channels +): stack, service, _ = _fixture(tmp_path, channels) query = _query(service, tmp_path) assert query.total_count == channels - record = next(record for record in query.records if int(record.metadata["channel"]) == 3) + record = next( + record for record in query.records if int(record.metadata["channel"]) == 3 + ) sample = _sample(service, tmp_path, record.virtual_path) assert int(sample.source_metadata["channel"]) == 3 assert sample.sample_values == stack[0, 0, 2, 2:4, 3:5].tolist() @@ -129,14 +138,16 @@ def test_physical_c3_remains_available_after_c1_only_preparation(tmp_path, chann ) ) filemanager = factory.create() - BioFormatsHandler(filemanager, source_bindings_config=bindings).initialize_workspace( - tmp_path, filemanager - ) + BioFormatsHandler( + filemanager, source_bindings_config=bindings + ).initialize_workspace(tmp_path, filemanager) metadata_path = tmp_path / "openhcs_metadata.json" before = metadata_path.read_bytes() persisted = json.loads(before)["subdirectories"]["."] assert len(persisted["image_files"]) == 1 - assert {int(value["channel"]) for value in persisted["source_metadata"].values()} == {1} + assert { + int(value["channel"]) for value in persisted["source_metadata"].values() + } == {1} try: # Ordinary auto/workspace routing must keep the persisted C1 selection. selected = _query(service, tmp_path, source_format="auto") @@ -149,14 +160,18 @@ def test_physical_c3_remains_available_after_c1_only_preparation(tmp_path, chann assert sample.sample_values == stack[0, 0, 0, 2:4, 3:5].tolist() query = _query(service, tmp_path) - observed_channels = {int(record.metadata["channel"]) for record in query.records} + observed_channels = { + int(record.metadata["channel"]) for record in query.records + } # No invented C3 path, receipt, source-ref override, or projection mutation. assert 3 in observed_channels, ( "Explicit physical-source inspection hides C3 after C1 preparation: " f"observed_channels={sorted(observed_channels)}, total_count={query.total_count}" ) assert observed_channels == set(range(1, channels + 1)) - record = next(record for record in query.records if int(record.metadata["channel"]) == 3) + record = next( + record for record in query.records if int(record.metadata["channel"]) == 3 + ) physical = _sample(service, tmp_path, record.virtual_path) assert int(physical.source_metadata["channel"]) == 3 assert physical.sample_values == stack[0, 0, 2, 2:4, 3:5].tolist() @@ -187,7 +202,9 @@ def test_physical_c3_remains_available_after_c1_only_preparation(tmp_path, chann from openhcs.core.viewer_streaming_service import ViewerStreamingSource context, errors, _ = service.open_context( - PlatePathInspectionRequest(plate_path=str(tmp_path), source_format="bioformats") + PlatePathInspectionRequest( + plate_path=str(tmp_path), source_format="bioformats" + ) ) assert errors == () inventory, warnings = service.file_inventory(context, kind=PlateFileKind.IMAGE) @@ -241,7 +258,14 @@ def test_independent_metadata_hook_flows_through_unchanged_inventory_consumer(tm inventory = service._image_inventory(handler, tmp_path, filemanager, warnings) assert warnings == [] assert len(inventory.records) == 4 - assert {record.metadata["inspection_witness"] for record in inventory.records} == {"declared"} - assert {int(record.metadata["channel"]) for record in inventory.records} == {1, 2, 3, 4} + assert {record.metadata["inspection_witness"] for record in inventory.records} == { + "declared" + } + assert {int(record.metadata["channel"]) for record in inventory.records} == { + 1, + 2, + 3, + 4, + } assert all(record.source_ref is not None for record in inventory.records) assert not (tmp_path / "openhcs_metadata.json").exists() diff --git a/tests/unit/agent/test_plate_inspection_cold_runtime.py b/tests/unit/agent/test_plate_inspection_cold_runtime.py index dcebf0fa47..770ff78be4 100644 --- a/tests/unit/agent/test_plate_inspection_cold_runtime.py +++ b/tests/unit/agent/test_plate_inspection_cold_runtime.py @@ -87,9 +87,9 @@ def inspect(self, request): built.call_tool("openhcs_list_capabilities", {}), 0.5 ) assert result is not None - assert not pending.done(), ( - "Discovery must finish while inspection is still held" - ) + assert ( + not pending.done() + ), "Discovery must finish while inspection is still held" finally: release.set() await asyncio.wait_for(pending, 1) diff --git a/tests/unit/agent/test_plate_inspection_service.py b/tests/unit/agent/test_plate_inspection_service.py index 66cb9b206f..60071768c3 100644 --- a/tests/unit/agent/test_plate_inspection_service.py +++ b/tests/unit/agent/test_plate_inspection_service.py @@ -430,14 +430,8 @@ def test_registered_handler_selection_roles_are_owned_polymorphically() -> None: from openhcs.microscopes.imagexpress import ImageXpressHandler from openhcs.microscopes.opera_phenix import OperaPhenixHandler - assert ( - BioFormatsHandler.source_selection_role() - is BroadStoreSource - ) - assert ( - SourceBindingsSource.source_selection_role() - is DeclaredFileSource - ) + assert BioFormatsHandler.source_selection_role() is BroadStoreSource + assert SourceBindingsSource.source_selection_role() is DeclaredFileSource assert "structured or rich container" in ( BioFormatsHandler.source_selection_guidance() ) @@ -1254,7 +1248,8 @@ def test_plate_inspection_reads_no_main_openhcs_output_subdirectories( for image_name in image_names ], "source_metadata": _calibrated_source_metadata( - f"checkpoints_step1/{image_name}" for image_name in image_names + f"checkpoints_step1/{image_name}" + for image_name in image_names ), "results_dir": "checkpoints_step1_results", }, @@ -1265,7 +1260,8 @@ def test_plate_inspection_reads_no_main_openhcs_output_subdirectories( for image_name in image_names ], "source_metadata": _calibrated_source_metadata( - f"checkpoints_step2/{image_name}" for image_name in image_names + f"checkpoints_step2/{image_name}" + for image_name in image_names ), "results_dir": "checkpoints_step2_results", }, diff --git a/tests/unit/agent/test_plate_streaming_service.py b/tests/unit/agent/test_plate_streaming_service.py index 8e4b895f86..b73e832abb 100644 --- a/tests/unit/agent/test_plate_streaming_service.py +++ b/tests/unit/agent/test_plate_streaming_service.py @@ -51,7 +51,6 @@ ) from openhcs.domains.microscopy.axes import Microscopy - # A metadata owner that anchors workspace-relative addresses at the plate. PLATE_ROOT_METADATA = SimpleNamespace(source_workspace_root=Path) @@ -72,19 +71,28 @@ def test_saved_site_free_image_inventory_and_loading_preserve_original_scope(tmp from openhcs.core.plate_image_inventory import PlateImageInventory from openhcs.core.runtime_image_values import ImagePayloadMetadata from openhcs.core.source_image_provenance import ( - SourceImageIdentity, SourceImageProvenance, SourceImageProvenanceContributor, + SourceImageIdentity, + SourceImageProvenance, + SourceImageProvenanceContributor, ) from openhcs.core.source_projection import SourceArtifactProjection from openhcs.core.source_metadata import SourceVoxelSpacing - from openhcs.core.source_workspace_projection import VirtualWorkspaceSourceProjectionBuilder - from openhcs.core.virtual_workspace_metadata import VirtualWorkspaceSourceProjectionEntries + from openhcs.core.source_workspace_projection import ( + VirtualWorkspaceSourceProjectionBuilder, + ) + from openhcs.core.virtual_workspace_metadata import ( + VirtualWorkspaceSourceProjectionEntries, + ) from openhcs.core.dataset_sources.source_schema import SourceSchemaFilenameParser filename = "A01_s001_w1_z001_t001.tif" - pixels = np.concatenate(( - np.full((4, 6), 11, dtype=np.uint16), - np.full((4, 6), 13, dtype=np.uint16), - ), axis=1) + pixels = np.concatenate( + ( + np.full((4, 6), 11, dtype=np.uint16), + np.full((4, 6), 13, dtype=np.uint16), + ), + axis=1, + ) tifffile.imwrite(tmp_path / filename, pixels) components = {"well": "A01", "channel": "1", "z_index": "1", "timepoint": "1"} contributors = tuple( @@ -93,7 +101,8 @@ def test_saved_site_free_image_inventory_and_loading_preserve_original_scope(tmp str(tmp_path / f"original-s{site}.tif"), {**components, "site": site}, ), - ) for site in ("1", "3") + ) + for site in ("1", "3") ) metadata = ImagePayloadMetadata( source_provenance=SourceImageProvenance( @@ -104,45 +113,72 @@ def test_saved_site_free_image_inventory_and_loading_preserve_original_scope(tmp ) ref = SourcePixelRef(backend="disk", backend_address=filename) source = SourceArtifactProjection( - address=None, ref=ref, source_alias="Signal", artifact_kind=ImageArtifactType, - source_metadata=components, image_metadata=metadata, + address=None, + ref=ref, + source_alias="Signal", + artifact_kind=ImageArtifactType, + source_metadata=components, + image_metadata=metadata, execution_scope=RuntimeExecutionAxisScope.from_raw( - "A01", component=Microscopy.Channel, value="1", - fixed_component_values=((Microscopy.ZIndex, "1"), (Microscopy.Timepoint, "1")), + "A01", + component=Microscopy.Channel, + value="1", + fixed_component_values=( + (Microscopy.ZIndex, "1"), + (Microscopy.Timepoint, "1"), + ), ), ) builder = VirtualWorkspaceSourceProjectionBuilder(tmp_path) builder.record_workspace_source_path(filename, ref) builder.record_source_metadata(filename, components) - builder.ingest_source_projections(VirtualWorkspaceSourceProjectionEntries({filename: source})) + builder.ingest_source_projections( + VirtualWorkspaceSourceProjectionEntries({filename: source}) + ) filemanager = FileManager({"disk": DiskStorageBackend()}) handler = SimpleNamespace( parse_image_path=SourceSchemaFilenameParser().parse_filename, metadata_handler=PLATE_ROOT_METADATA, ) record = PlateImageInventory._record( - plate_path=tmp_path, image_file=filename, handler=handler, - projection=builder.projection(), filemanager=filemanager, backend="disk", + plate_path=tmp_path, + image_file=filename, + handler=handler, + projection=builder.projection(), + filemanager=filemanager, + backend="disk", ) assert "site" not in record.metadata context = PlateInspectionContext( - plate_path=tmp_path, filemanager=filemanager, handler=handler, parser=None, + plate_path=tmp_path, + filemanager=filemanager, + handler=handler, + parser=None, ) projection = PlateStreamingService._inventory_source_projection( - (PlateFileRecord.from_image(record),), context, + (PlateFileRecord.from_image(record),), + context, ) stream_source = ViewerStreamingSource( - plate_path=tmp_path, filemanager=filemanager, microscope_handler=handler, + plate_path=tmp_path, + filemanager=filemanager, + microscope_handler=handler, ) actual = stream_source.load_image( - filename, "disk", source_projection=projection, component_metadata=record.metadata, + filename, + "disk", + source_projection=projection, + component_metadata=record.metadata, ) np.testing.assert_array_equal(actual.data, pixels) actual_metadata = actual.metadata assert "site" not in actual_metadata.source_component_metadata assert actual_metadata.plane_axis is None assert actual_metadata.source_voxel_spacing.values_zyx == (0.25, 0.5) - assert actual_metadata.source_provenance.source_image_provenance_planes == metadata.source_provenance.source_image_provenance_planes + assert ( + actual_metadata.source_provenance.source_image_provenance_planes + == metadata.source_provenance.source_image_provenance_planes + ) class FakeInspectionService: diff --git a/tests/unit/agent/test_progressive_authoring_context.py b/tests/unit/agent/test_progressive_authoring_context.py index a3a6c09580..60a7c318a5 100644 --- a/tests/unit/agent/test_progressive_authoring_context.py +++ b/tests/unit/agent/test_progressive_authoring_context.py @@ -193,7 +193,9 @@ def test_task_authorization_routes_to_the_existing_canonical_knowledge_owner() - assert document.selected_section_id == target.section_id assert document.document.source_path == "docs/source/architecture/quick_start.rst" for kind in ("first_use", "ui_visible_workflow"): - context = service.get_bounded_authoring_context(AuthoringContextRequest(kind=kind)) + context = service.get_bounded_authoring_context( + AuthoringContextRequest(kind=kind) + ) assert context == service.get_authoring_context(kind) assert target.document_id in context.content assert agent_capabilities.get_knowledge_document.name in context.content @@ -241,9 +243,10 @@ class _CombinedReviewContext( source = Path(__file__).resolve().parents[3] / document.document.source_path lines = tuple(source.read_text(encoding="utf-8").splitlines()) section = target.find_section(document.sections) - assert document.content.strip() == "\n".join( - section.span.line_slice(lines) - ).strip() + assert ( + document.content.strip() + == "\n".join(section.span.line_slice(lines)).strip() + ) for declaration in ( ImageAnalysisWorkflowAuthoringContext, ViewerReviewAuthoringContext, @@ -284,10 +287,7 @@ def test_knowledge_summary_growth_does_not_expand_context_deepening_links() -> N target_id = target.document_id if target.section_id is not None: target_id = f"{target_id}#{target.section_id}" - assert ( - f"{target_id} — Title for {target.document_id}" - in delivered.content - ) + assert f"{target_id} — Title for {target.document_id}" in delivered.content @pytest.mark.parametrize("kind", ("first_use", "pipeline")) @@ -552,7 +552,10 @@ def test_task_contexts_expose_only_the_next_relevant_boundary() -> None: assert "openhcs_get_viewer_window_payloads" in debugging assert "openhcs_summarize_viewer_window_rois" in debugging assert "schema-bearing per-object measurement rows" in debugging - assert "follow the 'show_live_results' action's `related_state_surface_ids`" in debugging + assert ( + "follow the 'show_live_results' action's `related_state_surface_ids`" + in debugging + ) assert "never guess from a title substring" in debugging assert "bounded retained-table authority" in debugging assert "row/object cardinality" in debugging diff --git a/tests/unit/agent/test_public_callable_enum_ingress.py b/tests/unit/agent/test_public_callable_enum_ingress.py index d9b1515160..3a4cafabf9 100644 --- a/tests/unit/agent/test_public_callable_enum_ingress.py +++ b/tests/unit/agent/test_public_callable_enum_ingress.py @@ -55,10 +55,14 @@ class IndependentProjectionRequest: @numpy @callable_request(IndependentProjectionRequest) def independent_request_projection(request: IndependentProjectionRequest): - return request.image if request.mode is None else request.mode.project(request.image) + return ( + request.image if request.mode is None else request.mode.project(request.image) + ) -@pytest.fixture(params=(independent_declared_projection, independent_request_projection)) +@pytest.fixture( + params=(independent_declared_projection, independent_request_projection) +) def author(monkeypatch, request): # Declarations outside the registry's modules are custom functions. metadata = project_custom_function(request.param) @@ -67,7 +71,9 @@ def author(monkeypatch, request): monkeypatch.setattr(RegistryService, "_metadata_cache", {function_id: metadata}) monkeypatch.setattr(RegistryService, "_resolved_reference_callables", {}) try: - yield PipelineAuthoringService(function_catalog=SelectedDeclarations()), function_id + yield PipelineAuthoringService( + function_catalog=SelectedDeclarations() + ), function_id finally: CustomFunctionRuntimeRegistry.remove(request.param.__name__) @@ -77,13 +83,21 @@ def author(monkeypatch, request): def test_new_callable_json_roundtrip_exercises_cooperative_member(author, clean, mode): service, function_id = author ref = service.create_pipeline() - service.add_function_step_from_request(FunctionStepAddRequest.from_fields( - pipeline_id=ref.pipeline_id, function_id=function_id, kwargs={"mode": mode}, - )) + service.add_function_step_from_request( + FunctionStepAddRequest.from_fields( + pipeline_id=ref.pipeline_id, + function_id=function_id, + kwargs={"mode": mode}, + ) + ) validation = service.validate(ref) assert validation.valid, validation.errors - document = PipelineDocumentCodec.from_source(service.render_source(ref, clean=clean).source) - invocation = next(normalize_function_pattern(document.pipeline_steps[0].func).iter_items()) + document = PipelineDocumentCodec.from_source( + service.render_source(ref, clean=clean).source + ) + invocation = next( + normalize_function_pattern(document.pipeline_steps[0].func).iter_items() + ) expected = None if mode is None else IndependentProjection.DOUBLE_WITH_OFFSET if mode is None: # Clean source legitimately omits a parameter equal to its declared default. @@ -91,24 +105,38 @@ def test_new_callable_json_roundtrip_exercises_cooperative_member(author, clean, else: assert invocation.kwargs_dict == {"mode": expected} contract = invocation.contract - assert contract.validate_public_kwargs(invocation.kwargs_dict) == tuple(invocation.kwargs_dict.items()) + assert contract.validate_public_kwargs(invocation.kwargs_dict) == tuple( + invocation.kwargs_dict.items() + ) # Execute both actual hooks through the decoded original enum member's MRO. assert contract.resolve_raw_runtime_callable()(5, **invocation.kwargs_dict) == ( 5 if mode is None else 13 ) - assert IndependentProjection.__mro__.index(_OffsetCapability) < IndependentProjection.__mro__.index(_ScaleBehavior) - - -@pytest.mark.parametrize("kwargs", ( - {"mode": "not_declared"}, {"mode": True}, {"unknown": "double_with_offset"}, - {"dtype_config": {}}, {"image": 5}, -)) + assert IndependentProjection.__mro__.index( + _OffsetCapability + ) < IndependentProjection.__mro__.index(_ScaleBehavior) + + +@pytest.mark.parametrize( + "kwargs", + ( + {"mode": "not_declared"}, + {"mode": True}, + {"unknown": "double_with_offset"}, + {"dtype_config": {}}, + {"image": 5}, + ), +) def test_public_ingress_rejects_unknown_and_runtime_owned_kwargs(author, kwargs): service, function_id = author ref = service.create_pipeline() - service.add_function_step_from_request(FunctionStepAddRequest.from_fields( - pipeline_id=ref.pipeline_id, function_id=function_id, kwargs=kwargs, - )) + service.add_function_step_from_request( + FunctionStepAddRequest.from_fields( + pipeline_id=ref.pipeline_id, + function_id=function_id, + kwargs=kwargs, + ) + ) assert not service.validate(ref).valid with pytest.raises((TypeError, ValueError)): service.render_source(ref) diff --git a/tests/unit/agent/test_selected_plane_reopen_stream.py b/tests/unit/agent/test_selected_plane_reopen_stream.py index f6416bf0ec..9f95643b93 100644 --- a/tests/unit/agent/test_selected_plane_reopen_stream.py +++ b/tests/unit/agent/test_selected_plane_reopen_stream.py @@ -39,7 +39,10 @@ ) from openhcs.runtime.napari_viewer_server import NapariBatchPayload, NapariImagePayload -from tests.unit.agent.test_persisted_output_reopen import NoRuntimeBridge, declared_output +from tests.unit.agent.test_persisted_output_reopen import ( + NoRuntimeBridge, + declared_output, +) @pytest.fixture diff --git a/tests/unit/agent/test_u1_g8_guards.py b/tests/unit/agent/test_u1_g8_guards.py index daafdb09f2..2566215b7b 100644 --- a/tests/unit/agent/test_u1_g8_guards.py +++ b/tests/unit/agent/test_u1_g8_guards.py @@ -34,7 +34,9 @@ def test_agent_dtos_declare_no_well_or_microscope_type_field() -> None: and isinstance(statement.target, ast.Name) and statement.target.id in {"well", "microscope_type"} ): - offenders.append(f"{path.relative_to(ROOT)}:{node.name}.{statement.target.id}") + offenders.append( + f"{path.relative_to(ROOT)}:{node.name}.{statement.target.id}" + ) assert offenders == [] @@ -43,10 +45,9 @@ def test_gui_restates_no_plate_grid_default_or_row_letter_decode() -> None: for path in _modules("pyqt_gui"): tree = ast.parse(path.read_text()) for node in ast.walk(tree): - if ( - isinstance(node, ast.Tuple) - and [getattr(element, "value", None) for element in node.elts] == [8, 12] - ): + if isinstance(node, ast.Tuple) and [ + getattr(element, "value", None) for element in node.elts + ] == [8, 12]: offenders.append(f"{path.relative_to(ROOT)}:{node.lineno} (8, 12)") if ( isinstance(node, ast.BinOp) @@ -56,7 +57,9 @@ def test_gui_restates_no_plate_grid_default_or_row_letter_decode() -> None: for side in (node.left, node.right) ) ): - offenders.append(f"{path.relative_to(ROOT)}:{node.lineno} base-26 decode") + offenders.append( + f"{path.relative_to(ROOT)}:{node.lineno} base-26 decode" + ) assert offenders == [] @@ -76,6 +79,10 @@ def test_grid_placement_is_declared_only_by_grid_addressed_axes() -> None: base.id if isinstance(base, ast.Name) else getattr(base, "attr", "") for base in node.bases } - if declares and node.name != "GridAddressed" and "GridAddressed" not in bases: + if ( + declares + and node.name != "GridAddressed" + and "GridAddressed" not in bases + ): offenders.append(f"{path.relative_to(ROOT)}:{node.name}") assert offenders == [] diff --git a/tests/unit/agent/test_ui_bridge_operation_family.py b/tests/unit/agent/test_ui_bridge_operation_family.py index c1b3b75623..c61085b5bc 100644 --- a/tests/unit/agent/test_ui_bridge_operation_family.py +++ b/tests/unit/agent/test_ui_bridge_operation_family.py @@ -119,7 +119,8 @@ def test_every_wire_operation_is_one_mcp_tool_deriving_its_contracts() -> None: capabilities = tuple( capability for capability in agent_capability_declarations() - if issubclass(capability, UiBridgeCapability) and "operation" in vars(capability) + if issubclass(capability, UiBridgeCapability) + and "operation" in vars(capability) ) named = [capability.operation for capability in capabilities] assert len(named) == len(set(named)) diff --git a/tests/unit/agent/test_ui_bridge_service.py b/tests/unit/agent/test_ui_bridge_service.py index 986576bdf2..3b50be84ee 100644 --- a/tests/unit/agent/test_ui_bridge_service.py +++ b/tests/unit/agent/test_ui_bridge_service.py @@ -1020,7 +1020,9 @@ def test_descriptor_resolver_rejects_dead_process(monkeypatch, tmp_path): staticmethod(lambda _pid: None), ) - status = UiBridgeService(gateway=_FakeUiBridgeGateway()).invoke(UiBridgeStatusOperation) + status = UiBridgeService(gateway=_FakeUiBridgeGateway()).invoke( + UiBridgeStatusOperation + ) assert status.reachable is False assert status.descriptor_status == "stale_ui_bridge_descriptor" @@ -1109,7 +1111,9 @@ def test_status_rechecks_descriptor_process_liveness_after_gateway_call( staticmethod(lambda _pid: next(process_start_times)), ) - status = UiBridgeService(gateway=_FakeUiBridgeGateway()).invoke(UiBridgeStatusOperation) + status = UiBridgeService(gateway=_FakeUiBridgeGateway()).invoke( + UiBridgeStatusOperation + ) assert status.reachable is False assert status.descriptor_status == "stale_ui_bridge_descriptor" @@ -1136,7 +1140,9 @@ def status(self, connection: UiBridgeConnectionSpec) -> UiBridgeStatus: ).write() monkeypatch.setenv("OPENHCS_UI_BRIDGE_DESCRIPTOR", str(descriptor_path)) - status = UiBridgeService(gateway=MismatchedIdentityGateway()).invoke(UiBridgeStatusOperation) + status = UiBridgeService(gateway=MismatchedIdentityGateway()).invoke( + UiBridgeStatusOperation + ) assert status.reachable is False assert status.descriptor_status == "ui_bridge_endpoint_identity_mismatch" @@ -1270,7 +1276,9 @@ def test_configured_descriptor_directory_includes_process_advertised_descriptor( proc_root, ) - status = UiBridgeService(gateway=_FakeUiBridgeGateway()).invoke(UiBridgeStatusOperation) + status = UiBridgeService(gateway=_FakeUiBridgeGateway()).invoke( + UiBridgeStatusOperation + ) assert status.reachable is True assert status.descriptor_status == "ok" @@ -1308,7 +1316,9 @@ def test_descriptor_catalog_unions_default_and_process_advertised_live_bridges( ) catalog = UiBridgeService().list_bridges() - status = UiBridgeService(gateway=_FakeUiBridgeGateway()).invoke(UiBridgeStatusOperation) + status = UiBridgeService(gateway=_FakeUiBridgeGateway()).invoke( + UiBridgeStatusOperation + ) assert [bridge.bridge_instance_id for bridge in catalog.bridges] == [ "default-bridge", @@ -1372,7 +1382,9 @@ def test_service_forwards_fake_gateway_requests(monkeypatch, tmp_path): ) catalog = service.invoke(UiBridgeListDocumentsOperation, connection=connection) - state_catalog = service.invoke(UiBridgeListStateSurfacesOperation, connection=connection) + state_catalog = service.invoke( + UiBridgeListStateSurfacesOperation, connection=connection + ) document = service.invoke( UiBridgeGetDocumentOperation, UiCodeDocumentRequest(document_id=DOCUMENT_ID), @@ -1820,7 +1832,7 @@ def list_object_state_scopes( scope_ids=("global_config",), field_path_contains=("napari_streaming_config",), field_filter="semantic", - ) + ), ) assert result.field_filter == "semantic" @@ -1849,9 +1861,7 @@ def close_window( errors=( AgentError( code="unsupported_ui_bridge_operation", - message=( - "Unsupported UI bridge operation: close_window" - ), + message=("Unsupported UI bridge operation: close_window"), exception_type="UiBridgeUnsupportedOperationError", ), ) @@ -1881,7 +1891,7 @@ def test_restore_request_rejects_multiple_selectors(monkeypatch, tmp_path): result = service.invoke( UiBridgeRestoreSnapshotOperation, - UiSnapshotRestoreRequest(snapshot_id="snap-1", index=0) + UiSnapshotRestoreRequest(snapshot_id="snap-1", index=0), ) assert result.restored is False diff --git a/tests/unit/agent/test_user_facing_help_projection.py b/tests/unit/agent/test_user_facing_help_projection.py index 7804584e2e..09e6e86dc0 100644 --- a/tests/unit/agent/test_user_facing_help_projection.py +++ b/tests/unit/agent/test_user_facing_help_projection.py @@ -296,7 +296,9 @@ def test_preprocessing_guidance_projects_from_callable_owners(monkeypatch) -> No normalized_shared_text = " ".join(shared_text.split()) # Catalog prose is the semantic declaration's, not the memory wrapper's. - raw = CallableContract.from_callable(metadata.func).resolve_raw_runtime_callable() + raw = CallableContract.from_callable( + metadata.func + ).resolve_raw_runtime_callable() assert detail.doc == inspect.getdoc(raw) assert detail.entry.summary == shared_help.summary for phrase in expected_phrases[metadata.func]: diff --git a/tests/unit/benchmark/test_agent_validation_attempt_recorder.py b/tests/unit/benchmark/test_agent_validation_attempt_recorder.py index 77037adc67..05129ddfa8 100644 --- a/tests/unit/benchmark/test_agent_validation_attempt_recorder.py +++ b/tests/unit/benchmark/test_agent_validation_attempt_recorder.py @@ -160,20 +160,35 @@ def _ui_pipeline_document(source: str) -> UiCodeDocument: def _accepted(operation_id: str) -> SessionOperationResult: return SessionOperationResult( - operation_id=operation_id, status="accepted", target_scope_ids=("/plate",), + operation_id=operation_id, + status="accepted", + target_scope_ids=("/plate",), ) def _finished_datasets() -> DatasetListState: row = DatasetRowState( - scope_id="/plate", name="plate", root="/plate", pipeline_path=None, - selected=True, initialized=True, compiled=True, init_pending=False, - compile_pending=False, execution_active=False, status_prefix="", - orchestrator_state="completed", execution_id="execution-1", - terminal_status="complete", runtime_state=None, runtime_percent=None, + scope_id="/plate", + name="plate", + root="/plate", + pipeline_path=None, + selected=True, + initialized=True, + compiled=True, + init_pending=False, + compile_pending=False, + execution_active=False, + status_prefix="", + orchestrator_state="completed", + execution_id="execution-1", + terminal_status="complete", + runtime_state=None, + runtime_percent=None, queue_position=None, ) - return DatasetListState(rows=(row,), selected_scope_ids=("/plate",), execution_state="idle") + return DatasetListState( + rows=(row,), selected_scope_ids=("/plate",), execution_state="idle" + ) def test_attempt_recorder_derives_dsl_evidence_from_mcp_receipts(tmp_path: Path): diff --git a/tests/unit/cellprofiler_runtime_test_support.py b/tests/unit/cellprofiler_runtime_test_support.py index d80059e9a7..e159ba234f 100644 --- a/tests/unit/cellprofiler_runtime_test_support.py +++ b/tests/unit/cellprofiler_runtime_test_support.py @@ -48,7 +48,7 @@ class CellProfilerRuntimeTestContext: filemanager: object | None microscope_handler: object source_image_set_identity_policy: SourceImageSetIdentityPolicy - dataset_root: Path = Path('/plate/Images') + dataset_root: Path = Path("/plate/Images") completed_step_outputs: StepExecutionObservation = field( default_factory=StepExecutionObservation.empty, ) diff --git a/tests/unit/desktop/test_desktop_restart.py b/tests/unit/desktop/test_desktop_restart.py index f288587b7a..95c1bd1f3c 100644 --- a/tests/unit/desktop/test_desktop_restart.py +++ b/tests/unit/desktop/test_desktop_restart.py @@ -116,7 +116,9 @@ def test_session_restart_admission_uses_existing_declaration_work_guard( window = SimpleNamespace(session=restart_session) assert not DesktopSessionRestart.available(window) - with pytest.raises(DesktopUpdateError, match="active initialization or compilation"): + with pytest.raises( + DesktopUpdateError, match="active initialization or compilation" + ): DesktopRestartSession.require_capture_allowed(window) diff --git a/tests/unit/progress/test_debug_projection.py b/tests/unit/progress/test_debug_projection.py index 72c87472d9..c13a79fe70 100644 --- a/tests/unit/progress/test_debug_projection.py +++ b/tests/unit/progress/test_debug_projection.py @@ -78,7 +78,9 @@ def test_debug_progress_record_fails_loudly_for_malformed_debug_context() -> Non with pytest.raises(ValueError, match="missing required field"): DebugProgressRecord.from_progress_event( _event( - context={DebugProgressContext.context_key: {"debug_session_id": "debug-1"}} + context={ + DebugProgressContext.context_key: {"debug_session_id": "debug-1"} + } ) ) diff --git a/tests/unit/pyqt_gui/session_harness.py b/tests/unit/pyqt_gui/session_harness.py index a01c385592..ee75752c5b 100644 --- a/tests/unit/pyqt_gui/session_harness.py +++ b/tests/unit/pyqt_gui/session_harness.py @@ -79,7 +79,9 @@ def caller_session(**kwargs) -> Iterator[Session]: if persisted is not None: write_dataset_scope_ids(list(persisted)) session = Session( - transport_config=OpenHCSZMQConfig(default_port=session_port(), persistent=False), + transport_config=OpenHCSZMQConfig( + default_port=session_port(), persistent=False + ), global_config=global_config, main_thread=kwargs.pop("main_thread", CallerThread()), **kwargs, diff --git a/tests/unit/pyqt_gui/test_concurrent_initialization_progress.py b/tests/unit/pyqt_gui/test_concurrent_initialization_progress.py index dffca9c157..c2b2416475 100644 --- a/tests/unit/pyqt_gui/test_concurrent_initialization_progress.py +++ b/tests/unit/pyqt_gui/test_concurrent_initialization_progress.py @@ -15,7 +15,9 @@ def test_init_signals_and_completion_preserve_running_progress(): app = QApplication.instance() or QApplication([]) projection = ExecutionRuntimeProjection() plate = SubjectRuntimeProjection( - identity=SubjectRuntimeIdentity(subject_id="/running", execution_id="owned-run"), + identity=SubjectRuntimeIdentity( + subject_id="/running", execution_id="owned-run" + ), state=ExecutingSubjectRuntimeState, percent=43, axis_progress=(), diff --git a/tests/unit/pyqt_gui/test_config_preview_formatters.py b/tests/unit/pyqt_gui/test_config_preview_formatters.py index b392867baf..e253bc4ef0 100644 --- a/tests/unit/pyqt_gui/test_config_preview_formatters.py +++ b/tests/unit/pyqt_gui/test_config_preview_formatters.py @@ -24,7 +24,10 @@ def test_partition_filter_preview_label_is_owned_by_config_declaration() -> None def test_plain_partition_filter_preview_requires_a_selection() -> None: - assert format_partition_filter_config(PartitionFilterConfig(), "partition_filter") is None + assert ( + format_partition_filter_config(PartitionFilterConfig(), "partition_filter") + is None + ) assert ( format_partition_filter_config( PartitionFilterConfig( @@ -42,7 +45,10 @@ def test_lazy_plain_partition_filter_preserves_base_preview_policy() -> None: assert resolution is not None assert resolution.owner is PartitionFilterConfig - assert format_partition_filter_config(LazyPartitionFilterConfig(), "partition_filter") is None + assert ( + format_partition_filter_config(LazyPartitionFilterConfig(), "partition_filter") + is None + ) def test_lazy_specialized_config_uses_its_declared_preview_policy() -> None: diff --git a/tests/unit/pyqt_gui/test_debug_toolbar.py b/tests/unit/pyqt_gui/test_debug_toolbar.py index c291aaf739..6844eb17f6 100644 --- a/tests/unit/pyqt_gui/test_debug_toolbar.py +++ b/tests/unit/pyqt_gui/test_debug_toolbar.py @@ -579,9 +579,11 @@ def test_pipeline_change_marks_the_debug_session_dirty(tmp_path, monkeypatch) -> ).with_cursor(_cursor(1, "segment")) statuses = [] harness.session.subscribe( - lambda record: statuses.append(record.event.text) - if isinstance(record.event, StatusReported) - else None + lambda record: ( + statuses.append(record.event.text) + if isinstance(record.event, StatusReported) + else None + ) ) harness.session.set_pipeline( diff --git a/tests/unit/pyqt_gui/test_main_config_propagation.py b/tests/unit/pyqt_gui/test_main_config_propagation.py index db75ee27fb..cbdb766b14 100644 --- a/tests/unit/pyqt_gui/test_main_config_propagation.py +++ b/tests/unit/pyqt_gui/test_main_config_propagation.py @@ -437,11 +437,13 @@ def test_lifecycle_workflow_adopts_a_saved_global_config_into_the_session( (scope_id,) = add_datasets(session, Path(tempfile.mkdtemp()), "plate") changed = [] session.subscribe( - lambda record: changed.append( - (record.event.scope_id, record.event.effective_config.num_workers) + lambda record: ( + changed.append( + (record.event.scope_id, record.event.effective_config.num_workers) + ) + if isinstance(record.event, DatasetConfigChanged) + else None ) - if isinstance(record.event, DatasetConfigChanged) - else None ) workflow.propagate_config(new_config) diff --git a/tests/unit/pyqt_gui/test_main_status_bar.py b/tests/unit/pyqt_gui/test_main_status_bar.py index ab8b22056d..9c7e9805a6 100644 --- a/tests/unit/pyqt_gui/test_main_status_bar.py +++ b/tests/unit/pyqt_gui/test_main_status_bar.py @@ -129,7 +129,10 @@ def test_endpoint_and_application_status_have_one_visual_owner( assert window._status_message_label.text() == application_message assert window._zmq_status_indicator._label.text() == "ZMQ: Connected" - assert window.button_refreshes == ["plate_manager_widget", "pipeline_editor_widget"] + assert window.button_refreshes == [ + "plate_manager_widget", + "pipeline_editor_widget", + ] assert ( window._zmq_status_indicator.toolTip() == "Execution endpoint 7777: Connected" diff --git a/tests/unit/pyqt_gui/test_main_window_file_actions.py b/tests/unit/pyqt_gui/test_main_window_file_actions.py index aaa1a11b50..d78a5d182b 100644 --- a/tests/unit/pyqt_gui/test_main_window_file_actions.py +++ b/tests/unit/pyqt_gui/test_main_window_file_actions.py @@ -21,8 +21,14 @@ def show_dataset_code(self, selection_mode) -> None: @pytest.mark.parametrize( "action, mode", ( - (OpenHCSMainWindow.load_orchestrator_configuration, SelectedAllSelectionMode.SELECTED), - (OpenHCSMainWindow.save_orchestrator_configuration, SelectedAllSelectionMode.ALL), + ( + OpenHCSMainWindow.load_orchestrator_configuration, + SelectedAllSelectionMode.SELECTED, + ), + ( + OpenHCSMainWindow.save_orchestrator_configuration, + SelectedAllSelectionMode.ALL, + ), ), ) def test_file_actions_open_the_dataset_code_document(action, mode) -> None: diff --git a/tests/unit/pyqt_gui/test_one_persistence_format.py b/tests/unit/pyqt_gui/test_one_persistence_format.py index 233d6f82e0..2b4e7a2eaf 100644 --- a/tests/unit/pyqt_gui/test_one_persistence_format.py +++ b/tests/unit/pyqt_gui/test_one_persistence_format.py @@ -84,7 +84,10 @@ def test_pipeline_round_trips_through_python_file(tmp_path: Path) -> None: assert (function, {key: kwargs[key] for key in _normalize(2.0)[1]}) == ( _normalize(2.0) ) - assert [step.name for step in editor.displayed_steps] == ["Normalize", "Crop"] + assert [step.name for step in editor.displayed_steps] == [ + "Normalize", + "Crop", + ] assert editor.code_document_source(clean=True) == target.read_text( encoding="utf-8" ) @@ -161,8 +164,7 @@ def test_gui_persistence_modules_do_not_pickle() -> None: for path in sorted(tree.rglob("*.py")): if path in TRANSPORT_MODULES: assert not any( - use.endswith((".dump", ".load")) - for use in _pickle_uses(path) + use.endswith((".dump", ".load")) for use in _pickle_uses(path) ), f"{path} may pickle in memory for ZMQ only" continue offenders.extend(_pickle_uses(path)) @@ -170,7 +172,7 @@ def test_gui_persistence_modules_do_not_pickle() -> None: def test_gui_offers_no_pickle_file_formats() -> None: - retired = ("*.func", "*.pipeline", "*.step", ".func\"", ".pipeline\"", ".step\"") + retired = ("*.func", "*.pipeline", "*.step", '.func"', '.pipeline"', '.step"') offenders = [ f"{path}: {marker}" for tree in PERSISTENCE_TREES diff --git a/tests/unit/pyqt_gui/test_parameter_form_manager_cross_window.py b/tests/unit/pyqt_gui/test_parameter_form_manager_cross_window.py index 33266987cc..555b75014c 100644 --- a/tests/unit/pyqt_gui/test_parameter_form_manager_cross_window.py +++ b/tests/unit/pyqt_gui/test_parameter_form_manager_cross_window.py @@ -147,7 +147,9 @@ def _on_live_context_changed(self): QTimer.singleShot(240, loop.quit) loop.exec() - assert manager.state.updated == [("step_partition_filter_config.partition_filter", "B02")] + assert manager.state.updated == [ + ("step_partition_filter_config.partition_filter", "B02") + ] assert manager.synced == ["partition_filter"] assert manager.flash_requests == [] assert manager.live_refreshes == 1 @@ -291,14 +293,20 @@ def get_resolved_value(self, path): partition_filter.form_tree = form_tree path_planning.form_tree = form_tree - root.chrome_sync.refresh_widgets_for_paths({"partition_filter_config.partition_filter"}) + root.chrome_sync.refresh_widgets_for_paths( + {"partition_filter_config.partition_filter"} + ) - assert widget_service.calls == [("partition_filter_config", "partition_filter", "A01", False)] + assert widget_service.calls == [ + ("partition_filter_config", "partition_filter", "A01", False) + ] assert partition_filter.widgets["partition_filter"].values == ["A01"] assert path_planning.widgets["partition_filter"].values == [] assert placeholder_service.calls == [] - root.chrome_sync.refresh_widgets_for_paths({"path_planning_config.partition_filter"}) + root.chrome_sync.refresh_widgets_for_paths( + {"path_planning_config.partition_filter"} + ) assert widget_service.calls[-1] == ( "path_planning_config", @@ -377,9 +385,15 @@ def _flush_resolved_values_changed(self): manager.deleteLater() qapp.processEvents() - assert manager.chrome_sync.widget_paths == [{"partition_filter_config.partition_filter"}] - assert manager.chrome_sync.state_paths == [{"partition_filter_config.partition_filter"}] - assert manager.flash_registrations == [("partition_filter_config.partition_filter", False)] + assert manager.chrome_sync.widget_paths == [ + {"partition_filter_config.partition_filter"} + ] + assert manager.chrome_sync.state_paths == [ + {"partition_filter_config.partition_filter"} + ] + assert manager.flash_registrations == [ + ("partition_filter_config.partition_filter", False) + ] assert manager.flash_batches == [("partition_filter_config.partition_filter",)] diff --git a/tests/unit/pyqt_gui/test_pipeline_code_document_transactionality.py b/tests/unit/pyqt_gui/test_pipeline_code_document_transactionality.py index 88e6421267..80095dc957 100644 --- a/tests/unit/pyqt_gui/test_pipeline_code_document_transactionality.py +++ b/tests/unit/pyqt_gui/test_pipeline_code_document_transactionality.py @@ -22,9 +22,11 @@ def _editor(tmp_path): gui.settle() changes = [] gui.session.subscribe( - lambda record: changes.append(record.event.scope_id) - if isinstance(record.event, PipelineChanged) - else None + lambda record: ( + changes.append(record.event.scope_id) + if isinstance(record.event, PipelineChanged) + else None + ) ) yield gui, scope_id, gui.pipeline_editor.code_document_driver(), changes diff --git a/tests/unit/pyqt_gui/test_pipeline_editor_widget.py b/tests/unit/pyqt_gui/test_pipeline_editor_widget.py index 75818291c8..9928b253a7 100644 --- a/tests/unit/pyqt_gui/test_pipeline_editor_widget.py +++ b/tests/unit/pyqt_gui/test_pipeline_editor_widget.py @@ -840,9 +840,11 @@ def test_code_document_driver_reads_validates_and_applies(tmp_path) -> None: harness.set_steps([FunctionStep(name="Original")]) changed = [] harness.session.subscribe( - lambda record: changed.append(record.event.scope_id) - if isinstance(record.event, PipelineChanged) - else None + lambda record: ( + changed.append(record.event.scope_id) + if isinstance(record.event, PipelineChanged) + else None + ) ) driver = harness.editor.code_document_driver() assert driver is not None @@ -1023,7 +1025,12 @@ def test_time_travel_broadcasts_only_pipeline_level_restores( assert broadcasts == [] harness.editor.on_time_travel_complete( - [(PipelineScopeIdentity.from_plate_scope(harness.scope_id).scope_id, object())], + [ + ( + PipelineScopeIdentity.from_plate_scope(harness.scope_id).scope_id, + object(), + ) + ], None, ) assert [[step.name for step in steps] for steps in broadcasts] == [ @@ -1254,7 +1261,9 @@ def test_step_partition_filter_live_resolution_is_visible_in_the_row(tmp_path) - ) dataset_state = ObjectStateRegistry.get_by_scope(harness.scope_id) - dataset_state.update_parameter("partition_filter_config.partition_filter", "A01") + dataset_state.update_parameter( + "partition_filter_config.partition_filter", "A01" + ) ObjectStateRegistry._notify_change() (step,) = harness.editor.displayed_steps styled_text, _ = harness.editor.format_item_for_display(step, step_index=0) @@ -1264,7 +1273,9 @@ def test_step_partition_filter_live_resolution_is_visible_in_the_row(tmp_path) - for segment in styled_text.layout.preview_segments if segment.field_path } - assert preview_by_path["step_partition_filter_config.partition_filter"] == ":A01" + assert ( + preview_by_path["step_partition_filter_config.partition_filter"] == ":A01" + ) def test_a_pipeline_config_scope_is_not_a_current_orchestrator() -> None: diff --git a/tests/unit/pyqt_gui/test_plate_compile_connection_action.py b/tests/unit/pyqt_gui/test_plate_compile_connection_action.py index 5a959f2e60..6621377350 100644 --- a/tests/unit/pyqt_gui/test_plate_compile_connection_action.py +++ b/tests/unit/pyqt_gui/test_plate_compile_connection_action.py @@ -59,9 +59,9 @@ def test_endpoint_phase_resolves_the_compile_slot(monkeypatch, phase, expected): def test_connect_needs_no_dataset_and_startup_is_never_available(tmp_path): with caller_session() as session: (scope_id,) = add_datasets(session, tmp_path, "plate") - session.orchestrator(scope_id)._state = ( - session.orchestrator(scope_id)._state.READY - ) + session.orchestrator(scope_id)._state = session.orchestrator( + scope_id + )._state.READY session.set_pipeline(scope_id, [FunctionStep(name="Step")]) assert ConnectServer.available(session, ConnectServer.request()) is None for selection in ((), (scope_id,)): diff --git a/tests/unit/pyqt_gui/test_plate_manager_widget.py b/tests/unit/pyqt_gui/test_plate_manager_widget.py index d6fd7d8429..cf280cb893 100644 --- a/tests/unit/pyqt_gui/test_plate_manager_widget.py +++ b/tests/unit/pyqt_gui/test_plate_manager_widget.py @@ -438,7 +438,9 @@ def test_workspace_import_replaces_steps_keeps_public_functions_and_applies_conf ) with caller_session() as session: session.add_dataset_roots((root,)) - session.set_pipeline(scope_id, [FunctionStep(func=percentile_normalize, name="Existing")]) + session.set_pipeline( + scope_id, [FunctionStep(func=percentile_normalize, name="Existing")] + ) session.import_workspace_pipeline(scope_id, _workspace([crop], pipeline_config)) @@ -533,7 +535,9 @@ def test_workspace_import_never_writes_into_the_displayed_dataset(tmp_path) -> N assert _names(gui.pipeline_editor.displayed_steps) == ["StartOnly"] -@pytest.mark.parametrize("state", (OrchestratorState.READY, OrchestratorState.EXECUTING)) +@pytest.mark.parametrize( + "state", (OrchestratorState.READY, OrchestratorState.EXECUTING) +) def test_source_binding_context_projects_the_current_orchestrator_config( tmp_path, state ) -> None: @@ -587,7 +591,9 @@ def test_source_binding_context_projects_the_current_orchestrator_config( def test_compile_needs_an_initialized_dataset(tmp_path) -> None: with caller_session() as session: (scope_id,) = add_datasets(session, tmp_path, "plate") - session.set_pipeline(scope_id, [FunctionStep(func=percentile_normalize, name="Defined")]) + session.set_pipeline( + scope_id, [FunctionStep(func=percentile_normalize, name="Defined")] + ) error = CompileDatasets.available( session, CompileDatasets.request_for_selection(session, (scope_id,)) @@ -664,7 +670,9 @@ def navigate(item_id, field_path=None): ): result = navigate(item_id, field_path) assert result.focused and not result.navigated - assert result.errors[0].code == "ui_window_navigation_target_unsupported" + assert ( + result.errors[0].code == "ui_window_navigation_target_unsupported" + ) assert gui.session.current_scope_id == paths[0] assert selected == list(reversed(paths)) @@ -703,7 +711,9 @@ def test_restart_restores_a_missing_dataset_without_initializing_it(tmp_path) -> payload = PlateManagerCodeDocumentAuthority.from_values( plate_paths=(scope_id,), global_pipeline_config=GlobalPipelineConfig(), - per_plate_configs={scope_id: PipelineConfig(dataset_source=OpenHCSDatasetSource)}, + per_plate_configs={ + scope_id: PipelineConfig(dataset_source=OpenHCSDatasetSource) + }, pipeline_data={scope_id: []}, ) restart.session_document.write_text( @@ -1026,7 +1036,9 @@ def test_adopting_a_global_config_preserves_a_dirty_dataset_override(tmp_path) - session.adopt_global_config(global_state.to_object(update_delegate=False)) gui.settle() - assert object.__getattribute__(orchestrator.pipeline_config, "num_workers") is None + assert ( + object.__getattribute__(orchestrator.pipeline_config, "num_workers") is None + ) assert dataset_state.parameters["num_workers"] == 3 assert dataset_state._saved_parameters["num_workers"] is None assert dataset_state.get_resolved_value("num_workers") == 3 @@ -1152,7 +1164,9 @@ def test_code_document_keeps_an_authored_dataset_override(tmp_path) -> None: def test_code_document_reads_the_global_config_from_object_state() -> None: - initial = GlobalPipelineConfig(partition_filter_config=PartitionFilterConfig(partition_filter="A01")) + initial = GlobalPipelineConfig( + partition_filter_config=PartitionFilterConfig(partition_filter="A01") + ) with caller_session(global_config=initial) as session: global_state = _global_state(session) global_state.update_parameter("partition_filter_config.partition_filter", "A02") @@ -1169,7 +1183,9 @@ def test_code_document_reads_the_global_config_from_object_state() -> None: == "A02" ) assert ( - object.__getattribute__(session.global_config.partition_filter_config, "partition_filter") + object.__getattribute__( + session.global_config.partition_filter_config, "partition_filter" + ) == "A01" ) @@ -1204,7 +1220,9 @@ def test_a_pipeline_change_clears_stale_compilation_and_terminal_tracking( global_pipeline_config=live_global_config(session), per_plate_configs={scope_id: authored_pipeline_config(scope_id)}, pipeline_data={ - scope_id: [FunctionStep(func=percentile_normalize, name="Replacement")] + scope_id: [ + FunctionStep(func=percentile_normalize, name="Replacement") + ] }, ), AllDatasetDocumentScope(), @@ -1230,7 +1248,9 @@ def _failed(execution_id: str, detail: str) -> ExecutionCompletionPayload: def _executing_dataset(gui, tmp_path) -> str: session = gui.session (scope_id,) = add_datasets(session, tmp_path, "plate") - session.set_pipeline(scope_id, [FunctionStep(func=percentile_normalize, name="Failing")]) + session.set_pipeline( + scope_id, [FunctionStep(func=percentile_normalize, name="Failing")] + ) orchestrator = session.orchestrator(scope_id) orchestrator._initialized = True orchestrator._state = OrchestratorState.EXECUTING @@ -1304,7 +1324,9 @@ def test_a_failure_is_presented_only_after_the_run_finalizes(tmp_path) -> None: replacement = PipelineDocumentCodec.render( PipelineDocumentCodec.from_values( pipeline_config=PipelineConfig(), - pipeline_steps=[FunctionStep(func=percentile_normalize, name="Replacement")], + pipeline_steps=[ + FunctionStep(func=percentile_normalize, name="Replacement") + ], ) ) @@ -1327,14 +1349,19 @@ def apply(source): ) try: - assert apply(replacement.replace("Replacement", "Edited during run")).applied + assert apply( + replacement.replace("Replacement", "Edited during run") + ).applied assert session.batch.is_active(scope_id) assert session.batch.execution_id(scope_id) == "execution-1" assert _names(session.pipeline_steps(scope_id)) == ["Edited during run"] worker = threading.Thread( target=session.finish_dataset_execution, - args=(_failed("execution-1", "expected asynchronous failure"), scope_id), + args=( + _failed("execution-1", "expected asynchronous failure"), + scope_id, + ), ) worker.start() worker.join(timeout=5) @@ -1440,10 +1467,14 @@ def test_a_produced_output_dataset_selects_prepared_replay_once( assert output_root in session.dataset_scope_ids() orchestrator = session.orchestrator(output_root) - assert orchestrator.get_effective_config().dataset_source is OpenHCSDatasetSource + assert ( + orchestrator.get_effective_config().dataset_source is OpenHCSDatasetSource + ) orchestrator.apply_pipeline_config( - replace_raw(orchestrator.pipeline_config, dataset_source=SourceBindingsSource) + replace_raw( + orchestrator.pipeline_config, dataset_source=SourceBindingsSource + ) ) _running(session, source, "produced-again") session.finish_dataset_execution( @@ -1459,7 +1490,9 @@ def test_a_produced_output_dataset_selects_prepared_replay_once( ) assert session.orchestrator(output_root) is orchestrator - assert orchestrator.get_effective_config().dataset_source is SourceBindingsSource + assert ( + orchestrator.get_effective_config().dataset_source is SourceBindingsSource + ) def test_a_standard_run_retires_only_its_targets_debug_summaries( diff --git a/tests/unit/pyqt_gui/test_session_batch_workflows.py b/tests/unit/pyqt_gui/test_session_batch_workflows.py index 016708c015..67d2fc63d6 100644 --- a/tests/unit/pyqt_gui/test_session_batch_workflows.py +++ b/tests/unit/pyqt_gui/test_session_batch_workflows.py @@ -160,9 +160,11 @@ def test_run_and_debug_reserve_before_connect_and_release_on_connection_failure( session.compile_pending.add(unrelated) errors = [] session.subscribe( - lambda record: errors.append(record.event) - if type(record.event).__name__ == "ErrorReported" - else None + lambda record: ( + errors.append(record.event) + if type(record.event).__name__ == "ErrorReported" + else None + ) ) async def connect(): @@ -185,9 +187,7 @@ async def connect(): @pytest.mark.parametrize("fails", (False, True)) -def test_compiling_another_dataset_keeps_the_active_batch( - tmp_path, monkeypatch, fails -): +def test_compiling_another_dataset_keeps_the_active_batch(tmp_path, monkeypatch, fails): with caller_session() as session: running, other = add_datasets(session, tmp_path, "running", "other") _ready(session, running, other) @@ -250,9 +250,11 @@ def test_compile_publishes_its_inspection_and_clears_the_previous_one( _ready(session, scope_id) compiled_states = [] session.subscribe( - lambda record: compiled_states.append(record.event.compiled) - if isinstance(record.event, CompiledStateChanged) - else None + lambda record: ( + compiled_states.append(record.event.compiled) + if isinstance(record.event, CompiledStateChanged) + else None + ) ) def during_compile(_scope_id): @@ -326,9 +328,11 @@ def test_failed_compile_reports_the_dataset(tmp_path, monkeypatch): _ready(session, scope_id) failures = [] session.subscribe( - lambda record: failures.append(record.event) - if isinstance(record.event, CompilationFailed) - else None + lambda record: ( + failures.append(record.event) + if isinstance(record.event, CompilationFailed) + else None + ) ) StandInServer(session, monkeypatch, failing={scope_id}) asyncio.run(session.compile_datasets((scope_id,))) @@ -486,16 +490,19 @@ def _debug_progress_event(*, snapshot_id: str | None, event_type=None): callable_name="IdentifyPrimaryObjects", axis_id="A01", ) - return event, DebugProgressEventRequest( - debug_session_id="debug-1", - debug_event=event, - progress_context=ProgressExecutionContext("exec-1", "plate-1"), - **( - {} - if snapshot_id is None - else {"snapshot_id": snapshot_id, "snapshot_store_ref": "/tmp/debug"} - ), - ).to_progress_event() + return ( + event, + DebugProgressEventRequest( + debug_session_id="debug-1", + debug_event=event, + progress_context=ProgressExecutionContext("exec-1", "plate-1"), + **( + {} + if snapshot_id is None + else {"snapshot_id": snapshot_id, "snapshot_store_ref": "/tmp/debug"} + ), + ).to_progress_event(), + ) def _events_of(session, kind): @@ -587,7 +594,9 @@ def test_runtime_artifact_progress_is_published(): artifact_type=MeasurementsArtifactType, scope=RuntimeExecutionAxisScope(axis_id="A01"), ), - location=RuntimeArtifactLocation(path="/memory/result.pkl", backend="memory"), + location=RuntimeArtifactLocation( + path="/memory/result.pkl", backend="memory" + ), value_type="ndarray", ) event = _progress_event( @@ -611,7 +620,9 @@ def test_progress_the_registry_rejects_publishes_nothing(monkeypatch): artifact_type=MeasurementsArtifactType, scope=RuntimeExecutionAxisScope(axis_id="A01"), ), - location=RuntimeArtifactLocation(path="/memory/result.pkl", backend="memory"), + location=RuntimeArtifactLocation( + path="/memory/result.pkl", backend="memory" + ), ) session.progress.on_progress( _progress_event( @@ -628,7 +639,6 @@ def test_server_queue_snapshot_projects_queued_executions(monkeypatch): from openhcs.core.progress.projection import QueuedSubjectRuntimeState - with caller_session() as session: monkeypatch.setattr( session.progress._server_info_poller, diff --git a/tests/unit/pyqt_gui/test_synthetic_plate_pipeline_document.py b/tests/unit/pyqt_gui/test_synthetic_plate_pipeline_document.py index e5e00c7f07..331f7bcb9f 100644 --- a/tests/unit/pyqt_gui/test_synthetic_plate_pipeline_document.py +++ b/tests/unit/pyqt_gui/test_synthetic_plate_pipeline_document.py @@ -96,9 +96,11 @@ def test_main_window_loads_emitted_synthetic_pipeline_document( main_window = _MainWindowHarness(session) changed = [] session.subscribe( - lambda record: changed.append(record.event.scope_id) - if isinstance(record.event, PipelineChanged) - else None + lambda record: ( + changed.append(record.event.scope_id) + if isinstance(record.event, PipelineChanged) + else None + ) ) OpenHCSMainWindow._on_synthetic_plate_generated( @@ -109,7 +111,9 @@ def test_main_window_loads_emitted_synthetic_pipeline_document( assert main_window.plate_manager_shown == 1 assert main_window.pipeline_editor_shown == 1 assert session.current_scope_id == scope_id - assert len(session.pipeline_steps(scope_id)) == len(document.pipeline_steps) == 8 + assert ( + len(session.pipeline_steps(scope_id)) == len(document.pipeline_steps) == 8 + ) assert [step.name for step in session.pipeline_steps(scope_id)] == [ step.name for step in document.pipeline_steps ] diff --git a/tests/unit/pyqt_gui/test_ui_agent_bridge.py b/tests/unit/pyqt_gui/test_ui_agent_bridge.py index 885f55da94..e4c0e23c27 100644 --- a/tests/unit/pyqt_gui/test_ui_agent_bridge.py +++ b/tests/unit/pyqt_gui/test_ui_agent_bridge.py @@ -2648,8 +2648,6 @@ def test_object_state_exact_field_paths_include_path_type_and_description() -> N ) - - def test_managed_window_save_action_returns_before_deferred_save_runs() -> None: app = QtApplicationAuthority.app() dispatcher = QueuedPostDispatcher() @@ -2697,13 +2695,17 @@ def test_managed_window_save_action_returns_before_deferred_save_runs() -> None: assert len(action.target_scope_ids) == 2 assert len(dispatcher.callbacks) == 1 assert ( - bridge.get_operation_status(UiBridgeOperationStatusRequest(result.receipt.bridge_operation_id)).status + bridge.get_operation_status( + UiBridgeOperationStatusRequest(result.receipt.bridge_operation_id) + ).status == "running" ) dispatcher.run_next() - operation = bridge.get_operation_status(UiBridgeOperationStatusRequest(result.receipt.bridge_operation_id)) + operation = bridge.get_operation_status( + UiBridgeOperationStatusRequest(result.receipt.bridge_operation_id) + ) assert window.save_count == 1 assert window.saved_close_window is False assert operation.status == "completed" @@ -3112,11 +3114,18 @@ def test_apply_document_returns_running_operation_before_queued_ui_apply_runs() assert result.receipt.accepted is True assert result.receipt.bridge_operation_id == result.operation_id assert operations.pre_count == 0 - assert bridge.get_operation_status(UiBridgeOperationStatusRequest(result.operation_id)).status == "running" + assert ( + bridge.get_operation_status( + UiBridgeOperationStatusRequest(result.operation_id) + ).status + == "running" + ) dispatcher.run_next() - operation = bridge.get_operation_status(UiBridgeOperationStatusRequest(result.operation_id)) + operation = bridge.get_operation_status( + UiBridgeOperationStatusRequest(result.operation_id) + ) assert operations.pre_count == 1 assert operations.post_count == 1 assert operation.status == "completed" @@ -3149,11 +3158,18 @@ def test_queued_apply_document_error_updates_operation_status() -> None: assert result.applied is False assert result.operation_id is not None assert result.receipt.accepted is True - assert bridge.get_operation_status(UiBridgeOperationStatusRequest(result.operation_id)).status == "running" + assert ( + bridge.get_operation_status( + UiBridgeOperationStatusRequest(result.operation_id) + ).status + == "running" + ) dispatcher.run_next() - operation = bridge.get_operation_status(UiBridgeOperationStatusRequest(result.operation_id)) + operation = bridge.get_operation_status( + UiBridgeOperationStatusRequest(result.operation_id) + ) assert operation.status == "completed" assert operation.outcome == "not_applied" assert operation.errors @@ -3435,15 +3451,15 @@ def test_two_ipc_ui_bridges_route_each_descriptor_to_its_exact_instance( service = UiBridgeService() first_status = service.invoke( UiBridgeStatusOperation, - connection= service.connection_from_args( + connection=service.connection_from_args( descriptor_file_path=first_binding.descriptor_file_path - ) + ), ) second_status = service.invoke( UiBridgeStatusOperation, - connection= service.connection_from_args( + connection=service.connection_from_args( descriptor_file_path=second_binding.descriptor_file_path - ) + ), ) assert first_status.reachable is True @@ -3894,9 +3910,7 @@ def document(mode): assert newly_selected_document.summary.current_selection_count == 1 assert newly_selected_state.selected_scope_ids == (second,) assert newly_selected_state.summary.current_selection_count == 1 - assert [ - row["scope_id"] for row in newly_selected_state.payload["rows"] - ] == [second] + assert [row["scope_id"] for row in newly_selected_state.payload["rows"]] == [second] assert newly_selected_state.payload["rows"][0]["selected"] is True assert ( newly_selected_state.current_revision_token @@ -4035,7 +4049,8 @@ def test_plate_manager_intermediate_plate_keeps_both_path_relations( tmp_path: Path, reverse_rows: bool ) -> None: roots = tuple( - str(tmp_path / name) for name in ("raw", "raw_stitched", "raw_stitched_analysis") + str(tmp_path / name) + for name in ("raw", "raw_stitched", "raw_stitched_analysis") ) ordered = roots[::-1] if reverse_rows else roots with dataset_gui(*ordered, selected=(1,)) as (gui, _scope_ids): @@ -4209,12 +4224,19 @@ def test_selected_workflow_returns_before_the_queued_operation_runs( ) assert started == [] assert len(dispatcher.callbacks) == 1 - assert bridge.get_operation_status(UiBridgeOperationStatusRequest(operation_id)).status == "running" + assert ( + bridge.get_operation_status( + UiBridgeOperationStatusRequest(operation_id) + ).status + == "running" + ) dispatcher.run_next() assert started == [((DATASET_SCOPE_ID,),)] - operation = bridge.get_operation_status(UiBridgeOperationStatusRequest(operation_id)) + operation = bridge.get_operation_status( + UiBridgeOperationStatusRequest(operation_id) + ) assert operation.status == "completed" assert operation.outcome == "accepted" @@ -4456,9 +4478,7 @@ def function_reference(function): assert state.payload["steps"][0]["function_ids"] == ["test:function_0"] assert state.payload["steps"][1]["function_ids"] == ["test:function_1"] assert selected_state.payload["steps"][0]["name"] == "step_two" - assert selected_state.payload["steps"][0]["function_ids"] == [ - "test:function_1" - ] + assert selected_state.payload["steps"][0]["function_ids"] == ["test:function_1"] assert selected_state.unchanged is False @@ -4466,9 +4486,7 @@ def _debug_actions(bridge): return _actions(bridge, PipelineDebugToolbarWidgetIdentity.require_value()) -def test_pipeline_debug_toolbar_actions_are_exposed_from_toolbar_declarations() -> ( - None -): +def test_pipeline_debug_toolbar_actions_are_exposed_from_toolbar_declarations() -> None: with pipeline_gui() as gui: actions = _debug_actions(_pipeline_editor_bridge(gui.pipeline_editor)) @@ -4556,7 +4574,9 @@ def test_pipeline_debug_toolbar_runtime_values_action_requires_debug_session( ) bridge = _pipeline_editor_bridge(gui.pipeline_editor) widget_id = PipelineDebugToolbarWidgetIdentity.require_value() - action = _debug_actions(bridge)[DebugToolbarAuxiliaryAction.RUNTIME_VALUES.value] + action = _debug_actions(bridge)[ + DebugToolbarAuxiliaryAction.RUNTIME_VALUES.value + ] result = bridge.invoke_action( UiActionInvokeRequest( @@ -4729,5 +4749,6 @@ def test_pipeline_debug_session_terminal_summary_retires_the_inspected_session() assert actions[DebugCommandType.RESTART.value]["enabled"] is False assert actions[DebugCommandType.STOP.value]["enabled"] is False assert ( - actions[DebugToolbarAuxiliaryAction.RUNTIME_VALUES.value]["enabled"] is False + actions[DebugToolbarAuxiliaryAction.RUNTIME_VALUES.value]["enabled"] + is False ) diff --git a/tests/unit/pyqt_gui/test_ui_bridge_live_measurements.py b/tests/unit/pyqt_gui/test_ui_bridge_live_measurements.py index 7c44f54be7..25d02cdba6 100644 --- a/tests/unit/pyqt_gui/test_ui_bridge_live_measurements.py +++ b/tests/unit/pyqt_gui/test_ui_bridge_live_measurements.py @@ -293,7 +293,9 @@ def test_live_measurement_surface_honors_plate_selection_and_populates_overview( "value": "1", } assert section["metrics"][1]["value"] == "3" - assert section["items"][0]["label"] == ("Neurite Outgrowth: PerNeuronMeasurements") + assert section["items"][0]["label"] == ( + "Neurite Outgrowth: PerNeuronMeasurements" + ) manager.selected = [manager.plates[1]] filtered = bridge.get_state_surface( diff --git a/tests/unit/pyqt_gui/test_ui_window_flash_snapshot.py b/tests/unit/pyqt_gui/test_ui_window_flash_snapshot.py index a91f70f080..fc5be727a2 100644 --- a/tests/unit/pyqt_gui/test_ui_window_flash_snapshot.py +++ b/tests/unit/pyqt_gui/test_ui_window_flash_snapshot.py @@ -94,24 +94,24 @@ def test_async_snapshot_uses_existing_operation_and_registered_response_contract bridge._mutation_gate._lock.release() form.update_parameter("number", 8) qtbot.waitUntil( - lambda: bridge.get_operation_status(UiBridgeOperationStatusRequest(accepted.operation_id)).completed_at_unix + lambda: bridge.get_operation_status( + UiBridgeOperationStatusRequest(accepted.operation_id) + ).completed_at_unix is not None, timeout=2000, ) - operation = bridge.get_operation_status(UiBridgeOperationStatusRequest(accepted.operation_id)) + operation = bridge.get_operation_status( + UiBridgeOperationStatusRequest(accepted.operation_id) + ) assert operation.status == "completed" contract = UiBridgeOperation.for_name(operation.identity.operation_name) - result = dataclass_from_mapping( - contract.result_type, operation.result_payload - ) + result = dataclass_from_mapping(contract.result_type, operation.result_payload) assert isinstance(result, UiWindowSnapshotResult) assert result.captured assert result.frame_condition is request.frame_condition assert result.observation.frame.has_maximum_alpha assert result.resource.path.endswith(".png") - wire_operation = dataclass_from_mapping( - type(operation), to_jsonable(operation) - ) + wire_operation = dataclass_from_mapping(type(operation), to_jsonable(operation)) assert wire_operation == operation wire_result = dataclass_from_mapping( contract.result_type, @@ -152,7 +152,12 @@ def test_async_snapshot_invalid_baseline_is_terminal_failed( ) assert not result.captured assert result.errors - assert bridge.get_operation_status(UiBridgeOperationStatusRequest(result.operation_id)).status == "failed" + assert ( + bridge.get_operation_status( + UiBridgeOperationStatusRequest(result.operation_id) + ).status + == "failed" + ) def test_noop_reset_quiet_operation_preserves_actual_interval_receipt( @@ -171,15 +176,17 @@ def test_noop_reset_quiet_operation_preserves_actual_interval_receipt( ) form.reset_buttons["number"].click() qtbot.waitUntil( - lambda: bridge.get_operation_status(UiBridgeOperationStatusRequest(accepted.operation_id)).completed_at_unix + lambda: bridge.get_operation_status( + UiBridgeOperationStatusRequest(accepted.operation_id) + ).completed_at_unix is not None, timeout=2000, ) - operation = bridge.get_operation_status(UiBridgeOperationStatusRequest(accepted.operation_id)) - contract = UiBridgeOperation.for_name(operation.identity.operation_name) - result = dataclass_from_mapping( - contract.result_type, operation.result_payload + operation = bridge.get_operation_status( + UiBridgeOperationStatusRequest(accepted.operation_id) ) + contract = UiBridgeOperation.for_name(operation.identity.operation_name) + result = dataclass_from_mapping(contract.result_type, operation.result_payload) assert result.captured assert ( result.observation.flash_start_count @@ -212,26 +219,25 @@ def test_failed_observation_frame_and_existing_trace_decode_through_registered_o ) form.update_parameter("number", 8) qtbot.waitUntil( - lambda: bridge.get_operation_status(UiBridgeOperationStatusRequest(accepted.operation_id)).completed_at_unix + lambda: bridge.get_operation_status( + UiBridgeOperationStatusRequest(accepted.operation_id) + ).completed_at_unix is not None, timeout=2000, ) - operation = bridge.get_operation_status(UiBridgeOperationStatusRequest(accepted.operation_id)) + operation = bridge.get_operation_status( + UiBridgeOperationStatusRequest(accepted.operation_id) + ) assert operation.status == "failed" contract = UiBridgeOperation.for_name(operation.identity.operation_name) - result = dataclass_from_mapping( - contract.result_type, operation.result_payload - ) + result = dataclass_from_mapping(contract.result_type, operation.result_payload) assert not result.captured and result.resource is None assert isinstance(result.observation.frame, FlashPaintFrame) assert result.observation.trace and len(result.observation.trace) <= 300 assert all( isinstance(record, FlashTraceRecord) for record in result.observation.trace ) - assert ( - dataclass_from_mapping(type(result), to_jsonable(result)) - == result - ) + assert dataclass_from_mapping(type(result), to_jsonable(result)) == result def test_snapshot_cli_derives_typed_observation_arguments(tmp_path): @@ -303,16 +309,16 @@ def _route_index(self): assert not accepted.navigated assert accepted.target_exposed is None qtbot.waitUntil( - lambda: bridge.get_operation_status(UiBridgeOperationStatusRequest(accepted.operation_id)).completed_at_unix + lambda: bridge.get_operation_status( + UiBridgeOperationStatusRequest(accepted.operation_id) + ).completed_at_unix is not None ) - operation = bridge.get_operation_status(UiBridgeOperationStatusRequest(accepted.operation_id)) - contract = UiBridgeOperation.for_name( - operation.identity.operation_name - ) - result = dataclass_from_mapping( - contract.result_type, operation.result_payload + operation = bridge.get_operation_status( + UiBridgeOperationStatusRequest(accepted.operation_id) ) + contract = UiBridgeOperation.for_name(operation.identity.operation_name) + result = dataclass_from_mapping(contract.result_type, operation.result_payload) assert isinstance(result, UiWindowNavigateResult) assert operation.status == "completed" assert selected == ["number"] @@ -338,7 +344,12 @@ def test_unowned_target_navigation_is_terminal_failed_not_pending( ) assert accepted.errors assert accepted.operation_id - assert bridge.get_operation_status(UiBridgeOperationStatusRequest(accepted.operation_id)).status == "failed" + assert ( + bridge.get_operation_status( + UiBridgeOperationStatusRequest(accepted.operation_id) + ).status + == "failed" + ) def test_tracker_observation_failure_uses_declared_operation_error_owner( diff --git a/tests/unit/pyqt_gui/test_zmq_client_service.py b/tests/unit/pyqt_gui/test_zmq_client_service.py index 668f86ef54..b62e3d094b 100644 --- a/tests/unit/pyqt_gui/test_zmq_client_service.py +++ b/tests/unit/pyqt_gui/test_zmq_client_service.py @@ -93,7 +93,9 @@ def endpoint_compatibility(self) -> EndpointApplicationCompatibility: ) -def test_async_connect_cancellation_retires_same_attempt_and_releases_ownership(monkeypatch): +def test_async_connect_cancellation_retires_same_attempt_and_releases_ownership( + monkeypatch, +): import openhcs.runtime.zmq_execution_client as client_module SlowFakeExecutionClient.instances = [] @@ -179,6 +181,7 @@ def new_connection_attempt(self): class BlockingConnectionAttempt(EndpointConnectionAttempt): def __init__(self): pass + def cancel(self) -> None: BlockingExecutionClient.attempt_cancelled.set() diff --git a/tests/unit/remote_sensing_witness_pipeline.py b/tests/unit/remote_sensing_witness_pipeline.py index b051cf76c7..68e78185d2 100644 --- a/tests/unit/remote_sensing_witness_pipeline.py +++ b/tests/unit/remote_sensing_witness_pipeline.py @@ -94,7 +94,8 @@ def pipeline_steps(napari_streaming_config=None) -> list: ) streaming = ( - {} if napari_streaming_config is None + {} + if napari_streaming_config is None else {"napari_streaming_config": napari_streaming_config} ) return [ diff --git a/tests/unit/test_ack_return_route_journey.py b/tests/unit/test_ack_return_route_journey.py index 37241227fd..2894eb767a 100644 --- a/tests/unit/test_ack_return_route_journey.py +++ b/tests/unit/test_ack_return_route_journey.py @@ -49,10 +49,15 @@ def stream_producer(pipe, port, mode, config): identity=ViewerStreamSourceIdentity(SyntheticMicroscope(), None), metadata=BatchViewerStreamSourceMetadata({"well": "synthetic"}), ), - producer=ViewerStreamProducer.from_identity(StreamProducerIdentity.pipeline_output( - output_kind="main", output_key="main", projection_key="main", - step_name="SyntheticTransport", pipeline_position=0, - )), + producer=ViewerStreamProducer.from_identity( + StreamProducerIdentity.pipeline_output( + output_kind="main", + output_key="main", + projection_key="main", + step_name="SyntheticTransport", + pipeline_position=0, + ) + ), ) try: for _ in range(2): @@ -62,10 +67,19 @@ def stream_producer(pipe, port, mode, config): break assert command == "send" completed.clear() - backend.save_batch([np.arange(12, dtype=np.uint16).reshape(3, 4)], - ["synthetic.tif"], stream_request=request) + backend.save_batch( + [np.arange(12, dtype=np.uint16).reshape(3, 4)], + ["synthetic.tif"], + stream_request=request, + ) tracker = GlobalQueueTrackerRegistry().get_tracker(port) - pipe.send((listener.return_route, tracker.get_progress(), len(backend._shared_memory_blocks))) + pipe.send( + ( + listener.return_route, + tracker.get_progress(), + len(backend._shared_memory_blocks), + ) + ) assert pipe.poll(6), "No controlled ACK check" command = pipe.recv() if command == "close": @@ -88,9 +102,13 @@ def test_two_native_stream_producers_keep_deferred_ack_ownership(mode, unix_sock directory = unix_socket_dir config = ZMQConfig(default_port=48000, ipc_socket_dir=str(directory)) pair = DataControlPortPairAuthority.acquire(config, transport_mode=mode) - server = NapariViewerServer(NapariViewerServerRequest( - port=pair.data_port, viewer_title="synthetic ACK transport", transport_mode=mode, - )) + server = NapariViewerServer( + NapariViewerServerRequest( + port=pair.data_port, + viewer_title="synthetic ACK transport", + transport_mode=mode, + ) + ) server.config = config server._running = True owners = [] @@ -100,9 +118,17 @@ def test_two_native_stream_producers_keep_deferred_ack_ownership(mode, unix_sock parent, child = Pipe() # Separate native interpreters, like the application: multiprocessing # siblings would share one resource-tracker and invalidate this proof. - process = Popen([sys.executable, __file__, str(child.fileno()), - str(pair.data_port), mode.value, str(directory)], - pass_fds=(child.fileno(),)) + process = Popen( + [ + sys.executable, + __file__, + str(child.fileno()), + str(pair.data_port), + mode.value, + str(directory), + ], + pass_fds=(child.fileno(),), + ) child.close() owners.append((process, parent)) routes = [] @@ -115,7 +141,9 @@ def test_two_native_stream_producers_keep_deferred_ack_ownership(mode, unix_sock accepted = server.accepted_stream_batches.get(timeout=3) item = accepted.items[0] assert item.payload.transfer.return_route == route - np.testing.assert_array_equal(item.data, np.arange(12, dtype=np.uint16).reshape(3, 4)) + np.testing.assert_array_equal( + item.data, np.arange(12, dtype=np.uint16).reshape(3, 4) + ) with pytest.raises(FileNotFoundError): SharedMemory(name=item.payload.shm_name) assert server.send_ack(item.payload.transfer) @@ -124,7 +152,9 @@ def test_two_native_stream_producers_keep_deferred_ack_ownership(mode, unix_sock if index == 0: pipe.send("send") route_again, progress, remaining_memory = receive(pipe) - assert route_again == route # One warm process owner, not a port per batch. + assert ( + route_again == route + ) # One warm process owner, not a port per batch. assert progress == (1, 2) second = server.accepted_stream_batches.get(timeout=3).items[0] assert server.send_ack(second.payload.transfer) @@ -157,5 +187,9 @@ def test_two_native_stream_producers_keep_deferred_ack_ownership(mode, unix_sock if __name__ == "__main__": - stream_producer(Connection(int(sys.argv[1])), int(sys.argv[2]), - TransportMode(sys.argv[3]), ZMQConfig(ipc_socket_dir=sys.argv[4])) + stream_producer( + Connection(int(sys.argv[1])), + int(sys.argv[2]), + TransportMode(sys.argv[3]), + ZMQConfig(ipc_socket_dir=sys.argv[4]), + ) diff --git a/tests/unit/test_acquisition_calibration_pipeline_journey.py b/tests/unit/test_acquisition_calibration_pipeline_journey.py index dd34e649c5..48a622e510 100644 --- a/tests/unit/test_acquisition_calibration_pipeline_journey.py +++ b/tests/unit/test_acquisition_calibration_pipeline_journey.py @@ -28,9 +28,7 @@ def test_htd_spacing_survives_gaussian_compile_execute_and_persist(tmp_path): plate = tmp_path / "plate" image_dir = plate / "TimePoint_1" image_dir.mkdir(parents=True) - (plate / "plate.HTD").write_text( - '"XSites", 1\n"YSites", 1\n"PixelSizeUM", 0.65' - ) + (plate / "plate.HTD").write_text('"XSites", 1\n"YSites", 1\n"PixelSizeUM", 0.65') pixels = np.random.default_rng(31).integers(0, 1000, (96, 96), dtype=np.uint16) filename = "A01_s001_w1_z001_t001.tif" tifffile.imwrite(image_dir / filename, pixels) diff --git a/tests/unit/test_aggregate_source_scope.py b/tests/unit/test_aggregate_source_scope.py index 318d135176..1b590e100a 100644 --- a/tests/unit/test_aggregate_source_scope.py +++ b/tests/unit/test_aggregate_source_scope.py @@ -7,26 +7,44 @@ import numpy as np import pytest -from openhcs.core.artifacts import ArtifactSpec, ImageArtifactType, MeasurementsArtifactType +from openhcs.core.artifacts import ( + ArtifactSpec, + ImageArtifactType, + MeasurementsArtifactType, +) from openhcs.core.component_group_scope import RuntimeExecutionAxisScope from openhcs.core.measurement_row_materialization import MeasurementSparseColumnarRows from openhcs.core.orchestrator.execution_result import ( - RuntimeContextObservation, RuntimeExecutionObservation, + RuntimeContextObservation, + RuntimeExecutionObservation, ) from openhcs.core.projected_image_output import SourceProjectedImageOutput from openhcs.core.runtime_artifact_values import ArtifactKey, RuntimeValue from openhcs.core.runtime_exports import ( - RuntimeExportExpectation, RuntimeExportObservation, runtime_export_failures, + RuntimeExportExpectation, + RuntimeExportObservation, + runtime_export_failures, ) from openhcs.core.runtime_image_values import ImagePayloadMetadata -from openhcs.core.runtime_measurements import MeasurementScope, MeasurementSubject, MeasurementTable -from openhcs.core.runtime_plane_projection import RuntimePlaneAxis, RuntimePlaneAxisValueProjection +from openhcs.core.runtime_measurements import ( + MeasurementScope, + MeasurementSubject, + MeasurementTable, +) +from openhcs.core.runtime_plane_projection import ( + RuntimePlaneAxis, + RuntimePlaneAxisValueProjection, +) from openhcs.core.runtime_stores import ( - RuntimeArtifactAddress, RuntimeArtifactLocation, RuntimeValueStore, + RuntimeArtifactAddress, + RuntimeArtifactLocation, + RuntimeValueStore, ) from openhcs.core.runtime_tabular_values import FieldSpec from openhcs.core.source_image_provenance import ( - RuntimeSourceImageProvenancePlane, SourceImageIdentity, SourceImageProvenancePlanes, + RuntimeSourceImageProvenancePlane, + SourceImageIdentity, + SourceImageProvenancePlanes, ) from openhcs.core.source_metadata import SourceVoxelSpacing from openhcs.core.steps.abstract import StepExecutionObservation @@ -45,12 +63,19 @@ def _source(sites): source_image_provenance_planes=SourceImageProvenancePlanes.from_components( paths=tuple(f"/input/field_{i}.tif" for i in range(len(sites))), component_metadata=tuple( - {"well": "A01", "channel": 2, "z_index": 1, "timepoint": 1, - **({"site": site} if site is not None else {})} + { + "well": "A01", + "channel": 2, + "z_index": 1, + "timepoint": 1, + **({"site": site} if site is not None else {}), + } for site in sites ), ), - ).payload_with(np.arange(len(sites) * 12, dtype=np.uint16).reshape(len(sites), 3, 4)) + ).payload_with( + np.arange(len(sites) * 12, dtype=np.uint16).reshape(len(sites), 3, 4) + ) class ProjectionAudit: @@ -80,8 +105,12 @@ def test_mixed_acquired_and_reduced_batch_uses_distinct_original_display_scopes( source = _source((1, 3)) declaration = DeclaredReducedField(np.full((3, 4), 720, dtype=np.uint16), []) reduced = ImageArtifactType.contextualize_output( - source, declaration, None, - RuntimePlaneAxisValueProjection.preserve(axis=RuntimePlaneAxis.SOURCE_BINDING, axis_size=2), + source, + declaration, + None, + RuntimePlaneAxisValueProjection.preserve( + axis=RuntimePlaneAxis.SOURCE_BINDING, axis_size=2 + ), ) assert declaration.calls == ["before", "after"] acquired_metadata = source.metadata.for_leading_source_plane(0) @@ -95,33 +124,52 @@ def test_mixed_acquired_and_reduced_batch_uses_distinct_original_display_scopes( request = kwargs["stream_request"] single = adapter.to_filemanager_kwargs(output)["stream_request"] assert request.display_config == single.display_config - assert request.source.metadata.component_metadata_for_item(output.path, 0) == single.source.metadata.component_metadata_for_item(output.path, 0) + assert request.source.metadata.component_metadata_for_item( + output.path, 0 + ) == single.source.metadata.component_metadata_for_item(output.path, 0) assert request.producer == adapter.values.stream_request.producer - assert request.viewer_transport == adapter.values.stream_request.viewer_transport - assert request.display_config.component_modes() == adapter.values.stream_request.display_config.component_modes() + assert ( + request.viewer_transport == adapter.values.stream_request.viewer_transport + ) + assert ( + request.display_config.component_modes() + == adapter.values.stream_request.display_config.component_modes() + ) assert output.metadata.source_voxel_spacing == SourceVoxelSpacing((0.5, 0.5)) - acquired_request, reduced_request = (kwargs["stream_request"] for _, kwargs in batches) + acquired_request, reduced_request = ( + kwargs["stream_request"] for _, kwargs in batches + ) assert "site" in acquired_request.display_config.COMPONENT_ORDER assert "site" not in reduced_request.display_config.COMPONENT_ORDER - assert aggregate.metadata.source_provenance.source_image_provenance_planes.contributor_count == 2 + assert ( + aggregate.metadata.source_provenance.source_image_provenance_planes.contributor_count + == 2 + ) np.testing.assert_array_equal(aggregate.content, declaration.data) @pytest.mark.parametrize("sites", ((), (None,), (1,), (1, 1), (1, None), (None, None))) -def test_absence_one_equal_or_incomplete_contributors_cannot_exempt_acquired_plane(sites): +def test_absence_one_equal_or_incomplete_contributors_cannot_exempt_acquired_plane( + sites, +): metadata = _source(sites).metadata.collapse_leading_plane_axis() components = { - "well": "A01", "channel": 2, "z_index": 1, "timepoint": 1, + "well": "A01", + "channel": 2, + "z_index": 1, + "timepoint": 1, } # Explicitly absent scalar identity, not a selected plane with a valid common site. metadata = ImagePayloadMetadata( source_component_metadata=components, - source_image_provenance_planes=SourceImageProvenancePlanes(( - RuntimeSourceImageProvenancePlane( - SourceImageIdentity(component_metadata=components), - contributors=metadata.source_image_provenance_planes.contributors, - ), - )), + source_image_provenance_planes=SourceImageProvenancePlanes( + ( + RuntimeSourceImageProvenancePlane( + SourceImageIdentity(component_metadata=components), + contributors=metadata.source_image_provenance_planes.contributors, + ), + ) + ), ) output = Output("unaddressed.tif", np.zeros((3, 4), dtype=np.uint16), metadata) adapter = _viewer_stream_backend_kwargs() @@ -134,64 +182,119 @@ def test_absence_one_equal_or_incomplete_contributors_cannot_exempt_acquired_pla def test_present_scalar_override_remains_required_despite_varied_contributors(): metadata = (_source((1, 3))).metadata.collapse_leading_plane_axis() - metadata = metadata.replace_fields(source_component_metadata={ - "well": "A01", "site": 7, "channel": 2, "z_index": 1, "timepoint": 1, - }) + metadata = metadata.replace_fields( + source_component_metadata={ + "well": "A01", + "site": 7, + "channel": 2, + "z_index": 1, + "timepoint": 1, + } + ) assert metadata.source_provenance.required_scalar_components(("site",)) == ("site",) output = Output("override.tif", np.zeros((3, 4), dtype=np.uint16), metadata) - request = _viewer_stream_backend_kwargs().to_filemanager_kwargs(output)["stream_request"] + request = _viewer_stream_backend_kwargs().to_filemanager_kwargs(output)[ + "stream_request" + ] assert "site" in request.display_config.COMPONENT_ORDER - assert request.source.metadata.component_metadata_for_item(output.path, 0)["site"] == 7 + assert ( + request.source.metadata.component_metadata_for_item(output.path, 0)["site"] == 7 + ) def _saved_table(tmp_path, *, producer, fields): table = MeasurementTable( - name="Cells", subject=MeasurementSubject(MeasurementScope.ARTIFACT, "Cells"), + name="Cells", + subject=MeasurementSubject(MeasurementScope.ARTIFACT, "Cells"), rows=MeasurementSparseColumnarRows.from_rows( - ({field: 1 for field in fields},), fields=tuple(FieldSpec(field) for field in fields), + ({field: 1 for field in fields},), + fields=tuple(FieldSpec(field) for field in fields), ), ) store = RuntimeValueStore() - record = store.record(RuntimeValue( - key=ArtifactKey(name="Cells", artifact_type=MeasurementsArtifactType, - scope=RuntimeExecutionAxisScope(axis_id="A01")), - data=table, - ), path=f"/runtime/{producer}/Cells", backend="memory") + record = store.record( + RuntimeValue( + key=ArtifactKey( + name="Cells", + artifact_type=MeasurementsArtifactType, + scope=RuntimeExecutionAxisScope(axis_id="A01"), + ), + data=table, + ), + path=f"/runtime/{producer}/Cells", + backend="memory", + ) filemanager = FileManager({"memory": MemoryStorageBackend()}) outputs = materialization_outputs( - MaterializationSpec(CsvOptions(filename_suffix=".csv")), table, - str(tmp_path / f"Cells_{producer}"), filemanager, + MaterializationSpec(CsvOptions(filename_suffix=".csv")), + table, + str(tmp_path / f"Cells_{producer}"), + filemanager, ) for output in outputs: # Persist exactly the original writer's content and declared path. Path(output.path).write_text(output.require_text_content(), encoding="utf-8") observation = StepExecutionObservation( - {RuntimeArtifactAddress.from_record(record): tuple(RuntimeArtifactLocation(output.path, "disk") for output in outputs)}, + { + RuntimeArtifactAddress.from_record(record): tuple( + RuntimeArtifactLocation(output.path, "disk") for output in outputs + ) + }, tuple(Path(output.path) for output in outputs), ) return record, observation def test_same_named_field_and_aggregate_tables_keep_exact_producer_ownership(tmp_path): - acquired, acquired_outputs = _saved_table(tmp_path, producer="step1", fields=("site", "source_image_name")) - aggregate, aggregate_outputs = _saved_table(tmp_path, producer="step3", fields=("area",)) - execution = RuntimeExecutionObservation(( - RuntimeContextObservation("A01", (acquired, aggregate), StepExecutionObservation.combine((acquired_outputs, aggregate_outputs))), - )) + acquired, acquired_outputs = _saved_table( + tmp_path, producer="step1", fields=("site", "source_image_name") + ) + aggregate, aggregate_outputs = _saved_table( + tmp_path, producer="step3", fields=("area",) + ) + execution = RuntimeExecutionObservation( + ( + RuntimeContextObservation( + "A01", + (acquired, aggregate), + StepExecutionObservation.combine((acquired_outputs, aggregate_outputs)), + ), + ) + ) transported = pickle.loads(pickle.dumps(execution)) exports = RuntimeExportObservation.from_runtime_observations((transported,)) - spec = ArtifactSpec.output(name="Cells", artifact_type=MeasurementsArtifactType, - materialization=MaterializationSpec(CsvOptions(filename_suffix=".csv"))) + spec = ArtifactSpec.output( + name="Cells", + artifact_type=MeasurementsArtifactType, + materialization=MaterializationSpec(CsvOptions(filename_suffix=".csv")), + ) expectation = RuntimeExportExpectation.from_output_specs((spec,)) - assert exports.matching_table_outputs(acquired) == acquired_outputs.runtime_export_paths - assert exports.matching_table_outputs(aggregate) == aggregate_outputs.runtime_export_paths - assert runtime_export_failures(expectation, exports, {"A01": (acquired, aggregate)}) == () + assert ( + exports.matching_table_outputs(acquired) + == acquired_outputs.runtime_export_paths + ) + assert ( + exports.matching_table_outputs(aggregate) + == aggregate_outputs.runtime_export_paths + ) + assert ( + runtime_export_failures(expectation, exports, {"A01": (acquired, aggregate)}) + == () + ) # Files and a same-named neighbor cannot replace the missing producer relation. - unowned = RuntimeExportObservation.from_output_paths(exports.output_files, outputs=aggregate_outputs) - failures = runtime_export_failures(expectation, unowned, {"A01": (acquired, aggregate)}) + unowned = RuntimeExportObservation.from_output_paths( + exports.output_files, outputs=aggregate_outputs + ) + failures = runtime_export_failures( + expectation, unowned, {"A01": (acquired, aggregate)} + ) assert len(failures) == 1 and "no matching table output" in failures[0] # A correct address does not waive its own required columns. acquired_outputs.runtime_export_paths[0].write_text("area\n1\n", encoding="utf-8") invalid = RuntimeExportObservation.from_runtime_observations((transported,)) - failures = runtime_export_failures(expectation, invalid, {"A01": (acquired, aggregate)}) - assert any("site" in failure and "source_image_name" in failure for failure in failures) + failures = runtime_export_failures( + expectation, invalid, {"A01": (acquired, aggregate)} + ) + assert any( + "site" in failure and "source_image_name" in failure for failure in failures + ) diff --git a/tests/unit/test_artifact_publication_journey.py b/tests/unit/test_artifact_publication_journey.py index 811d66ac05..52f4fca2cb 100644 --- a/tests/unit/test_artifact_publication_journey.py +++ b/tests/unit/test_artifact_publication_journey.py @@ -18,16 +18,24 @@ from openhcs.core.orchestrator.orchestrator import PipelineOrchestrator from openhcs.core.progress import set_progress_queue from openhcs.core.artifacts import ( - ArtifactSpec, ObjectLabelsArtifactType, MeasurementsArtifactType, + ArtifactSpec, + ObjectLabelsArtifactType, + MeasurementsArtifactType, ArtifactMeasurementSubjectRelation, ) from openhcs.core.function_patterns import compile_function_pattern from openhcs.core.aligned_image_payload import AlignedImageSliceContext from openhcs.core.measurement_row_materialization import MeasurementSparseColumnarRows from openhcs.core.runtime_tabular_values import FieldSpec -from openhcs.core.pipeline.function_contracts import (artifact_inputs, artifact_outputs, object_label_input_execution_mode) +from openhcs.core.pipeline.function_contracts import ( + artifact_inputs, + artifact_outputs, + object_label_input_execution_mode, +) from openhcs.core.memory import numpy as numpy_decorator -from openhcs.core.runtime_object_label_building import SourceImageObjectLabelBuildRequest +from openhcs.core.runtime_object_label_building import ( + SourceImageObjectLabelBuildRequest, +) from openhcs.core.runtime_object_labels import ObjectLabelValue from openhcs.core.processing_contracts import ( Pure2DContract, @@ -59,22 +67,29 @@ @numpy_decorator(contract=Pure2DContract) def tiny_objects(image: np.ndarray) -> ObjectLabelValue: return SourceImageObjectLabelBuildRequest( - image=image, labels=(np.asarray(image) > 0).astype(np.int32), - declared_object_count=1, declared_object_ids=(1,), + image=image, + labels=(np.asarray(image) > 0).astype(np.int32), + declared_object_count=1, + declared_object_ids=(1,), ).payload() -@artifact_inputs(ArtifactSpec.input("objects", ObjectLabelsArtifactType, parameter_name="labels")) +@artifact_inputs( + ArtifactSpec.input("objects", ObjectLabelsArtifactType, parameter_name="labels") +) @numpy_decorator(contract=Pure3DContract) @object_label_input_execution_mode(FullStackLabels) def render_labels(image: np.ndarray, labels: ObjectLabelValue) -> np.ndarray: return convert_objects_to_image(image, labels, image_mode=ImageMode.UINT16) -@artifact_outputs(ArtifactSpec.output( - "counts", MeasurementsArtifactType, - relations=(ArtifactMeasurementSubjectRelation(),), -)) +@artifact_outputs( + ArtifactSpec.output( + "counts", + MeasurementsArtifactType, + relations=(ArtifactMeasurementSubjectRelation(),), + ) +) @numpy_decorator(contract=Pure2DContract) def tiny_counts(image: np.ndarray): return image, MeasurementSparseColumnarRows.from_rows( @@ -99,10 +114,21 @@ def _plate(root: Path, site_count: int = 1) -> Path: for site in range(1, site_count + 1): path = root / f"A01_s{site:03d}_w1_z001_t001.tif" tifffile.imwrite(path, pixels) - address = OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, site), (Microscopy.Channel, 1), (Microscopy.ZIndex, 1), (Microscopy.Timepoint, 1))) + address = OpenHCSPlaneAddress( + ( + (Microscopy.Well, "A01"), + (Microscopy.Site, site), + (Microscopy.Channel, 1), + (Microscopy.ZIndex, 1), + (Microscopy.Timepoint, 1), + ) + ) metadata = ImagePayloadMetadata( source_path=str(path), - source_component_metadata={**address.as_component_metadata(), "extension": ".tif"}, + source_component_metadata={ + **address.as_component_metadata(), + "extension": ".tif", + }, source_voxel_spacing=SourceVoxelSpacing((0.65, 0.65)), ) projection = SourcePlaneProjection( @@ -113,7 +139,9 @@ def _plate(root: Path, site_count: int = 1) -> Path: ) projections.append(projection) projection_paths.append((projection, path.name)) - document = SourceProjectionMetadataSerializer(SourceSchemaFilenameParser()).metadata_dict( + document = SourceProjectionMetadataSerializer( + SourceSchemaFilenameParser() + ).metadata_dict( SourceProjectionSet(tuple(projections)), microscope_handler_name="openhcsdata", source_filename_parser_name="SourceSchemaFilenameParser", @@ -122,7 +150,9 @@ def _plate(root: Path, site_count: int = 1) -> Path: main=True, projection_paths=tuple(projection_paths), ) - (root / "openhcs_metadata.json").write_text(json.dumps({"subdirectories": {".": document}})) + (root / "openhcs_metadata.json").write_text( + json.dumps({"subdirectories": {".": document}}) + ) return root @@ -130,7 +160,9 @@ def _compile(root: Path, results: Path, main_filter=0): plate = _plate(root / "source") config = PipelineConfig( materialization_results_path=results, - path_planning_config=LazyPathPlanningConfig(output_dir_suffix="_out", partition_filter=main_filter), + path_planning_config=LazyPathPlanningConfig( + output_dir_suffix="_out", partition_filter=main_filter + ), processing_config=LazyProcessingConfig( group_by=Microscopy.Channel, variable_components=[Microscopy.Site], @@ -145,7 +177,9 @@ def _compile(root: Path, results: Path, main_filter=0): func=render_labels, name="render", step_materialization_config=LazyStepMaterializationConfig( - enabled=True, output_dir_suffix="_out", sub_dir="converted", + enabled=True, + output_dir_suffix="_out", + sub_dir="converted", ), ), FunctionStep(func=tiny_counts, name="counts"), @@ -154,13 +188,17 @@ def _compile(root: Path, results: Path, main_filter=0): def test_public_planning_rejects_external_results_root(tmp_path): - with pytest.raises(ValueError, match="materialization_results_path.*output plate root"): + with pytest.raises( + ValueError, match="materialization_results_path.*output plate root" + ): _compile(tmp_path, tmp_path / "external-results") @pytest.mark.parametrize("absolute", (False, True)) @pytest.mark.parametrize("main_filter", (0, 1)) -def test_converted_checkpoint_has_typed_address_on_final_reconciliation(tmp_path, absolute, main_filter): +def test_converted_checkpoint_has_typed_address_on_final_reconciliation( + tmp_path, absolute, main_filter +): results = tmp_path / "source_out" / "results" if absolute else Path("results") orchestrator, steps, bundle = _compile(tmp_path, results, main_filter) for context in bundle.runtime_contexts.values(): @@ -183,11 +221,27 @@ def test_converted_checkpoint_has_typed_address_on_final_reconciliation(tmp_path assert entry["image_files"] == [projection["virtual_path"]] # Reopen the durable typed projection. Zero automatic main-flow output must # not invent a default main when independently saved targets coexist. - reopened_metadata = orchestrator.microscope_handler.metadata_handler.load_metadata_document(saved.parent) - reopened = VirtualWorkspaceSourceProjection.from_openhcs_metadata(saved.parent, reopened_metadata) + reopened_metadata = ( + orchestrator.microscope_handler.metadata_handler.load_metadata_document( + saved.parent + ) + ) + reopened = VirtualWorkspaceSourceProjection.from_openhcs_metadata( + saved.parent, reopened_metadata + ) retained = reopened.source_projections_by_virtual_path[projection["virtual_path"]] - assert retained.address == OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, 1), (Microscopy.Channel, 1), (Microscopy.ZIndex, 1), (Microscopy.Timepoint, 1))) - assert retained.image_metadata.source_voxel_spacing == SourceVoxelSpacing((0.65, 0.65)) + assert retained.address == OpenHCSPlaneAddress( + ( + (Microscopy.Well, "A01"), + (Microscopy.Site, 1), + (Microscopy.Channel, 1), + (Microscopy.ZIndex, 1), + (Microscopy.Timepoint, 1), + ) + ) + assert retained.image_metadata.source_voxel_spacing == SourceVoxelSpacing( + (0.65, 0.65) + ) assert retained.image_metadata.source_path == projected_metadata.source_path if main_filter: reopened_plate = PipelineOrchestrator(saved.parent).initialize() @@ -205,7 +259,9 @@ def test_converted_checkpoint_has_typed_address_on_final_reconciliation(tmp_path def test_planning_rejects_relative_results_escape(tmp_path): - with pytest.raises(ValueError, match="materialization_results_path.*output plate root"): + with pytest.raises( + ValueError, match="materialization_results_path.*output plate root" + ): _compile(tmp_path, Path("../external-results")) @@ -217,7 +273,9 @@ def test_planning_preserves_symlinked_plate_identity(tmp_path): _orchestrator, _steps, bundle = _compile(alias, Path("results")) for context in bundle.runtime_contexts.values(): for plan in context.step_plans.values(): - assert Path(plan.analysis_results_dir).is_relative_to(plan.output_dataset_root) + assert Path(plan.analysis_results_dir).is_relative_to( + plan.output_dataset_root + ) def test_planning_rejects_symlinked_results_escape(tmp_path): @@ -226,7 +284,9 @@ def test_planning_rejects_symlinked_results_escape(tmp_path): output = tmp_path / "source_out" output.mkdir() (output / "results").symlink_to(outside, target_is_directory=True) - with pytest.raises(ValueError, match="materialization_results_path.*output plate root"): + with pytest.raises( + ValueError, match="materialization_results_path.*output plate root" + ): _compile(tmp_path, Path("results")) @@ -234,10 +294,13 @@ def test_new_unnamed_replacement_and_table_passthrough_contracts(): def another_image(image): return image + 1 - @artifact_outputs(ArtifactSpec.output( - "counts", MeasurementsArtifactType, - relations=(ArtifactMeasurementSubjectRelation(),), - )) + @artifact_outputs( + ArtifactSpec.output( + "counts", + MeasurementsArtifactType, + relations=(ArtifactMeasurementSubjectRelation(),), + ) + ) def table_only(image): return image diff --git a/tests/unit/test_automatic_image_role_materialization_435.py b/tests/unit/test_automatic_image_role_materialization_435.py index bb666aa7f8..7f89a73318 100644 --- a/tests/unit/test_automatic_image_role_materialization_435.py +++ b/tests/unit/test_automatic_image_role_materialization_435.py @@ -11,14 +11,25 @@ from polystore.filemanager import FileManager from polystore.memory import MemoryStorageBackend -from tests.unit.test_function_outputs import context_stub, function_step_plan, record_output_path +from tests.unit.test_function_outputs import ( + context_stub, + function_step_plan, + record_output_path, +) from openhcs.constants.constants import Backend -from openhcs.core.aligned_image_payload import AlignedImageSliceContext, ImagePayloadBundleContext +from openhcs.core.aligned_image_payload import ( + AlignedImageSliceContext, + ImagePayloadBundleContext, +) from openhcs.core.artifacts import ArtifactOutputPlan, ArtifactSpec, ImageArtifactType from openhcs.core.compiled_step_plan import RuntimeArtifactMaterializationPlan from openhcs.core.pipeline.artifact_planning import ArtifactOutputMaterializationPlanner from openhcs.core.runtime_artifact_values import RuntimeValue -from openhcs.core.runtime_image_values import ImageMetadataPayload, ImagePayloadMetadata, ImagePayloadMetadataCompositionMode +from openhcs.core.runtime_image_values import ( + ImageMetadataPayload, + ImagePayloadMetadata, + ImagePayloadMetadataCompositionMode, +) from openhcs.core.runtime_stores import RuntimeValueStore from openhcs.core.source_metadata import SourceVoxelSpacing from openhcs.core.source_projection import SourceProjectionMetadataSerializer @@ -26,7 +37,9 @@ from openhcs.core.steps.function_output_identity import ( FunctionOutputIdentity, ) -from openhcs.core.steps.function_artifact_materialization import ArtifactMaterializationTargetPlan +from openhcs.core.steps.function_artifact_materialization import ( + ArtifactMaterializationTargetPlan, +) from openhcs.core.steps.function_outputs import ( PrimaryImageMetadataTarget, ) @@ -35,11 +48,15 @@ @pytest.mark.parametrize("plane_count", (1, 2)) -def test_automatic_two_role_images_publish_their_actual_saved_occurrences(tmp_path, plane_count): - filemanager = FileManager({ - Backend.DISK.value: DiskStorageBackend(), - Backend.MEMORY.value: MemoryStorageBackend(), - }) +def test_automatic_two_role_images_publish_their_actual_saved_occurrences( + tmp_path, plane_count +): + filemanager = FileManager( + { + Backend.DISK.value: DiskStorageBackend(), + Backend.MEMORY.value: MemoryStorageBackend(), + } + ) context = context_stub(filemanager, parser=SourceSchemaFilenameParser()) context.microscope_handler.source_name = "openhcsdata" context.runtime_value_store = RuntimeValueStore() @@ -56,17 +73,25 @@ def test_automatic_two_role_images_publish_their_actual_saved_occurrences(tmp_pa plan.sub_dir = "images" plan.analysis_results_dir = str(plan.output_dir) plan.runtime_artifact_materialization = RuntimeArtifactMaterializationPlan( - persistent_enabled=True, persistent_backend=Backend.DISK.value, + persistent_enabled=True, + persistent_backend=Backend.DISK.value, ) context.step_plans = {plan.step_index: plan} plans = tuple( ArtifactOutputPlan( - name=name, path=f"/memory/{name}.pkl", artifact_type=ImageArtifactType, + name=name, + path=f"/memory/{name}.pkl", + artifact_type=ImageArtifactType, producer_step_index=plan.step_index, producer_step_scope_id=plan.step_scope_id, producer_step_name=plan.step_name, materialization=ArtifactOutputMaterializationPlanner.materialization_for( - ArtifactSpec(name=name, artifact_type=ImageArtifactType, plan_type=ArtifactOutputPlan), (), + ArtifactSpec( + name=name, + artifact_type=ImageArtifactType, + plan_type=ArtifactOutputPlan, + ), + (), ), ) for name in ("raw_body_processing_role", "capped_outgrowth_processing_role") @@ -80,8 +105,12 @@ def test_automatic_two_role_images_publish_their_actual_saved_occurrences(tmp_pa (np.arange(20, dtype=np.float32).reshape(4, 5) + index) * factor, ImagePayloadMetadata( source_component_metadata={ - "well": "A01", "site": "1", "channel": "2", "z_index": str(index), - "timepoint": "1", "extension": ".tif", + "well": "A01", + "site": "1", + "channel": "2", + "z_index": str(index), + "timepoint": "1", + "extension": ".tif", }, source_image_names=(output.name,), source_voxel_spacing=SourceVoxelSpacing((1.3556, 1.3556)), @@ -90,115 +119,195 @@ def test_automatic_two_role_images_publish_their_actual_saved_occurrences(tmp_pa for index in range(1, plane_count + 1) ) value = ( - payloads[0] if plane_count == 1 else ImagePayloadBundleContext.from_payloads( - payloads, metadata_mode=ImagePayloadMetadataCompositionMode.STACK, + payloads[0] + if plane_count == 1 + else ImagePayloadBundleContext.from_payloads( + payloads, + metadata_mode=ImagePayloadMetadataCompositionMode.STACK, ).compose() ) context.runtime_value_store.record( RuntimeValue.normalize(output, value, axis_id="A01"), - path=output.path, backend=Backend.MEMORY.value, + path=output.path, + backend=Backend.MEMORY.value, ) expected[output.name] = [] for payload in payloads: identity = FunctionOutputIdentity.from_filename_metadata( - context.microscope_handler.parser, payload.metadata, + context.microscope_handler.parser, + payload.metadata, ).with_filename_qualifier(output_context.output_key) - destination = plan.output_dir / identity.filename(context.microscope_handler.parser) + destination = plan.output_dir / identity.filename( + context.microscope_handler.parser + ) filemanager.ensure_directory(plan.output_dir, Backend.MEMORY.value) filemanager.ensure_directory(plan.output_dir, Backend.DISK.value) filemanager.save(payload, str(destination), Backend.MEMORY.value) filemanager.save(payload.data, str(destination), Backend.DISK.value) record_output_path( - context, plan, str(destination), output_context=output_context, - image_metadata=payload.metadata, identity=identity, + context, + plan, + str(destination), + output_context=output_context, + image_metadata=payload.metadata, + identity=identity, ) expected[output.name].append((destination, payload.data)) saved = ArtifactMaterializationTargetPlan.materialize(context, plan) - target = replace(PrimaryImageMetadataTarget.from_plan(plan), artifact_materializations=saved) + target = replace( + PrimaryImageMetadataTarget.from_plan(plan), artifact_materializations=saved + ) # The original implementation raises its real duplicate projection guard # here: both automatic materializations used the same unqualified path. structured = target.produced_projection_entries(context, plan) structured = SourceProjectionMetadataSerializer.projection_fields( structured.projection_paths ) - assert {item["source_alias"] for item in structured["source_projection"]} == set(expected) + assert {item["source_alias"] for item in structured["source_projection"]} == set( + expected + ) assert len(structured["source_projection"]) == 2 * plane_count for materialized in saved: outputs = materialized.outputs_for_backend(Backend.DISK.value) assert len(outputs) == plane_count - for actual, (path, pixels) in zip(outputs, expected[materialized.materialization.output_plan.name], strict=True): + for actual, (path, pixels) in zip( + outputs, + expected[materialized.materialization.output_plan.name], + strict=True, + ): assert actual.path == str(path) np.testing.assert_array_equal(tifffile.imread(path), pixels) target.write(context, produced_plan=plan) document = json.loads((tmp_path / "openhcs_metadata.json").read_text()) - reopened = VirtualWorkspaceSourceProjection.from_openhcs_metadata(tmp_path, document) + reopened = VirtualWorkspaceSourceProjection.from_openhcs_metadata( + tmp_path, document + ) for name, expected_planes in expected.items(): for path, pixels in expected_planes: - projection = reopened.source_projections_by_virtual_path[str(path.relative_to(tmp_path))] + projection = reopened.source_projections_by_virtual_path[ + str(path.relative_to(tmp_path)) + ] assert projection.source_alias == name assert projection.address.as_component_metadata()["channel"] == "2" - assert projection.image_metadata.source_voxel_spacing == SourceVoxelSpacing((1.3556, 1.3556)) - np.testing.assert_array_equal(tifffile.imread(tmp_path / projection.ref.backend_address), pixels) + assert projection.image_metadata.source_voxel_spacing == SourceVoxelSpacing( + (1.3556, 1.3556) + ) + np.testing.assert_array_equal( + tifffile.imread(tmp_path / projection.ref.backend_address), pixels + ) -@pytest.mark.parametrize("purpose", ( - "explicit_source", "explicit_template", "terminal_template", - "explicit_artifact", "terminal_artifact", "unnamed_terminal", "unnamed_explicit", - "explicit_source_declared_terminal", "terminal_source_declared_explicit", -)) -def test_retained_role_policy_preserves_authored_and_unnamed_image_paths(tmp_path, purpose): +@pytest.mark.parametrize( + "purpose", + ( + "explicit_source", + "explicit_template", + "terminal_template", + "explicit_artifact", + "terminal_artifact", + "unnamed_terminal", + "unnamed_explicit", + "explicit_source_declared_terminal", + "terminal_source_declared_explicit", + ), +) +def test_retained_role_policy_preserves_authored_and_unnamed_image_paths( + tmp_path, purpose +): from openhcs.processing.materialization import ( - ImageFileOptions, MaterializationSpec, MaterializedFilenameIdentity, - TerminalMaterializationSpec, materialization_outputs, + ImageFileOptions, + MaterializationSpec, + MaterializedFilenameIdentity, + TerminalMaterializationSpec, + materialization_outputs, ) options = ImageFileOptions( filename_suffix=".tif", - relative_path_template=("authored/plane_{index}.tif" if purpose.endswith("template") else None), + relative_path_template=( + "authored/plane_{index}.tif" if purpose.endswith("template") else None + ), filename_identity=( - MaterializedFilenameIdentity.ARTIFACT_NAME if purpose.endswith("artifact") + MaterializedFilenameIdentity.ARTIFACT_NAME + if purpose.endswith("artifact") else MaterializedFilenameIdentity.SOURCE_IDENTITY ), ) spec = ( - TerminalMaterializationSpec(options) if purpose.startswith("terminal") or purpose == "unnamed_terminal" + TerminalMaterializationSpec(options) + if purpose.startswith("terminal") or purpose == "unnamed_terminal" else MaterializationSpec(options) ) declared_spec = ( - TerminalMaterializationSpec(options) if purpose == "explicit_source_declared_terminal" - else MaterializationSpec(options) if purpose == "terminal_source_declared_explicit" - else spec + TerminalMaterializationSpec(options) + if purpose == "explicit_source_declared_terminal" + else ( + MaterializationSpec(options) + if purpose == "terminal_source_declared_explicit" + else spec + ) ) - output_plan = None if purpose in ("unnamed_terminal", "unnamed_explicit") else ArtifactOutputPlan( - name="DeclaredRole", path="/memory/DeclaredRole.pkl", - artifact_type=ImageArtifactType, materialization=declared_spec, + output_plan = ( + None + if purpose in ("unnamed_terminal", "unnamed_explicit") + else ArtifactOutputPlan( + name="DeclaredRole", + path="/memory/DeclaredRole.pkl", + artifact_type=ImageArtifactType, + materialization=declared_spec, + ) ) payloads = tuple( - ImageMetadataPayload(np.ones((4, 5), dtype=np.float32) * index, ImagePayloadMetadata( - source_component_metadata={ - "well": "A01", "site": "1", "channel": "2", "z_index": str(index), - "timepoint": "1", "extension": ".tif", - }, - )) + ImageMetadataPayload( + np.ones((4, 5), dtype=np.float32) * index, + ImagePayloadMetadata( + source_component_metadata={ + "well": "A01", + "site": "1", + "channel": "2", + "z_index": str(index), + "timepoint": "1", + "extension": ".tif", + }, + ), + ) for index in (1, 2) ) value = ImagePayloadBundleContext.from_payloads( - payloads, metadata_mode=ImagePayloadMetadataCompositionMode.STACK, + payloads, + metadata_mode=ImagePayloadMetadataCompositionMode.STACK, ).compose() filemanager = FileManager({Backend.MEMORY.value: MemoryStorageBackend()}) context = context_stub(filemanager, parser=SourceSchemaFilenameParser()) outputs = materialization_outputs( - spec, value, str(tmp_path / "AuthoredBase"), filemanager, - context=context, output_plan=output_plan, + spec, + value, + str(tmp_path / "AuthoredBase"), + filemanager, + context=context, + output_plan=output_plan, variable_components=(Microscopy.ZIndex,), ) expected_paths = ( - ("authored/plane_1.tif", "authored/plane_2.tif") if purpose.endswith("template") - else ("AuthoredBase.tif",) if purpose.endswith("artifact") - else ("A01_s001_w2_z001_t001_DeclaredRole.tif", "A01_s001_w2_z002_t001_DeclaredRole.tif") if purpose == "terminal_source_declared_explicit" - else ("A01_s001_w2_z001_t001.tif", "A01_s001_w2_z002_t001.tif") + ("authored/plane_1.tif", "authored/plane_2.tif") + if purpose.endswith("template") + else ( + ("AuthoredBase.tif",) + if purpose.endswith("artifact") + else ( + ( + "A01_s001_w2_z001_t001_DeclaredRole.tif", + "A01_s001_w2_z002_t001_DeclaredRole.tif", + ) + if purpose == "terminal_source_declared_explicit" + else ("A01_s001_w2_z001_t001.tif", "A01_s001_w2_z002_t001.tif") + ) + ) + ) + assert ( + tuple(Path(output.path).relative_to(tmp_path).as_posix() for output in outputs) + == expected_paths ) - assert tuple(Path(output.path).relative_to(tmp_path).as_posix() for output in outputs) == expected_paths if len(outputs) == 2: for actual, expected in zip(outputs, payloads, strict=True): np.testing.assert_array_equal(actual.content, expected.data) @@ -207,60 +316,130 @@ def test_retained_role_policy_preserves_authored_and_unnamed_image_paths(tmp_pat def test_authored_duplicate_template_still_fails_before_any_save(tmp_path): - from openhcs.processing.materialization import ImageFileOptions, TerminalMaterializationSpec, materialization_outputs + from openhcs.processing.materialization import ( + ImageFileOptions, + TerminalMaterializationSpec, + materialization_outputs, + ) - spec = TerminalMaterializationSpec(ImageFileOptions( - filename_suffix=".tif", relative_path_template="same.tif", - )) - output_plan = ArtifactOutputPlan(name="NamedRole", path="/memory/role.pkl", artifact_type=ImageArtifactType, materialization=spec) + spec = TerminalMaterializationSpec( + ImageFileOptions( + filename_suffix=".tif", + relative_path_template="same.tif", + ) + ) + output_plan = ArtifactOutputPlan( + name="NamedRole", + path="/memory/role.pkl", + artifact_type=ImageArtifactType, + materialization=spec, + ) payloads = tuple( - ImageMetadataPayload(np.ones((4, 5), dtype=np.float32), ImagePayloadMetadata( - source_component_metadata={"well": "A01", "site": "1", "channel": "2", "z_index": str(index), "timepoint": "1", "extension": ".tif"}, - )) for index in (1, 2) + ImageMetadataPayload( + np.ones((4, 5), dtype=np.float32), + ImagePayloadMetadata( + source_component_metadata={ + "well": "A01", + "site": "1", + "channel": "2", + "z_index": str(index), + "timepoint": "1", + "extension": ".tif", + }, + ), + ) + for index in (1, 2) ) - value = ImagePayloadBundleContext.from_payloads(payloads, metadata_mode=ImagePayloadMetadataCompositionMode.STACK).compose() + value = ImagePayloadBundleContext.from_payloads( + payloads, metadata_mode=ImagePayloadMetadataCompositionMode.STACK + ).compose() manager = FileManager({Backend.MEMORY.value: MemoryStorageBackend()}) with pytest.raises(ValueError, match="produced duplicate paths"): - materialization_outputs(spec, value, str(tmp_path / "role"), manager, - context=context_stub(manager, parser=SourceSchemaFilenameParser()), - output_plan=output_plan) + materialization_outputs( + spec, + value, + str(tmp_path / "role"), + manager, + context=context_stub(manager, parser=SourceSchemaFilenameParser()), + output_plan=output_plan, + ) assert not (tmp_path / "same.tif").exists() @pytest.mark.parametrize("suffix", (".tif", ".png", ".ome.tif", ".labels.tif")) @pytest.mark.parametrize("plane_count", (1, 2)) -def test_retained_image_qualifier_respects_complete_writer_suffix(tmp_path, suffix, plane_count): - from openhcs.core.steps.function_artifact_materialization import RuntimeArtifactMaterialization - from openhcs.processing.materialization import ImageFileOptions, TerminalMaterializationSpec +def test_retained_image_qualifier_respects_complete_writer_suffix( + tmp_path, suffix, plane_count +): + from openhcs.core.steps.function_artifact_materialization import ( + RuntimeArtifactMaterialization, + ) + from openhcs.processing.materialization import ( + ImageFileOptions, + TerminalMaterializationSpec, + ) manager = FileManager({Backend.MEMORY.value: MemoryStorageBackend()}) context = context_stub(manager, parser=SourceSchemaFilenameParser()) - plan = function_step_plan("Retained format role", variable_components=(Microscopy.ZIndex,) if plane_count > 1 else ()) + plan = function_step_plan( + "Retained format role", + variable_components=(Microscopy.ZIndex,) if plane_count > 1 else (), + ) plan.output_dir = tmp_path / "images" plan.analysis_results_dir = str(plan.output_dir) output_plan = ArtifactOutputPlan( - name="Role Name", path="/memory/role.pkl", artifact_type=ImageArtifactType, - materialization=TerminalMaterializationSpec(ImageFileOptions(filename_suffix=suffix)), + name="Role Name", + path="/memory/role.pkl", + artifact_type=ImageArtifactType, + materialization=TerminalMaterializationSpec( + ImageFileOptions(filename_suffix=suffix) + ), ) payloads = tuple( - ImageMetadataPayload(np.ones((4, 5), dtype=np.uint8) * index, ImagePayloadMetadata( - source_component_metadata={"well": "A01", "site": "1", "channel": "2", "z_index": str(index), "timepoint": "1", "extension": ".tif"}, - )) for index in range(1, plane_count + 1) + ImageMetadataPayload( + np.ones((4, 5), dtype=np.uint8) * index, + ImagePayloadMetadata( + source_component_metadata={ + "well": "A01", + "site": "1", + "channel": "2", + "z_index": str(index), + "timepoint": "1", + "extension": ".tif", + }, + ), + ) + for index in range(1, plane_count + 1) + ) + value = ( + payloads[0] + if plane_count == 1 + else ImagePayloadBundleContext.from_payloads( + payloads, + metadata_mode=ImagePayloadMetadataCompositionMode.STACK, + ).compose() ) - value = payloads[0] if plane_count == 1 else ImagePayloadBundleContext.from_payloads( - payloads, metadata_mode=ImagePayloadMetadataCompositionMode.STACK, - ).compose() store = RuntimeValueStore() - record = store.record(RuntimeValue.normalize(output_plan, value, axis_id="A01"), path=output_plan.path, backend=Backend.MEMORY.value) + record = store.record( + RuntimeValue.normalize(output_plan, value, axis_id="A01"), + path=output_plan.path, + backend=Backend.MEMORY.value, + ) outputs = RuntimeArtifactMaterialization.from_record( - output_plan=output_plan, record=record, plan=plan, context=context, + output_plan=output_plan, + record=record, + plan=plan, + context=context, ).outputs(plan, context) assert len(outputs) == plane_count for index, (output, payload) in enumerate(zip(outputs, payloads, strict=True), 1): identity = FunctionOutputIdentity.from_filename_metadata( - context.microscope_handler.parser, payload.metadata, + context.microscope_handler.parser, + payload.metadata, ).with_filename_qualifier(output_plan.name) - expected = replace(identity, extension=suffix).filename(context.microscope_handler.parser) + expected = replace(identity, extension=suffix).filename( + context.microscope_handler.parser + ) assert Path(output.path).name == expected assert Path(output.path).name.endswith("_Role_Name" + suffix) assert output.metadata.source_component_metadata["channel"] == "2" @@ -269,60 +448,121 @@ def test_retained_image_qualifier_respects_complete_writer_suffix(tmp_path, suff def test_distinct_aliases_that_normalize_to_one_filename_still_conflict(tmp_path): - from openhcs.core.steps.function_artifact_materialization import RuntimeArtifactMaterialization - from openhcs.processing.materialization import ImageFileOptions, TerminalMaterializationSpec + from openhcs.core.steps.function_artifact_materialization import ( + RuntimeArtifactMaterialization, + ) + from openhcs.processing.materialization import ( + ImageFileOptions, + TerminalMaterializationSpec, + ) manager = FileManager({Backend.MEMORY.value: MemoryStorageBackend()}) context = context_stub(manager, parser=SourceSchemaFilenameParser()) plan = function_step_plan("Ambiguous role spelling") plan.output_dir = tmp_path / "images" plan.analysis_results_dir = str(plan.output_dir) - value = ImageMetadataPayload(np.ones((4, 5), dtype=np.float32), ImagePayloadMetadata( - source_component_metadata={"well": "A01", "site": "1", "channel": "2", "z_index": "1", "timepoint": "1", "extension": ".tif"}, - )) + value = ImageMetadataPayload( + np.ones((4, 5), dtype=np.float32), + ImagePayloadMetadata( + source_component_metadata={ + "well": "A01", + "site": "1", + "channel": "2", + "z_index": "1", + "timepoint": "1", + "extension": ".tif", + }, + ), + ) paths = [] store = RuntimeValueStore() for name in ("a/b", "a b"): - output_plan = ArtifactOutputPlan(name=name, path=f"/memory/{name}.pkl", artifact_type=ImageArtifactType, - materialization=TerminalMaterializationSpec(ImageFileOptions(filename_suffix=".tif"))) - record = store.record(RuntimeValue.normalize(output_plan, value, axis_id="A01"), path=output_plan.path, backend=Backend.MEMORY.value) - outputs = RuntimeArtifactMaterialization.from_record(output_plan=output_plan, record=record, plan=plan, context=context).outputs(plan, context) + output_plan = ArtifactOutputPlan( + name=name, + path=f"/memory/{name}.pkl", + artifact_type=ImageArtifactType, + materialization=TerminalMaterializationSpec( + ImageFileOptions(filename_suffix=".tif") + ), + ) + record = store.record( + RuntimeValue.normalize(output_plan, value, axis_id="A01"), + path=output_plan.path, + backend=Backend.MEMORY.value, + ) + outputs = RuntimeArtifactMaterialization.from_record( + output_plan=output_plan, record=record, plan=plan, context=context + ).outputs(plan, context) paths.append(outputs[0].path) assert paths[0] == paths[1] # Existing main-flow destination authority rejects this declared naming # collision instead of choosing a new suffix or inventing a channel. from openhcs.core.steps.function_output_identity import FunctionOutputIdentity + with pytest.raises(ValueError, match="duplicate"): - FunctionOutputIdentity.validate_output_paths(paths, input_paths=[], step_name=plan.step_name, pattern_repr="declared roles") + FunctionOutputIdentity.validate_output_paths( + paths, + input_paths=[], + step_name=plan.step_name, + pattern_repr="declared roles", + ) -def test_batch_binds_actual_purpose_once_without_rewriting_plan_or_source(monkeypatch, tmp_path): - from openhcs.processing.materialization import ImageFileOptions, MaterializationSpec, TerminalMaterializationSpec - from openhcs.processing.materialization.core import MaterializationBatch, MaterializationContext, Output, WriterSpec, _WRITERS_BY_OPTIONS +def test_batch_binds_actual_purpose_once_without_rewriting_plan_or_source( + monkeypatch, tmp_path +): + from openhcs.processing.materialization import ( + ImageFileOptions, + MaterializationSpec, + TerminalMaterializationSpec, + ) + from openhcs.processing.materialization.core import ( + MaterializationBatch, + MaterializationContext, + Output, + WriterSpec, + _WRITERS_BY_OPTIONS, + ) options = ImageFileOptions(filename_suffix=".tif") declared = TerminalMaterializationSpec(options) explicit = MaterializationSpec(options) - plan = ArtifactOutputPlan(name="Role", path="/memory/role", artifact_type=ImageArtifactType, materialization=declared) + plan = ArtifactOutputPlan( + name="Role", + path="/memory/role", + artifact_type=ImageArtifactType, + materialization=declared, + ) manager = FileManager({Backend.MEMORY.value: MemoryStorageBackend()}) source = {"physical_source": "original"} context = MaterializationContext( - base_path=str(tmp_path / "role"), backends=[], backend_kwargs={}, - filemanager=manager, extra_inputs=source, output_plan=plan, + base_path=str(tmp_path / "role"), + backends=[], + backend_kwargs={}, + filemanager=manager, + extra_inputs=source, + output_plan=plan, materialization_spec=declared, ) seen = [] + def custom_writer(data, actual_options, actual_context): assert actual_options is options assert actual_context.output_plan is plan assert actual_context.extra_inputs is source seen.append(actual_context) return [Output(path=str(tmp_path / "custom.tif"), content=data)] - monkeypatch.setitem(_WRITERS_BY_OPTIONS, ImageFileOptions, WriterSpec( - options_type=ImageFileOptions, - write=custom_writer, primary_path=lambda outputs: outputs[0].path, - candidate_paths=lambda _options, base: (base,), - )) + + monkeypatch.setitem( + _WRITERS_BY_OPTIONS, + ImageFileOptions, + WriterSpec( + options_type=ImageFileOptions, + write=custom_writer, + primary_path=lambda outputs: outputs[0].path, + candidate_paths=lambda _options, base: (base,), + ), + ) pixels = np.ones((4, 5), dtype=np.float32) first = MaterializationBatch.render(explicit, pixels, context) second = MaterializationBatch.render(declared, pixels, first.context) @@ -335,20 +575,48 @@ def custom_writer(data, actual_options, actual_context): assert source == {"physical_source": "original"} -def test_direct_image_writer_without_purpose_binding_retains_legacy_source_path(tmp_path): - from openhcs.processing.materialization import ImageFileOptions, TerminalMaterializationSpec - from openhcs.processing.materialization.core import MaterializationContext, write_image_file +def test_direct_image_writer_without_purpose_binding_retains_legacy_source_path( + tmp_path, +): + from openhcs.processing.materialization import ( + ImageFileOptions, + TerminalMaterializationSpec, + ) + from openhcs.processing.materialization.core import ( + MaterializationContext, + write_image_file, + ) options = ImageFileOptions(filename_suffix=".tif") - plan = ArtifactOutputPlan(name="Role", path="/memory/role", artifact_type=ImageArtifactType, - materialization=TerminalMaterializationSpec(options)) + plan = ArtifactOutputPlan( + name="Role", + path="/memory/role", + artifact_type=ImageArtifactType, + materialization=TerminalMaterializationSpec(options), + ) manager = FileManager({Backend.MEMORY.value: MemoryStorageBackend()}) - value = ImageMetadataPayload(np.ones((4, 5), dtype=np.float32), ImagePayloadMetadata( - source_component_metadata={"well": "A01", "site": "1", "channel": "2", "z_index": "1", "timepoint": "1", "extension": ".tif"}, - )) - context = MaterializationContext(base_path=str(tmp_path / "role"), backends=[], backend_kwargs={}, - filemanager=manager, extra_inputs={}, output_plan=plan, - context=context_stub(manager, parser=SourceSchemaFilenameParser())) + value = ImageMetadataPayload( + np.ones((4, 5), dtype=np.float32), + ImagePayloadMetadata( + source_component_metadata={ + "well": "A01", + "site": "1", + "channel": "2", + "z_index": "1", + "timepoint": "1", + "extension": ".tif", + }, + ), + ) + context = MaterializationContext( + base_path=str(tmp_path / "role"), + backends=[], + backend_kwargs={}, + filemanager=manager, + extra_inputs={}, + output_plan=plan, + context=context_stub(manager, parser=SourceSchemaFilenameParser()), + ) outputs = write_image_file(value, options, context) assert Path(outputs[0].path).name == "A01_s001_w2_z001_t001.tif" assert context.materialization_spec is None @@ -357,23 +625,50 @@ def test_direct_image_writer_without_purpose_binding_retains_legacy_source_path( @pytest.mark.parametrize("purpose", ("named_terminal", "explicit", "unnamed_terminal")) -def test_parser_free_render_requires_parser_only_for_automatic_named_role(tmp_path, purpose): - from openhcs.processing.materialization import ImageFileOptions, MaterializationSpec, TerminalMaterializationSpec, materialization_outputs +def test_parser_free_render_requires_parser_only_for_automatic_named_role( + tmp_path, purpose +): + from openhcs.processing.materialization import ( + ImageFileOptions, + MaterializationSpec, + TerminalMaterializationSpec, + materialization_outputs, + ) options = ImageFileOptions(filename_suffix=".tif") - spec = MaterializationSpec(options) if purpose == "explicit" else TerminalMaterializationSpec(options) - plan = None if purpose == "unnamed_terminal" else ArtifactOutputPlan( - name="Role", path="/memory/role", artifact_type=ImageArtifactType, materialization=spec, + spec = ( + MaterializationSpec(options) + if purpose == "explicit" + else TerminalMaterializationSpec(options) + ) + plan = ( + None + if purpose == "unnamed_terminal" + else ArtifactOutputPlan( + name="Role", + path="/memory/role", + artifact_type=ImageArtifactType, + materialization=spec, + ) ) manager = FileManager({Backend.MEMORY.value: MemoryStorageBackend()}) - value = ImageMetadataPayload(np.ones((4, 5), dtype=np.float32), ImagePayloadMetadata( - source_path="/physical/OriginalSource.tif", - )) + value = ImageMetadataPayload( + np.ones((4, 5), dtype=np.float32), + ImagePayloadMetadata( + source_path="/physical/OriginalSource.tif", + ), + ) if purpose == "named_terminal": - with pytest.raises(ValueError, match="Parser-backed source-stem authority requires a parser"): - materialization_outputs(spec, value, str(tmp_path / "Role"), manager, output_plan=plan) + with pytest.raises( + ValueError, match="Parser-backed source-stem authority requires a parser" + ): + materialization_outputs( + spec, value, str(tmp_path / "Role"), manager, output_plan=plan + ) else: - outputs = materialization_outputs(spec, value, str(tmp_path / "Role"), manager, output_plan=plan) + outputs = materialization_outputs( + spec, value, str(tmp_path / "Role"), manager, output_plan=plan + ) assert Path(outputs[0].path).name == "OriginalSource.tif" np.testing.assert_array_equal(outputs[0].content, value.data) assert value.metadata.source_path == "/physical/OriginalSource.tif" diff --git a/tests/unit/test_axis_family_guards.py b/tests/unit/test_axis_family_guards.py index 0ca8f4efd1..de6f22ce44 100644 --- a/tests/unit/test_axis_family_guards.py +++ b/tests/unit/test_axis_family_guards.py @@ -52,7 +52,9 @@ def _python_files(*roots: str) -> list[Path]: for root in roots: base = REPO_ROOT / root files.extend( - path for path in base.rglob("*.py") if "benchmark/results" not in path.as_posix() + path + for path in base.rglob("*.py") + if "benchmark/results" not in path.as_posix() ) return files @@ -77,7 +79,9 @@ def _identifiers(tree: ast.AST) -> set[str]: def test_deleted_component_enums_and_factory_do_not_exist() -> None: offenders = { - _relative(path): sorted(DELETED_NAMES & _identifiers(ast.parse(path.read_text()))) + _relative(path): sorted( + DELETED_NAMES & _identifiers(ast.parse(path.read_text())) + ) for path in _python_files("openhcs", "tests", "benchmark", "scripts") if path.name != Path(__file__).name } @@ -90,7 +94,10 @@ def test_kernel_modules_name_no_microscopy_member() -> None: offenders: dict[str, list[str]] = {} for path in _python_files("openhcs"): relative = _relative(path) - if relative.startswith(DOMAIN_MODULE_PREFIXES) or relative == DOMAIN_ENTRY_POINT: + if ( + relative.startswith(DOMAIN_MODULE_PREFIXES) + or relative == DOMAIN_ENTRY_POINT + ): continue tree = ast.parse(path.read_text()) found: set[str] = set() @@ -146,13 +153,17 @@ def _domain_member_spellings() -> frozenset[str]: ) -def _member_spellings_in(tree: ast.AST, spellings: frozenset[str]) -> set[tuple[str, str]]: +def _member_spellings_in( + tree: ast.AST, spellings: frozenset[str] +) -> set[tuple[str, str]]: """(enclosing class qualname, spelling) for literals, attributes and class fields.""" docstrings = { id(node.body[0].value) for node in ast.walk(tree) - if isinstance(node, (ast.Module, ast.ClassDef, ast.FunctionDef, ast.AsyncFunctionDef)) + if isinstance( + node, (ast.Module, ast.ClassDef, ast.FunctionDef, ast.AsyncFunctionDef) + ) and node.body and isinstance(node.body[0], ast.Expr) and isinstance(node.body[0].value, ast.Constant) @@ -168,9 +179,11 @@ def visit(node: ast.AST, scope: str) -> None: targets = ( [statement.target] if isinstance(statement, ast.AnnAssign) - else statement.targets - if isinstance(statement, ast.Assign) - else [] + else ( + statement.targets + if isinstance(statement, ast.Assign) + else [] + ) ) found.update( (child_scope, f"field {target.id}") @@ -204,7 +217,9 @@ def test_kernel_modules_spell_no_microscopy_member() -> None: continue found.update( (relative, scope, spelling) - for scope, spelling in _member_spellings_in(ast.parse(path.read_text()), spellings) + for scope, spelling in _member_spellings_in( + ast.parse(path.read_text()), spellings + ) ) assert found == EXTERNAL_SPELLINGS @@ -327,7 +342,10 @@ def test_runtime_foreign_names_name_existing_modules() -> None: ("openhcs/core/config.py", "PlateOutputDirectory"), ("openhcs/core/config.py", "PlateOutputFile"), ("openhcs/core/dataset_sources/openhcs_format.py", "PLATE_NAME"), - ("openhcs/core/dataset_sources/openhcs_format.py", "get_plate_acquisition_datetime"), + ( + "openhcs/core/dataset_sources/openhcs_format.py", + "get_plate_acquisition_datetime", + ), ("openhcs/core/dataset_sources/openhcs_format.py", "get_plate_name"), ("openhcs/core/dataset_sources/source.py", "plate_root"), ("openhcs/core/debug_session_projection.py", "NO_PLATE"), @@ -374,11 +392,23 @@ def test_runtime_foreign_names_name_existing_modules() -> None: ("openhcs/core/source_tile_geometry.py", "sites"), ("openhcs/core/steps/stream_component_semantics.py", "plate_path"), ("openhcs/core/streaming_config_factory.py", "plate_path"), - ("openhcs/core/synthetic_plate_generation.py", "SYNTHETIC_PLATE_GENERATION_PROFILE"), + ( + "openhcs/core/synthetic_plate_generation.py", + "SYNTHETIC_PLATE_GENERATION_PROFILE", + ), ("openhcs/core/synthetic_plate_generation.py", "SyntheticPlateFormat"), - ("openhcs/core/synthetic_plate_generation.py", "SyntheticPlateGenerationBounds"), - ("openhcs/core/synthetic_plate_generation.py", "SyntheticPlateGenerationParameters"), - ("openhcs/core/synthetic_plate_generation.py", "SyntheticPlateGenerationProfile"), + ( + "openhcs/core/synthetic_plate_generation.py", + "SyntheticPlateGenerationBounds", + ), + ( + "openhcs/core/synthetic_plate_generation.py", + "SyntheticPlateGenerationParameters", + ), + ( + "openhcs/core/synthetic_plate_generation.py", + "SyntheticPlateGenerationProfile", + ), ("openhcs/core/vfs_protocol.py", "PlateInputDirectory"), ("openhcs/core/vfs_protocol.py", "PlateInputDirectoryDeclaration"), ("openhcs/core/vfs_protocol.py", "PlateInputFile"), @@ -428,7 +458,9 @@ def test_kernel_domain_references_only_shrink() -> None: for name in _names_and_identifier_literals(ast.parse(path.read_text())) if _identifier_words(name) & domain_words } - assert sorted(found - KERNEL_DOMAIN_REFERENCES) == [], "new domain word in the kernel" + assert ( + sorted(found - KERNEL_DOMAIN_REFERENCES) == [] + ), "new domain word in the kernel" assert sorted(KERNEL_DOMAIN_REFERENCES - found) == [], "removed: shrink the pin" assert len(KERNEL_DOMAIN_REFERENCES) <= KERNEL_DOMAIN_REFERENCE_CEILING @@ -467,4 +499,3 @@ def test_renamed_dataset_names_do_not_exist() -> None: if path.name != Path(__file__).name } assert {path: names for path, names in offenders.items() if names} == {} - diff --git a/tests/unit/test_axis_family_witness.py b/tests/unit/test_axis_family_witness.py index ad13de24d3..f09956c9d7 100644 --- a/tests/unit/test_axis_family_witness.py +++ b/tests/unit/test_axis_family_witness.py @@ -66,7 +66,10 @@ def test_child_processes_serve_the_parent_family(remote_sensing) -> None: capture_output=True, text=True, timeout=300, - env={**os.environ, "PYTHONPATH": os.pathsep.join(map(str, (import_root, repo_root)))}, + env={ + **os.environ, + "PYTHONPATH": os.pathsep.join(map(str, (import_root, repo_root))), + }, check=True, ) assert child.stdout.strip().splitlines()[-1] == RemoteSensing.family_name @@ -147,9 +150,9 @@ def test_viewer_decodes_a_stream_by_its_own_declared_axes(remote_sensing) -> Non assert fiji.components_in(FijiSlots.HyperstackChannel) == ("band",) assert fiji.components_in(FijiSlots.HyperstackSlice) == () assert fiji.components_in(FijiSlots.HyperstackFrame) == ("tile", "date", "scene") - assert FijiDisplaySettings.from_display_payload(fiji_payload) == FijiDisplaySettings( - lut="Fire" - ) + assert FijiDisplaySettings.from_display_payload( + fiji_payload + ) == FijiDisplaySettings(lut="Fire") napari = ViewerMappingDisplayConfigInput(napari_payload).layout() assert napari.components_in(NapariSlots.Stack) == ("tile", "band", "date", "scene") @@ -239,8 +242,12 @@ def test_step_validation_and_compiler_axis_resolution(remote_sensing) -> None: ) validator = GenericValidator(AxisFamily.active()) - assert validator.validate_step([RemoteSensing.Tile], RemoteSensing.Band, True, "s").is_valid - rejected = validator.validate_step([RemoteSensing.Band], RemoteSensing.Band, True, "s") + assert validator.validate_step( + [RemoteSensing.Tile], RemoteSensing.Band, True, "s" + ).is_valid + rejected = validator.validate_step( + [RemoteSensing.Band], RemoteSensing.Band, True, "s" + ) assert not rejected.is_valid partition = validator.validate_step([RemoteSensing.Scene], Ungrouped, False, "s") assert "multiprocessing axis: scene" in partition.error_message @@ -334,7 +341,10 @@ def test_pipeline_runs_end_to_end_through_the_kernel( timeout=900, # The kernel this test imports, then the checkout for the witness domain; # processes the kernel spawns inherit both. - env={**os.environ, "PYTHONPATH": os.pathsep.join(map(str, (import_root, repo_root)))}, + env={ + **os.environ, + "PYTHONPATH": os.pathsep.join(map(str, (import_root, repo_root))), + }, check=False, ) assert completed.returncode == 0, completed.stderr[-4000:] diff --git a/tests/unit/test_benchmark_control.py b/tests/unit/test_benchmark_control.py index 5a2587dba2..38d8973460 100644 --- a/tests/unit/test_benchmark_control.py +++ b/tests/unit/test_benchmark_control.py @@ -63,7 +63,10 @@ InstalledDistributionVersion, RuntimeEnvironmentSnapshot, ) -from openhcs.runtime.zmq_execution_client import OpenHCSExecutionSubmission, ZMQExecutionRequestBuilder +from openhcs.runtime.zmq_execution_client import ( + OpenHCSExecutionSubmission, + ZMQExecutionRequestBuilder, +) from openhcs.runtime.zmq_execution_signature import ZMQAuxiliaryExecutionParams diff --git a/tests/unit/test_bioformats_microscope_handler.py b/tests/unit/test_bioformats_microscope_handler.py index fdc40481c8..1d5d6b2859 100644 --- a/tests/unit/test_bioformats_microscope_handler.py +++ b/tests/unit/test_bioformats_microscope_handler.py @@ -217,7 +217,9 @@ def test_openhcs_replay_registers_declared_source_handler_backends( reopened_filemanager = bioformats_filemanager() assert Backend.BIOFORMATS.value not in reopened_filemanager.registry - handler = DatasetSourceChoice.named("auto").open(tmp_path, filemanager=reopened_filemanager) + handler = DatasetSourceChoice.named("auto").open( + tmp_path, filemanager=reopened_filemanager + ) assert handler.dataset_root == tmp_path handler.initialize_workspace(tmp_path, reopened_filemanager) @@ -304,7 +306,9 @@ def test_openhcs_output_subdirectory_initializes_from_metadata_root( def test_create_microscope_handler_supports_explicit_bioformats(tmp_path: Path) -> None: - handler = DatasetSourceChoice.named("bioformats").open(tmp_path, filemanager=bioformats_filemanager()) + handler = DatasetSourceChoice.named("bioformats").open( + tmp_path, filemanager=bioformats_filemanager() + ) assert isinstance(handler, BioFormatsHandler) diff --git a/tests/unit/test_calibrated_gaussian_spatial_axes.py b/tests/unit/test_calibrated_gaussian_spatial_axes.py index ba152e6163..09c4bb99b8 100644 --- a/tests/unit/test_calibrated_gaussian_spatial_axes.py +++ b/tests/unit/test_calibrated_gaussian_spatial_axes.py @@ -13,9 +13,14 @@ ) from openhcs.core.runtime_plane_projection import RuntimePlaneAxis from openhcs.core.source_metadata import SourceVoxelSpacing -from openhcs.core.source_spatial_domain import SourceSpatialDomain, VolumeSourceSpatialDomain +from openhcs.core.source_spatial_domain import ( + SourceSpatialDomain, + VolumeSourceSpatialDomain, +) from openhcs.interop.cellprofiler.runtime.adapter import CellProfilerRuntimeAdapter -from openhcs.interop.cellprofiler.runtime.function_contract_execution import CellProfilerFunctionContractExecutor +from openhcs.interop.cellprofiler.runtime.function_contract_execution import ( + CellProfilerFunctionContractExecutor, +) from openhcs.processing.backends.cellprofiler.gaussian_filter import gaussian_filter from openhcs.core.payload_axes import PayloadAxes from openhcs.core.axes import ColourAxis @@ -26,16 +31,25 @@ def _execute(source): contract = CallableContract.from_callable(gaussian_filter).with_prepared_signature() - contract = replace(contract, metadata=replace( - contract.metadata, runtime_adapter=CellProfilerRuntimeAdapter.runtime_adapter_spec(), - )) + contract = replace( + contract, + metadata=replace( + contract.metadata, + runtime_adapter=CellProfilerRuntimeAdapter.runtime_adapter_spec(), + ), + ) return CellProfilerFunctionContractExecutor().execute( - contract, contract.resolve_canonical_raw_callable(), source, {"sigma": 1.5}, + contract, + contract.resolve_canonical_raw_callable(), + source, + {"sigma": 1.5}, execution_mode=FullStackExecution, ) -def _source(pixels, *, domain=None, spacing=(0.5, 0.75), channel_axis=None, cohort=False): +def _source( + pixels, *, domain=None, spacing=(0.5, 0.75), channel_axis=None, cohort=False +): metadata = ImagePayloadMetadata( source_path="/synthetic/original.tif", source_image_names=("Original",), @@ -68,12 +82,18 @@ def test_anisotropic_physical_z_matches_library_without_site_blur(cohort): volume[2, 4, 5] = 1 pixels = np.stack((volume, np.zeros_like(volume))) if cohort else volume source = _source( - pixels, domain=VolumeSourceSpatialDomain(source_depth=5), - spacing=(2, 0.5, 0.75), cohort=cohort, + pixels, + domain=VolumeSourceSpatialDomain(source_depth=5), + spacing=(2, 0.5, 0.75), + cohort=cohort, ) result = _execute(source) expected_volume = gaussian(volume, sigma=(0.75, 3, 2)) - expected = np.stack((expected_volume, np.zeros_like(volume))) if cohort else expected_volume + expected = ( + np.stack((expected_volume, np.zeros_like(volume))) + if cohort + else expected_volume + ) np.testing.assert_allclose(result.data, expected) assert result.metadata == source.metadata assert np.any(result.data[0, 1] if cohort else result.data[1]) @@ -85,9 +105,9 @@ def test_declared_channel_axis_is_not_smoothed(channel_axis): channels[0, 4, 5] = 1 pixels = np.moveaxis(channels, 0, channel_axis) result = _execute(_source(pixels, channel_axis=channel_axis)) - expected = np.moveaxis(np.stack([ - gaussian(plane, sigma=(3, 2)) for plane in channels - ]), 0, channel_axis) + expected = np.moveaxis( + np.stack([gaussian(plane, sigma=(3, 2)) for plane in channels]), 0, channel_axis + ) np.testing.assert_allclose(result.data, expected) @@ -95,8 +115,11 @@ def test_physical_volume_with_channels_and_independent_sites(): pixels = np.zeros((2, 5, 9, 11, 2), dtype=np.float32) pixels[0, 2, 4, 5, 0] = 1 source = _source( - pixels, domain=VolumeSourceSpatialDomain(source_depth=5), - spacing=(2, 0.5, 0.75), channel_axis=-1, cohort=True, + pixels, + domain=VolumeSourceSpatialDomain(source_depth=5), + spacing=(2, 0.5, 0.75), + channel_axis=-1, + cohort=True, ) result = _execute(source) expected = np.zeros_like(pixels) @@ -106,7 +129,9 @@ def test_physical_volume_with_channels_and_independent_sites(): def test_missing_physical_z_spacing_still_rejects(): - source = _source(np.zeros((3, 9, 11)), domain=VolumeSourceSpatialDomain(source_depth=3)) + source = _source( + np.zeros((3, 9, 11)), domain=VolumeSourceSpatialDomain(source_depth=3) + ) with pytest.raises(ValueError, match="Cannot project"): _execute(source) @@ -115,7 +140,9 @@ def test_invalid_channel_and_spatial_rank_still_reject(): with pytest.raises(ValueError, match="invalid for payload rank"): _execute(_source(np.zeros((9, 11)), channel_axis=4)) with pytest.raises(ValueError, match="spatial rank"): - _execute(_source(np.zeros((9, 11)), domain=VolumeSourceSpatialDomain(source_depth=3))) + _execute( + _source(np.zeros((9, 11)), domain=VolumeSourceSpatialDomain(source_depth=3)) + ) @pytest.mark.parametrize( @@ -128,7 +155,10 @@ def test_invalid_channel_and_spatial_rank_still_reject(): ), ) def test_optional_yx_and_strict_intrinsic_share_undeclared_axes( - shape, channel_axis, undeclared_axes, yx, + shape, + channel_axis, + undeclared_axes, + yx, ): metadata = ImagePayloadMetadata(axes=PayloadAxes.colour_samples(channel_axis)) pixels = np.zeros(shape) @@ -151,11 +181,17 @@ def test_yx_does_not_inherit_strict_volume_rank_requirement(): def test_axis_capability_is_inherited_without_metadata_overrides(): for name in ( - "undeclared_axes", "axis_position", "axis_index", "spatial_axes_yx", - "declares_colour_samples_plane", "declares_colour_samples_stack", + "undeclared_axes", + "axis_position", + "axis_index", + "spatial_axes_yx", + "declares_colour_samples_plane", + "declares_colour_samples_stack", ): assert name not in ImagePayloadMetadata.__dict__ - assert getattr(ImagePayloadMetadata, name) is getattr(ImagePayloadAxisFields, name) + assert getattr(ImagePayloadMetadata, name) is getattr( + ImagePayloadAxisFields, name + ) metadata = ImagePayloadMetadata() assert metadata.axis_position(ColourAxis) is None assert metadata.plane_axis is None @@ -182,7 +218,8 @@ def test_original_cohort_composition_keeps_every_source_plane(physical_volume): source_path=f"/synthetic/plane{index}.tif", source_image_names=("Original",), source_component_metadata={ - "well": "A01", "site": 1 if physical_volume else index + 1, + "well": "A01", + "site": 1 if physical_volume else index + 1, "z_index": index if physical_volume else 0, }, source_voxel_spacing=SourceVoxelSpacing( @@ -197,13 +234,15 @@ def test_original_cohort_composition_keeps_every_source_plane(physical_volume): if physical_volume: metadata = metadata.replace_fields( source_spatial_domain=VolumeSourceSpatialDomain().with_source_cohort( - metadata.source_spatial_domain, depth=len(planes), + metadata.source_spatial_domain, + depth=len(planes), ) ) pixels = np.stack([plane.data for plane in planes]) result = _execute(metadata.payload_with(pixels, None)) expected = ( - gaussian(pixels, sigma=(0.75, 3, 2)) if physical_volume + gaussian(pixels, sigma=(0.75, 3, 2)) + if physical_volume else np.stack([gaussian(plane, sigma=(3, 2)) for plane in pixels]) ) np.testing.assert_allclose(result.data, expected) diff --git a/tests/unit/test_callable_abi_preparation.py b/tests/unit/test_callable_abi_preparation.py index 80b4504618..6fc524966d 100644 --- a/tests/unit/test_callable_abi_preparation.py +++ b/tests/unit/test_callable_abi_preparation.py @@ -15,20 +15,29 @@ import openhcs.core.callable_contract as contract_module from openhcs.core.callable_contract import ( - CallableContract, CallableImportIdentity, CallableMetadata, CallableProjection, - attach_callable_contract_metadata, attach_processing_prepare, + CallableContract, + CallableImportIdentity, + CallableMetadata, + CallableProjection, + attach_callable_contract_metadata, + attach_processing_prepare, ) from openhcs.core.function_contract_metadata import FunctionContractAttribute from openhcs.core.function_patterns import NormalizedFunctionGroup from openhcs.core.function_reference import ImportableFunctionReference -from openhcs.core.processing_preparation import CallablePreparation, PreparationCacheBatch +from openhcs.core.processing_preparation import ( + CallablePreparation, + PreparationCacheBatch, +) from openhcs.core.runtime_image_values import ImagePayloadMetadata from openhcs.core.runtime_plane_projection import RuntimePlaneAxis from openhcs.core.pipeline.function_contracts import composed_image_payload from openhcs.processing.backends.lib_registry.registry_service import RegistryService from openhcs.core.runtime_batch_contracts import ( - Pure2DSliceBatchExecutor, RuntimeBatchExecutionDomain, - SerialPure2DSliceBatchExecutor, runtime_callable_defaults, + Pure2DSliceBatchExecutor, + RuntimeBatchExecutionDomain, + SerialPure2DSliceBatchExecutor, + runtime_callable_defaults, ) from openhcs.core.pipeline.function_contracts import resolved_callable_parameter from openhcs.core.processing_contracts import ( @@ -48,19 +57,26 @@ def unannotated_composed_echo(image): @pytest.mark.parametrize( ("composed", "annotation", "retains_payload"), - ((True, inspect.Parameter.empty, True), - (False, inspect.Parameter.empty, False), - (True, np.ndarray, False), - (True, Any, False), - (True, ArrayPayload, True)), + ( + (True, inspect.Parameter.empty, True), + (False, inspect.Parameter.empty, False), + (True, np.ndarray, False), + (True, Any, False), + (True, ArrayPayload, True), + ), ) def test_composition_default_respects_explicit_raw_carrier_boundary( - composed, annotation, retains_payload, fake_preparation, + composed, + annotation, + retains_payload, + fake_preparation, ): received = [] + def raw(image): received.append(image) return image + if annotation is not inspect.Parameter.empty: raw.__annotations__["image"] = annotation if composed: @@ -75,7 +91,9 @@ def raw(image): ).payload_with(pixels, mask) expected = source if retains_payload else pixels - result = RuntimeCallablePolicy().contract_invocation(contract, raw, source, {}).call() + result = ( + RuntimeCallablePolicy().contract_invocation(contract, raw, source, {}).call() + ) assert result is expected assert received[0] is expected @@ -85,16 +103,23 @@ def raw(image): @pytest.mark.parametrize("serializer", (pickle, cloudpickle)) def test_prepared_composition_default_survives_transport_without_introspection( - serializer, fake_preparation, monkeypatch, + serializer, + fake_preparation, + monkeypatch, ): contract = CallableContract.from_prepared_callable(unannotated_composed_echo) reference = ImportableFunctionReference( import_identity=CallableImportIdentity.from_callable(unannotated_composed_echo), - composite_key="test:unannotated_composed_echo", metadata=contract.metadata, + composite_key="test:unannotated_composed_echo", + metadata=contract.metadata, ) - restored = CallableContract.from_callable(serializer.loads(serializer.dumps(reference))) + restored = CallableContract.from_callable( + serializer.loads(serializer.dumps(reference)) + ) + def forbidden(*args, **kwargs): pytest.fail("Prepared composed ABI must not inspect live callable declarations") + monkeypatch.setattr(contract_module, "get_type_hints", forbidden) monkeypatch.setattr(inspect, "signature", forbidden) payload = ImagePayloadMetadata( @@ -109,7 +134,9 @@ def forbidden(*args, **kwargs): def fake_preparation(monkeypatch): prepare = Mock() monkeypatch.setattr(CallablePreparation, "prepare", prepare) - monkeypatch.setattr(PreparationCacheBatch, "populate_child_caches", lambda *a, **k: None) + monkeypatch.setattr( + PreparationCacheBatch, "populate_child_caches", lambda *a, **k: None + ) return prepare @@ -123,9 +150,14 @@ def last(image: np.ndarray): return image first.__dict__[FunctionContractAttribute.raw_processing_function] = last - metadata = {"first": SimpleNamespace(func=first), "alias": SimpleNamespace(func=first)} + metadata = { + "first": SimpleNamespace(func=first), + "alias": SimpleNamespace(func=first), + } monkeypatch.setattr(RegistryService, "_metadata_cache", metadata) - monkeypatch.setattr(PreparationCacheBatch, "populate_child_caches", lambda *a, **k: None) + monkeypatch.setattr( + PreparationCacheBatch, "populate_child_caches", lambda *a, **k: None + ) def prepare(owner): events.append(("prepare", owner.projection.func)) @@ -140,17 +172,40 @@ def hints(func, **kwargs): monkeypatch.setattr(CallablePreparation, "prepare", prepare) monkeypatch.setattr(contract_module, "get_type_hints", hints) - assert RegistryService.prepare_in_current_process(status_callback=events.append) is metadata - preparations = [i for i, item in enumerate(events) if isinstance(item, tuple) and item[0] == "prepare"] - signatures = [i for i, item in enumerate(events) if isinstance(item, tuple) and item[0] == "signature"] + assert ( + RegistryService.prepare_in_current_process(status_callback=events.append) + is metadata + ) + preparations = [ + i + for i, item in enumerate(events) + if isinstance(item, tuple) and item[0] == "prepare" + ] + signatures = [ + i + for i, item in enumerate(events) + if isinstance(item, tuple) and item[0] == "signature" + ] assert len(preparations) == 2 assert max(preparations) < min(signatures) - assert max(signatures) < next(i for i, item in enumerate(events) if isinstance(item, str) and "kernels ready" in item) - assert CallableContract.from_callable(first).canonical_parameter_annotations["image"] is ArrayPayload - assert CallableContract.from_callable(last).canonical_parameter_annotations["image"] is ArrayPayload + assert max(signatures) < next( + i + for i, item in enumerate(events) + if isinstance(item, str) and "kernels ready" in item + ) + assert ( + CallableContract.from_callable(first).canonical_parameter_annotations["image"] + is ArrayPayload + ) + assert ( + CallableContract.from_callable(last).canonical_parameter_annotations["image"] + is ArrayPayload + ) -def test_explicit_library_rewarm_refreshes_new_contract_not_existing_snapshot(monkeypatch, fake_preparation): +def test_explicit_library_rewarm_refreshes_new_contract_not_existing_snapshot( + monkeypatch, fake_preparation +): def raw(image: np.ndarray): return image @@ -159,14 +214,19 @@ def raw(image: np.ndarray): RegistryService.prepare_in_current_process() first = CallableContract.from_callable(raw) raw.__annotations__["image"] = ArrayPayload - assert CallableContract.from_callable(raw).canonical_parameter_annotations["image"] is np.ndarray + assert ( + CallableContract.from_callable(raw).canonical_parameter_annotations["image"] + is np.ndarray + ) RegistryService.prepare_in_current_process() second = CallableContract.from_callable(raw) assert first.canonical_parameter_annotations["image"] is np.ndarray assert second.canonical_parameter_annotations["image"] is ArrayPayload -def test_authored_compilation_prepares_before_signature_and_refreshes_each_compilation(monkeypatch): +def test_authored_compilation_prepares_before_signature_and_refreshes_each_compilation( + monkeypatch, +): def raw(image: np.ndarray, scale=2): return image @@ -186,6 +246,7 @@ def prepare(owner): def test_authored_preparation_precedes_metadata_validation(monkeypatch): def raw(image: np.ndarray): return image + events = [] original = CallableMetadata.from_projection.__func__ @@ -193,25 +254,33 @@ def validate(cls, projection): assert events == ["prepared"] return original(cls, projection) - monkeypatch.setattr(CallablePreparation, "prepare", lambda self: events.append("prepared")) + monkeypatch.setattr( + CallablePreparation, "prepare", lambda self: events.append("prepared") + ) monkeypatch.setattr(CallableMetadata, "from_projection", classmethod(validate)) contract = CallableContract.from_prepared_callable(raw) assert contract.canonical_parameter_annotations["image"] is np.ndarray -def test_unprepared_function_reference_compilation_reads_prepared_live_declaration(monkeypatch, fake_preparation): +def test_unprepared_function_reference_compilation_reads_prepared_live_declaration( + monkeypatch, fake_preparation +): reference = ImportableFunctionReference( import_identity=CallableImportIdentity.from_callable(raw_numeric), composite_key="test:raw_numeric", ) - contract = NormalizedFunctionGroup.from_pattern("group", reference).items[0].contract + contract = ( + NormalizedFunctionGroup.from_pattern("group", reference).items[0].contract + ) assert contract.func is reference assert contract.canonical_parameter_annotations["image"] is np.ndarray assert fake_preparation.call_count == 1 @pytest.mark.parametrize("serializer", [pickle, cloudpickle]) -def test_prepared_signature_metadata_namespace_and_reference_transport(serializer, fake_preparation): +def test_prepared_signature_metadata_namespace_and_reference_transport( + serializer, fake_preparation +): contract = CallableContract.from_prepared_callable(raw_numeric) metadata = contract.metadata @@ -219,10 +288,14 @@ def proxy(image): return image proxy.__dict__.update(metadata.as_namespace()) - assert CallableMetadata.from_callable(proxy).canonical_signature == metadata.canonical_signature + assert ( + CallableMetadata.from_callable(proxy).canonical_signature + == metadata.canonical_signature + ) reference = ImportableFunctionReference( import_identity=CallableImportIdentity.from_callable(raw_numeric), - composite_key="test:raw_numeric", metadata=metadata, + composite_key="test:raw_numeric", + metadata=metadata, ) restored = serializer.loads(serializer.dumps(reference)) assert restored.metadata.canonical_signature == metadata.canonical_signature @@ -232,49 +305,76 @@ def proxy(image): assert restored_contract.canonical_signature.return_annotation is np.ndarray -def test_compiled_raw_projection_never_queries_signature_or_annotations(monkeypatch, fake_preparation): +def test_compiled_raw_projection_never_queries_signature_or_annotations( + monkeypatch, fake_preparation +): contract = CallableContract.from_prepared_callable(raw_numeric) reference = ImportableFunctionReference( import_identity=CallableImportIdentity.from_callable(raw_numeric), - composite_key="test:raw_numeric", metadata=contract.metadata, + composite_key="test:raw_numeric", + metadata=contract.metadata, ) restored = CallableContract.from_callable(pickle.loads(pickle.dumps(reference))) + def forbidden(*args, **kwargs): raise AssertionError("Compiled runtime queried a live signature") + monkeypatch.setattr(contract_module, "get_type_hints", forbidden) monkeypatch.setattr(inspect, "signature", forbidden) - payload = ImagePayloadMetadata(source_image_names=("DNA",)).payload_with(np.ones((2, 3)), None) + payload = ImagePayloadMetadata(source_image_names=("DNA",)).payload_with( + np.ones((2, 3)), None + ) for _ in range(40): assert restored.raw_main_flow_call_argument(payload) is payload.data assert restored.primary_input_parameter_name == "image" assert restored.canonical_parameter_annotations["image"] is np.ndarray -def test_custom_signature_and_mutable_default_are_retained_until_recompilation(monkeypatch, fake_preparation): +def test_custom_signature_and_mutable_default_are_retained_until_recompilation( + monkeypatch, fake_preparation +): default = [] + def raw(image, values=None): return image - raw.__signature__ = inspect.Signature([ - inspect.Parameter("image", inspect.Parameter.POSITIONAL_ONLY, annotation=ArrayPayload), - inspect.Parameter("values", inspect.Parameter.KEYWORD_ONLY, default=default), - ]) + + raw.__signature__ = inspect.Signature( + [ + inspect.Parameter( + "image", inspect.Parameter.POSITIONAL_ONLY, annotation=ArrayPayload + ), + inspect.Parameter( + "values", inspect.Parameter.KEYWORD_ONLY, default=default + ), + ] + ) first = CallableContract.from_prepared_callable(raw) assert first.canonical_signature.parameters["values"].default is default - raw.__signature__ = raw.__signature__.replace(parameters=[ - inspect.Parameter("image", inspect.Parameter.POSITIONAL_ONLY, annotation=np.ndarray), - ]) + raw.__signature__ = raw.__signature__.replace( + parameters=[ + inspect.Parameter( + "image", inspect.Parameter.POSITIONAL_ONLY, annotation=np.ndarray + ), + ] + ) second = CallableContract.from_prepared_callable(raw) assert first.canonical_parameter_annotations["image"] is ArrayPayload assert second.canonical_parameter_annotations["image"] is np.ndarray -@pytest.mark.parametrize("mutation", [ - lambda func: attach_processing_prepare(func, lambda: None), - lambda func: attach_callable_contract_metadata(func, raw_processing_function=raw_numeric), -]) +@pytest.mark.parametrize( + "mutation", + [ + lambda func: attach_processing_prepare(func, lambda: None), + lambda func: attach_callable_contract_metadata( + func, raw_processing_function=raw_numeric + ), + ], +) def test_supported_declaration_changes_invalidate_published_signature(mutation): def raw(image: np.ndarray): return image + CallableProjection.from_callable(raw).warm_canonical_signature() assert FunctionContractAttribute.canonical_signature in vars(raw) mutation(raw) @@ -284,6 +384,7 @@ def raw(image: np.ndarray): def test_malformed_signature_declaration_is_rejected(): def raw(image): return image + raw.__dict__[FunctionContractAttribute.canonical_signature] = "not a signature" with pytest.raises(TypeError, match="inspect.Signature"): CallableMetadata.from_callable(raw) @@ -292,11 +393,17 @@ def raw(image): def test_exact_raw_default_filter_and_type_views_refresh_with_library_snapshot(): def raw(image: np.ndarray, scale: float = 2): return image * scale + CallableProjection.from_callable(raw).warm_canonical_signature() policy = RuntimeCallablePolicy(kwarg_policy=SignatureFilteredKwargs) assert runtime_callable_defaults(raw)["scale"] == 2 assert resolved_callable_parameter(raw, "scale").annotation is float - assert policy.invocation(raw, (np.ones((2, 3)),), {"scale":3, "unknown":9}).call()[0,0] == 3 + assert ( + policy.invocation(raw, (np.ones((2, 3)),), {"scale": 3, "unknown": 9}).call()[ + 0, 0 + ] + == 3 + ) raw.__defaults__ = (4,) raw.__annotations__["scale"] = int assert runtime_callable_defaults(raw)["scale"] == 2 @@ -309,48 +416,78 @@ def raw(image: np.ndarray, scale: float = 2): def test_different_wrapper_signature_remains_live_for_default_filter_and_parameter_queries(): def raw(image: np.ndarray, private: int = 8): return image + def wrapper(image: np.ndarray, public: int = 2): return image * public + wrapper.__dict__[FunctionContractAttribute.raw_processing_function] = raw CallableProjection.from_callable(wrapper).warm_canonical_signature() - assert runtime_callable_defaults(wrapper) == {"public":2} + assert runtime_callable_defaults(wrapper) == {"public": 2} assert resolved_callable_parameter(wrapper, "public").annotation is int policy = RuntimeCallablePolicy(kwarg_policy=SignatureFilteredKwargs) - assert policy.invocation(wrapper, (np.ones((2, 3)),), {"public":3, "private":99}).call()[0,0] == 3 + assert ( + policy.invocation( + wrapper, (np.ones((2, 3)),), {"public": 3, "private": 99} + ).call()[0, 0] + == 3 + ) wrapper.__defaults__ = (5,) - assert runtime_callable_defaults(wrapper) == {"public":5} - assert policy.invocation(wrapper, (np.ones((2, 3)),), {"private":99}).call()[0,0] == 5 + assert runtime_callable_defaults(wrapper) == {"public": 5} + assert ( + policy.invocation(wrapper, (np.ones((2, 3)),), {"private": 99}).call()[0, 0] + == 5 + ) def test_prepared_type_query_preserves_legacy_nested_annotation_stripping(): from typing import Annotated, get_type_hints + def raw(image, labels): return image - raw.__annotations__ = {"image":Annotated[np.ndarray,"image"],"labels":tuple[Annotated[int,"id"],...],"return":Annotated[np.ndarray,"result"]} + + raw.__annotations__ = { + "image": Annotated[np.ndarray, "image"], + "labels": tuple[Annotated[int, "id"], ...], + "return": Annotated[np.ndarray, "result"], + } expected = get_type_hints(raw) CallableProjection.from_callable(raw).warm_canonical_signature() - assert resolved_callable_parameter(raw,"image").annotation == expected["image"] - assert resolved_callable_parameter(raw,"labels").annotation == expected["labels"] - assert CallableContract.from_callable(raw).canonical_parameter_annotations["labels"] == raw.__annotations__["labels"] + assert resolved_callable_parameter(raw, "image").annotation == expected["image"] + assert resolved_callable_parameter(raw, "labels").annotation == expected["labels"] + assert ( + CallableContract.from_callable(raw).canonical_parameter_annotations["labels"] + == raw.__annotations__["labels"] + ) -def test_distinct_authored_runtime_signature_transport_and_new_compilation_refresh(fake_preparation): +def test_distinct_authored_runtime_signature_transport_and_new_compilation_refresh( + fake_preparation, +): def raw(image: np.ndarray, scale: int = 2) -> np.ndarray: return image * scale + @wraps(raw) def wrapper(*args, **kwargs): return raw(*args, **kwargs) - wrapper.__signature__ = inspect.signature(raw).replace(parameters=[ - *inspect.signature(raw).parameters.values(), - inspect.Parameter("injected",inspect.Parameter.KEYWORD_ONLY,default=False), - ]) + + wrapper.__signature__ = inspect.signature(raw).replace( + parameters=[ + *inspect.signature(raw).parameters.values(), + inspect.Parameter( + "injected", inspect.Parameter.KEYWORD_ONLY, default=False + ), + ] + ) first = CallableContract.from_prepared_callable(wrapper) assert "injected" in first.canonical_signature.parameters assert "injected" not in first.raw_runtime_signature.parameters assert first.metadata.raw_runtime_signature is not None restored = pickle.loads(pickle.dumps(first.metadata)) assert restored.raw_runtime_signature == first.raw_runtime_signature - assert restored.as_namespace()[FunctionContractAttribute.raw_runtime_signature] == first.raw_runtime_signature + assert ( + restored.as_namespace()[FunctionContractAttribute.raw_runtime_signature] + == first.raw_runtime_signature + ) assert FunctionContractAttribute.canonical_signature not in vars(raw) assert FunctionContractAttribute.canonical_signature not in vars(wrapper) raw.__defaults__ = (4,) @@ -361,23 +498,31 @@ def wrapper(*args, **kwargs): assert second.raw_runtime_signature.parameters["scale"].annotation is float -def test_distinct_targets_own_signatures_even_when_layouts_match_without_comparing_defaults(fake_preparation): +def test_distinct_targets_own_signatures_even_when_layouts_match_without_comparing_defaults( + fake_preparation, +): class RejectEquality: def __eq__(self, other): raise AssertionError("Signature reuse compared a mutable default") + default = RejectEquality() + def raw(image: np.ndarray, values=default) -> np.ndarray: return image + @wraps(raw) def wrapper(*args, **kwargs): return raw(*args, **kwargs) + contract = CallableContract.from_prepared_callable(wrapper) assert contract.metadata.raw_runtime_signature is not None assert contract.raw_runtime_signature is not contract.canonical_signature assert contract.raw_runtime_signature.parameters["values"].default is default raw.__defaults__ = ("new raw default",) second = CallableContract.from_prepared_callable(wrapper) - assert second.raw_runtime_signature.parameters["values"].default == "new raw default" + assert ( + second.raw_runtime_signature.parameters["values"].default == "new raw default" + ) assert contract.raw_runtime_signature.parameters["values"].default is default @@ -387,24 +532,34 @@ def test_same_actual_target_shares_canonical_signature(fake_preparation): assert contract.raw_runtime_signature is contract.canonical_signature -@pytest.mark.parametrize("executors", (None, {}, {RuntimeBatchExecutionDomain.PURE_2D_SLICES: None})) +@pytest.mark.parametrize( + "executors", (None, {}, {RuntimeBatchExecutionDomain.PURE_2D_SLICES: None}) +) def test_pure_2d_executor_selection_retains_serial_fallback(executors): - assert isinstance(Pure2DSliceBatchExecutor.from_executors(executors), SerialPure2DSliceBatchExecutor) + assert isinstance( + Pure2DSliceBatchExecutor.from_executors(executors), + SerialPure2DSliceBatchExecutor, + ) def test_pure_2d_executor_selection_retains_exact_declared_executor(): def declared(*args, **kwargs): raise AssertionError("Selecting an executor must not execute it") - selected = Pure2DSliceBatchExecutor.from_executors({ - RuntimeBatchExecutionDomain.PURE_2D_SLICES: declared, - }) + selected = Pure2DSliceBatchExecutor.from_executors( + { + RuntimeBatchExecutionDomain.PURE_2D_SLICES: declared, + } + ) assert selected is declared -def test_compiled_input_places_named_bundle_once_before_cpu_identity_shortcut(monkeypatch): +def test_compiled_input_places_named_bundle_once_before_cpu_identity_shortcut( + monkeypatch, +): from openhcs.core.aligned_image_payload import ( - AlignedImageSliceContext, ImageOutputBundle, + AlignedImageSliceContext, + ImageOutputBundle, ) from openhcs.core.function_patterns import compile_function_pattern from openhcs.processing.backends.processors.numpy_processor import gaussian_blur @@ -412,10 +567,14 @@ def test_compiled_input_places_named_bundle_once_before_cpu_identity_shortcut(mo pixels = tuple(np.full((3, 4), value, dtype=np.float32) for value in (0.25, 0.75)) masks = tuple(np.full((3, 4), value, dtype=bool) for value in (True, False)) source = ImageOutputBundle( - tuple(ImagePayloadMetadata().payload_with(data, mask) - for data, mask in zip(pixels, masks, strict=True)), - tuple(AlignedImageSliceContext.independent_main_flow(name) - for name in ("DNA", "Membrane")), + tuple( + ImagePayloadMetadata().payload_with(data, mask) + for data, mask in zip(pixels, masks, strict=True) + ), + tuple( + AlignedImageSliceContext.independent_main_flow(name) + for name in ("DNA", "Membrane") + ), ) calls = [] original = ImageOutputBundle.compose @@ -425,7 +584,9 @@ def observed(value, **destination): return original(value, **destination) monkeypatch.setattr(ImageOutputBundle, "compose", observed) - invocation = compile_function_pattern(gaussian_blur, {}, {}).default_group.invocations[0] + invocation = compile_function_pattern( + gaussian_blur, {}, {} + ).default_group.invocations[0] placed = invocation.convert_input(source, "numpy") invocation.main_flow_call_argument(placed) placed.data @@ -441,16 +602,20 @@ def observed(value, **destination): def test_compiled_input_reuses_produced_stack_realization_without_copy(): from openhcs.core.aligned_image_payload import ( - AlignedImageSliceContext, ProducedImageStack, + AlignedImageSliceContext, + ProducedImageStack, ) from openhcs.core.function_patterns import compile_function_pattern from openhcs.processing.backends.processors.numpy_processor import gaussian_blur source = ProducedImageStack( tuple(np.full((3, 4), value, dtype=np.float32) for value in (1, 2)), - memory_type="numpy", plane_axis=RuntimePlaneAxis.RUNTIME_SLICE, + memory_type="numpy", + plane_axis=RuntimePlaneAxis.RUNTIME_SLICE, ) - invocation = compile_function_pattern(gaussian_blur, {}, {}).default_group.invocations[0] + invocation = compile_function_pattern( + gaussian_blur, {}, {} + ).default_group.invocations[0] placed = invocation.convert_input(source, "numpy") assert invocation.convert_input(source, "numpy") is placed assert placed.data is source.data diff --git a/tests/unit/test_callable_contract.py b/tests/unit/test_callable_contract.py index e5017a2346..4f4d50e56b 100644 --- a/tests/unit/test_callable_contract.py +++ b/tests/unit/test_callable_contract.py @@ -147,16 +147,10 @@ def process(image): contract = CallableContract.from_callable(process) - assert ( - contract.payload_axis_requirement - == PayloadAxisRequirement(ColourAxis) - ) - assert ( - contract.metadata.as_namespace()[ - FunctionContractAttribute.payload_axis_requirement - ] - == PayloadAxisRequirement(ColourAxis) - ) + assert contract.payload_axis_requirement == PayloadAxisRequirement(ColourAxis) + assert contract.metadata.as_namespace()[ + FunctionContractAttribute.payload_axis_requirement + ] == PayloadAxisRequirement(ColourAxis) def test_callable_metadata_rejects_string_payload_axis_requirement() -> None: @@ -182,9 +176,7 @@ def test_callable_metadata_reader_returns_declared_requirement_kinds() -> None: FunctionContractAttribute.payload_axis_requirement: ( PayloadAxisRequirement(ColourAxis) ), - FunctionContractAttribute.payload_axis_transition: ( - PreservedPayloadAxes() - ), + FunctionContractAttribute.payload_axis_transition: (PreservedPayloadAxes()), }, "process", ) @@ -209,10 +201,7 @@ def crop_like(image): contract = CallableContract.from_callable(crop_like) - assert ( - contract.payload_axis_transition - == PreservedPayloadAxes() - ) + assert contract.payload_axis_transition == PreservedPayloadAxes() assert ( contract.metadata.as_namespace()[ FunctionContractAttribute.payload_axis_transition @@ -266,9 +255,15 @@ def project(image, method: ProjectionMethod = ProjectionMethod.MAX): with pytest.raises(TypeError, match="project.method must be ProjectionMethod"): contract.validate_public_kwargs({"method": "max"}) - assert contract.decode_public_kwargs({"method": "max"}) == {"method": ProjectionMethod.MAX} - assert contract.decode_public_kwargs({"method": "MAX"}) == {"method": ProjectionMethod.MAX} - assert contract.decode_public_kwargs({"method": ProjectionMethod.MAX}) == {"method": ProjectionMethod.MAX} + assert contract.decode_public_kwargs({"method": "max"}) == { + "method": ProjectionMethod.MAX + } + assert contract.decode_public_kwargs({"method": "MAX"}) == { + "method": ProjectionMethod.MAX + } + assert contract.decode_public_kwargs({"method": ProjectionMethod.MAX}) == { + "method": ProjectionMethod.MAX + } with pytest.raises(TypeError, match="project.method must be ProjectionMethod"): contract.decode_public_kwargs({"method": None}) with pytest.raises(ValueError, match="not a valid"): @@ -298,14 +293,18 @@ def process(image, *, slice_index: int = 0): contract.validate_public_kwargs({"slice_index": 3}) -def test_callable_contract_reads_wrapper_declared_config_parameters(monkeypatch) -> None: +def test_callable_contract_reads_wrapper_declared_config_parameters( + monkeypatch, +) -> None: @numpy(contract=Pure3DContract) def process(image): return image contract = CallableContract.from_callable(process) declared_parameter = contract.canonical_signature.parameters["dtype_config"] - assert isinstance(signature(process).parameters["dtype_config"].default, LazyDtypeConfig) + assert isinstance( + signature(process).parameters["dtype_config"].default, LazyDtypeConfig + ) def forbidden_config_value(*args, **kwargs): raise AssertionError("Config schema discovery constructed a value") @@ -333,19 +332,29 @@ def forbidden_signature_read(owner): assert graph.config_parameters_for_step("process") is parameters -def test_config_parameter_schema_refreshes_from_replaced_pipeline_declaration(monkeypatch): +def test_config_parameter_schema_refreshes_from_replaced_pipeline_declaration( + monkeypatch, +): def forbidden_default(): raise AssertionError("Config field discovery evaluated a default factory") original_default = object() parameter = Parameter( - "custom_config", Parameter.KEYWORD_ONLY, - annotation=LazyDtypeConfig, default=original_default, + "custom_config", + Parameter.KEYWORD_ONLY, + annotation=LazyDtypeConfig, + default=original_default, ) for declared_type, admitted in ((LazyDtypeConfig, True), (int, False)): replacement = make_dataclass( "PipelineConfig", - [("custom_config", declared_type, field(default_factory=forbidden_default))], + [ + ( + "custom_config", + declared_type, + field(default_factory=forbidden_default), + ) + ], ) monkeypatch.setattr(config_module, "PipelineConfig", replacement) resolved = config_module.runtime_config_parameter(parameter) @@ -431,16 +440,13 @@ def test_callable_contract_metadata_preserves_explicit_nominal_processing_contra def process(image): return image - vars(process)[ - FunctionContractAttribute.processing_contract - ] = FlexibleContract + vars(process)[FunctionContractAttribute.processing_contract] = FlexibleContract attach_callable_contract_metadata(process, processing_contract=Pure2DContract) contract = CallableContract.from_callable(process) assert ( - vars(process)[FunctionContractAttribute.processing_contract] - is FlexibleContract + vars(process)[FunctionContractAttribute.processing_contract] is FlexibleContract ) assert contract.processing_contract is FlexibleContract @@ -601,14 +607,18 @@ def test_function_reference_preserves_batching_through_compiler_and_transport( restored = pickle.loads(pickle.dumps(contract)) assert restored.func == reference expected_metadata = reference.metadata.with_prepared_signatures(func, func) - assert restored.metadata.canonical_signature == expected_metadata.canonical_signature + assert ( + restored.metadata.canonical_signature == expected_metadata.canonical_signature + ) assert restored.metadata == expected_metadata - assert restored.runtime_batch_executor( - RuntimeBatchExecutionDomain.PURE_2D_SLICES - ) is _batch_executor - assert restored.runtime_batch_executor( - RuntimeBatchExecutionDomain.MEASUREMENT_IMAGES - ) is _measurement_batch_executor + assert ( + restored.runtime_batch_executor(RuntimeBatchExecutionDomain.PURE_2D_SLICES) + is _batch_executor + ) + assert ( + restored.runtime_batch_executor(RuntimeBatchExecutionDomain.MEASUREMENT_IMAGES) + is _measurement_batch_executor + ) def test_reference_batch_family_preserves_wrapper_precedence_and_raw_inheritance( @@ -627,12 +637,14 @@ def test_reference_batch_family_preserves_wrapper_precedence_and_raw_inheritance ) contract = normalize_function_pattern(reference).groups[0].items[0].contract - assert contract.runtime_batch_executor( - RuntimeBatchExecutionDomain.PURE_2D_SLICES - ) is _override_batch_executor - assert contract.runtime_batch_executor( - RuntimeBatchExecutionDomain.MEASUREMENT_IMAGES - ) is _measurement_batch_executor + assert ( + contract.runtime_batch_executor(RuntimeBatchExecutionDomain.PURE_2D_SLICES) + is _override_batch_executor + ) + assert ( + contract.runtime_batch_executor(RuntimeBatchExecutionDomain.MEASUREMENT_IMAGES) + is _measurement_batch_executor + ) def test_registry_reference_preserves_real_intensity_batch_declarations() -> None: diff --git a/tests/unit/test_canonical_custom_source_lookup.py b/tests/unit/test_canonical_custom_source_lookup.py index 7067b036b2..780eb61e91 100644 --- a/tests/unit/test_canonical_custom_source_lookup.py +++ b/tests/unit/test_canonical_custom_source_lookup.py @@ -30,7 +30,11 @@ def source_owner(monkeypatch, tmp_path): store = tmp_path / "custom_functions" store.mkdir() - monkeypatch.setattr(CustomFunctionManager, "default_storage_directory", classmethod(lambda cls: store)) + monkeypatch.setattr( + CustomFunctionManager, + "default_storage_directory", + classmethod(lambda cls: store), + ) monkeypatch.setattr(CustomFunctionRuntimeRegistry, "_declarations_by_name", {}) monkeypatch.setattr(CustomFunctionRuntimeRegistry, "_published_exports", {}) monkeypatch.setattr(CustomFunctionRuntimeRegistry, "_preparation_outcomes", {}) @@ -49,14 +53,21 @@ def source(name: str, expression: str = "image") -> str: def forbid_catalog(monkeypatch) -> None: def forbidden(cls, **kwargs): pytest.fail("a selected custom source must not prepare the global catalog") - monkeypatch.setattr(RegistryService, "get_all_functions_with_metadata", classmethod(forbidden)) + + monkeypatch.setattr( + RegistryService, "get_all_functions_with_metadata", classmethod(forbidden) + ) @pytest.mark.parametrize("cache", [None, {}]) -def test_persisted_lookup_loads_only_exact_source_and_reuses_original_owner(source_owner, monkeypatch, cache): +def test_persisted_lookup_loads_only_exact_source_and_reuses_original_owner( + source_owner, monkeypatch, cache +): name = "canonical_selected_source" (source_owner / f"{name}.py").write_text(source(name), encoding="utf-8") - (source_owner / "unrelated_invalid.py").write_text("invalid source !!!", encoding="utf-8") + (source_owner / "unrelated_invalid.py").write_text( + "invalid source !!!", encoding="utf-8" + ) monkeypatch.setattr(RegistryService, "_metadata_cache", cache) forbid_catalog(monkeypatch) function = get_function(f"openhcs:{name}") @@ -65,41 +76,65 @@ def test_persisted_lookup_loads_only_exact_source_and_reuses_original_owner(sour assert get_function(metadata.composite_key) is function assert RegistryService._metadata_cache is cache assert tuple(CustomFunctionRuntimeRegistry.metadata_by_name()) == (name,) - assert metadata.import_identity.import_path == f"openhcs.processing.custom_functions.{name}" + assert ( + metadata.import_identity.import_path + == f"openhcs.processing.custom_functions.{name}" + ) @pytest.mark.parametrize("persist", [False, True]) -def test_published_lookup_does_not_reexecute_declaration_or_warm_catalog(source_owner, monkeypatch, persist): - [function] = CustomFunctionManager().register_from_code(source("canonical_published"), persist=persist) +def test_published_lookup_does_not_reexecute_declaration_or_warm_catalog( + source_owner, monkeypatch, persist +): + [function] = CustomFunctionManager().register_from_code( + source("canonical_published"), persist=persist + ) monkeypatch.setattr(RegistryService, "_metadata_cache", None) - monkeypatch.setattr(CustomFunctionManager, "_prepare_source", lambda *a: pytest.fail("reexecuted source")) + monkeypatch.setattr( + CustomFunctionManager, + "_prepare_source", + lambda *a: pytest.fail("reexecuted source"), + ) forbid_catalog(monkeypatch) assert get_function("openhcs:canonical_published") is function -@pytest.mark.parametrize("mutation", ["source_changed", "source_removed", "export_displaced", "declaration_removed"]) -def test_cached_custom_claim_never_rescues_stale_source(source_owner, monkeypatch, mutation): +@pytest.mark.parametrize( + "mutation", + ["source_changed", "source_removed", "export_displaced", "declaration_removed"], +) +def test_cached_custom_claim_never_rescues_stale_source( + source_owner, monkeypatch, mutation +): manager = CustomFunctionManager() name = "canonical_stale" manager.register_from_code(source(name)) metadata = CustomFunctionRuntimeRegistry.metadata_by_name()[name] - monkeypatch.setattr(RegistryService, "_metadata_cache", {metadata.composite_key: metadata}) + monkeypatch.setattr( + RegistryService, "_metadata_cache", {metadata.composite_key: metadata} + ) with monkeypatch.context() as mutation_patch: if mutation == "source_changed": - (source_owner / f"{name}.py").write_text(source(name, "image + 1"), encoding="utf-8") + (source_owner / f"{name}.py").write_text( + source(name, "image + 1"), encoding="utf-8" + ) elif mutation == "source_removed": (source_owner / f"{name}.py").unlink() elif mutation == "export_displaced": mutation_patch.setattr(custom_functions, name, object()) else: manager.delete_custom_function(name) - mutation_patch.setattr(RegistryService, "_metadata_cache", {metadata.composite_key: metadata}) + mutation_patch.setattr( + RegistryService, "_metadata_cache", {metadata.composite_key: metadata} + ) with pytest.raises(RuntimeError, match="changed; recompile"): get_function(metadata.composite_key) assert RegistryService._metadata_cache[metadata.composite_key] is metadata -@pytest.mark.parametrize("name", ["missing_canonical", "bad:name", "_private", "../outside"]) +@pytest.mark.parametrize( + "name", ["missing_canonical", "bad:name", "_private", "../outside"] +) def test_missing_or_noncanonical_source_is_rejected(source_owner, name): with pytest.raises(KeyError, match="Unknown canonical function ID"): get_function(f"openhcs:{name}") @@ -108,13 +143,19 @@ def test_missing_or_noncanonical_source_is_rejected(source_owner, name): def test_filename_declaration_mismatch_is_not_accepted(source_owner): - (source_owner / "canonical_mismatch.py").write_text(source("other_declaration"), encoding="utf-8") - with pytest.raises(ValidationError, match="filename and declaration name must match"): + (source_owner / "canonical_mismatch.py").write_text( + source("other_declaration"), encoding="utf-8" + ) + with pytest.raises( + ValidationError, match="filename and declaration name must match" + ): get_function("openhcs:canonical_mismatch") assert not CustomFunctionRuntimeRegistry.metadata_by_name() -def test_public_canonical_source_evaluation_and_render_roundtrip_use_selected_owner(source_owner, monkeypatch): +def test_public_canonical_source_evaluation_and_render_roundtrip_use_selected_owner( + source_owner, monkeypatch +): from openhcs.core.pipeline_document import PipelineDocumentCodec from openhcs.core.function_reference import FunctionReferenceTransportAuthority @@ -129,7 +170,10 @@ def test_public_canonical_source_evaluation_and_render_roundtrip_use_selected_ow ) function = CustomFunctionRuntimeRegistry.metadata_by_name()[name].func assert document.pipeline_steps[0].func is function - assert FunctionReferenceTransportAuthority.function_reference(function).resolve() is function + assert ( + FunctionReferenceTransportAuthority.function_reference(function).resolve() + is function + ) rendered = PipelineDocumentCodec.render(document) assert f"from openhcs.processing.custom_functions import {name}" in rendered assert "get_function" not in rendered @@ -137,29 +181,38 @@ def test_public_canonical_source_evaluation_and_render_roundtrip_use_selected_ow assert restored.pipeline_steps[0].func is function reference = FunctionReferenceTransportAuthority.function_reference(function) from openhcs.core.function_reference import ModuleExportRegistryFunctionReference + assert isinstance(reference, ModuleExportRegistryFunctionReference) assert reference.composite_key == f"openhcs:{name}" assert reference.resolve() is function namespace = {} - exec(compile(rendered, '', 'exec'), namespace) - assert namespace['pipeline_steps'][0].func is function + exec(compile(rendered, "", "exec"), namespace) + assert namespace["pipeline_steps"][0].func is function -def test_catalog_key_cannot_contradict_original_metadata_identity(source_owner, monkeypatch): +def test_catalog_key_cannot_contradict_original_metadata_identity( + source_owner, monkeypatch +): @numpy def independent_identity(image): raise AssertionError("lookup must not execute processing") metadata = FunctionMetadata( - name="actual_declared_name", func=independent_identity, - contract=FlexibleContract, registry=OpenHCSRegistry(), + name="actual_declared_name", + func=independent_identity, + contract=FlexibleContract, + registry=OpenHCSRegistry(), + ) + monkeypatch.setattr( + RegistryService, "_metadata_cache", {"openhcs:wrong_name": metadata} ) - monkeypatch.setattr(RegistryService, "_metadata_cache", {"openhcs:wrong_name": metadata}) with pytest.raises(ValueError, match="contradicts declaration"): get_function("openhcs:wrong_name") -def test_cached_native_collision_rejects_ambiguous_current_custom_owner(source_owner, monkeypatch): +def test_cached_native_collision_rejects_ambiguous_current_custom_owner( + source_owner, monkeypatch +): name = "canonical_ambiguous" CustomFunctionManager().register_from_code(source(name), persist=False) custom = CustomFunctionRuntimeRegistry.metadata_by_name()[name] @@ -169,15 +222,23 @@ def independent_native(image): raise AssertionError("lookup must not execute processing") native = FunctionMetadata( - name=name, func=independent_native, contract=FlexibleContract, - registry=OpenHCSRegistry(), original_name="independent_native", module=__name__, + name=name, + func=independent_native, + contract=FlexibleContract, + registry=OpenHCSRegistry(), + original_name="independent_native", + module=__name__, + ) + monkeypatch.setattr( + RegistryService, "_metadata_cache", {custom.composite_key: native} ) - monkeypatch.setattr(RegistryService, "_metadata_cache", {custom.composite_key: native}) with pytest.raises(ValueError, match="ambiguous owners"): get_function(custom.composite_key) -def test_new_nominal_registry_and_independent_capabilities_use_real_cooperative_mro(source_owner): +def test_new_nominal_registry_and_independent_capabilities_use_real_cooperative_mro( + source_owner, +): events = [] class TraceClaims: @@ -209,20 +270,31 @@ def independently_declared(image): registry = IndependentCanonicalRegistry() registry_type = type(registry) registry_type.probe = FunctionMetadata( - name="independently_declared", func=independently_declared, - contract=FlexibleContract, registry=registry, - original_name="independently_declared", module=__name__, + name="independently_declared", + func=independently_declared, + contract=FlexibleContract, + registry=registry, + original_name="independently_declared", + module=__name__, ) assert get_function(registry_type.probe.composite_key) is independently_declared assert events == ["trace-enter", "declaration", "trace-exit"] - assert registry_type.__mro__.index(TraceClaims) < registry_type.__mro__.index(DeclaredClaims) - assert registry_type.__mro__.index(OpenHCSRegistry) < registry_type.__mro__.index(LibraryRegistryBase) + assert registry_type.__mro__.index(TraceClaims) < registry_type.__mro__.index( + DeclaredClaims + ) + assert registry_type.__mro__.index( + OpenHCSRegistry + ) < registry_type.__mro__.index(LibraryRegistryBase) events.clear() with pytest.raises(KeyError, match="Unknown canonical function ID"): get_function(f"{registry.library_name}:missing") assert events == ["trace-enter", "declaration", "trace-exit"] finally: - dict.pop(LibraryRegistryBase.__registry__, IndependentCanonicalRegistry._registry_name, None) + dict.pop( + LibraryRegistryBase.__registry__, + IndependentCanonicalRegistry._registry_name, + None, + ) def test_shared_consumer_has_no_concrete_owner_dispatch(): @@ -232,7 +304,8 @@ def test_shared_consumer_has_no_concrete_owner_dispatch(): syntax = ast.parse(dedent(inspect.getsource(method))) assert not any(isinstance(node, ast.Match) for node in ast.walk(syntax)) assert not any( - isinstance(node, ast.Name) and node.id in {"OpenHCSRegistry", "CustomFunctionRuntimeRegistry"} + isinstance(node, ast.Name) + and node.id in {"OpenHCSRegistry", "CustomFunctionRuntimeRegistry"} for node in ast.walk(syntax) ) assert "get_all_functions_with_metadata" not in inspect.getsource(get_function) diff --git a/tests/unit/test_catalog_current_readiness.py b/tests/unit/test_catalog_current_readiness.py index 6dd712f033..640e32f059 100644 --- a/tests/unit/test_catalog_current_readiness.py +++ b/tests/unit/test_catalog_current_readiness.py @@ -26,7 +26,9 @@ FlexibleContract, ) from openhcs.processing.custom_functions.manager import CustomFunctionManager -from openhcs.processing.custom_functions.runtime_registry import CustomFunctionRuntimeRegistry +from openhcs.processing.custom_functions.runtime_registry import ( + CustomFunctionRuntimeRegistry, +) from openhcs.runtime.function_catalog_preparation import FunctionCatalogPreparation @@ -40,13 +42,19 @@ def catalog(monkeypatch, tmp_path): monkeypatch.setenv("XDG_DATA_HOME", str(tmp_path / "data")) manager = CustomFunctionManager() metadata = FunctionMetadata( - name="declaration", func=declaration, + name="declaration", + func=declaration, contract=FlexibleContract, registry=SimpleNamespace(library_name="probe"), - module=__name__, doc=declaration.__doc__, + module=__name__, + doc=declaration.__doc__, + ) + monkeypatch.setattr( + RegistryService, "_metadata_cache", {"probe:declaration": metadata} + ) + monkeypatch.setattr( + CustomFunctionRuntimeRegistry, "_source_revision", manager.source_revision() ) - monkeypatch.setattr(RegistryService, "_metadata_cache", {"probe:declaration": metadata}) - monkeypatch.setattr(CustomFunctionRuntimeRegistry, "_source_revision", manager.source_revision()) def projected_metadata(self, **kwargs): # Model the original registry/source owners publishing their revision. @@ -56,7 +64,8 @@ def projected_metadata(self, **kwargs): monkeypatch.setattr(FunctionCatalogService, "_all_metadata", projected_metadata) kernel_calls = [] monkeypatch.setattr( - RegistryService, "prepare_persistent_catalog", + RegistryService, + "prepare_persistent_catalog", lambda **kwargs: kernel_calls.append("kernel"), ) return FunctionCatalogService(), manager, kernel_calls @@ -73,7 +82,10 @@ def test_both_public_views_are_warm(catalog, monkeypatch): service, _, _ = catalog calls = [] original = service._entry - from openhcs.agent.services.function_catalog_service import PARAMETER_DOCUMENTATION_POLICY + from openhcs.agent.services.function_catalog_service import ( + PARAMETER_DOCUMENTATION_POLICY, + ) + parameter_calls = [] original_parameters = PARAMETER_DOCUMENTATION_POLICY.parameter_specs @@ -81,7 +93,9 @@ def parameter_specs(*args, **kwargs): parameter_calls.append(args[0]) return original_parameters(*args, **kwargs) - monkeypatch.setattr(PARAMETER_DOCUMENTATION_POLICY, "parameter_specs", parameter_specs) + monkeypatch.setattr( + PARAMETER_DOCUMENTATION_POLICY, "parameter_specs", parameter_specs + ) def entry(*args, **kwargs): calls.append(args[2:4]) @@ -94,7 +108,10 @@ def entry(*args, **kwargs): assert len(calls) == 1 for compact in (False, True, False, True): page = service.catalog(compact_signatures=compact) - assert service.search(query="declaration", compact_signatures=compact).items == page.items + assert ( + service.search(query="declaration", compact_signatures=compact).items + == page.items + ) service.require_revision(page.revision) assert len(calls) == 1 assert parameter_calls == [declaration] @@ -112,16 +129,21 @@ def test_stale_success_refreshes_without_kernel_rewarm(catalog, mutation): assert preparation.observe(handle()).outcome is Outcome.READY old = RegistryService._metadata_cache if mutation == "body_only": - manager.storage_dir.joinpath("source_probe.py").write_text("# source revision changed\n") + manager.storage_dir.joinpath("source_probe.py").write_text( + "# source revision changed\n" + ) elif mutation == "delete": RegistryService._metadata_cache = {} elif mutation == "update": RegistryService._metadata_cache = { - "probe:declaration": replace(old["probe:declaration"], doc="Updated declaration") + "probe:declaration": replace( + old["probe:declaration"], doc="Updated declaration" + ) } else: RegistryService._metadata_cache = { - **old, "probe:second": replace(old["probe:declaration"], name="second") + **old, + "probe:second": replace(old["probe:declaration"], name="second"), } state = preparation.observe(handle()) assert state.outcome is Outcome.NOT_STARTED @@ -216,7 +238,9 @@ def entry(*args, **kwargs): "probe:declaration": replace(old, doc="Current declaration prose") } if source_change: - manager.storage_dir.joinpath("midway.py").write_text("# current source\n") + manager.storage_dir.joinpath("midway.py").write_text( + "# current source\n" + ) changed.set() return result @@ -255,7 +279,9 @@ def cancelling_entry(*args, **kwargs): preparation.cancel_and_join() -def test_overlapping_projection_cannot_commit_old_view_into_new_metadata(catalog, monkeypatch): +def test_overlapping_projection_cannot_commit_old_view_into_new_metadata( + catalog, monkeypatch +): service, _, _ = catalog changed = Event() original = service._entry @@ -279,7 +305,10 @@ def overlapping_entry(*args, **kwargs): assert service.projections_current() assert all(projection.metadata is current for projection in service._projections) for compact in (False, True): - assert service.catalog(compact_signatures=compact).items[0].summary == declaration.__doc__ + assert ( + service.catalog(compact_signatures=compact).items[0].summary + == declaration.__doc__ + ) preparation.cancel_and_join() @@ -311,7 +340,8 @@ def test_startup_uses_original_main_thread_warmup(catalog, monkeypatch): service, _, child_kernels = catalog main_kernels = [] monkeypatch.setattr( - FunctionCatalogPreparation, "prepare_persistent_catalog", + FunctionCatalogPreparation, + "prepare_persistent_catalog", staticmethod(lambda **kwargs: main_kernels.append("main")), ) preparation = FunctionCatalogPreparation(service) @@ -324,22 +354,40 @@ def test_startup_uses_original_main_thread_warmup(catalog, monkeypatch): def test_original_abc_owns_refresh_and_both_implementations_are_concrete(): - from openhcs.agent.services.function_catalog_service import FunctionCatalogServiceABC - from openhcs.agent.services.endpoint_function_catalog_service import ZMQFunctionCatalogService + from openhcs.agent.services.function_catalog_service import ( + FunctionCatalogServiceABC, + ) + from openhcs.agent.services.endpoint_function_catalog_service import ( + ZMQFunctionCatalogService, + ) assert "projections_current" in FunctionCatalogServiceABC.__abstractmethods__ - assert FunctionCatalogService.prepare_projections is FunctionCatalogServiceABC.prepare_projections - assert ZMQFunctionCatalogService.prepare_projections is FunctionCatalogServiceABC.prepare_projections + assert ( + FunctionCatalogService.prepare_projections + is FunctionCatalogServiceABC.prepare_projections + ) + assert ( + ZMQFunctionCatalogService.prepare_projections + is FunctionCatalogServiceABC.prepare_projections + ) assert not FunctionCatalogService.__abstractmethods__ assert not ZMQFunctionCatalogService.__abstractmethods__ -@pytest.mark.parametrize("native_outcome", (Outcome.READY, Outcome.PENDING, Outcome.FAILED)) +@pytest.mark.parametrize( + "native_outcome", (Outcome.READY, Outcome.PENDING, Outcome.FAILED) +) def test_endpoint_current_readiness_uses_bound_native_status(native_outcome): from openhcs.agent.dto.common import SCHEMA_VERSION - from openhcs.agent.dto.functions import FunctionCatalogPreparationState, FunctionCatalogEntry, catalog_page + from openhcs.agent.dto.functions import ( + FunctionCatalogPreparationState, + FunctionCatalogEntry, + catalog_page, + ) from openhcs.agent.services.endpoint_function_catalog_service import ( - FunctionCatalogClientSession, FunctionCatalogProjection, ZMQFunctionCatalogService, + FunctionCatalogClientSession, + FunctionCatalogProjection, + ZMQFunctionCatalogService, ) from openhcs.runtime.zmq_config import OPENHCS_ZMQ_CONFIG from zmqruntime.startup import EndpointStartupPhase, EndpointStartupStatus @@ -350,11 +398,14 @@ def test_endpoint_current_readiness_uses_bound_native_status(native_outcome): def status(request): seen.append(request) return FunctionCatalogPreparationState( - schema_version=SCHEMA_VERSION, handle=request.handle, + schema_version=SCHEMA_VERSION, + handle=request.handle, outcome=native_outcome, progress=EndpointStartupStatus( - sequence=1, phase=EndpointStartupPhase.PREPARING_CAPABILITIES, - message="Native readiness witness", timestamp=0, + sequence=1, + phase=EndpointStartupPhase.PREPARING_CAPABILITIES, + message="Native readiness witness", + timestamp=0, ), ) @@ -362,20 +413,34 @@ def status(request): connected_endpoint=SimpleNamespace(process_identity=ProcessIdentity.current()), function_catalog_preparation=status, ) - service = ZMQFunctionCatalogService(lambda: endpoint, client_factory=lambda _: client) + service = ZMQFunctionCatalogService( + lambda: endpoint, client_factory=lambda _: client + ) assert not service.projections_current() assert not seen entry = FunctionCatalogEntry( - function_id="probe:declaration", import_path=__name__ + ".declaration", - name="declaration", module=__name__, library="probe", - signature="declaration(image)", summary="Witness", backend_tags=(), + function_id="probe:declaration", + import_path=__name__ + ".declaration", + name="declaration", + module=__name__, + library="probe", + signature="declaration(image)", + summary="Witness", + backend_tags=(), ) page = catalog_page( - items=(entry,), catalog_items=(entry,), total=1, limit=1, query=None, library=None, + items=(entry,), + catalog_items=(entry,), + total=1, + limit=1, + query=None, + library=None, ) service._client_session = FunctionCatalogClientSession(endpoint, client) service._endpoint_state = FunctionCatalogProjection.from_page( - endpoint, page, compact_signatures=True, + endpoint, + page, + compact_signatures=True, ) assert service.projections_current() is (native_outcome is Outcome.READY) assert len(seen) == 1 @@ -387,11 +452,17 @@ def test_original_control_family_stays_pending_during_refresh( catalog, monkeypatch, request_kind ): from openhcs.agent.dto.functions import ( - FunctionCatalogControlPayload, FunctionCatalogControlRequest, - FunctionCatalogPreparationControlResponse, FunctionDetailControlRequest, - FunctionReferenceControlRequest, FunctionSearchRequest, + FunctionCatalogControlPayload, + FunctionCatalogControlRequest, + FunctionCatalogPreparationControlResponse, + FunctionDetailControlRequest, + FunctionReferenceControlRequest, + FunctionSearchRequest, + ) + from openhcs.runtime.zmq_control import ( + ZMQControlMessageRouter, + ZMQControlRequestContext, ) - from openhcs.runtime.zmq_control import ZMQControlMessageRouter, ZMQControlRequestContext service, _, _ = catalog preparation = FunctionCatalogPreparation(service) @@ -425,13 +496,18 @@ def delayed(*args, **kwargs): try: assert entered.wait(timeout=2) response = ZMQControlMessageRouter.handle( - FunctionCatalogControlPayload.from_request(requests[request_kind]).to_dict(), + FunctionCatalogControlPayload.from_request( + requests[request_kind] + ).to_dict(), ZMQControlRequestContext( - compiled_artifacts={}, function_catalog=service, + compiled_artifacts={}, + function_catalog=service, function_catalog_preparation=preparation, ), ) - pending = FunctionCatalogPreparationControlResponse.from_control_response(response) + pending = FunctionCatalogPreparationControlResponse.from_control_response( + response + ) assert pending.status is not None assert not future.done() finally: diff --git a/tests/unit/test_cellprofiler_adapter.py b/tests/unit/test_cellprofiler_adapter.py index 90f8da35bb..70054dde60 100644 --- a/tests/unit/test_cellprofiler_adapter.py +++ b/tests/unit/test_cellprofiler_adapter.py @@ -207,7 +207,8 @@ def test_native_cellprofiler_imported_metadata_places_files_by_path_columns( with csv_path.open("a", encoding="utf-8") as handle: handle.write("unselected.tif,20585/\n") universe = NativeCellProfilerSelectedSourceUniverse( - (image_path, csv_path), placements=placements, + (image_path, csv_path), + placements=placements, imported_metadata_paths=(csv_path,), ) staged = universe.materialize_flat_input_dir(tmp_path / "staged") diff --git a/tests/unit/test_cellprofiler_additional_prepared_payload_abi.py b/tests/unit/test_cellprofiler_additional_prepared_payload_abi.py index c6439f57d4..849fa9a421 100644 --- a/tests/unit/test_cellprofiler_additional_prepared_payload_abi.py +++ b/tests/unit/test_cellprofiler_additional_prepared_payload_abi.py @@ -1,4 +1,5 @@ """Prepared raw arguments retain context consumed by nominal CP request owners.""" + from dataclasses import replace import numpy as np @@ -14,16 +15,27 @@ CellProfilerFunctionContractExecutor, ) from openhcs.processing.backends.cellprofiler.alignment import AlignModule, align -from openhcs.processing.backends.cellprofiler.area_occupied import measure_image_area_occupied +from openhcs.processing.backends.cellprofiler.area_occupied import ( + measure_image_area_occupied, +) from openhcs.processing.backends.cellprofiler.crop import CropModule, crop -from openhcs.processing.backends.cellprofiler.grid import ShapeChoice, identify_objects_in_grid -from openhcs.core.runtime_object_label_building import SourceImageObjectLabelBuildRequest +from openhcs.processing.backends.cellprofiler.grid import ( + ShapeChoice, + identify_objects_in_grid, +) +from openhcs.core.runtime_object_label_building import ( + SourceImageObjectLabelBuildRequest, +) from openhcs.core.runtime_spatial_grid import SpatialGrid from openhcs.core.source_metadata import SourceVoxelSpacing from openhcs.processing.backends.cellprofiler.image_quality import measure_image_quality from openhcs.processing.backends.cellprofiler.intensity import measure_object_intensity -from openhcs.processing.backends.cellprofiler.manual_objects import identify_objects_manually -from openhcs.processing.backends.cellprofiler.object_images import convert_image_to_objects +from openhcs.processing.backends.cellprofiler.manual_objects import ( + identify_objects_manually, +) +from openhcs.processing.backends.cellprofiler.object_images import ( + convert_image_to_objects, +) from openhcs.processing.backends.cellprofiler.thresholding import threshold from openhcs.processing.backends.cellprofiler.worms import ( identify_dead_worms, @@ -92,8 +104,13 @@ def test_prepared_primary_context_consumers_keep_nominal_payload_and_bare_pixels @pytest.mark.parametrize("shape", ((5, 5), (4, 7))) -@pytest.mark.parametrize("shape_choice", (ShapeChoice.RECTANGLE, ShapeChoice.NATURAL, ShapeChoice.CIRCLE_NATURAL)) -def test_prepared_grid_objects_preserve_rgb_mask_calibration_and_crop_domain(shape, shape_choice): +@pytest.mark.parametrize( + "shape_choice", + (ShapeChoice.RECTANGLE, ShapeChoice.NATURAL, ShapeChoice.CIRCLE_NATURAL), +) +def test_prepared_grid_objects_preserve_rgb_mask_calibration_and_crop_domain( + shape, shape_choice +): pixels = np.arange(np.prod(shape) * 3, dtype=np.float32).reshape(*shape, 3) mask = np.ones(shape, dtype=bool) mask[0] = False @@ -102,22 +119,35 @@ def test_prepared_grid_objects_preserve_rgb_mask_calibration_and_crop_domain(sha axes=PayloadAxes.colour_samples(-1), source_voxel_spacing=SourceVoxelSpacing((2.0, 0.7, 0.9)), source_spatial_domain=SourceSpatialDomain( - origin_yx=(2, 3), source_shape_yx=(shape[0] + 4, shape[1] + 6), + origin_yx=(2, 3), + source_shape_yx=(shape[0] + 4, shape[1] + 6), ), ) source = metadata.payload_with(pixels, mask) grid = SpatialGrid( - name="Grid", rows=1, columns=1, x_spacing=4.0, y_spacing=4.0, - x_origin=2.0, y_origin=2.0, source_spatial_shape_yx=shape, + name="Grid", + rows=1, + columns=1, + x_spacing=4.0, + y_spacing=4.0, + x_origin=2.0, + y_origin=2.0, + source_spatial_shape_yx=shape, ) guide_pixels = np.zeros(shape, dtype=np.int32) guide_pixels[1:3, 1:4] = 1 - guides = SourceImageObjectLabelBuildRequest(image=source, labels=guide_pixels).payload() - topology_inputs = (grid,) if shape_choice is ShapeChoice.RECTANGLE else (grid, guides) + guides = SourceImageObjectLabelBuildRequest( + image=source, labels=guide_pixels + ).payload() + topology_inputs = ( + (grid,) if shape_choice is ShapeChoice.RECTANGLE else (grid, guides) + ) contract = _prepared_contract(identify_objects_in_grid) returned_image, _rows, objects = CellProfilerFunctionContractExecutor().execute( - contract, contract.resolve_canonical_raw_callable(), source, + contract, + contract.resolve_canonical_raw_callable(), + source, {"topology_inputs": topology_inputs, "shape_choice": shape_choice}, execution_mode=FullStackExecution, ) @@ -126,14 +156,21 @@ def test_prepared_grid_objects_preserve_rgb_mask_calibration_and_crop_domain(sha assert returned_image.mask is mask assert objects.source_provenance == metadata.source_provenance assert objects.source_spatial_domain.origin_yx == (2, 3) - assert objects.source_spatial_domain.source_shape_yx == metadata.source_spatial_domain.source_shape_yx + assert ( + objects.source_spatial_domain.source_shape_yx + == metadata.source_spatial_domain.source_shape_yx + ) assert objects.parent_image_source_voxel_spacing == metadata.source_voxel_spacing assert objects.labels.shape == shape assert objects.domain.declared_object_count == 1 assert objects.domain.declared_object_ids == (1,) - expanded = objects.source_spatial_domain.materialize(objects.labels, spatial_axes_yx=(0, 1)) + expanded = objects.source_spatial_domain.materialize( + objects.labels, spatial_axes_yx=(0, 1) + ) assert expanded.shape == metadata.source_spatial_domain.source_shape_yx - np.testing.assert_array_equal(expanded[2:2 + shape[0], 3:3 + shape[1]], objects.labels) + np.testing.assert_array_equal( + expanded[2 : 2 + shape[0], 3 : 3 + shape[1]], objects.labels + ) @pytest.mark.parametrize("color", (False, True)) @@ -147,7 +184,10 @@ def test_prepared_align_preserves_named_plane_axis_masks_and_source_identity(col source_image_names=("First", "Second"), source_image_provenance_planes=SourceImageProvenancePlanes.from_components( paths=("/input/first.tif", "/input/second.tif"), - component_metadata=({"well": "A01", "channel": "1"}, {"well": "A01", "channel": "2"}), + component_metadata=( + {"well": "A01", "channel": "1"}, + {"well": "A01", "channel": "2"}, + ), ), plane_axis=RuntimePlaneAxis.SOURCE_BINDING, axes=PayloadAxes.colour_samples(3 if color else None), @@ -162,8 +202,10 @@ def test_prepared_align_preserves_named_plane_axis_masks_and_source_identity(col contract, contract.resolve_canonical_raw_callable(), source, - {"method": AlignModule.Method.NORMALIZED_CROSS_CORRELATION, - "crop_mode": AlignModule.CropMode.KEEP_SIZE}, + { + "method": AlignModule.Method.NORMALIZED_CROSS_CORRELATION, + "crop_mode": AlignModule.CropMode.KEEP_SIZE, + }, execution_mode=FullStackExecution, ) @@ -173,7 +215,9 @@ def test_prepared_align_preserves_named_plane_axis_masks_and_source_identity(col np.testing.assert_array_equal(output.data, pixels[index]) np.testing.assert_array_equal(output.mask, masks[index]) output_metadata = output.metadata - assert output_metadata.source_image_paths == (("/input/first.tif", "/input/second.tif")[index],) + assert output_metadata.source_image_paths == ( + ("/input/first.tif", "/input/second.tif")[index], + ) assert output_metadata.source_spatial_domain == metadata.source_spatial_domain assert output_metadata.axis_position(ColourAxis) == (2 if color else None) @@ -195,9 +239,11 @@ def test_prepared_crop_preserves_rgb_spatial_mask_and_calibrated_source_domain() contract, contract.resolve_canonical_raw_callable(), source, - {"left_right_rectangle_positions": (1, 4), - "top_bottom_rectangle_positions": (0, 3), - "removal_method": CropModule.RemovalMethod.NO}, + { + "left_right_rectangle_positions": (1, 4), + "top_bottom_rectangle_positions": (0, 3), + "removal_method": CropModule.RemovalMethod.NO, + }, execution_mode=NaturalExecution, ) @@ -219,7 +265,13 @@ def test_prepared_threshold_keeps_owned_unit_interval_proof_and_mask(): pixels = np.arange(20, dtype=np.uint8).reshape(4, 5) mask = np.ones(pixels.shape, dtype=bool) mask[0] = False - source = ImagePayloadMetadata.for_array(pixels).payload_with(pixels, mask).normalize_intensity_payload(dtype=np.float32,) + source = ( + ImagePayloadMetadata.for_array(pixels) + .payload_with(pixels, mask) + .normalize_intensity_payload( + dtype=np.float32, + ) + ) contract = _prepared_contract(threshold) projected = contract.raw_main_flow_call_argument(source) diff --git a/tests/unit/test_cellprofiler_analyst_export.py b/tests/unit/test_cellprofiler_analyst_export.py index b91206e86a..de9047d74e 100644 --- a/tests/unit/test_cellprofiler_analyst_export.py +++ b/tests/unit/test_cellprofiler_analyst_export.py @@ -142,7 +142,10 @@ def count_projection( ) -> FieldSpec | None: calls.append((table.name, subject, field_name)) return original( - self, table, field_name, subject=subject, + self, + table, + field_name, + subject=subject, project_database_field=project_database_field, ) @@ -168,7 +171,9 @@ def count_projection( subject=subject, ) - first_subject, first_rows, first_fields = next(projection.measurement_projections(first, scope=None)) + first_subject, first_rows, first_fields = next( + projection.measurement_projections(first, scope=None) + ) assert first_subject == subject assert first_fields == (FieldSpec("Count_Nuclei", int),) assert first_rows.row_mappings() == ( @@ -176,7 +181,9 @@ def count_projection( {"Count_Nuclei": 2}, {"Count_Nuclei": 3}, ) - second_subject, second_rows, second_fields = next(projection.measurement_projections(second, scope=None)) + second_subject, second_rows, second_fields = next( + projection.measurement_projections(second, scope=None) + ) assert second_subject == subject assert second_fields == (FieldSpec("Count_Nuclei", float),) assert second_rows.row_mappings() == ( @@ -203,7 +210,9 @@ def count_projection( ).row_mappings()[0] == {"Cells_AreaShape_Area": 1.0} assert projection._project_runtime_rows( object_table, - MeasurementProjectedColumnarRows({"AreaShape_Area": (2.0,)}, fields=(FieldSpec("AreaShape_Area", float),)), + MeasurementProjectedColumnarRows( + {"AreaShape_Area": (2.0,)}, fields=(FieldSpec("AreaShape_Area", float),) + ), subject=nuclei, field_projection_cache=object_field_cache, project_database_field=None, @@ -244,16 +253,21 @@ def test_database_field_projection_refreshes_declarations_and_live_dtype_hooks( table = MeasurementTable( name="EmptyDistances", rows=MeasurementProjectedColumnarRows( - {field.name: ()}, fields=(field,), + {field.name: ()}, + fields=(field,), ), subject=MeasurementSubject(MeasurementScope.RUN), measurement_feature_owner=RelateObjectsModule, ) - _subject, actual_rows, actual_fields = next(projection.measurement_projections(table, scope=None)) + _subject, actual_rows, actual_fields = next( + projection.measurement_projections(table, scope=None) + ) assert actual_rows.row_mappings() == () assert actual_fields == (field,) monkeypatch.undo() - _subject, actual_rows, actual_fields = next(projection.measurement_projections(table, scope=None)) + _subject, actual_rows, actual_fields = next( + projection.measurement_projections(table, scope=None) + ) assert actual_rows.row_mappings() == () assert actual_fields == (replace(field, dtype=int),) @@ -265,7 +279,9 @@ def database_measurement_field_projection(cls): return lambda source_field: replace(source_field, dtype=str) custom_table = replace(table, measurement_feature_owner=CustomFieldOwner) - _subject, actual_rows, actual_fields = next(projection.measurement_projections(custom_table, scope=None)) + _subject, actual_rows, actual_fields = next( + projection.measurement_projections(custom_table, scope=None) + ) assert actual_rows.row_mappings() == () assert actual_fields == (replace(field, dtype=str),) assert CustomFieldOwner.database_measurement_field(field).dtype is str @@ -858,13 +874,29 @@ def _borrowed_source_export_fixture( sources.extend( ( SourcePlaneProjection( - address=OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, site), (Microscopy.Channel, 1), (Microscopy.ZIndex, 1), (Microscopy.Timepoint, 1))), + address=OpenHCSPlaneAddress( + ( + (Microscopy.Well, "A01"), + (Microscopy.Site, site), + (Microscopy.Channel, 1), + (Microscopy.ZIndex, 1), + (Microscopy.Timepoint, 1), + ) + ), ref=SourcePixelRef("disk", str(original)), source_alias="DNA", source_metadata=metadata, ), SourceArtifactProjection( - address=OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, site), (Microscopy.Channel, 2), (Microscopy.ZIndex, 1), (Microscopy.Timepoint, 1))), + address=OpenHCSPlaneAddress( + ( + (Microscopy.Well, "A01"), + (Microscopy.Site, site), + (Microscopy.Channel, 2), + (Microscopy.ZIndex, 1), + (Microscopy.Timepoint, 1), + ) + ), ref=SourcePixelRef( "disk", str(calibration), (site - 1,) if embedded_tiff else () ), @@ -894,7 +926,7 @@ def _borrowed_source_export_fixture( metadata_handler=SimpleNamespace( source_workspace_root=lambda _plate_path: tmp_path, source_workspace_metadata_document=lambda _plate_path: document, - ) + ), ) context.filemanager = SimpleNamespace( exists=lambda *_args: False, @@ -963,7 +995,10 @@ def test_source_only_calibration_occurrences_export_each_executed_site( tmp_path ) assert batch.records_of_type(ImageArtifactType)["A01"] == () - from openhcs.core.source_workspace_projection import VirtualWorkspaceSourceProjection + from openhcs.core.source_workspace_projection import ( + VirtualWorkspaceSourceProjection, + ) + monkeypatch.setattr( VirtualWorkspaceSourceProjection, "load_binding_payloads", @@ -1936,13 +1971,27 @@ def _record_relationship( def _external_rows( table: CellProfilerProjectedTable, ) -> tuple[dict[str, object], ...]: - return tuple(dict(row) for row in (table.rows.iter_row_mappings() if isinstance(table.rows, ColumnarRows) else table.rows)) + return tuple( + dict(row) + for row in ( + table.rows.iter_row_mappings() + if isinstance(table.rows, ColumnarRows) + else table.rows + ) + ) def _field_rows( table: CellProfilerProjectedTable, ) -> tuple[dict[str, object], ...]: - return tuple(dict(row) for row in (table.rows.iter_row_mappings() if isinstance(table.rows, ColumnarRows) else table.rows)) + return tuple( + dict(row) + for row in ( + table.rows.iter_row_mappings() + if isinstance(table.rows, ColumnarRows) + else table.rows + ) + ) def test_database_projection_includes_derived_grid_measurements() -> None: @@ -1990,7 +2039,8 @@ def test_database_projection_includes_derived_grid_measurements() -> None: def test_source_digest_is_shared_across_image_numbers_and_channel_aliases( - tmp_path, monkeypatch, + tmp_path, + monkeypatch, ): from collections import defaultdict from hashlib import md5 @@ -2046,8 +2096,10 @@ def test_source_projection_frame_error_precedes_digest_read(tmp_path, monkeypatc source = tmp_path / "source.tif" tifffile.imwrite( - source, np.zeros((3, 4, 5), dtype=np.uint16), - photometric="minisblack", metadata={"axes": "ZYX"}, + source, + np.zeros((3, 4, 5), dtype=np.uint16), + photometric="minisblack", + metadata={"axes": "ZYX"}, ) def forbidden_read(self): @@ -2057,7 +2109,9 @@ def forbidden_read(self): monkeypatch.setattr(Path, "read_bytes", forbidden_read) with pytest.raises(ValueError, match="Source frame index True"): export._source_image_projection_values( - source, "DNA", CellProfilerDatabaseColumnDialect(), + source, + "DNA", + CellProfilerDatabaseColumnDialect(), source_axis_indices=(True,), ) assert ImageFileRevision.md5_digest.cache_info().currsize == 0 @@ -2071,12 +2125,16 @@ def test_repeated_source_contributors_share_physical_projection(tmp_path, monkey source = tmp_path / "DNA.tif" import tifffile + tifffile.imwrite( - source, np.arange(192, dtype=np.uint16).reshape(3, 8, 8), - photometric="minisblack", metadata={"axes": "ZYX"}, + source, + np.arange(192, dtype=np.uint16).reshape(3, 8, 8), + photometric="minisblack", + metadata={"axes": "ZYX"}, ) provenance = SourceImageProvenance( - source_path=str(source), source_component_metadata={"well": "A01", "site": "1"}, + source_path=str(source), + source_component_metadata={"well": "A01", "site": "1"}, source_image_names=("DNA",), ) owner = CPATableRowProjection( @@ -2086,77 +2144,126 @@ def test_repeated_source_contributors_share_physical_projection(tmp_path, monkey rows, metadata = {}, defaultdict(list) manufacture = export._source_image_projection_values calls = [] + def counted(*args, **kwargs): calls.append(args[0]) return manufacture(*args, **kwargs) + monkeypatch.setattr(export, "_source_image_projection_values", counted) for _ in range(3): owner.collect_image_provenance( - provenance, scope=RuntimeExecutionAxisScope(AXIS_ID), source_image_name="DNA", - image_rows_by_number=rows, source_metadata_by_image_number=metadata, + provenance, + scope=RuntimeExecutionAxisScope(AXIS_ID), + source_image_name="DNA", + image_rows_by_number=rows, + source_metadata_by_image_number=metadata, ) assert calls == [source] assert len(metadata[1]) == 3 # Every contributor's correlation evidence survives. conflicting = provenance.with_source_path(str(tmp_path / "other.tif")) with pytest.raises(ValueError, match="conflicting values"): owner.collect_image_provenance( - conflicting, scope=RuntimeExecutionAxisScope(AXIS_ID), source_image_name="DNA", - image_rows_by_number=rows, source_metadata_by_image_number=metadata, + conflicting, + scope=RuntimeExecutionAxisScope(AXIS_ID), + source_image_name="DNA", + image_rows_by_number=rows, + source_metadata_by_image_number=metadata, ) with pytest.raises(ValueError, match="conflicting values"): owner.collect_image_provenance( - provenance, scope=RuntimeExecutionAxisScope(AXIS_ID), source_image_name="DNA", - image_rows_by_number=rows, source_metadata_by_image_number=metadata, + provenance, + scope=RuntimeExecutionAxisScope(AXIS_ID), + source_image_name="DNA", + image_rows_by_number=rows, + source_metadata_by_image_number=metadata, source_axis_indices=(1,), ) assert calls == [source] # Explicit frame admission still rejects contradiction. -def test_produced_thumbnail_uses_named_pixels_not_measurement_source(tmp_path, monkeypatch): +def test_produced_thumbnail_uses_named_pixels_not_measurement_source( + tmp_path, monkeypatch +): import openhcs.interop.cellprofiler.analyst_export as export + source = tmp_path / "DNA.tif" Image.fromarray(np.arange(64, dtype=np.uint8).reshape(8, 8)).save(source) store = RuntimeValueStore() - dna = _record_image(store, name="DNA", source_path=str(source), metadata={"well": "A01", "site": "1"}) - planes = SourceImageProvenancePlanes.from_components( - paths=(str(source),), component_metadata=({"well": "A01", "site": "1"},), + dna = _record_image( + store, + name="DNA", + source_path=str(source), + metadata={"well": "A01", "site": "1"}, ) - measurements = tuple(_record_measurements( - store, table=MeasurementTable( - name=f"Measurements{index}", source_image_names=("DNA",), - subject=MeasurementSubject(MeasurementScope.SAMPLE, "Image"), - rows=MeasurementSparseColumnarRows.from_rows( - ({RUNTIME_IMAGE_FIELD: 0, f"Value{index}": float(index)},), - fields=(FieldSpec(RUNTIME_IMAGE_FIELD, int), FieldSpec(f"Value{index}", float)), + planes = SourceImageProvenancePlanes.from_components( + paths=(str(source),), + component_metadata=({"well": "A01", "site": "1"},), + ) + measurements = tuple( + _record_measurements( + store, + table=MeasurementTable( + name=f"Measurements{index}", + source_image_names=("DNA",), + subject=MeasurementSubject(MeasurementScope.SAMPLE, "Image"), + rows=MeasurementSparseColumnarRows.from_rows( + ({RUNTIME_IMAGE_FIELD: 0, f"Value{index}": float(index)},), + fields=( + FieldSpec(RUNTIME_IMAGE_FIELD, int), + FieldSpec(f"Value{index}", float), + ), + ), ), - ), source_image_provenance_planes=planes, - ) for index in range(3)) + source_image_provenance_planes=planes, + ) + for index in range(3) + ) batch = RuntimeArtifactBatch( - input_specs=(dna, *measurements), records_by_axis={AXIS_ID: store.values()}, + input_specs=(dna, *measurements), + records_by_axis={AXIS_ID: store.values()}, source_image_set_identity_policy=SourceImageSetIdentityPolicy(), ) - settings = replace(_settings(), write_image_thumbnails=True, thumbnail_image_names=("DNA",), auto_scale_thumbnail_intensities=False) + settings = replace( + _settings(), + write_image_thumbnails=True, + thumbnail_image_names=("DNA",), + auto_scale_thumbnail_intensities=False, + ) encode = export._thumbnail_png_base64 pixels = [] + def counted(image, **kwargs): pixels.append(np.asarray(image).copy()) return encode(image, **kwargs) + monkeypatch.setattr(export, "_thumbnail_png_base64", counted) - projection = _projection_builder().build(batch, settings, (CPAImageChannelSpec(alias="DNA", image_name="DNA", channel_color="none"),)) + projection = _projection_builder().build( + batch, + settings, + (CPAImageChannelSpec(alias="DNA", image_name="DNA", channel_color="none"),), + ) assert len(pixels) == 1 assert np.array_equal(pixels[0], np.zeros((8, 8), dtype=np.uint8)) - assert _external_rows(projection.image_table)[0]["Image_Thumbnail_DNA"] == encode(pixels[0], auto_scale=False) + assert _external_rows(projection.image_table)[0]["Image_Thumbnail_DNA"] == encode( + pixels[0], auto_scale=False + ) def test_source_bound_thumbnail_only_ref_preserves_executed_sites(tmp_path): - builder, batch, _channels, calibration, _document = _borrowed_source_export_fixture(tmp_path) + builder, batch, _channels, calibration, _document = _borrowed_source_export_fixture( + tmp_path + ) builder.context.filemanager.load_batch = lambda paths, _backend: [ np.load(calibration) for _path in paths ] - builder.context.filemanager.physical_source_path = lambda address, _backend, *, base_path: address + builder.context.filemanager.physical_source_path = ( + lambda address, _backend, *, base_path: address + ) settings = replace( - _settings(), write_image_thumbnails=True, thumbnail_image_names=("IllumDNA",), + _settings(), + write_image_thumbnails=True, + thumbnail_image_names=("IllumDNA",), auto_scale_thumbnail_intensities=False, ) projection = builder.build(batch, settings, ()) @@ -2164,5 +2271,7 @@ def test_source_bound_thumbnail_only_ref_preserves_executed_sites(tmp_path): assert len(rows) == 2 # The third declared source occurrence was not executed. assert all(row["Image_Thumbnail_IllumDNA"] for row in rows) assert all("Image_FileName_IllumDNA" not in row for row in rows) - disabled = replace(settings, write_image_thumbnails=False, thumbnail_image_names=("Missing",)) + disabled = replace( + settings, write_image_thumbnails=False, thumbnail_image_names=("Missing",) + ) assert len(_external_rows(builder.build(batch, disabled, ()).image_table)) == 2 diff --git a/tests/unit/test_cellprofiler_artifact_declarations.py b/tests/unit/test_cellprofiler_artifact_declarations.py index 29144593ee..c3810635bb 100644 --- a/tests/unit/test_cellprofiler_artifact_declarations.py +++ b/tests/unit/test_cellprofiler_artifact_declarations.py @@ -596,30 +596,35 @@ def test_calculate_math_retains_measurement_subjects_across_producer_channels() ), ) contract = _callable_contract( - _module(8, "CalculateMath", { - "Name the output measurement": "Ratio", - "Operation": "Divide", - "Select the numerator objects": "Nuclei", - "Select the numerator measurement": "Intensity_MeanIntensity_CorrProtein", - "Select the denominator objects": "Cells", - "Select the denominator measurement": "Intensity_MeanIntensity_CorrProtein", - }), + _module( + 8, + "CalculateMath", + { + "Name the output measurement": "Ratio", + "Operation": "Divide", + "Select the numerator objects": "Nuclei", + "Select the numerator measurement": "Intensity_MeanIntensity_CorrProtein", + "Select the denominator objects": "Cells", + "Select the denominator measurement": "Intensity_MeanIntensity_CorrProtein", + }, + ), step_index=7, available_artifacts=ArtifactSpecCollection((image, *objects, measurements)), main_flow_artifacts=ArtifactSpecCollection((image,)), available_artifact_producers=artifact_producers_for_outputs( (measurements,), groups=("2", "1"), - invocation_keys=(FunctionInvocationKey( - "measure_object_intensity", DEFAULT_GROUP_KEY, 0 - ),), + invocation_keys=( + FunctionInvocationKey("measure_object_intensity", DEFAULT_GROUP_KEY, 0), + ), ), ) (measurement_input,) = contract.artifact_inputs.of_artifact_type( MeasurementsArtifactType ) assert tuple( - relation.source.name for relation in measurement_input.relations + relation.source.name + for relation in measurement_input.relations if isinstance(relation, InputObjectMeasurementSourceRelation) ) == ("Nuclei", "Cells") storage = ArtifactInputPlan( diff --git a/tests/unit/test_cellprofiler_backend_public_api.py b/tests/unit/test_cellprofiler_backend_public_api.py index 0a6e9298f6..778290f49e 100644 --- a/tests/unit/test_cellprofiler_backend_public_api.py +++ b/tests/unit/test_cellprofiler_backend_public_api.py @@ -284,7 +284,6 @@ def test_callable_ownership_uses_complete_import_identity() -> None: assert CellProfilerModule.for_callable_contract(numpy_contract) is None - @pytest.mark.parametrize("mutation", ("replacement", "unowned_exclusion")) def test_compiled_callable_owner_keeps_current_declaration_admission( monkeypatch: pytest.MonkeyPatch, @@ -313,6 +312,7 @@ def test_compiled_callable_owner_keeps_current_declaration_admission( finally: set_parameter_exclusions(raw, original) + def test_duplicate_function_ownership_fails_during_module_declaration() -> None: existing_owner = CellProfilerModule.require_module("Crop") diff --git a/tests/unit/test_cellprofiler_display_manual_projection_declarations.py b/tests/unit/test_cellprofiler_display_manual_projection_declarations.py index 305a42a2ee..818484e397 100644 --- a/tests/unit/test_cellprofiler_display_manual_projection_declarations.py +++ b/tests/unit/test_cellprofiler_display_manual_projection_declarations.py @@ -550,7 +550,9 @@ def test_display_callables_resolve_values_from_declared_measurement_tables() -> assert density.row_mappings()[0]["num_points"] == 2 assert histogram.row_mappings()[0]["total_count"] == 2 assert scatter.row_mappings()[0]["point_count"] == 2 - assert {row["well_name"] for row in platemap.row_mappings() if "well_name" in row} == { + assert { + row["well_name"] for row in platemap.row_mappings() if "well_name" in row + } == { "A01", "A02", } diff --git a/tests/unit/test_cellprofiler_export_equivalence.py b/tests/unit/test_cellprofiler_export_equivalence.py index a2703aeaa3..5adb3dc374 100644 --- a/tests/unit/test_cellprofiler_export_equivalence.py +++ b/tests/unit/test_cellprofiler_export_equivalence.py @@ -224,7 +224,9 @@ def test_native_shards_merge_cpa_declared_image_and_object_rows( _write_sharded_native_output(second, (2,)) report = cellprofiler_native_shard_equivalence( - reference, (first, second), policy=RuntimeEquivalencePolicy(measurement_dialect=SAVED_OUTPUT_DIALECT) + reference, + (first, second), + policy=RuntimeEquivalencePolicy(measurement_dialect=SAVED_OUTPUT_DIALECT), ) assert report.is_equivalent @@ -265,7 +267,9 @@ def test_native_shards_merge_declared_relationship_rows_and_view( ) report = cellprofiler_native_shard_equivalence( - reference, (first, second), policy=RuntimeEquivalencePolicy(measurement_dialect=SAVED_OUTPUT_DIALECT) + reference, + (first, second), + policy=RuntimeEquivalencePolicy(measurement_dialect=SAVED_OUTPUT_DIALECT), ) assert report.is_equivalent @@ -280,7 +284,9 @@ def test_native_shards_reject_overlapping_image_sets(tmp_path: Path) -> None: _write_sharded_native_output(second, (1,)) report = cellprofiler_native_shard_equivalence( - reference, (first, second), policy=RuntimeEquivalencePolicy(measurement_dialect=SAVED_OUTPUT_DIALECT) + reference, + (first, second), + policy=RuntimeEquivalencePolicy(measurement_dialect=SAVED_OUTPUT_DIALECT), ) assert not report.is_equivalent @@ -305,7 +311,9 @@ def test_native_shards_reject_nonpartitionable_experiment_aggregate( ) report = cellprofiler_native_shard_equivalence( - reference, (first, second), policy=RuntimeEquivalencePolicy(measurement_dialect=SAVED_OUTPUT_DIALECT) + reference, + (first, second), + policy=RuntimeEquivalencePolicy(measurement_dialect=SAVED_OUTPUT_DIALECT), ) assert any( @@ -324,7 +332,9 @@ def test_native_shards_reject_undeclared_side_writes(tmp_path: Path) -> None: (second / "unexpected.txt").write_text("side write") report = cellprofiler_native_shard_equivalence( - reference, (first, second), policy=RuntimeEquivalencePolicy(measurement_dialect=SAVED_OUTPUT_DIALECT) + reference, + (first, second), + policy=RuntimeEquivalencePolicy(measurement_dialect=SAVED_OUTPUT_DIALECT), ) assert any("unexpected files" in message for message in report.failure_messages()) @@ -634,7 +644,9 @@ def test_database_export_equivalence_applies_numeric_tolerance( report = cellprofiler_database_export_equivalence( reference, RuntimeExportObservation.from_output_root(candidate), - policy=RuntimeEquivalencePolicy(numeric_abs_tolerance=0.001, measurement_dialect=SAVED_OUTPUT_DIALECT), + policy=RuntimeEquivalencePolicy( + numeric_abs_tolerance=0.001, measurement_dialect=SAVED_OUTPUT_DIALECT + ), ) assert report.is_equivalent @@ -892,7 +904,9 @@ def test_database_column_declarations_own_export_and_equivalence_names() -> None ) -def test_database_measurements_keep_complete_relationship_cohort(tmp_path: Path) -> None: +def test_database_measurements_keep_complete_relationship_cohort( + tmp_path: Path, +) -> None: import pytest from benchmark.cellprofiler_export_equivalence import _sqlite_database_differences @@ -901,7 +915,9 @@ def test_database_measurements_keep_complete_relationship_cohort(tmp_path: Path) "Per_Parents": MeasurementSubject(MeasurementScope.OBJECT, "Parents"), "Per_Children": MeasurementSubject(MeasurementScope.OBJECT, "Children"), } - reference, candidate = (tmp_path / name for name in ("reference.db", "candidate.db")) + reference, candidate = ( + tmp_path / name for name in ("reference.db", "candidate.db") + ) for path, value in ((reference, 1.5), (candidate, 1.5 + 1e-13)): with sqlite3.connect(path) as connection: connection.execute( diff --git a/tests/unit/test_cellprofiler_full_stack_materialization.py b/tests/unit/test_cellprofiler_full_stack_materialization.py index 325733c61f..550d4713ae 100644 --- a/tests/unit/test_cellprofiler_full_stack_materialization.py +++ b/tests/unit/test_cellprofiler_full_stack_materialization.py @@ -5,7 +5,14 @@ import numpy as np import pytest -from openhcs.core.aligned_image_payload import (AlignedImageSliceContext, AlignedImageStack, ImageOutputBundle, ImagePayloadStackComposition, ImagePayloadSliceStack, compose_aligned_image_payload) +from openhcs.core.aligned_image_payload import ( + AlignedImageSliceContext, + AlignedImageStack, + ImageOutputBundle, + ImagePayloadStackComposition, + ImagePayloadSliceStack, + compose_aligned_image_payload, +) from openhcs.core.artifacts import ArtifactSpec, ImageArtifactType from openhcs.core.callable_contract import CallableContract, CallableMetadata from openhcs.core.measurement_row_materialization import MeasurementSparseColumnarRows @@ -14,7 +21,10 @@ MeasurementSubject, MeasurementTable, ) -from openhcs.core.runtime_object_labels import ObjectLabelPayload, ObjectLabelVariantData +from openhcs.core.runtime_object_labels import ( + ObjectLabelPayload, + ObjectLabelVariantData, +) from openhcs.core.runtime_image_values import ImagePayloadMetadata from openhcs.core.runtime_plane_projection import ( RuntimePlaneAxis, @@ -31,7 +41,10 @@ CellProfilerFunctionContractExecutor, ) from openhcs.processing.backends.cellprofiler.intensity import rescale_intensity -from openhcs.processing.backends.cellprofiler.morphology import remove_holes, remove_holes_3d +from openhcs.processing.backends.cellprofiler.morphology import ( + remove_holes, + remove_holes_3d, +) from openhcs.core.processing_contracts import ( ProcessingContract, Pure3DContract, @@ -65,7 +78,9 @@ def test_real_rescale_full_stack_materializes_composed_runtime_sources(): @pytest.mark.parametrize("function", (remove_holes, remove_holes_3d)) -def test_literal_volume_keeps_dense_contract_semantics_without_bundle_classification(function): +def test_literal_volume_keeps_dense_contract_semantics_without_bundle_classification( + function, +): pixels = np.ones((60, 6, 8), dtype=np.float32) pixels[20:22, 2:4, 3:5] = 0 mask = np.ones(pixels.shape, dtype=bool) @@ -73,7 +88,8 @@ def test_literal_volume_keeps_dense_contract_semantics_without_bundle_classifica metadata = ImagePayloadMetadata( plane_axis=RuntimePlaneAxis.RUNTIME_SLICE, source_voxel_spacing=SourceVoxelSpacing( - (1.25, 2.5, 3.0), SourceVoxelSpacingUnit.MICROMETERS, + (1.25, 2.5, 3.0), + SourceVoxelSpacingUnit.MICROMETERS, ), source_spatial_domain=SourceSpatialDomain(source_shape_yx=(6, 8)), source_image_provenance_planes=SourceImageProvenancePlanes.from_components( @@ -83,12 +99,18 @@ def test_literal_volume_keeps_dense_contract_semantics_without_bundle_classifica ) dense = metadata.payload_with(pixels, mask) projection = RuntimePlaneAxisValueProjection.preserve( - axis=RuntimePlaneAxis.RUNTIME_SLICE, axis_size=60, + axis=RuntimePlaneAxis.RUNTIME_SLICE, + axis_size=60, ) literal = ImagePayloadSliceStack.from_output_slices( - tuple(RuntimeSliceProjection.value_for_slice(dense, projection.selected_plane(index)) - for index in range(60)), - memory_type="numpy", plane_axis=RuntimePlaneAxis.RUNTIME_SLICE, + tuple( + RuntimeSliceProjection.value_for_slice( + dense, projection.selected_plane(index) + ) + for index in range(60) + ), + memory_type="numpy", + plane_axis=RuntimePlaneAxis.RUNTIME_SLICE, ) assert not isinstance(literal, AlignedImageStack) composition = compose_aligned_image_payload("literal volume", (literal,)) @@ -96,24 +118,39 @@ def test_literal_volume_keeps_dense_contract_semantics_without_bundle_classifica assert composition.plane_axis is RuntimePlaneAxis.RUNTIME_SLICE assert literal._composed_payload is None contract = CallableContract.from_callable(function) - contract = replace(contract, metadata=replace( - contract.metadata, artifact_outputs=(ArtifactSpec.output("Filled", ImageArtifactType),), - )) + contract = replace( + contract, + metadata=replace( + contract.metadata, + artifact_outputs=(ArtifactSpec.output("Filled", ImageArtifactType),), + ), + ) executor = CellProfilerFunctionContractExecutor() outputs = tuple( executor.execute( - contract, contract.resolve_canonical_raw_callable(), value, {}, - execution_mode=compose_aligned_image_payload("volume", (value,)).execution_mode, + contract, + contract.resolve_canonical_raw_callable(), + value, + {}, + execution_mode=compose_aligned_image_payload( + "volume", (value,) + ).execution_mode, plane_projection=projection, ) for value in (dense, literal) ) - np.testing.assert_array_equal(ImagePayload.of(outputs[1]).data, ImagePayload.of(outputs[0]).data) - np.testing.assert_array_equal(ImagePayload.of(outputs[1]).mask, ImagePayload.of(outputs[0]).mask) + np.testing.assert_array_equal( + ImagePayload.of(outputs[1]).data, ImagePayload.of(outputs[0]).data + ) + np.testing.assert_array_equal( + ImagePayload.of(outputs[1]).mask, ImagePayload.of(outputs[0]).mask + ) assert ImagePayload.of(outputs[1]).metadata == ImagePayload.of(outputs[0]).metadata -@pytest.mark.parametrize("processing_contract", tuple(ProcessingContract.__registry__.values())) +@pytest.mark.parametrize( + "processing_contract", tuple(ProcessingContract.__registry__.values()) +) @pytest.mark.parametrize("slice_count", (1, 2)) def test_every_full_stack_processing_family_materializes_image_and_image_kwargs( processing_contract, slice_count @@ -213,12 +250,7 @@ def test_aligned_materialization_retains_masks_calibration_and_exact_contributor } == set(expected_paths) np.testing.assert_array_equal( dense.mask, - np.stack( - [ - np.broadcast_to(plane.mask, (2, 3, 4)) - for plane in composed.slices - ] - ), + np.stack([np.broadcast_to(plane.mask, (2, 3, 4)) for plane in composed.slices]), ) for index in range(2): projected = RuntimeSliceProjection.value_for_slice( @@ -227,9 +259,7 @@ def test_aligned_materialization_retains_masks_calibration_and_exact_contributor axis=RuntimePlaneAxis.RUNTIME_SLICE, plane_index=index, axis_size=2 ), ) - np.testing.assert_array_equal( - projected.data, composed.slices[index].data - ) + np.testing.assert_array_equal(projected.data, composed.slices[index].data) expected = {f"/synthetic/A01_s{index + 1}_w{channel}.tif" for channel in (1, 2)} assert { identity.path @@ -248,7 +278,10 @@ def test_named_output_bundle_materializes_its_own_source_binding_axis(): ) dense = RuntimeSliceProjection.full_stack_value(bundle) assert dense.metadata.plane_axis is RuntimePlaneAxis.SOURCE_BINDING - assert dense.metadata.source_provenance.represented_source_image_names == ("red", "green") + assert dense.metadata.source_provenance.represented_source_image_names == ( + "red", + "green", + ) np.testing.assert_array_equal(dense.data, np.stack(planes)) assert all( slice_payload.data is plane @@ -289,9 +322,7 @@ def test_named_bundle_preserves_outer_aliases_with_real_inner_plane_provenance() axis=RuntimePlaneAxis.SOURCE_BINDING, plane_index=index, axis_size=2 ), ) - np.testing.assert_array_equal( - selected.data, payloads[index].data - ) + np.testing.assert_array_equal(selected.data, payloads[index].data) selected_metadata = selected.metadata assert selected_metadata.source_image_names == (alias,) assert { @@ -307,8 +338,10 @@ class CalibratedAlignedImageStack(AlignedImageStack): """A generated acquisition grid owns one additional metadata fact.""" def composition_payload_metadata(self, metadata): - return super().composition_payload_metadata(metadata).replace_fields( - source_voxel_spacing=spacing + return ( + super() + .composition_payload_metadata(metadata) + .replace_fields(source_voxel_spacing=spacing) ) data = np.arange(24, dtype=np.float32).reshape(2, 3, 4) @@ -384,7 +417,9 @@ def test_full_stack_preserves_nonimage_identity_domains_and_dense_images(): assert materialized[name] is value -@pytest.mark.parametrize("processing_contract", tuple(ProcessingContract.__registry__.values())) +@pytest.mark.parametrize( + "processing_contract", tuple(ProcessingContract.__registry__.values()) +) def test_full_stack_raw_callable_keeps_opaque_nonimage_kwargs(processing_contract): opaque = object() image = np.ones((2, 3, 4), dtype=np.float32) diff --git a/tests/unit/test_cellprofiler_function_contract_execution.py b/tests/unit/test_cellprofiler_function_contract_execution.py index 26bca67424..726e05ca53 100644 --- a/tests/unit/test_cellprofiler_function_contract_execution.py +++ b/tests/unit/test_cellprofiler_function_contract_execution.py @@ -13,7 +13,11 @@ from arraybridge import ArrayPayload import openhcs.core.callable_contract as callable_contract_module -from openhcs.core.aligned_image_payload import (AlignedImageStack, ImageOutputBundle, compose_aligned_image_payload) +from openhcs.core.aligned_image_payload import ( + AlignedImageStack, + ImageOutputBundle, + compose_aligned_image_payload, +) from openhcs.core.image_payload_execution_mode import ImagePayloadExecutionMode from openhcs.core.artifacts import ( ArtifactSpec, @@ -66,8 +70,9 @@ ) - -def _execute_one_slice(contract, raw, projection, value, kwargs, slice_index, slice_count): +def _execute_one_slice( + contract, raw, projection, value, kwargs, slice_index, slice_count +): """Run one plane of a per-plane contract in the CellProfiler environment.""" call = CellProfilerContractCall(contract, raw, projection) return contextualize_main_image_output( @@ -75,6 +80,7 @@ def _execute_one_slice(contract, raw, projection, value, kwargs, slice_index, sl call.invoke(value, call.slice_kwargs(None, kwargs, slice_index, slice_count)), ) + def _compiled_contract( func: Callable[..., object], processing_contract: ProcessingContract, @@ -112,7 +118,9 @@ def test_morphological_skeleton_executes_one_planar_runtime_image() -> None: assert result.data.shape == image.shape -def test_registered_morph_distance_projects_raw_abi_and_retains_source_context() -> None: +def test_registered_morph_distance_projects_raw_abi_and_retains_source_context() -> ( + None +): contract = CallableContract.from_callable(morph) contract = replace( contract, @@ -144,7 +152,8 @@ def test_registered_morph_distance_projects_raw_abi_and_retains_source_context() {"operation": MorphOperation.DISTANCE}, execution_mode=NaturalExecution, plane_projection=RuntimePlaneAxisValueProjection.preserve( - axis=RuntimePlaneAxis.RUNTIME_SLICE, axis_size=1, + axis=RuntimePlaneAxis.RUNTIME_SLICE, + axis_size=1, ), ) @@ -174,7 +183,8 @@ def payload_raw(image: ArrayPayload, *, labels: np.ndarray) -> ArrayPayload: assert isinstance(image, ArrayPayload) seen.append((image, labels)) return image.metadata.payload_with( - image.data + labels, image.mask, + image.data + labels, + image.mask, ) raw = payload_raw if payload_annotation else array_raw @@ -187,19 +197,29 @@ def payload_raw(image: ArrayPayload, *, labels: np.ndarray) -> ArrayPayload: ), ) source = ImagePayloadMetadata(source_image_names=("DNA",)).payload_with( - np.ones((3, 4), dtype=np.float32), np.ones((3, 4), dtype=bool), + np.ones((3, 4), dtype=np.float32), + np.ones((3, 4), dtype=bool), ) selected_labels = np.full((3, 4), 5, dtype=np.float32) projection = RuntimePlaneAxisValueProjection.preserve( - axis=RuntimePlaneAxis.RUNTIME_SLICE, axis_size=2, + axis=RuntimePlaneAxis.RUNTIME_SLICE, + axis_size=2, ) assert contract.runtime_main_flow_call_argument(source) is source result = _execute_one_slice( - contract, raw, projection, source, - {"labels": RuntimeSliceAlignedValues(slices=(np.zeros((3, 4)), selected_labels)), - "adapter_only_control": 1}, - 1, 2, + contract, + raw, + projection, + source, + { + "labels": RuntimeSliceAlignedValues( + slices=(np.zeros((3, 4)), selected_labels) + ), + "adapter_only_control": 1, + }, + 1, + 2, ) assert seen[0][0] is (source if payload_annotation else source.data) @@ -216,10 +236,12 @@ def bad_shape(image: np.ndarray) -> np.ndarray: contract = _compiled_contract(bad_shape, Pure2DContract) source = ImagePayloadMetadata(source_image_names=("DNA",)).payload_with( - np.ones((3, 4)), np.ones((3, 4), dtype=bool), + np.ones((3, 4)), + np.ones((3, 4), dtype=bool), ) projection = RuntimePlaneAxisValueProjection.preserve( - axis=RuntimePlaneAxis.RUNTIME_SLICE, axis_size=1, + axis=RuntimePlaneAxis.RUNTIME_SLICE, + axis_size=1, ) with pytest.raises(ValueError, match="[Mm]ask.*shape|shape.*[Mm]ask"): _execute_one_slice(contract, bad_shape, projection, source, {}, 0, 1) @@ -263,11 +285,16 @@ def payload_raw(image: ArrayPayload) -> ArrayPayload: ), ) source = ImagePayloadMetadata(source_image_names=("DNA",)).payload_with( - np.ones((3, 4)), None, + np.ones((3, 4)), + None, ) result = CellProfilerFunctionContractExecutor().execute( - contract, raw, source, {}, execution_mode=execution_mode, + contract, + raw, + source, + {}, + execution_mode=execution_mode, ) assert seen == [source if payload_annotation else source.data] @@ -294,7 +321,8 @@ def payload_raw(image: ArrayPayload) -> ArrayPayload: ), ) def test_canonical_argument_projection_uses_declared_nominal_annotation( - annotation: object, retains_payload: bool, + annotation: object, + retains_payload: bool, ) -> None: def raw(image): return image @@ -303,7 +331,8 @@ def raw(image): raw.__annotations__["image"] = annotation contract = _compiled_contract(raw, Pure2DContract) source = ImagePayloadMetadata(source_image_names=("DNA",)).payload_with( - np.ones((3, 4)), None, + np.ones((3, 4)), + None, ) expected = source if retains_payload else source.data @@ -320,7 +349,9 @@ def raw(image): assert adapted.runtime_main_flow_call_argument(source) is source -def test_raw_slice_argument_retains_buffer_identity_for_inplace_array_callable() -> None: +def test_raw_slice_argument_retains_buffer_identity_for_inplace_array_callable() -> ( + None +): def inplace(image: np.ndarray) -> np.ndarray: image[0, 0] = 7 return image @@ -328,10 +359,12 @@ def inplace(image: np.ndarray) -> np.ndarray: contract = _compiled_contract(inplace, Pure2DContract) pixels = np.zeros((3, 4)) source = ImagePayloadMetadata(source_image_names=("DNA",)).payload_with( - pixels, None, + pixels, + None, ) projection = RuntimePlaneAxisValueProjection.preserve( - axis=RuntimePlaneAxis.RUNTIME_SLICE, axis_size=1, + axis=RuntimePlaneAxis.RUNTIME_SLICE, + axis_size=1, ) result = _execute_one_slice(contract, inplace, projection, source, {}, 0, 1) @@ -349,14 +382,17 @@ def raw(image: np.ndarray) -> np.ndarray: return image contract = _compiled_contract( - raw, Pure2DContract, + raw, + Pure2DContract, artifact_outputs=(ArtifactSpec.output("Result", ImageArtifactType),), ) source = ImagePayloadMetadata( - plane_axis=RuntimePlaneAxis.RUNTIME_SLICE, source_image_names=("DNA",), + plane_axis=RuntimePlaneAxis.RUNTIME_SLICE, + source_image_names=("DNA",), ).payload_with(np.ones((12, 3, 4)), None) projection = RuntimePlaneAxisValueProjection.preserve( - axis=RuntimePlaneAxis.RUNTIME_SLICE, axis_size=12, + axis=RuntimePlaneAxis.RUNTIME_SLICE, + axis_size=12, ) executor = CellProfilerFunctionContractExecutor() hints = [] @@ -381,7 +417,10 @@ def counted_signature(func, **kwargs): raw.__annotations__["image"] = annotation contract = contract.with_prepared_signature() result = executor.execute( - contract, raw, source, {}, + contract, + raw, + source, + {}, execution_mode=NaturalExecution, plane_projection=projection, ) @@ -403,23 +442,31 @@ def reject_premature_annotations(self): raise AssertionError("Input guard resolved the raw ABI prematurely") monkeypatch.setattr( - CallableContract, "canonical_parameter_declarations", + CallableContract, + "canonical_parameter_declarations", property(reject_premature_annotations), ) with pytest.raises(TypeError, match="requires AlignedImageStack"): CellProfilerFunctionContractExecutor().execute( - contract, raw, np.zeros((2, 3)), {}, + contract, + raw, + np.zeros((2, 3)), + {}, execution_mode=AlignedStackExecution, ) projection = RuntimePlaneAxisValueProjection.preserve( - axis=RuntimePlaneAxis.RUNTIME_SLICE, axis_size=2, + axis=RuntimePlaneAxis.RUNTIME_SLICE, + axis_size=2, ) mismatched_image = ImagePayloadMetadata( plane_axis=RuntimePlaneAxis.SOURCE_BINDING, ).payload_with(np.zeros((2, 3, 4)), None) with pytest.raises(RuntimeSliceProjectionDeclarationError, match="conflicts"): CellProfilerFunctionContractExecutor().execute( - contract, raw, mismatched_image, {}, + contract, + raw, + mismatched_image, + {}, execution_mode=NaturalExecution, plane_projection=projection, ) @@ -434,16 +481,25 @@ def raw(image: np.ndarray) -> np.ndarray: return image contract = _compiled_contract( - raw, Pure2DContract, + raw, + Pure2DContract, artifact_outputs=(ArtifactSpec.output("Result", ImageArtifactType),), ) - source = ImagePayloadMetadata(plane_axis=RuntimePlaneAxis.RUNTIME_SLICE).payload_with( - np.ones((2, 3, 4)), None, + source = ImagePayloadMetadata( + plane_axis=RuntimePlaneAxis.RUNTIME_SLICE + ).payload_with( + np.ones((2, 3, 4)), + None, ) result = CellProfilerFunctionContractExecutor().execute( - contract, raw, source, {}, execution_mode=NaturalExecution, + contract, + raw, + source, + {}, + execution_mode=NaturalExecution, plane_projection=RuntimePlaneAxisValueProjection.preserve( - axis=RuntimePlaneAxis.RUNTIME_SLICE, axis_size=2, + axis=RuntimePlaneAxis.RUNTIME_SLICE, + axis_size=2, ), ) @@ -456,66 +512,107 @@ def foreign_raw(image: ArrayPayload) -> ArrayPayload: foreign_contract = _compiled_contract(foreign_raw, Pure2DContract) policy = RuntimeCallablePolicy(kwarg_policy=SignatureFilteredKwargs) - assert policy.contract_invocation(foreign_contract, foreign_raw, source, {}).call() is source + assert ( + policy.contract_invocation(foreign_contract, foreign_raw, source, {}).call() + is source + ) -def test_actual_prepared_executor_planes_make_no_signature_or_hint_queries(monkeypatch) -> None: +def test_actual_prepared_executor_planes_make_no_signature_or_hint_queries( + monkeypatch, +) -> None: calls = [] + def raw(image: np.ndarray, scale: int = 2) -> np.ndarray: calls.append(image.shape) return image * scale + CallableProjection.from_callable(raw).warm_canonical_signature() contract = _compiled_contract( - raw, Pure2DContract, - artifact_outputs=(ArtifactSpec.output("Result",ImageArtifactType),), + raw, + Pure2DContract, + artifact_outputs=(ArtifactSpec.output("Result", ImageArtifactType),), ) source = ImagePayloadMetadata( - plane_axis=RuntimePlaneAxis.RUNTIME_SLICE,source_image_names=("DNA",), - ).payload_with(np.ones((12,3,4)),None) - projection = RuntimePlaneAxisValueProjection.preserve(axis=RuntimePlaneAxis.RUNTIME_SLICE,axis_size=12) + plane_axis=RuntimePlaneAxis.RUNTIME_SLICE, + source_image_names=("DNA",), + ).payload_with(np.ones((12, 3, 4)), None) + projection = RuntimePlaneAxisValueProjection.preserve( + axis=RuntimePlaneAxis.RUNTIME_SLICE, axis_size=12 + ) + def forbidden(*args, **kwargs): raise AssertionError("Prepared execution introspected a live callable") - monkeypatch.setattr(inspect,"signature",forbidden) - monkeypatch.setattr(callable_contract_module,"get_type_hints",forbidden) + + monkeypatch.setattr(inspect, "signature", forbidden) + monkeypatch.setattr(callable_contract_module, "get_type_hints", forbidden) result = CellProfilerFunctionContractExecutor().execute( - contract,raw,source,{},execution_mode=NaturalExecution, + contract, + raw, + source, + {}, + execution_mode=NaturalExecution, plane_projection=projection, ) - np.testing.assert_array_equal(result.data,np.full((12,3,4),2)) - assert calls == [(3,4)]*12 + np.testing.assert_array_equal(result.data, np.full((12, 3, 4), 2)) + assert calls == [(3, 4)] * 12 -def test_authored_distinct_raw_abi_batch_has_no_runtime_queries_and_filters_controls(monkeypatch) -> None: +def test_authored_distinct_raw_abi_batch_has_no_runtime_queries_and_filters_controls( + monkeypatch, +) -> None: from openhcs.core.processing_preparation import CallablePreparation + calls = [] + def raw(image: np.ndarray, scale: int = 2) -> np.ndarray: calls.append(image.shape) return image * scale + @wraps(raw) def wrapper(*args, **kwargs): return raw(*args, **kwargs) - wrapper.__signature__ = inspect.signature(raw).replace(parameters=[ - *inspect.signature(raw).parameters.values(), - inspect.Parameter("injected",inspect.Parameter.KEYWORD_ONLY,default=False), - ]) - monkeypatch.setattr(CallablePreparation,"prepare",lambda self:None) + + wrapper.__signature__ = inspect.signature(raw).replace( + parameters=[ + *inspect.signature(raw).parameters.values(), + inspect.Parameter( + "injected", inspect.Parameter.KEYWORD_ONLY, default=False + ), + ] + ) + monkeypatch.setattr(CallablePreparation, "prepare", lambda self: None) contract = CallableContract.from_prepared_callable(wrapper) - contract = replace(contract,metadata=replace( - contract.metadata,processing_contract=Pure2DContract, - artifact_outputs=(ArtifactSpec.output("Result",ImageArtifactType),), - )) - source = ImagePayloadMetadata(plane_axis=RuntimePlaneAxis.RUNTIME_SLICE,source_image_names=("DNA",)).payload_with(np.ones((12,3,4)),None) - projection = RuntimePlaneAxisValueProjection.preserve(axis=RuntimePlaneAxis.RUNTIME_SLICE,axis_size=12) + contract = replace( + contract, + metadata=replace( + contract.metadata, + processing_contract=Pure2DContract, + artifact_outputs=(ArtifactSpec.output("Result", ImageArtifactType),), + ), + ) + source = ImagePayloadMetadata( + plane_axis=RuntimePlaneAxis.RUNTIME_SLICE, source_image_names=("DNA",) + ).payload_with(np.ones((12, 3, 4)), None) + projection = RuntimePlaneAxisValueProjection.preserve( + axis=RuntimePlaneAxis.RUNTIME_SLICE, axis_size=12 + ) + def forbidden(*args, **kwargs): raise AssertionError("Prepared authored raw target queried a live signature") - monkeypatch.setattr(inspect,"signature",forbidden) - monkeypatch.setattr(callable_contract_module,"get_type_hints",forbidden) + + monkeypatch.setattr(inspect, "signature", forbidden) + monkeypatch.setattr(callable_contract_module, "get_type_hints", forbidden) result = CellProfilerFunctionContractExecutor().execute( - contract,wrapper,source,{"injected":True}, - execution_mode=NaturalExecution,plane_projection=projection, + contract, + wrapper, + source, + {"injected": True}, + execution_mode=NaturalExecution, + plane_projection=projection, ) - np.testing.assert_array_equal(result.data,np.full((12,3,4),2)) - assert calls == [(3,4)]*12 + np.testing.assert_array_equal(result.data, np.full((12, 3, 4), 2)) + assert calls == [(3, 4)] * 12 def test_compiled_slice_execution_resolves_raw_target_once_and_preserves_request_binding( @@ -601,9 +698,11 @@ def decorated(*args: object, **kwargs: object) -> object: signature = inspect.signature(raw) decorated.__signature__ = signature.replace( parameters=( - parameter.replace(default=canonical_default) - if parameter.name == "slice_by_slice" - else parameter + ( + parameter.replace(default=canonical_default) + if parameter.name == "slice_by_slice" + else parameter + ) for parameter in signature.parameters.values() ) ) @@ -624,14 +723,12 @@ def decorated(*args: object, **kwargs: object) -> object: ), ) - assert seen == ( - [((4, 5), True)] * 3 - if canonical_default - else [((3, 4, 5), False)] - ) + assert seen == ([((4, 5), True)] * 3 if canonical_default else [((3, 4, 5), False)]) -@pytest.mark.parametrize("processing_contract", tuple(ProcessingContract.__registry__.values())) +@pytest.mark.parametrize( + "processing_contract", tuple(ProcessingContract.__registry__.values()) +) @pytest.mark.parametrize( ("image_mode", "expected_call_count"), ( @@ -671,10 +768,7 @@ def dispatch_probe(image: np.ndarray) -> np.ndarray: ) ) - if ( - image_mode is AlignedStackExecution - and processing_contract is Pure3DContract - ): + if image_mode is AlignedStackExecution and processing_contract is Pure3DContract: with pytest.raises( ValueError, match=( diff --git a/tests/unit/test_cellprofiler_geometry_prepared_payload_abi.py b/tests/unit/test_cellprofiler_geometry_prepared_payload_abi.py index 4a9e6d8e65..03801cf2de 100644 --- a/tests/unit/test_cellprofiler_geometry_prepared_payload_abi.py +++ b/tests/unit/test_cellprofiler_geometry_prepared_payload_abi.py @@ -43,7 +43,8 @@ def _prepared_contract(func): def test_prepared_resize_preserves_typed_primary_and_bare_array(func): pixels = np.arange(24, dtype=np.float32).reshape(4, 6) source = ImagePayloadMetadata( - source_path="/images/input.tif", source_image_names=("Original",), + source_path="/images/input.tif", + source_image_names=("Original",), ).payload_with(pixels, pixels % 2 == 0) contract = _prepared_contract(func) @@ -65,9 +66,14 @@ def test_prepared_full_stack_resize_preserves_existing_metadata_and_mask( ImagePayloadMetadata( source_path=f"/images/z{index}.tif", source_image_names=("Original",), - source_component_metadata={"z_index": index, "timepoint": 1, "well": "W001"}, + source_component_metadata={ + "z_index": index, + "timepoint": 1, + "well": "W001", + }, source_spatial_domain=SourceSpatialDomain( - origin_yx=(2, 3), source_shape_yx=(8, 10), + origin_yx=(2, 3), + source_shape_yx=(8, 10), ), ).payload_with(np.full((4, 6), index, dtype=np.float32), None) for index in (3, 1, 2) @@ -89,21 +95,22 @@ def test_prepared_full_stack_resize_preserves_existing_metadata_and_mask( contract = _prepared_contract(resize_volumetric) canonical = contract.resolve_canonical_raw_callable() kwargs = dict( - resizing_factor_x=0.5, resizing_factor_y=0.5, resizing_factor_z=z_factor, + resizing_factor_x=0.5, + resizing_factor_y=0.5, + resizing_factor_z=z_factor, ) expected = canonical(source, **kwargs) result = CellProfilerFunctionContractExecutor().execute( - contract, canonical, source, kwargs, + contract, + canonical, + source, + kwargs, execution_mode=FullStackExecution, ) - np.testing.assert_array_equal( - result.data, expected.data - ) - np.testing.assert_array_equal( - result.mask, expected.mask - ) + np.testing.assert_array_equal(result.data, expected.data) + np.testing.assert_array_equal(result.mask, expected.mask) assert result.metadata == expected.metadata assert result.metadata.plane_axis is axis assert result.metadata.source_image_paths == metadata.source_image_paths @@ -129,7 +136,10 @@ def test_prepared_resize_bare_pixels_preserve_canonical_return_contract(func): expected = canonical(pixels, **kwargs) result = CellProfilerFunctionContractExecutor().execute( - contract, canonical, pixels, kwargs, + contract, + canonical, + pixels, + kwargs, execution_mode=FullStackExecution, ) diff --git a/tests/unit/test_cellprofiler_identify_secondary_replacement_primary.py b/tests/unit/test_cellprofiler_identify_secondary_replacement_primary.py index 57baed1854..ca7552197b 100644 --- a/tests/unit/test_cellprofiler_identify_secondary_replacement_primary.py +++ b/tests/unit/test_cellprofiler_identify_secondary_replacement_primary.py @@ -229,7 +229,10 @@ def test_replacement_primary_runtime_emits_typed_labels_and_relationships() -> N primary_labels = SourceImageObjectLabelBuildRequest( image=image, labels=primary_array, - variants={UneditedLabels: primary_array.copy(), SmallRemovedLabels: primary_array.copy()}, + variants={ + UneditedLabels: primary_array.copy(), + SmallRemovedLabels: primary_array.copy(), + }, declared_object_ids=(1, 2), ).payload() @@ -299,7 +302,9 @@ def test_replacement_primary_runtime_emits_typed_labels_and_relationships() -> N variantless_replacement = variantless_result[-1] assert isinstance(variantless_replacement, ObjectLabelValue) assert variantless_replacement.variant_data.variant_labels(UneditedLabels) is None - assert variantless_replacement.variant_data.variant_labels(SmallRemovedLabels) is None + assert ( + variantless_replacement.variant_data.variant_labels(SmallRemovedLabels) is None + ) assert ( variantless_replacement.source_provenance == primary_without_variants.source_provenance diff --git a/tests/unit/test_cellprofiler_image_quality.py b/tests/unit/test_cellprofiler_image_quality.py index d5716ae0df..14eb0df4e3 100644 --- a/tests/unit/test_cellprofiler_image_quality.py +++ b/tests/unit/test_cellprofiler_image_quality.py @@ -544,7 +544,9 @@ def test_quality_control_measurements_keep_distinct_site_image_identities() -> N object_name=None, source_image_name=None, source_metadata=ImagePayloadMetadata( - source_provenance=MeasurementTable.joined_source_provenance("quality_metrics", tables), + source_provenance=MeasurementTable.joined_source_provenance( + "quality_metrics", tables + ), ), ) @@ -706,8 +708,6 @@ def test_running_variance_preserves_numpy_accumulation(dtype, reverse): if reverse: values = values[::-1] means = values.cumsum() / np.arange(1, len(values) + 1) - accumulated = ( - (values[1:] - means[:-1]) * (values[1:] - means[1:]) - ).cumsum() + accumulated = ((values[1:] - means[:-1]) * (values[1:] - means[1:])).cumsum() expected = np.hstack(([0], accumulated / np.arange(1, len(values)))) np.testing.assert_array_equal(running_variance_numba(values), expected) diff --git a/tests/unit/test_cellprofiler_library_loading.py b/tests/unit/test_cellprofiler_library_loading.py index 19af8406fa..5f6cec73fd 100644 --- a/tests/unit/test_cellprofiler_library_loading.py +++ b/tests/unit/test_cellprofiler_library_loading.py @@ -11,7 +11,10 @@ import skimage.morphology import skimage.segmentation from python_introspect import declared_enum_type -from openhcs.core.aligned_image_payload import (AlignedImageStack, ImagePayloadBundleContext) +from openhcs.core.aligned_image_payload import ( + AlignedImageStack, + ImagePayloadBundleContext, +) from openhcs.core.callable_contract import CallableContract from openhcs.core.runtime_tabular_values import FieldSpec from openhcs.processing.backends.cellprofiler.alignment import ( @@ -199,7 +202,6 @@ ) - def test_module_registry_resolves_every_declared_function(): for module_type in CellProfilerModule.__registry__.values(): if not module_type.emits_function_step(): @@ -261,16 +263,14 @@ def test_measure_colocalization_declares_composed_stack_execution() -> None: def test_track_objects_callable_requires_timepoint(): track_objects = CellProfilerModule.__registry__["TrackObjects"].require_callable() - assert CallableContract.from_callable( - track_objects - ).required_axis_roles == (TimeAxis,) + assert CallableContract.from_callable(track_objects).required_axis_roles == ( + TimeAxis, + ) def test_correct_illumination_apply_inherits_resolved_stack_grouping() -> None: assert ( - CallableContract.from_callable( - correct_illumination_apply - ).required_axis_roles + CallableContract.from_callable(correct_illumination_apply).required_axis_roles == () ) assert ( @@ -280,9 +280,7 @@ def test_correct_illumination_apply_inherits_resolved_stack_grouping() -> None: def test_mask_objects_has_no_unconditional_image_stack_axis() -> None: - assert ( - CallableContract.from_callable(mask_objects).required_axis_roles == () - ) + assert CallableContract.from_callable(mask_objects).required_axis_roles == () def test_all_cellprofiler_module_aliases_canonicalize(): @@ -902,7 +900,9 @@ def test_edge_direction_round_trips_through_compiled_worker_transport(direction) assert EdgeDirection(direction.value) is direction restored = pickle.loads(pickle.dumps(direction, protocol=pickle.HIGHEST_PROTOCOL)) assert restored is direction - assert restored.includes_horizontal_response == direction.includes_horizontal_response + assert ( + restored.includes_horizontal_response == direction.includes_horizontal_response + ) assert restored.includes_vertical_response == direction.includes_vertical_response @@ -2103,7 +2103,9 @@ def test_watershed_xy_downsample_factors_preserve_leading_axes(): assert watershed_xy_downsample_factors(2, 2) == (2.0, 2.0) assert watershed_xy_downsample_factors(3, 2) == (1.0, 2.0, 2.0) assert watershed_xy_downsample_factors(4, 2) == (1.0, 1.0, 2.0, 2.0) - from openhcs.processing.backends.cellprofiler.morphology import MorphologyBackendStrategy + from openhcs.processing.backends.cellprofiler.morphology import ( + MorphologyBackendStrategy, + ) labels = MorphologyBackendStrategy.for_memory_type().label_equal_values( np.ones((2, 3, 4, 5), dtype=bool) @@ -3706,10 +3708,7 @@ def test_absorbed_processing_contract_metadata_does_not_act_as_validator(): image = np.ones((8, 8), dtype=np.float32) result = correct_illumination_calculate(image, dtype_config=DtypeConfig()) assert result.shape == image.shape - assert ( - correct_illumination_calculate.__processing_contract__ - is FlexibleContract - ) + assert correct_illumination_calculate.__processing_contract__ is FlexibleContract assert opening.__processing_contract__ is FlexibleContract @@ -4363,7 +4362,9 @@ def flexible(image): def test_unmix_colors_returns_one_output_per_stain_row(): image = np.full((8, 9, 3), 0.5, dtype=np.float32) outputs = unmix_colors( - ImagePayloadMetadata(axes=PayloadAxes.colour_samples(-1)).payload_with(image, None), + ImagePayloadMetadata(axes=PayloadAxes.colour_samples(-1)).payload_with( + image, None + ), stain_names=(StainType.HEMATOXYLIN, StainType.EOSIN, StainType.CUSTOM), custom_absorbances=((0.5, 0.5, 0.5), (0.5, 0.5, 0.5), (0.1, 0.2, 0.3)), dtype_config=DtypeConfig(), @@ -4372,8 +4373,7 @@ def test_unmix_colors_returns_one_output_per_stain_row(): assert [output.shape for output in outputs.slices] == [(8, 9), (8, 9), (8, 9)] assert all((output.dtype == np.float32 for output in outputs.slices)) assert all( - output.metadata.axis_position(ColourAxis) is None - for output in outputs.slices + output.metadata.axis_position(ColourAxis) is None for output in outputs.slices ) assert unmix_colors.__processing_contract__ is FlexibleContract @@ -4408,9 +4408,9 @@ def test_binary_mask_output_preserves_declared_color_channel_axis() -> None: def test_crop_preserves_hwc_color_image_domain() -> None: image = np.arange(8 * 9 * 3, dtype=np.uint8).reshape(8, 9, 3) - image_payload = ImagePayloadMetadata(axes=PayloadAxes.colour_samples(-1)).payload_with( - image, None - ) + image_payload = ImagePayloadMetadata( + axes=PayloadAxes.colour_samples(-1) + ).payload_with(image, None) cropped, mask, measurements = crop( image_payload, removal_method=CropModule.RemovalMethod.ALL, @@ -4539,10 +4539,7 @@ def test_measure_image_area_occupied_runs_mixed_rows(): "DNA", "Nuclei", ] - assert ( - measure_image_area_occupied.__processing_contract__ - is FlexibleContract - ) + assert measure_image_area_occupied.__processing_contract__ is FlexibleContract def test_measure_image_area_occupied_uses_runtime_slice_for_every_operand_row(): @@ -5079,7 +5076,10 @@ def test_overlay_outlines_renders_exact_projected_empty_label_plane(): ), ) def test_measurement_batch_admission_uses_declared_processing_domain( - contract, mode, slice_by_slice, admitted, + contract, + mode, + slice_by_slice, + admitted, ) -> None: image = np.stack((np.ones((2, 3)), np.full((2, 3), 2.0))) request = RuntimeBatchInvocationRequest( diff --git a/tests/unit/test_cellprofiler_measurement_rows.py b/tests/unit/test_cellprofiler_measurement_rows.py index 21ff0d08bc..335e731b63 100644 --- a/tests/unit/test_cellprofiler_measurement_rows.py +++ b/tests/unit/test_cellprofiler_measurement_rows.py @@ -695,7 +695,9 @@ def test_concatenated_measurement_rows_preserve_structural_missing_cells() -> No ) -def test_declared_wide_admission_preserves_features_before_absent_object_column() -> None: +def test_declared_wide_admission_preserves_features_before_absent_object_column() -> ( + None +): rows = MeasurementSparseColumnarRows.from_rows( ({"slice_index": 1, "contrast": 0.25, "entropy": 0.75},), fields=( @@ -857,7 +859,9 @@ def test_wide_measurement_projection_uses_row_owned_scope_for_artifact_table() - @pytest.mark.parametrize("layout", ["identity", "wide", "long"]) def test_measurement_row_contract_owns_output_identity_without_renumbering(layout): - from openhcs.interop.cellprofiler.measurement_dialect import CELLPROFILER_MEASUREMENT_DIALECT + from openhcs.interop.cellprofiler.measurement_dialect import ( + CELLPROFILER_MEASUREMENT_DIALECT, + ) columns = {"slice_index": (1, 1), "object_label": (2, 17)} if layout == "wide": @@ -866,8 +870,10 @@ def test_measurement_row_contract_owns_output_identity_without_renumbering(layou columns.update(feature_name=("Area", "Area"), result_value=(3.0, 4.0)) rows = MeasurementSparseColumnarRows( columns, - fields=tuple(FieldSpec(name, str if name == "feature_name" else float) - for name in columns), + fields=tuple( + FieldSpec(name, str if name == "feature_name" else float) + for name in columns + ), ) for contract, output_name in ( (PlainMeasurementDialect.row_identity_contract, "object_label"), @@ -875,10 +881,15 @@ def test_measurement_row_contract_owns_output_identity_without_renumbering(layou ): accumulator = WideMeasurementRowAccumulator(contract) accumulator.add( - rows, lambda feature, qualifiers: feature, - default_subject="Objects", default_scope=MeasurementScope.OBJECT, + rows, + lambda feature, qualifiers: feature, + default_subject="Objects", + default_scope=MeasurementScope.OBJECT, ) projected = accumulator.row_mappings_by_subject()["Objects"] assert tuple(row[output_name] for row in projected) == (2, 17) - assert all(("object_label" if output_name == "object_number" else "object_number") - not in row for row in projected) + assert all( + ("object_label" if output_name == "object_number" else "object_number") + not in row + for row in projected + ) diff --git a/tests/unit/test_cellprofiler_module_execution.py b/tests/unit/test_cellprofiler_module_execution.py index 610c22a693..b46bc4a2bd 100644 --- a/tests/unit/test_cellprofiler_module_execution.py +++ b/tests/unit/test_cellprofiler_module_execution.py @@ -20,7 +20,16 @@ DirectParentReferenceMeasurementFeature, ) from openhcs.constants.constants import Backend, MemoryType -from openhcs.core.aligned_image_payload import (AlignedImageSliceContext, AlignedImageStack, ProducedImageStack, ImageOutputBundle, ImagePayloadBundleContext, aligned_image_stack_kwargs, compose_aligned_image_payload, pack_aligned_image_outputs) +from openhcs.core.aligned_image_payload import ( + AlignedImageSliceContext, + AlignedImageStack, + ProducedImageStack, + ImageOutputBundle, + ImagePayloadBundleContext, + aligned_image_stack_kwargs, + compose_aligned_image_payload, + pack_aligned_image_outputs, +) from openhcs.core.image_payload_execution_mode import ImagePayloadExecutionMode from openhcs.core.artifacts import ( ArtifactInputPlan, @@ -66,7 +75,12 @@ DataclassMeasurementColumnarRows, MeasurementSparseColumnarRows, ) -from openhcs.core.pipeline.function_contracts import (object_label_input_execution_mode, object_label_input_execution_mode_from_callable, runtime_bound_parameters, special_inputs) +from openhcs.core.pipeline.function_contracts import ( + object_label_input_execution_mode, + object_label_input_execution_mode_from_callable, + runtime_bound_parameters, + special_inputs, +) from openhcs.core.runtime_artifact_queries import ( MeasurementTableAxisProjection, RuntimeArtifactQueryContext, @@ -467,7 +481,8 @@ def reject_authored_preparation(*args, **kwargs): ) executor = CellProfilerModuleExecutor(raw, contract) image_request = executor._image_request( - image, adapter, + image, + adapter, module_type=executor.module_type(), active_input_specs=contract.artifact_inputs.specs, ) @@ -475,6 +490,7 @@ def reject_authored_preparation(*args, **kwargs): assert image_request.source_aliases == (source_spec.name,) np.testing.assert_array_equal(image_request.payload.data, image) + def test_default_invocation_keeps_compiled_source_bindings_outside_anchor_group() -> ( None ): @@ -1403,9 +1419,7 @@ def test_image_artifact_resolution_uses_declared_artifact_alias( ).payload_with(np.ones((4, 5), dtype=np.float32), None) payload = CellProfilerOutputRecorder.for_artifact_type( ImageArtifactType - ).raw_runtime_input_value( - spec=spec, value=source - ) + ).raw_runtime_input_value(spec=spec, value=source) metadata = payload.metadata assert metadata.source_image_names == ("IllumStain1",) @@ -1465,11 +1479,12 @@ def test_main_flow_image_artifact_selects_declared_alias_plane() -> None: payload = CellProfilerOutputRecorder.for_artifact_type( ImageArtifactType ).raw_runtime_input_value( - first_spec, RuntimeInputBindingRequest( + first_spec, + RuntimeInputBindingRequest( adapter=adapter, kwargs={}, current_image=current_image, - ).artifact_value_for_spec(first_spec) + ).artifact_value_for_spec(first_spec), ) np.testing.assert_array_equal(payload.data, np.zeros((4, 5))) @@ -1505,7 +1520,8 @@ def test_single_main_flow_image_artifact_selects_declared_source_binding_plane() payload = CellProfilerOutputRecorder.for_artifact_type( ImageArtifactType ).raw_runtime_input_value( - spec, RuntimeInputBindingRequest( + spec, + RuntimeInputBindingRequest( adapter=_FakeCellProfilerRuntime( {}, callable_contract=contract, @@ -1518,7 +1534,7 @@ def test_single_main_flow_image_artifact_selects_declared_source_binding_plane() ), kwargs={}, current_image=current_image, - ).artifact_value_for_spec(spec) + ).artifact_value_for_spec(spec), ) np.testing.assert_array_equal(payload.data, np.zeros((4, 5))) @@ -1556,7 +1572,8 @@ def test_main_flow_image_artifact_projects_named_provenance_plane() -> None: payload = CellProfilerOutputRecorder.for_artifact_type( ImageArtifactType ).raw_runtime_input_value( - first_spec, RuntimeInputBindingRequest( + first_spec, + RuntimeInputBindingRequest( adapter=_FakeCellProfilerRuntime( {}, callable_contract=contract, @@ -1569,7 +1586,7 @@ def test_main_flow_image_artifact_projects_named_provenance_plane() -> None: ), kwargs={}, current_image=current_image, - ).artifact_value_for_spec(first_spec) + ).artifact_value_for_spec(first_spec), ) np.testing.assert_array_equal(payload.data, np.zeros((4, 5))) @@ -1601,7 +1618,8 @@ def test_main_flow_image_artifact_preserves_declared_runtime_slice_stack() -> No payload = CellProfilerOutputRecorder.for_artifact_type( ImageArtifactType ).raw_runtime_input_value( - first_spec, RuntimeInputBindingRequest( + first_spec, + RuntimeInputBindingRequest( adapter=_FakeCellProfilerRuntime( {}, callable_contract=contract, @@ -1614,12 +1632,10 @@ def test_main_flow_image_artifact_preserves_declared_runtime_slice_stack() -> No ), kwargs={}, current_image=current_image, - ).artifact_value_for_spec(first_spec) + ).artifact_value_for_spec(first_spec), ) - np.testing.assert_array_equal( - payload.data, current_image.data - ) + np.testing.assert_array_equal(payload.data, current_image.data) assert payload.metadata.source_image_names == ("Stain1",) @@ -1709,7 +1725,9 @@ def record_resolution(self, spec: ArtifactSpec, value): assert runtime_kwargs["illumination_function"].data.shape == (4, 5) -def test_composed_measurement_images_keep_declared_aliases_and_resolved_payload() -> None: +def test_composed_measurement_images_keep_declared_aliases_and_resolved_payload() -> ( + None +): image_spec = ArtifactSpec.input("DeclaredImage", ImageArtifactType) contract = _compiled_callable_contract( CellProfilerModule.require_module("MeasureObjectIntensity").require_callable(), @@ -1750,7 +1768,9 @@ def test_composed_measurement_images_keep_declared_aliases_and_resolved_payload( assert measurement_image.execution_mode is image_request.execution_mode assert measurement_image.plane_projection is image_request.plane_projection assert not measurement_image.align_to_labels - with pytest.raises(ValueError, match="requires a composed measurement image request"): + with pytest.raises( + ValueError, match="requires a composed measurement image request" + ): executor._measurement_image_inputs( runtime, image, @@ -1871,7 +1891,8 @@ def test_single_main_flow_image_preserves_runtime_slice_axis() -> None: payload = CellProfilerOutputRecorder.for_artifact_type( ImageArtifactType ).raw_runtime_input_value( - spec, RuntimeInputBindingRequest( + spec, + RuntimeInputBindingRequest( adapter=_FakeCellProfilerRuntime( {}, callable_contract=contract, @@ -1884,7 +1905,7 @@ def test_single_main_flow_image_preserves_runtime_slice_axis() -> None: ), kwargs={}, current_image=current_image, - ).artifact_value_for_spec(spec) + ).artifact_value_for_spec(spec), ) assert payload.data.shape == (1, 4, 5) @@ -2895,7 +2916,10 @@ def test_watershed_output_source_policy_uses_declared_image_as_parent() -> None: image_payload.data, ) assert source_context.parent_image_payload is source_context.source_payload - assert source_context.source_payload.metadata.source_voxel_spacing == SourceVoxelSpacing((2.0, 1.0, 1.0)) + assert ( + source_context.source_payload.metadata.source_voxel_spacing + == SourceVoxelSpacing((2.0, 1.0, 1.0)) + ) def test_object_label_recorder_suppresses_parent_spacing_when_policy_declares_no_parent_image() -> ( @@ -4991,7 +5015,10 @@ def test_object_row_binding_honors_authoritative_singleton_runtime_projection( assert projection is not None and projection.plane_index is None measurement_values = RuntimeSliceAlignedValues((np.asarray([0.25]),)) projected = RuntimeSliceProjection.kwargs_for_slice( - {"object_labels": (bound_labels,), "measurement_values": measurement_values}, + { + "object_labels": (bound_labels,), + "measurement_values": measurement_values, + }, projection.selected_plane(0), ) assert object_label_dense_array(projected["object_labels"][0]).shape == (2, 2) @@ -6796,20 +6823,27 @@ def test_multi_image_photometry_consumes_each_named_current_source(aliases): ).payload_with(np.stack(tuple(images[name] for name in aliases))) image_specs = tuple(ArtifactSpec.input(name, ImageArtifactType) for name in aliases) contract = _compiled_callable_contract( - MeasureObjectIntensityModule.require_callable(), artifact_inputs=image_specs, + MeasureObjectIntensityModule.require_callable(), + artifact_inputs=image_specs, ) executor = _module_executor(contract) runtime = _FakeCellProfilerRuntime( - {}, callable_contract=contract, + {}, + callable_contract=contract, source_bindings=tuple(NamedSourceBinding(alias=name) for name in aliases), - artifact_input_edges=tuple(_artifact_input_edge_for_test(spec, stored=False) for spec in image_specs), + artifact_input_edges=tuple( + _artifact_input_edge_for_test(spec, stored=False) for spec in image_specs + ), ) measurement_images = executor._resolved_measurement_images( - image_specs, runtime, payload, + image_specs, + runtime, + payload, reference_domain=CellProfilerMeasurementImageDomain.OBJECT_LABELS, ) labels = ObjectLabelSet( - name="Cells", variant_data=ObjectLabelVariantData(labels=np.ones((4, 5), dtype=np.int32)), + name="Cells", + variant_data=ObjectLabelVariantData(labels=np.ones((4, 5), dtype=np.int32)), domain=ObjectLabelDomain(declared_object_count=1), ) for source in measurement_images: @@ -6817,31 +6851,56 @@ def test_multi_image_photometry_consumes_each_named_current_source(aliases): image = prepared.aligned_image.data np.testing.assert_allclose(image, images[source.source_image_name] / 255.0) arrays = object_intensity_backend().measure_prepared( - image, ObjectIntensityPreparedLabels.from_source(prepared.completion_payload, prepared.measurement_labels), + image, + ObjectIntensityPreparedLabels.from_source( + prepared.completion_payload, prepared.measurement_labels + ), ) - expected = {"Process": (221 / 255.0, 1, 2), "Nuclear": (1.0, 2, 3)}[source.source_image_name] + expected = {"Process": (221 / 255.0, 1, 2), "Nuclear": (1.0, 2, 3)}[ + source.source_image_name + ] np.testing.assert_allclose(arrays.max_intensity, (expected[0],)) np.testing.assert_allclose(arrays.max_intensity_y, (expected[1],)) np.testing.assert_allclose(arrays.max_intensity_x, (expected[2],)) -@pytest.mark.parametrize("aliases", (("Process",), ("Process", "Nuclear"), ("Nuclear", "Process"), ("Process", "Nuclear", "Third"))) +@pytest.mark.parametrize( + "aliases", + ( + ("Process",), + ("Process", "Nuclear"), + ("Nuclear", "Process"), + ("Process", "Nuclear", "Third"), + ), +) @pytest.mark.parametrize("slice_count", (1, 2)) -def test_registered_object_photometry_retains_each_source_at_each_slice(aliases, slice_count): +def test_registered_object_photometry_retains_each_source_at_each_slice( + aliases, slice_count +): image_specs = tuple(ArtifactSpec.input(name, ImageArtifactType) for name in aliases) object_spec = ArtifactSpec.input("Cells", ObjectLabelsArtifactType) output_spec = ArtifactSpec.output("Photometry", MeasurementsArtifactType) images = {} for name in aliases: value = {"Process": 100, "Nuclear": 200, "Third": 300}[name] - paths = tuple(f"/synthetic/{name}_z{index + 1}.tif" for index in range(slice_count)) - fields = tuple({"well": "A01", "site": "1", "channel": str(value // 100), - "z_index": str(index + 1), "pixel_size": 0.5} - for index in range(slice_count)) + paths = tuple( + f"/synthetic/{name}_z{index + 1}.tif" for index in range(slice_count) + ) + fields = tuple( + { + "well": "A01", + "site": "1", + "channel": str(value // 100), + "z_index": str(index + 1), + "pixel_size": 0.5, + } + for index in range(slice_count) + ) metadata = ImagePayloadMetadata( source_image_names=(name,), source_image_provenance_planes=SourceImageProvenancePlanes.from_components( - paths=paths, component_metadata=fields), + paths=paths, component_metadata=fields + ), plane_axis=RuntimePlaneAxis.RUNTIME_SLICE if slice_count > 1 else None, ) shape = (slice_count, 4, 5) if slice_count > 1 else (4, 5) @@ -6849,39 +6908,67 @@ def test_registered_object_photometry_retains_each_source_at_each_slice(aliases, label_shape = (slice_count, 4, 5) if slice_count > 1 else (4, 5) labels = ObjectLabelSet( name="Cells", - variant_data=ObjectLabelVariantData(labels=np.ones(label_shape, dtype=np.int32)), + variant_data=ObjectLabelVariantData( + labels=np.ones(label_shape, dtype=np.int32) + ), plane_axis=RuntimePlaneAxis.RUNTIME_SLICE if slice_count > 1 else None, domain=ObjectLabelDomain( - scope=ObjectLabelDomainScope.PLANE if slice_count > 1 else ObjectLabelDomainScope.PAYLOAD, + scope=( + ObjectLabelDomainScope.PLANE + if slice_count > 1 + else ObjectLabelDomainScope.PAYLOAD + ), declared_object_id_domains=((1,),) * slice_count if slice_count > 1 else (), declared_object_count=None if slice_count > 1 else 1, ), ) contract = _compiled_callable_contract( MeasureObjectIntensityModule.require_callable(), - artifact_inputs=(*image_specs, object_spec), artifact_outputs=(output_spec,), + artifact_inputs=(*image_specs, object_spec), + artifact_outputs=(output_spec,), ) runtime = _FakeCellProfilerRuntime( - images, objects={"Cells": labels}, callable_contract=contract, - artifact_input_edges=tuple(_artifact_input_edge_for_test(spec) for spec in (*image_specs, object_spec)), + images, + objects={"Cells": labels}, + callable_contract=contract, + artifact_input_edges=tuple( + _artifact_input_edge_for_test(spec) for spec in (*image_specs, object_spec) + ), artifact_output_bindings=((output_spec, _artifact_output_plan(output_spec)),), plane_projection=RuntimePlaneProjection.stack(slice_count), ) - _run_module(_module_executor(contract), images[aliases[0]], cellprofiler_runtime=runtime) + _run_module( + _module_executor(contract), images[aliases[0]], cellprofiler_runtime=runtime + ) assert len(runtime.measurements) == 1 table = runtime.measurements[0] assert table.subject.object_name == "Cells" assert set(table.source_provenance.represented_source_image_names) == set(aliases) - assert {row["slice_index"] for row in table.rows.iter_row_mappings()} == set(range(slice_count)) + assert {row["slice_index"] for row in table.rows.iter_row_mappings()} == set( + range(slice_count) + ) for name in aliases: provenance = table.source_provenance.for_source_image(name) assert provenance.source_image_provenance_planes.paths == tuple( - f"/synthetic/{name}_z{index + 1}.tif" for index in range(slice_count)) - assert all(fields["pixel_size"] == 0.5 for fields in provenance.source_image_provenance_planes.component_metadata) - assert tuple(fields["z_index"] for fields in provenance.source_image_provenance_planes.component_metadata) == tuple(str(index + 1) for index in range(slice_count)) + f"/synthetic/{name}_z{index + 1}.tif" for index in range(slice_count) + ) + assert all( + fields["pixel_size"] == 0.5 + for fields in provenance.source_image_provenance_planes.component_metadata + ) + assert tuple( + fields["z_index"] + for fields in provenance.source_image_provenance_planes.component_metadata + ) == tuple(str(index + 1) for index in range(slice_count)) column = f"Intensity_MaxIntensity_{name}" - values = tuple(row[column] for row in table.rows.iter_row_mappings() if column in row) - np.testing.assert_allclose(values, ({"Process": 100, "Nuclear": 200, "Third": 300}[name] / 65535,) * slice_count) + values = tuple( + row[column] for row in table.rows.iter_row_mappings() if column in row + ) + np.testing.assert_allclose( + values, + ({"Process": 100, "Nuclear": 200, "Third": 300}[name] / 65535,) + * slice_count, + ) def test_aligned_measurement_provenance_uses_cooperative_source_hook(): @@ -6893,22 +6980,35 @@ def source_provenances(cls, payload): calls.append(payload) return super().source_provenances(payload) - sources = tuple(DeclaredMeasurementSource( - source_image_name=name, - payload=ImagePayloadMetadata(source_path=f"/{name}.tif", source_image_names=(name,)).payload_with(np.ones((2, 3))), - ) for name in ("First", "Independent")) + sources = tuple( + DeclaredMeasurementSource( + source_image_name=name, + payload=ImagePayloadMetadata( + source_path=f"/{name}.tif", source_image_names=(name,) + ).payload_with(np.ones((2, 3))), + ) + for name in ("First", "Independent") + ) metadata = DeclaredMeasurementSource.aligned_source_metadata(sources) assert calls == [source.payload for source in sources] assert metadata.source_provenance.source_plane_count == 1 - assert metadata.source_provenance.represented_source_image_names == ("First", "Independent") - assert metadata.source_provenance.for_source_image("Independent").source_image_provenance_planes.paths == ("/Independent.tif",) + assert metadata.source_provenance.represented_source_image_names == ( + "First", + "Independent", + ) + assert metadata.source_provenance.for_source_image( + "Independent" + ).source_image_provenance_planes.paths == ("/Independent.tif",) def test_aligned_measurement_provenance_rejects_unequal_and_empty_axes(): - sources = tuple(CellProfilerMeasurementImage( - source_image_name="Image", - payload=AlignedImageStack(tuple(np.ones((2, 3)) for _ in range(count))), - ) for count in (1, 2)) + sources = tuple( + CellProfilerMeasurementImage( + source_image_name="Image", + payload=AlignedImageStack(tuple(np.ones((2, 3)) for _ in range(count))), + ) + for count in (1, 2) + ) with pytest.raises(ValueError, match="zip"): CellProfilerMeasurementImage.aligned_source_metadata(sources) with pytest.raises(ValueError, match="requires source images"): @@ -7961,7 +8061,9 @@ def test_aligned_payload_slices_runtime_slice_aligned_values() -> None: first = np.zeros((4, 5), dtype=np.int32) second = np.ones((4, 5), dtype=np.int32) - slices = RuntimeSliceProjection.alignment_slices(RuntimeSliceAlignedValues((first, second))) + slices = RuntimeSliceProjection.alignment_slices( + RuntimeSliceAlignedValues((first, second)) + ) assert slices[0] is first assert slices[1] is second @@ -8087,9 +8189,7 @@ def test_cellprofiler_auxiliary_payload_stack_preserves_metadata() -> None: stacked = RuntimeSliceProjection.full_stack_value(stacked) assert isinstance(stacked, ImageMetadataPayload) assert stacked.data.shape == (2, 1, 4, 5) - assert ( - stacked.metadata.for_source_plane(0).intensity_scale == 65535.0 - ) + assert stacked.metadata.for_source_plane(0).intensity_scale == 65535.0 assert stacked.metadata.for_source_plane(1).source_dtype == "uint8" @@ -8101,14 +8201,16 @@ def test_cellprofiler_image_aggregation_uses_nominal_image_payload_type() -> Non aggregated = RuntimeSliceProjection.full_stack_value(aggregated) assert aggregated.data.shape == (1, 4, 5) - assert aggregated.metadata.plane_axis is ( - RuntimePlaneAxis.RUNTIME_SLICE - ) + assert aggregated.metadata.plane_axis is (RuntimePlaneAxis.RUNTIME_SLICE) @pytest.mark.parametrize("owner", (AlignedImageStack, ImageOutputBundle)) -def test_cellprofiler_aligned_main_output_aggregation_transposes_surfaces(owner) -> None: - contexts = tuple(AlignedImageSliceContext.main_flow(name) for name in ("First", "Second")) +def test_cellprofiler_aligned_main_output_aggregation_transposes_surfaces( + owner, +) -> None: + contexts = tuple( + AlignedImageSliceContext.main_flow(name) for name in ("First", "Second") + ) outputs = ( owner( ( @@ -9076,7 +9178,10 @@ def keep_payload(image: np.ndarray): ObjectLabelPayload( variant_data=ObjectLabelVariantData( labels=labels, - variants={UneditedLabels: labels + 10, SmallRemovedLabels: labels + 20}, + variants={ + UneditedLabels: labels + 10, + SmallRemovedLabels: labels + 20, + }, ), domain=ObjectLabelDomain(declared_object_ids=(object_id,)), ), @@ -9206,9 +9311,7 @@ def segment(image: object): def test_aligned_stack_object_label_auxiliary_aggregates_on_runtime_slice_axis(): def segment(image: object): - site_index = int( - image.metadata.source_component_metadata["site"] - ) + site_index = int(image.metadata.source_component_metadata["site"]) labels = np.full( image.data.shape, site_index, @@ -9502,7 +9605,11 @@ def source_payload(alias: str, channel: int) -> AlignedImageStack: (RuntimePlaneAxis.RUNTIME_SLICE, ImagePayloadMetadataCompositionMode.STACK, 2), (RuntimePlaneAxis.RUNTIME_SLICE, ImagePayloadMetadataCompositionMode.BUNDLE, 1), (RuntimePlaneAxis.SOURCE_BINDING, ImagePayloadMetadataCompositionMode.STACK, 1), - (RuntimePlaneAxis.SOURCE_BINDING, ImagePayloadMetadataCompositionMode.BUNDLE, 2), + ( + RuntimePlaneAxis.SOURCE_BINDING, + ImagePayloadMetadataCompositionMode.BUNDLE, + 2, + ), ), ) def test_image_source_composition_preserves_only_compatible_incoming_topology( @@ -9525,7 +9632,10 @@ def test_image_source_composition_preserves_only_compatible_incoming_topology( if expected_planes == 2: assert planes.paths == paths else: - assert tuple(contributor.path for contributor in planes.plane(0).contributors) == paths + assert ( + tuple(contributor.path for contributor in planes.plane(0).contributors) + == paths + ) def test_measurement_source_composition_reads_dense_plane_facts_without_pixel_projection( @@ -9696,8 +9806,12 @@ def _pattern_group_runtime_for_output_memory( context.microscope_handler = SimpleNamespace(parser=SourceSchemaFilenameParser()) return PatternGroupExecutionRequest( context=context, - compiled_group=compile_function_pattern(lambda image: image, {}, {}).default_group, - pattern_group_info="", component_index=0, component_count=1, + compiled_group=compile_function_pattern( + lambda image: image, {}, {} + ).default_group, + pattern_group_info="", + component_index=0, + component_count=1, execution_plan=CompiledStepPlan( step_index=0, step_name="pattern output", @@ -9722,7 +9836,9 @@ def test_pattern_group_runtime_unstacks_aligned_image_stack_output(): output = runtime._project_output_slices(aligned, matching_files) stack_payload = aligned.copy_projected_output_stack( - output, memory_type="numpy", device_id=None, + output, + memory_type="numpy", + device_id=None, ) assert [payload for payload, _context in output] == [dna_payload, rna_payload] assert stack_payload.metadata.source_plane_intensity_scales == (65535, 65535) @@ -9778,12 +9894,9 @@ def test_pattern_group_runtime_does_not_invent_nested_axes_from_image_rank(): assert len(output_slices) == 2 np.testing.assert_array_equal(output_slices[0].data, first_site.data) - np.testing.assert_array_equal( - output_slices[1].data, second_site.data - ) + np.testing.assert_array_equal(output_slices[1].data, second_site.data) assert [ - payload.metadata.source_image_provenance_planes - for payload in output_slices + payload.metadata.source_image_provenance_planes for payload in output_slices ] == [ first_site.metadata.source_image_provenance_planes, second_site.metadata.source_image_provenance_planes, @@ -9802,15 +9915,22 @@ def test_pattern_group_runtime_leaves_variable_shape_aligned_outputs_uncached(): payload.data is pixels for (payload, _context), pixels in zip(output, (first, second), strict=True) ) - assert aligned.copy_projected_output_stack( - output, memory_type="numpy", device_id=None, - ) is None + assert ( + aligned.copy_projected_output_stack( + output, + memory_type="numpy", + device_id=None, + ) + is None + ) def test_invocation_request_keeps_original_aliases_when_labels_change_image_domain( monkeypatch, ): - contract = _compiled_callable_contract(IdentifyTertiaryObjectsModule.require_callable()) + contract = _compiled_callable_contract( + IdentifyTertiaryObjectsModule.require_callable() + ) executor = _module_executor(contract) larger = ObjectLabelSet( name="Larger", @@ -11247,9 +11367,7 @@ def test_enhance_or_suppress_features_tubeness_matches_hessian_reference(): expected = (-roots[..., 0] * (roots[..., 0] < 0) * (smoothing_value**2)).astype( np.float32 ) - np.testing.assert_allclose( - result.data, expected, rtol=1e-6, atol=1e-7 - ) + np.testing.assert_allclose(result.data, expected, rtol=1e-6, atol=1e-7) def test_cellprofiler_contract_executor_preserves_multi_image_stack_payload(): @@ -11341,9 +11459,7 @@ def test_compose_image_payload_broadcasts_only_from_declared_stack_owner(): stack_broadcast_source_indices=(None, 0), ) - assert composition.execution_mode is ( - AlignedStackExecution - ) + assert composition.execution_mode is (AlignedStackExecution) assert isinstance(composition.payload, AlignedImageStack) assert len(composition.payload.slices) == 2 first_pair = composition.payload.slices[0].data @@ -12019,8 +12135,7 @@ def test_track_objects_record_builder_uses_nominal_image_table_ownership() -> No for feature_name in object_feature_names ) assert all( - rows_by_feature[feature_name]["source_image_name"] - == "image" + rows_by_feature[feature_name]["source_image_name"] == "image" and "object_name" not in rows_by_feature[feature_name] for feature_name in image_feature_names | mean_feature_names ) @@ -12096,12 +12211,18 @@ def test_object_label_output_recorder_uses_output_label_domain() -> None: @pytest.mark.parametrize("carrier_axis", tuple(RuntimePlaneAxis)) -@pytest.mark.parametrize("module_name", ( - "ExpandOrShrinkObjects", "DilateObjects", "ShrinkToObjectCenters", - "ClassifyObjectsSingleMeasurement", -)) +@pytest.mark.parametrize( + "module_name", + ( + "ExpandOrShrinkObjects", + "DilateObjects", + "ShrinkToObjectCenters", + "ClassifyObjectsSingleMeasurement", + ), +) def test_object_only_module_uses_label_planes_instead_of_carrier_axis( - carrier_axis, module_name, + carrier_axis, + module_name, ) -> None: labels = np.zeros((2, 7, 7), dtype=np.int32) labels[0, 2, 2] = 1 @@ -12123,46 +12244,70 @@ def test_object_only_module_uses_label_planes_instead_of_carrier_axis( "InputObjects", ObjectLabelsArtifactType, parameter_name="labels" ) runtime = _FakeCellProfilerRuntime( - {}, objects={"InputObjects": input_payload}, + {}, + objects={"InputObjects": input_payload}, artifact_input_edges=(_artifact_input_edge_for_test(input_spec),), plane_projection=RuntimePlaneProjection.stack(2), ) output_spec = _output_from_input("ExpandedObjects", "InputObjects") classification = module_name == "ClassifyObjectsSingleMeasurement" measurement_spec = ArtifactSpec.output( - f"{module_name}_measurements", MeasurementsArtifactType, - relations=(ArtifactSpecRelation( - source=input_spec.ref() if classification else output_spec.ref() - ),), - ) - executor = _module_executor(_compiled_callable_contract( - CellProfilerModule.require_module(module_name).require_callable(), - artifact_inputs=(input_spec,), - artifact_outputs=(measurement_spec,) if classification else (measurement_spec, output_spec), - )) - runtime.request = replace(runtime.request, artifact_outputs={ - spec.ref(): _artifact_output_plan(spec) - for spec in executor.callable_contract.artifact_outputs - }) + f"{module_name}_measurements", + MeasurementsArtifactType, + relations=( + ArtifactSpecRelation( + source=input_spec.ref() if classification else output_spec.ref() + ), + ), + ) + executor = _module_executor( + _compiled_callable_contract( + CellProfilerModule.require_module(module_name).require_callable(), + artifact_inputs=(input_spec,), + artifact_outputs=( + (measurement_spec,) + if classification + else (measurement_spec, output_spec) + ), + ) + ) + runtime.request = replace( + runtime.request, + artifact_outputs={ + spec.ref(): _artifact_output_plan(spec) + for spec in executor.callable_contract.artifact_outputs + }, + ) # The image only carries main flow; its three planes do not define the # independent two-plane object domain. carrier = ImagePayloadMetadata(plane_axis=carrier_axis).payload_with( np.zeros((3, 7, 7), dtype=np.float32) ) result = _run_module( - executor, carrier, cellprofiler_runtime=runtime, + executor, + carrier, + cellprofiler_runtime=runtime, dtype_config=DtypeConfig(), - **({ - "measurement_values": RuntimeSliceAlignedValues(slices=( - np.array([0.25]), np.array([0.75, 0.75, 0.75, 0.75]), - )), - "bin_count": 2, - } if classification else {}), + **( + { + "measurement_values": RuntimeSliceAlignedValues( + slices=( + np.array([0.25]), + np.array([0.75, 0.75, 0.75, 0.75]), + ) + ), + "bin_count": 2, + } + if classification + else {} + ), ) assert result is carrier if classification: rows = tuple(runtime.measurements[0].rows.row_mappings()) - feature = ClassifyObjectsSingleMeasurementModule.MeasurementFeatureTemplate.OBJECT_CLASS + feature = ( + ClassifyObjectsSingleMeasurementModule.MeasurementFeatureTemplate.OBJECT_CLASS + ) first_bin = feature.feature_name(bin_name="Bin_1") second_bin = feature.feature_name(bin_name="Bin_2") classified = tuple( @@ -12182,8 +12327,8 @@ def test_object_only_module_uses_label_planes_instead_of_carrier_axis( expected = labels.copy() if module_name != "ShrinkToObjectCenters": for plane, coordinate, object_id in ((0, 2, 1), (1, 4, 4)): - expected[plane, coordinate - 1:coordinate + 2, coordinate] = object_id - expected[plane, coordinate, coordinate - 1:coordinate + 2] = object_id + expected[plane, coordinate - 1 : coordinate + 2, coordinate] = object_id + expected[plane, coordinate, coordinate - 1 : coordinate + 2] = object_id np.testing.assert_array_equal(recorded.labels, expected) @@ -12745,13 +12890,17 @@ def test_literal_measurement_stack_preserves_pixels_in_object_reference_domain( ) stack = ProducedImageStack( (metadata.payload_with(pixels, mask),), - memory_type="numpy", plane_axis=RuntimePlaneAxis.RUNTIME_SLICE, + memory_type="numpy", + plane_axis=RuntimePlaneAxis.RUNTIME_SLICE, ) labels = ObjectLabelPayload( - variant_data=ObjectLabelVariantData(labels=np.ones(pixels.shape, dtype=np.int32)), + variant_data=ObjectLabelVariantData( + labels=np.ones(pixels.shape, dtype=np.int32) + ), ) carrier = CellProfilerMeasurementImage( - source_image_name="Source", payload=stack, + source_image_name="Source", + payload=stack, reference_domain=CellProfilerMeasurementImageDomain.OBJECT_LABELS, execution_mode=execution_mode, ) @@ -12919,16 +13068,23 @@ def test_filterobjects_binds_selection_measurement_values_to_label_slices( assert filtered_array[1, 3, 3] == 0 -@pytest.mark.parametrize("carrier_axis, binary_operand", ( - (RuntimePlaneAxis.SOURCE_BINDING, False), - (RuntimePlaneAxis.RUNTIME_SLICE, False), - (RuntimePlaneAxis.RUNTIME_SLICE, True), -)) -def test_area_occupied_rows_preserve_the_selected_domain(carrier_axis, binary_operand) -> None: +@pytest.mark.parametrize( + "carrier_axis, binary_operand", + ( + (RuntimePlaneAxis.SOURCE_BINDING, False), + (RuntimePlaneAxis.RUNTIME_SLICE, False), + (RuntimePlaneAxis.RUNTIME_SLICE, True), + ), +) +def test_area_occupied_rows_preserve_the_selected_domain( + carrier_axis, binary_operand +) -> None: from openhcs.processing.backends.cellprofiler.area_occupied import OperandChoice object_specs = tuple( - ArtifactSpec.input(name, ObjectLabelsArtifactType, parameter_name="object_labels") + ArtifactSpec.input( + name, ObjectLabelsArtifactType, parameter_name="object_labels" + ) for name in ("Primary", "Other") ) objects = {} @@ -12946,15 +13102,19 @@ def test_area_occupied_rows_preserve_the_selected_domain(carrier_axis, binary_op plane_axis=RuntimePlaneAxis.RUNTIME_SLICE, ) measurement_spec = ArtifactSpec.output( - "AreaMeasurements", MeasurementsArtifactType, + "AreaMeasurements", + MeasurementsArtifactType, measurement_feature_owner=MeasureImageAreaOccupiedBinaryModule, - relations=tuple(ArtifactSpecRelation(source=spec.ref()) for spec in object_specs), + relations=tuple( + ArtifactSpecRelation(source=spec.ref()) for spec in object_specs + ), ) binary_spec = ArtifactSpec.input("Binary", ImageArtifactType) input_specs = ((binary_spec,) if binary_operand else ()) + object_specs contract = _compiled_callable_contract( MeasureImageAreaOccupiedBinaryModule.require_callable(), - artifact_inputs=input_specs, artifact_outputs=(measurement_spec,), + artifact_inputs=input_specs, + artifact_outputs=(measurement_spec,), ) assert contract.invocation_domain_inputs.specs == ( (binary_spec,) if binary_operand else object_specs @@ -12968,15 +13128,23 @@ def test_area_occupied_rows_preserve_the_selected_domain(carrier_axis, binary_op source_image_names=("Binary",) if binary_operand else (), ).payload_with(carrier_pixels) runtime = _FakeCellProfilerRuntime( - {"Binary": carrier} if binary_operand else {}, objects=objects, - artifact_input_edges=tuple(_artifact_input_edge_for_test(spec) for spec in input_specs), + {"Binary": carrier} if binary_operand else {}, + objects=objects, + artifact_input_edges=tuple( + _artifact_input_edge_for_test(spec) for spec in input_specs + ), plane_projection=RuntimePlaneProjection.stack(2), ) - runtime.request = replace(runtime.request, artifact_outputs={ - measurement_spec.ref(): _artifact_output_plan(measurement_spec), - }) + runtime.request = replace( + runtime.request, + artifact_outputs={ + measurement_spec.ref(): _artifact_output_plan(measurement_spec), + }, + ) result = _run_module( - _module_executor(contract), carrier, cellprofiler_runtime=runtime, + _module_executor(contract), + carrier, + cellprofiler_runtime=runtime, operand_choices=((OperandChoice.BINARY_IMAGE,) if binary_operand else ()) + (OperandChoice.OBJECTS, OperandChoice.OBJECTS), dtype_config=DtypeConfig(), @@ -12984,15 +13152,26 @@ def test_area_occupied_rows_preserve_the_selected_domain(carrier_axis, binary_op assert result is carrier rows = tuple(runtime.measurements[0].rows.row_mappings()) measured = tuple( - (row["slice_index"], row["source_image_name"], - row[f"AreaOccupied_AreaOccupied_{row['source_image_name']}"]) + ( + row["slice_index"], + row["source_image_name"], + row[f"AreaOccupied_AreaOccupied_{row['source_image_name']}"], + ) for row in rows ) - expected = ((0, "Primary", 1.0), (0, "Other", 2.0), - (1, "Primary", 4.0), (1, "Other", 3.0)) + expected = ( + (0, "Primary", 1.0), + (0, "Other", 2.0), + (1, "Primary", 4.0), + (1, "Other", 3.0), + ) if binary_operand: - expected = ((0, "Binary", 3.0), *expected[:2], - (1, "Binary", 6.0), *expected[2:]) + expected = ( + (0, "Binary", 3.0), + *expected[:2], + (1, "Binary", 6.0), + *expected[2:], + ) assert measured == expected @@ -14229,7 +14408,10 @@ def test_relateobjects_parent_means_align_scoped_child_tables_by_source_plane( changed_rows = MeasurementSparseColumnarRows( columns={ **first.rows.columns, - colocalization_feature: (123.0, *first.rows.column_values(colocalization_feature)[1:]), + colocalization_feature: ( + 123.0, + *first.rows.column_values(colocalization_feature)[1:], + ), }, fields=first.rows.fields, ) @@ -14344,7 +14526,8 @@ def test_relateobjects_parent_means_align_scoped_child_tables_by_source_plane( if conflicting_upstream: with pytest.raises(ValueError, match="conflicting values"): projector.parent_mean_upstream_measurement_rows( - parent_spec=parent_spec, child_spec=child_spec, + parent_spec=parent_spec, + child_spec=child_spec, payload=runtime.relationships[0], ) return @@ -14618,9 +14801,10 @@ def test_spatial_grid_output_recorder_accepts_pure_2d_grid_sequence() -> None: recorded = runtime.spatial_grids["Grid"] assert isinstance(recorded, RuntimeSliceAlignedValues) - assert [ - recorded.value_at(slice_index).x_origin for slice_index in range(2) - ] == [1.0, 2.0] + assert [recorded.value_at(slice_index).x_origin for slice_index in range(2)] == [ + 1.0, + 2.0, + ] def test_identify_objects_in_grid_respects_row_primary_ordering() -> None: @@ -16963,11 +17147,12 @@ def test_object_label_payload_preserves_source_metadata_for_measurements() -> No payload = CellProfilerOutputRecorder.for_artifact_type( spec.artifact_type ).raw_runtime_input_value( - spec, RuntimeInputBindingRequest( + spec, + RuntimeInputBindingRequest( adapter=runtime, kwargs={}, current_image=image, - ).artifact_value_for_spec(spec) + ).artifact_value_for_spec(spec), ) np.testing.assert_array_equal(payload.labels, labels.labels) @@ -17021,11 +17206,12 @@ def test_runtime_object_label_payload_ignores_measurement_image_as_selector() -> payload = CellProfilerOutputRecorder.for_artifact_type( spec.artifact_type ).raw_runtime_input_value( - spec, RuntimeInputBindingRequest( + spec, + RuntimeInputBindingRequest( adapter=runtime, kwargs={}, current_image=measurement_image, - ).artifact_value_for_spec(spec) + ).artifact_value_for_spec(spec), ) np.testing.assert_array_equal(payload.labels, labels.labels) @@ -17072,11 +17258,12 @@ def test_full_stack_object_measurement_resolves_complete_label_artifact() -> Non label_payload = CellProfilerOutputRecorder.for_artifact_type( object_spec.artifact_type ).raw_runtime_input_value( - object_spec, RuntimeInputBindingRequest( + object_spec, + RuntimeInputBindingRequest( adapter=runtime, kwargs={}, current_image=image, - ).artifact_value_for_spec(object_spec) + ).artifact_value_for_spec(object_spec), ) _, executable_labels, _, _, _, _, _ = object_measurement_runtime_inputs( object_label_execution=object_label_input_execution_mode_from_callable( @@ -17152,11 +17339,12 @@ def test_object_label_payload_for_measurement_image_projects_source_spatial_crop label_payload = CellProfilerOutputRecorder.for_artifact_type( spec.artifact_type ).raw_runtime_input_value( - spec, RuntimeInputBindingRequest( + spec, + RuntimeInputBindingRequest( adapter=runtime, kwargs={}, current_image=image, - ).artifact_value_for_spec(spec) + ).artifact_value_for_spec(spec), ) payload = ( CellProfilerMeasurementImage( @@ -18242,7 +18430,9 @@ def test_convert_objects_to_image_uses_declared_label_source_for_runtime_plane_d variable_components=(Microscopy.ZIndex,), relations=(GroupLineageSourceRelation(source_spec.ref()),), ) - result = (ImageArtifactType if output_plan is None else output_plan.artifact_type).contextualize_output_from_projector( + result = ( + ImageArtifactType if output_plan is None else output_plan.artifact_type + ).contextualize_output_from_projector( label_payload, raw_result, output_plan, @@ -18278,7 +18468,9 @@ def test_object_label_image_output_rejects_source_plane_count_drift() -> None: ValueError, match="source-plane provenance must match the declared runtime plane axis", ): - (ImageArtifactType if output_plan is None else output_plan.artifact_type).contextualize_output_from_projector( + ( + ImageArtifactType if output_plan is None else output_plan.artifact_type + ).contextualize_output_from_projector( label_payload, ImagePayloadMetadata().payload_with( np.zeros(labels.shape, dtype=np.uint16) @@ -18309,7 +18501,9 @@ def test_object_label_singleton_volume_output_declares_runtime_plane_axis() -> N relations=(GroupLineageSourceRelation(source_spec.ref()),), ) - result = (ImageArtifactType if output_plan is None else output_plan.artifact_type).contextualize_output_from_projector( + result = ( + ImageArtifactType if output_plan is None else output_plan.artifact_type + ).contextualize_output_from_projector( label_payload, raw_result, output_plan, @@ -18341,7 +18535,9 @@ def test_object_label_scalar_image_output_does_not_invent_runtime_plane_axis() - relations=(GroupLineageSourceRelation(source_spec.ref()),), ) - result = (ImageArtifactType if output_plan is None else output_plan.artifact_type).contextualize_output_from_projector( + result = ( + ImageArtifactType if output_plan is None else output_plan.artifact_type + ).contextualize_output_from_projector( label_payload, raw_result, output_plan, @@ -18595,7 +18791,9 @@ def test_measure_object_size_shape_surface_areas_match_marching_cubes_oracle() - expected.append(_surface_area(labels[bounds] == label_id)) features, measured_labels = measure_object_size_shape_feature_arrays( - labels, calculate_advanced=False, calculate_zernikes=False, + labels, + calculate_advanced=False, + calculate_zernikes=False, ) np.testing.assert_array_equal(measured_labels, label_ids) np.testing.assert_allclose( @@ -18670,8 +18868,7 @@ def test_natural_object_measurement_resolves_one_roster_without_eager_compositio images, {"Cells": labels}, artifact_input_edges=tuple( - _artifact_input_edge_for_test(spec) - for spec in (*image_specs, object_spec) + _artifact_input_edge_for_test(spec) for spec in (*image_specs, object_spec) ), artifact_output_bindings=( (measurement_spec, _artifact_output_plan(measurement_spec)), @@ -18717,7 +18914,8 @@ def table_name(self, measurement_images, source_image_name): assert resolutions == ["DNA", "ER"][:image_count] assert fallbacks == [None if image_count == 1 else "dna__er"] table = next( - table for table in runtime.measurements + table + for table in runtime.measurements if table.subject == MeasurementSubject(MeasurementScope.OBJECT, "Cells") ) rows = table.rows.row_mappings() @@ -18732,11 +18930,14 @@ def test_natural_measurement_fallback_preserves_selected_main_flow_plane(monkeyp image_spec = ArtifactSpec.input("DNA", ImageArtifactType) object_spec = ArtifactSpec.input("Cells", ObjectLabelsArtifactType) - output_spec = ArtifactSpec.output("SelectedPlaneIntensity", MeasurementsArtifactType) + output_spec = ArtifactSpec.output( + "SelectedPlaneIntensity", MeasurementsArtifactType + ) planes = tuple( ImagePayloadMetadata(source_image_names=(name,)).payload_with( np.ones((3, 4), dtype=np.float32) - ) for name in ("DNA", "ER") + ) + for name in ("DNA", "ER") ) image = ImagePayloadMetadata.compose( planes, mode=ImagePayloadMetadataCompositionMode.STACK @@ -18751,7 +18952,8 @@ def test_natural_measurement_fallback_preserves_selected_main_flow_plane(monkeyp ), ) runtime = _FakeCellProfilerRuntime( - {}, {"Cells": labels}, + {}, + {"Cells": labels}, artifact_input_edges=( replace( _artifact_input_edge_for_test(image_spec, stored=False), @@ -18762,26 +18964,36 @@ def test_natural_measurement_fallback_preserves_selected_main_flow_plane(monkeyp artifact_output_bindings=((output_spec, _artifact_output_plan(output_spec)),), plane_projection=RuntimePlaneProjection(plane_index=0, plane_count=2), ) - executor = _module_executor(_compiled_callable_contract( - MeasureObjectIntensityModule.require_callable(), - artifact_inputs=(image_spec, object_spec), artifact_outputs=(output_spec,), - )) + executor = _module_executor( + _compiled_callable_contract( + MeasureObjectIntensityModule.require_callable(), + artifact_inputs=(image_spec, object_spec), + artifact_outputs=(output_spec,), + ) + ) _activate_runtime_contract(executor.callable_contract, runtime) original_fallback = executor._image_request( - image, runtime, module_type=MeasureObjectIntensityModule, + image, + runtime, + module_type=MeasureObjectIntensityModule, active_input_specs=(image_spec, object_spec), ).source_image_name assert original_fallback == "DNA" monkeypatch.setattr( - CellProfilerModuleExecutor, "_image_request", + CellProfilerModuleExecutor, + "_image_request", lambda *_args, **_kwargs: pytest.fail("unused composed request"), ) - policy_type = type(MeasureObjectIntensityModule.runtime_object_measurement_row_policy()) + policy_type = type( + MeasureObjectIntensityModule.runtime_object_measurement_row_policy() + ) original_table_name = policy_type.table_source_image_name fallbacks = [] + def table_name(self, measurement_images, source_image_name): fallbacks.append(source_image_name) return original_table_name(self, measurement_images, source_image_name) + monkeypatch.setattr(policy_type, "table_source_image_name", table_name) assert _run_module(executor, image, cellprofiler_runtime=runtime) is image assert fallbacks == [original_fallback] @@ -18838,11 +19050,11 @@ def execute_batch(func, requests, execute_one): contract = replace( contract, - runtime_batch_executors={ - RuntimeBatchExecutionDomain.MEASUREMENT_IMAGES: execute_batch - } - if batched - else {}, + runtime_batch_executors=( + {RuntimeBatchExecutionDomain.MEASUREMENT_IMAGES: execute_batch} + if batched + else {} + ), ) runtime = _FakeCellProfilerRuntime( images, diff --git a/tests/unit/test_cellprofiler_module_execution_semantics.py b/tests/unit/test_cellprofiler_module_execution_semantics.py index 967be27b01..1c869df72e 100644 --- a/tests/unit/test_cellprofiler_module_execution_semantics.py +++ b/tests/unit/test_cellprofiler_module_execution_semantics.py @@ -40,7 +40,9 @@ ) from openhcs.core.runtime_plane_projection import RuntimePlaneAxis from openhcs.core.runtime_stores import RuntimeValueStore -from openhcs.core.pipeline.function_contracts import object_label_input_execution_mode_from_callable +from openhcs.core.pipeline.function_contracts import ( + object_label_input_execution_mode_from_callable, +) from openhcs.interop.cellprofiler.runtime.module_execution import ( CellProfilerModuleExecutor, ) @@ -246,8 +248,7 @@ def test_composed_image_measurement_callable_retains_standard_execution() -> Non raw_func = MeasureColocalizationModule.require_callable() assert ( - object_label_input_execution_mode_from_callable(raw_func) - is SliceAlignedLabels + object_label_input_execution_mode_from_callable(raw_func) is SliceAlignedLabels ) assert not MeasureColocalizationModule.executes_per_image_measurements( raw_func, diff --git a/tests/unit/test_cellprofiler_morphology_public_contracts.py b/tests/unit/test_cellprofiler_morphology_public_contracts.py index 734f0c6ac4..6b74e6a6b8 100644 --- a/tests/unit/test_cellprofiler_morphology_public_contracts.py +++ b/tests/unit/test_cellprofiler_morphology_public_contracts.py @@ -266,7 +266,10 @@ def test_registry_morphology_owners_compile_exact_public_function_step_abi( assert tuple(spec.artifact_type for spec in contract.artifact_outputs) == ( case.output_types ) - assert CallableContract.from_callable(case.function).artifact_input_parameter_names == case.special_inputs + assert ( + CallableContract.from_callable(case.function).artifact_input_parameter_names + == case.special_inputs + ) def test_fill_objects_returns_its_declared_object_label_artifact() -> None: diff --git a/tests/unit/test_cellprofiler_output_record_occurrence.py b/tests/unit/test_cellprofiler_output_record_occurrence.py index b63cec28aa..5bb5d55d27 100644 --- a/tests/unit/test_cellprofiler_output_record_occurrence.py +++ b/tests/unit/test_cellprofiler_output_record_occurrence.py @@ -269,13 +269,18 @@ def _image_record_request() -> tuple[CellProfilerOutputRecordRequest, ArtifactSp spec = ArtifactSpec.input("Original", ImageArtifactType) request, (edge,) = _output_record_request((spec,), ((0, spec),)) edge = replace(edge, main_flow_projection=MainFlowInputProjection.COMPLETE_PAYLOAD) - adapter = SimpleNamespace(request=replace( - request.adapter.request, artifact_inputs={edge.key: edge} - )) - return replace( - request, adapter=adapter, active_input_edges=(edge,), - current_image=np.zeros((3, 4), dtype=np.float32), - ), spec + adapter = SimpleNamespace( + request=replace(request.adapter.request, artifact_inputs={edge.key: edge}) + ) + return ( + replace( + request, + adapter=adapter, + active_input_edges=(edge,), + current_image=np.zeros((3, 4), dtype=np.float32), + ), + spec, + ) def test_record_binding_keeps_output_admission_before_live_input_validation( @@ -287,7 +292,9 @@ def test_record_binding_keeps_output_admission_before_live_input_validation( def fail_selection(_self): raise error - monkeypatch.setattr(RuntimeAdapterRequest, "selected_artifact_input_specs", fail_selection) + monkeypatch.setattr( + RuntimeAdapterRequest, "selected_artifact_input_specs", fail_selection + ) # Output-only recording does not observe input declaration callbacks. assert replace(request).output_value is request.output_value with pytest.raises(ValueError, match="does not match active output"): @@ -299,17 +306,23 @@ def fail_selection(_self): assert caught.value is error -def test_record_binding_reads_current_pixels_and_mutable_kwargs_without_holder_aliases() -> None: +def test_record_binding_reads_current_pixels_and_mutable_kwargs_without_holder_aliases() -> ( + None +): request, spec = _image_record_request() assert isinstance(request, RuntimeInputBindingRequest) assert "call_kwargs" not in {item.name for item in fields(request)} assert not hasattr(request, "__dict__") with pytest.raises(TypeError): CellProfilerOutputRecordRequest( - request.callable_contract, request.active_input_edges, - adapter=request.adapter, kwargs=request.kwargs, - current_image=request.current_image, spec=request.spec, - output_plan=request.output_plan, output_value=request.output_value, + request.callable_contract, + request.active_input_edges, + adapter=request.adapter, + kwargs=request.kwargs, + current_image=request.current_image, + spec=request.spec, + output_plan=request.output_plan, + output_value=request.output_value, source=request.source, ) with pytest.raises(FrozenInstanceError): @@ -350,9 +363,7 @@ def observe_bind(strategy, input_spec, value): binding_inputs.append(value) return original_bind(strategy, input_spec, value) - monkeypatch.setattr( - ImageOutputRecorder, "raw_runtime_input_value", observe_bind - ) + monkeypatch.setattr(ImageOutputRecorder, "raw_runtime_input_value", observe_bind) for pixel_value in (65535, 32768): pixels[:] = pixel_value binding_inputs.clear() @@ -360,18 +371,22 @@ def observe_bind(strategy, input_spec, value): assert binding_inputs == [payload] assert source.data.dtype == np.float32 np.testing.assert_allclose( - source.data, np.float32(pixel_value) / 65535, + source.data, + np.float32(pixel_value) / 65535, ) assert source.mask is mask metadata = source.metadata assert metadata.source_image_names == (spec.name,) assert metadata.source_image_provenance_planes.paths == ("/input/actual.tif",) assert set(metadata.source_provenance.represented_source_image_names) == { - "Acquired", spec.name, + "Acquired", + spec.name, } -def test_record_source_returns_the_actual_bound_value_without_rebinding(monkeypatch) -> None: +def test_record_source_returns_the_actual_bound_value_without_rebinding( + monkeypatch, +) -> None: request, _spec = _image_record_request() bound = ImagePayloadMetadata( source_image_names=("Original",), @@ -394,7 +409,9 @@ def reject_rebinding(*_args, **_kwargs): assert reads == [(edge, edge.spec.parameter_name)] -def test_record_endpoint_identity_does_not_change_reference_broadcast_selection() -> None: +def test_record_endpoint_identity_does_not_change_reference_broadcast_selection() -> ( + None +): request, spec = _image_record_request() equivalent = replace(spec) with pytest.raises(RuntimeError, match="has no exact compiled input edge"): @@ -425,7 +442,8 @@ def observe_selection(adapter_request): request.kwargs["epoch"] = epoch events.clear() original = RuntimeInputBindingRequest( - adapter=request.adapter, kwargs=request.kwargs, + adapter=request.adapter, + kwargs=request.kwargs, current_image=request.current_image, ).runtime_value(request.active_input_edges[0]) expected_events = tuple(events) @@ -439,7 +457,9 @@ def observe_selection(adapter_request): # in the map cannot excuse a live selector removing its declaration. edge = request.active_input_edges[0] output = ArtifactSpec.output_preserving_source_stack_scope( - request.spec.name, request.spec.artifact_type, spec, + request.spec.name, + request.spec.artifact_type, + spec, ) consumed = CellProfilerImageRequest( source_image_name=spec.name, @@ -452,12 +472,14 @@ def observe_selection(adapter_request): primary_input_edges=(edge,), ) request = replace( - request, spec=output, + request, + spec=output, output_plan=replace(request.output_plan, relations=output.relations), source=consumed, ) monkeypatch.setattr( - RuntimeAdapterRequest, "selected_artifact_input_specs", + RuntimeAdapterRequest, + "selected_artifact_input_specs", lambda _adapter_request: ArtifactSpecCollection(()), ) with pytest.raises(ValueError, match="does not declare artifact input"): @@ -469,7 +491,9 @@ def test_record_explicit_object_subset_preserves_binding_admission() -> None: second = ArtifactSpec.input("Second", ObjectLabelsArtifactType) request, _edges = _output_record_request((first, second), ((0, first), (1, second))) standalone = RuntimeInputBindingRequest( - adapter=request.adapter, kwargs=request.kwargs, current_image=request.current_image, + adapter=request.adapter, + kwargs=request.kwargs, + current_image=request.current_image, ) for subset in ((first,), (replace(second),), ()): actual = request.with_object_inputs(subset) @@ -495,11 +519,15 @@ def test_record_subset_validation_runs_after_output_admission_and_preserves_erro def fail_selection(_self): raise error - monkeypatch.setattr(RuntimeAdapterRequest, "selected_artifact_input_specs", fail_selection) + monkeypatch.setattr( + RuntimeAdapterRequest, "selected_artifact_input_specs", fail_selection + ) for change in ( lambda: RuntimeInputBindingRequest( - adapter=request.adapter, kwargs=request.kwargs, - current_image=request.current_image, selected_object_inputs=(), + adapter=request.adapter, + kwargs=request.kwargs, + current_image=request.current_image, + selected_object_inputs=(), ), lambda: request.with_object_inputs(()), ): @@ -508,7 +536,8 @@ def fail_selection(_self): assert caught.value is error with pytest.raises(ValueError, match="does not match active output"): replace( - request, selected_object_inputs=(), + request, + selected_object_inputs=(), output_plan=replace(request.output_plan, name="Wrong"), ) # The ordinary post-call record still observes no input selector at birth. diff --git a/tests/unit/test_cellprofiler_prepared_payload_abi.py b/tests/unit/test_cellprofiler_prepared_payload_abi.py index 43e4b77799..5df50b3e40 100644 --- a/tests/unit/test_cellprofiler_prepared_payload_abi.py +++ b/tests/unit/test_cellprofiler_prepared_payload_abi.py @@ -7,14 +7,19 @@ from openhcs.core.function_patterns import NormalizedFunctionGroup from openhcs.core.runtime_image_values import ImagePayloadMetadata -from openhcs.core.runtime_object_label_building import SourceImageObjectLabelBuildRequest +from openhcs.core.runtime_object_label_building import ( + SourceImageObjectLabelBuildRequest, +) from openhcs.core.runtime_object_labels import object_label_dense_array from openhcs.core.source_spatial_domain import SourceSpatialDomain from openhcs.interop.cellprofiler.runtime.adapter import CellProfilerRuntimeAdapter from openhcs.interop.cellprofiler.runtime.function_contract_execution import ( CellProfilerFunctionContractExecutor, ) -from openhcs.processing.backends.cellprofiler.color import ColorToGrayMode, color_to_gray +from openhcs.processing.backends.cellprofiler.color import ( + ColorToGrayMode, + color_to_gray, +) from openhcs.processing.backends.cellprofiler.median_filter import medianfilter from openhcs.processing.backends.cellprofiler.secondary import ( SecondaryMethod, @@ -62,12 +67,18 @@ def test_prepared_color_to_gray_preserves_channel_mask_and_spatial_domain(): ).payload_with(pixels, mask) result = _execute_prepared( - color_to_gray, source, - {"mode": ColorToGrayMode.COMBINE, - "channel_indices": (0, 2), "contributions": (1.0, 3.0)}, + color_to_gray, + source, + { + "mode": ColorToGrayMode.COMBINE, + "channel_indices": (0, 2), + "contributions": (1.0, 3.0), + }, ) - np.testing.assert_array_equal(result.data, (pixels[..., 0] + 3 * pixels[..., 2]) / 4) + np.testing.assert_array_equal( + result.data, (pixels[..., 0] + 3 * pixels[..., 2]) / 4 + ) np.testing.assert_array_equal(result.mask, mask) metadata = result.metadata assert metadata.axis_position(ColourAxis) is None @@ -79,10 +90,19 @@ def test_prepared_medianfilter_keeps_original_intensity_scale_and_mask(): pixels = np.arange(25, dtype=np.uint16).reshape(5, 5) mask = np.ones(pixels.shape, dtype=bool) mask[0] = False - source = ImagePayloadMetadata.for_array(pixels).payload_with(pixels, mask).normalize_intensity_payload(dtype=np.float32,) + source = ( + ImagePayloadMetadata.for_array(pixels) + .payload_with(pixels, mask) + .normalize_intensity_payload( + dtype=np.float32, + ) + ) result = _execute_prepared( - medianfilter, source, {"window_size": 3}, FullStackExecution, + medianfilter, + source, + {"window_size": 3}, + FullStackExecution, ) np.testing.assert_array_equal( @@ -98,24 +118,34 @@ def test_prepared_secondary_object_output_keeps_actual_source_spatial_domain(): pixels = np.zeros((9, 9), dtype=np.float32) domain = SourceSpatialDomain(origin_yx=(3, 4), source_shape_yx=(15, 16)) source = ImagePayloadMetadata( - source_image_names=("DNA",), source_path="/input/dna.tif", + source_image_names=("DNA",), + source_path="/input/dna.tif", source_spatial_domain=domain, ).payload_with(pixels, None) seed_pixels = np.zeros(pixels.shape, dtype=np.int32) seed_pixels[4, 4] = 1 seeds = SourceImageObjectLabelBuildRequest( - image=source, labels=seed_pixels, declared_object_ids=(1,), + image=source, + labels=seed_pixels, + declared_object_ids=(1,), ).payload() result = _execute_prepared( - identify_secondary_objects, source, - {"primary_labels": seeds, "method": SecondaryMethod.DISTANCE_N, - "distance_to_dilate": 1, "fill_holes": False}, + identify_secondary_objects, + source, + { + "primary_labels": seeds, + "method": SecondaryMethod.DISTANCE_N, + "distance_to_dilate": 1, + "fill_holes": False, + }, ) _image, _rows, relationship, objects = result expected = ndimage.distance_transform_edt(seed_pixels == 0) <= 1 - np.testing.assert_array_equal(object_label_dense_array(objects), expected.astype(np.int32)) + np.testing.assert_array_equal( + object_label_dense_array(objects), expected.astype(np.int32) + ) assert objects.source_spatial_domain.origin_yx == domain.origin_yx assert objects.source_spatial_domain.source_shape_yx == domain.source_shape_yx assert objects.source_provenance == source.metadata.source_provenance diff --git a/tests/unit/test_cellprofiler_runtime_adapter.py b/tests/unit/test_cellprofiler_runtime_adapter.py index 3d43c9b698..81ac50f779 100644 --- a/tests/unit/test_cellprofiler_runtime_adapter.py +++ b/tests/unit/test_cellprofiler_runtime_adapter.py @@ -49,7 +49,12 @@ MeasurementSparseColumnarRows, measurement_rows, ) -from openhcs.core.pipeline.function_contracts import (composed_image_payload, object_label_input_execution_mode, runtime_bound_parameters, special_inputs) +from openhcs.core.pipeline.function_contracts import ( + composed_image_payload, + object_label_input_execution_mode, + runtime_bound_parameters, + special_inputs, +) from openhcs.core.runtime_artifact_queries import ( MeasurementLabelSliceFeatureQuery, MeasurementTableAxisProjection, @@ -382,8 +387,6 @@ def runtime_axis_scope( ) - - @dataclass(frozen=True) class StaticFilenameParser: metadata: Mapping[str, object] @@ -695,44 +698,52 @@ def _pipeline_start_contains_binding(alias): def _source_bound_image_adapter(output_bindings, images): from polystore.virtual_workspace import SourcePixelRef from openhcs.core.source_projection import ( - OpenHCSPlaneAddress, SourcePlaneProjection, SourceProjectionSet, + OpenHCSPlaneAddress, + SourcePlaneProjection, + SourceProjectionSet, + ) + from openhcs.core.source_workspace_projection import ( + VirtualWorkspaceSourceProjection, ) - from openhcs.core.source_workspace_projection import VirtualWorkspaceSourceProjection from openhcs.core.dataset_sources.source_schema import SourceSchemaFilenameParser filemanager = FileManagerStub() for alias, image in images.items(): filemanager.saved[("memory", f"/src/{alias}.tif")] = image context = ContextStub(filemanager) - projections = SourceProjectionSet(tuple( - SourcePlaneProjection( - address=OpenHCSPlaneAddress( - ( - (Microscopy.Well, 'A01'), - (Microscopy.Site, '1'), - (Microscopy.Channel, str(index)), - (Microscopy.ZIndex, '1'), - (Microscopy.Timepoint, '1'), - ) - ), - ref=SourcePixelRef('memory', f'/src/{alias}.tif'), - source_alias=alias, + projections = SourceProjectionSet( + tuple( + SourcePlaneProjection( + address=OpenHCSPlaneAddress( + ( + (Microscopy.Well, "A01"), + (Microscopy.Site, "1"), + (Microscopy.Channel, str(index)), + (Microscopy.ZIndex, "1"), + (Microscopy.Timepoint, "1"), + ) + ), + ref=SourcePixelRef("memory", f"/src/{alias}.tif"), + source_alias=alias, + ) + for index, alias in enumerate(images, start=1) ) - for index, alias in enumerate(images, start=1) - )) + ) subdirectory = projections.metadata_dict( parser=SourceSchemaFilenameParser(), - microscope_handler_name='SourceBindingsSource', - source_filename_parser_name='SourceSchemaFilenameParser', - grid_dimensions=[1, 1], pixel_size=1., + microscope_handler_name="SourceBindingsSource", + source_filename_parser_name="SourceSchemaFilenameParser", + grid_dimensions=[1, 1], + pixel_size=1.0, ) - document = {'subdirectories': {'.': subdirectory}} + document = {"subdirectories": {".": subdirectory}} context.microscope_handler.metadata_handler = SimpleNamespace( source_workspace_root=lambda _plate: Path(context.input_dir), source_workspace_metadata_document=lambda _plate: document, ) workspace = VirtualWorkspaceSourceProjection.from_openhcs_metadata( - Path(context.input_dir), document, + Path(context.input_dir), + document, ) for path, projection in workspace.source_projections_by_virtual_path.items(): pixels = images[projection.source_alias] @@ -740,11 +751,14 @@ def _source_bound_image_adapter(output_bindings, images): source_path=projection.ref.backend_address, source_image_names=(projection.source_alias,), source_spatial_domain=SourceSpatialDomain( - origin_yx=(0, 0), source_shape_yx=pixels.shape[-2:], + origin_yx=(0, 0), + source_shape_yx=pixels.shape[-2:], ), ).payload_with(pixels) - filemanager.saved[('virtual_workspace', path)] = payload - filemanager.saved[('virtual_workspace', str(Path(context.input_dir) / path))] = payload + filemanager.saved[("virtual_workspace", path)] = payload + filemanager.saved[ + ("virtual_workspace", str(Path(context.input_dir) / path)) + ] = payload return cellprofiler_runtime_adapter_for_test( runtime_value_store=RuntimeValueStore(), axis_scope=runtime_axis_scope(AXIS_ID), @@ -1247,9 +1261,7 @@ def test_cellprofiler_adapter_rejects_unaddressable_source_aligned_label_stack() def test_cellprofiler_adapter_does_not_cache_current_image_object_selection(): store = RuntimeValueStore() outputs = { - NUCLEI: _plan( - NUCLEI, ObjectLabelsArtifactType, group_component=Microscopy.Site - ) + NUCLEI: _plan(NUCLEI, ObjectLabelsArtifactType, group_component=Microscopy.Site) } output_bindings = ( _output_binding( @@ -1415,9 +1427,7 @@ def test_cellprofiler_adapter_does_not_select_relationship_from_current_source_p ("1", "2"), component=Microscopy.Site ), component_scopes=( - ComponentGroupScope.from_raw( - ("1", "2"), component=Microscopy.Site - ), + ComponentGroupScope.from_raw(("1", "2"), component=Microscopy.Site), ), consumer_variable_components=(Microscopy.Site,), ), @@ -1515,9 +1525,7 @@ def test_cellprofiler_adapter_aligns_grouped_relationships_to_runtime_slices(): ("1", "2"), component=Microscopy.Site ), component_scopes=( - ComponentGroupScope.from_raw( - ("1", "2"), component=Microscopy.Site - ), + ComponentGroupScope.from_raw(("1", "2"), component=Microscopy.Site), ), consumer_variable_components=(Microscopy.Site,), ), @@ -1550,9 +1558,7 @@ def test_cellprofiler_adapter_aligns_grouped_relationships_to_runtime_slices(): def test_cellprofiler_adapter_does_not_source_scope_default_image_records(): store = RuntimeValueStore() outputs = { - DNA_IMAGE: _plan( - DNA_IMAGE, ImageArtifactType, group_component=Microscopy.Site - ) + DNA_IMAGE: _plan(DNA_IMAGE, ImageArtifactType, group_component=Microscopy.Site) } output_bindings = ( _output_binding(DNA_IMAGE, ImageArtifactType, plan=outputs[DNA_IMAGE]), @@ -1589,9 +1595,7 @@ def test_cellprofiler_adapter_does_not_source_scope_default_image_records(): group_component=Microscopy.Site, ), invocation_scope=ComponentGroupScope.ungrouped(), - producer_selection_scope=ComponentGroupScope.dynamic( - Microscopy.Site - ), + producer_selection_scope=ComponentGroupScope.dynamic(Microscopy.Site), component_scopes=(ComponentGroupScope.dynamic(Microscopy.Site),), consumer_variable_components=(Microscopy.Site,), ), @@ -1678,13 +1682,9 @@ def test_cellprofiler_adapter_stacks_declared_default_image_input_runtime_groups invocation_scope=ComponentGroupScope.from_raw( (AXIS_ID,), component=Microscopy.Well ), - producer_selection_scope=ComponentGroupScope.dynamic( - Microscopy.Site - ), + producer_selection_scope=ComponentGroupScope.dynamic(Microscopy.Site), component_scopes=( - ComponentGroupScope.from_raw( - (AXIS_ID,), component=Microscopy.Well - ), + ComponentGroupScope.from_raw((AXIS_ID,), component=Microscopy.Well), ComponentGroupScope.dynamic(Microscopy.Site), ), consumer_variable_components=(Microscopy.Site,), @@ -1778,9 +1778,7 @@ def test_cellprofiler_adapter_keeps_multisource_current_image_grouped_when_files group_component=Microscopy.Site, ), invocation_scope=ComponentGroupScope.ungrouped(), - producer_selection_scope=ComponentGroupScope.dynamic( - Microscopy.Site - ), + producer_selection_scope=ComponentGroupScope.dynamic(Microscopy.Site), component_scopes=(ComponentGroupScope.dynamic(Microscopy.Site),), consumer_variable_components=(Microscopy.Site,), ), @@ -1869,9 +1867,7 @@ def test_cellprofiler_adapter_stacks_declared_image_input_for_pattern_group(): group_component=Microscopy.Site, ), invocation_scope=ComponentGroupScope.ungrouped(), - producer_selection_scope=ComponentGroupScope.dynamic( - Microscopy.Site - ), + producer_selection_scope=ComponentGroupScope.dynamic(Microscopy.Site), component_scopes=(ComponentGroupScope.dynamic(Microscopy.Site),), consumer_variable_components=(Microscopy.Site,), ), @@ -2292,12 +2288,8 @@ def test_cellprofiler_adapter_uses_grouped_input_when_consumer_group_is_differen ("1", "2"), component=Microscopy.Site ), component_scopes=( - ComponentGroupScope.from_raw( - ("1",), component=Microscopy.Channel - ), - ComponentGroupScope.from_raw( - ("1", "2"), component=Microscopy.Site - ), + ComponentGroupScope.from_raw(("1",), component=Microscopy.Channel), + ComponentGroupScope.from_raw(("1", "2"), component=Microscopy.Site), ), consumer_variable_components=(Microscopy.Site,), ), @@ -2602,9 +2594,7 @@ def test_cellprofiler_adapter_does_not_select_image_record_from_current_source_s ("1", "2"), component=Microscopy.Site ), component_scopes=( - ComponentGroupScope.from_raw( - ("1", "2"), component=Microscopy.Site - ), + ComponentGroupScope.from_raw(("1", "2"), component=Microscopy.Site), ), consumer_variable_components=(Microscopy.Site,), ), @@ -2664,9 +2654,7 @@ def test_cellprofiler_adapter_does_not_select_image_record_from_current_source_s def test_cellprofiler_adapter_keeps_template_scoped_object_records_grouped(): store = RuntimeValueStore() outputs = { - NUCLEI: _plan( - NUCLEI, ObjectLabelsArtifactType, group_component=Microscopy.Site - ) + NUCLEI: _plan(NUCLEI, ObjectLabelsArtifactType, group_component=Microscopy.Site) } output_bindings = ( _output_binding( @@ -2710,9 +2698,7 @@ def test_cellprofiler_adapter_keeps_template_scoped_object_records_grouped(): group_component=Microscopy.Site, ), invocation_scope=ComponentGroupScope.ungrouped(), - producer_selection_scope=ComponentGroupScope.dynamic( - Microscopy.Site - ), + producer_selection_scope=ComponentGroupScope.dynamic(Microscopy.Site), component_scopes=(ComponentGroupScope.dynamic(Microscopy.Site),), consumer_variable_components=(Microscopy.Site,), ), @@ -3065,9 +3051,7 @@ def test_cellprofiler_adapter_availability_accepts_grouped_runtime_inputs(): ("1", "2"), component=Microscopy.Site ), component_scopes=( - ComponentGroupScope.from_raw( - ("1", "2"), component=Microscopy.Site - ), + ComponentGroupScope.from_raw(("1", "2"), component=Microscopy.Site), ), consumer_variable_components=(Microscopy.Site,), ), @@ -3335,9 +3319,7 @@ def test_cellprofiler_adapter_composes_object_input_across_declared_site_axis(): ("1", "2"), component=Microscopy.Site ), component_scopes=( - ComponentGroupScope.from_raw( - ("1", "2"), component=Microscopy.Site - ), + ComponentGroupScope.from_raw(("1", "2"), component=Microscopy.Site), ), consumer_variable_components=(Microscopy.Site,), ), @@ -3429,9 +3411,7 @@ def test_cellprofiler_adapter_does_not_resolve_object_input_from_source_context( ("1", "2"), component=Microscopy.Site ), component_scopes=( - ComponentGroupScope.from_raw( - ("1", "2"), component=Microscopy.Site - ), + ComponentGroupScope.from_raw(("1", "2"), component=Microscopy.Site), ), consumer_variable_components=(Microscopy.Site,), ), @@ -3515,9 +3495,7 @@ def test_cellprofiler_adapter_preserves_ungrouped_runtime_slice_output_stack(): invocation_scope=ComponentGroupScope.ungrouped(), producer_selection_scope=ComponentGroupScope.ungrouped(), component_scopes=( - ComponentGroupScope.from_raw( - ("1", "2"), component=Microscopy.Site - ), + ComponentGroupScope.from_raw(("1", "2"), component=Microscopy.Site), ), consumer_variable_components=(Microscopy.Site,), ), @@ -3608,9 +3586,7 @@ def test_cellprofiler_adapter_stacks_dynamic_compiled_grouped_images(): paths_by_group=group_paths, ), invocation_scope=ComponentGroupScope.ungrouped(), - producer_selection_scope=ComponentGroupScope.dynamic( - Microscopy.Site - ), + producer_selection_scope=ComponentGroupScope.dynamic(Microscopy.Site), component_scopes=(ComponentGroupScope.dynamic(Microscopy.Site),), consumer_variable_components=(Microscopy.Site,), ), @@ -4011,17 +3987,18 @@ def test_cellprofiler_adapter_adds_and_reads_slice_aligned_spatial_grids(): assert isinstance(stored, RuntimeSliceAlignedValues) assert stored.slice_count == 2 - assert [ - stored.value_at(index).name for index in range(stored.slice_count) - ] == ["Grid", "Grid"] - assert [ - stored.value_at(index).x_origin for index in range(stored.slice_count) - ] == [1.0, 2.0] + assert [stored.value_at(index).name for index in range(stored.slice_count)] == [ + "Grid", + "Grid", + ] + assert [stored.value_at(index).x_origin for index in range(stored.slice_count)] == [ + 1.0, + 2.0, + ] saved_grids = filemanager.saved[("memory", "/memory/Grid.pkl")] assert isinstance(saved_grids, RuntimeSliceAlignedValues) assert [ - saved_grids.value_at(index).x_origin - for index in range(saved_grids.slice_count) + saved_grids.value_at(index).x_origin for index in range(saved_grids.slice_count) ] == [ 1.0, 2.0, @@ -4380,9 +4357,7 @@ def test_cellprofiler_adapter_does_not_select_measurement_record_from_current_so ("1", "2"), component=Microscopy.Site ), component_scopes=( - ComponentGroupScope.from_raw( - ("1", "2"), component=Microscopy.Site - ), + ComponentGroupScope.from_raw(("1", "2"), component=Microscopy.Site), ), consumer_variable_components=(Microscopy.Site,), ), @@ -4849,9 +4824,7 @@ def test_cellprofiler_adapter_aligns_multiplane_measurements_across_groups(): group_component=Microscopy.Site, ), invocation_scope=ComponentGroupScope.ungrouped(), - producer_selection_scope=ComponentGroupScope.dynamic( - Microscopy.Site - ), + producer_selection_scope=ComponentGroupScope.dynamic(Microscopy.Site), component_scopes=(ComponentGroupScope.dynamic(Microscopy.Site),), consumer_variable_components=(Microscopy.Site,), ), @@ -5277,9 +5250,7 @@ def test_cellprofiler_adapter_multiplane_measurement_alignment_is_feature_scoped group_component=Microscopy.Site, ), invocation_scope=ComponentGroupScope.ungrouped(), - producer_selection_scope=ComponentGroupScope.dynamic( - Microscopy.Site - ), + producer_selection_scope=ComponentGroupScope.dynamic(Microscopy.Site), component_scopes=(ComponentGroupScope.dynamic(Microscopy.Site),), consumer_variable_components=(Microscopy.Site,), ), @@ -6109,9 +6080,7 @@ def test_cellprofiler_module_executor_reads_objects_for_measurements( def measure_object_size_shape( image_arg: ImagePayload, *, labels: ObjectLabelValue ) -> tuple[object, DataclassMeasurementColumnarRows]: - np.testing.assert_array_equal( - image_arg.data, image.data - ) + np.testing.assert_array_equal(image_arg.data, image.data) np.testing.assert_array_equal(object_label_dense_array(labels), labels_array) return image_arg, rows @@ -6266,7 +6235,10 @@ def test_cellprofiler_object_only_pure_2d_module_executes_label_runtime_slices( @runtime_bound_parameters(SliceIndexRuntimeParameter) @with_processing_contract(Pure2DContract) def measure_object_size_shape( - image_arg: ImagePayload, *, labels: ObjectLabelValue, slice_index: int | None = None + image_arg: ImagePayload, + *, + labels: ObjectLabelValue, + slice_index: int | None = None, ) -> tuple[object, DataclassMeasurementColumnarRows]: image_pixels = np.asarray(image_arg.data) label_pixels = object_label_dense_array(labels) @@ -6981,9 +6953,7 @@ def test_measurement_lookup_uses_canonical_runtime_identifier_for_numbered_featu def test_cellprofiler_child_count_lookup_uses_parent_row_domain(): - lookup = CELLPROFILER_MEASUREMENT_DIALECT.feature_lookup( - "Children_PH3_Count" - ) + lookup = CELLPROFILER_MEASUREMENT_DIALECT.feature_lookup("Children_PH3_Count") assert lookup.query_object_name(NUCLEI) is None @@ -9685,9 +9655,7 @@ def relate_objects( ), calculate_per_parent_means: bool = False, ): - np.testing.assert_array_equal( - image_arg.data, image.data - ) + np.testing.assert_array_equal(image_arg.data, image.data) np.testing.assert_array_equal(object_label_dense_array(parent_labels), cells) np.testing.assert_array_equal(object_label_dense_array(child_labels), nuclei) return ( diff --git a/tests/unit/test_cellprofiler_runtime_callable_introspection.py b/tests/unit/test_cellprofiler_runtime_callable_introspection.py index e737aadada..5d7a3db38b 100644 --- a/tests/unit/test_cellprofiler_runtime_callable_introspection.py +++ b/tests/unit/test_cellprofiler_runtime_callable_introspection.py @@ -187,7 +187,9 @@ def test_step_invocation_contract_provider_validates_public_cellprofiler_config( ValueError, match="unknown object_labels artifact 'Objects1'", ): - CellProfilerInvocationContractProviderFactory.provider_for_pipeline(session.pipeline) + CellProfilerInvocationContractProviderFactory.provider_for_pipeline( + session.pipeline + ) def test_public_compile_time_provider_binds_cellprofiler_runtime_from_step_declarations(): diff --git a/tests/unit/test_cellprofiler_save_images.py b/tests/unit/test_cellprofiler_save_images.py index cef5b6aba2..086c4d3c27 100644 --- a/tests/unit/test_cellprofiler_save_images.py +++ b/tests/unit/test_cellprofiler_save_images.py @@ -217,7 +217,9 @@ def test_save_images_is_an_adapter_free_executable_axis_module() -> None: assert callable_contract.processing_contract is Pure3DContract assert callable_contract.runtime_adapter is None assert callable_contract.resolve_runtime_callable() is save_images - assert CallableContract.from_callable(save_images).artifact_input_parameter_names == ("image_to_save",) + assert CallableContract.from_callable( + save_images + ).artifact_input_parameter_names == ("image_to_save",) def test_save_images_contract_consumes_runtime_image_and_declares_export_only() -> None: @@ -267,7 +269,9 @@ def test_save_images_selected_image_binding_is_runtime_special_identity() -> Non assert bound.kwargs["file_format"] is SaveImagesFileFormat.PNG assert bound.kwargs["bit_depth"] is SaveImagesBitDepth.UINT8 assert bound.kwargs["filename_method"] is SaveImagesFilenameMethod.SINGLE_NAME - assert CallableContract.from_callable(save_images).artifact_input_parameter_names == ("image_to_save",) + assert CallableContract.from_callable( + save_images + ).artifact_input_parameter_names == ("image_to_save",) def test_save_images_public_contract_selects_upstream_runtime_image() -> None: @@ -954,8 +958,12 @@ def test_image_output_context_projects_declared_group_lineage_source() -> None: ), ) - contextualized = (ImageArtifactType if output_plan is None else output_plan.artifact_type).contextualize_output( - (source_slices).project_declared_source((output_plan.source_context_source()).name), + contextualized = ( + ImageArtifactType if output_plan is None else output_plan.artifact_type + ).contextualize_output( + (source_slices).project_declared_source( + (output_plan.source_context_source()).name + ), np.zeros((3, 2, 2, 3), dtype=np.uint8), output_plan, RuntimePlaneAxisValueProjection.preserve( @@ -1029,8 +1037,12 @@ def test_declared_group_lineage_source_overrides_complete_output_identity() -> N ), ) - contextualized = (ImageArtifactType if output_plan is None else output_plan.artifact_type).contextualize_output( - (source_slices).project_declared_source((output_plan.source_context_source()).name), + contextualized = ( + ImageArtifactType if output_plan is None else output_plan.artifact_type + ).contextualize_output( + (source_slices).project_declared_source( + (output_plan.source_context_source()).name + ), complete_output, output_plan, RuntimePlaneAxisValueProjection.preserve( @@ -1072,7 +1084,9 @@ def test_planned_image_output_restores_axis_without_duplicate_provenance() -> No ), ) - contextualized = (ImageArtifactType if output_plan is None else output_plan.artifact_type).contextualize_output( + contextualized = ( + ImageArtifactType if output_plan is None else output_plan.artifact_type + ).contextualize_output( source_payload, source_payload, output_plan, @@ -1120,8 +1134,12 @@ def test_image_output_context_removes_selected_source_runtime_axis() -> None: ), ) - contextualized = (ImageArtifactType if output_plan is None else output_plan.artifact_type).contextualize_output( - (source_payload).project_declared_source((output_plan.source_context_source()).name), + contextualized = ( + ImageArtifactType if output_plan is None else output_plan.artifact_type + ).contextualize_output( + (source_payload).project_declared_source( + (output_plan.source_context_source()).name + ), np.zeros((2, 2, 3), dtype=np.uint8), output_plan, RuntimePlaneAxisValueProjection.preserve( @@ -1157,8 +1175,12 @@ def test_image_output_context_preserves_repeated_declared_source_planes() -> Non ), ) - contextualized = (ImageArtifactType if output_plan is None else output_plan.artifact_type).contextualize_output( - (source_payload).project_declared_source((output_plan.source_context_source()).name), + contextualized = ( + ImageArtifactType if output_plan is None else output_plan.artifact_type + ).contextualize_output( + (source_payload).project_declared_source( + (output_plan.source_context_source()).name + ), np.zeros((2, 2, 2), dtype=np.uint8), output_plan, RuntimePlaneAxisValueProjection.preserve( @@ -1192,7 +1214,9 @@ def test_image_output_context_accepts_exact_loaded_derived_artifact() -> None: relations=(SourceStackLineageSourceRelation(source=selected_artifact.ref()),), ) - contextualized = (ImageArtifactType if output_plan is None else output_plan.artifact_type).contextualize_output( + contextualized = ( + ImageArtifactType if output_plan is None else output_plan.artifact_type + ).contextualize_output( source_payload, np.ones((2, 2), dtype=np.uint8), output_plan, diff --git a/tests/unit/test_cellprofiler_source_bindings.py b/tests/unit/test_cellprofiler_source_bindings.py index c33bb9c172..bfa19fa2bb 100644 --- a/tests/unit/test_cellprofiler_source_bindings.py +++ b/tests/unit/test_cellprofiler_source_bindings.py @@ -1065,18 +1065,28 @@ def test_setup_fold_is_source_ordered_and_uses_module_registry() -> None: ) -@pytest.mark.parametrize("dtype,foreground", [(np.uint8, 255), (np.uint16, 65535), (np.float32, 0.25)]) -def test_declared_binary_source_preserves_foreground_through_cp_normalization(dtype, foreground): - from openhcs.interop.cellprofiler.image_normalization import normalize_cellprofiler_image_payload - from openhcs.processing.backends.cellprofiler.image_geometry import binary_mask_plane +@pytest.mark.parametrize( + "dtype,foreground", [(np.uint8, 255), (np.uint16, 65535), (np.float32, 0.25)] +) +def test_declared_binary_source_preserves_foreground_through_cp_normalization( + dtype, foreground +): + from openhcs.interop.cellprofiler.image_normalization import ( + normalize_cellprofiler_image_payload, + ) + from openhcs.processing.backends.cellprofiler.image_geometry import ( + binary_mask_plane, + ) raw = np.asarray([[0, foreground], [foreground, 0]], dtype=dtype) - context = ImagePayloadSourceMetadataContext(SourceImageIdentity('/input/mask.tif')) - loaded = NamedSourceBinding(alias='Mask', load_as_mask=True).apply_loaded_payload(raw, context) + context = ImagePayloadSourceMetadataContext(SourceImageIdentity("/input/mask.tif")) + loaded = NamedSourceBinding(alias="Mask", load_as_mask=True).apply_loaded_payload( + raw, context + ) source_metadata = loaded.metadata normalized = normalize_cellprofiler_image_payload(loaded) np.testing.assert_array_equal(normalized.data, raw != 0) np.testing.assert_array_equal(binary_mask_plane(normalized), raw != 0) assert source_metadata.unit_interval_intensity.scale == 1 assert normalized.metadata.source_provenance == source_metadata.source_provenance - assert source_metadata.source_path == '/input/mask.tif' + assert source_metadata.source_path == "/input/mask.tif" diff --git a/tests/unit/test_cellprofiler_source_bindings_ingestion.py b/tests/unit/test_cellprofiler_source_bindings_ingestion.py index d5efb73a41..2d02e9c5c6 100644 --- a/tests/unit/test_cellprofiler_source_bindings_ingestion.py +++ b/tests/unit/test_cellprofiler_source_bindings_ingestion.py @@ -134,9 +134,12 @@ def test_axis_projection_reads_each_axis_through_its_declarations() -> None: assert source_metadata_component(alias) is axis primary = axis.metadata_aliases[0] assert SourceAxisProjection.project(axis, {primary: "B07"}, 0) == "B07" - assert SourceAxisProjection.project( - Microscopy.Well, {"Metadata_Row": "c", "Metadata_Column": "4"}, 0 - ) == "C04" + assert ( + SourceAxisProjection.project( + Microscopy.Well, {"Metadata_Row": "c", "Metadata_Column": "4"}, 0 + ) + == "C04" + ) assert SourceAxisProjection.project(Microscopy.Well, {}, 3) == "A01" assert SourceAxisProjection.project(Microscopy.Channel, {}, 3) == "4" assert SourceAxisProjection.project(Microscopy.ZIndex, {}, 3) == "1" diff --git a/tests/unit/test_cellprofiler_special_io_declarations.py b/tests/unit/test_cellprofiler_special_io_declarations.py index cf2b61bacb..e41ad30ef0 100644 --- a/tests/unit/test_cellprofiler_special_io_declarations.py +++ b/tests/unit/test_cellprofiler_special_io_declarations.py @@ -731,10 +731,7 @@ def test_fixed_return_slots_follow_nominal_contract_output_order() -> None: f"{diagnostic_prefix}__{stage}" for stage in PrimaryObjectDiagnosticPlanes._fields ) - assert all( - spec.sidecar_role is QaCheckpoint - for spec in diagnostic_outputs - ) + assert all(spec.sidecar_role is QaCheckpoint for spec in diagnostic_outputs) def test_contract_order_owns_guided_grid_and_tertiary_input_mapping() -> None: @@ -830,7 +827,13 @@ def test_object_colocalization_rank_provider_is_runtime_bound() -> None: parameter = parameters["rank_provider"] assert parameter.kind is inspect.Parameter.KEYWORD_ONLY - assert CallableContract.from_callable(measure_colocalization_objects).runtime_bound_parameters == ("rank_provider", "threshold_mask_outputs", "costes_threshold_batch") + assert CallableContract.from_callable( + measure_colocalization_objects + ).runtime_bound_parameters == ( + "rank_provider", + "threshold_mask_outputs", + "costes_threshold_batch", + ) contract = CallableContract.from_callable(measure_colocalization_objects) assert contract.config_bound_parameter_names == ("dtype_config",) assert parameters["dtype_config"].annotation is LazyDtypeConfig diff --git a/tests/unit/test_cellprofiler_spreadsheet_export.py b/tests/unit/test_cellprofiler_spreadsheet_export.py index 85e5679579..a4c86829f3 100644 --- a/tests/unit/test_cellprofiler_spreadsheet_export.py +++ b/tests/unit/test_cellprofiler_spreadsheet_export.py @@ -108,6 +108,7 @@ from openhcs.core.axes import Axis from openhcs.domains.microscopy.axes import Microscopy + def test_export_to_spreadsheet_declares_exact_plate_callable_abi() -> None: contract = CallableContract.from_callable(export_to_spreadsheet) parameter = inspect.signature(export_to_spreadsheet).parameters["artifact_batch"] @@ -421,136 +422,176 @@ def test_export_to_spreadsheet_renders_only_declared_batch_records() -> None: @pytest.mark.parametrize("add_files", (False, True)) @pytest.mark.parametrize("axisless", (False, True)) def test_spreadsheet_projects_source_identity_without_upstream_image_features( - add_metadata: bool, add_files: bool, axisless: bool, + add_metadata: bool, + add_files: bool, + axisless: bool, ) -> None: - provenance = SourceImageProvenancePlanes(( - RuntimeSourceImageProvenancePlane( - SourceImageIdentity('/acquisition/body.tif', { - 'well': 'A01', 'site': '2', 'channel': '3', 'z_index': '4', - 'timepoint': '5', DECLARED_SOURCE_METADATA_FIELD: {'Treatment': 'control'}, - }), - source_image_name='Body', - ), - )) + provenance = SourceImageProvenancePlanes( + ( + RuntimeSourceImageProvenancePlane( + SourceImageIdentity( + "/acquisition/body.tif", + { + "well": "A01", + "site": "2", + "channel": "3", + "z_index": "4", + "timepoint": "5", + DECLARED_SOURCE_METADATA_FIELD: {"Treatment": "control"}, + }, + ), + source_image_name="Body", + ), + ) + ) cells = _measurement_record( - 'cells', axis_id='A01', - subject=MeasurementSubject(MeasurementScope.OBJECT, 'Cells', 'object_number'), - rows=({**({} if axisless else {'slice_index': 0}), 'object_number': 7, 'Area': 12.0},), + "cells", + axis_id="A01", + subject=MeasurementSubject(MeasurementScope.OBJECT, "Cells", "object_number"), + rows=( + { + **({} if axisless else {"slice_index": 0}), + "object_number": 7, + "Area": 12.0, + }, + ), source_image_provenance_planes=provenance, ) bundle = render_spreadsheet_bundle( artifact_batch=RuntimeArtifactBatch( - input_specs=(ArtifactSpec.input('cells', MeasurementsArtifactType),), - records_by_axis={'A01': (cells,)}, + input_specs=(ArtifactSpec.input("cells", MeasurementsArtifactType),), + records_by_axis={"A01": (cells,)}, source_image_set_identity_policy=SourceImageSetIdentityPolicy(), ), - add_image_metadata=add_metadata, add_image_file_names=add_files, + add_image_metadata=add_metadata, + add_image_file_names=add_files, add_filename_prefix=False, ) - image, = csv.DictReader(io.StringIO(bundle['Image.csv'])) - cell, = csv.DictReader(io.StringIO(bundle['Cells.csv'])) - assert image['Metadata_Treatment'] == 'control' - assert ('Metadata_site' in image) is add_metadata - assert ('FileName_Body' in image) is add_files + (image,) = csv.DictReader(io.StringIO(bundle["Image.csv"])) + (cell,) = csv.DictReader(io.StringIO(bundle["Cells.csv"])) + assert image["Metadata_Treatment"] == "control" + assert ("Metadata_site" in image) is add_metadata + assert ("FileName_Body" in image) is add_files if add_metadata: - assert image['Metadata_well'] == 'A01' - assert image['Metadata_site'] == '2' - assert image['Metadata_z_index'] == '4' - assert image['Metadata_timepoint'] == '5' + assert image["Metadata_well"] == "A01" + assert image["Metadata_site"] == "2" + assert image["Metadata_z_index"] == "4" + assert image["Metadata_timepoint"] == "5" if add_files: - assert image['FileName_Body'] == 'body.tif' - assert image['PathName_Body'] == '/acquisition' - assert cell['object_number'] == '7' - assert cell['Area'] == '12.0' - assert ('Metadata_site' in cell) is add_metadata - assert ('Image_FileName_Body' in cell) is add_files + assert image["FileName_Body"] == "body.tif" + assert image["PathName_Body"] == "/acquisition" + assert cell["object_number"] == "7" + assert cell["Area"] == "12.0" + assert ("Metadata_site" in cell) is add_metadata + assert ("Image_FileName_Body" in cell) is add_files if add_files: - assert cell['Image_FileName_Body'] == 'body.tif' - assert cell['Image_PathName_Body'] == '/acquisition' - - -@pytest.mark.parametrize('names', (('Body', 'Nuclear'), ('Nuclear', 'Body'))) -def test_spreadsheet_preserves_independent_contributor_filenames(names: tuple[str, ...]) -> None: - provenance = SourceImageProvenancePlanes(( - RuntimeSourceImageProvenancePlane( - SourceImageIdentity(component_metadata={'well': 'A01', 'site': '2'}), - contributors=tuple( - SourceImageProvenanceContributor( - SourceImageIdentity(f'/acquisition/{name}.tif', {'well': 'A01', 'site': '2'}), - source_image_name=name, - ) - for name in names + assert cell["Image_FileName_Body"] == "body.tif" + assert cell["Image_PathName_Body"] == "/acquisition" + + +@pytest.mark.parametrize("names", (("Body", "Nuclear"), ("Nuclear", "Body"))) +def test_spreadsheet_preserves_independent_contributor_filenames( + names: tuple[str, ...], +) -> None: + provenance = SourceImageProvenancePlanes( + ( + RuntimeSourceImageProvenancePlane( + SourceImageIdentity(component_metadata={"well": "A01", "site": "2"}), + contributors=tuple( + SourceImageProvenanceContributor( + SourceImageIdentity( + f"/acquisition/{name}.tif", {"well": "A01", "site": "2"} + ), + source_image_name=name, + ) + for name in names + ), ), - ), - )) + ) + ) record = _measurement_record( - 'cells', axis_id='A01', - subject=MeasurementSubject(MeasurementScope.OBJECT, 'Cells', 'object_number'), - rows=({'slice_index': 0, 'object_number': 1, 'Area': 12.0},), + "cells", + axis_id="A01", + subject=MeasurementSubject(MeasurementScope.OBJECT, "Cells", "object_number"), + rows=({"slice_index": 0, "object_number": 1, "Area": 12.0},), source_image_provenance_planes=provenance, ) bundle = render_spreadsheet_bundle( artifact_batch=RuntimeArtifactBatch( - input_specs=(ArtifactSpec.input('cells', MeasurementsArtifactType),), - records_by_axis={'A01': (record,)}, + input_specs=(ArtifactSpec.input("cells", MeasurementsArtifactType),), + records_by_axis={"A01": (record,)}, source_image_set_identity_policy=SourceImageSetIdentityPolicy(), ), - add_image_metadata=True, add_image_file_names=True, add_filename_prefix=False, + add_image_metadata=True, + add_image_file_names=True, + add_filename_prefix=False, ) - row, = csv.DictReader(io.StringIO(bundle['Cells.csv'])) + (row,) = csv.DictReader(io.StringIO(bundle["Cells.csv"])) for name in names: - assert row[f'Image_FileName_{name}'] == f'{name}.tif' - assert row[f'Image_PathName_{name}'] == '/acquisition' - assert 'Metadata_channel' not in row - assert 'Metadata_z_index' not in row + assert row[f"Image_FileName_{name}"] == f"{name}.tif" + assert row[f"Image_PathName_{name}"] == "/acquisition" + assert "Metadata_channel" not in row + assert "Metadata_z_index" not in row def test_requested_source_columns_preserve_original_extraction_path_template() -> None: image = _measurement_record( - 'image', axis_id='A01', - subject=MeasurementSubject(MeasurementScope.SAMPLE, 'Image'), - rows=({'slice_index': 0, 'Count_Cells': 1},), + "image", + axis_id="A01", + subject=MeasurementSubject(MeasurementScope.SAMPLE, "Image"), + rows=({"slice_index": 0, "Count_Cells": 1},), source_image_provenance_planes=SourceImageProvenancePlanes.from_components( - component_metadata=({'site': '1', DECLARED_SOURCE_METADATA_FIELD: {'Run': 'run1'}},), + component_metadata=( + {"site": "1", DECLARED_SOURCE_METADATA_FIELD: {"Run": "run1"}}, + ), ), ) cells = _measurement_record( - 'cells', axis_id='A01', - subject=MeasurementSubject(MeasurementScope.OBJECT, 'Cells', 'object_number'), - rows=({'slice_index': 0, 'object_number': 1, 'Area': 12.0},), + "cells", + axis_id="A01", + subject=MeasurementSubject(MeasurementScope.OBJECT, "Cells", "object_number"), + rows=({"slice_index": 0, "object_number": 1, "Area": 12.0},), ) bundle = render_spreadsheet_bundle( artifact_batch=RuntimeArtifactBatch( - input_specs=tuple(ArtifactSpec.input(record.key.name, MeasurementsArtifactType) for record in (image, cells)), - records_by_axis={'A01': (image, cells)}, + input_specs=tuple( + ArtifactSpec.input(record.key.name, MeasurementsArtifactType) + for record in (image, cells) + ), + records_by_axis={"A01": (image, cells)}, source_image_set_identity_policy=SourceImageSetIdentityPolicy(), ), - add_image_metadata=True, add_image_file_names=True, - output_directory='{Run}', add_filename_prefix=False, + add_image_metadata=True, + add_image_file_names=True, + output_directory="{Run}", + add_filename_prefix=False, ) - assert set(bundle) == {'run1/Image.csv', 'run1/Cells.csv'} - row, = csv.DictReader(io.StringIO(bundle['run1/Cells.csv'])) - assert row['Metadata_Run'] == 'run1' - assert row['Metadata_site'] == '1' + assert set(bundle) == {"run1/Image.csv", "run1/Cells.csv"} + (row,) = csv.DictReader(io.StringIO(bundle["run1/Cells.csv"])) + assert row["Metadata_Run"] == "run1" + assert row["Metadata_site"] == "1" def test_spreadsheet_rejects_existing_filename_conflicting_with_provenance() -> None: record = _measurement_record( - 'image', axis_id='A01', - subject=MeasurementSubject(MeasurementScope.SAMPLE, 'Image'), - rows=({'slice_index': 0, 'FileName_Body': 'other.tif'},), - source_image_provenance_planes=SourceImageProvenancePlanes(( - RuntimeSourceImageProvenancePlane( - SourceImageIdentity('/acquisition/body.tif', {'site': '1'}), - source_image_name='Body', - ), - )), + "image", + axis_id="A01", + subject=MeasurementSubject(MeasurementScope.SAMPLE, "Image"), + rows=({"slice_index": 0, "FileName_Body": "other.tif"},), + source_image_provenance_planes=SourceImageProvenancePlanes( + ( + RuntimeSourceImageProvenancePlane( + SourceImageIdentity("/acquisition/body.tif", {"site": "1"}), + source_image_name="Body", + ), + ) + ), ) - with pytest.raises(ValueError, match='Conflicting sparse measurement values'): + with pytest.raises(ValueError, match="Conflicting sparse measurement values"): render_spreadsheet_bundle( artifact_batch=RuntimeArtifactBatch( - input_specs=(ArtifactSpec.input('image', MeasurementsArtifactType),), - records_by_axis={'A01': (record,)}, + input_specs=(ArtifactSpec.input("image", MeasurementsArtifactType),), + records_by_axis={"A01": (record,)}, source_image_set_identity_policy=SourceImageSetIdentityPolicy(), ), add_image_file_names=True, @@ -582,6 +623,7 @@ def test_export_to_spreadsheet_bundle_uses_generic_file_materialization() -> Non context=context, ) from openhcs.processing.materialization.core import ColumnarCsvOutput + assert all(isinstance(output, ColumnarCsvOutput) for output in bundle.values()) outputs = materialization_outputs( MaterializationSpec(FileBundleOptions()), @@ -1902,11 +1944,11 @@ def _measurement_record( axis_id=axis_id, ) return StoredRuntimeValue( - key=value.key, - data=value.data, - materialization_source_metadata=value.materialization_source_metadata, - location=RuntimeArtifactLocation(path=output_plan.path, backend="memory"), - ) + key=value.key, + data=value.data, + materialization_source_metadata=value.materialization_source_metadata, + location=RuntimeArtifactLocation(path=output_plan.path, backend="memory"), + ) def _fixture_field_dtype( @@ -1953,11 +1995,11 @@ def _relationship_record(name: str, *, axis_id: str) -> StoredRuntimeValue: axis_id=axis_id, ) return StoredRuntimeValue( - key=value.key, - data=value.data, - materialization_source_metadata=value.materialization_source_metadata, - location=RuntimeArtifactLocation(path=output_plan.path, backend="memory"), - ) + key=value.key, + data=value.data, + materialization_source_metadata=value.materialization_source_metadata, + location=RuntimeArtifactLocation(path=output_plan.path, backend="memory"), + ) @pytest.mark.parametrize("cycle_aligned", (False, True)) @@ -2077,19 +2119,26 @@ def test_image_number_references_follow_exact_source_numbering( from openhcs.core.measurement_row_materialization import ConcatenatedColumnarRows fields = tuple( - FieldSpec(name, int) - if name in ("slice_index", "object_number") - else FieldSpec(name, required=False) + ( + FieldSpec(name, int) + if name in ("slice_index", "object_number") + else FieldSpec(name, required=False) + ) for name in columns ) source_rows = ( - ConcatenatedColumnarRows(tuple( - MeasurementSparseColumnarRows( - {name: values[start:start + 3] for name, values in columns.items()}, - fields=fields, + ConcatenatedColumnarRows( + tuple( + MeasurementSparseColumnarRows( + { + name: values[start : start + 3] + for name, values in columns.items() + }, + fields=fields, + ) + for start in (0, 3) ) - for start in (0, 3) - )) + ) if concatenated else MeasurementSparseColumnarRows(columns, fields=fields) ) @@ -2136,9 +2185,7 @@ def columns(**values): name = "Mean_Cells_Tracking_ParentImageNumber_50" source = OrderedDict( Image=columns(image_number=(23,), **{name: (0.5,)}), - Cells=columns( - image_number=(23, 23), Tracking_ParentImageNumber_50=(0, 22) - ), + Cells=columns(image_number=(23, 23), Tracking_ParentImageNumber_50=(0, 22)), ) selected = OrderedDict(Image=source["Image"]) result = _with_requested_aggregates( diff --git a/tests/unit/test_cellprofiler_watershed_provider.py b/tests/unit/test_cellprofiler_watershed_provider.py index f94ae6e23d..c1ac6c0eea 100644 --- a/tests/unit/test_cellprofiler_watershed_provider.py +++ b/tests/unit/test_cellprofiler_watershed_provider.py @@ -6,7 +6,9 @@ import pytest from openhcs.constants.constants import MemoryType -from openhcs.processing.backends.cellprofiler._backend import CellProfilerBackendProvider +from openhcs.processing.backends.cellprofiler._backend import ( + CellProfilerBackendProvider, +) from openhcs.processing.backends.cellprofiler.watershed import ( LegacyWatershedBackendStrategy, NumbaNumpyLegacyWatershedBackendStrategy, @@ -31,8 +33,9 @@ def reject_centrosome(name, *args, **kwargs): MemoryType.NUMPY, backend_provider=provider ) assert strategy.backend_provider is provider.provider - assert provider.provider in LegacyWatershedBackendStrategy.available_backend_providers( - MemoryType.NUMPY + assert ( + provider.provider + in LegacyWatershedBackendStrategy.available_backend_providers(MemoryType.NUMPY) ) image = np.array([[0.0, 1.0, 0.0]]) markers = marker_sign * np.array([[1, 0, 2]], dtype=np.int32) @@ -132,25 +135,27 @@ def observe_request(self, image, *, markers, mask, connectivity=1): ) ).astype(np.float32) original_image = image.copy() - returned_image, measurements, payload, *_diagnostic_planes = identify_primary_objects( - image, - min_diameter=4, - max_diameter=14, - exclude_size=False, - exclude_border_objects=False, - unclump_method=UnclumpMethod.INTENSITY, - watershed_method=WatershedMethod.INTENSITY, - automatic_smoothing=False, - smoothing_filter_size=0, - automatic_suppression=False, - maxima_suppression_size=3, - low_res_maxima=False, - fill_holes=FillHolesOption.NEVER, - threshold_method=CellProfilerThresholdMethod.MANUAL, - manual_threshold=0.2, - threshold_smoothing_scale=0, - morphology_backend_provider=CellProfilerBackendProvider.CENTROSOME, - watershed_backend_provider=provider, + returned_image, measurements, payload, *_diagnostic_planes = ( + identify_primary_objects( + image, + min_diameter=4, + max_diameter=14, + exclude_size=False, + exclude_border_objects=False, + unclump_method=UnclumpMethod.INTENSITY, + watershed_method=WatershedMethod.INTENSITY, + automatic_smoothing=False, + smoothing_filter_size=0, + automatic_suppression=False, + maxima_suppression_size=3, + low_res_maxima=False, + fill_holes=FillHolesOption.NEVER, + threshold_method=CellProfilerThresholdMethod.MANUAL, + manual_threshold=0.2, + threshold_smoothing_scale=0, + morphology_backend_provider=CellProfilerBackendProvider.CENTROSOME, + watershed_backend_provider=provider, + ) ) (selected_backend,) = selected (request,) = requests @@ -161,7 +166,10 @@ def observe_request(self, image, *, markers, mask, connectivity=1): assert labels.dtype == np.int32 assert labels.shape == image.shape assert np.unique(labels).tolist() == [0, 1, 2] - assert payload.domain.require_explicit_id_domain(context="primary journey") == (1, 2) + assert payload.domain.require_explicit_id_domain(context="primary journey") == ( + 1, + 2, + ) np.testing.assert_array_equal(labels > 0, image > 0.2) np.testing.assert_array_equal(returned_image, original_image) np.testing.assert_array_equal(image, original_image) diff --git a/tests/unit/test_cellprofiler_workspace_export.py b/tests/unit/test_cellprofiler_workspace_export.py index ad64ca2a3c..b755968869 100644 --- a/tests/unit/test_cellprofiler_workspace_export.py +++ b/tests/unit/test_cellprofiler_workspace_export.py @@ -1,4 +1,3 @@ - from tests.unit.saved_output_dialect import SAVED_OUTPUT_DIALECT from dataclasses import replace from pathlib import Path @@ -254,7 +253,9 @@ def test_workspace_comparison_coverage_is_physical_and_separate_from_science(tmp for root in (reference, candidate) ) report = cellprofiler_database_export_equivalence( - reference, exports[1], policy=RuntimeEquivalencePolicy(measurement_dialect=SAVED_OUTPUT_DIALECT) + reference, + exports[1], + policy=RuntimeEquivalencePolicy(measurement_dialect=SAVED_OUTPUT_DIALECT), ) assert report.is_equivalent assert report.compared_output_files == frozenset(paths) @@ -271,7 +272,9 @@ def test_workspace_comparison_coverage_is_physical_and_separate_from_science(tmp _require_compared_output_inventory(**kwargs, compared_file_report=report) paths[1].write_text(paths[1].read_text().replace("Image_", "Changed_", 1)) changed = cellprofiler_database_export_equivalence( - reference, exports[1], policy=RuntimeEquivalencePolicy(measurement_dialect=SAVED_OUTPUT_DIALECT) + reference, + exports[1], + policy=RuntimeEquivalencePolicy(measurement_dialect=SAVED_OUTPUT_DIALECT), ) assert not changed.is_equivalent assert changed.compared_output_files == frozenset(paths) @@ -300,7 +303,9 @@ def test_unmatched_workspace_and_reader_failures_never_claim_coverage( extra.write_bytes(fixture.read_bytes()) exports = RuntimeExportObservation.from_output_root(candidate) report = cellprofiler_database_export_equivalence( - reference, exports, policy=RuntimeEquivalencePolicy(measurement_dialect=SAVED_OUTPUT_DIALECT) + reference, + exports, + policy=RuntimeEquivalencePolicy(measurement_dialect=SAVED_OUTPUT_DIALECT), ) assert not report.is_equivalent assert not report.compared_output_files @@ -310,7 +315,9 @@ def test_unmatched_workspace_and_reader_failures_never_claim_coverage( extra.write_text("not a CPA workspace") with pytest.raises(ValueError): cellprofiler_database_export_equivalence( - reference, exports, policy=RuntimeEquivalencePolicy(measurement_dialect=SAVED_OUTPUT_DIALECT) + reference, + exports, + policy=RuntimeEquivalencePolicy(measurement_dialect=SAVED_OUTPUT_DIALECT), ) extra.write_bytes(fixture.read_bytes()) original_read = Path.read_text @@ -324,7 +331,9 @@ def fail_read(path, *args, **kwargs): monkeypatch.setattr(Path, "read_text", fail_read) with pytest.raises(OSError) as raised: cellprofiler_database_export_equivalence( - reference, exports, policy=RuntimeEquivalencePolicy(measurement_dialect=SAVED_OUTPUT_DIALECT) + reference, + exports, + policy=RuntimeEquivalencePolicy(measurement_dialect=SAVED_OUTPUT_DIALECT), ) assert raised.value is failure diff --git a/tests/unit/test_classification_two_vector_authoring.py b/tests/unit/test_classification_two_vector_authoring.py index 9acf7d0341..5ce03e08a5 100644 --- a/tests/unit/test_classification_two_vector_authoring.py +++ b/tests/unit/test_classification_two_vector_authoring.py @@ -11,7 +11,11 @@ import numpy as np import pytest -from openhcs.agent.dto.pipeline import FunctionSpecRef, FunctionStepAddRequest, FunctionStepSpec +from openhcs.agent.dto.pipeline import ( + FunctionSpecRef, + FunctionStepAddRequest, + FunctionStepSpec, +) from openhcs.agent.services.pipeline_authoring_service import PipelineAuthoringService from openhcs.core.function_patterns import normalize_function_pattern from openhcs.core.runtime_image_values import ImagePayloadMetadata @@ -76,9 +80,9 @@ def test_original_mode_projections_have_no_case_dispatch(projection): for node in ast.walk(tree) if isinstance(node, (ast.If, ast.IfExp, ast.Match, ast.Compare, ast.Dict)) ) - assert not forbidden, ( - f"{projection} still decides case behavior: {[type(node).__name__ for node in forbidden]}" - ) + assert ( + not forbidden + ), f"{projection} still decides case behavior: {[type(node).__name__ for node in forbidden]}" member_names = frozenset(ClassificationMethod.__members__) assert not any( isinstance(node, ast.Attribute) and node.attr in member_names @@ -296,7 +300,9 @@ def test_original_parsed_pair_runtime_binds_only_its_declared_signature(): @pytest.mark.parametrize("clean", (False, True)) -def test_original_public_016_json_kwargs_reach_strict_020_callable_abi(clean, monkeypatch): +def test_original_public_016_json_kwargs_reach_strict_020_callable_abi( + clean, monkeypatch +): """Exact live request's kwargs, without plate/native/registration replay.""" function_id, metadata = RegistryService.declared_metadata_for_callable( classify_objects_two_measurements @@ -305,7 +311,7 @@ def test_original_public_016_json_kwargs_reach_strict_020_callable_abi(clean, mo monkeypatch.setattr(RegistryService, "_resolved_reference_callables", {}) author = PipelineAuthoringService(function_catalog=SelectedDeclarations()) ref = author.create_pipeline() - kwargs = json.loads('''{ + kwargs = json.loads("""{ "select_the_object_to_be_classified": "engineering_cells", "measurement1_feature": "pixel_count", "measurement2_feature": "calibration_um", @@ -313,22 +319,39 @@ def test_original_public_016_json_kwargs_reach_strict_020_callable_abi(clean, mo "threshold2_method": "custom", "threshold2_value": 2.0, "low_low_name": "low_low", "high_low_name": "high_low", "low_high_name": "low_high", "high_high_name": "high_high" - }''') - author.add_function_step_from_request(FunctionStepAddRequest.from_fields( - pipeline_id=ref.pipeline_id, function_id=function_id, - name="engineering_pair_classification", kwargs=kwargs, - )) + }""") + author.add_function_step_from_request( + FunctionStepAddRequest.from_fields( + pipeline_id=ref.pipeline_id, + function_id=function_id, + name="engineering_pair_classification", + kwargs=kwargs, + ) + ) assert author.validate(ref).valid - document = PipelineDocumentCodec.from_source(author.render_source(ref, clean=clean).source) - invocation = next(normalize_function_pattern(document.pipeline_steps[0].func).iter_items()) + document = PipelineDocumentCodec.from_source( + author.render_source(ref, clean=clean).source + ) + invocation = next( + normalize_function_pattern(document.pipeline_steps[0].func).iter_items() + ) values = invocation.kwargs_dict assert values["threshold1_method"] is ClassificationThresholdMethod.CUSTOM assert values["threshold2_method"] is ClassificationThresholdMethod.CUSTOM assert (values["threshold1_value"], values["threshold2_value"]) == (100.0, 2.0) - assert (values["measurement1_feature"], values["measurement2_feature"]) == ("pixel_count", "calibration_um") + assert (values["measurement1_feature"], values["measurement2_feature"]) == ( + "pixel_count", + "calibration_um", + ) assert values["select_the_object_to_be_classified"] == "engineering_cells" - executable = {name: value for name, value in values.items() if name != "select_the_object_to_be_classified"} - assert invocation.contract.validate_public_kwargs(executable) == tuple(executable.items()) + executable = { + name: value + for name, value in values.items() + if name != "select_the_object_to_be_classified" + } + assert invocation.contract.validate_public_kwargs(executable) == tuple( + executable.items() + ) @pytest.mark.parametrize( @@ -571,10 +594,12 @@ def test_paired_author_validate_render_parse_compile_and_original_runtime( author = PipelineAuthoringService(function_catalog=SelectedDeclarations()) authored_kwargs = paired_kwargs() if json_kwargs: - authored_kwargs = json.loads(json.dumps( - authored_kwargs, - default=lambda member: member.value, - )) + authored_kwargs = json.loads( + json.dumps( + authored_kwargs, + default=lambda member: member.value, + ) + ) ref = author.create_pipeline( steps=( FunctionStepSpec( @@ -589,8 +614,14 @@ def test_paired_author_validate_render_parse_compile_and_original_runtime( invocation = next( normalize_function_pattern(restored.pipeline_steps[0].func).iter_items() ) - assert invocation.kwargs_dict["threshold1_method"] is ClassificationThresholdMethod.CUSTOM - assert invocation.kwargs_dict["threshold2_method"] is ClassificationThresholdMethod.CUSTOM + assert ( + invocation.kwargs_dict["threshold1_method"] + is ClassificationThresholdMethod.CUSTOM + ) + assert ( + invocation.kwargs_dict["threshold2_method"] + is ClassificationThresholdMethod.CUSTOM + ) contract = _public_function_step_contract( ClassifyObjectsSingleMeasurementModule, invocation.contract.func, diff --git a/tests/unit/test_collapsed_persisted_occurrence_435.py b/tests/unit/test_collapsed_persisted_occurrence_435.py index e513f32f86..dd4e4070cf 100644 --- a/tests/unit/test_collapsed_persisted_occurrence_435.py +++ b/tests/unit/test_collapsed_persisted_occurrence_435.py @@ -37,7 +37,9 @@ ) from openhcs.core.source_workspace_projection import VirtualWorkspaceSourceProjection from openhcs.core.steps.function_output_identity import FunctionOutputIdentity -from openhcs.core.steps.function_artifact_materialization import ArtifactMaterializationTargetPlan +from openhcs.core.steps.function_artifact_materialization import ( + ArtifactMaterializationTargetPlan, +) from openhcs.core.steps.function_outputs import ( PrimaryImageMetadataTarget, ) diff --git a/tests/unit/test_colocalization_metric_setting_authority.py b/tests/unit/test_colocalization_metric_setting_authority.py index 59d99fede6..e5ae55a195 100644 --- a/tests/unit/test_colocalization_metric_setting_authority.py +++ b/tests/unit/test_colocalization_metric_setting_authority.py @@ -153,7 +153,7 @@ def test_recorded_source_pair_features_exclude_disabled_metrics(disabled, famili for name in enabled.columns if not any( name.startswith( - f"Correlation_{(family if family == "RWC" else family.title())}_" + f"Correlation_{(family if family == 'RWC' else family.title())}_" ) for family in families ) diff --git a/tests/unit/test_colocalization_preparation_readiness.py b/tests/unit/test_colocalization_preparation_readiness.py index 0299706bfd..762f6b4a32 100644 --- a/tests/unit/test_colocalization_preparation_readiness.py +++ b/tests/unit/test_colocalization_preparation_readiness.py @@ -79,10 +79,16 @@ def reject_late_work(*args, **kwargs): """) environment = os.environ.copy() environment.update( - OPENHCS_CPU_ONLY="true", NUMBA_CACHE_DIR=str(tmp_path / "kernels"), + OPENHCS_CPU_ONLY="true", + NUMBA_CACHE_DIR=str(tmp_path / "kernels"), ) result = subprocess.run( - (sys.executable, "-c", script), cwd=Path(__file__).parents[2], - env=environment, check=False, capture_output=True, text=True, timeout=90, + (sys.executable, "-c", script), + cwd=Path(__file__).parents[2], + env=environment, + check=False, + capture_output=True, + text=True, + timeout=90, ) assert result.returncode == 0, result.stdout + result.stderr diff --git a/tests/unit/test_compilation_paths.py b/tests/unit/test_compilation_paths.py index 67b0949dcf..522ed0f381 100644 --- a/tests/unit/test_compilation_paths.py +++ b/tests/unit/test_compilation_paths.py @@ -235,7 +235,9 @@ def test_callable_compilation_resolves_authored_and_default_declared_paths() -> captured = CallableContract.from_callable(_path_callable).with_prepared_signature() restored = pickle.loads(pickle.dumps(captured)) - assert restored.metadata.canonical_signature == captured.metadata.canonical_signature + assert ( + restored.metadata.canonical_signature == captured.metadata.canonical_signature + ) assert restored.declared_path_parameters == captured.declared_path_parameters assert restored.resolve_declared_paths({}, resolver) == {"template_path": expected} @@ -244,19 +246,28 @@ def test_path_declarations_follow_live_or_captured_signature_epoch() -> None: def operation(image, *, path=None): return image - operation.__signature__ = inspect.Signature(( - inspect.Parameter("image", inspect.Parameter.POSITIONAL_OR_KEYWORD), - inspect.Parameter("path", inspect.Parameter.KEYWORD_ONLY, - annotation=PlateInputFile, default=None), - )) + operation.__signature__ = inspect.Signature( + ( + inspect.Parameter("image", inspect.Parameter.POSITIONAL_OR_KEYWORD), + inspect.Parameter( + "path", + inspect.Parameter.KEYWORD_ONLY, + annotation=PlateInputFile, + default=None, + ), + ) + ) live = CallableContract.from_callable(operation) captured = live.with_prepared_signature() layout = captured.declared_path_parameters - operation.__signature__ = operation.__signature__.replace(parameters=( - inspect.Parameter("image", inspect.Parameter.POSITIONAL_OR_KEYWORD), - inspect.Parameter("path", inspect.Parameter.KEYWORD_ONLY, - annotation=str, default=None), - )) + operation.__signature__ = operation.__signature__.replace( + parameters=( + inspect.Parameter("image", inspect.Parameter.POSITIONAL_OR_KEYWORD), + inspect.Parameter( + "path", inspect.Parameter.KEYWORD_ONLY, annotation=str, default=None + ), + ) + ) assert captured.declared_path_parameters is layout assert isinstance(layout["path"], PlateInputFileDeclaration) assert not live.declared_path_parameters diff --git a/tests/unit/test_compilation_session.py b/tests/unit/test_compilation_session.py index 4af3639b8a..be21c1f5fa 100644 --- a/tests/unit/test_compilation_session.py +++ b/tests/unit/test_compilation_session.py @@ -126,7 +126,8 @@ def full_plate_keys(component, *, resolved_config): assert plans[0].zarr_config is None if backend is MaterializationBackend.ZARR: assert plans[1].zarr_config == { - "all_wells": ("A01", "A02"), "needs_initialization": True + "all_wells": ("A01", "A02"), + "needs_initialization": True, } assert len(requests) == 1 and requests[0][1] is config else: @@ -312,7 +313,8 @@ def _compile_source_plans_for_contract( ) compiled = ( replace(execution_bindings, metadata_fields=fields) - if execution_bindings.bindings else execution_bindings + if execution_bindings.bindings + else execution_bindings ) return execution_bindings, ( compiled, @@ -357,12 +359,14 @@ def test_compilation_session_shares_resolved_pipeline_and_owns_axis_plans(monkey orchestrator=_orchestrator(), global_config=GlobalPipelineConfig(), pipeline=ResolvedPipelineDefinition( - steps=(_resolved_step( - step, - source_bindings=StepSourceBindingsConfig( - enabled=True, bindings=(NamedSourceBinding(alias="DNA"),) + steps=( + _resolved_step( + step, + source_bindings=StepSourceBindingsConfig( + enabled=True, bindings=(NamedSourceBinding(alias="DNA"),) + ), ), - ),), + ), step_scope_ids={0: step_state.scope_id}, step_provenance={0: {}}, ), @@ -413,9 +417,9 @@ def infer_fields(cls, records): source_workspace_projection=VirtualWorkspaceSourceProjection( source_refs_by_virtual_path={}, source_metadata_by_path={ - "image": DurableSourceMetadata.from_mapping({ - DECLARED_SOURCE_METADATA_FIELD: {"Dose": value} - }), + "image": DurableSourceMetadata.from_mapping( + {DECLARED_SOURCE_METADATA_FIELD: {"Dose": value}} + ), }, ), ) @@ -426,7 +430,9 @@ def infer_fields(cls, records): assert len(calls) == 2 -def test_resolved_declarations_reuse_contracts_but_keep_axis_and_author_epochs(monkeypatch): +def test_resolved_declarations_reuse_contracts_but_keep_axis_and_author_epochs( + monkeypatch, +): from openhcs.core.invocation_artifacts import InvocationContractPlan from openhcs.core.pipeline.artifact_planning import extract_artifact_declarations @@ -496,9 +502,7 @@ def test_compiler_keeps_variable_components_as_stack_source(): orchestrator=_orchestrator(), global_config=GlobalPipelineConfig(), pipeline=ResolvedPipelineDefinition( - steps=( - _resolved_step(step, variable_components=(Microscopy.Channel,)), - ), + steps=(_resolved_step(step, variable_components=(Microscopy.Channel,)),), step_scope_ids={0: "plate::functionstep_0"}, step_provenance={0: {}}, ), @@ -688,7 +692,9 @@ def test_path_planner_activates_declared_source_binding_for_pipeline_start(): assert source_binding_plan.bindings == (binding,) -def test_plate_export_contract_construction_projects_inputs_to_runtime_batch(monkeypatch): +def test_plate_export_contract_construction_projects_inputs_to_runtime_batch( + monkeypatch, +): from openhcs.interop.cellprofiler.compile_time_contracts import ( CellProfilerInvocationContractProviderFactory, ) @@ -729,7 +735,9 @@ def counted_provider(cls, pipeline): classmethod(counted_provider), ) final_pipeline = replace(session.pipeline) - assert calls == [] # Intermediate saved/path-resolved owners do not build providers. + assert ( + calls == [] + ) # Intermediate saved/path-resolved owners do not build providers. session.pipeline = final_pipeline other_context = _context() other_context.axis_id = "B01" diff --git a/tests/unit/test_compiled_execution.py b/tests/unit/test_compiled_execution.py index ed2eda228a..4e863650bb 100644 --- a/tests/unit/test_compiled_execution.py +++ b/tests/unit/test_compiled_execution.py @@ -248,7 +248,9 @@ def execute_axis(_pipeline, context, _lane, **_kwargs): return ExecutionResult.success(context.axis_id) monkeypatch.setattr(worker_execution, "_execute_single_axis_static", execute_axis) - monkeypatch.setattr(progress_types, "_progress_queue", SimpleNamespace(put=lambda _event: None)) + monkeypatch.setattr( + progress_types, "_progress_queue", SimpleNamespace(put=lambda _event: None) + ) results = worker_execution.execute_worker_lane( pipeline_definition=[object()], @@ -372,7 +374,9 @@ def execute_current_run( "_execute_single_axis_static", execute_current_run, ) - monkeypatch.setattr(progress_types, "_progress_queue", SimpleNamespace(put=lambda _event: None)) + monkeypatch.setattr( + progress_types, "_progress_queue", SimpleNamespace(put=lambda _event: None) + ) context = ProcessingContext( axis_id="A01", step_plans={ @@ -433,7 +437,9 @@ def fail_reset() -> None: ("image",), memory_type="numpy", stack=pixels ) del pixels - monkeypatch.setattr(progress_types, "_progress_queue", SimpleNamespace(put=lambda _event: None)) + monkeypatch.setattr( + progress_types, "_progress_queue", SimpleNamespace(put=lambda _event: None) + ) monkeypatch.setattr( worker_execution, "_execute_single_axis_static", @@ -489,7 +495,9 @@ def execute_axis(_pipeline, context, _lane, **_kwargs): return ExecutionResult.success(context.axis_id) monkeypatch.setattr(worker_execution, "_execute_single_axis_static", execute_axis) - monkeypatch.setattr(progress_types, "_progress_queue", SimpleNamespace(put=lambda _event: None)) + monkeypatch.setattr( + progress_types, "_progress_queue", SimpleNamespace(put=lambda _event: None) + ) results = worker_execution.execute_worker_lane( pipeline_definition=[object()], lane_axis_contexts=[ @@ -540,7 +548,9 @@ def execute_axis(_pipeline, context, _lane, **_kwargs): return ExecutionResult.success(context.axis_id) monkeypatch.setattr(worker_execution, "_execute_single_axis_static", execute_axis) - monkeypatch.setattr(progress_types, "_progress_queue", SimpleNamespace(put=lambda _event: None)) + monkeypatch.setattr( + progress_types, "_progress_queue", SimpleNamespace(put=lambda _event: None) + ) results = worker_execution.execute_worker_lane( pipeline_definition=[object()], lane_axis_contexts=[ diff --git a/tests/unit/test_completed_output_publication_lifecycle.py b/tests/unit/test_completed_output_publication_lifecycle.py index dd9746b8d8..d32331698a 100644 --- a/tests/unit/test_completed_output_publication_lifecycle.py +++ b/tests/unit/test_completed_output_publication_lifecycle.py @@ -1,4 +1,5 @@ """Completed persisted facts survive cleanup, failure and deferred publication.""" + import openhcs.core.progress.types as progress_types from types import SimpleNamespace from pathlib import Path @@ -27,39 +28,64 @@ ) from openhcs.core.steps.abstract import StepExecutionObservation from openhcs.core.steps.function_outputs import PrimaryImageMetadataTarget -from openhcs.core.virtual_workspace_metadata import VirtualWorkspaceSourceProjectionEntries +from openhcs.core.virtual_workspace_metadata import ( + VirtualWorkspaceSourceProjectionEntries, +) from openhcs.core.dataset_sources.source_schema import SourceSchemaFilenameParser from openhcs.domains.microscopy.axes import Microscopy def _facts(plate_root: Path) -> StepExecutionObservation: target = PrimaryImageMetadataTarget( - output_dir=plate_root / "images", backend="disk", dataset_root=str(plate_root), - sub_dir="images", results_dir=None, + output_dir=plate_root / "images", + backend="disk", + dataset_root=str(plate_root), + sub_dir="images", + results_dir=None, ) projection = SourcePlaneProjection( - address=OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, 1), (Microscopy.Channel, 0), (Microscopy.ZIndex, 1), (Microscopy.Timepoint, 1))), - ref=SourcePixelRef("disk", "images/saved.tif"), source_alias="saved", + address=OpenHCSPlaneAddress( + ( + (Microscopy.Well, "A01"), + (Microscopy.Site, 1), + (Microscopy.Channel, 0), + (Microscopy.ZIndex, 1), + (Microscopy.Timepoint, 1), + ) + ), + ref=SourcePixelRef("disk", "images/saved.tif"), + source_alias="saved", image_metadata=ImagePayloadMetadata(source_dtype="uint16"), ) return StepExecutionObservation( - MappingProxyType({}), source_projection_entries_by_target=MappingProxyType({ - target: VirtualWorkspaceSourceProjectionEntries.from_projection_paths( - ((projection, "images/saved.tif"),) - ) - }), + MappingProxyType({}), + source_projection_entries_by_target=MappingProxyType( + { + target: VirtualWorkspaceSourceProjectionEntries.from_projection_paths( + ((projection, "images/saved.tif"),) + ) + } + ), ) def _context(plate_root: Path, count: int) -> ProcessingContext: context = ProcessingContext( - axis_id="A01", filemanager=SimpleNamespace(exists=lambda *_args: False), - step_plans={index: SimpleNamespace( - compiled_function_pattern=None, execution_scope=FunctionStepExecutionScope.AXIS, - step_name=str(index), visualize=False, streaming_configs={}, - artifact_outputs={}, future_artifact_inputs=None, - requires_main_flow_checkpoint=lambda _plans: False, - ) for index in range(count)}, + axis_id="A01", + filemanager=SimpleNamespace(exists=lambda *_args: False), + step_plans={ + index: SimpleNamespace( + compiled_function_pattern=None, + execution_scope=FunctionStepExecutionScope.AXIS, + step_name=str(index), + visualize=False, + streaming_configs={}, + artifact_outputs={}, + future_artifact_inputs=None, + requires_main_flow_checkpoint=lambda _plans: False, + ) + for index in range(count) + }, ) context.dataset_root = plate_root context.microscope_handler = SimpleNamespace( @@ -76,15 +102,21 @@ def _context(plate_root: Path, count: int) -> ProcessingContext: def _lane() -> WorkerLaneExecutionContext: return WorkerLaneExecutionContext( - execution_id="output-facts", subject_id="plate", worker_slot="worker_0", + execution_id="output-facts", + subject_id="plate", + worker_slot="worker_0", debug_execution_policy=NoOpDebugExecutionPolicy(), worker_assignments={"worker_0": ["A01"]}, ) @pytest.mark.parametrize("cancel", [False, True]) -def test_completed_projection_facts_survive_error_or_cancel_and_omit(tmp_path, monkeypatch, cancel): - monkeypatch.setattr(progress_types, "_progress_queue", SimpleNamespace(put=lambda _event: None)) +def test_completed_projection_facts_survive_error_or_cancel_and_omit( + tmp_path, monkeypatch, cancel +): + monkeypatch.setattr( + progress_types, "_progress_queue", SimpleNamespace(put=lambda _event: None) + ) facts = _facts(tmp_path) signal = ExecutionCancellationSignal() pixel_refs = [] @@ -94,7 +126,9 @@ def process(self, context, index): pixels = np.ones((8, 8), dtype=np.float32) pixel_refs.append(weakref.ref(pixels)) context.runtime_image_stack_cache.store( - ("temporary-image",), memory_type="numpy", stack=pixels, + ("temporary-image",), + memory_type="numpy", + stack=pixels, ) if cancel: signal.request() @@ -106,8 +140,12 @@ def process(self, context, index): context = _context(tmp_path, 2) result = worker_execution._execute_axis_with_sequential_combinations( - [Save(), Fail()], [("A01", context)], _lane(), RuntimeObservationMode.OMIT, - cancellation=signal, release_axis_resources=False, + [Save(), Fail()], + [("A01", context)], + _lane(), + RuntimeObservationMode.OMIT, + cancellation=signal, + release_axis_resources=False, ) assert result.is_cancelled() if cancel else result.is_error() assert not context.runtime_value_store.observed_values @@ -115,12 +153,23 @@ def process(self, context, index): assert len(result.runtime_observation.contexts) == 1 observation = result.runtime_observation.contexts[0] assert not observation.records - assert observation.outputs.source_projection_entries_by_target == facts.source_projection_entries_by_target + assert ( + observation.outputs.source_projection_entries_by_target + == facts.source_projection_entries_by_target + ) restored = pickle.loads(pickle.dumps(result)) - assert restored.runtime_observation.contexts[0].outputs.source_projection_entries_by_target == facts.source_projection_entries_by_target + assert ( + restored.runtime_observation.contexts[ + 0 + ].outputs.source_projection_entries_by_target + == facts.source_projection_entries_by_target + ) parent = ProcessingContext(axis_id="A01") restored.runtime_observation.merge_into({"A01": parent}) - assert parent.completed_step_outputs.source_projection_entries_by_target == facts.source_projection_entries_by_target + assert ( + parent.completed_step_outputs.source_projection_entries_by_target + == facts.source_projection_entries_by_target + ) def test_same_plate_reader_uses_completed_facts_before_durable_publication(tmp_path): @@ -130,12 +179,17 @@ def test_same_plate_reader_uses_completed_facts_before_durable_publication(tmp_p assert authority.projection_if_available() is None context.record_completed_step_outputs(_facts(tmp_path)) projection = authority.projection_or_empty() - lookup = VirtualWorkspacePathLookup.from_paths("images/saved.tif", str(tmp_path / "images/saved.tif")) - assert projection.source_projection_for(lookup).image_metadata.source_dtype == "uint16" + lookup = VirtualWorkspacePathLookup.from_paths( + "images/saved.tif", str(tmp_path / "images/saved.tif") + ) + assert ( + projection.source_projection_for(lookup).image_metadata.source_dtype == "uint16" + ) assert projection.source_metadata_for(lookup)["source_alias"] == "saved" # A direct durable reader retains its published-boundary contract. direct = WorkspaceSourceProjections.from_dataset_metadata( - dataset_root=tmp_path, metadata_handler=context.microscope_handler.metadata_handler, + dataset_root=tmp_path, + metadata_handler=context.microscope_handler.metadata_handler, filemanager=context.filemanager, ) assert direct.projection_if_available() is None @@ -155,22 +209,46 @@ def test_runtime_overlay_excludes_another_output_plate_and_other_context(tmp_pat assert not authority.is_bound_to_context(other) -def test_runtime_axis_query_is_explicit_and_full_plate_outputs_remain_available(tmp_path): +def test_runtime_axis_query_is_explicit_and_full_plate_outputs_remain_available( + tmp_path, +): context = _context(tmp_path, 1) first = _facts(tmp_path) target, entries = next(iter(first.source_projection_entries_by_target.items())) projection, _ = entries.projection_paths[0] - second = replace(projection, address=OpenHCSPlaneAddress(((Microscopy.Well, "A02"), (Microscopy.Site, 1), (Microscopy.Channel, 0), (Microscopy.ZIndex, 1), (Microscopy.Timepoint, 1)))) - outputs = replace(first, source_projection_entries_by_target=MappingProxyType({ - target: VirtualWorkspaceSourceProjectionEntries.from_projection_paths(( - (projection, "images/A01.tif"), (second, "images/A02.tif"), - )), - })) + second = replace( + projection, + address=OpenHCSPlaneAddress( + ( + (Microscopy.Well, "A02"), + (Microscopy.Site, 1), + (Microscopy.Channel, 0), + (Microscopy.ZIndex, 1), + (Microscopy.Timepoint, 1), + ) + ), + ) + outputs = replace( + first, + source_projection_entries_by_target=MappingProxyType( + { + target: VirtualWorkspaceSourceProjectionEntries.from_projection_paths( + ( + (projection, "images/A01.tif"), + (second, "images/A02.tif"), + ) + ), + } + ), + ) context.record_completed_step_outputs(outputs) authority = context.runtime_source_workspace_projections full = authority.projection_or_empty() axis = authority.projection_or_empty(axis_id="A01") - assert set(full.pipeline_start_files()) == {str(tmp_path / "images/A01.tif"), str(tmp_path / "images/A02.tif")} + assert set(full.pipeline_start_files()) == { + str(tmp_path / "images/A01.tif"), + str(tmp_path / "images/A02.tif"), + } assert axis.pipeline_start_files() == (str(tmp_path / "images/A01.tif"),) # Another context shares declaration admission, never the completed facts. other = _context(tmp_path, 1) diff --git a/tests/unit/test_custom_function_lifecycle.py b/tests/unit/test_custom_function_lifecycle.py index 1a07d75e86..2317153959 100644 --- a/tests/unit/test_custom_function_lifecycle.py +++ b/tests/unit/test_custom_function_lifecycle.py @@ -30,7 +30,7 @@ def _source(name: str, expression: str = "image") -> str: def _plate_source(name: str) -> str: - return f'''from openhcs.core.artifacts import ArtifactSpec, SpecialArtifactType + return f"""from openhcs.core.artifacts import ArtifactSpec, SpecialArtifactType from openhcs.core.callable_contract import FunctionStepExecutionScope from openhcs.core.pipeline.function_contracts import ( artifact_outputs, execution_scope, runtime_bound_parameters, @@ -42,22 +42,32 @@ def _plate_source(name: str) -> str: @artifact_outputs(ArtifactSpec.output("EngineeringBundle", SpecialArtifactType)) def {name}(*, artifact_batch: RuntimeArtifactBatch): return {{"engineering.txt": b"independent ABI fixture"}} -''' +""" @pytest.mark.parametrize("persist", (True, False)) def test_custom_plate_uses_native_abi_projection_and_source_lifecycle( - isolated_custom_runtime, persist, + isolated_custom_runtime, + persist, ) -> None: - from openhcs.core.callable_contract import CallableContract, FunctionStepExecutionScope - from openhcs.core.pipeline.funcstep_contract_validator import FuncStepContractValidator - from openhcs.processing.custom_functions.runtime_registry import CustomFunctionMetadata + from openhcs.core.callable_contract import ( + CallableContract, + FunctionStepExecutionScope, + ) + from openhcs.core.pipeline.funcstep_contract_validator import ( + FuncStepContractValidator, + ) + from openhcs.processing.custom_functions.runtime_registry import ( + CustomFunctionMetadata, + ) from openhcs.processing.func_registry import get_function source = _plate_source("engineering_plate_probe") manager = CustomFunctionManager() [function] = manager.register_from_code(source, persist=persist) - metadata = CustomFunctionRuntimeRegistry.metadata_by_name()["engineering_plate_probe"] + metadata = CustomFunctionRuntimeRegistry.metadata_by_name()[ + "engineering_plate_probe" + ] assert isinstance(metadata, CustomFunctionMetadata) assert get_function(metadata.composite_key) is function contract = CallableContract.from_callable(function) @@ -65,7 +75,9 @@ def test_custom_plate_uses_native_abi_projection_and_source_lifecycle( assert not contract.declared_memory_types assert contract.processing_contract is None assert metadata.tags == ["openhcs", "custom"] - FuncStepContractValidator.validate_dataset_callable_contracts((contract,), "engineering") + FuncStepContractValidator.validate_dataset_callable_contracts( + (contract,), "engineering" + ) reference = FunctionReferenceTransportAuthority.function_reference(function) assert reference.resolve() is function @@ -79,7 +91,9 @@ def test_custom_plate_uses_native_abi_projection_and_source_lifecycle( assert manager.load_all_custom_functions() == 1 reloaded = get_function(metadata.composite_key) assert CallableContract.from_callable(reloaded).processing_contract is None - manager.update_custom_function("engineering_plate_probe", source.replace("fixture", "replacement")) + manager.update_custom_function( + "engineering_plate_probe", source.replace("fixture", "replacement") + ) with pytest.raises(RuntimeError, match="changed"): reference.resolve() else: @@ -95,7 +109,8 @@ def test_custom_plate_uses_native_abi_projection_and_source_lifecycle( ), ) def test_custom_plate_rejects_invalid_original_batch_abi_without_publication( - isolated_custom_runtime, replacement, + isolated_custom_runtime, + replacement, ) -> None: source = _plate_source("engineering_invalid_plate_probe") source = source.replace("*, artifact_batch: RuntimeArtifactBatch", replacement) @@ -110,7 +125,8 @@ def test_custom_plate_does_not_admit_axis_memory_contract( isolated_custom_runtime, ) -> None: source = _plate_source("engineering_mixed_scope_probe").replace( - "@execution_scope", "@numpy\n@execution_scope", + "@execution_scope", + "@numpy\n@execution_scope", ) with pytest.raises(ValidationError, match="cannot declare axis-local"): CustomFunctionManager().register_from_code(source) @@ -401,7 +417,9 @@ def pause_reconstruction(self, declared, contract): def isolated_custom_runtime(monkeypatch, tmp_path): storage_dir = tmp_path / "custom_functions" storage_dir.mkdir() - monkeypatch.setattr(manager_module, "get_data_file_path", lambda _name, *, create=True: storage_dir) + monkeypatch.setattr( + manager_module, "get_data_file_path", lambda _name, *, create=True: storage_dir + ) monkeypatch.setattr(CustomFunctionRuntimeRegistry, "_declarations_by_name", {}) monkeypatch.setattr(CustomFunctionRuntimeRegistry, "_published_exports", {}) monkeypatch.setattr(CustomFunctionRuntimeRegistry, "_preparation_outcomes", {}) @@ -413,11 +431,14 @@ def isolated_custom_runtime(monkeypatch, tmp_path): @pytest.mark.parametrize("persist", (False, True)) def test_registration_observation_uses_current_owners_without_loading( - isolated_custom_runtime, monkeypatch, persist, + isolated_custom_runtime, + monkeypatch, + persist, ): from dataclasses import replace from openhcs.agent.dto.functions import ( - CustomFunctionRegistrationHandle, CustomFunctionRegistrationRequest, + CustomFunctionRegistrationHandle, + CustomFunctionRegistrationRequest, CustomFunctionRegistrationObservationOutcome, ) from openhcs.agent.path_policy import AgentPathPolicy @@ -428,29 +449,60 @@ def test_registration_observation_uses_current_owners_without_loading( policy = AgentPathPolicy.with_roots(readable_roots=(root,), writable_roots=(root,)) manager = CustomFunctionManager(create_storage=False) request = CustomFunctionRegistrationRequest.from_fields( - source_code=_source("observation_probe"), function_name="observation_probe", - persist=persist, storage_dir=str(root), port=22319, + source_code=_source("observation_probe"), + function_name="observation_probe", + persist=persist, + storage_dir=str(root), + port=22319, + ) + handle = CustomFunctionRegistrationHandle.from_request( + replace(request, server_identity=ProcessIdentity.current()) ) - handle = CustomFunctionRegistrationHandle.from_request(replace(request, server_identity=ProcessIdentity.current())) service = FunctionCatalogService(path_policy=policy) before = service.observe_custom_function_registration(handle) assert before.outcome is CustomFunctionRegistrationObservationOutcome.NOT_OBSERVED - [function] = manager.register_from_code(request.source_code, persist=persist, clear_caches=False, emit_signal=False) - monkeypatch.setattr(CustomFunctionManager, "_prepare_source", lambda *_: pytest.fail("Read-only observation cannot evaluate source")) - monkeypatch.setattr(CustomFunctionManager, "load_custom_function", lambda *_a, **_k: pytest.fail("Read-only observation cannot lazy load")) + [function] = manager.register_from_code( + request.source_code, persist=persist, clear_caches=False, emit_signal=False + ) + monkeypatch.setattr( + CustomFunctionManager, + "_prepare_source", + lambda *_: pytest.fail("Read-only observation cannot evaluate source"), + ) + monkeypatch.setattr( + CustomFunctionManager, + "load_custom_function", + lambda *_a, **_k: pytest.fail("Read-only observation cannot lazy load"), + ) observed = service.observe_custom_function_registration(handle) assert observed.outcome is CustomFunctionRegistrationObservationOutcome.REGISTERED assert observed.published_sources == (handle.require_named_source(),) - assert CustomFunctionRuntimeRegistry.metadata_by_name()["observation_probe"].func is function - assert observed.persisted_source == (handle.require_named_source() if persist else None) - changed = service.observe_custom_function_registration(replace(handle, content_sha256="0" * 64)) + assert ( + CustomFunctionRuntimeRegistry.metadata_by_name()["observation_probe"].func + is function + ) + assert observed.persisted_source == ( + handle.require_named_source() if persist else None + ) + changed = service.observe_custom_function_registration( + replace(handle, content_sha256="0" * 64) + ) assert changed.outcome is CustomFunctionRegistrationObservationOutcome.NOT_OBSERVED assert not changed.published_sources with pytest.raises(RuntimeError, match="stale"): - service.observe_custom_function_registration(replace(handle, server_identity=replace(ProcessIdentity.current(), create_time=0))) + service.observe_custom_function_registration( + replace( + handle, + server_identity=replace(ProcessIdentity.current(), create_time=0), + ) + ) CustomFunctionRuntimeRegistry.remove("observation_probe") after = service.observe_custom_function_registration(handle) - assert after.outcome is (CustomFunctionRegistrationObservationOutcome.PERSISTED_ONLY if persist else CustomFunctionRegistrationObservationOutcome.NOT_OBSERVED) + assert after.outcome is ( + CustomFunctionRegistrationObservationOutcome.PERSISTED_ONLY + if persist + else CustomFunctionRegistrationObservationOutcome.NOT_OBSERVED + ) def test_register_rejects_multi_declaration_source_without_partial_publication( @@ -667,7 +719,9 @@ def load_one(): def test_failed_preparation_keeps_evidence_not_payload_or_reader_frames( isolated_custom_runtime, ) -> None: - from openhcs.processing.custom_functions.source_namespace import CustomFunctionSource + from openhcs.processing.custom_functions.source_namespace import ( + CustomFunctionSource, + ) source = CustomFunctionSource("failed_payload_probe", "failure-revision") references = [] @@ -691,7 +745,9 @@ def prepare(): errors.append(error) assert str(error) == "source failed | Line 7 | Code: raise cause" assert (error.message, error.line_number, error.code_snippet) == ( - "source failed", 7, "raise cause", + "source failed", + 7, + "raise cause", ) assert "ValueError: original source cause" in error.__notes__[0] assert "raise ValidationError" in error.__notes__[0] @@ -1086,11 +1142,17 @@ def nested_helper_transport_probe(image): pickle.dumps(helper) -@pytest.mark.parametrize("operation", ["replace", "remove", "reconcile", "clear", "stale_preparation"]) +@pytest.mark.parametrize( + "operation", ["replace", "remove", "reconcile", "clear", "stale_preparation"] +) def test_source_retirement_thaws_before_dropping_captured_owners( - isolated_custom_runtime, monkeypatch, operation, + isolated_custom_runtime, + monkeypatch, + operation, ): - from openhcs.processing.custom_functions.source_namespace import CustomFunctionSource + from openhcs.processing.custom_functions.source_namespace import ( + CustomFunctionSource, + ) manager = CustomFunctionManager() name = "startup_gc_retirement_probe" @@ -1104,7 +1166,10 @@ def unfreeze(): assert vars(custom_functions)[name] is previous.func calls.append("thaw") - monkeypatch.setattr("openhcs.processing.backends.lib_registry.registry_service.gc.unfreeze", unfreeze) + monkeypatch.setattr( + "openhcs.processing.backends.lib_registry.registry_service.gc.unfreeze", + unfreeze, + ) if operation == "replace": manager.update_custom_function(name, _source(name, "image + 1")) elif operation == "remove": diff --git a/tests/unit/test_czi_brain_axon_cellbody_demo.py b/tests/unit/test_czi_brain_axon_cellbody_demo.py index 53f02de4c0..8582ce7620 100644 --- a/tests/unit/test_czi_brain_axon_cellbody_demo.py +++ b/tests/unit/test_czi_brain_axon_cellbody_demo.py @@ -75,7 +75,15 @@ def _candidate( Microscopy.Site.name: site_label, Microscopy.Channel.name: channel_label, }, - declared_address=OpenHCSPlaneAddress(((Microscopy.Well, "."), (Microscopy.Site, site), (Microscopy.Channel, channel), (Microscopy.ZIndex, "1"), (Microscopy.Timepoint, "1"))), + declared_address=OpenHCSPlaneAddress( + ( + (Microscopy.Well, "."), + (Microscopy.Site, site), + (Microscopy.Channel, channel), + (Microscopy.ZIndex, "1"), + (Microscopy.Timepoint, "1"), + ) + ), dataset_identity=identity, store_identity=SourcePlaneStoreIdentity( container_paths=(source_path,), @@ -165,25 +173,17 @@ def test_czi_demo_declares_bounded_three_channel_analysis_and_artifacts( assert get_core_callable(steps[2].func) is tophat assert get_core_callable(steps[3].func) is skan_axon_skeletonize_and_analyze - assert steps[0].processing_config.variable_components == [ - Microscopy.Channel - ] + assert steps[0].processing_config.variable_components == [Microscopy.Channel] assert steps[0].processing_config.input_source is InputSource.PIPELINE_START - assert steps[1].processing_config.variable_components == [ - Microscopy.Channel - ] + assert steps[1].processing_config.variable_components == [Microscopy.Channel] assert steps[1].processing_config.input_source is InputSource.PREVIOUS_STEP assert steps[2].processing_config.group_by is Ungrouped - assert steps[2].processing_config.variable_components == [ - Microscopy.Channel - ] + assert steps[2].processing_config.variable_components == [Microscopy.Channel] assert steps[2].processing_config.input_source is InputSource.PIPELINE_START assert steps[2].source_bindings.enabled is True assert [binding.alias for binding in steps[2].source_bindings.bindings] == ["Axon"] assert steps[3].processing_config.group_by is Ungrouped - assert steps[3].processing_config.variable_components == [ - Microscopy.Channel - ] + assert steps[3].processing_config.variable_components == [Microscopy.Channel] assert steps[3].processing_config.input_source is InputSource.PREVIOUS_STEP count_kwargs = steps[1].func[1] diff --git a/tests/unit/test_dataset_source_family.py b/tests/unit/test_dataset_source_family.py index 145735000b..16c18aa2c0 100644 --- a/tests/unit/test_dataset_source_family.py +++ b/tests/unit/test_dataset_source_family.py @@ -11,7 +11,9 @@ from openhcs.agent.dto.plate import PlatePathInspectionRequest from openhcs.agent.path_policy import AgentPathPolicy from openhcs.agent.services.config_service import ConfigService -from openhcs.agent.services.artifact_plan_inspection_service import ArtifactPlanInspectionService +from openhcs.agent.services.artifact_plan_inspection_service import ( + ArtifactPlanInspectionService, +) from openhcs.agent.services.plate_inspection_service import PlateInspectionService from openhcs.core.config import LazyPartitionFilterConfig, PipelineConfig from openhcs.core.config_document import ConfigDocumentAuthority @@ -79,7 +81,10 @@ def test_every_registered_source_declares_its_identity_role_and_backends() -> No ) @pytest.mark.parametrize("missing", (False, True)) def test_metadata_detection_uses_each_declared_owner( - monkeypatch, tmp_path: Path, handler_type, missing: bool, + monkeypatch, + tmp_path: Path, + handler_type, + missing: bool, ) -> None: from polystore.exceptions import MetadataNotFoundError @@ -134,12 +139,16 @@ def test_config_schema_patch_and_source_share_opera_handler_identity() -> None: def test_handler_factory_uses_exact_declared_identity(tmp_path: Path) -> None: - handler = DatasetSourceChoice.named("opera_phenix").open(tmp_path, filemanager=bioformats_filemanager()) + handler = DatasetSourceChoice.named("opera_phenix").open( + tmp_path, filemanager=bioformats_filemanager() + ) assert isinstance(handler, OperaPhenixHandler) assert handler.source_name == "opera_phenix" with pytest.raises(ValueError, match="Unknown dataset source 'OperaPhenix'"): - DatasetSourceChoice.named("OperaPhenix").open(tmp_path, filemanager=bioformats_filemanager()) + DatasetSourceChoice.named("OperaPhenix").open( + tmp_path, filemanager=bioformats_filemanager() + ) def test_source_selection_role_owns_local_availability_contract( @@ -150,13 +159,9 @@ def test_source_selection_role_owns_local_availability_contract( file_path.touch() with pytest.raises(FileNotFoundError, match=str(missing_path)): - FormatSpecificSource.require_available_source( - missing_path - ) + FormatSpecificSource.require_available_source(missing_path) with pytest.raises(NotADirectoryError, match=str(file_path)): - FormatSpecificSource.require_available_source( - file_path - ) + FormatSpecificSource.require_available_source(file_path) RemoteServiceSource.require_available_source(missing_path) diff --git a/tests/unit/test_dataset_source_guards.py b/tests/unit/test_dataset_source_guards.py index 2693a55228..963daa61e5 100644 --- a/tests/unit/test_dataset_source_guards.py +++ b/tests/unit/test_dataset_source_guards.py @@ -98,7 +98,9 @@ def test_kernel_imports_no_domain_package() -> None: def test_deleted_source_names_do_not_exist() -> None: offenders = { - _relative(path): sorted(DELETED_NAMES & _identifiers(ast.parse(path.read_text()))) + _relative(path): sorted( + DELETED_NAMES & _identifiers(ast.parse(path.read_text())) + ) for root in ("openhcs", "tests", "benchmark", "scripts") for path in (REPO_ROOT / root).rglob("*.py") if "benchmark/results" not in path.as_posix() @@ -113,9 +115,7 @@ def test_kernel_spells_no_domain_root_or_filename_token() -> None: for path in _kernel_files() for node in ast.walk(ast.parse(path.read_text())) if isinstance(node, (ast.Constant, ast.JoinedStr)) - and any( - token in ast.unparse(node) for token in ('"/omero/', "'/omero/", "_w{") - ) + and any(token in ast.unparse(node) for token in ('"/omero/', "'/omero/", "_w{")) } assert offenders == set() diff --git a/tests/unit/test_debug_runtime.py b/tests/unit/test_debug_runtime.py index e12751fe23..61910405a6 100644 --- a/tests/unit/test_debug_runtime.py +++ b/tests/unit/test_debug_runtime.py @@ -177,6 +177,7 @@ def lane(cls) -> WorkerLaneExecutionContext: worker_slot="worker_0", worker_assignments={"worker_0": ("A01",)}, ) + SEGMENT_NAME = "segment" @staticmethod @@ -1001,8 +1002,7 @@ def test_debug_step_executes_one_function_pattern_invocation(): np.testing.assert_array_equal(result, np.array([2, 4, 6])) assert [ - DebugProgressContext.from_context(event.context).event_type - for event in emitted + DebugProgressContext.from_context(event.context).event_type for event in emitted ] == [DebugEventType.BEFORE_INVOCATION, DebugEventType.AFTER_INVOCATION] @@ -1038,7 +1038,9 @@ def test_debug_step_can_advance_past_current_function_pattern_invocation(): np.testing.assert_array_equal(result, np.array([2, 3, 4])) assert len(emitted) == 2 - assert DebugProgressContext.from_context(emitted[0].context).cursor.invocation_key.endswith(":1:plus_one") + assert DebugProgressContext.from_context( + emitted[0].context + ).cursor.invocation_key.endswith(":1:plus_one") def test_execute_chain_emits_debug_exception_event(): @@ -1117,10 +1119,7 @@ def test_projection_exception_contains_complete_invocation_context_without_debug assert "declared_cardinality=2" in message assert "selected_runtime_plane=preserved_stack" in message assert "execution_axis_cardinality=3" in message - assert ( - "image_payload_execution_mode=FullStackExecution" - in message - ) + assert "image_payload_execution_mode=FullStackExecution" in message assert "processing_contract=Pure2DContract" in message assert isinstance(exc.__cause__, RuntimeSliceProjectionDeclarationError) assert str(exc.__cause__) == "projection preflight mismatch" @@ -1306,7 +1305,8 @@ def test_progress_debug_event_sink_uses_injected_progress_emitter(): assert emitted[0].status is ProgressStatus.STARTED assert emitted[0].context is not None assert ( - emitted[0].context[DebugProgressContext.context_key]["debug_session_id"] == DebugRuntimeFixture.DEBUG_SESSION_ID + emitted[0].context[DebugProgressContext.context_key]["debug_session_id"] + == DebugRuntimeFixture.DEBUG_SESSION_ID ) assert ( emitted[0].context[DebugProgressContext.context_key]["snapshot_store_ref"] @@ -1484,9 +1484,9 @@ def test_debug_execution_policy_skips_before_start_step_index(): def test_debug_session_marks_current_cursor_dirty_nominally(): cursor = DebugRuntimeFixture.cursor() - session = DebugSession.create(subject_id=DebugRuntimeFixture.SUBJECT_ID).with_cursor( - cursor - ) + session = DebugSession.create( + subject_id=DebugRuntimeFixture.SUBJECT_ID + ).with_cursor(cursor) dirty_session = session.mark_dirty_from_cursor() diff --git a/tests/unit/test_declared_singleton_composition_437.py b/tests/unit/test_declared_singleton_composition_437.py index 48725ef09e..23ce5f9f5e 100644 --- a/tests/unit/test_declared_singleton_composition_437.py +++ b/tests/unit/test_declared_singleton_composition_437.py @@ -5,7 +5,10 @@ import numpy as np import pytest -from openhcs.core.aligned_image_payload import (AlignedImageSliceContext, AlignedImageStack) +from openhcs.core.aligned_image_payload import ( + AlignedImageSliceContext, + AlignedImageStack, +) from openhcs.core.callable_contract import CallableContract, ImagePayloadConsumption from openhcs.core.component_group_scope import RuntimeExecutionAxisScope from openhcs.core.function_patterns import ( @@ -197,9 +200,7 @@ def test_new_declaration_reuses_composition_and_executor(slice_count, aligned): assert metadata.source_image_names == ("FITC",) assert metadata.source_component_metadata["channel"] == "2" assert metadata.source_image_paths == (f"/synthetic/A01_s{index + 1}_w2.tif",) - np.testing.assert_array_equal( - plane.data[0], original_pixels[index] - ) + np.testing.assert_array_equal(plane.data[0], original_pixels[index]) np.testing.assert_array_equal(plane.mask, original_masks[index]) result = CellProfilerFunctionContractExecutor().execute( contract, @@ -214,23 +215,13 @@ def test_new_declaration_reuses_composition_and_executor(slice_count, aligned): result_plane = RuntimeSliceProjection.value_for_slice( result, projection.selected_plane(index) ) - np.testing.assert_array_equal( - result_plane.data, original_pixels[index] - ) - np.testing.assert_array_equal( - result_plane.mask, original_masks[index] - ) + np.testing.assert_array_equal(result_plane.data, original_pixels[index]) + np.testing.assert_array_equal(result_plane.mask, original_masks[index]) metadata = result_plane.metadata assert metadata.plane_axis is None assert metadata.axis_position(ColourAxis) is None - assert ( - metadata.source_voxel_spacing - == source.metadata.source_voxel_spacing - ) - assert ( - metadata.source_spatial_domain - == source.metadata.source_spatial_domain - ) + assert metadata.source_voxel_spacing == source.metadata.source_voxel_spacing + assert metadata.source_spatial_domain == source.metadata.source_spatial_domain np.testing.assert_array_equal(source.data, original_pixels) np.testing.assert_array_equal(source.mask, original_masks) @@ -252,10 +243,7 @@ def test_declared_composed_scalar_introduces_one_source_axis(): (scalar,), ) assert composition.execution_mode is FullStackExecution - assert ( - composition.payload.metadata.plane_axis - is RuntimePlaneAxis.SOURCE_BINDING - ) + assert composition.payload.metadata.plane_axis is RuntimePlaneAxis.SOURCE_BINDING contract = CallableContract.from_callable(independently_declared_source_echo) result = CellProfilerFunctionContractExecutor().execute( contract, @@ -264,9 +252,7 @@ def test_declared_composed_scalar_introduces_one_source_axis(): {}, execution_mode=composition.execution_mode, ) - np.testing.assert_array_equal( - result.data, source.data[0] - ) + np.testing.assert_array_equal(result.data, source.data[0]) @pytest.mark.parametrize("aligned", (False, True)) @@ -280,9 +266,7 @@ def test_natural_declaration_retains_exact_singleton(aligned): ) assert composition.payload is current assert composition.execution_mode is ( - AlignedStackExecution - if aligned - else NaturalExecution + AlignedStackExecution if aligned else NaturalExecution ) if not aligned: result = CellProfilerFunctionContractExecutor().execute( @@ -297,12 +281,8 @@ def test_natural_declaration_retains_exact_singleton(aligned): result.data, source.data * source.mask, ) - np.testing.assert_array_equal( - result.mask, source.mask - ) - assert ( - result.metadata.plane_axis is RuntimePlaneAxis.RUNTIME_SLICE - ) + np.testing.assert_array_equal(result.mask, source.mask) + assert result.metadata.plane_axis is RuntimePlaneAxis.RUNTIME_SLICE def test_kernel_still_rejects_unprojected_runtime_axis(): diff --git a/tests/unit/test_derived_role_publication_435.py b/tests/unit/test_derived_role_publication_435.py index 4acdffdfef..7ac6a14ded 100644 --- a/tests/unit/test_derived_role_publication_435.py +++ b/tests/unit/test_derived_role_publication_435.py @@ -11,57 +11,102 @@ from polystore.memory import MemoryStorageBackend from tests.unit.test_function_outputs import ( - context_stub, function_step_plan, record_output_path, + context_stub, + function_step_plan, + record_output_path, ) from openhcs.core.dataset_sources.source_schema import SourceSchemaFilenameParser from openhcs.constants.constants import Backend -from openhcs.core.aligned_image_payload import AlignedImageSliceContext, stack_image_payloads -from openhcs.core.artifacts import ArtifactOutputPlan, ImageArtifactType, ObjectLabelsArtifactType -from openhcs.core.component_group_scope import ComponentGroupScope, RuntimeExecutionAxisScope +from openhcs.core.aligned_image_payload import ( + AlignedImageSliceContext, + stack_image_payloads, +) +from openhcs.core.artifacts import ( + ArtifactOutputPlan, + ImageArtifactType, + ObjectLabelsArtifactType, +) +from openhcs.core.component_group_scope import ( + ComponentGroupScope, + RuntimeExecutionAxisScope, +) from openhcs.core.compiled_step_plan import RuntimeArtifactMaterializationPlan from openhcs.core.runtime_artifact_values import RuntimeValue from openhcs.core.runtime_image_values import ( - ImageMetadataPayload, ImagePayloadMetadata, ImagePayloadMetadataCompositionMode, + ImageMetadataPayload, + ImagePayloadMetadata, + ImagePayloadMetadataCompositionMode, ) from openhcs.core.runtime_stores import RuntimeValueStore -from openhcs.core.runtime_object_label_building import SourceImageObjectLabelBuildRequest +from openhcs.core.runtime_object_label_building import ( + SourceImageObjectLabelBuildRequest, +) from openhcs.core.source_metadata import SourceVoxelSpacing from openhcs.core.source_image_provenance import ( - SourceImageProvenance, SourceImageProvenancePlanes, + SourceImageProvenance, + SourceImageProvenancePlanes, ) from openhcs.core.source_projection import SourceProjectionMetadataSerializer from openhcs.core.source_workspace_projection import VirtualWorkspaceSourceProjection -from openhcs.core.steps.function_artifact_materialization import ArtifactMaterializationTargetPlan +from openhcs.core.steps.function_artifact_materialization import ( + ArtifactMaterializationTargetPlan, +) from openhcs.core.steps.function_outputs import ( PrimaryImageMetadataTarget, ) from openhcs.core.steps.function_output_identity import FunctionOutputIdentity -from openhcs.core.runtime_plane_projection import RuntimePlaneAxis, RuntimePlaneAxisValueProjection +from openhcs.core.runtime_plane_projection import ( + RuntimePlaneAxis, + RuntimePlaneAxisValueProjection, +) from python_introspect import to_jsonable from openhcs.processing.materialization import ( - ImageFileOptions, MaterializationSpec, MaterializedFilenameIdentity, + ImageFileOptions, + MaterializationSpec, + MaterializedFilenameIdentity, ) from openhcs.domains.microscopy.axes import Microscopy -@pytest.mark.parametrize("scenario,aggregate", ( - *((scenario, False) for scenario in ( - "same_occurrence", "different_path", "stale_scope", "conflicting_metadata", "different_kind", - )), - *((scenario, True) for scenario in ( - "same_occurrence", "different_path", "stale_scope", "conflicting_metadata", "different_kind", - )), - ("conflicting_scope", True), - ("mosaic_context", False), -)) -def test_saved_roles_publish_once_per_persisted_occurrence(tmp_path, scenario, aggregate): +@pytest.mark.parametrize( + "scenario,aggregate", + ( + *( + (scenario, False) + for scenario in ( + "same_occurrence", + "different_path", + "stale_scope", + "conflicting_metadata", + "different_kind", + ) + ), + *( + (scenario, True) + for scenario in ( + "same_occurrence", + "different_path", + "stale_scope", + "conflicting_metadata", + "different_kind", + ) + ), + ("conflicting_scope", True), + ("mosaic_context", False), + ), +) +def test_saved_roles_publish_once_per_persisted_occurrence( + tmp_path, scenario, aggregate +): mosaic = scenario == "mosaic_context" same_persisted_path = scenario != "different_path" - filemanager = FileManager({ - Backend.DISK.value: DiskStorageBackend(), - Backend.MEMORY.value: MemoryStorageBackend(), - }) + filemanager = FileManager( + { + Backend.DISK.value: DiskStorageBackend(), + Backend.MEMORY.value: MemoryStorageBackend(), + } + ) context = context_stub(filemanager, parser=SourceSchemaFilenameParser()) context.microscope_handler.source_name = "openhcsdata" context.runtime_value_store = RuntimeValueStore() @@ -75,24 +120,29 @@ def test_saved_roles_publish_once_per_persisted_occurrence(tmp_path, scenario, a plan.sub_dir = "images" plan.analysis_results_dir = str(plan.output_dir) plan.runtime_artifact_materialization = RuntimeArtifactMaterializationPlan( - persistent_enabled=True, persistent_backend=Backend.DISK.value, + persistent_enabled=True, + persistent_backend=Backend.DISK.value, ) context.step_plans = {plan.step_index: plan} filemanager.ensure_directory(plan.output_dir, Backend.MEMORY.value) components = { - "well": "A01", "site": "1", "channel": "2", - "z_index": "1", "timepoint": "1", "extension": ".tif", + "well": "A01", + "site": "1", + "channel": "2", + "z_index": "1", + "timepoint": "1", + "extension": ".tif", } roles = ( - (("MosaicRole", 1.0),) if mosaic - else (("RawRole", 1.0), ("CappedRole", 0.5)) + (("MosaicRole", 1.0),) if mosaic else (("RawRole", 1.0), ("CappedRole", 0.5)) ) if aggregate: components.update(channel="1", z_index="0") plan.variable_components = (Microscopy.ZIndex,) plan.group_by = Microscopy.Channel plan.execution_group_scope = ComponentGroupScope.from_raw( - ("1",), component=Microscopy.Channel, + ("1",), + component=Microscopy.Channel, ) source_planes = [] source_dir = tmp_path / "physical_source" @@ -101,13 +151,16 @@ def test_saved_roles_publish_once_per_persisted_occurrence(tmp_path, scenario, a for z_index, plane in enumerate(pixels): path = source_dir / f"A01_s001_w1_z{z_index:03d}_t001.tif" filemanager.save(plane, str(path), Backend.DISK.value) - source_planes.append(ImagePayloadMetadata( - source_path=str(path), - source_component_metadata={**components, "z_index": str(z_index)}, - source_voxel_spacing=SourceVoxelSpacing((2.0, 0.65, 0.65)), - ).payload_with(filemanager.load(str(path), Backend.DISK.value))) + source_planes.append( + ImagePayloadMetadata( + source_path=str(path), + source_component_metadata={**components, "z_index": str(z_index)}, + source_voxel_spacing=SourceVoxelSpacing((2.0, 0.65, 0.65)), + ).payload_with(filemanager.load(str(path), Backend.DISK.value)) + ) source_stack = stack_image_payloads( - source_planes, metadata_mode=ImagePayloadMetadataCompositionMode.STACK, + source_planes, + metadata_mode=ImagePayloadMetadataCompositionMode.STACK, ) roles = (("fixture_image_494", 1.0),) for role, scale in roles: @@ -117,29 +170,41 @@ def test_saved_roles_publish_once_per_persisted_occurrence(tmp_path, scenario, a f"_z{int(components['z_index']):03d}_t001_{role}.tif" ) output_plan = ArtifactOutputPlan( - name=role, path=f"/memory/{filename.removesuffix('.tif')}.pkl", + name=role, + path=f"/memory/{filename.removesuffix('.tif')}.pkl", artifact_type=( ObjectLabelsArtifactType - if scenario == "different_kind" and tested_role else ImageArtifactType + if scenario == "different_kind" and tested_role + else ImageArtifactType ), producer_step_index=plan.step_index, producer_step_scope_id=( "different-step-scope" - if scenario == "stale_scope" and tested_role else plan.step_scope_id + if scenario == "stale_scope" and tested_role + else plan.step_scope_id ), producer_step_name=plan.step_name, - materialization=MaterializationSpec(ImageFileOptions( - filename_suffix=".tif", - filename_identity=( - MaterializedFilenameIdentity.ARTIFACT_NAME if aggregate - else MaterializedFilenameIdentity.SOURCE_IDENTITY - ), - relative_path_template=(filename if same_persisted_path else filename.replace(".tif", "_independent.tif")), - )), + materialization=MaterializationSpec( + ImageFileOptions( + filename_suffix=".tif", + filename_identity=( + MaterializedFilenameIdentity.ARTIFACT_NAME + if aggregate + else MaterializedFilenameIdentity.SOURCE_IDENTITY + ), + relative_path_template=( + filename + if same_persisted_path + else filename.replace(".tif", "_independent.tif") + ), + ) + ), ) if aggregate: output_plan = replace( - output_plan, group_keys=("1",), group_component=Microscopy.Channel, + output_plan, + group_keys=("1",), + group_component=Microscopy.Channel, variable_components=(Microscopy.ZIndex,), ) plan.artifact_outputs[output_plan.ref()] = output_plan @@ -172,128 +237,205 @@ def test_saved_roles_publish_once_per_persisted_occurrence(tmp_path, scenario, a component_metadata=({"site": "1"}, {"site": "2"}), ), ), - ).payload_with(np.stack([ - filemanager.load(str(path), Backend.DISK.value) - for path in contributor_paths - ])) + ).payload_with( + np.stack( + [ + filemanager.load(str(path), Backend.DISK.value) + for path in contributor_paths + ] + ) + ) if aggregate: payload = ImageArtifactType.contextualize_output( - source_stack, pixels.copy(), output_plan, - RuntimePlaneAxisValueProjection(RuntimePlaneAxis.RUNTIME_SLICE, (), None, 4), + source_stack, + pixels.copy(), + output_plan, + RuntimePlaneAxisValueProjection( + RuntimePlaneAxis.RUNTIME_SLICE, (), None, 4 + ), ) if aggregate: execution_scope = RuntimeExecutionAxisScope.from_raw( - "A01", component=Microscopy.Channel, value="1", + "A01", + component=Microscopy.Channel, + value="1", fixed_component_values=( - (Microscopy.Site, "1"), (Microscopy.Timepoint, "1"), + (Microscopy.Site, "1"), + (Microscopy.Timepoint, "1"), ), ) payload = RuntimeValue.normalize_for_execution_scope( - replace(output_plan, artifact_type=ImageArtifactType), payload, + replace(output_plan, artifact_type=ImageArtifactType), + payload, execution_scope=execution_scope, ).data artifact_payload = payload if tested_role and scenario == "conflicting_metadata": conflicting_fields = ( {"source_voxel_spacing": SourceVoxelSpacing((3.0, 0.65, 0.65))} - if aggregate else {"source_image_names": ("IndependentRole",)} + if aggregate + else {"source_image_names": ("IndependentRole",)} ) artifact_payload = payload.metadata.replace_fields( **conflicting_fields ).payload_with(payload.data) if tested_role and scenario == "different_kind": artifact_payload = SourceImageObjectLabelBuildRequest( - image=payload, labels=np.ones(payload.data.shape, dtype=np.int32), - declared_object_count=1, declared_object_ids=(1,), + image=payload, + labels=np.ones(payload.data.shape, dtype=np.int32), + declared_object_count=1, + declared_object_ids=(1,), ).payload() output_identity = ( FunctionOutputIdentity.from_metadata( - context.microscope_handler.parser, payload.metadata, + context.microscope_handler.parser, + payload.metadata, variable_components=plan.variable_components, - ) if aggregate else None + ) + if aggregate + else None ) value = ( RuntimeValue.normalize_for_execution_scope( - output_plan, artifact_payload, + output_plan, + artifact_payload, execution_scope=( RuntimeExecutionAxisScope.from_raw( - "A01", component=Microscopy.Channel, value="1", + "A01", + component=Microscopy.Channel, + value="1", fixed_component_values=( - (Microscopy.Site, "2"), (Microscopy.Timepoint, "1"), + (Microscopy.Site, "2"), + (Microscopy.Timepoint, "1"), ), - ) if scenario == "conflicting_scope" else execution_scope + ) + if scenario == "conflicting_scope" + else execution_scope ), - ) if aggregate else ( + ) + if aggregate + else ( RuntimeValue.normalize_for_execution_scope( - output_plan, artifact_payload, + output_plan, + artifact_payload, execution_scope=RuntimeExecutionAxisScope.from_raw( - "A01", component=None, value=None, fixed_component_values=( - (Microscopy.Channel, "2"), (Microscopy.ZIndex, "1"), + "A01", + component=None, + value=None, + fixed_component_values=( + (Microscopy.Channel, "2"), + (Microscopy.ZIndex, "1"), (Microscopy.Timepoint, "1"), ), ), - ) if mosaic else RuntimeValue.normalize(output_plan, artifact_payload, axis_id="A01") + ) + if mosaic + else RuntimeValue.normalize( + output_plan, artifact_payload, axis_id="A01" + ) ) ) if mosaic: payload = value.data output_identity = FunctionOutputIdentity( component_values={ - key: value for key, value in components.items() + key: value + for key, value in components.items() if key not in ("site", "extension") }, filename_component_values={ - key: value for key, value in components.items() if key != "extension" + key: value + for key, value in components.items() + if key != "extension" }, - extension=".tif", source="collapsed Mosaic contributors", + extension=".tif", + source="collapsed Mosaic contributors", ) context.runtime_value_store.record( value, - path=output_plan.path, backend=Backend.MEMORY.value, + path=output_plan.path, + backend=Backend.MEMORY.value, ) destination = plan.output_dir / filename filemanager.save(payload, str(destination), Backend.MEMORY.value) filemanager.ensure_directory(plan.output_dir, Backend.DISK.value) filemanager.save(payload.data, str(destination), Backend.DISK.value) record_output_path( - context, plan, str(destination), image_metadata=payload.metadata, + context, + plan, + str(destination), + image_metadata=payload.metadata, output_context=AlignedImageSliceContext.main_flow( - output_key=role, artifact_kind=ImageArtifactType.value, + output_key=role, + artifact_kind=ImageArtifactType.value, ), identity=output_identity, ) saved = ArtifactMaterializationTargetPlan.materialize(context, plan) assert len(saved) == len(roles) - target = replace(PrimaryImageMetadataTarget.from_plan(plan), artifact_materializations=saved) + target = replace( + PrimaryImageMetadataTarget.from_plan(plan), artifact_materializations=saved + ) records = target.produced_records(context, plan) artifacts = target.runtime_artifact_projection_paths(context, plan) receipt = { "source_scope": plan.step_scope_id, - "produced": [{"path": str(record.path_under(plan.output_dir)), - "identity": asdict(record.producer_identity), - "address": record.filename_address.as_component_metadata()} for record in records], - "materialized": [{"path": output.path, - "role": item.materialization.output_plan.name, - "semantic_source": None if item.materialization.source_identity is None else { - "path": item.materialization.source_identity.path, - "component_metadata": dict(item.materialization.source_identity.component_metadata or {}), - }, - "producer_scope": item.materialization.output_plan.producer_step_scope_id, - "stored_key": repr(item.materialization.record.key)} - for item in saved for output in item.outputs_for_backend(Backend.DISK.value)], - "artifact_projections": [{"path": path, - "role": projection.projection_role.value, - "address": None if projection.address is None else projection.address.as_component_metadata(), - "alias": projection.source_alias, - "kind": projection.artifact_kind.value} - for projection, path in artifacts], + "produced": [ + { + "path": str(record.path_under(plan.output_dir)), + "identity": asdict(record.producer_identity), + "address": record.filename_address.as_component_metadata(), + } + for record in records + ], + "materialized": [ + { + "path": output.path, + "role": item.materialization.output_plan.name, + "semantic_source": ( + None + if item.materialization.source_identity is None + else { + "path": item.materialization.source_identity.path, + "component_metadata": dict( + item.materialization.source_identity.component_metadata + or {} + ), + } + ), + "producer_scope": item.materialization.output_plan.producer_step_scope_id, + "stored_key": repr(item.materialization.record.key), + } + for item in saved + for output in item.outputs_for_backend(Backend.DISK.value) + ], + "artifact_projections": [ + { + "path": path, + "role": projection.projection_role.value, + "address": ( + None + if projection.address is None + else projection.address.as_component_metadata() + ), + "alias": projection.source_alias, + "kind": projection.artifact_kind.value, + } + for projection, path in artifacts + ], "pre_join_main_flow": SourceProjectionMetadataSerializer.projection_records( - replace(target, artifact_materializations=()).produced_projection_entries(context, plan).projection_paths, + replace(target, artifact_materializations=()) + .produced_projection_entries(context, plan) + .projection_paths, + ), + "pre_join_artifacts": SourceProjectionMetadataSerializer.projection_records( + artifacts ), - "pre_join_artifacts": SourceProjectionMetadataSerializer.projection_records(artifacts), "produced_records": [to_jsonable(record) for record in records], } - (tmp_path / "persisted-occurrence-witness.json").write_text(json.dumps(receipt, indent=2)) + (tmp_path / "persisted-occurrence-witness.json").write_text( + json.dumps(receipt, indent=2) + ) produced_paths = {item["path"] for item in receipt["produced"]} materialized_paths = {item["path"] for item in receipt["materialized"]} if same_persisted_path: @@ -303,29 +445,44 @@ def test_saved_roles_publish_once_per_persisted_occurrence(tmp_path, scenario, a if mosaic: expected_sums = {780.0} elif aggregate: - expected_sums = {140.0} if scenario == "different_kind" else {float(pixels.sum())} + expected_sums = ( + {140.0} if scenario == "different_kind" else {float(pixels.sum())} + ) else: expected_sums = {20.0} if scenario == "different_kind" else {20.0, 10.0} - assert {float(tifffile.imread(item["path"]).sum()) for item in receipt["materialized"]} == expected_sums - if scenario not in ("same_occurrence", "mosaic_context") and not (aggregate and scenario == "different_path"): + assert { + float(tifffile.imread(item["path"]).sum()) for item in receipt["materialized"] + } == expected_sums + if scenario not in ("same_occurrence", "mosaic_context") and not ( + aggregate and scenario == "different_path" + ): message = ( "Conflicting metadata for persisted image" - if scenario in ("conflicting_metadata", "conflicting_scope", "mosaic_context") + if scenario + in ("conflicting_metadata", "conflicting_scope", "mosaic_context") else "Duplicate source projection address" ) with pytest.raises(ValueError, match=message) as rejection: target.produced_projection_entries(context, plan) receipt["rejection"] = str(rejection.value) - (tmp_path / "persisted-occurrence-witness.json").write_text(json.dumps(receipt, indent=2)) + (tmp_path / "persisted-occurrence-witness.json").write_text( + json.dumps(receipt, indent=2) + ) return if aggregate: (main_flow,) = receipt["pre_join_main_flow"] (artifact,) = receipt["pre_join_artifacts"] assert main_flow["address"] is artifact["address"] is None - assert main_flow["execution_scope"] == artifact["execution_scope"] == { - "axis_id": "A01", "component": "channel", "value": "1", - "fixed_component_values": [["site", "1"], ["timepoint", "1"]], - } + assert ( + main_flow["execution_scope"] + == artifact["execution_scope"] + == { + "axis_id": "A01", + "component": "channel", + "value": "1", + "fixed_component_values": [["site", "1"], ["timepoint", "1"]], + } + ) assert main_flow["image_metadata"] == artifact["image_metadata"] assert main_flow["source_metadata"] == artifact["source_metadata"] assert receipt["produced"][0]["address"]["z_index"] == "0" @@ -333,14 +490,20 @@ def test_saved_roles_publish_once_per_persisted_occurrence(tmp_path, scenario, a metadata = main_flow["image_metadata"] assert metadata["source_dtype"] == "uint16" assert metadata["source_voxel_spacing"] == { - "values_zyx": [2.0, 0.65, 0.65], "unit": "micrometers", + "values_zyx": [2.0, 0.65, 0.65], + "unit": "micrometers", } provenance = metadata["source_provenance"] assert "z_index" not in provenance["source_component_metadata"] - assert [plane["component_metadata"]["z_index"] for plane in - provenance["source_image_provenance_planes"]] == ["0", "1", "2", "3"] - assert [plane["path"] for plane in provenance["source_image_provenance_planes"]] == [ - str(source_dir / f"A01_s001_w1_z{z_index:03d}_t001.tif") for z_index in range(4) + assert [ + plane["component_metadata"]["z_index"] + for plane in provenance["source_image_provenance_planes"] + ] == ["0", "1", "2", "3"] + assert [ + plane["path"] for plane in provenance["source_image_provenance_planes"] + ] == [ + str(source_dir / f"A01_s001_w1_z{z_index:03d}_t001.tif") + for z_index in range(4) ] assert metadata["source_spatial_domain"]["spatial_dimensions"] == 2 for path in produced_paths | materialized_paths: @@ -356,20 +519,32 @@ def test_saved_roles_publish_once_per_persisted_occurrence(tmp_path, scenario, a assert main_flow["image_metadata"] == artifact["image_metadata"] assert main_flow["source_metadata"] == artifact["source_metadata"] assert receipt["produced"][0]["address"]["site"] == "1" - assert "site" not in receipt["materialized"][0]["semantic_source"]["component_metadata"] + assert ( + "site" + not in receipt["materialized"][0]["semantic_source"]["component_metadata"] + ) structured = SourceProjectionMetadataSerializer.projection_fields( structured.projection_paths ) occurrence_count = 2 if aggregate and scenario == "different_path" else 1 assert len(structured["source_projection"]) == len(roles) * occurrence_count - assert {item["source_alias"] for item in structured["source_projection"]} == {role for role, _ in roles} + assert {item["source_alias"] for item in structured["source_projection"]} == { + role for role, _ in roles + } target.write(context, produced_plan=plan) document = json.loads((tmp_path / "openhcs_metadata.json").read_text()) - reopened = VirtualWorkspaceSourceProjection.from_openhcs_metadata(tmp_path, document) + reopened = VirtualWorkspaceSourceProjection.from_openhcs_metadata( + tmp_path, document + ) projections = reopened.source_projections_by_virtual_path - relative_paths = {path.removeprefix(str(tmp_path) + "/") for path in produced_paths | materialized_paths} + relative_paths = { + path.removeprefix(str(tmp_path) + "/") + for path in produced_paths | materialized_paths + } assert set(projections) == relative_paths | produced_paths | materialized_paths - assert {projection.source_alias for projection in projections.values()} == {role for role, _ in roles} + assert {projection.source_alias for projection in projections.values()} == { + role for role, _ in roles + } for relative_path in relative_paths: projection = projections[relative_path] if mosaic: @@ -380,15 +555,26 @@ def test_saved_roles_publish_once_per_persisted_occurrence(tmp_path, scenario, a assert metadata.source_image_provenance_planes.contributor_count == 2 assert metadata.source_voxel_spacing.values_zyx == (0.5, 0.5) assert projection.execution_scope.fixed_component_values == ( - (Microscopy.Channel, "2"), (Microscopy.ZIndex, "1"), + (Microscopy.Channel, "2"), + (Microscopy.ZIndex, "1"), (Microscopy.Timepoint, "1"), ) - np.testing.assert_array_equal(tifffile.imread(tmp_path / relative_path), mosaic_pixels) + np.testing.assert_array_equal( + tifffile.imread(tmp_path / relative_path), mosaic_pixels + ) continue if aggregate: assert projection.address is None - np.testing.assert_array_equal(tifffile.imread(tmp_path / relative_path), pixels) + np.testing.assert_array_equal( + tifffile.imread(tmp_path / relative_path), pixels + ) continue - assert projection.address.as_component_metadata() == {key: components[key] for key in ("well", "site", "channel", "z_index", "timepoint")} + assert projection.address.as_component_metadata() == { + key: components[key] + for key in ("well", "site", "channel", "z_index", "timepoint") + } expected_scale = 1.0 if projection.source_alias == "RawRole" else 0.5 - np.testing.assert_array_equal(tifffile.imread(tmp_path / relative_path), np.ones((4, 5), dtype=np.float32) * expected_scale) + np.testing.assert_array_equal( + tifffile.imread(tmp_path / relative_path), + np.ones((4, 5), dtype=np.float32) * expected_scale, + ) diff --git a/tests/unit/test_documented_dimensionality.py b/tests/unit/test_documented_dimensionality.py index 5a2e513f1a..8ae8c71c0c 100644 --- a/tests/unit/test_documented_dimensionality.py +++ b/tests/unit/test_documented_dimensionality.py @@ -3,7 +3,9 @@ from pathlib import Path from openhcs.core.callable_contract import CallableContract -from openhcs.core.pipeline.function_contracts import object_label_input_execution_mode_from_callable +from openhcs.core.pipeline.function_contracts import ( + object_label_input_execution_mode_from_callable, +) from openhcs.processing.backends.cellprofiler.area_occupied import ( measure_image_volume_occupied_binary, measure_image_volume_occupied_objects, diff --git a/tests/unit/test_experimental_analysis_engine.py b/tests/unit/test_experimental_analysis_engine.py index 1109908af6..e199b58504 100644 --- a/tests/unit/test_experimental_analysis_engine.py +++ b/tests/unit/test_experimental_analysis_engine.py @@ -81,7 +81,9 @@ def test_desktop_workflow_does_not_copy_config_filenames_or_use_legacy_runner() def test_engine_projects_declaration_owned_sheet_names(monkeypatch) -> None: - from openhcs.processing.backends.experimental_analysis import analysis as experimental_analysis + from openhcs.processing.backends.experimental_analysis import ( + analysis as experimental_analysis, + ) config = ExperimentalAnalysisConfig( design_sheet_name="design", @@ -110,7 +112,9 @@ def test_engine_projects_declaration_owned_sheet_names(monkeypatch) -> None: def test_engine_delegates_to_declared_normalization_method(monkeypatch) -> None: - from openhcs.processing.backends.experimental_analysis import analysis as experimental_analysis + from openhcs.processing.backends.experimental_analysis import ( + analysis as experimental_analysis, + ) engine = ExperimentalAnalysisEngine( ExperimentalAnalysisConfig(normalization_method=NormalizationMethod.Z_SCORE) diff --git a/tests/unit/test_exported_relationship_equivalence.py b/tests/unit/test_exported_relationship_equivalence.py index 10b79f331c..13b1707ce0 100644 --- a/tests/unit/test_exported_relationship_equivalence.py +++ b/tests/unit/test_exported_relationship_equivalence.py @@ -1,5 +1,5 @@ - from tests.unit.saved_output_dialect import SAVED_OUTPUT_POLICY + """Saved directed edges must retain the native object-row correlations.""" from dataclasses import replace @@ -61,12 +61,17 @@ def _edit_table(snapshot: RuntimeOutputSnapshot, stem: str, **changes): return replace( snapshot, tables=tuple( - replace( - table, - **{name: change(getattr(table, name)) for name, change in changes.items()}, + ( + replace( + table, + **{ + name: change(getattr(table, name)) + for name, change in changes.items() + }, + ) + if table.path.stem == stem + else table ) - if table.path.stem == stem - else table for table in snapshot.tables ), ) @@ -85,7 +90,9 @@ def test_saved_relationship_edges_join_native_parent_and_zero_child_counts( candidate = RuntimeMeasurementSnapshot.from_output_snapshot( _saved_outputs(tmp_path / "openhcs", edges=True), policy=SAVED_OUTPUT_POLICY ) - assert runtime_measurement_equivalence(reference, candidate, policy=SAVED_OUTPUT_POLICY).is_equivalent + assert runtime_measurement_equivalence( + reference, candidate, policy=SAVED_OUTPUT_POLICY + ).is_equivalent def test_relationship_endpoint_permutation_cannot_hide_in_equal_value_histograms( @@ -95,9 +102,12 @@ def test_relationship_endpoint_permutation_cannot_hide_in_equal_value_histograms _saved_outputs(tmp_path / "native", edges=False), policy=SAVED_OUTPUT_POLICY ) candidate = RuntimeMeasurementSnapshot.from_output_snapshot( - _saved_outputs(tmp_path / "openhcs", edges=True, swapped=True), policy=SAVED_OUTPUT_POLICY + _saved_outputs(tmp_path / "openhcs", edges=True, swapped=True), + policy=SAVED_OUTPUT_POLICY, ) - assert not runtime_measurement_equivalence(reference, candidate, policy=SAVED_OUTPUT_POLICY).is_equivalent + assert not runtime_measurement_equivalence( + reference, candidate, policy=SAVED_OUTPUT_POLICY + ).is_equivalent def test_relationship_rows_must_agree_with_explicit_parent_measurements( @@ -115,7 +125,9 @@ def test_relationship_rows_must_agree_with_explicit_parent_measurements( ), ) with pytest.raises(ValueError, match="relationship|Relationship"): - RuntimeMeasurementSnapshot.from_output_snapshot(snapshot, policy=SAVED_OUTPUT_POLICY) + RuntimeMeasurementSnapshot.from_output_snapshot( + snapshot, policy=SAVED_OUTPUT_POLICY + ) def test_ordinary_image_row_conflicts_remain_rejected(tmp_path: Path) -> None: @@ -210,7 +222,9 @@ def test_saved_edge_schema_and_coverage_rejects_malformed_inputs( snapshot, "Relationships", header=lambda _: header, rows=lambda _: rows ) with pytest.raises((ValueError, TypeError), match=message): - RuntimeMeasurementSnapshot.from_output_snapshot(snapshot, policy=SAVED_OUTPUT_POLICY) + RuntimeMeasurementSnapshot.from_output_snapshot( + snapshot, policy=SAVED_OUTPUT_POLICY + ) def test_saved_relationship_counts_include_noncontiguous_zero_child_parents( @@ -223,19 +237,28 @@ def test_saved_relationship_counts_include_noncontiguous_zero_child_parents( snapshot, "Cells", rows=lambda rows: tuple( - tuple("30" if i == 1 and value == "3" else value for i, value in enumerate(row)) + tuple( + "30" if i == 1 and value == "3" else value + for i, value in enumerate(row) + ) for row in rows ), ) for snapshot in (native, candidate) ) - a = RuntimeMeasurementSnapshot.from_output_snapshot(native, policy=SAVED_OUTPUT_POLICY) - b = RuntimeMeasurementSnapshot.from_output_snapshot(candidate, policy=SAVED_OUTPUT_POLICY) + a = RuntimeMeasurementSnapshot.from_output_snapshot( + native, policy=SAVED_OUTPUT_POLICY + ) + b = RuntimeMeasurementSnapshot.from_output_snapshot( + candidate, policy=SAVED_OUTPUT_POLICY + ) assert ( a.correlated_relationships is not None and b.correlated_relationships is not None ) - assert runtime_measurement_equivalence(a, b, policy=SAVED_OUTPUT_POLICY).is_equivalent + assert runtime_measurement_equivalence( + a, b, policy=SAVED_OUTPUT_POLICY + ).is_equivalent def test_snapshot_cache_retains_known_correlations_and_unknown_typed_scope( @@ -290,7 +313,9 @@ def test_saved_relationship_omitted_slice_count_preserves_optional_payload_contr header=lambda header: header[:-1], rows=lambda rows: tuple(row[:-1] for row in rows), ) - result = RuntimeMeasurementSnapshot.from_output_snapshot(snapshot, policy=SAVED_OUTPUT_POLICY) + result = RuntimeMeasurementSnapshot.from_output_snapshot( + snapshot, policy=SAVED_OUTPUT_POLICY + ) assert result.correlated_relationships is not None @@ -305,8 +330,12 @@ def test_saved_relationship_row_order_does_not_change_directed_correlation( "Relationships", rows=lambda rows: rows[:4] + rows[4:][::-1], ) - result = RuntimeMeasurementSnapshot.from_output_snapshot(snapshot, policy=SAVED_OUTPUT_POLICY) - assert runtime_measurement_equivalence(native, result, policy=SAVED_OUTPUT_POLICY).is_equivalent + result = RuntimeMeasurementSnapshot.from_output_snapshot( + snapshot, policy=SAVED_OUTPUT_POLICY + ) + assert runtime_measurement_equivalence( + native, result, policy=SAVED_OUTPUT_POLICY + ).is_equivalent def test_unparented_zero_edges_still_validate_actual_child_domain( @@ -322,7 +351,9 @@ def test_unparented_zero_edges_still_validate_actual_child_domain( "Relationships", rows=lambda rows: tuple((*row[:6], "0", *row[7:]) for row in rows), ) - result = RuntimeMeasurementSnapshot.from_output_snapshot(snapshot, policy=SAVED_OUTPUT_POLICY) + result = RuntimeMeasurementSnapshot.from_output_snapshot( + snapshot, policy=SAVED_OUTPUT_POLICY + ) assert result.correlated_relationships is not None assert all( not graph.source_keys for graph in result.correlated_relationships.values() @@ -333,7 +364,9 @@ def test_unparented_zero_edges_still_validate_actual_child_domain( rows=lambda rows: tuple((*row[:7], "999", *row[8:]) for row in rows), ) with pytest.raises(ValueError, match="absent child"): - RuntimeMeasurementSnapshot.from_output_snapshot(snapshot, policy=SAVED_OUTPUT_POLICY) + RuntimeMeasurementSnapshot.from_output_snapshot( + snapshot, policy=SAVED_OUTPUT_POLICY + ) @pytest.mark.parametrize( @@ -348,7 +381,9 @@ def test_unparented_zero_edges_still_validate_actual_child_domain( def test_child_count_parser_and_core_declaration_share_original_grammar( feature: str, identity: str | None ) -> None: - from openhcs.interop.cellprofiler.measurement_lookup import ChildCountFeatureDeclaration + from openhcs.interop.cellprofiler.measurement_lookup import ( + ChildCountFeatureDeclaration, + ) from openhcs.interop.cellprofiler.measurement_lookup import ( CellProfilerChildCountFeatureParser, ) @@ -397,7 +432,9 @@ def test_native_parent_rows_require_real_integral_parent_domain( rows=lambda rows: tuple((*row[:2], parent_value) for row in rows), ) with pytest.raises(ValueError, match=message): - RuntimeMeasurementSnapshot.from_output_snapshot(snapshot, policy=SAVED_OUTPUT_POLICY) + RuntimeMeasurementSnapshot.from_output_snapshot( + snapshot, policy=SAVED_OUTPUT_POLICY + ) def test_native_child_count_permutation_cannot_hide_in_equal_histograms( @@ -406,10 +443,14 @@ def test_native_child_count_permutation_cannot_hide_in_equal_histograms( snapshot = _edit_table( _saved_outputs(tmp_path / "native", edges=False), "Cells", - rows=lambda rows: tuple((*row[:2], "0" if row[1] == "1" else "1") for row in rows), + rows=lambda rows: tuple( + (*row[:2], "0" if row[1] == "1" else "1") for row in rows + ), ) with pytest.raises(ValueError, match="disagree with.*children_nuclei_count"): - RuntimeMeasurementSnapshot.from_output_snapshot(snapshot, policy=SAVED_OUTPUT_POLICY) + RuntimeMeasurementSnapshot.from_output_snapshot( + snapshot, policy=SAVED_OUTPUT_POLICY + ) def test_snapshot_json_cache_retains_correlation_scope(tmp_path: Path) -> None: diff --git a/tests/unit/test_exported_table_axis_snapshot.py b/tests/unit/test_exported_table_axis_snapshot.py index aaf5d67d97..df5369e665 100644 --- a/tests/unit/test_exported_table_axis_snapshot.py +++ b/tests/unit/test_exported_table_axis_snapshot.py @@ -1,10 +1,10 @@ - from tests.unit.saved_output_dialect import SAVED_OUTPUT_POLICY from benchmark.equivalence.outputs import ExportedTableAxis from openhcs.interop.cellprofiler.measurement_dialect import ( CELLPROFILER_MEASUREMENT_DIALECT, ) + """Exporter-owned row domains use physical paths before semantic namespaces.""" from pathlib import Path @@ -56,7 +56,8 @@ def test_prefixed_axis_snapshot_preserves_exact_relationships(tmp_path, prefix, candidate = _exports(tmp_path / "candidate", prefix, (1, 2)) physical_contents = {path: path.read_bytes() for path in candidate.output_files} snapshot = RuntimeOutputSnapshot.from_export_observation( - candidate.for_execution_axis(axis), execution_axis=ExportedTableAxis(axis, CELLPROFILER_MEASUREMENT_DIALECT) + candidate.for_execution_axis(axis), + execution_axis=ExportedTableAxis(axis, CELLPROFILER_MEASUREMENT_DIALECT), ) assert all(len(table.rows) == 1 for table in snapshot.tables) number = "1" if axis == "W001" else "2" @@ -67,9 +68,12 @@ def test_prefixed_axis_snapshot_preserves_exact_relationships(tmp_path, prefix, "Image.csv", } reference = RuntimeMeasurementSnapshot.from_output_snapshot( - RuntimeOutputSnapshot.from_export_observation(native), policy=SAVED_OUTPUT_POLICY + RuntimeOutputSnapshot.from_export_observation(native), + policy=SAVED_OUTPUT_POLICY, + ) + actual = RuntimeMeasurementSnapshot.from_output_snapshot( + snapshot, policy=SAVED_OUTPUT_POLICY ) - actual = RuntimeMeasurementSnapshot.from_output_snapshot(snapshot, policy=SAVED_OUTPUT_POLICY) assert ( actual.required_relationship_correlations() == reference.required_relationship_correlations() @@ -84,10 +88,13 @@ def test_prefixed_axis_snapshot_still_rejects_absent_parent(tmp_path): path = tmp_path / "candidate/experiment_Children.csv" path.write_text("ImageNumber,ObjectNumber,Parent_Parents\n1,11,7\n2,11,8\n") snapshot = RuntimeOutputSnapshot.from_export_observation( - candidate.for_execution_axis("W002"), execution_axis=ExportedTableAxis("W002", CELLPROFILER_MEASUREMENT_DIALECT) + candidate.for_execution_axis("W002"), + execution_axis=ExportedTableAxis("W002", CELLPROFILER_MEASUREMENT_DIALECT), ) with pytest.raises(ValueError, match="absent parent endpoint"): - RuntimeMeasurementSnapshot.from_output_snapshot(snapshot, policy=SAVED_OUTPUT_POLICY) + RuntimeMeasurementSnapshot.from_output_snapshot( + snapshot, policy=SAVED_OUTPUT_POLICY + ) def test_prefixed_axis_snapshot_still_rejects_unowned_image_id(tmp_path): @@ -96,5 +103,6 @@ def test_prefixed_axis_snapshot_still_rejects_unowned_image_id(tmp_path): path.write_text("ImageNumber,ObjectNumber,Parent_Parents\n1,11,7\n3,11,7\n") with pytest.raises(ValueError, match="unowned image identity"): RuntimeOutputSnapshot.from_export_observation( - candidate.for_execution_axis("W002"), execution_axis=ExportedTableAxis("W002", CELLPROFILER_MEASUREMENT_DIALECT) + candidate.for_execution_axis("W002"), + execution_axis=ExportedTableAxis("W002", CELLPROFILER_MEASUREMENT_DIALECT), ) diff --git a/tests/unit/test_fast_speckle_opening.py b/tests/unit/test_fast_speckle_opening.py index 4464708d5e..5c617c366f 100644 --- a/tests/unit/test_fast_speckle_opening.py +++ b/tests/unit/test_fast_speckle_opening.py @@ -25,11 +25,13 @@ def reflected_opening_reference(image, footprint): """Explicit reflect-index oracle, independent of either native filter.""" padding = tuple((size // 2, size // 2) for size in footprint.shape) erode_windows = np.lib.stride_tricks.sliding_window_view( - np.pad(image, padding, mode="symmetric"), footprint.shape, + np.pad(image, padding, mode="symmetric"), + footprint.shape, ) eroded = erode_windows[..., footprint.astype(bool)].min(axis=-1) dilate_windows = np.lib.stride_tricks.sliding_window_view( - np.pad(eroded, padding, mode="symmetric"), footprint.shape, + np.pad(eroded, padding, mode="symmetric"), + footprint.shape, ) return dilate_windows[..., footprint.astype(bool)].max(axis=-1) @@ -43,13 +45,18 @@ def test_declared_fast_speckles_matches_original_disk_opening(shape, dtype, radi image[-1, -1] = -5 footprint = morphology.disk(max(1, round(radius))) expected = image - ndimage.maximum_filter( - ndimage.minimum_filter(image, footprint=footprint), footprint=footprint, + ndimage.minimum_filter(image, footprint=footprint), + footprint=footprint, ) actual = enhance_or_suppress_features( - image, radius=radius, speckle_accuracy=SpeckleAccuracy.FAST, + image, + radius=radius, + speckle_accuracy=SpeckleAccuracy.FAST, dtype_config=DtypeConfig(), ) - np.testing.assert_array_equal(ImagePayload.of(actual).data, expected.astype(np.float32)) + np.testing.assert_array_equal( + ImagePayload.of(actual).data, expected.astype(np.float32) + ) assert ImagePayload.of(actual).data.dtype == np.float32 @@ -66,11 +73,15 @@ def test_speckle_mask_background_and_metadata_remain_owned(accuracy, masked): masked_image = np.where(mask, image, 0) footprint = morphology.disk(5) expected = masked_image - ndimage.maximum_filter( - ndimage.minimum_filter(masked_image, footprint=footprint), footprint=footprint, + ndimage.minimum_filter(masked_image, footprint=footprint), + footprint=footprint, ) expected[~mask] = image[~mask] actual = enhance_or_suppress_features( - original, radius=5, speckle_accuracy=accuracy, dtype_config=DtypeConfig(), + original, + radius=5, + speckle_accuracy=accuracy, + dtype_config=DtypeConfig(), ) np.testing.assert_array_equal(ImagePayload.of(actual).data, expected) np.testing.assert_array_equal(actual.mask, mask) @@ -80,15 +91,22 @@ def test_speckle_mask_background_and_metadata_remain_owned(accuracy, masked): def test_fast_speckles_registered_callable_keeps_independent_planes(): image = np.random.default_rng(616).uniform(0, 1, (2, 17, 23)).astype(np.float32) - expected = np.stack([ - plane - ndimage.maximum_filter( - ndimage.minimum_filter(plane, footprint=morphology.disk(5)), - footprint=morphology.disk(5), - ) for plane in image - ]) + expected = np.stack( + [ + plane + - ndimage.maximum_filter( + ndimage.minimum_filter(plane, footprint=morphology.disk(5)), + footprint=morphology.disk(5), + ) + for plane in image + ] + ) actual = enhance_or_suppress_features(image, radius=5, dtype_config=DtypeConfig()) np.testing.assert_array_equal(ImagePayload.of(actual).data, expected) - assert CallableContract.from_callable(enhance_or_suppress_features).processing_contract is Pure2DContract + assert ( + CallableContract.from_callable(enhance_or_suppress_features).processing_contract + is Pure2DContract + ) @pytest.mark.parametrize("shape,radius", [((1, 9), 150), ((5, 7), 12), ((17, 23), 5)]) @@ -99,8 +117,12 @@ def test_fast_speckles_matches_reflected_opening_math(shape, radius, dtype): image[-1, -1] = -5 before = image.copy() expected = image - reflected_opening_reference(image, morphology.disk(radius)) - actual = enhance_or_suppress_features(image, radius=radius, dtype_config=DtypeConfig()) - np.testing.assert_array_equal(ImagePayload.of(actual).data, expected.astype(np.float32)) + actual = enhance_or_suppress_features( + image, radius=radius, dtype_config=DtypeConfig() + ) + np.testing.assert_array_equal( + ImagePayload.of(actual).data, expected.astype(np.float32) + ) np.testing.assert_array_equal(image, before) @@ -114,7 +136,8 @@ def test_fast_oversized_opening_preserves_constant_negative_plane(): def test_speckle_module_keeps_original_feature_size_binding(): bound = BoundModuleSettings({"feature_size": 11}) actual = EnhanceOrSuppressFeaturesModule.postprocess_bound_settings( - ModuleBlock("EnhanceOrSuppressFeatures", 1), bound, + ModuleBlock("EnhanceOrSuppressFeatures", 1), + bound, ) assert actual.kwargs == {"radius": 5.5} assert actual.unmapped_kwargs == bound.unmapped_kwargs diff --git a/tests/unit/test_fiji_viewer_server.py b/tests/unit/test_fiji_viewer_server.py index 29f16608a4..fc0e3f0ed2 100644 --- a/tests/unit/test_fiji_viewer_server.py +++ b/tests/unit/test_fiji_viewer_server.py @@ -61,6 +61,7 @@ ) from tests.unit.viewer_axes_fixture import STREAM_AXES + def fiji_control_server(*, settlement=None, ij=None, process_launch=None): """A Fiji server with only the state its control actions read.""" @@ -89,8 +90,11 @@ def fiji_control_server(*, settlement=None, ij=None, process_launch=None): "artifact_kind": None, } -ACK_ROUTE = AckReturnRoute("tcp://127.0.0.1:8111", "00000000-0000-0000-0000-000000000001", - ProcessIdentity.current()) +ACK_ROUTE = AckReturnRoute( + "tcp://127.0.0.1:8111", + "00000000-0000-0000-0000-000000000001", + ProcessIdentity.current(), +) def _component_value_domain(payload: dict) -> ViewerComponentValueDomainPayload: @@ -223,14 +227,6 @@ def test_fiji_server_stop_closes_transport_before_owned_jvm(monkeypatch) -> None assert server.ij is None - - - - - - - - def test_fiji_settlement_tracks_queued_and_bounded_native_work() -> None: settlement = FijiBatchSettlementState() @@ -326,8 +322,6 @@ def test_fiji_window_registry_closes_replaced_hyperstack() -> None: assert registry.open_hyperstack("A14") is new_image_plus - - def test_fiji_macro_control_executes_inside_managed_imagej_runtime(tmp_path) -> None: macro_path = tmp_path / "copy.ijm" macro_path.write_text("// managed macro", encoding="utf-8") @@ -656,7 +650,9 @@ def test_fiji_payload_local_plane_axis_rejects_malformed_declarations() -> None: } with pytest.raises(ValueError, match="exactly one component declaration"): - FijiPayloadLocalPlaneExpander(STREAM_AXES).expand_item(FijiWireItem.from_payload(payload)) + FijiPayloadLocalPlaneExpander(STREAM_AXES).expand_item( + FijiWireItem.from_payload(payload) + ) payload_without_axis = dict(payload) payload_without_axis["plane_component_values"] = {"site": [1, 2]} diff --git a/tests/unit/test_funcstep_contract_validator.py b/tests/unit/test_funcstep_contract_validator.py index 22871cd1f4..19e5d3b4dd 100644 --- a/tests/unit/test_funcstep_contract_validator.py +++ b/tests/unit/test_funcstep_contract_validator.py @@ -326,7 +326,8 @@ def process(image): contract = CallableContract.from_callable(process) with pytest.raises( - ValueError, match=r"requires variable_components with role TimeAxis \(timepoint\)" + ValueError, + match=r"requires variable_components with role TimeAxis \(timepoint\)", ): FuncStepContractValidator.validate_required_axis_roles( (), @@ -776,5 +777,7 @@ def process(image): ), ) - with pytest.raises(ValueError, match="requires variable_components with role TimeAxis"): + with pytest.raises( + ValueError, match="requires variable_components with role TimeAxis" + ): FuncStepContractValidator.validate_funcstep(step) diff --git a/tests/unit/test_function_artifact_materialization.py b/tests/unit/test_function_artifact_materialization.py index 17096f587f..8d7b4f565b 100644 --- a/tests/unit/test_function_artifact_materialization.py +++ b/tests/unit/test_function_artifact_materialization.py @@ -428,7 +428,6 @@ def _plan( AxisFamily.active().named(group_by_value) if group_by_value is not None else Ungrouped - ), execution_group_scope=execution_group_scope, variable_components=variable_components, @@ -649,7 +648,9 @@ def test_slice_aligned_object_label_arrays_preserve_source_slice_metadata(): ) label_slices = RuntimeSliceAlignedValues(tuple(expected_labels)) - contextualized = (ImageArtifactType if output_plan is None else output_plan.artifact_type).contextualize_output( + contextualized = ( + ImageArtifactType if output_plan is None else output_plan.artifact_type + ).contextualize_output( source, label_slices, output_plan, @@ -728,7 +729,9 @@ def test_image_outputs_merge_source_provenance_when_output_already_has_metadata( metadata=ImagePayloadMetadata(source_dtype="float32"), ) - contextualized = (ImageArtifactType if output_plan is None else output_plan.artifact_type).contextualize_output( + contextualized = ( + ImageArtifactType if output_plan is None else output_plan.artifact_type + ).contextualize_output( source, output, output_plan, @@ -784,7 +787,9 @@ def test_object_label_payload_stack_preserves_source_slice_metadata(): ), ) - contextualized = (ImageArtifactType if output_plan is None else output_plan.artifact_type).contextualize_output( + contextualized = ( + ImageArtifactType if output_plan is None else output_plan.artifact_type + ).contextualize_output( source, labels, output_plan, @@ -809,9 +814,7 @@ def test_object_label_payload_stack_preserves_source_slice_metadata(): assert runtime_value.data.source_spatial_domain == SourceSpatialDomain( value_name="Object-label" ) - assert runtime_value.data.source_provenance == ( - source.metadata.source_provenance - ) + assert runtime_value.data.source_provenance == (source.metadata.source_provenance) assert runtime_value.data.dimensions == () assert runtime_value.data.source_image_name is None assert runtime_value.materialization_payload() is runtime_value.data @@ -851,7 +854,9 @@ def fake_materialize(_spec, data, path, *_args, **_kwargs): _observe_materialization(monkeypatch, fake_materialize) - PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs(filemanager, _plan(output_plan), context) + PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( + filemanager, _plan(output_plan), context + ) assert materialized == [ ({"x": "from-runtime"}, "/analysis/A01_positions_step7.roi.zip") @@ -892,13 +897,17 @@ def fake_materialize(_spec, data, path, *_args, **_kwargs): _observe_materialization(monkeypatch, fake_materialize) - PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs(filemanager, _plan( + PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( + filemanager, + _plan( output_plan, variable_components=( Microscopy.ZIndex, Microscopy.Channel, ), - ), context) + ), + context, + ) assert len(materialized) == 1 data, path = materialized[0] @@ -986,7 +995,9 @@ def fake_materialize(spec, data, path, manager, *_args, **kwargs): == expected_paths ) - PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs(filemanager, plan, context) + PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( + filemanager, plan, context + ) assert output_paths == [str(path) for path in expected_paths] @@ -1002,7 +1013,9 @@ def test_materialize_artifact_outputs_requires_runtime_store_record(): context = _context(filemanager) with pytest.raises(RuntimeError, match="Missing RuntimeValueStore record"): - PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs(filemanager, _plan(output_plan), context) + PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( + filemanager, _plan(output_plan), context + ) def test_materialize_artifact_outputs_does_not_require_vfs_payload_for_store_record( @@ -1027,7 +1040,9 @@ def fake_materialize(_spec, data, path, *_args, **_kwargs): _observe_materialization(monkeypatch, fake_materialize) - PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs(filemanager, _plan(output_plan), context) + PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( + filemanager, _plan(output_plan), context + ) assert materialized == [({"x": 1}, "/analysis/A01_positions_step7.roi.zip")] @@ -1073,10 +1088,15 @@ def fake_materialize(_spec, data, path, *_args, **_kwargs): _observe_materialization(monkeypatch, fake_materialize) - PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs(filemanager, _plan(output_plan), context) + PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( + filemanager, _plan(output_plan), context + ) assert [(data.row_mappings(), path) for data, path in materialized] == [ - (({"object_id": 1, "area": 42},), "/analysis/A01_channel-1_measurements_step7.roi.zip") + ( + ({"object_id": 1, "area": 42},), + "/analysis/A01_channel-1_measurements_step7.roi.zip", + ) ] @@ -1118,7 +1138,9 @@ def fake_materialize(spec, data, path, *_args, **_kwargs): _observe_materialization(monkeypatch, fake_materialize) - PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs(filemanager, _plan(output_plan), context) + PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( + filemanager, _plan(output_plan), context + ) spec, data, path = materialized[0] assert isinstance(spec.outputs[0], CsvOptions) @@ -1199,7 +1221,9 @@ def test_runtime_artifact_plan_materializes_3d_point_measurements(source_z_origi assert [item["z_index"] for item in domain.values] == list( range(source_z_origin, source_z_origin + 4) ) - assert result.viewer_source_identity.component_metadata["z_index"] == source_z_origin + assert ( + result.viewer_source_identity.component_metadata["z_index"] == source_z_origin + ) config = streaming_config_stub() plan = _plan( output_plan, @@ -1207,13 +1231,18 @@ def test_runtime_artifact_plan_materializes_3d_point_measurements(source_z_origi streaming_configs={"napari_stream": config}, ) [saved] = StreamingOnlyArtifactMaterializationTargetPlan().materialize_outputs( - context.filemanager, plan, context, + context.filemanager, + plan, + context, ) [streamed] = saved.outputs_for_backend("napari_stream") assert streamed.content[0].metadata[ROIFractionalZ.FIELD] == 2.375 request = context.filemanager.saved[-1][3]["stream_request"] assert "z_index" in request.display_config.COMPONENT_ORDER - assert request.source.metadata.component_metadata_for_item(streamed.path, 0)["z_index"] == source_z_origin + assert ( + request.source.metadata.component_metadata_for_item(streamed.path, 0)["z_index"] + == source_z_origin + ) z_domain = request.message_extra["component_value_domain"] assert z_domain["z_index"] == list(range(source_z_origin, source_z_origin + 4)) @@ -1294,7 +1323,9 @@ def test_multi_plane_measurement_materialization_uses_aggregate_artifact_name(): ) -@pytest.mark.parametrize("parser_type", [ImageXpressFilenameParser, SourceSchemaFilenameParser]) +@pytest.mark.parametrize( + "parser_type", [ImageXpressFilenameParser, SourceSchemaFilenameParser] +) def test_multi_plane_special_output_uses_aggregate_artifact_name(parser_type): output_plan = ArtifactOutputPlan( name="cell_counts", @@ -1345,7 +1376,9 @@ def test_multi_plane_special_output_uses_aggregate_artifact_name(parser_type): ) -@pytest.mark.parametrize("parser_type", [ImageXpressFilenameParser, SourceSchemaFilenameParser]) +@pytest.mark.parametrize( + "parser_type", [ImageXpressFilenameParser, SourceSchemaFilenameParser] +) def test_scalar_special_output_preserves_complete_source_identity(parser_type): output_plan = ArtifactOutputPlan( name="cell_counts", @@ -1389,7 +1422,9 @@ def test_scalar_special_output_preserves_complete_source_identity(parser_type): ) -@pytest.mark.parametrize("parser_type", [ImageXpressFilenameParser, SourceSchemaFilenameParser]) +@pytest.mark.parametrize( + "parser_type", [ImageXpressFilenameParser, SourceSchemaFilenameParser] +) def test_incomplete_scalar_special_output_keeps_strict_filename_failure(parser_type): output_plan = ArtifactOutputPlan( name="cell_counts", @@ -1427,7 +1462,9 @@ def test_incomplete_scalar_special_output_keeps_strict_filename_failure(parser_t runtime_artifact_materializations(_plan(output_plan), context) -@pytest.mark.parametrize("parser_type", [ImageXpressFilenameParser, SourceSchemaFilenameParser]) +@pytest.mark.parametrize( + "parser_type", [ImageXpressFilenameParser, SourceSchemaFilenameParser] +) def test_grouped_special_output_retains_group_coordinate_in_aggregate_name(parser_type): output_plan = ArtifactOutputPlan( name="cell_counts", @@ -1561,9 +1598,7 @@ def test_multi_plane_roi_aggregate_defers_source_filenames_to_plane_writer(monke ) roi_outputs = materialization.outputs(plan, context) assert tuple( - output.path - for output in roi_outputs - if output.path.endswith(".roi.zip") + output.path for output in roi_outputs if output.path.endswith(".roi.zip") ) == tuple( ( f"/analysis/A01_s{site:03d}_w{channel}_z001_t001" @@ -1584,8 +1619,12 @@ def test_multi_plane_roi_aggregate_defers_source_filenames_to_plane_writer(monke "Consolidation must not reconstruct unrelated ROI geometry" ), ) - hook = AnalysisConsolidationHook(AnalysisConsolidationConfig(), PlateMetadataConfig()) - assert hook.observe_reused_outputs(context, plan, materialization, roi_outputs) is None + hook = AnalysisConsolidationHook( + AnalysisConsolidationConfig(), PlateMetadataConfig() + ) + assert ( + hook.observe_reused_outputs(context, plan, materialization, roi_outputs) is None + ) def test_multi_plane_measurement_aggregate_names_retain_runtime_group_coordinate(): @@ -1735,9 +1774,12 @@ def record_for_site(site: str) -> None: current_execution_records = context.runtime_value_store.observed_values_after( current_execution_cursor ) - saved = PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs(context.filemanager, plan, context) + saved = PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( + context.filemanager, plan, context + ) selected_saved = tuple( - artifact for artifact in saved + artifact + for artifact in saved if artifact.materialization.record.key == current_execution_records[0].key ) assert len(selected_saved) == 1 @@ -1746,7 +1788,12 @@ def record_for_site(site: str) -> None: context.filemanager.save(historical_content, selected_path, "disk") reused = preview_reused_step_outputs(plan, context, current_execution_records) assert reused.hook_observations.get("analysis_consolidation") is not None - assert reused.hook_observations.get("analysis_consolidation").outputs_by_directory[Path("/analysis")][0].csv_content == historical_content + assert ( + reused.hook_observations.get("analysis_consolidation") + .outputs_by_directory[Path("/analysis")][0] + .csv_content + == historical_content + ) assert tuple( Path(location.path) for locations in reused.materialized_locations_by_address.values() @@ -1769,7 +1816,11 @@ def record_for_site(site: str) -> None: ), } context.step_plans = {plan.step_index: plan} - consolidation_inputs = selected_saved[0].observation(plan, context).hook_observations.get("analysis_consolidation") + consolidation_inputs = ( + selected_saved[0] + .observation(plan, context) + .hook_observations.get("analysis_consolidation") + ) assert consolidation_inputs is not None runtime_output = consolidation_inputs.outputs_by_directory[Path("/analysis")][0] assert runtime_output.path == Path( @@ -1786,14 +1837,17 @@ def record_for_site(site: str) -> None: @pytest.mark.parametrize("sites", [("1", "3"), ("3", "1")]) @pytest.mark.parametrize("automatic", [True, False]) -def test_scalar_acquired_tables_keep_fixed_site_addresses_on_shared_runtime_path(sites, automatic): +def test_scalar_acquired_tables_keep_fixed_site_addresses_on_shared_runtime_path( + sites, automatic +): output_plan = ArtifactOutputPlan( name="AcquiredFieldMeasurements", path="/memory/A01_w1_AcquiredFieldMeasurements_step0.pkl", artifact_type=MeasurementsArtifactType, materialization=( AutomaticMeasurementsArtifactOutputMaterializationStrategy().materialization() - if automatic else csv_only() + if automatic + else csv_only() ), variable_components=(Microscopy.ZIndex,), ) @@ -1809,8 +1863,11 @@ def test_scalar_acquired_tables_keep_fixed_site_addresses_on_shared_runtime_path ) for site in sites: metadata = { - "well": "A01", "site": site, "channel": "1", - "z_index": "1", "timepoint": "1", + "well": "A01", + "site": site, + "channel": "1", + "z_index": "1", + "timepoint": "1", } context.runtime_value_store.record( RuntimeValue.normalize_for_execution_scope( @@ -1830,7 +1887,9 @@ def test_scalar_acquired_tables_keep_fixed_site_addresses_on_shared_runtime_path subject=MeasurementSubject(MeasurementScope.ARTIFACT), ), execution_scope=RuntimeExecutionAxisScope.from_raw( - "A01", component=Microscopy.Channel, value="1", + "A01", + component=Microscopy.Channel, + value="1", fixed_component_values=( (Microscopy.Site, site), (Microscopy.Timepoint, "1"), @@ -1845,7 +1904,9 @@ def test_scalar_acquired_tables_keep_fixed_site_addresses_on_shared_runtime_path assert len(records) == 2 assert len({record.location.path for record in records}) == 1 saved = PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( - context.filemanager, plan, context, + context.filemanager, + plan, + context, ) assert len(saved) == 2 saved_paths = set() @@ -1856,7 +1917,8 @@ def test_scalar_acquired_tables_keep_fixed_site_addresses_on_shared_runtime_path paths = observation.paths_for(record) expected_stem = ( f"A01_site-{site}_channel-1_timepoint-1" - if automatic else f"A01_s{int(site):03d}_w1_z001_t001" + if automatic + else f"A01_s{int(site):03d}_w1_z001_t001" ) assert paths == ( Path( @@ -1955,7 +2017,9 @@ def test_consolidation_preserves_export_bundle_and_text_tables(options, payload) persistent_backend="disk", ) context.step_plans = {plan.step_index: plan} - saved = PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs(context.filemanager, plan, context) + saved = PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( + context.filemanager, plan, context + ) consolidation = StepExecutionObservation.combine( artifact.observation(plan, context) for artifact in saved ).hook_observations.get("analysis_consolidation") @@ -1976,33 +2040,44 @@ def test_consolidation_preserves_export_bundle_and_text_tables(options, payload) (RuntimeObservationMode.OMIT, RuntimeObservationMode.MERGE_INTO_PARENT), ) def test_completed_observation_projects_tables_before_worker_payload_release( - observation_mode, monkeypatch, + observation_mode, + monkeypatch, ): image_plan = ArtifactOutputPlan( - name="Corrected", path="/memory/Corrected.pkl", artifact_type=ImageArtifactType, + name="Corrected", + path="/memory/Corrected.pkl", + artifact_type=ImageArtifactType, ) table_plan = ArtifactOutputPlan( - name="cell_counts", path="/memory/cell_counts.pkl", - artifact_type=MeasurementsArtifactType, materialization=csv_only(), + name="cell_counts", + path="/memory/cell_counts.pkl", + artifact_type=MeasurementsArtifactType, + materialization=csv_only(), ) context = _context(FileManagerStub()) image_step = replace(_plan(image_plan), step_index=5) table_step = _plan(table_plan) table_step.runtime_artifact_materialization = RuntimeArtifactMaterializationPlan( - persistent_enabled=True, persistent_backend="disk", + persistent_enabled=True, + persistent_backend="disk", ) - context.step_plans = {image_step.step_index: image_step, table_step.step_index: table_step} + context.step_plans = { + image_step.step_index: image_step, + table_step.step_index: table_step, + } pixels = np.ones((8, 8), dtype=np.float32) pixel_reference = weakref.ref(pixels) context.runtime_value_store.record( RuntimeValue.normalize( image_plan, ImageMetadataPayload( - data=pixels, metadata=ImagePayloadMetadata(source_dtype="float32"), + data=pixels, + metadata=ImagePayloadMetadata(source_dtype="float32"), ), axis_id="A01", ), - path=image_plan.path, backend="memory", + path=image_plan.path, + backend="memory", ) context.runtime_value_store.record( RuntimeValue.normalize( @@ -2010,24 +2085,30 @@ def test_completed_observation_projects_tables_before_worker_payload_release( MeasurementTable( name=table_plan.name, rows=MeasurementSparseColumnarRows.from_rows( - ({"cell_count": 2},), fields=(FieldSpec("cell_count", int),), + ({"cell_count": 2},), + fields=(FieldSpec("cell_count", int),), ), subject=MeasurementSubject(MeasurementScope.ARTIFACT), ), axis_id="A01", ), - path=table_plan.path, backend="memory", + path=table_plan.path, + backend="memory", + ) + saved = PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( + context.filemanager, table_step, context ) - saved = PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs(context.filemanager, table_step, context) monkeypatch.setattr( - RuntimeArtifactMaterialization, "outputs", + RuntimeArtifactMaterialization, + "outputs", lambda *_args, **_kwargs: pytest.fail("Saved tables must not render again"), ) step_observation = StepExecutionObservation.combine( artifact.observation(table_step, context) for artifact in saved ) observation = RuntimeContextObservation.from_context( - context_key="A01", context=context, + context_key="A01", + context=context, records=context.runtime_value_store.observed_values, runtime_observation_mode=observation_mode, outputs=step_observation, @@ -2042,9 +2123,21 @@ def test_completed_observation_projects_tables_before_worker_payload_release( ) assert len(observation.outputs.runtime_export_paths) == 1 assert observation.outputs.runtime_export_paths[0].suffix == ".csv" - assert observation.outputs.hook_observations.get("analysis_consolidation") is not None - assert observation.outputs.hook_observations.get("analysis_consolidation").destination.backend == "disk" - assert observation.outputs.hook_observations.get("analysis_consolidation").destination.images_dir == "/images" + assert ( + observation.outputs.hook_observations.get("analysis_consolidation") is not None + ) + assert ( + observation.outputs.hook_observations.get( + "analysis_consolidation" + ).destination.backend + == "disk" + ) + assert ( + observation.outputs.hook_observations.get( + "analysis_consolidation" + ).destination.images_dir + == "/images" + ) execution_observation = RuntimeExecutionObservation(contexts=(observation,)) transported = pickle.loads(pickle.dumps(execution_observation)) consolidated = PostExecuteHook.combine_all( @@ -2119,7 +2212,9 @@ def fake_materialize(_spec, data, path, *_args, **_kwargs): _observe_materialization(monkeypatch, fake_materialize) - PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs(filemanager, _plan(output_plan), context) + PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( + filemanager, _plan(output_plan), context + ) assert materialized == [ ( @@ -2413,9 +2508,13 @@ def fake_materialize( _observe_materialization(monkeypatch, fake_materialize) - PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs(filemanager, _plan( + PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( + filemanager, + _plan( output_plan, streaming_configs={"napari_stream": streaming_config_stub()} - ), context) + ), + context, + ) spec, data, path, backends, backend_kwargs = materialized[0] assert isinstance(spec.outputs[0], CsvOptions) @@ -2470,7 +2569,9 @@ def fake_materialize(spec, data, path, *_args, **_kwargs): _observe_materialization(monkeypatch, fake_materialize) - PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs(filemanager, _plan(output_plan), context) + PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( + filemanager, _plan(output_plan), context + ) assert len(materialized) == 1 spec, data, path = materialized[0] @@ -2741,7 +2842,9 @@ def fake_materialize(spec, data, path, *_args, **_kwargs): _observe_materialization(monkeypatch, fake_materialize) - PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs(filemanager, _plan(output_plan), context) + PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( + filemanager, _plan(output_plan), context + ) assert [path for _spec, _data, path in materialized] == [ "/analysis/A01_s001_w5_z001_t001_measurements_step7.roi.zip", @@ -2796,7 +2899,9 @@ def fake_materialize(spec, data, path, *_args, **_kwargs): _observe_materialization(monkeypatch, fake_materialize) - PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs(filemanager, _plan(output_plan, group_by_value="channel"), context) + PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( + filemanager, _plan(output_plan, group_by_value="channel"), context + ) assert [path for _spec, _data, path in materialized] == [ "/analysis/A01_channel-2_measurements_step7.roi.zip", @@ -2853,9 +2958,13 @@ def fake_materialize( _observe_materialization(monkeypatch, fake_materialize) - StreamingOnlyArtifactMaterializationTargetPlan().materialize_outputs(filemanager, _plan( + StreamingOnlyArtifactMaterializationTargetPlan().materialize_outputs( + filemanager, + _plan( output_plan, streaming_configs={"napari_stream": streaming_config_stub()} - ), context) + ), + context, + ) _spec, _data, path, _backends, backend_kwargs = materialized[0] assert path == "/analysis/A01_s001_w2_z001_t001_Nuclei_step7.roi.zip" @@ -2916,7 +3025,11 @@ def fake_materialize( streaming_config = streaming_config_stub() - StreamingOnlyArtifactMaterializationTargetPlan().materialize_outputs(filemanager, _plan(output_plan, streaming_configs={"napari_stream": streaming_config}), context) + StreamingOnlyArtifactMaterializationTargetPlan().materialize_outputs( + filemanager, + _plan(output_plan, streaming_configs={"napari_stream": streaming_config}), + context, + ) _spec, _data, path, _backends, backend_kwargs = materialized[0] assert path == "/analysis/A01_Nuclei_step7.roi.zip" @@ -3056,7 +3169,9 @@ def fake_materialize(spec, data, path, *_args, **_kwargs): _observe_materialization(monkeypatch, fake_materialize) - PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs(filemanager, _plan(output_plan), context) + PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( + filemanager, _plan(output_plan), context + ) spec, data, _path = materialized[0] assert isinstance(spec.outputs[0], JsonOptions) @@ -3081,7 +3196,9 @@ def fake_materialize(*args, **kwargs): _observe_materialization(monkeypatch, fake_materialize) - PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs(filemanager, _plan(output_plan), context) + PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( + filemanager, _plan(output_plan), context + ) assert materialized == [] @@ -3137,14 +3254,18 @@ def test_tile_positions_runtime_materializes_native_json_without_changing_payloa analysis_results_dir=str(tmp_path / "results"), ) - PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs(filemanager, plan, context) + PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( + filemanager, plan, context + ) (retained,) = tuple((tmp_path / "results").rglob("*.json")) assert json.loads(retained.read_text()) == [list(pair) for pair in positions] assert positions == [(-1.25, 0.000000001), (921.600000001, -3.5), (1.0, 1843.2)] assert record.key is value.key assert record.data is value.data - assert record.materialization_source_metadata is value.materialization_source_metadata + assert ( + record.materialization_source_metadata is value.materialization_source_metadata + ) def test_materialize_artifact_outputs_skips_explicitly_disabled_artifact_without_record( @@ -3165,7 +3286,9 @@ def fake_materialize(*args, **kwargs): _observe_materialization(monkeypatch, fake_materialize) - PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs(filemanager, _plan(output_plan), context) + PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( + filemanager, _plan(output_plan), context + ) assert materialized == [] @@ -3197,7 +3320,9 @@ def fake_materialize(spec, data, path, *_args, **_kwargs): _observe_materialization(monkeypatch, fake_materialize) - PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs(filemanager, _plan(output_plan), context) + PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( + filemanager, _plan(output_plan), context + ) spec, data, path = materialized[0] assert isinstance(spec.outputs[0], ROIOptions) @@ -3245,7 +3370,11 @@ def fake_materialize( _observe_materialization(monkeypatch, fake_materialize) - StreamingOnlyArtifactMaterializationTargetPlan().materialize_outputs(filemanager, _plan(output_plan, streaming_configs={"napari_stream": streaming_config}), context) + StreamingOnlyArtifactMaterializationTargetPlan().materialize_outputs( + filemanager, + _plan(output_plan, streaming_configs={"napari_stream": streaming_config}), + context, + ) spec, data, path, backends, backend_kwargs = materialized[0] assert isinstance(spec.outputs[0], ROIOptions) @@ -3325,11 +3454,15 @@ def fake_materialize( _observe_materialization(monkeypatch, fake_materialize) - PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs(filemanager, _plan( + PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( + filemanager, + _plan( output_plan, streaming_configs={"napari_stream": streaming_config_stub()}, compiled_function_pattern=_main_flow_output_compiled_pattern(output_plan), - ), context) + ), + context, + ) assert len(materialized) == 1 backends, backend_kwargs = materialized[0] @@ -3381,10 +3514,14 @@ def fake_materialize( _observe_materialization(monkeypatch, fake_materialize) - StreamingOnlyArtifactMaterializationTargetPlan().materialize_outputs(filemanager, _plan( + StreamingOnlyArtifactMaterializationTargetPlan().materialize_outputs( + filemanager, + _plan( output_plan, streaming_configs={"napari_stream": streaming_config}, - ), context) + ), + context, + ) _spec, _data, path, _backends, backend_kwargs = materialized[0] assert path == "/analysis/A01_s002_w3_z001_t001_labels_step7.roi.zip" @@ -3458,7 +3595,9 @@ def fake_materialize( _observe_materialization(monkeypatch, fake_materialize) - StreamingOnlyArtifactMaterializationTargetPlan().materialize_outputs(filemanager, plan, context) + StreamingOnlyArtifactMaterializationTargetPlan().materialize_outputs( + filemanager, plan, context + ) _spec, _data, path, _backends, backend_kwargs = materialized[0] assert path == "/analysis/01_POS002_D_Nuclei_step7.roi.zip" @@ -3511,11 +3650,15 @@ def fake_materialize(_spec, data, path, *_args, **_kwargs): _observe_materialization(monkeypatch, fake_materialize) - PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs(filemanager, _plan( + PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( + filemanager, + _plan( output_plan, group_by_value="timepoint", variable_components=(Microscopy.Timepoint,), - ), context) + ), + context, + ) assert materialized == [ ( @@ -3524,9 +3667,7 @@ def fake_materialize(_spec, data, path, *_args, **_kwargs): ) ] assert materialized[0][0] is runtime_value.data - assert materialized[0][0].metadata.source_image_names == ( - "AdjacentImage", - ) + assert materialized[0][0].metadata.source_image_names == ("AdjacentImage",) def test_materialize_artifact_outputs_merges_parser_axes_into_source_metadata( @@ -3573,7 +3714,11 @@ def fake_materialize( _observe_materialization(monkeypatch, fake_materialize) - StreamingOnlyArtifactMaterializationTargetPlan().materialize_outputs(filemanager, _plan(output_plan, streaming_configs={"napari_stream": streaming_config}), context) + StreamingOnlyArtifactMaterializationTargetPlan().materialize_outputs( + filemanager, + _plan(output_plan, streaming_configs={"napari_stream": streaming_config}), + context, + ) _spec, _data, path, _backends, backend_kwargs = materialized[0] assert path == "/analysis/A01_s002_w3_z001_t001_Nuclei_step7.roi.zip" @@ -3665,11 +3810,15 @@ def fake_materialize( _observe_materialization(monkeypatch, fake_materialize) - StreamingOnlyArtifactMaterializationTargetPlan().materialize_outputs(filemanager, _plan( + StreamingOnlyArtifactMaterializationTargetPlan().materialize_outputs( + filemanager, + _plan( output_plan, streaming_configs={"napari_stream": streaming_config}, variable_components=(Microscopy.ZIndex,), - ), context) + ), + context, + ) _spec, _data, path, _backends, backend_kwargs = materialized[0] assert path == "/analysis/A01_Nuclei_step7.roi.zip" @@ -3729,11 +3878,15 @@ def test_materialize_artifact_outputs_streams_singleton_roi_plane_from_output_pl backend="memory", ) - StreamingOnlyArtifactMaterializationTargetPlan().materialize_outputs(filemanager, _plan( + StreamingOnlyArtifactMaterializationTargetPlan().materialize_outputs( + filemanager, + _plan( output_plan, streaming_configs={"napari_stream": streaming_config_stub()}, variable_components=(Microscopy.ZIndex,), - ), context) + ), + context, + ) roi_saves = [item for item in filemanager.saved if item[1].endswith(".roi.zip")] assert len(roi_saves) == 1 @@ -3823,11 +3976,15 @@ def fake_materialize( _observe_materialization(monkeypatch, fake_materialize) - StreamingOnlyArtifactMaterializationTargetPlan().materialize_outputs(filemanager, _plan( + StreamingOnlyArtifactMaterializationTargetPlan().materialize_outputs( + filemanager, + _plan( output_plan, streaming_configs={"napari_stream": streaming_config}, variable_components=(Microscopy.Channel,), - ), context) + ), + context, + ) _spec, _data, path, _backends, backend_kwargs = materialized[0] assert path == "/analysis/A01_segmentation_masks_step7.roi.zip" @@ -3867,8 +4024,11 @@ def test_materialize_rgb_artifact_streams_filename_channel_identity( # contributors alone do not establish a scalar channel on the result. source_path="/input/A01_s001_w3_z001_t001.TIF", source_component_metadata={ - "well": "A01", "site": "1", "channel": "3", - "z_index": "1", "timepoint": "1", + "well": "A01", + "site": "1", + "channel": "3", + "z_index": "1", + "timepoint": "1", }, source_image_provenance_planes=SourceImageProvenancePlanes.from_components( paths=( @@ -3924,11 +4084,15 @@ def fake_materialize( _observe_materialization(monkeypatch, fake_materialize) - StreamingOnlyArtifactMaterializationTargetPlan().materialize_outputs(filemanager, _plan( + StreamingOnlyArtifactMaterializationTargetPlan().materialize_outputs( + filemanager, + _plan( output_plan, streaming_configs={"napari_stream": streaming_config}, variable_components=(Microscopy.Channel,), - ), context) + ), + context, + ) _spec, _data, path, _backends, backend_kwargs = materialized[0] assert path == "/images/A01_s001_w3_z001_t001.TIF" @@ -4001,7 +4165,11 @@ def fake_materialize(_spec, _data, path, *_args, **_kwargs): _observe_materialization(monkeypatch, fake_materialize) - PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs(filemanager, _plan(output_plan, variable_components=(Microscopy.Channel,)), context) + PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( + filemanager, + _plan(output_plan, variable_components=(Microscopy.Channel,)), + context, + ) assert materialized == ["/images/A01_s001_w1_z001_t001.TIF"] @@ -4085,18 +4253,28 @@ def test_compiled_z_axis_reaches_source_named_image_materialization() -> None: source_image_names=(selected_image.name,), source_path="/derived/A01_s001_w2_z001_t001.tif", source_component_metadata={ - "well": "A01", "site": "1", "channel": "2", + "well": "A01", + "site": "1", + "channel": "2", "timepoint": "1", }, source_image_provenance_planes=SourceImageProvenancePlanes.from_components( paths=tuple(f"/derived/A01_s001_w2_z00{z}_t001.tif" for z in (1, 2)), component_metadata=tuple( - {"well": "A01", "site": "1", "channel": "2", - "z_index": str(z), "timepoint": "1"} for z in (1, 2) + { + "well": "A01", + "site": "1", + "channel": "2", + "z_index": str(z), + "timepoint": "1", + } + for z in (1, 2) ), ), ).payload_with(np.ones((2, 5, 7), dtype=np.uint16), None) - saved_payload = (ImageArtifactType if output_plan is None else output_plan.artifact_type).contextualize_output_from_projector( + saved_payload = ( + ImageArtifactType if output_plan is None else output_plan.artifact_type + ).contextualize_output_from_projector( selected_payload, ImagePayloadMetadata( plane_axis=RuntimePlaneAxis.RUNTIME_SLICE, @@ -4159,9 +4337,7 @@ def test_compiled_z_axis_reaches_source_named_image_materialization() -> None: persistent_backend="disk", ) - assert saved_payload.metadata.plane_axis is ( - RuntimePlaneAxis.RUNTIME_SLICE - ) + assert saved_payload.metadata.plane_axis is (RuntimePlaneAxis.RUNTIME_SLICE) assert tuple( Path(output.path) for materialization in runtime_artifact_materializations(plan, context) @@ -4239,11 +4415,15 @@ def fake_materialize( _observe_materialization(monkeypatch, fake_materialize) - StreamingOnlyArtifactMaterializationTargetPlan().materialize_outputs(filemanager, _plan( + StreamingOnlyArtifactMaterializationTargetPlan().materialize_outputs( + filemanager, + _plan( output_plan, streaming_configs={"napari_stream": streaming_config}, variable_components=(Microscopy.Site,), - ), context) + ), + context, + ) _spec, _data, path, _backends, backend_kwargs = materialized[0] assert path == "/images/A01_s001_w2_z001_t001.TIF" diff --git a/tests/unit/test_function_artifact_outputs.py b/tests/unit/test_function_artifact_outputs.py index 949a45933c..71dbac8717 100644 --- a/tests/unit/test_function_artifact_outputs.py +++ b/tests/unit/test_function_artifact_outputs.py @@ -299,9 +299,7 @@ def assemble(image, **kwargs): assert result.data.shape == (4, 8) assert result.data.dtype == tiles.dtype - assert result.metadata.source_path == ( - "/input/A01_s001_w1_z001_t001.tif" - ) + assert result.metadata.source_path == ("/input/A01_s001_w1_z001_t001.tif") def test_function_core_preserves_output_declared_runtime_slice_axis(): @@ -345,7 +343,9 @@ def test_image_output_context_preserves_stack_after_exact_source_projection(): relations=(GroupLineageSourceRelation(source_spec.ref()),), ) - result = (ImageArtifactType if output_plan is None else output_plan.artifact_type).contextualize_output( + result = ( + ImageArtifactType if output_plan is None else output_plan.artifact_type + ).contextualize_output( source, output, output_plan, @@ -395,7 +395,9 @@ def test_image_output_context_does_not_infer_axis_for_unmarked_payload(): relations=(GroupLineageSourceRelation(source_spec.ref()),), ) - result = (ImageArtifactType if output_plan is None else output_plan.artifact_type).contextualize_output_from_projector( + result = ( + ImageArtifactType if output_plan is None else output_plan.artifact_type + ).contextualize_output_from_projector( source, np.ones((2, 4, 5), dtype=np.uint16), output_plan, @@ -427,7 +429,9 @@ def test_image_output_context_rejects_declared_output_axis_shape_drift(): ValueError, match="declared 'runtime_slice' axis of size 2", ): - (ImageArtifactType if output_plan is None else output_plan.artifact_type).contextualize_output( + ( + ImageArtifactType if output_plan is None else output_plan.artifact_type + ).contextualize_output( np.ones((1, 4, 5), dtype=np.float32), output, output_plan, @@ -452,7 +456,9 @@ def test_image_output_context_preserves_complete_scalar_rgb_identity() -> None: source_image_names=(source_spec.name,), axes=PayloadAxes.colour_samples(2), ).payload_with(np.ones((4, 5, 3), dtype=np.float32), None) - output = source.with_pixels(np.ones((4, 5, 3), dtype=np.uint8),) + output = source.with_pixels( + np.ones((4, 5, 3), dtype=np.uint8), + ) output_plan = ArtifactOutputPlan( name="SavedColorNeighbors", path="/memory/SavedColorNeighbors.png", @@ -461,7 +467,9 @@ def test_image_output_context_preserves_complete_scalar_rgb_identity() -> None: relations=(GroupLineageSourceRelation(source_spec.ref()),), ) - result = (ImageArtifactType if output_plan is None else output_plan.artifact_type).contextualize_output_from_projector( + result = ( + ImageArtifactType if output_plan is None else output_plan.artifact_type + ).contextualize_output_from_projector( source, output, output_plan, @@ -547,7 +555,9 @@ def test_image_output_context_projects_complete_multi_plane_identity() -> None: relations=(GroupLineageSourceRelation(source_spec.ref()),), ) - result = (ImageArtifactType if output_plan is None else output_plan.artifact_type).contextualize_output_from_projector( + result = ( + ImageArtifactType if output_plan is None else output_plan.artifact_type + ).contextualize_output_from_projector( source, output, output_plan, @@ -673,9 +683,7 @@ def compose_channels(_image): def test_crop_mask_sidecar_names_derive_from_core_artifact_role(): - assert CropMask.name_for("CroppedImage") == ( - "CroppedImage__crop_mask" - ) + assert CropMask.name_for("CroppedImage") == ("CroppedImage__crop_mask") def test_unstack_payload_context_slices_volume_stack_mask_with_volume_data(): @@ -771,10 +779,10 @@ def test_unstack_payload_context_preserves_declared_axis_over_default(): default_plane_axis=RuntimePlaneAxis.RUNTIME_SLICE, ) - assert [ - slice_payload.metadata.source_image_names - for slice_payload in slices - ] == [("DNA",), ("Actin",)] + assert [slice_payload.metadata.source_image_names for slice_payload in slices] == [ + ("DNA",), + ("Actin",), + ] def test_stack_payload_context_projects_single_payload_mask_to_stack_domain(): @@ -1385,9 +1393,7 @@ def produce_corrected_image(image): stored = context.runtime_value_store.find(name="CorrectedImage", axis_id="A01") assert len(stored) == 1 np.testing.assert_array_equal(stored[0].data.data, output) - assert stored[0].data.metadata.source_image_names == ( - "CorrectedImage", - ) + assert stored[0].data.metadata.source_image_names == ("CorrectedImage",) def test_saved_canonical_source_projection_survives_main_flow_return(): @@ -1401,22 +1407,37 @@ def test_saved_canonical_source_projection_survives_main_flow_return(): source_image_provenance_planes=SourceImageProvenancePlanes.from_components( paths=paths, component_metadata=tuple( - {"well": "A01", "site": site, "channel": "2", "z_index": "1", "timepoint": "1"} + { + "well": "A01", + "site": site, + "channel": "2", + "z_index": "1", + "timepoint": "1", + } for site in ("1", "2") ), ), ).payload_with(pixels) output = ArtifactSpec.output("Selected", ImageArtifactType) - plan = ArtifactOutputPlan(name=output.name, path="/memory/selected.pkl", artifact_type=ImageArtifactType) + plan = ArtifactOutputPlan( + name=output.name, path="/memory/selected.pkl", artifact_type=ImageArtifactType + ) @artifact_outputs(output) def select_second_plane(image): return SelectedPlaneImageOutput(image[1:2], (1,)) - result = _execute_function_core(CoreExecutionRequest( - func_callable=select_second_plane, main_data_arg=source, base_kwargs={}, context=context, - artifact_inputs={}, artifact_outputs={plan.ref(): plan}, runtime_plane_count=2, - )) + result = _execute_function_core( + CoreExecutionRequest( + func_callable=select_second_plane, + main_data_arg=source, + base_kwargs={}, + context=context, + artifact_inputs={}, + artifact_outputs={plan.ref(): plan}, + runtime_plane_count=2, + ) + ) stored = context.runtime_value_store.find(name=output.name, axis_id="A01") assert len(stored) == 1 for payload in (result, stored[0].data): @@ -1430,7 +1451,9 @@ def select_second_plane(image): @pytest.mark.parametrize("declared_canonical", [False, True]) -def test_unsaved_main_return_reads_source_context_after_artifact_save(declared_canonical): +def test_unsaved_main_return_reads_source_context_after_artifact_save( + declared_canonical, +): context = ContextStub() source = ImagePayloadMetadata( source_path="/source/before.tif", @@ -1438,13 +1461,21 @@ def test_unsaved_main_return_reads_source_context_after_artifact_save(declared_c ).payload_with(np.zeros((2, 3), dtype=np.float32)) sidecar = ArtifactSpec.output("Sidecar", MetadataArtifactType) canonical = ArtifactSpec.output("Unselected", ImageArtifactType) - plan = ArtifactOutputPlan(name=sidecar.name, path="/memory/sidecar.pkl", artifact_type=MetadataArtifactType) + plan = ArtifactOutputPlan( + name=sidecar.name, + path="/memory/sidecar.pkl", + artifact_type=MetadataArtifactType, + ) original_save = context.filemanager.save def save_and_change_source(value, path, backend): original_save(value, path, backend) source.metadata.source_path = "/source/after.tif" - source.metadata.source_component_metadata = {"well": "A01", "site": "9", "channel": "2"} + source.metadata.source_component_metadata = { + "well": "A01", + "site": "9", + "channel": "2", + } context.filemanager.save = save_and_change_source @@ -1452,10 +1483,16 @@ def save_and_change_source(value, path, backend): def produce(image): return image + 1, {"note": "saved"} - result = _execute_function_core(CoreExecutionRequest( - func_callable=produce, main_data_arg=source, base_kwargs={}, context=context, - artifact_inputs={}, artifact_outputs={plan.ref(): plan}, - )) + result = _execute_function_core( + CoreExecutionRequest( + func_callable=produce, + main_data_arg=source, + base_kwargs={}, + context=context, + artifact_inputs={}, + artifact_outputs={plan.ref(): plan}, + ) + ) np.testing.assert_array_equal(result.data, np.ones((2, 3))) assert result.metadata.source_path == "/source/after.tif" assert result.metadata.source_component_metadata["site"] == "9" @@ -1521,9 +1558,7 @@ def produce_image(image): output_payload.data, output_slices[index], ) - assert ( - metadata.source_path == source_payload.metadata.source_path - ) + assert metadata.source_path == source_payload.metadata.source_path assert metadata.source_image_names == (output_spec.name,) assert metadata.source_provenance.represented_source_image_names == ( output_spec.name, @@ -1775,8 +1810,10 @@ def rgb_output(image): ) from openhcs.core.aligned_image_payload import ImagePayloadStackComposition - stack_payload = (scalar_output).copied(memory_type=MEMORY_TYPE_NUMPY, - device_id=None,) + stack_payload = (scalar_output).copied( + memory_type=MEMORY_TYPE_NUMPY, + device_id=None, + ) assert stack_payload.data.shape == (2, 3, 3) stack_metadata = stack_payload.metadata assert stack_metadata.plane_axis is None @@ -1811,10 +1848,7 @@ def test_pattern_group_runtime_uses_declared_output_slice_cardinality(): ) assert len(output) == 1 assert output[0][0].data.shape == (4, 5) - assert ( - declared_output.metadata.plane_axis - is RuntimePlaneAxis.RUNTIME_SLICE - ) + assert declared_output.metadata.plane_axis is RuntimePlaneAxis.RUNTIME_SLICE def test_pattern_group_runtime_projects_nominal_object_label_stack(): @@ -2311,9 +2345,7 @@ def segment(image): assert label_set.declared_plane_count() == 2 np.testing.assert_array_equal( label_set.labels, - np.stack( - tuple(labels.value_at(index) for index in range(labels.slice_count)) - ), + np.stack(tuple(labels.value_at(index) for index in range(labels.slice_count))), ) assert label_set.source_provenance.source_image_provenance_planes.paths == tuple( item.metadata.source_path for item in source_slices @@ -2338,11 +2370,15 @@ def test_measurement_subject_is_required_before_schema_rows_execute(boundary): def count(image): raise AssertionError("Invalid measurement declarations must not execute") - with pytest.raises(ValueError, match="cell_counts.*no declared measurement subject") as exc: + with pytest.raises( + ValueError, match="cell_counts.*no declared measurement subject" + ) as exc: if boundary == "contract": from openhcs.core.callable_contract import CallableContract - CallableContract.from_callable(count).validate_artifact_output_declarations() + CallableContract.from_callable( + count + ).validate_artifact_output_declarations() else: compile_function_pattern(count, {}, {}) assert "ImageMeasurementSubjectRelation" in str(exc.value) @@ -2925,7 +2961,9 @@ def declared_source_origin(image, *, image_to_save=None): value=None, ), execution_plan=CompiledStepPlan( - step_index=0, step_name="source-bound", axis_id="A01", + step_index=0, + step_name="source-bound", + axis_id="A01", variable_components=(), ), ), @@ -3135,5 +3173,6 @@ def test_declared_source_payload_rejects_ambiguous_stored_and_source_bound_origi np.zeros((1, 2, 3), dtype=np.uint16) ) with pytest.raises(ValueError, match="must resolve to exactly one .* resolved 2"): - executor.declared_source_payload(spec.ref(), primary, - loaded_artifact_payloads={spec.ref(): primary}) + executor.declared_source_payload( + spec.ref(), primary, loaded_artifact_payloads={spec.ref(): primary} + ) diff --git a/tests/unit/test_function_execution_lifecycle_anchors.py b/tests/unit/test_function_execution_lifecycle_anchors.py index 00134ac514..d09ec72dad 100644 --- a/tests/unit/test_function_execution_lifecycle_anchors.py +++ b/tests/unit/test_function_execution_lifecycle_anchors.py @@ -50,7 +50,8 @@ def _anchor_executor( ) executor.context.runtime_source_workspace_projections = ( WorkspaceSourceProjections.from_context( - executor.context, cache=source_workspace_projection_cache, + executor.context, + cache=source_workspace_projection_cache, ) ) if output_manifest is not None: diff --git a/tests/unit/test_function_io.py b/tests/unit/test_function_io.py index 79e391490d..794e11c239 100644 --- a/tests/unit/test_function_io.py +++ b/tests/unit/test_function_io.py @@ -307,7 +307,8 @@ def test_bulk_preload_preserves_nested_virtual_workspace_paths(tmp_path: Path) - assert virtual_paths == expected_paths SourceFileUniverse( - tuple(expected_paths), Backend.VIRTUAL_WORKSPACE, + tuple(expected_paths), + Backend.VIRTUAL_WORKSPACE, ).load_images(filemanager) loaded = filemanager.load_batch(expected_paths, Backend.MEMORY.value) @@ -526,7 +527,9 @@ def test_bulk_preload_loads_matlab_pixels_through_declared_vfs_backend( str(workspace_root / virtual_path) for virtual_path in materialization.plane_mappings ) - SourceFileUniverse(virtual_paths, Backend.VIRTUAL_WORKSPACE).load_images(filemanager) + SourceFileUniverse(virtual_paths, Backend.VIRTUAL_WORKSPACE).load_images( + filemanager + ) loaded = filemanager.load_batch( list(virtual_paths), diff --git a/tests/unit/test_function_outputs.py b/tests/unit/test_function_outputs.py index d14c6c2f0d..6a5cd3782f 100644 --- a/tests/unit/test_function_outputs.py +++ b/tests/unit/test_function_outputs.py @@ -75,7 +75,9 @@ MaterializedRuntimeArtifact, RuntimeArtifactMaterialization, ) -from openhcs.core.steps.function_artifact_materialization import ArtifactMaterializationTargetPlan +from openhcs.core.steps.function_artifact_materialization import ( + ArtifactMaterializationTargetPlan, +) from openhcs.core.steps.function_outputs import ( MaterializedImageOutputWriter, MemoryOutputWriter, @@ -123,7 +125,8 @@ def _publish_saved_step(context, plan, *, artifact_materializations=()): METADATA_CONFIG.metadata_path(target.dataset_root) for target in facts ): entries = { - target: value for target, value in facts.items() + target: value + for target, value in facts.items() if METADATA_CONFIG.metadata_path(target.dataset_root) == metadata_path } AtomicMetadataWriter().reconcile_completed_dataset( @@ -174,7 +177,8 @@ def test_function_step_metadata_follows_runtime_artifact_persistence( authority, method_name, lambda _context, _plan, event=event, **_kwargs: ( - events.append(event), {} if event == "metadata" else () + events.append(event), + {} if event == "metadata" else (), )[1], ) @@ -305,7 +309,10 @@ def parse_filename(self, name): } ) return FilenameParseResult( - ((component, metadata.get(component.name)) for component in AxisFamily.active().axes), + ( + (component, metadata.get(component.name)) + for component in AxisFamily.active().axes + ), extension=str(metadata["extension"]), ) @@ -510,7 +517,9 @@ def test_memory_output_writer_materializes_preserved_source_at_current_destinati @pytest.mark.parametrize("source_extension", (None, ".czi")) -def test_function_output_identity_preserves_non_axis_source_metadata(source_extension) -> None: +def test_function_output_identity_preserves_non_axis_source_metadata( + source_extension, +) -> None: identity = FunctionOutputIdentity( component_values={"channel": "2", "site": "1"}, extension=".tif", @@ -518,10 +527,10 @@ def test_function_output_identity_preserves_non_axis_source_metadata(source_exte ) source_metadata = { - "Run": "Sequence1", - "Specimen": "DrosophilaEmbryo", - "ChannelNumber": "1", - "OpenHCSOriginalSourceMetadata": {"FrameNumber": "0007"}, + "Run": "Sequence1", + "Specimen": "DrosophilaEmbryo", + "ChannelNumber": "1", + "OpenHCSOriginalSourceMetadata": {"FrameNumber": "0007"}, } if source_extension is not None: source_metadata["extension"] = source_extension @@ -536,7 +545,9 @@ def test_function_output_identity_preserves_non_axis_source_metadata(source_exte **({"extension": source_extension} if source_extension is not None else {}), } assert identity.filename_component_metadata() == { - "channel": "2", "site": "1", "extension": ".tif", + "channel": "2", + "site": "1", + "extension": ".tif", } assert source_metadata["ChannelNumber"] == "1" assert source_metadata.get("extension") == source_extension @@ -1665,7 +1676,9 @@ def test_produced_projection_metadata_persists_typed_collapsed_semantics( assert len(restored.source_provenance.represented_source_identities) == 2 assert record["address"] is None assert record["execution_scope"]["fixed_component_values"] == [ - ["channel", "1"], ["z_index", "1"], ["timepoint", "1"], + ["channel", "1"], + ["z_index", "1"], + ["timepoint", "1"], ] assert "site" not in record["source_metadata"] assert "site" not in subdirectory[FIELDS.SOURCE_METADATA]["images/A01_s1_w1.tif"] @@ -1944,7 +1957,10 @@ def parse_filename(self, name): } ) return FilenameParseResult( - ((component, metadata.get(component.name)) for component in AxisFamily.active().axes), + ( + (component, metadata.get(component.name)) + for component in AxisFamily.active().axes + ), extension=str(metadata["extension"]), ) @@ -2084,9 +2100,7 @@ def test_produced_address_publication_never_parses_generated_filenames( }, source_voxel_spacing=SourceVoxelSpacing((0.5, 0.5)), ) - identity = FunctionOutputIdentity.from_metadata( - parser, metadata - ) + identity = FunctionOutputIdentity.from_metadata(parser, metadata) assert identity is not None assert identity.extension == extension identity = identity.with_filename_qualifier("centre_dots") @@ -2258,11 +2272,15 @@ def test_completed_plate_metadata_includes_outputs_written_after_owner_axis( follower_plan.create_openhcs_metadata = True filemanager.save(first_pixels, str(later_image), Backend.MEMORY.value) record_output_path(follower_context, follower_plan, str(later_image)) - later_facts = OpenHCSMetadataTarget.observe_for_step(follower_context, follower_plan) + later_facts = OpenHCSMetadataTarget.observe_for_step( + follower_context, follower_plan + ) combined_facts = { target: VirtualWorkspaceSourceProjectionEntries.combine( - (first_facts.get(target, VirtualWorkspaceSourceProjectionEntries({})), - later_facts.get(target, VirtualWorkspaceSourceProjectionEntries({}))) + ( + first_facts.get(target, VirtualWorkspaceSourceProjectionEntries({})), + later_facts.get(target, VirtualWorkspaceSourceProjectionEntries({})), + ) ) for target in dict.fromkeys((*first_facts, *later_facts)) } @@ -2389,7 +2407,9 @@ def test_runtime_image_metadata_target_requires_persisted_images(tmp_path, conte context, plan, artifact_materializations=saved_artifacts ) if contents == "images": - assert selected == (replace(target, create_openhcs_metadata=plan.create_openhcs_metadata),) + assert selected == ( + replace(target, create_openhcs_metadata=plan.create_openhcs_metadata), + ) else: assert selected == () plan.create_openhcs_metadata = True @@ -2459,12 +2479,8 @@ def test_declared_image_destinations_publish_and_reconcile_after_value_cleanup( path=output_plan.path, backend=Backend.MEMORY.value, ) - materializations = ArtifactMaterializationTargetPlan.materialize( - context, plan - ) - _publish_saved_step( - context, plan, artifact_materializations=materializations - ) + materializations = ArtifactMaterializationTargetPlan.materialize(context, plan) + _publish_saved_step(context, plan, artifact_materializations=materializations) context.runtime_value_store.clear() if stray_image: tifffile.imwrite(tmp_path / "images/review/unowned.tif", pixels) @@ -2556,9 +2572,7 @@ def test_actual_array_exports_only_publish_declared_raster_inventory( backend=Backend.MEMORY.value, ) - materializations = ArtifactMaterializationTargetPlan.materialize( - context, plan - ) + materializations = ArtifactMaterializationTargetPlan.materialize(context, plan) saved_outputs = tuple( output for artifact in materializations @@ -2576,15 +2590,15 @@ def test_actual_array_exports_only_publish_declared_raster_inventory( location for artifact in materializations for locations in ( - artifact.observation(plan, context).materialized_locations_by_address.values() + artifact.observation( + plan, context + ).materialized_locations_by_address.values() ) for location in locations ) assert {Path(location.path) for location in observed_locations} == expected_paths - _publish_saved_step( - context, plan, artifact_materializations=materializations - ) + _publish_saved_step(context, plan, artifact_materializations=materializations) context.runtime_value_store.clear() OpenHCSMetadataTarget.finalize_completed_dataset({"A01": context}) metadata_path = tmp_path / "openhcs_metadata.json" @@ -2623,8 +2637,13 @@ def test_actual_array_exports_only_publish_declared_raster_inventory( def _stream_batch_sources(projections): plan = function_step_plan("StreamOwned") - arrays = [np.full((2, 3), index, dtype=np.float32) for index in range(len(projections))] - paths = [str(plan.output_dir / f"A01_s1_w{index}.tif") for index in range(len(projections))] + arrays = [ + np.full((2, 3), index, dtype=np.float32) for index in range(len(projections)) + ] + paths = [ + str(plan.output_dir / f"A01_s1_w{index}.tif") + for index in range(len(projections)) + ] records = tuple( ProducedOutputSemantics.from_output( plan, @@ -2635,7 +2654,8 @@ def _stream_batch_sources(projections): source="stream source", ), output_context=AlignedImageSliceContext.main_flow( - output_key=f"image-{index}", projection_key=projection, + output_key=f"image-{index}", + projection_key=projection, artifact_kind=ImageArtifactType.value, ), ) @@ -2644,8 +2664,12 @@ def _stream_batch_sources(projections): return arrays, paths, records -def test_stream_batch_owns_correlated_projection_order_and_frozen_input_lists(monkeypatch): - arrays, paths, records = _stream_batch_sources(("second", "first", "second", "first")) +def test_stream_batch_owns_correlated_projection_order_and_frozen_input_lists( + monkeypatch, +): + arrays, paths, records = _stream_batch_sources( + ("second", "first", "second", "first") + ) expected_arrays, expected_paths = tuple(arrays), tuple(paths) observed = [] original_project = StreamOutputBatch.project_item @@ -2658,7 +2682,9 @@ def project(request): monkeypatch.setattr(StreamOutputBatch, "project_item", staticmethod(project)) batches = StreamOutputBatch.from_projection_groups( - parser=ImageXpressFilenameParser(), payloads=arrays, paths=paths, + parser=ImageXpressFilenameParser(), + payloads=arrays, + paths=paths, produced_outputs=records, ) assert observed == [expected_paths[index] for index in (0, 2, 1, 3)] @@ -2668,14 +2694,20 @@ def project(request): records[index].producer_identity for index in (0, 2, 1, 3) ] for item, index in zip( - (item for batch in batches for item in batch.items), (0, 2, 1, 3), strict=True, + (item for batch in batches for item in batch.items), + (0, 2, 1, 3), + strict=True, ): assert item.data is expected_arrays[index] assert item.source_component_metadata["channel"] == str(index) -def test_stream_batch_keeps_first_projection_error_and_prior_callback_effects(monkeypatch): - arrays, paths, records = _stream_batch_sources(("first", "second", "first", "second")) +def test_stream_batch_keeps_first_projection_error_and_prior_callback_effects( + monkeypatch, +): + arrays, paths, records = _stream_batch_sources( + ("first", "second", "first", "second") + ) observed = [] failure = ValueError("actual projector failure") original_project = StreamOutputBatch.project_item @@ -2689,7 +2721,9 @@ def project(request): monkeypatch.setattr(StreamOutputBatch, "project_item", staticmethod(project)) with pytest.raises(ValueError) as raised: StreamOutputBatch.from_projection_groups( - parser=ImageXpressFilenameParser(), payloads=arrays, paths=paths, + parser=ImageXpressFilenameParser(), + payloads=arrays, + paths=paths, produced_outputs=records, ) assert raised.value is failure @@ -2702,23 +2736,37 @@ def test_stream_batch_validates_cardinality_and_routes_before_projecting(monkeyp def forbidden_project(request): raise AssertionError("invalid inputs reached projection") - monkeypatch.setattr(StreamOutputBatch, "project_item", staticmethod(forbidden_project)) + monkeypatch.setattr( + StreamOutputBatch, "project_item", staticmethod(forbidden_project) + ) parser = ImageXpressFilenameParser() with pytest.raises(ValueError, match="payload/path cardinality mismatch"): StreamOutputBatch.from_projection_groups( - parser=parser, payloads=arrays, paths=paths[:1], produced_outputs=(), + parser=parser, + payloads=arrays, + paths=paths[:1], + produced_outputs=(), ) with pytest.raises(ValueError, match="payload/output-record cardinality mismatch"): StreamOutputBatch.from_projection_groups( - parser=parser, payloads=arrays, paths=paths, produced_outputs=records[:1], + parser=parser, + payloads=arrays, + paths=paths, + produced_outputs=records[:1], ) with pytest.raises(ValueError, match="at least one produced output record"): StreamOutputBatch.from_projection_groups( - parser=parser, payloads=(), paths=(), produced_outputs=(), + parser=parser, + payloads=(), + paths=(), + produced_outputs=(), ) with pytest.raises(ValueError, match="cannot mix producer projections"): StreamOutputBatch.from_projection( - parser=parser, payloads=arrays, paths=paths, produced_outputs=records, + parser=parser, + payloads=arrays, + paths=paths, + produced_outputs=records, ) @@ -2780,7 +2828,10 @@ def test_whole_volume_checkpoint_publication_retains_exact_producer_scope( ) assert str(contextualized.source_component_metadata["timepoint"]) == "1" assert "z_index" not in contextualized.source_component_metadata - assert contextualized.source_image_provenance_planes == metadata.source_image_provenance_planes + assert ( + contextualized.source_image_provenance_planes + == metadata.source_image_provenance_planes + ) manager = FileManager( {"disk": DiskStorageBackend(), "memory": MemoryStorageBackend()} ) @@ -2850,9 +2901,7 @@ def test_whole_volume_checkpoint_publication_retains_exact_producer_scope( path=output_plan.path, backend=Backend.MEMORY.value, ) - materializations = ArtifactMaterializationTargetPlan.materialize( - context, plan - ) + materializations = ArtifactMaterializationTargetPlan.materialize(context, plan) assert { Path(output.path) for artifact in materializations @@ -2866,19 +2915,14 @@ def test_whole_volume_checkpoint_publication_retains_exact_producer_scope( assert projection.address is None assert projection.source_alias == "DNA" assert ( - projection.execution_scope.value_text_for_component(Microscopy.Channel) - == "2" - ) - assert ( - projection.execution_scope.value_text_for_component(Microscopy.Site) == "1" + projection.execution_scope.value_text_for_component(Microscopy.Channel) == "2" ) + assert projection.execution_scope.value_text_for_component(Microscopy.Site) == "1" assert ( - projection.execution_scope.value_text_for_component(Microscopy.Timepoint) - == "1" + projection.execution_scope.value_text_for_component(Microscopy.Timepoint) == "1" ) assert ( - projection.execution_scope.value_text_for_component(Microscopy.ZIndex) - is None + projection.execution_scope.value_text_for_component(Microscopy.ZIndex) is None ) assert ( projection.image_metadata.source_image_provenance_planes.component_metadata diff --git a/tests/unit/test_function_reference_identity.py b/tests/unit/test_function_reference_identity.py index fbf967e1e4..e7af6b453b 100644 --- a/tests/unit/test_function_reference_identity.py +++ b/tests/unit/test_function_reference_identity.py @@ -99,8 +99,7 @@ def test_cold_external_resolution_retains_registry_classified_contract( resolved = reference.resolve() assert ( - CallableContract.from_callable(resolved).processing_contract - is FlexibleContract + CallableContract.from_callable(resolved).processing_contract is FlexibleContract ) @@ -143,11 +142,14 @@ def test_module_export_source_keeps_exact_registry_transport_identity(monkeypatc metadata = RegistryService.declared_metadata_for_callable( cellprofiler_backend.crop )[1] - monkeypatch.setattr(RegistryService, "_metadata_cache", {metadata.composite_key: metadata}) + monkeypatch.setattr( + RegistryService, "_metadata_cache", {metadata.composite_key: metadata} + ) monkeypatch.setattr(RegistryService, "_resolved_reference_callables", {}) reference = FunctionReferenceTransportAuthority.function_reference(metadata.func) exported = FunctionReferenceTransportAuthority.importable_function( - metadata.import_identity.module_name, metadata.import_identity.function_name, + metadata.import_identity.module_name, + metadata.import_identity.function_name, ) if exported is metadata.func: assert isinstance(reference, ModuleExportRegistryFunctionReference) diff --git a/tests/unit/test_function_runtime_source_projection.py b/tests/unit/test_function_runtime_source_projection.py index af776f9a81..3e52a96930 100644 --- a/tests/unit/test_function_runtime_source_projection.py +++ b/tests/unit/test_function_runtime_source_projection.py @@ -143,7 +143,8 @@ def _anchor_executor( ) executor.context.runtime_source_workspace_projections = ( WorkspaceSourceProjections.from_context( - executor.context, cache=source_workspace_projection_cache, + executor.context, + cache=source_workspace_projection_cache, ) ) if output_manifest is not None: @@ -295,7 +296,15 @@ def test_workspace_source_files_select_exact_projection_roles( plate_path = tmp_path / "plate" canonical_path = "A01_s001_w1_z001_t001.tif" artifact_path = f"_source/Illumination/{canonical_path}" - address = OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, "1"), (Microscopy.Channel, "1"), (Microscopy.ZIndex, "1"), (Microscopy.Timepoint, "1"))) + address = OpenHCSPlaneAddress( + ( + (Microscopy.Well, "A01"), + (Microscopy.Site, "1"), + (Microscopy.Channel, "1"), + (Microscopy.ZIndex, "1"), + (Microscopy.Timepoint, "1"), + ) + ) plane_projection = SourcePlaneProjection( address=address, ref=SourcePixelRef("disk", "/source/image.tif"), @@ -334,7 +343,15 @@ def test_workspace_logical_identity_requires_paired_projection_declaration( path = "A01_s001_w1_z001_t001.tif" full_path = str(tmp_path / path) first = SourcePlaneProjection( - address=OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, 1), (Microscopy.Channel, 1), (Microscopy.ZIndex, 1), (Microscopy.Timepoint, 1))), + address=OpenHCSPlaneAddress( + ( + (Microscopy.Well, "A01"), + (Microscopy.Site, 1), + (Microscopy.Channel, 1), + (Microscopy.ZIndex, 1), + (Microscopy.Timepoint, 1), + ) + ), ref=SourcePixelRef("disk", "/physical/shared.tif"), source_alias="Original", ) @@ -350,7 +367,16 @@ def test_workspace_logical_identity_requires_paired_projection_declaration( # Sharing a backend reference does not prove a shared logical declaration. declarations[full_path] = replace( - first, address=OpenHCSPlaneAddress(((Microscopy.Well, "B01"), (Microscopy.Site, 1), (Microscopy.Channel, 1), (Microscopy.ZIndex, 1), (Microscopy.Timepoint, 1))) + first, + address=OpenHCSPlaneAddress( + ( + (Microscopy.Well, "B01"), + (Microscopy.Site, 1), + (Microscopy.Channel, 1), + (Microscopy.ZIndex, 1), + (Microscopy.Timepoint, 1), + ) + ), ) assert projection.logical_path_for(lookup) == full_path assert projection.source_path_for(lookup) == "/physical/shared.tif" @@ -367,7 +393,15 @@ def test_workspace_source_projection_carries_exact_aliases_into_stack_provenance projection_set = SourceProjectionSet( tuple( SourcePlaneProjection( - address=OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, "1"), (Microscopy.Channel, str(channel)), (Microscopy.ZIndex, "1"), (Microscopy.Timepoint, "1"))), + address=OpenHCSPlaneAddress( + ( + (Microscopy.Well, "A01"), + (Microscopy.Site, "1"), + (Microscopy.Channel, str(channel)), + (Microscopy.ZIndex, "1"), + (Microscopy.Timepoint, "1"), + ) + ), ref=SourcePixelRef("disk", str(source_path)), source_alias=alias, ) @@ -424,12 +458,10 @@ def test_workspace_source_projection_carries_exact_aliases_into_stack_provenance for virtual_path in subdirectory["image_files"] ) == ("Raw", "Illum") assert tuple( - payload.metadata.source_image_names - for payload in projected_payloads + payload.metadata.source_image_names for payload in projected_payloads ) == (("Raw",), ("Illum",)) assert all( - "source_alias" - not in (payload.metadata.source_component_metadata or {}) + "source_alias" not in (payload.metadata.source_component_metadata or {}) for payload in projected_payloads ) @@ -468,7 +500,15 @@ def test_workspace_replay_preserves_collapsed_semantic_identity( projection_set = SourceProjectionSet( ( SourcePlaneProjection( - address=OpenHCSPlaneAddress(((Microscopy.Well, "A14"), (Microscopy.Site, 1), (Microscopy.Channel, 1), (Microscopy.ZIndex, 1), (Microscopy.Timepoint, 1))), + address=OpenHCSPlaneAddress( + ( + (Microscopy.Well, "A14"), + (Microscopy.Site, 1), + (Microscopy.Channel, 1), + (Microscopy.ZIndex, 1), + (Microscopy.Timepoint, 1), + ) + ), ref=SourcePixelRef("disk", "/outputs/A14_mosaic.tif"), source_alias="neurite", image_metadata=persisted_metadata, @@ -538,7 +578,15 @@ def test_workspace_source_loading_preserves_declared_tiff_intensity_scale( ) virtual_path = "A01_s001_w1_z001_t001.tif" source_plane = SourcePlaneProjection( - address=OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, "1"), (Microscopy.Channel, "1"), (Microscopy.ZIndex, "1"), (Microscopy.Timepoint, "1"))), + address=OpenHCSPlaneAddress( + ( + (Microscopy.Well, "A01"), + (Microscopy.Site, "1"), + (Microscopy.Channel, "1"), + (Microscopy.ZIndex, "1"), + (Microscopy.Timepoint, "1"), + ) + ), ref=SourcePixelRef("disk", str(source_path)), source_alias="OrigBlue", ) @@ -1066,9 +1114,10 @@ def test_step_output_manifest_pattern_lookup_returns_producer_memory_paths( ) index = store.producer_record_index_for(consumer, SourceSchemaFilenameParser()) - assert [record.output_path for record in index.matching_records( - "A14_s{iii}_w3_z001_t001.tif" - )] == [ + assert [ + record.output_path + for record in index.matching_records("A14_s{iii}_w3_z001_t001.tif") + ] == [ str(output_dir / "A14_s001_w3_z001_t001.tif"), str(output_dir / "A14_s002_w3_z001_t001.tif"), ] @@ -2848,7 +2897,11 @@ def main_flow_with_auxiliary_source(image): spec=spec, storage_plan=None, projection=None, - main_flow_projection=(MainFlowInputProjection.DECLARED_SOURCE_IMAGE if input_index == 0 else None), + main_flow_projection=( + MainFlowInputProjection.DECLARED_SOURCE_IMAGE + if input_index == 0 + else None + ), ) for input_index, (edge_key, spec) in enumerate( zip( @@ -3152,7 +3205,6 @@ def measure_neurites(image, pixel_size=1.0): spec=pixel_size_spec, storage_plan=None, projection=None, - ), ) ) @@ -3290,7 +3342,12 @@ def test_pipeline_start_main_flow_survives_prior_producer_image_input( assert mask_edge.storage_plan is not None assert (mask_edge.main_flow_projection is not None) is False assert invocation.contract.accepts_implicit_main_flow_input is True - assert compiled_pattern.default_group.main_flow_input_refs(source_bindings=CompiledSourceBindingPlan.empty()) is None + assert ( + compiled_pattern.default_group.main_flow_input_refs( + source_bindings=CompiledSourceBindingPlan.empty() + ) + is None + ) assert tuple( binding.alias for binding in scope.main_flow_source_binding_plan.binding_declarations @@ -3733,7 +3790,15 @@ def test_alias_only_workspace_filter_excludes_unselected_source_and_orders_stack ) source_planes = tuple( SourcePlaneProjection( - address=OpenHCSPlaneAddress(((Microscopy.Well, "R04C09"), (Microscopy.Site, "11"), (Microscopy.Channel, channel), (Microscopy.ZIndex, "1"), (Microscopy.Timepoint, "1"))), + address=OpenHCSPlaneAddress( + ( + (Microscopy.Well, "R04C09"), + (Microscopy.Site, "11"), + (Microscopy.Channel, channel), + (Microscopy.ZIndex, "1"), + (Microscopy.Timepoint, "1"), + ) + ), ref=SourcePixelRef("disk", f"/source/ch{channel}.tiff"), source_alias=binding.alias, ) @@ -3819,7 +3884,15 @@ def test_unbound_workspace_source_keeps_filename_component_provenance( source_metadata_by_path={virtual_path: source_metadata}, source_projections_by_virtual_path={ virtual_path: SourcePlaneProjection( - address=OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, "2"), (Microscopy.Channel, "1"), (Microscopy.ZIndex, "3"), (Microscopy.Timepoint, "4"))), + address=OpenHCSPlaneAddress( + ( + (Microscopy.Well, "A01"), + (Microscopy.Site, "2"), + (Microscopy.Channel, "1"), + (Microscopy.ZIndex, "3"), + (Microscopy.Timepoint, "4"), + ) + ), ref=source_ref, source_alias="OrigDNA", source_metadata=source_metadata, @@ -3882,7 +3955,9 @@ def resolve_address(backend_address, backend, *, base_path): Microscopy.Well.name: "A01", SourceFilterSubject.EXTENSION.value: ".tif", } - assert projected.metadata.source_provenance.represented_source_image_names == ("OrigDNA",) + assert projected.metadata.source_provenance.represented_source_image_names == ( + "OrigDNA", + ) def test_step_output_load_filter_skips_source_binding_filter() -> None: @@ -3966,7 +4041,15 @@ def test_producer_anchored_pipeline_start_paths_use_exact_source_projection_bund ) source_planes = tuple( SourcePlaneProjection( - address=OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, "1"), (Microscopy.Channel, str(channel)), (Microscopy.ZIndex, "1"), (Microscopy.Timepoint, "1"))), + address=OpenHCSPlaneAddress( + ( + (Microscopy.Well, "A01"), + (Microscopy.Site, "1"), + (Microscopy.Channel, str(channel)), + (Microscopy.ZIndex, "1"), + (Microscopy.Timepoint, "1"), + ) + ), ref=SourcePixelRef("disk", str(source_path)), source_alias=alias, ) @@ -4044,7 +4127,9 @@ def resolve_address(backend_address, backend, *, base_path): "source_workspace_projection_authority", lambda _self: SimpleNamespace( projection_if_available=lambda: projection, - projection_or_empty=lambda *, axis_id=None: projection.filtered_by_axis(axis_id=axis_id), + projection_or_empty=lambda *, axis_id=None: projection.filtered_by_axis( + axis_id=axis_id + ), ), ) @@ -4944,12 +5029,14 @@ def test_function_output_path_uses_payload_identity_over_input_carrier( }, ).payload_with(np.zeros((4, 5), dtype=np.float32), None) - output_path = FunctionOutputIdentity.path_from_request(FunctionOutputPathRequest( + output_path = FunctionOutputIdentity.path_from_request( + FunctionOutputPathRequest( parser=SourceSchemaFilenameParser(), output_dir=tmp_path, output_payload=payload, input_path="A14_s001_w1_z001_t001.tif", - )) + ) + ) assert output_path.name == "A14_s002_w3_z001_t001.tif" @@ -4968,12 +5055,14 @@ def test_function_output_path_uses_payload_identity_without_input_path( }, ).payload_with(np.zeros((4, 5), dtype=np.float32), None) - output_path = FunctionOutputIdentity.path_from_request(FunctionOutputPathRequest( + output_path = FunctionOutputIdentity.path_from_request( + FunctionOutputPathRequest( parser=SourceSchemaFilenameParser(), output_dir=tmp_path, output_payload=payload, input_path=None, - )) + ) + ) assert output_path.name == "A14_s001_w5_z001_t001.tif" @@ -4999,10 +5088,7 @@ def test_function_output_identity_completes_partial_payload_metadata_from_fallba ) assert identity is not None - assert ( - identity.filename(parser) - == "Sequence1_s001_w1_z001_t000.tif" - ) + assert identity.filename(parser) == "Sequence1_s001_w1_z001_t000.tif" def test_function_output_identity_uses_fallback_path_extension_for_payload_identity() -> ( @@ -5026,10 +5112,7 @@ def test_function_output_identity_uses_fallback_path_extension_for_payload_ident ) assert identity is not None - assert ( - identity.filename(parser) - == "A01_s001_w2_z001_t001.png" - ) + assert identity.filename(parser) == "A01_s001_w2_z001_t001.png" @pytest.mark.parametrize( @@ -5048,9 +5131,7 @@ def test_source_stack_output_takes_image_extension_not_artifact_key( metadata = ImagePayloadMetadata( source_provenance=SourceImageProvenance( source_image_provenance_planes=SourceImageProvenancePlanes.from_components( - paths=tuple( - f"A01_s001_w0_z{z:03d}_t001.tif" for z in (1, 2, 3) - ), + paths=tuple(f"A01_s001_w0_z{z:03d}_t001.tif" for z in (1, 2, 3)), component_metadata=tuple( { "well": "A01", @@ -5099,12 +5180,14 @@ def test_function_output_path_uses_input_identity_for_multi_plane_carrier( ), ).payload_with(np.zeros((2, 4, 5), dtype=np.float32), None) - output_path = FunctionOutputIdentity.path_from_request(FunctionOutputPathRequest( + output_path = FunctionOutputIdentity.path_from_request( + FunctionOutputPathRequest( parser=SourceSchemaFilenameParser(), output_dir=tmp_path, output_payload=payload, input_path="A14_s001_w1_z001_t001.tif", - )) + ) + ) assert output_path.name == "A14_s001_w1_z001_t001.tif" @@ -5122,12 +5205,14 @@ def test_function_output_path_rejects_multi_plane_carrier_without_input_path( ).payload_with(np.zeros((2, 4, 5), dtype=np.float32), None) with pytest.raises(ValueError, match="multi-plane source provenance"): - FunctionOutputIdentity.path_from_request(FunctionOutputPathRequest( + FunctionOutputIdentity.path_from_request( + FunctionOutputPathRequest( parser=SourceSchemaFilenameParser(), output_dir=tmp_path, output_payload=payload, input_path=None, - )) + ) + ) def test_function_output_path_uses_variable_component_identity( @@ -5239,10 +5324,7 @@ def test_collapsed_output_identity_uses_retained_source_contributors( assert identity.filename_component_values is not None assert identity.filename_component_values["site"] == 1 assert filename_identity is not None - assert ( - filename_identity.filename(request.parser) - == "A14_s001_w1_z001_t001.tif" - ) + assert filename_identity.filename(request.parser) == "A14_s001_w1_z001_t001.tif" def test_composite_then_z_collapse_uses_current_scalar_identity( @@ -5386,10 +5468,7 @@ def test_function_output_path_rejects_variation_outside_identity_components( ).payload_with(np.zeros((2, 4, 5), dtype=np.float32), None) assert payload.metadata.source_provenance.source_plane_count == 2 - assert ( - payload.metadata.source_image_provenance_planes.contributor_count - == 0 - ) + assert payload.metadata.source_image_provenance_planes.contributor_count == 0 with pytest.raises(ValueError, match="varies outside identity components"): FunctionOutputIdentity.from_request( @@ -5602,15 +5681,14 @@ def save_batch( expected += "_Corrected" assert Path(filemanager.saved_paths[0]).name == expected + ".tif" assert ( - filemanager.saved_payloads[0].metadata.source_component_metadata[ - "site" - ] - == "1" + filemanager.saved_payloads[0].metadata.source_component_metadata["site"] == "1" ) assert records[0].component_values["site"] == 1 if named_topology != "anonymous": assert records[0].output_context == named_context - assert filemanager.saved_payloads[0].metadata.source_image_names == ("Corrected",) + assert filemanager.saved_payloads[0].metadata.source_image_names == ( + "Corrected", + ) assert filemanager.saved_payloads[0].data is payload.data @@ -5644,8 +5722,7 @@ def qualified_producer_manifest(tmp_path): records.append( ProducedOutputSemantics.from_output( producer, - tmp_path - / identity.filename(parser), + tmp_path / identity.filename(parser), identity, output_context=AlignedImageSliceContext.main_flow( output_key=f"Output{plane}", @@ -5671,9 +5748,7 @@ def count_filename(identity, parser): calls.append(identity) return original(identity, parser) - monkeypatch.setattr( - FunctionOutputIdentity, "filename", count_filename - ) + monkeypatch.setattr(FunctionOutputIdentity, "filename", count_filename) paths = ( records[1].output_path, "A01_s001_w2_z001_t001.tif", @@ -5795,7 +5870,9 @@ def test_producer_admission_rederives_live_storage_aliases_each_epoch( ): store, producer, consumer, records, parser = qualified_producer_manifest storage_components = dict(records[0].filename_values) - record = replace(records[0], filename_component_values=storage_components).published() + record = replace( + records[0], filename_component_values=storage_components + ).published() store.record_outputs(producer, (record,)) old_index = store.producer_record_index_for(consumer, parser) old_alias = record.without_filename_qualifier().filename(parser) diff --git a/tests/unit/test_function_step_execution_plan.py b/tests/unit/test_function_step_execution_plan.py index cb7a2bd523..39134d3cdf 100644 --- a/tests/unit/test_function_step_execution_plan.py +++ b/tests/unit/test_function_step_execution_plan.py @@ -521,7 +521,8 @@ def test_invocation_output_selection_omits_inactive_component_outputs(): {channel_one.ref(): channel_one}, compiled_output_plans=invocation.output_plans_for_component( ComponentGroupScope.from_raw( - ("1", "2"), component=Microscopy.Channel, + ("1", "2"), + component=Microscopy.Channel, ), "1", ), @@ -711,7 +712,9 @@ def publish_derived_channels(image, *, illumination_function=None, runtime): ) selected_outputs = invocation.select_outputs( {active_output.ref(): active_output}, - compiled_output_plans=invocation.output_plans_for_component(execution_scope, "1"), + compiled_output_plans=invocation.output_plans_for_component( + execution_scope, "1" + ), ) assert tuple(edge.spec for edge in selected_inputs.values()) == ( @@ -728,7 +731,9 @@ def publish_derived_channels(image, *, illumination_function=None, runtime): ) second_selected_outputs = invocation.select_outputs( {second_active_output.ref(): second_active_output}, - compiled_output_plans=invocation.output_plans_for_component(execution_scope, "2"), + compiled_output_plans=invocation.output_plans_for_component( + execution_scope, "2" + ), ) assert tuple(edge.spec for edge in second_selected_inputs.values()) == ( @@ -747,7 +752,8 @@ def test_invocation_output_selection_rejects_missing_active_component_output(): {}, compiled_output_plans=invocation.output_plans_for_component( ComponentGroupScope.from_raw( - ("1", "2"), component=Microscopy.Channel, + ("1", "2"), + component=Microscopy.Channel, ), "1", ), @@ -770,7 +776,8 @@ def test_invocation_output_selection_rejects_active_projection_drift(): {drifted_channel_one.ref(): drifted_channel_one}, compiled_output_plans=invocation.output_plans_for_component( ComponentGroupScope.from_raw( - ("1", "2"), component=Microscopy.Channel, + ("1", "2"), + component=Microscopy.Channel, ), "1", ), diff --git a/tests/unit/test_function_step_execution_scope.py b/tests/unit/test_function_step_execution_scope.py index bb7f3409a3..086e9aa4d9 100644 --- a/tests/unit/test_function_step_execution_scope.py +++ b/tests/unit/test_function_step_execution_scope.py @@ -141,17 +141,25 @@ def test_runtime_axis_scope_returns_none_only_without_declared_component_axis() def test_runtime_axis_scope_joins_correlated_rows_without_inventing_states() -> None: rows = tuple( RuntimeExecutionAxisScope.from_raw( - "A01", component=None, value=None, - fixed_component_values=((Microscopy.Site, site), (Microscopy.Timepoint, timepoint)), + "A01", + component=None, + value=None, + fixed_component_values=( + (Microscopy.Site, site), + (Microscopy.Timepoint, timepoint), + ), ) for site, timepoint in (("1", "3"), ("2", "4")) ) time_three = RuntimeExecutionAxisScope.from_raw( - "A01", component=None, value=None, + "A01", + component=None, + value=None, fixed_component_values=((Microscopy.Timepoint, "3"),), ) joined = tuple( - result for row in rows + result + for row in rows if (result := row.join_execution_cohort(time_three)) is not None ) assert joined == (rows[0],) @@ -161,7 +169,9 @@ def test_runtime_axis_scope_joins_correlated_rows_without_inventing_states() -> def test_runtime_axis_scope_empty_context_broadcasts_to_actual_correlated_row() -> None: empty = RuntimeExecutionAxisScope.from_raw("A01", component=None, value=None) source = RuntimeExecutionAxisScope.from_raw( - "A01", component=None, value=None, + "A01", + component=None, + value=None, fixed_component_values=((Microscopy.Site, "1"), (Microscopy.Timepoint, "3")), ) assert empty.join_execution_cohort(source) == source @@ -169,29 +179,46 @@ def test_runtime_axis_scope_empty_context_broadcasts_to_actual_correlated_row() assert empty.join_execution_cohort(empty) == empty -@pytest.mark.parametrize("axis,component,value", [ - ("B01", Microscopy.Channel, "1"), - ("A01", Microscopy.Site, "1"), - ("A01", Microscopy.Channel, "2"), -]) -def test_runtime_axis_scope_cannot_join_other_execution_group(axis, component, value) -> None: - own = RuntimeExecutionAxisScope.from_raw("A01", component=Microscopy.Channel, value="1") +@pytest.mark.parametrize( + "axis,component,value", + [ + ("B01", Microscopy.Channel, "1"), + ("A01", Microscopy.Site, "1"), + ("A01", Microscopy.Channel, "2"), + ], +) +def test_runtime_axis_scope_cannot_join_other_execution_group( + axis, component, value +) -> None: + own = RuntimeExecutionAxisScope.from_raw( + "A01", component=Microscopy.Channel, value="1" + ) other = RuntimeExecutionAxisScope.from_raw(axis, component=component, value=value) assert own.join_execution_cohort(other) is None def test_runtime_axis_scope_join_derives_partial_fixed_context_union() -> None: left = RuntimeExecutionAxisScope.from_raw( - "A01", component=None, value=None, + "A01", + component=None, + value=None, fixed_component_values=((Microscopy.Site, "1"), (Microscopy.Timepoint, "3")), ) right = RuntimeExecutionAxisScope.from_raw( - "A01", component=None, value=None, + "A01", + component=None, + value=None, fixed_component_values=((Microscopy.Timepoint, "3"), (Microscopy.ZIndex, "2")), ) expected = RuntimeExecutionAxisScope.from_raw( - "A01", component=None, value=None, - fixed_component_values=((Microscopy.Site, "1"), (Microscopy.Timepoint, "3"), (Microscopy.ZIndex, "2")), + "A01", + component=None, + value=None, + fixed_component_values=( + (Microscopy.Site, "1"), + (Microscopy.Timepoint, "3"), + (Microscopy.ZIndex, "2"), + ), ) assert left.join_execution_cohort(right) == expected assert right.join_execution_cohort(left) == expected @@ -912,7 +939,8 @@ def export(*, artifact_batch: RuntimeArtifactBatch): metadata_writer=True, ) plan.runtime_artifact_materialization = RuntimeArtifactMaterializationPlan( - persistent_enabled=True, persistent_backend=Backend.MEMORY.value, + persistent_enabled=True, + persistent_backend=Backend.MEMORY.value, ) plan.output_dataset_root = str(tmp_path) plan.output_dir = tmp_path / "images" @@ -920,7 +948,8 @@ def export(*, artifact_batch: RuntimeArtifactBatch): context = _plate_context("A01", (plan,)) context.metadata_cache = {} context.microscope_handler = SimpleNamespace( - parser=SourceSchemaFilenameParser(), source_name="test", + parser=SourceSchemaFilenameParser(), + source_name="test", ) _record_measurements( context, @@ -943,11 +972,14 @@ def export(*, artifact_batch: RuntimeArtifactBatch): assert {record.key.name for record in observation.contexts[0].records} == { output_spec.name } - analysis_inputs = observation.contexts[0].outputs.hook_observations.get("analysis_consolidation") + analysis_inputs = observation.contexts[0].outputs.hook_observations.get( + "analysis_consolidation" + ) assert analysis_inputs is not None assert analysis_inputs.destination.backend == Backend.MEMORY.value (table,) = tuple( - output for group in analysis_inputs.outputs_by_directory.values() + output + for group in analysis_inputs.outputs_by_directory.values() for output in group ) assert table.csv_content == "ImageNumber,Count\r\n1,7\r\n" diff --git a/tests/unit/test_function_step_transport.py b/tests/unit/test_function_step_transport.py index 498f73a3e8..c52b07b209 100644 --- a/tests/unit/test_function_step_transport.py +++ b/tests/unit/test_function_step_transport.py @@ -22,7 +22,10 @@ PipelineConfig, ) from openhcs.core.callable_contract import CallableContract -from openhcs.core.function_patterns import CompiledFunctionInvocation, FunctionInvocationKey +from openhcs.core.function_patterns import ( + CompiledFunctionInvocation, + FunctionInvocationKey, +) from openhcs.core.function_reference import ( FunctionReference, FunctionReferenceTransportAuthority, @@ -312,9 +315,7 @@ def test_transport_preserves_explicit_lazy_processing_defaults() -> None: exec(compile(source, "", "exec"), namespace) restored = FunctionStepTransportAuthority.pipeline_steps_from_namespace(namespace) - assert restored[0].processing_config.variable_components == [ - Microscopy.Channel - ] + assert restored[0].processing_config.variable_components == [Microscopy.Channel] assert restored[0].processing_config.group_by is Microscopy.Site @@ -400,7 +401,10 @@ def test_bundle_transport_preserves_prepared_runtime_contract_identity() -> None ) from openhcs.core.compiled_step_plan import CompiledStepPlan from openhcs.core.context.processing_context import ProcessingContext - from openhcs.core.function_patterns import compile_function_pattern, normalize_function_pattern + from openhcs.core.function_patterns import ( + compile_function_pattern, + normalize_function_pattern, + ) from openhcs.core.orchestrator.execution_result import ( RuntimeExecutionTransportSerialization, ) @@ -447,59 +451,85 @@ def test_bundle_transport_preserves_prepared_runtime_contract_identity() -> None ) shared_items = replace( captured, - groups=(replace( - captured.groups[0], - items=(captured_item, replace(captured_item, key=replace(captured_item.key, position=1))), - ),), + groups=( + replace( + captured.groups[0], + items=( + captured_item, + replace(captured_item, key=replace(captured_item.key, position=1)), + ), + ), + ), ) first_item, second_item = tuple( - FunctionStepTransportAuthority.normalize_function_spec(shared_items).iter_items() + FunctionStepTransportAuthority.normalize_function_spec( + shared_items + ).iter_items() ) assert first_item.contract is second_item.contract repeated_group = replace( pattern.groups[0], - invocations=(invocation, replace(invocation, key=replace(invocation.key, position=1))), + invocations=( + invocation, + replace(invocation, key=replace(invocation.key, position=1)), + ), ) first_invocation, second_invocation = ( - FunctionStepTransportAuthority.normalize_compiled_group(repeated_group).invocations + FunctionStepTransportAuthority.normalize_compiled_group( + repeated_group + ).invocations ) assert first_invocation.contract is second_invocation.contract prepared_callable = invocation.contract.resolve_runtime_callable() plan = CompiledStepPlan( - step_index=0, step_name="Crop", axis_id="A01", + step_index=0, + step_name="Crop", + axis_id="A01", compiled_function_pattern=pattern, ) context = ProcessingContext( - axis_id="A01", step_plans={0: plan}, + axis_id="A01", + step_plans={0: plan}, filemanager=FileManager({Backend.MEMORY.value: MemoryStorageBackend()}), ) context.freeze() second_invocation = replace( - invocation, kwargs=(("left_right_rectangle_positions", (2, 5)),), + invocation, + kwargs=(("left_right_rectangle_positions", (2, 5)),), ) second_pattern = replace( - pattern, groups=(replace(pattern.groups[0], invocations=(second_invocation,)),), + pattern, + groups=(replace(pattern.groups[0], invocations=(second_invocation,)),), ) second_plan = replace(plan, axis_id="A02", compiled_function_pattern=second_pattern) second_context = ProcessingContext( - axis_id="A02", step_plans={0: second_plan}, filemanager=context.filemanager, + axis_id="A02", + step_plans={0: second_plan}, + filemanager=context.filemanager, ) second_context.freeze() contexts = {"A01": context, "A02": second_context} bundle = CompiledExecutionBundle.from_runtime_contexts( - pipeline_definition=(), runtime_contexts=contexts, + pipeline_definition=(), + runtime_contexts=contexts, worker_assignments={"worker_0": ["A01", "A02"]}, runtime_environment=CompiledRuntimeEnvironmentPlan.from_global_config( - GlobalPipelineConfig(), compiled_contexts=contexts, server_mode=False, + GlobalPipelineConfig(), + compiled_contexts=contexts, + server_mode=False, ), ) transport_context = bundle.transport_contexts["A01"] transport_plan = transport_context.step_plans[0] assert transport_context is not context assert transport_plan is not plan - (transport_invocation,) = tuple(transport_plan.compiled_function_pattern.iter_invocations()) + (transport_invocation,) = tuple( + transport_plan.compiled_function_pattern.iter_invocations() + ) second_transport_invocation = next( - bundle.transport_contexts["A02"].step_plans[0].compiled_function_pattern.iter_invocations() + bundle.transport_contexts["A02"] + .step_plans[0] + .compiled_function_pattern.iter_invocations() ) assert transport_invocation.contract is second_transport_invocation.contract assert transport_invocation.func is second_transport_invocation.func @@ -512,7 +542,9 @@ def test_bundle_transport_preserves_prepared_runtime_contract_identity() -> None assert plan.compiled_function_pattern is pattern assert invocation.contract.resolve_runtime_callable() is prepared_callable assert transport_context.runtime_value_store is context.runtime_value_store - assert transport_context.runtime_image_stack_cache is context.runtime_image_stack_cache + assert ( + transport_context.runtime_image_stack_cache is context.runtime_image_stack_cache + ) assert transport_context.filemanager is context.filemanager transport_plan.step_name = "Transport-only name" assert plan.step_name == "Crop" @@ -537,10 +569,15 @@ def test_bundle_transport_preserves_prepared_runtime_contract_identity() -> None is restored_invocation.contract.resolve_runtime_callable() ) second_restored_invocation = next( - restored.runtime_contexts["A02"].step_plans[0].compiled_function_pattern.iter_invocations() + restored.runtime_contexts["A02"] + .step_plans[0] + .compiled_function_pattern.iter_invocations() ) assert restored_invocation.contract is second_restored_invocation.contract - assert restored_invocation.runtime_callable is second_restored_invocation.runtime_callable + assert ( + restored_invocation.runtime_callable + is second_restored_invocation.runtime_callable + ) assert second_restored_invocation.kwargs == second_invocation.kwargs assert restored_invocation.contract is not transport_invocation.contract assert ( @@ -569,7 +606,9 @@ def test_generated_source_resolves_catalog_owned_cellprofiler_callable() -> None restored = FunctionStepTransportAuthority.pipeline_steps_from_namespace(namespace) assert len(restored) == 1 assert restored[0].name == "Crop" - assert restored[0].func is RegistryService.registered_callable(cellprofiler_backend.crop) + assert restored[0].func is RegistryService.registered_callable( + cellprofiler_backend.crop + ) def test_registered_custom_function_is_function_step_picklable() -> None: @@ -620,7 +659,10 @@ def codex_pickle_probe(image): restored_invocation = pickle.loads(pickle.dumps(invocation)) assert restored_invocation.runtime_callable is custom_func assert restored_invocation.contract is not contract - assert restored_invocation.artifact_input_edges == invocation.artifact_input_edges + assert ( + restored_invocation.artifact_input_edges + == invocation.artifact_input_edges + ) assert ( CellProfilerModule.require_module("Crop").require_callable() is canonical_crop @@ -648,7 +690,9 @@ def codex_lazy_import_probe(image): encoding="utf-8", ) vars(custom_functions).pop(func_name, None) - monkeypatch.setattr(manager_module, "get_data_file_path", lambda _name, *, create=True: storage_dir) + monkeypatch.setattr( + manager_module, "get_data_file_path", lambda _name, *, create=True: storage_dir + ) namespace: dict[str, object] = {} try: diff --git a/tests/unit/test_g5_processing_contract_guards.py b/tests/unit/test_g5_processing_contract_guards.py index e04da345bc..67b43b8d8c 100644 --- a/tests/unit/test_g5_processing_contract_guards.py +++ b/tests/unit/test_g5_processing_contract_guards.py @@ -182,7 +182,9 @@ def _docstring_ids(tree: ast.AST) -> set[int]: ids = set() for node in ast.walk(tree): if ( - isinstance(node, (ast.Module, ast.ClassDef, ast.FunctionDef, ast.AsyncFunctionDef)) + isinstance( + node, (ast.Module, ast.ClassDef, ast.FunctionDef, ast.AsyncFunctionDef) + ) and node.body and isinstance(node.body[0], ast.Expr) and isinstance(node.body[0].value, ast.Constant) diff --git a/tests/unit/test_g7_runtime_protocol_guards.py b/tests/unit/test_g7_runtime_protocol_guards.py index 7c6c5457a1..ab2f7d42c8 100644 --- a/tests/unit/test_g7_runtime_protocol_guards.py +++ b/tests/unit/test_g7_runtime_protocol_guards.py @@ -86,13 +86,19 @@ def test_worker_connections_carry_declared_messages() -> None: and isinstance(node.func, ast.Attribute) and node.func.attr == "send" and node.args - and isinstance(node.args[0], (ast.Tuple, ast.Constant, ast.List, ast.Dict)) + and isinstance( + node.args[0], (ast.Tuple, ast.Constant, ast.List, ast.Dict) + ) ): offenders.append(f"{path.name}:{node.lineno}") - if isinstance(node, ast.Compare) and any( - isinstance(operand, ast.Constant) and isinstance(operand.value, str) - for operand in (node.left, *node.comparators) - ) and re.search(r"kind|operation|command|message", ast.unparse(node)): + if ( + isinstance(node, ast.Compare) + and any( + isinstance(operand, ast.Constant) and isinstance(operand.value, str) + for operand in (node.left, *node.comparators) + ) + and re.search(r"kind|operation|command|message", ast.unparse(node)) + ): offenders.append(f"{path.name}:{node.lineno}") assert offenders == [] diff --git a/tests/unit/test_gray_to_color_binding_axis_432.py b/tests/unit/test_gray_to_color_binding_axis_432.py index 7e940241bd..370136c258 100644 --- a/tests/unit/test_gray_to_color_binding_axis_432.py +++ b/tests/unit/test_gray_to_color_binding_axis_432.py @@ -6,7 +6,10 @@ import pytest from openhcs.core.component_group_scope import RuntimeExecutionAxisScope -from openhcs.core.aligned_image_payload import (AlignedImageStack, ImagePayloadBundleContext) +from openhcs.core.aligned_image_payload import ( + AlignedImageStack, + ImagePayloadBundleContext, +) from openhcs.core.callable_contract import CallableContract from openhcs.core.artifacts import ArtifactSpec, ImageArtifactType from openhcs.core.config import StepSourceBindingsConfig @@ -25,13 +28,22 @@ from openhcs.interop.cellprofiler.runtime.function_contract_execution import ( CellProfilerFunctionContractExecutor, ) -from openhcs.interop.cellprofiler.runtime.module_execution import CellProfilerModuleExecutor +from openhcs.interop.cellprofiler.runtime.module_execution import ( + CellProfilerModuleExecutor, +) from openhcs.interop.cellprofiler.runtime.output_recording import ( CellProfilerOutputRecorder, ) -from openhcs.processing.backends.cellprofiler.color import GrayToColorModule, gray_to_color -from tests.unit.test_cellprofiler_generic_special_input_binding import _compile_public_step -from tests.unit.cellprofiler_runtime_test_support import cellprofiler_runtime_adapter_for_test +from openhcs.processing.backends.cellprofiler.color import ( + GrayToColorModule, + gray_to_color, +) +from tests.unit.test_cellprofiler_generic_special_input_binding import ( + _compile_public_step, +) +from tests.unit.cellprofiler_runtime_test_support import ( + cellprofiler_runtime_adapter_for_test, +) from openhcs.domains.microscopy.axes import Microscopy from openhcs.core.axes import ColourAxis from openhcs.core.image_payload_execution_mode import ( @@ -41,7 +53,9 @@ @pytest.mark.parametrize("explicit_selector", [False, True]) -def test_declared_fitc_runtime_plane_retains_values_and_physical_identity(explicit_selector): +def test_declared_fitc_runtime_plane_retains_values_and_physical_identity( + explicit_selector, +): kwargs = { "color_scheme": GrayToColorModule.Scheme.STACK, "rescale_intensity": False, @@ -50,40 +64,64 @@ def test_declared_fitc_runtime_plane_retains_values_and_physical_identity(explic if explicit_selector: kwargs["image_name"] = ("FITC",) step = FunctionStep( - func=(gray_to_color, kwargs), name="RetainRawReference", + func=(gray_to_color, kwargs), + name="RetainRawReference", source_bindings=StepSourceBindingsConfig( - enabled=True, bindings=(NamedSourceBinding("FITC"),), + enabled=True, + bindings=(NamedSourceBinding("FITC"),), ), ) invocation = _compile_public_step(step) - contract = replace(invocation.contract, metadata=replace( - invocation.contract.metadata, - runtime_adapter=CellProfilerRuntimeAdapter.runtime_adapter_spec(), - )) + contract = replace( + invocation.contract, + metadata=replace( + invocation.contract.metadata, + runtime_adapter=CellProfilerRuntimeAdapter.runtime_adapter_spec(), + ), + ) pixels = np.arange(20, dtype=np.float32).reshape(4, 5) * 1000 - physical = {"well": "A01", "site": "1", "channel": "2", "z_index": "1", "timepoint": "1"} + physical = { + "well": "A01", + "site": "1", + "channel": "2", + "z_index": "1", + "timepoint": "1", + } metadata = ImagePayloadMetadata( plane_axis=RuntimePlaneAxis.RUNTIME_SLICE, source_image_provenance_planes=SourceImageProvenancePlanes.from_components( - paths=("/synthetic/A01_s1_w2_z1_t1.tif",), component_metadata=(physical,), + paths=("/synthetic/A01_s1_w2_z1_t1.tif",), + component_metadata=(physical,), ), source_image_names=("FITC",), source_voxel_spacing=SourceVoxelSpacing((1.0, 1.0)), - source_spatial_domain=SourceSpatialDomain(origin_yx=(0, 0), source_shape_yx=(4, 5)), + source_spatial_domain=SourceSpatialDomain( + origin_yx=(0, 0), source_shape_yx=(4, 5) + ), ) source = metadata.payload_with(pixels[None], None) runtime = cellprofiler_runtime_adapter_for_test( - runtime_value_store=RuntimeValueStore(), callable_contract=contract, + runtime_value_store=RuntimeValueStore(), + callable_contract=contract, variable_components=(Microscopy.Channel,), - axis_scope=RuntimeExecutionAxisScope.from_raw("synthetic", component=None, value=None), - artifact_inputs={edge.key: InvocationArtifactInputEdgePlan.from_source_declarations( - key=edge.key, spec=edge.spec, main_flow_artifacts=contract.artifact_inputs, - invocation_sources=contract.artifact_inputs, - ) for edge in invocation.artifact_input_edges}, + axis_scope=RuntimeExecutionAxisScope.from_raw( + "synthetic", component=None, value=None + ), + artifact_inputs={ + edge.key: InvocationArtifactInputEdgePlan.from_source_declarations( + key=edge.key, + spec=edge.spec, + main_flow_artifacts=contract.artifact_inputs, + invocation_sources=contract.artifact_inputs, + ) + for edge in invocation.artifact_input_edges + }, ) executor = CellProfilerModuleExecutor(gray_to_color, contract) request = executor._image_request( - source, runtime, module_type=GrayToColorModule, + source, + runtime, + module_type=GrayToColorModule, active_input_specs=contract.artifact_inputs.specs, ) assert contract.artifact_inputs.names() == ("FITC",) @@ -95,22 +133,38 @@ def test_declared_fitc_runtime_plane_retains_values_and_physical_identity(explic assert request.plane_projection.axis is RuntimePlaneAxis.RUNTIME_SLICE assert request.plane_projection.axis_size == 1 assert request.plane_projection.plane_index is None - assert request.payload.slices[0].metadata.plane_axis is RuntimePlaneAxis.SOURCE_BINDING + assert ( + request.payload.slices[0].metadata.plane_axis is RuntimePlaneAxis.SOURCE_BINDING + ) assert request.payload.slices[0].metadata.source_image_names == ("FITC",) execution = executor._invocation_request( - image_request=request, adapter=runtime, current_image=source, + image_request=request, + adapter=runtime, + current_image=source, module_type=GrayToColorModule, - kwargs={"color_scheme": GrayToColorModule.Scheme.STACK, "rescale_intensity": False}, + kwargs={ + "color_scheme": GrayToColorModule.Scheme.STACK, + "rescale_intensity": False, + }, ) result = CellProfilerFunctionContractExecutor().execute( - contract, contract.resolve_canonical_raw_callable(), execution.payload, execution.kwargs, - execution_mode=execution.execution_mode, plane_projection=execution.plane_projection, + contract, + contract.resolve_canonical_raw_callable(), + execution.payload, + execution.kwargs, + execution_mode=execution.execution_mode, + plane_projection=execution.plane_projection, ) # Float32 is the original Stack runner's output contract, not unit normalization. assert result.metadata.plane_axis is RuntimePlaneAxis.RUNTIME_SLICE - plane = RuntimeSliceProjection.value_for_slice(result, RuntimePlaneAxisValueProjection.from_selected_plane( - axis=RuntimePlaneAxis.RUNTIME_SLICE, plane_index=0, axis_size=1, - )) + plane = RuntimeSliceProjection.value_for_slice( + result, + RuntimePlaneAxisValueProjection.from_selected_plane( + axis=RuntimePlaneAxis.RUNTIME_SLICE, + plane_index=0, + axis_size=1, + ), + ) assert plane.data.shape == (4, 5, 1) np.testing.assert_array_equal(plane.data[..., 0], pixels) np.testing.assert_array_equal(source.data[0], pixels) @@ -124,17 +178,27 @@ def test_declared_fitc_runtime_plane_retains_values_and_physical_identity(explic def _source_plane(pixels, alias): return ImagePayloadMetadata( source_path="/synthetic/A01_s1_w2_z1_t1.tif", - source_component_metadata={"well": "A01", "site": "1", "channel": "2", "z_index": "1", "timepoint": "1"}, + source_component_metadata={ + "well": "A01", + "site": "1", + "channel": "2", + "z_index": "1", + "timepoint": "1", + }, source_image_names=(alias,), source_voxel_spacing=SourceVoxelSpacing((1.0, 1.0)), - source_spatial_domain=SourceSpatialDomain(origin_yx=(0, 0), source_shape_yx=pixels.shape[:2]), + source_spatial_domain=SourceSpatialDomain( + origin_yx=(0, 0), source_shape_yx=pixels.shape[:2] + ), ).payload_with(pixels, None) def _execute_bound_stack(bundle): contract = CallableContract.from_callable(gray_to_color) return CellProfilerFunctionContractExecutor().execute( - contract, contract.resolve_canonical_raw_callable(), bundle, + contract, + contract.resolve_canonical_raw_callable(), + bundle, {"color_scheme": GrayToColorModule.Scheme.STACK, "rescale_intensity": False}, execution_mode=FullStackExecution, plane_projection=RuntimePlaneAxisValueProjection.preserve( @@ -145,11 +209,17 @@ def _execute_bound_stack(bundle): ) -@pytest.mark.parametrize("aliases", [("RawBody", "CappedOutgrowth"), ("IndependentRoleA", "IndependentRoleB")]) +@pytest.mark.parametrize( + "aliases", + [("RawBody", "CappedOutgrowth"), ("IndependentRoleA", "IndependentRoleB")], +) def test_two_processing_roles_share_physical_channel_without_fabrication(aliases): raw = np.arange(20, dtype=np.float32).reshape(4, 5) * 1000 capped = np.clip(raw / 6000, 0, 1) - inputs = tuple(_source_plane(pixels, alias) for pixels, alias in zip((raw, capped), aliases, strict=True)) + inputs = tuple( + _source_plane(pixels, alias) + for pixels, alias in zip((raw, capped), aliases, strict=True) + ) bundle = ImagePayloadBundleContext.from_payloads(inputs).compose() assert bundle.metadata.plane_axis is RuntimePlaneAxis.SOURCE_BINDING result = _execute_bound_stack(bundle) @@ -169,10 +239,17 @@ def test_projected_runtime_plane_needs_a_separate_named_binding_axis(): pixels = np.arange(20, dtype=np.float32).reshape(4, 5) * 1000 scalar = _source_plane(pixels, "FITC") metadata = scalar.metadata - retained = replace(metadata, plane_axis=RuntimePlaneAxis.RUNTIME_SLICE).payload_with(pixels[None], None) - projected = RuntimeSliceProjection.value_for_slice(retained, RuntimePlaneAxisValueProjection.from_selected_plane( - axis=RuntimePlaneAxis.RUNTIME_SLICE, plane_index=0, axis_size=1, - )) + retained = replace( + metadata, plane_axis=RuntimePlaneAxis.RUNTIME_SLICE + ).payload_with(pixels[None], None) + projected = RuntimeSliceProjection.value_for_slice( + retained, + RuntimePlaneAxisValueProjection.from_selected_plane( + axis=RuntimePlaneAxis.RUNTIME_SLICE, + plane_index=0, + axis_size=1, + ), + ) assert projected.metadata.plane_axis is None bundle = ImagePayloadBundleContext.from_payloads((projected,)).compose() assert bundle.metadata.plane_axis is RuntimePlaneAxis.SOURCE_BINDING @@ -184,7 +261,9 @@ def test_projected_runtime_plane_needs_a_separate_named_binding_axis(): def test_stack_color_output_is_not_a_second_scalar_gray_binding(): pixels = np.arange(20, dtype=np.float32).reshape(4, 5) - named = ImagePayloadBundleContext.from_payloads((_source_plane(pixels, "FITC"),)).compose() + named = ImagePayloadBundleContext.from_payloads( + (_source_plane(pixels, "FITC"),) + ).compose() color = _execute_bound_stack(named) # The existing STACK contract emits YXC even for one role. A second grayscale # binding introduces C,Y,X,source-color rather than the required C,Y,X. diff --git a/tests/unit/test_gray_to_color_payload_axis_compile_432.py b/tests/unit/test_gray_to_color_payload_axis_compile_432.py index c4d80351e3..219cb58b00 100644 --- a/tests/unit/test_gray_to_color_payload_axis_compile_432.py +++ b/tests/unit/test_gray_to_color_payload_axis_compile_432.py @@ -7,25 +7,40 @@ import tifffile from openhcs.core.callable_contract import ( - CallableContract, CreatedPayloadAxis, PayloadAxisRequirement, PreservedPayloadAxes, - creates_payload_axis, preserves_payload_axes, + CallableContract, + CreatedPayloadAxis, + PayloadAxisRequirement, + PreservedPayloadAxes, + creates_payload_axis, + preserves_payload_axes, ) from openhcs.core.compiled_step_plan import CompiledStepPlan from openhcs.core.function_patterns import ( - CompiledFunctionGroup, CompiledFunctionPattern, PayloadAxisProof, - InheritedPayloadAxisProof, CreatedPayloadAxisProof, + CompiledFunctionGroup, + CompiledFunctionPattern, + PayloadAxisProof, + InheritedPayloadAxisProof, + CreatedPayloadAxisProof, UnprovedPayloadAxisProof, ) from openhcs.core.pipeline.compiler import PipelineCompiler from openhcs.core.aligned_image_payload import ImagePayloadBundleContext from openhcs.core.image_payload_execution_mode import NaturalExecution from openhcs.core.step_dependencies import StepInputDependency -from openhcs.interop.cellprofiler.runtime.function_contract_execution import CellProfilerFunctionContractExecutor +from openhcs.interop.cellprofiler.runtime.function_contract_execution import ( + CellProfilerFunctionContractExecutor, +) from openhcs.processing.backends.cellprofiler.color import ( - ColorToGrayMode, ImageChannelType, color_to_gray, gray_to_color, + ColorToGrayMode, + ImageChannelType, + color_to_gray, + gray_to_color, split_color_to_gray, ) -from tests.unit.test_gray_to_color_binding_axis_432 import _execute_bound_stack, _source_plane +from tests.unit.test_gray_to_color_binding_axis_432 import ( + _execute_bound_stack, + _source_plane, +) from tests.unit.test_payload_axis_compile_gate import _compiled_pattern, _session from openhcs.core.payload_axes import ColourSampleAxisSpec from openhcs.core.axes import ColourAxis @@ -36,13 +51,18 @@ def _gray_creator_session(tmp_path): tifffile.imwrite(path, np.zeros((4, 5), dtype=np.uint16)) session = _session(tmp_path, (path,)) producer = replace( - session.plans[0], step_name="ComposeScalarRoles", step_scope_id="step-0", + session.plans[0], + step_name="ComposeScalarRoles", + step_scope_id="step-0", compiled_function_pattern=_compiled_pattern(gray_to_color), ) consumer = CompiledStepPlan( - step_index=1, step_name="SplitComposedRoles", axis_id="A01", + step_index=1, + step_name="SplitComposedRoles", + axis_id="A01", main_input_dependency=StepInputDependency.step_output( - source_step_index=0, source_step_scope_id="step-0", + source_step_index=0, + source_step_scope_id="step-0", ), compiled_function_pattern=_compiled_pattern(color_to_gray), ) @@ -59,14 +79,23 @@ def test_declared_gray_creator_proves_color_consumer_before_execution(tmp_path): def test_declared_gray_creator_proves_same_group_color_consumer(tmp_path): session = _gray_creator_session(tmp_path) - group = CompiledFunctionGroup("default", tuple( - invocation - for plan in session.plans.values() - for invocation in plan.compiled_function_pattern.iter_invocations() - )) - session.plans = {0: replace(session.plans[0], compiled_function_pattern=CompiledFunctionPattern( - groups=(group,), is_grouped=False, - ))} + group = CompiledFunctionGroup( + "default", + tuple( + invocation + for plan in session.plans.values() + for invocation in plan.compiled_function_pattern.iter_invocations() + ), + ) + session.plans = { + 0: replace( + session.plans[0], + compiled_function_pattern=CompiledFunctionPattern( + groups=(group,), + is_grouped=False, + ), + ) + } session.context.step_plans = session.plans PipelineCompiler.validate_payload_axis_requirements(session) @@ -76,12 +105,23 @@ def test_preservation_annotation_cannot_create_missing_source_payload_axis(tmp_p producer = session.plans[0] (invocation,) = tuple(producer.compiled_function_pattern.iter_invocations()) # A local counterexample only: do not mutate production callable/registry. - falsely_preserving = replace(invocation, contract=replace(invocation.contract, metadata=replace( - invocation.contract.metadata, payload_axis_transition=PreservedPayloadAxes(), - ))) - session.plans[0] = replace(producer, compiled_function_pattern=CompiledFunctionPattern( - groups=(CompiledFunctionGroup("default", (falsely_preserving,)),), is_grouped=False, - )) + falsely_preserving = replace( + invocation, + contract=replace( + invocation.contract, + metadata=replace( + invocation.contract.metadata, + payload_axis_transition=PreservedPayloadAxes(), + ), + ), + ) + session.plans[0] = replace( + producer, + compiled_function_pattern=CompiledFunctionPattern( + groups=(CompiledFunctionGroup("default", (falsely_preserving,)),), + is_grouped=False, + ), + ) session.context.step_plans = session.plans with pytest.raises(ValueError, match="requires a declared ColourAxis payload axis"): PipelineCompiler.validate_payload_axis_requirements(session) @@ -90,18 +130,26 @@ def test_preservation_annotation_cannot_create_missing_source_payload_axis(tmp_p def _runtime_scalar_role_split(): raw = np.arange(20, dtype=np.float32).reshape(4, 5) * 1000 / 65535 capped = np.clip((raw - 39 / 65535) / (5961 / 65535), 0, 1) - bundle = ImagePayloadBundleContext.from_payloads(tuple( - _source_plane(pixels, name) - for pixels, name in ((raw, "RawBody"), (capped, "CappedOutgrowth")) - )).compose() + bundle = ImagePayloadBundleContext.from_payloads( + tuple( + _source_plane(pixels, name) + for pixels, name in ((raw, "RawBody"), (capped, "CappedOutgrowth")) + ) + ).compose() composite = _execute_bound_stack(bundle) assert composite.data.shape == (4, 5, 2) assert composite.metadata.axis_position(ColourAxis) == -1 contract = CallableContract.from_callable(color_to_gray) result = CellProfilerFunctionContractExecutor().execute( - contract, contract.resolve_canonical_raw_callable(), composite, - {"mode": ColorToGrayMode.SPLIT, "image_type": ImageChannelType.CHANNELS, - "channel_indices": (0, 1), "contributions": (1.0, 1.0)}, + contract, + contract.resolve_canonical_raw_callable(), + composite, + { + "mode": ColorToGrayMode.SPLIT, + "image_type": ImageChannelType.CHANNELS, + "channel_indices": (0, 1), + "contributions": (1.0, 1.0), + }, execution_mode=NaturalExecution, ) return result, (raw, capped) @@ -114,7 +162,9 @@ def test_original_runtime_split_preserves_both_role_values_and_physical_channel( assert output.data.shape == (4, 5) np.testing.assert_array_equal(output.data, expected) assert output.metadata.source_component_metadata["channel"] == "2" - assert set(output.metadata.source_image_paths) == {"/synthetic/A01_s1_w2_z1_t1.tif"} + assert set(output.metadata.source_image_paths) == { + "/synthetic/A01_s1_w2_z1_t1.tif" + } def test_scalar_role_outputs_consume_the_color_payload_axis(): @@ -125,7 +175,9 @@ def test_scalar_role_outputs_consume_the_color_payload_axis(): @pytest.mark.parametrize("same_group", [False, True]) -def test_independent_creator_declaration_needs_no_consumer_edits(tmp_path, monkeypatch, same_group): +def test_independent_creator_declaration_needs_no_consumer_edits( + tmp_path, monkeypatch, same_group +): @creates_payload_axis(ColourAxis) def independent_lane_creator(image): return np.stack((image, image * 2), axis=-1) @@ -146,15 +198,30 @@ def independent_lane_preserver(image): for invocation in _compiled_pattern(callable_).iter_invocations() ) if same_group: - consumer_invocations = tuple(session.plans[1].compiled_function_pattern.iter_invocations()) - session.plans = {0: replace(session.plans[0], compiled_function_pattern=CompiledFunctionPattern( - groups=(CompiledFunctionGroup("default", producer_invocations + consumer_invocations),), - is_grouped=False, - ))} + consumer_invocations = tuple( + session.plans[1].compiled_function_pattern.iter_invocations() + ) + session.plans = { + 0: replace( + session.plans[0], + compiled_function_pattern=CompiledFunctionPattern( + groups=( + CompiledFunctionGroup( + "default", producer_invocations + consumer_invocations + ), + ), + is_grouped=False, + ), + ) + } else: - session.plans[0] = replace(session.plans[0], compiled_function_pattern=CompiledFunctionPattern( - groups=(CompiledFunctionGroup("default", producer_invocations),), is_grouped=False, - )) + session.plans[0] = replace( + session.plans[0], + compiled_function_pattern=CompiledFunctionPattern( + groups=(CompiledFunctionGroup("default", producer_invocations),), + is_grouped=False, + ), + ) session.context.step_plans = session.plans def reject_source_backtracking(*args, **kwargs): @@ -172,9 +239,12 @@ def test_group_proof_stops_at_creator_but_rejects_later_unknown(unknown_after_cr def unknown(image): return image - functions = (gray_to_color, unknown) if unknown_after_creator else (unknown, gray_to_color) + functions = ( + (gray_to_color, unknown) if unknown_after_creator else (unknown, gray_to_color) + ) invocations = tuple( - invocation for callable_ in functions + invocation + for callable_ in functions for invocation in _compiled_pattern(callable_).iter_invocations() ) proof = CompiledFunctionGroup("default", invocations).payload_axis_proof( @@ -183,7 +253,9 @@ def unknown(image): failures = [] assert proof.validate_obligation( failure_message=lambda invocation: invocation.contract.function_name, - source_validation=lambda: pytest.fail("Creation/rejection must stop source validation"), + source_validation=lambda: pytest.fail( + "Creation/rejection must stop source validation" + ), failures=failures, ) assert failures == (["unknown"] if unknown_after_creator else []) @@ -199,7 +271,9 @@ def validate_source(): complete = InheritedPayloadAxisProof().validate_obligation( source_validation=validate_source, - failure_message=lambda invocation: pytest.fail("Inherited proof has no failed invocation"), + failure_message=lambda invocation: pytest.fail( + "Inherited proof has no failed invocation" + ), failures=failures, ) assert complete is source_complete @@ -208,14 +282,25 @@ def validate_source(): @pytest.mark.parametrize("capability_first", [False, True]) -@pytest.mark.parametrize("proof_type,args,expected_visits,expected_failures", [ - (InheritedPayloadAxisProof, (), ["before", "source", "after"], []), - (CreatedPayloadAxisProof, (), ["before", "after"], []), - (UnprovedPayloadAxisProof, - tuple(_compiled_pattern(color_to_gray).iter_invocations()), ["before", "after"], ["rejected color_to_gray"]), -]) +@pytest.mark.parametrize( + "proof_type,args,expected_visits,expected_failures", + [ + (InheritedPayloadAxisProof, (), ["before", "source", "after"], []), + (CreatedPayloadAxisProof, (), ["before", "after"], []), + ( + UnprovedPayloadAxisProof, + tuple(_compiled_pattern(color_to_gray).iter_invocations()), + ["before", "after"], + ["rejected color_to_gray"], + ), + ], +) def test_independent_same_node_validation_capability_composes_both_mro_orders( - capability_first, proof_type, args, expected_visits, expected_failures, + capability_first, + proof_type, + args, + expected_visits, + expected_failures, ): visits, failures = [], [] @@ -228,7 +313,11 @@ def validate_obligation(self, **kwargs): # Same nominal owner in both bases puts the independent capability before # the shared operation in C3, regardless of its position beside the leaf. - bases = (ValidationVisitCapability, proof_type) if capability_first else (proof_type, ValidationVisitCapability) + bases = ( + (ValidationVisitCapability, proof_type) + if capability_first + else (proof_type, ValidationVisitCapability) + ) declared_proof = type("IndependentlyObservedProof", bases, {})(*args) def validate_source(): @@ -237,7 +326,8 @@ def validate_source(): assert declared_proof.validate_obligation( source_validation=validate_source, - failure_message=lambda invocation: "rejected " + invocation.contract.function_name, + failure_message=lambda invocation: "rejected " + + invocation.contract.function_name, failures=failures, ) assert visits == expected_visits @@ -258,18 +348,25 @@ def reject_source(): assert failures == ["exact source missing its declared payload axis"] -def test_color_to_gray_does_not_preserve_consumed_payload_axis_for_later_consumer(tmp_path): +def test_color_to_gray_does_not_preserve_consumed_payload_axis_for_later_consumer( + tmp_path, +): session = _gray_creator_session(tmp_path) consumer = session.plans[1] session.plans[2] = replace( - consumer, step_index=2, step_name="RejectSecondScalarSplit", + consumer, + step_index=2, + step_name="RejectSecondScalarSplit", main_input_dependency=StepInputDependency.step_output( - source_step_index=1, source_step_scope_id="step-1", + source_step_index=1, + source_step_scope_id="step-1", ), ) session.plans[1] = replace(consumer, step_scope_id="step-1") session.context.step_plans = session.plans - with pytest.raises(ValueError, match="not preserved by producer step 1 callable 'color_to_gray'"): + with pytest.raises( + ValueError, match="not preserved by producer step 1 callable 'color_to_gray'" + ): PipelineCompiler.validate_payload_axis_requirements(session) @@ -283,9 +380,15 @@ def test_scalar_projection_modes_keep_pixels_mask_and_original_source(image_type expected = split_color_to_gray(source, image_type, (0, 1)) contract = CallableContract.from_callable(color_to_gray) result = CellProfilerFunctionContractExecutor().execute( - contract, contract.resolve_canonical_raw_callable(), source, - {"mode": ColorToGrayMode.SPLIT, "image_type": image_type, - "channel_indices": (0, 1), "contributions": (1.0, 1.0)}, + contract, + contract.resolve_canonical_raw_callable(), + source, + { + "mode": ColorToGrayMode.SPLIT, + "image_type": image_type, + "channel_indices": (0, 1), + "contributions": (1.0, 1.0), + }, execution_mode=NaturalExecution, ) for output, expected_pixels in zip(result.slices, expected, strict=True): @@ -294,7 +397,10 @@ def test_scalar_projection_modes_keep_pixels_mask_and_original_source(image_type output_metadata = output.metadata assert output_metadata.axis_position(ColourAxis) is None assert output_metadata.source_path == metadata.source_path - assert output_metadata.source_component_metadata == metadata.source_component_metadata + assert ( + output_metadata.source_component_metadata + == metadata.source_component_metadata + ) assert output_metadata.source_image_names == metadata.source_image_names assert output_metadata.source_voxel_spacing == metadata.source_voxel_spacing assert output_metadata.source_spatial_domain == metadata.source_spatial_domain diff --git a/tests/unit/test_grid_addressed_axis.py b/tests/unit/test_grid_addressed_axis.py index 678d20f49b..8db3b47487 100644 --- a/tests/unit/test_grid_addressed_axis.py +++ b/tests/unit/test_grid_addressed_axis.py @@ -67,7 +67,10 @@ def test_component_filters_validate_against_the_family_and_match_generically() - assert filters.matches({"well": "B02", "channel": 1, "site": 3}) assert not filters.matches({"well": "C03", "channel": 1}) assert not filters.matches({"channel": 1}) - assert ComponentFilters.from_assignments(["well=A01", "well=B02", "channel=1"]) == filters + assert ( + ComponentFilters.from_assignments(["well=A01", "well=B02", "channel=1"]) + == filters + ) assert ComponentFilters().matches({}) with pytest.raises(ValueError, match="unknown axes"): ComponentFilters.from_mapping({"plot": "1"}) diff --git a/tests/unit/test_image_plane_contracts.py b/tests/unit/test_image_plane_contracts.py index 033419b84f..ce7e5f4511 100644 --- a/tests/unit/test_image_plane_contracts.py +++ b/tests/unit/test_image_plane_contracts.py @@ -1,7 +1,14 @@ import numpy as np import pytest -from openhcs.core.aligned_image_payload import (AlignedImageStack, AlignedImageSliceContext, ImageOutputBundle, compose_aligned_image_payload, pack_aligned_image_outputs, stack_image_payloads) +from openhcs.core.aligned_image_payload import ( + AlignedImageStack, + AlignedImageSliceContext, + ImageOutputBundle, + compose_aligned_image_payload, + pack_aligned_image_outputs, + stack_image_payloads, +) from openhcs.core.memory import ( MEMORY_TYPE_NUMPY, stack_runtime_slices, @@ -76,9 +83,12 @@ def test_aligned_image_stack_nominally_declares_runtime_alignment() -> None: ) payload = AlignedImageStack(slices) - assert tuple( - member.data for member in RuntimeSliceProjection.alignment_slices(payload) - ) == slices + assert ( + tuple( + member.data for member in RuntimeSliceProjection.alignment_slices(payload) + ) + == slices + ) def test_runtime_plane_projection_requires_matching_payload_axis() -> None: @@ -182,9 +192,10 @@ def test_scalar_mask_projection_requires_metadata_owner_to_declare_plane_axis( else: metadata.payload_with(data, mask).slice_payload(data, 0) - assert ImagePayloadSliceProjector(mask=None, metadata=metadata).mask_for_slice( - data, 0 - ) is None + assert ( + ImagePayloadSliceProjector(mask=None, metadata=metadata).mask_for_slice(data, 0) + is None + ) @pytest.mark.parametrize("batch", (False, True)) @@ -201,7 +212,9 @@ def test_slice_projection_keeps_fresh_source_metadata_across_calls(batch: bool) def project(): if batch: return projector.payloads_for_slices(tuple(data))[1] - return payload.slice_payload(data[1], 1, plane_axis=RuntimePlaneAxis.RUNTIME_SLICE) + return payload.slice_payload( + data[1], 1, plane_axis=RuntimePlaneAxis.RUNTIME_SLICE + ) first = project() metadata.source_component_metadata = {"well": "B02"} @@ -275,9 +288,7 @@ def test_image_bundle_composition_declares_source_binding_axis() -> None: composition = compose_aligned_image_payload("ImageMath", (first, second)) assert composition.execution_mode is FullStackExecution - assert composition.payload.metadata.plane_axis is ( - RuntimePlaneAxis.SOURCE_BINDING - ) + assert composition.payload.metadata.plane_axis is (RuntimePlaneAxis.SOURCE_BINDING) assert composition.payload.data.shape == (2, 4, 5) @@ -290,9 +301,7 @@ def test_single_runtime_slice_payload_is_not_repacked_as_source_binding() -> Non assert composition.execution_mode is NaturalExecution assert composition.payload is payload - assert composition.payload.metadata.plane_axis is ( - RuntimePlaneAxis.RUNTIME_SLICE - ) + assert composition.payload.metadata.plane_axis is (RuntimePlaneAxis.RUNTIME_SLICE) def test_composed_runtime_slice_projection_ignores_outer_file_cardinality() -> None: @@ -364,9 +373,7 @@ def test_single_aligned_payload_is_preserved_without_rebundling() -> None: composition = compose_aligned_image_payload("MeasureImage", (payload,)) - assert composition.execution_mode is ( - AlignedStackExecution - ) + assert composition.execution_mode is (AlignedStackExecution) assert composition.payload is payload @@ -386,9 +393,7 @@ def test_declared_output_contexts_preserve_aligned_multi_image_payloads() -> Non slice_contexts=contexts, ) - assert composition.execution_mode is ( - AlignedStackExecution - ) + assert composition.execution_mode is (AlignedStackExecution) assert isinstance(composition.payload, ImageOutputBundle) assert tuple(member.data for member in composition.payload.slices) == payloads assert composition.payload.slice_contexts == contexts diff --git a/tests/unit/test_implicit_main_flow_compilation.py b/tests/unit/test_implicit_main_flow_compilation.py index 91fb138d72..c35faa2972 100644 --- a/tests/unit/test_implicit_main_flow_compilation.py +++ b/tests/unit/test_implicit_main_flow_compilation.py @@ -95,6 +95,6 @@ def test_native_grouped_main_flow_compiles_into_cellprofiler_consumer(tmp_path): assert threshold_plan.compiled_function_pattern is not None invocation = next(threshold_plan.compiled_function_pattern.iter_invocations()) (edge,) = invocation.artifact_input_edges - assert (edge.main_flow_projection is not None) + assert edge.main_flow_projection is not None assert edge.storage_plan is None assert invocation.contract.execution_scope is FunctionStepExecutionScope.AXIS diff --git a/tests/unit/test_installed_mcp_demo.py b/tests/unit/test_installed_mcp_demo.py index 4625f61f71..4ba2c1803a 100644 --- a/tests/unit/test_installed_mcp_demo.py +++ b/tests/unit/test_installed_mcp_demo.py @@ -74,7 +74,9 @@ def _execution(capability, payload, *, returncode: int = 0) -> McpDevCommandExec response = McpDevToolBatchResponse( server=McpDevServerIdentity(command="python", module="openhcs.mcp"), results=( - McpDevToolResult(tool=capability.name, mcp_error=False, payloads=(payload,)), + McpDevToolResult( + tool=capability.name, mcp_error=False, payloads=(payload,) + ), ), ) return McpDevCommandExecution( @@ -335,11 +337,23 @@ def test_command_payload_rejects_a_missing_declared_result() -> None: def _dataset_row(root: Path, terminal_status: str | None) -> DatasetRowState: return DatasetRowState( - scope_id=str(root), name=root.name, root=str(root), pipeline_path=None, - selected=True, initialized=True, compiled=True, init_pending=False, - compile_pending=False, execution_active=False, status_prefix="", - orchestrator_state=None, execution_id=None, terminal_status=terminal_status, - runtime_state=None, runtime_percent=None, queue_position=None, + scope_id=str(root), + name=root.name, + root=str(root), + pipeline_path=None, + selected=True, + initialized=True, + compiled=True, + init_pending=False, + compile_pending=False, + execution_active=False, + status_prefix="", + orchestrator_state=None, + execution_id=None, + terminal_status=terminal_status, + runtime_state=None, + runtime_percent=None, + queue_position=None, ) @@ -359,7 +373,9 @@ def test_execute_pipeline_runs_the_session_journey_and_reads_its_row( def fake_run_mcp(client, argv, *, capability, payload_type, timeout_seconds): calls.append((client, tuple(argv), capability, payload_type, timeout_seconds)) return DatasetListState( - rows=rows, selected_scope_ids=(str(plate),), execution_state="idle", + rows=rows, + selected_scope_ids=(str(plate),), + execution_state="idle", ) monkeypatch.setattr(installed_demo, "_run_mcp", fake_run_mcp) diff --git a/tests/unit/test_invocation_contract_provider.py b/tests/unit/test_invocation_contract_provider.py index 879d721a1b..98be7c4e0c 100644 --- a/tests/unit/test_invocation_contract_provider.py +++ b/tests/unit/test_invocation_contract_provider.py @@ -1164,7 +1164,9 @@ def test_cellprofiler_provider_rejects_under_specified_one_image_align() -> None ValueError, match=r"Align.*cannot reconstruct an exact module block", ): - CellProfilerInvocationContractProviderFactory.provider_for_pipeline(session.pipeline) + CellProfilerInvocationContractProviderFactory.provider_for_pipeline( + session.pipeline + ) def test_cellprofiler_invocation_contract_allows_source_binding_supersets() -> None: diff --git a/tests/unit/test_invocation_input_source_context_identity.py b/tests/unit/test_invocation_input_source_context_identity.py index afee352f86..fe95048984 100644 --- a/tests/unit/test_invocation_input_source_context_identity.py +++ b/tests/unit/test_invocation_input_source_context_identity.py @@ -94,7 +94,10 @@ def test_compiled_input_edges_own_exact_main_flow_membership() -> None: assert tuple(edge.spec for edge in edges) == (plate_template, combined_image) assert tuple(edge.storage_plan for edge in edges) == (None, None) - assert tuple((edge.main_flow_projection is not None) for edge in edges) == (False, True) + assert tuple((edge.main_flow_projection is not None) for edge in edges) == ( + False, + True, + ) assert tuple(edge.main_flow_projection for edge in edges) == ( None, MainFlowInputProjection.COMPLETE_PAYLOAD, @@ -240,12 +243,8 @@ def source_artifact_payload(_request, ref): np.testing.assert_array_equal(source_request.data, 7.0) np.testing.assert_array_equal(main_flow_request.data, 11.0) - assert source_request.metadata.source_image_names == ( - plate_template.name, - ) - assert main_flow_request.metadata.source_image_names == ( - combined_image.name, - ) + assert source_request.metadata.source_image_names == (plate_template.name,) + assert main_flow_request.metadata.source_image_names == (combined_image.name,) def test_storage_backed_input_keeps_exact_runtime_authority() -> None: @@ -294,9 +293,7 @@ def test_storage_backed_input_keeps_exact_runtime_authority() -> None: runtime_request = request.artifact_value(edge) np.testing.assert_array_equal(runtime_request.data, 13.0) - assert runtime_request.metadata.source_image_names == ( - combined_image.name, - ) + assert runtime_request.metadata.source_image_names == (combined_image.name,) def test_compiled_producer_edge_overrides_matching_source_binding() -> None: @@ -348,6 +345,4 @@ def test_compiled_producer_edge_overrides_matching_source_binding() -> None: runtime_request = request.artifact_value(edge) np.testing.assert_array_equal(runtime_request.data, 19.0) - assert runtime_request.metadata.source_image_names == ( - combined_image.name, - ) + assert runtime_request.metadata.source_image_names == (combined_image.name,) diff --git a/tests/unit/test_l3_zmqruntime_vocabulary_guards.py b/tests/unit/test_l3_zmqruntime_vocabulary_guards.py index cdec3ed7c1..4d4dbe5914 100644 --- a/tests/unit/test_l3_zmqruntime_vocabulary_guards.py +++ b/tests/unit/test_l3_zmqruntime_vocabulary_guards.py @@ -35,14 +35,20 @@ def test_messages_inherit_their_codec_from_the_declared_bases() -> None: for node in tree.body if isinstance(node, ast.ClassDef) and node.name not in bases for member in node.body - if isinstance(member, ast.FunctionDef) and member.name in {"to_dict", "from_dict"} + if isinstance(member, ast.FunctionDef) + and member.name in {"to_dict", "from_dict"} ] assert offenders == [] def test_launch_policy_lives_in_zmqruntime() -> None: assert not ( - REPO / "external" / "pyqt-reactive" / "src" / "pyqt_reactive" / "process_launch.py" + REPO + / "external" + / "pyqt-reactive" + / "src" + / "pyqt_reactive" + / "process_launch.py" ).exists() importers = [ str(path.relative_to(REPO)) diff --git a/tests/unit/test_library_registry_discovery.py b/tests/unit/test_library_registry_discovery.py index a0f3403be1..73fc7c4082 100644 --- a/tests/unit/test_library_registry_discovery.py +++ b/tests/unit/test_library_registry_discovery.py @@ -109,7 +109,11 @@ def test_native_catalog_excludes_modules_whose_distribution_markers_exclude_here @pytest.mark.parametrize( "platform_system, platform_machine, excluded", - [("Darwin", "x86_64", True), ("Darwin", "arm64", False), ("Linux", "x86_64", False)], + [ + ("Darwin", "x86_64", True), + ("Darwin", "arm64", False), + ("Linux", "x86_64", False), + ], ) def test_basicpy_catalog_module_follows_the_package_basicpy_marker( monkeypatch, platform_system, platform_machine, excluded diff --git a/tests/unit/test_logical_image_output_snapshot.py b/tests/unit/test_logical_image_output_snapshot.py index 27c12c578b..1b335b2b62 100644 --- a/tests/unit/test_logical_image_output_snapshot.py +++ b/tests/unit/test_logical_image_output_snapshot.py @@ -1,5 +1,5 @@ - from tests.unit.saved_output_dialect import SAVED_OUTPUT_DIALECT + """Typed image sets retain physical coverage across volume/plane exports.""" from dataclasses import replace @@ -38,7 +38,11 @@ from openhcs.core.payload_axes import PayloadAxes Z_STACK = SourceImageSetIdentityPolicy(frozenset({Microscopy.ZIndex})) -EXACT = RuntimeEquivalencePolicy(image_abs_tolerance=0, image_rel_tolerance=0, measurement_dialect=SAVED_OUTPUT_DIALECT) +EXACT = RuntimeEquivalencePolicy( + image_abs_tolerance=0, + image_rel_tolerance=0, + measurement_dialect=SAVED_OUTPUT_DIALECT, +) def _write_metadata(root, projections): @@ -73,7 +77,15 @@ def exported_volume(tmp_path): for plane_index, plane in enumerate(volume): path = candidate / f"opaque-{plane_index}.tiff" imageio.imwrite(path, plane) - address = OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, "1"), (Microscopy.Channel, "2"), (Microscopy.ZIndex, plane_index + 1), (Microscopy.Timepoint, "1"))) + address = OpenHCSPlaneAddress( + ( + (Microscopy.Well, "A01"), + (Microscopy.Site, "1"), + (Microscopy.Channel, "2"), + (Microscopy.ZIndex, plane_index + 1), + (Microscopy.Timepoint, "1"), + ) + ) projections.append( SourceArtifactProjection( address=address, diff --git a/tests/unit/test_matched_cellprofiler_batch.py b/tests/unit/test_matched_cellprofiler_batch.py index c9c7a8bef2..647eb158d5 100644 --- a/tests/unit/test_matched_cellprofiler_batch.py +++ b/tests/unit/test_matched_cellprofiler_batch.py @@ -551,9 +551,12 @@ def reject_read_bytes(_path: Path) -> bytes: monkeypatch.setattr(Path, "read_bytes", reject_read_bytes) expected = hashlib.sha256(b"microscopy pixels").hexdigest() - assert _source_input_inventory( - NativeCellProfilerSelectedSourceUniverse((staged / "image.tif",)) - )[0]["sha256"] == expected + assert ( + _source_input_inventory( + NativeCellProfilerSelectedSourceUniverse((staged / "image.tif",)) + )[0]["sha256"] + == expected + ) assert _output_inventory(tmp_path, frozenset({source}))[0]["sha256"] == expected diff --git a/tests/unit/test_materialization_batch_outcomes.py b/tests/unit/test_materialization_batch_outcomes.py index 16961b7e02..1c4b1bee50 100644 --- a/tests/unit/test_materialization_batch_outcomes.py +++ b/tests/unit/test_materialization_batch_outcomes.py @@ -174,7 +174,9 @@ def test_worker_exports_actual_step_outcomes_after_resource_release( worker_slot="worker-0", worker_assignments={"worker-0": ("A01",)}, ) - monkeypatch.setattr(progress_types, "_progress_queue", SimpleNamespace(put=lambda _event: None)) + monkeypatch.setattr( + progress_types, "_progress_queue", SimpleNamespace(put=lambda _event: None) + ) def forbidden_reconstruction(*_args, **_kwargs): pytest.fail("Ordinary execution must consume actual step outcomes") @@ -195,7 +197,9 @@ def forbidden_reconstruction(*_args, **_kwargs): assert context.released assert transported.is_success() - assert transported.runtime_observation.contexts[0].outputs.runtime_export_paths == (path,) + assert transported.runtime_observation.contexts[0].outputs.runtime_export_paths == ( + path, + ) assert transported.runtime_observation.contexts[0].records == () assert RuntimeExportObservation.from_runtime_observations( (transported.runtime_observation,) @@ -209,10 +213,18 @@ def test_parent_and_worker_exports_share_existing_observation_authority(tmp_path for path in (worker_path, plate_path, unrelated): path.write_text("Area\n42\n") worker = RuntimeExecutionObservation( - contexts=(RuntimeContextObservation("A01", (), StepExecutionObservation({}, (worker_path,))),) + contexts=( + RuntimeContextObservation( + "A01", (), StepExecutionObservation({}, (worker_path,)) + ), + ) ) parent = RuntimeExecutionObservation( - contexts=(RuntimeContextObservation("A01", (), StepExecutionObservation({}, (plate_path,))),) + contexts=( + RuntimeContextObservation( + "A01", (), StepExecutionObservation({}, (plate_path,)) + ), + ) ) results = BundleExecutionResults( {"A01": ExecutionResult.success("A01", runtime_observation=worker)}, diff --git a/tests/unit/test_materialization_flag_planner.py b/tests/unit/test_materialization_flag_planner.py index e2302a2409..297c65ed8d 100644 --- a/tests/unit/test_materialization_flag_planner.py +++ b/tests/unit/test_materialization_flag_planner.py @@ -249,7 +249,9 @@ def new_submission(): def test_unused_materialization_auto_does_not_query_source(): - config = _pipeline_config(path_planning_config=PathPlanningConfig(partition_filter=0)) + config = _pipeline_config( + path_planning_config=PathPlanningConfig(partition_filter=0) + ) config.vfs_config = VFSConfig( read_backend=Backend.DISK, materialization_backend=MaterializationBackend.AUTO, diff --git a/tests/unit/test_mcp_release_workflow.py b/tests/unit/test_mcp_release_workflow.py index 8fb58f3cae..cc1241d6a1 100644 --- a/tests/unit/test_mcp_release_workflow.py +++ b/tests/unit/test_mcp_release_workflow.py @@ -385,9 +385,7 @@ def test_pypi_wheel_smoke_uses_the_canonical_pipeline_document_boundary(): smoke = headless_step["run"] assert "from openhcs.core.config import PipelineConfig" in smoke - assert ( - "from openhcs.core.pipeline_document import PipelineDocumentCodec" in smoke - ) + assert "from openhcs.core.pipeline_document import PipelineDocumentCodec" in smoke assert "from openhcs.core.steps.function_step import FunctionStep" in smoke assert "PipelineDocumentCodec.from_values(" in smoke assert "pipeline_config=PipelineConfig()" in smoke diff --git a/tests/unit/test_measurement_dialect.py b/tests/unit/test_measurement_dialect.py index b9f57eb61a..4a7c5fa70f 100644 --- a/tests/unit/test_measurement_dialect.py +++ b/tests/unit/test_measurement_dialect.py @@ -64,26 +64,26 @@ def survey() -> Iterator[type[Survey]]: def test_a_family_without_its_own_dialect_writes_kernel_names() -> None: assert type(MeasurementDialect.for_family(Microscopy)) is PlainMeasurementDialect - assert _render_csv_rows(({"slice_index": 0, "count": 3},), ("slice_index", "count")) == ( - "slice_index,count\r\n0,3\r\n" - ) + assert _render_csv_rows( + ({"slice_index": 0, "count": 3},), ("slice_index", "count") + ) == ("slice_index,count\r\n0,3\r\n") def test_the_active_family_names_written_rows_through_its_dialect(survey) -> None: assert type(MeasurementDialect.for_active_family()) is SurveyDialect - assert _render_csv_rows(({"slice_index": 0, "count": 3},), ("slice_index", "count")) == ( - "scan_index,count\r\n0,3\r\n" - ) + assert _render_csv_rows( + ({"slice_index": 0, "count": 3},), ("slice_index", "count") + ) == ("scan_index,count\r\n0,3\r\n") def test_each_dialect_spells_the_unqualified_sample() -> None: names = MeasurementDialect.unqualified_sample_names() assert {"", "sample", "image", "scan"} <= names for name in ("sample", "Image", "scan"): - assert MeasurementSubject(MeasurementScope.SAMPLE, name).source_image_name is None - assert ( - MeasurementSubject(MeasurementScope.SAMPLE, "DNA").source_image_name == "DNA" - ) + assert ( + MeasurementSubject(MeasurementScope.SAMPLE, name).source_image_name is None + ) + assert MeasurementSubject(MeasurementScope.SAMPLE, "DNA").source_image_name == "DNA" def test_cellprofiler_spells_scopes_and_sample_numbers_in_its_own_dialect() -> None: diff --git a/tests/unit/test_measureobjectintensitydistribution.py b/tests/unit/test_measureobjectintensitydistribution.py index ee78d01a33..d40c3fa025 100644 --- a/tests/unit/test_measureobjectintensitydistribution.py +++ b/tests/unit/test_measureobjectintensitydistribution.py @@ -9,7 +9,9 @@ MeasurementFeatureQuery, MeasurementFeatureValueIndex, ) -from openhcs.core.pipeline.function_contracts import object_label_input_execution_mode_from_callable +from openhcs.core.pipeline.function_contracts import ( + object_label_input_execution_mode_from_callable, +) from openhcs.core.measurement_row_materialization import columnar_row_values from openhcs.core.runtime_image_values import ImagePayloadMetadata from openhcs.core.runtime_measurements import ( diff --git a/tests/unit/test_named_checkpoint_publication.py b/tests/unit/test_named_checkpoint_publication.py index 5561e66be1..fe720be850 100644 --- a/tests/unit/test_named_checkpoint_publication.py +++ b/tests/unit/test_named_checkpoint_publication.py @@ -13,18 +13,26 @@ from tests.unit.test_artifact_publication_journey import _plate, progress_events from openhcs.core.artifacts import ( - QaCheckpoint, ImageArtifactType, MainFlowPlaneProjectionOutputSpec, + QaCheckpoint, + ImageArtifactType, + MainFlowPlaneProjectionOutputSpec, MainFlowStackOutputSpec, ) from openhcs.core.config import ( - LazyPathPlanningConfig, LazyProcessingConfig, LazyStepMaterializationConfig, + LazyPathPlanningConfig, + LazyProcessingConfig, + LazyStepMaterializationConfig, PipelineConfig, ) from openhcs.core.memory import numpy as numpy_decorator from openhcs.core.orchestrator.orchestrator import PipelineOrchestrator -from openhcs.core.orchestrator.execution_result import RuntimeContextObservation, RuntimeExecutionObservation +from openhcs.core.orchestrator.execution_result import ( + RuntimeContextObservation, + RuntimeExecutionObservation, +) from openhcs.core.pipeline.function_contracts import ( - artifact_outputs, required_axis_roles, + artifact_outputs, + required_axis_roles, ) from openhcs.core.projected_image_output import SourceProjectedImageOutput from openhcs.core.runtime_image_values import ImagePayloadMetadata @@ -38,13 +46,14 @@ Pure3DContract, ) from openhcs.processing.materialization import ( - ImageFileOptions, MaterializationSpec, MaterializedFilenameIdentity, + ImageFileOptions, + MaterializationSpec, + MaterializedFilenameIdentity, ) from openhcs.core.axes import TileAxis from openhcs.domains.microscopy.axes import Microscopy - @dataclass(frozen=True) class SyntheticAggregateField(SourceProjectedImageOutput): data: np.ndarray @@ -55,15 +64,22 @@ def with_data(self, data): def resolve_source_context(self, source, projection): assert projection is not None and projection.plane_index is None assert projection.axis_size == 24 - return source.metadata.collapse_leading_plane_axis().payload_with(self.data, None) + return source.metadata.collapse_leading_plane_axis().payload_with( + self.data, None + ) def _field(name): return MainFlowPlaneProjectionOutputSpec.output( - name, ImageArtifactType, sidecar_role=QaCheckpoint, - materialization=MaterializationSpec(ImageFileOptions( - filename_suffix=".tif", filename_identity=MaterializedFilenameIdentity.ARTIFACT_NAME, - )), + name, + ImageArtifactType, + sidecar_role=QaCheckpoint, + materialization=MaterializationSpec( + ImageFileOptions( + filename_suffix=".tif", + filename_identity=MaterializedFilenameIdentity.ARTIFACT_NAME, + ) + ), ) @@ -71,7 +87,8 @@ def _field(name): @required_axis_roles(TileAxis) @artifact_outputs( MainFlowStackOutputSpec.output("corrected", ImageArtifactType), - _field("flat"), _field("dark"), + _field("flat"), + _field("dark"), ) def synthetic_three_outputs(image): assert image.shape == (24, 8, 8) @@ -83,25 +100,35 @@ def synthetic_three_outputs(image): @pytest.mark.parametrize("main_filter", (0, 1)) -def test_named_and_ordinary_checkpoint_inventory_owns_every_address(tmp_path, main_filter): +def test_named_and_ordinary_checkpoint_inventory_owns_every_address( + tmp_path, main_filter +): plate = _plate(tmp_path / "source", site_count=24) config = PipelineConfig( - path_planning_config=LazyPathPlanningConfig(output_dir_suffix="_out", partition_filter=main_filter), - num_workers=1, use_threading=True, + path_planning_config=LazyPathPlanningConfig( + output_dir_suffix="_out", partition_filter=main_filter + ), + num_workers=1, + use_threading=True, ) orchestrator = PipelineOrchestrator(plate, pipeline_config=config).initialize() step = FunctionStep( func=synthetic_three_outputs, processing_config=LazyProcessingConfig( - group_by=Microscopy.Channel, variable_components=[Microscopy.Site], + group_by=Microscopy.Channel, + variable_components=[Microscopy.Site], ), step_materialization_config=LazyStepMaterializationConfig(enabled=True), ) bundle = orchestrator.compile_pipelines([step]) - observations = RuntimeExecutionObservation(contexts=tuple( - RuntimeContextObservation(context_key=key, records=(), outputs=step.process(context, 0)) - for key, context in bundle.runtime_contexts.items() - )) + observations = RuntimeExecutionObservation( + contexts=tuple( + RuntimeContextObservation( + context_key=key, records=(), outputs=step.process(context, 0) + ) + for key, context in bundle.runtime_contexts.items() + ) + ) OpenHCSMetadataTarget.finalize_completed_dataset( bundle.runtime_contexts, runtime_observations=(observations,) ) @@ -110,10 +137,14 @@ def test_named_and_ordinary_checkpoint_inventory_owns_every_address(tmp_path, ma reopened = VirtualWorkspaceSourceProjection.from_openhcs_metadata(root, metadata) for target, count in (("checkpoints", 48), ("checkpoints_results", 2)): entry = metadata["subdirectories"][target] - inventory = {str(path.relative_to(root)) for path in (root / target).glob("*.tif")} + inventory = { + str(path.relative_to(root)) for path in (root / target).glob("*.tif") + } assert len(inventory) == count assert set(entry["image_files"]) == inventory - assert {item["virtual_path"] for item in entry["source_projection"]} == inventory + assert { + item["virtual_path"] for item in entry["source_projection"] + } == inventory for item in entry["source_projection"]: retained = ImagePayloadMetadata.from_mapping(item["image_metadata"]) assert retained.source_voxel_spacing == SourceVoxelSpacing((0.65, 0.65)) @@ -123,13 +154,23 @@ def test_named_and_ordinary_checkpoint_inventory_owns_every_address(tmp_path, ma projection for item in checkpoints if isinstance( - projection := reopened.source_projections_by_virtual_path[item["virtual_path"]], + projection := reopened.source_projections_by_virtual_path[ + item["virtual_path"] + ], SourceArtifactProjection, ) ] assert len(ordinary) == len(checkpoints) - len(ordinary) == 24 assert {projection.address for projection in ordinary} == { - OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, site), (Microscopy.Channel, 1), (Microscopy.ZIndex, 1), (Microscopy.Timepoint, 1))) + OpenHCSPlaneAddress( + ( + (Microscopy.Well, "A01"), + (Microscopy.Site, site), + (Microscopy.Channel, 1), + (Microscopy.ZIndex, 1), + (Microscopy.Timepoint, 1), + ) + ) for site in range(1, 25) } for item in checkpoints: @@ -151,9 +192,19 @@ def test_named_and_ordinary_checkpoint_inventory_owns_every_address(tmp_path, ma assert all_inventory.issubset(reopened.source_projections_by_virtual_path) if main_filter: assert len(metadata["subdirectories"]["images"]["image_files"]) == 24 - assert PipelineOrchestrator(root).initialize().microscope_handler.get_pixel_size(root) == 0.65 + assert ( + PipelineOrchestrator(root) + .initialize() + .microscope_handler.get_pixel_size(root) + == 0.65 + ) # Keep fail-closed reconciliation: an actually unowned image is still an # error. This is not a missing-address permissive fallback. - tifffile.imwrite(root / "checkpoints" / "unaddressed.tif", np.zeros((8, 8), dtype=np.uint16)) - with pytest.raises(MetadataWriteError, match="Saved images lack typed produced addresses.*unaddressed"): + tifffile.imwrite( + root / "checkpoints" / "unaddressed.tif", np.zeros((8, 8), dtype=np.uint16) + ) + with pytest.raises( + MetadataWriteError, + match="Saved images lack typed produced addresses.*unaddressed", + ): OpenHCSMetadataTarget.finalize_completed_dataset(bundle.runtime_contexts) diff --git a/tests/unit/test_napari_accepted_work_settlement.py b/tests/unit/test_napari_accepted_work_settlement.py index 958d9b25b5..53d5fd72a8 100644 --- a/tests/unit/test_napari_accepted_work_settlement.py +++ b/tests/unit/test_napari_accepted_work_settlement.py @@ -105,7 +105,9 @@ def wire_batch(item): return json.dumps( { "type": "batch", - "images": [{**row, "producer_identity": producer.to_payload()} for row in items], + "images": [ + {**row, "producer_identity": producer.to_payload()} for row in items + ], "display_config": ViewerBatchDisplayPayload( component_modes=config.component_modes(), component_order=config.COMPONENT_ORDER, @@ -182,9 +184,13 @@ def test_saved_roi_reopen_reports_pre_route_failure_and_recovers( # A new accepted batch starts the next existing settlement cycle. The # previous failed cycle must not become a permanent unrelated failure. assert wire_receiver(image_item())["status"] == "success" - admitted = ViewerSettleProgress.from_response(ViewerControlResponse( - NapariControlAction.for_message_type("settle").transport_thread_response(receiver, {}) - )) + admitted = ViewerSettleProgress.from_response( + ViewerControlResponse( + NapariControlAction.for_message_type("settle").transport_thread_response( + receiver, {} + ) + ) + ) assert admitted.phase is ViewerSettlePhase.RUNNING assert admitted.total_update_count == 0 receiver.process_accepted_stream_messages() @@ -205,15 +211,23 @@ def test_previous_complete_cannot_settle_new_accepted_work(receiver): == "success" ) assert not receiver.accepted_stream_batches.empty() - admitted = ViewerSettleProgress.from_response(ViewerControlResponse( - NapariControlAction.for_message_type("settle").transport_thread_response(receiver, {}) - )) + admitted = ViewerSettleProgress.from_response( + ViewerControlResponse( + NapariControlAction.for_message_type("settle").transport_thread_response( + receiver, {} + ) + ) + ) assert admitted.phase is ViewerSettlePhase.RUNNING assert admitted.total_update_count == 0 def test_initial_settlement_is_observable_before_qt_intake_and_completes( - native_opengl, receiver, wire_receiver, qtbot, tmp_path, + native_opengl, + receiver, + wire_receiver, + qtbot, + tmp_path, ): """Use actual transport, native layers and Qt service, without delayed mocks.""" from qtpy.QtCore import QTimer @@ -239,11 +253,22 @@ def test_initial_settlement_is_observable_before_qt_intake_and_completes( items = [] for path in sorted(Path(directory).glob("*_rois.roi.zip")): parts = path.name.split("_") - metadata = {**image_item()["metadata"], "well": parts[0], - "site": int(parts[1][1:]), "channel": int(parts[2][1:])} - items.append({"path": str(path), "data_type": "shapes", - "shapes": NapariROIConverter.rois_to_shapes(load_rois_from_zip(path)), - "metadata": metadata}) + metadata = { + **image_item()["metadata"], + "well": parts[0], + "site": int(parts[1][1:]), + "channel": int(parts[2][1:]), + } + items.append( + { + "path": str(path), + "data_type": "shapes", + "shapes": NapariROIConverter.rois_to_shapes( + load_rois_from_zip(path) + ), + "metadata": metadata, + } + ) assert items item = items assert wire_receiver(item)["status"] == "success" @@ -256,10 +281,14 @@ def test_initial_settlement_is_observable_before_qt_intake_and_completes( control.setsockopt(zmq.RCVTIMEO, 2000) receiver.control_transport_pump.start() control_port = get_control_port(receiver.port, receiver.config) - control.connect(get_zmq_transport_url( - control_port, host="localhost", mode=receiver.transport_mode, - config=receiver.config, - )) + control.connect( + get_zmq_transport_url( + control_port, + host="localhost", + mode=receiver.transport_mode, + config=receiver.config, + ) + ) service = QTimer() service.timeout.connect(receiver.process_accepted_stream_messages) service.timeout.connect(receiver.process_messages) @@ -267,69 +296,104 @@ def test_initial_settlement_is_observable_before_qt_intake_and_completes( filemanager=FileManager({}), runtime_config=StreamingViewerRuntimeConfig( transport_endpoint=ViewerTransportEndpoint( - port=receiver.port, host="localhost", transport_mode=receiver.transport_mode, - ), persistent=False, viewer_family=NapariViewer, + port=receiver.port, + host="localhost", + transport_mode=receiver.transport_mode, + ), + persistent=False, + viewer_family=NapariViewer, ), ) client.lifecycle_state.mark_connected_external() observations = [] client_worker = ThreadPoolExecutor(max_workers=1) - late_observer = os.environ.get("OPENHCS_SETTLEMENT_QUALIFICATION_LATE_OBSERVER") == "true" + late_observer = ( + os.environ.get("OPENHCS_SETTLEMENT_QUALIFICATION_LATE_OBSERVER") == "true" + ) client_started = [] delivery = None control_observation = [] - inspect_pending_control = os.environ.get("OPENHCS_SETTLEMENT_QUALIFICATION_CONTROL") == "true" + inspect_pending_control = ( + os.environ.get("OPENHCS_SETTLEMENT_QUALIFICATION_CONTROL") == "true" + ) def deliver(): if late_observer or inspect_pending_control: deadline = time.monotonic() + 90 while True: progress = receiver.layer_route_state.existing_settlement_progress() - if (progress is not None and progress.active_route is not None - and progress.active_route_work_unit_active - and ((inspect_pending_control and receiver.layer_route_state.layers) - or (late_observer and "channel_2" in progress.active_route))): + if ( + progress is not None + and progress.active_route is not None + and progress.active_route_work_unit_active + and ( + (inspect_pending_control and receiver.layer_route_state.layers) + or (late_observer and "channel_2" in progress.active_route) + ) + ): break if late_observer and time.monotonic() >= deadline: raise AssertionError("No real channel-2 native work was observed.") time.sleep(0.001) if inspect_pending_control: from openhcs.runtime.viewer_controls import ViewerNavigationControlOptions - route = next(key for key in receiver.layer_route_state.layers if "channel_1" in key) + + route = next( + key for key in receiver.layer_route_state.layers if "channel_1" in key + ) native_layer = receiver.layer_route_state.layers[route] started = time.perf_counter() try: reply = ViewerControlMessageRequest( - client.runtime_endpoint, "navigate", - ViewerNavigationControlOptions(route_key=route, visible=False), timeout=0.1, + client.runtime_endpoint, + "navigate", + ViewerNavigationControlOptions(route_key=route, visible=False), + timeout=0.1, ).send() assert not reply.succeeded() except zmq.Again: pass # The caller's original observation budget expired. expired_after = time.perf_counter() - started started = time.perf_counter() - progress = ViewerSettleProgress.from_response(ViewerControlMessageRequest( - client.runtime_endpoint, "settle", timeout=2, - ).send()) - control_observation.append((native_layer, expired_after, time.perf_counter() - started, progress)) + progress = ViewerSettleProgress.from_response( + ViewerControlMessageRequest( + client.runtime_endpoint, + "settle", + timeout=2, + ).send() + ) + control_observation.append( + (native_layer, expired_after, time.perf_counter() - started, progress) + ) assert progress.phase is ViewerSettlePhase.RUNNING client_started.append(time.perf_counter()) return client.settle_viewer_state( - progress_callback=lambda progress: observations.append((time.perf_counter(), progress)), + progress_callback=lambda progress: observations.append( + (time.perf_counter(), progress) + ), ) def observe(): - control.send(pickle.dumps(ViewerControlMessageRequest( - client.runtime_endpoint, "settle", timeout=2, - ).to_wire_mapping())) - return ViewerSettleProgress.from_response(ViewerControlResponse( - pickle.loads(control.recv()) - )) + control.send( + pickle.dumps( + ViewerControlMessageRequest( + client.runtime_endpoint, + "settle", + timeout=2, + ).to_wire_mapping() + ) + ) + return ViewerSettleProgress.from_response( + ViewerControlResponse(pickle.loads(control.recv())) + ) try: started = time.perf_counter() - first = (receiver.layer_route_state.existing_settlement_progress() - if late_observer else observe()) + first = ( + receiver.layer_route_state.existing_settlement_progress() + if late_observer + else observe() + ) elapsed = time.perf_counter() - started assert first.phase is ViewerSettlePhase.RUNNING assert first.completed_update_count == first.total_update_count == 0 @@ -364,28 +428,46 @@ def observe(): assert receiver.accepted_stream_batches.empty() assert receiver.viewer.layers if inspect_pending_control: - native_layer, expired_after, settlement_reply_seconds, pending = control_observation[0] - assert not native_layer.visible # Expiry did not cancel the real Qt mutation. - print(f"native control expired after {expired_after:.3f}s; subsequent settlement " - f"reply {settlement_reply_seconds:.3f}s; pending={pending}; " - "original navigation applied after observation expiry") - assert final.processed_intake_item_count == (len(item) if isinstance(item, list) else 1) + 1 - awaiting = [(t, p) for t, p in observations - if p.phase is ViewerSettlePhase.RUNNING and p.total_update_count == 0] + native_layer, expired_after, settlement_reply_seconds, pending = ( + control_observation[0] + ) + assert ( + not native_layer.visible + ) # Expiry did not cancel the real Qt mutation. + print( + f"native control expired after {expired_after:.3f}s; subsequent settlement " + f"reply {settlement_reply_seconds:.3f}s; pending={pending}; " + "original navigation applied after observation expiry" + ) + assert ( + final.processed_intake_item_count + == (len(item) if isinstance(item, list) else 1) + 1 + ) + awaiting = [ + (t, p) + for t, p in observations + if p.phase is ViewerSettlePhase.RUNNING and p.total_update_count == 0 + ] awaiting_seconds = (awaiting[-1][0] - awaiting[0][0]) if awaiting else 0 receiver.viewer.screenshot(path=str(tmp_path / "native-settlement.png")) - first_reply_seconds = observations[0][0] - client_started[0] if late_observer else elapsed - print(f"native first settlement {first_reply_seconds:.4f}s; pending intake admitted; " - f"terminal {final.completed_update_count}/{final.total_update_count}; " - f"native layers {len(receiver.viewer.layers)}; archive={archive}; " - f"intake items={final.processed_intake_item_count}; " - f"client delivery={time.perf_counter() - started_delivery:.3f}s; " - f"unbound observations={len(awaiting)} over {awaiting_seconds:.3f}s; " - f"first={awaiting[0][1] if awaiting else None}; " - f"last={awaiting[-1][1] if awaiting else None}") + first_reply_seconds = ( + observations[0][0] - client_started[0] if late_observer else elapsed + ) + print( + f"native first settlement {first_reply_seconds:.4f}s; pending intake admitted; " + f"terminal {final.completed_update_count}/{final.total_update_count}; " + f"native layers {len(receiver.viewer.layers)}; archive={archive}; " + f"intake items={final.processed_intake_item_count}; " + f"client delivery={time.perf_counter() - started_delivery:.3f}s; " + f"unbound observations={len(awaiting)} over {awaiting_seconds:.3f}s; " + f"first={awaiting[0][1] if awaiting else None}; " + f"last={awaiting[-1][1] if awaiting else None}" + ) if late_observer: assert observations[0][1].active_route_work_unit_active - assert awaiting_seconds > 30, f"Actual unbound work was only {awaiting_seconds:.3f}s" + assert ( + awaiting_seconds > 30 + ), f"Actual unbound work was only {awaiting_seconds:.3f}s" finally: # Preserve Qt service until the original observation reaches terminal; # never block its callback owner on the observer thread's join. diff --git a/tests/unit/test_napari_roi_manager_integration.py b/tests/unit/test_napari_roi_manager_integration.py index 98bea8e64b..4a8bff9375 100644 --- a/tests/unit/test_napari_roi_manager_integration.py +++ b/tests/unit/test_napari_roi_manager_integration.py @@ -143,7 +143,9 @@ def test_installed_manager_binds_and_selects_native_shapes_without_copying( name="Native ROIs", ) - class ResultSelectionController(napari_viewer_server.NapariResultSelectionController): + class ResultSelectionController( + napari_viewer_server.NapariResultSelectionController + ): @staticmethod def bind(_layer) -> None: return None @@ -189,10 +191,17 @@ def bind(_layer) -> None: @pytest.mark.unit -@pytest.mark.parametrize("destroy", [ - "napari_close", "manager", "immediate_manager", "window", - "manager_then_dock", "manager_then_window", -]) +@pytest.mark.parametrize( + "destroy", + [ + "napari_close", + "manager", + "immediate_manager", + "window", + "manager_then_dock", + "manager_then_window", + ], +) def test_result_surface_native_destruction_releases_and_remounts(qtbot, destroy): napari = pytest.importorskip("napari") module = pytest.importorskip("openhcs.runtime.napari_viewer_server") @@ -212,12 +221,16 @@ def test_result_surface_native_destruction_releases_and_remounts(qtbot, destroy) controller = server.result_selection_controller layer = viewer.add_shapes( [np.array([[0, 0], [0, 4], [4, 4]], dtype=float)], - shape_type="polygon", features={"name": ["first"]}, name="First result", + shape_type="polygon", + features={"name": ["first"]}, + name="First result", ) server.layer_route_state.set_layer("first", layer) server.bind_result_selection_layer(layer) original = server.require_result_selection_surface() - toolbars = viewer.window._qt_window.findChildren(QToolBar, "openhcs_roi_selection_toolbar") + toolbars = viewer.window._qt_window.findChildren( + QToolBar, "openhcs_roi_selection_toolbar" + ) assert len(toolbars) == 1 assert len(controller._selection_observers) == 1 original.manager.hide() @@ -254,7 +267,9 @@ def test_result_surface_native_destruction_releases_and_remounts(qtbot, destroy) next_layer = viewer.add_shapes( [np.array([[8, 8], [8, 12], [12, 12]], dtype=float)], - shape_type="polygon", features={"name": ["next"]}, name="Next streamed result", + shape_type="polygon", + features={"name": ["next"]}, + name="Next streamed result", ) server.layer_route_state.set_layer("next", next_layer) server.bind_result_selection_layer(next_layer) @@ -263,7 +278,14 @@ def test_result_surface_native_destruction_releases_and_remounts(qtbot, destroy) assert replacement.manager._layer is next_layer assert replacement.manager._roilist.rowCount() == 1 assert len(controller._selection_observers) == 1 - assert len(viewer.window._qt_window.findChildren(QToolBar, "openhcs_roi_selection_toolbar")) == 1 + assert ( + len( + viewer.window._qt_window.findChildren( + QToolBar, "openhcs_roi_selection_toolbar" + ) + ) + == 1 + ) get_settings().appearance.highlight.highlight_thickness = 5 controller._notify_selection_observers() viewer.close() diff --git a/tests/unit/test_napari_streaming_handlers.py b/tests/unit/test_napari_streaming_handlers.py index 8b3d124c65..1dc1c309a6 100644 --- a/tests/unit/test_napari_streaming_handlers.py +++ b/tests/unit/test_napari_streaming_handlers.py @@ -108,7 +108,9 @@ def _component_name_metadata(payload, context="test component metadata"): def _component_value_domain(payload, context="test component value domain"): - return ViewerComponentValueDomainPayload.from_wire_mapping(payload, context=context, declared_axes=STREAM_AXES) + return ViewerComponentValueDomainPayload.from_wire_mapping( + payload, context=context, declared_axes=STREAM_AXES + ) def _layer_item( @@ -879,7 +881,12 @@ def add_layer(self, layer): ) def _add_layer(self, layer_type, data, name, kwargs): - layer_attributes = {"name": name, "data": data, "kwargs": kwargs, "visible": True} + layer_attributes = { + "name": name, + "data": data, + "kwargs": kwargs, + "visible": True, + } if layer_type == "shapes": layer_attributes.update( { @@ -1350,7 +1357,12 @@ def test_napari_layer_update_authority_replaces_existing_image_without_global_ax "image", "image-1", "Nuclei", - {"blending": "additive", "rgb": False, "colormap": "gray", "axis_labels": ("z", "y", "x")}, + { + "blending": "additive", + "rgb": False, + "colormap": "gray", + "axis_labels": ("z", "y", "x"), + }, ) assert viewer.calls[1] == ( "image", @@ -1480,7 +1492,9 @@ def test_napari_image_display_stacks_sites_without_rebasing_payload_color_axis() _layer_item( {"site": site}, data=np.full((16, 16, 3), site, dtype=np.uint8), - image_metadata=ImagePayloadMetadata(axes=PayloadAxes.colour_samples(2)), + image_metadata=ImagePayloadMetadata( + axes=PayloadAxes.colour_samples(2) + ), ) for site in (1, 2) ], @@ -1588,7 +1602,10 @@ def test_napari_viewer_model_keeps_declared_plate_axis_sizes_across_routes() -> layout=ViewerComponentLayout.from_parts( component_modes={ **ALL_STACKED, - **{component: NapariSlots.Stack.wire_value for component in full_route_axes}, + **{ + component: NapariSlots.Stack.wire_value + for component in full_route_axes + }, }, component_order=full_route_axes, declared_axes=STREAM_AXES, @@ -1617,7 +1634,10 @@ def test_napari_viewer_model_keeps_declared_plate_axis_sizes_across_routes() -> layout=ViewerComponentLayout.from_parts( component_modes={ **ALL_STACKED, - **{component: NapariSlots.Stack.wire_value for component in reduced_route_axes}, + **{ + component: NapariSlots.Stack.wire_value + for component in reduced_route_axes + }, }, component_order=reduced_route_axes, declared_axes=STREAM_AXES, @@ -1735,7 +1755,8 @@ def test_napari_display_pipeline_rejects_unbound_payload_local_image_axes(): ) assert ( policy.axis_labels( - np.zeros((16, 16, 3)), ImagePayloadMetadata(axes=PayloadAxes.colour_samples(-1)) + np.zeros((16, 16, 3)), + ImagePayloadMetadata(axes=PayloadAxes.colour_samples(-1)), ) == () ) @@ -1785,17 +1806,15 @@ def test_napari_display_pipeline_projects_route_axes_locally(): component_value_domain = _component_value_domain( {"channel": [1, 2, 3, 4, 5], "site": [1, 2]} ) - component_axis_semantics = ( - ViewerComponentAxisSemanticsFactory.from_display_config( - ViewerMappingDisplayConfigInput( - { - "component_modes": {"channel": "stack", "site": "stack"}, - "component_order": ["channel", "site"], - "declared_axes": STREAM_AXES.to_wire(), - } - ), - component_value_domain, - ) + component_axis_semantics = ViewerComponentAxisSemanticsFactory.from_display_config( + ViewerMappingDisplayConfigInput( + { + "component_modes": {"channel": "stack", "site": "stack"}, + "component_order": ["channel", "site"], + "declared_axes": STREAM_AXES.to_wire(), + } + ), + component_value_domain, ) first = pipeline.display_axis_projection( @@ -1836,17 +1855,15 @@ def test_napari_display_pipeline_projects_route_axes_locally(): def test_napari_display_pipeline_uses_declared_domain_independent_of_arrival_order(): napari_viewer_server = pytest.importorskip("openhcs.runtime.napari_viewer_server") pipeline = napari_viewer_server.NapariLayerDisplayPipeline(_FakeNapariServer()) - component_axis_semantics = ( - ViewerComponentAxisSemanticsFactory.from_display_config( - ViewerMappingDisplayConfigInput( - { - "component_modes": {"channel": "stack"}, - "component_order": ["channel"], - "declared_axes": STREAM_AXES.to_wire(), - } - ), - _component_value_domain({"channel": [1, 2, 4]}), - ) + component_axis_semantics = ViewerComponentAxisSemanticsFactory.from_display_config( + ViewerMappingDisplayConfigInput( + { + "component_modes": {"channel": "stack"}, + "component_order": ["channel"], + "declared_axes": STREAM_AXES.to_wire(), + } + ), + _component_value_domain({"channel": [1, 2, 4]}), ) derived = pipeline.display_axis_projection( @@ -2022,51 +2039,77 @@ def test_paired_raw_stack_review_accepts_layer_pipeline_at_same_channel_coordina server.viewer = ViewerModel() pipeline = NapariLayerDisplayPipeline(server) for origin in (("raw", "pipeline") if raw_first else ("pipeline", "raw")): - config = config_type(**{ - mode_field: ( - NapariDimensionMode.STACK if origin == stack_origin - else NapariDimensionMode.LAYER - ) - }) - producer = StreamProducerIdentity("manual", "image", "raw", "raw") if origin == "raw" else None + config = config_type( + **{ + mode_field: ( + NapariDimensionMode.STACK + if origin == stack_origin + else NapariDimensionMode.LAYER + ) + } + ) + producer = ( + StreamProducerIdentity("manual", "image", "raw", "raw") + if origin == "raw" + else None + ) for channel in (1, 2): route = f"{origin}-{channel}" item = _layer_item( - {"site": 1, "channel": 1, "z_index": 1, - "timepoint": 1, "well": "A01", component: channel}, + { + "site": 1, + "channel": 1, + "z_index": 1, + "timepoint": 1, + "well": "A01", + component: channel, + }, data=np.full((4, 4), channel, dtype=np.uint16), producer=producer, ) semantics = ViewerComponentAxisSemantics( - entries=_component_value_domain({ - component: [value] for component, value in item.address.components.items() - }).entries, + entries=_component_value_domain( + { + component: [value] + for component, value in item.address.components.items() + } + ).entries, layout=ViewerObjectDisplayConfigInput(config).layout(), ) server.layer_route_state.set_title(route, route) work = pipeline.display_layer_batch( - layer_key=route, items=[item], + layer_key=route, + items=[item], display_payload=NapariPendingLayerUpdate.from_semantics( - timer=_FakeTimer(), data_type=StreamingDataType.IMAGE, - semantics=semantics, display_config=config, + timer=_FakeTimer(), + data_type=StreamingDataType.IMAGE, + semantics=semantics, + display_config=config, ), component_names_metadata=ViewerComponentNameMetadata.empty(), ) assert work.advance() assert ViewerObjectDisplayConfigInput(config).layout() == semantics.layout for channel in (1, 2): - raw = server.layer_route_state.dimension_state_for(f"raw-{channel}").presentation - result = server.layer_route_state.dimension_state_for(f"pipeline-{channel}").presentation + raw = server.layer_route_state.dimension_state_for( + f"raw-{channel}" + ).presentation + result = server.layer_route_state.dimension_state_for( + f"pipeline-{channel}" + ).presentation axis = raw.display_axis_components.index(component) assert result.display_axis_components == raw.display_axis_components assert raw.viewer_step(0, axis) == result.viewer_step(0, axis) == channel - 1 assert raw.projection.routed_component_values[component] == [channel] assert result.projection.routed_component_values[component] == [channel] assert np.max(server.layer_route_state.layer(f"raw-{channel}").data) == channel - assert np.max(server.layer_route_state.layer(f"pipeline-{channel}").data) == channel - assert tuple(server.layer_route_state.layer(f"raw-{channel}").translate) == tuple( - server.layer_route_state.layer(f"pipeline-{channel}").translate + assert ( + np.max(server.layer_route_state.layer(f"pipeline-{channel}").data) + == channel ) + assert tuple( + server.layer_route_state.layer(f"raw-{channel}").translate + ) == tuple(server.layer_route_state.layer(f"pipeline-{channel}").translate) def test_display_batch_composes_domain_and_names_for_new_declared_component(): @@ -2184,17 +2227,15 @@ def test_napari_display_pipeline_projects_aggregate_payload_axes_into_route_doma component_value_domain = _component_value_domain( {"channel": [1], "z_index": [1, 2]} ) - component_axis_semantics = ( - ViewerComponentAxisSemanticsFactory.from_display_config( - ViewerMappingDisplayConfigInput( - { - "component_modes": {"channel": "stack", "z_index": "stack"}, - "component_order": ["z_index", "channel"], - "declared_axes": STREAM_AXES.to_wire(), - } - ), - component_value_domain, - ) + component_axis_semantics = ViewerComponentAxisSemanticsFactory.from_display_config( + ViewerMappingDisplayConfigInput( + { + "component_modes": {"channel": "stack", "z_index": "stack"}, + "component_order": ["z_index", "channel"], + "declared_axes": STREAM_AXES.to_wire(), + } + ), + component_value_domain, ) projection = pipeline.display_axis_projection( @@ -3407,10 +3448,16 @@ def test_clear_state_preserves_mounted_payload_and_domains_until_native_removal( server.component_groups.items_for("cancelled").append(_layer_item({"well": "A14"})) server.component_values = ViewerRouteComponentValueTracker() server.component_values.update("retained", ["well"], [item]) - server.component_values.declare_component_values("retained", ["well"], {"well": ["A01", "A02"]}) - server.component_values.update("cancelled", ["well"], [_layer_item({"well": "A14"})]) + server.component_values.declare_component_values( + "retained", ["well"], {"well": ["A01", "A02"]} + ) + server.component_values.update( + "cancelled", ["well"], [_layer_item({"well": "A14"})] + ) domain = server.component_values.domain_for("retained", ["well"]) - server.component_name_metadata = _component_name_metadata({"well": {"A01": "source well"}}) + server.component_name_metadata = _component_name_metadata( + {"well": {"A01": "source well"}} + ) server.layer_batch_processor_debounce_policy = NapariLayerBatchDebouncePolicy() server.display_pipeline = NapariLayerDisplayPipeline(server) server.batch_processors = NapariBatchProcessorStore() @@ -3421,7 +3468,9 @@ def test_clear_state_preserves_mounted_payload_and_domains_until_native_removal( assert server.component_groups.existing_items_for("cancelled") is None assert server.component_values.domain_for("retained", ["well"]) is domain assert domain.coordinate_values("well") == {"A01", "A02"} - assert server.component_values.shared_values_for(["well"]) == {"well": ["A01", "A02"]} + assert server.component_values.shared_values_for(["well"]) == { + "well": ["A01", "A02"] + } assert server.component_name_metadata.display_name("well", "A01") == "source well" server.viewer.layers.remove(layer) @@ -3484,7 +3533,8 @@ def require_result_selection_surface(self): ) events.append("result_selection_surface_open") napari_viewer_server._install_result_selection_toolbar( - self.result_selection_surface, self.result_selection_controller, + self.result_selection_surface, + self.result_selection_controller, ) return self.result_selection_surface @@ -4196,7 +4246,9 @@ def test_roi_selection_toolbar_adjusts_native_highlight_setting(qtbot, monkeypat server = type("ResultSelectionServer", (), {"viewer": None})() controller = napari_viewer_server.NapariResultSelectionController(server) - surface = napari_viewer_server.NapariResultSelectionSurface(result_selection_dock, object()) + surface = napari_viewer_server.NapariResultSelectionSurface( + result_selection_dock, object() + ) qt_window.destroyed.connect(surface.release) toolbar = napari_viewer_server._install_result_selection_toolbar( surface, @@ -4263,7 +4315,9 @@ def test_roi_layer_color_button_recolors_every_shape(qtbot, monkeypatch): lambda *_args, **_kwargs: QColor.fromRgbF(0.2, 0.8, 0.3, 1.0), ) - surface = napari_viewer_server.NapariResultSelectionSurface(result_selection_dock, object()) + surface = napari_viewer_server.NapariResultSelectionSurface( + result_selection_dock, object() + ) qt_window.destroyed.connect(surface.release) toolbar = napari_viewer_server._install_result_selection_toolbar( surface, @@ -4333,8 +4387,6 @@ def test_napari_entrypoint_bind_failure_quits_event_loop_and_stops_server(monkey assert events[-1] == "server_stop" - - @pytest.mark.parametrize("message_type", ("shutdown", "force_shutdown")) def test_napari_shutdown_cancels_pending_updates_before_scheduling_viewer_close( monkeypatch, @@ -4408,12 +4460,6 @@ def close(self): assert not any(event.startswith("display:") for event in events) - - - - - - def test_napari_axis_projector_drops_only_globally_singleton_axes(): pytest.importorskip("openhcs.runtime.napari_viewer_server") component_values = { @@ -4537,7 +4583,9 @@ def __init__(self): self.component_groups = NapariComponentGroupStore() self.component_values = ViewerRouteComponentValueTracker() self.batch_processors = napari_viewer_server.NapariBatchProcessorStore() - self.display_pipeline = napari_viewer_server.NapariLayerDisplayPipeline(self) + self.display_pipeline = napari_viewer_server.NapariLayerDisplayPipeline( + self + ) server = _FakeServer() route_key = "deleted-channel-1" @@ -4634,11 +4682,9 @@ def __init__(self): "component_order": ["well"], "declared_axes": STREAM_AXES.to_wire(), } - component_axis_semantics = ( - ViewerComponentAxisSemanticsFactory.from_display_config( - ViewerMappingDisplayConfigInput(display_config), - _component_value_domain({"well": ["A01"]}), - ) + component_axis_semantics = ViewerComponentAxisSemanticsFactory.from_display_config( + ViewerMappingDisplayConfigInput(display_config), + _component_value_domain({"well": ["A01"]}), ) for data, image_metadata in ( @@ -4716,17 +4762,15 @@ def __init__(self): step_name="OverlayOutlines", pipeline_position=7, ) - component_axis_semantics = ( - ViewerComponentAxisSemanticsFactory.from_display_config( - ViewerMappingDisplayConfigInput( - { - "component_modes": {"well": "stack", "channel": "stack"}, - "component_order": ["well", "channel"], - "declared_axes": STREAM_AXES.to_wire(), - } - ), - _component_value_domain({"well": ["A01"], "channel": [2]}), - ) + component_axis_semantics = ViewerComponentAxisSemanticsFactory.from_display_config( + ViewerMappingDisplayConfigInput( + { + "component_modes": {"well": "stack", "channel": "stack"}, + "component_order": ["well", "channel"], + "declared_axes": STREAM_AXES.to_wire(), + } + ), + _component_value_domain({"well": ["A01"], "channel": [2]}), ) coordinator.display( @@ -5393,17 +5437,15 @@ def test_napari_display_projection_derives_sparse_coordinates_from_routed_items( "well": ["A02", "B04", "B12", "B17"], "site": [1, 2, 4, 7], } - component_axis_semantics = ( - ViewerComponentAxisSemanticsFactory.from_display_config( - ViewerMappingDisplayConfigInput( - { - "component_modes": {"well": "stack", "site": "stack"}, - "component_order": ["well", "site"], - "declared_axes": STREAM_AXES.to_wire(), - } - ), - _component_value_domain(component_values), - ) + component_axis_semantics = ViewerComponentAxisSemanticsFactory.from_display_config( + ViewerMappingDisplayConfigInput( + { + "component_modes": {"well": "stack", "site": "stack"}, + "component_order": ["well", "site"], + "declared_axes": STREAM_AXES.to_wire(), + } + ), + _component_value_domain(component_values), ) projection = pipeline.display_axis_projection( @@ -5767,7 +5809,9 @@ def test_napari_shape_feature_columns_fill_sparse_late_columns_in_order(): columns = NapariShapeFeatureColumns() columns.append({"area": 4.0}, label=1, path="first", element_identity="first:0") - columns.append({"circularity": 0.8}, label=2, path="second", element_identity="second:0") + columns.append( + {"circularity": 0.8}, label=2, path="second", element_identity="second:0" + ) columns.append( {"area": 9.0, "circularity": 0.6}, label=3, @@ -5779,7 +5823,11 @@ def test_napari_shape_feature_columns_fill_sparse_late_columns_in_order(): "area": [4.0, None, 9.0], "label": [1, 2, 3], "path": ["first", "second", "third"], - NapariStreamLayerItem.ELEMENT_IDENTITY_FEATURE: ["first:0", "second:0", "third:0"], + NapariStreamLayerItem.ELEMENT_IDENTITY_FEATURE: [ + "first:0", + "second:0", + "third:0", + ], "circularity": [None, 0.8, 0.6], } @@ -5823,17 +5871,15 @@ def test_napari_aggregate_axis_binding_uses_declared_component_not_equal_extent( component_value_domain = _component_value_domain( {"site": [1, 2], "channel": [1, 2]} ) - component_axis_semantics = ( - ViewerComponentAxisSemanticsFactory.from_display_config( - ViewerMappingDisplayConfigInput( - { - "component_modes": {"site": "stack", "channel": "stack"}, - "component_order": ["site", "channel"], - "declared_axes": STREAM_AXES.to_wire(), - } - ), - component_value_domain, - ) + component_axis_semantics = ViewerComponentAxisSemanticsFactory.from_display_config( + ViewerMappingDisplayConfigInput( + { + "component_modes": {"site": "stack", "channel": "stack"}, + "component_order": ["site", "channel"], + "declared_axes": STREAM_AXES.to_wire(), + } + ), + component_value_domain, ) items = [ _layer_item( @@ -5878,17 +5924,15 @@ def test_napari_shape_plane_metadata_requires_declared_component_axis(): component_value_domain = _component_value_domain( {"site": [1, 2], "channel": [1, 2]} ) - component_axis_semantics = ( - ViewerComponentAxisSemanticsFactory.from_display_config( - ViewerMappingDisplayConfigInput( - { - "component_modes": {"site": "stack", "channel": "stack"}, - "component_order": ["site", "channel"], - "declared_axes": STREAM_AXES.to_wire(), - } - ), - component_value_domain, - ) + component_axis_semantics = ViewerComponentAxisSemanticsFactory.from_display_config( + ViewerMappingDisplayConfigInput( + { + "component_modes": {"site": "stack", "channel": "stack"}, + "component_order": ["site", "channel"], + "declared_axes": STREAM_AXES.to_wire(), + } + ), + component_value_domain, ) with pytest.raises(ValueError, match="plane_component_values"): @@ -5918,17 +5962,15 @@ def test_napari_shape_label_rasterizer_rejects_mixed_aggregate_plane_metadata(): component_value_domain = _component_value_domain( {"channel": [1], "z_index": [1, 2]} ) - component_axis_semantics = ( - ViewerComponentAxisSemanticsFactory.from_display_config( - ViewerMappingDisplayConfigInput( - { - "component_modes": {"channel": "stack", "z_index": "stack"}, - "component_order": ["z_index", "channel"], - "declared_axes": STREAM_AXES.to_wire(), - } - ), - component_value_domain, - ) + component_axis_semantics = ViewerComponentAxisSemanticsFactory.from_display_config( + ViewerMappingDisplayConfigInput( + { + "component_modes": {"channel": "stack", "z_index": "stack"}, + "component_order": ["z_index", "channel"], + "declared_axes": STREAM_AXES.to_wire(), + } + ), + component_value_domain, ) with pytest.raises(ValueError, match="mixes plane-indexed and unindexed"): @@ -6166,8 +6208,13 @@ def test_napari_points_layer_display_applies_route_global_axis_translate(): ) item = _layer_item( {"channel": 4}, - [{"type": "points", "coordinates": [[1, 2]], - "metadata": {"label": 7, "component": 4}}], + [ + { + "type": "points", + "coordinates": [[1, 2]], + "metadata": {"label": 7, "component": 4}, + } + ], stream_layer_data_type=StreamingDataType.POINTS, ) @@ -6188,7 +6235,8 @@ def test_napari_points_layer_display_applies_route_global_axis_translate(): assert layer_kwargs["axis_labels"] == ("channel", "y", "x") assert layer_kwargs["translate"] == (3.0, 0.0, 0.0) assert layer_kwargs["properties"] == { - "label": [7], "component": [4], + "label": [7], + "component": [4], NapariStreamLayerItem.ELEMENT_IDENTITY_FEATURE: [item.element_identity(0)], } @@ -6243,14 +6291,30 @@ def test_napari_points_layer_uses_exact_fractional_z_from_native_roi_metadata(): @pytest.mark.parametrize( "values,unit,expected_scale,expected_units", ( - ((2.0, 0.65, 0.65), "micrometers", (1.0, 2.0, 0.65, 0.65), - ("dimensionless", "micrometer", "micrometer", "micrometer")), - ((2.0, 1.0, 1.0), "relative", (1.0, 2.0, 1.0, 1.0), - ("dimensionless", "dimensionless", "dimensionless", "dimensionless")), - ((0.65, 0.65), "micrometers", (1.0, 1.0, 0.65, 0.65), - ("dimensionless", "dimensionless", "micrometer", "micrometer")), - ((), "micrometers", (1.0, 1.0, 1.0, 1.0), - ("dimensionless", "dimensionless", "pixel", "pixel")), + ( + (2.0, 0.65, 0.65), + "micrometers", + (1.0, 2.0, 0.65, 0.65), + ("dimensionless", "micrometer", "micrometer", "micrometer"), + ), + ( + (2.0, 1.0, 1.0), + "relative", + (1.0, 2.0, 1.0, 1.0), + ("dimensionless", "dimensionless", "dimensionless", "dimensionless"), + ), + ( + (0.65, 0.65), + "micrometers", + (1.0, 1.0, 0.65, 0.65), + ("dimensionless", "dimensionless", "micrometer", "micrometer"), + ), + ( + (), + "micrometers", + (1.0, 1.0, 1.0, 1.0), + ("dimensionless", "dimensionless", "pixel", "pixel"), + ), ), ) def test_spatial_presentation_owns_scale_and_world_translation( @@ -6264,19 +6328,26 @@ def test_spatial_presentation_owns_scale_and_world_translation( component_values={"channel": [4], "z_index": [2, 3, 4, 5]}, axis_offsets=(3, 2), ) - item = _layer_item( - {"channel": 4, "z_index": 2}, np.zeros((5, 7), dtype=np.uint16) + item = _layer_item({"channel": 4, "z_index": 2}, np.zeros((5, 7), dtype=np.uint16)) + item = replace( + item, + image_metadata=ImagePayloadMetadata( + source_voxel_spacing=SourceVoxelSpacing( + values, SourceVoxelSpacingUnit(unit) + ) + ), ) - item = replace(item, image_metadata=ImagePayloadMetadata( - source_voxel_spacing=SourceVoxelSpacing(values, SourceVoxelSpacingUnit(unit)) - )) kwargs = presentation.spatial_layer_kwargs([item]) assert kwargs["scale"] == expected_scale assert kwargs["units"] == expected_units assert kwargs["translate"] == (3.0, 2 * expected_scale[1], 0.0, 0.0) payload_kwargs = presentation.spatial_layer_kwargs([item], ("band",)) assert payload_kwargs["scale"] == (*expected_scale[:2], 1.0, *expected_scale[2:]) - assert payload_kwargs["units"] == (*expected_units[:2], "dimensionless", *expected_units[2:]) + assert payload_kwargs["units"] == ( + *expected_units[:2], + "dimensionless", + *expected_units[2:], + ) def test_calibrated_native_image_point_and_semantic_navigation_agree_without_qt(): @@ -6317,12 +6388,17 @@ def test_calibrated_native_image_point_and_semantic_navigation_agree_without_qt( def test_shape_features_exclude_source_transport_without_mutating_payload(): from openhcs.core.roi_source_metadata import ROIArchiveSourceMetadata - full_metadata = {"label": 7, "response": 0.25, - ROIArchiveSourceMetadata.FIELD: {"source_provenance": "retained"}} + full_metadata = { + "label": 7, + "response": 0.25, + ROIArchiveSourceMetadata.FIELD: {"source_provenance": "retained"}, + } features = NapariShapeFeatureColumns() item = _layer_item({}, stream_layer_data_type=StreamingDataType.SHAPES) features.append( - full_metadata, label=7, path=item.address.path, + full_metadata, + label=7, + path=item.address.path, element_identity=item.element_identity(0), ) assert ROIArchiveSourceMetadata.FIELD in full_metadata @@ -6358,7 +6434,9 @@ def test_napari_fractional_z_points_are_relative_to_the_declared_anchor(): ) projection = pipeline.display_axis_projection("centres", semantics, [item]) assert projection.component_values == {"z_index": [2, 3, 4]} - points, _properties = napari_viewer_server._build_nd_points([item], projection, ("z_index",)) + points, _properties = napari_viewer_server._build_nd_points( + [item], projection, ("z_index",) + ) assert tuple(points[0]) == (1.25, 1.25, 3.5) assert projection.axis_offsets == (2,) diff --git a/tests/unit/test_native_viewer_presentation.py b/tests/unit/test_native_viewer_presentation.py index f384f17f4e..08923183f7 100644 --- a/tests/unit/test_native_viewer_presentation.py +++ b/tests/unit/test_native_viewer_presentation.py @@ -64,7 +64,9 @@ def presentation(): semantics = ViewerComponentAxisSemanticsFactory.empty() return NapariAxisPresentation( entries=semantics.entries, - layout=ViewerComponentLayout.from_parts(component_modes={}, component_order=(), declared_axes=STREAM_AXES), + layout=ViewerComponentLayout.from_parts( + component_modes={}, component_order=(), declared_axes=STREAM_AXES + ), route_key="paired", projection=ViewerLayerAxisProjection( projected_axis_components=(), @@ -308,7 +310,7 @@ def test_native_display_handlers_apply_shared_calibration_after_wire_roundtrip() def test_two_dimensional_calibration_does_not_calibrate_component_z_axis(): - + axis = replace( presentation(), layout=ViewerComponentLayout.from_parts( @@ -579,14 +581,25 @@ def test_selected_file_stream_restores_persisted_crop_calibration_and_native_bat spacing = SourceVoxelSpacing((1.3556, 1.3556)) domain = SourceSpatialDomain((7, 11), (20, 30), 0, "Retained crop") for channel in (1, 2): - address = OpenHCSPlaneAddress(((Microscopy.Well, well), (Microscopy.Site, 1), (Microscopy.Channel, channel), (Microscopy.ZIndex, 1), (Microscopy.Timepoint, 1))) + address = OpenHCSPlaneAddress( + ( + (Microscopy.Well, well), + (Microscopy.Site, 1), + (Microscopy.Channel, channel), + (Microscopy.ZIndex, 1), + (Microscopy.Timepoint, 1), + ) + ) component_metadata = { component.name: value for component, value in address.component_values().items() } contributors = tuple( SourceImageProvenanceContributor( - SourceImageIdentity(str(tmp_path / f"synthetic-site{site}-channel{channel}.tif"), {**component_metadata, "site": site}), + SourceImageIdentity( + str(tmp_path / f"synthetic-site{site}-channel{channel}.tif"), + {**component_metadata, "site": site}, + ), source_image_name="neurite" if channel == 1 else "nucleus", ) for site in range(1, 10) @@ -597,7 +610,9 @@ def test_selected_file_stream_restores_persisted_crop_calibration_and_native_bat source_provenance=SourceImageProvenance( source_component_metadata=component_metadata, source_image_names=("neurite" if channel == 1 else "nucleus",), - source_image_provenance_planes=SourceImageProvenancePlanes(contributors), + source_image_provenance_planes=SourceImageProvenancePlanes( + contributors + ), ), ) path = tmp_path / f"stored-channel{channel}.tif" diff --git a/tests/unit/test_neurite_outgrowth.py b/tests/unit/test_neurite_outgrowth.py index eddc18f647..401719724b 100644 --- a/tests/unit/test_neurite_outgrowth.py +++ b/tests/unit/test_neurite_outgrowth.py @@ -78,7 +78,12 @@ def _declared_shaft_admission(response, mask=None): return NeuriteAdmissionResult( mask=support, planes=NeuriteAdmissionPlanes( - response, support, support, response, response > 0, response, + response, + support, + support, + response, + response > 0, + response, ), outgrowth_width_px=2.0, ) @@ -100,7 +105,8 @@ def _cell_body_count_implementation(): def test_admission_exports_the_original_independent_gates(seed_factor): image = _draw_fluorescent_neuron(branched=True)[0] settings = MetaXpressOutgrowthSettings( - maximum_width=3, intensity_above_local_background=100, + maximum_width=3, + intensity_above_local_background=100, candidate_hysteresis_seed_correction_factor=seed_factor, ) result = _identify_neurites_cellprofiler( @@ -109,9 +115,7 @@ def test_admission_exports_the_original_independent_gates(seed_factor): assert isinstance(result, NeuriteAdmissionResult) planes = result.planes assert result.response is planes.local_response - np.testing.assert_array_equal( - planes.local_support, planes.local_response >= 100 - ) + np.testing.assert_array_equal(planes.local_support, planes.local_response >= 100) np.testing.assert_array_equal( result.mask, planes.retained_support & planes.local_support ) @@ -141,19 +145,26 @@ def test_pixel_recipe_matches_physical_detection_and_declares_metric_units( image = np.concatenate((body_and_process, nuclei[None]), axis=0) before = image.copy() pixel_body = PixelCellBodySettings( - approximate_max_width=30, minimum_area=100, + approximate_max_width=30, + minimum_area=100, intensity_above_local_background=100, channel_index=0 if with_nuclei else None, ) pixel_growth = PixelOutgrowthSettings( - maximum_width=3, intensity_above_local_background=100, + maximum_width=3, + intensity_above_local_background=100, minimum_cell_growth_to_log_as_significant=20, ) contract = CallableContract.from_callable(neurite_outgrowth_metaxpress_pixels) assert contract.artifact_inputs.names() == () - assert "pixel_size" not in signature(contract.resolve_raw_runtime_callable()).parameters + assert ( + "pixel_size" + not in signature(contract.resolve_raw_runtime_callable()).parameters + ) pixel = contract.resolve_raw_runtime_callable()( - image, cell_body=pixel_body, outgrowth=pixel_growth, + image, + cell_body=pixel_body, + outgrowth=pixel_growth, neurite_channel_index=1 if with_nuclei else 0, use_nuclear_stain=with_nuclei, nuclear_stain=PixelNuclearSettings(channel_index=2), @@ -161,12 +172,14 @@ def test_pixel_recipe_matches_physical_detection_and_declares_metric_units( physical = _implementation()( image, cell_body=MetaXpressCellBodySettings( - approximate_max_width=15, minimum_area=25, + approximate_max_width=15, + minimum_area=25, intensity_above_local_background=100, channel_index=0 if with_nuclei else None, ), outgrowth=MetaXpressOutgrowthSettings( - maximum_width=1.5, intensity_above_local_background=100, + maximum_width=1.5, + intensity_above_local_background=100, minimum_cell_growth_to_log_as_significant=10, ), pixel_size=0.5, @@ -202,9 +215,12 @@ def test_pixel_recipe_matches_physical_detection_and_declares_metric_units( assert tuple(type(option) for option in morphology.materialization.outputs) == ( SpatialGraphROIOptions, ) - assert morphology.relations == CallableContract.from_callable( - neurite_outgrowth_metaxpress - ).artifact_outputs[-1].relations + assert ( + morphology.relations + == CallableContract.from_callable(neurite_outgrowth_metaxpress) + .artifact_outputs[-1] + .relations + ) @pytest.mark.parametrize( @@ -231,11 +247,13 @@ def test_pixel_recipe_rows_csv_and_native_graph_roi_keep_distinct_source_units( ).resolve_raw_runtime_callable()( image, cell_body=PixelCellBodySettings( - approximate_max_width=30, minimum_area=100, + approximate_max_width=30, + minimum_area=100, intensity_above_local_background=100, ), outgrowth=PixelOutgrowthSettings( - maximum_width=3, intensity_above_local_background=100, + maximum_width=3, + intensity_above_local_background=100, minimum_cell_growth_to_log_as_significant=20, ), ) @@ -243,10 +261,15 @@ def test_pixel_recipe_rows_csv_and_native_graph_roi_keep_distinct_source_units( source_voxel_spacing=spacing, source_image_provenance_planes=SourceImageProvenancePlanes.from_components( paths=("/engineering/A01_s1_w2.tif",), - component_metadata=({ - "well": "A01", "site": 1, "channel": 2, - "z_index": 1, "timepoint": 1, - },), + component_metadata=( + { + "well": "A01", + "site": 1, + "channel": 2, + "z_index": 1, + "timepoint": 1, + }, + ), ), ) source = metadata.payload_with(image) @@ -260,9 +283,12 @@ def test_pixel_recipe_rows_csv_and_native_graph_roi_keep_distinct_source_units( neurite_outgrowth_metaxpress_pixels ).artifact_outputs csv_path = materialize( - declarations[1].materialization, data=result[2], - path=str(tmp_path / "cells.csv"), filemanager=manager, - backends=["disk"], backend_kwargs={}, + declarations[1].materialization, + data=result[2], + path=str(tmp_path / "cells.csv"), + filemanager=manager, + backends=["disk"], + backend_kwargs={}, ) with open(csv_path, newline="") as stream: rows = tuple(csv.DictReader(stream)) @@ -270,9 +296,12 @@ def test_pixel_recipe_rows_csv_and_native_graph_roi_keep_distinct_source_units( assert {row["coordinate_unit"] for row in rows} == {"pixels"} assert float(rows[0]["total_outgrowth"]) > 0 zip_path = materialize( - declarations[-1].materialization, data=graph, - path=str(tmp_path / "morphology.roi.zip"), filemanager=manager, - backends=["disk"], backend_kwargs={}, + declarations[-1].materialization, + data=graph, + path=str(tmp_path / "morphology.roi.zip"), + filemanager=manager, + backends=["disk"], + backend_kwargs={}, ) restored = load_rois_from_zip(Path(zip_path)) saved = ROIArchiveSourceMetadata.decode(restored) @@ -283,13 +312,18 @@ def test_pixel_recipe_rows_csv_and_native_graph_roi_keep_distinct_source_units( for roi, edge in zip(geometry, graph.edges, strict=True): np.testing.assert_array_equal(roi.shapes[0].coordinates, edge.coordinates) assert roi.metadata["coordinate_unit"] == "pixels" - assert roi.metadata["branch_distance"] == edge.feature_mapping()["branch_distance"] + assert ( + roi.metadata["branch_distance"] == edge.feature_mapping()["branch_distance"] + ) assert not tuple(tmp_path.glob("*.swc")) with pytest.raises(ValueError, match="explicit micrometer spacing"): materialize( - MaterializationSpec(SWCOptions()), data=graph, - path=str(tmp_path / "forbidden.swc"), filemanager=manager, - backends=["disk"], backend_kwargs={}, + MaterializationSpec(SWCOptions()), + data=graph, + path=str(tmp_path / "forbidden.swc"), + filemanager=manager, + backends=["disk"], + backend_kwargs={}, ) assert not tuple(tmp_path.glob("*.swc")) @@ -332,7 +366,10 @@ class AuditedPixelRecipe(MetricAudit, PixelCellProfilerNeuriteEngineProfile): SpatialGraphROIOptions, ) assert events == [ - "export-before", "export-after", "analysis-before", "analysis-after" + "export-before", + "export-after", + "analysis-before", + "analysis-after", ] @@ -399,7 +436,9 @@ def minimum_area_px(self, pixel_size_um): @dataclass(frozen=True) class DeclaredBody( - ProjectionAudit, HalfWidthProjection, QuarterAreaProjection, + ProjectionAudit, + HalfWidthProjection, + QuarterAreaProjection, MetaXpressCellBodySettings, ): pass @@ -413,17 +452,21 @@ class DeclaredOutgrowth( image = _draw_fluorescent_neuron(branched=True) original = image.copy() expected = _implementation()( - image, cell_body=_cell_body_settings(), outgrowth=_outgrowth_settings(), + image, + cell_body=_cell_body_settings(), + outgrowth=_outgrowth_settings(), pixel_size=1.3556, ) actual = _implementation()( image, cell_body=DeclaredBody( - approximate_max_width=60.0, minimum_area=400.0, + approximate_max_width=60.0, + minimum_area=400.0, intensity_above_local_background=100.0, ), outgrowth=DeclaredOutgrowth( - maximum_width=6.0, intensity_above_local_background=100.0, + maximum_width=6.0, + intensity_above_local_background=100.0, minimum_cell_growth_to_log_as_significant=20.0, ), pixel_size=1.3556, @@ -588,7 +631,9 @@ def test_cell_body_minimum_area_does_not_impose_hidden_roundness(monkeypatch): monkeypatch.setattr( "openhcs.processing.backends.analysis.neurite_outgrowth.identify_primary_objects", lambda *_args, **_kwargs: ( - image, image, detected_labels, + image, + image, + detected_labels, *(object() for _ in PrimaryObjectDiagnosticPlanes._fields), ), ) @@ -623,7 +668,9 @@ def test_nuclear_support_ignores_rejected_nearer_body_candidate(monkeypatch): monkeypatch.setattr( "openhcs.processing.backends.analysis.neurite_outgrowth.identify_primary_objects", lambda *_args, **_kwargs: ( - image, image, detected_labels, + image, + image, + detected_labels, *(object() for _ in PrimaryObjectDiagnosticPlanes._fields), ), ) @@ -768,9 +815,7 @@ def test_topology_metrics_and_significant_threshold_is_scoring_only(): assert low_cell["significant_growth"] is True assert high_cell["significant_growth"] is False assert high_summary["cells_significant_growth"] == 0 - assert high_cell["total_outgrowth"] == pytest.approx( - low_cell["total_outgrowth"] - ) + assert high_cell["total_outgrowth"] == pytest.approx(low_cell["total_outgrowth"]) assert np.array_equal( low_threshold[4], high_threshold[4], @@ -809,7 +854,11 @@ def test_edgeless_skeleton_has_empty_topology(coordinates): skeleton[coordinate] = True original = skeleton.copy() topology = _analyze_topology( - skeleton, np.zeros(skeleton.shape, np.int32), 1.3556, 3.0, branch_support=skeleton, + skeleton, + np.zeros(skeleton.shape, np.int32), + 1.3556, + 3.0, + branch_support=skeleton, ) assert all(len(getattr(topology, field.name)) == 0 for field in fields(topology)) assert np.array_equal(skeleton, original) @@ -994,7 +1043,12 @@ def test_crossing_cluster_counts_three_owned_arms_across_physical_nodes(): bodies[30:35, 3:9] = 1 bodies[7:13, 47:53] = 2 topology = _analyze_topology( - labels > 0, bodies, 1.0, 8.0, assigned_path_labels=labels, branch_support=labels > 0, + labels > 0, + bodies, + 1.0, + 8.0, + assigned_path_labels=labels, + branch_support=labels > 0, ) # Three same-owner arms are now a branch, not a retained two-arm crossover. assert topology.crossing_nodes == frozenset() @@ -1063,23 +1117,33 @@ def test_short_two_junction_crossing_resolves_opposite_rooted_traces(pixel_size_ owned = _render_owned_skeleton(skeleton.shape, topology) response = np.where(skeleton, 150.0, 0.0) repaired = _repair_signal_supported_skeleton( - owned, _declared_shaft_admission(response), np.zeros_like(cell_bodies), cell_bodies, - minimum_response=100.0, crossing_topology=topology, + owned, + _declared_shaft_admission(response), + np.zeros_like(cell_bodies), + cell_bodies, + minimum_response=100.0, + crossing_topology=topology, ) core = topology.crossing_core_mask(skeleton.shape) np.testing.assert_array_equal(repaired[core], owned[core]) # Renumbering reverses physical owner traversal without changing geometry. reversed_topology = replace( - topology, path_owners=np.where(topology.path_owners > 0, 3 - topology.path_owners, 0), + topology, + path_owners=np.where(topology.path_owners > 0, 3 - topology.path_owners, 0), ) reversed_owned = np.where(owned > 0, 3 - owned, 0) reversed_bodies = np.where(cell_bodies > 0, 3 - cell_bodies, 0) reversed_repair = _repair_signal_supported_skeleton( - reversed_owned, _declared_shaft_admission(response), np.zeros_like(cell_bodies), reversed_bodies, - minimum_response=100.0, crossing_topology=reversed_topology, + reversed_owned, + _declared_shaft_admission(response), + np.zeros_like(cell_bodies), + reversed_bodies, + minimum_response=100.0, + crossing_topology=reversed_topology, ) np.testing.assert_array_equal( - reversed_repair, np.where(repaired > 0, 3 - repaired, 0), + reversed_repair, + np.where(repaired > 0, 3 - repaired, 0), ) foreign_owned = owned.copy() foreign_bodies = cell_bodies.copy() @@ -1091,8 +1155,12 @@ def test_short_two_junction_crossing_resolves_opposite_rooted_traces(pixel_size_ foreign_response = response.copy() foreign_response[foreign_rows, foreign_columns] = 150.0 foreign_repair = _repair_signal_supported_skeleton( - foreign_owned, _declared_shaft_admission(foreign_response), np.zeros_like(cell_bodies), foreign_bodies, - minimum_response=100.0, crossing_topology=topology, + foreign_owned, + _declared_shaft_admission(foreign_response), + np.zeros_like(cell_bodies), + foreign_bodies, + minimum_response=100.0, + crossing_topology=topology, ) assert np.all(foreign_repair[core] == 99) foreign_trace = (foreign_owned == 99) & (foreign_bodies == 0) @@ -1100,10 +1168,16 @@ def test_short_two_junction_crossing_resolves_opposite_rooted_traces(pixel_size_ for path_index in topology.crossing_paths: coordinates = topology.path_coordinates[path_index] outside = ~core[tuple(coordinates.T)] & (cell_bodies[tuple(coordinates.T)] == 0) - assert np.all(repaired[tuple(coordinates[outside].T)] == topology.path_owners[path_index]) + assert np.all( + repaired[tuple(coordinates[outside].T)] == topology.path_owners[path_index] + ) repaired[cell_bodies > 0] = 0 # The profile excludes soma pixels from traces. final = _analyze_owned_topology( - repaired, cell_bodies, pixel_size_um, 8.0, crossing_topology=topology, + repaired, + cell_bodies, + pixel_size_um, + 8.0, + crossing_topology=topology, branch_support=response > 0, ) final_terminals = { @@ -1113,7 +1187,8 @@ def test_short_two_junction_crossing_resolves_opposite_rooted_traces(pixel_size_ if tuple(coordinate) in terminals and cell_bodies[tuple(coordinate)] == 0 } assert final_terminals == { - coordinate: label for coordinate, label in terminal_owners.items() + coordinate: label + for coordinate, label in terminal_owners.items() if cell_bodies[coordinate] == 0 } assert final.root_paths_by_cell.keys() == {1, 2} @@ -1121,13 +1196,20 @@ def test_short_two_junction_crossing_resolves_opposite_rooted_traces(pixel_size_ for path_index in topology.crossing_paths: coordinates = topology.path_coordinates[path_index] outside = ~core[tuple(coordinates.T)] & (cell_bodies[tuple(coordinates.T)] == 0) - assert np.all(final_rendered[tuple(coordinates[outside].T)] == topology.path_owners[path_index]) + assert np.all( + final_rendered[tuple(coordinates[outside].T)] + == topology.path_owners[path_index] + ) unsupported_response = response.copy() unsupported_response[core] = 0 unsupported = _repair_signal_supported_skeleton( - owned, _declared_shaft_admission(unsupported_response), np.zeros_like(cell_bodies), cell_bodies, - minimum_response=100.0, crossing_topology=topology, + owned, + _declared_shaft_admission(unsupported_response), + np.zeros_like(cell_bodies), + cell_bodies, + minimum_response=100.0, + crossing_topology=topology, ) unsupported[cell_bodies > 0] = 0 # Merely declaring a crossing does not license a below-threshold bridge. @@ -1191,20 +1273,31 @@ def test_branch_events_require_three_paths_of_the_same_final_owner( def test_branch_qualification_rejects_medial_stars_inside_a_single_cap(): center = np.array([20, 20]) - paths = tuple(np.array([center, center + offset]) for offset in ( - [0, 1], [-1, -1], [1, -1], - )) + paths = tuple( + np.array([center, center + offset]) + for offset in ( + [0, 1], + [-1, -1], + [1, -1], + ) + ) foreground = np.zeros((41, 41), dtype=bool) rows, columns = disk(tuple(center), 3, shape=foreground.shape) foreground[rows, columns] = True topology = replace( - _empty_topology(), path_owners=np.ones(3, dtype=np.int32), - path_coordinates=paths, path_endpoint_groups=((0, 1), (0, 2), (0, 3)), - endpoint_group_coordinates={0: tuple(center), **{i + 1: tuple(path[-1]) for i, path in enumerate(paths)}}, + _empty_topology(), + path_owners=np.ones(3, dtype=np.int32), + path_coordinates=paths, + path_endpoint_groups=((0, 1), (0, 2), (0, 3)), + endpoint_group_coordinates={ + 0: tuple(center), + **{i + 1: tuple(path[-1]) for i, path in enumerate(paths)}, + }, ) branch_types, branches = topology.classify_owned_endpoints( branch_support=foreground, - cell_body_labels=np.zeros(foreground.shape, dtype=np.int32), merge_radius=3, + cell_body_labels=np.zeros(foreground.shape, dtype=np.int32), + merge_radius=3, outgrowth_width_px=3, ) assert branches == {} @@ -1226,16 +1319,25 @@ def test_branch_qualification_keeps_a_short_real_soma_attachment(body_owner): bodies = np.zeros(foreground.shape, dtype=np.int32) bodies[22:27, 19:22] = body_owner topology = replace( - _empty_topology(), path_owners=np.ones(3, dtype=np.int32), - path_coordinates=paths, path_endpoint_groups=((0, 1), (0, 2), (0, 3)), - endpoint_group_coordinates={0: (20, 20), **{i + 1: tuple(path[-1]) for i, path in enumerate(paths)}}, + _empty_topology(), + path_owners=np.ones(3, dtype=np.int32), + path_coordinates=paths, + path_endpoint_groups=((0, 1), (0, 2), (0, 3)), + endpoint_group_coordinates={ + 0: (20, 20), + **{i + 1: tuple(path[-1]) for i, path in enumerate(paths)}, + }, ) branch_types, branches = topology.classify_owned_endpoints( - branch_support=foreground, cell_body_labels=bodies, merge_radius=3, + branch_support=foreground, + cell_body_labels=bodies, + merge_radius=3, outgrowth_width_px=3, ) assert branches == ({1: (0,)} if body_owner == 1 else {}) - np.testing.assert_array_equal(branch_types, [1, 1, 1] if body_owner == 1 else [0, 0, 0]) + np.testing.assert_array_equal( + branch_types, [1, 1, 1] if body_owner == 1 else [0, 0, 0] + ) def test_branch_qualification_resolves_a_short_daughter_beyond_a_thick_junction(): @@ -1252,14 +1354,17 @@ def test_branch_qualification_resolves_a_short_daughter_beyond_a_thick_junction( foreground[11, 8:11] = True foreground[12:14, 2:6] = True topology = replace( - _empty_topology(), path_owners=np.ones(3, dtype=np.int32), - path_coordinates=paths, path_endpoint_groups=((1, 0), (0, 2), (0, 3)), + _empty_topology(), + path_owners=np.ones(3, dtype=np.int32), + path_coordinates=paths, + path_endpoint_groups=((1, 0), (0, 2), (0, 3)), endpoint_group_coordinates={0: (8, 6), 1: (4, 6), 2: (11, 10), 3: (13, 3)}, ) branch_types, branches = topology.classify_owned_endpoints( branch_support=foreground, cell_body_labels=np.zeros(foreground.shape, dtype=np.int32), - merge_radius=2.95, outgrowth_width_px=2.95, + merge_radius=2.95, + outgrowth_width_px=2.95, ) assert branches == {1: (0,)} np.testing.assert_array_equal(branch_types, [1, 1, 1]) @@ -1600,7 +1705,8 @@ def test_explicit_body_nuclear_and_neurite_channels_are_aligned(): assert np.asarray(topology_dropped_trace).shape == (1, *image.shape[1:]) assert np.asarray(topology_added_trace).shape == (1, *image.shape[1:]) assert len(admission_planes) == len( - NeuriteAdmissionPlanes.artifact_specs() + NeuriteOwnershipPlanes.artifact_specs() + NeuriteAdmissionPlanes.artifact_specs() + + NeuriteOwnershipPlanes.artifact_specs() ) for plane in admission_planes: assert plane.source_indices == (2,) @@ -2093,16 +2199,17 @@ def reject_hidden_secondary_threshold(*args, **kwargs): assert neurites[1, 80, 37] > 0 for owner in (1, 2): components, count = ndi.label( - cell_bodies[1] == owner, structure=np.ones((3, 3), dtype=bool), + cell_bodies[1] == owner, + structure=np.ones((3, 3), dtype=bool), ) assert count == 1 assert np.any((components > 0) & (nuclei[0] == owner)) lengths = sorted(row["total_outgrowth"] for row in cell_rows) assert lengths[0] == 0.0 assert lengths[1] > 89.0 - assert lengths[1] == pytest.approx(sum( - edge.feature_mapping()["branch_distance"] for edge in result[-1].edges - )) + assert lengths[1] == pytest.approx( + sum(edge.feature_mapping()["branch_distance"] for edge in result[-1].edges) + ) zero_growth_cell = next( row["cell"] for row in cell_rows if row["total_outgrowth"] == 0.0 ) @@ -2145,7 +2252,8 @@ def test_provisional_owner_cannot_reach_a_shaft_through_unadmitted_background(): support = np.zeros(response.shape, dtype=bool) support[16, 3:29] = True owner_regions = _propagate_neurite_owner_regions( - _declared_shaft_admission(response, support), bodies, + _declared_shaft_admission(response, support), + bodies, minimum_response=100.0, ) assert np.all(owner_regions[16, 3:29] == 1) @@ -2387,7 +2495,9 @@ def test_signal_supported_repair_follows_curved_trace_instead_of_chord(): @pytest.mark.parametrize("attachment_response, retained", [(150.0, True), (0.0, False)]) -def test_signal_supported_repair_qualifies_the_short_soma_transition(attachment_response, retained): +def test_signal_supported_repair_qualifies_the_short_soma_transition( + attachment_response, retained +): labels = np.zeros((32, 48), dtype=np.int32) bodies = np.zeros_like(labels) bodies[13:20, 3:10] = 1 @@ -2397,14 +2507,22 @@ def test_signal_supported_repair_qualifies_the_short_soma_transition(attachment_ admission = _declared_shaft_admission(response) soma_response = response.copy() soma_response[16, 10:13] = attachment_response - admission = replace(admission, planes=replace( - admission.planes, soma_attachment_response=soma_response, - )) + admission = replace( + admission, + planes=replace( + admission.planes, + soma_attachment_response=soma_response, + ), + ) regions = np.where(response > 0, 1, 0).astype(np.int32) repaired = _repair_signal_supported_skeleton( - labels, admission, regions, bodies, - minimum_response=100.0, crossing_topology=_empty_topology(), + labels, + admission, + regions, + bodies, + minimum_response=100.0, + crossing_topology=_empty_topology(), ) assert bool(np.all(repaired[16, 13:35] == 1)) == retained @@ -2422,8 +2540,12 @@ def test_signal_supported_repair_cannot_use_old_ownership_to_cross_a_foreign_som regions = np.where(response > 0, 1, 0).astype(np.int32) repaired = _repair_signal_supported_skeleton( - labels, _declared_shaft_admission(response), regions, bodies, - minimum_response=100.0, crossing_topology=_empty_topology(), + labels, + _declared_shaft_admission(response), + regions, + bodies, + minimum_response=100.0, + crossing_topology=_empty_topology(), ) assert np.all(repaired[16, 10:20] == 1) @@ -2442,10 +2564,16 @@ def test_process_attachment_groups_daughters_but_preserves_independent_exits(): # All three are ownership evidence, but only the proximal stem starts a # process. Final same-owner transitions determine biological grouping. assert _TopologyResult.soma_ownership_seeds(bodies, 2.0, paths) == { - 0: (1,), 1: (1,), 2: (1,), + 0: (1,), + 1: (1,), + 2: (1,), } assert _TopologyResult.soma_process_roots( - bodies, 2.0, paths, transitions, np.ones(3, dtype=np.int32), + bodies, + 2.0, + paths, + transitions, + np.ones(3, dtype=np.int32), ) == {1: (0,)} exits = ( @@ -2454,7 +2582,10 @@ def test_process_attachment_groups_daughters_but_preserves_independent_exits(): np.array([[12, 16], [18, 16]]), ) assert _TopologyResult.soma_process_roots( - bodies, 2.0, exits, {0: (2,), 1: (2,), 2: (0, 1)}, + bodies, + 2.0, + exits, + {0: (2,), 1: (2,), 2: (0, 1)}, np.ones(3, dtype=np.int32), ) == {1: (0, 1)} @@ -2612,15 +2743,17 @@ def repaired_topology(*, connected: bool): assert np.all(connected_topology.path_owners == 1) -def test_signal_supported_repair_projects_crossings_regionally_on_eight_megapixels(monkeypatch): +def test_signal_supported_repair_projects_crossings_regionally_on_eight_megapixels( + monkeypatch, +): labels = np.zeros((2000, 4000), dtype=np.int32) bodies = np.zeros_like(labels) response = np.zeros(labels.shape, dtype=np.float32) for index in range(1000): y, x = 5 + 40 * (index // 20), 5 + 200 * (index % 20) bodies[y, x] = index + 1 - labels[y, x + 1:x + 4] = index + 1 - response[y, x:x + 4] = 5 + labels[y, x + 1 : x + 4] = index + 1 + response[y, x : x + 4] = 5 projected_shapes = [] original_projection = _TopologyResult.crossing_core_mask @@ -2628,9 +2761,13 @@ def regional_projection(self, shape, **kwargs): projected_shapes.append(shape) return original_projection(self, shape, **kwargs) - monkeypatch.setattr(_TopologyResult, 'crossing_core_mask', regional_projection) + monkeypatch.setattr(_TopologyResult, "crossing_core_mask", regional_projection) repaired = _repair_signal_supported_skeleton( - labels, _declared_shaft_admission(response), labels, bodies, minimum_response=3, + labels, + _declared_shaft_admission(response), + labels, + bodies, + minimum_response=3, crossing_topology=_empty_topology(), ) np.testing.assert_array_equal(repaired, np.where(bodies > 0, bodies, labels)) @@ -2649,14 +2786,21 @@ def test_signal_supported_repair_reserves_core_from_unrelated_earlier_owner(): response[20, 20] = 150.0 regions = np.where(response > 0, 1, 0) topology = replace( - _empty_topology(), path_owners=np.array([2, 2, 0]), - path_coordinates=(np.array([[20, x] for x in range(5, 20)]), - np.array([[20, x] for x in range(21, 36)]), - np.array([[20, 19], [20, 20], [20, 21]])), + _empty_topology(), + path_owners=np.array([2, 2, 0]), + path_coordinates=( + np.array([[20, x] for x in range(5, 20)]), + np.array([[20, x] for x in range(21, 36)]), + np.array([[20, 19], [20, 20], [20, 21]]), + ), resolved_crossings=(_ResolvedCrossing(0, (0, 1), (2,)),), ) repaired = _repair_signal_supported_skeleton( - labels, _declared_shaft_admission(response), regions, bodies, minimum_response=100.0, + labels, + _declared_shaft_admission(response), + regions, + bodies, + minimum_response=100.0, crossing_topology=topology, ) assert np.all(repaired[20, 8:20] == 2) @@ -2666,15 +2810,20 @@ def test_signal_supported_repair_reserves_core_from_unrelated_earlier_owner(): reversed_topology = replace(topology, path_owners=np.array([1, 1, 0])) reverse = lambda image: np.where(image > 0, 3 - image, 0) reversed_repair = _repair_signal_supported_skeleton( - reverse(labels), _declared_shaft_admission(response), reverse(regions), reverse(bodies), - minimum_response=100.0, crossing_topology=reversed_topology, + reverse(labels), + _declared_shaft_admission(response), + reverse(regions), + reverse(bodies), + minimum_response=100.0, + crossing_topology=reversed_topology, ) np.testing.assert_array_equal(reversed_repair, reverse(repaired)) def test_crossing_resolution_keeps_branch_arms_in_shared_support_declaration(): topology = replace( - _empty_topology(), path_owners=np.array([1, 1, 2, 2, 2, 0]), + _empty_topology(), + path_owners=np.array([1, 1, 2, 2, 2, 0]), path_coordinates=tuple(np.array([[4, 5]]) for _ in range(6)), resolved_crossings=(_ResolvedCrossing(0, (0, 1, 2, 3, 4), (5,)),), ) @@ -2834,7 +2983,10 @@ def test_morphology_retains_topology_owners_when_shared_endpoints_collide(): assert rendered[tuple(paths[1].T)].tolist() == [1, 2, 1] graph = _build_neurite_morphology_graph( - topology, bodies, coordinate_spacing=SourceVoxelSpacing((1.0, 1.0)), outgrowth_width_px=1.0 + topology, + bodies, + coordinate_spacing=SourceVoxelSpacing((1.0, 1.0)), + outgrowth_width_px=1.0, ) graph.require_directed_forest() assert len(graph.edges) == 2 diff --git a/tests/unit/test_neuroncyto_ii_neurite_demo.py b/tests/unit/test_neuroncyto_ii_neurite_demo.py index a93141116e..ccb710f768 100644 --- a/tests/unit/test_neuroncyto_ii_neurite_demo.py +++ b/tests/unit/test_neuroncyto_ii_neurite_demo.py @@ -150,10 +150,14 @@ def test_neuroncyto_demo_declares_exact_crossover_channel_semantics( "neurite_secondary_ownership": ArtifactViewerStreaming.ON_DEMAND, "neurite_topology_dropped_trace": ArtifactViewerStreaming.ON_DEMAND, "neurite_topology_added_trace": ArtifactViewerStreaming.ON_DEMAND, - **{spec.name: ArtifactViewerStreaming.ON_DEMAND - for spec in NeuriteAdmissionPlanes.artifact_specs()}, - **{spec.name: ArtifactViewerStreaming.ON_DEMAND - for spec in NeuriteOwnershipPlanes.artifact_specs()}, + **{ + spec.name: ArtifactViewerStreaming.ON_DEMAND + for spec in NeuriteAdmissionPlanes.artifact_specs() + }, + **{ + spec.name: ArtifactViewerStreaming.ON_DEMAND + for spec in NeuriteOwnershipPlanes.artifact_specs() + }, "neurite_morphology": ArtifactViewerStreaming.AUTOMATIC, } @@ -208,10 +212,14 @@ def test_neuroncyto_demo_compiles_exact_loose_tiff_pair(tmp_path: Path) -> None: "neurite_secondary_ownership": ArtifactViewerStreaming.ON_DEMAND, "neurite_topology_dropped_trace": ArtifactViewerStreaming.ON_DEMAND, "neurite_topology_added_trace": ArtifactViewerStreaming.ON_DEMAND, - **{spec.name: ArtifactViewerStreaming.ON_DEMAND - for spec in NeuriteAdmissionPlanes.artifact_specs()}, - **{spec.name: ArtifactViewerStreaming.ON_DEMAND - for spec in NeuriteOwnershipPlanes.artifact_specs()}, + **{ + spec.name: ArtifactViewerStreaming.ON_DEMAND + for spec in NeuriteAdmissionPlanes.artifact_specs() + }, + **{ + spec.name: ArtifactViewerStreaming.ON_DEMAND + for spec in NeuriteOwnershipPlanes.artifact_specs() + }, "neurite_morphology": ArtifactViewerStreaming.AUTOMATIC, } diff --git a/tests/unit/test_nlm_slice_contract.py b/tests/unit/test_nlm_slice_contract.py index 437e6c3082..bd4d0dd2e2 100644 --- a/tests/unit/test_nlm_slice_contract.py +++ b/tests/unit/test_nlm_slice_contract.py @@ -21,7 +21,9 @@ def _torch_nlm(): @pytest.mark.parametrize( ("func", "expected_default"), ( - pytest.param(lambda: jax_nlm_processor.non_local_means_denoise_jax, False, id="jax"), + pytest.param( + lambda: jax_nlm_processor.non_local_means_denoise_jax, False, id="jax" + ), pytest.param( lambda: _torch_nlm().non_local_means_denoise_torch, True, diff --git a/tests/unit/test_openhcs_adapter.py b/tests/unit/test_openhcs_adapter.py index c2f94a805c..56aabeab8d 100644 --- a/tests/unit/test_openhcs_adapter.py +++ b/tests/unit/test_openhcs_adapter.py @@ -389,13 +389,15 @@ def poll_status(self, execution_id): == FakeZMQExecutionClient.submitted[1].pipeline_code ) assert all( - submission.global_config_code == ConfigDocumentAuthority.render( + submission.global_config_code + == ConfigDocumentAuthority.render( global_config, expected_config_type=GlobalPipelineConfig ) for submission in FakeZMQExecutionClient.submitted ) assert [ - submission.compile_control.compile_only for submission in FakeZMQExecutionClient.submitted + submission.compile_control.compile_only + for submission in FakeZMQExecutionClient.submitted ] == [ True, False, diff --git a/tests/unit/test_openhcs_measured_run.py b/tests/unit/test_openhcs_measured_run.py index b5f47b386c..00a3464097 100644 --- a/tests/unit/test_openhcs_measured_run.py +++ b/tests/unit/test_openhcs_measured_run.py @@ -23,7 +23,10 @@ from openhcs.core.orchestrator.execution_result import ExecutionResult from openhcs.core.pipeline_document import PipelineDocumentCodec from openhcs.runtime.zmq_application import OPENHCS_ENDPOINT_APPLICATION -from openhcs.runtime.zmq_execution_client import OpenHCSExecutionSubmission, ZMQExecutionRequestBuilder +from openhcs.runtime.zmq_execution_client import ( + OpenHCSExecutionSubmission, + ZMQExecutionRequestBuilder, +) from openhcs.runtime.zmq_execution_observation import ZMQRuntimeExecutionOutcomeExport from openhcs.runtime.zmq_execution_signature import ( ZMQAuxiliaryExecutionParams, @@ -237,7 +240,10 @@ def submit_compile(self, submission): } def submit_pipeline(self, submission): - assert submission.compile_control.compile_artifact_id == f"compile-{self.run_number}" + assert ( + submission.compile_control.compile_artifact_id + == f"compile-{self.run_number}" + ) observer({"phase": "axis_started", "timestamp": 11.0}) return { "status": "accepted", diff --git a/tests/unit/test_orchestrator_lane_planning.py b/tests/unit/test_orchestrator_lane_planning.py index 927331ea22..e0bd2ab32f 100644 --- a/tests/unit/test_orchestrator_lane_planning.py +++ b/tests/unit/test_orchestrator_lane_planning.py @@ -1125,7 +1125,9 @@ def test_analysis_consolidation_propagates_runtime_failures(monkeypatch): "consolidate_runtime_analysis_table_output_groups", lambda **kwargs: ([], [("results", "invalid CSV")]), ) - hook = AnalysisConsolidationHook(AnalysisConsolidationConfig(), PlateMetadataConfig()) + hook = AnalysisConsolidationHook( + AnalysisConsolidationConfig(), PlateMetadataConfig() + ) with pytest.raises(RuntimeError, match="invalid CSV"): hook.run({"A01": context}, inputs) @@ -1381,9 +1383,7 @@ def force_stop(self): self.running = False progress_queue = SimpleNamespace( - put=lambda raw_event: events.append( - raw_event.phase.value - ) + put=lambda raw_event: events.append(raw_event.phase.value) ) results = _execute_with_visualizer( monkeypatch, diff --git a/tests/unit/test_path_planner_axis_source_scope.py b/tests/unit/test_path_planner_axis_source_scope.py index 88532d3d46..e9a598e0e1 100644 --- a/tests/unit/test_path_planner_axis_source_scope.py +++ b/tests/unit/test_path_planner_axis_source_scope.py @@ -46,7 +46,9 @@ def test_path_planner_scopes_source_metadata_to_compilation_axis() -> None: ).source_metadata_by_path.values() ), step_count=0, - pipeline=ResolvedPipelineDefinition(steps=(), step_scope_ids={}, step_provenance={}), + pipeline=ResolvedPipelineDefinition( + steps=(), step_scope_ids={}, step_provenance={} + ), ) planner = PathPlanner(session) diff --git a/tests/unit/test_path_planner_materialization.py b/tests/unit/test_path_planner_materialization.py index 10fe29a3b3..6a44d62239 100644 --- a/tests/unit/test_path_planner_materialization.py +++ b/tests/unit/test_path_planner_materialization.py @@ -196,25 +196,31 @@ def resolve(self, handler, dataset_root): def _prepare_step_declarations(planner, step, step_index): context = replace( - planner.artifact_context, step_name=step.name, step_index=step_index, + planner.artifact_context, + step_name=step.name, + step_index=step_index, ).with_source_binding_scope( source_bindings=step.source_bindings, group_by=PathPlannerExecutionGroups.normalized_group_by(step), input_source=step.processing_config.input_source, ) graph = extract_artifact_declarations( - step.func, invocation_contract_provider=planner.invocation_contract_provider, + step.func, + invocation_contract_provider=planner.invocation_contract_provider, step_context=context, ) contracts = tuple(item.contract for item in graph.pattern.iter_items()) - return graph, graph.pattern, FunctionStepExecutionScope.require_uniform(contracts), contracts + return ( + graph, + graph.pattern, + FunctionStepExecutionScope.require_uniform(contracts), + contracts, + ) def _compile_metadata_pattern(planner, pattern): snapshot = _resolved_step(func=pattern) - declarations, pattern, _, _ = _prepare_step_declarations(planner, - snapshot, 3 - ) + declarations, pattern, _, _ = _prepare_step_declarations(planner, snapshot, 3) input_plans = planner.artifacts.process_artifact_inputs( declarations, 3, @@ -372,8 +378,8 @@ def metadata_consumer(image, grid_dimensions): bindings=(NamedSourceBinding(alias="DNA"),), ), ) - declarations, pattern, _, contracts = _prepare_step_declarations(planner, - snapshot, 3 + declarations, pattern, _, contracts = _prepare_step_declarations( + planner, snapshot, 3 ) execution_bindings = CompiledSourceBindingPlan.from_contracts( snapshot.source_bindings, @@ -431,9 +437,13 @@ def metadata_consumer(image, grid_dimensions): "artifact_name, expected", [("pixel_size", 1.3556), ("engineering_exposure_duration", 17.25)], ) -def test_registered_metadata_provider_value_reaches_runtime_unchanged(artifact_name, expected): +def test_registered_metadata_provider_value_reaches_runtime_unchanged( + artifact_name, expected +): received = [] - spec = ArtifactSpec.input(artifact_name, SpecialArtifactType, parameter_name="calibration") + spec = ArtifactSpec.input( + artifact_name, SpecialArtifactType, parameter_name="calibration" + ) @numpy @artifact_inputs(spec) @@ -445,13 +455,18 @@ def metadata_consumer(image, *, calibration): planner.ctx.microscope_handler = _EngineeringMetadataHandler(filemanager=object()) planner.ctx.dataset_root = planner.dataset_root assert planner.ctx.microscope_handler.can_resolve_metadata_artifact(artifact_name) - compiled, input_plans = _compile_metadata_pattern(planner, (metadata_consumer, {"calibration": -7.0})) + compiled, input_plans = _compile_metadata_pattern( + planner, (metadata_consumer, {"calibration": -7.0}) + ) (invocation,) = compiled.default_group.invocations (edge,) = invocation.artifact_input_edges assert isinstance(edge, CompiledMetadataArtifactInputEdgePlan) assert input_plans == {} assert invocation.kwargs_dict == {"calibration": expected} - assert all(binding.parameter_name != "calibration" for binding in invocation.runtime_parameter_bindings) + assert all( + binding.parameter_name != "calibration" + for binding in invocation.runtime_parameter_bindings + ) _execute_compiled_metadata_pattern(compiled) assert received == [expected] @@ -476,7 +491,8 @@ def resolve(self, handler, dataset_root): return value class CheckedExposureMetadataProvider( - ResolutionReceipt, PositiveMetadataValue, + ResolutionReceipt, + PositiveMetadataValue, _EngineeringExposureMetadataArtifactProvider, ): artifact_name = "engineering_checked_exposure_duration" @@ -544,7 +560,9 @@ def metadata_consumer(image, pixel_size): planner.ctx.dataset_root = planner.dataset_root compiled, _ = _compile_metadata_pattern(planner, metadata_consumer) if value is None: - with pytest.raises(ValueError, match="Required metadata artifact .* has no value"): + with pytest.raises( + ValueError, match="Required metadata artifact .* has no value" + ): _execute_compiled_metadata_pattern(compiled) else: _execute_compiled_metadata_pattern(compiled) @@ -558,7 +576,9 @@ def metadata_consumer(image, unknown_metadata): planner = _artifact_planner_stub() with pytest.raises(MissingArtifactInputError): - _compile_metadata_pattern(planner, (metadata_consumer, {"unknown_metadata": 1.3556})) + _compile_metadata_pattern( + planner, (metadata_consumer, {"unknown_metadata": 1.3556}) + ) def test_exact_producer_takes_precedence_over_metadata_provider(): @@ -571,11 +591,16 @@ def metadata_consumer(image, pixel_size): planner = _artifact_planner_stub() planner.ctx.microscope_handler = _EngineeringMetadataHandler(filemanager=object()) planner.ctx.dataset_root = planner.dataset_root - producer = _record_declared_output(planner, ArtifactOutputPlan( - name="pixel_size", path="/memory/pixel_size.pkl", - artifact_type=SpecialArtifactType, producer_step_index=2, - producer_step_scope_id="plate::functionstep_2", - )) + producer = _record_declared_output( + planner, + ArtifactOutputPlan( + name="pixel_size", + path="/memory/pixel_size.pkl", + artifact_type=SpecialArtifactType, + producer_step_index=2, + producer_step_scope_id="plate::functionstep_2", + ), + ) compiled, input_plans = _compile_metadata_pattern(planner, metadata_consumer) (invocation,) = compiled.default_group.invocations (edge,) = invocation.artifact_input_edges @@ -789,12 +814,13 @@ def __call__(self, invocation, step_context): match="Conflicting compiled invocation output artifact sidecar role", ): compile_function_pattern( - pattern, {}, output_plans, + pattern, + {}, + output_plans, invocation_contract_provider=planner.invocation_contract_provider, ).coalesced_artifact_output_specs() - def test_materialization_collision_updates_results_dir_and_config(): planner = PathPlanner.__new__(PathPlanner) planner.dataset_root = Path("/data/plate1") @@ -1065,9 +1091,7 @@ def test_artifact_output_plan_only_preserves_explicit_source_stack_scope(): ) assert outputs[output_specs[0].ref()].variable_components == () - assert outputs[output_specs[1].ref()].variable_components == ( - Microscopy.Channel, - ) + assert outputs[output_specs[1].ref()].variable_components == (Microscopy.Channel,) def test_artifact_output_source_lookup_combines_repeated_main_flow_inputs(): @@ -1113,7 +1137,8 @@ def test_artifact_output_source_lookup_combines_repeated_main_flow_inputs(): @pytest.mark.parametrize("stored_main_flow", [False, True]) @pytest.mark.parametrize("stored_secondary", [False, True]) def test_compiled_source_edges_only_consume_relation_owned_main_flow( - stored_main_flow, stored_secondary, + stored_main_flow, + stored_secondary, ): source_specs = ( ArtifactSpec.input("DNA", ImageArtifactType), @@ -1163,7 +1188,11 @@ def combine_sources(image, secondary=None): edges = next(compiled.iter_invocations()).artifact_input_edges assert tuple(edge.spec for edge in edges) == source_specs - assert tuple((edge.main_flow_projection is not None) for edge in edges) == (True, False, False) + assert tuple((edge.main_flow_projection is not None) for edge in edges) == ( + True, + False, + False, + ) assert edges[0].storage_plan is None assert edges[0].projection is None assert edges[1].storage_plan is stored_inputs.get(source_specs[1].ref()) @@ -1249,10 +1278,10 @@ def test_implicit_native_main_flow_provenance_drives_artifact_owned_scope(): ) native_pattern = compile_function_pattern(percentile_normalize, {}, {}) native_invocation = next(native_pattern.iter_invocations()) - planner.artifact_context = ( - extract_artifact_declarations(percentile_normalize).advance_declaration_context( - planner.artifact_context, - ) + planner.artifact_context = extract_artifact_declarations( + percentile_normalize + ).advance_declaration_context( + planner.artifact_context, ) cursor_name = unnamed_main_flow_artifact_name(0, native_invocation.key) cursor = ArtifactSpec.input(cursor_name, ImageArtifactType) @@ -1498,9 +1527,7 @@ def test_artifact_output_source_lookup_ignores_shared_input_broadcast_projection execution_scope=FunctionStepExecutionScope.AXIS, ) - assert projected[ambiguous_output.ref()].variable_components == ( - Microscopy.Site, - ) + assert projected[ambiguous_output.ref()].variable_components == (Microscopy.Site,) def test_enabled_source_bindings_preserve_previous_step_main_flow(): @@ -1626,9 +1653,7 @@ def identify(main_image, Stain1): assert maps.group_scope.keys == expected_channel_scope.keys assert maps.group_scope.component is expected_channel_scope.component assert maps.outputs[object_output.ref()].group_keys == ("1",) - assert maps.outputs[object_output.ref()].variable_components == ( - Microscopy.Site, - ) + assert maps.outputs[object_output.ref()].variable_components == (Microscopy.Site,) edge = next(compiled.iter_invocations()).artifact_input_edges[0] assert edge.projection.invocation_scope == expected_channel_scope assert edge.projection.component_scope(Microscopy.Channel) == ( @@ -1866,11 +1891,17 @@ def identify(image): scopes = planner.artifacts.output_groups_from_declared_relations( namespaced, - group_scope=PathPlannerGroupScope.from_raw(("1", "2"), component=Microscopy.Channel), - relation_source_scopes={}, consumer_variable_components=ComponentSet(), - step_index=0, step_name="identify", + group_scope=PathPlannerGroupScope.from_raw( + ("1", "2"), component=Microscopy.Channel + ), + relation_source_scopes={}, + consumer_variable_components=ComponentSet(), + step_index=0, + step_name="identify", ) - assert scopes[ArtifactSpec.output("nuclei", ObjectLabelsArtifactType).ref()].keys == ("1", "2") + assert scopes[ + ArtifactSpec.output("nuclei", ObjectLabelsArtifactType).ref() + ].keys == ("1", "2") def test_artifact_graph_preserves_same_name_outputs_of_different_types(): @@ -1919,9 +1950,13 @@ def declared_artifact_owner(image): assert declarations.output_groups[output_ref] == {None} scopes = planner.artifacts.output_groups_from_declared_relations( namespaced, - group_scope=PathPlannerGroupScope.from_raw(("1", "2"), component=Microscopy.Channel), - relation_source_scopes={}, consumer_variable_components=ComponentSet(), - step_index=0, step_name="correct_illumination", + group_scope=PathPlannerGroupScope.from_raw( + ("1", "2"), component=Microscopy.Channel + ), + relation_source_scopes={}, + consumer_variable_components=ComponentSet(), + step_index=0, + step_name="correct_illumination", ) assert scopes[output_ref].keys == ("1", "2") @@ -2251,7 +2286,11 @@ def filter_objects(main_image, MeasureObjectIntensity_3_measurements): consumer_maps.outputs, consumer_maps.relation_source_scopes, consumer_maps.group_scope, - declarations=extract_artifact_declarations(filter_objects, invocation_contract_provider=planner.invocation_contract_provider, step_context=planner.artifact_context), + declarations=extract_artifact_declarations( + filter_objects, + invocation_contract_provider=planner.invocation_contract_provider, + step_context=planner.artifact_context, + ), ) edge = next(compiled.iter_invocations()).artifact_input_edges[0] @@ -3403,8 +3442,10 @@ def declared_artifact_owner(image): ) pipeline = ResolvedPipelineDefinition( - steps=(snapshot,), step_scope_ids={0: "plate::identify_cells"}, - step_provenance={0: {}}, declaration_provider=declarations_for_invocation, + steps=(snapshot,), + step_scope_ids={0: "plate::identify_cells"}, + step_provenance={0: {}}, + declaration_provider=declarations_for_invocation, ) (declarations,) = pipeline.artifact_graphs func_pattern = declarations.pattern @@ -3470,11 +3511,10 @@ def filter_objects(image, *, runtime): } ) - _declarations, _pattern, execution_scope, _contracts = ( - _prepare_step_declarations(planner, - snapshot, - 2, - ) + _declarations, _pattern, execution_scope, _contracts = _prepare_step_declarations( + planner, + snapshot, + 2, ) execution_groups = planner.execution_groups.get_execution_groups( snapshot, @@ -3491,11 +3531,10 @@ def filter_objects(image, *, runtime): "1": [filter_objects], "2": [filter_objects], } - _declarations, _pattern, execution_scope, _contracts = ( - _prepare_step_declarations(planner, - snapshot, - 2, - ) + _declarations, _pattern, execution_scope, _contracts = _prepare_step_declarations( + planner, + snapshot, + 2, ) execution_groups = planner.execution_groups.get_execution_groups( snapshot, @@ -4949,7 +4988,11 @@ def consume(main_image, image): maps.outputs, maps.relation_source_scopes, maps.group_scope, - declarations=extract_artifact_declarations(consume, invocation_contract_provider=planner.invocation_contract_provider, step_context=planner.artifact_context), + declarations=extract_artifact_declarations( + consume, + invocation_contract_provider=planner.invocation_contract_provider, + step_context=planner.artifact_context, + ), ) @@ -5011,7 +5054,11 @@ def consume(main_image, image): maps.outputs, maps.relation_source_scopes, maps.group_scope, - declarations=extract_artifact_declarations(consume, invocation_contract_provider=planner.invocation_contract_provider, step_context=planner.artifact_context), + declarations=extract_artifact_declarations( + consume, + invocation_contract_provider=planner.invocation_contract_provider, + step_context=planner.artifact_context, + ), ) edge = next(compiled.iter_invocations()).artifact_input_edges[0] @@ -5116,9 +5163,7 @@ def produce(image, source_value): PathPlannerComponentScopes.empty(), source_bindings=source_bindings, ) - declarations = ( - extract_artifact_declarations(produce) - ) + declarations = extract_artifact_declarations(produce) maps = planner.artifacts.compile_plan_maps( snapshot, step_index, diff --git a/tests/unit/test_payload_axis_compile_gate.py b/tests/unit/test_payload_axis_compile_gate.py index ea2ba16dc5..4c443afddb 100644 --- a/tests/unit/test_payload_axis_compile_gate.py +++ b/tests/unit/test_payload_axis_compile_gate.py @@ -119,7 +119,15 @@ def _session( ) projections = { path.name: SourcePlaneProjection( - address=OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, index), (Microscopy.Channel, 1), (Microscopy.ZIndex, 1), (Microscopy.Timepoint, 1))), + address=OpenHCSPlaneAddress( + ( + (Microscopy.Well, "A01"), + (Microscopy.Site, index), + (Microscopy.Channel, 1), + (Microscopy.ZIndex, 1), + (Microscopy.Timepoint, 1), + ) + ), ref=SourcePixelRef("disk", str(path)), source_alias=binding.alias, ) @@ -202,9 +210,7 @@ def test_all_rgb_sources_prove_color_to_gray_payload_axis_requirement( photometric="rgb", ) - PipelineCompiler.validate_payload_axis_requirements( - _session(tmp_path, paths) - ) + PipelineCompiler.validate_payload_axis_requirements(_session(tmp_path, paths)) def test_grouped_color_requirement_inspects_only_compatible_binding( @@ -248,7 +254,15 @@ def identity(image): ) projections = { path.name: SourcePlaneProjection( - address=OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, index), (Microscopy.Channel, binding.component_identity[0].value), (Microscopy.ZIndex, 1), (Microscopy.Timepoint, 1))), + address=OpenHCSPlaneAddress( + ( + (Microscopy.Well, "A01"), + (Microscopy.Site, index), + (Microscopy.Channel, binding.component_identity[0].value), + (Microscopy.ZIndex, 1), + (Microscopy.Timepoint, 1), + ) + ), ref=SourcePixelRef("disk", str(path)), source_alias=binding.alias, ) @@ -339,9 +353,7 @@ def test_unknown_or_unreadable_exact_source_fails_compile_closed( path.write_bytes(b"not an image header") with pytest.raises(ValueError, match="failed before execution") as error: - PipelineCompiler.validate_payload_axis_requirements( - _session(tmp_path, (path,)) - ) + PipelineCompiler.validate_payload_axis_requirements(_session(tmp_path, (path,))) assert str(path) in str(error.value) or str(path.name) in str(error.value) assert "ColourAxis" in str(error.value) @@ -613,12 +625,28 @@ def test_unknown_routed_group_cannot_fall_back_to_all_primary_sources( ) projections = { grayscale_path.name: SourcePlaneProjection( - address=OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, 1), (Microscopy.Channel, 1), (Microscopy.ZIndex, 1), (Microscopy.Timepoint, 1))), + address=OpenHCSPlaneAddress( + ( + (Microscopy.Well, "A01"), + (Microscopy.Site, 1), + (Microscopy.Channel, 1), + (Microscopy.ZIndex, 1), + (Microscopy.Timepoint, 1), + ) + ), ref=SourcePixelRef("disk", str(grayscale_path)), source_alias=grayscale_binding.alias, ), rgb_path.name: SourcePlaneProjection( - address=OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, 2), (Microscopy.Channel, 2), (Microscopy.ZIndex, 1), (Microscopy.Timepoint, 1))), + address=OpenHCSPlaneAddress( + ( + (Microscopy.Well, "A01"), + (Microscopy.Site, 2), + (Microscopy.Channel, 2), + (Microscopy.ZIndex, 1), + (Microscopy.Timepoint, 1), + ) + ), ref=SourcePixelRef("disk", str(rgb_path)), source_alias=rgb_binding.alias, ), diff --git a/tests/unit/test_persisted_image_metadata.py b/tests/unit/test_persisted_image_metadata.py index 125cadcbf9..b496b4904b 100644 --- a/tests/unit/test_persisted_image_metadata.py +++ b/tests/unit/test_persisted_image_metadata.py @@ -163,8 +163,11 @@ def test_saved_two_channel_planes_keep_runtime_axis_not_contributor_axis( def receive_projection_entries(writer, metadata_path, subdirectory, entries): target = RuntimeArtifactMetadataTarget( output_dir=Path(metadata_path).parent / subdirectory, - dataset_root=str(Path(metadata_path).parent), sub_dir=subdirectory, - backend="disk", results_dir=None, create_openhcs_metadata=False, + dataset_root=str(Path(metadata_path).parent), + sub_dir=subdirectory, + backend="disk", + results_dir=None, + create_openhcs_metadata=False, ) writer.reconcile_completed_dataset( metadata_path, {}, produced_entries_by_target={target: entries} @@ -179,11 +182,20 @@ def test_atomic_perwell_projection_merge_preserves_all_records(tmp_path): def merge(well): virtual_path = f"{well}.tif" projection = SourcePlaneProjection( - OpenHCSPlaneAddress(((Microscopy.Well, well), (Microscopy.Site, 1), (Microscopy.Channel, 1), (Microscopy.ZIndex, 1), (Microscopy.Timepoint, 1))), + OpenHCSPlaneAddress( + ( + (Microscopy.Well, well), + (Microscopy.Site, 1), + (Microscopy.Channel, 1), + (Microscopy.ZIndex, 1), + (Microscopy.Timepoint, 1), + ) + ), SourcePixelRef("disk", virtual_path), image_metadata=metadata_fixture(), ) - receive_projection_entries(writer, + receive_projection_entries( + writer, path, ".", VirtualWorkspaceSourceProjectionEntries.from_projection_paths( @@ -205,7 +217,15 @@ def calibrated_projection(well, path, pixel_size): source_metadata = {} spacing.merge_into(source_metadata, path=path) return SourcePlaneProjection( - OpenHCSPlaneAddress(((Microscopy.Well, well), (Microscopy.Site, 1), (Microscopy.Channel, 1), (Microscopy.ZIndex, 1), (Microscopy.Timepoint, 1))), + OpenHCSPlaneAddress( + ( + (Microscopy.Well, well), + (Microscopy.Site, 1), + (Microscopy.Channel, 1), + (Microscopy.ZIndex, 1), + (Microscopy.Timepoint, 1), + ) + ), SourcePixelRef("disk", path), source_metadata=source_metadata, image_metadata=metadata_fixture().replace_fields(source_voxel_spacing=spacing), @@ -247,8 +267,10 @@ def test_atomic_projection_merge_refreshes_geometry_from_current_document( writer.merge_subdirectory_metadata(path, {"images": second_fields}) expected_pixel_size = 2.0 else: - receive_projection_entries(writer, - path, "images", + receive_projection_entries( + writer, + path, + "images", VirtualWorkspaceSourceProjectionEntries.from_projection_paths( ((second, "images/second.tif"),) ), @@ -270,9 +292,11 @@ def test_final_publication_prunes_geometry_and_preserves_other_directories(tmp_p (deleted, "images/deleted.tif"), (elsewhere, "other/retained.tif"), ) - receive_projection_entries(writer, - path, "images", - VirtualWorkspaceSourceProjectionEntries.from_projection_paths(projection_paths) + receive_projection_entries( + writer, + path, + "images", + VirtualWorkspaceSourceProjectionEntries.from_projection_paths(projection_paths), ) before = json.loads(path.read_text())[FIELDS.SUBDIRECTORIES]["images"] assert before[FIELDS.PIXEL_SIZE] == 1.0 @@ -389,7 +413,8 @@ def test_invalid_unreplaced_projection_aborts_typed_transaction(tmp_path, publis path = tmp_path / "openhcs_metadata.json" writer = AtomicMetadataWriter() writer.replace_subdirectory_metadata( - path, "images", + path, + "images", {FIELDS.SOURCE_PROJECTION: [{"virtual_path": "images/broken.tif"}]}, ) replacement = calibrated_projection("A02", "images/new.tif", 0.5) @@ -438,9 +463,10 @@ def test_typed_transaction_retains_wire_fields_and_derives_workspace_views( assert "legacy.tif" not in subdir[FIELDS.WORKSPACE_MAPPING] assert "legacy.tif" not in subdir[FIELDS.SOURCE_METADATA] assert subdir[FIELDS.SOURCE_METADATA]["images/original.tif"]["well"] == "A01" - assert "manual_override" not in subdir[FIELDS.SOURCE_METADATA][ - "images/original.tif" - ] + assert ( + "manual_override" + not in subdir[FIELDS.SOURCE_METADATA]["images/original.tif"] + ) assert "well" not in subdir[FIELDS.SOURCE_PROJECTION][0]["source_metadata"] publish_projection_inventory( writer, path, None, ("images/original.tif", "images/new.tif") @@ -519,7 +545,14 @@ def test_native_header_reload_preserves_declared_crop_and_alias(tmp_path, origin context = ImagePayloadSourceMetadataContext(SourceImageIdentity(str(path), {})) from openhcs.core.source_bindings import NamedSourceBinding - current = NamedSourceBinding(alias="neurite").apply_loaded_payload( reloaded, context, ).metadata + current = ( + NamedSourceBinding(alias="neurite") + .apply_loaded_payload( + reloaded, + context, + ) + .metadata + ) assert current.source_dtype == "float32" assert current.source_spatial_domain == metadata.source_spatial_domain assert current.source_image_names == ("neurite",) @@ -545,10 +578,12 @@ def test_native_mosaic_replaces_incompatible_old_tile_domain_without_losing_sour reloaded = ImageMetadataPayload( image_format.read(path), image_format.persisted_metadata(path, payload) ) - current = (NamedSourceBinding(alias="neurite").apply_loaded_payload( + current = ( + NamedSourceBinding(alias="neurite").apply_loaded_payload( reloaded, ImagePayloadSourceMetadataContext(SourceImageIdentity(str(path), {})), - )).metadata + ) + ).metadata assert current.source_spatial_domain.origin_yx == (0, 0) assert current.source_spatial_domain.source_shape_yx == (10, 16) assert current.source_image_names == ("neurite",) diff --git a/tests/unit/test_persisted_roi_only_publication.py b/tests/unit/test_persisted_roi_only_publication.py index 36e68e53c6..f2bff30df0 100644 --- a/tests/unit/test_persisted_roi_only_publication.py +++ b/tests/unit/test_persisted_roi_only_publication.py @@ -36,7 +36,11 @@ ProducedImageMetadataCapability, RuntimeArtifactMetadataTarget, ) -from openhcs.core.virtual_workspace_metadata import AtomicMetadataWriter, METADATA_CONFIG, MetadataWriteError +from openhcs.core.virtual_workspace_metadata import ( + AtomicMetadataWriter, + METADATA_CONFIG, + MetadataWriteError, +) from openhcs.processing.materialization.core import ( MaterializationSpec, _WRITERS_BY_OPTIONS, @@ -55,7 +59,8 @@ def _publish_saved_step(context, plan, *, artifact_materializations=()): METADATA_CONFIG.metadata_path(target.dataset_root) for target in facts ): entries = { - target: value for target, value in facts.items() + target: value + for target, value in facts.items() if METADATA_CONFIG.metadata_path(target.dataset_root) == metadata_path } AtomicMetadataWriter().reconcile_completed_dataset( @@ -103,8 +108,13 @@ def plan_for(plate, backend=Backend.DISK.value): def save_archive_outcome( - context, plate, destination, monkeypatch, - *, filename="independent-outline.roi.zip", content=b"synthetic admission only", + context, + plate, + destination, + monkeypatch, + *, + filename="independent-outline.roi.zip", + content=b"synthetic admission only", backend=Backend.DISK.value, ): """Use the original declared bundle writer, batch and disk save boundary.""" @@ -225,9 +235,7 @@ def test_saved_roi_only_batch_publishes_and_reconciles_without_rendering( artifact_materializations=(outcome,), ) assert tuple(target.output_dir for target in targets) == (plate / destination,) - _publish_saved_step( - publication_context, plan, artifact_materializations=(outcome,) - ) + _publish_saved_step(publication_context, plan, artifact_materializations=(outcome,)) assert result_paths(plate, publication_context) == (output.path,) published = json.loads(METADATA_CONFIG.metadata_path(plate).read_text()) assert published["subdirectories"][destination]["image_files"] == [] @@ -305,58 +313,91 @@ def test_shared_writer_accepts_result_only_but_not_unaddressed_images( def test_measurement_only_step_does_not_reconcile_pending_image_producer( - tmp_path, publication_context, monkeypatch, + tmp_path, + publication_context, + monkeypatch, ): from polystore.virtual_workspace import SourcePixelRef from openhcs.core.artifacts import ImageArtifactType - from openhcs.core.source_projection import OpenHCSPlaneAddress, SourceArtifactProjection + from openhcs.core.source_projection import ( + OpenHCSPlaneAddress, + SourceArtifactProjection, + ) from openhcs.core.virtual_workspace_metadata import ( - AtomicMetadataWriter, FIELDS, VirtualWorkspaceSourceProjectionEntries, + AtomicMetadataWriter, + FIELDS, + VirtualWorkspaceSourceProjectionEntries, ) plate = tmp_path / "plate" destination = "shared_results" outcome, table_output = save_archive_outcome( - publication_context, plate, destination, monkeypatch, - filename="measurements.csv", content=b"ObjectNumber,Area\n1,6\n", + publication_context, + plate, + destination, + monkeypatch, + filename="measurements.csv", + content=b"ObjectNumber,Area\n1,6\n", ) pending = plate / destination / "independent-image.tif" # This lifecycle fixture never reads or derives addresses from the bytes. pending.write_bytes(b"producer pixel save precedes its typed publication") plan = plan_for(plate) (target,) = OpenHCSMetadataTarget.for_execution( - publication_context, plan, artifact_materializations=(outcome,), + publication_context, + plan, + artifact_materializations=(outcome,), ) assert target.produced_projection_entries(publication_context, plan).entries == {} staged = OpenHCSMetadataTarget.observe_for_step( - publication_context, plan, artifact_materializations=(outcome,), + publication_context, + plan, + artifact_materializations=(outcome,), ) metadata_path = METADATA_CONFIG.metadata_path(plate) assert not metadata_path.exists() assert table_output.path == str(plate / destination / "measurements.csv") - assert (plate / destination / "measurements.csv").read_bytes() == b"ObjectNumber,Area\n1,6\n" + assert ( + plate / destination / "measurements.csv" + ).read_bytes() == b"ObjectNumber,Area\n1,6\n" with pytest.raises(MetadataWriteError, match="lack typed produced addresses"): AtomicMetadataWriter().reconcile_completed_dataset( - metadata_path, {target: publication_context}, + metadata_path, + {target: publication_context}, produced_entries_by_target=staged, ) assert not metadata_path.exists() virtual_path = str(pending.relative_to(plate)) projection = SourceArtifactProjection( - address=OpenHCSPlaneAddress(((Microscopy.Well, "A04"), (Microscopy.Site, 1), (Microscopy.Channel, 2), (Microscopy.ZIndex, 1), (Microscopy.Timepoint, 1))), + address=OpenHCSPlaneAddress( + ( + (Microscopy.Well, "A04"), + (Microscopy.Site, 1), + (Microscopy.Channel, 2), + (Microscopy.ZIndex, 1), + (Microscopy.Timepoint, 1), + ) + ), ref=SourcePixelRef(Backend.DISK.value, virtual_path), - source_alias="declared_image", artifact_kind=ImageArtifactType, + source_alias="declared_image", + artifact_kind=ImageArtifactType, ) from openhcs.core.source_projection import SourceProjectionMetadataSerializer + AtomicMetadataWriter().replace_subdirectory_metadata( - metadata_path, destination, - SourceProjectionMetadataSerializer.projection_fields(((projection, virtual_path),)), + metadata_path, + destination, + SourceProjectionMetadataSerializer.projection_fields( + ((projection, virtual_path),) + ), ) publication_context.step_plans = {0: plan} OpenHCSMetadataTarget.finalize_completed_dataset({"A01": publication_context}) - reconciled = json.loads(metadata_path.read_text())[FIELDS.SUBDIRECTORIES][destination] + reconciled = json.loads(metadata_path.read_text())[FIELDS.SUBDIRECTORIES][ + destination + ] assert reconciled[FIELDS.IMAGE_FILES] == [virtual_path] assert len(reconciled[FIELDS.SOURCE_PROJECTION]) == 1 restored = VirtualWorkspaceSourceProjectionEntries.from_subdirectory(reconciled) @@ -379,7 +420,15 @@ def test_reconciliation_keeps_artifact_destination_without_results_field( plate = tmp_path / "plate" plate.mkdir() projection = SourceArtifactProjection( - address=OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, "1"), (Microscopy.Channel, "2"), (Microscopy.ZIndex, "1"), (Microscopy.Timepoint, "1"))), + address=OpenHCSPlaneAddress( + ( + (Microscopy.Well, "A01"), + (Microscopy.Site, "1"), + (Microscopy.Channel, "2"), + (Microscopy.ZIndex, "1"), + (Microscopy.Timepoint, "1"), + ) + ), ref=SourcePixelRef(Backend.DISK.value, "physical/acquisition-source.tif"), source_alias="Declared", artifact_kind=MetadataArtifactType, @@ -394,7 +443,8 @@ def test_reconciliation_keeps_artifact_destination_without_results_field( assert handler.analysis_result_directories(plate) == () owner = RuntimeArtifactMetadataTarget.from_plan(plan_for(plate)) assert plate / "nested" in tuple( - target.output_dir for target in owner.reconciliation_targets(publication_context) + target.output_dir + for target in owner.reconciliation_targets(publication_context) ) assert metadata_path.read_bytes() == before @@ -488,12 +538,16 @@ def from_plan(cls, plan): pending.write_bytes(b"never decoded") target.write(publication_context, produced_plan=plan) assert publication_context.publication_audit == [ - ("before", destination), ("after", destination), - ("before", destination), ("after", destination), + ("before", destination), + ("after", destination), + ("before", destination), + ("after", destination), ] # Independent MI hooks reach the inherited empty-step behavior even # while another image producer has not published its typed address. - assert target.produced_projection_entries(publication_context, plan).entries == {} + assert ( + target.produced_projection_entries(publication_context, plan).entries == {} + ) before = METADATA_CONFIG.metadata_path(plate).read_bytes() with pytest.raises(MetadataWriteError, match="lack typed produced addresses"): target.write(publication_context) diff --git a/tests/unit/test_persisted_semantic_metadata.py b/tests/unit/test_persisted_semantic_metadata.py index 5912a215ea..66f37bbb7b 100644 --- a/tests/unit/test_persisted_semantic_metadata.py +++ b/tests/unit/test_persisted_semantic_metadata.py @@ -31,7 +31,9 @@ VirtualWorkspacePathLookup, VirtualWorkspaceSourceProjection, ) -from openhcs.core.virtual_workspace_metadata import VirtualWorkspaceSourceProjectionEntries +from openhcs.core.virtual_workspace_metadata import ( + VirtualWorkspaceSourceProjectionEntries, +) from openhcs.core.dataset_sources.source_schema import SourceSchemaFilenameParser from python_introspect import to_jsonable from openhcs.domains.microscopy.axes import Microscopy @@ -62,7 +64,15 @@ def _projection( source_metadata: Mapping[str, object] | None = None, ) -> SourcePlaneProjection: return SourcePlaneProjection( - address=OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, 1), (Microscopy.Channel, 1), (Microscopy.ZIndex, 1), (Microscopy.Timepoint, 1))), + address=OpenHCSPlaneAddress( + ( + (Microscopy.Well, "A01"), + (Microscopy.Site, 1), + (Microscopy.Channel, 1), + (Microscopy.ZIndex, 1), + (Microscopy.Timepoint, 1), + ) + ), ref=SourcePixelRef("disk", VIRTUAL_PATH), image_metadata=image_metadata, source_metadata={} if source_metadata is None else source_metadata, @@ -135,7 +145,9 @@ def test_publication_retains_typed_provenance_and_exact_wire_document() -> None: original = json.dumps(subdirectory) typed = VirtualWorkspaceSourceProjectionEntries.from_subdirectory(subdirectory) retained = typed.publish_into_subdirectory( - subdirectory, saved_image_paths=(VIRTUAL_PATH,), reconcile_directory=".", + subdirectory, + saved_image_paths=(VIRTUAL_PATH,), + reconcile_directory=".", accepted_entries=typed, ) @@ -164,7 +176,15 @@ def test_image_artifact_projection_round_trips_typed_pixel_metadata() -> None: source_dtype="uint8", ) projection = SourceArtifactProjection( - address=OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, 1), (Microscopy.Channel, 2), (Microscopy.ZIndex, 1), (Microscopy.Timepoint, 1))), + address=OpenHCSPlaneAddress( + ( + (Microscopy.Well, "A01"), + (Microscopy.Site, 1), + (Microscopy.Channel, 2), + (Microscopy.ZIndex, 1), + (Microscopy.Timepoint, 1), + ) + ), ref=SourcePixelRef("disk", "analysis/A01_candidate.checkpoint.tif"), source_alias="neurite_candidate_mask", artifact_kind=ImageArtifactType, diff --git a/tests/unit/test_plane_nlm_declaration.py b/tests/unit/test_plane_nlm_declaration.py index 7af11cad80..3ab67024de 100644 --- a/tests/unit/test_plane_nlm_declaration.py +++ b/tests/unit/test_plane_nlm_declaration.py @@ -26,7 +26,9 @@ from openhcs.core.source_metadata import SourceVoxelSpacing from openhcs.core.source_spatial_domain import SourceSpatialDomain from openhcs.processing.backends.lib_registry.openhcs_registry import OpenHCSRegistry -from openhcs.processing.backends.lib_registry.scikit_image_registry import SkimageRegistry +from openhcs.processing.backends.lib_registry.scikit_image_registry import ( + SkimageRegistry, +) from openhcs.processing.custom_functions.runtime_registry import project_custom_function from openhcs.core.processing_contracts import ( FlexibleContract, @@ -36,10 +38,13 @@ non_local_means_denoise_planes, ) - NLM_KWARGS = dict( - patch_size=3, patch_distance=2, h=0.075, - fast_mode=True, sigma=0.0, preserve_range=True, + patch_size=3, + patch_distance=2, + h=0.075, + fast_mode=True, + sigma=0.0, + preserve_range=True, ) @@ -47,15 +52,17 @@ def _source(plane_count): rng = np.random.default_rng(458) pixels = rng.uniform(0.1, 0.9, (plane_count, 16, 20)).astype(np.float32) provenance = SourceImageProvenancePlanes.from_components( - paths=tuple(f"/synthetic/A01_s{site + 1}_w1.tif" for site in range(plane_count)), + paths=tuple( + f"/synthetic/A01_s{site + 1}_w1.tif" for site in range(plane_count) + ), component_metadata=tuple( {"well": "A01", "site": str(site + 1), "channel": "1"} for site in range(plane_count) ), ) - provenance = SourceImageProvenancePlanes(tuple( - replace(plane, source_image_name="C1") for plane in provenance.planes - )) + provenance = SourceImageProvenancePlanes( + tuple(replace(plane, source_image_name="C1") for plane in provenance.planes) + ) metadata = ImagePayloadMetadata( plane_axis=RuntimePlaneAxis.RUNTIME_SLICE, source_image_names=("C1",) * plane_count, @@ -63,7 +70,8 @@ def _source(plane_count): source_plane_dtypes=("float32",) * plane_count, source_voxel_spacing=SourceVoxelSpacing((0.5, 0.75)), source_spatial_domain=SourceSpatialDomain( - origin_yx=(0, 0), source_shape_yx=(16, 20), + origin_yx=(0, 0), + source_shape_yx=(16, 20), ), ) return metadata.payload_with(pixels) @@ -83,13 +91,17 @@ def _registered(metadata=None): @pytest.mark.parametrize("plane_count", (1, 3)) @pytest.mark.parametrize("fast_mode", (True, False)) def test_real_nlm_kernel_receives_only_planes_and_restores_context( - monkeypatch, plane_count, fast_mode, + monkeypatch, + plane_count, + fast_mode, ): source = _source(plane_count) before = source.data.copy() original = restoration.denoise_nl_means kwargs = dict(NLM_KWARGS, fast_mode=fast_mode) - expected = np.stack([original(plane, channel_axis=None, **kwargs) for plane in before]) + expected = np.stack( + [original(plane, channel_axis=None, **kwargs) for plane in before] + ) seen = [] def observed_kernel(image, **kernel_kwargs): @@ -98,7 +110,11 @@ def observed_kernel(image, **kernel_kwargs): monkeypatch.setattr(restoration, "denoise_nl_means", observed_kernel) result = _registered().func(source, **kwargs) - assert seen == [((16, 20), np.dtype("float32"), 1280, dict(kwargs, channel_axis=None))] * plane_count + assert ( + seen + == [((16, 20), np.dtype("float32"), 1280, dict(kwargs, channel_axis=None))] + * plane_count + ) np.testing.assert_array_equal(result.data, expected) np.testing.assert_array_equal(source.data, before) assert result.data.dtype == expected.dtype == np.dtype("float32") @@ -108,7 +124,10 @@ def observed_kernel(image, **kernel_kwargs): assert current.plane_axis is prior.plane_axis is RuntimePlaneAxis.RUNTIME_SLICE assert current.source_voxel_spacing == prior.source_voxel_spacing assert current.source_spatial_domain == prior.source_spatial_domain - assert current.source_image_provenance_planes.identity == prior.source_image_provenance_planes.identity + assert ( + current.source_image_provenance_planes.identity + == prior.source_image_provenance_planes.identity + ) assert current.source_image_names == prior.source_image_names @@ -145,7 +164,10 @@ def observed_volume(image, *args, **kwargs): def test_declared_catalog_identity_and_public_kwargs_are_original_projections(): metadata = _registered() function_id = f"{metadata.registry.library_name}:{metadata.name}" - assert function_id == "openhcs:processors_numpy_processor_non_local_means_denoise_planes" + assert ( + function_id + == "openhcs:processors_numpy_processor_non_local_means_denoise_planes" + ) assert metadata.original_name == "non_local_means_denoise_planes" assert "slice_by_slice" not in inspect.signature(metadata.func).parameters _validate_callable_kwargs(function_id, metadata.func, NLM_KWARGS) @@ -159,9 +181,14 @@ def test_nominal_transport_resolves_original_declaration_without_global_catalog( metadata = _registered() reference = FunctionReferenceTransportAuthority.function_reference(metadata.func) assert isinstance(reference, RegistryFunctionReference) - assert reference.composite_key == "openhcs:processors_numpy_processor_non_local_means_denoise_planes" + assert ( + reference.composite_key + == "openhcs:processors_numpy_processor_non_local_means_denoise_planes" + ) resolved = reference.resolve() - assert CallableContract.from_callable(resolved).processing_contract is Pure2DContract + assert ( + CallableContract.from_callable(resolved).processing_contract is Pure2DContract + ) result = resolved(_source(1), **NLM_KWARGS) assert result.data.shape == (1, 16, 20) @@ -170,12 +197,16 @@ def test_new_independent_declaration_executes_same_consumer_without_edits(): seen = [] @numpy_contract(contract=Pure2DContract) - def independent_plane_offset(image: np.ndarray, *, offset: float = 0.25) -> np.ndarray: + def independent_plane_offset( + image: np.ndarray, *, offset: float = 0.25 + ) -> np.ndarray: seen.append(image.shape) return image + offset source = _source(3) - result = _registered(project_custom_function(independent_plane_offset)).func(source, offset=0.5) + result = _registered(project_custom_function(independent_plane_offset)).func( + source, offset=0.5 + ) assert seen == [(16, 20)] * 3 np.testing.assert_array_equal(result.data, source.data + 0.5) assert result.metadata.source_image_provenance_planes.identity == ( diff --git a/tests/unit/test_plane_store_sources.py b/tests/unit/test_plane_store_sources.py index 0c43812468..16f38fff96 100644 --- a/tests/unit/test_plane_store_sources.py +++ b/tests/unit/test_plane_store_sources.py @@ -321,7 +321,9 @@ def test_mixed_plane_stores_bind_and_load_through_virtual_workspace( ) ) filemanager = _filemanager() - handler = DatasetSourceChoice.named("auto").open(tmp_path, filemanager=filemanager, source_bindings_config=source_bindings) + handler = DatasetSourceChoice.named("auto").open( + tmp_path, filemanager=filemanager, source_bindings_config=source_bindings + ) assert isinstance(handler, BioFormatsHandler) assert handler.parser.extract_component_coordinates("plain.tif") == ( "S", @@ -354,7 +356,9 @@ def test_saved_source_bindings_rebuild_canonical_store_projection( replace_config: str, ) -> None: def reject_editor(*args, **kwargs): - raise AssertionError("Saved runtime config capture must not construct an editor") + raise AssertionError( + "Saved runtime config capture must not construct an editor" + ) if replace_config != "delegate": monkeypatch.setattr(ObjectState, "__init__", reject_editor) diff --git a/tests/unit/test_plate_image_inventory.py b/tests/unit/test_plate_image_inventory.py index 55cc08a6f4..c4bda9ab52 100644 --- a/tests/unit/test_plate_image_inventory.py +++ b/tests/unit/test_plate_image_inventory.py @@ -76,7 +76,9 @@ def test_image_browser_inventory_uses_declared_virtual_workspace_address( ensure_storage_registry() filemanager = FileManager(dict(storage_registry)) - handler = DatasetSourceChoice.named("imagexpress").open(plate, filemanager=filemanager) + handler = DatasetSourceChoice.named("imagexpress").open( + plate, filemanager=filemanager + ) handler.initialize_workspace(plate, filemanager) assert ( handler.get_primary_backend(plate, filemanager) diff --git a/tests/unit/test_primary_object_diagnostics.py b/tests/unit/test_primary_object_diagnostics.py index 56bb53ca71..2d0af04c71 100644 --- a/tests/unit/test_primary_object_diagnostics.py +++ b/tests/unit/test_primary_object_diagnostics.py @@ -21,7 +21,9 @@ ObjectLabelsArtifactType, ) from openhcs.core.runtime_image_values import ImagePayloadMetadata, MaskedImagePayload -from openhcs.core.runtime_object_label_building import SourceImageObjectLabelBuildRequest +from openhcs.core.runtime_object_label_building import ( + SourceImageObjectLabelBuildRequest, +) from openhcs.core.source_spatial_domain import SourceSpatialDomain from openhcs.processing.backends.cellprofiler.primary_object_diagnostics import ( ExecutedDeclumpingEvidence, @@ -71,7 +73,8 @@ def _objects(source): def test_selected_response_keeps_source_frame_without_acquisition_intensity_scale(): from openhcs.core.runtime_plane_projection import ( - RuntimePlaneAxis, RuntimePlaneAxisValueProjection, + RuntimePlaneAxis, + RuntimePlaneAxisValueProjection, ) source = _source() @@ -88,7 +91,9 @@ def test_selected_response_keeps_source_frame_without_acquisition_intensity_scal np.testing.assert_array_equal(resolved.data, pixels[0]) np.testing.assert_array_equal(resolved.normalize_intensity_payload(), pixels[0]) assert resolved.metadata.source_path == source.metadata.source_path - assert resolved.metadata.source_spatial_domain == source.metadata.source_spatial_domain + assert ( + resolved.metadata.source_spatial_domain == source.metadata.source_spatial_domain + ) assert resolved.metadata.source_dtype == "float32" assert resolved.metadata.intensity_scale is None with pytest.raises(ValueError, match="preserve the source spatial shape"): @@ -106,7 +111,8 @@ def test_planes_preserve_exact_source_coordinates_and_owned_variants(executed): markers = maxima.astype(np.int32) evidence = ( ExecutedDeclumpingEvidence(response, maxima, markers) - if executed else UnexecutedDeclumpingEvidence() + if executed + else UnexecutedDeclumpingEvidence() ) threshold_support = source.data > 15 initial_components = objects.labels.copy() @@ -129,7 +135,9 @@ def test_planes_preserve_exact_source_coordinates_and_owned_variants(executed): assert diagnostics.threshold_support.data is threshold_support assert diagnostics.initial_components.data is initial_components assert diagnostics.unedited_objects.data is objects.variant_labels(UneditedLabels) - assert diagnostics.small_removed_objects.data is objects.variant_labels(SmallRemovedLabels) + assert diagnostics.small_removed_objects.data is objects.variant_labels( + SmallRemovedLabels + ) assert objects.domain.declared_object_count == 1 assert diagnostics.threshold_support.mask is diagnostics.initial_components.mask assert diagnostics.unedited_objects.mask is diagnostics.small_removed_objects.mask @@ -153,7 +161,9 @@ def test_planes_preserve_exact_source_coordinates_and_owned_variants(executed): def test_diagnostic_contract_has_source_producer_stage_identity_not_new_objects(): source = ArtifactSpec.input("DNA", ImageArtifactType) objects = ArtifactSpec.output("Nuclei", ObjectLabelsArtifactType) - specs = PrimaryObjectDiagnosticPlanes.artifact_specs(source_image=source, objects=objects) + specs = PrimaryObjectDiagnosticPlanes.artifact_specs( + source_image=source, objects=objects + ) assert len(specs) == len(PrimaryObjectDiagnosticPlanes._fields) for field, spec in zip(PrimaryObjectDiagnosticPlanes._fields, specs, strict=True): assert spec.name.endswith(f"__{field}") @@ -172,19 +182,22 @@ def test_diagnostic_contract_has_source_producer_stage_identity_not_new_objects( assert spec.source_context_sources() == (source.ref(),) assert ArtifactSidecarSourceRelation(source=objects.ref()) in spec.relations assert len(get_args(PrimaryObjectsRuntimeTuple)) == 3 + len(specs) - assert get_args(PrimaryObjectsRuntimeTuple)[3:] == (MaskedImagePayload,) * len(specs) + assert get_args(PrimaryObjectsRuntimeTuple)[3:] == (MaskedImagePayload,) * len( + specs + ) def test_ordinary_image_contextualization_preserves_unexecuted_masks(): from openhcs.core.artifacts import ArtifactOutputPlan - source = _source() objects = _objects(source) diagnostics = PrimaryObjectDiagnosticPlanes.from_execution( - image=source, threshold_support=source.data > 15, + image=source, + threshold_support=source.data > 15, initial_components=objects.labels, - declumping=UnexecutedDeclumpingEvidence(), objects=objects, + declumping=UnexecutedDeclumpingEvidence(), + objects=objects, ) specs = PrimaryObjectDiagnosticPlanes.artifact_specs( source_image=ArtifactSpec.input("DNA", ImageArtifactType), @@ -192,13 +205,15 @@ def test_ordinary_image_contextualization_preserves_unexecuted_masks(): ) for spec, plane in zip(specs, diagnostics, strict=True): plan = ArtifactOutputPlan( - name=spec.name, path=f"/memory/{spec.name}.pkl", - artifact_type=spec.artifact_type, sidecar_role=spec.sidecar_role, + name=spec.name, + path=f"/memory/{spec.name}.pkl", + artifact_type=spec.artifact_type, + sidecar_role=spec.sidecar_role, relations=spec.relations, ) - value = (ImageArtifactType if plan is None else plan.artifact_type).contextualize_output( - source, plane, plan, None - ) + value = ( + ImageArtifactType if plan is None else plan.artifact_type + ).contextualize_output(source, plane, plan, None) np.testing.assert_array_equal(value.mask, plane.mask) np.testing.assert_array_equal(value.data, plane.data) assert value.metadata.source_provenance == source.metadata.source_provenance @@ -210,18 +225,24 @@ def test_existing_pure2d_aggregation_retains_stage_pixels_masks_and_plane_identi from openhcs.core.processing_contracts import ( Pure2DAuxiliaryOutputAggregator, ) - from openhcs.processing.backends.cellprofiler.primary_object_diagnostics import DiagnosticPlaneSource + from openhcs.processing.backends.cellprofiler.primary_object_diagnostics import ( + DiagnosticPlaneSource, + ) first = _source() second = MaskedImagePayload( - first.data + 10, ~first.mask, + first.data + 10, + ~first.mask, ImagePayloadMetadata( - source_path="/synthetic/A01_s002_DNA.tif", source_image_names=("DNA",), + source_path="/synthetic/A01_s002_DNA.tif", + source_image_names=("DNA",), source_spatial_domain=first.metadata.source_spatial_domain, ), ) - planes = [DiagnosticPlaneSource.from_image(source).plane(source.data) - for source in (first, second)] + planes = [ + DiagnosticPlaneSource.from_image(source).plane(source.data) + for source in (first, second) + ] value = Pure2DAuxiliaryOutputAggregator.aggregate( planes, "numpy", plane_axis=RuntimePlaneAxis.RUNTIME_SLICE ) @@ -250,28 +271,42 @@ def test_existing_pure2d_aggregation_retains_stage_pixels_masks_and_plane_identi @pytest.mark.parametrize("shared_mask", (False, True)) @pytest.mark.parametrize("spatial_mask", (False, True)) -def test_produced_input_snapshot_owns_canonical_pixels_and_masks(shared_mask, spatial_mask): +def test_produced_input_snapshot_owns_canonical_pixels_and_masks( + shared_mask, spatial_mask +): import pickle from openhcs.core.aligned_image_payload import ProducedImageStack from openhcs.core.runtime_plane_projection import RuntimePlaneAxis - pixels = (np.full((3, 4), 7, dtype=np.uint16), - np.full((3, 4), 0.25, dtype=np.float32)) + pixels = ( + np.full((3, 4), 7, dtype=np.uint16), + np.full((3, 4), 0.25, dtype=np.float32), + ) first_mask = np.ones((3, 4), dtype=bool) masks = (first_mask, first_mask if shared_mask else ~first_mask) - metadata = ImagePayloadMetadata( - source_spatial_domain=SourceSpatialDomain(source_shape_yx=(3, 4)) - ) if spatial_mask else ImagePayloadMetadata() + metadata = ( + ImagePayloadMetadata( + source_spatial_domain=SourceSpatialDomain(source_shape_yx=(3, 4)) + ) + if spatial_mask + else ImagePayloadMetadata() + ) source = ProducedImageStack( - tuple(MaskedImagePayload(data, mask, metadata) - for data, mask in zip(pixels, masks, strict=True)), - memory_type="numpy", plane_axis=( - RuntimePlaneAxis.SOURCE_BINDING if spatial_mask + tuple( + MaskedImagePayload(data, mask, metadata) + for data, mask in zip(pixels, masks, strict=True) + ), + memory_type="numpy", + plane_axis=( + RuntimePlaneAxis.SOURCE_BINDING + if spatial_mask else RuntimePlaneAxis.RUNTIME_SLICE ), ) expected = np.stack(pixels) - expected_mask = first_mask.copy() if shared_mask and spatial_mask else np.stack(masks) + expected_mask = ( + first_mask.copy() if shared_mask and spatial_mask else np.stack(masks) + ) copied = source.copied(memory_type="numpy", device_id=None) assert source._composed_payload is None canonical = copied.compose() @@ -299,9 +334,11 @@ def test_typed_stage_payloads_roundtrip_existing_pickle_boundary(): source = _source() diagnostics = PrimaryObjectDiagnosticPlanes.from_execution( - image=source, threshold_support=source.data > 15, + image=source, + threshold_support=source.data > 15, initial_components=_objects(source).labels, - declumping=UnexecutedDeclumpingEvidence(), objects=_objects(source), + declumping=UnexecutedDeclumpingEvidence(), + objects=_objects(source), ) reloaded = pickle.loads(pickle.dumps(diagnostics)) for before, after in zip(diagnostics, reloaded, strict=True): @@ -326,27 +363,34 @@ class ExtendedPlanes(NamedTuple): def test_focused_real_source_antipattern_guards(): root = Path(__file__).resolve().parents[2] - source = root / "openhcs/processing/backends/cellprofiler/primary_object_diagnostics.py" + source = ( + root / "openhcs/processing/backends/cellprofiler/primary_object_diagnostics.py" + ) tree = ast.parse(source.read_text()) # IMPL-1/2/3/7 and BOUND-1/2/7: evidence owners have no kind/type # dispatcher, raw string-key consumers or duck-typed fallback readers. assert not any(isinstance(node, ast.Match) for node in ast.walk(tree)) assert not any( - isinstance(node, ast.Call) and isinstance(node.func, ast.Name) + isinstance(node, ast.Call) + and isinstance(node.func, ast.Name) and node.func.id in {"isinstance", "getattr", "hasattr", "eval", "exec"} for node in ast.walk(tree) ) assert not any( - isinstance(node, ast.Subscript) and isinstance(node.slice, ast.Constant) + isinstance(node, ast.Subscript) + and isinstance(node.slice, ast.Constant) and isinstance(node.slice.value, str) for node in ast.walk(tree) ) projection = next( - node for node in ast.walk(tree) + node + for node in ast.walk(tree) if isinstance(node, ast.FunctionDef) and node.name == "artifact_specs" ) # MEMB-1/3: no hand-maintained roster alongside the typed stage record. - assert not any(isinstance(node, (ast.List, ast.Set, ast.Dict)) for node in ast.walk(projection)) + assert not any( + isinstance(node, (ast.List, ast.Set, ast.Dict)) for node in ast.walk(projection) + ) assert any( isinstance(node, ast.Attribute) and node.attr == "_fields" for node in ast.walk(projection) @@ -358,18 +402,35 @@ def test_processing_body_is_unchanged_except_same_run_evidence_capture(): relative = "openhcs/processing/backends/cellprofiler/primary_objects.py" baseline = subprocess.run( ["git", "show", f"a0263e82a1b3296f34f8c5cdd50ffae5c33ef7dd:{relative}"], - cwd=root, check=True, capture_output=True, text=True, + cwd=root, + check=True, + capture_output=True, + text=True, ).stdout - before = next(n for n in ast.parse(baseline).body if isinstance(n, ast.FunctionDef) - and n.name == "identify_primary_objects") - after = next(n for n in ast.parse((root / relative).read_text()).body - if isinstance(n, ast.FunctionDef) and n.name == before.name) + before = next( + n + for n in ast.parse(baseline).body + if isinstance(n, ast.FunctionDef) and n.name == "identify_primary_objects" + ) + after = next( + n + for n in ast.parse((root / relative).read_text()).body + if isinstance(n, ast.FunctionDef) and n.name == before.name + ) class RemoveEvidence(ast.NodeTransformer): def visit_Assign(self, node): - if any(isinstance(target, ast.Name) and target.id in { - "threshold_support", "initial_components", "declumping", "diagnostics" - } for target in node.targets): + if any( + isinstance(target, ast.Name) + and target.id + in { + "threshold_support", + "initial_components", + "declumping", + "diagnostics", + } + for target in node.targets + ): return None return self.generic_visit(node) @@ -393,14 +454,19 @@ def visit_Call(self, node): and not node.keywords ): return ast.Attribute( - value=node.args[0], attr=self.ACCESSORS[node.func.id], ctx=ast.Load() + value=node.args[0], + attr=self.ACCESSORS[node.func.id], + ctx=ast.Load(), ) return node class DeclaredLabelVariants(ast.NodeTransformer): """The label-variant keywords became one variants mapping (G5).""" - VARIANTS = {"unedited_labels": "UneditedLabels", "small_removed_labels": "SmallRemovedLabels"} + VARIANTS = { + "unedited_labels": "UneditedLabels", + "small_removed_labels": "SmallRemovedLabels", + } def visit_Call(self, node): self.generic_visit(node) @@ -408,7 +474,9 @@ def visit_Call(self, node): if not variant_keywords: return node mapping = ast.Dict( - keys=[ast.Name(self.VARIANTS[k.arg], ast.Load()) for k in variant_keywords], + keys=[ + ast.Name(self.VARIANTS[k.arg], ast.Load()) for k in variant_keywords + ], values=[k.value for k in variant_keywords], ) first = node.keywords.index(variant_keywords[0]) @@ -420,4 +488,5 @@ def visit_Call(self, node): before = DeclaredLabelVariants().visit(PayloadAttributes().visit(before)) # Compare the original algorithm, not docstring, signature or output ABI. assert ast.dump(ast.Module(body=before.body[1:-1], type_ignores=[])) == ast.dump( - ast.Module(body=after.body[1:-1], type_ignores=[])) + ast.Module(body=after.body[1:-1], type_ignores=[]) + ) diff --git a/tests/unit/test_process_local_cache.py b/tests/unit/test_process_local_cache.py index 171a0c957d..90d2ff02a8 100644 --- a/tests/unit/test_process_local_cache.py +++ b/tests/unit/test_process_local_cache.py @@ -40,7 +40,10 @@ def test_process_local_domains_join_family_cleanup_and_store_queries_do_not(): assert len({id(cache) for cache in caches}) == len(process_local) assert all(cache.max_entries == 16 for cache in caches[: len(numerical)]) for cache_type in process_local: - assert ProcessLocalBoundedCache.__registry__[cache_type.process_cache_key] is cache_type + assert ( + ProcessLocalBoundedCache.__registry__[cache_type.process_cache_key] + is cache_type + ) owner = object() for cache in caches[:-1]: diff --git a/tests/unit/test_processor_operations_written_once.py b/tests/unit/test_processor_operations_written_once.py index 3a1f5bd2c4..5896e30125 100644 --- a/tests/unit/test_processor_operations_written_once.py +++ b/tests/unit/test_processor_operations_written_once.py @@ -117,7 +117,9 @@ def test_projection_choices_are_served_by_the_projection_family() -> None: } for backend in BACKENDS: create_projection = _processor_module(backend).create_projection - method_type = type(inspect.signature(create_projection).parameters["method"].default) + method_type = type( + inspect.signature(create_projection).parameters["method"].default + ) assert {member.value for member in method_type} <= served, backend @@ -125,7 +127,11 @@ def test_numpy_registrations_compute_the_shared_operations() -> None: numpy_processor = _processor_module("numpy") stack = np.arange(2 * 4 * 4, dtype=np.uint16).reshape(2, 4, 4) - assert numpy_processor.spatial_bin_2d(stack, 2, SpatialBinMethod.MEAN).shape == (2, 2, 2) + assert numpy_processor.spatial_bin_2d(stack, 2, SpatialBinMethod.MEAN).shape == ( + 2, + 2, + 2, + ) np.testing.assert_array_equal( numpy_processor.spatial_bin_3d(stack, 2, SpatialBinMethod.MAX), stack.reshape(1, 2, 2, 2, 2, 2).max(axis=(1, 3, 5)), @@ -138,12 +144,15 @@ def test_numpy_registrations_compute_the_shared_operations() -> None: numpy_processor.create_projection(stack, method), reduce(stack, axis=0) ) np.testing.assert_array_equal( - numpy_processor.create_projection(stack, numpy_processor.NumpyStackProjectionMethod.MEAN), + numpy_processor.create_projection( + stack, numpy_processor.NumpyStackProjectionMethod.MEAN + ), stack.mean(axis=0).astype(stack.dtype), ) assert ( - CallableContract.from_callable(numpy_processor.create_projection) - .require_processing_contract() + CallableContract.from_callable( + numpy_processor.create_projection + ).require_processing_contract() is VolumetricToSliceContract ) mask = np.array([[1, 0, 1, 0]] * 4, dtype=np.uint16) diff --git a/tests/unit/test_pycodify_formatters.py b/tests/unit/test_pycodify_formatters.py index 1d8c7bf527..aeea134b1a 100644 --- a/tests/unit/test_pycodify_formatters.py +++ b/tests/unit/test_pycodify_formatters.py @@ -48,7 +48,11 @@ def selected_declaration_catalog(monkeypatch): """Use existing declaration metadata, not a native catalog preparation.""" metadata = ( RegistryService.declared_metadata_for_callable(func) - for func in (count_cells_simple, measure_colocalization_objects, measure_object_size_shape) + for func in ( + count_cells_simple, + measure_colocalization_objects, + measure_object_size_shape, + ) ) monkeypatch.setattr(RegistryService, "_metadata_cache", dict(metadata)) @@ -123,15 +127,19 @@ def test_function_reference_formats_from_declared_identity_without_resolution( assert "remote_backend.filters" not in source -def test_module_export_registry_reference_uses_direct_import_without_resolution(monkeypatch): +def test_module_export_registry_reference_uses_direct_import_without_resolution( + monkeypatch, +): reference = ModuleExportRegistryFunctionReference( import_identity=CallableImportIdentity( - module_name="declared.functions", function_name="exported_filter", + module_name="declared.functions", + function_name="exported_filter", ), composite_key="registry:exported_filter", ) monkeypatch.setattr( - RegistryFunctionReference, "resolve", + RegistryFunctionReference, + "resolve", lambda self: (_ for _ in ()).throw(AssertionError("must not resolve")), ) source = _source(reference) @@ -167,7 +175,9 @@ def test_clean_pipeline_config_keeps_explicit_nested_lazy_config_values(): def test_clean_pipeline_config_keeps_explicit_empty_nested_lazy_config(): - source = _source(PipelineConfig(partition_filter_config=LazyPartitionFilterConfig())) + source = _source( + PipelineConfig(partition_filter_config=LazyPartitionFilterConfig()) + ) assert "partition_filter_config=LazyPartitionFilterConfig()" in source @@ -197,7 +207,9 @@ def test_clean_function_step_omits_objectstate_reconstructed_empty_lazy_configs( clean_mode=True, ) - reference = FunctionReferenceTransportAuthority.function_reference(count_cells_simple) + reference = FunctionReferenceTransportAuthority.function_reference( + count_cells_simple + ) assert f"func={reference.source_literal()}" in source assert "name='count_cells'" in source assert "dtype_config=" not in source diff --git a/tests/unit/test_registered_configuration_source_readiness.py b/tests/unit/test_registered_configuration_source_readiness.py index 9580f922c4..f3ff4ee445 100644 --- a/tests/unit/test_registered_configuration_source_readiness.py +++ b/tests/unit/test_registered_configuration_source_readiness.py @@ -6,7 +6,10 @@ import objectstate.lazy_factory as lazy_factory from python_introspect import SignatureAnalyzer -from openhcs.core.processing_preparation import CallablePreparation, PreparationCacheBatch +from openhcs.core.processing_preparation import ( + CallablePreparation, + PreparationCacheBatch, +) from openhcs.processing.backends.lib_registry.registry_service import RegistryService @@ -41,7 +44,9 @@ def raw(image): monkeypatch.setattr(RegistryService, "_metadata_cache", metadata) monkeypatch.setattr(lazy_factory, "_lazy_type_registry", {}) monkeypatch.setattr(lazy_factory, "_base_to_lazy_registry", {}) - monkeypatch.setattr(PreparationCacheBatch, "populate_child_caches", lambda *a, **k: None) + monkeypatch.setattr( + PreparationCacheBatch, "populate_child_caches", lambda *a, **k: None + ) pairs = iter(((LazyConfig, PublicConfig), (LaterLazyConfig, LaterPublicConfig))) def prepare(owner): @@ -60,16 +65,27 @@ def declaration(config_type): monkeypatch.setattr(SignatureAnalyzer, "_extract_inline_field_docs", declaration) # The registry admits types that are not dataclasses; readiness filters them. lazy_factory.register_lazy_type_mapping(dict, list) - assert RegistryService.prepare_in_current_process(status_callback=events.append) is metadata + assert ( + RegistryService.prepare_in_current_process(status_callback=events.append) + is metadata + ) assert factories == [] - first_ready = next(i for i, item in enumerate(events) if isinstance(item, str) and "kernels ready" in item) + first_ready = next( + i + for i, item in enumerate(events) + if isinstance(item, str) and "kernels ready" in item + ) assert events.index(LazyConfig) < first_ready assert events.index(PublicConfig) < first_ready assert LaterPublicConfig not in events and dict not in events and list not in events events.clear() # An already loaded callable catalog must not freeze the configuration roster. RegistryService.prepare_in_current_process(status_callback=events.append) - second_ready = next(i for i, item in enumerate(events) if isinstance(item, str) and "kernels ready" in item) + second_ready = next( + i + for i, item in enumerate(events) + if isinstance(item, str) and "kernels ready" in item + ) for config_type in (LazyConfig, PublicConfig, LaterLazyConfig, LaterPublicConfig): assert events.count(config_type) == 1 assert events.index(config_type) < second_ready diff --git a/tests/unit/test_rescale_stream_plane_declaration.py b/tests/unit/test_rescale_stream_plane_declaration.py index 9fc43bdbb5..7b3a727640 100644 --- a/tests/unit/test_rescale_stream_plane_declaration.py +++ b/tests/unit/test_rescale_stream_plane_declaration.py @@ -13,8 +13,13 @@ from openhcs.core.runtime_stores import RuntimeValueStore from openhcs.core.component_group_scope import RuntimeExecutionAxisScope from openhcs.core.function_patterns import InvocationArtifactInputEdgePlan -from openhcs.core.runtime_object_label_building import SourceImageObjectLabelBuildRequest -from openhcs.core.runtime_plane_projection import RuntimePlaneAxis, RuntimePlaneAxisValueProjection +from openhcs.core.runtime_object_label_building import ( + SourceImageObjectLabelBuildRequest, +) +from openhcs.core.runtime_plane_projection import ( + RuntimePlaneAxis, + RuntimePlaneAxisValueProjection, +) from openhcs.core.runtime_object_label_domains import ObjectLabelDomainScope from openhcs.core.source_image_provenance import SourceImageProvenancePlanes from openhcs.core.config import StepSourceBindingsConfig @@ -23,44 +28,69 @@ from openhcs.core.source_metadata import SourceVoxelSpacing, SourceVoxelSpacingUnit from openhcs.core.steps.function_outputs import StreamOutputBatch from openhcs.core.steps.function_step import FunctionStep -from openhcs.core.steps.stream_component_semantics import StreamImagePayloadMetadataProjector +from openhcs.core.steps.stream_component_semantics import ( + StreamImagePayloadMetadataProjector, +) from openhcs.processing.backends.cellprofiler.intensity import ( - AutomaticHigh, AutomaticLow, RescaleIntensityModule, RescaleMethod, rescale_intensity, + AutomaticHigh, + AutomaticLow, + RescaleIntensityModule, + RescaleMethod, + rescale_intensity, ) from openhcs.processing.backends.cellprofiler.shape import measure_object_size_shape from openhcs.interop.cellprofiler.runtime.adapter import CellProfilerRuntimeAdapter -from openhcs.interop.cellprofiler.runtime.module_execution import CellProfilerModuleExecutor -from openhcs.interop.cellprofiler.runtime.function_contract_execution import CellProfilerFunctionContractExecutor +from openhcs.interop.cellprofiler.runtime.module_execution import ( + CellProfilerModuleExecutor, +) +from openhcs.interop.cellprofiler.runtime.function_contract_execution import ( + CellProfilerFunctionContractExecutor, +) from openhcs.runtime.napari_streaming_handlers import ( - NapariAggregateAxisBindingBuilder, NapariStreamLayerAddress, NapariStreamLayerItem, + NapariAggregateAxisBindingBuilder, + NapariStreamLayerAddress, + NapariStreamLayerItem, ) from openhcs.runtime.viewer_component_system import ( - ViewerComponentAxisSemanticsFactory, ViewerComponentValueDomainPayload, + ViewerComponentAxisSemanticsFactory, + ViewerComponentValueDomainPayload, ) # Reuse the original provider-free compiler and runtime fixtures, not a second ABI. -from tests.unit.test_cellprofiler_generic_special_input_binding import _compile_public_step +from tests.unit.test_cellprofiler_generic_special_input_binding import ( + _compile_public_step, +) from tests.unit.cellprofiler_runtime_test_support import ( cellprofiler_runtime_adapter_for_test, ) from tests.unit.viewer_axes_fixture import STREAM_AXES -@pytest.mark.parametrize("method", [RescaleMethod.DIVIDE_BY_VALUE, RescaleMethod.MANUAL_INPUT_RANGE]) +@pytest.mark.parametrize( + "method", [RescaleMethod.DIVIDE_BY_VALUE, RescaleMethod.MANUAL_INPUT_RANGE] +) @pytest.mark.parametrize("matching", [None, "Bright", "Other"]) -def test_public_rescale_preserves_one_source_plane_through_stream_binding(method, matching): +def test_public_rescale_preserves_one_source_plane_through_stream_binding( + method, matching +): kwargs = { - "select_the_input_image": "Bright", "name_the_output_image": "Detection", - "rescale_method": method, "automatic_low": AutomaticLow.CUSTOM, - "automatic_high": AutomaticHigh.CUSTOM, "source_low": 0.0, - "source_high": 20.0, "divisor_value": 2.0, + "select_the_input_image": "Bright", + "name_the_output_image": "Detection", + "rescale_method": method, + "automatic_low": AutomaticLow.CUSTOM, + "automatic_high": AutomaticHigh.CUSTOM, + "source_low": 0.0, + "source_high": 20.0, + "divisor_value": 2.0, } if matching is not None: kwargs["select_image_to_match_in_maximum_intensity"] = matching step = FunctionStep( - func=(rescale_intensity, kwargs), name="SyntheticRescale", + func=(rescale_intensity, kwargs), + name="SyntheticRescale", source_bindings=StepSourceBindingsConfig( - enabled=True, bindings=(NamedSourceBinding("Bright"),), + enabled=True, + bindings=(NamedSourceBinding("Bright"),), ), ) invocation = _compile_public_step(step) @@ -74,18 +104,32 @@ def test_public_rescale_preserves_one_source_plane_through_stream_binding(method pixels = np.arange(20, dtype=np.float32).reshape(4, 5) metadata = ImagePayloadMetadata( source_path="/synthetic/A01_s1_w1_z1_t1.tif", - source_component_metadata={"well": "A01", "site": "1", "channel": "1", "z_index": "1", "timepoint": "1"}, + source_component_metadata={ + "well": "A01", + "site": "1", + "channel": "1", + "z_index": "1", + "timepoint": "1", + }, source_image_names=("Bright",), - source_voxel_spacing=SourceVoxelSpacing((1.0, 1.0, 1.0), SourceVoxelSpacingUnit.RELATIVE), - source_spatial_domain=SourceSpatialDomain(origin_yx=(0, 0), source_shape_yx=(4, 5)), + source_voxel_spacing=SourceVoxelSpacing( + (1.0, 1.0, 1.0), SourceVoxelSpacingUnit.RELATIVE + ), + source_spatial_domain=SourceSpatialDomain( + origin_yx=(0, 0), source_shape_yx=(4, 5) + ), ) source = metadata.payload_with(pixels, None) runtime = cellprofiler_runtime_adapter_for_test( - runtime_value_store=RuntimeValueStore(), callable_contract=contract, - axis_scope=RuntimeExecutionAxisScope.from_raw("synthetic", component=None, value=None), + runtime_value_store=RuntimeValueStore(), + callable_contract=contract, + axis_scope=RuntimeExecutionAxisScope.from_raw( + "synthetic", component=None, value=None + ), artifact_inputs={ edge.key: InvocationArtifactInputEdgePlan.from_source_declarations( - key=edge.key, spec=edge.spec, + key=edge.key, + spec=edge.spec, main_flow_artifacts=contract.artifact_inputs, invocation_sources=contract.artifact_inputs, ) @@ -94,50 +138,85 @@ def test_public_rescale_preserves_one_source_plane_through_stream_binding(method ) executor = CellProfilerModuleExecutor(rescale_intensity, contract) request = executor._image_request( - source, runtime, module_type=RescaleIntensityModule, + source, + runtime, + module_type=RescaleIntensityModule, active_input_specs=contract.artifact_inputs.specs, ) execution = executor._invocation_request( - image_request=request, adapter=runtime, current_image=source, + image_request=request, + adapter=runtime, + current_image=source, module_type=RescaleIntensityModule, - kwargs={"rescale_method": method, "automatic_low": AutomaticLow.CUSTOM, - "automatic_high": AutomaticHigh.CUSTOM, "source_low": 0.0, - "source_high": 20.0, "divisor_value": 2.0}, + kwargs={ + "rescale_method": method, + "automatic_low": AutomaticLow.CUSTOM, + "automatic_high": AutomaticHigh.CUSTOM, + "source_low": 0.0, + "source_high": 20.0, + "divisor_value": 2.0, + }, ) result = CellProfilerFunctionContractExecutor().execute( - contract, rescale_intensity, execution.payload, execution.kwargs, - execution_mode=execution.execution_mode, plane_projection=execution.plane_projection, + contract, + rescale_intensity, + execution.payload, + execution.kwargs, + execution_mode=execution.execution_mode, + plane_projection=execution.plane_projection, + ) + (item,) = tuple( + StreamOutputBatch.project_item( + RuntimeProjectionSourceIdentityRequest( + value=result, + source_description="/synthetic/Detection.tif", + ) + ) + ) + fields = StreamImagePayloadMetadataProjector.item_fields( + item.metadata, ("well", "site", "channel", "z_index", "timepoint") ) - (item,) = tuple(StreamOutputBatch.project_item(RuntimeProjectionSourceIdentityRequest( - value=result, source_description="/synthetic/Detection.tif", - ))) - fields = StreamImagePayloadMetadataProjector.item_fields(item.metadata, ("well", "site", "channel", "z_index", "timepoint")) native_item = NapariStreamLayerItem( data=item.data, producer=StreamProducerIdentity.pipeline_output( - output_kind="main", output_key="main", projection_key="main", - step_name="SyntheticRescale", pipeline_position=0, step_scope_id="synthetic", + output_kind="main", + output_key="main", + projection_key="main", + step_name="SyntheticRescale", + pipeline_position=0, + step_scope_id="synthetic", ), address=NapariStreamLayerAddress( components=dict(item.require_source_component_metadata()), - path="/synthetic/Detection.tif", stream_layer_data_type=StreamingDataType.IMAGE, + path="/synthetic/Detection.tif", + stream_layer_data_type=StreamingDataType.IMAGE, ), image_metadata=item.metadata, plane_component_domain=ViewerComponentValueDomainPayload.from_wire_mapping( - fields.get(ViewerWireField.PLANE_COMPONENT_VALUES.value, {}), context="synthetic producer fields", + fields.get(ViewerWireField.PLANE_COMPONENT_VALUES.value, {}), + context="synthetic producer fields", declared_axes=STREAM_AXES, ), ) # The exact original strict receiver is reached, not replaced by a mock. - NapariAggregateAxisBindingBuilder.bindings((native_item,), ViewerComponentAxisSemanticsFactory.empty()) + NapariAggregateAxisBindingBuilder.bindings( + (native_item,), ViewerComponentAxisSemanticsFactory.empty() + ) assert contract.artifact_inputs.names() == ("Bright",) assert request.image_count == 1 assert result.data.shape == (4, 5) assert result.metadata.source_spatial_domain == metadata.source_spatial_domain assert result.metadata.source_image_paths == metadata.source_image_paths assert result.metadata.source_voxel_spacing == metadata.source_voxel_spacing - assert SourceVoxelSpacing.common_physical_pixel_size((result.metadata.source_voxel_spacing,)) is None - assert item.require_source_component_metadata() == metadata.source_component_metadata + assert ( + SourceVoxelSpacing.common_physical_pixel_size( + (result.metadata.source_voxel_spacing,) + ) + is None + ) + assert ( + item.require_source_component_metadata() == metadata.source_component_metadata + ) expected = pixels / (2.0 if method is RescaleMethod.DIVIDE_BY_VALUE else 20.0) np.testing.assert_array_equal(item.data, expected) @@ -150,14 +229,17 @@ def test_declared_source_plane_has_area_not_volume(retained_site): metadata = ImagePayloadMetadata( source_path="/synthetic/site.tif", source_component_metadata={"site": "7"}, - source_spatial_domain=SourceSpatialDomain(origin_yx=(0, 0), source_shape_yx=(4, 5)), + source_spatial_domain=SourceSpatialDomain( + origin_yx=(0, 0), source_shape_yx=(4, 5) + ), ) if retained_site: pixels = pixels[None] labels = labels[None] metadata = ImagePayloadMetadata( source_image_provenance_planes=SourceImageProvenancePlanes.from_components( - paths=("/synthetic/site.tif",), component_metadata=({"site": "7"},), + paths=("/synthetic/site.tif",), + component_metadata=({"site": "7"},), ), source_spatial_domain=metadata.source_spatial_domain, plane_axis=RuntimePlaneAxis.RUNTIME_SLICE, @@ -169,19 +251,26 @@ def test_declared_source_plane_has_area_not_volume(retained_site): domain_scope=ObjectLabelDomainScope.PLANE if retained_site else None, plane_projection=( RuntimePlaneAxisValueProjection.from_source_declaration( - metadata.plane_axis, metadata.source_provenance, + metadata.plane_axis, + metadata.source_provenance, ) - if retained_site else None + if retained_site + else None ), ).payload() assert objects.plane_axis is ( RuntimePlaneAxis.RUNTIME_SLICE if retained_site else None ) assert objects.object_label_domain().scope is ( - ObjectLabelDomainScope.PLANE if retained_site else ObjectLabelDomainScope.PAYLOAD + ObjectLabelDomainScope.PLANE + if retained_site + else ObjectLabelDomainScope.PAYLOAD ) _, rows = measure_object_size_shape.__wrapped__( - payload, objects, calculate_advanced=False, calculate_zernikes=False, + payload, + objects, + calculate_advanced=False, + calculate_zernikes=False, ) fields = tuple(field.name for field in rows.fields) assert "Area" in fields @@ -194,10 +283,15 @@ def test_real_volume_is_not_squeezed_or_relabelled_as_area(): pixels = np.ones((3, 4, 5), dtype=np.float32) labels = np.zeros_like(pixels, dtype=np.int32) labels[1:, 1:3, 1:4] = 1 - payload = ImagePayloadMetadata(source_path="/synthetic/volume.tif").payload_with(pixels, None) + payload = ImagePayloadMetadata(source_path="/synthetic/volume.tif").payload_with( + pixels, None + ) objects = SourceImageObjectLabelBuildRequest(image=payload, labels=labels).payload() _, rows = measure_object_size_shape.__wrapped__( - payload, objects, calculate_advanced=False, calculate_zernikes=False, + payload, + objects, + calculate_advanced=False, + calculate_zernikes=False, ) fields = tuple(field.name for field in rows.fields) assert "Volume" in fields diff --git a/tests/unit/test_returned_output_abi.py b/tests/unit/test_returned_output_abi.py index 95a5f07151..2d3b8ca180 100644 --- a/tests/unit/test_returned_output_abi.py +++ b/tests/unit/test_returned_output_abi.py @@ -6,7 +6,11 @@ import pytest from openhcs.processing.backends.cellprofiler.crop import CropMask -from openhcs.core.aligned_image_payload import (AlignedImageSliceContext, AlignedImageStack, ImageOutputBundle) +from openhcs.core.aligned_image_payload import ( + AlignedImageSliceContext, + AlignedImageStack, + ImageOutputBundle, +) from openhcs.core.artifacts import ( ArtifactOutputPlan, ArtifactSpec, diff --git a/tests/unit/test_runner_cellprofiler_compatibility.py b/tests/unit/test_runner_cellprofiler_compatibility.py index 77728f2fdc..d4e7190c32 100644 --- a/tests/unit/test_runner_cellprofiler_compatibility.py +++ b/tests/unit/test_runner_cellprofiler_compatibility.py @@ -288,7 +288,9 @@ def test_native_reference_lookup_separates_public_partition_filter_scope( ) assert ( - native_cellprofiler_partition_filter_scope_slug(global_config.partition_filter_config) + native_cellprofiler_partition_filter_scope_slug( + global_config.partition_filter_config + ) == "wells_include_first1" ) assert location.output_dir == ( diff --git a/tests/unit/test_runtime_adapter_source_artifacts.py b/tests/unit/test_runtime_adapter_source_artifacts.py index 50105b5c0b..a3d6ed038f 100644 --- a/tests/unit/test_runtime_adapter_source_artifacts.py +++ b/tests/unit/test_runtime_adapter_source_artifacts.py @@ -130,7 +130,9 @@ def binding( ) # This opens the persisted projection; it does not ingest workspace files # as a new raw source folder. Physical inputs remain in source_root. - microscope_handler = DatasetSourceChoice.named("auto").open(workspace_root, filemanager=filemanager, source_bindings_config=source_bindings) + microscope_handler = DatasetSourceChoice.named("auto").open( + workspace_root, filemanager=filemanager, source_bindings_config=source_bindings + ) assert isinstance(microscope_handler, OpenHCSDatasetSource) microscope_handler.initialize_workspace(workspace_root, filemanager) projection_cache = VirtualWorkspaceSourceProjectionCache() @@ -144,8 +146,11 @@ def binding( frozenset((Microscopy.Channel,)) ), ) - context.runtime_source_workspace_projections = WorkspaceSourceProjections.from_context( - context, cache=projection_cache, + context.runtime_source_workspace_projections = ( + WorkspaceSourceProjections.from_context( + context, + cache=projection_cache, + ) ) projection = WorkspaceSourceProjections.from_context( context, @@ -229,9 +234,7 @@ def request() -> RuntimeAdapterRequest: labels_payload = labels_request.source_artifact_payload(labels_spec.ref()) label_set = CellProfilerOutputRecorder.for_artifact_type( ObjectLabelsArtifactType - ).raw_runtime_input_value( - spec=labels_spec, value=labels_payload - ) + ).raw_runtime_input_value(spec=labels_spec, value=labels_payload) assert isinstance(label_set, ObjectLabelSet) np.testing.assert_array_equal(label_set.labels, labels[np.newaxis, ...]) @@ -254,9 +257,7 @@ def request() -> RuntimeAdapterRequest: (illumination_payload, live_illumination), (labels_payload, live_labels), ): - np.testing.assert_array_equal( - produced.data, live.data - ) + np.testing.assert_array_equal(produced.data, live.data) produced_metadata = produced.metadata live_metadata = live.metadata assert produced_metadata == live_metadata @@ -274,6 +275,7 @@ def request() -> RuntimeAdapterRequest: def test_workspace_materialization_preserves_declared_source_pixels(tmp_path, mode): from PIL import Image from skimage.color import rgb2gray + source_root, workspace_root = tmp_path / "source", tmp_path / "workspace" source_root.mkdir() path = source_root / "A01_s1_w1_z001_t001.tif" @@ -282,26 +284,51 @@ def test_workspace_materialization_preserves_declared_source_pixels(tmp_path, mo pixels = pixels[..., 0] Image.fromarray(pixels).save(path) binding = NamedSourceBinding( - alias="Raw", load_as_monochrome=mode == "monochrome", load_as_mask=mode == "mask", + alias="Raw", + load_as_monochrome=mode == "monochrome", + load_as_mask=mode == "mask", ) bindings = SourceBindingsConfig(bindings=(binding,)) filemanager = _filemanager() - SourceBindingWorkspaceProjector(bindings, parser=SourceSchemaFilenameParser()).materialize( - source_root, workspace_root, filemanager=filemanager, - source_backend=Backend.DISK, workspace_backend=Backend.DISK, source_files=(path,), + SourceBindingWorkspaceProjector( + bindings, parser=SourceSchemaFilenameParser() + ).materialize( + source_root, + workspace_root, + filemanager=filemanager, + source_backend=Backend.DISK, + workspace_backend=Backend.DISK, + source_files=(path,), + ) + microscope = DatasetSourceChoice.named("auto").open( + workspace_root, filemanager=filemanager, source_bindings_config=bindings ) - microscope = DatasetSourceChoice.named("auto").open(workspace_root, filemanager=filemanager, source_bindings_config=bindings) microscope.initialize_workspace(workspace_root, filemanager) cache = VirtualWorkspaceSourceProjectionCache() context = SimpleNamespace( completed_step_outputs=StepExecutionObservation.empty(), - dataset_root=workspace_root, filemanager=filemanager, microscope_handler=microscope, + dataset_root=workspace_root, + filemanager=filemanager, + microscope_handler=microscope, + ) + workspace = WorkspaceSourceProjections.from_context( + context, cache=cache + ).projection_or_empty() + paths = tuple( + path + for path, _projection in workspace.source_occurrences_for_binding( + binding, axis_id="A01" + ) ) - workspace = WorkspaceSourceProjections.from_context(context, cache=cache).projection_or_empty() - paths = tuple(path for path, _projection in workspace.source_occurrences_for_binding(binding, axis_id="A01")) assert len(paths) == 1 - (payload,) = workspace.load_binding_payloads(paths, binding=binding, filemanager=filemanager) - expected = rgb2gray(pixels.astype(np.float32) / 255) if mode == "monochrome" else pixels.astype(bool) if mode == "mask" else pixels + (payload,) = workspace.load_binding_payloads( + paths, binding=binding, filemanager=filemanager + ) + expected = ( + rgb2gray(pixels.astype(np.float32) / 255) + if mode == "monochrome" + else pixels.astype(bool) if mode == "mask" else pixels + ) assert np.allclose(payload.data, expected) metadata = payload.metadata assert metadata.axis_position(ColourAxis) == (-1 if mode == "rgb" else None) diff --git a/tests/unit/test_runtime_context_declaration.py b/tests/unit/test_runtime_context_declaration.py index 8095b1a954..bbdeb131be 100644 --- a/tests/unit/test_runtime_context_declaration.py +++ b/tests/unit/test_runtime_context_declaration.py @@ -54,9 +54,9 @@ def test_absent_declaration_retains_signature_inference(): assert FunctionContractAttribute.runtime_context_parameter not in vars( _inferred_context ) - assert CallableContract.from_callable(_inferred_context).runtime_context_parameter == ( - "context" - ) + assert CallableContract.from_callable( + _inferred_context + ).runtime_context_parameter == ("context") def test_none_declaration_preserves_callable_identity_and_public_signature(): @@ -75,8 +75,11 @@ def process(image, *, context=None): @pytest.mark.parametrize( ("func", "expected"), - [(_inferred_context, "context"), (_without_context, None), - (_named_context, "execution_context")], + [ + (_inferred_context, "context"), + (_without_context, None), + (_named_context, "execution_context"), + ], ) def test_namespace_reconstruction_retains_captured_selection(func, expected): metadata = CallableMetadata.from_callable(func) @@ -96,9 +99,9 @@ def test_reference_and_pickle_reconstruction_retain_selection(func): reference = FunctionReferenceTransportAuthority.function_reference(func) restored_reference = pickle.loads(pickle.dumps(reference)) assert restored_reference.resolve() is func - assert CallableContract.from_callable(restored_reference).runtime_context_parameter == ( - expected - ) + assert CallableContract.from_callable( + restored_reference + ).runtime_context_parameter == (expected) contract = CallableContract.from_callable(restored_reference) assert pickle.loads(pickle.dumps(contract)).runtime_context_parameter == expected @@ -115,9 +118,12 @@ def process(image, *, context=None): assert contract.runtime_context_parameter is None assert "context" not in contract.runtime_owned_parameter_names assert "context" not in parameter_exclusions(process) - assert inspect.signature(contract.resolve_canonical_raw_callable()).parameters[ - "context" - ].default is None + assert ( + inspect.signature(contract.resolve_canonical_raw_callable()) + .parameters["context"] + .default + is None + ) @pytest.mark.parametrize( @@ -168,7 +174,10 @@ def test_save_images_variants_keep_independent_injection_declarations(): for func in (save_images, save_images_with_measurements): reference = FunctionReferenceTransportAuthority.function_reference(func) expected = CallableContract.from_callable(func).runtime_context_parameter - assert CallableContract.from_callable(reference).runtime_context_parameter == expected + assert ( + CallableContract.from_callable(reference).runtime_context_parameter + == expected + ) def test_measurement_variant_keeps_real_axis_scoped_filename_context(): @@ -202,8 +211,12 @@ def test_named_context_declaration_does_not_resolve_unrelated_forward_references assert inspect.signature(process).parameters["image"].annotation == ( "DefinedAfterDeclaration" ) - assert vars(process)[FunctionContractAttribute.runtime_context_parameter] == "context" - assert CallableMetadata.from_callable(process).runtime_context_parameter == "context" + assert ( + vars(process)[FunctionContractAttribute.runtime_context_parameter] == "context" + ) + assert ( + CallableMetadata.from_callable(process).runtime_context_parameter == "context" + ) marker = namespace["DefinedAfterDeclaration"]() assert process(marker, context=marker) == (marker, marker) diff --git a/tests/unit/test_runtime_equivalence.py b/tests/unit/test_runtime_equivalence.py index b4dcce0efd..195de14507 100644 --- a/tests/unit/test_runtime_equivalence.py +++ b/tests/unit/test_runtime_equivalence.py @@ -1,6 +1,8 @@ from __future__ import annotations -from openhcs.interop.cellprofiler.measurement_dialect import CellProfilerEquivalencePolicy +from openhcs.interop.cellprofiler.measurement_dialect import ( + CellProfilerEquivalencePolicy, +) from openhcs.core.measurement_dialect import PlainMeasurementDialect @@ -1496,7 +1498,9 @@ def test_runtime_execution_equivalence_detects_artifact_count_mismatch( RuntimeExportObservation.from_output_root(tmp_path / "candidate"), ) - report = runtime_artifact_execution_equivalence(reference, candidate, policy=RuntimeEquivalencePolicy()) + report = runtime_artifact_execution_equivalence( + reference, candidate, policy=RuntimeEquivalencePolicy() + ) assert report.failure_messages() == ( "runtime artifact counts differ: " @@ -11453,7 +11457,7 @@ class DeclaredIdentityDialect(CellProfilerMeasurementDialect): snapshot = RuntimeMeasurementSnapshot.from_output_snapshot( RuntimeOutputSnapshot(tables=(parents, children)), policy=policy ) - relationship, = snapshot.correlated_relationships.values() + (relationship,) = snapshot.correlated_relationships.values() assert relationship.source_keys == (ObjectInstanceKey(7, 0),) assert relationship.target_keys == (ObjectInstanceKey(11, 0),) diff --git a/tests/unit/test_runtime_equivalence_package.py b/tests/unit/test_runtime_equivalence_package.py index 194a3046e6..6961b884bd 100644 --- a/tests/unit/test_runtime_equivalence_package.py +++ b/tests/unit/test_runtime_equivalence_package.py @@ -1,4 +1,5 @@ from openhcs.core.measurement_dialect import PlainMeasurementDialect + """Runtime equivalence package boundary tests.""" import ast @@ -45,10 +46,10 @@ PROJECT_ROOT = Path(__file__).parents[2] - - def test_runtime_equivalence_policy_types_have_package_owner() -> None: - policy = RuntimeEquivalencePolicy(measurement_dialect=PlainMeasurementDialect.shared()) + policy = RuntimeEquivalencePolicy( + measurement_dialect=PlainMeasurementDialect.shared() + ) assert policy.measurement_dialect is not None assert normalize_runtime_identifier("MeanIntensity_OrigBlue") == ( @@ -180,8 +181,6 @@ def test_runtime_equivalence_cell_counter_comparison_has_package_owner() -> None ) - - def test_runtime_equivalence_array_payloads_use_canonical_equivalence_surface() -> None: array = np.array([[1, 2], [3, 4]], dtype=np.int16) diff --git a/tests/unit/test_runtime_execution_validation.py b/tests/unit/test_runtime_execution_validation.py index 6306a45e9a..ab8e4652d5 100644 --- a/tests/unit/test_runtime_execution_validation.py +++ b/tests/unit/test_runtime_execution_validation.py @@ -519,8 +519,7 @@ def test_runtime_execution_observation_reads_plate_export_from_exact_owner( assert observation.exports.table_outputs == (contracted_output,) assert expectation.artifact_kinds == frozenset((MeasurementsArtifactType,)) assert tuple( - (item.axis_id, item.artifact_kinds) - for item in expectation.axis_expectations + (item.axis_id, item.artifact_kinds) for item in expectation.axis_expectations ) == ( ("A01", frozenset((MeasurementsArtifactType,))), ("A02", frozenset()), diff --git a/tests/unit/test_runtime_image_singleton_alignment.py b/tests/unit/test_runtime_image_singleton_alignment.py index 0bdb70631c..86cfef04ad 100644 --- a/tests/unit/test_runtime_image_singleton_alignment.py +++ b/tests/unit/test_runtime_image_singleton_alignment.py @@ -3,7 +3,10 @@ import numpy as np import pytest -from openhcs.core.aligned_image_payload import (AlignedImageStack, compose_aligned_image_payload) +from openhcs.core.aligned_image_payload import ( + AlignedImageStack, + compose_aligned_image_payload, +) from openhcs.core.runtime_image_values import ( ImagePayloadMetadata, MaskedImagePayload, @@ -63,9 +66,7 @@ def test_scalar_image_aligns_with_nominal_singleton_owner( composition = compose_aligned_image_payload("Measurement consumer", payloads) - assert composition.execution_mode is ( - AlignedStackExecution - ) + assert composition.execution_mode is (AlignedStackExecution) assert isinstance(composition.payload, AlignedImageStack) assert len(composition.payload.slices) == 1 bundle = composition.payload.slices[0] @@ -115,9 +116,7 @@ def test_same_shaped_scalar_images_do_not_invent_runtime_alignment() -> None: assert composition.execution_mode is FullStackExecution assert not isinstance(composition.payload, AlignedImageStack) - assert composition.payload.metadata.plane_axis is ( - RuntimePlaneAxis.SOURCE_BINDING - ) + assert composition.payload.metadata.plane_axis is (RuntimePlaneAxis.SOURCE_BINDING) def test_singleton_runtime_projection_consumes_rgb_image_and_mask_axis_together() -> ( diff --git a/tests/unit/test_runtime_slice_projection.py b/tests/unit/test_runtime_slice_projection.py index 880f0c79cf..278014c5a9 100644 --- a/tests/unit/test_runtime_slice_projection.py +++ b/tests/unit/test_runtime_slice_projection.py @@ -7,7 +7,12 @@ from arraybridge.decorators import PRESERVE_INPUT_DTYPE_CONFIG import openhcs.processing.backends.cellprofiler # noqa: F401 -from openhcs.core.aligned_image_payload import (AlignedImageSliceContext, AlignedImageStack, ImageOutputBundle, ImagePayloadComposition) +from openhcs.core.aligned_image_payload import ( + AlignedImageSliceContext, + AlignedImageStack, + ImageOutputBundle, + ImagePayloadComposition, +) from openhcs.core.callable_contract import KeywordRuntimeParameter from openhcs.core.measurement_row_materialization import ( DataclassMeasurementColumnarRows, @@ -127,7 +132,9 @@ class UndeclaredValue: RuntimeSliceProjection.value_for_slice( UndeclaredValue(), RuntimePlaneAxisValueProjection.from_selected_plane( - axis=RuntimePlaneAxis.RUNTIME_SLICE, plane_index=0, axis_size=1, + axis=RuntimePlaneAxis.RUNTIME_SLICE, + plane_index=0, + axis_size=1, ), ) @@ -200,7 +207,9 @@ def test_object_label_payload_selects_nominal_object_label_projection() -> None: projected = RuntimeSliceProjection.value_for_slice( payload, RuntimePlaneAxisValueProjection.from_selected_plane( - axis=RuntimePlaneAxis.RUNTIME_SLICE, plane_index=0, axis_size=1, + axis=RuntimePlaneAxis.RUNTIME_SLICE, + plane_index=0, + axis_size=1, ), ) assert projected is payload @@ -656,9 +665,7 @@ def test_variable_component_projection_requires_declared_plane_provenance() -> N RuntimeSliceProjectionDeclarationError, match="nominal payload.*RUNTIME_SLICE", ): - OptionalSourceIdentity.project_payload_items( - request - ) + OptionalSourceIdentity.project_payload_items(request) def test_variable_component_projection_validates_declared_cardinality() -> None: @@ -676,9 +683,7 @@ def test_variable_component_projection_validates_declared_cardinality() -> None: ) with pytest.raises(ValueError, match="declared 'runtime_slice' axis of size 2"): - OptionalSourceIdentity.project_payload_items( - request - ) + OptionalSourceIdentity.project_payload_items(request) def test_variable_component_projection_rejects_expanded_source_provenance() -> None: @@ -699,9 +704,7 @@ def test_variable_component_projection_rejects_expanded_source_provenance() -> N ValueError, match="source provenance must exactly match.*9 != 3", ): - OptionalSourceIdentity.project_payload_items( - request - ) + OptionalSourceIdentity.project_payload_items(request) def test_variable_component_projection_uses_declared_plane_axis() -> None: @@ -710,17 +713,15 @@ def test_variable_component_projection_uses_declared_plane_axis() -> None: ) payload = _declared_plane_metadata(2).payload_with(stack, None) - projected = ( - OptionalSourceIdentity.project_payload_items( - RuntimeProjectionSourceIdentityRequest( - value=payload, - source_description="declared image stack", - variable_components=(Microscopy.Site,), - plane_projection=RuntimePlaneAxisValueProjection.preserve( - axis=RuntimePlaneAxis.RUNTIME_SLICE, - axis_size=2, - ), - ) + projected = OptionalSourceIdentity.project_payload_items( + RuntimeProjectionSourceIdentityRequest( + value=payload, + source_description="declared image stack", + variable_components=(Microscopy.Site,), + plane_projection=RuntimePlaneAxisValueProjection.preserve( + axis=RuntimePlaneAxis.RUNTIME_SLICE, + axis_size=2, + ), ) ) @@ -785,13 +786,11 @@ def test_source_identity_request_projects_explicit_source_binding_label_planes() ), ) - projected = ( - OptionalSourceIdentity.project_payload_items( - RuntimeProjectionSourceIdentityRequest( - value=labels, - source_description="source-bound labels", - plane_projection=labels.declared_plane_projection(), - ) + projected = OptionalSourceIdentity.project_payload_items( + RuntimeProjectionSourceIdentityRequest( + value=labels, + source_description="source-bound labels", + plane_projection=labels.declared_plane_projection(), ) ) diff --git a/tests/unit/test_runtime_value_store.py b/tests/unit/test_runtime_value_store.py index 3a5a54a771..0067a7fe58 100644 --- a/tests/unit/test_runtime_value_store.py +++ b/tests/unit/test_runtime_value_store.py @@ -70,9 +70,16 @@ ComponentSelector, NamedSourceBinding, ) -from openhcs.core.source_matching import SourceImageSetIdentityCompatibility, SourceImageSetIdentityPolicy -from openhcs.interop.cellprofiler.runtime.artifact_binding import RuntimeInputBindingRequest -from tests.unit.cellprofiler_runtime_test_support import cellprofiler_runtime_adapter_for_test +from openhcs.core.source_matching import ( + SourceImageSetIdentityCompatibility, + SourceImageSetIdentityPolicy, +) +from openhcs.interop.cellprofiler.runtime.artifact_binding import ( + RuntimeInputBindingRequest, +) +from tests.unit.cellprofiler_runtime_test_support import ( + cellprofiler_runtime_adapter_for_test, +) from openhcs.core.axes import Axis, AxisFamily from openhcs.domains.microscopy.axes import Microscopy @@ -213,10 +220,7 @@ def test_store_transport_excludes_derived_caches_and_retains_records(): restored = pickle.loads(serialized_without_cache) assert restored.revision == store.revision assert len(restored) == len(store) == 1 - assert ( - restored.observed_values[0].data.row_mappings() - == value.data.row_mappings() - ) + assert restored.observed_values[0].data.row_mappings() == value.data.row_mappings() assert ( restored.query_cache(RuntimeObjectLabelMeasurementQueryCache).cached_value( query @@ -929,7 +933,9 @@ def test_runtime_artifact_input_projection_transposes_producer_stack_axis() -> N group_plan, payload, execution_scope=RuntimeExecutionAxisScope.from_raw( - "A01", component=Microscopy.Channel, value=channel, + "A01", + component=Microscopy.Channel, + value=channel, ), ), path=group_plan.path, @@ -986,11 +992,15 @@ def test_artifact_candidate_scopes_preserve_projected_site_time_correlation() -> path = "/memory/processed_pixels.pkl" variables = (Microscopy.Site, Microscopy.Timepoint) output_plan = ArtifactOutputPlan( - name="image", path=path, artifact_type=ImageArtifactType, + name="image", + path=path, + artifact_type=ImageArtifactType, variable_components=variables, ) storage_plan = ArtifactInputPlan( - name="image", path=path, artifact_type=ImageArtifactType, + name="image", + path=path, + artifact_type=ImageArtifactType, variable_components=variables, ) payload = ImagePayloadMetadata( @@ -1007,8 +1017,11 @@ def test_artifact_candidate_scopes_preserve_projected_site_time_correlation() -> np.asarray([[[True, False], [False, True]], [[False, True], [True, False]]]), ) store = RuntimeValueStore() - store.record(RuntimeValue.normalize(output_plan, payload, axis_id="A01"), - path=path, backend="memory") + store.record( + RuntimeValue.normalize(output_plan, payload, axis_id="A01"), + path=path, + backend="memory", + ) edge = _runtime_input_edge( storage_plan, invocation_scope=ComponentGroupScope.dynamic(Microscopy.Site), @@ -1020,11 +1033,15 @@ def test_artifact_candidate_scopes_preserve_projected_site_time_correlation() -> consumer_variable_components=(), ) runtime_input = RuntimeArtifactInput( - edge_plan=edge, backend="memory", - axis_scope=RuntimeExecutionAxisScope.from_raw("A01", component=None, value=None), + edge_plan=edge, + backend="memory", + axis_scope=RuntimeExecutionAxisScope.from_raw( + "A01", component=None, value=None + ), ) candidates = runtime_input.candidate_execution_scopes( - store, ComponentGroupScope.dynamic(Microscopy.Site), + store, + ComponentGroupScope.dynamic(Microscopy.Site), variable_components=ComponentSet(), ) (coordinates,) = tuple(candidates) @@ -1036,30 +1053,51 @@ def test_artifact_candidate_scopes_preserve_projected_site_time_correlation() -> np.testing.assert_array_equal(selected.mask, payload.mask[0]) -def test_artifact_candidate_scopes_keep_actual_producer_group_as_fixed_context() -> None: +def test_artifact_candidate_scopes_keep_actual_producer_group_as_fixed_context() -> ( + None +): path = "/memory/produced_pixels.pkl" output_plan = ArtifactOutputPlan( - name="image", path=path, artifact_type=ImageArtifactType, - group_component=Microscopy.Channel, group_keys=("1",), + name="image", + path=path, + artifact_type=ImageArtifactType, + group_component=Microscopy.Channel, + group_keys=("1",), ) storage_plan = ArtifactInputPlan( - name="image", path=path, artifact_type=ImageArtifactType, - group_component=Microscopy.Channel, group_keys=("1",), + name="image", + path=path, + artifact_type=ImageArtifactType, + group_component=Microscopy.Channel, + group_keys=("1",), ) payload = ImagePayloadMetadata( source_path="/source/old_filename_w9.tif", - source_component_metadata={"site": "2", "timepoint": "3", "channel": "1", "z_index": "1"}, + source_component_metadata={ + "site": "2", + "timepoint": "3", + "channel": "1", + "z_index": "1", + }, ).payload_with(np.full((2, 2), 17.0), None) store = RuntimeValueStore() record = store.record( RuntimeValue.normalize_for_execution_scope( - output_plan, payload, + output_plan, + payload, execution_scope=RuntimeExecutionAxisScope.from_raw( - "A01", component=Microscopy.Channel, value="1", - fixed_component_values=((Microscopy.Site, "2"), (Microscopy.Timepoint, "3"), (Microscopy.ZIndex, "1")), + "A01", + component=Microscopy.Channel, + value="1", + fixed_component_values=( + (Microscopy.Site, "2"), + (Microscopy.Timepoint, "3"), + (Microscopy.ZIndex, "1"), + ), ), ), - path=path, backend="memory", + path=path, + backend="memory", ) runtime_input = RuntimeArtifactInput( edge_plan=_runtime_input_edge( @@ -1069,17 +1107,22 @@ def test_artifact_candidate_scopes_keep_actual_producer_group_as_fixed_context() component_scopes=(ComponentGroupScope.dynamic(Microscopy.Site),), consumer_variable_components=(), ), - axis_scope=RuntimeExecutionAxisScope.from_raw("A01", component=None, value=None), + axis_scope=RuntimeExecutionAxisScope.from_raw( + "A01", component=None, value=None + ), backend="memory", ) candidates = runtime_input.candidate_execution_scopes( - store, ComponentGroupScope.dynamic(Microscopy.Site), + store, + ComponentGroupScope.dynamic(Microscopy.Site), variable_components=ComponentSet(), ) (coordinates,) = tuple(candidates) assert coordinates.value_text == "2" assert dict(coordinates.fixed_component_values) == { - Microscopy.Channel: "1", Microscopy.Timepoint: "3", Microscopy.ZIndex: "1", + Microscopy.Channel: "1", + Microscopy.Timepoint: "3", + Microscopy.ZIndex: "1", } selected = replace(runtime_input, axis_scope=coordinates).records(store) assert len(selected) == 1 @@ -1087,26 +1130,39 @@ def test_artifact_candidate_scopes_keep_actual_producer_group_as_fixed_context() second_payload = ImagePayloadMetadata( source_path="/source/another_original.tif", - source_component_metadata={"site": "2", "timepoint": "3", "channel": "1", "z_index": "2"}, + source_component_metadata={ + "site": "2", + "timepoint": "3", + "channel": "1", + "z_index": "2", + }, ).payload_with(np.full((2, 2), 29.0), None) second_record = store.record( RuntimeValue.normalize_for_execution_scope( - output_plan, second_payload, + output_plan, + second_payload, execution_scope=RuntimeExecutionAxisScope.from_raw( - "A01", component=Microscopy.Channel, value="1", - fixed_component_values=((Microscopy.Site, "2"), (Microscopy.Timepoint, "3"), (Microscopy.ZIndex, "2")), + "A01", + component=Microscopy.Channel, + value="1", + fixed_component_values=( + (Microscopy.Site, "2"), + (Microscopy.Timepoint, "3"), + (Microscopy.ZIndex, "2"), + ), ), ), - path=path, backend="memory", + path=path, + backend="memory", ) candidates = runtime_input.candidate_execution_scopes( - store, ComponentGroupScope.dynamic(Microscopy.Site), + store, + ComponentGroupScope.dynamic(Microscopy.Site), variable_components=ComponentSet(), ) assert len(candidates) == 2 selected_records = tuple( - replace(runtime_input, axis_scope=scope).records(store) - for scope in candidates + replace(runtime_input, axis_scope=scope).records(store) for scope in candidates ) assert selected_records == ((record,), (second_record,)) @@ -1534,9 +1590,7 @@ def test_runtime_artifact_input_projection_uses_compiled_group_for_plane_scope() ) resolved = runtime_input.resolve_value(store) - np.testing.assert_array_equal( - resolved.data, payload.data - ) + np.testing.assert_array_equal(resolved.data, payload.data) def test_runtime_value_store_observation_cursor_returns_delta_only(): @@ -1736,7 +1790,9 @@ def _ungrouped_runtime_artifact_input( ) -def _paired_channel_label_input(*, producer_axis="A01", producer_values=(), producer_path=None): +def _paired_channel_label_input( + *, producer_axis="A01", producer_values=(), producer_path=None +): """One exact DNA producer consumed in its paired actin image-set context.""" fixed_values = { @@ -1761,7 +1817,9 @@ def _paired_channel_label_input(*, producer_axis="A01", producer_values=(), prod output_plan, ObjectLabelSet( name=storage_plan.name, - variant_data=ObjectLabelVariantData(labels=np.ones((2, 2), dtype=np.uint16)), + variant_data=ObjectLabelVariantData( + labels=np.ones((2, 2), dtype=np.uint16) + ), ), execution_scope=RuntimeExecutionAxisScope.from_raw( producer_axis, @@ -1784,10 +1842,12 @@ def _paired_channel_label_input(*, producer_axis="A01", producer_values=(), prod ), ) source_bindings = CompiledSourceBindingPlan( - bindings=(NamedSourceBinding( - alias="Actin", - component_identity=(ComponentSelector(Microscopy.Channel, "2"),), - ),), + bindings=( + NamedSourceBinding( + alias="Actin", + component_identity=(ComponentSelector(Microscopy.Channel, "2"),), + ), + ), ) runtime_input = _ungrouped_runtime_artifact_input( storage_plan, @@ -1799,9 +1859,13 @@ def _paired_channel_label_input(*, producer_axis="A01", producer_values=(), prod source=source_bindings.binding_declarations[0].input_spec().ref() ) ) - return store, record, replace( - runtime_input, - edge_plan=replace(runtime_input.edge_plan, spec=declared_spec), + return ( + store, + record, + replace( + runtime_input, + edge_plan=replace(runtime_input.edge_plan, spec=declared_spec), + ), ) @@ -1835,12 +1899,17 @@ def _grouped_label_input_with_distinct_consumer_channel(): output_plan, ObjectLabelSet( name="Cells", - variant_data=ObjectLabelVariantData(labels=np.ones((2, 2), dtype=np.uint16)), + variant_data=ObjectLabelVariantData( + labels=np.ones((2, 2), dtype=np.uint16) + ), ), execution_scope=RuntimeExecutionAxisScope.from_raw( - "W001", component=Microscopy.Channel, value="0", + "W001", + component=Microscopy.Channel, + value="0", fixed_component_values=( - (Microscopy.Site, "1"), (Microscopy.Timepoint, "1"), + (Microscopy.Site, "1"), + (Microscopy.Timepoint, "1"), ), ), ) @@ -1855,9 +1924,12 @@ def _grouped_label_input_with_distinct_consumer_channel(): consumer_variable_components=(Microscopy.ZIndex,), ), axis_scope=RuntimeExecutionAxisScope.from_raw( - "W001", component=None, value=None, + "W001", + component=None, + value=None, fixed_component_values=( - (Microscopy.Site, "1"), (Microscopy.Channel, "2"), + (Microscopy.Site, "1"), + (Microscopy.Channel, "2"), (Microscopy.Timepoint, "1"), ), ), @@ -1872,27 +1944,32 @@ def test_exact_producer_group_does_not_constrain_consumer_source_channel(): assert runtime_input.records(store) == (record,) assert runtime_input.resolve_value(store) is record.data candidates = runtime_input.candidate_execution_scopes( - store, ComponentGroupScope.ungrouped(), + store, + ComponentGroupScope.ungrouped(), variable_components=ComponentSet((Microscopy.ZIndex,)), ) (scope,) = candidates # Discovery still owns the producer's semantic channel, rather than adopting # a later consumer's independently selected source-image channel. assert dict(scope.fixed_component_values) == { - Microscopy.Site: "1", Microscopy.Channel: "0", + Microscopy.Site: "1", + Microscopy.Channel: "0", Microscopy.Timepoint: "1", } @pytest.mark.parametrize("component", [Microscopy.Site, Microscopy.Timepoint]) def test_exact_producer_group_preserves_shared_fixed_context_constraints(component): - store, _record, runtime_input = _grouped_label_input_with_distinct_consumer_channel() + store, _record, runtime_input = ( + _grouped_label_input_with_distinct_consumer_channel() + ) fixed_values = dict(runtime_input.axis_scope.fixed_component_values) fixed_values[component] = "2" wrong_context = replace( runtime_input, axis_scope=replace( - runtime_input.axis_scope, fixed_component_values=tuple(fixed_values.items()), + runtime_input.axis_scope, + fixed_component_values=tuple(fixed_values.items()), ), ) with pytest.raises(RuntimeError, match="Missing RuntimeValueStore record"): @@ -1901,7 +1978,9 @@ def test_exact_producer_group_preserves_shared_fixed_context_constraints(compone @pytest.mark.parametrize("mismatch", ["group", "well", "path", "fixed-z"]) def test_exact_producer_group_rejects_other_producer_or_fixed_plane(mismatch): - _store, record, runtime_input = _grouped_label_input_with_distinct_consumer_channel() + _store, record, runtime_input = ( + _grouped_label_input_with_distinct_consumer_channel() + ) scope = record.key.scope path = record.location.path if mismatch == "group": @@ -1912,8 +1991,13 @@ def test_exact_producer_group_rejects_other_producer_or_fixed_plane(mismatch): path = "/memory/other/Cells_channel_0.pkl" else: scope = RuntimeExecutionAxisScope.from_raw( - scope.axis_id, component=scope.component, value=scope.value, - fixed_component_values=(*scope.fixed_component_values, (Microscopy.ZIndex, "2")), + scope.axis_id, + component=scope.component, + value=scope.value, + fixed_component_values=( + *scope.fixed_component_values, + (Microscopy.ZIndex, "2"), + ), ) runtime_input = replace( runtime_input, @@ -1930,25 +2014,32 @@ def test_exact_producer_group_rejects_other_producer_or_fixed_plane(mismatch): store = RuntimeValueStore() store.record( replace(record, key=replace(record.key, scope=scope)), - path=path, backend=record.location.backend, + path=path, + backend=record.location.backend, ) with pytest.raises(RuntimeError, match="Missing RuntimeValueStore record"): runtime_input.records(store) def test_different_producer_group_axis_preserves_fixed_source_channel_constraint(): - _store, record, runtime_input = _grouped_label_input_with_distinct_consumer_channel() + _store, record, runtime_input = ( + _grouped_label_input_with_distinct_consumer_channel() + ) # SITE is now the selected producer group. CHANNEL remains a genuine fixed # image context and cannot use the selected-group exemption. scope = RuntimeExecutionAxisScope.from_raw( - "W001", component=Microscopy.Site, value="1", + "W001", + component=Microscopy.Site, + value="1", fixed_component_values=( - (Microscopy.Channel, "0"), (Microscopy.Timepoint, "1"), + (Microscopy.Channel, "0"), + (Microscopy.Timepoint, "1"), ), ) storage_plan = replace( runtime_input.edge_plan.storage_plan, - group_component=Microscopy.Site, group_keys=("1",), + group_component=Microscopy.Site, + group_keys=("1",), paths_by_group={"1": record.location.path}, ) runtime_input = replace( @@ -1965,7 +2056,8 @@ def test_different_producer_group_axis_preserves_fixed_source_channel_constraint store = RuntimeValueStore() store.record( replace(record, key=replace(record.key, scope=scope)), - path=record.location.path, backend=record.location.backend, + path=record.location.path, + backend=record.location.backend, ) with pytest.raises(RuntimeError, match="Missing RuntimeValueStore record"): runtime_input.records(store) @@ -2005,14 +2097,18 @@ def consume_labels(image, labels): assert projected.axis_scope is adapter.request.axis_scope assert projected.source_binding_plan is runtime_input.source_binding_plan assert projected.records(store) == (record,) - assert adapter.artifact_input_records("Nuclei", ObjectLabelsArtifactType) == (record,) + assert adapter.artifact_input_records("Nuclei", ObjectLabelsArtifactType) == ( + record, + ) request = RuntimeInputBindingRequest( adapter=adapter, kwargs={}, current_image=np.zeros((2, 2)) ) assert request.artifact_value(edge) is record.data -@pytest.mark.parametrize("component", [Microscopy.Site, Microscopy.ZIndex, Microscopy.Timepoint]) +@pytest.mark.parametrize( + "component", [Microscopy.Site, Microscopy.ZIndex, Microscopy.Timepoint] +) def test_paired_channel_input_rejects_other_context_coordinate(component): store, _record, runtime_input = _paired_channel_label_input( producer_values=((component, "2"),), @@ -2022,13 +2118,19 @@ def test_paired_channel_input_rejects_other_context_coordinate(component): runtime_input.records(store) -@pytest.mark.parametrize("producer_axis,producer_path", [ - ("B01", None), - ("A01", "/memory/other_producer/Nuclei.pkl"), -]) -def test_paired_channel_input_rejects_other_well_or_producer(producer_axis, producer_path): +@pytest.mark.parametrize( + "producer_axis,producer_path", + [ + ("B01", None), + ("A01", "/memory/other_producer/Nuclei.pkl"), + ], +) +def test_paired_channel_input_rejects_other_well_or_producer( + producer_axis, producer_path +): store, _record, runtime_input = _paired_channel_label_input( - producer_axis=producer_axis, producer_path=producer_path, + producer_axis=producer_axis, + producer_path=producer_path, ) with pytest.raises(RuntimeError, match="Missing RuntimeValueStore record"): @@ -2038,7 +2140,8 @@ def test_paired_channel_input_rejects_other_well_or_producer(producer_axis, prod def test_cross_channel_input_without_source_declaration_remains_exact(): store, _record, runtime_input = _paired_channel_label_input() strict_input = replace( - runtime_input, source_binding_plan=CompiledSourceBindingPlan.empty(), + runtime_input, + source_binding_plan=CompiledSourceBindingPlan.empty(), ) with pytest.raises(RuntimeError, match="Missing RuntimeValueStore record"): @@ -2067,9 +2170,11 @@ def test_context_relation_to_another_source_does_not_borrow_visible_plane_member runtime_input.edge_plan, spec=replace( runtime_input.edge_plan.spec, - relations=(InputGroupLineageSourceRelation( - ArtifactSpec.input("UnrelatedImage", ImageArtifactType).ref() - ),), + relations=( + InputGroupLineageSourceRelation( + ArtifactSpec.input("UnrelatedImage", ImageArtifactType).ref() + ), + ), ), ), ) @@ -2089,7 +2194,9 @@ def test_stored_producer_context_retains_its_declared_source_membership(): spec=replace(edge.spec, relations=()), storage_plan=replace( edge.storage_plan, - relations=(SourceStackLineageSourceRelation(binding.input_spec().ref()),), + relations=( + SourceStackLineageSourceRelation(binding.input_spec().ref()), + ), ), ), ) @@ -2108,12 +2215,19 @@ def test_explicit_consumer_context_does_not_borrow_stored_producer_membership(): runtime_input, edge_plan=replace( edge, - spec=replace(edge.spec, relations=(InputGroupLineageSourceRelation( - ArtifactSpec.input("UnrelatedImage", ImageArtifactType).ref(), - ),)), + spec=replace( + edge.spec, + relations=( + InputGroupLineageSourceRelation( + ArtifactSpec.input("UnrelatedImage", ImageArtifactType).ref(), + ), + ), + ), storage_plan=replace( edge.storage_plan, - relations=(SourceStackLineageSourceRelation(binding.input_spec().ref()),), + relations=( + SourceStackLineageSourceRelation(binding.input_spec().ref()), + ), ), ), ) @@ -2133,16 +2247,22 @@ def test_paired_channel_projection_rejects_a_different_producer_site_plane(): ), ) store.replace( - replace(record, data=payload), path=record.location.path, backend=record.location.backend + replace(record, data=payload), + path=record.location.path, + backend=record.location.backend, ) projected_input = replace( runtime_input, edge_plan=replace( edge, - storage_plan=replace(edge.storage_plan, variable_components=(Microscopy.Site,)), + storage_plan=replace( + edge.storage_plan, variable_components=(Microscopy.Site,) + ), projection=replace( edge.projection, - component_scopes=(ComponentGroupScope(("1",), component=Microscopy.Site),), + component_scopes=( + ComponentGroupScope(("1",), component=Microscopy.Site), + ), ), ), ) @@ -2156,7 +2276,11 @@ def test_paired_channel_input_rejects_ambiguous_address_matched_contexts(): _other_store, other_record, _other_input = _paired_channel_label_input( producer_values=((Microscopy.Channel, "3"),), ) - store.record(other_record, path=other_record.location.path, backend=other_record.location.backend) + store.record( + other_record, + path=other_record.location.path, + backend=other_record.location.backend, + ) with pytest.raises(RuntimeError, match="Ambiguous RuntimeValueStore records"): runtime_input.records(store) @@ -2170,23 +2294,27 @@ def test_runtime_artifact_input_preserves_same_scope_semantic_partitions(): ) -@pytest.mark.parametrize("producer_values,consumer_values", [ - pytest.param((), (), id="exact-unscoped"), - pytest.param((), ((Microscopy.ZIndex, "1"),), id="consumer-only"), - pytest.param(((Microscopy.Timepoint, "1"),), (), id="producer-only"), - pytest.param( - ((Microscopy.Timepoint, "1"),), - ((Microscopy.ZIndex, "1"),), - id="disjoint-partial", - ), - pytest.param( - ((Microscopy.Timepoint, "1"),), - ((Microscopy.ZIndex, "1"), (Microscopy.Timepoint, "1")), - id="shared-partial", - ), -]) +@pytest.mark.parametrize( + "producer_values,consumer_values", + [ + pytest.param((), (), id="exact-unscoped"), + pytest.param((), ((Microscopy.ZIndex, "1"),), id="consumer-only"), + pytest.param(((Microscopy.Timepoint, "1"),), (), id="producer-only"), + pytest.param( + ((Microscopy.Timepoint, "1"),), + ((Microscopy.ZIndex, "1"),), + id="disjoint-partial", + ), + pytest.param( + ((Microscopy.Timepoint, "1"),), + ((Microscopy.ZIndex, "1"), (Microscopy.Timepoint, "1")), + id="shared-partial", + ), + ], +) def test_runtime_artifact_input_accepts_exact_unscoped_and_partial_coordinates( - producer_values, consumer_values, + producer_values, + consumer_values, ): store = RuntimeValueStore() storage_plan = ArtifactInputPlan( @@ -2230,23 +2358,30 @@ def test_runtime_artifact_input_accepts_exact_unscoped_and_partial_coordinates( ).records(store) == (record,) -@pytest.mark.parametrize("producer_values,consumer_values", [ - pytest.param((), (), id="empty-projected-context"), - pytest.param(((Microscopy.Timepoint, "1"),), (), id="producer-only"), - pytest.param((), ((Microscopy.ZIndex, "1"),), id="consumer-only"), - pytest.param( - ((Microscopy.Timepoint, "1"),), - ((Microscopy.ZIndex, "1"),), - id="disjoint-partial", - ), -]) +@pytest.mark.parametrize( + "producer_values,consumer_values", + [ + pytest.param((), (), id="empty-projected-context"), + pytest.param(((Microscopy.Timepoint, "1"),), (), id="producer-only"), + pytest.param((), ((Microscopy.ZIndex, "1"),), id="consumer-only"), + pytest.param( + ((Microscopy.Timepoint, "1"),), + ((Microscopy.ZIndex, "1"),), + id="disjoint-partial", + ), + ], +) def test_exact_input_admits_no_shared_projected_context_constraints( - producer_values, consumer_values, + producer_values, + consumer_values, ): _store, original, original_input = _paired_channel_label_input() storage_plan = original_input.edge_plan.storage_plan producer_scope = RuntimeExecutionAxisScope.from_raw( - "A01", component=None, value=None, fixed_component_values=producer_values, + "A01", + component=None, + value=None, + fixed_component_values=producer_values, ) store = RuntimeValueStore() record = store.record( @@ -2255,17 +2390,24 @@ def test_exact_input_admits_no_shared_projected_context_constraints( data=original.data, materialization_source_metadata=original.materialization_source_metadata, ), - path=original.location.path, backend=original.location.backend, + path=original.location.path, + backend=original.location.backend, ) source = NamedSourceBinding( alias="Reference", - component_identity=(ComponentSelector(AxisFamily.active().partition_axis(), "A01"),), + component_identity=( + ComponentSelector(AxisFamily.active().partition_axis(), "A01"), + ), ) consumer_scope = RuntimeExecutionAxisScope.from_raw( - "A01", component=None, value=None, fixed_component_values=consumer_values, + "A01", + component=None, + value=None, + fixed_component_values=consumer_values, ) runtime_input = _ungrouped_runtime_artifact_input( - storage_plan, axis_scope=consumer_scope, + storage_plan, + axis_scope=consumer_scope, source_binding_plan=CompiledSourceBindingPlan(bindings=(source,)), ) runtime_input = replace( @@ -2281,8 +2423,12 @@ def test_exact_input_admits_no_shared_projected_context_constraints( # Plane-identity compatibility still requires evidence of a shared identity. # Exact artifact admission instead has a proven producer/address and only # checks the additional coordinates constrained on both sides. - assert runtime_input.edge_plan.spec.source_context_sources() == (source.input_spec().ref(),) - policy = SourceImageSetIdentityPolicy.from_source_bindings(runtime_input.source_binding_plan) + assert runtime_input.edge_plan.spec.source_context_sources() == ( + source.input_spec().ref(), + ) + policy = SourceImageSetIdentityPolicy.from_source_bindings( + runtime_input.source_binding_plan + ) components = ComponentSet.collect( (component for component, _value in producer_scope.source_component_values), (component for component, _value in consumer_scope.source_component_values), @@ -2293,7 +2439,8 @@ def test_exact_input_admits_no_shared_projected_context_constraints( ).matches() assert runtime_input.records(store) == (record,) wrong_well = replace( - runtime_input, axis_scope=replace(consumer_scope, axis_id="B01"), + runtime_input, + axis_scope=replace(consumer_scope, axis_id="B01"), ) wrong_producer = replace( runtime_input, @@ -2410,14 +2557,14 @@ def test_runtime_value_store_merges_observed_records_from_worker_boundary(): def test_runtime_measurement_observation_axis_accepts_table_record_once(): value = _runtime_value() record = StoredRuntimeValue( - key=value.key, - data=value.data, - materialization_source_metadata=value.materialization_source_metadata, - location=RuntimeArtifactLocation( + key=value.key, + data=value.data, + materialization_source_metadata=value.materialization_source_metadata, + location=RuntimeArtifactLocation( path="/memory/measurements.pkl", backend="memory", ), - ) + ) axis = RuntimeMeasurementObservationAxis("A01") axis.accept_measurement_table(record, PlainMeasurementDialect.shared()) @@ -2454,7 +2601,10 @@ def test_dynamic_input_query_distinguishes_compiled_paths_with_same_backend(): path=record.location.path, artifact_type=MeasurementsArtifactType, group_component=Microscopy.Channel, - paths_by_group={None: record.location.path, "DAPI": record.location.path}, + paths_by_group={ + None: record.location.path, + "DAPI": record.location.path, + }, ), axis_id="A01", backend="memory", @@ -2582,9 +2732,7 @@ def test_store_transport_excludes_all_derived_lookup_caches(): {"object_id": (1,)}, fields=(FieldSpec("object_id", int),) ) record = store.record(value, path="/memory/measurements.pkl", backend="memory") - latest = store.replace( - value, path="/later/measurements.pkl", backend="memory" - ) + latest = store.replace(value, path="/later/measurements.pkl", backend="memory") original_transport = pickle.dumps(store, protocol=5) query = RuntimeArtifactQuery.from_input_plan( ArtifactInputPlan( @@ -2626,7 +2774,9 @@ def test_unified_store_cache_keeps_both_query_domains_and_empty_results(): from metaclass_registry.caches import BoundedCache store = RuntimeValueStore() - record = store.record(_runtime_value(), path="/memory/measurements.pkl", backend="memory") + record = store.record( + _runtime_value(), path="/memory/measurements.pkl", backend="memory" + ) query = RuntimeArtifactQuery.from_input_plan( ArtifactInputPlan( name="measurements", @@ -2694,7 +2844,8 @@ def test_changed_stored_measurement_subject_requires_a_transient_value(): store = RuntimeValueStore() record = store.record(value, path="/memory/measurements.pkl", backend="memory") plan = ArtifactOutputPlan( - name=value.name, artifact_type=value.artifact_type, + name=value.name, + artifact_type=value.artifact_type, path="/memory/measurements.pkl", ) record.data.subject = MeasurementSubject(MeasurementScope.SAMPLE, "NextImage") @@ -2707,6 +2858,13 @@ def test_changed_stored_measurement_subject_requires_a_transient_value(): assert store.get(record.key) is record assert record.location.path == "/memory/measurements.pkl" assert admitted.data is record.data - assert type(StoredRuntimeValue.from_output_plan( - plan, admitted.data, execution_scope=record.key.scope, - )) is RuntimeValue + assert ( + type( + StoredRuntimeValue.from_output_plan( + plan, + admitted.data, + execution_scope=record.key.scope, + ) + ) + is RuntimeValue + ) diff --git a/tests/unit/test_runtime_values.py b/tests/unit/test_runtime_values.py index ef7e293a2f..e44d7450a2 100644 --- a/tests/unit/test_runtime_values.py +++ b/tests/unit/test_runtime_values.py @@ -240,7 +240,9 @@ def test_object_label_set_rejects_nested_payload_and_implicit_authority_merge() labels = np.array([[0, 1], [2, 0]], dtype=np.int16) unedited = np.array([[0, 1], [1, 0]], dtype=np.int16) payload = ObjectLabelPayload( - variant_data=ObjectLabelVariantData(labels=labels, variants={UneditedLabels: unedited}), + variant_data=ObjectLabelVariantData( + labels=labels, variants={UneditedLabels: unedited} + ), domain=ObjectLabelDomain( declared_object_ids=(1, 2), ), @@ -396,8 +398,9 @@ def test_object_label_pure_2d_aggregator_preserves_dense_payload_domains() -> No assert aggregated.domain.declared_object_id_domains == ((1,), (2,)) - -def test_object_label_aggregation_preserves_variant_plane_writes_and_transport() -> None: +def test_object_label_aggregation_preserves_variant_plane_writes_and_transport() -> ( + None +): planes = (np.array([[0, 1]], dtype=np.int16), np.array([[0, 2]], dtype=np.int32)) values = tuple( ObjectLabelPayload( @@ -409,7 +412,9 @@ def test_object_label_aggregation_preserves_variant_plane_writes_and_transport() for index, plane in enumerate(planes) ) aggregate = ObjectLabelPure2DSliceAggregator.aggregate( - values, "numpy", plane_axis=RuntimePlaneAxis.RUNTIME_SLICE, + values, + "numpy", + plane_axis=RuntimePlaneAxis.RUNTIME_SLICE, ) assert aggregate.shape == (2, 1, 2) assert aggregate.dtype == np.dtype(np.int32) @@ -939,7 +944,9 @@ def test_object_label_source_plane_projection_preserves_projected_variants() -> projected = source.project_source_plane(1, labels=labels[1]) np.testing.assert_array_equal(projected.labels, labels[1]) - np.testing.assert_array_equal(projected.variant_labels(SmallRemovedLabels), small_removed[1]) + np.testing.assert_array_equal( + projected.variant_labels(SmallRemovedLabels), small_removed[1] + ) assert projected.domain.scope is ObjectLabelDomainScope.PAYLOAD assert projected.domain.declared_object_ids == (2, 3) assert projected.source_image_names == ("rawGFP",) @@ -959,7 +966,9 @@ def test_object_label_replacement_identity_preserves_planar_variants() -> None: projected = source.with_measurement_labels(labels) np.testing.assert_array_equal(projected.labels, labels) - np.testing.assert_array_equal(projected.variant_labels(SmallRemovedLabels), small_removed) + np.testing.assert_array_equal( + projected.variant_labels(SmallRemovedLabels), small_removed + ) def test_object_label_plane_projection_preserves_ordered_domains_and_variants() -> None: @@ -1138,9 +1147,7 @@ def test_empty_sparse_ijv_stack_preserves_runtime_slice_count() -> None: np.zeros((2, *shape), dtype=np.int32), ) - projected_items = ( - OptionalSourceIdentity - ).project_payload_items( + projected_items = (OptionalSourceIdentity).project_payload_items( RuntimeProjectionSourceIdentityRequest( value=aggregated, source_description="empty sparse labels", @@ -1203,7 +1210,9 @@ def test_shape_object_feature_table_rejects_undeclared_dense_feature_domain() -> table.rows() -def test_shape_descriptors_align_with_measured_ids_not_missing_declared_labels() -> None: +def test_shape_descriptors_align_with_measured_ids_not_missing_declared_labels() -> ( + None +): table = ShapeObjectFeatureValueTable.from_feature_arrays( { MeasureObjectSizeShapeModule.MeasurementFeature.MAX_FERET_DIAMETER.value: np.asarray( @@ -1527,11 +1536,9 @@ def test_source_context_preserves_target_payload_channel_axis_domain() -> None: plane_axis=RuntimePlaneAxis.RUNTIME_SLICE, ).payload_with(np.zeros((1, 2, 4, 5, 3), dtype=np.float32), None) - contextualized = ( - target.metadata - .with_source_context_from(source.metadata) - .attach_source_context_to(target) - ) + contextualized = target.metadata.with_source_context_from( + source.metadata + ).attach_source_context_to(target) metadata = contextualized.metadata assert metadata.axis_index(ColourAxis, contextualized) == 4 @@ -2527,7 +2534,8 @@ def test_object_label_payload_with_measurement_labels_preserves_domain_and_varia small_removed = np.full_like(labels, 2) payload = ObjectLabelPayload( variant_data=ObjectLabelVariantData( - labels=labels, variants={UneditedLabels: unedited, SmallRemovedLabels: small_removed} + labels=labels, + variants={UneditedLabels: unedited, SmallRemovedLabels: small_removed}, ), source_spatial_domain=SourceSpatialDomain( origin_yx=(4, 5), @@ -2587,9 +2595,7 @@ def test_runtime_projection_requirement_projects_singleton_object_label_stack() ), ) - (item,) = ( - OptionalSourceIdentity - ).project_payload_items( + (item,) = (OptionalSourceIdentity).project_payload_items( RuntimeProjectionSourceIdentityRequest( value=payload, source_description="singleton labels", @@ -2624,9 +2630,7 @@ def test_runtime_projection_requirement_tracks_multi_plane_runtime_coordinates() ), ) - first, second = ( - OptionalSourceIdentity - ).project_payload_items( + first, second = (OptionalSourceIdentity).project_payload_items( RuntimeProjectionSourceIdentityRequest( value=payload, source_description="labels", @@ -2669,9 +2673,7 @@ def test_runtime_projection_does_not_expand_indexed_scalar_source_metadata() -> }, ).payload_with(np.arange(12, dtype=np.uint16).reshape(3, 2, 2), None) - projected = ( - RequiredSourceComponentMetadata - ).project_payload_items( + projected = (RequiredSourceComponentMetadata).project_payload_items( RuntimeProjectionSourceIdentityRequest( value=payload, source_description=source_path, @@ -2696,9 +2698,7 @@ def test_runtime_projection_keeps_unindexed_scalar_volume_as_one_payload() -> No }, ).payload_with(np.arange(8, dtype=np.uint16).reshape(2, 2, 2), None) - (projected,) = ( - RequiredSourceComponentMetadata - ).project_payload_items( + (projected,) = (RequiredSourceComponentMetadata).project_payload_items( RuntimeProjectionSourceIdentityRequest( value=payload, source_description=source_path, @@ -2945,9 +2945,7 @@ def test_group_scoped_stack_artifact_preserves_declared_payload_axis() -> None: ) assert value.data.data.shape == (2, 4, 5) - assert ( - value.data.metadata.plane_axis is RuntimePlaneAxis.RUNTIME_SLICE - ) + assert value.data.metadata.plane_axis is RuntimePlaneAxis.RUNTIME_SLICE def test_group_scoped_measurements_preserve_runtime_row_axis() -> None: @@ -3321,12 +3319,14 @@ def test_spatial_grid_normalizes_pure_2d_sequence_from_canonical_mappings(): value = RuntimeValue.normalize(output_plan, list(grids), axis_id="A01") assert isinstance(value.data, RuntimeSliceAlignedValues) - assert [ - value.data.value_at(slice_index).name for slice_index in range(2) - ] == ["Grid", "Grid"] - assert [ - value.data.value_at(slice_index).x_origin for slice_index in range(2) - ] == [1.0, 2.0] + assert [value.data.value_at(slice_index).name for slice_index in range(2)] == [ + "Grid", + "Grid", + ] + assert [value.data.value_at(slice_index).x_origin for slice_index in range(2)] == [ + 1.0, + 2.0, + ] assert [mapping["x_origin"] for mapping in value.materialization_payload()] == [ 1.0, 2.0, @@ -3540,16 +3540,14 @@ def test_derived_image_payload_context_preserves_declared_resized_spatial_domain source_shape_yx=(4, 5), ) ).payload_with(np.zeros((4, 5), dtype=np.float32), None) - output = ( - source.metadata - .with_spatial_resize((4, 15)) - .payload_with( - np.ones((4, 15), dtype=np.float32), - None, - ) + output = source.metadata.with_spatial_resize((4, 15)).payload_with( + np.ones((4, 15), dtype=np.float32), + None, ) - result = source.metadata.derive_payload(ImagePayload.of(source), ImagePayload.of(output)) + result = source.metadata.derive_payload( + ImagePayload.of(source), ImagePayload.of(output) + ) assert result.metadata.source_spatial_domain == SourceSpatialDomain( origin_yx=(0, 0), @@ -3580,7 +3578,9 @@ def test_derived_image_payload_context_merges_source_provenance_into_output_meta metadata=ImagePayloadMetadata(source_dtype="float32"), ) - result = source.metadata.derive_payload(ImagePayload.of(source), ImagePayload.of(output)) + result = source.metadata.derive_payload( + ImagePayload.of(source), ImagePayload.of(output) + ) result_metadata = result.metadata assert result_metadata.source_dtype == "float32" @@ -3677,7 +3677,9 @@ def test_derived_image_payload_context_replaces_stale_scalar_source_with_planes( ), ) - result = source.metadata.derive_payload(ImagePayload.of(source), ImagePayload.of(output)) + result = source.metadata.derive_payload( + ImagePayload.of(source), ImagePayload.of(output) + ) metadata = result.metadata assert metadata.source_path is None @@ -3731,7 +3733,9 @@ def test_derived_image_payload_context_preserves_output_name_while_replacing_sta ), ) - result = source.metadata.derive_payload(ImagePayload.of(source), ImagePayload.of(output)) + result = source.metadata.derive_payload( + ImagePayload.of(source), ImagePayload.of(output) + ) metadata = result.metadata assert metadata.source_image_provenance_planes.paths == ( @@ -3768,7 +3772,9 @@ def test_derived_image_payload_context_preserves_output_name_while_replacing_sta ), ) - result = source.metadata.derive_payload(ImagePayload.of(source), ImagePayload.of(output)) + result = source.metadata.derive_payload( + ImagePayload.of(source), ImagePayload.of(output) + ) metadata = result.metadata assert metadata.source_path == "/input/A01_s001_w3_z001_t001.tif" @@ -3817,7 +3823,9 @@ def test_derived_image_payload_context_preserves_output_name_while_replacing_sin ), ) - result = source.metadata.derive_payload(ImagePayload.of(source), ImagePayload.of(output)) + result = source.metadata.derive_payload( + ImagePayload.of(source), ImagePayload.of(output) + ) metadata = result.metadata assert metadata.source_path == "/input/Sequence1_s001_w1_z001_t007.tif" @@ -3858,7 +3866,9 @@ def test_derived_image_payload_context_expands_indexed_scalar_source_metadata_fo ) output = np.ones((3, 4, 5), dtype=np.float32) - result = source.metadata.derive_payload(ImagePayload.of(source), ImagePayload.of(output)) + result = source.metadata.derive_payload( + ImagePayload.of(source), ImagePayload.of(output) + ) metadata = result.metadata assert metadata.source_image_provenance_planes.paths == ( @@ -3997,7 +4007,9 @@ def test_derived_image_payload_context_uses_source_provenance_atomically() -> No ), ) - result = source.metadata.derive_payload(ImagePayload.of(source), ImagePayload.of(output)) + result = source.metadata.derive_payload( + ImagePayload.of(source), ImagePayload.of(output) + ) metadata = result.metadata assert metadata.source_path is None @@ -4470,9 +4482,7 @@ def test_object_label_runtime_slice_projection_projects_source_path() -> None: "channel": "1", } assert projected.source_image_provenance_planes.paths == () - assert projected.metadata.source_path == ( - "/input/A01_s002_w1_z001_t001.TIF" - ) + assert projected.metadata.source_path == ("/input/A01_s002_w1_z001_t001.TIF") def test_runtime_slice_projection_does_not_infer_raw_array_axis() -> None: @@ -4528,9 +4538,7 @@ def test_object_label_set_runtime_slice_projection_projects_source_path() -> Non "channel": "1", } assert projected.source_image_provenance_planes.paths == () - assert projected.metadata.source_path == ( - "/input/A01_s003_w1_z001_t001.TIF" - ) + assert projected.metadata.source_path == ("/input/A01_s003_w1_z001_t001.TIF") def test_measurement_table_runtime_slice_projection_projects_source_identity() -> None: @@ -4671,9 +4679,7 @@ def test_pure_2d_auxiliary_aggregator_preserves_image_payload_metadata() -> None # PURE2D image stacks stay deferred slices (70a79fa38); any ImagePayload. assert isinstance(stacked, ImagePayload) assert stacked.data.shape == (2, 2, 3) - assert ( - stacked.metadata.for_source_plane(0).intensity_scale == 65535.0 - ) + assert stacked.metadata.for_source_plane(0).intensity_scale == 65535.0 assert stacked.metadata.for_source_plane(1).source_dtype == "uint8" @@ -4749,8 +4755,7 @@ def test_pure_2d_auxiliary_aggregator_preserves_slice_aligned_spatial_grids() -> assert isinstance(aggregated, RuntimeSliceAlignedValues) assert tuple( - aggregated.value_at(index).x_origin - for index in range(aggregated.slice_count) + aggregated.value_at(index).x_origin for index in range(aggregated.slice_count) ) == (1.0, 3.0) @@ -4969,7 +4974,10 @@ def test_normalize_object_label_set_preserves_dense_label_variants(): name="Nuclei", variant_data=ObjectLabelVariantData( labels=labels, - variants={UneditedLabels: unedited_labels, SmallRemovedLabels: small_removed_labels}, + variants={ + UneditedLabels: unedited_labels, + SmallRemovedLabels: small_removed_labels, + }, ), ), axis_id="A01", @@ -4982,8 +4990,12 @@ def test_normalize_object_label_set_preserves_dense_label_variants(): UneditedLabels, ) np.testing.assert_array_equal(restored.labels, labels) - np.testing.assert_array_equal(restored.variant_labels(UneditedLabels), unedited_labels) - np.testing.assert_array_equal(restored.variant_labels(SmallRemovedLabels), small_removed_labels) + np.testing.assert_array_equal( + restored.variant_labels(UneditedLabels), unedited_labels + ) + np.testing.assert_array_equal( + restored.variant_labels(SmallRemovedLabels), small_removed_labels + ) def test_normalize_object_label_set_accepts_sparse_ijv_representation(): diff --git a/tests/unit/test_saved_source_label_plane_domain.py b/tests/unit/test_saved_source_label_plane_domain.py index 3cf532baf7..8c96cb52db 100644 --- a/tests/unit/test_saved_source_label_plane_domain.py +++ b/tests/unit/test_saved_source_label_plane_domain.py @@ -9,54 +9,81 @@ from openhcs.core.callable_contract import CallableContract from openhcs.core.runtime_image_values import ImagePayloadMetadata from openhcs.core.runtime_measurements import MeasurementRowAxisField -from openhcs.core.runtime_object_label_building import SourceImageObjectLabelBuildRequest +from openhcs.core.runtime_object_label_building import ( + SourceImageObjectLabelBuildRequest, +) from openhcs.core.runtime_object_label_domains import ObjectLabelDomainScope from openhcs.core.runtime_tabular_values import MeasurementObjectRowIdentity -from openhcs.core.runtime_plane_projection import RuntimePlaneAxis, RuntimePlaneAxisValueProjection +from openhcs.core.runtime_plane_projection import ( + RuntimePlaneAxis, + RuntimePlaneAxisValueProjection, +) from openhcs.core.source_image_provenance import SourceImageProvenancePlanes from openhcs.core.source_metadata import SourceVoxelSpacing from openhcs.interop.cellprofiler.runtime.output_recording import ( CellProfilerOutputRecorder, ) -from openhcs.interop.cellprofiler.runtime.function_contract_execution import CellProfilerFunctionContractExecutor -from openhcs.processing.backends.cellprofiler.shape import MeasureObjectSizeShapeModule, measure_object_size_shape +from openhcs.interop.cellprofiler.runtime.function_contract_execution import ( + CellProfilerFunctionContractExecutor, +) +from openhcs.processing.backends.cellprofiler.shape import ( + MeasureObjectSizeShapeModule, + measure_object_size_shape, +) from openhcs.core.image_payload_execution_mode import ( FullStackExecution, ) -def _source(labels, *, axis=RuntimePlaneAxis.RUNTIME_SLICE, plane_count=None, spacing=(1.3556, 1.3556)): +def _source( + labels, + *, + axis=RuntimePlaneAxis.RUNTIME_SLICE, + plane_count=None, + spacing=(1.3556, 1.3556), +): count = labels.shape[0] if plane_count is None else plane_count return ImagePayloadMetadata( plane_axis=axis, source_image_names=("saved_labels",) * count, source_image_provenance_planes=SourceImageProvenancePlanes.from_components( - paths=tuple(f"/synthetic/A01_s001_w{index + 1}_labels.tif" for index in range(count)), - component_metadata=tuple({"well": "A01", "site": "1", "channel": str(index + 1)} for index in range(count)), + paths=tuple( + f"/synthetic/A01_s001_w{index + 1}_labels.tif" for index in range(count) + ), + component_metadata=tuple( + {"well": "A01", "site": "1", "channel": str(index + 1)} + for index in range(count) + ), ), source_voxel_spacing=SourceVoxelSpacing(spacing), ).payload_with(labels, None) def _saved_label_input(source): - spec = ArtifactSpec.input("saved_labels", ObjectLabelsArtifactType, parameter_name="labels") - return CellProfilerOutputRecorder.for_artifact_type(spec.artifact_type).runtime_input_value( - spec=spec, value=source + spec = ArtifactSpec.input( + "saved_labels", ObjectLabelsArtifactType, parameter_name="labels" ) + return CellProfilerOutputRecorder.for_artifact_type( + spec.artifact_type + ).runtime_input_value(spec=spec, value=source) @pytest.mark.parametrize("axis", tuple(RuntimePlaneAxis)) @pytest.mark.parametrize("plane_count", (1, 2)) def test_explicit_object_projection_preserves_axis_without_acquisition_provenance( - axis, plane_count, + axis, + plane_count, ): labels = np.ones((plane_count, 6, 7), dtype=np.int32) source = ImagePayloadMetadata(plane_axis=axis).payload_with(labels) projection = RuntimePlaneAxisValueProjection.preserve( - axis=axis, axis_size=plane_count, + axis=axis, + axis_size=plane_count, ) payload = SourceImageObjectLabelBuildRequest( - image=source, labels=labels, plane_projection=projection, + image=source, + labels=labels, + plane_projection=projection, ).payload() assert payload.plane_axis is axis @@ -66,9 +93,11 @@ def test_explicit_object_projection_preserves_axis_without_acquisition_provenanc other_axis = next(member for member in RuntimePlaneAxis if member is not axis) with pytest.raises(ValueError, match="conflicts with the source-image axis"): SourceImageObjectLabelBuildRequest( - image=source, labels=labels, + image=source, + labels=labels, plane_projection=RuntimePlaneAxisValueProjection.preserve( - axis=other_axis, axis_size=plane_count, + axis=other_axis, + axis_size=plane_count, ), ).payload() @@ -80,23 +109,32 @@ def test_original_artifact_admission_and_full_stack_shape_keep_2d_planes(axis): labels[1, 2:5, 3:7] = 106 source = _source(labels, axis=axis) source_bytes = labels.tobytes() - spec = ArtifactSpec.input("saved_labels", ObjectLabelsArtifactType, parameter_name="labels") - value = CellProfilerOutputRecorder.for_artifact_type(spec.artifact_type).runtime_input_value( - spec=spec, value=source + spec = ArtifactSpec.input( + "saved_labels", ObjectLabelsArtifactType, parameter_name="labels" ) + value = CellProfilerOutputRecorder.for_artifact_type( + spec.artifact_type + ).runtime_input_value(spec=spec, value=source) assert value.domain.scope is ObjectLabelDomainScope.PLANE assert value.plane_axis is axis assert value.domain.declared_object_id_domains == ((29,), (106,)) - assert value.source_provenance.identity() == source.metadata.source_provenance.identity() + assert ( + value.source_provenance.identity() + == source.metadata.source_provenance.identity() + ) planes = value.measurement_planes() assert tuple(plane.labels.shape for plane in planes) == ((8, 9), (8, 9)) for index, plane in enumerate(planes): np.testing.assert_array_equal(plane.labels, labels[index]) - assert plane.parent_image_source_voxel_spacing == SourceVoxelSpacing((1.3556, 1.3556)) + assert plane.parent_image_source_voxel_spacing == SourceVoxelSpacing( + (1.3556, 1.3556) + ) contract = CallableContract.from_callable(measure_object_size_shape) _image, rows = CellProfilerFunctionContractExecutor().execute( - contract, contract.resolve_canonical_raw_callable(), source, + contract, + contract.resolve_canonical_raw_callable(), + source, {"labels": value, "calculate_advanced": False, "calculate_zernikes": False}, execution_mode=FullStackExecution, ) @@ -127,8 +165,15 @@ def test_singleton_runtime_slice_is_not_a_physical_volume(): def test_declared_runtime_planes_can_each_contain_a_genuine_calibrated_volume(): labels = np.ones((2, 3, 6, 7), dtype=np.int32) value = _saved_label_input(_source(labels, spacing=(2.0, 1.3556, 1.3556))) - assert tuple(plane.labels.shape for plane in value.measurement_planes()) == ((3, 6, 7), (3, 6, 7)) - assert value.parent_image_source_voxel_spacing.spacing_for_ndim(3) == (2.0, 1.3556, 1.3556) + assert tuple(plane.labels.shape for plane in value.measurement_planes()) == ( + (3, 6, 7), + (3, 6, 7), + ) + assert value.parent_image_source_voxel_spacing.spacing_for_ndim(3) == ( + 2.0, + 1.3556, + 1.3556, + ) def test_undeclared_volume_and_explicit_payload_override_are_not_guessed_planar(): @@ -137,11 +182,15 @@ def test_undeclared_volume_and_explicit_payload_override_are_not_guessed_planar( (_source(labels, axis=None, spacing=(2.0, 1.3556, 1.3556)), {}), (_source(labels), {"domain_scope": ObjectLabelDomainScope.PAYLOAD}), ): - value = SourceImageObjectLabelBuildRequest(image=source, labels=labels, **options).payload() + value = SourceImageObjectLabelBuildRequest( + image=source, labels=labels, **options + ).payload() assert value.domain.scope is ObjectLabelDomainScope.PAYLOAD assert value.plane_axis is None assert value.measurement_planes()[0].labels.shape == (2, 6, 7) - with pytest.raises(ValueError, match="Cannot project 2-D source voxel spacing onto 3-D data"): + with pytest.raises( + ValueError, match="Cannot project 2-D source voxel spacing onto 3-D data" + ): SourceVoxelSpacing((1.3556, 1.3556)).spacing_for_ndim(3) @@ -156,16 +205,20 @@ def test_declared_axis_rejects_payload_wide_ids_and_mismatched_label_geometry(): labels = np.ones((2, 6, 7), dtype=np.int32) source = _source(labels) projection = RuntimePlaneAxisValueProjection.from_source_declaration( - source.metadata.plane_axis, source.metadata.source_provenance, + source.metadata.plane_axis, + source.metadata.source_provenance, ) with pytest.raises(ValueError, match="per-plane domains"): SourceImageObjectLabelBuildRequest( - image=source, labels=labels, declared_object_ids=(1,), + image=source, + labels=labels, + declared_object_ids=(1,), plane_projection=projection, ).payload() with pytest.raises(ValueError, match="spatial shape"): SourceImageObjectLabelBuildRequest( - image=source, labels=np.ones((2, 5, 7), dtype=np.int32), + image=source, + labels=np.ones((2, 5, 7), dtype=np.int32), plane_projection=projection, ).payload() @@ -174,8 +227,11 @@ def test_explicit_projection_conflicting_with_source_declaration_is_rejected(): labels = np.ones((2, 6, 7), dtype=np.int32) with pytest.raises(ValueError, match="conflicts"): SourceImageObjectLabelBuildRequest( - image=_source(labels), labels=labels, - plane_projection=RuntimePlaneAxisValueProjection.preserve(axis=RuntimePlaneAxis.SOURCE_BINDING, axis_size=2), + image=_source(labels), + labels=labels, + plane_projection=RuntimePlaneAxisValueProjection.preserve( + axis=RuntimePlaneAxis.SOURCE_BINDING, axis_size=2 + ), ).payload() @@ -183,11 +239,15 @@ class PhysicalCalibrationRequired: """Independent admission capability for a new physical-measurement leaf.""" def payload(self, **kwargs): - SourceVoxelSpacing.require_physical_pixel_size((self.metadata.source_voxel_spacing,)) + SourceVoxelSpacing.require_physical_pixel_size( + (self.metadata.source_voxel_spacing,) + ) return super().payload(**kwargs) -class CalibratedSavedLabels(PhysicalCalibrationRequired, SourceImageObjectLabelBuildRequest): +class CalibratedSavedLabels( + PhysicalCalibrationRequired, SourceImageObjectLabelBuildRequest +): """A new declaration composes calibration admission with the existing builder.""" @@ -195,16 +255,26 @@ def test_new_calibrated_leaf_executes_cooperative_admission_and_plane_building() labels = np.ones((2, 6, 7), dtype=np.int32) source = _source(labels) projection = RuntimePlaneAxisValueProjection.from_source_declaration( - source.metadata.plane_axis, source.metadata.source_provenance, + source.metadata.plane_axis, + source.metadata.source_provenance, ) value = CalibratedSavedLabels( - image=source, labels=labels, plane_projection=projection, + image=source, + labels=labels, + plane_projection=projection, ).payload() assert value.domain.scope is ObjectLabelDomainScope.PLANE assert len(value.measurement_planes()) == 2 with pytest.raises(ValueError, match="Physical scalar pixel size requires"): - CalibratedSavedLabels(image=_source(labels, spacing=()), labels=labels, plane_projection=projection).payload() - assert _saved_label_input(_source(labels, spacing=())).domain.scope is ObjectLabelDomainScope.PLANE + CalibratedSavedLabels( + image=_source(labels, spacing=()), + labels=labels, + plane_projection=projection, + ).payload() + assert ( + _saved_label_input(_source(labels, spacing=())).domain.scope + is ObjectLabelDomainScope.PLANE + ) class PairedSourceAdmission: @@ -223,17 +293,31 @@ class PairedSourceProjection(PairedSourceAdmission, RuntimePlaneAxisValueProject @pytest.mark.parametrize("axis", tuple(RuntimePlaneAxis)) def test_source_constructor_admits_a_new_cooperative_projection_capability(axis): source = _source(np.ones((2, 6, 7), dtype=np.int32), axis=axis) - value = PairedSourceProjection.from_source_declaration(axis, source.metadata.source_provenance) + value = PairedSourceProjection.from_source_declaration( + axis, source.metadata.source_provenance + ) assert isinstance(value, PairedSourceProjection) assert value.axis is axis assert value.axis_size == 2 assert value.selected_plane(1).require_plane_index() == 1 bigger = _source(np.ones((3, 6, 7), dtype=np.int32), axis=axis) with pytest.raises(ValueError, match="at most two source planes"): - PairedSourceProjection.from_source_declaration(axis, bigger.metadata.source_provenance) - assert RuntimePlaneAxisValueProjection.from_source_declaration(axis, bigger.metadata.source_provenance).axis_size == 3 + PairedSourceProjection.from_source_declaration( + axis, bigger.metadata.source_provenance + ) + assert ( + RuntimePlaneAxisValueProjection.from_source_declaration( + axis, bigger.metadata.source_provenance + ).axis_size + == 3 + ) # Absence is a declaration, not inferred from three source records. - assert PairedSourceProjection.from_source_declaration(None, bigger.metadata.source_provenance) is None + assert ( + PairedSourceProjection.from_source_declaration( + None, bigger.metadata.source_provenance + ) + is None + ) @pytest.mark.parametrize("axis", tuple(RuntimePlaneAxis)) @@ -254,8 +338,13 @@ def test_produced_volume_global_object_domain_is_independent_of_source_plane_axi assert value.plane_axis is None assert len(value.measurement_planes()) == 1 np.testing.assert_array_equal(value.measurement_planes()[0].labels, labels) - assert value.source_provenance.identity() == source.metadata.source_provenance.identity() - assert value.parent_image_source_voxel_spacing == source.metadata.source_voxel_spacing + assert ( + value.source_provenance.identity() + == source.metadata.source_provenance.identity() + ) + assert ( + value.parent_image_source_voxel_spacing == source.metadata.source_voxel_spacing + ) @pytest.mark.parametrize("axis", tuple(RuntimePlaneAxis)) @@ -281,15 +370,21 @@ def test_explicit_plane_domain_still_requires_projection_and_rejects_global_coun source = _source(labels, axis=axis) with pytest.raises(ValueError, match="requires an exact plane projection"): SourceImageObjectLabelBuildRequest( - image=source, labels=labels, domain_scope=ObjectLabelDomainScope.PLANE, + image=source, + labels=labels, + domain_scope=ObjectLabelDomainScope.PLANE, ).payload() projection = RuntimePlaneAxisValueProjection.from_source_declaration( - source.metadata.plane_axis, source.metadata.source_provenance, + source.metadata.plane_axis, + source.metadata.source_provenance, ) with pytest.raises(ValueError, match="per-plane domains"): SourceImageObjectLabelBuildRequest( - image=source, labels=labels, domain_scope=ObjectLabelDomainScope.PLANE, - plane_projection=projection, declared_object_count=1, + image=source, + labels=labels, + domain_scope=ObjectLabelDomainScope.PLANE, + plane_projection=projection, + declared_object_count=1, ).payload() @@ -306,7 +401,8 @@ def test_generic_output_context_uses_execution_projection_not_image_storage_axis assert volume.domain.declared_object_ids == (29, 106) np.testing.assert_array_equal(volume.measurement_planes()[0].labels, labels) projection = RuntimePlaneAxisValueProjection.from_source_declaration( - source.metadata.plane_axis, source.metadata.source_provenance, + source.metadata.plane_axis, + source.metadata.source_provenance, ) planes = ImagePayload.of(labels).object_label_output(source, projection) assert planes.domain.scope is ObjectLabelDomainScope.PLANE @@ -321,7 +417,9 @@ def test_saved_ingress_preserves_already_typed_object_domain_and_identity(): labels = np.ones((2, 6, 7), dtype=np.int32) source = _source(labels, spacing=(2.0, 1.3556, 1.3556)) produced = SourceImageObjectLabelBuildRequest( - image=source, labels=labels, declared_object_count=1, + image=source, + labels=labels, + declared_object_count=1, ).label_set(name="saved_labels") admitted = _saved_label_input(produced) assert admitted is produced diff --git a/tests/unit/test_selected_plane_stream_materialization.py b/tests/unit/test_selected_plane_stream_materialization.py index 081bfd6b17..55fabeb3c8 100644 --- a/tests/unit/test_selected_plane_stream_materialization.py +++ b/tests/unit/test_selected_plane_stream_materialization.py @@ -99,7 +99,9 @@ def _source_stack(axis, component=Microscopy.Channel.name): ) def test_public_selected_checkpoint_materializes_and_streams_without_storage_axes( axis, - output_type, artifact_name, dtype, + output_type, + artifact_name, + dtype, tmp_path, monkeypatch, ): @@ -165,11 +167,10 @@ def capture_stream(data_list, paths, backend, **kwargs): request.source.metadata.component_metadata_for_item(path, 0)["channel"] == 2 ) assert payload.metadata.source_image_names == ("FITC",) - assert ( - payload.metadata.source_path - == "/input/A01_s001_w2_z001_t001.tif" + assert payload.metadata.source_path == "/input/A01_s001_w2_z001_t001.tif" + assert payload.metadata.source_voxel_spacing == SourceVoxelSpacing( + (1.3556, 1.3556) ) - assert payload.metadata.source_voxel_spacing == SourceVoxelSpacing((1.3556, 1.3556)) finally: viewer.cleanup() @@ -238,9 +239,7 @@ def with_data(self, data): ) assert calls == ["observe", "reverse", "declaration"] assert payload.metadata.source_image_names == ("FITC", "DAPI") - assert payload.metadata.retained_plane_component_values() == { - "channel": ("2", "1") - } + assert payload.metadata.retained_plane_component_values() == {"channel": ("2", "1")} np.testing.assert_array_equal(np.asarray(payload), selected.data) @@ -262,11 +261,13 @@ class DeclaredResponse(ObservedProjection, SelectedDiagnosticPlaneImageOutput): pass source = _source_stack(RuntimePlaneAxis.SOURCE_BINDING) - pixels = np.full((1, 7, 8), .75, dtype=np.float32) + pixels = np.full((1, 7, 8), 0.75, dtype=np.float32) response = DeclaredResponse(pixels, (1,)) calls.clear() payload = ImageArtifactType.contextualize_output( - source, response, None, + source, + response, + None, RuntimePlaneAxisValueProjection.preserve( axis=RuntimePlaneAxis.SOURCE_BINDING, axis_size=2 ), @@ -291,11 +292,27 @@ def test_all_public_declared_outputs_persist_with_selected_qa_streams( image=source, labels=labels, plane_projection=projection ).payload() summary = NeuriteOutgrowthSummary( - **{field.name: 0 for field in fields(NeuriteOutgrowthSummary) if field.name != "coordinate_unit"}, + **{ + field.name: 0 + for field in fields(NeuriteOutgrowthSummary) + if field.name != "coordinate_unit" + }, coordinate_unit=SourceVoxelSpacingUnit.MICROMETERS, ) cell = NeuriteOutgrowthCellResult( - 1, 1, SourceVoxelSpacingUnit.MICROMETERS, 0.0, 0, 0.0, 0.0, 0.0, 0, 0.0, 1.0, 0.0, False + 1, + 1, + SourceVoxelSpacingUnit.MICROMETERS, + 0.0, + 0, + 0.0, + 0.0, + 0.0, + 0, + 0.0, + 1.0, + 0.0, + False, ) selected = ImageArtifactType.contextualize_output( source, @@ -324,8 +341,10 @@ def test_all_public_declared_outputs_persist_with_selected_qa_streams( ), *NeuriteAdmissionPlanes( np.asarray(source)[1].astype(np.float32), - labels[1] > 0, labels[1] > 0, - np.asarray(source)[1].astype(np.float32), labels[1] > 0, + labels[1] > 0, + labels[1] > 0, + np.asarray(source)[1].astype(np.float32), + labels[1] > 0, np.asarray(source)[1].astype(np.float32), ).selected_outputs(1), *NeuriteOwnershipPlanes(labels[1], labels[1], labels[1]).selected_outputs(1), @@ -362,7 +381,9 @@ def test_all_public_declared_outputs_persist_with_selected_qa_streams( output_dir=tmp_path / "images", ) for output, value in zip(plans, values, strict=True): - value = (ImageArtifactType if output is None else output.artifact_type).contextualize_output( + value = ( + ImageArtifactType if output is None else output.artifact_type + ).contextualize_output( source, value, output, @@ -386,13 +407,16 @@ def capture_stream(data_list, paths, backend, **kwargs): monkeypatch.setattr(filemanager, "save_batch", capture_stream) try: - PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs(filemanager, plan, context) + PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( + filemanager, plan, context + ) materializations = runtime_artifact_materializations(plan, context) assert {item.output_plan.name for item in materializations} == set( specs.names() ) assert len(materializations) == 12 + len( - NeuriteAdmissionPlanes.artifact_specs() + NeuriteOwnershipPlanes.artifact_specs() + NeuriteAdmissionPlanes.artifact_specs() + + NeuriteOwnershipPlanes.artifact_specs() ) for item in materializations: outputs = item.outputs(plan, context) @@ -404,7 +428,8 @@ def capture_stream(data_list, paths, backend, **kwargs): if paths[0].endswith(".checkpoint.tif") ] assert len(checkpoints) == 5 + len( - NeuriteAdmissionPlanes.artifact_specs() + NeuriteOwnershipPlanes.artifact_specs() + NeuriteAdmissionPlanes.artifact_specs() + + NeuriteOwnershipPlanes.artifact_specs() ) for (data,), (path,), request in checkpoints: assert ( diff --git a/tests/unit/test_singleton_rgb_label_invocation_477.py b/tests/unit/test_singleton_rgb_label_invocation_477.py index 7ee07aea04..cfe05f87b9 100644 --- a/tests/unit/test_singleton_rgb_label_invocation_477.py +++ b/tests/unit/test_singleton_rgb_label_invocation_477.py @@ -1,15 +1,32 @@ """Scalar RGB invocations consume only an explicitly declared singleton root.""" + import numpy as np import pytest -from openhcs.core.artifacts import ArtifactSpec, ImageArtifactType, ObjectLabelsArtifactType +from openhcs.core.artifacts import ( + ArtifactSpec, + ImageArtifactType, + ObjectLabelsArtifactType, +) from openhcs.core.function_contract_metadata import FunctionContractAttribute from openhcs.core.runtime_image_values import ImageMetadataPayload, ImagePayloadMetadata -from openhcs.core.runtime_object_label_domains import ObjectLabelDomain, ObjectLabelDomainScope -from openhcs.core.runtime_object_labels import ObjectLabelSet, ObjectLabelVariantData, object_label_dense_array -from openhcs.core.runtime_plane_projection import RuntimePlaneAxis, RuntimePlaneProjection +from openhcs.core.runtime_object_label_domains import ( + ObjectLabelDomain, + ObjectLabelDomainScope, +) +from openhcs.core.runtime_object_labels import ( + ObjectLabelSet, + ObjectLabelVariantData, + object_label_dense_array, +) +from openhcs.core.runtime_plane_projection import ( + RuntimePlaneAxis, + RuntimePlaneProjection, +) from openhcs.core.runtime_slice_projection import RuntimeSliceProjectionDeclarationError -from openhcs.interop.cellprofiler.runtime.function_contract_execution import CellProfilerFunctionContractExecutor +from openhcs.interop.cellprofiler.runtime.function_contract_execution import ( + CellProfilerFunctionContractExecutor, +) from openhcs.processing.backends.cellprofiler.outlines import OverlayOutlinesModule from tests.unit.test_cellprofiler_module_execution import ( _FakeCellProfilerRuntime, @@ -29,42 +46,69 @@ def _invocation_case(*, root_count=1, label_count=1): - image_spec = ArtifactSpec.input('GFPandDNA', ImageArtifactType) - labels_spec = ArtifactSpec.input('Nuclei', ObjectLabelsArtifactType) + image_spec = ArtifactSpec.input("GFPandDNA", ImageArtifactType) + labels_spec = ArtifactSpec.input("Nuclei", ObjectLabelsArtifactType) raw = OverlayOutlinesModule.require_callable() - contract = _compiled_callable_contract(raw, artifact_inputs=(image_spec, labels_spec)) + contract = _compiled_callable_contract( + raw, artifact_inputs=(image_spec, labels_spec) + ) data = np.linspace(0, 1, 8 * 9 * 3, dtype=np.float32).reshape(8, 9, 3) - image = ImageMetadataPayload(data, ImagePayloadMetadata(axes=PayloadAxes.colour_samples(2), source_image_names=('GFPandDNA',))) + image = ImageMetadataPayload( + data, + ImagePayloadMetadata( + axes=PayloadAxes.colour_samples(2), source_image_names=("GFPandDNA",) + ), + ) label_data = np.zeros((label_count, 8, 9), dtype=np.int32) label_data[:, 2:5, 3:6] = 1 labels = ObjectLabelSet( - name='Nuclei', variant_data=ObjectLabelVariantData(labels=label_data), + name="Nuclei", + variant_data=ObjectLabelVariantData(labels=label_data), plane_axis=RuntimePlaneAxis.RUNTIME_SLICE, - domain=ObjectLabelDomain(declared_object_id_domains=((1,),) * label_count, scope=ObjectLabelDomainScope.PLANE), + domain=ObjectLabelDomain( + declared_object_id_domains=((1,),) * label_count, + scope=ObjectLabelDomainScope.PLANE, + ), ) adapter = _FakeCellProfilerRuntime( - {'GFPandDNA': image}, objects={'Nuclei': labels}, callable_contract=contract, - artifact_input_edges=tuple(_artifact_input_edge_for_test(spec) for spec in (image_spec, labels_spec)), + {"GFPandDNA": image}, + objects={"Nuclei": labels}, + callable_contract=contract, + artifact_input_edges=tuple( + _artifact_input_edge_for_test(spec) for spec in (image_spec, labels_spec) + ), plane_projection=RuntimePlaneProjection.stack(root_count), ) executor = _module_executor(contract) - request = executor._image_request(image, adapter, module_type=OverlayOutlinesModule, active_input_specs=(image_spec, labels_spec)) + request = executor._image_request( + image, + adapter, + module_type=OverlayOutlinesModule, + active_input_specs=(image_spec, labels_spec), + ) return raw, contract, image, labels, adapter, executor, request def _bind(case): raw, contract, image, labels, adapter, executor, request = case return executor._invocation_request( - image_request=request, adapter=adapter, current_image=image, - kwargs={'outline_colors': ('#21FFFF',)}, module_type=OverlayOutlinesModule, + image_request=request, + adapter=adapter, + current_image=image, + kwargs={"outline_colors": ("#21FFFF",)}, + module_type=OverlayOutlinesModule, ) def _execute(case, invocation): raw, contract, *_ = case return CellProfilerFunctionContractExecutor().execute( - contract, raw, invocation.payload, invocation.kwargs, - execution_mode=invocation.execution_mode, plane_projection=invocation.plane_projection, + contract, + raw, + invocation.payload, + invocation.kwargs, + execution_mode=invocation.execution_mode, + plane_projection=invocation.plane_projection, ) @@ -74,15 +118,21 @@ def test_match_image_labels_consume_declared_singleton_root_for_scalar_rgb(): image_before = image.data.copy() labels_before = object_label_dense_array(labels).copy() invocation = _bind(case) - selected = invocation.kwargs['object_labels'][0] + selected = invocation.kwargs["object_labels"][0] assert invocation.execution_mode is NaturalExecution assert invocation.plane_projection is None assert invocation.payload.metadata.axis_position(ColourAxis) == 2 assert selected.plane_axis is None assert object_label_dense_array(selected).shape == (8, 9) output = _execute(case, invocation) - scalar_labels = ObjectLabelSet(name='Nuclei', variant_data=ObjectLabelVariantData(labels=labels_before[0]), domain=ObjectLabelDomain(declared_object_ids=(1,), scope=ObjectLabelDomainScope.PAYLOAD)) - expected = raw(image, object_labels=(scalar_labels,), outline_colors=('#21FFFF',)) + scalar_labels = ObjectLabelSet( + name="Nuclei", + variant_data=ObjectLabelVariantData(labels=labels_before[0]), + domain=ObjectLabelDomain( + declared_object_ids=(1,), scope=ObjectLabelDomainScope.PAYLOAD + ), + ) + expected = raw(image, object_labels=(scalar_labels,), outline_colors=("#21FFFF",)) np.testing.assert_array_equal(output.data, expected.data) np.testing.assert_array_equal(image.data, image_before) np.testing.assert_array_equal(object_label_dense_array(labels), labels_before) @@ -90,49 +140,76 @@ def test_match_image_labels_consume_declared_singleton_root_for_scalar_rgb(): assert image.metadata.plane_axis is None -@pytest.mark.parametrize('root_count', (None, 2)) +@pytest.mark.parametrize("root_count", (None, 2)) def test_scalar_rgb_cannot_authorize_unknown_or_multiple_root_planes(root_count): case = _invocation_case(root_count=root_count) invocation = _bind(case) - assert invocation.kwargs['object_labels'][0].plane_axis is RuntimePlaneAxis.RUNTIME_SLICE - with pytest.raises(RuntimeSliceProjectionDeclarationError, match='Kwargs cannot create image-axis execution semantics'): + assert ( + invocation.kwargs["object_labels"][0].plane_axis + is RuntimePlaneAxis.RUNTIME_SLICE + ) + with pytest.raises( + RuntimeSliceProjectionDeclarationError, + match="Kwargs cannot create image-axis execution semantics", + ): _execute(case, invocation) -def test_explicit_full_stack_label_contract_preserves_scalar_boundary_refusal(monkeypatch): +def test_explicit_full_stack_label_contract_preserves_scalar_boundary_refusal( + monkeypatch, +): raw = OverlayOutlinesModule.require_callable() - monkeypatch.setattr(raw, FunctionContractAttribute.object_label_input_execution_mode, FullStackLabels) + monkeypatch.setattr( + raw, + FunctionContractAttribute.object_label_input_execution_mode, + FullStackLabels, + ) case = _invocation_case() invocation = _bind(case) - assert invocation.kwargs['object_labels'][0].plane_axis is RuntimePlaneAxis.RUNTIME_SLICE - with pytest.raises(RuntimeSliceProjectionDeclarationError, match='Kwargs cannot create image-axis execution semantics'): + assert ( + invocation.kwargs["object_labels"][0].plane_axis + is RuntimePlaneAxis.RUNTIME_SLICE + ) + with pytest.raises( + RuntimeSliceProjectionDeclarationError, + match="Kwargs cannot create image-axis execution semantics", + ): _execute(case, invocation) def test_declared_singleton_root_refuses_mismatched_label_cardinality(): case = _invocation_case(label_count=2) - with pytest.raises(ValueError, match='Object-label runtime plane-axis cardinality mismatch'): + with pytest.raises( + ValueError, match="Object-label runtime plane-axis cardinality mismatch" + ): _bind(case) def test_final_module_mode_sees_full_labels_before_scalar_projection(monkeypatch): case = _invocation_case() + def mode(cls, default, *, image, kwargs, variable_components): - assert kwargs['object_labels'][0].plane_axis is RuntimePlaneAxis.RUNTIME_SLICE - assert object_label_dense_array(kwargs['object_labels'][0]).shape == (1, 8, 9) + assert kwargs["object_labels"][0].plane_axis is RuntimePlaneAxis.RUNTIME_SLICE + assert object_label_dense_array(kwargs["object_labels"][0]).shape == (1, 8, 9) return FullStackExecution - monkeypatch.setattr(OverlayOutlinesModule, 'execution_mode', classmethod(mode)) + + monkeypatch.setattr(OverlayOutlinesModule, "execution_mode", classmethod(mode)) invocation = _bind(case) assert invocation.execution_mode is FullStackExecution - assert invocation.kwargs['object_labels'][0].plane_axis is RuntimePlaneAxis.RUNTIME_SLICE + assert ( + invocation.kwargs["object_labels"][0].plane_axis + is RuntimePlaneAxis.RUNTIME_SLICE + ) def test_module_mode_failure_precedes_projection_guard(monkeypatch): case = _invocation_case(label_count=2) - error = RuntimeError('module image-domain selection failed') + error = RuntimeError("module image-domain selection failed") + def mode(cls, default, *, image, kwargs, variable_components): raise error - monkeypatch.setattr(OverlayOutlinesModule, 'execution_mode', classmethod(mode)) + + monkeypatch.setattr(OverlayOutlinesModule, "execution_mode", classmethod(mode)) with pytest.raises(RuntimeError) as caught: _bind(case) assert caught.value is error diff --git a/tests/unit/test_smoke_installed_gui.py b/tests/unit/test_smoke_installed_gui.py index 359a1dc2e0..7461331098 100644 --- a/tests/unit/test_smoke_installed_gui.py +++ b/tests/unit/test_smoke_installed_gui.py @@ -174,7 +174,9 @@ def test_gui_smoke_waits_for_gui_endpoint_before_reading_catalog() -> None: class UnreachableFunctionCatalogService: @staticmethod def prepare() -> Future[FunctionCatalogPage]: - raise AssertionError("The catalog attaches only after the GUI endpoint starts.") + raise AssertionError( + "The catalog attaches only after the GUI endpoint starts." + ) with pytest.raises( OperationTimeoutError, diff --git a/tests/unit/test_snapshot_deadline.py b/tests/unit/test_snapshot_deadline.py index 09460a9580..3fc216571f 100644 --- a/tests/unit/test_snapshot_deadline.py +++ b/tests/unit/test_snapshot_deadline.py @@ -56,7 +56,9 @@ def close(self, *args, **kwargs): pass def send(self, data, flags): - accepted = NapariAcceptedControlRequest.from_wire_mapping(pickle.loads(data)) + accepted = NapariAcceptedControlRequest.from_wire_mapping( + pickle.loads(data) + ) self.message = accepted.message server.accepted_control_requests.put(replace(accepted, response=reply)) server.process_messages() @@ -110,28 +112,38 @@ def poll(self, timeout): def test_snapshot_wire_offer_is_admitted_once_and_capture_uses_local_identity(tmp_path): from zmqruntime.messages import MessageFields - from openhcs.runtime.viewer_protocol import ViewerControlMessageRequest, ViewerRuntimeEndpoint + from openhcs.runtime.viewer_protocol import ( + ViewerControlMessageRequest, + ViewerRuntimeEndpoint, + ) from openhcs.runtime.zmq_config import OPENHCS_ZMQ_CONFIG original = ViewerWindowSnapshotRequest.from_fields( - connection=ExecutionConnectionSpec(port=5584), output_dir_path=str(tmp_path), + connection=ExecutionConnectionSpec(port=5584), + output_dir_path=str(tmp_path), ).start_operation() envelope = ViewerControlMessageRequest( endpoint=ViewerRuntimeEndpoint( transport=original.connection.transport_endpoint(OPENHCS_ZMQ_CONFIG), config=OPENHCS_ZMQ_CONFIG, ), - message_type="screenshot", payload=original, + message_type="screenshot", + payload=original, operation_deadline=original.operation_deadline, ) wire = pickle.loads(pickle.dumps(envelope.to_wire_mapping())) assert wire["payload"].operation_deadline is None - assert 0 < wire[MessageFields.OBSERVATION_BUDGET_SECONDS] <= original.timeout_ms / 1000 + assert ( + 0 < wire[MessageFields.OBSERVATION_BUDGET_SECONDS] <= original.timeout_ms / 1000 + ) admitted_at = time.monotonic() accepted = NapariAcceptedControlRequest.from_wire_mapping(wire) assert accepted.observation_deadline is not original.operation_deadline assert accepted.observation_deadline.expires_at >= admitted_at - assert accepted.message["payload"].snapshot_operation_deadline() is accepted.observation_deadline + assert ( + accepted.message["payload"].snapshot_operation_deadline() + is accepted.observation_deadline + ) assert MessageFields.OBSERVATION_BUDGET_SECONDS not in accepted.message # Qt/action consumers receive an admitted object, not a reusable duration. with pytest.raises(KeyError): diff --git a/tests/unit/test_source_binding_workspace.py b/tests/unit/test_source_binding_workspace.py index 0b07be04d8..20343d21ee 100644 --- a/tests/unit/test_source_binding_workspace.py +++ b/tests/unit/test_source_binding_workspace.py @@ -11,7 +11,9 @@ from openhcs.constants.constants import Backend from openhcs.core.artifacts import ImageArtifactType, ObjectLabelsArtifactType -from openhcs.processing.backends.cellprofiler.infrastructure import relative_voxel_spacing +from openhcs.processing.backends.cellprofiler.infrastructure import ( + relative_voxel_spacing, +) from openhcs.core.source_binding_workspace import SourceBindingWorkspaceProjector from openhcs.core.runtime_tabular_values import FieldSpec from openhcs.core.source_metadata import ( @@ -69,29 +71,46 @@ def test_transient_axis_views_observe_each_new_projection_without_reusing_releas cache = VirtualWorkspaceSourceProjectionCache() refs = MappingProxyType({"A01.tif": SourcePixelRef("disk", "source.tif")}) for generation in range(20): - metadata = MappingProxyType({"A01.tif": MappingProxyType({ - "well": "A01", "generation": str(generation), - })}) + metadata = MappingProxyType( + { + "A01.tif": MappingProxyType( + { + "well": "A01", + "generation": str(generation), + } + ) + } + ) projection = VirtualWorkspaceSourceProjection(refs, metadata, "/plate") filtered = cache.filtered_by_axis(projection, axis_id="A01") - assert filtered.source_metadata_by_path["A01.tif"]["generation"] == str(generation) + assert filtered.source_metadata_by_path["A01.tif"]["generation"] == str( + generation + ) del projection -def test_source_projection_document_replacement_preserves_absence_and_schema_rejection(tmp_path): +def test_source_projection_document_replacement_preserves_absence_and_schema_rejection( + tmp_path, +): cache = VirtualWorkspaceSourceProjectionCache() - native_document = {FIELDS.SUBDIRECTORIES: { - "images": {FIELDS.IMAGE_FILES: ["native.tif"]}, - }} - empty_workspace = {FIELDS.SUBDIRECTORIES: { - "images": {FIELDS.WORKSPACE_MAPPING: {}}, - }} + native_document = { + FIELDS.SUBDIRECTORIES: { + "images": {FIELDS.IMAGE_FILES: ["native.tif"]}, + } + } + empty_workspace = { + FIELDS.SUBDIRECTORIES: { + "images": {FIELDS.WORKSPACE_MAPPING: {}}, + } + } for document in (native_document, empty_workspace): assert cache.projection_for(tmp_path, document) is None assert cache.projection_for(tmp_path, document) is None - malformed_document = {FIELDS.SUBDIRECTORIES: { - "images": {FIELDS.WORKSPACE_MAPPING: []}, - }} + malformed_document = { + FIELDS.SUBDIRECTORIES: { + "images": {FIELDS.WORKSPACE_MAPPING: []}, + } + } with pytest.raises(RuntimeError, match="workspace_mapping must be a mapping"): cache.projection_for(tmp_path, malformed_document) assert cache.projection_for(tmp_path, native_document) is None @@ -110,68 +129,113 @@ def _filemanager() -> FileManager: return FileManager(dict(storage_registry)) -@pytest.mark.parametrize("selection,expected_sites", ((None, tuple(range(1, 10))), (7, (7,)), (3, (3,)))) +@pytest.mark.parametrize( + "selection,expected_sites", ((None, tuple(range(1, 10))), (7, (7,)), (3, (3,))) +) def test_prepared_workspace_admits_declared_source_universe_without_rewriting_provenance( - tmp_path, monkeypatch, selection, expected_sites, + tmp_path, + monkeypatch, + selection, + expected_sites, ): source = tmp_path / "source" source.mkdir() - files = tuple(source / f"A01_s{site:03}_w{channel}.tif" - for site in range(1, 10) for channel in (1, 2)) + files = tuple( + source / f"A01_s{site:03}_w{channel}.tif" + for site in range(1, 10) + for channel in (1, 2) + ) for path in files: _write_tiff_stack(path, (11,)) filemanager = _filemanager() workspace = tmp_path / "prepared" materialization = SourceBindingWorkspaceProjector( - SourceBindingsConfig(metadata_rules=(MetadataExtractionRule( - source=MetadataSource.FILE_NAME, - pattern=r"(?PA\d+)_s(?P\d+)_w(?P\d+)", - ),)), + SourceBindingsConfig( + metadata_rules=( + MetadataExtractionRule( + source=MetadataSource.FILE_NAME, + pattern=r"(?PA\d+)_s(?P\d+)_w(?P\d+)", + ), + ) + ), parser=SourceSchemaFilenameParser(), - ).materialize(source, workspace, filemanager=filemanager, - source_backend=Backend.DISK, workspace_backend=Backend.DISK, - source_files=files) + ).materialize( + source, + workspace, + filemanager=filemanager, + source_backend=Backend.DISK, + workspace_backend=Backend.DISK, + source_files=files, + ) (workspace / FIELDS.DEFAULT_SUBDIRECTORY).mkdir(exist_ok=True) original_bytes = materialization.metadata_path.read_bytes() - config = SourceBindingsConfig(source_filters=() if selection is None else ( - SourceFilterClause(subject=SourceFilterSubject.FILE, - match_type=SourceFilterMatchType.CONTAINS, - value=f"_s{selection:03}_"), - )) - handler = DatasetSourceChoice.named("auto").open(workspace, filemanager=filemanager, source_bindings_config=config) + config = SourceBindingsConfig( + source_filters=( + () + if selection is None + else ( + SourceFilterClause( + subject=SourceFilterSubject.FILE, + match_type=SourceFilterMatchType.CONTAINS, + value=f"_s{selection:03}_", + ), + ) + ) + ) + handler = DatasetSourceChoice.named("auto").open( + workspace, filemanager=filemanager, source_bindings_config=config + ) from openhcs.core.dataset_sources.openhcs_format import OpenHCSDatasetSource + assert isinstance(handler, OpenHCSDatasetSource) handler.initialize_workspace(workspace, filemanager) cache = VirtualWorkspaceSourceProjectionCache() full = WorkspaceSourceProjections.from_dataset_metadata( - dataset_root=workspace, metadata_handler=handler.metadata_handler, - filemanager=filemanager, cache=cache, + dataset_root=workspace, + metadata_handler=handler.metadata_handler, + filemanager=filemanager, + cache=cache, ).projection_or_empty() admissions = [] original_admit = SourceBindingWorkspaceProjector.accept_prepared_projection + def observe_admission(projector, projection): admissions.append(projection) return original_admit(projector, projection) - monkeypatch.setattr(SourceBindingWorkspaceProjector, "accept_prepared_projection", observe_admission) + + monkeypatch.setattr( + SourceBindingWorkspaceProjector, "accept_prepared_projection", observe_admission + ) selected = WorkspaceSourceProjections.from_dataset_metadata( - dataset_root=workspace, metadata_handler=handler.metadata_handler, - filemanager=filemanager, cache=cache, source_bindings=config, + dataset_root=workspace, + metadata_handler=handler.metadata_handler, + filemanager=filemanager, + cache=cache, + source_bindings=config, ).projection_or_empty() - assert set(selected.component_values(Microscopy.Site)) == set(map(str, expected_sites)) + assert set(selected.component_values(Microscopy.Site)) == set( + map(str, expected_sites) + ) assert set(selected.component_values(Microscopy.Channel)) == {"1", "2"} assert len(selected.pipeline_start_files()) == len(expected_sites) * 2 for path in selected.pipeline_start_files(): lookup = VirtualWorkspacePathLookup.from_paths(path, path) assert selected.source_ref_for(lookup) is full.source_ref_for(lookup) - assert selected.require_source_projection_for(lookup) is full.require_source_projection_for(lookup) + assert selected.require_source_projection_for( + lookup + ) is full.require_source_projection_for(lookup) assert selected.source_metadata_for(lookup) == full.source_metadata_for(lookup) assert len(full.pipeline_start_files()) == 18 from openhcs.core.context.processing_context import ProcessingContext + context = ProcessingContext(filemanager=filemanager, axis_id="A01") context.dataset_root = workspace context.microscope_handler = handler from openhcs.core import source_workspace_projection - monkeypatch.setattr(source_workspace_projection, "DEFAULT_SOURCE_PROJECTION_CACHE", cache) + + monkeypatch.setattr( + source_workspace_projection, "DEFAULT_SOURCE_PROJECTION_CACHE", cache + ) runtime_projection = WorkspaceSourceProjections.from_context( context, ).projection_or_empty() @@ -180,70 +244,129 @@ def observe_admission(projector, projection): other = ProcessingContext(filemanager=filemanager, axis_id="A01") other.dataset_root = workspace other.microscope_handler = handler - assert context.runtime_source_workspace_projections.cache is other.runtime_source_workspace_projections.cache is cache + assert ( + context.runtime_source_workspace_projections.cache + is other.runtime_source_workspace_projections.cache + is cache + ) assert other.runtime_source_workspace_projections.projection_or_empty() is selected equivalent = WorkspaceSourceProjections.from_dataset_metadata( - dataset_root=workspace, metadata_handler=handler.metadata_handler, - filemanager=filemanager, cache=cache, source_bindings=replace(config), + dataset_root=workspace, + metadata_handler=handler.metadata_handler, + filemanager=filemanager, + cache=cache, + source_bindings=replace(config), ).projection_or_empty() assert equivalent is selected compiled_views = cache.partition_by_axes(selected, axis_ids=("A01",)) - assert context.runtime_source_workspace_projections.projection_or_empty(axis_id="A01") is compiled_views["A01"] + assert ( + context.runtime_source_workspace_projections.projection_or_empty(axis_id="A01") + is compiled_views["A01"] + ) assert len(admissions) == (0 if selection is None else 1) with monkeypatch.context() as admitted_query: + def reject_redecode(_cls, _subdirectory): - raise AssertionError("An admitted document must not decode its whole workspace again.") - admitted_query.setattr(VirtualWorkspaceMapping, "from_subdirectory", classmethod(reject_redecode)) - assert context.runtime_source_workspace_projections.projection_or_empty() is selected - assert context.runtime_source_workspace_projections.projection_or_empty(axis_id="A01") is compiled_views["A01"] + raise AssertionError( + "An admitted document must not decode its whole workspace again." + ) + + admitted_query.setattr( + VirtualWorkspaceMapping, "from_subdirectory", classmethod(reject_redecode) + ) + assert ( + context.runtime_source_workspace_projections.projection_or_empty() + is selected + ) + assert ( + context.runtime_source_workspace_projections.projection_or_empty( + axis_id="A01" + ) + is compiled_views["A01"] + ) prior_authority = context.runtime_source_workspace_projections - handler._source_bindings_config = replace(config, source_filters=( - SourceFilterClause(subject=SourceFilterSubject.FILE, - match_type=SourceFilterMatchType.CONTAINS, value="_s008_"), - )) + handler._source_bindings_config = replace( + config, + source_filters=( + SourceFilterClause( + subject=SourceFilterSubject.FILE, + match_type=SourceFilterMatchType.CONTAINS, + value="_s008_", + ), + ), + ) changed = context.runtime_source_workspace_projections assert changed is not prior_authority assert changed.projection_or_empty().component_values(Microscopy.Site) == ("8",) handler._source_bindings_config = replace(config, source_filters=()) assert context.runtime_source_workspace_projections.projection_or_empty() is full - handler._source_bindings_config = replace(config, source_filters=( - SourceFilterClause(subject=SourceFilterSubject.FILE, - match_type=SourceFilterMatchType.CONTAINS, value="_s099_"), - )) + handler._source_bindings_config = replace( + config, + source_filters=( + SourceFilterClause( + subject=SourceFilterSubject.FILE, + match_type=SourceFilterMatchType.CONTAINS, + value="_s099_", + ), + ), + ) with pytest.raises(ValueError, match="matched no prepared workspace sources"): context.runtime_source_workspace_projections.projection_or_empty() handler._source_bindings_config = config - assert context.runtime_source_workspace_projections.projection_or_empty() is selected + assert ( + context.runtime_source_workspace_projections.projection_or_empty() is selected + ) handler.metadata_handler.invalidate_metadata_cache() - replacement = context.runtime_source_workspace_projections.projection_or_empty(axis_id="A01") + replacement = context.runtime_source_workspace_projections.projection_or_empty( + axis_id="A01" + ) assert replacement is not compiled_views["A01"] - assert replacement.source_metadata_by_path == compiled_views["A01"].source_metadata_by_path + assert ( + replacement.source_metadata_by_path + == compiled_views["A01"].source_metadata_by_path + ) from openhcs.core.config import PipelineConfig, LazySourceBindingsConfig from openhcs.core.orchestrator.orchestrator import PipelineOrchestrator + orchestrator = PipelineOrchestrator( - workspace, pipeline_config=PipelineConfig( - source_bindings_config=LazySourceBindingsConfig(source_filters=config.source_filters), + workspace, + pipeline_config=PipelineConfig( + source_bindings_config=LazySourceBindingsConfig( + source_filters=config.source_filters + ), ), ).initialize() assert orchestrator.source_workspace_files() == selected.pipeline_start_files() - assert set(orchestrator.get_component_keys(Microscopy.Site)) == set(map(str, expected_sites)) - assert orchestrator.get_component_keys(Microscopy.Site, [str(expected_sites[0])]) == [str(expected_sites[0])] + assert set(orchestrator.get_component_keys(Microscopy.Site)) == set( + map(str, expected_sites) + ) + assert orchestrator.get_component_keys( + Microscopy.Site, [str(expected_sites[0])] + ) == [str(expected_sites[0])] assert orchestrator.get_component_keys(Microscopy.Site, ["99"]) == [] from tests.unit.test_completed_output_publication_lifecycle import _facts + context.record_completed_step_outputs(_facts(workspace)) with_output = WorkspaceSourceProjections.from_context(context).projection_or_empty() assert set(with_output.pipeline_start_files()) == { - *selected.pipeline_start_files(), str(workspace / "images/saved.tif"), + *selected.pipeline_start_files(), + str(workspace / "images/saved.tif"), } assert materialization.metadata_path.read_bytes() == original_bytes with pytest.raises(ValueError, match="matched no prepared workspace sources"): WorkspaceSourceProjections.from_dataset_metadata( - dataset_root=workspace, metadata_handler=handler.metadata_handler, - filemanager=filemanager, source_bindings=SourceBindingsConfig(source_filters=( - SourceFilterClause(subject=SourceFilterSubject.FILE, - match_type=SourceFilterMatchType.CONTAINS, - value="_s099_"), - )), + dataset_root=workspace, + metadata_handler=handler.metadata_handler, + filemanager=filemanager, + source_bindings=SourceBindingsConfig( + source_filters=( + SourceFilterClause( + subject=SourceFilterSubject.FILE, + match_type=SourceFilterMatchType.CONTAINS, + value="_s099_", + ), + ) + ), ).projection_if_available() @@ -627,7 +750,10 @@ def binding(alias: str, channel: str) -> NamedSourceBinding: for projection in projection_set.projections } == {("B02", "3")} assert metadata[Microscopy.Well.metadata_collection_field] == {"B02": None} - assert metadata[Microscopy.Channel.metadata_collection_field] == {"1": "DNA", "2": "RNA"} + assert metadata[Microscopy.Channel.metadata_collection_field] == { + "1": "DNA", + "2": "RNA", + } def test_source_binding_workspace_projects_declared_groups_to_wells(tmp_path): @@ -796,9 +922,13 @@ def test_group_address_preserves_distinct_source_well_as_literal_metadata(tmp_pa def test_nonempty_source_bindings_select_their_declared_microscope_handler( tmp_path, ): - handler = DatasetSourceChoice.named("auto").open(tmp_path, filemanager=_filemanager(), source_bindings_config=SourceBindingsConfig( + handler = DatasetSourceChoice.named("auto").open( + tmp_path, + filemanager=_filemanager(), + source_bindings_config=SourceBindingsConfig( bindings=(NamedSourceBinding(alias="DNA"),) - )) + ), + ) assert isinstance(handler, SourceBindingsSource) @@ -806,9 +936,13 @@ def test_nonempty_source_bindings_select_their_declared_microscope_handler( def test_nonempty_source_bindings_override_format_specific_microscope( tmp_path, ): - handler = DatasetSourceChoice.named("imagexpress").open(tmp_path, filemanager=_filemanager(), source_bindings_config=SourceBindingsConfig( + handler = DatasetSourceChoice.named("imagexpress").open( + tmp_path, + filemanager=_filemanager(), + source_bindings_config=SourceBindingsConfig( bindings=(NamedSourceBinding(alias="DNA"),) - )) + ), + ) assert isinstance(handler, SourceBindingsSource) @@ -954,9 +1088,7 @@ def test_order_source_sets_join_imported_metadata_across_aliases(tmp_path): assert {projection.source_alias for projection in projections} == {"DNA", "Actin"} for projection in projections: - original = dict( - SourceMetadataFields.declared_items(projection.source_metadata) - ) + original = dict(SourceMetadataFields.declared_items(projection.source_metadata)) assert original["Compound"] == "DMSO" assert original["Dose"] == "0" @@ -1452,8 +1584,7 @@ def test_source_bindings_reinitialization_resolves_current_imported_metadata(tmp ) ) assert ( - dict(SourceMetadataFields.declared_items(first_metadata))["Compound"] - == "First" + dict(SourceMetadataFields.declared_items(first_metadata))["Compound"] == "First" ) metadata_path = tmp_path / "openhcs_metadata.json" @@ -2024,9 +2155,13 @@ def test_openhcs_metadata_handler_resolves_subdirectory_inputs(tmp_path): def test_source_bindings_handler_is_registry_constructed(): - handler = DatasetSourceChoice.named("source_bindings").open(Path("."), filemanager=object(), source_bindings_config=SourceBindingsConfig( + handler = DatasetSourceChoice.named("source_bindings").open( + Path("."), + filemanager=object(), + source_bindings_config=SourceBindingsConfig( bindings=(NamedSourceBinding(alias="DNA"),), - )) + ), + ) assert isinstance(handler, SourceBindingsSource) diff --git a/tests/unit/test_source_bindings.py b/tests/unit/test_source_bindings.py index 24dc22a791..798ad622ce 100644 --- a/tests/unit/test_source_bindings.py +++ b/tests/unit/test_source_bindings.py @@ -703,7 +703,8 @@ def fake_open(cls, root, **kwargs): def test_admitted_request_config_survives_next_live_global_context( - tmp_path, monkeypatch, + tmp_path, + monkeypatch, ): from openhcs.core.orchestrator.orchestrator import PipelineOrchestrator from openhcs.core.pipeline.compiler import AxisCompilationRequest @@ -732,7 +733,9 @@ def test_admitted_request_config_survives_next_live_global_context( ), ) orchestrator = PipelineOrchestrator( - tmp_path, pipeline_config=authored, resolved_config=admitted, + tmp_path, + pipeline_config=authored, + resolved_config=admitted, ) orchestrator.initialize_microscope_handler(admitted) assert received["source_bindings_config"] is admitted.source_bindings_config @@ -744,8 +747,10 @@ def test_admitted_request_config_survives_next_live_global_context( request = AxisCompilationRequest( orchestrator=orchestrator, global_config=admitted, - pipeline=SimpleNamespace(), path_resolver=SimpleNamespace(), - global_step_axis_filters={}, enable_visualizer_override=False, + pipeline=SimpleNamespace(), + path_resolver=SimpleNamespace(), + global_step_axis_filters={}, + enable_visualizer_override=False, source_projections_by_axis={}, materialization_planner=MaterializationFlagPlanner( pipeline_config=admitted, @@ -761,7 +766,9 @@ def test_admitted_request_config_survives_next_live_global_context( assert context.tiff_config is admitted.tiff_config # The next authored request admits current global and local changes. - orchestrator.pipeline_config = PipelineConfig(dataset_source=OpenHCSDatasetSource, num_workers=3) + orchestrator.pipeline_config = PipelineConfig( + dataset_source=OpenHCSDatasetSource, num_workers=3 + ) next_request = orchestrator.get_effective_config() assert next_request.num_workers == 3 assert next_request.auto_add_output_dataset is False @@ -830,9 +837,7 @@ def test_realized_component_values_are_scoped_by_source_selector(monkeypatch) -> {"channel": 2, "site": 7, "z_index": 9}, ] expected = { - component: binding.component_values( - component, realized_source_metadata=records - ) + component: binding.component_values(component, realized_source_metadata=records) for component in AxisFamily.active().axes } matched = [] @@ -842,9 +847,13 @@ def record_match(owner, metadata): matched.append(metadata) return original_match(owner, metadata) - monkeypatch.setattr(NamedSourceBinding, "matches_realized_source_metadata", record_match) + monkeypatch.setattr( + NamedSourceBinding, "matches_realized_source_metadata", record_match + ) domains = binding.component_domains(realized_source_metadata=iter(records)) - assert {component: domains.get(component, ()) for component in AxisFamily.active().axes} == expected + assert { + component: domains.get(component, ()) for component in AxisFamily.active().axes + } == expected assert matched == records assert domains[Microscopy.Site] == ("3",) assert domains[Microscopy.ZIndex] == ("4", "5") @@ -999,7 +1008,15 @@ def test_alias_only_step_bindings_filter_and_order_workspace_projections() -> No ) source_projections = { virtual_path: SourcePlaneProjection( - address=OpenHCSPlaneAddress(((Microscopy.Well, "R04C09"), (Microscopy.Site, "11"), (Microscopy.Channel, channel), (Microscopy.ZIndex, "1"), (Microscopy.Timepoint, "1"))), + address=OpenHCSPlaneAddress( + ( + (Microscopy.Well, "R04C09"), + (Microscopy.Site, "11"), + (Microscopy.Channel, channel), + (Microscopy.ZIndex, "1"), + (Microscopy.Timepoint, "1"), + ) + ), ref=SourcePixelRef("disk", f"/source/ch{channel}.tiff"), source_alias=binding.alias, ) @@ -1335,7 +1352,9 @@ def test_source_binding_members_canonicalize_declared_physical_spellings(aliases selector=SourceSelector(metadata=(MetadataSelector("channel", channel),)), component_identity=(ComponentSelector(Microscopy.Channel, channel),), ) - for alias, channel in zip(aliases, map(str, range(1, len(aliases) + 1)), strict=True) + for alias, channel in zip( + aliases, map(str, range(1, len(aliases) + 1)), strict=True + ) ) context = SourcePatternResolutionContext.from_sources( parser=SourceSchemaFilenameParser(), source_paths_by_virtual_path=paths @@ -1351,19 +1370,23 @@ def test_source_binding_members_canonicalize_declared_physical_spellings(aliases for binding, virtual_path in zip(bindings[1:], virtual_paths[1:], strict=True): assert matched_set.members_for_binding( binding, - anchor_provenance=SourceImageProvenance(source_path=paths[virtual_paths[0]]), + anchor_provenance=SourceImageProvenance( + source_path=paths[virtual_paths[0]] + ), source_universe=tuple(paths.values()), ) == (paths[virtual_path],) def test_declared_positions_keep_store_planes_and_same_basename_sources_distinct(): positions = ( - "A01_s001_w1_z001_t001.tif", "A01_s001_w1_z002_t001.tif", + "A01_s001_w1_z001_t001.tif", + "A01_s001_w1_z002_t001.tif", "A02_s001_w1_z001_t001.tif", ) paths = dict(zip(positions, ("/a/raw.tif", "/a/raw.tif", "/b/raw.tif"))) context = SourceIdentityResolutionContext.from_sources( - parser=SourceSchemaFilenameParser(), source_paths_by_virtual_path=paths, + parser=SourceSchemaFilenameParser(), + source_paths_by_virtual_path=paths, ) candidates = (*positions, *paths.values()) assert context.virtual_paths_for_sources( @@ -1371,14 +1394,16 @@ def test_declared_positions_keep_store_planes_and_same_basename_sources_distinct ) == (positions[:2], (positions[2],), ()) assert context.declared_positions_for_candidates(candidates) == positions assert context.matching_candidates_for_source_identities( - (SourceImageIdentity("/a/raw.tif"), - SourceImageIdentity("/a/raw.tif", {"z_index": "2"}), - SourceImageIdentity("/b/raw.tif"), SourceImageIdentity("/elsewhere/raw.tif")), + ( + SourceImageIdentity("/a/raw.tif"), + SourceImageIdentity("/a/raw.tif", {"z_index": "2"}), + SourceImageIdentity("/b/raw.tif"), + SourceImageIdentity("/elsewhere/raw.tif"), + ), candidates, ) == (positions[:2], (positions[1],), (positions[2],), ()) - def test_source_binding_members_collapse_lookup_aliases_of_same_projection(): relative = tuple(f"A01_s001_w1_z{z:03d}_t001.tif" for z in (0, 1)) full = tuple(f"/workspace/{path}" for path in relative) @@ -1386,8 +1411,17 @@ def test_source_binding_members_collapse_lookup_aliases_of_same_projection(): binding = NamedSourceBinding(alias="OriginalPhysical") projections = tuple( SourcePlaneProjection( - address=OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, "1"), (Microscopy.Channel, "1"), (Microscopy.ZIndex, z), (Microscopy.Timepoint, "1"))), - ref=SourcePixelRef("disk", source_path), source_alias=binding.alias, + address=OpenHCSPlaneAddress( + ( + (Microscopy.Well, "A01"), + (Microscopy.Site, "1"), + (Microscopy.Channel, "1"), + (Microscopy.ZIndex, z), + (Microscopy.Timepoint, "1"), + ) + ), + ref=SourcePixelRef("disk", source_path), + source_alias=binding.alias, ) for z in (0, 1) ) @@ -1397,15 +1431,20 @@ def test_source_binding_members_collapse_lookup_aliases_of_same_projection(): for spelling in (path, loadable) } workspace = VirtualWorkspaceSourceProjection( - source_refs_by_virtual_path={path: projection.ref for path, projection in typed.items()}, + source_refs_by_virtual_path={ + path: projection.ref for path, projection in typed.items() + }, source_metadata_by_path={full[0]: {"channel": "9"}}, source_projections_by_virtual_path=typed, ) context = SourcePatternResolutionContext.from_projection( - parser=SourceSchemaFilenameParser(), projection=workspace, + parser=SourceSchemaFilenameParser(), + projection=workspace, ) matched = SourceBindingMatchedImageSet.from_plan( - bindings=(binding,), match_plan=None, source_context=context, + bindings=(binding,), + match_plan=None, + source_context=context, identity_policy=SourceImageSetIdentityPolicy(), ) candidates = (*typed, source_path) @@ -1415,12 +1454,18 @@ def test_source_binding_members_collapse_lookup_aliases_of_same_projection(): source_universe=candidates, ) == (relative[0],) assert matched.matching_candidates_for_source_identities( - (SourceImageIdentity(source_path), SourceImageIdentity(full[0]), - SourceImageIdentity(source_path, {"channel": "9"})), candidates, + ( + SourceImageIdentity(source_path), + SourceImageIdentity(full[0]), + SourceImageIdentity(source_path, {"channel": "9"}), + ), + candidates, ) == (relative, (relative[0],), (relative[0],)) # A later declaration replacement is a new position even with equal fields. typed[full[0]] = SourcePlaneProjection( - address=projections[0].address, ref=projections[0].ref, source_alias=binding.alias, + address=projections[0].address, + ref=projections[0].ref, + source_alias=binding.alias, ) with pytest.raises(ValueError, match="one exact declared source-set position"): matched.members_for_binding( @@ -1429,6 +1474,7 @@ def test_source_binding_members_collapse_lookup_aliases_of_same_projection(): source_universe=candidates, ) + def test_source_binding_exact_position_projection_inherits_owning_context(): """Surface policy guard: declaration resolution is not reimplemented in leaves.""" assert ( @@ -1616,7 +1662,15 @@ def test_virtual_workspace_source_matching_uses_declared_filter_identity(): def test_virtual_workspace_source_matching_requires_exact_projection_binding(): - address = OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, "1"), (Microscopy.Channel, "1"), (Microscopy.ZIndex, "1"), (Microscopy.Timepoint, "1"))) + address = OpenHCSPlaneAddress( + ( + (Microscopy.Well, "A01"), + (Microscopy.Site, "1"), + (Microscopy.Channel, "1"), + (Microscopy.ZIndex, "1"), + (Microscopy.Timepoint, "1"), + ) + ) original_path = "A01_s001_w1_z001_t001.tif" illumination_path = f"_source/Illumination/{original_path}" original = SourcePlaneProjection( diff --git a/tests/unit/test_source_bound_matched_image_set_anchors.py b/tests/unit/test_source_bound_matched_image_set_anchors.py index 0949a7c74e..7db4fb9cc6 100644 --- a/tests/unit/test_source_bound_matched_image_set_anchors.py +++ b/tests/unit/test_source_bound_matched_image_set_anchors.py @@ -39,7 +39,8 @@ ) from openhcs.core.source_projection import OpenHCSPlaneAddress, SourcePlaneProjection from openhcs.core.source_workspace_projection import ( - VirtualWorkspaceSourceProjection, WorkspaceSourceProjections, + VirtualWorkspaceSourceProjection, + WorkspaceSourceProjections, ) from openhcs.core.step_dependencies import StepInputDependency from openhcs.core.steps.function_execution import ( @@ -76,7 +77,8 @@ def _anchor_executor( ) executor.context.runtime_source_workspace_projections = ( WorkspaceSourceProjections.from_context( - executor.context, cache=source_workspace_projection_cache, + executor.context, + cache=source_workspace_projection_cache, ) ) if output_manifest is not None: @@ -505,7 +507,15 @@ def identify_bodies(image): aliases = ("Hoechst", "SMI312") projections = { path: SourcePlaneProjection( - address=OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, "1"), (Microscopy.Channel, channel), (Microscopy.ZIndex, "1"), (Microscopy.Timepoint, "1"))), + address=OpenHCSPlaneAddress( + ( + (Microscopy.Well, "A01"), + (Microscopy.Site, "1"), + (Microscopy.Channel, channel), + (Microscopy.ZIndex, "1"), + (Microscopy.Timepoint, "1"), + ) + ), ref=SourcePixelRef("disk", f"/source/{path}"), source_alias=alias, ) @@ -617,7 +627,15 @@ def align_source_set(image): aliases = ("OrigStain1", "OrigStain2") * 2 projections = { path: SourcePlaneProjection( - address=OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, str(site)), (Microscopy.Channel, str(channel)), (Microscopy.ZIndex, "1"), (Microscopy.Timepoint, "1"))), + address=OpenHCSPlaneAddress( + ( + (Microscopy.Well, "A01"), + (Microscopy.Site, str(site)), + (Microscopy.Channel, str(channel)), + (Microscopy.ZIndex, "1"), + (Microscopy.Timepoint, "1"), + ) + ), ref=SourcePixelRef("disk", f"/source/{path}"), source_alias=alias, ) @@ -740,7 +758,15 @@ def align_source_set(image): ) projections = { path: SourcePlaneProjection( - address=OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, str(site)), (Microscopy.Channel, str(channel)), (Microscopy.ZIndex, "1"), (Microscopy.Timepoint, "1"))), + address=OpenHCSPlaneAddress( + ( + (Microscopy.Well, "A01"), + (Microscopy.Site, str(site)), + (Microscopy.Channel, str(channel)), + (Microscopy.ZIndex, "1"), + (Microscopy.Timepoint, "1"), + ) + ), ref=SourcePixelRef("disk", f"/source/{path}"), source_alias=f"OrigStain{channel}", ) diff --git a/tests/unit/test_source_provenance_constructor_defaults.py b/tests/unit/test_source_provenance_constructor_defaults.py index 1436a629b4..c5765616f7 100644 --- a/tests/unit/test_source_provenance_constructor_defaults.py +++ b/tests/unit/test_source_provenance_constructor_defaults.py @@ -51,7 +51,10 @@ def _carriers(): subject=MeasurementSubject(MeasurementScope.SAMPLE, "DNA"), ), partial( - SpatialGraph, name="Graph", nodes=(), edges=(), + SpatialGraph, + name="Graph", + nodes=(), + edges=(), coordinate_spacing=SourceVoxelSpacing((1.0, 1.0)), ), partial( diff --git a/tests/unit/test_source_spatial_domain.py b/tests/unit/test_source_spatial_domain.py index 5936690655..74f3c01a09 100644 --- a/tests/unit/test_source_spatial_domain.py +++ b/tests/unit/test_source_spatial_domain.py @@ -133,8 +133,11 @@ def test_xy_alignment_preserves_intrinsic_volume_owner(as_labels): ), ).payload_with(pixels, mask) value = ( - SourceImageObjectLabelBuildRequest(image=source, labels=pixels).label_set(name="Objects") - if as_labels else source + SourceImageObjectLabelBuildRequest(image=source, labels=pixels).label_set( + name="Objects" + ) + if as_labels + else source ) target = ImagePayloadMetadata( source_spatial_domain=SourceSpatialDomain( diff --git a/tests/unit/test_source_tile_geometry.py b/tests/unit/test_source_tile_geometry.py index 03ef0c1669..c914bbde2a 100644 --- a/tests/unit/test_source_tile_geometry.py +++ b/tests/unit/test_source_tile_geometry.py @@ -425,9 +425,16 @@ def for_source_plane(self, plane_index): return super().for_source_plane(plane_index) geometries = tuple( - SourceTileGeometry(column * 6 - 1.25, row * 4 + 0.5, - row=row, column=column, width_pixels=6, height_pixels=4) - for row in range(3) for column in range(3) + SourceTileGeometry( + column * 6 - 1.25, + row * 4 + 0.5, + row=row, + column=column, + width_pixels=6, + height_pixels=4, + ) + for row in range(3) + for column in range(3) ) function = OpenHCSRegistry.metadata_for_declared_callable( acquisition_tile_positions @@ -436,8 +443,10 @@ def for_source_plane(self, plane_index): raw_assemble = CallableContract.from_callable( assemble_stack_cpu ).resolve_raw_runtime_callable() - for channel, order in (("1", (8, 0, 4, 3, 1, 7, 2, 6, 5)), - ("2", (5, 2, 6, 1, 3, 4, 7, 0, 8))): + for channel, order in ( + ("1", (8, 0, 4, 3, 1, 7, 2, 6, 5)), + ("2", (5, 2, 6, 1, 3, 4, 7, 0, 8)), + ): original = _metadata(geometries, order, channel) metadata = ProjectionAuditMetadata( source_provenance=original.source_provenance, @@ -445,16 +454,20 @@ def for_source_plane(self, plane_index): source_voxel_spacing=SourceVoxelSpacing((0.25, 0.5)), ) pixels = np.stack([np.full((4, 6), site + 1, np.uint16) for site in order]) - result, positions = function(ImageMetadataPayload(pixels, metadata), - source_metadata=metadata) + result, positions = function( + ImageMetadataPayload(pixels, metadata), source_metadata=metadata + ) np.testing.assert_array_equal(result, pixels) - assert positions == [(geometries[site].x_pixels, geometries[site].y_pixels) - for site in order] + assert positions == [ + (geometries[site].x_pixels, geometries[site].y_pixels) for site in order + ] assert result.metadata.source_voxel_spacing == metadata.source_voxel_spacing assert result.metadata.source_provenance == metadata.source_provenance assert calls == list(range(9)) calls.clear() - assembled.append(raw_assemble(pixels, positions, blend_method=TileBlendMethod.NONE)) + assembled.append( + raw_assemble(pixels, positions, blend_method=TileBlendMethod.NONE) + ) assert assembled[0].shape == assembled[1].shape == (13, 19) @@ -601,9 +614,7 @@ def test_synthetic_pipeline_compiles_and_executes_positions_artifact_then_paired assert any( edge.spec.name == "positions" for edge in consumer.artifact_input_edges ) - assert context.step_plans[2].variable_components == [ - Microscopy.Channel - ] + assert context.step_plans[2].variable_components == [Microscopy.Channel] result = orchestrator.execute_compiled_bundle( execution_bundle=compilation, progress_queue=progress, @@ -662,9 +673,7 @@ def test_synthetic_pipeline_compiles_and_executes_positions_artifact_then_paired (0.25, 0.5) ) assert metadata.plane_axis is None - replay_config = PreparedWorkspaceSource.pipeline_config_for_source( - config - ) + replay_config = PreparedWorkspaceSource.pipeline_config_for_source(config) reopened = PipelineOrchestrator( output_dataset, pipeline_config=replay_config ).initialize() diff --git a/tests/unit/test_stream_component_semantics.py b/tests/unit/test_stream_component_semantics.py index 3def580922..76d8b53712 100644 --- a/tests/unit/test_stream_component_semantics.py +++ b/tests/unit/test_stream_component_semantics.py @@ -42,7 +42,9 @@ def test_streaming_domain_spans_worker_lanes_without_mirroring_worker_identity() context.bind_execution_runtime(lane) assert context.execution_runtime is lane assert lane.worker_assignments is assignments.worker_assignments - assert lane.owned_partition_values == tuple(assignments.worker_assignments[slot]) + assert lane.owned_partition_values == tuple( + assignments.worker_assignments[slot] + ) provider = StreamExecutionAxisDomainProvider.build_for_component( context=context, component=AxisFamily.active().partition_axis().name, @@ -50,9 +52,7 @@ def test_streaming_domain_spans_worker_lanes_without_mirroring_worker_identity() ) domains.append(provider.domain_metadata_items()) assert domains[0] == domains[1] - assert { - item[AxisFamily.active().partition_axis().name] for item in domains[0] - } == { + assert {item[AxisFamily.active().partition_axis().name] for item in domains[0]} == { "R02C05", "R04C04", "R04C02", @@ -215,17 +215,21 @@ def test_retained_stream_domain_includes_exact_planes_not_scalar_address(compone ) scalar = SourceImageIdentity(component_metadata={"well": "A01"}) observations = StreamSourceComponentMetadataItems.from_image_metadata( - metadata, fallback_source_identity=scalar, + metadata, + fallback_source_identity=scalar, ) assert observations.complete_component_order(("well", component, "timepoint")) == ( - "well", component, + "well", + component, ) assert observations.domain_metadata_items(("well", component)) == ( - {"well": "A01", component: 3}, {"well": "A01", component: 1}, + {"well": "A01", component: 3}, + {"well": "A01", component: 1}, ) fields = StreamImagePayloadMetadataProjector.item_fields(metadata, (component,)) route_metadata = StreamViewerComponentMetadataProjector.for_item_fields( - ("well", component), fields, + ("well", component), + fields, ).indexed_source_metadata((scalar.component_metadata,)) # Two plane observations still describe ONE streamed image/route address. assert route_metadata.metadata_by_index == ({"well": "A01"},) @@ -233,10 +237,12 @@ def test_retained_stream_domain_includes_exact_planes_not_scalar_address(compone def test_collapsed_contributors_do_not_restore_a_stream_pixel_axis(): metadata = _plane_metadata( - RuntimePlaneAxis.RUNTIME_SLICE, ({"site": "3"}, {"site": "1"}), + RuntimePlaneAxis.RUNTIME_SLICE, + ({"site": "3"}, {"site": "1"}), ).collapse_leading_plane_axis() observations = StreamSourceComponentMetadataItems.from_image_metadata( - metadata, fallback_source_identity=SourceImageIdentity( + metadata, + fallback_source_identity=SourceImageIdentity( component_metadata={"well": "A01"}, ), ) diff --git a/tests/unit/test_streaming_service.py b/tests/unit/test_streaming_service.py index 4acc8a7537..ee33547c0b 100644 --- a/tests/unit/test_streaming_service.py +++ b/tests/unit/test_streaming_service.py @@ -700,7 +700,10 @@ def source_path( "A49", component=None, value=None, - fixed_component_values=((Microscopy.ZIndex, "1"), (Microscopy.Timepoint, "1")), + fixed_component_values=( + (Microscopy.ZIndex, "1"), + (Microscopy.Timepoint, "1"), + ), ), ) workspace_projection = VirtualWorkspaceSourceProjection( @@ -1270,11 +1273,13 @@ def lifecycle_manager(monkeypatch): ManagedViewerLifecycleMixin, "wait_for_ready", lambda self, timeout: True ) monkeypatch.setattr( - ManagedViewerLifecycleMixin, "force_stop", + ManagedViewerLifecycleMixin, + "force_stop", lambda self: self.lifecycle_state.mark_stopped(), ) monkeypatch.setattr( - ManagedViewerLifecycleMixin, "start", + ManagedViewerLifecycleMixin, + "start", lambda self: (_ for _ in ()).throw(AssertionError("must not restart")), ) manager = ViewerStateManager.get_instance() @@ -1283,7 +1288,8 @@ def lifecycle_manager(monkeypatch): def test_streaming_viewer_lifecycle_attaches_existing_viewer_without_restart( - monkeypatch, lifecycle_manager, + monkeypatch, + lifecycle_manager, ) -> None: monkeypatch.setattr( NapariStreamVisualizer, @@ -1396,19 +1402,25 @@ def fail_after_factory(*, factory, **_kwargs): def test_streaming_viewer_lifecycle_admits_new_launch_inside_managed_acquisition( monkeypatch, lifecycle_manager, config_type, owns_process, requested_host, reusable ) -> None: - active_config = config_type(enabled=True, port=5563, persistent=True, listen_host="*") + active_config = config_type( + enabled=True, port=5563, persistent=True, listen_host="*" + ) existing_viewer = active_config.create_visualizer(FakeFileManager()) existing_viewer.lifecycle_state.mark_connected_external() monkeypatch.setattr( - ManagedViewerLifecycleMixin, "owned_viewer_process_is_alive", + ManagedViewerLifecycleMixin, + "owned_viewer_process_is_alive", lambda self: owns_process, ) monkeypatch.setattr( - ViewerControlMessageRequest, "send", - lambda self: ViewerControlResponse({ - "status": "success", - "process_launch": active_config.viewer_process_launch_config().to_wire_mapping(), - }), + ViewerControlMessageRequest, + "send", + lambda self: ViewerControlResponse( + { + "status": "success", + "process_launch": active_config.viewer_process_launch_config().to_wire_mapping(), + } + ), ) lifecycle_manager.get_or_create_viewer( active_config.viewer_family.wire_value, 5563, lambda: existing_viewer @@ -1425,14 +1437,24 @@ def test_streaming_viewer_lifecycle_admits_new_launch_inside_managed_acquisition enabled=True, port=5563, persistent=True, listen_host=requested_host ) if reusable: - assert StreamingViewerLifecycle.get_or_create_visualizer( - filemanager=FakeFileManager(), config=requested, fresh=False, - ) is existing_viewer + assert ( + StreamingViewerLifecycle.get_or_create_visualizer( + filemanager=FakeFileManager(), + config=requested, + fresh=False, + ) + is existing_viewer + ) else: - with pytest.raises(RuntimeError, match="does not match the requested process launch"): + with pytest.raises( + RuntimeError, match="does not match the requested process launch" + ): StreamingViewerLifecycle.get_or_create_visualizer( - filemanager=FakeFileManager(), config=requested, fresh=False, + filemanager=FakeFileManager(), + config=requested, + fresh=False, ) - assert lifecycle_manager.get_viewer( - active_config.viewer_family.wire_value, 5563 - ) is existing_viewer + assert ( + lifecycle_manager.get_viewer(active_config.viewer_family.wire_value, 5563) + is existing_viewer + ) diff --git a/tests/unit/test_u1_session_guards.py b/tests/unit/test_u1_session_guards.py index cdcbdbdf3f..e6418f545a 100644 --- a/tests/unit/test_u1_session_guards.py +++ b/tests/unit/test_u1_session_guards.py @@ -56,9 +56,11 @@ def test_no_widget_holds_session_state() -> None: targets = ( node.targets if isinstance(node, ast.Assign) - else [node.target] - if isinstance(node, (ast.AnnAssign, ast.AugAssign)) - else [] + else ( + [node.target] + if isinstance(node, (ast.AnnAssign, ast.AugAssign)) + else [] + ) ) for target in targets: if ( @@ -67,7 +69,9 @@ def test_no_widget_holds_session_state() -> None: and target.value.id == "self" and target.attr in SESSION_STATE_ATTRIBUTES ): - offenders.append(f"{path.relative_to(PACKAGE)}:{node.lineno} self.{target.attr}") + offenders.append( + f"{path.relative_to(PACKAGE)}:{node.lineno} self.{target.attr}" + ) assert offenders == [] @@ -78,13 +82,13 @@ def test_session_package_imports_no_qt_and_no_gui() -> None: names = ( [alias.name for alias in node.names] if isinstance(node, ast.Import) - else [node.module or ""] - if isinstance(node, ast.ImportFrom) - else [] + else [node.module or ""] if isinstance(node, ast.ImportFrom) else [] ) for name in names: if name.split(".")[0] == "PyQt6" or name.startswith("openhcs.pyqt_gui"): - offenders.append(f"{path.relative_to(PACKAGE)}:{node.lineno} {name}") + offenders.append( + f"{path.relative_to(PACKAGE)}:{node.lineno} {name}" + ) assert offenders == [] @@ -115,7 +119,9 @@ def test_actions_are_operation_classes_not_enum_rosters() -> None: if isinstance(node, ast.Lambda) and [ argument.arg for argument in node.args.args ] in (["widget"], ["window"], ["manager"]): - offenders.append(f"{path.relative_to(PACKAGE)}:{node.lineno} lambda") + offenders.append( + f"{path.relative_to(PACKAGE)}:{node.lineno} lambda" + ) for path, tree in _modules(SESSION_PACKAGE): for node in ast.walk(tree): if isinstance(node, ast.ClassDef) and any( @@ -129,12 +135,19 @@ def test_actions_are_operation_classes_not_enum_rosters() -> None: def test_every_operation_declares_itself_and_headless_ones_are_tools() -> None: from openhcs.agent.capabilities import get_capability_registry - tools = {capability.name: capability for capability in get_capability_registry().capabilities} + tools = { + capability.name: capability + for capability in get_capability_registry().capabilities + } for operation in SessionOperation.all(): - assert operation.label and operation.tooltip and operation.description, operation + assert ( + operation.label and operation.tooltip and operation.description + ), operation assert isinstance(operation.request, type), operation tool_name = f"openhcs_{operation.operation_id}" - assert (tool_name in tools) is issubclass(operation, HeadlessOperation), operation + assert (tool_name in tools) is issubclass( + operation, HeadlessOperation + ), operation if issubclass(operation, RendererOperation): assert not issubclass(operation, HeadlessOperation), operation if tool_name in tools: @@ -147,7 +160,7 @@ def test_pipeline_formats_are_importers_not_suffix_branches() -> None: for root in (PACKAGE / "pyqt_gui", SESSION_PACKAGE): for path in sorted(root.rglob("*.py")): text = path.read_text(encoding="utf-8") - for marker in ('== ".cppipe"', '!= ".cppipe"', '#openhcs-cppipe='): + for marker in ('== ".cppipe"', '!= ".cppipe"', "#openhcs-cppipe="): if marker in text: offenders.append(f"{path.relative_to(PACKAGE)} {marker}") assert offenders == [] diff --git a/tests/unit/test_unified_registry_runtime_payloads.py b/tests/unit/test_unified_registry_runtime_payloads.py index c403f5ca63..2869b9a344 100644 --- a/tests/unit/test_unified_registry_runtime_payloads.py +++ b/tests/unit/test_unified_registry_runtime_payloads.py @@ -233,7 +233,9 @@ def test_native_flexible_callable_receives_declared_array_abi_and_keeps_slice_co ) -> None: stack = np.arange(40, dtype=np.float32).reshape(2, 4, 5) mask = np.ones_like(stack, dtype=bool) - payload = ImagePayloadMetadata(plane_axis=RuntimePlaneAxis.RUNTIME_SLICE).payload_with(stack, mask) + payload = ImagePayloadMetadata( + plane_axis=RuntimePlaneAxis.RUNTIME_SLICE + ).payload_with(stack, mask) seen_shapes = [] @numpy @@ -243,10 +245,13 @@ def native_image_operation(image: np.ndarray) -> np.ndarray: return image + 1 invocation = compile_function_pattern( - (native_image_operation, {"slice_by_slice": slice_by_slice}), {}, {}, + (native_image_operation, {"slice_by_slice": slice_by_slice}), + {}, + {}, ).default_group.invocations[0] result = invocation.runtime_callable( - invocation.main_flow_call_argument(payload), **dict(invocation.runtime_kwargs), + invocation.main_flow_call_argument(payload), + **dict(invocation.runtime_kwargs), ) assert seen_shapes == ([(4, 5), (4, 5)] if slice_by_slice else [(2, 4, 5)]) np.testing.assert_array_equal(result, stack + 1) @@ -254,13 +259,17 @@ def native_image_operation(image: np.ndarray) -> np.ndarray: def test_native_flexible_callable_preserves_explicitly_declared_carrier_abi() -> None: - payload = ImagePayloadMetadata().payload_with(np.ones((2, 4, 5)), np.ones((2, 4, 5), dtype=bool)) + payload = ImagePayloadMetadata().payload_with( + np.ones((2, 4, 5)), np.ones((2, 4, 5), dtype=bool) + ) @numpy def carrier_operation(image: RuntimeArrayPayload) -> RuntimeArrayPayload: return image - invocation = compile_function_pattern(carrier_operation, {}, {}).default_group.invocations[0] + invocation = compile_function_pattern( + carrier_operation, {}, {} + ).default_group.invocations[0] assert invocation.main_flow_call_argument(payload) is payload @@ -287,7 +296,8 @@ def add_one(image: np.ndarray) -> np.ndarray: return image + 1 wrapped = MinimalRegistry("minimal").apply_contract_wrapper( - add_one, FlexibleContract, + add_one, + FlexibleContract, ) contract = CallableContract.from_callable(wrapped) call_argument = contract.main_flow_call_argument(payload) @@ -297,7 +307,8 @@ def add_one(image: np.ndarray) -> np.ndarray: np.testing.assert_array_equal(result.data, stack + 1) assert result.metadata.plane_axis is RuntimePlaneAxis.RUNTIME_SLICE assert result.metadata.source_image_provenance_planes.paths == ( - "z0.tif", "z1.tif", + "z0.tif", + "z1.tif", ) @@ -327,7 +338,9 @@ def apply_mask( seen_shapes.append(mask_data.shape) true_y, true_x = np.argwhere(mask_data)[0] seen_true_indices.append((int(true_y), int(true_x))) - return image.with_pixels(np.where(mask_data, image_data + 100, image_data),) + return image.with_pixels( + np.where(mask_data, image_data + 100, image_data), + ) apply_mask.output_memory_type = MEMORY_TYPE_NUMPY @@ -456,7 +469,9 @@ def test_with_pixels_rejects_implicit_channel_mask_collapse() -> None: ) with pytest.raises(ValueError, match="does not match declared image mask domain"): - source.with_pixels(np.ones((4, 5), dtype=np.float32),) + source.with_pixels( + np.ones((4, 5), dtype=np.float32), + ) def test_runtime_callable_invocation_can_call_raw_signature_filtered_callable() -> None: @@ -466,9 +481,7 @@ def test_runtime_callable_invocation_can_call_raw_signature_filtered_callable() def raw(image: RuntimeArrayPayload, *, scale: int) -> RuntimeArrayPayload: assert isinstance(image, RuntimeArrayPayload) - return image.metadata.payload_with( - image.data * scale, image.mask - ) + return image.metadata.payload_with(image.data * scale, image.mask) @wraps(raw) def decorated(image: Any, **kwargs: Any) -> np.ndarray: @@ -484,9 +497,7 @@ def decorated(image: Any, **kwargs: Any) -> np.ndarray: assert isinstance(result, RuntimeArrayPayload) np.testing.assert_array_equal(result.data, np.full((4, 5), 3.0)) - np.testing.assert_array_equal( - result.mask, np.ones((4, 5), dtype=bool) - ) + np.testing.assert_array_equal(result.mask, np.ones((4, 5), dtype=bool)) def test_raw_runtime_callable_preserves_request_binding_semantics() -> None: diff --git a/tests/unit/test_viewer_receipt_replay.py b/tests/unit/test_viewer_receipt_replay.py index 4b0efd499b..d26a67a5e8 100644 --- a/tests/unit/test_viewer_receipt_replay.py +++ b/tests/unit/test_viewer_receipt_replay.py @@ -49,16 +49,18 @@ from openhcs.core.axes import ColourAxis - def _plate_context(plate_path, filemanager) -> PlateInspectionContext: """A plate whose metadata owner anchors workspace addresses at the plate.""" return PlateInspectionContext( plate_path=plate_path, filemanager=filemanager, - handler=SimpleNamespace(metadata_handler=SimpleNamespace(source_workspace_root=Path)), + handler=SimpleNamespace( + metadata_handler=SimpleNamespace(source_workspace_root=Path) + ), parser=None, ) + def receipt_state(path, *, summary=None): fixed = { component.name: 1 @@ -66,16 +68,22 @@ def receipt_state(path, *, summary=None): if component is not Microscopy.Channel } fixed[Microscopy.Well.name] = "1" - summary = summary if summary is not None else ViewerPayloadSummary.from_wire_mapping(dict( - path=str(path), - data_type="image", - components=fixed, - aggregate_component_values={"channel": [2, 1]}, - shape=[2, 8, 9], - source_spatial_shape_yx=[8, 9], - spatial_origin_yx=[0, 0], - dtype="int32", - )) + summary = ( + summary + if summary is not None + else ViewerPayloadSummary.from_wire_mapping( + dict( + path=str(path), + data_type="image", + components=fixed, + aggregate_component_values={"channel": [2, 1]}, + shape=[2, 8, 9], + source_spatial_shape_yx=[8, 9], + spatial_origin_yx=[0, 0], + dtype="int32", + ) + ) + ) layer = ViewerWindowLayerState( route_key="native_historical_labels", title="Labels", @@ -121,9 +129,7 @@ def test_receipt_codec_and_exact_path_admission(tmp_path): ) -@pytest.mark.parametrize( - "change", ["transform", "truncated", "window", "producer"] -) +@pytest.mark.parametrize("change", ["transform", "truncated", "window", "producer"]) def test_receipt_incomplete_or_conflicting_facts_refuse(tmp_path, change): state = receipt_state(tmp_path / "aggregate.labels.tif") layer = state.layers[0] @@ -142,10 +148,16 @@ def test_receipt_incomplete_or_conflicting_facts_refuse(tmp_path, change): ) -@pytest.mark.parametrize("field,value", [ - ("spatial_origin_yx", [False, 0]), ("shape", [True, 8, 9]), -]) -def test_receipt_boolean_spatial_facts_refuse_at_original_typed_ingress(tmp_path, field, value): +@pytest.mark.parametrize( + "field,value", + [ + ("spatial_origin_yx", [False, 0]), + ("shape", [True, 8, 9]), + ], +) +def test_receipt_boolean_spatial_facts_refuse_at_original_typed_ingress( + tmp_path, field, value +): wire = to_jsonable(receipt_state(tmp_path / "aggregate.labels.tif")) wire["layers"][0]["payload_summaries"][0][field] = value with pytest.raises(TypeError): @@ -174,28 +186,34 @@ def test_observed_and_declared_wire_domains_have_distinct_order_contracts(): ) observed = ViewerComponentValueDomainPayload.from_wire_mapping( - {"channel": ["2", 1, 2]}, context="observed", + {"channel": ["2", 1, 2]}, + context="observed", declared_axes=STREAM_AXES, ) assert observed.to_wire_mapping() == {"channel": [1, 2]} declared = ViewerComponentValueDomainPayload.from_ordered_wire_mapping( - {"channel": ["2", 1]}, context="declared", + {"channel": ["2", 1]}, + context="declared", declared_axes=STREAM_AXES, ) assert declared.to_wire_mapping() == {"channel": [2, 1]} with pytest.raises(ValueError, match="unique after normalization"): ViewerComponentValueDomainPayload.from_ordered_wire_mapping( - {"channel": ["01", 1]}, context="malformed declared", + {"channel": ["01", 1]}, + context="malformed declared", declared_axes=STREAM_AXES, ) @pytest.mark.parametrize("historical_dtype", [None, "int32"]) -@pytest.mark.parametrize("spacing", ( - SourceVoxelSpacing(), - SourceVoxelSpacing((2.0, 0.65, 0.65)), - SourceVoxelSpacing((2.0, 1.0, 0.75), SourceVoxelSpacingUnit.RELATIVE), -)) +@pytest.mark.parametrize( + "spacing", + ( + SourceVoxelSpacing(), + SourceVoxelSpacing((2.0, 0.65, 0.65)), + SourceVoxelSpacing((2.0, 1.0, 0.75), SourceVoxelSpacingUnit.RELATIVE), + ), +) def test_native_persisted_aggregate_source_projection_preserves_order( tmp_path, historical_dtype, spacing ): @@ -206,11 +224,14 @@ def test_native_persisted_aggregate_source_projection_preserves_order( from openhcs.core.runtime_image_values import ImagePayloadMetadata from openhcs.core.source_spatial_domain import SourceSpatialDomain from openhcs.runtime.napari_streaming_handlers import ( - NapariAggregateAxisBindingBuilder, NapariStreamLayerAddress, NapariStreamLayerItem, + NapariAggregateAxisBindingBuilder, + NapariStreamLayerAddress, + NapariStreamLayerItem, ) from openhcs.runtime.napari_viewer_server import NapariViewerStateProjection from openhcs.runtime.viewer_component_system import ( - ViewerComponentAxisSemanticsFactory, ViewerComponentValueDomainPayload, + ViewerComponentAxisSemanticsFactory, + ViewerComponentValueDomainPayload, ViewerMappingDisplayConfigInput, ) @@ -220,32 +241,43 @@ def test_native_persisted_aggregate_source_projection_preserves_order( producer=fixture.layers[0].producer_identities[0], address=NapariStreamLayerAddress( components=fixture.layers[0].payload_summaries[0].components, - path=str(path), stream_layer_data_type=StreamingDataType.IMAGE, + path=str(path), + stream_layer_data_type=StreamingDataType.IMAGE, ), image_metadata=ImagePayloadMetadata( source_voxel_spacing=spacing, source_spatial_domain=SourceSpatialDomain( - origin_yx=(0, 0), source_shape_yx=(8, 9), + origin_yx=(0, 0), + source_shape_yx=(8, 9), ), ), plane_component_domain=ViewerComponentValueDomainPayload.from_ordered_wire_mapping( - {"channel": [2, 1]}, context="synthetic native receipt", + {"channel": [2, 1]}, + context="synthetic native receipt", declared_axes=STREAM_AXES, ), ) native_semantics = ViewerComponentAxisSemanticsFactory.from_display_config( ViewerMappingDisplayConfigInput( - {"component_modes": {"channel": "stack"}, "component_order": ["channel"], "declared_axes": STREAM_AXES.to_wire()} + { + "component_modes": {"channel": "stack"}, + "component_order": ["channel"], + "declared_axes": STREAM_AXES.to_wire(), + } ), ViewerComponentValueDomainPayload.from_wire_mapping( - {"channel": [1, 2]}, context="synthetic observed receipt domain", + {"channel": [1, 2]}, + context="synthetic observed receipt domain", declared_axes=STREAM_AXES, ), ) summary = NapariViewerStateProjection.payload_summary( - native_item, native_item.address.components, expected, + native_item, + native_item.address.components, + expected, aggregate_axis_bindings=NapariAggregateAxisBindingBuilder.bindings( - [native_item], native_semantics, + [native_item], + native_semantics, ), ) assert summary.full_image_plane_count == 2 @@ -272,14 +304,16 @@ def test_native_persisted_aggregate_source_projection_preserves_order( state = replace( state, viewer_ndim=len(presentation.axis_labels), - layers=(replace( - state.layers[0], - axis_labels=presentation.axis_labels, - native_transform=ViewerNativeLayerTransform( - scale=coordinate_kwargs["scale"], - translate=coordinate_kwargs["translate"], + layers=( + replace( + state.layers[0], + axis_labels=presentation.axis_labels, + native_transform=ViewerNativeLayerTransform( + scale=coordinate_kwargs["scale"], + translate=coordinate_kwargs["translate"], + ), ), - ),), + ), ) assert summary.voxel_spacing == spacing data = json.dumps(to_jsonable(state), sort_keys=True).encode() @@ -335,19 +369,29 @@ def test_native_persisted_aggregate_source_projection_preserves_order( assert metadata.source_dtype == "int64" from openhcs.agent.capabilities import GetViewerWindowStateCapability from openhcs.mcp.dev_client_core import ( - McpDevServerIdentity, McpDevToolBatchResponse, McpDevToolResult, + McpDevServerIdentity, + McpDevToolBatchResponse, + McpDevToolResult, ) from openhcs.mcp.dev_client_rendering import McpDevOutputRenderer - public_wire = to_jsonable(McpDevToolBatchResponse( - server=McpDevServerIdentity(command="source-only", module="openhcs.mcp"), - results=(McpDevToolResult(GetViewerWindowStateCapability.name, False, (state,)),), - )) + public_wire = to_jsonable( + McpDevToolBatchResponse( + server=McpDevServerIdentity(command="source-only", module="openhcs.mcp"), + results=( + McpDevToolResult(GetViewerWindowStateCapability.name, False, (state,)), + ), + ) + ) decoded = McpDevToolBatchResponse.for_rendering(public_wire) received = decoded.payload_for(GetViewerWindowStateCapability) assert isinstance(received.layers[0].payload_summaries[0], ViewerPayloadSummary) - assert received.layers[0].payload_summaries[0].require_plane_components() == {"channel": (2, 1)} - renderer = McpDevOutputRenderer.for_output_contract(GetViewerWindowStateCapability.output_contract) + assert received.layers[0].payload_summaries[0].require_plane_components() == { + "channel": (2, 1) + } + renderer = McpDevOutputRenderer.for_output_contract( + GetViewerWindowStateCapability.output_contract + ) compact = renderer.render(decoded) assert '"aggregate_component_values": {"channel": [2, 1]}' in compact assert '"source_spatial_shape_yx": [8, 9]' in compact @@ -356,7 +400,8 @@ def test_native_persisted_aggregate_source_projection_preserves_order( ) domain = ViewerComponentValueDomainPayload.from_ordered_wire_mapping( - metadata.retained_plane_component_values(), context="native receipt domain", + metadata.retained_plane_component_values(), + context="native receipt domain", declared_axes=STREAM_AXES, ) assert domain.to_wire_mapping() == {"channel": [2, 1]} @@ -388,12 +433,17 @@ def test_native_persisted_aggregate_source_projection_preserves_order( from zmqruntime.viewer_protocol import ViewerWirePayload observed = ViewerComponentValueDomainPayload.from_wire_mapping( - {"channel": [1, 2]}, context="observed stack domain", + {"channel": [1, 2]}, + context="observed stack domain", declared_axes=STREAM_AXES, ) semantics = ViewerComponentAxisSemanticsFactory.from_display_config( ViewerMappingDisplayConfigInput( - {"component_modes": {"channel": "stack"}, "component_order": ["channel"], "declared_axes": STREAM_AXES.to_wire()} + { + "component_modes": {"channel": "stack"}, + "component_order": ["channel"], + "declared_axes": STREAM_AXES.to_wire(), + } ), observed, ) @@ -459,8 +509,13 @@ def test_native_persisted_aggregate_source_projection_preserves_order( ) -@pytest.mark.parametrize("refusal", ["unowned", "missing_domain", "null_domain", "empty_axis", "ambiguous_domain"]) -def test_receipt_projection_refuses_unowned_or_invalid_declared_source(tmp_path, refusal): +@pytest.mark.parametrize( + "refusal", + ["unowned", "missing_domain", "null_domain", "empty_axis", "ambiguous_domain"], +) +def test_receipt_projection_refuses_unowned_or_invalid_declared_source( + tmp_path, refusal +): path = tmp_path / "aggregate.labels.tif" state = receipt_state(path) wire = to_jsonable(state) @@ -490,8 +545,13 @@ def test_receipt_projection_refuses_unowned_or_invalid_declared_source(tmp_path, ) service = PlateStreamingService(inspection, object()) context = _plate_context(tmp_path, FileManager({"disk": DiskStorageBackend()})) - source_ref = None if refusal == "unowned" else SourcePixelRef( - backend="disk", backend_address=str(path), + source_ref = ( + None + if refusal == "unowned" + else SourcePixelRef( + backend="disk", + backend_address=str(path), + ) ) expected_message = { "unowned": "inventory source reference", @@ -503,7 +563,11 @@ def test_receipt_projection_refuses_unowned_or_invalid_declared_source(tmp_path, with pytest.raises(ValueError, match=expected_message): service._receipt_source_projection( PlateFileStreamRequest(plate_path=str(tmp_path), source_receipt=resource), - (PlateFileRecord(kind=PlateFileKind.IMAGE, key=str(path), source_ref=source_ref),), + ( + PlateFileRecord( + kind=PlateFileKind.IMAGE, key=str(path), source_ref=source_ref + ), + ), context, ) @@ -558,11 +622,15 @@ def stream_files(self, request, *, ui_bridge_connection): assert admitted_connection.descriptor_file_path == str(tmp_path / "exact-ui.json") -@pytest.mark.parametrize("change", ("scale", "missing_axes", "duplicate_axes", "translation", "crop")) +@pytest.mark.parametrize( + "change", ("scale", "missing_axes", "duplicate_axes", "translation", "crop") +) def test_calibrated_receipt_rejects_conflicting_native_placement(tmp_path, change): spacing = SourceVoxelSpacing((2.0, 0.65, 0.65)) state = receipt_state(tmp_path / "aggregate.labels.tif") - summary = replace(state.layers[0].payload_summaries[0], source_voxel_spacing=spacing) + summary = replace( + state.layers[0].payload_summaries[0], source_voxel_spacing=spacing + ) axes = ("channel", "z_index", "timepoint", "site", "well", "y", "x") transform = ViewerNativeLayerTransform( scale=spacing.layer_coordinate_kwargs(axes)["scale"], translate=(0,) * 7 @@ -577,9 +645,16 @@ def test_calibrated_receipt_rejects_conflicting_native_placement(tmp_path, chang axes = ("z_index", "z_index", "timepoint", "site", "well", "y", "x") else: summary = replace(summary, spatial_origin_yx=(1, 0)) - state = replace(state, layers=(replace( - state.layers[0], axis_labels=axes, - payload_summaries=(summary,), native_transform=transform, - ),)) + state = replace( + state, + layers=( + replace( + state.layers[0], + axis_labels=axes, + payload_summaries=(summary,), + native_transform=transform, + ), + ), + ) with pytest.raises(ValueError): state.image_payload_binding_for(str(tmp_path / "aggregate.labels.tif")) diff --git a/tests/unit/test_viewer_result_element_selection.py b/tests/unit/test_viewer_result_element_selection.py index 37bb3b8707..1c92a04679 100644 --- a/tests/unit/test_viewer_result_element_selection.py +++ b/tests/unit/test_viewer_result_element_selection.py @@ -174,13 +174,17 @@ def _viewer_server(viewer, layer, route_key: str = "result-rois", group_binding= NapariStreamLayerItem( data=layer.data, producer=StreamProducerIdentity( - origin="manual", output_kind="fixture", output_key=route_key, + origin="manual", + output_kind="fixture", + output_key=route_key, projection_key=route_key, ), address=NapariStreamLayerAddress( - components={}, path=f"fixture/{route_key}", + components={}, + path=f"fixture/{route_key}", stream_layer_data_type=( - StreamingDataType.POINTS if isinstance(layer, Points) + StreamingDataType.POINTS + if isinstance(layer, Points) else StreamingDataType.SHAPES ), ), @@ -242,7 +246,9 @@ def test_linked_selection_projects_off_slice_members_without_changing_subject(qt paths = [np.array([[0, i, i], [0, i + 1, i + 1]], dtype=float) for i in range(2)] layer = viewer.add_shapes(paths, shape_type="path", features={"owner": [8, 8]}) linked_paths = [coordinates + [1, 0, 0] for coordinates in paths] - linked = viewer.add_shapes(linked_paths, shape_type="path", features={"owner": [8, 8]}) + linked = viewer.add_shapes( + linked_paths, shape_type="path", features={"owner": [8, 8]} + ) viewer.dims.current_step = (0, 0, 0) binding = NapariResultSelectionGroupBinding("native-test-subject", "owner") server, _, _, _ = _viewer_server(viewer, layer, group_binding=binding) @@ -254,7 +260,8 @@ def test_linked_selection_projects_off_slice_members_without_changing_subject(qt assert state.selected_data_indices == (0, 1) assert linked.selected_data == set() assert server.result_selection_controller._linked_group_members(layer, 1) == ( - (layer, (0, 1)), (linked, (0, 1)), + (layer, (0, 1)), + (linked, (0, 1)), ) assert viewer.dims.current_step == (0, 0, 0) for before, after in zip(original_data, linked.data, strict=True): @@ -268,10 +275,12 @@ def test_linked_points_use_the_same_native_slice_projection(qtbot): viewer.add_image(np.zeros((2, 8, 8), dtype=np.uint8)) layer = viewer.add_shapes( [np.array([[0, 0, 0], [0, 1, 1]], dtype=float)], - shape_type="path", features={"owner": [8]}, + shape_type="path", + features={"owner": [8]}, ) linked = viewer.add_points( - np.array([[0, 2, 2], [1, 3, 3]], dtype=float), features={"owner": [8, 8]}, + np.array([[0, 2, 2], [1, 3, 3]], dtype=float), + features={"owner": [8, 8]}, ) viewer.dims.current_step = (0, 0, 0) binding = NapariResultSelectionGroupBinding("native-test-subject", "owner") @@ -283,7 +292,8 @@ def test_linked_points_use_the_same_native_slice_projection(qtbot): assert state.selected_data_indices == (0,) assert linked.selected_data == {0} assert server.result_selection_controller._linked_group_members(layer, 0) == ( - (layer, (0,)), (linked, (0, 1)), + (layer, (0,)), + (linked, (0, 1)), ) @@ -681,8 +691,10 @@ def test_point_coordinate_owner_preserves_strict_acquisition_admission(coordinat def test_shapes_coordinate_owner_stays_strict_for_hidden_spatial_axes(displayed): with pytest.raises(ValueError, match="integral slice"): ViewerResultElementCoordinates.axis_indices( - coordinates=((1.5, 2.5, 3.5),), axis_labels=("z_index", "y", "x"), - displayed_axis_indices=displayed, spatial_axis_labels=("z_index", "y", "x"), + coordinates=((1.5, 2.5, 3.5),), + axis_labels=("z_index", "y", "x"), + displayed_axis_indices=displayed, + spatial_axis_labels=("z_index", "y", "x"), ) diff --git a/tests/unit/test_viewer_spatial_sample.py b/tests/unit/test_viewer_spatial_sample.py index 903e7d7b6c..e589928945 100644 --- a/tests/unit/test_viewer_spatial_sample.py +++ b/tests/unit/test_viewer_spatial_sample.py @@ -97,7 +97,9 @@ def test_raw_array_slices_keep_trailing_dimension_contract_in_same_native_projec rgb = np.stack((gray, gray + 1000, gray + 2000), axis=-1) items = server.component_groups.items_for("source") items[0] = replace( - items[0], data=rgb, image_metadata=ImagePayloadMetadata(axes=PayloadAxes.colour_samples(2)) + items[0], + data=rgb, + image_metadata=ImagePayloadMetadata(axes=PayloadAxes.colour_samples(2)), ) projection = NapariViewerPayloadProjection( server=server, diff --git a/tests/unit/test_well_throughput_scaling.py b/tests/unit/test_well_throughput_scaling.py index 644cf7b1c8..d900972949 100644 --- a/tests/unit/test_well_throughput_scaling.py +++ b/tests/unit/test_well_throughput_scaling.py @@ -605,7 +605,15 @@ def test_repeated_wells_keep_all_declared_projection_fields_coherent( ) -> None: parser = SourceSchemaFilenameParser() original = SourcePlaneProjection( - address=OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, 1), (Microscopy.Channel, 1), (Microscopy.ZIndex, 1), (Microscopy.Timepoint, 1))), + address=OpenHCSPlaneAddress( + ( + (Microscopy.Well, "A01"), + (Microscopy.Site, 1), + (Microscopy.Channel, 1), + (Microscopy.ZIndex, 1), + (Microscopy.Timepoint, 1), + ) + ), ref=SourcePixelRef("disk", "/source/blue.tif"), source_alias="Blue", source_metadata={"Well": "A01", "site": "1", "channel": "1"}, @@ -659,7 +667,15 @@ def test_repeated_wells_use_manifest_source_well_scope( serializer = SourceProjectionMetadataSerializer(parser=parser) sources = tuple( SourcePlaneProjection( - address=OpenHCSPlaneAddress(((Microscopy.Well, well), (Microscopy.Site, 1), (Microscopy.Channel, 1), (Microscopy.ZIndex, 1), (Microscopy.Timepoint, 1))), + address=OpenHCSPlaneAddress( + ( + (Microscopy.Well, well), + (Microscopy.Site, 1), + (Microscopy.Channel, 1), + (Microscopy.ZIndex, 1), + (Microscopy.Timepoint, 1), + ) + ), ref=SourcePixelRef("disk", f"/source/{well}.tif"), source_metadata={"Well": well}, ) diff --git a/tests/unit/test_zarr_input_conversion_subdirectory.py b/tests/unit/test_zarr_input_conversion_subdirectory.py index c391a6e00f..5f8e1f9ffd 100644 --- a/tests/unit/test_zarr_input_conversion_subdirectory.py +++ b/tests/unit/test_zarr_input_conversion_subdirectory.py @@ -62,7 +62,9 @@ def test_zarr_conversion_keys_store_outputs_by_plate_relative_virtual_path(tmp_p orchestrator = PipelineOrchestrator( plate, pipeline_config=PipelineConfig( - vfs_config=LazyVFSConfig(materialization_backend=MaterializationBackend.ZARR) + vfs_config=LazyVFSConfig( + materialization_backend=MaterializationBackend.ZARR + ) ), ).initialize() wells = orchestrator.get_component_keys(AxisFamily.active().partition_axis()) @@ -96,6 +98,6 @@ def test_zarr_conversion_keys_store_outputs_by_plate_relative_virtual_path(tmp_p } assert {Path(path).name for path in converted["image_files"]} == source_files assert all(path.startswith("zarr/") for path in converted["image_files"]) - assert { - record["ref"]["backend"] for record in converted["source_projection"] - } == {Backend.ZARR.value} + assert {record["ref"]["backend"] for record in converted["source_projection"]} == { + Backend.ZARR.value + } diff --git a/tests/unit/test_zmq_compiled_pipeline_run.py b/tests/unit/test_zmq_compiled_pipeline_run.py index 27dac3a465..77e0de8cc8 100644 --- a/tests/unit/test_zmq_compiled_pipeline_run.py +++ b/tests/unit/test_zmq_compiled_pipeline_run.py @@ -57,7 +57,11 @@ def wait_for_completion(self, execution_id, poll_interval=0.5): "execution": { "execution_id": execution_id, "subject_id": "/tmp/plate", - "status": self.compile_status if execution_id == "compile-1" else self.execute_status, + "status": ( + self.compile_status + if execution_id == "compile-1" + else self.execute_status + ), "start_time": 10.0, "end_time": 12.0, }, @@ -106,7 +110,9 @@ def observe(phase): "wait:execute-1", ] assert client.execution_submission.pipeline_code == result.request.pipeline_code - assert client.execution_submission.compile_control.compile_artifact_id == "compile-1" + assert ( + client.execution_submission.compile_control.compile_artifact_id == "compile-1" + ) assert observed_phases == [ (event, phase) for phase in ZMQPipelineRunPhase for event in ("start", "end") ] @@ -117,7 +123,9 @@ class ClientWithResults(FakeExecutionClient): def wait_for_completion(self, execution_id, poll_interval=0.5): response = super().wait_for_completion(execution_id, poll_interval) if execution_id == "execute-1": - response["execution"]["results_summary"] = {"output_dataset_root": "/output/plate"} + response["execution"]["results_summary"] = { + "output_dataset_root": "/output/plate" + } return response run = run_compiled_pipeline(ClientWithResults(), _submission()) @@ -131,7 +139,9 @@ class ClientWithLegacyResults(FakeExecutionClient): def wait_for_completion(self, execution_id, poll_interval=0.5): response = super().wait_for_completion(execution_id, poll_interval) if execution_id == "execute-1": - response["execution"]["results_summary"] = {"output_dataset_root": "/legacy/plate"} + response["execution"]["results_summary"] = { + "output_dataset_root": "/legacy/plate" + } return response run = run_compiled_pipeline(ClientWithLegacyResults(), _submission()) @@ -229,7 +239,9 @@ def test_compiled_pipeline_run_does_not_report_failed_execution_as_complete(): assert client.events[-1] == "wait:execute-1" -def test_compiled_run_retains_its_admitted_wire_and_standalone_request_stays_fresh(monkeypatch): +def test_compiled_run_retains_its_admitted_wire_and_standalone_request_stays_fresh( + monkeypatch, +): submission = _submission() rendered = [] original_render = PipelineDocumentCodec.render @@ -256,7 +268,9 @@ def submit_compile(self, request): assert len(rendered) == 1 assert run.request.pipeline_code == client.compile_submission.pipeline_code - assert run.request.global_config_code == client.compile_submission.global_config_code + assert ( + run.request.global_config_code == client.compile_submission.global_config_code + ) fresh = ZMQExecutionRequestBuilder.from_task(submission) assert fresh.pipeline_code != run.request.pipeline_code assert fresh.global_config_code != run.request.global_config_code diff --git a/tests/unit/test_zmq_execution_server.py b/tests/unit/test_zmq_execution_server.py index bdb9ccb019..e89e81b9cd 100644 --- a/tests/unit/test_zmq_execution_server.py +++ b/tests/unit/test_zmq_execution_server.py @@ -161,7 +161,9 @@ def test_axis_outcomes_qualify_server_terminal_after_export(monkeypatch, failed) if failed: assert "A02: error: producer lineage missing" in terminal[MessageFields.ERROR] else: - assert terminal[MessageFields.RESULTS_SUMMARY][MessageFields.PARTITION_VALUES] == [ + assert terminal[MessageFields.RESULTS_SUMMARY][ + MessageFields.PARTITION_VALUES + ] == [ "A01", "A02", ] @@ -333,7 +335,9 @@ def test_server_exports_outcomes_without_projecting_compiled_values( RuntimeContextObservation( "context", (), - outputs=StepExecutionObservation({}, (declared_output,)), + outputs=StepExecutionObservation( + {}, (declared_output,) + ), ), ) ), @@ -537,7 +541,9 @@ def test_zmq_server_admits_all_progress_through_generic_terminal_watermark() -> step_name="ExportToDatabase", phase=ProgressPhase.RUNNING, status=ProgressStatus.RUNNING, - completed=32, total=33, percent=(32 / 33) * 100.0, + completed=32, + total=33, + percent=(32 / 33) * 100.0, ) server._enqueue_progress(parent) worker_queue.put(parent) @@ -557,7 +563,11 @@ def test_zmq_server_admits_all_progress_through_generic_terminal_watermark() -> subscriber = ProgressStreamSubscriber(lambda: None, client._record_progress) for payload in published: assert subscriber._dispatch_message(json.dumps(payload)) - assert [payload[MessageFields.PROGRESS_SEQUENCE] for payload in received] == [1, 2, 3] + assert [payload[MessageFields.PROGRESS_SEQUENCE] for payload in received] == [ + 1, + 2, + 3, + ] assert client.progress_observation(record.execution_id).sequence == 3 assert record.to_dict()[MessageFields.PROGRESS_SEQUENCE] == 3 assert record.progress_event == published[-1] @@ -600,8 +610,7 @@ def test_zmq_server_records_the_compilation_output_plate_value_without_rebuildin ExecutionOutputSummary.EXECUTION_RECORD_KEY: output_dataset } assert ( - record.get_extra(ExecutionOutputSummary.EXECUTION_RECORD_KEY) - is output_dataset + record.get_extra(ExecutionOutputSummary.EXECUTION_RECORD_KEY) is output_dataset ) @@ -636,7 +645,8 @@ def fail_transport_stop(_server) -> None: def test_compiled_source_adoption_owns_fresh_runtime_services_and_live_source_gate( - tmp_path, monkeypatch, + tmp_path, + monkeypatch, ): import json from polystore.base import reset_memory_backend @@ -706,7 +716,9 @@ def test_compiled_source_adoption_owns_fresh_runtime_services_and_live_source_ga def reject_configuration_work(*args, **kwargs): raise AssertionError("Compiled adoption must use the admitted domain") - monkeypatch.setattr(PipelineOrchestrator, "get_effective_config", reject_configuration_work) + monkeypatch.setattr( + PipelineOrchestrator, "get_effective_config", reject_configuration_work + ) monkeypatch.setattr( "openhcs.core.orchestrator.orchestrator.ZarrStorageBackend", reject_configuration_work, @@ -770,7 +782,9 @@ def reject_source_discovery(*args, **kwargs): ) == ["A02"] -def test_compiled_request_keeps_config_scope_without_resolving_configuration(monkeypatch): +def test_compiled_request_keeps_config_scope_without_resolving_configuration( + monkeypatch, +): server = ZMQExecutionServer(port=5555) global_config = GlobalPipelineConfig( processing_config=ProcessingConfig(group_by=Microscopy.Channel) @@ -800,35 +814,45 @@ def test_compiled_request_keeps_config_scope_without_resolving_configuration(mon monkeypatch.setattr( zmq_execution_server_module, "ZMQOrchestratorEnvironmentRequest", - lambda **kwargs: SimpleNamespace(prepare=lambda: SimpleNamespace( - debug_execution_policy=policy, - debug_execution_config=None, - dataset_root=context.subject_id, - )), + lambda **kwargs: SimpleNamespace( + prepare=lambda: SimpleNamespace( + debug_execution_policy=policy, + debug_execution_config=None, + dataset_root=context.subject_id, + ) + ), ) def reject_resolution(*args, **kwargs): raise AssertionError("Compiled request must not resolve the saved config again") monkeypatch.setattr( - zmq_execution_server_module.ObjectState, "resolve_saved_object", reject_resolution + zmq_execution_server_module.ObjectState, + "resolve_saved_object", + reject_resolution, ) initialized = [] monkeypatch.setattr( - server, "_initialize_orchestrator", + server, + "_initialize_orchestrator", lambda *args, **kwargs: initialized.append(kwargs) or object(), ) compilations = [] monkeypatch.setattr( - server, "_resolve_compilation", + server, + "_resolve_compilation", lambda **kwargs: compilations.append(kwargs) or object(), ) monkeypatch.setattr(server, "_record_compilation_outputs", lambda *args: None) marker = object() - monkeypatch.setattr(server, "_finish_compilation_or_execute", lambda **kwargs: marker) + monkeypatch.setattr( + server, "_finish_compilation_or_execute", lambda **kwargs: marker + ) assert server._execute_with_orchestrator(context) is marker assert initialized[0]["execution_bundle"] is bundle assert initialized[0]["resolved_config"] is None assert compilations[0]["resolved_config"] is None assert compilations[0]["partition_values"] == ["A01"] - assert get_current_global_config(GlobalPipelineConfig, use_live=False) is global_config + assert ( + get_current_global_config(GlobalPipelineConfig, use_live=False) is global_config + ) diff --git a/tests/unit/test_zmq_progress.py b/tests/unit/test_zmq_progress.py index 8e3a21d3f9..9a356541bd 100644 --- a/tests/unit/test_zmq_progress.py +++ b/tests/unit/test_zmq_progress.py @@ -65,17 +65,20 @@ def test_compile_heartbeat_emits_running_pipeline_compile_event(): _emitter(emitted).compile_heartbeat(step_count=18) assert len(emitted) == 1 - assert emitted[0].identity == ProgressExecutionContext( - "exec-1", "/tmp/plate" - ).event( - axis_id="", - step_name="pipeline", - phase=ProgressPhase.COMPILE, - status=ProgressStatus.RUNNING, - percent=0.0, - completed=0, - total=18, - ).identity + assert ( + emitted[0].identity + == ProgressExecutionContext("exec-1", "/tmp/plate") + .event( + axis_id="", + step_name="pipeline", + phase=ProgressPhase.COMPILE, + status=ProgressStatus.RUNNING, + percent=0.0, + completed=0, + total=18, + ) + .identity + ) assert emitted[0].phase is ProgressPhase.COMPILE assert emitted[0].status is ProgressStatus.RUNNING assert emitted[0].total == 18 diff --git a/tests/unit/test_zmq_submission_timeout.py b/tests/unit/test_zmq_submission_timeout.py index 5f2cdaa16f..aabd9ac9c6 100644 --- a/tests/unit/test_zmq_submission_timeout.py +++ b/tests/unit/test_zmq_submission_timeout.py @@ -51,7 +51,9 @@ def ensure_progress_subscription(self, *, timeout_ms: int): observed.append(("progress", timeout_ms)) def serialize_task(self, _task, _config=None): - return ExecuteRequest(subject_id="subject", pipeline_code="pass", config_code="") + return ExecuteRequest( + subject_id="subject", pipeline_code="pass", config_code="" + ) def send_control_request(self, request, *, timeout_ms: int): observed.append((request[MessageFields.TYPE], timeout_ms)) @@ -67,7 +69,9 @@ def send_control_request(self, request, *, timeout_ms: int): client, ) - client._submit_submission(lambda: client.serialize_task(_Submission().to_task()), timeout_ms=15000) + client._submit_submission( + lambda: client.serialize_task(_Submission().to_task()), timeout_ms=15000 + ) assert [phase for phase, _timeout_ms in observed] == [ "progress", @@ -117,7 +121,9 @@ def connect( client.connect = MethodType(connect, client) try: - client._submit_submission(lambda: client.serialize_task(_Submission().to_task()), timeout_ms=15000) + client._submit_submission( + lambda: client.serialize_task(_Submission().to_task()), timeout_ms=15000 + ) except RuntimeError as error: assert str(error) == "Failed to connect to execution server" else: @@ -164,7 +170,9 @@ def send_control_request(self, request, *, timeout_ms: int): ExecutionSubmissionPreparationTimeoutError, match="no execute request was sent", ): - client._submit_submission(lambda: client.serialize_task(_Submission().to_task()), timeout_ms=10) + client._submit_submission( + lambda: client.serialize_task(_Submission().to_task()), timeout_ms=10 + ) assert observed == ["connect"] diff --git a/validation/public_two_vector_388.py b/validation/public_two_vector_388.py index f0271c40ab..25c4d63c4b 100644 --- a/validation/public_two_vector_388.py +++ b/validation/public_two_vector_388.py @@ -20,7 +20,9 @@ import time ROOT = Path("/home/ts/wt/openhcs-issue-batch-20260929") -FIXTURE = ROOT / "calibration371-classification372-installed-20261001/engineering_fixture.py" +FIXTURE = ( + ROOT / "calibration371-classification372-installed-20261001/engineering_fixture.py" +) FIXTURE_SHA = "3157e246de9348b0522f8e4d2712b84df06db71185d8c09ce868b6f74c7349ce" CLASSIFICATION_SHA = "3ca76fb8a479c2701e367bce1db1935771b818bc4be9662a958ab4e4f7912b74" PLATE = ROOT / "classification382-installed-20261001/synthetic-plate" @@ -36,20 +38,40 @@ def require(condition, message): def expected_labels(): - return [[1 if 8 <= y < 16 and 8 <= x < 16 else - 2 if 32 <= y < 48 and 32 <= x < 48 else 0 - for x in range(64)] for y in range(64)] + return [ + [ + ( + 1 + if 8 <= y < 16 and 8 <= x < 16 + else 2 if 32 <= y < 48 and 32 <= x < 48 else 0 + ) + for x in range(64) + ] + for y in range(64) + ] def assert_label_identity(values, shape): require(tuple(shape) == (64, 64), f"Labels are not a 64x64 plane: {shape!r}") - require(len(values) == 64 and all(len(row) == 64 for row in values), - "Incomplete pixels (or RGB/stack substituted for labels)") - require(all(type(value) is int for row in values for value in row), - "Label payload must retain integral object IDs") - require([list(row) for row in values] == expected_labels(), "Full 4096-value object-label identity differs") - return {"shape": [64, 64], "pixels_checked": 4096, - "label_0_pixels": 3776, "label_1_pixels": 64, "label_2_pixels": 256} + require( + len(values) == 64 and all(len(row) == 64 for row in values), + "Incomplete pixels (or RGB/stack substituted for labels)", + ) + require( + all(type(value) is int for row in values for value in row), + "Label payload must retain integral object IDs", + ) + require( + [list(row) for row in values] == expected_labels(), + "Full 4096-value object-label identity differs", + ) + return { + "shape": [64, 64], + "pixels_checked": 4096, + "label_0_pixels": 3776, + "label_1_pixels": 64, + "label_2_pixels": 256, + } def assert_measurement_rows(objects, classes): @@ -64,39 +86,69 @@ def assert_measurement_rows(objects, classes): require(len(classes) == 3, "Expected one aggregate + two per-object class rows") aggregate = [row for row in classes if row["object_label"] == ""] require(len(aggregate) == 1, "Missing/duplicate aggregate row") - per_object = {int(row["object_label"]): row for row in classes - if row["object_label"] != ""} + per_object = { + int(row["object_label"]): row for row in classes if row["object_label"] != "" + } require(set(per_object) == {1, 2}, "Classification subject differs/duplicate") require(int(aggregate[0]["slice_index"]) == 0, "Aggregate slice changed") for name in QUADRANTS: expected_count = int(name in ("low_low", "high_low")) - require(int(aggregate[0][f"Classify_{name}_NumObjectsPerBin"]) == expected_count, - f"Wrong quadrant count: {name}") - require(float(aggregate[0][f"Classify_{name}_PctObjectsPerBin"]) == 50 * expected_count, - f"Wrong quadrant percentage: {name}") + require( + int(aggregate[0][f"Classify_{name}_NumObjectsPerBin"]) == expected_count, + f"Wrong quadrant count: {name}", + ) + require( + float(aggregate[0][f"Classify_{name}_PctObjectsPerBin"]) + == 50 * expected_count, + f"Wrong quadrant percentage: {name}", + ) for label, expected_name in ((1, "low_low"), (2, "high_low")): row = per_object[label] - require(row["object_name"] == "engineering_cells", "Classification object subject changed") + require( + row["object_name"] == "engineering_cells", + "Classification object subject changed", + ) require(int(row["slice_index"]) == 0, "Classification slice changed") - require(int(row[f"Classify_{name}"]) == int(name == expected_name), - f"Wrong per-object quadrant: label={label}, quadrant={name}") - return {"label_1": "low_low", "label_2": "high_low", - "counts": dict(zip(QUADRANTS, (1, 1, 0, 0))), - "pixel_count": [64, 256], "calibration_um": [1.3556, 1.3556], - "custom_splits": [100.0, 2.0]} + require( + int(row[f"Classify_{name}"]) == int(name == expected_name), + f"Wrong per-object quadrant: label={label}, quadrant={name}", + ) + return { + "label_1": "low_low", + "label_2": "high_low", + "counts": dict(zip(QUADRANTS, (1, 1, 0, 0))), + "pixel_count": [64, 256], + "calibration_um": [1.3556, 1.3556], + "custom_splits": [100.0, 2.0], + } def pipeline_patch(suffix): return { - "config_type": "pipeline", "values": { - "num_workers": 1, "use_threading": True, - "microscope": "source_bindings", "materialize_runtime_artifacts": True, + "config_type": "pipeline", + "values": { + "num_workers": 1, + "use_threading": True, + "microscope": "source_bindings", + "materialize_runtime_artifacts": True, "well_filter_config": {"well_filter": ["A01"]}, "source_bindings_config": { - "bindings": [{"alias": "engineering_raw", "selector": { - "components": [{"component": "channel", "value": "1"}], - "filters": [{"subject": "file", "match_type": "contains", "value": "_w1_"}], - }}], "source_voxel_spacing": {"values_zyx": [1.3556, 1.3556]}, + "bindings": [ + { + "alias": "engineering_raw", + "selector": { + "components": [{"component": "channel", "value": "1"}], + "filters": [ + { + "subject": "file", + "match_type": "contains", + "value": "_w1_", + } + ], + }, + } + ], + "source_voxel_spacing": {"values_zyx": [1.3556, 1.3556]}, }, "path_planning_config": {"output_dir_suffix": suffix}, }, @@ -107,9 +159,13 @@ def step_overrides(viewer_port, *, select_source=False): values = { "processing_config": {"variable_components": ["channel"], "group_by": "NONE"}, "step_materialization_config": {"enabled": True}, - "napari_streaming_config": {"enabled": True, "persistent": True, - "host": "127.0.0.1", "port": viewer_port, - "transport_mode": "tcp"}, + "napari_streaming_config": { + "enabled": True, + "persistent": True, + "host": "127.0.0.1", + "port": viewer_port, + "transport_mode": "tcp", + }, } if select_source: values["source_bindings"] = {"enabled": True} @@ -118,12 +174,19 @@ def step_overrides(viewer_port, *, select_source=False): def paired_kwargs(): - return {"select_the_object_to_be_classified": "engineering_cells", - "measurement1_feature": "pixel_count", "measurement2_feature": "calibration_um", - "threshold1_method": "custom", "threshold1_value": 100.0, - "threshold2_method": "custom", "threshold2_value": 2.0, - "low_low_name": "low_low", "high_low_name": "high_low", - "low_high_name": "low_high", "high_high_name": "high_high"} + return { + "select_the_object_to_be_classified": "engineering_cells", + "measurement1_feature": "pixel_count", + "measurement2_feature": "calibration_um", + "threshold1_method": "custom", + "threshold1_value": 100.0, + "threshold2_method": "custom", + "threshold2_value": 2.0, + "low_low_name": "low_low", + "high_low_name": "high_low", + "low_high_name": "low_high", + "high_high_name": "high_high", + } class PublicJourney: @@ -136,6 +199,7 @@ def __init__(self, client, receipt_dir): def save(self, filename, value): from python_introspect import to_jsonable + with (self.root / filename).open("x") as stream: stream.write(json.dumps(to_jsonable(value), indent=2) + "\n") @@ -143,21 +207,35 @@ def call(self, name, arguments): from openhcs.agent.capabilities import get_agent_capability from openhcs.mcp.dev_client_core import McpDevToolBatchResponse from python_introspect import to_jsonable + capability = get_agent_capability(name) self.sequence += 1 stem = f"{self.sequence:03d}-{name}" self.save(stem + "-request.json", arguments) - result = self.client.execute([ - "--timeout-seconds", "10", "call", capability.name, - "--arguments", json.dumps(to_jsonable(arguments)), "--json", - ]) + result = self.client.execute( + [ + "--timeout-seconds", + "10", + "call", + capability.name, + "--arguments", + json.dumps(to_jsonable(arguments)), + "--json", + ] + ) self.save(stem + "-response.json", result.payload) - self.save(stem + "-transport.json", { - "returncode": result.returncode, "server_stderr_tail": result.server_stderr_tail, - }) + self.save( + stem + "-transport.json", + { + "returncode": result.returncode, + "server_stderr_tail": result.server_stderr_tail, + }, + ) response = McpDevToolBatchResponse.for_rendering(result.payload) - require(result.returncode == 0 and not response.has_errors(), - f"{name} failed/uncertain; retained response; NEVER replay") + require( + result.returncode == 0 and not response.has_errors(), + f"{name} failed/uncertain; retained response; NEVER replay", + ) payload = response.payload_for(capability) require(payload is not None, f"Missing declared output for {name}") return payload @@ -167,22 +245,39 @@ def execute_source(self, source_request, connection): from openhcs.agent.capabilities import agent_capabilities from openhcs.agent.dto.session import DatasetListState from openhcs.mcp.dev_client_core import McpDevToolBatchResponse - result = self.client.execute([ - "--timeout-seconds", "1800", "execute-source", source_request["plate_path"], - "--source-text", source_request["pipeline_source"], - "--host", connection["host"], "--port", str(connection["port"]), - "--transport-mode", connection["transport_mode"], "--wait", "--json", - ]) + + result = self.client.execute( + [ + "--timeout-seconds", + "1800", + "execute-source", + source_request["plate_path"], + "--source-text", + source_request["pipeline_source"], + "--host", + connection["host"], + "--port", + str(connection["port"]), + "--transport-mode", + connection["transport_mode"], + "--wait", + "--json", + ] + ) self.sequence += 1 self.save(f"{self.sequence:03d}-execute-source-response.json", result.payload) response = McpDevToolBatchResponse.for_rendering(result.payload) - require(result.returncode == 0 and not response.has_errors(), - "execute-source failed/uncertain; retained response; NEVER replay") + require( + result.returncode == 0 and not response.has_errors(), + "execute-source failed/uncertain; retained response; NEVER replay", + ) state = response.payload_for(agent_capabilities.session_datasets) require(isinstance(state, DatasetListState), "Missing final dataset list") return state - def observe_until(self, name, arguments, ready, terminal, *, seconds=60, on_observation=None): + def observe_until( + self, name, arguments, ready, terminal, *, seconds=60, on_observation=None + ): deadline = None if seconds is None else time.monotonic() + seconds while True: value = self.call(name, arguments) @@ -192,7 +287,10 @@ def observe_until(self, name, arguments, ready, terminal, *, seconds=60, on_obse return value require(not terminal(value), f"{name}: original attempt became terminal") if deadline is not None: - require(time.monotonic() < deadline, f"{name}: pending handle retained; no replay") + require( + time.monotonic() < deadline, + f"{name}: pending handle retained; no replay", + ) time.sleep(0.5) def readiness_observer(self, bootstrap_handle, phase, validate_handle): @@ -204,6 +302,7 @@ def readiness_observer(self, bootstrap_handle, phase, validate_handle): """ import psutil from python_introspect import to_jsonable + started = time.monotonic() last_progress = None last_report = float("-inf") @@ -212,7 +311,10 @@ def observed(value): nonlocal last_progress, last_report validate_handle(value) identity = bootstrap_handle.process_identity - require(identity.is_alive() is True, "Original runtime is terminal/unknown; stop and retain handle") + require( + identity.is_alive() is True, + "Original runtime is terminal/unknown; stop and retain handle", + ) progress = to_jsonable(value.progress) elapsed = time.monotonic() - started if progress != last_progress or elapsed - last_report >= 5: @@ -220,6 +322,7 @@ def observed(value): native_processes = (native, *native.children(recursive=True)) driver = psutil.Process() driver_processes = (driver, *driver.children(recursive=True)) + # Ephemeral OS observation, no mirrored native authority/cache. def rss(processes): total = 0 @@ -229,274 +332,608 @@ def rss(processes): except psutil.NoSuchProcess: continue return round(total / 1024**2, 2) - require(identity.is_alive() is True, "Runtime incarnation changed during resource observation") - print(json.dumps({"phase": phase, - "elapsed_seconds": round(elapsed, 2), "progress": progress, - "original_process_identity": to_jsonable(identity), - "native_tree_rss_mib": rss(native_processes), - "driver_tree_rss_mib": rss(driver_processes), - "resource_scope": "trees may overlap; parent owns combined live budget", - "ordinary_mcp_timeout_seconds": 10}), flush=True) + + require( + identity.is_alive() is True, + "Runtime incarnation changed during resource observation", + ) + print( + json.dumps( + { + "phase": phase, + "elapsed_seconds": round(elapsed, 2), + "progress": progress, + "original_process_identity": to_jsonable(identity), + "native_tree_rss_mib": rss(native_processes), + "driver_tree_rss_mib": rss(driver_processes), + "resource_scope": "trees may overlap; parent owns combined live budget", + "ordinary_mcp_timeout_seconds": 10, + } + ), + flush=True, + ) last_progress, last_report = progress, elapsed + return observed def csv_preview(query, suffix): require(query.truncated_count == 0, "File inventory truncated") - matches = [record for record in query.records - if record.relative_path is not None and record.relative_path.endswith(suffix)] + matches = [ + record + for record in query.records + if record.relative_path is not None and record.relative_path.endswith(suffix) + ] require(len(matches) == 1, f"Expected exactly one persisted {suffix}: {matches!r}") preview = matches[0].preview - require(preview is not None and not preview.truncated and not preview.omitted_reason, - "CSV preview incomplete; no count-only acceptance") + require( + preview is not None and not preview.truncated and not preview.omitted_reason, + "CSV preview incomplete; no count-only acceptance", + ) return matches[0].full_path, preview.csv_rows def run(args): - from openhcs.agent.dto.execution import RuntimeBootstrapCloseRequest, RuntimeBootstrapHandle + from openhcs.agent.dto.execution import ( + RuntimeBootstrapCloseRequest, + RuntimeBootstrapHandle, + ) from openhcs.mcp.dev_client import McpDevClient from openhcs.processing.custom_functions.manager import CustomFunctionManager from python_introspect import to_jsonable from python_introspect import dataclass_from_mapping - require(args.parent_serialized_slot_authorized, "Parent must authorize the released serialized slot") - require(args.receipt_dir is not None and args.receipt_dir.is_absolute(), "Explicit persistent receipt-dir required") - require(args.receipt_dir.resolve().is_relative_to(Path("/home/ts/wt")), "Receipts must be under persistent /home/ts/wt") + require( + args.parent_serialized_slot_authorized, + "Parent must authorize the released serialized slot", + ) + require( + args.receipt_dir is not None and args.receipt_dir.is_absolute(), + "Explicit persistent receipt-dir required", + ) + require( + args.receipt_dir.resolve().is_relative_to(Path("/home/ts/wt")), + "Receipts must be under persistent /home/ts/wt", + ) require(not args.receipt_dir.exists(), "Refuse to overwrite original receipts") - require(args.output_suffix.startswith("_classification388_") and - all(char.isalnum() or char == "_" for char in args.output_suffix), "Invalid unique output suffix") + require( + args.output_suffix.startswith("_classification388_") + and all(char.isalnum() or char == "_" for char in args.output_suffix), + "Invalid unique output suffix", + ) output_plate = PLATE.with_name(PLATE.name + args.output_suffix) - require(not output_plate.exists(), "Refuse to reuse/overwrite an earlier output plate") - require(args.runtime_port != args.viewer_port and - 1 <= args.runtime_port <= 64535 and 1 <= args.viewer_port <= 64535, - "Invalid/disjoint runtime and viewer endpoint ports") - require(hashlib.sha256(FIXTURE.read_bytes()).hexdigest() == FIXTURE_SHA, "Original fixture changed") + require( + not output_plate.exists(), "Refuse to reuse/overwrite an earlier output plate" + ) + require( + args.runtime_port != args.viewer_port + and 1 <= args.runtime_port <= 64535 + and 1 <= args.viewer_port <= 64535, + "Invalid/disjoint runtime and viewer endpoint ports", + ) + require( + hashlib.sha256(FIXTURE.read_bytes()).hexdigest() == FIXTURE_SHA, + "Original fixture changed", + ) require(PLATE.is_dir(), "Original reviewed synthetic plate is unavailable") - classification_spec = importlib.util.find_spec("openhcs.processing.backends.cellprofiler.classification") - require(classification_spec is not None and classification_spec.origin is not None and - hashlib.sha256(Path(classification_spec.origin).read_bytes()).hexdigest() == CLASSIFICATION_SHA, - "Installed classification bytes are not reviewed PR388") - require(os.environ.get("XDG_DATA_HOME") and os.environ.get("OPENHCS_UI_CONFIG_CACHE_FILE"), - "Parent must supply the exact controlled launch data/cache selectors") + classification_spec = importlib.util.find_spec( + "openhcs.processing.backends.cellprofiler.classification" + ) + require( + classification_spec is not None + and classification_spec.origin is not None + and hashlib.sha256(Path(classification_spec.origin).read_bytes()).hexdigest() + == CLASSIFICATION_SHA, + "Installed classification bytes are not reviewed PR388", + ) + require( + os.environ.get("XDG_DATA_HOME") + and os.environ.get("OPENHCS_UI_CONFIG_CACHE_FILE"), + "Parent must supply the exact controlled launch data/cache selectors", + ) intended_store = CustomFunctionManager.default_storage_directory() original_handle = None if args.runtime_handle is not None: - original_handle = dataclass_from_mapping(RuntimeBootstrapHandle, json.loads(args.runtime_handle.read_text())) - require(original_handle.connection.port == args.runtime_port and - original_handle.connection.host == "127.0.0.1" and - original_handle.launch_plan.storage_dir == intended_store, - "Same-context handle disagrees with parent endpoint/store") + original_handle = dataclass_from_mapping( + RuntimeBootstrapHandle, json.loads(args.runtime_handle.read_text()) + ) + require( + original_handle.connection.port == args.runtime_port + and original_handle.connection.host == "127.0.0.1" + and original_handle.launch_plan.storage_dir == intended_store, + "Same-context handle disagrees with parent endpoint/store", + ) else: # Read-only endpoint check; NEVER take over Dalton's or another listener. - for port in (args.runtime_port, args.runtime_port + 1000, args.viewer_port, args.viewer_port + 1000): + for port in ( + args.runtime_port, + args.runtime_port + 1000, + args.viewer_port, + args.viewer_port + 1000, + ): with socket.socket() as probe: probe.settimeout(0.2) - require(probe.connect_ex(("127.0.0.1", port)) != 0, f"Endpoint {port} occupied: stop") - persisted_fixture = CustomFunctionManager.source_path_for_name(intended_store, FIXTURE_NAME) + require( + probe.connect_ex(("127.0.0.1", port)) != 0, + f"Endpoint {port} occupied: stop", + ) + persisted_fixture = CustomFunctionManager.source_path_for_name( + intended_store, FIXTURE_NAME + ) if args.fixture_function_id is None: - require(not persisted_fixture.exists(), "Fixture already persisted: reconcile/reuse explicitly; NEVER register again") + require( + not persisted_fixture.exists(), + "Fixture already persisted: reconcile/reuse explicitly; NEVER register again", + ) else: - require(persisted_fixture.is_file() and hashlib.sha256(persisted_fixture.read_bytes()).hexdigest() == FIXTURE_SHA, - "Explicit reused fixture is not original reviewed source") + require( + persisted_fixture.is_file() + and hashlib.sha256(persisted_fixture.read_bytes()).hexdigest() + == FIXTURE_SHA, + "Explicit reused fixture is not original reviewed source", + ) args.receipt_dir.mkdir() with (args.receipt_dir / "mcp-server.stderr").open("x") as stderr: - with McpDevClient(python_executable=sys.executable, use_resident_server=False, - server_stderr=stderr) as client: + with McpDevClient( + python_executable=sys.executable, + use_resident_server=False, + server_stderr=stderr, + ) as client: journey = PublicJourney(client, args.receipt_dir) - journey.save("engineering-input-selection.json", { - "fixture": str(FIXTURE), "sha256": FIXTURE_SHA, "plate": str(PLATE), - "output_plate": str(output_plate), "paired_kwargs": paired_kwargs(), - "scope": "reviewed engineering fixture only; not biology or frozen-blind replay", - "installed_python": sys.executable, "classification_source": classification_spec.origin, - "classification_sha256": CLASSIFICATION_SHA, - "runtime_ownership": "parent retained same-context handle" if original_handle else "driver bootstrapped exact handle", - }) + journey.save( + "engineering-input-selection.json", + { + "fixture": str(FIXTURE), + "sha256": FIXTURE_SHA, + "plate": str(PLATE), + "output_plate": str(output_plate), + "paired_kwargs": paired_kwargs(), + "scope": "reviewed engineering fixture only; not biology or frozen-blind replay", + "installed_python": sys.executable, + "classification_source": classification_spec.origin, + "classification_sha256": CLASSIFICATION_SHA, + "runtime_ownership": ( + "parent retained same-context handle" + if original_handle + else "driver bootstrapped exact handle" + ), + }, + ) try: journey.call("openhcs_health_check", {}) journey.call("openhcs_get_authoring_context", {"kind": "first_use"}) - journey.call("openhcs_get_authoring_context", {"kind": "custom_function"}) + journey.call( + "openhcs_get_authoring_context", {"kind": "custom_function"} + ) journey.call("openhcs_search_capabilities", {"text": "owned runtime"}) - connection = {"host": "127.0.0.1", "port": args.runtime_port, - "transport_mode": "tcp", "persistent": True} + connection = { + "host": "127.0.0.1", + "port": args.runtime_port, + "transport_mode": "tcp", + "persistent": True, + } if original_handle is None: - startup = journey.call("openhcs_start_owned_runtime", {**connection, "timeout_ms": 500}) + startup = journey.call( + "openhcs_start_owned_runtime", {**connection, "timeout_ms": 500} + ) original_handle = startup.handle journey.save("original-runtime-handle.json", original_handle) handle = to_jsonable(original_handle) - observed = journey.observe_until("openhcs_observe_owned_runtime", {"handle": handle}, - lambda value: value.ready, lambda value: value.process_alive is False, - seconds=None, on_observation=journey.readiness_observer( - original_handle, "startup", lambda value: require( - value.handle == original_handle, "Startup observation changed exact handle"))) - require(observed.handle == original_handle, "Runtime incarnation changed") - journey.call("openhcs_search_capabilities", {"text": "catalog preparation"}) - preparation = journey.call("openhcs_start_function_catalog_preparation", connection) + observed = journey.observe_until( + "openhcs_observe_owned_runtime", + {"handle": handle}, + lambda value: value.ready, + lambda value: value.process_alive is False, + seconds=None, + on_observation=journey.readiness_observer( + original_handle, + "startup", + lambda value: require( + value.handle == original_handle, + "Startup observation changed exact handle", + ), + ), + ) + require( + observed.handle == original_handle, "Runtime incarnation changed" + ) + journey.call( + "openhcs_search_capabilities", {"text": "catalog preparation"} + ) + preparation = journey.call( + "openhcs_start_function_catalog_preparation", connection + ) journey.save("original-catalog-handle.json", preparation.handle) - prepared = journey.observe_until("openhcs_get_function_catalog_preparation_status", + prepared = journey.observe_until( + "openhcs_get_function_catalog_preparation_status", to_jsonable(preparation.handle), - lambda value: value.outcome.ready, lambda value: value.outcome.terminal, - seconds=None, on_observation=journey.readiness_observer( - original_handle, "catalog", lambda value: value.require_handle(preparation.handle))) + lambda value: value.outcome.ready, + lambda value: value.outcome.terminal, + seconds=None, + on_observation=journey.readiness_observer( + original_handle, + "catalog", + lambda value: value.require_handle(preparation.handle), + ), + ) prepared.require_handle(preparation.handle) - require(prepared.handle.server_identity == original_handle.process_identity, - "Catalog warmed a different native owner") + require( + prepared.handle.server_identity == original_handle.process_identity, + "Catalog warmed a different native owner", + ) fixture_function_id = args.fixture_function_id if fixture_function_id is None: - registered = journey.call("openhcs_register_custom_function", { - **connection, "source_code": FIXTURE.read_text(), "persist": True, - "function_name": FIXTURE_NAME, "storage_dir": original_handle.launch_plan.storage_dir, - }) - require(registered.server_identity == original_handle.process_identity, - "Fixture registered on a different native owner") - functions = [item for item in registered.functions if item.name == FIXTURE_NAME] - require(registered.registered_count == 1 and len(functions) == 1, "Fixture registration differs") + registered = journey.call( + "openhcs_register_custom_function", + { + **connection, + "source_code": FIXTURE.read_text(), + "persist": True, + "function_name": FIXTURE_NAME, + "storage_dir": original_handle.launch_plan.storage_dir, + }, + ) + require( + registered.server_identity == original_handle.process_identity, + "Fixture registered on a different native owner", + ) + functions = [ + item + for item in registered.functions + if item.name == FIXTURE_NAME + ] + require( + registered.registered_count == 1 and len(functions) == 1, + "Fixture registration differs", + ) fixture_function_id = functions[0].function_id - fixture_detail = journey.call("openhcs_describe_function", {"function_id": fixture_function_id}) - require(fixture_detail.entry.name == FIXTURE_NAME, "Reused registry identity names a different fixture") + fixture_detail = journey.call( + "openhcs_describe_function", {"function_id": fixture_function_id} + ) + require( + fixture_detail.entry.name == FIXTURE_NAME, + "Reused registry identity names a different fixture", + ) journey.call("openhcs_describe_function", {"function_id": PAIR_ID}) - journey.call("openhcs_describe_config_schema", {"config_type": "pipeline"}) + journey.call( + "openhcs_describe_config_schema", {"config_type": "pipeline"} + ) journey.call("openhcs_describe_config_schema", {"config_type": "step"}) patch = pipeline_patch(args.output_suffix) validated_config = journey.call("openhcs_validate_config_patch", patch) - require(validated_config.valid and validated_config.config_ref is not None, "Invalid complete pipeline config") - pipeline = journey.call("openhcs_create_pipeline", {"pipeline_config_id": validated_config.config_ref.config_id}) - journey.call("openhcs_add_function_step", { - "pipeline_id": pipeline.pipeline_id, "function_id": fixture_function_id, - "name": "engineering_fixture", "kwargs": {}, - "step_config_overrides": step_overrides(args.viewer_port, select_source=True), - }) - authored = journey.call("openhcs_add_function_step", { - "pipeline_id": pipeline.pipeline_id, "function_id": PAIR_ID, - "name": PAIR_STEP, "kwargs": paired_kwargs(), - "step_config_overrides": step_overrides(args.viewer_port), - }) - require(len(authored.steps) == 2 and authored.steps[1].functions[0].kwargs == paired_kwargs(), - "Public authoring lost selectors/settings") - validated = journey.call("openhcs_validate_pipeline", {"pipeline_id": pipeline.pipeline_id}) + require( + validated_config.valid and validated_config.config_ref is not None, + "Invalid complete pipeline config", + ) + pipeline = journey.call( + "openhcs_create_pipeline", + {"pipeline_config_id": validated_config.config_ref.config_id}, + ) + journey.call( + "openhcs_add_function_step", + { + "pipeline_id": pipeline.pipeline_id, + "function_id": fixture_function_id, + "name": "engineering_fixture", + "kwargs": {}, + "step_config_overrides": step_overrides( + args.viewer_port, select_source=True + ), + }, + ) + authored = journey.call( + "openhcs_add_function_step", + { + "pipeline_id": pipeline.pipeline_id, + "function_id": PAIR_ID, + "name": PAIR_STEP, + "kwargs": paired_kwargs(), + "step_config_overrides": step_overrides(args.viewer_port), + }, + ) + require( + len(authored.steps) == 2 + and authored.steps[1].functions[0].kwargs == paired_kwargs(), + "Public authoring lost selectors/settings", + ) + validated = journey.call( + "openhcs_validate_pipeline", {"pipeline_id": pipeline.pipeline_id} + ) require(validated.valid, "Public complete-document validation failed") - clean = journey.call("openhcs_render_pipeline_source", {"pipeline_id": pipeline.pipeline_id, "clean": True}) - resolved = journey.call("openhcs_render_pipeline_source", {"pipeline_id": pipeline.pipeline_id, "clean": False}) - for name, rendered in (("public-clean.py", clean), ("public-resolved.py", resolved)): + clean = journey.call( + "openhcs_render_pipeline_source", + {"pipeline_id": pipeline.pipeline_id, "clean": True}, + ) + resolved = journey.call( + "openhcs_render_pipeline_source", + {"pipeline_id": pipeline.pipeline_id, "clean": False}, + ) + for name, rendered in ( + ("public-clean.py", clean), + ("public-resolved.py", resolved), + ): with (args.receipt_dir / name).open("x") as source: source.write(rendered.source) - require("pipeline_config" in rendered.source and "pipeline_steps" in rendered.source, - "Renderer returned a steps-only fragment") + require( + "pipeline_config" in rendered.source + and "pipeline_steps" in rendered.source, + "Renderer returned a steps-only fragment", + ) # Native plan/source reconstruction receives EXACT public clean source, not edited source. - source_request = {"plate_path": str(PLATE), "pipeline_source": clean.source} - plan = journey.call("openhcs_inspect_pipeline_source_artifact_plan", source_request) - require(plan.axis_count == 1 and plan.step_count == 2 and len(plan.steps) == 2, - "Unexpected/truncated axes or steps") - require(not plan.truncated_axis_count and not plan.truncated_step_count, - "Plan is truncated") + source_request = { + "plate_path": str(PLATE), + "pipeline_source": clean.source, + } + plan = journey.call( + "openhcs_inspect_pipeline_source_artifact_plan", source_request + ) + require( + plan.axis_count == 1 + and plan.step_count == 2 + and len(plan.steps) == 2, + "Unexpected/truncated axes or steps", + ) + require( + not plan.truncated_axis_count and not plan.truncated_step_count, + "Plan is truncated", + ) final = next(step for step in plan.steps if step.step_name == PAIR_STEP) - require(final.axis_id == "A01" and final.step_index == 1 and - not final.truncated_artifact_input_count and not final.truncated_artifact_output_count, - "Paired compiled axis/order/artifact coverage changed") - require({"engineering_cells", "engineering_object_rows"} <= - {artifact.name for artifact in final.artifact_inputs}, - "Compiled pair lost original object subject/prior measurement producer") - require(final.main_flow_axis_persistence_enabled and - final.main_flow_materialization.plate_root == str(output_plate), - "Final unchanged-label persistence disabled/wrong destination") - require(len(final.artifact_outputs) == 1 and - final.artifact_outputs[0].name == PAIR_STEP + "_1_measurements" and - final.artifact_outputs[0].materialization is not None and - final.artifact_outputs[0].materialization.persistent_enabled, - "Pair must declare ONLY classification rows (no retained RGB output)") - journey.save("exact-output-slots.json", { - "callable_return_0": "unchanged engineering_cells dense object labels, int32[64,64]", - "callable_return_1": final.artifact_outputs[0], - "fixture_rows": "engineering_object_rows", "final_main_flow": final.main_flow_materialization, - }) + require( + final.axis_id == "A01" + and final.step_index == 1 + and not final.truncated_artifact_input_count + and not final.truncated_artifact_output_count, + "Paired compiled axis/order/artifact coverage changed", + ) + require( + {"engineering_cells", "engineering_object_rows"} + <= {artifact.name for artifact in final.artifact_inputs}, + "Compiled pair lost original object subject/prior measurement producer", + ) + require( + final.main_flow_axis_persistence_enabled + and final.main_flow_materialization.plate_root == str(output_plate), + "Final unchanged-label persistence disabled/wrong destination", + ) + require( + len(final.artifact_outputs) == 1 + and final.artifact_outputs[0].name == PAIR_STEP + "_1_measurements" + and final.artifact_outputs[0].materialization is not None + and final.artifact_outputs[0].materialization.persistent_enabled, + "Pair must declare ONLY classification rows (no retained RGB output)", + ) + journey.save( + "exact-output-slots.json", + { + "callable_return_0": "unchanged engineering_cells dense object labels, int32[64,64]", + "callable_return_1": final.artifact_outputs[0], + "fixture_rows": "engineering_object_rows", + "final_main_flow": final.main_flow_materialization, + }, + ) run = journey.execute_source(source_request, connection) journey.save("original-session-run.json", run) (row,) = (row for row in run.rows if row.root == str(PLATE)) - require(row.compiled and row.terminal_status == "complete", - "Original native compile or execution failed") - results = journey.call("openhcs_query_plate_files", { - "plate_path": str(output_plate), "kind": "result", "limit": 50, - "include_previews": True, "max_preview_lines": 10, - }) - object_file, object_rows = csv_preview(results, "engineering_object_rows_step0_details.csv") - class_file, class_rows = csv_preview(results, PAIR_STEP + "_1_measurements_step1_details.csv") - declared_csvs = {path for group in final.artifact_outputs[0].materialization.paths - for path in group.candidate_paths} - require(class_file in declared_csvs, "Classification CSV is not the current declared output slot") + require( + row.compiled and row.terminal_status == "complete", + "Original native compile or execution failed", + ) + results = journey.call( + "openhcs_query_plate_files", + { + "plate_path": str(output_plate), + "kind": "result", + "limit": 50, + "include_previews": True, + "max_preview_lines": 10, + }, + ) + object_file, object_rows = csv_preview( + results, "engineering_object_rows_step0_details.csv" + ) + class_file, class_rows = csv_preview( + results, PAIR_STEP + "_1_measurements_step1_details.csv" + ) + declared_csvs = { + path + for group in final.artifact_outputs[0].materialization.paths + for path in group.candidate_paths + } + require( + class_file in declared_csvs, + "Classification CSV is not the current declared output slot", + ) rows_evidence = assert_measurement_rows(object_rows, class_rows) # Semantic producer identity, NOT a hardcoded route/submission address. - viewer_args = {"host": "127.0.0.1", "port": args.viewer_port, "transport_mode": "tcp"} - state = journey.call("openhcs_get_viewer_window_state", {**viewer_args, "include_response": False}) - layers = [layer for layer in state.layers if any( - owner.step_name == PAIR_STEP and owner.pipeline_position == 1 and owner.output_kind == "main" - for owner in layer.producer_identities)] - require(state.observed and len(layers) == 1, "Missing/ambiguous paired main-flow viewer layer") - payloads = journey.call("openhcs_get_viewer_window_payloads", { - **viewer_args, "route_key": layers[0].route_key, "axis_indices": [], - "include_array_values": True, "max_array_elements": 4096, "include_response": False, - }) - require(payloads.observed and len(payloads.layers) == 1 and len(payloads.layers[0].payloads) == 1, - "Missing/ambiguous full paired viewer pixels") + viewer_args = { + "host": "127.0.0.1", + "port": args.viewer_port, + "transport_mode": "tcp", + } + state = journey.call( + "openhcs_get_viewer_window_state", + {**viewer_args, "include_response": False}, + ) + layers = [ + layer + for layer in state.layers + if any( + owner.step_name == PAIR_STEP + and owner.pipeline_position == 1 + and owner.output_kind == "main" + for owner in layer.producer_identities + ) + ] + require( + state.observed and len(layers) == 1, + "Missing/ambiguous paired main-flow viewer layer", + ) + payloads = journey.call( + "openhcs_get_viewer_window_payloads", + { + **viewer_args, + "route_key": layers[0].route_key, + "axis_indices": [], + "include_array_values": True, + "max_array_elements": 4096, + "include_response": False, + }, + ) + require( + payloads.observed + and len(payloads.layers) == 1 + and len(payloads.layers[0].payloads) == 1, + "Missing/ambiguous full paired viewer pixels", + ) pixels = payloads.layers[0].payloads[0] - require(pixels.array_value_summary["included"] is True and - pixels.array_value_summary["size"] == 4096, "Viewer pixels truncated") - pixel_evidence = assert_label_identity(pixels.array_values, pixels.array_value_summary["shape"]) + require( + pixels.array_value_summary["included"] is True + and pixels.array_value_summary["size"] == 4096, + "Viewer pixels truncated", + ) + pixel_evidence = assert_label_identity( + pixels.array_values, pixels.array_value_summary["shape"] + ) # Persistent file identity derives from the actual compiled final checkpoint. - image_query = journey.call("openhcs_query_plate_files", { - "plate_path": str(output_plate), "kind": "image", "limit": 50, "include_previews": False, - }) - require(image_query.truncated_count == 0, "Persisted image inventory truncated") + image_query = journey.call( + "openhcs_query_plate_files", + { + "plate_path": str(output_plate), + "kind": "image", + "limit": 50, + "include_previews": False, + }, + ) + require( + image_query.truncated_count == 0, + "Persisted image inventory truncated", + ) # The last step publishes the plate's final main image under its # compiled output_dir; checkpoint directories can include other # artifact projections and are not the plate image inventory. final_images = Path(final.output_dir) - images = [record for record in image_query.records if record.source_path is not None and - Path(record.source_path).is_relative_to(final_images)] - require(len(images) == 1, "Missing/ambiguous persisted final label image") - saved = journey.call("openhcs_sample_plate_image", { - "plate_path": str(output_plate), "image_path": images[0].virtual_path, - "height": 64, "width": 64, "include_array_values": True, "max_array_elements": 4096, - }) - require(saved.sample_included and saved.sample_origin_yx == (0, 0), "Persisted pixels omitted/cropped") - saved_evidence = assert_label_identity(saved.sample_values, saved.sample_shape) - require(to_jsonable(saved.sample_values) == to_jsonable(pixels.array_values), "Streamed/saved label pixels differ") - journey.save("assertions-before-close.json", { - "rows": rows_evidence, "streamed_labels": pixel_evidence, - "saved_labels": saved_evidence, "saved_image": images[0].source_path, - "fixture_csv": object_file, "classification_csv": class_file, - "source_sha256": hashlib.sha256(clean.source.encode()).hexdigest(), - }) + images = [ + record + for record in image_query.records + if record.source_path is not None + and Path(record.source_path).is_relative_to(final_images) + ] + require( + len(images) == 1, "Missing/ambiguous persisted final label image" + ) + saved = journey.call( + "openhcs_sample_plate_image", + { + "plate_path": str(output_plate), + "image_path": images[0].virtual_path, + "height": 64, + "width": 64, + "include_array_values": True, + "max_array_elements": 4096, + }, + ) + require( + saved.sample_included and saved.sample_origin_yx == (0, 0), + "Persisted pixels omitted/cropped", + ) + saved_evidence = assert_label_identity( + saved.sample_values, saved.sample_shape + ) + require( + to_jsonable(saved.sample_values) + == to_jsonable(pixels.array_values), + "Streamed/saved label pixels differ", + ) + journey.save( + "assertions-before-close.json", + { + "rows": rows_evidence, + "streamed_labels": pixel_evidence, + "saved_labels": saved_evidence, + "saved_image": images[0].source_path, + "fixture_csv": object_file, + "classification_csv": class_file, + "source_sha256": hashlib.sha256( + clean.source.encode() + ).hexdigest(), + }, + ) closed_by_driver = args.runtime_handle is None if closed_by_driver: - closed_viewer = journey.call("openhcs_close_viewer_window", {**viewer_args, "confirmed": True}) - require(closed_viewer.process_exited is True, "Original viewer process exit unproven") - closed_runtime = journey.call("openhcs_close_owned_runtime", to_jsonable(RuntimeBootstrapCloseRequest(handle=original_handle))) - require(closed_runtime.handle == original_handle and closed_runtime.outcome.process_exited is True, - "Original runtime process exit unproven") - journey.save("ACCEPTANCE.json", {"engineering_journey_passed": True, - "native_compiled": row.compiled, - "native_execution_id": row.finished_execution_id, - "native_execution_status": row.terminal_status, - "scope": "public paired engineering only; not installed biology/global FULL", - "processes_exited": True if closed_by_driver else None, - "lifecycle_owner": "driver exact owned close" if closed_by_driver else "parent retains handed-off runtime/viewer; no close attempted"}) + closed_viewer = journey.call( + "openhcs_close_viewer_window", + {**viewer_args, "confirmed": True}, + ) + require( + closed_viewer.process_exited is True, + "Original viewer process exit unproven", + ) + closed_runtime = journey.call( + "openhcs_close_owned_runtime", + to_jsonable( + RuntimeBootstrapCloseRequest(handle=original_handle) + ), + ) + require( + closed_runtime.handle == original_handle + and closed_runtime.outcome.process_exited is True, + "Original runtime process exit unproven", + ) + journey.save( + "ACCEPTANCE.json", + { + "engineering_journey_passed": True, + "native_compiled": row.compiled, + "native_execution_id": row.finished_execution_id, + "native_execution_status": row.terminal_status, + "scope": "public paired engineering only; not installed biology/global FULL", + "processes_exited": True if closed_by_driver else None, + "lifecycle_owner": ( + "driver exact owned close" + if closed_by_driver + else "parent retains handed-off runtime/viewer; no close attempted" + ), + }, + ) except BaseException as error: - journey.save("FAILED-OR-UNCERTAIN.json", {"exception": type(error).__name__, - "message": str(error), "disposition": "STOP: no replay; parent retains original handles and cleanup ownership"}) + journey.save( + "FAILED-OR-UNCERTAIN.json", + { + "exception": type(error).__name__, + "message": str(error), + "disposition": "STOP: no replay; parent retains original handles and cleanup ownership", + }, + ) raise def offline_check(): labels = expected_labels() assert_label_identity(labels, (64, 64)) - objects = [{"slice_index": "0", "object_label": str(label), "pixel_count": str(area), - "calibration_um": "1.3556"} for label, area in ((1, 64), (2, 256))] + objects = [ + { + "slice_index": "0", + "object_label": str(label), + "pixel_count": str(area), + "calibration_um": "1.3556", + } + for label, area in ((1, 64), (2, 256)) + ] aggregate = {"object_label": "", "object_name": "", "slice_index": "0"} for name in QUADRANTS: aggregate[f"Classify_{name}_NumObjectsPerBin"] = str(int(name in QUADRANTS[:2])) - aggregate[f"Classify_{name}_PctObjectsPerBin"] = str(50 * int(name in QUADRANTS[:2])) - classes = [aggregate] + [{"slice_index": "0", "object_name": "engineering_cells", "object_label": str(label), - **{f"Classify_{name}": str(int(name == selected)) for name in QUADRANTS}} - for label, selected in ((1, "low_low"), (2, "high_low"))] + aggregate[f"Classify_{name}_PctObjectsPerBin"] = str( + 50 * int(name in QUADRANTS[:2]) + ) + classes = [aggregate] + [ + { + "slice_index": "0", + "object_name": "engineering_cells", + "object_label": str(label), + **{f"Classify_{name}": str(int(name == selected)) for name in QUADRANTS}, + } + for label, selected in ((1, "low_low"), (2, "high_low")) + ] assert_measurement_rows(objects, classes) bad_labels = copy.deepcopy(labels) bad_labels[8][8] = 2 @@ -505,19 +942,23 @@ def offline_check(): bad_classes[1]["Classify_low_high"] = "1" wrong_calibration = copy.deepcopy(objects) wrong_calibration[1]["calibration_um"] = "256" - probes = [lambda: assert_label_identity(bad_labels, (64, 64)), - lambda: assert_label_identity(labels, (64, 64, 3)), - lambda: assert_label_identity(labels[:32], (64, 64)), - lambda: assert_measurement_rows(objects, bad_classes), - lambda: assert_measurement_rows(wrong_calibration, classes), - lambda: assert_measurement_rows(objects, classes[:2])] + probes = [ + lambda: assert_label_identity(bad_labels, (64, 64)), + lambda: assert_label_identity(labels, (64, 64, 3)), + lambda: assert_label_identity(labels[:32], (64, 64)), + lambda: assert_measurement_rows(objects, bad_classes), + lambda: assert_measurement_rows(wrong_calibration, classes), + lambda: assert_measurement_rows(objects, classes[:2]), + ] for probe in probes: try: probe() except AssertionError: continue raise AssertionError("Oracle accepted a counterexample") - print("OFFLINE ONLY: label/row positives + 6 rejection oracles; no MCP/native/fixture invocation") + print( + "OFFLINE ONLY: label/row positives + 6 rejection oracles; no MCP/native/fixture invocation" + ) def main(): @@ -528,9 +969,15 @@ def main(): parser.add_argument("--output-suffix", default="_classification388_public_pair") parser.add_argument("--runtime-port", type=int, default=5993) parser.add_argument("--viewer-port", type=int, default=5992) - parser.add_argument("--runtime-handle", type=Path, - help="Parent's exact retained bootstrap handle JSON; observe/reuse, never startup or close that owner's processes") - parser.add_argument("--fixture-function-id", help="Explicit original registered fixture identity; verify original bytes and never register again") + parser.add_argument( + "--runtime-handle", + type=Path, + help="Parent's exact retained bootstrap handle JSON; observe/reuse, never startup or close that owner's processes", + ) + parser.add_argument( + "--fixture-function-id", + help="Explicit original registered fixture identity; verify original bytes and never register again", + ) args = parser.parse_args() if args.run: run(args) From 74e74e67e3fe13d0de0277a5125b70804ee9eab7 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 04:08:14 -0400 Subject: [PATCH 37/84] B6: re-pin documentation authorities after B6's own code changes Formatting-only changes are AST-equal; the semantic B6 changes were compared against each dependent page and change none of their claims. Co-Authored-By: Claude Opus 5.5 --- docs/audits/architecture_a_m.json | 266 +++++---- docs/audits/architecture_n_z_development.json | 387 +++++++------ docs/audits/project_surface.json | 27 +- docs/audits/user_docs.json | 527 ++++++++++-------- 4 files changed, 669 insertions(+), 538 deletions(-) diff --git a/docs/audits/architecture_a_m.json b/docs/audits/architecture_a_m.json index 446ecc6101..ff6bc9e8e0 100644 --- a/docs/audits/architecture_a_m.json +++ b/docs/audits/architecture_a_m.json @@ -81,32 +81,32 @@ }, { "path": "openhcs/interop/cellprofiler/module_declarations.py", - "sha256": "6bbf41c158496a02439be2ee5aef12d7712fb632419c226a64c9259a55438b95", + "sha256": "493efb9c749b10ba54713d0c16ee7118f07ff35afbafc9ff156b7100dbd63b19", "role": "Defines CellProfilerModule leaves and require_module as owners of module identity, lowering, headless-import validation and artifact semantics." }, { "path": "openhcs/interop/cellprofiler/module_artifact_declarations.py", - "sha256": "f70b52114fb5d5e2c2fc49cd9a2b46986bde48ce02d82f1a34a67f072ca2ab20", + "sha256": "286638babca312c85ee0b8319fd44ba86be5868a3c1a20b74b1b48a0bfdbc652", "role": "Defines the nominal interactive-module family that rejects headless import." }, { "path": "openhcs/core/callable_contract.py", - "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", + "sha256": "2060f856bcb0cc8ef57b92fc17a1e558a676e4033075950b1a4e41dc8656d330", "role": "Declares CallableContract, the declaration-owned callable metadata at the nominal boundary." }, { "path": "openhcs/core/artifacts.py", - "sha256": "65eb0ec5c903366243ca8bd5ea07412e8d34bdb81e6015fa76b3c4c59838733f", + "sha256": "907551ae0ce4bf42a557554f5df0252b0a88fc9b7c1b7ac19705aade66b6656f", "role": "Declares the ArtifactSpecRelation family for cross-artifact semantics." }, { "path": "openhcs/core/dataset_sources/source.py", - "sha256": "097fde4d2b74031eacf53a88e79f499417e8c1c97ba0753e8a9df831b307efa6", + "sha256": "f736728d2f715b72cd5479fcc79520911b037d3410a42ebfc9febef2e51ffda6", "role": "Defines the DatasetSource registry and its layout detection and workspace projection boundary." }, { "path": "openhcs/core/dataset_sources/source_bindings_source.py", - "sha256": "fccda877f39e7b2f99d6cc95143480e5955186264e316e832ca9a69c2ae90b8c", + "sha256": "7d2ac740ebb081869ad98573032849732f9b64cbbfb4fc7bd9778becc328c78c", "role": "Defines SourceBindingsSource, which composes a dataset source with SourceBindingsConfig declarations." }, { @@ -116,7 +116,7 @@ }, { "path": "openhcs/agent/capabilities.py", - "sha256": "eaebe33af01d057d1d21b3701f958bc96ede3ec5d0bddddca9917c1164ef0042", + "sha256": "87c8e2e5aeb425b9a2eb690d48bdd6f5af7a327c389deac261d7e4e0da5d609f", "role": "Defines AgentCapabilityDeclaration and CapabilityTransport members carrying their transport semantics." }, { @@ -136,6 +136,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/abstraction_lattices.rst against the later B6 changes to openhcs/interop/cellprofiler/module_declarations.py, openhcs/interop/cellprofiler/module_artifact_declarations.py, openhcs/core/callable_contract.py, openhcs/core/artifacts.py, openhcs/core/dataset_sources/source.py, openhcs/core/dataset_sources/source_bindings_source.py, openhcs/agent/capabilities.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/abstraction_lattices.rst in full and compared it against the changed authorities: openhcs/core/registry_strategies.py (deleted; mixins now in metaclass_registry.strategies), openhcs/microscopes/microscope_base.py (deleted; DatasetSource family in openhcs/core/dataset_sources), module_declarations.py, module_artifact_declarations.py, callable_contract.py, openhcs/agent/capabilities.py and the two CellProfiler tests; replaced the microscope-handler owner and SourceBindingsHandler with DatasetSource and SourceBindingsSource, attributed the strategy mixins to metaclass-registry, and confirmed CellProfilerModule.require_module, ArtifactSpecRelation, RuntimePlaneAxisStrategy, for_enum_member and CapabilityTransport against current source. Historical commit references are left as history.", "Re-read docs/source/architecture/abstraction_lattices.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/registry_strategies.py; openhcs/interop/cellprofiler/module_declarations.py; openhcs/core/callable_contract.py; openhcs/microscopes/microscope_base.py; openhcs/agent/capabilities.py; tests/unit/test_cellprofiler_static_deletion_gates.py.", @@ -173,12 +174,12 @@ "authority": [ { "path": "openhcs/domains/microscopy/analysis_consolidation.py", - "sha256": "292f7138a00dc22dba5b8d8af5de6140d247b955723cc807cc982ffec7a7dd2d", + "sha256": "1084a6761a379417f86a042d635d4d43fbe55c4ac497998e48c94fe56807393a", "role": "Defines AnalysisConsolidationHook, its saved/reused output observations, the per-directory destinations and the FileManager summary writer." }, { "path": "openhcs/core/post_execute.py", - "sha256": "42183491eae6a5a17fd68c179970b1682343f1f73f355291f2ba4b02c784905d", + "sha256": "2d925b58e1fe242acf705f16fe4ac0ef34119da9827eca00c126c4194eed2919", "role": "Defines the PostExecuteHook family: binding from the global config, observation combine, and running after every partition succeeds." }, { @@ -198,7 +199,7 @@ }, { "path": "tests/unit/test_function_artifact_materialization.py", - "sha256": "e92bd682c105778d8bde9cb0f63ff1baed9f4c082a05d2c7f8f04bf539bd699f", + "sha256": "1f6c1aafba18111da085153bbbba8b8141096b94e626647ca717be172cb63a74", "role": "Proves the hook consumes saved CSV content, uses the compiled backend destination, skips a disabled config, and propagates failures." }, { @@ -213,6 +214,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/analysis_consolidation_system.rst against the later B6 changes to openhcs/domains/microscopy/analysis_consolidation.py, openhcs/core/post_execute.py, tests/unit/test_function_artifact_materialization.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/analysis_consolidation_system.rst in full and compared it against the changed authorities: openhcs/core/orchestrator/analysis_consolidation.py (moved to openhcs/domains/microscopy/analysis_consolidation.py as a post-execute hook), consolidate_analysis_results.py, openhcs/core/config.py (consolidation config moved to openhcs/domains/microscopy/config.py), the three tests and openhcs/processing/materialization/core.py; rewrote the flow paragraph for AnalysisConsolidationHook and PostExecuteHook, removed the obsolete output-selection/skip-rendering paragraph, and confirmed the writer, DataSink save context, output-root destination and failure propagation against current source. Dropped test_orchestrator_lane_planning.py and materialization/core.py, which no longer own a stated claim.", "Re-read docs/source/architecture/analysis_consolidation_system.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/orchestrator/analysis_consolidation.py; openhcs/core/config.py; tests/unit/test_orchestrator_lane_planning.py.", @@ -241,37 +243,37 @@ "authority": [ { "path": "openhcs/core/artifacts.py", - "sha256": "65eb0ec5c903366243ca8bd5ea07412e8d34bdb81e6015fa76b3c4c59838733f", + "sha256": "907551ae0ce4bf42a557554f5df0252b0a88fc9b7c1b7ac19705aade66b6656f", "role": "Defines the ArtifactType families, ArtifactSpec/ArtifactSpecRef, ArtifactSpecRelation leaves and materialisation declarations." }, { "path": "openhcs/core/callable_contract.py", - "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", + "sha256": "2060f856bcb0cc8ef57b92fc17a1e558a676e4033075950b1a4e41dc8656d330", "role": "Defines CallableContract and its ordered artifact input and output collections." }, { "path": "openhcs/core/function_patterns.py", - "sha256": "95b9056c6980e4f7943f909aeae8262252c9aa175247a534befc8d4cc459fdd2", + "sha256": "3dbdad68a4c7c765cc0dae056079929e5e4632f39bdbe2e746bea3acc171562d", "role": "Defines InvocationArtifactInputEdgePlan, its main_flow_projection and storage plan, and MainFlowInputProjection." }, { "path": "openhcs/core/pipeline/function_contracts.py", - "sha256": "13bcaf7c7a62654f512d2cc739edccd2deeff35d99970ca493f3e827afbda91a", + "sha256": "883d22780ab41f122d6ed651c9f59fea540120f8fcf50d30e348ac295bd4fca9", "role": "Defines the special_inputs, artifact_inputs and artifact_outputs decorators." }, { "path": "openhcs/core/pipeline/path_planner.py", - "sha256": "677d106969aae4d818745bff29b6f84e487ec79a044c8eaf26bea1916236fc6c", + "sha256": "bc1380d505dae574c6dc1255afcba59b0513d933716017c05217baabed104d24", "role": "Implements artifact satisfaction and exact compiled path planning." }, { "path": "tests/unit/test_implicit_main_flow_compilation.py", - "sha256": "9f8f3c44f096ce0318612df7d51affbb21a7bbc837c9846d599dbe20c0a8998b", + "sha256": "1feb8d922c8551500b7d38db1ca1d1aa43475af1bc9a3552443980c2662bf4e4", "role": "Compiles grouped native output into a CellProfiler Threshold consumer and asserts the main-flow edge, group scope and execution scope." }, { "path": "tests/unit/test_callable_contract.py", - "sha256": "cc220f53107a08b0406b6e0dd49896e3e4c3c0af6e527b0ef00d4fdaba8f0b2c", + "sha256": "b0ce0cba2e396e53f8cb9e95533ff918788e8c6dd60037a66e0aeda603bc64c2", "role": "Verifies callable contract construction, validation, and artifact semantics." }, { @@ -286,6 +288,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/artifact_contract_system.rst against the later B6 changes to openhcs/core/artifacts.py, openhcs/core/callable_contract.py, openhcs/core/function_patterns.py, openhcs/core/pipeline/function_contracts.py, openhcs/core/pipeline/path_planner.py, tests/unit/test_implicit_main_flow_compilation.py, tests/unit/test_callable_contract.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/artifact_contract_system.rst in full and compared it against the changed authorities: openhcs/core/artifacts.py, openhcs/core/callable_contract.py, openhcs/core/pipeline/path_planner.py, tests/unit/test_callable_contract.py, tests/unit/test_artifact_input_projection_plan.py; replaced the removed consumes_main_flow flag with main_flow_projection in the occurrence and implicit main-flow sections and the regression description, tidied the special_inputs sentence, and confirmed every other named declaration (ArtifactType families, relation leaves, SourceImageProvenance, ArtifactPlanKeySelector, SelectedPlaneImageOutput, RuntimeValueStore, StepMaterializationConfig) exists with the stated role.", "Re-read docs/source/architecture/artifact_contract_system.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/artifacts.py; openhcs/core/callable_contract.py; openhcs/core/pipeline/path_planner.py; tests/unit/test_callable_contract.py; tests/unit/test_artifact_input_projection_plan.py.", @@ -312,7 +315,7 @@ "authority": [ { "path": "openhcs/authoring/session/operations/datasets.py", - "sha256": "208274effc2e790775d90528211fa4f640eb01f7ca3fdd9c927c3bf71e39b632", + "sha256": "43d9edfed59abe6b88e242d39f1ddc8f5078559e04b3875da1da07b993295993", "role": "Declares CompileDatasets and its initialised, idle and non-empty-pipeline availability checks." }, { @@ -322,7 +325,7 @@ }, { "path": "openhcs/authoring/session/compile_batch.py", - "sha256": "0cbe5a6b86a0fb5e9b92c4fc45f858c979031ac93594b317b9761c28f5aaf9ab", + "sha256": "d539bcf8df15581d23bc69701c5d30d489c3fe5f600f19236b28ec35c1c2d92f", "role": "Runs compile-only batches and compile-before-execution through BatchSubmitWaitEngine." }, { @@ -332,22 +335,22 @@ }, { "path": "openhcs/authoring/session/session.py", - "sha256": "587047b63972df89ec46a01ed2c92ff3ef9577335f58a980297db02094e4e8e0", + "sha256": "458d71fe96ab75153977c554fb748e78f1773cd7794b625d7ec233c9b22e4453", "role": "Implements run_datasets (reserve, reset, connect, compile all, submit each with compile_artifact_id) and the endpoint status binding." }, { "path": "openhcs/authoring/session/submission.py", - "sha256": "710ac218d1db01f231b67bf0fe229dc03fb428bc6655b23671f3a026be91c673", + "sha256": "e04fdb017c3905903885b21007ebb4ceb530ec16267ea26577b3c9609b82e2ce", "role": "Submits each execution with its compile_artifact_id and auxiliary observation-export parameters." }, { "path": "openhcs/authoring/session/progress.py", - "sha256": "cd34f68461b50506600da7854a3bf57646f2f0ff7288d0627bc40cb852a16825", + "sha256": "eef42f1c12c9649c36e7d1a86d664f527e6c1aa28868eb5b348d1ebdb5d548bd", "role": "Defines ExecutionProgress: one registry-mutation subscription, dirty marking and interval rebuilds on the session main thread." }, { "path": "tests/unit/test_u1_session_guards.py", - "sha256": "98de1f6239607e92d2aa35bd5e95f3e1c1c42887cc238e1291ffc830ca2cd4a5", + "sha256": "54e03867fc65858f21384d4c98215dc9d9accec1c0965a85d563b8410085b8f9", "role": "Guards that session state lives in the Qt-free session package and its operations are declared classes." } ], @@ -356,6 +359,7 @@ ], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/batch_workflow_service.rst against the later B6 changes to openhcs/authoring/session/operations/datasets.py, openhcs/authoring/session/compile_batch.py, openhcs/authoring/session/session.py, openhcs/authoring/session/submission.py, openhcs/authoring/session/progress.py, tests/unit/test_u1_session_guards.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/batch_workflow_service.rst in full and compared it against the changed authorities: the deleted openhcs/pyqt_gui batch workflow, compile batch, components, client service and execution-state modules and their tests (replaced by openhcs/authoring/session), openhcs/pyqt_gui/main.py, test_main_zmq_window_routing.py, endpoint_function_catalog_service.py, zmqruntime client.py and test_concurrent_initialization_progress.py; checked CompileDatasets availability, DatasetPipelineRequest/CompiledDataset, BatchSubmitWaitEngine, run_datasets ordering and compile_artifact_id submission, ExecutionProgress listener and interval rebuild, and the endpoint status binding against the session package. The page text is accurate; the record now points at the session owners.", "Read the complete page and challenged accuracy, freshness, duplication, boundary, flow, and navigation against the listed current authorities.", "python scripts/validate_docs.py docs/source/architecture passed, and .venv Sphinx -E -W --keep-going rendered all 152 active pages without warnings.", @@ -391,17 +395,17 @@ "authority": [ { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "3439d96f3fbfcb104a53019dc150049021d9a8132bd27096f1ccf510eefbe37b", + "sha256": "5d3646e17f7de843a5b36164f7cef7505da7841636423ae4fb4e00d194a87e9c", "role": "Implements .cppipe parsing, declaration-owned headless validation, and lowering into PipelineConfig and FunctionStep declarations." }, { "path": "openhcs/interop/cellprofiler/module_declarations.py", - "sha256": "6bbf41c158496a02439be2ee5aef12d7712fb632419c226a64c9259a55438b95", + "sha256": "493efb9c749b10ba54713d0c16ee7118f07ff35afbafc9ff156b7100dbd63b19", "role": "Owns module-specific callable, setting, headless-import, scope, artifact, and function-catalogue declarations." }, { "path": "openhcs/interop/cellprofiler/module_artifact_declarations.py", - "sha256": "f70b52114fb5d5e2c2fc49cd9a2b46986bde48ce02d82f1a34a67f072ca2ab20", + "sha256": "286638babca312c85ee0b8319fd44ba86be5868a3c1a20b74b1b48a0bfdbc652", "role": "Defines the nominal interactive-module family that rejects headless import." }, { @@ -411,7 +415,7 @@ }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", + "sha256": "90d75f6287c5f21f93d82960729199261f7906e0f4912b7728527260b9723a66", "role": "Declares ResolvedPipelineDefinition, the resolved step sequence the provider walks." }, { @@ -421,12 +425,12 @@ }, { "path": "tests/unit/test_cellprofiler_artifact_declarations.py", - "sha256": "fe894cd2b592d0755c106be3363eeaf078bc26d0b698a7ccfe6662ef0c0c475b", + "sha256": "5792727b3fdb1a7bffffe43fa63e7d5944a6c538ab35e90dc86e1dcfa908b189", "role": "Verifies typed image, object, measurement, relationship, table, grid, graph, and special artifact declarations." }, { "path": "tests/integration/test_cellprofiler_official30_zmq.py", - "sha256": "dbd6e113a7135901f936e9bc8278d727d678e9971a7c951fced9080adfd9d6df", + "sha256": "3297b0e54ad3a349d35b613bd7d406e92c974320e12723be80dcfc0a90be2116", "role": "Defines the executable Official30 import, execution, equivalence, and viewer evidence boundary." }, { @@ -450,6 +454,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/cellprofiler_interop.rst against the later B6 changes to openhcs/interop/cellprofiler/pipeline_import.py, openhcs/interop/cellprofiler/module_declarations.py, openhcs/interop/cellprofiler/module_artifact_declarations.py, openhcs/core/pipeline/compilation_session.py, tests/unit/test_cellprofiler_artifact_declarations.py, tests/integration/test_cellprofiler_official30_zmq.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/cellprofiler_interop.rst in full and compared it against the changed authorities: pipeline_import.py, module_declarations.py, module_artifact_declarations.py, test_cellprofiler_artifact_declarations.py, the Official30 integration test, test_cellprofiler_pipeline_import.py and benchmark/cellprofiler_reference_exports.py; replaced the removed StepSnapshot with ResolvedPipelineDefinition and confirmed the import flow, interactive-module rejection, InputSource members, FunctionStepExecutionScope.PLATE, the provider's named methods, the CPA SQLite/properties route and CellProfilerReferenceExportPlan against current source.", "Re-read docs/source/architecture/cellprofiler_interop.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/interop/cellprofiler/pipeline_import.py; openhcs/interop/cellprofiler/module_declarations.py; tests/unit/test_cellprofiler_artifact_declarations.py; tests/integration/test_cellprofiler_official30_zmq.py.", @@ -480,12 +485,12 @@ }, { "path": "openhcs/pyqt_gui/services/ui_bridge_object_state.py", - "sha256": "e548b367507b3fb6830ce9ecb844e3b756243af96fe38e0d35de5da0b974c1fd", + "sha256": "81b5ffbbd19952f85430962cd1bb61918ac3abe4dd9ec843b8f67e4590ffd06b", "role": "Projects live ObjectState scopes, fields, revisions, document capabilities, and mutation receipts through the UI bridge." }, { "path": "openhcs/agent/dto/ui_bridge.py", - "sha256": "918b30f34e7d096b19ce04017d9a991d0a10e4088c310797e497fb2444fc37c0", + "sha256": "5f5df140f0a32fa7e0a169981b7f346f7335a2a517676eefc7b29bd8fdc52a8f", "role": "Carries generic endpoint application and operation-lifecycle declarations through UI bridge descriptors and request/response envelopes." }, { @@ -510,7 +515,7 @@ }, { "path": "openhcs/pyqt_gui/services/ui_agent_bridge.py", - "sha256": "ce650c52475764657bbc0221109879594f9cf1482de8245d6c141f00474d5d23", + "sha256": "5a03e683a0677166361440c6c78803ec2dadf753342e69996153005bccab5473", "role": "Projects ObjectState branch and snapshot restoration through the attached-agent surface." }, { @@ -520,7 +525,7 @@ }, { "path": "tests/unit/pyqt_gui/test_ui_agent_bridge.py", - "sha256": "de6758859d9743e826369162855327e4a84056cafdd7ddfdc33868d2a3093353", + "sha256": "b03744b1fa201f77de26d2e6050422b6422ebce78443c478c5e8c5f55e0a3e24", "role": "Verifies read-only rejection, semantic no-op application, and branch restoration through the UI bridge." }, { @@ -530,7 +535,7 @@ }, { "path": "tests/unit/pyqt_gui/test_pipeline_code_document_transactionality.py", - "sha256": "0bb9b43d867a4e26f13b51531a2b7a847c6c3685841f8e002e209200a37366b6", + "sha256": "139725010d983c7e8a98731b4e462339c4188b8d9c44383677191810ad7bf24f", "role": "Verifies revision checks, validation, snapshots, rollback, and transactional application." }, { @@ -565,7 +570,7 @@ }, { "path": "tests/unit/test_pycodify_formatters.py", - "sha256": "2f0c6683a5255d01442ae33f676f601d5e3f1e3e04d1b2c2fc945b3c564796ce", + "sha256": "1dd67d1a782d04dc24ac2ba9ec1e457769c1c6c9191c992f0a1cc0feb78487c5", "role": "Verifies clean/full parameter order and semantic default handling." }, { @@ -577,6 +582,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/code_ui_interconversion.rst against the later B6 changes to openhcs/pyqt_gui/services/ui_bridge_object_state.py, openhcs/agent/dto/ui_bridge.py, openhcs/pyqt_gui/services/ui_agent_bridge.py, tests/unit/pyqt_gui/test_ui_agent_bridge.py, tests/unit/pyqt_gui/test_pipeline_code_document_transactionality.py, tests/unit/test_pycodify_formatters.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/code_ui_interconversion.rst in full and compared it against the changed authorities: openhcs/core/pipeline_document.py (PipelineDocumentAuthority renamed PipelineDocumentCodec; PipelineDocument unchanged), ui_bridge_object_state.py, openhcs/agent/dto/ui_bridge.py, ui_bridge_transport.py, ui_bridge_server.py, zmqruntime messages.py, python-introspect annotation_types.py, ui_agent_bridge.py and its test, function_step_code_document.py (now FunctionStepDocumentCodec), the code-document transactionality test, step_parameter_editor.py (the pickled .step settings files are gone, which the page never described), pycodify_formatters.py and its test; checked the schema/bridge-protocol/application identity checks on both bridge ends, the field-mutation request token without base revision, enum values in field help, code-document revision apply, and Plate Manager naming. Every claim the page makes still holds; no text change.", "Read the complete page and challenged accuracy, freshness, duplication, boundary, flow, and navigation against the listed current authorities.", "python scripts/validate_docs.py docs/source/architecture passed, and .venv Sphinx -E -W --keep-going rendered all 152 active pages without warnings.", @@ -608,12 +614,12 @@ "authority": [ { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "sha256": "d1216010dec1804c51aec9c774f298513638ca016d500fe6328d76a9a9bfb4d6", "role": "Lowers declarations into typed step plans, including declaration-derived framework-device assignments." }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", + "sha256": "90d75f6287c5f21f93d82960729199261f7906e0f4912b7728527260b9723a66", "role": "Defines the per-context CompilationSession and ResolvedPipelineDefinition." }, { @@ -627,6 +633,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/compilation_service.rst against the later B6 changes to openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/compilation_session.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/compilation_service.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/compiled_execution.py; the page only names the session, typed-plan and execution-bundle model and redirects to pipeline_compilation_system, all of which still exist (CompilationSession, CompiledStepPlan, CompiledExecutionBundle), so the redirect stays unchanged.", "Re-read docs/source/architecture/compilation_service.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/pipeline/compiler.py; openhcs/core/pipeline/compilation_session.py; openhcs/core/compiled_execution.py.", @@ -647,17 +654,17 @@ "authority": [ { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "sha256": "d1216010dec1804c51aec9c774f298513638ca016d500fe6328d76a9a9bfb4d6", "role": "Implements the current compiler stages, including declaration-derived framework-device assignment." }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", + "sha256": "90d75f6287c5f21f93d82960729199261f7906e0f4912b7728527260b9723a66", "role": "Defines ResolvedPipelineDefinition, which replaced resolved step snapshots, and CompilationSession." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", + "sha256": "f4a851d7de2ea82e1d548f37e6be2511fd20d799580193496a3746e5484906ba", "role": "Defines typed step-plan fields, exact framework footprints, and framework-local device bindings." }, { @@ -671,6 +678,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/compilation_system_detailed.rst against the later B6 changes to openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/compiled_step_plan.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/compilation_system_detailed.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/step_snapshot.py (deleted in 0adb68108; resolved steps now live on ResolvedPipelineDefinition in compilation_session.py), openhcs/core/compiled_step_plan.py; replaced StepSnapshot with ResolvedPipelineDefinition in the redirect text and confirmed the redirect target page exists.", "Re-read docs/source/architecture/compilation_system_detailed.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/pipeline/compiler.py; openhcs/core/pipeline/step_snapshot.py; openhcs/core/compiled_step_plan.py.", @@ -691,12 +699,12 @@ "authority": [ { "path": "external/ObjectState/src/objectstate/lazy_factory.py", - "sha256": "64ae924c176757863a84eef19b9df0eff516e32443c2661134ec2ae72f234bf9", + "sha256": "4f86ec64d975b354203539581943f36a75e34fbf2da076a3ef9b32036e54bff9", "role": "Implements generic lazy dataclass creation and resolution owned by ObjectState." }, { "path": "openhcs/core/config.py", - "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "sha256": "7b5622b3d82b1e1d672eaedb14e054be81baef417a736adb06bea34fb6708b28", "role": "Declares the kernel OpenHCS configuration dataclasses that consume ObjectState mechanics." }, { @@ -710,6 +718,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/component_configuration_framework.rst against the later B6 changes to external/ObjectState/src/objectstate/lazy_factory.py, openhcs/core/config.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/component_configuration_framework.rst in full and compared it against the changed authorities: external/ObjectState/src/objectstate/lazy_factory.py, openhcs/core/config.py (domain sections such as analysis consolidation moved to openhcs/domains/microscopy/config.py, now added as an authority); the page only states that ObjectState owns lazy configuration and that OpenHCS documents its domain declarations, and redirects to component_system_integration and external_foundations, both of which exist, so the redirect stays unchanged.", "Re-read docs/source/architecture/component_configuration_framework.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: external/ObjectState/src/objectstate/lazy_factory.py; openhcs/core/config.py.", @@ -745,7 +754,7 @@ }, { "path": "openhcs/domains/microscopy/axes.py", - "sha256": "531695809259324032e0b88632813323b4fe6a9a176ad6f636aeef8dec332399", + "sha256": "28cc16a296d4580351cd9abec553155e59c4d56a6ea20d52cbd6abb2879b75c6", "role": "Declares the microscopy axis family, the OpenHCS domain declaration registered through metaclass-registry." } ], @@ -754,6 +763,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/component_processor_metaprogramming.rst against the later B6 changes to openhcs/domains/microscopy/axes.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/component_processor_metaprogramming.rst in full and compared it against the changed authorities: signature_analyzer.py, metaclass_registry/core.py, openhcs/constants/constants.py; confirmed signature analysis, callable-declaration projection and AutoRegisterMeta registration remain in the external packages, and replaced the deleted component-enum authority with the microscopy axis family declaration; no page edit was needed.", "Read the complete page and challenged accuracy, freshness, duplication, boundary, flow, and navigation against the listed current authorities.", "python scripts/validate_docs.py docs/source/architecture passed, and .venv Sphinx -E -W --keep-going rendered all 152 active pages without warnings.", @@ -777,12 +787,12 @@ "authority": [ { "path": "openhcs/core/axes.py", - "sha256": "8d93133639c576050f089fb8aab99937c8982aff07e742fea44f8a12e62bb7b2", + "sha256": "03e5a61d808a0221507b2be93f3672cc5f4421e53eb72ce4b683309cc673213b", "role": "Declares Axis, AxisFamily and the PartitionAxis role whose single member partitions compiled contexts." }, { "path": "openhcs/core/config.py", - "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "sha256": "7b5622b3d82b1e1d672eaedb14e054be81baef417a736adb06bea34fb6708b28", "role": "Declares the step-level variable_components and group_by fields and the pipeline-level sequential_components field." }, { @@ -797,12 +807,12 @@ }, { "path": "openhcs/core/dataset_sources/interfaces.py", - "sha256": "dd88254bfd5da8b0fdf0a038e358f36b49db5b8cf0b93213d56dc928af965f00", + "sha256": "31e3f5789c4a1603de12a0f0f95ceb529ff3190d45f2315e2674ef81a9177047", "role": "Defines MetadataComponentValueSet, the metadata-handler projection of complete axis values." }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "sha256": "d1216010dec1804c51aec9c774f298513638ca016d500fe6328d76a9a9bfb4d6", "role": "Consumes the axis declarations to construct execution-axis contexts, stacks, grouping, and typed plans." } ], @@ -812,6 +822,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/component_system_integration.rst against the later B6 changes to openhcs/core/axes.py, openhcs/core/config.py, openhcs/core/dataset_sources/interfaces.py, openhcs/core/pipeline/compiler.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/component_system_integration.rst in full and compared it against the changed authorities: constants.py, component_set.py, component_values.py, parser_metaprogramming.py, compiler.py; verified AxisValues, FilenameParseResult, MetadataComponentValueSet, variable_components/group_by/sequential_components and AxisFamily.partition_axis in current source, and replaced the nonexistent multiprocessing_axis declaration with the family's PartitionAxis.", "Re-read docs/source/architecture/component_system_integration.rst in full against repository base 32ca17c17 and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/constants/constants.py; openhcs/core/component_set.py; openhcs/core/pipeline/compiler.py.", @@ -834,7 +845,7 @@ "authority": [ { "path": "openhcs/core/axes.py", - "sha256": "8d93133639c576050f089fb8aab99937c8982aff07e742fea44f8a12e62bb7b2", + "sha256": "03e5a61d808a0221507b2be93f3672cc5f4421e53eb72ce4b683309cc673213b", "role": "Defines AxisFamily.named decoding, variable_axes as every axis but the partition axis, and the Ungrouped declaration." }, { @@ -844,17 +855,17 @@ }, { "path": "openhcs/core/pipeline/funcstep_contract_validator.py", - "sha256": "4a167f620be9ff6e6524101652b6dfa25984b3a389b19da3f5ce78c14d5947d4", + "sha256": "56968226fda4d3909aa581604e12e170d1b9b13ea1cd34536f6be32d6f4566a1", "role": "Validates allowed group_by roles, required callable axis roles, and declared memory contracts per callable." }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "sha256": "d1216010dec1804c51aec9c774f298513638ca016d500fe6328d76a9a9bfb4d6", "role": "Rejects pipeline sequential axes that are also a step's variable axes." }, { "path": "tests/unit/test_funcstep_contract_validator.py", - "sha256": "a208ca59a60b2d5bbcc9955e99592fd480368995336f6ef86a0a06f8f6991747", + "sha256": "e40f4cd60e418dcafcea5d8b6ddcd61e222478e14da3bd3d8a5d5f5263af9cc0", "role": "Covers accepted and rejected grouping, required-role, and callable-contract combinations." } ], @@ -865,6 +876,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/component_validation_system.rst against the later B6 changes to openhcs/core/axes.py, openhcs/core/pipeline/funcstep_contract_validator.py, openhcs/core/pipeline/compiler.py, tests/unit/test_funcstep_contract_validator.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/component_validation_system.rst in full and compared it against the changed authorities: validation.py, funcstep_contract_validator.py, test_funcstep_contract_validator.py; traced GenericValidator.validate_step, validate_dict_pattern_keys, validate_allowed_group_by, validate_required_axis_roles and the compiler's sequential-components check, and rewrote the check list and consumer guidance to match.", "Re-read docs/source/architecture/component_validation_system.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/components/validation.py; openhcs/core/pipeline/funcstep_contract_validator.py; tests/unit/test_funcstep_contract_validator.py.", @@ -894,7 +906,7 @@ }, { "path": "openhcs/core/orchestrator/bundle_execution.py", - "sha256": "f5ba09a2ec7c688b1ca08167ab398cc976a9e2c39b301ae98f3c5fd560a25c85", + "sha256": "7665bd4ab8b9ce73d3dcb4fb9679e836b23bffb81d70541baef2e114ee54b0d9", "role": "Runs one compiled bundle, then the parent-owned dataset-scoped steps and post-execute hooks." }, { @@ -904,7 +916,7 @@ }, { "path": "openhcs/core/orchestrator/worker_execution.py", - "sha256": "23a3ef2a70378dd1bb5c413a56e36bbb3fa013142741c9b45407bfc47de8c4a4", + "sha256": "1d71883f545f7f8702c64710f968b4138d0748fa0c58b6e1b87e06a7045e5ebf", "role": "Owns executor construction, the WorkerCommand family served by process workers, result collection, cancellation, and cleanup." }, { @@ -919,7 +931,7 @@ }, { "path": "tests/unit/test_orchestrator_lane_planning.py", - "sha256": "3d5c2f3662b8d4c330582af0c4a649b8f4d9582e064d9cf3159798c8934a2bbb", + "sha256": "4a83337fa35c68e4f641760ebbf86e4cc30f6624da55ccebd09be636a4b56bd0", "role": "Verifies that worker-lane grouping remains correct when context mapping keys do not encode axis identity." }, { @@ -936,6 +948,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/concurrency_model.rst against the later B6 changes to openhcs/core/orchestrator/bundle_execution.py, openhcs/core/orchestrator/worker_execution.py, tests/unit/test_orchestrator_lane_planning.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/concurrency_model.rst in full and compared it against the changed authorities: compiled_execution.py, processing_context.py, worker_lanes.py, test_orchestrator_lane_planning.py, worker_execution.py, test_worker_start_policy.py; confirmed CompiledExecutionBundle, WorkerLaneExecutionContext, resolve_worker_start_context, the cancellation signal and AXIS_COMPLETED, renamed compiled_plate_execution to bundle_execution, and corrected the lane, worker-command, progress and post-execute claims.", "Re-read docs/source/architecture/concurrency_model.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/compiled_execution.py; openhcs/core/orchestrator/worker_execution.py; openhcs/core/orchestrator/cancellation.py; openhcs/core/worker_start_policy.py; tests/unit/test_worker_start_policy.py.", @@ -958,7 +971,7 @@ "authority": [ { "path": "external/ObjectState/src/objectstate/lazy_factory.py", - "sha256": "64ae924c176757863a84eef19b9df0eff516e32443c2661134ec2ae72f234bf9", + "sha256": "4f86ec64d975b354203539581943f36a75e34fbf2da076a3ef9b32036e54bff9", "role": "Implements reusable lazy dataclass configuration mechanics." }, { @@ -968,7 +981,7 @@ }, { "path": "openhcs/core/config.py", - "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "sha256": "7b5622b3d82b1e1d672eaedb14e054be81baef417a736adb06bea34fb6708b28", "role": "Declares GlobalPipelineConfig and the step and standalone configuration dataclasses OpenHCS composes over ObjectState." } ], @@ -977,6 +990,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/configuration_framework.rst against the later B6 changes to external/ObjectState/src/objectstate/lazy_factory.py, openhcs/core/config.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/configuration_framework.rst in full and compared it against the changed authorities: lazy_factory.py, context_manager.py, openhcs/core/config.py; confirmed ObjectState still owns lazy configuration and context resolution, the linked system_overview and external_foundations pages exist, and no page edit was needed.", "Re-read docs/source/architecture/configuration_framework.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: external/ObjectState/src/objectstate/lazy_factory.py; external/ObjectState/src/objectstate/context_manager.py; openhcs/core/config.py.", @@ -1002,7 +1016,7 @@ }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", + "sha256": "90d75f6287c5f21f93d82960729199261f7906e0f4912b7728527260b9723a66", "role": "Consumes an already resolved GlobalPipelineConfig in a scoped compiler session." } ], @@ -1011,6 +1025,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/context_system.rst against the later B6 changes to openhcs/core/pipeline/compilation_session.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/context_system.rst in full and compared it against the changed authorities: context_manager.py, compilation_session.py; confirmed ObjectState owns context management and the compiler session consumes a resolved GlobalPipelineConfig; no page edit was needed.", "Read the complete page and challenged accuracy, freshness, duplication, boundary, flow, and navigation against the listed current authorities.", "python scripts/validate_docs.py docs/source/architecture passed, and .venv Sphinx -E -W --keep-going rendered all 152 active pages without warnings." @@ -1057,7 +1072,7 @@ "authority": [ { "path": "openhcs/processing/custom_functions/manager.py", - "sha256": "f87d622c4e03db3b6d7f977daa86187b892f59133be264ffa381472785247e56", + "sha256": "5cd7e955087f0bd9c527fc6307dff48f4aa959c607a3ec776b8379abac76fb36", "role": "Owns persisted custom-function lifecycle and projects validated declarations through the custom runtime registry." }, { @@ -1071,6 +1086,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/custom_function_registration_system.rst against the later B6 changes to openhcs/processing/custom_functions/manager.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/custom_function_registration_system.rst in full and compared it against the changed authority manager.py; confirmed the user_guide/custom_function_management target exists and CustomFunctionManager still owns persisted custom functions through the runtime registry; no page edit was needed.", "Re-read docs/source/architecture/custom_function_registration_system.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/processing/custom_functions/manager.py; openhcs/core/steps/function_step.py.", @@ -1105,7 +1121,7 @@ }, { "path": "openhcs/authoring/session/views.py", - "sha256": "8ce6cac27293ac9105f4d2ebc76d10eebeba1ef90e0e8954c0a117910e17e3b3", + "sha256": "2963a2c48856704ea8291075316c470ccc8eeed161b5a798181a519a2ceaef98", "role": "Declares the SessionView family, each view's title, and the operations its window binds." }, { @@ -1124,6 +1140,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/declarative_window_system.rst against the later B6 changes to openhcs/authoring/session/views.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/declarative_window_system.rst in full and compared it against the changed authorities: ui_bridge_identities.py, test_build_website.py; confirmed the pyqt-reactive window-manager and scope-window-factory pages and sources exist, traced UiSessionViewWidgetIdentityDeclaration to SessionView.rendered_in, and revised the OpenHCS-retained ownership sentence.", "Read the complete page and challenged accuracy, freshness, duplication, boundary, flow, and navigation against the listed current authorities.", "python scripts/validate_docs.py docs/source/architecture passed, and .venv Sphinx -E -W --keep-going rendered all 152 active pages without warnings.", @@ -1145,7 +1162,7 @@ "authority": [ { "path": "openhcs/core/function_patterns.py", - "sha256": "95b9056c6980e4f7943f909aeae8262252c9aa175247a534befc8d4cc459fdd2", + "sha256": "3dbdad68a4c7c765cc0dae056079929e5e4632f39bdbe2e746bea3acc171562d", "role": "Defines callable, sequential-chain, and dictionary-fan-out patterns and their grouped source keys." }, { @@ -1159,6 +1176,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/dict_pattern_case_study.rst against the later B6 changes to openhcs/core/function_patterns.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/dict_pattern_case_study.rst in full and compared it against the changed authorities: function_patterns.py, test_function_patterns.py; confirmed NormalizedFunctionPattern still carries grouped dictionary keys and the redirect target exists; no page edit was needed.", "Re-read docs/source/architecture/dict_pattern_case_study.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/function_patterns.py; tests/unit/test_function_patterns.py.", @@ -1178,7 +1196,7 @@ "authority": [ { "path": "external/ObjectState/src/objectstate/lazy_factory.py", - "sha256": "64ae924c176757863a84eef19b9df0eff516e32443c2661134ec2ae72f234bf9", + "sha256": "4f86ec64d975b354203539581943f36a75e34fbf2da076a3ef9b32036e54bff9", "role": "Implements dynamic lazy dataclass generation, validation, and resolution." }, { @@ -1192,6 +1210,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/dynamic_dataclass_factory.rst against the later B6 changes to external/ObjectState/src/objectstate/lazy_factory.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/dynamic_dataclass_factory.rst in full and compared it against the changed authority external/ObjectState/src/objectstate/lazy_factory.py; confirmed ObjectState still owns lazy dataclass generation and the linked pages exist; no page edit was needed.", "Read the complete page and challenged accuracy, freshness, duplication, boundary, flow, and navigation against the listed current authorities.", "python scripts/validate_docs.py docs/source/architecture passed, and .venv Sphinx -E -W --keep-going rendered all 152 active pages without warnings." @@ -1214,27 +1233,27 @@ }, { "path": "openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py", - "sha256": "2705d459204557c4f7092df3fd21ed687dcf56449ed361485382b1c0ce196444", + "sha256": "bd43ec00c265279585f6bcc9eb0a6b6a2eeedaacad6003a33a60ab181cd13a4d", "role": "Coordinates the workbook-declared format strategy, replicate-local normalization, exports, and the configuration-projected run_directory boundary." }, { "path": "openhcs/processing/backends/experimental_analysis/result_formats.py", - "sha256": "6951f41a2ff2bdac1945c7e56584a578c111c078cac1d684030c00f95a23f528", + "sha256": "b0790fa7050855c772f6817b3d538c86bbb161d3d0446020d808ada95351a3c0", "role": "Defines the scope-keyed CX5 and MetaXpress ExperimentalResultFormatStrategy leaves for result reading and plate population." }, { "path": "openhcs/processing/backends/experimental_analysis/analysis.py", - "sha256": "17a701e0740eac3e11319fe73f10302bbcab603218349bfba4e07014aabc27bb", + "sha256": "2fe9bcfba30e739cbf0b74112bfec6ce8714f173bdbd08b6dde1e3b6abe78d6e", "role": "Owns layout parsing, exclusion projection, replicate averaging, and table/plate-grid export." }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "e43085c1eaf0c836ce34e1a1dae7740488d91bdfea56844e7ddd4e6925561cf6", + "sha256": "00ff3bfe7a032bd3de59274697a69d12c9b898177cd8e839f9da3b130becaedf", "role": "Routes the desktop analysis action through ExperimentalAnalysisEngine.run_directory with the default configuration." }, { "path": "tests/unit/test_experimental_analysis_engine.py", - "sha256": "641fadecf8c940cc2316a06771c74db5f3f2db9dafdfbfebdc535760d6d776ec", + "sha256": "c79887f97615197c25c340dd277189ebc331bce1f3e9bfec7466c57c92ebd6cf", "role": "Verifies configured path projection, exclusions, normalization-method delegation, and an end-to-end MetaXpress directory run." } ], @@ -1244,6 +1263,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/experimental_analysis_system.rst against the later B6 changes to openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py, openhcs/processing/backends/experimental_analysis/result_formats.py, openhcs/processing/backends/experimental_analysis/analysis.py, openhcs/pyqt_gui/main.py, tests/unit/test_experimental_analysis_engine.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/experimental_analysis_system.rst in full and compared it against the changed authorities: core/config.py, unified_analysis_engine.py, the moved formats/experimental_result_formats.py, pyqt_gui/main.py, test_experimental_analysis_engine.py; traced ExperimentalAnalysisConfig and NormalizationMethod to domains/microscopy/config.py, the result-format strategies to result_formats.py and layout parsing to analysis.py, confirmed the desktop action calls ExperimentalAnalysisEngine().run_directory, and removed the stale deprecated-wrapper clause.", "Re-read docs/source/architecture/experimental_analysis_system.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/config.py; openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py; openhcs/formats/experimental_result_formats.py; openhcs/pyqt_gui/main.py; tests/unit/test_experimental_analysis_engine.py.", @@ -1271,7 +1291,7 @@ "authority": [ { "path": "pyproject.toml", - "sha256": "3592cc37444e2e1f5bb0284e939643039c0d513217861af8133dd8bd6e8a4325", + "sha256": "c3a2bea22aa6872940125f4aaac72c9eac72084fefc6c2cefdb54b5273361e4a", "role": "Pins the eight first-party package constraints and documents their installed integration surface." }, { @@ -1383,6 +1403,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/external_foundations.rst against the later B6 changes to pyproject.toml: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/external_foundations.rst in full and compared it against the changed authorities: pyproject.toml (zmqruntime 0.6, polystore 0.6, objectstate 1.3.1, python-introspect 0.2.5, metaclass-registry 0.3.0, arraybridge 0.4.1, pyqt-reactive 0.4 bounds; still eight first-party packages), the package READMEs, window_snapshot.py, client.py and arraybridge types.py; added the L1, L2, L3 and L5 ownership moves to the catalogue and confirmed the PolyStore, OMERO and pyqt-reactive claims against current source.", "Re-read docs/source/architecture/external_foundations.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: pyproject.toml; external/ObjectState/README.md; external/arraybridge/README.md; external/metaclass-registry/README.md; external/PolyStore/README.md; external/pyqt-reactive/README.md; external/python-introspect/README.md; external/zmqruntime/README.md; external/pycodify/README.md.", @@ -1432,7 +1453,7 @@ "authority": [ { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "3439d96f3fbfcb104a53019dc150049021d9a8132bd27096f1ccf510eefbe37b", + "sha256": "5d3646e17f7de843a5b36164f7cef7505da7841636423ae4fb4e00d194a87e9c", "role": "Implements CellProfiler lowering to FunctionStep and PipelineConfig declarations with declaration-owned headless validation." }, { @@ -1447,17 +1468,17 @@ }, { "path": "openhcs/interop/cellprofiler/module_declarations.py", - "sha256": "6bbf41c158496a02439be2ee5aef12d7712fb632419c226a64c9259a55438b95", + "sha256": "493efb9c749b10ba54713d0c16ee7118f07ff35afbafc9ff156b7100dbd63b19", "role": "Defines the nominal module validation hook queried by the importer." }, { "path": "openhcs/interop/cellprofiler/module_artifact_declarations.py", - "sha256": "f70b52114fb5d5e2c2fc49cd9a2b46986bde48ce02d82f1a34a67f072ca2ab20", + "sha256": "286638babca312c85ee0b8319fd44ba86be5868a3c1a20b74b1b48a0bfdbc652", "role": "Defines the interactive-module family that rejects headless import." }, { "path": "openhcs/core/dataset_sources/source.py", - "sha256": "097fde4d2b74031eacf53a88e79f499417e8c1c97ba0753e8a9df831b307efa6", + "sha256": "f736728d2f715b72cd5479fcc79520911b037d3410a42ebfc9febef2e51ffda6", "role": "Defines the registered DatasetSource family that microscope handlers implement." }, { @@ -1482,7 +1503,7 @@ }, { "path": "openhcs/runtime/zmq_server_hooks.py", - "sha256": "c3027f3495c5ec188a0f9176db7fd0fcbb22451892248c9fd41d73347ac236dd", + "sha256": "daf16cc68c8696f6c3680bd230c809be276ead00cfa84488954d328288f51f42", "role": "Maps generic targeted cancellation onto the exact OpenHCS orchestrator and its owned workers." }, { @@ -1497,6 +1518,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/external_integrations_overview.rst against the later B6 changes to openhcs/interop/cellprofiler/pipeline_import.py, openhcs/interop/cellprofiler/module_declarations.py, openhcs/interop/cellprofiler/module_artifact_declarations.py, openhcs/core/dataset_sources/source.py, openhcs/runtime/zmq_server_hooks.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/external_integrations_overview.rst in full and compared it against the changed authorities: pipeline_import.py, module_declarations.py, module_artifact_declarations.py, the deleted microscope_base.py, polystore/base.py, zmqruntime execution/server.py, zmq_server_hooks.py, test_cellprofiler_pipeline_import.py; replaced MicroscopeHandler with the DatasetSource family, named the PipelineImporter registration, and confirmed the CellProfiler, OMERO and cancellation claims and every linked page.", "Read the complete page and challenged accuracy, freshness, duplication, boundary, flow, and navigation against the listed current authorities.", "python scripts/validate_docs.py docs/source/architecture passed, and .venv Sphinx -E -W --keep-going rendered all 152 active pages without warnings.", @@ -1623,7 +1645,7 @@ "authority": [ { "path": "openhcs/core/function_patterns.py", - "sha256": "95b9056c6980e4f7943f909aeae8262252c9aa175247a534befc8d4cc459fdd2", + "sha256": "3dbdad68a4c7c765cc0dae056079929e5e4632f39bdbe2e746bea3acc171562d", "role": "Normalizes callable, (callable, kwargs), list-chain, and dict patterns into named invocations and compiles them with their memory and artifact plans." }, { @@ -1637,6 +1659,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/function_pattern_system.rst against the later B6 changes to openhcs/core/function_patterns.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/function_pattern_system.rst in full and compared it against the changed authorities: openhcs/core/function_patterns.py and tests/unit/test_function_patterns.py; confirmed the module still accepts callable, (callable, kwargs), list and dict patterns and that docs/source/concepts/function_patterns.rst exists; the page text needed no change.", "Re-read docs/source/architecture/function_pattern_system.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/function_patterns.py; tests/unit/test_function_patterns.py.", @@ -1666,12 +1689,12 @@ }, { "path": "openhcs/core/callable_contract.py", - "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", + "sha256": "2060f856bcb0cc8ef57b92fc17a1e558a676e4033075950b1a4e41dc8656d330", "role": "Adds OpenHCS processing, runtime, artifact, scope, and ArrayBridge memory semantics on top of python-introspect annotation helpers." }, { "path": "tests/unit/test_callable_contract.py", - "sha256": "cc220f53107a08b0406b6e0dd49896e3e4c3c0af6e527b0ef00d4fdaba8f0b2c", + "sha256": "b0ce0cba2e396e53f8cb9e95533ff918788e8c6dd60037a66e0aeda603bc64c2", "role": "Verifies OpenHCS callable contracts projected from declaration-owned metadata, including execution memory and runtime batching." } ], @@ -1680,6 +1703,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/function_reference_pattern.rst against the later B6 changes to openhcs/core/callable_contract.py, tests/unit/test_callable_contract.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/function_reference_pattern.rst in full and compared it against the changed authorities: external/python-introspect/src/python_introspect/signature_analyzer.py, openhcs/core/callable_contract.py and tests/unit/test_callable_contract.py; grepped OpenHCS imports of python_introspect and confirmed docs/source/architecture/processing_semantics.rst exists; the page text needed no change.", "Re-read docs/source/architecture/function_reference_pattern.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: external/python-introspect/src/python_introspect/signature_analyzer.py; openhcs/core/callable_contract.py; tests/unit/test_callable_contract.py.", @@ -1748,7 +1772,7 @@ }, { "path": "openhcs/core/callable_contract.py", - "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", + "sha256": "2060f856bcb0cc8ef57b92fc17a1e558a676e4033075950b1a4e41dc8656d330", "role": "Projects declaration-owned input, output, and execution memory into immutable callable contracts." }, { @@ -1763,17 +1787,17 @@ }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "sha256": "d1216010dec1804c51aec9c774f298513638ca016d500fe6328d76a9a9bfb4d6", "role": "Calls assign_framework_devices on each context's step plans before bundle assembly." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", + "sha256": "f4a851d7de2ea82e1d548f37e6be2511fd20d799580193496a3746e5484906ba", "role": "Owns FrameworkDeviceAssignment bindings, merged cleanup footprints, and the requires_gpu projection." }, { "path": "openhcs/core/orchestrator/worker_execution.py", - "sha256": "23a3ef2a70378dd1bb5c413a56e36bbb3fa013142741c9b45407bfc47de8c4a4", + "sha256": "1d71883f545f7f8702c64710f968b4138d0748fa0c58b6e1b87e06a7045e5ebf", "role": "Releases each axis's compiled footprint after process and inline contexts and the merged footprint only after thread-backed lanes join." }, { @@ -1808,6 +1832,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/gpu_resource_management.rst against the later B6 changes to openhcs/core/callable_contract.py, openhcs/core/pipeline/compiler.py, openhcs/core/compiled_step_plan.py, openhcs/core/orchestrator/worker_execution.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/gpu_resource_management.rst in full and compared it against the changed authorities: arraybridge decorators.py and types.py, openhcs/core/callable_contract.py, openhcs/core/pipeline/framework_device_assignment.py, openhcs/core/pipeline/compiler.py, openhcs/core/compiled_step_plan.py, openhcs/core/orchestrator/worker_execution.py and the worker-start and execution-scope tests; traced assign_framework_devices from the compiler, FrameworkDeviceResolver's gpu_imports_disabled check, MemoryType.subprocess_environment() callers, _release_runtime_resources per axis and after threaded lanes, and resolve_worker_start_context; replaced one jargon phrase and refreshed evidence roles.", "Re-read docs/source/architecture/gpu_resource_management.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: external/arraybridge/src/arraybridge/decorators.py; external/arraybridge/src/arraybridge/types.py; openhcs/core/callable_contract.py; openhcs/core/framework_device_resolver.py; openhcs/core/pipeline/framework_device_assignment.py; openhcs/core/pipeline/compiler.py; openhcs/core/compiled_step_plan.py; openhcs/core/orchestrator/worker_execution.py; openhcs/core/worker_start_policy.py; tests/unit/test_framework_device_resolver.py; tests/unit/test_framework_execution_scope.py; tests/unit/test_worker_start_policy.py.", @@ -1871,22 +1896,22 @@ }, { "path": "openhcs/core/config.py", - "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "sha256": "7b5622b3d82b1e1d672eaedb14e054be81baef417a736adb06bea34fb6708b28", "role": "Declares GlobalPipelineConfig and the step variable_components and group_by fields, with group_by defaulting from the active AxisFamily." }, { "path": "openhcs/core/processing_contracts.py", - "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", + "sha256": "9b999e9210baf292f8fbf5bb3758c41a95e74401bfb1e1c127bae2dba625b0ee", "role": "Defines the ProcessingContract family named as an independent declaration in the invariants." }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", + "sha256": "90d75f6287c5f21f93d82960729199261f7906e0f4912b7728527260b9723a66", "role": "Defines CompilationSession, the scoped compiler state named by the invariants." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", + "sha256": "f4a851d7de2ea82e1d548f37e6be2511fd20d799580193496a3746e5484906ba", "role": "Defines typed CompiledStepPlan fields, including framework footprints and device bindings." }, { @@ -1898,6 +1923,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/index.rst against the later B6 changes to openhcs/core/config.py, openhcs/core/processing_contracts.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/compiled_step_plan.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/index.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compilation_session.py, openhcs/core/compiled_step_plan.py and openhcs/core/compiled_execution.py; verified each architecture invariant against the current declarations and that every toctree target file exists; the page text needed no change.", "Re-read docs/source/architecture/index.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/steps/function_step.py; openhcs/core/pipeline/compilation_session.py; openhcs/core/compiled_step_plan.py; openhcs/core/compiled_execution.py.", @@ -1927,7 +1953,7 @@ }, { "path": "openhcs/core/config.py", - "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "sha256": "7b5622b3d82b1e1d672eaedb14e054be81baef417a736adb06bea34fb6708b28", "role": "Declares PartitionFilterConfig's show_preview_without_partition_filter policy and PartitionFilterMode members' preview_prefix operators." }, { @@ -1937,7 +1963,7 @@ }, { "path": "openhcs/pyqt_gui/services/reactor_providers.py", - "sha256": "aecf1c77636095408486cab53a95cb0fc044fcf2ead0d7b72ac674842cf69abe", + "sha256": "43634ee5d4ac6c65d681288c575ee0c9570140d592cb3856942ec69f47c59d90", "role": "Registers format_partition_filter_config for PartitionFilterConfig with the generic preview service." }, { @@ -1947,12 +1973,12 @@ }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "e43085c1eaf0c836ce34e1a1dae7740488d91bdfea56844e7ddd4e6925561cf6", + "sha256": "00ff3bfe7a032bd3de59274697a69d12c9b898177cd8e839f9da3b130becaedf", "role": "_apply_list_preview_config applies list_previews to Plate Manager and Pipeline Editor at startup and on UI config change." }, { "path": "tests/unit/pyqt_gui/test_main_config_propagation.py", - "sha256": "303d90b43bee7bd531a1c7664fbf9707a2e7be1696a5147614f93c40e5c7021f", + "sha256": "499b05f1249ffc522bf18e97de05d1bf2000ab159daa0685958bd37d9dc62cc4", "role": "Verifies the list preview config reaches both real lists." }, { @@ -1992,6 +2018,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/list_item_preview_system.rst against the later B6 changes to openhcs/core/config.py, openhcs/pyqt_gui/services/reactor_providers.py, openhcs/pyqt_gui/main.py, tests/unit/pyqt_gui/test_main_config_propagation.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/list_item_preview_system.rst in full and compared it against the changed authorities: openhcs/core/config.py, openhcs/pyqt_gui/widgets/config_preview_formatters.py, openhcs/pyqt_gui/services/reactor_providers.py, openhcs/pyqt_gui/main.py and tests/unit/pyqt_gui/test_main_config_propagation.py; confirmed PartitionFilterConfig/PartitionFilterMode preview declarations, the single formatter registration, UIConfig.list_previews, _apply_list_preview_config for both managers, and TogglePreviewAction; corrected one stale well-selection phrase.", "Read the complete page and challenged accuracy, freshness, duplication, boundary, flow, and navigation against the listed current authorities.", "python scripts/validate_docs.py docs/source/architecture passed, and .venv Sphinx -E -W --keep-going rendered all 152 active pages without warnings.", @@ -2016,17 +2043,17 @@ "authority": [ { "path": "openhcs/agent/capabilities.py", - "sha256": "eaebe33af01d057d1d21b3701f958bc96ede3ec5d0bddddca9917c1164ef0042", + "sha256": "87c8e2e5aeb425b9a2eb690d48bdd6f5af7a327c389deac261d7e4e0da5d609f", "role": "Declares AgentCapabilityDeclaration fields (service, side effects, security, data exposure, transport availability, progress heartbeat and worker-thread safety), CapabilityTransport semantics, local surface profiles, and one tool per headless session operation plus openhcs_session_events." }, { "path": "openhcs/mcp/server.py", - "sha256": "e1f236011df715da3d17574c0b80cbf50fcae45e3f82491c22fb57e98c6bb662", + "sha256": "927e64594a0fc174759eeb58496fed0ccf95e5d8d29bc248cde9a033efb69fbc", "role": "Binds the selected declaration registry into FastMCP with outputContract metadata and structured output, runs non-worker-safe capabilities through the main-thread dispatcher, and binds the session to the main thread." }, { "path": "openhcs/mcp/execution.py", - "sha256": "2fcf0db56f5bd613844c8361962c04715aaa9bfb1b0f1ab191543a99522a06ad", + "sha256": "e1e3b4c649e8b62e873c10b92f9e27eedb5196e8e4890700929e40ab406c2170", "role": "Runs the MCP SDK loop on a worker thread while a Qt core application and McpMainThreadDispatcher own main-thread calls." }, { @@ -2046,12 +2073,12 @@ }, { "path": "openhcs/mcp/dev_client_core.py", - "sha256": "41739b744b0e4287ca7d14cd133371d669e9bff12f4b622313996ae9bb5597da", + "sha256": "b5b1aea8a5a9352d828efd408408cb0ce867db4c5dc4f91e1fa9fbb7947013b2", "role": "Resolves tool identity and compatible profiles from capability declarations and renews the command deadline on each response or progress notification." }, { "path": "openhcs/mcp/dev_client_session.py", - "sha256": "2dceca7937087d05f72af1d037122ea393721d61d690803628a7b14ca70fd980", + "sha256": "0486edcefae169b51ee2baf4b6498d93411f8dd93d94bcfdfbf46ee21896c73d", "role": "Implements execute-source as add-datasets, set-pipeline, then initialize, compile and run stages followed through session events." }, { @@ -2061,12 +2088,12 @@ }, { "path": "openhcs/mcp/context.py", - "sha256": "1292c9bb7173e6afe5aa6237b861c5f39bc2e854f364fd722c75a3c6b76f534e", + "sha256": "af9ec43138e05762fc97b34825b96e15f183aa56781d697c601eb79f14a1bc42", "role": "Composes local stdio authoring and the headless session with the saved desktop execution endpoint while keeping hosted HTTP self-contained." }, { "path": "openhcs/authoring/session/session.py", - "sha256": "587047b63972df89ec46a01ed2c92ff3ef9577335f58a980297db02094e4e8e0", + "sha256": "458d71fe96ab75153977c554fb748e78f1773cd7794b625d7ec233c9b22e4453", "role": "Defines the headless Session whose datasets and pipelines live in the MCP process's ObjectState and whose events clients follow." }, { @@ -2126,7 +2153,7 @@ }, { "path": "openhcs/pyqt_gui/services/ui_bridge_windows.py", - "sha256": "e76298f78e3cbd949e8426a2b42cd6768cc54770a5f45c9dd6c9f788b257785e", + "sha256": "7a10b37aba9c5ea5cdbe46a4c9703b345e83a64554d6abacbfa1ee41cd3c4020", "role": "Resolves authorised OpenHCS windows, projects snapshot results, and distinguishes focus from accepted target navigation." }, { @@ -2171,12 +2198,12 @@ }, { "path": "benchmark/control_service.py", - "sha256": "a1256b349ffdc8d04cab927326d97821462b61f57b29e564b61cc627302be49c", + "sha256": "f894cc090597ca04c273090c99f26fd0e3cfeffd1b7d44bbb94858892e17813c", "role": "Finalises measured-run evidence from the session's finished execution." }, { "path": "benchmark/cellprofiler_benchmark_cli.py", - "sha256": "4025c591644e29f3df61e476ecae2143690e014336ea9f889aadeff4e692445e", + "sha256": "edc32916215a88b62702621dea9b692586572284caf9b781394b5784f62897ee", "role": "Owns the installed openhcs-benchmark execution, report and list-cases command surface." } ], @@ -2191,6 +2218,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/mcp_distribution.rst against the later B6 changes to openhcs/agent/capabilities.py, openhcs/mcp/server.py, openhcs/mcp/execution.py, openhcs/mcp/dev_client_core.py, openhcs/mcp/dev_client_session.py, openhcs/mcp/context.py, openhcs/authoring/session/session.py, openhcs/pyqt_gui/services/ui_bridge_windows.py, benchmark/control_service.py, benchmark/cellprofiler_benchmark_cli.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/mcp_distribution.rst in full and compared it against the changed authorities: openhcs/agent/capabilities.py, openhcs/mcp/server.py, stdio.py, context.py, http.py, client_registration.py, dev_client*.py, the deleted openhcs/desktop_deployment.py and desktop_installation.py (moved to openhcs.desktop in 73a2aece2), ui_bridge_windows.py, synthetic-plate and benchmark control owners and tests; verified every named symbol by grep, traced McpTransportExecutor, the binder's dispatcher path, SessionJourney.run_source and skill synchronisation; corrected the PyQt/threading, session-execution, desktop-deployment, client-registration and finalisation passages.", "Read the complete page and challenged accuracy, freshness, duplication, boundary, flow, and navigation against the listed current authorities.", "python scripts/validate_docs.py docs/source/architecture passed, and .venv Sphinx -E -W --keep-going rendered all 152 active pages without warnings.", @@ -2259,12 +2287,12 @@ }, { "path": "openhcs/core/equivalence/keys.py", - "sha256": "0ae79e4cc4e70804f9d2e948078ade7c271e142473821731b38eeccaef064b5d", + "sha256": "1f409e1a834afd15f8bff206ebaefc8ef904c1ac7a93d43222d19e1230b7a4cb", "role": "Defines RuntimeMeasurementFeatureKey and the semantic row, source, and aggregate identities." }, { "path": "openhcs/core/equivalence/measurement_features.py", - "sha256": "18ff75cdd4dad0f88870b90b9a91f2ea352372773bcc9a74cee288d4c110f05b", + "sha256": "2899f25b73c04a6f845a68d5e174088388978c4b21cb1fd8127ff7f100a1a0bd", "role": "Defines the RuntimeMeasurementFeatureSemanticProfile most-derived strategy family." }, { @@ -2274,7 +2302,7 @@ }, { "path": "openhcs/core/measurement_dialect.py", - "sha256": "96cac83e09e22eeaa02dbd14da2f9553c1f19f51962e83731936ba33429c4578", + "sha256": "ab20f9e1edd45021e6881497b342489e312f18bef9b7b749ea4359b4cb811375", "role": "Defines the registered MeasurementDialect family and PlainMeasurementDialect." }, { @@ -2289,12 +2317,12 @@ }, { "path": "benchmark/adapters/openhcs.py", - "sha256": "8f84d9b70037c59558c2534b1ef49dd888c0f1d77d75eae44646c58173fb001d", + "sha256": "8621fb5adcab09e7e376d2c5f4eddafc5f485a9328669985f7a36b0646b3a561", "role": "Builds benchmark runtime snapshots, the strict CellProfiler policy at 1e-6 tolerances, and digest-bound reference-export comparisons." }, { "path": "benchmark/openhcs_measured_run.py", - "sha256": "b7b2a3fd7b795db416005ad8b0b1661a960df0792c5d47e70de286d4cd08f9af", + "sha256": "58916c818b713cc6388a6941ce33cbbdf7160837792475ce76292f16bbe442b6", "role": "Wraps the ordinary compile-and-execute submission with observation validation, phase timing from completed server records, endpoint provenance, and the completed-run receipt." }, { @@ -2309,7 +2337,7 @@ }, { "path": "benchmark/control_service.py", - "sha256": "a1256b349ffdc8d04cab927326d97821462b61f57b29e564b61cc627302be49c", + "sha256": "f894cc090597ca04c273090c99f26fd0e3cfeffd1b7d44bbb94858892e17813c", "role": "Finalises benchmark evidence from the OpenHCS session's finished execution: its request, endpoint handshake, and completion record." }, { @@ -2334,7 +2362,7 @@ }, { "path": "tests/integration/test_cellprofiler_official30_zmq.py", - "sha256": "dbd6e113a7135901f936e9bc8278d727d678e9971a7c951fced9080adfd9d6df", + "sha256": "3297b0e54ad3a349d35b613bd7d406e92c974320e12723be80dcfc0a90be2116", "role": "Implements the headless and non-persistent Napari acceptance routes over all manifest cases, plus Fiji variant tests." }, { @@ -2363,6 +2391,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/measurement_equivalence_system.rst against the later B6 changes to openhcs/core/equivalence/keys.py, openhcs/core/equivalence/measurement_features.py, openhcs/core/measurement_dialect.py, benchmark/adapters/openhcs.py, benchmark/openhcs_measured_run.py, benchmark/control_service.py, tests/integration/test_cellprofiler_official30_zmq.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/measurement_equivalence_system.rst in full and compared it against the changed authorities: openhcs/core/equivalence/policy.py and keys.py, the deleted openhcs/core/equivalence/report.py (moved to benchmark/equivalence in f16177ed7), benchmark/adapters/openhcs.py, openhcs_measured_run.py, timing.py, control_service.py, openhcs/runtime/zmq_execution_observation.py, measured_run_receipt.py, the Official30 manifest and integration test, reference exports and their tests; corrected the dialect name, the comparison-package boundary, archived-export readability, the session-based finalisation path and the additive-total rule.", "Read the complete page and challenged accuracy, freshness, duplication, boundary, flow, and navigation against the listed current authorities.", "python scripts/validate_docs.py docs/source/architecture passed, and .venv Sphinx -E -W --keep-going rendered all 152 active pages without warnings.", @@ -2416,12 +2445,12 @@ }, { "path": "openhcs/core/callable_contract.py", - "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", + "sha256": "2060f856bcb0cc8ef57b92fc17a1e558a676e4033075950b1a4e41dc8656d330", "role": "Reads ArrayBridge metadata through MemoryContractAttribute into immutable callable input, output, and execution memory contracts." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", + "sha256": "f4a851d7de2ea82e1d548f37e6be2511fd20d799580193496a3746e5484906ba", "role": "Owns FrameworkDeviceAssignment framework-local device bindings and footprint cleanup." }, { @@ -2435,6 +2464,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/memory_type_system.rst against the later B6 changes to openhcs/core/callable_contract.py, openhcs/core/compiled_step_plan.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/memory_type_system.rst in full and compared it against the changed authorities: external/arraybridge/src/arraybridge/decorators.py, types.py, openhcs/core/callable_contract.py and openhcs/core/compiled_step_plan.py; confirmed MemoryContractAttribute's INPUT/OUTPUT/EXECUTION keys, FrameworkDeviceAssignment, openhcs/core/runtime_adapters.py and the gpu_resource_management and external_foundations targets; the page text needed no change.", "Re-read docs/source/architecture/memory_type_system.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: external/arraybridge/src/arraybridge/decorators.py; external/arraybridge/src/arraybridge/types.py; external/arraybridge/src/arraybridge/converters.py; openhcs/core/callable_contract.py; openhcs/core/compiled_step_plan.py; openhcs/core/framework_device_resolver.py.", @@ -2461,12 +2491,12 @@ "authority": [ { "path": "openhcs/core/dataset_sources/source.py", - "sha256": "097fde4d2b74031eacf53a88e79f499417e8c1c97ba0753e8a9df831b307efa6", + "sha256": "f736728d2f715b72cd5479fcc79520911b037d3410a42ebfc9febef2e51ffda6", "role": "Defines the DatasetSource family keyed by source_name, its selection-role mixins, detection_order, create hook, workspace backend registration, and virtual-workspace metadata recording." }, { "path": "openhcs/core/dataset_sources/choice.py", - "sha256": "7fb3c7ce701f0d629f288b79090044f43ac8ce1b51ca4f50937534407f226f1a", + "sha256": "ece43af801b4b02c302d9ac4bf6e5d07c0d9ebee5027433d12a4b548a19d11f0", "role": "Defines DatasetSourceChoice, AutoDetectedSource, and open(), which selects the source class, applies declared-file selection, calls create, and binds dataset_root." }, { @@ -2476,32 +2506,32 @@ }, { "path": "openhcs/core/dataset_sources/interfaces.py", - "sha256": "dd88254bfd5da8b0fdf0a038e358f36b49db5b8cf0b93213d56dc928af965f00", + "sha256": "31e3f5789c4a1603de12a0f0f95ceb529ff3190d45f2315e2674ef81a9177047", "role": "Defines MetadataHandler.component_value_set and MetadataComponentValueSet keyed by the active family's axes." }, { "path": "openhcs/core/dataset_sources/openhcs_format.py", - "sha256": "a8fa6a597104075d888a5b14cc4b24a9676d1a3918c30b5df2bad4a9abb26078", + "sha256": "f4976001de8dcfd598fd08ebe8cb0e2c8a412e03956210d81a392d327a11ce3e", "role": "Reopens a persisted workspace by looking up the recorded source_name and re-registering that source's workspace backends." }, { "path": "openhcs/core/config.py", - "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "sha256": "7b5622b3d82b1e1d672eaedb14e054be81baef417a736adb06bea34fb6708b28", "role": "Declares PipelineConfig.dataset_source as a DatasetSourceChoice class defaulting to AutoDetectedSource." }, { "path": "openhcs/microscopes/bioformats.py", - "sha256": "bda05411746852ea12ffe11b49e5298cb475e2137cd1a09cf9659b52a89f40e4", + "sha256": "478cac79bb2512a16f1419b6f7c899d06e6e4c0d1046bd13ac031fbc9cd21470", "role": "Shows a decoder-owned source metadata handler replaced by the workspace metadata handler after the workspace is written." }, { "path": "tests/unit/test_dataset_source_family.py", - "sha256": "4cb651b475e5e033ab94824567b8a35b0ee02f313254082e03e35cd96585d829", + "sha256": "e9e4decb541543619d6f6abc1dceffcfec590e916045b7a1396baf51c9bea3be", "role": "Verifies every registered source declares identity, selection role and backends, and that metadata detection and the factory use declared identity." }, { "path": "tests/unit/test_dataset_source_guards.py", - "sha256": "adfeac23f6196a0c5393dc15b71df04fda0ec34e5ae23bfdaa22e0a165f68170", + "sha256": "89b8cfc1492c909477e8f102d8e82baf5bb6f2d324a2866e319f90d7c97ce214", "role": "Guards that the kernel imports no domain package and that the deleted microscope-handler names do not return." } ], @@ -2513,6 +2543,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/microscope_handler_integration.rst against the later B6 changes to openhcs/core/dataset_sources/source.py, openhcs/core/dataset_sources/choice.py, openhcs/core/dataset_sources/interfaces.py, openhcs/core/dataset_sources/openhcs_format.py, openhcs/core/config.py, openhcs/microscopes/bioformats.py, tests/unit/test_dataset_source_family.py, tests/unit/test_dataset_source_guards.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/microscope_handler_integration.rst in full and compared it against the changed authorities: the deleted openhcs/microscopes/microscope_base.py, openhcs.py and microscope_interfaces.py (moved to openhcs/core/dataset_sources in c6d9a52ce), openhcs/core/source_bindings.py and the Bio-Formats and plate-inspection tests; traced DatasetSource registration and discovery, DatasetSourceChoice.open, detection_order, PipelineConfig.dataset_source, component_value_set, and workspace replay through the recorded source name; rewrote the page in DatasetSource vocabulary.", "Re-read docs/source/architecture/microscope_handler_integration.rst in full against repository base 32ca17c17 and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/microscopes/microscope_base.py; openhcs/core/source_bindings.py; tests/unit/test_microscope_handler_identity.py; tests/unit/test_bioformats_microscope_handler.py.", @@ -2537,12 +2568,12 @@ "authority": [ { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "sha256": "d1216010dec1804c51aec9c774f298513638ca016d500fe6328d76a9a9bfb4d6", "role": "Discovers partition-axis values through the active AxisFamily and compiles one frozen ProcessingContext per value." }, { "path": "openhcs/core/orchestrator/bundle_execution.py", - "sha256": "f5ba09a2ec7c688b1ca08167ab398cc976a9e2c39b301ae98f3c5fd560a25c85", + "sha256": "7665bd4ab8b9ce73d3dcb4fb9679e836b23bffb81d70541baef2e114ee54b0d9", "role": "Validates compiled contexts and worker-safe steps before worker setup and builds the worker-lane execution plan." }, { @@ -2552,7 +2583,7 @@ }, { "path": "openhcs/core/orchestrator/worker_execution.py", - "sha256": "23a3ef2a70378dd1bb5c413a56e36bbb3fa013142741c9b45407bfc47de8c4a4", + "sha256": "1d71883f545f7f8702c64710f968b4138d0748fa0c58b6e1b87e06a7045e5ebf", "role": "Submits one task per active lane, collects per-axis ExecutionResult values, ships or inherits the bundle, and terminates owned process workers on cancellation." }, { @@ -2579,6 +2610,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/multiprocessing_coordination_system.rst against the later B6 changes to openhcs/core/pipeline/compiler.py, openhcs/core/orchestrator/bundle_execution.py, openhcs/core/orchestrator/worker_execution.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/multiprocessing_coordination_system.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compiler.py, openhcs/core/compiled_execution.py and openhcs/core/orchestrator/worker_execution.py; traced _axis_values_to_process through AxisFamily.active().partition_axis(), validate_bundle_execution, CompiledContextLanePlanner, PooledWorkerLaneRunner._submit_lanes, ExecuteShippedLane and ProcessWorkerExecutorResources.cancel_execution; corrected the task granularity, axis vocabulary, bundle installation and cancellation wording.", "Re-read docs/source/architecture/multiprocessing_coordination_system.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/pipeline/compiler.py; openhcs/core/compiled_execution.py; openhcs/core/orchestrator/worker_execution.py; openhcs/core/orchestrator/cancellation.py; openhcs/core/worker_start_policy.py.", diff --git a/docs/audits/architecture_n_z_development.json b/docs/audits/architecture_n_z_development.json index 2470b38114..3d22254c16 100644 --- a/docs/audits/architecture_n_z_development.json +++ b/docs/audits/architecture_n_z_development.json @@ -48,7 +48,7 @@ "authority": [ { "path": "openhcs/core/viewer_streaming_service.py", - "sha256": "c96eca530f767002cfcc9aca4fe75c93d7a5848a7f42df9e5d336bf6712ed51e", + "sha256": "b85d6d9b0677bdbd99a657672e19e3a27ef418bfcddfb5550524a83e7598ec4b", "role": "Compiled output streaming parametrized by viewer type, viewer settlement, and lifecycle cleanup." }, { @@ -67,6 +67,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/napari_streaming_system.rst against the later B6 changes to openhcs/core/viewer_streaming_service.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/napari_streaming_system.rst in full and compared it against the changed authorities: openhcs/core/viewer_streaming_service.py and openhcs/runtime/napari_stream_visualizer.py; confirmed both redirect targets exist and the modules remain the OpenHCS streaming service and Napari adapter; the page text needed no change.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", "tests/unit/test_streaming_service.py and tests/unit/test_napari_streaming_handlers.py verify the current route", @@ -100,17 +101,17 @@ }, { "path": "openhcs/core/processing_contracts.py", - "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", + "sha256": "9b999e9210baf292f8fbf5bb3758c41a95e74401bfb1e1c127bae2dba625b0ee", "role": "ProcessingContract subclass family owning per-callable execution over the plane axis." }, { "path": "openhcs/processing/backends/lib_registry/openhcs_registry.py", - "sha256": "3fff5c3549ef8e1073d75e6d5544b2cd75fbb7cc9b17263207760ffca45cf885", + "sha256": "7872cf8b59a954472871a72614d67bb2658cc2f4ce67e08dba1f8088589ed760", "role": "Native callable catalogue derived from each callable's nominal catalogue declaration and the public module surface." }, { "path": "openhcs/interop/cellprofiler/module_declarations.py", - "sha256": "6bbf41c158496a02439be2ee5aef12d7712fb632419c226a64c9259a55438b95", + "sha256": "493efb9c749b10ba54713d0c16ee7118f07ff35afbafc9ff156b7100dbd63b19", "role": "CellProfilerModule declarations owning callable names, catalogue placement, and headless-import eligibility." }, { @@ -125,6 +126,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/nominal_ownership.rst against the later B6 changes to openhcs/core/processing_contracts.py, openhcs/processing/backends/lib_registry/openhcs_registry.py, openhcs/interop/cellprofiler/module_declarations.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/nominal_ownership.rst in full and compared it against the changed authorities: deleted openhcs/core/registry_strategies.py (now metaclass_registry/strategies.py), metaclass_registry/core.py, tests/unit/test_registry_owner_iteration.py, lib_registry/openhcs_registry.py, interop/cellprofiler/module_declarations.py; verified ArtifactType, ProcessingContract, CellProfilerModule, the three mixins and the equivalence MostDerivedContextStrategyMixin users exist; replaced the deleted RuntimeSliceProjectionStrategy example and named the mixins' new home.", "Re-read docs/source/architecture/nominal_ownership.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/registry_strategies.py; external/metaclass-registry/src/metaclass_registry/core.py; tests/unit/test_registry_owner_iteration.py; openhcs/processing/backends/lib_registry/openhcs_registry.py; openhcs/interop/cellprofiler/module_declarations.py.", @@ -146,7 +148,7 @@ "authority": [ { "path": "external/PolyStore/src/polystore/omero_local.py", - "sha256": "35d39564317510234c2a3ded00bb06a09417d1b6f7359e52e74c7bd0a086f642", + "sha256": "e549d4811539cec9842ceac007eb666380ef00d6838a9b69d91eb02de64d1daa", "role": "Generic OMERO virtual-backend addressing, listing, pixel access, and declaration-owned output dispatch." }, { @@ -197,6 +199,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/omero_backend_system.rst against the later B6 changes to external/PolyStore/src/polystore/omero_local.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "Re-read docs/source/architecture/omero_backend_system.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: external/PolyStore/src/polystore/omero_local.py; external/PolyStore/src/polystore/omero_tables.py; external/PolyStore/docs/source/guides/omero_backend.rst; openhcs/omero/docker-compose.yml; openhcs/runtime/omero_instance_manager.py; openhcs/omero/plugin/setup.py; openhcs/omero/plugin/omero_openhcs/apps.py.", "Focused feature-lattice QA passed 159 tests covering the N-Z architecture and developer workflow authorities.", @@ -222,12 +225,12 @@ "authority": [ { "path": "openhcs/core/orchestrator/bundle_execution.py", - "sha256": "f5ba09a2ec7c688b1ca08167ab398cc976a9e2c39b301ae98f3c5fd560a25c85", + "sha256": "7665bd4ab8b9ce73d3dcb4fb9679e836b23bffb81d70541baef2e114ee54b0d9", "role": "Parent compiled-bundle execution: executor shutdown, BrokenProcessPool handling, parent resource release, reverse-order viewer bootstrap rollback, and viewer settlement ordering." }, { "path": "openhcs/core/orchestrator/worker_execution.py", - "sha256": "23a3ef2a70378dd1bb5c413a56e36bbb3fa013142741c9b45407bfc47de8c4a4", + "sha256": "1d71883f545f7f8702c64710f968b4138d0748fa0c58b6e1b87e06a7045e5ebf", "role": "Inline, process and thread lane resources: per-axis image-cache release, process-local runtime/framework cleanup, and thread-lane deferral to the parent." }, { @@ -237,12 +240,12 @@ }, { "path": "openhcs/runtime/viewer_protocol.py", - "sha256": "c4bdae4bae6dee412f0d27a9cf75a95db53b3bd95c03cee7d046be16b52fe5b8", + "sha256": "f126aaf4f5c0dcc9429b7aae8ce67985260241ec7d2f3d5a948573658cf079a3", "role": "rollback_failed_bootstrap distinguishing connected external viewers from OpenHCS-owned viewer processes." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", + "sha256": "f4a851d7de2ea82e1d548f37e6be2511fd20d799580193496a3746e5484906ba", "role": "Framework/device assignment merge and cleanup_loaded used for the compiled framework/device footprint." }, { @@ -257,7 +260,7 @@ }, { "path": "tests/unit/test_orchestrator_lane_planning.py", - "sha256": "3d5c2f3662b8d4c330582af0c4a649b8f4d9582e064d9cf3159798c8934a2bbb", + "sha256": "4a83337fa35c68e4f641760ebbf86e4cc30f6624da55ccebd09be636a4b56bd0", "role": "Proves partial viewer acquisition and readiness failures roll back every acquired viewer." }, { @@ -271,6 +274,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/orchestrator_cleanup_guarantees.rst against the later B6 changes to openhcs/core/orchestrator/bundle_execution.py, openhcs/core/orchestrator/worker_execution.py, openhcs/runtime/viewer_protocol.py, openhcs/core/compiled_step_plan.py, tests/unit/test_orchestrator_lane_planning.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/orchestrator_cleanup_guarantees.rst in full and compared it against the changed authorities: compiled_plate_execution.py (renamed to bundle_execution.py, 4cd901c9f), worker_execution.py, orchestrator.py, viewer_protocol.py, arraybridge types.py, test_orchestrator_lane_planning.py, processing_context.py, test_runtime_stack_cache_lifetime.py; checked executor shutdown and BrokenProcessPool handling, reverse-order viewer rollback with external-client handling, thread-lane deferral of runtime release, per-axis image-cache release and cancellation-scope finish; renamed the boundary in the opening sentence. Dropped orchestrator.py and test_framework_device_resolver.py as authorities; added compiled_step_plan.py for the device-footprint cleanup.", "Re-read docs/source/architecture/orchestrator_cleanup_guarantees.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/orchestrator/compiled_plate_execution.py; openhcs/core/orchestrator/worker_execution.py; openhcs/core/orchestrator/orchestrator.py; openhcs/core/orchestrator/cancellation.py; external/arraybridge/src/arraybridge/types.py; tests/unit/test_framework_device_resolver.py.", @@ -296,17 +300,17 @@ "authority": [ { "path": "openhcs/core/orchestrator/orchestrator.py", - "sha256": "a61658b107f170c192eea505966c268ab6826826f03586bd64562a44800e1ccc", + "sha256": "4e55309de74e770b2e37b53f6dbedc19ffac6988ab74cdfe4b3dfc51082e8c1a", "role": "PipelineOrchestrator: __objectstate_delegate__ to pipeline_config, dataset_root identity, and initialize enforcing the source role's availability check." }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "sha256": "d1216010dec1804c51aec9c774f298513638ca016d500fe6328d76a9a9bfb4d6", "role": "Compiler consuming ObjectState-resolved steps and the resolved global configuration once per compilation." }, { "path": "openhcs/core/dataset_sources/source.py", - "sha256": "097fde4d2b74031eacf53a88e79f499417e8c1c97ba0753e8a9df831b307efa6", + "sha256": "f736728d2f715b72cd5479fcc79520911b037d3410a42ebfc9febef2e51ffda6", "role": "DatasetSource family and SourceSelectionRole.require_available_source, which owns the local-directory availability contract." }, { @@ -320,6 +324,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/orchestrator_configuration_management.rst against the later B6 changes to openhcs/core/orchestrator/orchestrator.py, openhcs/core/pipeline/compiler.py, openhcs/core/dataset_sources/source.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/orchestrator_configuration_management.rst in full and compared it against the changed authorities: orchestrator.py, pipeline/compiler.py, objectstate/object_state.py; checked the pipeline_config ObjectState delegation, once-per-compilation resolution, and the initialize availability check (now dataset_source.open plus SourceSelectionRole.require_available_source); corrected the availability paragraph's handler/plate-path vocabulary and added dataset_sources/source.py as authority.", "Re-read docs/source/architecture/orchestrator_configuration_management.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/orchestrator/orchestrator.py; openhcs/core/pipeline/compiler.py; external/ObjectState/src/objectstate/object_state.py.", @@ -458,7 +463,7 @@ }, { "path": "openhcs/processing/backends/lib_registry/registry_service.py", - "sha256": "daef8925338fddf8504a9faa066af42456a13fe4602976bf64be06547a45fcc8", + "sha256": "46ad34dea67bb0f7ba54e1f9f79736654df60ceefe1ef4dd523a226d3222f289", "role": "OpenHCS consumer: imports SignatureAnalyzer from python-introspect for callable catalogue preparation instead of keeping its own signature parser." } ], @@ -467,6 +472,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/parser_metaprogramming_system.rst against the later B6 changes to openhcs/processing/backends/lib_registry/registry_service.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/parser_metaprogramming_system.rst in full and compared it against the changed authorities: python_introspect/signature_analyzer.py, python-introspect docs index, lib_registry/registry_service.py; confirmed python-introspect still owns signature analysis and declaration projection, registry_service.py now imports SignatureAnalyzer from it, and the external_foundations target exists; no prose change needed. Disposition corrected from revise to redirect, since the page is only a route.", "Re-read docs/source/architecture/parser_metaprogramming_system.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: external/python-introspect/src/python_introspect/signature_analyzer.py; external/python-introspect/docs/source/index.rst; openhcs/processing/backends/lib_registry/registry_service.py.", @@ -491,17 +497,17 @@ "authority": [ { "path": "openhcs/core/source_bindings.py", - "sha256": "b1474cfd969bd2ab3af6caba7e6e36f300f59fba5eeaaada08fd2254aef36598", + "sha256": "1dfbd51ae4c64d6a6a8e5ff5ab2ba82465c5e0adacb98dddaf819e0fea05e241", "role": "Public typed source-binding declaration vocabulary (SourceBindingsConfig and binding kinds)." }, { "path": "openhcs/core/source_binding_selection.py", - "sha256": "f97a4475aa26e465679b89ff9a1fe96690ab7631649d71650608fc7a8aa4bf5a", + "sha256": "74061a9dd7d474c908cad4588f59c2bdd1211006b6f2cfe8a5c150d03a96424b", "role": "Source candidate matching, anchor-pattern policies, and source-universe requests that replaced pattern detection." }, { "path": "openhcs/core/dataset_sources/source_bindings_source.py", - "sha256": "fccda877f39e7b2f99d6cc95143480e5955186264e316e832ca9a69c2ae90b8c", + "sha256": "7d2ac740ebb081869ad98573032849732f9b64cbbfb4fc7bd9778becc328c78c", "role": "SourceBindingsSource: the dataset source that ingests arbitrary image folders through source-binding declarations." } ], @@ -510,6 +516,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/pattern_detection_system.rst against the later B6 changes to openhcs/core/source_bindings.py, openhcs/core/source_binding_selection.py, openhcs/core/dataset_sources/source_bindings_source.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/pattern_detection_system.rst in full and compared it against the changed authorities: source_bindings.py, source_binding_selection.py, and the deleted microscopes/source_bindings_handler.py (G4, c6d9a52ce); confirmed typed source matching still owns the model and the source_model target page exists; replaced the deleted handler authority with dataset_sources/source_bindings_source.py; no prose change needed.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", "tests/unit/test_source_bindings.py and tests/unit/test_source_projection.py passed against the current source model" @@ -526,12 +533,12 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "sha256": "7b5622b3d82b1e1d672eaedb14e054be81baef417a736adb06bea34fb6708b28", "role": "ProcessingConfig field docstrings: variable_components is the stack-axis authority and group_by partitions already assembled values, used for dictionary-pattern branch selection." }, { "path": "openhcs/core/processing_contracts.py", - "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", + "sha256": "9b999e9210baf292f8fbf5bb3758c41a95e74401bfb1e1c127bae2dba625b0ee", "role": "ProcessingContract family, which moved here from the library registry module (G5)." }, { @@ -541,7 +548,7 @@ }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "b6a7231bc2964d472070731018f6a4935e5f23cf8f238a232045c32244e6545a", + "sha256": "628a27fea26ed971cfdc4ab22d11df5491f21eb49b6b90397749b46e73e8145b", "role": "RuntimeValueStore and runtime artifact queries behind the runtime-value route." } ], @@ -550,6 +557,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/pattern_grouping_and_special_outputs.rst against the later B6 changes to openhcs/core/config.py, openhcs/core/processing_contracts.py, openhcs/core/runtime_stores.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/pattern_grouping_and_special_outputs.rst in full and compared it against the changed authorities: core/config.py, pipeline/artifact_planning.py, core/runtime_stores.py, lib_registry/unified_registry.py; checked the group_by/variable_components claims against the ProcessingConfig docstrings, confirmed ProcessingContract still exists (now in core/processing_contracts.py) and that all four target pages exist; replaced unified_registry.py with processing_contracts.py as authority; no prose change needed.", "Re-read docs/source/architecture/pattern_grouping_and_special_outputs.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/config.py; openhcs/core/pipeline/artifact_planning.py; openhcs/core/runtime_stores.py; openhcs/processing/backends/lib_registry/unified_registry.py.", @@ -572,17 +580,17 @@ "authority": [ { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "sha256": "d1216010dec1804c51aec9c774f298513638ca016d500fe6328d76a9a9bfb4d6", "role": "_resolve_pipeline_once ObjectState resolution, AxisCompilationRequest context projection, stage order, worker assignment, and bundle construction." }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", + "sha256": "90d75f6287c5f21f93d82960729199261f7906e0f4912b7728527260b9723a66", "role": "ResolvedPipelineDefinition scope/provenance validation and CompilationSession fields and missing-plan errors." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", + "sha256": "f4a851d7de2ea82e1d548f37e6be2511fd20d799580193496a3746e5484906ba", "role": "CompiledStepPlan typed field inventory and per-step framework-device assignment." }, { @@ -592,32 +600,32 @@ }, { "path": "openhcs/core/callable_contract.py", - "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", + "sha256": "2060f856bcb0cc8ef57b92fc17a1e558a676e4033075950b1a4e41dc8656d330", "role": "CallableContract.from_prepared_callable, process-local resolve_runtime_callable cache, and AXIS/PLATE FunctionStepExecutionScope." }, { "path": "openhcs/core/function_patterns.py", - "sha256": "95b9056c6980e4f7943f909aeae8262252c9aa175247a534befc8d4cc459fdd2", + "sha256": "3dbdad68a4c7c765cc0dae056079929e5e4632f39bdbe2e746bea3acc171562d", "role": "CompiledFunctionInvocation binds its runtime callable at construction and rebuilds it after process transport." }, { "path": "openhcs/processing/backends/lib_registry/registry_service.py", - "sha256": "daef8925338fddf8504a9faa066af42456a13fe4602976bf64be06547a45fcc8", + "sha256": "46ad34dea67bb0f7ba54e1f9f79736654df60ceefe1ef4dd523a226d3222f289", "role": "prepare_in_current_process endpoint catalogue preparation and registered_callable owner projection." }, { "path": "openhcs/core/orchestrator/bundle_execution.py", - "sha256": "f5ba09a2ec7c688b1ca08167ab398cc976a9e2c39b301ae98f3c5fd560a25c85", + "sha256": "7665bd4ab8b9ce73d3dcb4fb9679e836b23bffb81d70541baef2e114ee54b0d9", "role": "Parent ordering of plate-scoped steps, completed-dataset metadata finalization, and viewer settlement." }, { "path": "openhcs/core/steps/function_outputs.py", - "sha256": "89405feaa9092caa7ac3699de649eca5516c7bc54a757a24ce881264c21af262", + "sha256": "074f5c16c64956616d86353106c7d89849d448a2a80813ddb17d93d347faa595", "role": "OpenHCSMetadataTarget declarations, typed projection persistence, and completed-plate deduplication across axes." }, { "path": "tests/unit/test_compilation_session.py", - "sha256": "381adf3a87729928a225fd1db5a35f8c802340fb2a1e583501884dc5500f6b9d", + "sha256": "044f6224de217df92c553e216be50c4ac023d69b2a92b835d59a3a781ed432a2", "role": "Regression proof that axis sessions share one resolved pipeline and own their axis plans." } ], @@ -630,6 +638,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/pipeline_compilation_system.rst against the later B6 changes to openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/compiled_step_plan.py, openhcs/core/callable_contract.py, openhcs/core/function_patterns.py, openhcs/processing/backends/lib_registry/registry_service.py, openhcs/core/orchestrator/bundle_execution.py, openhcs/core/steps/function_outputs.py, tests/unit/test_compilation_session.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/pipeline_compilation_system.rst in full and compared it against the changed authorities: pipeline/compiler.py, compilation_session.py, compiled_execution.py, context/processing_context.py, test_compiled_execution.py, compiled_plate_execution.py (renamed bundle_execution.py), steps/function_outputs.py, test_compilation_session.py, callable_contract.py, compiled_step_plan.py, framework_device_assignment.py; verified each named class/function exists (FunctionStepTransportAuthority, RegistryService.registered_callable, AxisCompilationRequest, CompiledRuntimeEnvironmentPlan, RuntimeArtifactBatch, OpenHCSMetadataTarget, FunctionStepExecutionScope AXIS/PLATE); rewrote the resolve-once, preparation, session and plan-inventory passages that named deleted StepSnapshot, prepare_compiled_context_callables and source load plans. Dropped the framework-device resolver/assignment files and test_framework_device_resolver.py as authorities since the page states only that bindings stay on step plans.", "Re-read docs/source/architecture/pipeline_compilation_system.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/pipeline/compiler.py; openhcs/core/pipeline/compilation_session.py; openhcs/core/compiled_execution.py; openhcs/core/orchestrator/compiled_plate_execution.py; openhcs/core/steps/function_outputs.py; tests/unit/test_compilation_session.py; openhcs/core/callable_contract.py; openhcs/core/compiled_step_plan.py; openhcs/core/pipeline/framework_device_assignment.py; openhcs/core/framework_device_resolver.py; tests/unit/test_framework_device_resolver.py.", @@ -657,7 +666,7 @@ "authority": [ { "path": "openhcs/authoring/session/session.py", - "sha256": "587047b63972df89ec46a01ed2c92ff3ef9577335f58a980297db02094e4e8e0", + "sha256": "458d71fe96ab75153977c554fb748e78f1773cd7794b625d7ec233c9b22e4453", "role": "Session class, MainThread/DispatcherThread, DatasetAccess and Renderer ports, dataset add/delete/split and work/mutation checks." }, { @@ -667,22 +676,22 @@ }, { "path": "openhcs/authoring/session/views.py", - "sha256": "8ce6cac27293ac9105f4d2ebc76d10eebeba1ef90e0e8954c0a117910e17e3b3", + "sha256": "2963a2c48856704ea8291075316c470ccc8eeed161b5a798181a519a2ceaef98", "role": "SessionView family with DatasetListView and PipelineStepsView." }, { "path": "openhcs/authoring/session/events.py", - "sha256": "6f0b27e3c6d43de6f301de4c9bb80b713b433c7e10c351eda5b35c57d91ece98", + "sha256": "4bd9aa333019a770fd64dc4a42617148eb4e4a3081e488310ca860b68c11265c", "role": "SessionEvent family and event log with subscribe and events-after reads." }, { "path": "openhcs/authoring/session/compile_batch.py", - "sha256": "0cbe5a6b86a0fb5e9b92c4fc45f858c979031ac93594b317b9761c28f5aaf9ab", + "sha256": "d539bcf8df15581d23bc69701c5d30d489c3fe5f600f19236b28ec35c1c2d92f", "role": "Compile batches that check work is allowed and mark datasets compile-pending before connecting." }, { "path": "openhcs/authoring/session/dataset_document.py", - "sha256": "305da3c0bf7bb15a0f251a101696b721f9151d8620450a6b3e12413b79218574", + "sha256": "ca695d82a60dbee76ff660ae3710ac5033d2e55f51a99373ed8c0f0dccdbc16c", "role": "DatasetDocumentScope family and payload scope-id check for selected code documents." }, { @@ -692,7 +701,7 @@ }, { "path": "openhcs/agent/capabilities.py", - "sha256": "eaebe33af01d057d1d21b3701f958bc96ede3ec5d0bddddca9917c1164ef0042", + "sha256": "87c8e2e5aeb425b9a2eb690d48bdd6f5af7a327c389deac261d7e4e0da5d609f", "role": "SessionOperationInvocation deriving MCP tools from headless operations, and the openhcs_session_events capability." }, { @@ -707,12 +716,12 @@ }, { "path": "openhcs/pyqt_gui/session_rendering.py", - "sha256": "deee8c93c5bfd41c5ee25abe959938426ed17120275b3b6ee9ccc25b2dd0d690", + "sha256": "8059792f44b1b394ae0e2c68a5f614e55c9a7cc8c59e2e7c0ca7353f09f10cf4", "role": "QtSessionEventRelay GUI subscriber." }, { "path": "openhcs/mcp/server.py", - "sha256": "e1f236011df715da3d17574c0b80cbf50fcae45e3f82491c22fb57e98c6bb662", + "sha256": "927e64594a0fc174759eeb58496fed0ccf95e5d8d29bc248cde9a033efb69fbc", "role": "MCP binds the session to DispatcherThread over McpMainThreadDispatcher." } ], @@ -722,6 +731,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/plate_manager_services.rst against the later B6 changes to openhcs/authoring/session/session.py, openhcs/authoring/session/views.py, openhcs/authoring/session/events.py, openhcs/authoring/session/compile_batch.py, openhcs/authoring/session/dataset_document.py, openhcs/agent/capabilities.py, openhcs/pyqt_gui/session_rendering.py, openhcs/mcp/server.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/plate_manager_services.rst in full and compared it against the changed authorities: the page itself (rewritten in U1), deleted plate_manager_batch_workflow.py, batch_workflow_components.py, pipeline_object_state_binding.py and plate_manager_state_projection.py, changed plate_manager_workflows.py, pipeline_editor_workflows.py, endpoint_function_catalog_service.py, pyqt_gui/main.py, the Plate Manager and Pipeline Editor widget tests, and demo/synthetic_plate_pipeline.py; verified every named module in openhcs/authoring/session, SessionOperation/SessionView/SessionEvent families, DatasetListView, PipelineStepsView, DatasetDocumentScope, PipelineImporter keyed by suffix, the CellProfiler .cppipe dataset scope, QtSessionEventRelay, openhcs_session_events, DispatcherThread use in the GUI and MCP, compile-artifact ids on submission and compile-pending reservation; replaced all authorities with session-package owners.", "Re-read docs/source/architecture/plate_manager_services.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/pyqt_gui/services/plate_manager_batch_workflow.py; openhcs/pyqt_gui/widgets/shared/services/batch_workflow_components.py; tests/unit/pyqt_gui/test_plate_manager_widget.py; openhcs/pyqt_gui/widgets/shared/services/plate_manager_workflows.py; openhcs/pyqt_gui/services/pipeline_object_state_binding.py; openhcs/pyqt_gui/widgets/shared/services/pipeline_editor_workflows.py; openhcs/agent/services/endpoint_function_catalog_service.py; openhcs/pyqt_gui/main.py; tests/unit/pyqt_gui/test_pipeline_editor_widget.py; openhcs/demo/synthetic_plate_pipeline.py.", @@ -826,17 +836,17 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "sha256": "7b5622b3d82b1e1d672eaedb14e054be81baef417a736adb06bea34fb6708b28", "role": "ProcessingConfig variable_components, group_by and input_source field semantics." }, { "path": "openhcs/core/processing_contracts.py", - "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", + "sha256": "9b999e9210baf292f8fbf5bb3758c41a95e74401bfb1e1c127bae2dba625b0ee", "role": "ProcessingContract family: Pure2DContract, Pure3DContract, FlexibleContract, VolumetricToSliceContract and collapses_input_plane_axis." }, { "path": "openhcs/core/callable_contract.py", - "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", + "sha256": "2060f856bcb0cc8ef57b92fc17a1e558a676e4033075950b1a4e41dc8656d330", "role": "CallableContract memory types, allowed group_by roles, variable-component requirement, execution scope, runtime adapter, image mode and ImagePayloadConsumption." }, { @@ -856,7 +866,7 @@ }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", + "sha256": "f4a851d7de2ea82e1d548f37e6be2511fd20d799580193496a3746e5484906ba", "role": "CompiledStepPlan storage of resolved callable facts and framework-local device bindings." } ], @@ -868,6 +878,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/processing_semantics.rst against the later B6 changes to openhcs/core/config.py, openhcs/core/processing_contracts.py, openhcs/core/callable_contract.py, openhcs/core/compiled_step_plan.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/processing_semantics.rst in full and compared it against the changed authorities: core/config.py, lib_registry/unified_registry.py, callable_contract.py, runtime_slice_projection.py, compiled_step_plan.py, framework_device_assignment.py; verified the ProcessingConfig field semantics, the four ProcessingContract subclasses and keys, contract=Pure2DContract decorator use, CallableContract fields, FunctionStepExecutionScope and ImagePayloadConsumption; rewrote the payload-rank intro, contract names and the runtime-projection paragraph. Replaced unified_registry.py and framework_device_assignment.py with processing_contracts.py, payload_axes.py and runtime_plane_projection.py as authorities.", "Re-read docs/source/architecture/processing_semantics.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/config.py; openhcs/processing/backends/lib_registry/unified_registry.py; openhcs/core/callable_contract.py; openhcs/core/runtime_slice_projection.py; openhcs/core/compiled_step_plan.py; openhcs/core/pipeline/framework_device_assignment.py.", @@ -892,17 +903,17 @@ "authority": [ { "path": "openhcs/core/progress/projection.py", - "sha256": "0c931a096020b5622bcfbf92c856c7cb3ead193805b7276f5fb89bfb5613f8cb", + "sha256": "88851cf9e6704ad812208a3fb58e17897363ab46662fe0cb8f0dec6ecc685f2e", "role": "SubjectRuntimeState family (labels, terminal and server-lifecycle policy), SubjectRuntimeIdentity, state_counts and ExecutionRuntimeProjection heartbeat reconciliation." }, { "path": "openhcs/core/progress/runtime_tree.py", - "sha256": "016279f4135e2c9b10cb841e57ba0297c4197a195623f8a746ed90ab06533ce5", + "sha256": "27689de8557c87bcff5be713fa85ec0024171228c80b0cac6e86039114a18865", "role": "Runtime tree node kinds subject, worker, partition, step and compilation." }, { "path": "openhcs/core/progress/types.py", - "sha256": "beabbfeb8a6ed727b4bb09f6155de60c4e8b89e71e7ed053bb85574aa1675419", + "sha256": "b5f2ca1a053665a12cca78a6a6676f1ad7e157afafcc5e2161655fddf8bf34e5", "role": "ProgressEvent, set_progress_queue and the PATTERN_GROUP, AXIS_COMPLETED and SUCCESS phases." }, { @@ -912,7 +923,7 @@ }, { "path": "openhcs/core/debug.py", - "sha256": "95d365afefc46ee50da5abb8d5d246b3980feb32b85839f91e874cb7e689caaa", + "sha256": "4f02b36c242b7db3c4e0fa2060a852e11ded2ab497abde47494cba41ae7afdca", "role": "Debug command, boundary-event, cursor, session, snapshot, artifact identity, snapshot store, warm-replay and paused-worker declarations." }, { @@ -922,17 +933,17 @@ }, { "path": "openhcs/authoring/session/progress.py", - "sha256": "cd34f68461b50506600da7854a3bf57646f2f0ff7288d0627bc40cb852a16825", + "sha256": "eef42f1c12c9649c36e7d1a86d664f527e6c1aa28868eb5b348d1ebdb5d548bd", "role": "ExecutionProgress: dirty-marking registry subscription and interval rebuild on the session main thread, published as session events." }, { "path": "openhcs/agent/capabilities.py", - "sha256": "eaebe33af01d057d1d21b3701f958bc96ede3ec5d0bddddca9917c1164ef0042", + "sha256": "87c8e2e5aeb425b9a2eb690d48bdd6f5af7a327c389deac261d7e4e0da5d609f", "role": "openhcs_session_events and openhcs_session_datasets headless readers." }, { "path": "openhcs/core/orchestrator/bundle_execution.py", - "sha256": "f5ba09a2ec7c688b1ca08167ab398cc976a9e2c39b301ae98f3c5fd560a25c85", + "sha256": "7665bd4ab8b9ce73d3dcb4fb9679e836b23bffb81d70541baef2e114ee54b0d9", "role": "execute_dataset_scoped_steps and the parent ordering before the pipeline SUCCESS event." }, { @@ -954,6 +965,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/progress_runtime_projection_system.rst against the later B6 changes to openhcs/core/progress/projection.py, openhcs/core/progress/runtime_tree.py, openhcs/core/progress/types.py, openhcs/core/debug.py, openhcs/authoring/session/progress.py, openhcs/agent/capabilities.py, openhcs/core/orchestrator/bundle_execution.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/progress_runtime_projection_system.rst in full and compared it against the changed authorities: the page itself (edited in U1), core/progress/projection.py, core/debug.py, zmqruntime progress/projection.py and execution/progress_stream.py, deleted agent/services/execution_session_service.py, agent/dto/execution.py, tests/unit/agent/test_agent_services.py; verified every named class/function (all debug declarations, RuntimeProjectionBuilder, ExecutionProgress, LiveMeasurementProgressPayload, ZMQ progress emitters, phases) and corrected the plate-to-subject vocabulary, tree path, Qt-timer coalescing description and the dataset-scoped step function name.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", "tests/unit/progress/test_projection.py and tests/unit/progress/test_debug_projection.py passed; published owner URLs now replace checkout-local paths", @@ -990,27 +1002,27 @@ }, { "path": "openhcs/core/processing_contracts.py", - "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", + "sha256": "9b999e9210baf292f8fbf5bb3758c41a95e74401bfb1e1c127bae2dba625b0ee", "role": "Pure2DContract and Pure3DContract named in the minimal example and the locality summary." }, { "path": "openhcs/core/source_bindings.py", - "sha256": "b1474cfd969bd2ab3af6caba7e6e36f300f59fba5eeaaada08fd2254aef36598", + "sha256": "1dfbd51ae4c64d6a6a8e5ff5ab2ba82465c5e0adacb98dddaf819e0fea05e241", "role": "SourceBindingsConfig fields and the selector, filter and metadata-rule types used in the folder example." }, { "path": "openhcs/core/dataset_sources/source_bindings_source.py", - "sha256": "fccda877f39e7b2f99d6cc95143480e5955186264e316e832ca9a69c2ae90b8c", + "sha256": "7d2ac740ebb081869ad98573032849732f9b64cbbfb4fc7bd9778becc328c78c", "role": "SourceBindingsSource, selected when a non-empty source-binding declaration is present." }, { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "3439d96f3fbfcb104a53019dc150049021d9a8132bd27096f1ccf510eefbe37b", + "sha256": "5d3646e17f7de843a5b36164f7cef7505da7841636423ae4fb4e00d194a87e9c", "role": "import_cellprofiler_pipeline signature and headless-import rejection of interactive modules." }, { "path": "openhcs/agent/capabilities.py", - "sha256": "eaebe33af01d057d1d21b3701f958bc96ede3ec5d0bddddca9917c1164ef0042", + "sha256": "87c8e2e5aeb425b9a2eb690d48bdd6f5af7a327c389deac261d7e4e0da5d609f", "role": "MCP tool names used in the first-session sequence (health check, authoring context, capability and knowledge search, config schema, pipeline drafts)." }, { @@ -1020,7 +1032,7 @@ }, { "path": "openhcs/authoring/session/session.py", - "sha256": "587047b63972df89ec46a01ed2c92ff3ef9577335f58a980297db02094e4e8e0", + "sha256": "458d71fe96ab75153977c554fb748e78f1773cd7794b625d7ec233c9b22e4453", "role": "Session shared by the desktop application and MCP for progress, cancellation and results." }, { @@ -1043,6 +1055,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/quick_start.rst against the later B6 changes to openhcs/core/processing_contracts.py, openhcs/core/source_bindings.py, openhcs/core/dataset_sources/source_bindings_source.py, openhcs/interop/cellprofiler/pipeline_import.py, openhcs/agent/capabilities.py, openhcs/authoring/session/session.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/quick_start.rst in full and compared it against the changed authorities: the page itself (edited in G5/G1), pipeline_document.py, agent/authoring_contexts.py, cellprofiler/pipeline_import.py, test_cellprofiler_pipeline_import.py, test_pipeline_document.py, agent/services/llm_context_service.py, test_progressive_authoring_context.py; verified every MCP tool name in agent/capabilities.py, the analysis-strategy section id in the knowledge manifest, the code-example imports (Microscopy.Channel, stack_percentile_normalize, Lazy*Config, PipelineDocumentCodec) and the MFD preset paths; corrected StepSnapshot, PURE_2D/PURE_3D, SourceBindingsHandler, the missing InputSource import and the progress owner. scripts/validate_docs.py reports no code-block or import finding for the page.", "Reviewed the complete page against PipelineDocumentAuthority and the public pipeline declarations after the compatibility change.", "Passed the documentation validator and tests/unit/test_pipeline_document.py for parsing, defaulting, rejection, and canonical rendering behavior.", @@ -1069,7 +1082,7 @@ }, { "path": "openhcs/processing/materialization/core.py", - "sha256": "744573f1bafa67b7a81be5af425012ada47741292519863e250b79f1c778b265", + "sha256": "1e408b7b7c4ac406f5bce8a1fda8a8774cb049dbc2a646832b8c4d008ae2b5c3", "role": "ROI output writers, polyline graph-edge writer with its 2D-only rejection, and projected filename resolution." }, { @@ -1079,7 +1092,7 @@ }, { "path": "openhcs/core/artifacts.py", - "sha256": "65eb0ec5c903366243ca8bd5ea07412e8d34bdb81e6015fa76b3c4c59838733f", + "sha256": "907551ae0ce4bf42a557554f5df0252b0a88fc9b7c1b7ac19705aade66b6656f", "role": "ArtifactOutputPlan, ArtifactSpecRelation and ObjectArtifactSubjectBinding." }, { @@ -1089,7 +1102,7 @@ }, { "path": "tests/unit/test_napari_roi_manager_integration.py", - "sha256": "a4b7dee556f0ceccaa212751e29eb8160d3d0c08c741760715ee8372b0c0e927", + "sha256": "10d7079b3e10088bac81a8814b4d0f77854d20b7818f2c2a9d121589b295ca15", "role": "Regression proof for the native Shapes-layer ROI manager seam." } ], @@ -1098,6 +1111,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/roi_system.rst against the later B6 changes to openhcs/processing/materialization/core.py, openhcs/core/artifacts.py, tests/unit/test_napari_roi_manager_integration.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/roi_system.rst in full and compared it against the changed authorities: napari_roi_manager/widgets/_roi_manager.py (observer release on destroy), processing/materialization/core.py (new ROIOutput/PointROIOutput writers), polystore/roi.py, test_napari_roi_manager_integration.py; verified SpatialGraphROIOptions, SWCOptions, SpatialGraph, ArtifactOutputPlan, ArtifactSpecRelation, ObjectArtifactSubjectBinding and connect_layer exist and that the polyline writer rejects 3D graphs; reordered one misplaced paragraph. Added options.py and core/artifacts.py as authorities for the named declarations.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", "Source inspection confirmed native Shapes ownership and nominal materialisation strategies; the Napari ROI integration test covers the UI seam", @@ -1124,7 +1138,7 @@ }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "b6a7231bc2964d472070731018f6a4935e5f23cf8f238a232045c32244e6545a", + "sha256": "628a27fea26ed971cfdc4ab22d11df5491f21eb49b6b90397749b46e73e8145b", "role": "RuntimeValueStore recording, replacement, RuntimeArtifactQuery resolution, and observation semantics." }, { @@ -1139,7 +1153,7 @@ }, { "path": "openhcs/core/artifacts.py", - "sha256": "65eb0ec5c903366243ca8bd5ea07412e8d34bdb81e6015fa76b3c4c59838733f", + "sha256": "907551ae0ce4bf42a557554f5df0252b0a88fc9b7c1b7ac19705aade66b6656f", "role": "ArtifactType.contextualize_output root (unchanged) and its image, object-label, measurement, spatial-grid and spatial-graph overrides." }, { @@ -1149,7 +1163,7 @@ }, { "path": "openhcs/core/runtime_object_labels.py", - "sha256": "a8ca5cc37f7049bb0ed06f1600afab9943d515e5ccb43dac0fb39f8f1bc7fbf7", + "sha256": "c9581e4f9ba69cf40d2717745d82770b4b9da06ee59cef6bf60a68b72e576d23", "role": "ObjectLabelValue.contextualize_image_output plane-axis rule for label-rendered images." }, { @@ -1159,7 +1173,7 @@ }, { "path": "openhcs/core/callable_contract.py", - "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", + "sha256": "2060f856bcb0cc8ef57b92fc17a1e558a676e4033075950b1a4e41dc8656d330", "role": "CallableContract.resolve_returned_plan_values binding returned positions to exact output plan refs." }, { @@ -1169,12 +1183,12 @@ }, { "path": "tests/unit/test_function_artifact_outputs.py", - "sha256": "4c2b42cd9f31cc3cd63682039bb691cdd0af5bec3ca5b6f23f2ebea0fdbf0fac", + "sha256": "b5f103bd9e64f5685c593213edb8bb763b9502f899420eed75bec76b44fcb3d5", "role": "Proves image-output contextualization and that named artifacts bypass unrelated source-axis projection." }, { "path": "tests/unit/test_runtime_slice_projection.py", - "sha256": "d47cad11b66f1bb3a7e77796bf5cca83599ef8ef4f876ac685f06e002dc9552a", + "sha256": "80a39c9c110cfd54a3db6f0715a916601871fea2202cc505e9f7c27f209278d2", "role": "Proves undeclared value types are rejected and declared slices project without implicit device-to-host conversion." } ], @@ -1186,6 +1200,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/runtime_value_system.rst against the later B6 changes to openhcs/core/runtime_stores.py, openhcs/core/artifacts.py, openhcs/core/runtime_object_labels.py, openhcs/core/callable_contract.py, tests/unit/test_function_artifact_outputs.py, tests/unit/test_runtime_slice_projection.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/runtime_value_system.rst in full and compared it against the changed authorities: runtime_artifact_values.py, runtime_stores.py, runtime_slice_projection.py, runtime_image_values.py, steps/function_runtime.py and the two test files; verified every named class/function exists, replaced the deleted output matcher, slice-projection strategy registry and function-output context strategies with their current owners (resolve_returned_plan_values, RuntimeSliceProjectableValue, ArtifactType.contextualize_output and contextualize_image_output), and confirmed the persisted-metadata reload fallback in source_workspace_projection.py still holds.", "Re-read docs/source/architecture/runtime_value_system.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/runtime_artifact_values.py; openhcs/core/runtime_stores.py; openhcs/core/runtime_slice_projection.py; openhcs/core/steps/function_runtime.py; tests/unit/test_function_artifact_outputs.py.", @@ -1280,7 +1295,7 @@ }, { "path": "openhcs/core/image_file_serialization.py", - "sha256": "644a01f8cccdf8e3d663202e3207d8750b4cf2f03f42c5bcc82dcaffb2c3fb5d", + "sha256": "8640772ec0a5b011c525851f0a64204a23e02d4e5c753852d009ac839fea9235", "role": "ImageFileFormat registry: suffix matching, host preparation, read/write, source metadata, intensity scale, and declared pixel-band axes." }, { @@ -1294,6 +1309,7 @@ ], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/serialization_boundaries.rst against the later B6 changes to openhcs/core/image_file_serialization.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/serialization_boundaries.rst in full and compared it against the changed authorities: openhcs/serialization/json.py (deleted, L2) and openhcs/core/image_file_serialization.py; checked every to_jsonable dispatch case in python_introspect/jsonable.py, the dataclass_from_mapping decoder, OpenHCS to_jsonable.register users, and ImageFileFormat suffix/prepare/read/write/pixel-semantics members; no prose change was needed.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", "tests/unit/agent/test_agent_serialization.py and tests/unit/test_image_file_serialization.py cover the named boundaries" @@ -1368,22 +1384,22 @@ "authority": [ { "path": "openhcs/core/source_bindings.py", - "sha256": "b1474cfd969bd2ab3af6caba7e6e36f300f59fba5eeaaada08fd2254aef36598", + "sha256": "1dfbd51ae4c64d6a6a8e5ff5ab2ba82465c5e0adacb98dddaf819e0fea05e241", "role": "Public source-binding declaration vocabulary and compiled binding/universe plans." }, { "path": "openhcs/core/source_binding_workspace.py", - "sha256": "4d28066a9f73b283fc637662e05fb2632c6fa18af164476c2899fce58cd647fe", + "sha256": "33911fd2973176817f155eae78260b81085b4464c07907d28a678eb46de42b15", "role": "Source-binding workspace projector and materialization of the virtual workspace." }, { "path": "openhcs/core/dataset_sources/source_bindings_source.py", - "sha256": "fccda877f39e7b2f99d6cc95143480e5955186264e316e832ca9a69c2ae90b8c", + "sha256": "7d2ac740ebb081869ad98573032849732f9b64cbbfb4fc7bd9778becc328c78c", "role": "SourceBindingsSource: recursive folder inventory and declared source-binding projection." }, { "path": "openhcs/core/dataset_sources/choice.py", - "sha256": "7fb3c7ce701f0d629f288b79090044f43ac8ce1b51ca4f50937534407f226f1a", + "sha256": "ece43af801b4b02c302d9ac4bf6e5d07c0d9ebee5027433d12a4b548a19d11f0", "role": "AutoDetectedSource falls back to the registered declared-file source only when detection fails and bindings are non-empty." }, { @@ -1402,6 +1418,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/source_model.rst against the later B6 changes to openhcs/core/source_bindings.py, openhcs/core/source_binding_workspace.py, openhcs/core/dataset_sources/source_bindings_source.py, openhcs/core/dataset_sources/choice.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/source_model.rst in full and compared it against the changed authorities: openhcs/core/source_bindings.py, openhcs/core/source_binding_workspace.py, openhcs/microscopes/source_bindings_handler.py (deleted, G4) and PolyStore filemanager.py; checked every named declaration, compiled plan, projection, diagnostic and PolyStore method exists, corrected the arbitrary-folder fallback to AutoDetectedSource/SourceBindingsSource, and confirmed CompiledStepPlan carries no separate source-binding enabled flag.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", "tests/unit/test_source_bindings.py, tests/unit/test_source_projection.py, and tests/unit/test_source_binding_workspace.py cover declarations, projections, and generic-folder ingestion", @@ -1419,12 +1436,12 @@ "authority": [ { "path": "openhcs/core/callable_contract.py", - "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", + "sha256": "2060f856bcb0cc8ef57b92fc17a1e558a676e4033075950b1a4e41dc8656d330", "role": "CallableContract input/output declarations and runtime-owned parameter projection." }, { "path": "openhcs/core/pipeline/function_contracts.py", - "sha256": "13bcaf7c7a62654f512d2cc739edccd2deeff35d99970ca493f3e827afbda91a", + "sha256": "883d22780ab41f122d6ed651c9f59fea540120f8fcf50d30e348ac295bd4fca9", "role": "special_inputs decorator: records parameter names as runtime-managed callable metadata and excludes them from user parameters; no special_outputs decorator exists." }, { @@ -1434,12 +1451,12 @@ }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "b6a7231bc2964d472070731018f6a4935e5f23cf8f238a232045c32244e6545a", + "sha256": "628a27fea26ed971cfdc4ab22d11df5491f21eb49b6b90397749b46e73e8145b", "role": "RuntimeValueStore recording, replacement, query, and observation during execution." }, { "path": "openhcs/interop/cellprofiler/module_artifact_contracts.py", - "sha256": "982680cd36409ea6d90a07d4e60893b162847f5525ed516891e995877dc32fd1", + "sha256": "c9f1210146b23991dd5433ae2acac43c972b95cc95cf9bd785fac19f9d810928", "role": "CellProfilerModuleArtifactContracts: per-invocation artifact bindings from CellProfiler setting declarations." } ], @@ -1449,6 +1466,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/special_io_system.rst against the later B6 changes to openhcs/core/callable_contract.py, openhcs/core/pipeline/function_contracts.py, openhcs/core/runtime_stores.py, openhcs/interop/cellprofiler/module_artifact_contracts.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/special_io_system.rst in full, including its unreviewed special_outputs removal, and compared it against the changed authorities: callable_contract.py, pipeline/artifact_planning.py, runtime_stores.py, interop/cellprofiler/module_artifact_contracts.py; confirmed each named class and the special_inputs decorator exist with the roles the page assigns and that no special_outputs declaration remains; fixed the edited sentence.", "Re-read docs/source/architecture/special_io_system.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/callable_contract.py; openhcs/core/pipeline/artifact_planning.py; openhcs/core/runtime_stores.py; openhcs/interop/cellprofiler/module_artifact_contracts.py.", @@ -1527,13 +1545,14 @@ }, { "path": "pyproject.toml", - "sha256": "3592cc37444e2e1f5bb0284e939643039c0d513217861af8133dd8bd6e8a4325", + "sha256": "c3a2bea22aa6872940125f4aaac72c9eac72084fefc6c2cefdb54b5273361e4a", "role": "Declares polystore and arraybridge as separate runtime dependencies of OpenHCS." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/storage_and_memory_system.rst against the later B6 changes to pyproject.toml: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/storage_and_memory_system.rst in full and compared it against the changed authorities: external/PolyStore/src/polystore/filemanager.py (submodule advance) and pyproject.toml (polystore 0.6, arraybridge 0.4.1, JAX and openhcs-basicpy dependency changes); the redirect makes no version, backend or API claim those changes affect, and both packages remain separate dependencies with intersphinx targets in docs/source/conf.py; no prose change was needed.", "Re-read docs/source/architecture/storage_and_memory_system.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: external/PolyStore/src/polystore/filemanager.py; external/arraybridge/src/arraybridge/converters.py; pyproject.toml.", @@ -1569,7 +1588,7 @@ "authority": [ { "path": "openhcs/core/viewer_streaming_service.py", - "sha256": "c96eca530f767002cfcc9aca4fe75c93d7a5848a7f42df9e5d336bf6712ed51e", + "sha256": "b85d6d9b0677bdbd99a657672e19e3a27ef418bfcddfb5550524a83e7598ec4b", "role": "Compiled output streaming through StreamComponentMessageExtraAuthority and viewer settlement." }, { @@ -1579,22 +1598,22 @@ }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", + "sha256": "f4a851d7de2ea82e1d548f37e6be2511fd20d799580193496a3746e5484906ba", "role": "CompiledStepPlan.producer_identity_for_main_flow and producer_identity_for_artifact." }, { "path": "openhcs/core/runtime_object_labels.py", - "sha256": "a8ca5cc37f7049bb0ed06f1600afab9943d515e5ccb43dac0fb39f8f1bc7fbf7", + "sha256": "c9581e4f9ba69cf40d2717745d82770b4b9da06ee59cef6bf60a68b72e576d23", "role": "ObjectLabelValue.contextualize_image_output plane-axis and provenance rule for label-rendered images." }, { "path": "openhcs/core/orchestrator/bundle_execution.py", - "sha256": "f5ba09a2ec7c688b1ca08167ab398cc976a9e2c39b301ae98f3c5fd560a25c85", + "sha256": "7665bd4ab8b9ce73d3dcb4fb9679e836b23bffb81d70541baef2e114ee54b0d9", "role": "Viewer bootstrap with rollback of the acquired set, settle-then-read-state order, and stopping of execution-owned viewers." }, { "path": "openhcs/runtime/viewer_protocol.py", - "sha256": "c4bdae4bae6dee412f0d27a9cf75a95db53b3bd95c03cee7d046be16b52fe5b8", + "sha256": "f126aaf4f5c0dcc9429b7aae8ce67985260241ec7d2f3d5a948573658cf079a3", "role": "ManagedViewerLifecycleMixin, ViewerProcessPlatform Linux policy, ViewerSettleProgress phases, and the execution-owned EndpointProcessGroup." }, { @@ -1619,7 +1638,7 @@ }, { "path": "openhcs/runtime/zmq_execution_server.py", - "sha256": "e24bdcb3b086cc60197ceb63f29a6f6b639a93cad175218941148a207eb285f2", + "sha256": "0f26779f2d31a4478b9bf9f5d8e4a5b02d8d2e346861f404e7cc35628e64e25e", "role": "Releases execution-session process resources when the execution server stops." }, { @@ -1639,22 +1658,22 @@ }, { "path": "tests/unit/test_viewer_result_element_selection.py", - "sha256": "b1dae580f0291f53be9453df6f6c6df5458046bd3ff947b550f7ddea94b29f96", + "sha256": "a7b348fab106da4c89e58ed32225495aa27792132ef7488657f29821dfd6da92", "role": "Proves bulk isolation validates every route and navigation before any mutation." }, { "path": "tests/unit/test_napari_streaming_handlers.py", - "sha256": "93244d4d09c5cd6d7a2645fca2482578ad52646f3bb9e6de5ebe505b157fbfea", + "sha256": "860e234a39f36f143491e6f1edc9515c1d9cde88590759252378f8d23c0bab12", "role": "Proves one global ROI-member budget and metadata-only shape projection." }, { "path": "tests/unit/test_napari_accepted_work_settlement.py", - "sha256": "50cde8ceede31c3ef07db0c0aed69865d058617fd77f34d834bb48252a47fcf1", + "sha256": "86b37ee03cd1f3ca16d912a47365b4b2a0152263e4bac1f4f1ba08b8e55cfb08", "role": "Tests accepted display failures, settlement reporting and recovery." }, { "path": "tests/unit/test_orchestrator_lane_planning.py", - "sha256": "3d5c2f3662b8d4c330582af0c4a649b8f4d9582e064d9cf3159798c8934a2bbb", + "sha256": "4a83337fa35c68e4f641760ebbf86e4cc30f6624da55ccebd09be636a4b56bd0", "role": "Proves multi-viewer bootstrap rollback after partial acquisition and readiness failure." } ], @@ -1666,6 +1685,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/streaming_boundary_and_wrappers.rst against the later B6 changes to openhcs/core/viewer_streaming_service.py, openhcs/core/compiled_step_plan.py, openhcs/core/runtime_object_labels.py, openhcs/core/orchestrator/bundle_execution.py, openhcs/runtime/viewer_protocol.py, openhcs/runtime/zmq_execution_server.py, tests/unit/test_viewer_result_element_selection.py, tests/unit/test_napari_streaming_handlers.py, tests/unit/test_napari_accepted_work_settlement.py, tests/unit/test_orchestrator_lane_planning.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/streaming_boundary_and_wrappers.rst in full and compared it against the changed authorities: viewer_streaming_service.py, streaming_config_declarations.py, viewer_protocol.py, viewer_controls.py, napari_viewer_server.py, viewer_window_service.py, zmq_execution_server.py, compiled_plate_execution.py (renamed bundle_execution.py, G7), zmqruntime client.py and the two viewer tests; checked every named class exists, corrected the producer-identity and label-image context owners, and confirmed the bootstrap rollback, settle/read-state/stop order, Linux XCB/MIT-SHM/vblank policy and settle phases in current code.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", "tests/unit/test_streaming_service.py and tests/unit/test_viewer_protocol.py exercise the compiled stream, readiness, settlement, and platform policies", @@ -1709,7 +1729,7 @@ }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "sha256": "d1216010dec1804c51aec9c774f298513638ca016d500fe6328d76a9a9bfb4d6", "role": "PipelineCompiler phases from declaration resolution to framework device assignment." }, { @@ -1723,6 +1743,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/system_integration.rst against the later B6 changes to openhcs/core/pipeline/compiler.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/system_integration.rst in full and compared it against the changed authorities: openhcs/core/pipeline_document.py (PipelineDocumentAuthority renamed PipelineDocumentCodec), openhcs/core/pipeline/compiler.py and openhcs/core/compiled_execution.py; the redirect states no API detail and its target system_overview.rst exists; no prose change was needed.", "Re-read docs/source/architecture/system_integration.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/pipeline_document.py; openhcs/core/pipeline/compiler.py; openhcs/core/compiled_execution.py.", @@ -1748,12 +1769,12 @@ }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", + "sha256": "90d75f6287c5f21f93d82960729199261f7906e0f4912b7728527260b9723a66", "role": "ResolvedPipelineDefinition (resolved steps, scope ids, field provenance) and the per-axis CompilationSession." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", + "sha256": "f4a851d7de2ea82e1d548f37e6be2511fd20d799580193496a3746e5484906ba", "role": "CompiledStepPlan typed fields, including framework-local device assignment." }, { @@ -1768,7 +1789,7 @@ }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "b6a7231bc2964d472070731018f6a4935e5f23cf8f238a232045c32244e6545a", + "sha256": "628a27fea26ed971cfdc4ab22d11df5491f21eb49b6b90397749b46e73e8145b", "role": "RuntimeValueStore recording, replacement, query, and observation semantics." }, { @@ -1804,6 +1825,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/system_overview.rst against the later B6 changes to openhcs/core/pipeline/compilation_session.py, openhcs/core/compiled_step_plan.py, openhcs/core/runtime_stores.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/system_overview.rst in full and compared it against the changed authorities: pipeline_document.py, pipeline/compilation_session.py, compiled_execution.py, runtime_stores.py, compiled_step_plan.py, framework_device_assignment.py, pyproject.toml and the python-introspect/arraybridge/metaclass-registry owners; replaced StepSnapshot with ResolvedPipelineDefinition, restated the session invariant to what CompilationSession enforces, moved slice projection to value families, added the python-introspect JSON codec, and confirmed the CompiledStepPlan fields and bundle environment facts.", "Re-read docs/source/architecture/system_overview.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/pipeline_document.py; openhcs/core/pipeline/compilation_session.py; openhcs/core/compiled_execution.py; openhcs/core/runtime_stores.py; pyproject.toml; openhcs/core/compiled_step_plan.py; openhcs/core/pipeline/framework_device_assignment.py.", @@ -1841,12 +1863,12 @@ "authority": [ { "path": "pyproject.toml", - "sha256": "3592cc37444e2e1f5bb0284e939643039c0d513217861af8133dd8bd6e8a4325", + "sha256": "c3a2bea22aa6872940125f4aaac72c9eac72084fefc6c2cefdb54b5273361e4a", "role": "No Textual dependency or entry point; openhcs-gui gui-script points at the PyQt application." }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "e43085c1eaf0c836ce34e1a1dae7740488d91bdfea56844e7ddd4e6925561cf6", + "sha256": "00ff3bfe7a032bd3de59274697a69d12c9b898177cd8e839f9da3b130becaedf", "role": "PyQt desktop main window composition." }, { @@ -1860,6 +1882,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/tui_system.rst against the later B6 changes to pyproject.toml, openhcs/pyqt_gui/main.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/tui_system.rst in full and compared it against the changed authorities: pyproject.toml, openhcs/pyqt_gui/main.py and tests/unit/pyqt_gui/test_ui_config_lifecycle.py; confirmed no Textual dependency, package or entry point exists, openhcs-gui still launches openhcs.pyqt_gui, the textual_tui guard test remains and docs/archive still exists; no prose change was needed.", "Re-read docs/source/architecture/tui_system.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: pyproject.toml; openhcs/pyqt_gui/main.py; tests/unit/pyqt_gui/test_ui_config_lifecycle.py.", @@ -1948,7 +1971,7 @@ }, { "path": "openhcs/pyqt_gui/services/ui_bridge_session_actions.py", - "sha256": "033aee9296f6f1c75f01d1fc3a39e5aeb0e7c13c15a7e4396872d3d014673123", + "sha256": "0d06eb67771e683beeb5c2ca93d60d4dfcd132e992371f9261bca15acf415675", "role": "SessionOperationActionProvider projects session operation classes as UI-bridge actions." }, { @@ -1958,7 +1981,7 @@ }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "e43085c1eaf0c836ce34e1a1dae7740488d91bdfea56844e7ddd4e6925561cf6", + "sha256": "00ff3bfe7a032bd3de59274697a69d12c9b898177cd8e839f9da3b130becaedf", "role": "OpenHCSMainWindow.set_ui_config consumer reconciliation and rollback, and preview-policy propagation to both managers." }, { @@ -1968,7 +1991,7 @@ }, { "path": "tests/unit/pyqt_gui/test_main_config_propagation.py", - "sha256": "303d90b43bee7bd531a1c7664fbf9707a2e7be1696a5147614f93c40e5c7021f", + "sha256": "499b05f1249ffc522bf18e97de05d1bf2000ab159daa0685958bd37d9dc62cc4", "role": "Verifies live preview policy propagation through the main configuration lifecycle." }, { @@ -2025,6 +2048,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/ui_services_architecture.rst against the later B6 changes to openhcs/pyqt_gui/services/ui_bridge_session_actions.py, openhcs/pyqt_gui/main.py, tests/unit/pyqt_gui/test_main_config_propagation.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/ui_services_architecture.rst in full and compared it against the changed authorities: runtime/zmq_config.py, services/window_handlers.py, pyqt_gui/main.py, ui_thread_dispatch.py, the two UI config tests, agent/ui_bridge_actions.py (deleted, U1) and services/desktop_restart.py (deleted); verified every named class exists, replaced MainWindowAction with session operations, moved DesktopRestartSession to openhcs/desktop/update.py, corrected the OpenHCSZMQConfig base, and restated Plate Manager as a renderer of Session state; UIConfig fields, set_ui_config ordering and application shutdown were confirmed in code.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", "tests/unit/test_ui_zmq_config_authority.py passed; source inspection confirmed ScopeWindowRegistry routing and OpenHCSZMQConfig inheritance", @@ -2090,7 +2114,7 @@ }, { "path": "openhcs/runtime/zmq_execution_server.py", - "sha256": "e24bdcb3b086cc60197ceb63f29a6f6b639a93cad175218941148a207eb285f2", + "sha256": "0f26779f2d31a4478b9bf9f5d8e4a5b02d8d2e346861f404e7cc35628e64e25e", "role": "OpenHCS compiled-execution adapter over the generic execution server." }, { @@ -2125,7 +2149,7 @@ }, { "path": "external/zmqruntime/src/zmqruntime/transport_modes.py", - "sha256": "12bec1c13d87bb8f05b049dc76d3cb7c130414c9e69d6220228ceeb0087f9ad8", + "sha256": "5f97b45a78b759e454e2dad260579af32fbb3be42a1dff8b724220709911507f", "role": "Owns transport-specific endpoint operations, including IPC deadlines and data/control pair availability." }, { @@ -2140,7 +2164,7 @@ }, { "path": "openhcs/runtime/zmq_execution_client.py", - "sha256": "147fb663d3ea60b1348063440e05b77d54b40634d1ae81cf4fb4f3b5457fbc99", + "sha256": "202619246805f8d2dfbfd5ca5427af34716e424f551ebc800d8516f7b49e07e5", "role": "Spawns the execution server with the ArrayBridge-projected child environment and carries typed execution transport." }, { @@ -2150,12 +2174,12 @@ }, { "path": "openhcs/authoring/session/submission.py", - "sha256": "710ac218d1db01f231b67bf0fe229dc03fb428bc6655b23671f3a026be91c673", + "sha256": "e04fdb017c3905903885b21007ebb4ceb530ec16267ea26577b3c9609b82e2ce", "role": "ExecutionSubmission records each accepted submission with its request and endpoint and follows status to terminal." }, { "path": "openhcs/authoring/session/session.py", - "sha256": "587047b63972df89ec46a01ed2c92ff3ef9577335f58a980297db02094e4e8e0", + "sha256": "458d71fe96ab75153977c554fb748e78f1773cd7794b625d7ec233c9b22e4453", "role": "Session owns the one execution client and keeps FinishedExecution (request, endpoint, ExecutionRecord) per execution." }, { @@ -2165,7 +2189,7 @@ }, { "path": "openhcs/runtime/zmq_orchestrator_environment.py", - "sha256": "5301811d7a37fa500f4c511ce9e57ac3620b172331f0ffc76a29e3a63797bf85", + "sha256": "04857f72c1e82381d7fab8c2cb1e881ff693803d93310cd39a34df442b023d72", "role": "Opens execution_subject_id when declared, otherwise subject_id." }, { @@ -2175,7 +2199,7 @@ }, { "path": "tests/unit/test_zmq_submission_timeout.py", - "sha256": "acea5dcb72c820ed8c1f225dc1960dfd615ad3f002ef6aec8e78b75bf8e50b3e", + "sha256": "1d6508d6aecdde924bc9ca4b9aed53cba740af14c44e712b84e1abc048e73da0", "role": "Proves a total submit deadline prevents execute dispatch after slow startup and preserves the declared connection inactivity timeout." }, { @@ -2195,22 +2219,22 @@ }, { "path": "openhcs/processing/backends/lib_registry/registry_service.py", - "sha256": "daef8925338fddf8504a9faa066af42456a13fe4602976bf64be06547a45fcc8", + "sha256": "46ad34dea67bb0f7ba54e1f9f79736654df60ceefe1ef4dd523a226d3222f289", "role": "Prepares the canonical catalogue in a child environment projected by ArrayBridge declarations." }, { "path": "openhcs/processing/backends/lib_registry/unified_registry.py", - "sha256": "b1e9253021b9c3ab14a070b7d0f10909f641ba07784c794ad95d2ce36041b1d3", + "sha256": "f2bf83e045d40391446ca7629c91f32fccad9a102f0d57cbb0157f780d630cbc", "role": "Processing-contract execution, registry-owned catalogue admission and cache semantics, and external-signature authoring exclusions." }, { "path": "openhcs/processing/backends/lib_registry/openhcs_registry.py", - "sha256": "3fff5c3549ef8e1073d75e6d5544b2cd75fbb7cc9b17263207760ffca45cf885", + "sha256": "7872cf8b59a954472871a72614d67bb2658cc2f4ce67e08dba1f8088589ed760", "role": "Derives the public native callable catalogue while retaining explicit reconstruction from each callable declaration." }, { "path": "tests/unit/test_library_registry_discovery.py", - "sha256": "e612d181d0053bacb38cb7bd3a73e8af8b3f7cdbe3789c1b7deb8f5f6a655170", + "sha256": "a67eb8d91a44a3eb347b46608679f2f8632abfd5b54c6f85315d1ea71a164460", "role": "Proves source-revision cache invalidation, stable optional-runtime admission, declaration-owned projection, runtime-owned external-signature exclusions, and no partial catalogue after failure." }, { @@ -2240,7 +2264,7 @@ }, { "path": "tests/unit/desktop/test_desktop_restart.py", - "sha256": "50425f39e481a6fff384b9732bc0f2cd68f223d3ac980234a9142b6db981b48f", + "sha256": "6181c22b42f7c9aa2f72b0a5a1d66bf9e5c7dfd8cba6af657851f27e5ac02af4", "role": "Proves the themed mismatch decision and state-preserving version-restart transaction." }, { @@ -2299,6 +2323,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/zmq_execution_service_extracted.rst against the later B6 changes to openhcs/runtime/zmq_execution_server.py, external/zmqruntime/src/zmqruntime/transport_modes.py, openhcs/runtime/zmq_execution_client.py, openhcs/authoring/session/submission.py, openhcs/authoring/session/session.py, openhcs/runtime/zmq_orchestrator_environment.py, tests/unit/test_zmq_submission_timeout.py, openhcs/processing/backends/lib_registry/registry_service.py, openhcs/processing/backends/lib_registry/unified_registry.py, openhcs/processing/backends/lib_registry/openhcs_registry.py, tests/unit/test_library_registry_discovery.py, tests/unit/desktop/test_desktop_restart.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/zmq_execution_service_extracted.rst in full and compared it against the changed authorities: zmq_execution_server.py, zmq_execution_observation.py, zmq_execution_client.py, compiled_execution.py, function_catalog_preparation.py, the zmqruntime execution/messages/client/transport/timeout owners, tests/unit/test_zmq_submission_timeout.py and agent/services/execution_session_service.py (deleted, U1); verified every named class and the --prepare-capabilities mode exist, rewrote the headless job, cancellation and execution-subject paragraphs for Session/ExecutionSubmission and subject_id/execution_subject_id, moved the submission budget to ExecutionTransportConfig, and confirmed the compatibility check before submit and the benchmark execution-ID check.", "Re-read docs/source/architecture/zmq_execution_service_extracted.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: external/zmqruntime/src/zmqruntime/execution/client.py; openhcs/runtime/zmq_execution_server.py; openhcs/core/compiled_execution.py; external/zmqruntime/src/zmqruntime/messages.py; external/zmqruntime/src/zmqruntime/execution/lifecycle.py; external/zmqruntime/tests/test_execution.py; external/zmqruntime/src/zmqruntime/client.py; external/zmqruntime/src/zmqruntime/transport_modes.py; external/zmqruntime/tests/test_transport.py; openhcs/runtime/zmq_application.py; openhcs/runtime/zmq_execution_client.py; external/zmqruntime/src/zmqruntime/timeouts.py; openhcs/agent/services/execution_session_service.py; tests/unit/test_zmq_submission_timeout.py; openhcs/runtime/function_catalog_preparation.py; openhcs/runtime/zmq_execution_server_launcher.py; tests/unit/test_zmq_execution_server_process.py; openhcs/processing/backends/lib_registry/registry_service.py; openhcs/processing/backends/lib_registry/unified_registry.py; openhcs/processing/backends/lib_registry/openhcs_registry.py; tests/unit/test_library_registry_discovery.py; openhcs/pyqt_gui/services/desktop_restart.py; tests/unit/test_zmq_application.py; tests/unit/pyqt_gui/test_desktop_restart.py; openhcs/processing/custom_functions/runtime_registry.py.", @@ -2376,7 +2401,7 @@ }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "e43085c1eaf0c836ce34e1a1dae7740488d91bdfea56844e7ddd4e6925561cf6", + "sha256": "00ff3bfe7a032bd3de59274697a69d12c9b898177cd8e839f9da3b130becaedf", "role": "Projects the configured endpoint snapshot into its dedicated right-hand indicator, and routes endpoint termination to the client lifecycle owner." }, { @@ -2406,7 +2431,7 @@ }, { "path": "openhcs/core/progress/runtime_tree.py", - "sha256": "016279f4135e2c9b10cb841e57ba0297c4197a195623f8a746ed90ab06533ce5", + "sha256": "27689de8557c87bcff5be713fa85ec0024171228c80b0cac6e86039114a18865", "role": "RuntimeExecutionTopology, RuntimeTreeProjectionBuilder and the subject/worker/partition/step/compilation node kinds with mean or explicit percent policies." } ], @@ -2416,6 +2441,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/zmq_server_browser_system.rst against the later B6 changes to openhcs/pyqt_gui/main.py, openhcs/core/progress/runtime_tree.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/zmq_server_browser_system.rst in full and compared it against the changed authorities: server_browser/progress_projection.py, server_browser/live_tree_sync.py, pyqt_gui/main.py, tests/unit/pyqt_gui/test_main_zmq_window_routing.py, zmqruntime client.py and services/zmq_client_service.py (moved, U1); verified every named class and module path exists, corrected the progress subscriber owner and the subject/worker/partition/step tree path and aggregation kinds from core/progress/runtime_tree.py, and confirmed the main window still projects the endpoint snapshot into its indicator and manager button states.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", "The complete pyqt-reactive suite passed 320 tests, including shared non-overlapping browser and log-viewer scan lifecycle coverage.", @@ -2469,7 +2495,7 @@ }, { "path": "tests/unit/test_installed_mcp_demo.py", - "sha256": "8babc443f7ec59cebebd3772633dde9ae9c49b125066cd5e7c7d527dc74c2a09", + "sha256": "5f62d65c0add0936583e8e78bf65bf407ed304d5be15b38be4963332a5fe7914", "role": "Proves the installed demo runs the session journey and validates settled viewer state." }, { @@ -2481,6 +2507,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/agent_workflow_validation.rst against the later B6 changes to tests/unit/test_installed_mcp_demo.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/agent_workflow_validation.rst in full and compared it against the changed authorities: openhcs/mcp/client_registration.py, synthetic_plate_service.py, tests/unit/test_installed_mcp_demo.py, openhcs/demo/synthetic_data.py, openhcs/mcp/installed_demo.py; the client skill sync and session-based execute path do not touch the protocol, and the fixture request values, the registry the page names, the time-derived HTD identifier, the ZMQ Server Manager and Live Results windows and the retained baseline record all still match; updated the test's role because its old progress-polling watchdog tests were replaced by the session journey.", "Re-read docs/source/development/agent_workflow_validation.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/mcp/client_registration.py; openhcs/agent/services/synthetic_plate_service.py; website/assets/agent/cold-start-workflow-record.json; tests/unit/test_installed_mcp_demo.py; openhcs/demo/synthetic_data.py.", @@ -2535,42 +2562,42 @@ "authority": [ { "path": "openhcs/core/callable_contract.py", - "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", + "sha256": "2060f856bcb0cc8ef57b92fc17a1e558a676e4033075950b1a4e41dc8656d330", "role": "Declares CallableContract fields (required_axis_roles, allowed_group_by_roles, memory roles), resolve_returned_output, the return partition and the artifact-parameter binding errors quoted by the page." }, { "path": "openhcs/core/artifacts.py", - "sha256": "65eb0ec5c903366243ca8bd5ea07412e8d34bdb81e6015fa76b3c4c59838733f", + "sha256": "907551ae0ce4bf42a557554f5df0252b0a88fc9b7c1b7ac19705aade66b6656f", "role": "Declares ArtifactSpec, MainFlowStackOutputSpec, the artifact types, ObjectMeasurementSubjectRelation and MaterializationSourceIdentityRelation." }, { "path": "openhcs/core/processing_contracts.py", - "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", + "sha256": "9b999e9210baf292f8fbf5bb3758c41a95e74401bfb1e1c127bae2dba625b0ee", "role": "Declares Pure2DContract and Pure3DContract used in the examples." }, { "path": "openhcs/core/pipeline/function_contracts.py", - "sha256": "13bcaf7c7a62654f512d2cc739edccd2deeff35d99970ca493f3e827afbda91a", + "sha256": "883d22780ab41f122d6ed651c9f59fea540120f8fcf50d30e348ac295bd4fca9", "role": "Declares artifact_inputs, artifact_outputs, special_inputs, execution_scope and runtime_bound_parameters." }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "b6a7231bc2964d472070731018f6a4935e5f23cf8f238a232045c32244e6545a", + "sha256": "628a27fea26ed971cfdc4ab22d11df5491f21eb49b6b90397749b46e73e8145b", "role": "Declares RuntimeArtifactBatch.records and StoredRuntimeValue for plate-scope callables." }, { "path": "openhcs/core/orchestrator/bundle_execution.py", - "sha256": "f5ba09a2ec7c688b1ca08167ab398cc976a9e2c39b301ae98f3c5fd560a25c85", + "sha256": "7665bd4ab8b9ce73d3dcb4fb9679e836b23bffb81d70541baef2e114ee54b0d9", "role": "Requires a plate-scope callable to declare exactly one SpecialArtifactType output." }, { "path": "openhcs/processing/materialization/core.py", - "sha256": "744573f1bafa67b7a81be5af425012ada47741292519863e250b79f1c778b265", + "sha256": "1e408b7b7c4ac406f5bce8a1fda8a8774cb049dbc2a646832b8c4d008ae2b5c3", "role": "Point ROI writer: rejects missing fields, duplicate labels, non-finite coordinates and an empty archive." }, { "path": "openhcs/interop/cellprofiler/module_artifact_contracts.py", - "sha256": "982680cd36409ea6d90a07d4e60893b162847f5525ed516891e995877dc32fd1", + "sha256": "c9f1210146b23991dd5433ae2acac43c972b95cc95cf9bd785fac19f9d810928", "role": "Declares CellProfilerModuleArtifactContracts resolving setting-backed specs into the invocation CallableContract." }, { @@ -2585,7 +2612,7 @@ }, { "path": "tests/unit/test_callable_contract.py", - "sha256": "cc220f53107a08b0406b6e0dd49896e3e4c3c0af6e527b0ef00d4fdaba8f0b2c", + "sha256": "b0ce0cba2e396e53f8cb9e95533ff918788e8c6dd60037a66e0aeda603bc64c2", "role": "Proves exact enum validation and contract-derived runtime-owned parameter exclusions." } ], @@ -2597,6 +2624,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/callable_artifact_authoring.rst against the later B6 changes to openhcs/core/callable_contract.py, openhcs/core/artifacts.py, openhcs/core/processing_contracts.py, openhcs/core/pipeline/function_contracts.py, openhcs/core/runtime_stores.py, openhcs/core/orchestrator/bundle_execution.py, openhcs/processing/materialization/core.py, openhcs/interop/cellprofiler/module_artifact_contracts.py, tests/unit/test_callable_contract.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/callable_artifact_authoring.rst in full and compared it against the changed authorities: the page, openhcs/core/callable_contract.py, openhcs/core/artifacts.py, module_artifact_contracts.py, tests/unit/test_artifact_spec_collection.py, the ArrayBridge types and decorators, tests/unit/test_callable_contract.py; grepped every API the page names (SettingToKeywordBinding, SelectedPlaneImageOutput, canonical/trailing return specs, RuntimeArtifactBatch.records, PointROIOptions features, the custom-function capability and guide names) and the quoted error texts, confirmed the plate executor's single SpecialArtifactType rule and the empty-Points rejection, corrected the axis-role and processing-contract names, and added processing_contracts.py, function_contracts.py, runtime_stores.py, bundle_execution.py and materialization/core.py as owners while dropping arraybridge/types.py.", "Re-read docs/source/development/callable_artifact_authoring.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/callable_contract.py; openhcs/core/artifacts.py; openhcs/interop/cellprofiler/module_artifact_contracts.py; tests/unit/test_artifact_spec_collection.py; external/arraybridge/src/arraybridge/types.py; external/arraybridge/src/arraybridge/decorators.py; tests/unit/test_callable_contract.py.", @@ -2623,17 +2651,17 @@ "authority": [ { "path": "openhcs/interop/cellprofiler/module_declarations.py", - "sha256": "6bbf41c158496a02439be2ee5aef12d7712fb632419c226a64c9259a55438b95", + "sha256": "493efb9c749b10ba54713d0c16ee7118f07ff35afbafc9ff156b7100dbd63b19", "role": "CellProfiler module registry, catalogue placement, and headless-import eligibility." }, { "path": "openhcs/interop/cellprofiler/module_artifact_contracts.py", - "sha256": "982680cd36409ea6d90a07d4e60893b162847f5525ed516891e995877dc32fd1", + "sha256": "c9f1210146b23991dd5433ae2acac43c972b95cc95cf9bd785fac19f9d810928", "role": "Invocation-specific CellProfiler artifact contract derivation." }, { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "3439d96f3fbfcb104a53019dc150049021d9a8132bd27096f1ccf510eefbe37b", + "sha256": "5d3646e17f7de843a5b36164f7cef7505da7841636423ae4fb4e00d194a87e9c", "role": "Public CellProfiler pipeline lowering and declaration-owned headless-import validation." }, { @@ -2648,7 +2676,7 @@ }, { "path": "openhcs/processing/backends/lib_registry/openhcs_registry.py", - "sha256": "3fff5c3549ef8e1073d75e6d5544b2cd75fbb7cc9b17263207760ffca45cf885", + "sha256": "7872cf8b59a954472871a72614d67bb2658cc2f4ce67e08dba1f8088589ed760", "role": "Derives the public native callable catalogue while retaining explicit reconstruction from each callable declaration." }, { @@ -2660,6 +2688,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/cellprofiler_module_authoring.rst against the later B6 changes to openhcs/interop/cellprofiler/module_declarations.py, openhcs/interop/cellprofiler/module_artifact_contracts.py, openhcs/interop/cellprofiler/pipeline_import.py, openhcs/processing/backends/lib_registry/openhcs_registry.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/cellprofiler_module_authoring.rst in full and compared it against the changed authorities: module_declarations.py (axis-family imports, MeasurementDialect, lazy registry preparation, private _require_declared_callable with object-identity check, prepare_declared_callable), module_artifact_contracts.py (target_units_for_invocation replaces target_contract_for_invocation), pipeline_import.py, tests/unit/test_cellprofiler_pipeline_import.py and openhcs_registry.py (LazyDiscoveryDict declaration discovery, declared_callable_contract). Confirmed every name the page relies on still exists at its owner: AutoRegisterMeta, CellProfilerModule.__registry__, for_callable_contract (import identity then object identity), declared_function_names and the lazy CellProfilerBackendModule projection, validate_pipeline_import called by pipeline_import before lowering, CellProfilerModuleArtifactContracts, SettingToKeywordBinding.input/output, module_blocks_for_invocation, SourceBindingsConfig, ModuleBlock and CallableContract; no page change was needed.", "Re-read docs/source/development/cellprofiler_module_authoring.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/interop/cellprofiler/module_declarations.py; openhcs/interop/cellprofiler/module_artifact_contracts.py; openhcs/interop/cellprofiler/pipeline_import.py; tests/unit/test_cellprofiler_pipeline_import.py; tests/unit/test_cellprofiler_public_boundary_contracts.py; openhcs/processing/backends/lib_registry/openhcs_registry.py; openhcs/processing/backends/cellprofiler/__init__.py.", @@ -2683,22 +2712,22 @@ "authority": [ { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", + "sha256": "90d75f6287c5f21f93d82960729199261f7906e0f4912b7728527260b9723a66", "role": "ResolvedPipelineDefinition and CompilationSession fields, one-time artifact admission before axis fan-out, and the missing-plan failure of plan(index)." }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "sha256": "d1216010dec1804c51aec9c774f298513638ca016d500fe6328d76a9a9bfb4d6", "role": "Compilation order, declaration resolution, callable warm-up, framework-local device assignment, and bundle construction." }, { "path": "openhcs/core/pipeline/path_planner.py", - "sha256": "677d106969aae4d818745bff29b6f84e487ec79a044c8eaf26bea1916236fc6c", + "sha256": "bc1380d505dae574c6dc1255afcba59b0513d933716017c05217baabed104d24", "role": "Compiled source satisfaction, artifact edge, path, grouping, and plan decisions." }, { "path": "tests/unit/test_compilation_session.py", - "sha256": "381adf3a87729928a225fd1db5a35f8c802340fb2a1e583501884dc5500f6b9d", + "sha256": "044f6224de217df92c553e216be50c4ac023d69b2a92b835d59a3a781ed432a2", "role": "Regression proof that axis sessions share one resolved pipeline definition while owning their own plans." }, { @@ -2708,7 +2737,7 @@ }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", + "sha256": "f4a851d7de2ea82e1d548f37e6be2511fd20d799580193496a3746e5484906ba", "role": "Typed per-step framework-device binding authority." }, { @@ -2732,6 +2761,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/compiler_extension.rst against the later B6 changes to openhcs/core/pipeline/compilation_session.py, openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/path_planner.py, tests/unit/test_compilation_session.py, openhcs/core/compiled_step_plan.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/compiler_extension.rst in full and compared it against the changed authorities: compilation_session.py (StepSnapshot and snapshot validation deleted; ResolvedPipelineDefinition and CompilationDatasetScope added), compiler.py, path_planner.py, tests/unit/test_compilation_session.py, arraybridge types.py, compiled_step_plan.py and framework_device_assignment.py. Rewrote the Invariants section for the current session model; confirmed CompiledStepPlan, ArtifactSpecRef, assign_framework_devices, FrameworkDeviceResolver and the absence of a global GPU scheduler.", "Re-read docs/source/development/compiler_extension.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/pipeline/compilation_session.py; openhcs/core/pipeline/compiler.py; openhcs/core/pipeline/path_planner.py; tests/unit/test_compilation_session.py; external/arraybridge/src/arraybridge/types.py; openhcs/core/compiled_step_plan.py; openhcs/core/pipeline/framework_device_assignment.py; openhcs/core/framework_device_resolver.py; tests/unit/test_framework_device_resolver.py.", @@ -2816,7 +2846,7 @@ }, { "path": "pyproject.toml", - "sha256": "3592cc37444e2e1f5bb0284e939643039c0d513217861af8133dd8bd6e8a4325", + "sha256": "c3a2bea22aa6872940125f4aaac72c9eac72084fefc6c2cefdb54b5273361e4a", "role": "Current Python support, dependency, package discovery, and entry-point authority." }, { @@ -2831,7 +2861,7 @@ }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", + "sha256": "90d75f6287c5f21f93d82960729199261f7906e0f4912b7728527260b9723a66", "role": "ResolvedPipelineDefinition and CompilationSession consumed by compiler phases." }, { @@ -2841,12 +2871,12 @@ }, { "path": "openhcs/core/callable_contract.py", - "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", + "sha256": "2060f856bcb0cc8ef57b92fc17a1e558a676e4033075950b1a4e41dc8656d330", "role": "OpenHCS callable declaration projection for input, output, and execution frameworks." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", + "sha256": "f4a851d7de2ea82e1d548f37e6be2511fd20d799580193496a3746e5484906ba", "role": "Typed per-step framework-device binding authority." }, { @@ -2856,7 +2886,7 @@ }, { "path": "benchmark/cellprofiler_benchmark_cli.py", - "sha256": "4025c591644e29f3df61e476ecae2143690e014336ea9f889aadeff4e692445e", + "sha256": "edc32916215a88b62702621dea9b692586572284caf9b781394b5784f62897ee", "role": "Owns registered benchmark command declarations and the derived installed parser." }, { @@ -2866,7 +2896,7 @@ }, { "path": "tests/unit/test_benchmark_control.py", - "sha256": "093b584eeb37cbb12089d5af082ad45acd779e9d2bdd5b3feaea08234fc22183", + "sha256": "2798cf80df1222c80bb3aa9955b5c20c0c90e415a1795017779d22460f442bf2", "role": "Proves registered CLI derivation and declaration-derived receipt construction." } ], @@ -2877,6 +2907,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/extension_workflows.rst against the later B6 changes to pyproject.toml, openhcs/core/pipeline/compilation_session.py, openhcs/core/callable_contract.py, openhcs/core/compiled_step_plan.py, benchmark/cellprofiler_benchmark_cli.py, tests/unit/test_benchmark_control.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/extension_workflows.rst in full and compared it against the changed authorities: deleted openhcs/core/registry_strategies.py (strategy mixins moved to metaclass-registry, b221f8491), deleted openhcs/serialization/json.py (python-introspect codec, c0a53a1e0), compilation_session.py (StepSnapshot removed), arraybridge types.py, callable_contract.py, compiled_step_plan.py, framework_device_assignment.py, the benchmark CLI and run-receipt modules, and pyproject.toml. Corrected the StepSnapshot reference and the to_jsonable owner; confirmed ImageFileFormat, BenchmarkCliCommand, ComparisonSuiteRunDeclaration, ComparisonSuiteRunReceipt, MeasuredPipelineRunReceipt, ArtifactKey, CompiledExecutionBundle and MemoryType.import_environment exist at their owners.", "Re-read docs/source/development/extension_workflows.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/registry_strategies.py; pyproject.toml; openhcs/core/pipeline/compilation_session.py; openhcs/serialization/json.py; external/arraybridge/src/arraybridge/types.py; openhcs/core/callable_contract.py; openhcs/core/compiled_step_plan.py; openhcs/core/pipeline/framework_device_assignment.py.", @@ -2920,7 +2951,7 @@ }, { "path": "pyproject.toml", - "sha256": "3592cc37444e2e1f5bb0284e939643039c0d513217861af8133dd8bd6e8a4325", + "sha256": "c3a2bea22aa6872940125f4aaac72c9eac72084fefc6c2cefdb54b5273361e4a", "role": "Current Python support, dependency, package discovery, and entry-point authority." }, { @@ -2939,6 +2970,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/git_worktree_testing.rst against the later B6 changes to pyproject.toml: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/git_worktree_testing.rst in full and compared it against the changed authorities: pyproject.toml (first-party floors moved to zmqruntime 0.6, objectstate 1.3.1, python-introspect 0.2.5, metaclass-registry 0.3, arraybridge 0.4.1, polystore 0.6, pyqt-reactive 0.4; JAX extra now jax[cuda12-local]; openhcs-basicpy added); the page states no version floors. Checked the eight editable installs against .gitmodules, the dev and gui extras, tests/integration/test_main.py, and every --it-* option and value against tests/pytest_integration_options.py; corrected OPENHCS_CPU_ONLY=1 to OPENHCS_CPU_ONLY=true to match tests/conftest.py.", "Re-read docs/source/development/git_worktree_testing.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: .gitmodules; pyproject.toml; tests/pytest_integration_options.py.", @@ -2974,7 +3006,7 @@ "authority": [ { "path": "pyproject.toml", - "sha256": "3592cc37444e2e1f5bb0284e939643039c0d513217861af8133dd8bd6e8a4325", + "sha256": "c3a2bea22aa6872940125f4aaac72c9eac72084fefc6c2cefdb54b5273361e4a", "role": "Current Python support, dependency, package discovery, and entry-point authority." }, { @@ -2988,6 +3020,7 @@ ], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/index.rst against the later B6 changes to pyproject.toml: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/index.rst in full and compared it against the changed authorities: pyproject.toml (first-party dependency floors, JAX extra, openhcs-basicpy) and openhcs/core/compiled_execution.py; the page states neither versions nor compiled-bundle facts. Confirmed every toctree target exists and the recursive-clone guidance matches .gitmodules; dropped the unrelated compiled_execution.py and self-referential page authorities.", "Re-read docs/source/development/index.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: pyproject.toml; openhcs/core/compiled_execution.py; docs/source/development/index.rst.", @@ -3025,7 +3058,7 @@ "authority": [ { "path": "scripts/capture_media_gallery.py", - "sha256": "f4808f79d960a5cfcdb40edd37780120189f1c21e833e51f5a4878af36fec9f1", + "sha256": "00f9463a892bdf12845887641053dc72715a8964c90e2aa2e410deb42314efe7", "role": "Owns the typed capture manifest, host-tool discovery, trusted UI-bridge session, declaration-selected contextual fixture preparation, X11 capture, deterministic derivative construction, validation, and command-line surface." }, { @@ -3052,6 +3085,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/media_gallery_capture.rst against the later B6 changes to scripts/capture_media_gallery.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/media_gallery_capture.rst in full and compared it against the changed authorities: scripts/capture_media_gallery.py (UI-bridge calls now go through UiBridgeService.invoke with typed operation classes, to_jsonable moved to python_introspect), scripts/gallery_catalog.py (contextual actions are SessionOperation classes such as ShowDatasetImages and ShowLiveResults instead of PlateManagerAction values; StepEditorScope moved to openhcs.authoring.session.step_scopes) and tests/unit/test_capture_media_gallery.py (native-frame snapshot requests). None of these internals are named by the page; re-checked every subcommand and option (doctor, catalog, plan, build, validate, capture-still, capture-scenario-still, capture-ui-window-references, capture-ui-context-references with --fixture-root, record-window with --no-mouse), manifest schema_version 2, the 30-second trim limit, the documentation-fixture preparation path, the UI-bridge environment variables, the multi-plate-overview and lazy-inheritance scenario ids, and gallery_catalog --check; all remain accurate.", "Re-read the complete how-to against scripts/capture_media_gallery.py, scripts/gallery_catalog.py, scripts/website_gallery_projection.py, and tests/unit/test_capture_media_gallery.py; every capture, transformation, ownership, validation, and no-overwrite claim matches its current owner.", "Passed python -m scripts.capture_media_gallery catalog and verified the declaration-owned scenario inventory; python -m scripts.gallery_catalog --check confirmed the checked-in release projection.", @@ -3086,7 +3120,7 @@ }, { "path": "openhcs/mcp/dev_client_core.py", - "sha256": "41739b744b0e4287ca7d14cd133371d669e9bff12f4b622313996ae9bb5597da", + "sha256": "b5b1aea8a5a9352d828efd408408cb0ce867db4c5dc4f91e1fa9fbb7947013b2", "role": "MCP dev-client session, selected-workflow argument projection, revision-token terminal-state criteria with idempotent initialization, and transport timeout." }, { @@ -3116,12 +3150,12 @@ }, { "path": "openhcs/agent/services/ui_bridge_service.py", - "sha256": "c7bdf8b5caec1c21c986cdc2224524839fadfa8764dafcd05125d90c72351899", + "sha256": "dc2c4823cc9a611d384bba2785eb6e1b75338f0ec4f731292515bfded2218d63", "role": "Declares the default UI bridge connection spec whose timeout bounds UI development commands." }, { "path": "openhcs/mcp/dev_client_commands/ui.py", - "sha256": "be3bc9096e4a711cc6583667596ba25a60428711f2883248e82d5ac66d6dd28b", + "sha256": "4ee8f3a24dd24ed49a3489d7c195a0c620fc2daa7100ec04ce2c4e9abafd30aa", "role": "Owns declaration-derived UI development commands, including selected-workflow ordering and the typed output-directory projection for window snapshots." }, { @@ -3131,12 +3165,12 @@ }, { "path": "openhcs/agent/capabilities.py", - "sha256": "eaebe33af01d057d1d21b3701f958bc96ede3ec5d0bddddca9917c1164ef0042", + "sha256": "87c8e2e5aeb425b9a2eb690d48bdd6f5af7a327c389deac261d7e4e0da5d609f", "role": "Projects each SessionOperation as an openhcs_ tool and declares openhcs_session_events, openhcs_session_datasets and the local surface profiles." }, { "path": "openhcs/authoring/session/session.py", - "sha256": "587047b63972df89ec46a01ed2c92ff3ef9577335f58a980297db02094e4e8e0", + "sha256": "458d71fe96ab75153977c554fb748e78f1773cd7794b625d7ec233c9b22e4453", "role": "Session submission budget and runtime-observation export validation (writable root, no existing path, outcomes require a path)." }, { @@ -3146,7 +3180,7 @@ }, { "path": "tests/unit/agent/test_mcp_server.py", - "sha256": "4e6f6b1761b2c0316c99984baad6ea2c97e5c76c3f155111b8e498d7358e3175", + "sha256": "6a03cf2de856060f39d221127f41189a0a8094c65856cb1c4cfa765210099b67", "role": "Regression proof for selected-surface preflight, stale-row rejection, idempotent initialization, and every declared or malformed bridge-receipt continuation outcome." }, { @@ -3171,12 +3205,12 @@ }, { "path": "openhcs/runtime/zmq_execution_client.py", - "sha256": "147fb663d3ea60b1348063440e05b77d54b40634d1ae81cf4fb4f3b5457fbc99", + "sha256": "202619246805f8d2dfbfd5ca5427af34716e424f551ebc800d8516f7b49e07e5", "role": "Owns the total headless submission budget and the pre-dispatch timeout boundary documented for MCP developers." }, { "path": "tests/unit/test_zmq_submission_timeout.py", - "sha256": "acea5dcb72c820ed8c1f225dc1960dfd615ad3f002ef6aec8e78b75bf8e50b3e", + "sha256": "1d6508d6aecdde924bc9ca4b9aed53cba740af14c44e712b84e1abc048e73da0", "role": "Regression proof that deadline expiry after slow connection prevents execute dispatch." }, { @@ -3191,12 +3225,12 @@ }, { "path": "tests/unit/pyqt_gui/test_ui_agent_bridge.py", - "sha256": "de6758859d9743e826369162855327e4a84056cafdd7ddfdc33868d2a3093353", + "sha256": "b03744b1fa201f77de26d2e6050422b6422ebce78443c478c5e8c5f55e0a3e24", "role": "Proves stale protocol and application versions fail before UI operation dispatch." }, { "path": "openhcs/mcp/server.py", - "sha256": "e1f236011df715da3d17574c0b80cbf50fcae45e3f82491c22fb57e98c6bb662", + "sha256": "927e64594a0fc174759eeb58496fed0ccf95e5d8d29bc248cde9a033efb69fbc", "role": "Owns the mcp_server_stale guard, declaration-derived progress execution, and verbose delayed thread-stack diagnostics across the full invocation." }, { @@ -3212,6 +3246,7 @@ ], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/mcp_development.rst against the later B6 changes to openhcs/mcp/dev_client_core.py, openhcs/agent/services/ui_bridge_service.py, openhcs/mcp/dev_client_commands/ui.py, openhcs/agent/capabilities.py, openhcs/authoring/session/session.py, tests/unit/agent/test_mcp_server.py, openhcs/runtime/zmq_execution_client.py, tests/unit/test_zmq_submission_timeout.py, tests/unit/pyqt_gui/test_ui_agent_bridge.py, openhcs/mcp/server.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/mcp_development.rst in full and compared it against the changed authorities: the page's own U1 update (session-operation tools replacing the job-status paragraph; selected-workflow initialize_datasets), dev_client.py, dev_client_commanding.py, dev_client_core.py, zmq_config.py, knowledge_pipeline.py, agent/dto/ui_bridge.py, agent/dto/viewer.py, dev_client_commands/ui.py and viewer.py, bootstrap.py, stdio.py, zmq_execution_client.py, ui_bridge_transport.py, ui_bridge_server.py, progress_stream.py, agent/dto/execution.py, server.py, the deleted execution_session_service.py and their tests. Confirmed every dev-client command and option in the examples, the full/desktop/core/authoring surface names, initialize_datasets as a UiSelectedPlateWorkflowKind value, the revision-token and idempotent-initialization wait criteria, the 5000 ms UI and viewer control bounds, the 30000 ms submission default and pre-dispatch timeout error, the openhcs_ session tools, event_sequence, and the runtime-observation export checks; replaced the deleted and no-longer-relevant authorities.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", "Source search confirmed every documented dev-client command; persistent-session and MCP-server parser tests cover the command shapes", @@ -3275,12 +3310,12 @@ }, { "path": "openhcs/agent/capabilities.py", - "sha256": "eaebe33af01d057d1d21b3701f958bc96ede3ec5d0bddddca9917c1164ef0042", + "sha256": "87c8e2e5aeb425b9a2eb690d48bdd6f5af7a327c389deac261d7e4e0da5d609f", "role": "Declares the openhcs://knowledge resource and the three knowledge tools with their knowledge, knowledge-document, and knowledge-search dev-client commands." }, { "path": "tests/unit/agent/test_knowledge_base_service.py", - "sha256": "ce4636580893a8736fd4607c6ed0f431d25c97e0c44e2b95912af3fc36b1d750", + "sha256": "67a326b8473d18cc0596aa275ad2fa1e91261a24d5083ed332bc32335aeefaee", "role": "Regression proof for document IDs, search, bounded reads, and section selection." }, { @@ -3292,6 +3327,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/mcp_knowledge_base.rst against the later B6 changes to openhcs/agent/capabilities.py, tests/unit/agent/test_knowledge_base_service.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/mcp_knowledge_base.rst in full and compared it against the changed authorities: knowledge_base_service.py (comparison-manifest parsing moved to openhcs.agent.knowledge_manifest_schema.ComparisonManifestSnapshot; per-case module inventories are now cached by source mtime), knowledge_manifest.py (comparison_manifest_type_for_root) and both test files. Confirmed the generated Python conversion is still recomputed per request and never stored (the new cache holds only module-name inventories), the 12,000 default bound, the four MCP surface names and dev-client commands with --limit, --max-chars and --section-id, the four document ids in mcp_knowledge_base_manifest.json, benchmark/manifests/official30_portable_axis1.json, the first-use capability names, and every cited documentation path; no page change was needed. Added openhcs/agent/capabilities.py for the surface names.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", "tests/unit/agent/test_knowledge_base_service.py passed; source inspection confirmed lazy selected-section Official30 conversion", @@ -3366,7 +3402,7 @@ }, { "path": "tests/unit/test_mcp_release_workflow.py", - "sha256": "5433d76ab87c00f0047e43b65c7a590aab7a8e397a0a1fd7cf6dc3b01e062854", + "sha256": "d4d67e9591e7652fe00558337627d18bc9547a43af2a946e8ac671b583c90dd4", "role": "Proves immutable tag resolution, exact-SHA checkout, publication ordering, recovery, Registry policy, and publication-independent native candidate ownership." }, { @@ -3421,12 +3457,12 @@ }, { "path": "scripts/smoke_installed_gui.py", - "sha256": "d9bf24b9e5c631b4eb6a3de0fc7a79a5ee56a848693ed6692b60dc9c570d4efc", + "sha256": "a018e473245cee30747f86e85ab0410680bbab248753d20252cff727b22d381a", "role": "Owns the installed-GUI acceptance timing and single terminal completion, applies the declared logging configuration, constructs the application on declaration-allocated endpoints, proves the background function catalogue and advertised endpoint log are usable, drives the GUI through packaged MCP, validates retained window snapshots, and cleans up exact owned resources outside the checkout." }, { "path": "tests/unit/test_smoke_installed_gui.py", - "sha256": "b8d1a1fcd67c58e2b2749262b44c1dcccbb2e72a0d299d180823e94a0285ae9d", + "sha256": "2178b0b6cfcae83dc08574573e1b021b63e2390436fffb73cafea969ad415393", "role": "Proves source-checkout rejection, one derived smoke timing policy, deadline-owner error preservation under late cancellation, declaration-owned endpoint allocation, function-catalogue readiness, advertised execution-log validation, snapshot CLI projection, and emitted PNG identity validation for the installed GUI/MCP probe." }, { @@ -3471,6 +3507,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/mcp_release.rst against the later B6 changes to tests/unit/test_mcp_release_workflow.py, scripts/smoke_installed_gui.py, tests/unit/test_smoke_installed_gui.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/mcp_release.rst in full and compared it against the changed authorities: openhcs/__init__.py (version 0.8.7, microscopy axis-family activation; the version-authority claim holds), .github/workflows/publish.yml and tests/unit/test_mcp_release_workflow.py (new build-native-wheels job, sdist built separately, per-wheel validation, Linux x86_64 smoke selection), .github/workflows/integration-tests.yml (parity-scope gate, Official30 parity on Python 3.12 Linux and 3.14 on all three OSes, run_installed_tests.py coverage), scripts/resolve_successful_job_baseline.py (branch filter removed; ancestor rule unchanged), scripts/smoke_installed_gui.py (typed MCP payloads, ShowDatasetCode), tests/installer/test_windows_simple_installer.py, tests/installer/test_installer_signing.py, tests/unit/test_validate_local_release_floors.py and tests/unit/test_ci_package_boundaries.py. Corrected the single-wheel publication and recovery descriptions and the GUI smoke action name; confirmed the Python 3.14 wheel-candidate lane still excludes the Centrosome oracle and the remaining gate, recovery and signing claims.", "Re-read docs/source/development/mcp_release.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/__init__.py; scripts/sync_mcp_release_metadata.py; scripts/release.py; scripts/release_readiness.py; scripts/validate_local_release_floors.py; scripts/install_ci_candidate.py; .github/workflows/publish.yml; tests/unit/test_mcp_release_workflow.py; tests/unit/test_release_readiness.py; .github/workflows/integration-tests.yml; .github/dependabot.yml; tests/unit/test_ci_action_pinning.py; packaging/installers/windows/Sign-Installer.ps1; packaging/installers/macos/Sign-Installer.sh; tests/installer/test_installer_signing.py.", @@ -3550,7 +3587,7 @@ "authority": [ { "path": "external/PolyStore/src/polystore/omero_local.py", - "sha256": "35d39564317510234c2a3ded00bb06a09417d1b6f7359e52e74c7bd0a086f642", + "sha256": "e549d4811539cec9842ceac007eb666380ef00d6838a9b69d91eb02de64d1daa", "role": "Generic OMERO addressing, persistence dispatch, and image-plane batching." }, { @@ -3602,6 +3639,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/omero_testing.rst against the later B6 changes to external/PolyStore/src/polystore/omero_local.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/omero_testing.rst in full and compared it against the changed authorities: tests/integration/test_main.py (partition-filter config renames, Microscopy axis family, DatasetSourceChoice, per-test allocated ports, execute_compiled_bundle, subject_id, PipelineDocumentCodec) and .github/workflows/integration-tests.yml (parity-scope gate, run_installed_tests.py coverage). The page names none of the renamed test internals; confirmed the harness still registers OMEROLocalBackend and runs direct and ZMQ modes without opening a browser, the omero-tests-linux lane still builds submodule candidate wheels without a publication dependency and keeps Tables-0 and component diagnostics, and OMEROWellAddress, OMEROPlaneAddress, OMEROPlaneFilenameTemplate and FilenameParseResult exist at their owners.", "Re-read docs/source/development/omero_testing.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: external/PolyStore/src/polystore/omero_local.py; external/PolyStore/tests/test_omero_tables.py; tests/integration/test_main.py; openhcs/omero/docker-compose.yml; openhcs/runtime/omero_instance_manager.py; .github/workflows/integration-tests.yml.", @@ -3709,7 +3747,7 @@ "authority": [ { "path": "openhcs/core/debug.py", - "sha256": "95d365afefc46ee50da5abb8d5d246b3980feb32b85839f91e874cb7e689caaa", + "sha256": "4f02b36c242b7db3c4e0fa2060a852e11ded2ab497abde47494cba41ae7afdca", "role": "Nominal debug command, boundary, snapshot, replay, and paused-worker declarations." }, { @@ -3724,13 +3762,14 @@ }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "sha256": "d1216010dec1804c51aec9c774f298513638ca016d500fe6328d76a9a9bfb4d6", "role": "Compilation order, declaration resolution, callable warm-up, framework-local device assignment, and bundle construction." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/pipeline_debugging_guide.rst against the later B6 changes to openhcs/core/debug.py, openhcs/core/pipeline/compiler.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/pipeline_debugging_guide.rst in full and compared it against the changed authorities: openhcs/core/debug.py (DebugJsonCodec removed, DebugProgressContext now a declared progress-context payload, plate_id renamed subject_id), openhcs/agent/services/runtime_server_service.py and openhcs/core/pipeline/compiler.py (CompilationPlateScope and plate_path renamed to dataset scope and dataset_root). The page names none of the renamed fields; confirmed openhcs_inspect_pipeline_source_artifact_plan, openhcs_inspect_debug_runtime_values and its runtime-debug-values command, debug_session_id, pipeline_debug_toolbar, FunctionStep.debug_pause, the Run to Pause, Step and Inspect Runtime actions, StepMaterializationConfig, RuntimeValueStore, CompiledStepPlan, CompilationSession and CompiledExecutionBundle still exist with the described roles.", "Re-read docs/source/development/pipeline_debugging_guide.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/debug.py; openhcs/core/debug_session_projection.py; openhcs/agent/services/runtime_server_service.py; openhcs/core/pipeline/compiler.py.", @@ -3761,7 +3800,7 @@ }, { "path": "openhcs/processing/backends/processors/pyclesperanto_processor.py", - "sha256": "ae7b91db142b898f47831fcb3e44c67758e9e2f8c2d77ada3b347dadbc34af47", + "sha256": "d9c2d68eae058664321622ffa79113ee741cdb22a5eec6045a1ca537a95f622b", "role": "OpenHCS processing callables using the pyclesperanto integration." }, { @@ -3776,17 +3815,17 @@ }, { "path": "openhcs/core/memory/decorators.py", - "sha256": "76f234785f4865257fd19c935e0aace82d1f5586198737fec619ee243b79cbf6", + "sha256": "2740c030304107079f1fcb08c43f40950d0b752cbe10d8412d519b140c7c1b1c", "role": "OpenHCS metadata-preserving wrapper around ArrayBridge memory decorators, accepting a ProcessingContract class as contract." }, { "path": "openhcs/core/processing_contracts.py", - "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", + "sha256": "9b999e9210baf292f8fbf5bb3758c41a95e74401bfb1e1c127bae2dba625b0ee", "role": "ProcessingContract class family including Pure3DContract used in the example." }, { "path": "openhcs/core/callable_contract.py", - "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", + "sha256": "2060f856bcb0cc8ef57b92fc17a1e558a676e4033075950b1a4e41dc8656d330", "role": "Compiler-visible projection of all declared callable memory roles." }, { @@ -3796,7 +3835,7 @@ }, { "path": "openhcs/core/steps/function_runtime.py", - "sha256": "9f99f76c18669177abad9bfa4921c031875f4f9c3079f18aa3855f444b0bbe3b", + "sha256": "6b523a5b649c8f58e1b7ea242bcfc45c2524166b6ed0211fc51f54b178722820", "role": "Runtime conversion and invocation inside the compiled execution-framework device scope." }, { @@ -3806,13 +3845,14 @@ }, { "path": "tests/unit/test_unified_registry_runtime_payloads.py", - "sha256": "76e02cc26b40f5af984d31e874eab1b18d334259fb5cd44bc9fe5945c474b139", + "sha256": "935b3b19272e5a2f52b9fce8c60f9331b3b0adef2c24f8a3c20bd3f675c9a731", "role": "Proves pyclesperanto output buffers and device handles remain runtime-owned while algorithm parameters stay authored." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/pyclesperanto_simple_implementation.rst against the later B6 changes to openhcs/processing/backends/processors/pyclesperanto_processor.py, openhcs/core/memory/decorators.py, openhcs/core/processing_contracts.py, openhcs/core/callable_contract.py, openhcs/core/steps/function_runtime.py, tests/unit/test_unified_registry_runtime_payloads.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/pyclesperanto_simple_implementation.rst in full and compared it against the changed authorities: the page itself (example already moved from ProcessingContract.PURE_3D to the Pure3DContract class, G5), pyclesperanto_processor.py (shared operations moved to image_stack_operations), arraybridge types.py and decorators.py, openhcs/core/memory/decorators.py, callable_contract.py, function_runtime.py and test_unified_registry_runtime_payloads.py. Confirmed the example imports (openhcs.core.memory.pyclesperanto, openhcs.core.processing_contracts.Pure3DContract), that the OpenHCS decorator accepts a ProcessingContract subclass as contract, and the three runtime-owned PyclesperantoRegistry parameters; added processing_contracts.py as the owner of the example contract.", "Re-read docs/source/development/pyclesperanto_simple_implementation.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: external/arraybridge/src/arraybridge/converters.py; openhcs/processing/backends/lib_registry/pyclesperanto_registry.py; openhcs/processing/backends/processors/pyclesperanto_processor.py; external/arraybridge/src/arraybridge/types.py; external/arraybridge/src/arraybridge/decorators.py; openhcs/core/memory/decorators.py; openhcs/core/callable_contract.py; openhcs/core/framework_device_resolver.py; openhcs/core/steps/function_runtime.py; tests/unit/test_framework_device_resolver.py; tests/unit/test_unified_registry_runtime_payloads.py.", @@ -3842,13 +3882,14 @@ }, { "path": "pyproject.toml", - "sha256": "3592cc37444e2e1f5bb0284e939643039c0d513217861af8133dd8bd6e8a4325", + "sha256": "c3a2bea22aa6872940125f4aaac72c9eac72084fefc6c2cefdb54b5273361e4a", "role": "Current Python support, dependency, package discovery, and entry-point authority." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/repository_setup.rst against the later B6 changes to pyproject.toml: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/repository_setup.rst in full and compared it against the changed authorities: pyproject.toml (first-party floors moved to zmqruntime 0.6, objectstate 1.3.1, python-introspect 0.2.5, metaclass-registry 0.3, arraybridge 0.4.1, polystore 0.6, pyqt-reactive 0.4; JAX GPU extra now jax[cuda12-local]; openhcs-basicpy added as an ordinary dependency); challenged the eight-submodule count against .gitmodules, the 3.11-3.14 range against requires-python, the centrosome parity oracle's python_version < 3.14 marker, the named extras (dev, gui, dev-gui, cellprofiler-compat, viz, gpu, docs), setup.py deferring metadata to pyproject.toml, and the CUDA 12 statement for the gpu extra; all remain accurate and the page states no version floors.", "Re-read docs/source/development/repository_setup.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: .gitmodules; pyproject.toml.", @@ -3899,12 +3940,12 @@ }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", + "sha256": "90d75f6287c5f21f93d82960729199261f7906e0f4912b7728527260b9723a66", "role": "Resolved step/session identity and typed plan assembly invariants." }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "b6a7231bc2964d472070731018f6a4935e5f23cf8f238a232045c32244e6545a", + "sha256": "628a27fea26ed971cfdc4ab22d11df5491f21eb49b6b90397749b46e73e8145b", "role": "RuntimeValueStore recording, replacement, query, and observation semantics." }, { @@ -3918,6 +3959,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/respecting_codebase_architecture.rst against the later B6 changes to openhcs/core/pipeline/compilation_session.py, openhcs/core/runtime_stores.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/respecting_codebase_architecture.rst in full and compared it against the changed authorities: openhcs/core/registry_strategies.py (deleted in b221f8491), openhcs/core/pipeline/compilation_session.py, openhcs/core/runtime_stores.py, tests/unit/test_configuration_mirror_deletion.py; confirmed the three strategy mixins and the registry types named in the search command exist in metaclass-registry, replaced the deleted openhcs.core.registry_strategies reference with metaclass_registry.strategies, and found the typed-plan, RuntimeValueStore and mirror-deletion claims still accurate.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", "Source search confirmed the three named strategy mixins and direct typed-plan consumers; mirror-deletion tests protect removed parallel surfaces", @@ -3937,7 +3979,7 @@ "authority": [ { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", + "sha256": "90d75f6287c5f21f93d82960729199261f7906e0f4912b7728527260b9723a66", "role": "Declares ResolvedPipelineDefinition and CompilationSession, the resolved declaration view and per-context compiler boundary named in the assembly order." }, { @@ -3947,7 +3989,7 @@ }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "b6a7231bc2964d472070731018f6a4935e5f23cf8f238a232045c32244e6545a", + "sha256": "628a27fea26ed971cfdc4ab22d11df5491f21eb49b6b90397749b46e73e8145b", "role": "RuntimeValueStore recording, replacement, query, and observation semantics." }, { @@ -3957,12 +3999,12 @@ }, { "path": "openhcs/core/callable_contract.py", - "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", + "sha256": "2060f856bcb0cc8ef57b92fc17a1e558a676e4033075950b1a4e41dc8656d330", "role": "Callable ABI and input/output/execution memory-type declaration authority." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", + "sha256": "f4a851d7de2ea82e1d548f37e6be2511fd20d799580193496a3746e5484906ba", "role": "Typed per-step source-binding plan and framework-device decisions." }, { @@ -3978,6 +4020,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/runtime_system_assembly_rules.rst against the later B6 changes to openhcs/core/pipeline/compilation_session.py, openhcs/core/runtime_stores.py, openhcs/core/callable_contract.py, openhcs/core/compiled_step_plan.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/runtime_system_assembly_rules.rst in full and compared it against the changed authorities: compilation_session.py, compiled_execution.py, runtime_stores.py, runtime_slice_projection.py, callable_contract.py, compiled_step_plan.py, framework_device_assignment.py; replaced the deleted StepSnapshot with ResolvedPipelineDefinition, corrected the CompiledExecutionBundle field summary and the RuntimeSliceProjection description, and confirmed CallableContract memory roles, CellProfilerModuleArtifactContracts, SourceBindingsConfig, CompiledSourceUniversePlan, ArtifactSpecRef and ArtifactKey still exist as described.", "Re-read docs/source/development/runtime_system_assembly_rules.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/pipeline/compilation_session.py; openhcs/core/compiled_execution.py; openhcs/core/runtime_stores.py; openhcs/core/runtime_slice_projection.py; openhcs/core/callable_contract.py; openhcs/core/compiled_step_plan.py; openhcs/core/pipeline/framework_device_assignment.py.", @@ -4014,12 +4057,12 @@ }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "b6a7231bc2964d472070731018f6a4935e5f23cf8f238a232045c32244e6545a", + "sha256": "628a27fea26ed971cfdc4ab22d11df5491f21eb49b6b90397749b46e73e8145b", "role": "RuntimeValueStore recording, replacement, query, and observation semantics." }, { "path": "tests/unit/test_runtime_value_store.py", - "sha256": "4ea189341b761bda993eaa2b6e24b83917568d047f4834d9f44410f7545d1452", + "sha256": "844ff48fced863824fa73bd704d7348b2de79960364d0242eb5e57a9fb2591a7", "role": "Regression proof for typed runtime recording, replacement, queries, and failure behaviour." } ], @@ -4029,6 +4072,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/runtime_value_extension.rst against the later B6 changes to openhcs/core/runtime_stores.py, tests/unit/test_runtime_value_store.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/runtime_value_extension.rst in full and compared it against the changed authorities: runtime_artifact_values.py, runtime_stores.py, runtime_slice_projection.py, tests/unit/test_runtime_value_store.py; replaced the deleted RuntimeSliceProjectionStrategy registration step with the RuntimeSliceProjectableValue base classes from runtime_plane_projection.py, removed the MRO-selection and aggregation claims, and confirmed RuntimeValue, ArtifactKey, separate RuntimeArtifactLocation and fail-closed store queries remain as described.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", "tests/unit/test_runtime_value_store.py and tests/unit/test_runtime_slice_projection.py passed", @@ -4046,12 +4090,12 @@ "authority": [ { "path": "openhcs/core/source_bindings.py", - "sha256": "b1474cfd969bd2ab3af6caba7e6e36f300f59fba5eeaaada08fd2254aef36598", + "sha256": "1dfbd51ae4c64d6a6a8e5ff5ab2ba82465c5e0adacb98dddaf819e0fea05e241", "role": "Declares SourceFilterClause, MetadataExtractionRule, NamedSourceBinding, ImagePlaneSource, SourceBindingsConfig, StepSourceBindingsConfig and the compiled source universe and binding plans." }, { "path": "openhcs/core/source_binding_selection.py", - "sha256": "f97a4475aa26e465679b89ff9a1fe96690ab7631649d71650608fc7a8aa4bf5a", + "sha256": "74061a9dd7d474c908cad4588f59c2bdd1211006b6f2cfe8a5c150d03a96424b", "role": "SourceUniverseRequest family that selects source universes." }, { @@ -4061,12 +4105,12 @@ }, { "path": "openhcs/core/source_workspace_projection.py", - "sha256": "d833421b963550aa5fe428c68c0286216437aec97834e4204b0b88def011f98b", + "sha256": "edf67cc0c9f6d28ed8f6bd3a808c2bab704db212e432a0c9cd1490cc3b1c9856", "role": "VirtualWorkspaceSourceProjection mapping virtual paths to SourcePixelRef and source metadata." }, { "path": "openhcs/core/source_binding_workspace.py", - "sha256": "4d28066a9f73b283fc637662e05fb2632c6fa18af164476c2899fce58cd647fe", + "sha256": "33911fd2973176817f155eae78260b81085b4464c07907d28a678eb46de42b15", "role": "Compiled virtual source workspace and binding projection." }, { @@ -4081,6 +4125,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/source_binding_extension.rst against the later B6 changes to openhcs/core/source_bindings.py, openhcs/core/source_binding_selection.py, openhcs/core/source_workspace_projection.py, openhcs/core/source_binding_workspace.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/source_binding_extension.rst in full and compared it against the changed authorities: source_bindings.py (runtime-context and path-lookup classes removed), source_binding_workspace.py, pipeline/path_planner.py, PolyStore filemanager.py; confirmed every named declaration and compiled plan class still exists, rewrote the strategy-selection paragraph around the current SourceUniverseRequest / SourceFilterMatcher families, changed microscope metadata to dataset source metadata, and moved the PolyStore authority to virtual_workspace.py where SourcePixelRef is declared; path_planner.py and filemanager.py were dropped as the page states no fact they own.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", "tests/unit/test_source_bindings.py and tests/unit/test_source_projection.py passed; source inspection confirmed both compiled source plan types", diff --git a/docs/audits/project_surface.json b/docs/audits/project_surface.json index ea484ace6d..9c868fe80e 100644 --- a/docs/audits/project_surface.json +++ b/docs/audits/project_surface.json @@ -12,7 +12,7 @@ "authority": [ { "path": "pyproject.toml", - "sha256": "3592cc37444e2e1f5bb0284e939643039c0d513217861af8133dd8bd6e8a4325", + "sha256": "c3a2bea22aa6872940125f4aaac72c9eac72084fefc6c2cefdb54b5273361e4a", "role": "Declares the published README, Python range, package dependencies, optional extras, console commands, and project links." }, { @@ -32,42 +32,42 @@ }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "sha256": "d1216010dec1804c51aec9c774f298513638ca016d500fe6328d76a9a9bfb4d6", "role": "Owns compile-time resolution across selected axes, compile progress emission, and construction of the execution bundle." }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", + "sha256": "90d75f6287c5f21f93d82960729199261f7906e0f4912b7728527260b9723a66", "role": "Declares ResolvedPipelineDefinition and CompilationSession named in the compilation diagram." }, { "path": "openhcs/core/progress/types.py", - "sha256": "beabbfeb8a6ed727b4bb09f6155de60c4e8b89e71e7ed053bb85574aa1675419", + "sha256": "b5f2ca1a053665a12cca78a6a6676f1ad7e157afafcc5e2161655fddf8bf34e5", "role": "Declares set_progress_queue, which compilation requires before it emits progress events." }, { "path": "openhcs/core/artifacts.py", - "sha256": "65eb0ec5c903366243ca8bd5ea07412e8d34bdb81e6015fa76b3c4c59838733f", + "sha256": "907551ae0ce4bf42a557554f5df0252b0a88fc9b7c1b7ac19705aade66b6656f", "role": "Declares image, object, measurement, relationship, table, and file artifact semantics used by materialization." }, { "path": "openhcs/core/dataset_sources/source.py", - "sha256": "097fde4d2b74031eacf53a88e79f499417e8c1c97ba0753e8a9df831b307efa6", + "sha256": "f736728d2f715b72cd5479fcc79520911b037d3410a42ebfc9febef2e51ffda6", "role": "Declares the auto-registered DatasetSource family that the ImageXpress, Opera Phenix, Bio-Formats, OMERO and OpenHCS-format sources join." }, { "path": "openhcs/processing/backends/lib_registry/unified_registry.py", - "sha256": "b1e9253021b9c3ab14a070b7d0f10909f641ba07784c794ad95d2ce36041b1d3", + "sha256": "f2bf83e045d40391446ca7629c91f32fccad9a102f0d57cbb0157f780d630cbc", "role": "Owns declaration-based callable discovery, execution memory contracts, processing contracts, and registry caching." }, { "path": "openhcs/core/viewer_streaming_service.py", - "sha256": "c96eca530f767002cfcc9aca4fe75c93d7a5848a7f42df9e5d336bf6712ed51e", + "sha256": "b85d6d9b0677bdbd99a657672e19e3a27ef418bfcddfb5550524a83e7598ec4b", "role": "Owns manual and orchestrated image and ROI streaming through registered viewer declarations." }, { "path": "openhcs/agent/capabilities.py", - "sha256": "eaebe33af01d057d1d21b3701f958bc96ede3ec5d0bddddca9917c1164ef0042", + "sha256": "87c8e2e5aeb425b9a2eb690d48bdd6f5af7a327c389deac261d7e4e0da5d609f", "role": "Declares the MCP capability profiles, including the full profile and governed custom-function registration." }, { @@ -77,17 +77,17 @@ }, { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "3439d96f3fbfcb104a53019dc150049021d9a8132bd27096f1ccf510eefbe37b", + "sha256": "5d3646e17f7de843a5b36164f7cef7505da7841636423ae4fb4e00d194a87e9c", "role": "Declares import_cellprofiler_pipeline and its returned FunctionStep list and PipelineConfig." }, { "path": "tests/integration/test_cellprofiler_official30_zmq.py", - "sha256": "dbd6e113a7135901f936e9bc8278d727d678e9971a7c951fced9080adfd9d6df", + "sha256": "3297b0e54ad3a349d35b613bd7d406e92c974320e12723be80dcfc0a90be2116", "role": "Exercises all 30 declared CellProfiler comparison cases over ZMQ and validates live Napari and Fiji projections." }, { "path": "benchmark/cellprofiler_benchmark_cli.py", - "sha256": "4025c591644e29f3df61e476ecae2143690e014336ea9f889aadeff4e692445e", + "sha256": "edc32916215a88b62702621dea9b692586572284caf9b781394b5784f62897ee", "role": "Declares the openhcs-benchmark subcommands and options, including run-measured through the OpenHCS session." }, { @@ -107,7 +107,7 @@ }, { "path": "tests/unit/test_benchmark_control.py", - "sha256": "093b584eeb37cbb12089d5af082ad45acd779e9d2bdd5b3feaea08234fc22183", + "sha256": "2798cf80df1222c80bb3aa9955b5c20c0c90e415a1795017779d22460f442bf2", "role": "Proves the installed command derivation, typed receipts, MCP projection, path policy, and inspect-only import boundary." } ], @@ -121,6 +121,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared README.md against the later B6 changes to pyproject.toml, openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/progress/types.py, openhcs/core/artifacts.py, openhcs/core/dataset_sources/source.py, openhcs/processing/backends/lib_registry/unified_registry.py, openhcs/core/viewer_streaming_service.py, openhcs/agent/capabilities.py, openhcs/interop/cellprofiler/pipeline_import.py, tests/integration/test_cellprofiler_official30_zmq.py, benchmark/cellprofiler_benchmark_cli.py, tests/unit/test_benchmark_control.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read README.md in full and compared it against the changed authorities: pyproject.toml, openhcs/desktop_installation.py (moved to openhcs/desktop/installation.py), openhcs/core/pipeline_document.py, openhcs/core/pipeline/compiler.py, openhcs/core/artifacts.py, openhcs/constants/constants.py, unified_registry.py, viewer_streaming_service.py, openhcs/agent/capabilities.py, openhcs/mcp/client_registration.py, pipeline_import.py, the Official30 integration test and the benchmark CLI/receipt/control modules; installed a progress queue in the CellProfiler example and renamed its root to dataset_root, rewrote the run-measured and finalizer paragraphs for the session-backed run, replaced microscope-handler wording with dataset sources, replaced the deleted StepSnapshot with ResolvedPipelineDefinition in the validation card and compilation diagram, and confirmed extras, console scripts, benchmark subcommands/options, MCP tool names, the full profile, client targets, and the eight submodules against current source.", "Re-read README.md in full and challenged its explanation role, audience, capability claims, installation routes, package boundaries, flow, and links against the hash-pinned declaration owners.", "Removed the obsolete embedded-assistant claim while preserving the MCP-first agent workflow.", diff --git a/docs/audits/user_docs.json b/docs/audits/user_docs.json index a6a279080c..a6d881c7c8 100644 --- a/docs/audits/user_docs.json +++ b/docs/audits/user_docs.json @@ -11,7 +11,7 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "sha256": "7b5622b3d82b1e1d672eaedb14e054be81baef417a736adb06bea34fb6708b28", "role": "Declares PipelineConfig, GlobalPipelineConfig, ProcessingConfig and the lazy processing configuration used by the step example." }, { @@ -21,17 +21,17 @@ }, { "path": "openhcs/domains/microscopy/axes.py", - "sha256": "531695809259324032e0b88632813323b4fe6a9a176ad6f636aeef8dec332399", + "sha256": "28cc16a296d4580351cd9abec553155e59c4d56a6ea20d52cbd6abb2879b75c6", "role": "Declares the microscopy axis family members such as Microscopy.Site used as variable components." }, { "path": "openhcs/core/orchestrator/orchestrator.py", - "sha256": "a61658b107f170c192eea505966c268ab6826826f03586bd64562a44800e1ccc", + "sha256": "4e55309de74e770b2e37b53f6dbedc19ffac6988ab74cdfe4b3dfc51082e8c1a", "role": "Declares the PipelineOrchestrator constructor, compile_pipelines, progress_context and execute_compiled_bundle signatures." }, { "path": "openhcs/core/progress/types.py", - "sha256": "beabbfeb8a6ed727b4bb09f6155de60c4e8b89e71e7ed053bb85574aa1675419", + "sha256": "b5f2ca1a053665a12cca78a6a6676f1ad7e157afafcc5e2161655fddf8bf34e5", "role": "Declares ProgressQueue, set_progress_queue and ProgressExecutionContext, which compilation and execution emit through." }, { @@ -41,7 +41,7 @@ }, { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "3439d96f3fbfcb104a53019dc150049021d9a8132bd27096f1ccf510eefbe37b", + "sha256": "5d3646e17f7de843a5b36164f7cef7505da7841636423ae4fb4e00d194a87e9c", "role": "Declares import_cellprofiler_pipeline and its returned steps and PipelineConfig." }, { @@ -56,6 +56,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/api/index.rst against the later B6 changes to openhcs/core/config.py, openhcs/domains/microscopy/axes.py, openhcs/core/orchestrator/orchestrator.py, openhcs/core/progress/types.py, openhcs/interop/cellprofiler/pipeline_import.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/api/index.rst in full and compared it against the changed authorities: openhcs/core/config.py, openhcs/core/orchestrator/orchestrator.py, openhcs/core/compiled_execution.py, tests/unit/test_compiled_execution.py, openhcs/interop/cellprofiler/pipeline_import.py, tests/unit/test_cellprofiler_pipeline_import.py; rewrote the compile/execute example to install a progress queue, call execute_compiled_bundle with orchestrator.progress_context(), and name the constructor argument dataset_root, matching tests/unit/remote_sensing_witness_pipeline.py; confirmed the FunctionStep/ProcessingConfig example, the import_cellprofiler_pipeline signature and the interactive-module rejection against current source.", "Re-read docs/source/api/index.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/config.py; openhcs/core/steps/function_step.py; openhcs/core/orchestrator/orchestrator.py; openhcs/core/compiled_execution.py; tests/unit/test_compiled_execution.py; openhcs/interop/cellprofiler/pipeline_import.py; tests/unit/test_cellprofiler_pipeline_import.py.", @@ -80,12 +81,12 @@ "authority": [ { "path": "openhcs/core/artifacts.py", - "sha256": "65eb0ec5c903366243ca8bd5ea07412e8d34bdb81e6015fa76b3c4c59838733f", + "sha256": "907551ae0ce4bf42a557554f5df0252b0a88fc9b7c1b7ac19705aade66b6656f", "role": "Declares the nominal artefact types exchanged by compiled steps and runtime consumers." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", + "sha256": "f4a851d7de2ea82e1d548f37e6be2511fd20d799580193496a3746e5484906ba", "role": "Declares the per-step compiled plan read by runtime code." }, { @@ -95,37 +96,37 @@ }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", + "sha256": "90d75f6287c5f21f93d82960729199261f7906e0f4912b7728527260b9723a66", "role": "Owns compilation-scoped source, artifact and plan state." }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "b6a7231bc2964d472070731018f6a4935e5f23cf8f238a232045c32244e6545a", + "sha256": "628a27fea26ed971cfdc4ab22d11df5491f21eb49b6b90397749b46e73e8145b", "role": "Implements scoped typed runtime-value storage and lookup for compiled execution." }, { "path": "openhcs/core/source_bindings.py", - "sha256": "b1474cfd969bd2ab3af6caba7e6e36f300f59fba5eeaaada08fd2254aef36598", + "sha256": "1dfbd51ae4c64d6a6a8e5ff5ab2ba82465c5e0adacb98dddaf819e0fea05e241", "role": "Declares source bindings that connect callable inputs to sources or artifact plans." }, { "path": "openhcs/core/callable_contract.py", - "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", + "sha256": "2060f856bcb0cc8ef57b92fc17a1e558a676e4033075950b1a4e41dc8656d330", "role": "Declares the compiler-visible callable metadata defined by the glossary." }, { "path": "openhcs/core/processing_contracts.py", - "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", + "sha256": "9b999e9210baf292f8fbf5bb3758c41a95e74401bfb1e1c127bae2dba625b0ee", "role": "Declares the ProcessingContract family with Pure2DContract and Pure3DContract." }, { "path": "openhcs/core/axes.py", - "sha256": "8d93133639c576050f089fb8aab99937c8982aff07e742fea44f8a12e62bb7b2", + "sha256": "03e5a61d808a0221507b2be93f3672cc5f4421e53eb72ce4b683309cc673213b", "role": "Declares the PartitionAxis role and the axis family that names the partition axis." }, { "path": "openhcs/core/dataset_sources/source.py", - "sha256": "097fde4d2b74031eacf53a88e79f499417e8c1c97ba0753e8a9df831b307efa6", + "sha256": "f736728d2f715b72cd5479fcc79520911b037d3410a42ebfc9febef2e51ffda6", "role": "Declares DatasetSource with detection, parsing, metadata, virtual-workspace and backend behaviour." } ], @@ -136,6 +137,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/appendices/glossary.rst against the later B6 changes to openhcs/core/artifacts.py, openhcs/core/compiled_step_plan.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/runtime_stores.py, openhcs/core/source_bindings.py, openhcs/core/callable_contract.py, openhcs/core/processing_contracts.py, openhcs/core/axes.py, openhcs/core/dataset_sources/source.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/appendices/glossary.rst in full and compared it against the changed authorities: openhcs/core/artifacts.py, compiled_step_plan.py, pipeline/compilation_session.py, runtime_stores.py, source_bindings.py, callable_contract.py, openhcs/processing/backends/lib_registry/unified_registry.py, plus the G1/G4/G5/G7 owners; rewrote the Processing contract, Dataset source (was Microscope handler), Partition axis (was Execution axis), Compiled step plan, Compiled execution bundle and Source workspace projection entries, and confirmed the remaining terms (FunctionStep, CompiledExecutionBundle, SourceUniverse, RuntimeAdapterSpec, CellProfilerModuleArtifactContracts, variable_components, group_by) against current source.", "On 2026-08-27, re-read docs/source/appendices/glossary.rst in full against callable-owned form exclusions and registry-owned non-authorable controls; challenged its reference definitions for accuracy, completeness, owner boundaries, and navigation, and found no prose drift.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/artifacts.py; openhcs/core/compiled_step_plan.py; openhcs/core/pipeline/compilation_session.py; openhcs/core/runtime_stores.py; openhcs/core/source_bindings.py; openhcs/core/callable_contract.py; openhcs/processing/backends/lib_registry/unified_registry.py.", @@ -196,12 +198,12 @@ "authority": [ { "path": "pyproject.toml", - "sha256": "3592cc37444e2e1f5bb0284e939643039c0d513217861af8133dd8bd6e8a4325", + "sha256": "c3a2bea22aa6872940125f4aaac72c9eac72084fefc6c2cefdb54b5273361e4a", "role": "Declares the packaged openhcs-benchmark console script entry point." }, { "path": "benchmark/cellprofiler_benchmark_cli.py", - "sha256": "4025c591644e29f3df61e476ecae2143690e014336ea9f889aadeff4e692445e", + "sha256": "edc32916215a88b62702621dea9b692586572284caf9b781394b5784f62897ee", "role": "Owns installed benchmark execution and report commands." }, { @@ -216,13 +218,14 @@ }, { "path": "tests/unit/test_benchmark_control.py", - "sha256": "093b584eeb37cbb12089d5af082ad45acd779e9d2bdd5b3feaea08234fc22183", + "sha256": "2798cf80df1222c80bb3aa9955b5c20c0c90e415a1795017779d22460f442bf2", "role": "Proves receipt validation and the inspect-only MCP projection." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/appendices/research_impact.rst against the later B6 changes to pyproject.toml, benchmark/cellprofiler_benchmark_cli.py, tests/unit/test_benchmark_control.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/appendices/research_impact.rst in full and compared it against the changed authorities: pyproject.toml, benchmark/cellprofiler_benchmark_cli.py, benchmark/contracts/run_receipt.py, benchmark/control.py, tests/unit/test_benchmark_control.py; the CLI now runs through the OpenHCS session and receipts name a subject_id, but the openhcs-benchmark entry point, typed receipts, read-only inspect/report service and the archive path the page names all still exist, so no claim changed. Dropped benchmark/contracts/pipeline.py, which the page does not rely on.", "On 2026-08-27, re-read docs/source/appendices/research_impact.rst in full after the python-introspect dependency floor advanced to 0.1.12; challenged its explanation boundary and confirmed that retiring unsupported impact claims remains accurate.", "Compared every factual claim directly with the current hash-pinned authorities: benchmark/contracts/pipeline.py; pyproject.toml.", @@ -262,27 +265,27 @@ "authority": [ { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "sha256": "d1216010dec1804c51aec9c774f298513638ca016d500fe6328d76a9a9bfb4d6", "role": "Compiles pipeline declarations into validated step plans and execution bundles." }, { "path": "openhcs/core/source_bindings.py", - "sha256": "b1474cfd969bd2ab3af6caba7e6e36f300f59fba5eeaaada08fd2254aef36598", + "sha256": "1dfbd51ae4c64d6a6a8e5ff5ab2ba82465c5e0adacb98dddaf819e0fea05e241", "role": "Source-binding declarations that name pipeline inputs from files and metadata." }, { "path": "openhcs/core/dataset_sources/source.py", - "sha256": "097fde4d2b74031eacf53a88e79f499417e8c1c97ba0753e8a9df831b307efa6", + "sha256": "f736728d2f715b72cd5479fcc79520911b037d3410a42ebfc9febef2e51ffda6", "role": "DatasetSource family that owns recognized dataset layouts." }, { "path": "openhcs/core/processing_contracts.py", - "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", + "sha256": "9b999e9210baf292f8fbf5bb3758c41a95e74401bfb1e1c127bae2dba625b0ee", "role": "ProcessingContract family deciding per-plane versus whole-stack execution." }, { "path": "openhcs/core/artifacts.py", - "sha256": "65eb0ec5c903366243ca8bd5ea07412e8d34bdb81e6015fa76b3c4c59838733f", + "sha256": "907551ae0ce4bf42a557554f5df0252b0a88fc9b7c1b7ac19705aade66b6656f", "role": "Nominal artifact types exchanged by compiled steps and runtime consumers." }, { @@ -297,6 +300,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/concepts/building_intuition.rst against the later B6 changes to openhcs/core/pipeline/compiler.py, openhcs/core/source_bindings.py, openhcs/core/dataset_sources/source.py, openhcs/core/processing_contracts.py, openhcs/core/artifacts.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/concepts/building_intuition.rst in full and compared it against the changed authorities: pipeline/compiler.py, source_bindings.py, artifacts.py, lib_registry/unified_registry.py and external/arraybridge/README.md; confirmed FunctionStep, variable_components, group_by and ProcessingContract exist, that force_disk_output is absent, and renamed the source-integration owner to DatasetSource.", "On 2026-08-27, re-read docs/source/concepts/building_intuition.rst in full against the full-catalogue editor and owner-declared exclusion changes; challenged its explanation flow and ownership model and found no prose drift.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/pipeline/compiler.py; openhcs/core/source_bindings.py; openhcs/core/artifacts.py; openhcs/processing/backends/lib_registry/unified_registry.py; external/arraybridge/README.md.", @@ -322,12 +326,12 @@ "authority": [ { "path": "openhcs/core/axes.py", - "sha256": "8d93133639c576050f089fb8aab99937c8982aff07e742fea44f8a12e62bb7b2", + "sha256": "03e5a61d808a0221507b2be93f3672cc5f4421e53eb72ce4b683309cc673213b", "role": "Declares Axis, AxisFamily, the axis roles, Ungrouped, AxisFamily.active() and with_role()." }, { "path": "openhcs/domains/microscopy/axes.py", - "sha256": "531695809259324032e0b88632813323b4fe6a9a176ad6f636aeef8dec332399", + "sha256": "28cc16a296d4580351cd9abec553155e59c4d56a6ea20d52cbd6abb2879b75c6", "role": "Declares the Microscopy family with Site, Channel, ZIndex, Timepoint and the well partition axis." }, { @@ -337,7 +341,7 @@ }, { "path": "tests/unit/test_axis_family_guards.py", - "sha256": "a81f12f9057e2ed7393e6daeb30f1c086f5c867222289a2bb4af3cf297b59c5c", + "sha256": "1ee1fd528b9e95758ad787a791088bc56a3889cab92aefac2a1539db53b94a56", "role": "Proves the component enums are deleted and kernel modules name no microscopy member." } ], @@ -348,6 +352,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/concepts/component_system_concepts.rst against the later B6 changes to openhcs/core/axes.py, openhcs/domains/microscopy/axes.py, tests/unit/test_axis_family_guards.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/concepts/component_system_concepts.rst in full and compared it against the changed authorities: openhcs/constants/constants.py, openhcs/core/component_group_scope.py, tests/unit/test_configuration_mirror_deletion.py; the page had already been rewritten for the G1 axis family, so checked every named class, role, module path and the ProcessingConfig/LazyProcessingConfig.from_config example against openhcs/core/axes.py, openhcs/domains/microscopy/axes.py, openhcs/core/config.py and the ObjectState lazy factory, corrected the Site sentence, and moved the authorities to the axis owners.", "Re-read docs/source/concepts/component_system_concepts.rst in full against repository base 32ca17c17 and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared its factual claims directly with the hash-pinned files listed in authority: openhcs/constants/constants.py; openhcs/core/component_group_scope.py; tests/unit/test_configuration_mirror_deletion.py.", @@ -373,12 +378,12 @@ }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "sha256": "d1216010dec1804c51aec9c774f298513638ca016d500fe6328d76a9a9bfb4d6", "role": "Creates one CompilationSession per partition-axis context and emits the compiled bundle after validation." }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", + "sha256": "90d75f6287c5f21f93d82960729199261f7906e0f4912b7728527260b9723a66", "role": "Declares CompilationSession with the ResolvedPipelineDefinition, source workspace projection and CompiledStepPlan map." }, { @@ -388,22 +393,22 @@ }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "b6a7231bc2964d472070731018f6a4935e5f23cf8f238a232045c32244e6545a", + "sha256": "628a27fea26ed971cfdc4ab22d11df5491f21eb49b6b90397749b46e73e8145b", "role": "Declares RuntimeValueStore keyed by typed artifact keys and execution scope." }, { "path": "openhcs/core/callable_contract.py", - "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", + "sha256": "2060f856bcb0cc8ef57b92fc17a1e558a676e4033075950b1a4e41dc8656d330", "role": "Declares the CallableContract fields: memory types, required axis roles, allowed group-by roles, processing contract, execution scope, runtime adapter and artifact specs." }, { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "3439d96f3fbfcb104a53019dc150049021d9a8132bd27096f1ccf510eefbe37b", + "sha256": "5d3646e17f7de843a5b36164f7cef7505da7841636423ae4fb4e00d194a87e9c", "role": "Lowers parsed .cppipe modules into ordinary source bindings and FunctionStep declarations." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", + "sha256": "f4a851d7de2ea82e1d548f37e6be2511fd20d799580193496a3746e5484906ba", "role": "Declares the source, artifact, memory, materialisation, streaming and worker fields of CompiledStepPlan." } ], @@ -414,6 +419,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/concepts/core_model.rst against the later B6 changes to openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/runtime_stores.py, openhcs/core/callable_contract.py, openhcs/interop/cellprofiler/pipeline_import.py, openhcs/core/compiled_step_plan.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/concepts/core_model.rst in full and compared it against the changed authorities: openhcs/core/pipeline_document.py, openhcs/core/pipeline/compiler.py, openhcs/core/compiled_execution.py, openhcs/core/runtime_stores.py, openhcs/interop/cellprofiler/pipeline_import.py, openhcs/core/compiled_step_plan.py; grepped every named class (CompilationSession, CompiledExecutionBundle, RuntimeValueStore, CellProfilerModule, SourceBindingsConfig, ArtifactSpec, CompiledStepPlan, ProcessingContract), replaced the deleted StepSnapshot, corrected session scope and contract axis wording, and added compilation_session.py and callable_contract.py as owners.", "On 2026-08-27, re-read docs/source/concepts/core_model.rst in full against the explicit CellProfiler import-validation hook; challenged its declaration-to-runtime explanation and confirmed that its supported lowering model remains accurate.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/pipeline_document.py; openhcs/core/pipeline/compiler.py; openhcs/core/compiled_execution.py; openhcs/core/runtime_stores.py; openhcs/interop/cellprofiler/pipeline_import.py; openhcs/core/compiled_step_plan.py.", @@ -437,12 +443,12 @@ "authority": [ { "path": "openhcs/core/axes.py", - "sha256": "8d93133639c576050f089fb8aab99937c8982aff07e742fea44f8a12e62bb7b2", + "sha256": "03e5a61d808a0221507b2be93f3672cc5f4421e53eb72ce4b683309cc673213b", "role": "Declares AxisFamily, its single PartitionAxis and the variable axes a step may stack along." }, { "path": "openhcs/core/processing_contracts.py", - "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", + "sha256": "9b999e9210baf292f8fbf5bb3758c41a95e74401bfb1e1c127bae2dba625b0ee", "role": "Declares the Pure2DContract, Pure3DContract, FlexibleContract and VolumetricToSliceContract subclasses of ProcessingContract." }, { @@ -457,22 +463,22 @@ }, { "path": "openhcs/processing/backends/cellprofiler/watershed.py", - "sha256": "41c8cb9cb139f8b4041a25e1a7a2d19aa85cf0427422fc87b9551c8141069625", + "sha256": "4b8b3e9360c090ba9bbb1584ddb7749e3ff3ecb62d609ea66eb27d8a4b2762d2", "role": "Declares Watershed with FullStackExecution so it consumes the complete payload." }, { "path": "openhcs/processing/backends/cellprofiler/primary_objects.py", - "sha256": "b5b8a9b98dd9166b34a8f5cebf884e8af6da85f02fe878ef1227d6656d31ae4c", + "sha256": "825d2ed2dfe7f0248af0f4b840a80a3f053227fab4cf06896311e89839dfd6d8", "role": "Declares IdentifyPrimaryObjects with Pure2DContract." }, { "path": "openhcs/processing/backends/cellprofiler/secondary.py", - "sha256": "fc9d9b1bbc3baff7216c6abe3a7135ffddfc9f7e1212d4a391fa8534038a354c", + "sha256": "b0773976512067ec267df254111d305dbdec9694e8359c380cf4e28d327bc805", "role": "Declares IdentifySecondaryObjects and IdentifyTertiaryObjects with Pure2DContract." }, { "path": "tests/unit/test_documented_dimensionality.py", - "sha256": "45762e74bf5cecbbada509f378ddbfa47c20fa6a9f5bd668d05d003e59bb38e1", + "sha256": "0b83e470095a2c75996c58d26f8ed4b908ee7830b9546bfb7cc40bbd9cc2de52", "role": "Binds the published dimensionality boundary to the current callable declarations." } ], @@ -483,6 +489,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/concepts/data_dimensions.rst against the later B6 changes to openhcs/core/axes.py, openhcs/core/processing_contracts.py, openhcs/processing/backends/cellprofiler/watershed.py, openhcs/processing/backends/cellprofiler/primary_objects.py, openhcs/processing/backends/cellprofiler/secondary.py, tests/unit/test_documented_dimensionality.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/concepts/data_dimensions.rst in full and compared it against the changed authorities: openhcs/constants/constants.py, openhcs/core/component_group_scope.py, openhcs/processing/backends/lib_registry/unified_registry.py, openhcs/core/runtime_object_label_domains.py, the three CellProfiler backends, tests/unit/test_callable_contract.py and tests/unit/test_documented_dimensionality.py; confirmed Watershed still declares FullStackExecution and the Identify modules Pure2DContract, the PAYLOAD/PLANE scopes are unchanged, and corrected the processing-contract names and partition-axis wording.", "On 2026-08-27, re-read docs/source/concepts/data_dimensions.rst in full against registry form exclusions, explicit secondary-backend default selection, and expanded contract tests; challenged its dimensional explanation and found no semantic drift.", "On 2026-08-27, re-read the complete data-dimensions explanation after library registries gained declaration-owned runtime-parameter exclusions; dimensional and runtime-identity claims remain unchanged.", @@ -505,12 +512,12 @@ "authority": [ { "path": "openhcs/core/dataset_sources/source.py", - "sha256": "097fde4d2b74031eacf53a88e79f499417e8c1c97ba0753e8a9df831b307efa6", + "sha256": "f736728d2f715b72cd5479fcc79520911b037d3410a42ebfc9febef2e51ffda6", "role": "Declares the registered DatasetSource family that projects each microscope format into axis values, backends and metadata." }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "sha256": "d1216010dec1804c51aec9c774f298513638ca016d500fe6328d76a9a9bfb4d6", "role": "Resolves inputs, axes, locality, memory conversion and artifact dependencies before emitting compiled plans." }, { @@ -525,6 +532,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/concepts/domain_fundamentals.rst against the later B6 changes to openhcs/core/dataset_sources/source.py, openhcs/core/pipeline/compiler.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/concepts/domain_fundamentals.rst in full and compared it against the changed authorities: openhcs/microscopes/microscope_interfaces.py (deleted in G4), openhcs/core/pipeline/compiler.py, openhcs/core/runtime_artifact_values.py; replaced the deleted source interface with openhcs/core/dataset_sources/source.py, updated the dataset-source wording, and confirmed the compilation and typed-result claims still hold.", "Re-read docs/source/concepts/domain_fundamentals.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/microscopes/microscope_interfaces.py; openhcs/core/pipeline/compiler.py; openhcs/core/runtime_artifact_values.py.", @@ -546,7 +554,7 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "sha256": "7b5622b3d82b1e1d672eaedb14e054be81baef417a736adb06bea34fb6708b28", "role": "Declares DtypeConfig as a DtypeConversionConfig subclass with NATIVE_OUTPUT as its default_dtype_conversion." }, { @@ -556,7 +564,7 @@ }, { "path": "openhcs/core/memory/decorators.py", - "sha256": "76f234785f4865257fd19c935e0aace82d1f5586198737fec619ee243b79cbf6", + "sha256": "2740c030304107079f1fcb08c43f40950d0b752cbe10d8412d519b140c7c1b1c", "role": "Exports the OpenHCS memory decorators that delegate dtype conversion to ArrayBridge." }, { @@ -566,13 +574,14 @@ }, { "path": "tests/unit/test_callable_contract.py", - "sha256": "cc220f53107a08b0406b6e0dd49896e3e4c3c0af6e527b0ef00d4fdaba8f0b2c", + "sha256": "b0ce0cba2e396e53f8cb9e95533ff918788e8c6dd60037a66e0aeda603bc64c2", "role": "Proves dtype_config is the overridable runtime-bound parameter on a decorated callable's contract." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/concepts/dtype_config_system.rst against the later B6 changes to openhcs/core/config.py, openhcs/core/memory/decorators.py, tests/unit/test_callable_contract.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/concepts/dtype_config_system.rst in full and compared it against the changed authorities: openhcs/core/config.py, openhcs/core/memory/decorators.py, external/arraybridge/src/arraybridge/decorators.py, tests/unit/test_callable_contract.py; checked the DtypeConversion members, the NATIVE_OUTPUT default, the import paths in the example, the step-level dtype_config field (now owned by openhcs/core/steps/abstract.py, added as an authority) and the runtime-bound dtype_config contract parameter; no claim changed.", "On 2026-08-27, re-read docs/source/concepts/dtype_config_system.rst in full after callable contracts began projecting runtime-owned inputs into generic form exclusions; challenged the dtype ownership boundary and found no prose drift.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/config.py; openhcs/core/memory/decorators.py; external/arraybridge/src/arraybridge/decorators.py; tests/unit/test_callable_contract.py.", @@ -596,32 +605,32 @@ "authority": [ { "path": "openhcs/core/callable_contract.py", - "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", + "sha256": "2060f856bcb0cc8ef57b92fc17a1e558a676e4033075950b1a4e41dc8656d330", "role": "Declares the CallableContract fields the page lists: memory types, artifact specs, runtime-bound parameters, required_axis_roles, allowed_group_by_roles, processing contract, runtime adapter, execution scope and prepare hook." }, { "path": "openhcs/core/processing_contracts.py", - "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", + "sha256": "9b999e9210baf292f8fbf5bb3758c41a95e74401bfb1e1c127bae2dba625b0ee", "role": "Declares the ProcessingContract class family selecting per-slice, full-stack, flexible or volumetric-to-slice execution." }, { "path": "openhcs/processing/backends/lib_registry/unified_registry.py", - "sha256": "b1e9253021b9c3ab14a070b7d0f10909f641ba07784c794ad95d2ce36041b1d3", + "sha256": "f2bf83e045d40391446ca7629c91f32fccad9a102f0d57cbb0157f780d630cbc", "role": "Derives catalogue metadata and form exclusions from library registry declarations." }, { "path": "openhcs/processing/backends/lib_registry/registry_service.py", - "sha256": "daef8925338fddf8504a9faa066af42456a13fe4602976bf64be06547a45fcc8", + "sha256": "46ad34dea67bb0f7ba54e1f9f79736654df60ceefe1ef4dd523a226d3222f289", "role": "Owns backend discovery, availability filtering and the process catalogue projection." }, { "path": "openhcs/processing/backends/lib_registry/openhcs_registry.py", - "sha256": "3fff5c3549ef8e1073d75e6d5544b2cd75fbb7cc9b17263207760ffca45cf885", + "sha256": "7872cf8b59a954472871a72614d67bb2658cc2f4ce67e08dba1f8088589ed760", "role": "Derives the public native callable catalogue and excludes private decorated helpers." }, { "path": "openhcs/agent/services/function_catalog_service.py", - "sha256": "211c81f3747dcbc5cad2bde36191bf7ac90fab007f44c960c729ac33c2737df1", + "sha256": "5e6357e805f6506e0d26d675fb319bf377227baabc4fc735b04f88dad32bd340", "role": "Serves catalogue search and callable detail to MCP and GUI authoring from the execution endpoint." }, { @@ -636,12 +645,12 @@ }, { "path": "tests/unit/test_library_registry_discovery.py", - "sha256": "e612d181d0053bacb38cb7bd3a73e8af8b3f7cdbe3789c1b7deb8f5f6a655170", + "sha256": "a67eb8d91a44a3eb347b46608679f2f8632abfd5b54c6f85315d1ea71a164460", "role": "Proves declaration-derived catalogue discovery and process policy filtering." }, { "path": "tests/unit/test_unified_registry_runtime_payloads.py", - "sha256": "76e02cc26b40f5af984d31e874eab1b18d334259fb5cd44bc9fe5945c474b139", + "sha256": "935b3b19272e5a2f52b9fce8c60f9331b3b0adef2c24f8a3c20bd3f675c9a731", "role": "Proves external library runtime controls stay out of authored function parameters." }, { @@ -656,6 +665,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/concepts/function_library.rst against the later B6 changes to openhcs/core/callable_contract.py, openhcs/core/processing_contracts.py, openhcs/processing/backends/lib_registry/unified_registry.py, openhcs/processing/backends/lib_registry/registry_service.py, openhcs/processing/backends/lib_registry/openhcs_registry.py, openhcs/agent/services/function_catalog_service.py, tests/unit/test_library_registry_discovery.py, tests/unit/test_unified_registry_runtime_payloads.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/concepts/function_library.rst in full and compared it against the changed authorities: unified_registry.py, registry_service.py, openhcs/core/callable_contract.py, function_catalog_service.py, openhcs_registry.py, the ArrayBridge types and decorators, and the three registry/contract tests; the G5 processing-contract move and catalogue-service signature/doc projection changes leave the page's catalogue, exclusion and framework claims intact, corrected the contract field names, and added openhcs/core/processing_contracts.py as an owner.", "Re-read docs/source/concepts/function_library.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/processing/backends/lib_registry/unified_registry.py; openhcs/processing/backends/lib_registry/registry_service.py; openhcs/core/callable_contract.py; openhcs/agent/services/function_catalog_service.py; tests/unit/test_callable_contract.py; external/arraybridge/src/arraybridge/types.py; external/arraybridge/src/arraybridge/decorators.py; openhcs/processing/backends/lib_registry/openhcs_registry.py; tests/unit/test_library_registry_discovery.py; tests/unit/test_unified_registry_runtime_payloads.py; tests/unit/pyqt_gui/test_function_catalog_form_materialization.py.", @@ -689,12 +699,12 @@ }, { "path": "openhcs/core/function_patterns.py", - "sha256": "95b9056c6980e4f7943f909aeae8262252c9aa175247a534befc8d4cc459fdd2", + "sha256": "3dbdad68a4c7c765cc0dae056079929e5e4632f39bdbe2e746bea3acc171562d", "role": "Normalises callable, chain and dictionary patterns into CompiledFunctionPattern." }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "sha256": "d1216010dec1804c51aec9c774f298513638ca016d500fe6328d76a9a9bfb4d6", "role": "Validates per-invocation callable contracts and dictionary keys against compiled grouping." }, { @@ -708,6 +718,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/concepts/function_patterns.rst against the later B6 changes to openhcs/core/function_patterns.py, openhcs/core/pipeline/compiler.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/concepts/function_patterns.rst in full and compared it against the changed authorities: the page itself (example already moved to Microscopy axes), openhcs/core/pipeline/compiler.py, tests/unit/test_function_patterns.py; confirmed the example imports, CompiledFunctionPattern, and the grouping-versus-stacking claims, corrected the axis-role wording, and added openhcs/core/function_patterns.py as the pattern owner.", "Re-read docs/source/concepts/function_patterns.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/steps/function_step.py; openhcs/core/pipeline/compiler.py; tests/unit/test_function_patterns.py.", @@ -735,18 +746,19 @@ }, { "path": "openhcs/core/config.py", - "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "sha256": "7b5622b3d82b1e1d672eaedb14e054be81baef417a736adb06bea34fb6708b28", "role": "Declares the processing, dtype and storage configuration the processing-and-storage pages explain." }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "sha256": "d1216010dec1804c51aec9c774f298513638ca016d500fe6328d76a9a9bfb4d6", "role": "Compiles declarations into plans, the step the core-model and dimensions pages describe." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/concepts/index.rst against the later B6 changes to openhcs/core/config.py, openhcs/core/pipeline/compiler.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/concepts/index.rst in full and compared it against the changed authorities: openhcs/core/pipeline_document.py, openhcs/core/config.py, openhcs/core/pipeline/compiler.py; the page is a reading-order toctree whose every entry still exists under docs/source/concepts and makes no product claim affected by the G1/G4/G5 renames; no change.", "Re-read docs/source/concepts/index.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/pipeline_document.py; openhcs/core/config.py; openhcs/core/pipeline/compiler.py.", @@ -769,7 +781,7 @@ "authority": [ { "path": "pyproject.toml", - "sha256": "3592cc37444e2e1f5bb0284e939643039c0d513217861af8133dd8bd6e8a4325", + "sha256": "c3a2bea22aa6872940125f4aaac72c9eac72084fefc6c2cefdb54b5273361e4a", "role": "Declares the eight first-party foundation distributions as OpenHCS dependencies." }, { @@ -794,7 +806,7 @@ }, { "path": "openhcs/microscopes/omero.py", - "sha256": "a3ab5a257490f729e96ba413c3f11d12f10f68dae44a728206f81b850325da31", + "sha256": "11f486719055f05744b25076274361e332520af02cfa96bc37ec02d49e783149", "role": "Declares OMEROFilenameParser projecting PolyStore OMERO addresses into the parser family." }, { @@ -846,6 +858,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/concepts/module_structure.rst against the later B6 changes to pyproject.toml, openhcs/microscopes/omero.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/concepts/module_structure.rst in full and compared it against the changed authorities: pyproject.toml, external/python-introspect/README.md, openhcs/core/config.py, unified_registry.py, pipeline_import.py, openhcs/runtime/viewer_protocol.py, external/zmqruntime client, ArrayBridge README and types, metaclass-registry discovery, benchmark CLI, run_receipt.py and control.py; listed the current openhcs/ top-level packages, confirmed the OMERO address and parser classes, the eight foundation distributions and the benchmark MCP capabilities (now session-backed), and added the domains and authoring areas.", "On 2026-08-27, re-read docs/source/concepts/module_structure.rst in full against the python-introspect 0.1.12 floor, declaration-owned catalogue inputs, and CellProfiler import validation; challenged the owner map and found no prose drift.", "Compared every factual claim directly with the current hash-pinned authorities: pyproject.toml; .gitmodules; openhcs/core/config.py; openhcs/processing/backends/lib_registry/unified_registry.py; openhcs/interop/cellprofiler/pipeline_import.py; openhcs/runtime/viewer_protocol.py; external/arraybridge/README.md; external/arraybridge/src/arraybridge/types.py.", @@ -900,12 +913,12 @@ }, { "path": "openhcs/core/config.py", - "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "sha256": "7b5622b3d82b1e1d672eaedb14e054be81baef417a736adb06bea34fb6708b28", "role": "Declares ProcessingConfig (variable_components, group_by, input_source) and PipelineConfig." }, { "path": "openhcs/core/orchestrator/orchestrator.py", - "sha256": "a61658b107f170c192eea505966c268ab6826826f03586bd64562a44800e1ccc", + "sha256": "4e55309de74e770b2e37b53f6dbedc19ffac6988ab74cdfe4b3dfc51082e8c1a", "role": "Declares PipelineOrchestrator.compile_pipelines over selected partition values, returning CompiledExecutionBundle." }, { @@ -915,12 +928,12 @@ }, { "path": "openhcs/authoring/session/compile_batch.py", - "sha256": "0cbe5a6b86a0fb5e9b92c4fc45f858c979031ac93594b317b9761c28f5aaf9ab", + "sha256": "d539bcf8df15581d23bc69701c5d30d489c3fe5f600f19236b28ec35c1c2d92f", "role": "Compiles the selected datasets through the session before execution for GUI and MCP clients." }, { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "3439d96f3fbfcb104a53019dc150049021d9a8132bd27096f1ccf510eefbe37b", + "sha256": "5d3646e17f7de843a5b36164f7cef7505da7841636423ae4fb4e00d194a87e9c", "role": "import_cellprofiler_pipeline returns the lowered steps with a populated PipelineConfig." }, { @@ -936,6 +949,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/concepts/pipelines_and_steps.rst against the later B6 changes to openhcs/core/config.py, openhcs/core/orchestrator/orchestrator.py, openhcs/authoring/session/compile_batch.py, openhcs/interop/cellprofiler/pipeline_import.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/concepts/pipelines_and_steps.rst in full and compared it against the changed authorities: the page (example already on Microscopy axes), openhcs/core/pipeline_document.py, openhcs/core/config.py, openhcs/core/pipeline/compiler.py, tests/unit/test_pipeline_document.py; checked the example imports, compile_pipelines signature, CompiledExecutionBundle fields, the session compile path and the CellProfiler import return, and corrected the compile section and axis wording.", "Re-read docs/source/concepts/pipelines_and_steps.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/pipeline_document.py; openhcs/core/steps/function_step.py; openhcs/core/config.py; openhcs/core/pipeline/compiler.py; tests/unit/test_pipeline_document.py.", @@ -958,22 +972,22 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "sha256": "7b5622b3d82b1e1d672eaedb14e054be81baef417a736adb06bea34fb6708b28", "role": "Declares VFSConfig read/intermediate/materialization backends and the fieldless OpenHCS ZarrConfig and TiffConfig subtypes." }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "b6a7231bc2964d472070731018f6a4935e5f23cf8f238a232045c32244e6545a", + "sha256": "628a27fea26ed971cfdc4ab22d11df5491f21eb49b6b90397749b46e73e8145b", "role": "Declares RuntimeValueStore with explicit replace and retained observation history." }, { "path": "openhcs/core/steps/function_artifact_materialization.py", - "sha256": "2375f8e784d4d0a5116882ae588c0681940696289b252ec6bd4b53e503c8b20b", + "sha256": "4cbd772568286146214828cc83127866b33e3707aacf79a3daa5678773fe6700", "role": "Materialises planned function outputs and forms filenames without changing artifact metadata." }, { "path": "openhcs/core/pipeline/function_contracts.py", - "sha256": "13bcaf7c7a62654f512d2cc739edccd2deeff35d99970ca493f3e827afbda91a", + "sha256": "883d22780ab41f122d6ed651c9f59fea540120f8fcf50d30e348ac295bd4fca9", "role": "Declares the @artifact_outputs decorator named by the page." }, { @@ -998,7 +1012,7 @@ }, { "path": "tests/unit/test_runtime_value_store.py", - "sha256": "4ea189341b761bda993eaa2b6e24b83917568d047f4834d9f44410f7545d1452", + "sha256": "844ff48fced863824fa73bd704d7348b2de79960364d0242eb5e57a9fb2591a7", "role": "Proves scoped typed runtime-value insertion, replacement and lookup." }, { @@ -1012,6 +1026,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/concepts/storage_system.rst against the later B6 changes to openhcs/core/config.py, openhcs/core/runtime_stores.py, openhcs/core/steps/function_artifact_materialization.py, openhcs/core/pipeline/function_contracts.py, tests/unit/test_runtime_value_store.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/concepts/storage_system.rst in full and compared it against the changed authorities: the page, openhcs/core/config.py, openhcs/core/runtime_stores.py, function_artifact_materialization.py, PolyStore base/config/disk, ArrayBridge decorators, and the runtime-store and TIFF compression tests; confirmed the three VFSConfig backends and AUTO default, the fieldless ZarrConfig/TiffConfig subtypes, the TIFF NONE/DEFLATE codecs, the @artifact_outputs decorator and RuntimeValueStore replace/observation behaviour, and added openhcs/core/pipeline/function_contracts.py as the decorator owner.", "Re-read docs/source/concepts/storage_system.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/config.py; openhcs/core/runtime_stores.py; openhcs/core/steps/function_artifact_materialization.py; external/PolyStore/src/polystore/base.py; tests/unit/test_runtime_value_store.py.", @@ -1039,7 +1054,7 @@ "authority": [ { "path": "pyproject.toml", - "sha256": "3592cc37444e2e1f5bb0284e939643039c0d513217861af8133dd8bd6e8a4325", + "sha256": "c3a2bea22aa6872940125f4aaac72c9eac72084fefc6c2cefdb54b5273361e4a", "role": "Declares the package version, Python requirement, dependency extras, scripts, and distribution metadata." }, { @@ -1114,7 +1129,7 @@ }, { "path": "tests/unit/test_installed_mcp_demo.py", - "sha256": "8babc443f7ec59cebebd3772633dde9ae9c49b125066cd5e7c7d527dc74c2a09", + "sha256": "5f62d65c0add0936583e8e78bf65bf407ed304d5be15b38be4963332a5fe7914", "role": "Proves the documented installed MCP verification flow." } ], @@ -1124,6 +1139,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/getting_started/getting_started.rst against the later B6 changes to pyproject.toml, tests/unit/test_installed_mcp_demo.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/getting_started/getting_started.rst in full and compared it against the changed authorities: pyproject.toml, openhcs/cli.py, openhcs/mcp/installed_demo.py, render_installer_contract.py, both native installers, smoke_installed_desktop.py, installer tests, integration-tests.yml, test_installed_mcp_demo.py, and the desktop modules moved into openhcs/desktop by 73a2aece2; confirmed the extras, entry points (openhcs, openhcs-mcp-demo), --log-level handling in pyqt_gui/launch.py, the demo's generated plate / neurite preset / nonzero Napari payload check / output_root report, the desktop profile extras and pip/uv override suppression, and sibling-environment update with a preserved previous environment; removed the false Python 3.13 limit on cellprofiler-compat and replaced the moved authorities.", "On 2026-08-27, re-read docs/source/getting_started/getting_started.rst in full after the python-introspect dependency floor advanced to 0.1.12; challenged the installation how-to flow and confirmed that the supported procedure is unchanged.", "On 2026-08-27, re-read the complete installation how-to after the shared workflow made its Python quality baseline job-owned; native installation, staged update, and package-index isolation claims remain unchanged.", @@ -1204,7 +1220,7 @@ "authority": [ { "path": "openhcs/pyqt_gui/main.py", - "sha256": "e43085c1eaf0c836ce34e1a1dae7740488d91bdfea56844e7ddd4e6925561cf6", + "sha256": "00ff3bfe7a032bd3de59274697a69d12c9b898177cd8e839f9da3b130becaedf", "role": "Dock layout (system monitor above, Datasets and ZMQ server manager left, Pipeline Editor right), View/Tools/Help menus, and preview-policy application to both panes." }, { @@ -1214,17 +1230,17 @@ }, { "path": "openhcs/pyqt_gui/widgets/pipeline_editor.py", - "sha256": "8e418631fd9ba4f8ac3d6b257f9af127f9e8711281608516ea2617d7855d6f36", + "sha256": "abca0b56094615407fd81df039761ef9d48ddb57ddd19f31e87e804974f25b22", "role": "Implements step editing, pipeline compilation controls, and declaration feedback in the main UI." }, { "path": "openhcs/agent/services/ui_bridge_service.py", - "sha256": "c7bdf8b5caec1c21c986cdc2224524839fadfa8764dafcd05125d90c72351899", + "sha256": "dc2c4823cc9a611d384bba2785eb6e1b75338f0ec4f731292515bfded2218d63", "role": "Owns agent-to-UI declaration inspection and mutation, including code and form synchronisation." }, { "path": "tests/unit/pyqt_gui/test_ui_agent_bridge.py", - "sha256": "de6758859d9743e826369162855327e4a84056cafdd7ddfdc33868d2a3093353", + "sha256": "b03744b1fa201f77de26d2e6050422b6422ebce78443c478c5e8c5f55e0a3e24", "role": "Proves bidirectional UI-bridge inspection and mutation of live declarations, plus truthful rejection of unsupported window-navigation targets." }, { @@ -1234,12 +1250,12 @@ }, { "path": "tests/unit/pyqt_gui/test_pipeline_code_document_transactionality.py", - "sha256": "0bb9b43d867a4e26f13b51531a2b7a847c6c3685841f8e002e209200a37366b6", + "sha256": "139725010d983c7e8a98731b4e462339c4188b8d9c44383677191810ad7bf24f", "role": "Proves Pipeline Editor code validates and applies pipeline_steps without an explicit pipeline_config." }, { "path": "openhcs/pyqt_gui/widgets/shared/services/pipeline_editor_workflows.py", - "sha256": "c704ef7e6c5f8a5745c741e923bc4e620a321fdce069e1cdcdf521c0176e4a4f", + "sha256": "d77abd94b6db02b55486326d0bbd6c565dbea8dfedb41cd0ee03dcba6a4540b1", "role": "Applies Pipeline Editor documents and commits the reconciled editor state graph." }, { @@ -1249,7 +1265,7 @@ }, { "path": "tests/unit/pyqt_gui/test_pipeline_editor_widget.py", - "sha256": "0ce88271aa7f5f55a8fc4daec8279d732844e03dfd437c11e6e6265ca196b4a0", + "sha256": "10afc4ce49c91731a8dc7e52726c6a77198747a6ec3a765e7e88ceaf40967e3a", "role": "Proves Pipeline Editor code apply leaves the reconciled root, step, and nested function states saved." }, { @@ -1259,7 +1275,7 @@ }, { "path": "openhcs/pyqt_gui/services/ui_bridge_windows.py", - "sha256": "e76298f78e3cbd949e8426a2b42cd6768cc54770a5f45c9dd6c9f788b257785e", + "sha256": "7a10b37aba9c5ea5cdbe46a4c9703b345e83a64554d6abacbfa1ee41cd3c4020", "role": "Projects embedded manager navigation through the generic declared driver." }, { @@ -1274,7 +1290,7 @@ }, { "path": "tests/unit/pyqt_gui/test_main_config_propagation.py", - "sha256": "303d90b43bee7bd531a1c7664fbf9707a2e7be1696a5147614f93c40e5c7021f", + "sha256": "499b05f1249ffc522bf18e97de05d1bf2000ab159daa0685958bd37d9dc62cc4", "role": "Verifies live preview policy propagation through the main configuration lifecycle." }, { @@ -1309,12 +1325,12 @@ }, { "path": "openhcs/authoring/session/views.py", - "sha256": "8ce6cac27293ac9105f4d2ebc76d10eebeba1ef90e0e8954c0a117910e17e3b3", + "sha256": "2963a2c48856704ea8291075316c470ccc8eeed161b5a798181a519a2ceaef98", "role": "DatasetListView and PipelineStepsView titles and the operations each pane binds." }, { "path": "openhcs/authoring/session/operations/datasets.py", - "sha256": "208274effc2e790775d90528211fa4f640eb01f7ca3fdd9c927c3bf71e39b632", + "sha256": "43d9edfed59abe6b88e242d39f1ddc8f5078559e04b3875da1da07b993295993", "role": "Compile/Connect/Connecting slot, Run availability (compiled, endpoint accepting requests) and Stop while a batch runs." }, { @@ -1329,6 +1345,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/basic_interface.rst against the later B6 changes to openhcs/pyqt_gui/main.py, openhcs/pyqt_gui/widgets/pipeline_editor.py, openhcs/agent/services/ui_bridge_service.py, tests/unit/pyqt_gui/test_ui_agent_bridge.py, tests/unit/pyqt_gui/test_pipeline_code_document_transactionality.py, openhcs/pyqt_gui/widgets/shared/services/pipeline_editor_workflows.py, tests/unit/pyqt_gui/test_pipeline_editor_widget.py, openhcs/pyqt_gui/services/ui_bridge_windows.py, tests/unit/pyqt_gui/test_main_config_propagation.py, openhcs/authoring/session/views.py, openhcs/authoring/session/operations/datasets.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/basic_interface.rst in full and compared it against the changed authorities: pyqt_gui/main.py, plate_manager.py, pipeline_editor.py, ui_bridge_service.py, ui_bridge_windows.py, pipeline_document.py, pipeline_editor_workflows.py, endpoint_function_catalog_service.py, gallery_catalog.py and their tests; confirmed the dock layout, menus (Image Browser, Log Viewer, Global Configuration, Custom Functions, Analysis Consolidation), Connect/Connecting/Compile/Run/Stop rules in the session operations, selected/all dataset code documents, steps-only pipeline documents, background catalogue preparation, the ui-function-selector gallery scenario and unsupported-navigation reporting; renamed the Plate Manager pane to its current Datasets title and pointed agent step 6 at session events.", "Re-read docs/source/guide_for_biologists/basic_interface.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/pyqt_gui/main.py; tests/unit/pyqt_gui/test_ui_config_lifecycle.py; openhcs/pyqt_gui/widgets/plate_manager.py; openhcs/pyqt_gui/widgets/pipeline_editor.py; openhcs/agent/services/ui_bridge_service.py; tests/unit/pyqt_gui/test_ui_agent_bridge.py; openhcs/core/pipeline_document.py; tests/unit/pyqt_gui/test_pipeline_code_document_transactionality.py; openhcs/pyqt_gui/widgets/shared/services/pipeline_editor_workflows.py; openhcs/agent/services/endpoint_function_catalog_service.py; tests/unit/pyqt_gui/test_pipeline_editor_widget.py.", @@ -1372,7 +1389,7 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "sha256": "7b5622b3d82b1e1d672eaedb14e054be81baef417a736adb06bea34fb6708b28", "role": "Declares the supported OpenHCS configuration dataclasses, nested scopes, defaults, and field help metadata." }, { @@ -1392,12 +1409,12 @@ }, { "path": "openhcs/pyqt_gui/services/ui_bridge_object_state.py", - "sha256": "e548b367507b3fb6830ce9ecb844e3b756243af96fe38e0d35de5da0b974c1fd", + "sha256": "81b5ffbbd19952f85430962cd1bb61918ac3abe4dd9ec843b8f67e4590ffd06b", "role": "Projects declaration-derived accepted enum inputs through live ObjectState field help." }, { "path": "tests/unit/pyqt_gui/test_ui_agent_bridge.py", - "sha256": "de6758859d9743e826369162855327e4a84056cafdd7ddfdc33868d2a3093353", + "sha256": "b03744b1fa201f77de26d2e6050422b6422ebce78443c478c5e8c5f55e0a3e24", "role": "Proves live field help reports the exact accepted inputs for enum-backed configuration fields." }, { @@ -1438,6 +1455,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/configuration_reference.rst against the later B6 changes to openhcs/core/config.py, openhcs/pyqt_gui/services/ui_bridge_object_state.py, tests/unit/pyqt_gui/test_ui_agent_bridge.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/configuration_reference.rst in full and compared it against the changed authorities: the page's own unreviewed WellFilter-to-PartitionFilter rename, openhcs/core/config.py, ObjectState object_state.py, config_service.py, python-introspect annotation_types.py, ui_bridge_object_state.py, pipeline_document.py, gallery_catalog.py and their tests; imported GlobalPipelineConfig/PipelineConfig and AbstractStep to confirm every named config field and the step-overridable families, checked the PartitionFilterConfig inheritance tree, InputSource values, main_flow_axis_persistence_enabled, openhcs_describe_config_schema and the four gallery scenarios; replaced the step-snapshot wording, corrected the consolidation field summary and realigned the diagram.", "Re-read docs/source/guide_for_biologists/configuration_reference.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/config.py; external/ObjectState/src/objectstate/object_state.py; openhcs/agent/services/config_service.py; tests/unit/agent/test_mcp_self_description_contracts.py; tests/unit/test_configuration_mirror_deletion.py; openhcs/core/pipeline_document.py; tests/unit/test_pipeline_document.py.", @@ -1468,7 +1486,7 @@ "authority": [ { "path": "openhcs/agent/capabilities.py", - "sha256": "eaebe33af01d057d1d21b3701f958bc96ede3ec5d0bddddca9917c1164ef0042", + "sha256": "87c8e2e5aeb425b9a2eb690d48bdd6f5af7a327c389deac261d7e4e0da5d609f", "role": "Declares the agent capability catalogue and safety metadata exposed through MCP self-description." }, { @@ -1478,7 +1496,7 @@ }, { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "3439d96f3fbfcb104a53019dc150049021d9a8132bd27096f1ccf510eefbe37b", + "sha256": "5d3646e17f7de843a5b36164f7cef7505da7841636423ae4fb4e00d194a87e9c", "role": "Imports CellProfiler modules into ordinary OpenHCS pipeline declarations through the interop boundary." }, { @@ -1493,7 +1511,7 @@ }, { "path": "benchmark/cellprofiler_benchmark_cli.py", - "sha256": "4025c591644e29f3df61e476ecae2143690e014336ea9f889aadeff4e692445e", + "sha256": "edc32916215a88b62702621dea9b692586572284caf9b781394b5784f62897ee", "role": "Owns packaged comparison execution and reporting commands." }, { @@ -1508,7 +1526,7 @@ }, { "path": "tests/unit/test_benchmark_control.py", - "sha256": "093b584eeb37cbb12089d5af082ad45acd779e9d2bdd5b3feaea08234fc22183", + "sha256": "2798cf80df1222c80bb3aa9955b5c20c0c90e415a1795017779d22460f442bf2", "role": "Proves exact receipt, progress, artifact, path-policy, and MCP projection behavior." }, { @@ -1518,12 +1536,12 @@ }, { "path": "openhcs/interop/cellprofiler/module_artifact_declarations.py", - "sha256": "f70b52114fb5d5e2c2fc49cd9a2b46986bde48ce02d82f1a34a67f072ca2ab20", + "sha256": "286638babca312c85ee0b8319fd44ba86be5868a3c1a20b74b1b48a0bfdbc652", "role": "Raises the import error for CellProfiler modules that require interactive desktop input." }, { "path": "openhcs/authoring/session/views.py", - "sha256": "8ce6cac27293ac9105f4d2ebc76d10eebeba1ef90e0e8954c0a117910e17e3b3", + "sha256": "2963a2c48856704ea8291075316c470ccc8eeed161b5a798181a519a2ceaef98", "role": "DatasetListView title for the desktop pane that UI-owned work populates." } ], @@ -1532,6 +1550,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/domain_expert_onboarding.rst against the later B6 changes to openhcs/agent/capabilities.py, openhcs/interop/cellprofiler/pipeline_import.py, benchmark/cellprofiler_benchmark_cli.py, tests/unit/test_benchmark_control.py, openhcs/interop/cellprofiler/module_artifact_declarations.py, openhcs/authoring/session/views.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/domain_expert_onboarding.rst in full and compared it against the changed authorities: agent/capabilities.py, knowledge_base_service.py, interop/cellprofiler/pipeline_import.py, test_architecture_projection_service.py, test_cellprofiler_pipeline_import.py, benchmark/cellprofiler_benchmark_cli.py, benchmark/contracts/run_receipt.py, benchmark/control.py, test_benchmark_control.py; confirmed openhcs_explain_architecture with topic cellprofiler_translation, the 50,000-character knowledge document bound, the interactive-module import rejection, the full MCP profile, the openhcs-benchmark entry point and read-only receipt/artifact inspection in benchmark/control.py; renamed the Plate Manager reference to the Datasets pane.", "Reviewed docs/source/guide_for_biologists/domain_expert_onboarding.rst in full; the recorded findings and disposition are the resulting page-specific editorial decision.", "Compared its factual claims directly with the hash-pinned files listed in authority: openhcs/agent/capabilities.py; openhcs/agent/services/architecture_projection_service.py; openhcs/agent/services/knowledge_base_service.py; openhcs/interop/cellprofiler/pipeline_import.py; tests/unit/agent/test_architecture_projection_service.py; tests/unit/test_cellprofiler_pipeline_import.py.", @@ -1567,23 +1586,24 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "sha256": "7b5622b3d82b1e1d672eaedb14e054be81baef417a736adb06bea34fb6708b28", "role": "ProcessingConfig declares variable_components as the stack-axis authority and group_by as post-assembly partitioning." }, { "path": "openhcs/core/processing_contracts.py", - "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", + "sha256": "9b999e9210baf292f8fbf5bb3758c41a95e74401bfb1e1c127bae2dba625b0ee", "role": "ProcessingContract family (for example Pure2DContract and WholeStackInputContract) declares whether a callable runs per plane or on the whole stack." }, { "path": "openhcs/core/axes.py", - "sha256": "8d93133639c576050f089fb8aab99937c8982aff07e742fea44f8a12e62bb7b2", + "sha256": "03e5a61d808a0221507b2be93f3672cc5f4421e53eb72ce4b683309cc673213b", "role": "AxisFamily and Axis declarations that variable_components and group_by take their values from." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/glossary.rst against the later B6 changes to openhcs/core/config.py, openhcs/core/processing_contracts.py, openhcs/core/axes.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/glossary.rst in full and compared it against the changed authorities: openhcs/constants/constants.py (component enums removed by the axis-family refactor), lib_registry/unified_registry.py, openhcs/core/config.py; confirmed variable_components and group_by docstrings in ProcessingConfig and that ProcessingContract still exists as the per-plane versus whole-stack family in processing_contracts.py; the three distinctions remain accurate. Replaced constants.py and unified_registry.py, which no longer own these facts, with processing_contracts.py and core/axes.py.", "On 2026-08-27, re-read docs/source/guide_for_biologists/glossary.rst in full against the declaration-owned catalogue input changes; challenged its concise reference boundary and confirmed that the three axis-processing distinctions remain accurate.", "On 2026-08-27, re-read the complete biologist glossary after library registries gained declaration-owned runtime-parameter exclusions; its three axis-processing distinctions remain unchanged.", @@ -1607,27 +1627,27 @@ "authority": [ { "path": "openhcs/core/source_bindings.py", - "sha256": "b1474cfd969bd2ab3af6caba7e6e36f300f59fba5eeaaada08fd2254aef36598", + "sha256": "1dfbd51ae4c64d6a6a8e5ff5ab2ba82465c5e0adacb98dddaf819e0fea05e241", "role": "Declares source bindings that map logical pipeline inputs to microscope-backed workspaces." }, { "path": "openhcs/core/source_workspace_projection.py", - "sha256": "d833421b963550aa5fe428c68c0286216437aec97834e4204b0b88def011f98b", + "sha256": "edf67cc0c9f6d28ed8f6bd3a808c2bab704db212e432a0c9cd1490cc3b1c9856", "role": "Projects source bindings and persisted semantic metadata into the concrete workspace identities compiled for execution." }, { "path": "tests/unit/test_source_bindings.py", - "sha256": "3d2bca343054aa8aa2d66ce1136ad23d1805d2c800134ee5682fac91feabaadd", + "sha256": "926b6a8480791ac9f431caaaf039f0cc3a9ff0f32ca0480c5eefdbc866637eb8", "role": "Proves source-binding validation and projection into concrete workspace identities." }, { "path": "openhcs/core/source_binding_workspace.py", - "sha256": "4d28066a9f73b283fc637662e05fb2632c6fa18af164476c2899fce58cd647fe", + "sha256": "33911fd2973176817f155eae78260b81085b4464c07907d28a678eb46de42b15", "role": "Selects raw candidates and projects binding-owned component identity." }, { "path": "tests/unit/test_source_binding_workspace.py", - "sha256": "2d3582e109bfd2541a1c217cea4209674e7603dd0443f06733644d35ed80a55a", + "sha256": "45e3c25cd271cc6d0e06116d8c4e41473a1ca2ae09a96fd8ff3161aee675f2b9", "role": "Proves raw selector coordinates are retained as original metadata while the projected address uses semantic component identity." }, { @@ -1662,7 +1682,7 @@ }, { "path": "tests/unit/test_plane_store_sources.py", - "sha256": "f0f795651e4e3d48da30a6352088234d3ac8dcdce3a90c93e86c420491d02c1f", + "sha256": "ed55a14cfaefc1938a8c73f88df0e5689f9c219e8133553304368cfe59ded00b", "role": "Proves both physical NGFF formats retain axis cardinalities, pixels, source identities, bindings, materialisation, and reinitialisation through OpenHCS." }, { @@ -1682,12 +1702,12 @@ }, { "path": "openhcs/core/artifacts.py", - "sha256": "65eb0ec5c903366243ca8bd5ea07412e8d34bdb81e6015fa76b3c4c59838733f", + "sha256": "907551ae0ce4bf42a557554f5df0252b0a88fc9b7c1b7ac19705aade66b6656f", "role": "Owns InputImageSetContextSourceRelation and its exact input-image and context-bearing target constraints." }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "b6a7231bc2964d472070731018f6a4935e5f23cf8f238a232045c32244e6545a", + "sha256": "628a27fea26ed971cfdc4ab22d11df5491f21eb49b6b90397749b46e73e8145b", "role": "RuntimeArtifactInput resolves exact artifact addresses and checks execution context through the declared image-set relation; it does not select arbitrary same-shape labels." }, { @@ -1697,32 +1717,32 @@ }, { "path": "openhcs/interop/cellprofiler/module_artifact_declarations.py", - "sha256": "f70b52114fb5d5e2c2fc49cd9a2b46986bde48ce02d82f1a34a67f072ca2ab20", + "sha256": "286638babca312c85ee0b8319fd44ba86be5868a3c1a20b74b1b48a0bfdbc652", "role": "The existing object-measurement declaration ancestor derives label input image-set relations from its selected image specs." }, { "path": "openhcs/core/dataset_sources/source.py", - "sha256": "097fde4d2b74031eacf53a88e79f499417e8c1c97ba0753e8a9df831b307efa6", + "sha256": "f736728d2f715b72cd5479fcc79520911b037d3410a42ebfc9febef2e51ffda6", "role": "DatasetSource family with format-specific, broad structured-store and declared-file selection roles and their detection order." }, { "path": "openhcs/core/dataset_sources/source_bindings_source.py", - "sha256": "fccda877f39e7b2f99d6cc95143480e5955186264e316e832ca9a69c2ae90b8c", + "sha256": "7d2ac740ebb081869ad98573032849732f9b64cbbfb4fc7bd9778becc328c78c", "role": "SourceBindingsSource, the declared-file fallback for unrecognized folders driven by source_bindings_config." }, { "path": "openhcs/agent/dto/plate.py", - "sha256": "689e7e720190f7abc2e0229373b88c5c60b2059efa4f7a7de14aa2d74dea3e69", + "sha256": "11c92712caa3eb8dc7d19f3704ed691a6352a14f1733c7127a3e8e6a5fbca6c8", "role": "PlatePathInspectionRequest source_format field and format_specific_handler_candidates result field." }, { "path": "openhcs/domains/microscopy/axes.py", - "sha256": "531695809259324032e0b88632813323b4fe6a9a176ad6f636aeef8dec332399", + "sha256": "28cc16a296d4580351cd9abec553155e59c4d56a6ea20d52cbd6abb2879b75c6", "role": "Microscopy axis family (Well, Site, Channel, ZIndex, Timepoint) used for component selectors, variable_components and group_by." }, { "path": "openhcs/core/axes.py", - "sha256": "8d93133639c576050f089fb8aab99937c8982aff07e742fea44f8a12e62bb7b2", + "sha256": "03e5a61d808a0221507b2be93f3672cc5f4421e53eb72ce4b683309cc673213b", "role": "Ungrouped grouping declaration that overlapping non-dict group_by normalizes to." } ], @@ -1734,6 +1754,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/image_sources.rst against the later B6 changes to openhcs/core/source_bindings.py, openhcs/core/source_workspace_projection.py, tests/unit/test_source_bindings.py, openhcs/core/source_binding_workspace.py, tests/unit/test_source_binding_workspace.py, tests/unit/test_plane_store_sources.py, openhcs/core/artifacts.py, openhcs/core/runtime_stores.py, openhcs/interop/cellprofiler/module_artifact_declarations.py, openhcs/core/dataset_sources/source.py, openhcs/core/dataset_sources/source_bindings_source.py, openhcs/agent/dto/plate.py, openhcs/domains/microscopy/axes.py, openhcs/core/axes.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/image_sources.rst in full and compared it against the changed authorities: the page's own unreviewed axis-family edit, microscope_interfaces.py and source_bindings_handler.py (deleted in c6d9a52ce), source_bindings.py, source_workspace_projection.py, pipeline/compiler.py, source_binding_workspace.py, bioformats_adapter.py, PolyStore zarr.py and config.py, artifacts.py, runtime_stores.py, source_matching.py, module_artifact_declarations.py and their tests; replaced the deleted handler classes with DatasetSource / SourceBindingsSource, microscope_type with source_format, deleted component enum spellings with Microscopy axes and Ungrouped, and Plate Manager with the Datasets pane; confirmed source_stack_components, ImportedMetadataTable, InputImageSetContextSourceRelation, SourceFilterSubject/MatchType, viewer route keys and both code blocks' imports still exist.", "Re-read docs/source/guide_for_biologists/image_sources.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/microscopes/microscope_interfaces.py; openhcs/microscopes/source_bindings_handler.py; openhcs/core/source_bindings.py; openhcs/core/source_workspace_projection.py; openhcs/core/pipeline/compiler.py; tests/unit/test_source_bindings.py.", @@ -1762,7 +1783,7 @@ "authority": [ { "path": "openhcs/pyqt_gui/main.py", - "sha256": "e43085c1eaf0c836ce34e1a1dae7740488d91bdfea56844e7ddd4e6925561cf6", + "sha256": "00ff3bfe7a032bd3de59274697a69d12c9b898177cd8e839f9da3b130becaedf", "role": "Composes the main desktop window, menus, task windows, and registered UI services." }, { @@ -1779,6 +1800,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/index.rst against the later B6 changes to openhcs/pyqt_gui/main.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/index.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/main.py, openhcs/core/pipeline_document.py, openhcs/microscopes/microscope_interfaces.py (deleted in c6d9a52ce); the page states no fact owned by the deleted microscope interface or by the pipeline document, so both were replaced by the synthetic-plate generator window and the eight-step synthetic_plate_pipeline.py that back its only product claim; the toctree and links remain correct.", "Reviewed docs/source/guide_for_biologists/index.rst in full; the recorded findings and disposition are the resulting page-specific editorial decision.", "Compared its factual claims directly with the hash-pinned files listed in authority: openhcs/pyqt_gui/main.py; openhcs/core/pipeline_document.py; openhcs/microscopes/microscope_interfaces.py.", @@ -1801,7 +1823,7 @@ "authority": [ { "path": "pyproject.toml", - "sha256": "3592cc37444e2e1f5bb0284e939643039c0d513217861af8133dd8bd6e8a4325", + "sha256": "c3a2bea22aa6872940125f4aaac72c9eac72084fefc6c2cefdb54b5273361e4a", "role": "Declares the package version, Python requirement, dependency extras, scripts, and distribution metadata." }, { @@ -1825,6 +1847,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/installation_and_setup.rst against the later B6 changes to pyproject.toml: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/installation_and_setup.rst in full and compared it against the changed authorities: pyproject.toml, openhcs/cli.py, packaging/installers/windows/Install-OpenHCS.ps1, packaging/installers/macos/install-openhcs.sh; the page contains only the redirect sentence and its target getting_started.rst was re-reviewed in this batch, so the redirect disposition stands.", "On 2026-08-27, re-read docs/source/guide_for_biologists/installation_and_setup.rst in full after the python-introspect dependency floor advanced to 0.1.12; confirmed that its redirect still reaches the single maintained installation how-to.", "On 2026-08-26, re-read the complete compatibility page after native installer transcript and evidence handling changed; retaining one redirect to the canonical installation how-to remains accurate and avoids duplicated guidance.", @@ -1867,17 +1890,17 @@ "authority": [ { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "sha256": "d1216010dec1804c51aec9c774f298513638ca016d500fe6328d76a9a9bfb4d6", "role": "Compiles pipeline declarations into validated step plans and execution bundles." }, { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "3439d96f3fbfcb104a53019dc150049021d9a8132bd27096f1ccf510eefbe37b", + "sha256": "5d3646e17f7de843a5b36164f7cef7505da7841636423ae4fb4e00d194a87e9c", "role": "Imports CellProfiler modules into ordinary OpenHCS pipeline declarations through the interop boundary." }, { "path": "openhcs/processing/backends/lib_registry/registry_service.py", - "sha256": "daef8925338fddf8504a9faa066af42456a13fe4602976bf64be06547a45fcc8", + "sha256": "46ad34dea67bb0f7ba54e1f9f79736654df60ceefe1ef4dd523a226d3222f289", "role": "Owns backend discovery, availability filtering, and the canonical process catalogue projection." }, { @@ -1887,7 +1910,7 @@ }, { "path": "openhcs/core/dataset_sources/choice.py", - "sha256": "7fb3c7ce701f0d629f288b79090044f43ac8ce1b51ca4f50937534407f226f1a", + "sha256": "ece43af801b4b02c302d9ac4bf6e5d07c0d9ebee5027433d12a4b548a19d11f0", "role": "DatasetSourceChoices projects the registered dataset sources, plus auto, as the dataset_source field's choices." } ], @@ -1896,6 +1919,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/intro.rst against the later B6 changes to openhcs/core/pipeline/compiler.py, openhcs/interop/cellprofiler/pipeline_import.py, openhcs/processing/backends/lib_registry/registry_service.py, openhcs/core/dataset_sources/choice.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/intro.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compiler.py, openhcs/interop/cellprofiler/pipeline_import.py, lib_registry/registry_service.py, pyqt_gui/widgets/plate_manager.py; the fit list, CellProfiler import claim and compile-before-run flow remain accurate (Run requires a compiled selection in the session operations); replaced the Plate Manager format-catalogue pointer with the dataset_source choices declared in openhcs/core/config.py and dataset_sources/choice.py.", "On 2026-08-27, re-read docs/source/guide_for_biologists/intro.rst in full against the explicit CellProfiler import-validation hook; challenged its explanation of supported imports and found no prose drift.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/pipeline/compiler.py; openhcs/interop/cellprofiler/pipeline_import.py; openhcs/processing/backends/lib_registry/registry_service.py; openhcs/pyqt_gui/widgets/plate_manager.py.", @@ -1937,12 +1961,12 @@ }, { "path": "openhcs/authoring/session/operations/datasets.py", - "sha256": "208274effc2e790775d90528211fa4f640eb01f7ca3fdd9c927c3bf71e39b632", + "sha256": "43d9edfed59abe6b88e242d39f1ddc8f5078559e04b3875da1da07b993295993", "role": "Declares the Init, Compile, Run, Viewer, and Results dataset operations and their button labels named in the tutorial steps." }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "e43085c1eaf0c836ce34e1a1dae7740488d91bdfea56844e7ddd4e6925561cf6", + "sha256": "00ff3bfe7a032bd3de59274697a69d12c9b898177cd8e839f9da3b130becaedf", "role": "Places Generate Synthetic Plate in the View menu." }, { @@ -1957,6 +1981,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/intro_stitching.rst against the later B6 changes to openhcs/authoring/session/operations/datasets.py, openhcs/pyqt_gui/main.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/intro_stitching.rst in full and compared it against the changed authorities: openhcs/demo/synthetic_data.py, openhcs/demo/synthetic_plate_pipeline.py, tests/pyqt_gui/integration/test_end_to_end_workflow_foundation.py, scripts/gallery_catalog.py; confirmed the eight step names, the View menu entry, the generator dialog labels, the Init/Compile/Run/Results labels and the four gallery scenario ids, and replaced the unnamed metadata viewer with the Viewer button and Metadata tab.", "Re-read docs/source/guide_for_biologists/intro_stitching.rst in full against repository base bb4218fae and the current shared worktree; challenged its tutorial user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/pyqt_gui/windows/synthetic_plate_generator_window.py; openhcs/demo/synthetic_data.py; openhcs/demo/synthetic_plate_pipeline.py; tests/pyqt_gui/integration/test_end_to_end_workflow_foundation.py.", @@ -1980,7 +2005,7 @@ "authority": [ { "path": "openhcs/authoring/session/operations/datasets.py", - "sha256": "208274effc2e790775d90528211fa4f640eb01f7ca3fdd9c927c3bf71e39b632", + "sha256": "43d9edfed59abe6b88e242d39f1ddc8f5078559e04b3875da1da07b993295993", "role": "Declares the Init, Compile, Run, Connect, and Code dataset operations and the submit_timeout_ms request field that the recovery steps name." }, { @@ -1990,7 +2015,7 @@ }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "sha256": "d1216010dec1804c51aec9c774f298513638ca016d500fe6328d76a9a9bfb4d6", "role": "Stops before execution on unsatisfied declarations and applies materialize_runtime_artifacts to automatic terminal outputs." }, { @@ -2005,12 +2030,12 @@ }, { "path": "openhcs/runtime/viewer_protocol.py", - "sha256": "c4bdae4bae6dee412f0d27a9cf75a95db53b3bd95c03cee7d046be16b52fe5b8", + "sha256": "f126aaf4f5c0dcc9429b7aae8ce67985260241ec7d2f3d5a948573658cf079a3", "role": "Declares the viewer server contract whose readiness and dependency failures the viewer section diagnoses." }, { "path": "openhcs/runtime/zmq_execution_client.py", - "sha256": "147fb663d3ea60b1348063440e05b77d54b40634d1ae81cf4fb4f3b5457fbc99", + "sha256": "202619246805f8d2dfbfd5ca5427af34716e424f551ebc800d8516f7b49e07e5", "role": "Implements execution-server connection and retains the .startup.jsonl journal for a server that fails before readiness." }, { @@ -2020,7 +2045,7 @@ }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "e43085c1eaf0c836ce34e1a1dae7740488d91bdfea56844e7ddd4e6925561cf6", + "sha256": "00ff3bfe7a032bd3de59274697a69d12c9b898177cd8e839f9da3b130becaedf", "role": "Wires endpoint compatibility to the version-restart workflow and gates the restart operation until initialisation, compilation and execution have finished." }, { @@ -2045,7 +2070,7 @@ }, { "path": "tests/unit/desktop/test_desktop_restart.py", - "sha256": "50425f39e481a6fff384b9732bc0f2cd68f223d3ac980234a9142b6db981b48f", + "sha256": "6181c22b42f7c9aa2f72b0a5a1d66bf9e5c7dfd8cba6af657851f27e5ac02af4", "role": "Proves the mismatch dialog, saved session, and version-specific restart behavior." }, { @@ -2082,6 +2107,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/troubleshooting_FAQ.rst against the later B6 changes to openhcs/authoring/session/operations/datasets.py, openhcs/core/pipeline/compiler.py, openhcs/runtime/viewer_protocol.py, openhcs/runtime/zmq_execution_client.py, openhcs/pyqt_gui/main.py, tests/unit/desktop/test_desktop_restart.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/troubleshooting_FAQ.rst in full and compared it against the changed authorities: plate_manager.py, compiler.py, environment.py, viewer_protocol.py, zmq_execution_client.py, pyqt_gui/main.py, the moved desktop restart modules and tests, smoke_installed_desktop.py, the catalogue service and UI bridge transport/server; confirmed every named option, environment variable, journal file, button and action label in current source, replaced the receipt wording of the restart paragraph, and repointed restart authorities to openhcs/desktop.", "Re-read docs/source/guide_for_biologists/troubleshooting_FAQ.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/pyqt_gui/widgets/plate_manager.py; openhcs/core/pipeline/compiler.py; openhcs/utils/environment.py; openhcs/runtime/viewer_protocol.py; openhcs/runtime/zmq_execution_client.py; openhcs/runtime/zmq_application.py; openhcs/pyqt_gui/main.py; openhcs/pyqt_gui/services/desktop_restart.py; tests/unit/pyqt_gui/test_desktop_restart.py; openhcs/agent/services/endpoint_function_catalog_service.py; tests/unit/pyqt_gui/test_function_catalog_projection.py.", @@ -2138,7 +2164,7 @@ }, { "path": "openhcs/core/processing_contracts.py", - "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", + "sha256": "9b999e9210baf292f8fbf5bb3758c41a95e74401bfb1e1c127bae2dba625b0ee", "role": "Declares Pure2DContract used in the minimal declaration example." }, { @@ -2148,27 +2174,27 @@ }, { "path": "openhcs/processing/presets/pipelines/loose_operaphenix_neurite_outgrowth.py", - "sha256": "1997c15cf71b8c65c8b49f238fced11b2d7a6c8bd5725ea75bdcd8cf7ab14314", + "sha256": "d006e526d467b6bbcf686e5d2ea41980ddb88a0c866341e96a97fe45fc8e940e", "role": "Declares the example_inputs fields, including source_voxel_spacing, and the UnifiedNeurons result of the loose Opera Phenix example." }, { "path": "openhcs/microscopes/opera_phenix.py", - "sha256": "0af8f549988b0fb919d7b58121a3c260a47249e6cca6bf6d7e1a854171f2de38", + "sha256": "736f376bf8c1d06519a3f49ebc888dcea6b2e46d2f52e341e6dcba1445d59f07", "role": "Declares OperaPhenixHandler, which requires Index.xml for native plate semantics." }, { "path": "scripts/master_multi_plate_demo.py", - "sha256": "82f8f6e074625118ec9a96b0f67b5aa6ad39fcc95aed2b7e877d8f442770abe3", + "sha256": "e1d2a72c8bc24e6ac4d48a6c3517d9f607bfd90c3af15081af9bc52b358eb819", "role": "Implements the master showcase's command-line options, base port 5900, contributor loading and output root." }, { "path": "scripts/mcp_assay_showcase.py", - "sha256": "a1ea7a9b6ffdec553931fbe5e9f2b755e63a98b92561ff9edee13b296d926ff6", + "sha256": "1c908d0156ac9fbfc1192aee4ef22aacfce2c3666299ab949c663aa9e818c0de", "role": "Declares the seven built-in scenario blueprints the master showcase composes." }, { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "3439d96f3fbfcb104a53019dc150049021d9a8132bd27096f1ccf510eefbe37b", + "sha256": "5d3646e17f7de843a5b36164f7cef7505da7841636423ae4fb4e00d194a87e9c", "role": "Defines import_cellprofiler_pipeline, which lowers a .cppipe into steps and a pipeline config." } ], @@ -2179,6 +2205,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/guides/complete_examples.rst against the later B6 changes to openhcs/core/processing_contracts.py, openhcs/processing/presets/pipelines/loose_operaphenix_neurite_outgrowth.py, openhcs/microscopes/opera_phenix.py, scripts/master_multi_plate_demo.py, scripts/mcp_assay_showcase.py, openhcs/interop/cellprofiler/pipeline_import.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guides/complete_examples.rst in full and compared it against the changed authorities: knowledge_base_service.py, mfd_specs.py, loose_operaphenix_neurite_outgrowth.py, master_multi_plate_demo.py, test_official30_knowledge_examples.py; confirmed the code block's imports, the Official30 document and section ids, the MFD owner names and template path, the master demo flags, port, blueprint count and contributor callables, replaced Microscope.OPERAPHENIX with OperaPhenixHandler and added the source_voxel_spacing input.", "Reviewed docs/source/guides/complete_examples.rst in full; the recorded findings and disposition are the resulting page-specific editorial decision.", "Compared its factual claims directly with the hash-pinned files listed in authority: openhcs/agent/services/knowledge_base_service.py; openhcs/processing/presets/mfd_specs.py; openhcs/processing/presets/pipelines/loose_operaphenix_neurite_outgrowth.py; scripts/master_multi_plate_demo.py; tests/unit/agent/test_official30_knowledge_examples.py.", @@ -2211,12 +2238,12 @@ }, { "path": "openhcs/processing/presets/pipelines/loose_operaphenix_neurite_outgrowth.py", - "sha256": "1997c15cf71b8c65c8b49f238fced11b2d7a6c8bd5725ea75bdcd8cf7ab14314", + "sha256": "d006e526d467b6bbcf686e5d2ea41980ddb88a0c866341e96a97fe45fc8e940e", "role": "Declares the loose Opera Phenix example's exact Hoechst, MAP2 and SMI312 sources, one-well bound and unified neuron result." }, { "path": "openhcs/agent/capabilities.py", - "sha256": "eaebe33af01d057d1d21b3701f958bc96ede3ec5d0bddddca9917c1164ef0042", + "sha256": "87c8e2e5aeb425b9a2eb690d48bdd6f5af7a327c389deac261d7e4e0da5d609f", "role": "Declares openhcs_get_authoring_context and the inspect-plate, query-plate-files, selected-plate-images and selected-plate-files inspection tools." }, { @@ -2226,7 +2253,7 @@ }, { "path": "tests/unit/agent/test_knowledge_base_service.py", - "sha256": "ce4636580893a8736fd4607c6ed0f431d25c97e0c44e2b95912af3fc36b1d750", + "sha256": "67a326b8473d18cc0596aa275ad2fa1e91261a24d5083ed332bc32335aeefaee", "role": "Proves discovery and retrieval of the example corpus map and other maintained knowledge documents." } ], @@ -2235,6 +2262,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/guides/example_corpus_map.rst against the later B6 changes to openhcs/processing/presets/pipelines/loose_operaphenix_neurite_outgrowth.py, openhcs/agent/capabilities.py, tests/unit/agent/test_knowledge_base_service.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guides/example_corpus_map.rst in full and compared it against the changed authorities: knowledge_base_service.py, mfd_specs.py, benchmark/datasets/manifest.py, test_knowledge_base_service.py; confirmed every repository path, the thirty scoped native reference rows, the four MFD wrapper modules, the cellprofiler_translation topic, the authoring-context and plate inspection tool names and the Native Example Source Index; replaced 'parity receipts' with 'parity evidence'.", "Reviewed docs/source/guides/example_corpus_map.rst in full; the recorded findings and disposition are the resulting page-specific editorial decision.", "Compared its factual claims directly with the hash-pinned files listed in authority: openhcs/agent/services/knowledge_base_service.py; openhcs/processing/presets/mfd_specs.py; benchmark/datasets/manifest.py; tests/unit/agent/test_knowledge_base_service.py.", @@ -2256,7 +2284,7 @@ "authority": [ { "path": "openhcs/runtime/fiji_viewer_server.py", - "sha256": "ee246d32ed828bbf4665cc8a411e807da29bd43c2475012243f27e93cb550410", + "sha256": "d3c2114d16f71111562474b17a5ef4a54a7391275293c4678259e4410d11d15f", "role": "Initializes Fiji through FIJI_IMAGEJ_RUNTIME, tracks debounced display-batch settlement, registers the run-macro action and delegates gateway shutdown." }, { @@ -2266,7 +2294,7 @@ }, { "path": "openhcs/runtime/viewer_protocol.py", - "sha256": "c4bdae4bae6dee412f0d27a9cf75a95db53b3bd95c03cee7d046be16b52fe5b8", + "sha256": "f126aaf4f5c0dcc9429b7aae8ce67985260241ec7d2f3d5a948573658cf079a3", "role": "Declares ViewerSettleProgress and the viewer control message types named on the page." }, { @@ -2276,12 +2304,12 @@ }, { "path": "pyproject.toml", - "sha256": "3592cc37444e2e1f5bb0284e939643039c0d513217861af8133dd8bd6e8a4325", + "sha256": "c3a2bea22aa6872940125f4aaac72c9eac72084fefc6c2cefdb54b5273361e4a", "role": "Declares the fiji and viz extras named in setup." }, { "path": "tests/unit/test_fiji_viewer_server.py", - "sha256": "da575dfb02895ecc831b34d6bdf6b58112b986556459444e9701cbf43221ea22", + "sha256": "bed61ba6a34937e7827b5965a63ca1afe3874d6ecc27b3c8a32721628e8b4e36", "role": "Proves Fiji endpoint lifecycle, streamed image updates, settlement and control handling." } ], @@ -2291,6 +2319,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/guides/fiji_viewer_management.rst against the later B6 changes to openhcs/runtime/fiji_viewer_server.py, openhcs/runtime/viewer_protocol.py, pyproject.toml, tests/unit/test_fiji_viewer_server.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guides/fiji_viewer_management.rst in full and compared it against the changed authorities: fiji_viewer_server.py, viewer_protocol.py, zmqruntime __init__.py, test_fiji_viewer_server.py, plus the new shared viewer_control_actions.py; rewrote the settlement section for batch-tracked settle progress and the unsupported STATE/PAYLOADS answer, and confirmed the extras, FIJI_IMAGEJ_RUNTIME initialization and shutdown delegation.", "Reviewed docs/source/guides/fiji_viewer_management.rst in full; the recorded findings and disposition are the resulting page-specific editorial decision.", "Compared its factual claims directly with the hash-pinned files listed in authority: openhcs/runtime/fiji_viewer_server.py; openhcs/runtime/viewer_protocol.py; external/zmqruntime/src/zmqruntime/__init__.py; tests/unit/test_fiji_viewer_server.py.", @@ -2314,12 +2343,12 @@ "authority": [ { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "sha256": "d1216010dec1804c51aec9c774f298513638ca016d500fe6328d76a9a9bfb4d6", "role": "Compiles pipeline declarations into validated step plans and execution bundles." }, { "path": "openhcs/core/memory/decorators.py", - "sha256": "76f234785f4865257fd19c935e0aace82d1f5586198737fec619ee243b79cbf6", + "sha256": "2740c030304107079f1fcb08c43f40950d0b752cbe10d8412d519b140c7c1b1c", "role": "Exports OpenHCS memory and dtype decorators while delegating conversion semantics to ArrayBridge declarations." }, { @@ -2334,18 +2363,19 @@ }, { "path": "openhcs/runtime/viewer_protocol.py", - "sha256": "c4bdae4bae6dee412f0d27a9cf75a95db53b3bd95c03cee7d046be16b52fe5b8", + "sha256": "f126aaf4f5c0dcc9429b7aae8ce67985260241ec7d2f3d5a948573658cf079a3", "role": "Declares the transport-independent viewer messages, stream configuration, and server contract." }, { "path": "external/PolyStore/src/polystore/omero_local.py", - "sha256": "35d39564317510234c2a3ded00bb06a09417d1b6f7359e52e74c7bd0a086f642", + "sha256": "e549d4811539cec9842ceac007eb666380ef00d6838a9b69d91eb02de64d1daa", "role": "Implements the OMERO-backed PolyStore adapter, including connection and object materialisation behaviour." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/guides/index.rst against the later B6 changes to openhcs/core/pipeline/compiler.py, openhcs/core/memory/decorators.py, openhcs/runtime/viewer_protocol.py, external/PolyStore/src/polystore/omero_local.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guides/index.rst in full and compared it against the changed authorities: compiler.py, core/memory/decorators.py, viewer_protocol.py, arraybridge types.py; the five toctree targets exist, ArrayBridge still declares input, output and execution memory attributes, framework device assignment still plans worker scope, and the viewer and OMERO routes name current owners; no correction was needed.", "Re-read docs/source/guides/index.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/pipeline/compiler.py; openhcs/core/memory/decorators.py; openhcs/runtime/viewer_protocol.py; external/PolyStore/src/polystore/omero_local.py; external/arraybridge/src/arraybridge/types.py.", @@ -2386,22 +2416,22 @@ }, { "path": "openhcs/core/memory/decorators.py", - "sha256": "76f234785f4865257fd19c935e0aace82d1f5586198737fec619ee243b79cbf6", + "sha256": "2740c030304107079f1fcb08c43f40950d0b752cbe10d8412d519b140c7c1b1c", "role": "Exports the OpenHCS memory decorators used in the example while delegating conversion semantics to ArrayBridge." }, { "path": "openhcs/core/processing_contracts.py", - "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", + "sha256": "9b999e9210baf292f8fbf5bb3758c41a95e74401bfb1e1c127bae2dba625b0ee", "role": "Declares the ProcessingContract family and Pure2DContract used in the example." }, { "path": "openhcs/core/callable_contract.py", - "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", + "sha256": "2060f856bcb0cc8ef57b92fc17a1e558a676e4033075950b1a4e41dc8656d330", "role": "Reads input, output and execution memory types through MemoryContractAttribute." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", + "sha256": "f4a851d7de2ea82e1d548f37e6be2511fd20d799580193496a3746e5484906ba", "role": "Declares FrameworkDeviceAssignment and the typed per-step conversion decisions recorded by compilation." }, { @@ -2416,17 +2446,17 @@ }, { "path": "openhcs/core/steps/function_runtime.py", - "sha256": "9f99f76c18669177abad9bfa4921c031875f4f9c3079f18aa3855f444b0bbe3b", + "sha256": "6b523a5b649c8f58e1b7ea242bcfc45c2524166b6ed0211fc51f54b178722820", "role": "Enters the invocation's execution device scope around the function body." }, { "path": "openhcs/core/orchestrator/worker_execution.py", - "sha256": "23a3ef2a70378dd1bb5c413a56e36bbb3fa013142741c9b45407bfc47de8c4a4", + "sha256": "1d71883f545f7f8702c64710f968b4138d0748fa0c58b6e1b87e06a7045e5ebf", "role": "Cleans the exact compiled framework-device footprint during worker teardown." }, { "path": "openhcs/core/config.py", - "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "sha256": "7b5622b3d82b1e1d672eaedb14e054be81baef417a736adb06bea34fb6708b28", "role": "Declares variable_components and group_by on the step processing configuration." }, { @@ -2445,6 +2475,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/guides/memory_type_integration.rst against the later B6 changes to openhcs/core/memory/decorators.py, openhcs/core/processing_contracts.py, openhcs/core/callable_contract.py, openhcs/core/compiled_step_plan.py, openhcs/core/steps/function_runtime.py, openhcs/core/orchestrator/worker_execution.py, openhcs/core/config.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guides/memory_type_integration.rst in full and compared it against the changed authorities: arraybridge types.py and decorators.py, constants.py, core/memory/__init__.py and decorators.py, callable_contract.py, compiled_step_plan.py, test_callable_contract.py, framework_device_assignment.py, function_runtime.py, worker_execution.py, test_framework_execution_scope.py; confirmed the MemoryType re-exports, MemoryContractAttribute reads, FrameworkDeviceAssignment, execution device scope, subprocess_environment consumers and variable_components/group_by, and reworded the processing-contract paragraph for the subclass family.", "On 2026-08-27, re-read docs/source/guides/memory_type_integration.rst in full after callable contracts began projecting runtime-owned inputs into generic form exclusions; challenged the ArrayBridge/OpenHCS ownership boundary and found no prose drift.", "Compared every factual claim directly with the current hash-pinned authorities: external/arraybridge/src/arraybridge/types.py; openhcs/constants/constants.py; openhcs/core/memory/__init__.py; openhcs/core/memory/decorators.py; openhcs/core/callable_contract.py; openhcs/core/compiled_step_plan.py; external/arraybridge/src/arraybridge/decorators.py; tests/unit/test_callable_contract.py; tests/unit/test_memory_type_authority.py; openhcs/core/pipeline/framework_device_assignment.py; openhcs/core/framework_device_resolver.py; openhcs/core/steps/function_runtime.py; openhcs/core/orchestrator/worker_execution.py; tests/unit/test_framework_device_resolver.py; tests/unit/test_framework_execution_scope.py.", @@ -2470,7 +2501,7 @@ "authority": [ { "path": "external/PolyStore/src/polystore/omero_local.py", - "sha256": "35d39564317510234c2a3ded00bb06a09417d1b6f7359e52e74c7bd0a086f642", + "sha256": "e549d4811539cec9842ceac007eb666380ef00d6838a9b69d91eb02de64d1daa", "role": "Implements OMEROLocalBackend, its virtual images_dir projection, PurePosixPath normalization and save_batch plane resolution." }, { @@ -2500,17 +2531,17 @@ }, { "path": "openhcs/domains/microscopy/analysis_consolidation.py", - "sha256": "292f7138a00dc22dba5b8d8af5de6140d247b955723cc807cc982ffec7a7dd2d", + "sha256": "1084a6761a379417f86a042d635d4d43fbe55c4ac497998e48c94fe56807393a", "role": "Consolidates execution-ledger CSV content and writes summaries through FileManager and the compiled backend." }, { "path": "openhcs/core/dataset_sources/source.py", - "sha256": "097fde4d2b74031eacf53a88e79f499417e8c1c97ba0753e8a9df831b307efa6", + "sha256": "f736728d2f715b72cd5479fcc79520911b037d3410a42ebfc9febef2e51ffda6", "role": "Declares the DatasetSource interface the OMERO source implements for workspace and metadata access." }, { "path": "openhcs/microscopes/omero.py", - "sha256": "a3ab5a257490f729e96ba413c3f11d12f10f68dae44a728206f81b850325da31", + "sha256": "11f486719055f05744b25076274361e332520af02cfa96bc37ec02d49e783149", "role": "Declares OMEROHandler and OMEROFilenameParser, which map PolyStore's concrete and symbolic OMERO filenames into FilenameParseResult." }, { @@ -2535,7 +2566,7 @@ }, { "path": "pyproject.toml", - "sha256": "3592cc37444e2e1f5bb0284e939643039c0d513217861af8133dd8bd6e8a4325", + "sha256": "c3a2bea22aa6872940125f4aaac72c9eac72084fefc6c2cefdb54b5273361e4a", "role": "Includes the complete openhcs/omero deployment and plugin bundle in installed distributions." }, { @@ -2560,6 +2591,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/guides/omero_integration.rst against the later B6 changes to external/PolyStore/src/polystore/omero_local.py, openhcs/domains/microscopy/analysis_consolidation.py, openhcs/core/dataset_sources/source.py, openhcs/microscopes/omero.py, pyproject.toml: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guides/omero_integration.rst in full and compared it against the changed authorities: PolyStore filemanager.py and base.py, microscopes/omero.py, test_omero_microscope_metadata.py, pyproject.toml, tests/integration/test_main.py, ui_bridge_identities.py, plus the moved analysis consolidation and dataset-source owners; confirmed OMEROFilenameParser, FilenameParseResult, the PolyStore address types, DataSink contextual saves, ledger-based consolidation through FileManager and OMEROInstanceManager, and dropped 'canonical'.", "On 2026-08-27, re-read docs/source/guides/omero_integration.rst in full after the python-introspect dependency floor advanced to 0.1.12; challenged its OMERO ownership explanation and found no affected claim.", "Compared every factual claim directly with the current hash-pinned authorities: external/PolyStore/src/polystore/omero_local.py; external/PolyStore/src/polystore/base.py; external/PolyStore/src/polystore/omero_tables.py; openhcs/microscopes/microscope_interfaces.py; tests/unit/test_omero_microscope_metadata.py; openhcs/omero/plugin/README.md; openhcs/omero/docker-compose.yml; openhcs/runtime/omero_instance_manager.py; pyproject.toml; scripts/validate_wheel_deployment.py; tests/unit/test_omero_instance_manager.py; tests/unit/test_validate_wheel_deployment.py; tests/integration/test_main.py.", @@ -2607,22 +2639,22 @@ "authority": [ { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "sha256": "d1216010dec1804c51aec9c774f298513638ca016d500fe6328d76a9a9bfb4d6", "role": "Defines compile_pipelines over an orchestrator and list[AbstractStep] and returns a CompiledExecutionBundle." }, { "path": "openhcs/core/orchestrator/orchestrator.py", - "sha256": "a61658b107f170c192eea505966c268ab6826826f03586bd64562a44800e1ccc", + "sha256": "4e55309de74e770b2e37b53f6dbedc19ffac6988ab74cdfe4b3dfc51082e8c1a", "role": "Supplies the initialized orchestrator, its DatasetSource workspace and selected partition values to compilation." }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", + "sha256": "90d75f6287c5f21f93d82960729199261f7906e0f4912b7728527260b9723a66", "role": "Declares CompilationSession, which owns cross-step compile state." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", + "sha256": "f4a851d7de2ea82e1d548f37e6be2511fd20d799580193496a3746e5484906ba", "role": "Declares CompiledStepPlan and its typed per-step memory, conversion, device and runtime fields." }, { @@ -2637,7 +2669,7 @@ }, { "path": "tests/unit/test_compiled_execution.py", - "sha256": "cb6c7034e9534470daf99a471b27a9ae662b5d2828f01af8613534412e8eea81", + "sha256": "153a7ce595d55b3fde66c7ece1cdc26526fd9d91dba78a5a9bf51c91e870df2f", "role": "Proves compilation produces immutable step plans and execution bundles with the expected resolved values." } ], @@ -2646,6 +2678,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/guides/pipeline_compilation_workflow.rst against the later B6 changes to openhcs/core/pipeline/compiler.py, openhcs/core/orchestrator/orchestrator.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/compiled_step_plan.py, tests/unit/test_compiled_execution.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guides/pipeline_compilation_workflow.rst in full and compared it against the changed authorities: compiler.py, compilation_session.py, compiled_step_plan.py, compiled_execution.py, test_compiled_execution.py, framework_device_assignment.py; confirmed CompilationSession, CompiledStepPlan, CompiledExecutionBundle and the compile_pipelines inputs, and renamed the microscope/source workspace to the dataset-source workspace.", "Re-read docs/source/guides/pipeline_compilation_workflow.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/pipeline/compiler.py; openhcs/core/pipeline/compilation_session.py; openhcs/core/compiled_step_plan.py; openhcs/core/compiled_execution.py; tests/unit/test_compiled_execution.py; openhcs/core/pipeline/framework_device_assignment.py.", @@ -2768,12 +2801,12 @@ }, { "path": "scripts/smoke_installed_gui.py", - "sha256": "d9bf24b9e5c631b4eb6a3de0fc7a79a5ee56a848693ed6692b60dc9c570d4efc", + "sha256": "a018e473245cee30747f86e85ab0410680bbab248753d20252cff727b22d381a", "role": "Owns the installed-GUI acceptance timing and single terminal completion, applies the declared logging configuration, constructs the application on declaration-allocated endpoints, proves the background function catalogue and advertised endpoint log are usable, drives the GUI through packaged MCP, validates retained window snapshots, and cleans up exact owned resources outside the checkout." }, { "path": "tests/unit/test_smoke_installed_gui.py", - "sha256": "b8d1a1fcd67c58e2b2749262b44c1dcccbb2e72a0d299d180823e94a0285ae9d", + "sha256": "2178b0b6cfcae83dc08574573e1b021b63e2390436fffb73cafea969ad415393", "role": "Proves source-checkout rejection, one derived smoke timing policy, deadline-owner error preservation under late cancellation, declaration-owned endpoint allocation, function-catalogue readiness, advertised execution-log validation, snapshot CLI projection, and emitted PNG identity validation for the installed GUI/MCP probe." }, { @@ -2813,6 +2846,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/guides/testing_guide.rst against the later B6 changes to scripts/smoke_installed_gui.py, tests/unit/test_smoke_installed_gui.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guides/testing_guide.rst in full and compared it against the changed authorities: pytest.ini, integration-tests.yml, resolve_successful_job_baseline.py, test_windows_simple_installer.py, website-pages.yml, test_ci_package_boundaries.py, smoke_installed_desktop.py, test_installed_desktop_smoke.py, smoke_installed_gui.py, xdg_paths.py, tests/integration/conftest.py; confirmed the default discovery options, --it-* flags, artifact names, concurrency, actionlint, code-action probe and log-directory paths, and added the scoped Official30 parity lane and the installed-wheel test runner.", "Re-read docs/source/guides/testing_guide.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: pytest.ini; tests/pytest_integration_options.py; .github/workflows/integration-tests.yml; scripts/install_ci_candidate.py; .github/workflows/coverage-pages.yml; tests/unit/test_ci_package_boundaries.py; scripts/smoke_installed_desktop.py; tests/installer/test_installed_desktop_smoke.py; openhcs/pyqt_gui/config.py; openhcs/core/xdg_paths.py; tests/integration/conftest.py; .vscode/settings.json.", @@ -2890,7 +2924,7 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "sha256": "7b5622b3d82b1e1d672eaedb14e054be81baef417a736adb06bea34fb6708b28", "role": "Declares StreamingConfig, NapariStreamingConfig, FijiStreamingConfig and StepMaterializationConfig, including the partition filter inherited by streaming configuration." }, { @@ -2905,7 +2939,7 @@ }, { "path": "openhcs/runtime/viewer_protocol.py", - "sha256": "c4bdae4bae6dee412f0d27a9cf75a95db53b3bd95c03cee7d046be16b52fe5b8", + "sha256": "f126aaf4f5c0dcc9429b7aae8ce67985260241ec7d2f3d5a948573658cf079a3", "role": "Declares the SHAPES streaming data type and the settle progress fields." }, { @@ -2940,7 +2974,7 @@ }, { "path": "openhcs/core/artifacts.py", - "sha256": "65eb0ec5c903366243ca8bd5ea07412e8d34bdb81e6015fa76b3c4c59838733f", + "sha256": "907551ae0ce4bf42a557554f5df0252b0a88fc9b7c1b7ac19705aade66b6656f", "role": "Declares SpatialGraphArtifactType." }, { @@ -2955,12 +2989,12 @@ }, { "path": "tests/unit/test_napari_roi_manager_integration.py", - "sha256": "a4b7dee556f0ceccaa212751e29eb8160d3d0c08c741760715ee8372b0c0e927", + "sha256": "10d7079b3e10088bac81a8814b4d0f77854d20b7818f2c2a9d121589b295ca15", "role": "Proves the packaged Napari ROI manager is discovered and integrated with the viewer contract." }, { "path": "tests/unit/test_viewer_result_element_selection.py", - "sha256": "b1dae580f0291f53be9453df6f6c6df5458046bd3ff947b550f7ddea94b29f96", + "sha256": "a7b348fab106da4c89e58ed32225495aa27792132ef7488657f29821dfd6da92", "role": "Proves isolation preflights every requested route and selected-route axis before changing live Napari state." } ], @@ -2970,6 +3004,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/guides/viewer_management.rst against the later B6 changes to openhcs/core/config.py, openhcs/runtime/viewer_protocol.py, openhcs/core/artifacts.py, tests/unit/test_napari_roi_manager_integration.py, tests/unit/test_viewer_result_element_selection.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guides/viewer_management.rst in full and compared it against the changed authorities: napari_viewer_server.py, viewer_protocol.py, viewer_controls.py, viewer_window_service.py, napari.yaml, the ROI manager, streaming handler and element-selection tests, and napari_streaming_handlers.py; confirmed the streaming and materialization config classes, SHAPES boundary, ROI toolbar labels, plugin menu entries, license file, spatial-graph options, max_rois, layer isolation and transport-thread settle, renamed well selection to partition selection, and replaced 'custody'.", "Reviewed docs/source/guides/viewer_management.rst in full; the recorded findings and disposition are the resulting page-specific editorial decision.", "Compared its factual claims directly with the hash-pinned files listed in authority: openhcs/runtime/napari_viewer_server.py; openhcs/napari_roi_manager/__init__.py; openhcs/runtime/viewer_protocol.py; openhcs/napari.yaml; tests/unit/test_napari_roi_manager_integration.py; tests/unit/test_napari_streaming_handlers.py.", @@ -3002,12 +3037,12 @@ }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "e43085c1eaf0c836ce34e1a1dae7740488d91bdfea56844e7ddd4e6925561cf6", + "sha256": "00ff3bfe7a032bd3de59274697a69d12c9b898177cd8e839f9da3b130becaedf", "role": "Composes the main desktop window, menus, task windows, and registered UI services." }, { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "3439d96f3fbfcb104a53019dc150049021d9a8132bd27096f1ccf510eefbe37b", + "sha256": "5d3646e17f7de843a5b36164f7cef7505da7841636423ae4fb4e00d194a87e9c", "role": "Imports CellProfiler modules into ordinary OpenHCS pipeline declarations through the interop boundary." }, { @@ -3024,6 +3059,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/index.rst against the later B6 changes to openhcs/pyqt_gui/main.py, openhcs/interop/cellprofiler/pipeline_import.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/index.rst in full and compared it against the changed authorities: pipeline_document.py (PipelineDocumentAuthority renamed PipelineDocumentCodec), pyqt_gui/main.py, cellprofiler pipeline_import.py, mcp/bootstrap.py, gallery_catalog.py; the page names none of the renamed symbols, the multi-plate-overview scenario still exists, and the routing claims and toctree remain accurate; no correction was needed.", "On 2026-08-27, re-read docs/source/index.rst in full against the explicit CellProfiler import-validation hook; challenged its Diataxis routing and supported-import overview and found no prose drift.", "Compared its factual claims directly with the hash-pinned files listed in authority: openhcs/core/pipeline_document.py; openhcs/pyqt_gui/main.py; openhcs/interop/cellprofiler/pipeline_import.py; openhcs/mcp/bootstrap.py.", @@ -3049,7 +3085,7 @@ "authority": [ { "path": "pyproject.toml", - "sha256": "3592cc37444e2e1f5bb0284e939643039c0d513217861af8133dd8bd6e8a4325", + "sha256": "c3a2bea22aa6872940125f4aaac72c9eac72084fefc6c2cefdb54b5273361e4a", "role": "Declares the package version, Python requirement, dependency extras, scripts, and distribution metadata the redirected guide covers." }, { @@ -3074,6 +3110,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/installation.rst against the later B6 changes to pyproject.toml: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/installation.rst in full and compared it against the changed authorities: pyproject.toml (dependency floors, JAX extra), openhcs/cli.py (skills subcommand), the Windows and macOS installers (skill sync); the page copies none of those facts and still redirects to getting_started; replaced 'canonical' with 'maintained'.", "On 2026-08-27, re-read docs/source/installation.rst in full after the python-introspect dependency floor advanced to 0.1.12; confirmed that its compatibility redirect still reaches the canonical installation procedure.", "On 2026-08-26, re-read the complete compatibility page after native installer transcript and evidence handling changed; retaining one redirect to the canonical installation how-to remains accurate and avoids duplicated guidance.", @@ -3118,7 +3155,7 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "sha256": "7b5622b3d82b1e1d672eaedb14e054be81baef417a736adb06bea34fb6708b28", "role": "Declares GlobalPipelineConfig, PipelineConfig and the step configuration families with their defaults and field help." }, { @@ -3138,7 +3175,7 @@ }, { "path": "openhcs/agent/capabilities.py", - "sha256": "eaebe33af01d057d1d21b3701f958bc96ede3ec5d0bddddca9917c1164ef0042", + "sha256": "87c8e2e5aeb425b9a2eb690d48bdd6f5af7a327c389deac261d7e4e0da5d609f", "role": "Declares openhcs_describe_config_schema, the agent view of the same introspection." }, { @@ -3160,6 +3197,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/reference/configuration.rst against the later B6 changes to openhcs/core/config.py, openhcs/agent/capabilities.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/reference/configuration.rst in full and compared it against the changed authorities: core/config.py, test_config_reference_service.py, test_user_facing_help_surfaces.py; the four directive names still resolve through config_service aliases, the named config classes, openhcs_describe_config_schema and ConfigPatch exist, and the UIConfig overview matches its fields; no correction was needed. The generated field tables carry the G1/G4 renames without page edits.", "Re-read docs/source/reference/configuration.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/config.py; openhcs/pyqt_gui/config.py; openhcs/agent/services/config_reference_service.py; docs/source/_ext/openhcs_config_reference.py; tests/unit/agent/test_config_reference_service.py; tests/unit/pyqt_gui/test_user_facing_help_surfaces.py.", @@ -3185,12 +3223,12 @@ "authority": [ { "path": "openhcs/core/processing_contracts.py", - "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", + "sha256": "9b999e9210baf292f8fbf5bb3758c41a95e74401bfb1e1c127bae2dba625b0ee", "role": "Declares the ProcessingContract family: Pure2D, Pure3D, Flexible, and VolumetricToSlice contracts." }, { "path": "openhcs/core/image_payload_execution_mode.py", - "sha256": "7657995da0722cebfd0448c0cdce4cada1b00e0cff419f62e17dcc3654666ce0", + "sha256": "7be989333a4a58aff1184173a7b08b2d9cd821761af497575b65d9effb24ed38", "role": "Declares the image payload execution modes, including FullStackExecution, that preserve a nested volume under an outer plane axis." }, { @@ -3200,32 +3238,32 @@ }, { "path": "openhcs/processing/backends/cellprofiler/watershed.py", - "sha256": "41c8cb9cb139f8b4041a25e1a7a2d19aa85cf0427422fc87b9551c8141069625", + "sha256": "4b8b3e9360c090ba9bbb1584ddb7749e3ff3ecb62d609ea66eb27d8a4b2762d2", "role": "Declares the full-stack payload route for CellProfiler-compatible Watershed." }, { "path": "openhcs/processing/backends/cellprofiler/primary_objects.py", - "sha256": "b5b8a9b98dd9166b34a8f5cebf884e8af6da85f02fe878ef1227d6656d31ae4c", + "sha256": "825d2ed2dfe7f0248af0f4b840a80a3f053227fab4cf06896311e89839dfd6d8", "role": "Declares the current plane-local IdentifyPrimaryObjects route." }, { "path": "openhcs/processing/backends/cellprofiler/secondary.py", - "sha256": "fc9d9b1bbc3baff7216c6abe3a7135ffddfc9f7e1212d4a391fa8534038a354c", + "sha256": "b0773976512067ec267df254111d305dbdec9694e8359c380cf4e28d327bc805", "role": "Declares the current plane-local IdentifySecondaryObjects and IdentifyTertiaryObjects routes." }, { "path": "openhcs/processing/backends/cellprofiler/intensity.py", - "sha256": "43bfa771bb26453f864b633a48371c0e68cf5aae0e175ed6d7a1eabb38ec5dbe", + "sha256": "3086bf9a2cee76843fd2be8445799d29ec04f0f284570c52c20b3d55f61b62ab", "role": "Declares volumetric object-intensity execution and its Z measurement features." }, { "path": "openhcs/processing/backends/cellprofiler/shape.py", - "sha256": "69e82de4faa46adc1b5ea543e352edf9a253c4b3bdedec61e0e5e3a7d95b3019", + "sha256": "df062abfbdd8c69c9c9a309842cc2ab8917da02dd01a2e330e565515f0ed8695", "role": "Declares full-stack object-shape input and the exact 3D feature schema." }, { "path": "openhcs/processing/backends/cellprofiler/area_occupied.py", - "sha256": "8316d44a5ea2f88f855aeef1c3d4fd979fb01a95a6dc9f6d1bf005629f25b18b", + "sha256": "4f1b8ce875b3c8f67dad382868919375ee574952993772c7259e92892a882646", "role": "Declares 3D occupied-volume measurements and physical voxel-spacing input." }, { @@ -3235,7 +3273,7 @@ }, { "path": "tests/unit/test_documented_dimensionality.py", - "sha256": "45762e74bf5cecbbada509f378ddbfa47c20fa6a9f5bd668d05d003e59bb38e1", + "sha256": "0b83e470095a2c75996c58d26f8ed4b908ee7830b9546bfb7cc40bbd9cc2de52", "role": "Binds the published dimensionality claims to the current callable and feature declarations." }, { @@ -3245,12 +3283,12 @@ }, { "path": "tests/unit/test_cellprofiler_library_loading.py", - "sha256": "316732ba044a62b80711a0466d8ce5812cb882ff91fbf0417ecae90afa3ff610", + "sha256": "b0b5451d67ae9329f49e9108d7b834bc69415cb68d2a089bc0e4d6bfc6d9f0ff", "role": "Proves object intensity measures a 3D object as one volume and emits Z features." }, { "path": "tests/unit/test_cellprofiler_module_execution.py", - "sha256": "f5d678e29026578687a68f6ce7abc3184eec5223a7e4e2337038fb2079a98ea2", + "sha256": "39b15a579acf2097b0229d3394d19870e200523f92326b96b7d979bfd7cfd2f2", "role": "Proves payload-scoped size-and-shape rows use the 3D volume schema." }, { @@ -3264,6 +3302,7 @@ ], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/reference/dimensionality_and_measurements.rst against the later B6 changes to openhcs/core/processing_contracts.py, openhcs/core/image_payload_execution_mode.py, openhcs/processing/backends/cellprofiler/watershed.py, openhcs/processing/backends/cellprofiler/primary_objects.py, openhcs/processing/backends/cellprofiler/secondary.py, openhcs/processing/backends/cellprofiler/intensity.py, openhcs/processing/backends/cellprofiler/shape.py, openhcs/processing/backends/cellprofiler/area_occupied.py, tests/unit/test_documented_dimensionality.py, tests/unit/test_cellprofiler_library_loading.py, tests/unit/test_cellprofiler_module_execution.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/reference/dimensionality_and_measurements.rst in full and compared it against the changed authorities: aligned_image_payload.py, runtime_object_label_domains.py, unified_registry.py, the six CellProfiler backends, the Official30 axis1 manifest, and the four tests; confirmed primary/secondary/tertiary still declare Pure2DContract, Watershed and intensity use FullStackExecution, shape and area-occupied use FullStackLabels, occupied-volume callables use Pure3DContract, PAYLOAD/PLANE scopes remain, and every listed measurement family has a backend module; no page change was needed.", "On 2026-08-27, re-read docs/source/reference/dimensionality_and_measurements.rst in full against registry form exclusions, explicit secondary-backend default selection, and expanded import tests; challenged every dimensional capability claim and found no semantic drift.", "On 2026-08-27, re-read the complete capability reference after library registries gained declaration-owned runtime-parameter exclusions; segmentation and measurement boundaries remain unchanged.", @@ -3295,12 +3334,12 @@ }, { "path": "openhcs/processing/backends/experimental_analysis/analysis.py", - "sha256": "17a701e0740eac3e11319fe73f10302bbcab603218349bfba4e07014aabc27bb", + "sha256": "2fe9bcfba30e739cbf0b74112bfec6ce8714f173bdbd08b6dde1e3b6abe78d6e", "role": "Parses the drug_curve_map and plate_groups sheets, applies positional dose/well/plate-group mapping, controls, exclusions, and Per Well Datapoints." }, { "path": "openhcs/processing/backends/experimental_analysis/result_formats.py", - "sha256": "6951f41a2ff2bdac1945c7e56584a578c111c078cac1d684030c00f95a23f528", + "sha256": "b0790fa7050855c772f6817b3d538c86bbb161d3d0446020d808ada95351a3c0", "role": "Owns the CX5 workbook and MetaXpress CSV result readers selected by the workbook scope." }, { @@ -3310,7 +3349,7 @@ }, { "path": "openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py", - "sha256": "2705d459204557c4f7092df3fd21ed687dcf56449ed361485382b1c0ce196444", + "sha256": "bd43ec00c265279585f6bcc9eb0a6b6a2eeedaacad6003a33a60ab181cd13a4d", "role": "Exports ExperimentalAnalysisEngine, whose run_directory projects config filenames and returns the documented result dictionary." }, { @@ -3320,7 +3359,7 @@ }, { "path": "tests/unit/test_experimental_analysis_engine.py", - "sha256": "641fadecf8c940cc2316a06771c74db5f3f2db9dafdfbfebdc535760d6d776ec", + "sha256": "c79887f97615197c25c340dd277189ebc331bce1f3e9bfec7466c57c92ebd6cf", "role": "Exercises declaration-owned paths and a complete MetaXpress directory workflow with all three outputs." } ], @@ -3330,6 +3369,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/reference/experimental_config_syntax.rst against the later B6 changes to openhcs/processing/backends/experimental_analysis/analysis.py, openhcs/processing/backends/experimental_analysis/result_formats.py, openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py, tests/unit/test_experimental_analysis_engine.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/reference/experimental_config_syntax.rst in full and compared it against the changed authorities: the deleted openhcs/formats/experimental_{layout_rows,analysis,result_formats}.py (moved to processing/backends/experimental_analysis/), openhcs/core/config.py, unified_analysis_engine.py, and both experimental-analysis tests; checked the row roles, scope values, engine export and run_directory signature against the moved modules, and corrected the ExperimentalAnalysisConfig import to openhcs.domains.microscopy.config.", "Re-read docs/source/reference/experimental_config_syntax.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/formats/experimental_layout_rows.py; openhcs/formats/experimental_analysis.py; openhcs/formats/experimental_result_formats.py; openhcs/core/config.py; openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py; tests/unit/test_experimental_layout_rows.py; tests/unit/test_experimental_analysis_engine.py.", @@ -3350,12 +3390,12 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "sha256": "7b5622b3d82b1e1d672eaedb14e054be81baef417a736adb06bea34fb6708b28", "role": "Declares the global, pipeline, step, and desktop configuration dataclasses that the generated configuration reference lists." }, { "path": "openhcs/processing/backends/experimental_analysis/analysis.py", - "sha256": "17a701e0740eac3e11319fe73f10302bbcab603218349bfba4e07014aabc27bb", + "sha256": "2fe9bcfba30e739cbf0b74112bfec6ce8714f173bdbd08b6dde1e3b6abe78d6e", "role": "Parses the Excel experimental-design workbook that the syntax reference documents." }, { @@ -3365,7 +3405,7 @@ }, { "path": "openhcs/core/artifacts.py", - "sha256": "65eb0ec5c903366243ca8bd5ea07412e8d34bdb81e6015fa76b3c4c59838733f", + "sha256": "907551ae0ce4bf42a557554f5df0252b0a88fc9b7c1b7ac19705aade66b6656f", "role": "Declares the artifact types named by the shared terminology the index points to." } ], @@ -3374,6 +3414,7 @@ ], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/reference/index.rst against the later B6 changes to openhcs/core/config.py, openhcs/processing/backends/experimental_analysis/analysis.py, openhcs/core/artifacts.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/reference/index.rst in full and compared it against the changed authorities: openhcs/core/config.py, the deleted openhcs/formats/experimental_analysis.py, openhcs/agent/services/knowledge_base_service.py, and openhcs/core/artifacts.py; checked that every toctree target exists and each one-line description still matches its page and owner; no page change was needed.", "Re-read docs/source/reference/index.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/config.py; openhcs/formats/experimental_analysis.py; openhcs/agent/services/knowledge_base_service.py; openhcs/core/artifacts.py.", @@ -3405,12 +3446,12 @@ }, { "path": "openhcs/authoring/session/views.py", - "sha256": "8ce6cac27293ac9105f4d2ebc76d10eebeba1ef90e0e8954c0a117910e17e3b3", + "sha256": "2963a2c48856704ea8291075316c470ccc8eeed161b5a798181a519a2ceaef98", "role": "Owns the session view titles that the plate-manager and pipeline-editor window identities present." }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "e43085c1eaf0c836ce34e1a1dae7740488d91bdfea56844e7ddd4e6925561cf6", + "sha256": "00ff3bfe7a032bd3de59274697a69d12c9b898177cd8e839f9da3b130becaedf", "role": "Composes the main desktop window and its stable docked surfaces from the registered identities." }, { @@ -3442,6 +3483,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/reference/ui_windows.rst against the later B6 changes to openhcs/authoring/session/views.py, openhcs/pyqt_gui/main.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/reference/ui_windows.rst in full and compared it against the changed authorities: openhcs/agent/ui_bridge_identities.py, openhcs/pyqt_gui/main.py, openhcs/pyqt_gui/services/main_window_workflows.py, scripts/gallery_catalog.py, tests/unit/test_build_website.py, and tests/unit/test_capture_media_gallery.py; the U1 session refactor moved two window titles onto session views but the inventory, titles, and capture targets are still projected from the registered identity declarations, no OMERO window is registered, and all linked pages exist; added openhcs/authoring/session/views.py as the title owner; no page change was needed.", "On 2026-08-28, re-read docs/source/reference/ui_windows.rst in full and challenged its reference boundary, completeness, duplication, navigation, and current desktop claims against the registered identity and projection owners.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/agent/ui_bridge_identities.py; openhcs/pyqt_gui/main.py; openhcs/pyqt_gui/services/main_window_workflows.py; scripts/gallery_catalog.py; docs/source/_ext/openhcs_gallery.py; tests/unit/test_build_website.py; tests/unit/test_capture_media_gallery.py.", @@ -3464,7 +3506,7 @@ "authority": [ { "path": "openhcs/pyqt_gui/main.py", - "sha256": "e43085c1eaf0c836ce34e1a1dae7740488d91bdfea56844e7ddd4e6925561cf6", + "sha256": "00ff3bfe7a032bd3de59274697a69d12c9b898177cd8e839f9da3b130becaedf", "role": "Declares the four Tools > Analysis Consolidation actions and the experimental-analysis directory check." }, { @@ -3474,17 +3516,17 @@ }, { "path": "openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py", - "sha256": "2705d459204557c4f7092df3fd21ed687dcf56449ed361485382b1c0ce196444", + "sha256": "bd43ec00c265279585f6bcc9eb0a6b6a2eeedaacad6003a33a60ab181cd13a4d", "role": "Projects the five configured filenames and writes normalized, raw, and heatmap outputs." }, { "path": "tests/unit/test_experimental_analysis_engine.py", - "sha256": "641fadecf8c940cc2316a06771c74db5f3f2db9dafdfbfebdc535760d6d776ec", + "sha256": "c79887f97615197c25c340dd277189ebc331bce1f3e9bfec7466c57c92ebd6cf", "role": "Proves the documented directory contract and three output files end to end." }, { "path": "openhcs/domains/microscopy/analysis_consolidation.py", - "sha256": "292f7138a00dc22dba5b8d8af5de6140d247b955723cc807cc982ffec7a7dd2d", + "sha256": "1084a6761a379417f86a042d635d4d43fbe55c4ac497998e48c94fe56807393a", "role": "Owns the post-execute hook that consolidates execution-saved tables per directory and writes the merged summary beneath the compiled output plate root." } ], @@ -3494,6 +3536,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/user_guide/analysis_consolidation.rst against the later B6 changes to openhcs/pyqt_gui/main.py, openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py, tests/unit/test_experimental_analysis_engine.py, openhcs/domains/microscopy/analysis_consolidation.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/user_guide/analysis_consolidation.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/main.py, openhcs/core/config.py, the deleted openhcs/formats/experimental_analysis.py and openhcs/core/orchestrator/analysis_consolidation.py (now openhcs/domains/microscopy/analysis_consolidation.py), unified_analysis_engine.py, and the engine test; checked the menu actions, directory filenames, consolidation config fields, and per-directory plus merged summary behaviour; corrected the config description.", "Re-read docs/source/user_guide/analysis_consolidation.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/pyqt_gui/main.py; openhcs/core/config.py; openhcs/formats/experimental_analysis.py; openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py; tests/unit/test_experimental_analysis_engine.py.", @@ -3525,7 +3568,7 @@ }, { "path": "openhcs/authoring/session/dataset_document.py", - "sha256": "305da3c0bf7bb15a0f251a101696b721f9151d8620450a6b3e12413b79218574", + "sha256": "ca695d82a60dbee76ff660ae3710ac5033d2e55f51a99373ed8c0f0dccdbc16c", "role": "Owns the selected and all-datasets document scopes, selected-scope ID enforcement, and the global-draft and per-dataset change checks on apply." }, { @@ -3535,12 +3578,12 @@ }, { "path": "openhcs/authoring/session/views.py", - "sha256": "8ce6cac27293ac9105f4d2ebc76d10eebeba1ef90e0e8954c0a117910e17e3b3", + "sha256": "2963a2c48856704ea8291075316c470ccc8eeed161b5a798181a519a2ceaef98", "role": "Declares the Datasets and Pipeline Editor view titles shown for the two code-document owners." }, { "path": "openhcs/pyqt_gui/widgets/pipeline_editor.py", - "sha256": "8e418631fd9ba4f8ac3d6b257f9af127f9e8711281608516ea2617d7855d6f36", + "sha256": "abca0b56094615407fd81df039761ef9d48ddb57ddd19f31e87e804974f25b22", "role": "Makes the Pipeline Editor document writable only while a dataset is selected." }, { @@ -3560,17 +3603,17 @@ }, { "path": "openhcs/pyqt_gui/services/ui_bridge_object_state.py", - "sha256": "e548b367507b3fb6830ce9ecb844e3b756243af96fe38e0d35de5da0b974c1fd", + "sha256": "81b5ffbbd19952f85430962cd1bb61918ac3abe4dd9ec843b8f67e4590ffd06b", "role": "Enforces driver writability, returns unchanged for equivalent documents, and applies documents against revision tokens." }, { "path": "openhcs/pyqt_gui/services/ui_agent_bridge.py", - "sha256": "ce650c52475764657bbc0221109879594f9cf1482de8245d6c141f00474d5d23", + "sha256": "5a03e683a0677166361440c6c78803ec2dadf753342e69996153005bccab5473", "role": "Rejects wildcard imports and resolves catalogue-listed external functions in submitted documents." }, { "path": "openhcs/agent/services/ui_bridge_service.py", - "sha256": "c7bdf8b5caec1c21c986cdc2224524839fadfa8764dafcd05125d90c72351899", + "sha256": "dc2c4823cc9a611d384bba2785eb6e1b75338f0ec4f731292515bfded2218d63", "role": "Owns agent-to-UI document inspection and mutation and returns mutation receipts." }, { @@ -3600,7 +3643,7 @@ }, { "path": "openhcs/pyqt_gui/widgets/shared/services/pipeline_editor_workflows.py", - "sha256": "c704ef7e6c5f8a5745c741e923bc4e620a321fdce069e1cdcdf521c0176e4a4f", + "sha256": "d77abd94b6db02b55486326d0bbd6c565dbea8dfedb41cd0ee03dcba6a4540b1", "role": "Applies a complete Pipeline Editor document and commits the reconciled live state graph." }, { @@ -3615,12 +3658,12 @@ }, { "path": "tests/unit/pyqt_gui/test_ui_agent_bridge.py", - "sha256": "de6758859d9743e826369162855327e4a84056cafdd7ddfdc33868d2a3093353", + "sha256": "b03744b1fa201f77de26d2e6050422b6422ebce78443c478c5e8c5f55e0a3e24", "role": "Proves bidirectional UI-bridge inspection, capability enforcement, semantic no-op handling, and mutation of live declarations." }, { "path": "tests/unit/pyqt_gui/test_plate_manager_widget.py", - "sha256": "4daa91c0de965dc2dd9cd77ea4969f4fa0aff90fc5b69cbcadaba3bd2a1cfc6b", + "sha256": "5e66ccf63d5b422e3a241e8f5eb0e65cc5067edacb9656466dfb5f0994e3f07b", "role": "Proves selected code-document application preserves unselected dataset graphs." }, { @@ -3630,7 +3673,7 @@ }, { "path": "tests/unit/pyqt_gui/test_pipeline_editor_widget.py", - "sha256": "0ce88271aa7f5f55a8fc4daec8279d732844e03dfd437c11e6e6265ca196b4a0", + "sha256": "10afc4ce49c91731a8dc7e52726c6a77198747a6ec3a765e7e88ceaf40967e3a", "role": "Proves selected-dataset writability and that code apply advances saved baselines for the editor, step, and nested function states." }, { @@ -3640,7 +3683,7 @@ }, { "path": "tests/unit/test_pycodify_formatters.py", - "sha256": "2f0c6683a5255d01442ae33f676f601d5e3f1e3e04d1b2c2fc945b3c564796ce", + "sha256": "1dd67d1a782d04dc24ac2ba9ec1e457769c1c6c9191c992f0a1cc0feb78487c5", "role": "Verifies clean/full parameter order and semantic default handling." }, { @@ -3656,6 +3699,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/user_guide/code_ui_editing.rst against the later B6 changes to openhcs/authoring/session/dataset_document.py, openhcs/authoring/session/views.py, openhcs/pyqt_gui/widgets/pipeline_editor.py, openhcs/pyqt_gui/services/ui_bridge_object_state.py, openhcs/pyqt_gui/services/ui_agent_bridge.py, openhcs/agent/services/ui_bridge_service.py, openhcs/pyqt_gui/widgets/shared/services/pipeline_editor_workflows.py, tests/unit/pyqt_gui/test_ui_agent_bridge.py, tests/unit/pyqt_gui/test_plate_manager_widget.py, tests/unit/pyqt_gui/test_pipeline_editor_widget.py, tests/unit/test_pycodify_formatters.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/user_guide/code_ui_editing.rst in full and compared it against the changed authorities: pipeline_document.py, ui_bridge_object_state.py, ui_bridge_service.py, plate_manager_workflows.py and pipeline_editor_workflows.py, the deleted pipeline_object_state_binding.py (now authoring/session/pipelines.py), step_parameter_editor.py, pycodify_formatters.py, gallery_catalog.py, and the changed tests; checked selected/all document scopes, selected-dataset writability, global-draft protection while work runs, catalogue imports and wildcard rejection, unchanged no-op results, and saved-baseline commit; renamed Plate Manager to the Datasets window.", "Re-read docs/source/user_guide/code_ui_editing.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/pipeline_document.py; openhcs/pyqt_gui/services/pycodified_window_code_document.py; openhcs/agent/services/ui_bridge_service.py; external/pycodify/src/pycodify/__init__.py; tests/unit/pyqt_gui/test_ui_agent_bridge.py; openhcs/pyqt_gui/widgets/shared/services/plate_manager_workflows.py; openhcs/pyqt_gui/services/pipeline_object_state_binding.py; tests/unit/pyqt_gui/test_plate_manager_widget.py; tests/unit/test_pipeline_document.py; tests/unit/pyqt_gui/test_pipeline_code_document_transactionality.py; openhcs/pyqt_gui/widgets/shared/services/pipeline_editor_workflows.py; tests/unit/pyqt_gui/test_pipeline_editor_widget.py.", @@ -3697,23 +3741,24 @@ }, { "path": "openhcs/processing/backends/lib_registry/registry_service.py", - "sha256": "daef8925338fddf8504a9faa066af42456a13fe4602976bf64be06547a45fcc8", + "sha256": "46ad34dea67bb0f7ba54e1f9f79736654df60ceefe1ef4dd523a226d3222f289", "role": "Skips registries that do not support CPU-only mode during function discovery." }, { "path": "tests/unit/test_library_registry_discovery.py", - "sha256": "e612d181d0053bacb38cb7bd3a73e8af8b3f7cdbe3789c1b7deb8f5f6a655170", + "sha256": "a67eb8d91a44a3eb347b46608679f2f8632abfd5b54c6f85315d1ea71a164460", "role": "Proves backend discovery and availability filtering produce the expected callable registry." }, { "path": "pyproject.toml", - "sha256": "3592cc37444e2e1f5bb0284e939643039c0d513217861af8133dd8bd6e8a4325", + "sha256": "c3a2bea22aa6872940125f4aaac72c9eac72084fefc6c2cefdb54b5273361e4a", "role": "Declares the optional gpu extra and the openhcs console entry point." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/user_guide/cpu_only_mode.rst against the later B6 changes to openhcs/processing/backends/lib_registry/registry_service.py, tests/unit/test_library_registry_discovery.py, pyproject.toml: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/user_guide/cpu_only_mode.rst in full and compared it against the changed authorities: openhcs/utils/environment.py, registry_service.py, test_library_registry_discovery.py, and pyproject.toml; confirmed --no-gpu and OPENHCS_CPU_ONLY still select the same mode, discovery still filters registries by CPU-only support, and the gpu extra still exists; the new import-policy defaults (JAX platform, NumPy huge pages, worker profiling) do not contradict the page; no page change was needed.", "On 2026-08-27, re-read docs/source/user_guide/cpu_only_mode.rst in full against declaration-owned catalogue inputs and the python-introspect 0.1.12 floor; challenged the CPU-only workflow and found no behaviour change.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/utils/environment.py; openhcs/processing/backends/lib_registry/registry_service.py; tests/unit/test_library_registry_discovery.py; pyproject.toml.", @@ -3751,7 +3796,7 @@ "authority": [ { "path": "openhcs/processing/custom_functions/manager.py", - "sha256": "f87d622c4e03db3b6d7f977daa86187b892f59133be264ffa381472785247e56", + "sha256": "5cd7e955087f0bd9c527fc6307dff48f4aa959c607a3ec776b8379abac76fb36", "role": "Owns persisted source under the data directory, requires one decorated processing function per source, rejects collisions, and coordinates create, load, replace, and delete." }, { @@ -3771,7 +3816,7 @@ }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "e43085c1eaf0c836ce34e1a1dae7740488d91bdfea56844e7ddd4e6925561cf6", + "sha256": "00ff3bfe7a032bd3de59274697a69d12c9b898177cd8e839f9da3b130becaedf", "role": "Declares the Tools > Custom Functions > Manage Functions action." }, { @@ -3791,7 +3836,7 @@ }, { "path": "openhcs/mcp/context.py", - "sha256": "1292c9bb7173e6afe5aa6237b861c5f39bc2e854f364fd722c75a3c6b76f534e", + "sha256": "af9ec43138e05762fc97b34825b96e15f183aa56781d697c601eb79f14a1bc42", "role": "Supplies the local MCP tools with the session's endpoint function catalogue." }, { @@ -3811,7 +3856,7 @@ }, { "path": "tests/unit/test_custom_function_lifecycle.py", - "sha256": "70c87f0dbefd2c1a204d060c2211ec57f4f84023f7c6931d2cb7d81eb82e9252", + "sha256": "e333004437e29a3e44aeb93699f5fb0b61092bb9c48068e3bc0011781adbc885", "role": "Proves transactional create, load, replace, delete, collision, and source-revision behaviour." }, { @@ -3831,6 +3876,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/user_guide/custom_function_management.rst against the later B6 changes to openhcs/processing/custom_functions/manager.py, openhcs/pyqt_gui/main.py, openhcs/mcp/context.py, tests/unit/test_custom_function_lifecycle.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/user_guide/custom_function_management.rst in full and compared it against the changed authorities: custom_functions/manager.py and runtime_registry.py, custom_function_manager_dialog.py, openhcs/pyqt_gui/main.py, endpoint_function_catalog_service.py, openhcs/mcp/context.py, gallery_catalog.py, and the catalogue and lifecycle tests; checked the menu path, register_from_code signature and package export, source validation rules, ephemeral retention on reload, and catalogue invalidation; corrected the decorator requirement.", "Re-read docs/source/user_guide/custom_function_management.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/processing/custom_functions/manager.py; openhcs/processing/custom_functions/events.py; openhcs/processing/custom_functions/signals.py; tests/unit/test_custom_function_events.py; openhcs/pyqt_gui/dialogs/custom_function_manager_dialog.py; openhcs/pyqt_gui/main.py; tests/unit/test_function_catalog_zmq.py; openhcs/agent/services/endpoint_function_catalog_service.py; openhcs/pyqt_gui/dialogs/function_selector_dialog.py; openhcs/processing/custom_functions/runtime_registry.py; tests/unit/test_custom_function_lifecycle.py.", @@ -3865,12 +3911,12 @@ "authority": [ { "path": "openhcs/core/memory/decorators.py", - "sha256": "76f234785f4865257fd19c935e0aace82d1f5586198737fec619ee243b79cbf6", + "sha256": "2740c030304107079f1fcb08c43f40950d0b752cbe10d8412d519b140c7c1b1c", "role": "Exports one OpenHCS memory decorator per MemoryType, including numpy, accepting a contract argument." }, { "path": "openhcs/core/processing_contracts.py", - "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", + "sha256": "9b999e9210baf292f8fbf5bb3758c41a95e74401bfb1e1c127bae2dba625b0ee", "role": "Declares the ProcessingContract classes Pure2DContract, Pure3DContract, VolumetricToSliceContract, and FlexibleContract." }, { @@ -3880,7 +3926,7 @@ }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "e43085c1eaf0c836ce34e1a1dae7740488d91bdfea56844e7ddd4e6925561cf6", + "sha256": "00ff3bfe7a032bd3de59274697a69d12c9b898177cd8e839f9da3b130becaedf", "role": "Declares the Tools > Custom Functions > Create New Function action." }, { @@ -3890,7 +3936,7 @@ }, { "path": "openhcs/processing/custom_functions/manager.py", - "sha256": "f87d622c4e03db3b6d7f977daa86187b892f59133be264ffa381472785247e56", + "sha256": "5cd7e955087f0bd9c527fc6307dff48f4aa959c607a3ec776b8379abac76fb36", "role": "Validates and registers saved custom-function source into the function catalogue." }, { @@ -3900,7 +3946,7 @@ }, { "path": "tests/unit/test_custom_function_lifecycle.py", - "sha256": "70c87f0dbefd2c1a204d060c2211ec57f4f84023f7c6931d2cb7d81eb82e9252", + "sha256": "e333004437e29a3e44aeb93699f5fb0b61092bb9c48068e3bc0011781adbc885", "role": "Proves validated custom declarations publish atomically without mirrored catalogue state." }, { @@ -3915,6 +3961,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/user_guide/custom_functions.rst against the later B6 changes to openhcs/core/memory/decorators.py, openhcs/core/processing_contracts.py, openhcs/pyqt_gui/main.py, openhcs/processing/custom_functions/manager.py, tests/unit/test_custom_function_lifecycle.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/user_guide/custom_functions.rst in full and compared it against the changed authorities: the page's own earlier contract-import edit, openhcs/core/memory/decorators.py, unified_registry.py, custom_functions/manager.py and runtime_registry.py, external/arraybridge decorators.py, and the catalogue and lifecycle tests; checked the menu path, the decorator export, and the four contract classes in openhcs/core/processing_contracts.py; corrected the contract names in the list.", "On 2026-08-27, re-read docs/source/user_guide/custom_functions.rst in full against callable- and registry-owned form exclusions; challenged the authoring steps and confirmed that custom runtime inputs remain declaration-owned.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/memory/decorators.py; openhcs/processing/backends/lib_registry/unified_registry.py; openhcs/processing/custom_functions/manager.py; tests/unit/test_function_catalog_zmq.py; external/arraybridge/src/arraybridge/decorators.py; openhcs/processing/custom_functions/runtime_registry.py; tests/unit/test_custom_function_lifecycle.py.", @@ -3939,12 +3986,12 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "sha256": "7b5622b3d82b1e1d672eaedb14e054be81baef417a736adb06bea34fb6708b28", "role": "Declares DtypeConfig with default_dtype_conversion (NATIVE_OUTPUT default) and StepMaterializationConfig, from which the lazy step forms are generated." }, { "path": "openhcs/core/memory/decorators.py", - "sha256": "76f234785f4865257fd19c935e0aace82d1f5586198737fec619ee243b79cbf6", + "sha256": "2740c030304107079f1fcb08c43f40950d0b752cbe10d8412d519b140c7c1b1c", "role": "Exports OpenHCS memory decorators while delegating conversion semantics to ArrayBridge declarations." }, { @@ -3959,7 +4006,7 @@ }, { "path": "openhcs/processing/backends/processors/numpy_processor.py", - "sha256": "58df7297d1021decca547d9f5959b8d8e563544b4644ac4977452030ed8b30da", + "sha256": "9c3c8cc1d28188897e1f9d95941f1ef53db5daffb4247933816e7e596520f9b5", "role": "Declares the decorated create_projection function used in the 8-bit materialisation example." }, { @@ -3969,7 +4016,7 @@ }, { "path": "tests/unit/test_callable_contract.py", - "sha256": "cc220f53107a08b0406b6e0dd49896e3e4c3c0af6e527b0ef00d4fdaba8f0b2c", + "sha256": "b0ce0cba2e396e53f8cb9e95533ff918788e8c6dd60037a66e0aeda603bc64c2", "role": "Proves callable contract derivation for dimensions, memory types, dtype policy, and outputs." }, { @@ -3981,6 +4028,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/user_guide/dtype_conversion.rst against the later B6 changes to openhcs/core/config.py, openhcs/core/memory/decorators.py, openhcs/processing/backends/processors/numpy_processor.py, tests/unit/test_callable_contract.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/user_guide/dtype_conversion.rst in full and compared it against the changed authorities: openhcs/core/config.py, openhcs/core/memory/decorators.py, external/arraybridge decorators.py and array_operations.py, numpy_processor.py, external/PolyStore disk.py, test_callable_contract.py, and openhcs/core/steps/abstract.py; confirmed DtypeConfig.default_dtype_conversion and its NATIVE_OUTPUT default, the PRESERVE_INPUT and UINT8 members, LazyDtypeConfig and LazyStepMaterializationConfig imports from openhcs.core.config, create_projection, and step dtype_config/step_materialization_config parameters; no page change was needed.", "On 2026-08-27, re-read docs/source/user_guide/dtype_conversion.rst in full after callable contracts began projecting runtime-owned inputs into generic form exclusions; challenged the dtype task flow and found no behaviour change.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/config.py; openhcs/core/memory/decorators.py; external/arraybridge/src/arraybridge/decorators.py; tests/unit/test_callable_contract.py; openhcs/core/steps/abstract.py.", @@ -4004,12 +4052,12 @@ "authority": [ { "path": "openhcs/pyqt_gui/main.py", - "sha256": "e43085c1eaf0c836ce34e1a1dae7740488d91bdfea56844e7ddd4e6925561cf6", + "sha256": "00ff3bfe7a032bd3de59274697a69d12c9b898177cd8e839f9da3b130becaedf", "role": "Declares the Tools > Analysis Consolidation > Run Experimental Analysis action and its directory check for the config and results files." }, { "path": "openhcs/processing/backends/experimental_analysis/analysis.py", - "sha256": "17a701e0740eac3e11319fe73f10302bbcab603218349bfba4e07014aabc27bb", + "sha256": "2fe9bcfba30e739cbf0b74112bfec6ce8714f173bdbd08b6dde1e3b6abe78d6e", "role": "Parses the drug_curve_map and plate_groups sheets and applies replicate-local normalization and heatmap export." }, { @@ -4019,7 +4067,7 @@ }, { "path": "openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py", - "sha256": "2705d459204557c4f7092df3fd21ed687dcf56449ed361485382b1c0ce196444", + "sha256": "bd43ec00c265279585f6bcc9eb0a6b6a2eeedaacad6003a33a60ab181cd13a4d", "role": "Runs the directory workflow and writes the normalized, raw, and heatmap workbooks." }, { @@ -4029,7 +4077,7 @@ }, { "path": "tests/unit/test_experimental_analysis_engine.py", - "sha256": "641fadecf8c940cc2316a06771c74db5f3f2db9dafdfbfebdc535760d6d776ec", + "sha256": "c79887f97615197c25c340dd277189ebc331bce1f3e9bfec7466c57c92ebd6cf", "role": "Runs the documented MetaXpress directory task end to end." } ], @@ -4038,6 +4086,7 @@ ], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/user_guide/experimental_layouts.rst against the later B6 changes to openhcs/pyqt_gui/main.py, openhcs/processing/backends/experimental_analysis/analysis.py, openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py, tests/unit/test_experimental_analysis_engine.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/user_guide/experimental_layouts.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/main.py, the deleted openhcs/formats/experimental_analysis.py (moved to processing/backends/experimental_analysis/analysis.py), openhcs/core/config.py, unified_analysis_engine.py, and both experimental-analysis tests; checked the menu path, the five default filenames, normalization_method, and that variable_components and group_by remain step-config fields; no page change was needed.", "Re-read docs/source/user_guide/experimental_layouts.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/pyqt_gui/main.py; openhcs/formats/experimental_analysis.py; openhcs/core/config.py; openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py; tests/unit/test_experimental_layout_rows.py; tests/unit/test_experimental_analysis_engine.py.", @@ -4063,7 +4112,7 @@ "authority": [ { "path": "openhcs/pyqt_gui/main.py", - "sha256": "e43085c1eaf0c836ce34e1a1dae7740488d91bdfea56844e7ddd4e6925561cf6", + "sha256": "00ff3bfe7a032bd3de59274697a69d12c9b898177cd8e839f9da3b130becaedf", "role": "Composes the PyQt desktop main window that the openhcs command launches." }, { @@ -4078,18 +4127,19 @@ }, { "path": "openhcs/core/config.py", - "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "sha256": "7b5622b3d82b1e1d672eaedb14e054be81baef417a736adb06bea34fb6708b28", "role": "Declares the nested processing, materialization, and streaming configuration dataclasses on PipelineConfig and steps." }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "sha256": "d1216010dec1804c51aec9c774f298513638ca016d500fe6328d76a9a9bfb4d6", "role": "Compiles pipeline declarations into typed step plans and a CompiledExecutionBundle before execution." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/user_guide/index.rst against the later B6 changes to openhcs/pyqt_gui/main.py, openhcs/core/config.py, openhcs/core/pipeline/compiler.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/user_guide/index.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/main.py, test_ui_config_lifecycle.py, pipeline_document.py, and openhcs/core/pipeline/compiler.py; confirmed the Textual interface is still absent, the declaration is pipeline_steps plus PipelineConfig, nested step configuration still owns processing, input source, materialization and streaming options, and compilation still returns a CompiledExecutionBundle; all toctree targets exist; added openhcs/core/config.py for the nested-configuration claim; no page change was needed.", "Re-read docs/source/user_guide/index.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/pyqt_gui/main.py; tests/unit/pyqt_gui/test_ui_config_lifecycle.py; openhcs/core/pipeline_document.py; openhcs/core/pipeline/compiler.py.", @@ -4127,7 +4177,7 @@ }, { "path": "openhcs/pyqt_gui/services/reactor_providers.py", - "sha256": "aecf1c77636095408486cab53a95cb0fc044fcf2ead0d7b72ac674842cf69abe", + "sha256": "43634ee5d4ac6c65d681288c575ee0c9570140d592cb3856942ec69f47c59d90", "role": "Registers the log, process, status, and execution reactors consumed by desktop widgets." }, { @@ -4137,7 +4187,7 @@ }, { "path": "openhcs/runtime/zmq_execution_client.py", - "sha256": "147fb663d3ea60b1348063440e05b77d54b40634d1ae81cf4fb4f3b5457fbc99", + "sha256": "202619246805f8d2dfbfd5ca5427af34716e424f551ebc800d8516f7b49e07e5", "role": "Spawns the execution server with its own per-port log file and startup status file and reports connection state." }, { @@ -4149,6 +4199,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/user_guide/log_viewer.rst against the later B6 changes to openhcs/pyqt_gui/services/reactor_providers.py, openhcs/runtime/zmq_execution_client.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/user_guide/log_viewer.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/services/reactor_providers.py, openhcs/runtime/zmq_execution_client.py, and scripts/gallery_catalog.py; the reactor changes (axis-family grouping, session scope) and the execution server's per-port log and startup-status files are consistent with the page's claims that subprocesses write dedicated logs reported by the launching process and that readiness is recorded separately; no page change was needed.", "Reviewed docs/source/user_guide/log_viewer.rst in full; the recorded findings and disposition are the resulting page-specific editorial decision.", "Compared its factual claims directly with the hash-pinned files listed in authority: external/pyqt-reactive/src/pyqt_reactive/widgets/log_viewer.py; external/pyqt-reactive/src/pyqt_reactive/services/coalesced_scan_requests.py; openhcs/pyqt_gui/services/reactor_providers.py; openhcs/pyqt_gui/config.py; openhcs/runtime/zmq_execution_client.py.", @@ -4183,12 +4234,12 @@ }, { "path": "openhcs/agent/capabilities.py", - "sha256": "eaebe33af01d057d1d21b3701f958bc96ede3ec5d0bddddca9917c1164ef0042", + "sha256": "87c8e2e5aeb425b9a2eb690d48bdd6f5af7a327c389deac261d7e4e0da5d609f", "role": "Declares the full, desktop, authoring, and core surface profiles and derives the openhcs_ session tools from headless session operations." }, { "path": "openhcs/authoring/session/operations/datasets.py", - "sha256": "208274effc2e790775d90528211fa4f640eb01f7ca3fdd9c927c3bf71e39b632", + "sha256": "43d9edfed59abe6b88e242d39f1ddc8f5078559e04b3875da1da07b993295993", "role": "Declares the add, set-pipeline, initialize, compile, run, and stop dataset operations behind the session tools." }, { @@ -4218,7 +4269,7 @@ }, { "path": "benchmark/cellprofiler_benchmark_cli.py", - "sha256": "4025c591644e29f3df61e476ecae2143690e014336ea9f889aadeff4e692445e", + "sha256": "edc32916215a88b62702621dea9b692586572284caf9b781394b5784f62897ee", "role": "Declares the list-cases, inspect-run, inspect-measured, and run-measured commands, including the required --wait-timeout-ms." }, { @@ -4238,7 +4289,7 @@ }, { "path": "benchmark/control_service.py", - "sha256": "a1256b349ffdc8d04cab927326d97821462b61f57b29e564b61cc627302be49c", + "sha256": "f894cc090597ca04c273090c99f26fd0e3cfeffd1b7d44bbb94858892e17813c", "role": "Requires the manifest and every declared benchmark source location to satisfy the agent read policy." }, { @@ -4253,12 +4304,12 @@ }, { "path": "tests/unit/agent/test_mcp_server.py", - "sha256": "4e6f6b1761b2c0316c99984baad6ea2c97e5c76c3f155111b8e498d7358e3175", + "sha256": "6a03cf2de856060f39d221127f41189a0a8094c65856cb1c4cfa765210099b67", "role": "Proves MCP server tool registration, request handling, and lifecycle behaviour." }, { "path": "tests/unit/test_benchmark_control.py", - "sha256": "093b584eeb37cbb12089d5af082ad45acd779e9d2bdd5b3feaea08234fc22183", + "sha256": "2798cf80df1222c80bb3aa9955b5c20c0c90e415a1795017779d22460f442bf2", "role": "Proves benchmark capability binding, path policy, progress, artifacts, and lightweight imports." } ], @@ -4269,6 +4320,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/user_guide/mcp_clients.rst against the later B6 changes to openhcs/agent/capabilities.py, openhcs/authoring/session/operations/datasets.py, benchmark/cellprofiler_benchmark_cli.py, benchmark/control_service.py, tests/unit/agent/test_mcp_server.py, tests/unit/test_benchmark_control.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/user_guide/mcp_clients.rst in full and compared it against the changed authorities: the page's own session-tool edit, openhcs/mcp/bootstrap.py, client_registration.py, openhcs/agent/capabilities.py, the mcpb manifest, test_mcp_server.py, gallery_catalog.py, the benchmark CLI, contracts, control and control_service, openhcs/agent/path_policy.py, and test_benchmark_control.py; confirmed every named session tool derives from a declared headless operation, the four surface names and desktop default, the benchmark subcommands and flags, the path-policy variables, and the packaging paths; restored the --wait-timeout-ms requirement and renamed the window.", "Reviewed docs/source/user_guide/mcp_clients.rst in full; the recorded findings and disposition are the resulting page-specific editorial decision.", "Compared its factual claims directly with the hash-pinned files listed in authority: openhcs/mcp/bootstrap.py; openhcs/mcp/client_registration.py; openhcs/mcp/lifecycle.py; openhcs/agent/capabilities.py; packaging/mcpb/openhcs/manifest.json; tests/unit/agent/test_mcp_client_registration.py; tests/unit/agent/test_mcp_server.py.", @@ -4322,7 +4374,7 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "sha256": "7b5622b3d82b1e1d672eaedb14e054be81baef417a736adb06bea34fb6708b28", "role": "Declares StreamingDefaults (persistent, host, transport_mode, enabled), partition_filter, and the Napari and Fiji streaming configs with their ports." }, { @@ -4337,12 +4389,12 @@ }, { "path": "pyproject.toml", - "sha256": "3592cc37444e2e1f5bb0284e939643039c0d513217861af8133dd8bd6e8a4325", + "sha256": "c3a2bea22aa6872940125f4aaac72c9eac72084fefc6c2cefdb54b5273361e4a", "role": "Declares the gui, napari, fiji, and viz extras." }, { "path": "openhcs/runtime/viewer_protocol.py", - "sha256": "c4bdae4bae6dee412f0d27a9cf75a95db53b3bd95c03cee7d046be16b52fe5b8", + "sha256": "f126aaf4f5c0dcc9429b7aae8ce67985260241ec7d2f3d5a948573658cf079a3", "role": "Declares the transport-independent viewer messages and incremental settlement-progress fields." }, { @@ -4372,22 +4424,22 @@ }, { "path": "openhcs/runtime/fiji_viewer_server.py", - "sha256": "ee246d32ed828bbf4665cc8a411e807da29bd43c2475012243f27e93cb550410", + "sha256": "d3c2114d16f71111562474b17a5ef4a54a7391275293c4678259e4410d11d15f", "role": "Implements Fiji stream reception, image-stack updates, ROI handling, and endpoint lifecycle." }, { "path": "tests/unit/test_napari_streaming_handlers.py", - "sha256": "93244d4d09c5cd6d7a2645fca2482578ad52646f3bb9e6de5ebe505b157fbfea", + "sha256": "860e234a39f36f143491e6f1edc9515c1d9cde88590759252378f8d23c0bab12", "role": "Proves Napari streaming updates layers and carries ROI navigation through the declared protocol." }, { "path": "tests/unit/test_viewer_result_element_selection.py", - "sha256": "b1dae580f0291f53be9453df6f6c6df5458046bd3ff947b550f7ddea94b29f96", + "sha256": "a7b348fab106da4c89e58ed32225495aa27792132ef7488657f29821dfd6da92", "role": "Proves focused layer isolation is rejected before mutation when any route or axis is invalid." }, { "path": "tests/unit/test_fiji_viewer_server.py", - "sha256": "da575dfb02895ecc831b34d6bdf6b58112b986556459444e9701cbf43221ea22", + "sha256": "bed61ba6a34937e7827b5965a63ca1afe3874d6ecc27b3c8a32721628e8b4e36", "role": "Proves Fiji endpoint lifecycle, streamed image updates, and ROI message handling." }, { @@ -4402,6 +4454,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/user_guide/real_time_visualization.rst against the later B6 changes to openhcs/core/config.py, pyproject.toml, openhcs/runtime/viewer_protocol.py, openhcs/runtime/fiji_viewer_server.py, tests/unit/test_napari_streaming_handlers.py, tests/unit/test_viewer_result_element_selection.py, tests/unit/test_fiji_viewer_server.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/user_guide/real_time_visualization.rst in full and compared it against the changed authorities: the page's own partition_filter edit, openhcs/core/config.py, viewer_protocol.py, viewer_controls.py, viewer_window_service.py, napari_viewer_server.py, fiji_viewer_server.py, napari_streaming_handlers.py, the three viewer tests, and gallery_catalog.py; confirmed the extras, the per-step streaming fields, persistent and partition_filter, max_rois, the selected-layer overlay controller, and forward-progress settlement; corrected the step-3 option list.", "Re-read docs/source/user_guide/real_time_visualization.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: openhcs/core/config.py; openhcs/runtime/viewer_protocol.py; openhcs/runtime/napari_viewer_server.py; openhcs/runtime/fiji_viewer_server.py; tests/unit/test_napari_streaming_handlers.py; tests/unit/test_fiji_viewer_server.py; openhcs/runtime/napari_streaming_handlers.py.", From 68147644aa680aa12a8efeb3876e0c48dd51e4cf Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 04:15:39 -0400 Subject: [PATCH 38/84] Revert "B6: black and the 3.11 syntax check over every Python file changed since code-quality last passed" This reverts commit cf0b8eb8c6a2aceb3493a0aa9159745cddecc567. --- benchmark/adapters/cellprofiler.py | 7 +- .../agent_validation/mcp_attempt_recorder.py | 4 +- benchmark/cellprofiler_benchmark_cli.py | 8 +- benchmark/control_service.py | 4 +- .../equivalence/object_label_measurements.py | 6 +- benchmark/equivalence/runtime.py | 2 +- benchmark/equivalence/table_snapshots.py | 8 +- benchmark/openhcs_measured_run.py | 4 +- benchmark/session_measured_run.py | 8 +- benchmark/well_throughput_scaling.py | 21 +- openhcs/agent/capabilities.py | 37 +- openhcs/agent/dto/__init__.py | 8 +- openhcs/agent/dto/functions.py | 118 +-- openhcs/agent/dto/plate.py | 17 +- openhcs/agent/dto/ui_bridge.py | 4 + .../artifact_plan_inspection_service.py | 3 +- .../services/function_catalog_service.py | 83 +- .../services/plate_inspection_service.py | 30 +- .../agent/services/plate_streaming_service.py | 28 +- .../agent/services/selected_plate_service.py | 3 +- openhcs/agent/services/ui_bridge_service.py | 9 +- openhcs/authoring/session/compile_batch.py | 20 +- openhcs/authoring/session/dataset_document.py | 8 +- openhcs/authoring/session/debug_runs.py | 4 +- openhcs/authoring/session/events.py | 4 +- .../authoring/session/operations/datasets.py | 21 +- .../authoring/session/operations/pipelines.py | 4 +- openhcs/authoring/session/progress.py | 5 +- openhcs/authoring/session/session.py | 95 +- openhcs/authoring/session/submission.py | 9 +- openhcs/authoring/session/views.py | 18 +- openhcs/core/aligned_image_payload.py | 323 ++---- openhcs/core/artifacts.py | 12 +- openhcs/core/axes.py | 33 +- openhcs/core/callable_contract.py | 198 +--- openhcs/core/compiled_step_plan.py | 29 +- openhcs/core/component_filters.py | 10 +- openhcs/core/config.py | 34 +- openhcs/core/dataset_sources/choice.py | 5 +- openhcs/core/dataset_sources/interfaces.py | 9 +- .../core/dataset_sources/openhcs_format.py | 109 +- openhcs/core/dataset_sources/source.py | 7 +- .../dataset_sources/source_bindings_source.py | 7 +- openhcs/core/dataset_sources/source_schema.py | 1 + openhcs/core/debug.py | 6 +- openhcs/core/equivalence/keys.py | 4 +- openhcs/core/equivalence/measurement_facts.py | 7 +- .../core/equivalence/measurement_features.py | 8 +- openhcs/core/equivalence/measurement_rows.py | 5 +- openhcs/core/execution_state.py | 4 +- openhcs/core/function_patterns.py | 89 +- openhcs/core/function_reference.py | 12 +- openhcs/core/image_file_serialization.py | 8 +- openhcs/core/image_payload_execution_mode.py | 15 +- openhcs/core/measurement_dialect.py | 34 +- openhcs/core/measurement_feature_queries.py | 42 +- openhcs/core/measurement_image_alignment.py | 6 +- .../core/measurement_row_materialization.py | 1 + openhcs/core/memory/decorators.py | 12 +- openhcs/core/metadata_cache.py | 1 + openhcs/core/orchestrator/bundle_execution.py | 1 - openhcs/core/orchestrator/execution_result.py | 28 +- openhcs/core/orchestrator/orchestrator.py | 12 +- openhcs/core/orchestrator/worker_execution.py | 21 +- openhcs/core/orchestrator/worker_profiling.py | 7 +- openhcs/core/partition_filter.py | 23 +- openhcs/core/pipeline/compilation_session.py | 15 +- openhcs/core/pipeline/compiler.py | 82 +- .../pipeline/funcstep_contract_validator.py | 1 + openhcs/core/pipeline/function_contracts.py | 17 +- openhcs/core/pipeline/path_planner.py | 124 +-- openhcs/core/post_execute.py | 4 +- openhcs/core/processing_contracts.py | 122 +-- openhcs/core/progress/projection.py | 19 +- openhcs/core/progress/runtime_tree.py | 28 +- openhcs/core/progress/types.py | 20 +- openhcs/core/runtime_adapters.py | 3 +- openhcs/core/runtime_batch_contracts.py | 37 +- openhcs/core/runtime_exports.py | 4 +- openhcs/core/runtime_measurements.py | 11 +- .../core/runtime_object_label_aggregation.py | 8 +- openhcs/core/runtime_object_label_building.py | 3 +- openhcs/core/runtime_object_labels.py | 202 ++-- openhcs/core/runtime_relationships.py | 4 +- openhcs/core/runtime_stores.py | 86 +- openhcs/core/source_binding_context.py | 4 +- openhcs/core/source_binding_selection.py | 105 +- openhcs/core/source_binding_workspace.py | 98 +- openhcs/core/source_bindings.py | 87 +- openhcs/core/source_image_provenance.py | 33 +- openhcs/core/source_metadata.py | 41 +- openhcs/core/source_workspace_projection.py | 195 ++-- .../function_artifact_materialization.py | 65 +- openhcs/core/steps/function_execution.py | 10 +- openhcs/core/steps/function_io.py | 12 +- .../core/steps/function_output_identity.py | 7 +- openhcs/core/steps/function_outputs.py | 54 +- openhcs/core/steps/function_runtime.py | 53 +- .../core/steps/stream_component_semantics.py | 9 +- openhcs/core/viewer_streaming_service.py | 27 +- openhcs/core/virtual_workspace_metadata.py | 56 +- openhcs/demo/basic_pipeline.py | 12 +- .../microscopy/analysis_consolidation.py | 28 +- openhcs/domains/microscopy/axes.py | 8 +- openhcs/formats/pattern/pattern_discovery.py | 16 +- .../interop/cellprofiler/analyst_export.py | 25 +- .../cellprofiler/database_column_dialect.py | 4 +- .../cellprofiler/image_set_numbering.py | 12 +- .../cellprofiler/measurement_dialect.py | 4 +- .../cellprofiler/measurement_lookup.py | 1 + .../interop/cellprofiler/measurement_scope.py | 1 + .../cellprofiler/module_artifact_contracts.py | 1 + .../module_artifact_declarations.py | 37 +- .../cellprofiler/module_declarations.py | 20 +- .../interop/cellprofiler/pipeline_import.py | 10 +- .../runtime/function_contract_execution.py | 8 +- .../cellprofiler/runtime/invocation.py | 14 +- .../runtime/measurement_recording.py | 17 +- .../cellprofiler/runtime/module_execution.py | 67 +- .../interop/cellprofiler/workspace_export.py | 5 +- openhcs/mcp/context.py | 10 +- openhcs/mcp/dev_client_commands/ui.py | 12 +- openhcs/mcp/dev_client_core.py | 15 +- openhcs/mcp/dev_client_renderers/pipeline.py | 8 +- openhcs/mcp/dev_client_renderers/plate.py | 16 +- openhcs/mcp/dev_client_renderers/ui_bridge.py | 56 +- openhcs/mcp/dev_client_session.py | 6 +- openhcs/mcp/execution.py | 8 +- openhcs/mcp/server.py | 16 +- openhcs/microscopes/bioformats.py | 6 +- openhcs/microscopes/imagexpress.py | 10 +- openhcs/microscopes/omero.py | 8 +- openhcs/microscopes/opera_phenix.py | 2 + openhcs/microscopes/vendor_layout.py | 9 +- openhcs/omero/plugin/omero_openhcs/views.py | 8 +- .../backends/analysis/neurite_outgrowth.py | 397 +++---- .../backends/cellprofiler/alignment.py | 5 +- .../backends/cellprofiler/area_occupied.py | 39 +- .../cellprofiler/calculate_statistics.py | 5 +- .../backends/cellprofiler/classification.py | 117 +-- .../backends/cellprofiler/colocalization.py | 31 +- .../processing/backends/cellprofiler/color.py | 65 +- .../processing/backends/cellprofiler/crop.py | 5 +- .../backends/cellprofiler/display_modules.py | 6 +- .../processing/backends/cellprofiler/edge.py | 10 +- .../cellprofiler/feature_enhancement.py | 9 +- .../backends/cellprofiler/granularity.py | 1 + .../backends/cellprofiler/illumination.py | 12 +- .../backends/cellprofiler/image_geometry.py | 15 +- .../backends/cellprofiler/image_math.py | 4 +- .../backends/cellprofiler/image_quality.py | 1 + .../backends/cellprofiler/intensity.py | 19 +- .../cellprofiler/intensity_distribution.py | 26 +- .../backends/cellprofiler/median_filter.py | 7 +- .../backends/cellprofiler/morphology.py | 51 +- .../backends/cellprofiler/neighbors.py | 13 +- .../backends/cellprofiler/object_images.py | 11 +- .../backends/cellprofiler/object_overlap.py | 5 +- .../backends/cellprofiler/outlines.py | 25 +- .../primary_object_diagnostics.py | 16 +- .../backends/cellprofiler/primary_objects.py | 16 +- .../backends/cellprofiler/relationships.py | 5 +- .../backends/cellprofiler/secondary.py | 2 + .../processing/backends/cellprofiler/shape.py | 24 +- .../backends/cellprofiler/skeleton.py | 6 +- .../backends/cellprofiler/smoothing.py | 21 +- .../cellprofiler/spreadsheet_export.py | 55 +- .../backends/cellprofiler/texture.py | 2 + .../backends/cellprofiler/thresholding.py | 14 +- .../backends/cellprofiler/tracking.py | 39 +- .../backends/cellprofiler/watershed.py | 9 +- .../processing/backends/cellprofiler/worms.py | 19 +- .../backends/cellprofiler/zernike.py | 1 + .../backends/enhance/basic_processor_jax.py | 1 + .../experimental_analysis/analysis.py | 4 +- .../experimental_analysis/result_formats.py | 4 +- .../unified_analysis_engine.py | 36 +- .../backends/lib_registry/openhcs_registry.py | 10 +- .../backends/lib_registry/registry_service.py | 28 +- .../backends/lib_registry/unified_registry.py | 12 +- .../backends/processors/cupy_processor.py | 4 +- .../backends/processors/jax_processor.py | 4 +- .../backends/processors/numpy_processor.py | 25 +- .../processors/pyclesperanto_processor.py | 4 +- .../processors/tensorflow_processor.py | 4 +- .../backends/processors/torch_processor.py | 4 +- .../processing/custom_functions/manager.py | 25 +- .../processing/custom_functions/validation.py | 20 +- openhcs/processing/materialization/core.py | 127 +-- .../pipelines/czi_brain_axon_cellbody.py | 7 +- .../loose_operaphenix_neurite_outgrowth.py | 4 +- ...uroncyto_ii_crossover_neurite_outgrowth.py | 8 +- openhcs/pyqt_gui/main.py | 4 +- .../pyqt_gui/services/reactor_providers.py | 5 +- openhcs/pyqt_gui/services/ui_agent_bridge.py | 9 +- .../services/ui_bridge_composition.py | 1 + .../services/ui_bridge_object_state.py | 4 +- .../services/ui_bridge_plate_manager.py | 1 - .../services/ui_bridge_session_actions.py | 10 +- .../pyqt_gui/services/ui_bridge_windows.py | 1 + openhcs/pyqt_gui/session_rendering.py | 4 +- openhcs/pyqt_gui/widgets/image_browser.py | 9 +- openhcs/pyqt_gui/widgets/pipeline_editor.py | 28 +- .../services/pipeline_editor_workflows.py | 16 +- openhcs/runtime/fiji_viewer_server.py | 25 +- openhcs/runtime/viewer_protocol.py | 152 +-- openhcs/runtime/zmq_compilation.py | 4 +- openhcs/runtime/zmq_control.py | 17 +- openhcs/runtime/zmq_execution_client.py | 15 +- openhcs/runtime/zmq_execution_server.py | 17 +- openhcs/runtime/zmq_execution_signature.py | 4 +- .../runtime/zmq_orchestrator_environment.py | 4 +- openhcs/runtime/zmq_server_hooks.py | 4 +- ...notate_wells_with_experimental_metadata.py | 5 +- scripts/capture_media_gallery.py | 36 +- scripts/check_basicpy_source.py | 17 +- scripts/figures/generate_figure_overlays.py | 5 +- scripts/master_multi_plate_demo.py | 4 +- scripts/mcp_assay_showcase.py | 4 +- scripts/mcp_memory_diagnostic.py | 40 +- scripts/mcp_thesis_demo_live.py | 47 +- scripts/prototype_flatten_object_state.py | 7 +- scripts/smoke_installed_mcp.py | 9 +- scripts/test_real_global_config.py | 9 +- .../diagnostics/check_owned_bootstrap_live.py | 630 ++++-------- .../diagnostics/volume_projection_fixture.py | 43 +- .../test_agent_observation_export_zmq.py | 23 +- .../test_cellprofiler_generated_pipeline.py | 4 +- .../test_cellprofiler_official30_zmq.py | 3 +- .../test_chained_callable_lineage_journey.py | 72 +- .../test_custom_source_transport.py | 84 +- .../test_image_math_publication_620.py | 162 +-- .../integration/test_measured_cli_session.py | 8 +- .../test_measurement_declaration_journey.py | 297 ++---- .../test_neurite_outgrowth_synthetic_plate.py | 11 +- ...rimary_segmentation_diagnostics_journey.py | 10 +- tests/integration/test_session_journey.py | 10 +- ...est_source_schema_aggregate_publication.py | 86 +- .../test_end_to_end_workflow_foundation.py | 8 +- tests/pyqt_gui/test_session_gui_journey.py | 11 +- tests/unit/agent/test_agent_services.py | 53 +- .../test_authoring_roundtrip_boundaries.py | 4 +- .../agent/test_callable_artifact_reference.py | 188 +--- tests/unit/agent/test_capabilities.py | 57 +- .../agent/test_cold_inspection_feedback.py | 89 +- ...st_compile_inspection_declared_callable.py | 57 +- .../agent/test_compile_selector_authoring.py | 4 +- .../test_declared_callable_documentation.py | 64 +- .../agent/test_function_declaration_help.py | 16 +- .../agent/test_imagexpress_readonly_paths.py | 15 +- .../unit/agent/test_knowledge_base_service.py | 3 +- .../test_mcp_affine_inspection_progress.py | 153 +-- .../agent/test_mcp_affine_real_compile.py | 48 +- .../test_mcp_dev_client_config_renderer.py | 34 +- .../test_mcp_dev_client_pipeline_results.py | 134 +-- .../test_mcp_dev_client_plate_renderers.py | 202 +--- .../agent/test_mcp_dev_client_renderers.py | 24 +- .../test_mcp_dev_client_sampling_results.py | 289 ++---- .../test_mcp_dev_client_ui_action_results.py | 269 ++--- .../agent/test_mcp_dev_client_ui_renderers.py | 164 +-- .../test_mcp_dev_client_viewer_renderers.py | 777 ++++---------- .../agent/test_mcp_plate_stream_progress.py | 148 +-- .../test_mcp_process_resource_lifetime.py | 54 +- tests/unit/agent/test_mcp_server.py | 551 ++++------ .../test_physical_source_inspection_445.py | 50 +- .../test_plate_inspection_cold_runtime.py | 6 +- .../agent/test_plate_inspection_service.py | 16 +- .../agent/test_plate_streaming_service.py | 78 +- .../test_progressive_authoring_context.py | 21 +- .../test_public_callable_enum_ingress.py | 66 +- .../test_selected_plane_reopen_stream.py | 5 +- tests/unit/agent/test_u1_g8_guards.py | 21 +- .../agent/test_ui_bridge_operation_family.py | 3 +- tests/unit/agent/test_ui_bridge_service.py | 32 +- .../agent/test_user_facing_help_projection.py | 4 +- .../test_agent_validation_attempt_recorder.py | 29 +- .../unit/cellprofiler_runtime_test_support.py | 2 +- tests/unit/desktop/test_desktop_restart.py | 4 +- tests/unit/progress/test_debug_projection.py | 4 +- tests/unit/pyqt_gui/session_harness.py | 4 +- ...test_concurrent_initialization_progress.py | 4 +- .../test_config_preview_formatters.py | 10 +- tests/unit/pyqt_gui/test_debug_toolbar.py | 8 +- .../pyqt_gui/test_main_config_propagation.py | 10 +- tests/unit/pyqt_gui/test_main_status_bar.py | 5 +- .../pyqt_gui/test_main_window_file_actions.py | 10 +- .../pyqt_gui/test_one_persistence_format.py | 10 +- ...est_parameter_form_manager_cross_window.py | 28 +- ...pipeline_code_document_transactionality.py | 8 +- .../pyqt_gui/test_pipeline_editor_widget.py | 23 +- .../test_plate_compile_connection_action.py | 6 +- .../pyqt_gui/test_plate_manager_widget.py | 65 +- .../pyqt_gui/test_session_batch_workflows.py | 60 +- .../test_synthetic_plate_pipeline_document.py | 12 +- tests/unit/pyqt_gui/test_ui_agent_bridge.py | 73 +- .../test_ui_bridge_live_measurements.py | 4 +- .../pyqt_gui/test_ui_window_flash_snapshot.py | 73 +- .../unit/pyqt_gui/test_zmq_client_service.py | 5 +- tests/unit/remote_sensing_witness_pipeline.py | 3 +- tests/unit/test_ack_return_route_journey.py | 68 +- ...cquisition_calibration_pipeline_journey.py | 4 +- tests/unit/test_aggregate_source_scope.py | 211 +--- .../unit/test_artifact_publication_journey.py | 121 +-- ...utomatic_image_role_materialization_435.py | 555 +++------- tests/unit/test_axis_family_guards.py | 63 +- tests/unit/test_axis_family_witness.py | 24 +- tests/unit/test_benchmark_control.py | 5 +- .../test_bioformats_microscope_handler.py | 8 +- .../test_calibrated_gaussian_spatial_axes.py | 87 +- tests/unit/test_callable_abi_preparation.py | 325 ++---- tests/unit/test_callable_contract.py | 100 +- .../test_canonical_custom_source_lookup.py | 145 +-- tests/unit/test_catalog_current_readiness.py | 154 +-- tests/unit/test_cellprofiler_adapter.py | 3 +- ...rofiler_additional_prepared_payload_abi.py | 100 +- .../unit/test_cellprofiler_analyst_export.py | 205 +--- ...test_cellprofiler_artifact_declarations.py | 29 +- .../test_cellprofiler_backend_public_api.py | 2 +- ..._display_manual_projection_declarations.py | 4 +- .../test_cellprofiler_export_equivalence.py | 32 +- ...cellprofiler_full_stack_materialization.py | 101 +- ...ellprofiler_function_contract_execution.py | 248 ++--- ...lprofiler_geometry_prepared_payload_abi.py | 34 +- ..._identify_secondary_replacement_primary.py | 9 +- tests/unit/test_cellprofiler_image_quality.py | 8 +- .../unit/test_cellprofiler_library_loading.py | 56 +- .../test_cellprofiler_measurement_rows.py | 27 +- .../test_cellprofiler_module_execution.py | 582 ++++------- ...cellprofiler_module_execution_semantics.py | 7 +- ...ellprofiler_morphology_public_contracts.py | 5 +- ...t_cellprofiler_output_record_occurrence.py | 87 +- .../test_cellprofiler_prepared_payload_abi.py | 58 +- .../unit/test_cellprofiler_runtime_adapter.py | 192 ++-- ...profiler_runtime_callable_introspection.py | 4 +- tests/unit/test_cellprofiler_save_images.py | 48 +- .../unit/test_cellprofiler_source_bindings.py | 24 +- ..._cellprofiler_source_bindings_ingestion.py | 9 +- ...st_cellprofiler_special_io_declarations.py | 13 +- .../test_cellprofiler_spreadsheet_export.py | 263 ++--- .../test_cellprofiler_watershed_provider.py | 54 +- .../test_cellprofiler_workspace_export.py | 21 +- ...est_classification_two_vector_authoring.py | 75 +- ...test_collapsed_persisted_occurrence_435.py | 4 +- ...colocalization_metric_setting_authority.py | 2 +- ...st_colocalization_preparation_readiness.py | 12 +- tests/unit/test_compilation_paths.py | 33 +- tests/unit/test_compilation_session.py | 40 +- tests/unit/test_compiled_execution.py | 20 +- ..._completed_output_publication_lifecycle.py | 152 +-- tests/unit/test_custom_function_lifecycle.py | 127 +-- .../unit/test_czi_brain_axon_cellbody_demo.py | 26 +- tests/unit/test_dataset_source_family.py | 25 +- tests/unit/test_dataset_source_guards.py | 8 +- tests/unit/test_debug_runtime.py | 22 +- ...test_declared_singleton_composition_437.py | 48 +- .../unit/test_derived_role_publication_435.py | 434 +++----- tests/unit/test_documented_dimensionality.py | 4 +- .../unit/test_experimental_analysis_engine.py | 8 +- .../test_exported_relationship_equivalence.py | 89 +- .../unit/test_exported_table_axis_snapshot.py | 22 +- tests/unit/test_fast_speckle_opening.py | 57 +- tests/unit/test_fiji_viewer_server.py | 22 +- .../unit/test_funcstep_contract_validator.py | 7 +- .../test_function_artifact_materialization.py | 386 ++----- tests/unit/test_function_artifact_outputs.py | 137 +-- ...st_function_execution_lifecycle_anchors.py | 3 +- tests/unit/test_function_io.py | 7 +- tests/unit/test_function_outputs.py | 164 ++- .../unit/test_function_reference_identity.py | 10 +- ...test_function_runtime_source_projection.py | 197 ++-- .../unit/test_function_step_execution_plan.py | 17 +- .../test_function_step_execution_scope.py | 72 +- tests/unit/test_function_step_transport.py | 98 +- .../test_g5_processing_contract_guards.py | 4 +- tests/unit/test_g7_runtime_protocol_guards.py | 16 +- .../test_gray_to_color_binding_axis_432.py | 159 +-- ..._gray_to_color_payload_axis_compile_432.py | 230 ++--- tests/unit/test_grid_addressed_axis.py | 5 +- tests/unit/test_image_plane_contracts.py | 45 +- .../test_implicit_main_flow_compilation.py | 2 +- tests/unit/test_installed_mcp_demo.py | 30 +- .../unit/test_invocation_contract_provider.py | 4 +- ...nvocation_input_source_context_identity.py | 21 +- .../test_l3_zmqruntime_vocabulary_guards.py | 10 +- tests/unit/test_library_registry_discovery.py | 6 +- .../test_logical_image_output_snapshot.py | 18 +- tests/unit/test_matched_cellprofiler_batch.py | 9 +- .../test_materialization_batch_outcomes.py | 20 +- .../unit/test_materialization_flag_planner.py | 4 +- tests/unit/test_mcp_release_workflow.py | 4 +- tests/unit/test_measurement_dialect.py | 20 +- ...test_measureobjectintensitydistribution.py | 4 +- .../unit/test_named_checkpoint_publication.py | 105 +- .../test_napari_accepted_work_settlement.py | 200 ++-- .../test_napari_roi_manager_integration.py | 40 +- tests/unit/test_napari_streaming_handlers.py | 396 +++---- tests/unit/test_native_viewer_presentation.py | 25 +- tests/unit/test_neurite_outgrowth.py | 356 ++----- tests/unit/test_neuroncyto_ii_neurite_demo.py | 24 +- tests/unit/test_nlm_slice_contract.py | 4 +- tests/unit/test_openhcs_adapter.py | 6 +- tests/unit/test_openhcs_measured_run.py | 10 +- tests/unit/test_orchestrator_lane_planning.py | 8 +- .../test_path_planner_axis_source_scope.py | 4 +- .../unit/test_path_planner_materialization.py | 169 ++- tests/unit/test_payload_axis_compile_gate.py | 48 +- tests/unit/test_persisted_image_metadata.py | 69 +- .../test_persisted_roi_only_publication.py | 106 +- .../unit/test_persisted_semantic_metadata.py | 28 +- tests/unit/test_plane_nlm_declaration.py | 67 +- tests/unit/test_plane_store_sources.py | 8 +- tests/unit/test_plate_image_inventory.py | 4 +- tests/unit/test_primary_object_diagnostics.py | 169 +-- tests/unit/test_process_local_cache.py | 5 +- .../test_processor_operations_written_once.py | 19 +- tests/unit/test_pycodify_formatters.py | 24 +- ...gistered_configuration_source_readiness.py | 26 +- .../test_rescale_stream_plane_declaration.py | 192 +--- tests/unit/test_returned_output_abi.py | 6 +- .../test_runner_cellprofiler_compatibility.py | 4 +- .../test_runtime_adapter_source_artifacts.py | 65 +- .../unit/test_runtime_context_declaration.py | 41 +- tests/unit/test_runtime_equivalence.py | 10 +- .../unit/test_runtime_equivalence_package.py | 9 +- .../unit/test_runtime_execution_validation.py | 3 +- .../test_runtime_image_singleton_alignment.py | 13 +- tests/unit/test_runtime_slice_projection.py | 59 +- tests/unit/test_runtime_value_store.py | 430 +++----- tests/unit/test_runtime_values.py | 144 ++- .../test_saved_source_label_plane_domain.py | 194 +--- ...t_selected_plane_stream_materialization.py | 61 +- ...test_singleton_rgb_label_invocation_477.py | 147 +-- tests/unit/test_smoke_installed_gui.py | 4 +- tests/unit/test_snapshot_deadline.py | 24 +- tests/unit/test_source_binding_workspace.py | 299 ++---- tests/unit/test_source_bindings.py | 110 +- ..._source_bound_matched_image_set_anchors.py | 36 +- ..._source_provenance_constructor_defaults.py | 5 +- tests/unit/test_source_spatial_domain.py | 7 +- tests/unit/test_source_tile_geometry.py | 41 +- tests/unit/test_stream_component_semantics.py | 26 +- tests/unit/test_streaming_service.py | 60 +- tests/unit/test_u1_session_guards.py | 39 +- .../test_unified_registry_runtime_payloads.py | 41 +- tests/unit/test_viewer_receipt_replay.py | 203 ++-- .../test_viewer_result_element_selection.py | 32 +- tests/unit/test_viewer_spatial_sample.py | 4 +- tests/unit/test_well_throughput_scaling.py | 20 +- ...test_zarr_input_conversion_subdirectory.py | 10 +- tests/unit/test_zmq_compiled_pipeline_run.py | 26 +- tests/unit/test_zmq_execution_server.py | 62 +- tests/unit/test_zmq_progress.py | 25 +- tests/unit/test_zmq_submission_timeout.py | 16 +- validation/public_two_vector_388.py | 967 +++++------------- 454 files changed, 7379 insertions(+), 16436 deletions(-) diff --git a/benchmark/adapters/cellprofiler.py b/benchmark/adapters/cellprofiler.py index 178b19021f..fc3b77a141 100644 --- a/benchmark/adapters/cellprofiler.py +++ b/benchmark/adapters/cellprofiler.py @@ -827,8 +827,8 @@ def prepare( selected_input_dir = selected_input_path.input_dir() file_list_path = None if requires_flat_input_dir: - selected_source_universe = ( - selected_source_universe.materialize_flat_input_dir(selected_input_dir) + selected_source_universe = selected_source_universe.materialize_flat_input_dir( + selected_input_dir ) else: file_list_path = self._write_file_list( @@ -1285,8 +1285,7 @@ def native_cellprofiler_partition_filter_scope_slug( filter_slug = _scope_slug_fragment(partition_filter) else: filter_slug = ( - "_".join(_scope_slug_fragment(value) for value in partition_filter) - or "empty" + "_".join(_scope_slug_fragment(value) for value in partition_filter) or "empty" ) return f"wells_{partition_filter_config.partition_filter_mode.value}_{filter_slug}" diff --git a/benchmark/agent_validation/mcp_attempt_recorder.py b/benchmark/agent_validation/mcp_attempt_recorder.py index c607550efc..f8f5efc98d 100644 --- a/benchmark/agent_validation/mcp_attempt_recorder.py +++ b/benchmark/agent_validation/mcp_attempt_recorder.py @@ -253,7 +253,9 @@ def _derive_evidence( receipt_paths.append(receipt.payload_path) successful_results.extend(command_results) tool_names = tuple(result.tool for result in successful_results) - declarations = tuple(get_agent_capability(tool_name) for tool_name in tool_names) + declarations = tuple( + get_agent_capability(tool_name) for tool_name in tool_names + ) requirements: set[DslRequirement] = set() explanations: dict[DslRequirement, str] = {} diff --git a/benchmark/cellprofiler_benchmark_cli.py b/benchmark/cellprofiler_benchmark_cli.py index 22d3832e21..7a149cf0d0 100644 --- a/benchmark/cellprofiler_benchmark_cli.py +++ b/benchmark/cellprofiler_benchmark_cli.py @@ -321,9 +321,7 @@ def run(self, args: argparse.Namespace) -> int: ) return 130 except MeasuredRunEnded as ended: - print( - json.dumps(to_jsonable(ended.row), sort_keys=True), file=sys.stderr - ) + print(json.dumps(to_jsonable(ended.row), sort_keys=True), file=sys.stderr) raise RuntimeError( f"Ordinary pipeline run did not complete: {ended}" ) from ended @@ -826,9 +824,7 @@ def configure( required=True, help="MODE_LABEL=qualified_summary.csv; repeat for measured well/worker modes.", ) - parser.add_argument( - "--scope", choices=("execution", "total", "amortization"), required=True - ) + parser.add_argument("--scope", choices=("execution", "total", "amortization"), required=True) parser.add_argument("--output-dir", type=Path, required=True) return parser diff --git a/benchmark/control_service.py b/benchmark/control_service.py index 2461723171..9c5783fe0b 100644 --- a/benchmark/control_service.py +++ b/benchmark/control_service.py @@ -48,9 +48,7 @@ def finalize_measured_run( """Write benchmark evidence from the ordinary job's exact completion.""" if self._session is None: - raise RuntimeError( - "Measured run finalization requires the OpenHCS session." - ) + raise RuntimeError("Measured run finalization requires the OpenHCS session.") from benchmark.openhcs_measured_run import ( measured_endpoint_provenance, retain_measured_openhcs_completion, diff --git a/benchmark/equivalence/object_label_measurements.py b/benchmark/equivalence/object_label_measurements.py index c6386a5a2f..bbe8e8a5e2 100644 --- a/benchmark/equivalence/object_label_measurements.py +++ b/benchmark/equivalence/object_label_measurements.py @@ -63,7 +63,6 @@ RuntimeMeasurementFeatureSemanticMarker, ) from openhcs.core.runtime_stores import StoredRuntimeValue - if TYPE_CHECKING: pass from types import MappingProxyType @@ -78,6 +77,7 @@ ObjectLabelInstanceDomains, ) + RuntimeObjectValuesByLabel = dict[ RuntimeMeasurementFeatureKey, dict[ObjectInstanceKey, float], @@ -572,6 +572,10 @@ def _object_location_mean_feature_fact( return ((key, runtime_cell_signature(str(float(np.mean(finite_values))), policy)),) + + + + @dataclass(frozen=True, slots=True) class RuntimeObjectLabelMeasurementAuthority: """Nominal object-label ownership before primary-row fallback is allowed.""" diff --git a/benchmark/equivalence/runtime.py b/benchmark/equivalence/runtime.py index a1cdf60fb1..421f931fbf 100644 --- a/benchmark/equivalence/runtime.py +++ b/benchmark/equivalence/runtime.py @@ -1655,7 +1655,7 @@ def from_output_snapshots( for value in table.measurement_image_number_values( policy.measurement_dialect ) - ), + ) ) identity = tuple( table.measurement_input_key(policy, image_offset) diff --git a/benchmark/equivalence/table_snapshots.py b/benchmark/equivalence/table_snapshots.py index 6058b5fdb5..419c4e2d9f 100644 --- a/benchmark/equivalence/table_snapshots.py +++ b/benchmark/equivalence/table_snapshots.py @@ -460,10 +460,10 @@ def _subject_for_columns( normalized_name = normalize_runtime_identifier(self.path.stem) if normalized_name == _scope_token(dialect, MeasurementScope.RUN): return MeasurementSubject(MeasurementScope.RUN) - if normalized_name != _scope_token( - dialect, MeasurementScope.SAMPLE - ) and normalized_header & frozenset( - MeasurementRowAxisField.object_id_field_names() + if ( + normalized_name != _scope_token(dialect, MeasurementScope.SAMPLE) + and normalized_header + & frozenset(MeasurementRowAxisField.object_id_field_names()) ): return self._object_subject( self.path.stem, diff --git a/benchmark/openhcs_measured_run.py b/benchmark/openhcs_measured_run.py index c472c90ded..d13287a6d7 100644 --- a/benchmark/openhcs_measured_run.py +++ b/benchmark/openhcs_measured_run.py @@ -370,7 +370,9 @@ def retain_measured_openhcs_completion( if expected_axis_count is not None and expected_axis_count < 1: raise ValueError("Expected axis count must be positive when declared.") - auxiliary_params = ZMQAuxiliaryExecutionParams.from_transport(request.config_params) + auxiliary_params = ZMQAuxiliaryExecutionParams.from_transport( + request.config_params + ) observation_export_path = auxiliary_params.runtime_observation_export_path if observation_export_path is None: raise ValueError("Measured OpenHCS runs require runtime observation export.") diff --git a/benchmark/session_measured_run.py b/benchmark/session_measured_run.py index 4953822d56..fd44e217d0 100644 --- a/benchmark/session_measured_run.py +++ b/benchmark/session_measured_run.py @@ -70,9 +70,7 @@ def add(self, plate: Path, execution_plate: Path | None) -> str: AddDatasets, DatasetRootsRequest( roots=(str(plate),), - execution_root=( - None if execution_plate is None else str(execution_plate) - ), + execution_root=None if execution_plate is None else str(execution_plate), ), ) (self.scope_id,) = result.target_scope_ids @@ -81,9 +79,7 @@ def add(self, plate: Path, execution_plate: Path | None) -> str: def set_pipeline_source(self, source: str) -> None: self._require( SetDatasetPipeline, - DatasetPipelineSourceRequest( - scope_id=self._scope_id, pipeline_source=source - ), + DatasetPipelineSourceRequest(scope_id=self._scope_id, pipeline_source=source), ) def initialize(self) -> DatasetRowState: diff --git a/benchmark/well_throughput_scaling.py b/benchmark/well_throughput_scaling.py index a3dca4b5a4..3f1d117bfa 100644 --- a/benchmark/well_throughput_scaling.py +++ b/benchmark/well_throughput_scaling.py @@ -706,9 +706,7 @@ def generate_core_scaling_figures( for row in metric_rows ), filename_stem="03_core_scaling_average_with_pipeline_points_speedup", - methods=methods, - output_dir=self.output_dir, - output_formats=self.output_formats, + methods=methods, output_dir=self.output_dir, output_formats=self.output_formats, title="Average execution speedup by core count", ylabel="Execution speedup vs CellProfiler (x)", value_key="speedup", @@ -739,9 +737,7 @@ def generate_core_scaling_figures( outputs.extend( FIGURE_STYLE.generate_average_point_figures( metric_rows, - methods=methods, - output_dir=self.output_dir, - output_formats=self.output_formats, + methods=methods, output_dir=self.output_dir, output_formats=self.output_formats, filename_stem="04_core_scaling_average_with_pipeline_points_ram", title="Average RAM usage by core count", ylabel="Peak process-tree RSS (MB)", @@ -2115,9 +2111,7 @@ def run_case_well_throughput( use_threading=None, multiprocessing_start_method=None, materialize_runtime_artifacts=False, - partition_filter_config=LazyPartitionFilterConfig( - partition_filter=list(well_ids) - ), + partition_filter_config=LazyPartitionFilterConfig(partition_filter=list(well_ids)), path_planning_config=LazyPathPlanningConfig( partition_filter=0, global_output_folder=output_root, @@ -3150,10 +3144,7 @@ def _replicate_source_binding_workspace_wells( if projection.address is not None ) ) - if ( - source_partition_filter is None - or source_partition_filter.partition_filter is None - ): + if source_partition_filter is None or source_partition_filter.partition_filter is None: selected_source_wells = available_source_wells else: selected_source_wells = tuple( @@ -3251,9 +3242,7 @@ def _replicate_source_binding_workspace_wells( ) ) main_metadata[FIELDS.IMAGE_FILES] = expanded_image_files - main_metadata[Microscopy.Well.metadata_collection_field] = { - well_id: None for well_id in target_wells - } + main_metadata[Microscopy.Well.metadata_collection_field] = {well_id: None for well_id in target_wells} AtomicMetadataWriter().replace_subdirectory_metadata( metadata_path, FIELDS.DEFAULT_SUBDIRECTORY, main_metadata ) diff --git a/openhcs/agent/capabilities.py b/openhcs/agent/capabilities.py index 5cf4ac9586..18482e2405 100644 --- a/openhcs/agent/capabilities.py +++ b/openhcs/agent/capabilities.py @@ -461,8 +461,7 @@ class NonExpertCapabilitySurfaceMixin: def includes(self, capability: type["AgentCapabilityDeclaration"]) -> bool: return ( capability.exposition.visibility is not CapabilityVisibility.EXPERT - and capability.exposition.role - not in (CapabilityRole.EXPERT, CapabilityRole.FALLBACK) + and capability.exposition.role not in (CapabilityRole.EXPERT, CapabilityRole.FALLBACK) and super().includes(capability) ) @@ -473,9 +472,8 @@ class WorkflowGroupCapabilitySurfaceMixin: workflow_groups: ClassVar[frozenset[CapabilityWorkflowGroup]] def includes(self, capability: type["AgentCapabilityDeclaration"]) -> bool: - return ( - capability.exposition.workflow_group in self.workflow_groups - and super().includes(capability) + return capability.exposition.workflow_group in self.workflow_groups and super().includes( + capability ) @@ -656,9 +654,7 @@ def require_agent_type_contract(contract: AgentContract | None) -> type: def _keyword_parameter(name: str, annotation: object, default: object) -> Parameter: - return Parameter( - name, Parameter.KEYWORD_ONLY, default=default, annotation=annotation - ) + return Parameter(name, Parameter.KEYWORD_ONLY, default=default, annotation=annotation) class AgentCapabilityInvocation(ABC): @@ -790,6 +786,7 @@ class AgentResourceInvocationMixin: kind: ClassVar[CapabilityKind] = CapabilityKind.RESOURCE + def bind_mcp( self, declaration: type[AgentCapabilityDeclaration], @@ -1039,12 +1036,8 @@ def tool_parameters(self, declaration, binder): ) def invoke(self, declaration, binder, arguments): - return ( - super() - .invoke(declaration, binder, arguments) - .as_jsonable( - compact_actions=bool(arguments["compact_actions"]), - ) + return super().invoke(declaration, binder, arguments).as_jsonable( + compact_actions=bool(arguments["compact_actions"]), ) @@ -1159,6 +1152,8 @@ class AgentConfigPatchServiceInvocation( __slots__ = () + + class AgentConnectionRequestServiceInvocation( AgentUiConnectionMixin, AgentFromFieldsInputMixin, @@ -1167,6 +1162,7 @@ class AgentConnectionRequestServiceInvocation( __slots__ = () + class AgentViewerWindowConnectionServiceInvocation( AgentViewerWindowConnectionMixin, AgentServiceInvocation, @@ -1199,7 +1195,6 @@ def invoke(self, declaration, binder, arguments): *self.call_arguments(declaration, binder, arguments), ) - @dataclass(frozen=True, slots=True) class AgentCapabilitySearchRequest: """Bounded registry query over declaration-owned capability metadata.""" @@ -2167,9 +2162,7 @@ class ObserveCustomFunctionRegistrationCapability(FunctionCatalogCapability): output_contract = CustomFunctionRegistrationObservation invocation = AgentDataclassRequestServiceInvocation( service=lambda context: context.function_catalog, - method=lambda service, request: service.observe_custom_function_registration( - request - ), + method=lambda service, request: service.observe_custom_function_registration(request), ) @@ -2388,9 +2381,7 @@ class SamplePlateImageCapability(MainThreadProgressCapability, PlatePathCapabili ) -class StreamPlateFilesToViewerCapability( - MainThreadProgressCapability, PlatePathCapability -): +class StreamPlateFilesToViewerCapability(MainThreadProgressCapability, PlatePathCapability): name = "openhcs_stream_plate_files_to_viewer" cli_command = "stream-plate-files" title = "Stream plate files to viewer" @@ -2519,9 +2510,7 @@ class UiSampleSelectedPlateImageCapability(UiSelectedPlateCapability): ) -class UiStreamSelectedPlateFilesToViewerCapability( - MainThreadProgressCapability, UiSelectedPlateCapability -): +class UiStreamSelectedPlateFilesToViewerCapability(MainThreadProgressCapability, UiSelectedPlateCapability): name = "openhcs_ui_stream_selected_plate_files_to_viewer" cli_command = "selected-plate-stream" title = "Stream selected plate files to viewer" diff --git a/openhcs/agent/dto/__init__.py b/openhcs/agent/dto/__init__.py index a5e35e90c1..144a967fce 100644 --- a/openhcs/agent/dto/__init__.py +++ b/openhcs/agent/dto/__init__.py @@ -16,7 +16,9 @@ "AgentWarning", "RenderedSource", ), - "openhcs.agent.dto.mcp": ("McpServerHealthResult",), + "openhcs.agent.dto.mcp": ( + "McpServerHealthResult", + ), "pyqt_reactive.services.window_snapshot": ( "WindowSnapshotCaptureScope", "WindowSnapshotCaptureSpec", @@ -91,7 +93,9 @@ "SourceWorkspaceSummary", "ViewerStreamingPlanSummary", ), - "openhcs.agent.dto.execution_connection": ("ExecutionConnectionSpec",), + "openhcs.agent.dto.execution_connection": ( + "ExecutionConnectionSpec", + ), "openhcs.agent.dto.pipeline": ( "CreatePipelineRequest", "FunctionSpecRef", diff --git a/openhcs/agent/dto/functions.py b/openhcs/agent/dto/functions.py index 5045a176b5..6528f22aee 100644 --- a/openhcs/agent/dto/functions.py +++ b/openhcs/agent/dto/functions.py @@ -141,29 +141,21 @@ class CustomFunctionRegistrationHandle(FunctionCatalogOperationHandle): @classmethod def from_request(cls, request: CustomFunctionRegistrationRequest) -> Self: return cls( - connection=request.connection, - server_identity=request.require_selected_server_identity(), - content_sha256=CustomFunctionSource.content_digest( - request.source_code.encode("utf-8") - ), - function_name=request.function_name, - persist=request.persist, + connection=request.connection, server_identity=request.require_selected_server_identity(), + content_sha256=CustomFunctionSource.content_digest(request.source_code.encode("utf-8")), + function_name=request.function_name, persist=request.persist, storage_dir=request.storage_dir, ) def require_named_source(self) -> CustomFunctionSource: if self.function_name is None: - raise ValueError( - "Persisted registration observation requires its admitted name." - ) + raise ValueError("Persisted registration observation requires its admitted name.") return CustomFunctionSource(self.function_name, self.content_sha256) def require_storage_dir(self) -> Path: """Decode this handle's admitted source store once for native/path owners.""" if self.storage_dir is None: - raise ValueError( - "Persisted registration observation requires its admitted store." - ) + raise ValueError("Persisted registration observation requires its admitted store.") return Path(self.storage_dir) @@ -190,15 +182,10 @@ class CustomFunctionRegistrationObservation(AgentResultEnvelope): outcome: CustomFunctionRegistrationObservationOutcome = field(init=False) def __post_init__(self) -> None: - object.__setattr__( - self, - "outcome", - CustomFunctionRegistrationObservationOutcome.from_proofs( - published=bool(self.published_sources), - persisted=self.persisted_source is not None, - persist=self.handle.persist, - ), - ) + object.__setattr__(self, "outcome", CustomFunctionRegistrationObservationOutcome.from_proofs( + published=bool(self.published_sources), persisted=self.persisted_source is not None, + persist=self.handle.persist, + )) class FunctionCatalogPreparationOutcome(str, Enum): @@ -287,39 +274,25 @@ class CustomFunctionRegistrationRequest(FunctionCatalogControlRequestABC): connection: ExecutionConnectionSpec = field(default_factory=ExecutionConnectionSpec) function_name: str | None = None storage_dir: str | None = None - admission_policy: AgentPathPolicy | None = field( - default=None, repr=False, compare=False - ) + admission_policy: AgentPathPolicy | None = field(default=None, repr=False, compare=False) server_identity: ProcessIdentity | None = None message_type = FunctionCatalogControlMessageType.REGISTER_CUSTOM @classmethod def from_fields( - cls, - *, - source_code: str, - persist: bool = True, - compact_signature: bool = True, - function_name: str | None = None, - storage_dir: str | None = None, - host: str = ExecutionConnectionSpec().host, - port: int | None = None, - transport_mode: TransportMode | None = None, + cls, *, source_code: str, persist: bool = True, + compact_signature: bool = True, function_name: str | None = None, + storage_dir: str | None = None, host: str = ExecutionConnectionSpec().host, + port: int | None = None, transport_mode: TransportMode | None = None, persistent: bool = ExecutionConnectionSpec().persistent, ) -> Self: """Public boundary: callers cannot supply their own write authority.""" return cls( - source_code=source_code, - persist=persist, - compact_signature=compact_signature, - function_name=function_name, - storage_dir=storage_dir, + source_code=source_code, persist=persist, compact_signature=compact_signature, + function_name=function_name, storage_dir=storage_dir, connection=ExecutionConnectionSpec( - host=host, - port=port, - transport_mode=transport_mode, - persistent=persistent, + host=host, port=port, transport_mode=transport_mode, persistent=persistent, ), ) @@ -330,16 +303,12 @@ def admitted(self, policy: AgentPathPolicy) -> Self: self.connection.require_port("Custom function registration") if self.persist: if self.function_name is None or self.storage_dir is None: - raise ValueError( - "Persisted registration requires function_name and the endpoint's storage_dir." - ) + raise ValueError("Persisted registration requires function_name and the endpoint's storage_dir.") root = Path(self.storage_dir).expanduser() if not root.is_absolute(): raise ValueError("Custom function storage_dir must be absolute.") policy.assert_writable(root) - policy.assert_writable( - CustomFunctionManager.source_path_for_name(root, self.function_name) - ) + policy.assert_writable(CustomFunctionManager.source_path_for_name(root, self.function_name)) return replace(self, admission_policy=policy) def require_selected_server_identity(self) -> ProcessIdentity: @@ -392,15 +361,12 @@ def require_request(self, request: CustomFunctionRegistrationRequest) -> None: expected_root = Path(request.storage_dir).expanduser() expected_file = CustomFunctionManager.source_path_for_name( - expected_root, - request.function_name, + expected_root, request.function_name, ) if ( - Path(self.storage_dir).resolve(strict=False) - != expected_root.resolve(strict=False) + Path(self.storage_dir).resolve(strict=False) != expected_root.resolve(strict=False) or self.source_file_path is None - or Path(self.source_file_path).resolve(strict=False) - != expected_file.resolve(strict=False) + or Path(self.source_file_path).resolve(strict=False) != expected_file.resolve(strict=False) ): raise ValueError( f"Selected endpoint owns custom storage {self.storage_dir}, " @@ -409,15 +375,11 @@ def require_request(self, request: CustomFunctionRegistrationRequest) -> None: ) def require_result( - self, - request: CustomFunctionRegistrationRequest, + self, request: CustomFunctionRegistrationRequest, result: CustomFunctionRegistrationResult, ) -> None: """Require the returned mutation receipt to identify the admitted owner.""" - if ( - result.server_identity != self.server_identity - or result.connection != request.connection - ): + if result.server_identity != self.server_identity or result.connection != request.connection: raise ValueError("Registration returned a different execution owner.") if result.persisted != request.persist: raise ValueError("Registration returned a different persistence policy.") @@ -425,9 +387,7 @@ def require_result( result.storage_dir != self.storage_dir or result.source_file_paths != (self.source_file_path,) ): - raise ValueError( - "Registration returned a different persistence destination." - ) + raise ValueError("Registration returned a different persistence destination.") class FunctionParameterSource(str, Enum): @@ -787,32 +747,20 @@ class CustomFunctionRegistrationResult(AgentResultEnvelope): @classmethod def uncertain( - cls, - request: CustomFunctionRegistrationRequest, - error: Exception, + cls, request: CustomFunctionRegistrationRequest, error: Exception, ) -> Self: """Preserve the actual cause and same-owner read-only recovery identity.""" handle = CustomFunctionRegistrationHandle.from_request(request) - cause = ( - error.to_agent_error() - if isinstance(error, AgentFacingErrorMixin) - else AgentError.from_exception( - "custom_function_registration_uncertain", - error, - hint="Preserve the original request. Observe this handle; do not replay registration. Absence is not proof of no mutation.", - ) + cause = error.to_agent_error() if isinstance(error, AgentFacingErrorMixin) else AgentError.from_exception( + "custom_function_registration_uncertain", error, + hint="Preserve the original request. Observe this handle; do not replay registration. Absence is not proof of no mutation.", ) return cls( - schema_version=SCHEMA_VERSION, - persisted=request.persist, - storage_dir=request.storage_dir, - connection=request.connection, - server_identity=request.server_identity, - observation_handle=handle, + schema_version=SCHEMA_VERSION, persisted=request.persist, + storage_dir=request.storage_dir, connection=request.connection, + server_identity=request.server_identity, observation_handle=handle, errors=(cause,), - next_steps=( - "Observe this observation_handle with openhcs_get_custom_function_registration_status; do not replay source.", - ), + next_steps=("Observe this observation_handle with openhcs_get_custom_function_registration_status; do not replay source.",), ) diff --git a/openhcs/agent/dto/plate.py b/openhcs/agent/dto/plate.py index 44a1af39c4..3341c0459e 100644 --- a/openhcs/agent/dto/plate.py +++ b/openhcs/agent/dto/plate.py @@ -30,6 +30,7 @@ SyntheticPlateFormat, ) + COMPONENT_FILTERS_DESCRIPTION = ( "Accepted values per axis name of the active axis family, for example " '{"well": ["A01", "B02"]}; a file matches when every named axis holds one ' @@ -177,9 +178,7 @@ class SelectedPlateFileFilterOptions(SelectedPlateTargetOptions): path_contains: str | None = None component_filters: ComponentFilters = field( default=ComponentFilters(), - metadata={ - AGENT_PARAMETER_DESCRIPTION_METADATA_KEY: COMPONENT_FILTERS_DESCRIPTION - }, + metadata={AGENT_PARAMETER_DESCRIPTION_METADATA_KEY: COMPONENT_FILTERS_DESCRIPTION}, ) limit: int = 1 @@ -350,9 +349,7 @@ class PlateFileQueryRequest(DatasetTarget): path_contains: str | None = None component_filters: ComponentFilters = field( default=ComponentFilters(), - metadata={ - AGENT_PARAMETER_DESCRIPTION_METADATA_KEY: COMPONENT_FILTERS_DESCRIPTION - }, + metadata={AGENT_PARAMETER_DESCRIPTION_METADATA_KEY: COMPONENT_FILTERS_DESCRIPTION}, ) offset: int = 0 limit: int = 50 @@ -422,9 +419,7 @@ class PlateFileStreamRequest(DatasetTarget): path_contains: str | None = None component_filters: ComponentFilters = field( default=ComponentFilters(), - metadata={ - AGENT_PARAMETER_DESCRIPTION_METADATA_KEY: COMPONENT_FILTERS_DESCRIPTION - }, + metadata={AGENT_PARAMETER_DESCRIPTION_METADATA_KEY: COMPONENT_FILTERS_DESCRIPTION}, ) limit: int = 1 viewer_config_key: str = NapariViewer.config_key @@ -836,9 +831,7 @@ class SyntheticPlateGenerationRequest: shared_cell_fraction: float = ( SYNTHETIC_PLATE_GENERATION_PROFILE.default_request.shared_cell_fraction ) - partition_values: tuple[str, ...] = ( - SYNTHETIC_PLATE_GENERATION_PROFILE.default_request.partition_values - ) + partition_values: tuple[str, ...] = SYNTHETIC_PLATE_GENERATION_PROFILE.default_request.partition_values format: SyntheticPlateFormat = ( SYNTHETIC_PLATE_GENERATION_PROFILE.default_request.format ) diff --git a/openhcs/agent/dto/ui_bridge.py b/openhcs/agent/dto/ui_bridge.py index 736689ecfc..638bba5c3a 100644 --- a/openhcs/agent/dto/ui_bridge.py +++ b/openhcs/agent/dto/ui_bridge.py @@ -744,6 +744,10 @@ class UiCodeDocument( errors: tuple[AgentError, ...] = () + + + + @dataclass(frozen=True, kw_only=True) class UiStateSurfaceEnvelope(AgentResultEnvelope): """Shared envelope for typed UI state-surface payloads.""" diff --git a/openhcs/agent/services/artifact_plan_inspection_service.py b/openhcs/agent/services/artifact_plan_inspection_service.py index 6f08a06355..ee16315ac3 100644 --- a/openhcs/agent/services/artifact_plan_inspection_service.py +++ b/openhcs/agent/services/artifact_plan_inspection_service.py @@ -93,8 +93,7 @@ def compile(self, request: CompileInspectionInput) -> CompileInspectionResult: result = self._compile(request) except Exception: EndpointStartupStatus( - EndpointStartupPhase.FAILED, - "Source inspection compiler failed", + EndpointStartupPhase.FAILED, "Source inspection compiler failed", ).publish() raise EndpointStartupStatus( diff --git a/openhcs/agent/services/function_catalog_service.py b/openhcs/agent/services/function_catalog_service.py index f442844996..f406b1429e 100644 --- a/openhcs/agent/services/function_catalog_service.py +++ b/openhcs/agent/services/function_catalog_service.py @@ -235,8 +235,7 @@ def search_match( (CatalogSearchRank.SUMMARY_CONTAINS, entry.summary or ""), ( CatalogSearchRank.DOC_CONTAINS, - PARAMETER_DOCUMENTATION_POLICY.detail_doc(metadata.func, metadata.doc) - or "", + PARAMETER_DOCUMENTATION_POLICY.detail_doc(metadata.func, metadata.doc) or "", ), ): score = self._text_score(value) @@ -487,9 +486,7 @@ def agent_parameter_names( ) compile_only = tuple( name - for name in PipelineInvocationContractProviderAuthority.compile_time_parameter_names( - contract - ) + for name in PipelineInvocationContractProviderAuthority.compile_time_parameter_names(contract) if name not in runtime_names ) return tuple(dict.fromkeys((*ordinary, *compile_only))) @@ -609,8 +606,7 @@ def register_custom_function( @abstractmethod def observe_custom_function_registration( - self, - handle: CustomFunctionRegistrationHandle, + self, handle: CustomFunctionRegistrationHandle, ) -> CustomFunctionRegistrationObservation: """Observe exact existing publication/persistence without evaluating source.""" @@ -714,21 +710,17 @@ def register_custom_function( request.admitted(self._path_policy) server_identity = request.require_server_identity() manager = custom_function_manager.CustomFunctionManager(create_storage=False) - if request.persist and manager.storage_dir.resolve(strict=False) != Path( - request.storage_dir - ).resolve(strict=False): + if request.persist and manager.storage_dir.resolve(strict=False) != Path(request.storage_dir).resolve(strict=False): raise ValueError( f"Selected endpoint owns custom storage {manager.storage_dir}, " f"not requested {request.storage_dir}; no source was evaluated." ) - def admit_write(path: Path) -> Path: self._path_policy.assert_writable(path) return request.admission_policy.assert_writable(path) registered_functions = manager.register_from_code( - request.source_code, - persist=request.persist, + request.source_code, persist=request.persist, expected_function_name=request.function_name, write_admission=admit_write, ) @@ -772,53 +764,33 @@ def admit_write(path: Path) -> Path: ) def observe_custom_function_registration( - self, - handle: CustomFunctionRegistrationHandle, + self, handle: CustomFunctionRegistrationHandle, ) -> CustomFunctionRegistrationObservation: from openhcs.agent.dto.common import AgentWarning - from openhcs.processing.custom_functions.runtime_registry import ( - CustomFunctionRuntimeRegistry, - ) + from openhcs.processing.custom_functions.runtime_registry import CustomFunctionRuntimeRegistry handle.require_current_owner() published = CustomFunctionRuntimeRegistry.published_sources_for_content( - handle.content_sha256, - function_name=handle.function_name, + handle.content_sha256, function_name=handle.function_name, ) persisted = None warnings = () if handle.persist: - manager = custom_function_manager.CustomFunctionManager( - create_storage=False - ) - if handle.require_storage_dir().resolve( - strict=False - ) != manager.storage_dir.resolve(strict=False): - raise ValueError( - "Registration observation reached a different native source store." - ) + manager = custom_function_manager.CustomFunctionManager(create_storage=False) + if handle.require_storage_dir().resolve(strict=False) != manager.storage_dir.resolve(strict=False): + raise ValueError("Registration observation reached a different native source store.") source = handle.require_named_source() - self._path_policy.assert_readable_location( - manager.source_path_for_name(manager.storage_dir, source.function_name) - ) + self._path_policy.assert_readable_location(manager.source_path_for_name(manager.storage_dir, source.function_name)) try: manager.require_source(source) except (OSError, RuntimeError) as error: - warnings = ( - AgentWarning( - "registration_source_not_observed", - str(error), - "Missing/changed current source is not proof of no original mutation; do not replay.", - ), - ) + warnings = (AgentWarning("registration_source_not_observed", str(error), + "Missing/changed current source is not proof of no original mutation; do not replay."),) else: persisted = source return CustomFunctionRegistrationObservation( - schema_version=SCHEMA_VERSION, - handle=handle, - published_sources=published, - persisted_source=persisted, - warnings=warnings, + schema_version=SCHEMA_VERSION, handle=handle, published_sources=published, + persisted_source=persisted, warnings=warnings, ) def custom_function_registration_destination( @@ -830,13 +802,8 @@ def custom_function_registration_destination( return CustomFunctionRegistrationDestination( storage_dir=str(storage_dir), source_file_path=( - str( - manager_type.source_path_for_name( - storage_dir, request.function_name - ) - ) - if request.function_name is not None - else None + str(manager_type.source_path_for_name(storage_dir, request.function_name)) + if request.function_name is not None else None ), ) @@ -967,12 +934,8 @@ def _search_projections( metadata.func, contract ) entry = self._entry( - function_id, - metadata, - SignatureView.FULL, - SummaryView.FULL, - contract=contract, - visible_signature=signature, + function_id, metadata, SignatureView.FULL, SummaryView.FULL, + contract=contract, visible_signature=signature, ) projections.append( CatalogSearchProjection( @@ -988,8 +951,7 @@ def _search_projections( signature, entry.name, SignatureView.COMPACT ), summary=( - None - if entry.summary is None + None if entry.summary is None else _bounded_summary(entry.summary, SummaryView.COMPACT) ), ), @@ -1290,8 +1252,7 @@ def _runtime_contract_summary( return FunctionRuntimeContractSummary( callable_kind=callable_kind, processing_contract=( - None - if contract.processing_contract is None + None if contract.processing_contract is None else contract.processing_contract.key ), runtime_bound_parameters=contract.runtime_bound_parameters, diff --git a/openhcs/agent/services/plate_inspection_service.py b/openhcs/agent/services/plate_inspection_service.py index e417adcb47..df564102cf 100644 --- a/openhcs/agent/services/plate_inspection_service.py +++ b/openhcs/agent/services/plate_inspection_service.py @@ -63,7 +63,6 @@ ) from openhcs.core.axes import Axis, AxisFamily from openhcs.core.dataset_sources.choice import AutoDetectedSource, DatasetSourceChoice - if TYPE_CHECKING: from openhcs.core.components.parser_metaprogramming import ( FilenameParseResult, @@ -169,7 +168,10 @@ def candidates( if DatasetSourceChoice.named(requested_source_format) is not AutoDetectedSource: return () - if type(selected_handler).source_selection_role() is not BroadStoreSource: + if ( + type(selected_handler).source_selection_role() + is not BroadStoreSource + ): return () try: @@ -186,7 +188,10 @@ def candidates( candidates: list[PlateInspectionHandlerCandidate] = [] for handler_type in DatasetSource.__registry__.values(): - if handler_type.source_selection_role() is not FormatSpecificSource: + if ( + handler_type.source_selection_role() + is not FormatSpecificSource + ): continue candidate = cls._candidate( handler_type=handler_type, @@ -472,9 +477,7 @@ class PlateInspectionMutableParsedComponents: """Mutable accumulator before parsed component values are frozen.""" values: dict[type[Axis], set[str]] = field( - default_factory=lambda: { - component: set() for component in AxisFamily.active().axes - } + default_factory=lambda: {component: set() for component in AxisFamily.active().axes} ) def add_parsed_filename_values(self, parsed: "FilenameParseResult") -> None: @@ -870,8 +873,7 @@ def for_handler( selection_message = type(handler).source_selection_guidance() initialization_message = ( f"initialize with explicit source_format={ingestion_owner!r}" - if DatasetSourceChoice.named(requested_source_format) - is not AutoDetectedSource + if DatasetSourceChoice.named(requested_source_format) is not AutoDetectedSource else "initialize with auto-detection" ) message = ( @@ -1205,8 +1207,7 @@ def _query_files( parser = self._parser( handler, warnings, - warn=query_kind is not PlateFileKind.RESULT - or bool(request.component_filters), + warn=query_kind is not PlateFileKind.RESULT or bool(request.component_filters), ) file_inventory = self._plate_file_inventory_for_query( handler, @@ -1400,9 +1401,7 @@ def _sample_image( record, record.sample( filemanager, - plate_path=handler.metadata_handler.source_workspace_root( - plate_path - ), + plate_path=handler.metadata_handler.source_workspace_root(plate_path), request=ImageSamplingRequest( origin_yx=(request.y, request.x), shape_yx=(request.height, request.width), @@ -1574,7 +1573,10 @@ def _inspect( ) filemanager = self._filemanager_factory.create() - if request.detects_source_format and request.pattern_format is None: + if ( + request.detects_source_format + and request.pattern_format is None + ): result_only = self._result_only_inspection( request=request, available_types=available_types, diff --git a/openhcs/agent/services/plate_streaming_service.py b/openhcs/agent/services/plate_streaming_service.py index 139856757d..db62faf29c 100644 --- a/openhcs/agent/services/plate_streaming_service.py +++ b/openhcs/agent/services/plate_streaming_service.py @@ -148,9 +148,7 @@ def stream_files( requested_source_format=request.source_format, detected_source_format=context.source_format, handler_class=type(context.handler).__name__, - parser_class=( - None if context.parser is None else type(context.parser).__name__ - ), + parser_class=None if context.parser is None else type(context.parser).__name__, viewer_config_key=request.viewer_config_key, connection=connection, requested_paths=request.file_paths, @@ -166,15 +164,9 @@ def stream_files( transport_mode=config.transport_mode, persistent=config.persistent, ) - result = replace( - result, - viewer_type=config.viewer_family.viewer_type(), - connection=connection, - ) + result = replace(result, viewer_type=config.viewer_family.viewer_type(), connection=connection) if request.result_directory is not None: - if request.kind is not PlateFileKind.RESULT or bool( - request.component_filters - ): + if request.kind is not PlateFileKind.RESULT or bool(request.component_filters): raise ValueError( "Explicit result-directory streaming requires kind='result' " "and no acquisition-component filter." @@ -312,15 +304,11 @@ def stream_files( require_source_metadata=request.result_directory is not None, ) ) - self._report_progress( - "Selected artifacts published and viewer settlement completed" - ) + self._report_progress("Selected artifacts published and viewer settlement completed") except Exception as exc: return replace( result, - viewer_type=( - None if config is None else config.viewer_family.viewer_type() - ), + viewer_type=None if config is None else config.viewer_family.viewer_type(), connection=connection, errors=(self._stream_error(exc, plate_path=request.plate_path),), ) @@ -339,8 +327,7 @@ def stream_files( def _report_progress(message: str) -> None: """Use the original request-local relay, without a viewer observer/store.""" EndpointStartupStatus( - EndpointStartupPhase.PREPARING_CAPABILITIES, - message, + EndpointStartupPhase.PREPARING_CAPABILITIES, message, ).publish() def _record_stream_status(self, messages: list[str], message: str) -> None: @@ -421,8 +408,7 @@ def _receipt_source_projection( producers.append(producer) plane_domain = ViewerComponentValueDomainPayload.from_ordered_wire_mapping( record.summary.require_plane_components(), - declared_axes=_active_declared_axes(), - context="image receipt plane coordinates", + declared_axes=_active_declared_axes(), context="image receipt plane coordinates", ) planes = SourceImageProvenancePlanes.from_component_domain( path=image_path, diff --git a/openhcs/agent/services/selected_plate_service.py b/openhcs/agent/services/selected_plate_service.py index e9521e9091..ffdce1016e 100644 --- a/openhcs/agent/services/selected_plate_service.py +++ b/openhcs/agent/services/selected_plate_service.py @@ -392,7 +392,8 @@ def selected_plate_row_from_state_surface( matched_rows = tuple( row for row in rows_value - if isinstance(row, Mapping) and row.get("scope_id") == selected_scope_id + if isinstance(row, Mapping) + and row.get("scope_id") == selected_scope_id ) if len(matched_rows) != 1: return None, AgentError( diff --git a/openhcs/agent/services/ui_bridge_service.py b/openhcs/agent/services/ui_bridge_service.py index 5ebc57883b..4e659db252 100644 --- a/openhcs/agent/services/ui_bridge_service.py +++ b/openhcs/agent/services/ui_bridge_service.py @@ -400,9 +400,7 @@ def as_served(cls, result: UiBridgeStatus, binding: UiBridgeConnectionSpec): @classmethod def failed(cls, request, errors): del request - return UiBridgeStatus( - schema_version=SCHEMA_VERSION, reachable=False, errors=errors - ) + return UiBridgeStatus(schema_version=SCHEMA_VERSION, reachable=False, errors=errors) @staticmethod def _unreachable(resolution: "UiBridgeConnectionResolution") -> UiBridgeStatus: @@ -1069,6 +1067,7 @@ def invoke( raise NotImplementedError + class UiBridgeGatewayErrorABC(ABC): """Gateway-originated bridge failure that knows its agent-facing errors.""" @@ -1790,9 +1789,7 @@ def invoke( """Run one operation; failures come back as the operation's own result.""" return operation.respond(self, connection, request) - def resolve( - self, connection: UiBridgeConnectionSpec - ) -> UiBridgeConnectionResolution: + def resolve(self, connection: UiBridgeConnectionSpec) -> UiBridgeConnectionResolution: return self._descriptor_resolver.resolve(connection) def connection_from_args( diff --git a/openhcs/authoring/session/compile_batch.py b/openhcs/authoring/session/compile_batch.py index dcf7aa80e9..b0594a3d0d 100644 --- a/openhcs/authoring/session/compile_batch.py +++ b/openhcs/authoring/session/compile_batch.py @@ -50,9 +50,7 @@ async def compile_datasets(self, scope_ids: tuple[str, ...]) -> None: try: client = await session.connect_client() session.publish( - StatusReported( - f"Queueing compilation for {len(scope_ids)} dataset(s)..." - ) + StatusReported(f"Queueing compilation for {len(scope_ids)} dataset(s)...") ) requests: list[DatasetPipelineRequest] = [] for scope_id in scope_ids: @@ -131,9 +129,7 @@ def on_wait_start(_request, _index, _total) -> None: session.publish(DatasetsChanged()) def on_wait_success(request, _execution_id, index, total) -> None: - session.publish( - StatusReported(f"Compiled {index}/{total}: {request.scope_id}") - ) + session.publish(StatusReported(f"Compiled {index}/{total}: {request.scope_id}")) session.publish(DatasetsChanged()) def on_failure(request, error, _index, _total) -> None: @@ -143,12 +139,8 @@ def on_failure(request, error, _index, _total) -> None: error, exc_info=True, ) - session.batch.mark_terminal( - request.scope_id, TerminalExecutionStatus.FAILED - ) - session.publish( - ErrorReported(f"Compile failed for {request.scope_id}: {error}") - ) + session.batch.mark_terminal(request.scope_id, TerminalExecutionStatus.FAILED) + session.publish(ErrorReported(f"Compile failed for {request.scope_id}: {error}")) session.publish(DatasetsChanged()) return await self._engine.run( @@ -182,9 +174,7 @@ async def _submit(self, client, request: DatasetPipelineRequest) -> str: self._session.set_compiled(request.scope_id, None) return await submit_compile(client, request) - async def _wait( - self, client, execution_id: str, request: DatasetPipelineRequest - ) -> None: + async def _wait(self, client, execution_id: str, request: DatasetPipelineRequest) -> None: try: inspection = await wait_for_compile( client, diff --git a/openhcs/authoring/session/dataset_document.py b/openhcs/authoring/session/dataset_document.py index 2b1f68f945..e1de6c4d07 100644 --- a/openhcs/authoring/session/dataset_document.py +++ b/openhcs/authoring/session/dataset_document.py @@ -46,9 +46,7 @@ def from_carrier( default: SelectedAllSelectionMode = SelectedAllSelectionMode.SELECTED, ) -> "DatasetDocumentScope": mode = SelectedAllSelectionMode(carrier.resolved_selection_mode(default)) - return cls.__registry__[mode]( - selected_scope_ids=tuple(carrier.selected_scope_ids) - ) + return cls.__registry__[mode](selected_scope_ids=tuple(carrier.selected_scope_ids)) def require_payload_scope(self, scope_ids: tuple[str, ...]) -> None: if self.selected_scope_ids and frozenset(scope_ids) != frozenset( @@ -116,9 +114,7 @@ def authored_pipeline_config(scope_id: str) -> PipelineConfig: state = ObjectStateRegistry.get_by_scope(scope_id) if state is None or ObjectStateRegistry.get_object(scope_id) is None: - raise ValueError( - f"Dataset {scope_id!r} has no registered orchestrator ObjectState." - ) + raise ValueError(f"Dataset {scope_id!r} has no registered orchestrator ObjectState.") config = state.to_object(update_delegate=False) if not isinstance(config, PipelineConfig): raise TypeError("A dataset ObjectState must rebuild a PipelineConfig.") diff --git a/openhcs/authoring/session/debug_runs.py b/openhcs/authoring/session/debug_runs.py index 891e4aa94a..2c5622f5a9 100644 --- a/openhcs/authoring/session/debug_runs.py +++ b/openhcs/authoring/session/debug_runs.py @@ -134,9 +134,7 @@ async def inspect_runtime(self, *, debug_session_id: str): ) ) self._session.publish( - StatusReported( - f"Loaded runtime inspection for session {debug_session_id[:8]}" - ) + StatusReported(f"Loaded runtime inspection for session {debug_session_id[:8]}") ) return view_model diff --git a/openhcs/authoring/session/events.py b/openhcs/authoring/session/events.py index 8ba45c8760..1687356af7 100644 --- a/openhcs/authoring/session/events.py +++ b/openhcs/authoring/session/events.py @@ -324,6 +324,4 @@ def after( lambda: self._next_sequence - 1 > sequence, timeout=timeout_seconds, ) - return tuple( - record for record in self._records if record.sequence > sequence - ) + return tuple(record for record in self._records if record.sequence > sequence) diff --git a/openhcs/authoring/session/operations/datasets.py b/openhcs/authoring/session/operations/datasets.py index 74fce20b05..d9030c452f 100644 --- a/openhcs/authoring/session/operations/datasets.py +++ b/openhcs/authoring/session/operations/datasets.py @@ -148,9 +148,7 @@ class DeleteDatasets(IdleTargetsOperation, HeadlessOperation): operation_id = "delete_datasets" label = "Del" tooltip = "Delete selected datasets" - description = ( - "Removes datasets and their configuration and pipeline from the session." - ) + description = "Removes datasets and their configuration and pipeline from the session." side_effects = ("mutates_dataset_list",) @classmethod @@ -172,9 +170,7 @@ def available(cls, session, request) -> AgentError | None: error = super().available(session, request) if error is not None: return error - pending = [ - s for s in request.scope_ids if session.has_pending_definition_work(s) - ] + pending = [s for s in request.scope_ids if session.has_pending_definition_work(s)] if pending: return AgentError( code="dataset_definition_pending", @@ -183,9 +179,7 @@ def available(cls, session, request) -> AgentError | None: return None -class InitializeDatasets( - IdleTargetsOperation, DatasetWorkflowOperation, HeadlessOperation -): +class InitializeDatasets(IdleTargetsOperation, DatasetWorkflowOperation, HeadlessOperation): operation_id = "initialize_datasets" workflow_name = "INIT" label = "Init" @@ -336,9 +330,7 @@ def available(cls, session, request) -> AgentError | None: code="execution_batch_active", message="An execution batch is already running.", ) - pending = [ - s for s in request.scope_ids if session.has_pending_definition_work(s) - ] + pending = [s for s in request.scope_ids if session.has_pending_definition_work(s)] if pending: return AgentError( code="dataset_definition_pending", @@ -396,9 +388,7 @@ def button_label(cls, session) -> str: def available(cls, session, request) -> AgentError | None: state = session.execution_state if not state.busy: - return AgentError( - code="no_execution_running", message="Nothing is running." - ) + return AgentError(code="no_execution_running", message="Nothing is running.") if not state.run_button_enabled(True): return AgentError( code="execution_stopping", @@ -471,3 +461,4 @@ class ShowDatasetImages(InitializedTargetsOperation, RendererOperation): tooltip = "Browse dataset images and metadata" description = "Opens the image and metadata browser of the selected datasets." side_effects = ("opens_dataset_viewer_window",) + diff --git a/openhcs/authoring/session/operations/pipelines.py b/openhcs/authoring/session/operations/pipelines.py index 8027197a17..66dd730864 100644 --- a/openhcs/authoring/session/operations/pipelines.py +++ b/openhcs/authoring/session/operations/pipelines.py @@ -87,9 +87,7 @@ def available(cls, session, request) -> AgentError | None: return None -class AddPipelineStep( - CurrentDatasetRequestMixin, EditablePipelineOperation, RendererOperation -): +class AddPipelineStep(CurrentDatasetRequestMixin, EditablePipelineOperation, RendererOperation): operation_id = "add_step" label = "Add" tooltip = "Add new pipeline step" diff --git a/openhcs/authoring/session/progress.py b/openhcs/authoring/session/progress.py index 62f5d35e9a..360bd19e82 100644 --- a/openhcs/authoring/session/progress.py +++ b/openhcs/authoring/session/progress.py @@ -193,9 +193,7 @@ def on_progress(self, message: dict) -> None: session.live_measurements.add_notification(live) session.publish(LiveMeasurementAvailable(live)) try: - artifact = RuntimeArtifactAvailableNotification.from_progress_event( - event - ) + artifact = RuntimeArtifactAvailableNotification.from_progress_event(event) except RuntimeArtifactPayloadError as error: logger.warning( "Malformed runtime artifact progress context for " @@ -231,3 +229,4 @@ def _fetch_server_info_snapshot(self) -> ExecutionServerInfo: f"Expected ExecutionServerInfo, got {type(parsed).__name__}" ) return parsed + diff --git a/openhcs/authoring/session/session.py b/openhcs/authoring/session/session.py index 12d38523aa..01671cacae 100644 --- a/openhcs/authoring/session/session.py +++ b/openhcs/authoring/session/session.py @@ -310,9 +310,7 @@ def __init__( self.global_config = global_config self.client = ZMQClientService( config=transport_config, - status_callback=lambda status: self.publish( - ServerConnectionChanged(status) - ), + status_callback=lambda status: self.publish(ServerConnectionChanged(status)), compatibility_callback=lambda compatibility: self.publish( ServerCompatibilityObserved(compatibility) ), @@ -347,9 +345,7 @@ def __init__( self.submission = ExecutionSubmission(self) self.execution_control = ExecutionControl(self) self.debug_runs = DebugRuns(self) - self.progress = ExecutionProgress( - self, interval_seconds=progress_interval_seconds - ) + self.progress = ExecutionProgress(self, interval_seconds=progress_interval_seconds) self._work = AsyncOperationExecutor() self._closed = False @@ -361,9 +357,7 @@ def __init__( def publish(self, event: SessionEvent) -> SessionEventRecord: return self.event_log.publish(event) - def subscribe( - self, listener: Callable[[SessionEventRecord], None] - ) -> Callable[[], None]: + def subscribe(self, listener: Callable[[SessionEventRecord], None]) -> Callable[[], None]: return self.event_log.subscribe(listener) def events_after( @@ -513,9 +507,7 @@ def add_dataset_roots( on that separately prepared root. """ - roots = tuple( - self.dataset_access.require_readable(Path(root)) for root in roots - ) + roots = tuple(self.dataset_access.require_readable(Path(root)) for root in roots) if execution_root is not None: if len(roots) != 1: raise ValueError("A prepared execution root needs exactly one root.") @@ -530,9 +522,7 @@ def add_dataset_roots( ) else: offers = tuple( - offer - for root in roots - for offer in DatasetScopeKind.offers_for_root(root) + offer for root in roots for offer in DatasetScopeKind.offers_for_root(root) ) added = self._add_scopes( tuple(offer.scope for offer in offers), @@ -646,9 +636,7 @@ def _split_dataset_scopes(self) -> None: def delete_datasets(self, scope_ids: Iterable[str]) -> None: doomed = {str(scope_id) for scope_id in scope_ids} - active = sorted( - scope_id for scope_id in doomed if self.has_active_work(scope_id) - ) + active = sorted(scope_id for scope_id in doomed if self.has_active_work(scope_id)) if active: raise RuntimeError( "Cannot delete a dataset with active initialization, compilation, " @@ -716,9 +704,7 @@ def set_pipeline_document(self, scope_id: str, document: PipelineDocument) -> No PipelineObjectStateBinding.commit_plate_state(scope_id) self.pipeline_changed(scope_id) self.publish( - StatusReported( - f"Pipeline updated with {len(document.pipeline_steps)} steps" - ) + StatusReported(f"Pipeline updated with {len(document.pipeline_steps)} steps") ) def set_pipeline_source(self, scope_id: str, source: str) -> None: @@ -762,9 +748,7 @@ def replace_step( edited_step: FunctionStep, ) -> None: self.require_definition_mutation_allowed(scope_id) - PipelineObjectStateBinding.replace_plate_step( - scope_id, current_step, edited_step - ) + PipelineObjectStateBinding.replace_plate_step(scope_id, current_step, edited_step) self.pipeline_changed(scope_id) self.publish(StatusReported(f"Updated step: {edited_step.name}")) @@ -796,9 +780,7 @@ def delete_steps(self, scope_id: str, step_scope_ids: Iterable[str]) -> None: doomed = set(step_scope_ids) self.require_definition_mutation_allowed(scope_id) steps = self.pipeline_steps(scope_id) - kept = [ - step for step in steps if self.step_scope_id(scope_id, step) not in doomed - ] + kept = [step for step in steps if self.step_scope_id(scope_id, step) not in doomed] names = [step.name for step in steps if step not in kept] with ObjectStateRegistry.atomic(f"delete steps {', '.join(names)}"): for step_scope_id in doomed: @@ -901,9 +883,7 @@ def require_definition_mutation_allowed(self, scope_id: str | None = None) -> No "active initialization or compilation. Other datasets remain editable." ) - def require_definition_mutation_allowed_for_object_scope( - self, scope_id: str - ) -> None: + def require_definition_mutation_allowed_for_object_scope(self, scope_id: str) -> None: """Authorize an ObjectState mutation that may belong to a dataset.""" if scope_id == "": @@ -1044,13 +1024,10 @@ async def initialize(scope_id: str) -> None: self.publish(DatasetsChanged()) def run() -> InputWorkspacePreparationResult | None: - ensure_global_config_context( - GlobalPipelineConfig, self.global_config - ) + ensure_global_config_context(GlobalPipelineConfig, self.global_config) workspace = scope.prepare_input_workspace() - if ( - workspace is None - and not self.dataset_access.initializes_in_place(scope.root) + if workspace is None and not self.dataset_access.initializes_in_place( + scope.root ): workspace = self.mirrored_workspace(scope.root) if workspace is not None: @@ -1078,7 +1055,9 @@ def run() -> InputWorkspacePreparationResult | None: ) self.set_dataset_state(scope_id, OrchestratorState.READY) if scope_id == self.current_scope_id or not self.current_scope_id: - self.main_thread.call(lambda: self._reselect(scope_id)) + self.main_thread.call( + lambda: self._reselect(scope_id) + ) done += 1 self.publish(ProgressAdvanced(done)) @@ -1143,9 +1122,7 @@ def _arm_run_deadline( timer = threading.Timer( wait_timeout_ms / 1000, lambda: self.main_thread.post( - lambda: self._run_deadline_passed( - batch_number, scope_ids, wait_timeout_ms - ) + lambda: self._run_deadline_passed(batch_number, scope_ids, wait_timeout_ms) ), ) timer.daemon = True @@ -1230,9 +1207,7 @@ def observation_export_params( return None path = self.dataset_access.require_writable(Path(export_path)) if path.exists(): - raise FileExistsError( - f"Runtime observation export path already exists: {path}" - ) + raise FileExistsError(f"Runtime observation export path already exists: {path}") return ZMQAuxiliaryExecutionParams( runtime_observation_export_path=path, runtime_observation_export_scope=scope, @@ -1266,9 +1241,7 @@ def finish_dataset_execution( ) -> None: """Record one dataset's terminal status (from any thread).""" - self.main_thread.post( - lambda: self._finish_dataset_execution(completion, scope_id) - ) + self.main_thread.post(lambda: self._finish_dataset_execution(completion, scope_id)) def _finish_dataset_execution( self, @@ -1300,9 +1273,7 @@ def _finish_dataset_execution( if status.emit_failure: detail = completion.traceback_text or completion.message separator = ":\n\n" if completion.traceback_text else ": " - self.publish( - ErrorReported(f"Execution failed for {scope_id}{separator}{detail}") - ) + self.publish(ErrorReported(f"Execution failed for {scope_id}{separator}{detail}")) def _add_output_dataset( self, @@ -1338,17 +1309,12 @@ def _add_output_dataset( def finish_batch(self, completed_count: int, failed_count: int) -> None: self.execution_state = ManagerExecutionState.IDLE summary = f"All done: {completed_count} completed, {failed_count} failed" - if ( - completed_count > 1 - and self.global_config.analysis_consolidation_config.enabled - ): + if completed_count > 1 and self.global_config.analysis_consolidation_config.enabled: try: consolidate_batch_results(self) summary += ". Global summary created." except Exception as error: - logger.error( - "Failed to create global summary: %s", error, exc_info=True - ) + logger.error("Failed to create global summary: %s", error, exc_info=True) summary += ". Global summary failed." self.publish(StatusReported(summary)) self.publish(BatchFinished(completed_count, failed_count)) @@ -1381,9 +1347,7 @@ async def run_debug( return self.require_execution_allowed((scope_id,)) - debug_session = DebugSession.create( - subject_id=scope_id, command_type=command_type - ) + debug_session = DebugSession.create(subject_id=scope_id, command_type=command_type) self.debug_sessions[scope_id] = debug_session root = DatasetScope.parse(scope_id).root snapshot_root = (root if root.is_dir() else root.parent) / ".openhcs_debug" @@ -1404,9 +1368,7 @@ async def run_debug( replay_mode=DebugReplayMode.PERSISTENT_PAUSED_WORKER, ) request = dataset_pipeline_request(scope_id, self.global_config) - artifact_id = await self.debug_runs.compile_artifact_id( - request, debug_request - ) + artifact_id = await self.debug_runs.compile_artifact_id(request, debug_request) self.publish(StatusReported(f"Submitting debug run for {scope_id}...")) await self.debug_runs.submit( request, @@ -1468,9 +1430,7 @@ def inspect_debug_snapshot( snapshot_store_backend=context.snapshot_store_backend, axis_id=event.axis_id, ).with_cursor(context.cursor) - elif ( - summary is not None and summary.debug_session_id == context.debug_session_id - ): + elif summary is not None and summary.debug_session_id == context.debug_session_id: self.inspected_debug_sessions.pop(scope_id, None) if snapshot is not None: self.debug_terminal_summaries[scope_id] = summary.with_snapshot( @@ -1520,9 +1480,7 @@ def debug_context(self, scope_id: str) -> DebugSessionProjectionContext: return DebugSessionProjectionContext( target=DebugSessionTargetState( current_plate_scope_id=scope_id, - pipeline_scope_id=PipelineScopeIdentity.from_plate_scope( - scope_id - ).scope_id, + pipeline_scope_id=PipelineScopeIdentity.from_plate_scope(scope_id).scope_id, initialized=self.is_initialized(scope_id), compiled=scope_id in self.compiled, terminal_status=( @@ -1548,3 +1506,4 @@ def current_debug_context(self) -> DebugSessionProjectionContext | None: if not self.current_scope_id: return None return self.debug_context(self.current_scope_id) + diff --git a/openhcs/authoring/session/submission.py b/openhcs/authoring/session/submission.py index d882213f2e..da4e2f9b39 100644 --- a/openhcs/authoring/session/submission.py +++ b/openhcs/authoring/session/submission.py @@ -44,7 +44,6 @@ class SubmittedExecution: request: ZMQExecutionRequestBuilder endpoint: "PongResponse | None" - logger = logging.getLogger(__name__) @@ -75,9 +74,7 @@ async def submit_execution( execution_id = await self._submit( request, compile_artifact_id=compile_artifact_id, - send=lambda: client.submit_prepared_pipeline( - prepared, timeout_ms=timeout_ms - ), + send=lambda: client.submit_prepared_pipeline(prepared, timeout_ms=timeout_ms), label="execution", ) if execution_id is not None: @@ -135,9 +132,7 @@ async def _submit( error_text = response.require_failure_text(f"{label} submission") logger.error("%s submission failed for %s: %s", label, scope_id, error_text) - session.publish( - ErrorReported(f"Submission failed for {scope_id}: {error_text}") - ) + session.publish(ErrorReported(f"Submission failed for {scope_id}: {error_text}")) session.batch.mark_terminal(scope_id, TerminalExecutionStatus.FAILED) session.set_dataset_state(scope_id, OrchestratorState.EXEC_FAILED) return None diff --git a/openhcs/authoring/session/views.py b/openhcs/authoring/session/views.py index 6132a3b535..4f52d57c13 100644 --- a/openhcs/authoring/session/views.py +++ b/openhcs/authoring/session/views.py @@ -174,11 +174,7 @@ def orchestrator_state(self): if terminal is not None: return terminal.orchestrator_state orchestrator = ObjectStateRegistry.get_object(self.scope_id) - return ( - orchestrator.state - if isinstance(orchestrator, PipelineOrchestrator) - else None - ) + return orchestrator.state if isinstance(orchestrator, PipelineOrchestrator) else None @property def execution_id(self) -> str | None: @@ -259,9 +255,7 @@ def state_of(cls, session: "Session") -> DatasetListState: selected = set(session.selected_scope_ids) return DatasetListState( rows=tuple( - cls.row_state( - session, row, relations[row.scope_id], row.scope_id in selected - ) + cls.row_state(session, row, relations[row.scope_id], row.scope_id in selected) for row in rows ), selected_scope_ids=session.selected_scope_ids, @@ -334,9 +328,7 @@ def row_state( def _function_entries(function_spec) -> tuple: if function_spec is None: return () - entries = ( - tuple(function_spec) if isinstance(function_spec, list) else (function_spec,) - ) + entries = tuple(function_spec) if isinstance(function_spec, list) else (function_spec,) functions = [] for entry in entries: function = entry[0] if isinstance(entry, tuple) else entry @@ -401,9 +393,7 @@ def state_of( selected=step_scope_id in selected, dirty=bool(state.dirty_fields) if state is not None else False, default_diff=( - bool(state.signature_diff_fields) - if state is not None - else False + bool(state.signature_diff_fields) if state is not None else False ), description=step.description, debug_pause=step.debug_pause, diff --git a/openhcs/core/aligned_image_payload.py b/openhcs/core/aligned_image_payload.py index acf9a7d909..c7dd71391d 100644 --- a/openhcs/core/aligned_image_payload.py +++ b/openhcs/core/aligned_image_payload.py @@ -83,9 +83,7 @@ def metadata(self) -> ImagePayloadMetadata: return self.value.metadata @classmethod - def for_payload( - cls, value: ImagePayload - ) -> "ImagePayloadSourceSpatialDomainAdapter": + def for_payload(cls, value: ImagePayload) -> "ImagePayloadSourceSpatialDomainAdapter": """Place an image payload in the source domain its metadata declares.""" return cls( value, @@ -228,8 +226,7 @@ def value_in_payload_domain( ), ) materialized_mask = materialized.mask - return materialized.with_pixels( - target.extract_source_array( + return materialized.with_pixels(target.extract_source_array( materialized.data, spatial_axes_yx=self.spatial_axes_yx, ), @@ -241,11 +238,12 @@ def value_in_payload_domain( spatial_axes_yx=self.spatial_axes_yx, ) ), - metadata=metadata, - ) + metadata=metadata,) -class BareArraySourceSpatialDomainAdapter(ImagePayloadSourceSpatialDomainAdapter): +class BareArraySourceSpatialDomainAdapter( + ImagePayloadSourceSpatialDomainAdapter +): """Source-domain adapter for bare pixels, which declare no placement.""" @classmethod @@ -390,10 +388,9 @@ def validate_main_flow_cohort( producer_records: Sequence[ProducedOutputSemantics] | None, ) -> None: """One input cohort has one declared whole-image composition domain.""" - if ( - producer_records - and len({record.main_flow_plane_axis for record in producer_records}) != 1 - ): + if producer_records and len( + {record.main_flow_plane_axis for record in producer_records} + ) != 1: raise ValueError( "One main-flow cohort cannot combine different declared image axes." ) @@ -417,9 +414,7 @@ def from_loaded_images( ): main_data_stack = payloads[0].copied( memory_type=execution_plan.input_memory_type, - device_id=execution_plan.device_id_for( - execution_plan.input_memory_type - ), + device_id=execution_plan.device_id_for(execution_plan.input_memory_type), ) else: metadata_mode = ImagePayloadMetadataCompositionMode.STACK @@ -430,7 +425,9 @@ def from_loaded_images( metadata_mode = ( ImagePayloadMetadataCompositionMode.BUNDLE if declared_axis is None - and len({record.output_context for record in producer_records}) > 1 + and len( + {record.output_context for record in producer_records} + ) > 1 else ImagePayloadMetadataCompositionMode.for_plane_axis( declared_axis or RuntimePlaneAxis.RUNTIME_SLICE ) @@ -441,17 +438,13 @@ def from_loaded_images( ) if metadata_mode is ImagePayloadMetadataCompositionMode.STACK: main_data_stack = stack_image_payloads( - payloads, - metadata_mode=metadata_mode, + payloads, metadata_mode=metadata_mode, memory_type=execution_plan.input_memory_type, - device_id=execution_plan.device_id_for( - execution_plan.input_memory_type - ), + device_id=execution_plan.device_id_for(execution_plan.input_memory_type), ) elif metadata_mode is ImagePayloadMetadataCompositionMode.BUNDLE: main_data_stack = ImagePayloadBundleContext.from_payloads( - tuple(payloads), - metadata_mode=metadata_mode, + tuple(payloads), metadata_mode=metadata_mode, ).compose() return main_data_stack @@ -471,13 +464,8 @@ def with_saved_output_context( and single_output_plane_axis is metadata[0].plane_axis and np.shape(data) == np.shape(payloads[0].data) ): - member = ( - metadata[0] - .with_current_intensity_from(current_intensity) - .payload_with( - data, - stack_payload.mask, - ) + member = metadata[0].with_current_intensity_from(current_intensity).payload_with( + data, stack_payload.mask, ) if member.keeps_whole_as_single_member( declared_plane_axis=single_output_plane_axis, @@ -485,8 +473,7 @@ def with_saved_output_context( return member # Cache the cohort the next step composes from this one saved member. return stack_image_payloads( - (member,), - metadata_mode=ImagePayloadMetadataCompositionMode.STACK, + (member,), metadata_mode=ImagePayloadMetadataCompositionMode.STACK, ) if np.shape(data)[:1] != (len(payloads),): raise ValueError( @@ -500,21 +487,17 @@ def with_saved_output_context( else ImagePayloadMetadataCompositionMode.for_plane_axis(stack_plane_axis) ) output_metadata = ImagePayloadMetadata.compose( - tuple(payloads), - mode=mode, - source_metadata=tuple( + tuple(payloads), mode=mode, source_metadata=tuple( record.with_current_intensity_from(current_intensity, plane_index=index) for index, record in enumerate(metadata) ), ) return output_metadata.payload_with( - data, - _stack_image_payload_mask(tuple(payloads), data), + data, _stack_image_payload_mask(tuple(payloads), data), ) def composition_source_metadata( - self, - payloads: Sequence[Any] | None = None, + self, payloads: Sequence[Any] | None = None, ) -> tuple[ImagePayloadMetadata, ...]: return tuple( self.composition_payload_metadata(payload.metadata) @@ -529,10 +512,8 @@ def composition_payload_metadata( @staticmethod def composition_memory_domain( - payloads: tuple[RuntimeArrayData, ...], - *, - memory_type: str | None = None, - device_id: int | None = None, + payloads: tuple[RuntimeArrayData, ...], *, + memory_type: str | None = None, device_id: int | None = None, ) -> tuple[str, int | None]: """Resolve where a composition is placed: explicitly, else beside its first payload.""" if memory_type is not None: @@ -541,24 +522,17 @@ def composition_memory_domain( return memory_type, MemoryType(memory_type).device_id_of(payloads[0]) def compose( - self, - *, - memory_type: str | None = None, - device_id: int | None = None, + self, *, memory_type: str | None = None, device_id: int | None = None, ) -> Any: # Resolve the destination from the input payloads before reconciling intensity. memory_type, device_id = self.composition_memory_domain( tuple(payload.data for payload in self.composition_payloads), - memory_type=memory_type, - device_id=device_id, - ) - payloads = ImagePayloadMetadata.intensity_coherent_payloads( - self.composition_payloads + memory_type=memory_type, device_id=device_id, ) + payloads = ImagePayloadMetadata.intensity_coherent_payloads(self.composition_payloads) composed = self.compose_unmasked( tuple(payload.data for payload in payloads), - memory_type=memory_type, - device_id=device_id, + memory_type=memory_type, device_id=device_id, ) metadata = ImagePayloadMetadata.compose( payloads, @@ -568,21 +542,14 @@ def compose( return metadata.payload_with(composed, self.compose_mask(composed, metadata)) def compose_unmasked( - self, - payloads: tuple[RuntimeArrayData, ...], - *, - memory_type: str | None = None, - device_id: int | None = None, + self, payloads: tuple[RuntimeArrayData, ...], *, + memory_type: str | None = None, device_id: int | None = None, ) -> RuntimeArrayData: memory_type, device_id = self.composition_memory_domain( - payloads, - memory_type=memory_type, - device_id=device_id, + payloads, memory_type=memory_type, device_id=device_id, ) return stack_runtime_slices( - payloads, - memory_type, - device_id, + payloads, memory_type, device_id, ) def compose_mask(self, composed: Any, metadata: ImagePayloadMetadata) -> Any | None: @@ -625,8 +592,7 @@ def stack_image_payloads( """Stack image payloads in their declared memory domain with full context.""" return ImagePayloadStackContext(image_payloads, metadata_mode).compose( - memory_type=memory_type, - device_id=device_id, + memory_type=memory_type, device_id=device_id, ) @@ -663,7 +629,9 @@ def _stack_image_payload_mask( stacked_mask_shape ): resolved_masks = tuple( - payload.metadata.mask_domain(slice_domain).broadcast_to_data(mask) + payload.metadata + .mask_domain(slice_domain) + .broadcast_to_data(mask) for payload, slice_domain, mask in zip( payloads, output_slice_domains, resolved_masks, strict=True ) @@ -689,14 +657,10 @@ def _complete_image_payload_mask( f"{tuple(np.shape(payload_data))!r}." ) return project_image_mask_to_data_domain( - mask, - payload_data, - metadata=payload.metadata, + mask, payload_data, metadata=payload.metadata, ) return MemoryType(detect_memory_type(payload_data)).ones_like( - payload_data, - shape=mask_domain.default_mask_shape(), - dtype=bool, + payload_data, shape=mask_domain.default_mask_shape(), dtype=bool, ) @@ -861,9 +825,7 @@ def combined_mask(self, composed: Any) -> RuntimeArrayData | None: device_id = memory_type.device_id_of(data) prepared = tuple( memory_type.astype( - MemoryType(detect_memory_type(mask)).convert_to( - mask, memory_type, device_id - ), + MemoryType(detect_memory_type(mask)).convert_to(mask, memory_type, device_id), bool, ) for mask in masks @@ -879,15 +841,11 @@ def combined_mask(self, composed: Any) -> RuntimeArrayData | None: def compose_unmasked( self, payloads: tuple[RuntimeArrayData, ...], - *, - memory_type: str | None = None, - device_id: int | None = None, + *, memory_type: str | None = None, device_id: int | None = None, ) -> RuntimeArrayData: """Compose image payload arrays without mask/metadata wrapping.""" memory_type, device_id = self.composition_memory_domain( - payloads, - memory_type=memory_type, - device_id=device_id, + payloads, memory_type=memory_type, device_id=device_id, ) channel_axes = tuple( metadata.axis_index(ColourAxis, payload) @@ -900,9 +858,7 @@ def compose_unmasked( declared_channel_count = sum(axis is not None for axis in channel_axes) if declared_channel_count in {0, len(payloads)}: return super(ImagePayloadBundleContext, self).compose_unmasked( - payloads, - memory_type=memory_type, - device_id=device_id, + payloads, memory_type=memory_type, device_id=device_id, ) return self.compose_mixed_channel_payloads( payloads, @@ -972,17 +928,9 @@ def compose_mixed_channel_payloads( else target.broadcast_to( target.reshape( payload, - ( - *payload.shape[:channel_axis], - 1, - *payload.shape[channel_axis:], - ), - ), - ( - *payload.shape[:channel_axis], - channel_count, - *payload.shape[channel_axis:], + (*payload.shape[:channel_axis], 1, *payload.shape[channel_axis:]), ), + (*payload.shape[:channel_axis], channel_count, *payload.shape[channel_axis:]), ) ) for payload, axis in zip(prepared_payloads, channel_axes, strict=True) @@ -1146,7 +1094,6 @@ class ImagePayloadSliceStack(ImagePayloadStackComposition, ImagePayload): slices: tuple[Any, ...] slice_contexts: tuple[AlignedImageSliceContext, ...] = () - @classmethod def from_output_slices( cls, @@ -1157,9 +1104,7 @@ def from_output_slices( ) -> "ProducedImageStack": """Wrap produced image slices as a stack without allocating its pixels.""" return ProducedImageStack( - tuple(slices), - memory_type=memory_type, - plane_axis=plane_axis, + tuple(slices), memory_type=memory_type, plane_axis=plane_axis, ) @property @@ -1217,9 +1162,7 @@ def composition_payloads(self) -> tuple[Any, ...]: return self.slices @property - def projected_output_composition_mode( - self, - ) -> ImagePayloadMetadataCompositionMode | None: + def projected_output_composition_mode(self) -> ImagePayloadMetadataCompositionMode | None: """Declare the outer runtime axis retained by projected output members.""" return self.composition_metadata_mode @@ -1230,9 +1173,7 @@ def alignment_slices(self) -> tuple[Any, ...]: return self.slices def map_slices(self, transform: Any) -> "ImagePayloadSliceStack": - return self.with_slices( - tuple(transform(image_slice) for image_slice in self.slices) - ) + return self.with_slices(tuple(transform(image_slice) for image_slice in self.slices)) def output_slices(self) -> tuple[Any, ...]: return tuple(output for output, _context in self.projected_output_slices()) @@ -1254,16 +1195,13 @@ def value_for_slice(self, context: RuntimePlaneAxisValueProjection) -> Any: ) if all( projected is image_slice - for projected, image_slice in zip( - projected_slices, self.slices, strict=True - ) + for projected, image_slice in zip(projected_slices, self.slices, strict=True) ): return self return self.with_slices(projected_slices) def plane_axis_for_output_context( - self, - context: AlignedImageSliceContext, + self, context: AlignedImageSliceContext, ) -> RuntimePlaneAxis | None: """An explicitly aligned stack declares its outer runtime-slice domain.""" return RuntimePlaneAxis.RUNTIME_SLICE @@ -1274,8 +1212,7 @@ def owns_output_surfaces(self) -> bool: return True def main_output_slices( - self, - unstack: Any, + self, unstack: Any, ) -> tuple[tuple[Any, AlignedImageSliceContext | None], ...]: del unstack return tuple(self.projected_output_slices()) @@ -1288,9 +1225,7 @@ def output_stack_copy( device_id: int | None, ) -> RuntimeArrayData | None: return self.copy_projected_output_stack( - projected_outputs, - memory_type=memory_type, - device_id=device_id, + projected_outputs, memory_type=memory_type, device_id=device_id, ) def __post_init__(self) -> None: @@ -1331,10 +1266,7 @@ def with_slices(self, slices: Sequence[Any]) -> "ImagePayloadSliceStack": return type(self)(tuple(slices), self.slice_contexts) def copied( - self, - *, - memory_type: str, - device_id: int | None, + self, *, memory_type: str, device_id: int | None, ) -> "ImagePayloadSliceStack": """Retain aligned member domains while copying them into independent buffers.""" return self.with_slices( @@ -1373,21 +1305,15 @@ def copy_projected_output_stack( self.plane_axis_for_output_context(context) for _payload, context in projected_outputs } - if ( - len(declared_axes) != 1 - or len({tuple(np.shape(item)) for item in data}) != 1 - ): + if len(declared_axes) != 1 or len({tuple(np.shape(item)) for item in data}) != 1: return None if metadata_mode is ImagePayloadMetadataCompositionMode.BUNDLE: return ImagePayloadBundleContext.from_payloads( - payloads, - metadata_mode=metadata_mode, + payloads, metadata_mode=metadata_mode, ).compose() return stack_image_payloads( - payloads, - metadata_mode=metadata_mode, - memory_type=memory_type, - device_id=device_id, + payloads, metadata_mode=metadata_mode, + memory_type=memory_type, device_id=device_id, ) def output_values_for_artifact_specs( @@ -1527,14 +1453,9 @@ class ProducedImageStack(ImagePayloadSliceStack): plane_axis: RuntimePlaneAxis source_metadata: InitVar[ImagePayloadMetadata | None] = None _metadata: ImagePayloadMetadata = field(init=False, repr=False) - _composed_payload: RuntimeArrayData | None = field( - default=None, init=False, repr=False - ) + _composed_payload: RuntimeArrayData | None = field(default=None, init=False, repr=False) _realization_lock: LockType = field( - default_factory=Lock, - init=False, - repr=False, - compare=False, + default_factory=Lock, init=False, repr=False, compare=False, ) def __post_init__(self, source_metadata: ImagePayloadMetadata | None) -> None: @@ -1546,32 +1467,22 @@ def __post_init__(self, source_metadata: ImagePayloadMetadata | None) -> None: masks = tuple(payload.mask for payload in self.slices) present_masks = tuple(mask for mask in masks if mask is not None) if present_masks and len(present_masks) != len(masks): - raise ValueError( - "Cannot aggregate a mix of masked and unmasked image payloads." - ) + raise ValueError("Cannot aggregate a mix of masked and unmasked image payloads.") self._metadata = ( ImagePayloadMetadata.compose( self.slices, - mode=ImagePayloadMetadataCompositionMode.for_plane_axis( - self.plane_axis - ), + mode=ImagePayloadMetadataCompositionMode.for_plane_axis(self.plane_axis), ) - if source_metadata is None - else source_metadata + if source_metadata is None else source_metadata ) if self._metadata.plane_axis is not self.plane_axis: - raise ValueError( - "Produced image metadata must retain its declared plane axis." - ) + raise ValueError("Produced image metadata must retain its declared plane axis.") shared_mask = ( - self._shared_image_mask(present_masks, data_geometry) - if present_masks - else None + self._shared_image_mask(present_masks, data_geometry) if present_masks else None ) mask_geometry = ( array_geometry(shared_mask) - if shared_mask is not None - else ( + if shared_mask is not None else ( runtime_slice_stack_geometry(present_masks) if present_masks else None ) ) @@ -1595,16 +1506,14 @@ def composition_metadata_mode(self) -> ImagePayloadMetadataCompositionMode: def dtype(self) -> Any: return np.result_type( *( - MemoryType(detect_memory_type(payload.data)).canonical_dtype_name( - payload.data.dtype - ) + MemoryType(detect_memory_type(payload.data)) + .canonical_dtype_name(payload.data.dtype) for payload in self.slices ) ) def plane_axis_for_output_context( - self, - context: AlignedImageSliceContext, + self, context: AlignedImageSliceContext, ) -> RuntimePlaneAxis: return self.plane_axis @@ -1625,37 +1534,26 @@ def with_metadata(self, metadata: ImagePayloadMetadata) -> Any: return super(ProducedImageStack, self).with_metadata(metadata) slices = tuple( metadata.for_leading_source_plane(index).payload_with( - payload.data, - payload.mask, + payload.data, payload.mask, ) for index, payload in enumerate(self.slices) ) return type(self)( - slices, - self.slice_contexts, - memory_type=self.memory_type, - plane_axis=self.plane_axis, - source_metadata=metadata, + slices, self.slice_contexts, memory_type=self.memory_type, + plane_axis=self.plane_axis, source_metadata=metadata, ) def normalize_intensity_payload( - self, - *, - dtype: Any = None, - channel_index: int = 0, + self, *, dtype: Any = None, channel_index: int = 0, ) -> Any: if self._metadata.normalization_dtype(self.dtype, dtype) is None: return self return self._metadata.normalize_intensity_payload( - self.compose(), - dtype=dtype, - channel_index=channel_index, + self.compose(), dtype=dtype, channel_index=channel_index, ) def _shared_image_mask( - self, - masks: tuple[Any, ...], - data_geometry: ArrayGeometry, + self, masks: tuple[Any, ...], data_geometry: ArrayGeometry, ) -> Any | None: first = masks[0] if all(mask is first for mask in masks) and self._metadata.mask_domain( @@ -1679,14 +1577,8 @@ def _materialized_image_mask(self) -> Any | None: if masks[0] is None: return None shared = self._shared_image_mask(masks, self.geometry) - return ( - shared - if shared is not None - else stack_runtime_slices( - masks, - self.memory_type, - 0, - ) + return shared if shared is not None else stack_runtime_slices( + masks, self.memory_type, 0, ) def __getstate__(self) -> dict[str, Any]: @@ -1719,17 +1611,12 @@ def _retain_composed_slices(self) -> None: def with_slices(self, slices: Sequence[Any]) -> "ProducedImageStack": return type(self)( - tuple(slices), - self.slice_contexts, - memory_type=self.memory_type, - plane_axis=self.plane_axis, + tuple(slices), self.slice_contexts, + memory_type=self.memory_type, plane_axis=self.plane_axis, ) def copied( - self, - *, - memory_type: str, - device_id: int | None, + self, *, memory_type: str, device_id: int | None, ) -> "ProducedImageStack": """Snapshot produced pixels directly into one dense buffer. @@ -1739,8 +1626,7 @@ def copied( """ data = stack_runtime_slices( tuple(payload.data for payload in self.slices), - memory_type, - device_id, + memory_type, device_id, ) masks = tuple(payload.mask for payload in self.slices) mask = None @@ -1748,41 +1634,30 @@ def copied( shared = self._shared_image_mask(masks, self.geometry) mask = ( stack_runtime_slices(masks, memory_type, device_id) - if shared is None - else stack_runtime_slices( - (shared,), - memory_type, - device_id, + if shared is None else stack_runtime_slices( + (shared,), memory_type, device_id, )[0] ) metadata = self._metadata.replace_fields() copied = type(self)( - self.slices, - self.slice_contexts, - memory_type=memory_type, - plane_axis=self.plane_axis, - source_metadata=metadata, + self.slices, self.slice_contexts, memory_type=memory_type, + plane_axis=self.plane_axis, source_metadata=metadata, ) copied._composed_payload = metadata.payload_with(data, mask) copied._retain_composed_slices() return copied def compose( - self, - *, - memory_type: str | None = None, - device_id: int | None = None, + self, *, memory_type: str | None = None, device_id: int | None = None, ) -> Any: with self._realization_lock: if self._composed_payload is None: data = stack_runtime_slices( tuple(payload.data for payload in self.slices), - self.memory_type, - 0, + self.memory_type, 0, ) self._composed_payload = self._metadata.payload_with( - data, - self._materialized_image_mask(), + data, self._materialized_image_mask(), ) self._retain_composed_slices() if memory_type is None: @@ -1795,14 +1670,8 @@ def compose( mask = self._composed_payload.mask return self._metadata.payload_with( source.convert_to(data, target, device_id), - ( - None - if mask is None - else MemoryType(detect_memory_type(mask)).convert_to( - mask, - target, - device_id, - ) + None if mask is None else MemoryType(detect_memory_type(mask)).convert_to( + mask, target, device_id, ), ) @@ -1823,9 +1692,7 @@ def composition_metadata_mode(self) -> ImagePayloadMetadataCompositionMode: return ImagePayloadMetadataCompositionMode.BUNDLE @property - def projected_output_composition_mode( - self, - ) -> ImagePayloadMetadataCompositionMode | None: + def projected_output_composition_mode(self) -> ImagePayloadMetadataCompositionMode | None: """Flatten declared inner runtime planes, retaining other named image domains.""" if any( payload.metadata.plane_axis is RuntimePlaneAxis.RUNTIME_SLICE @@ -1837,15 +1704,11 @@ def projected_output_composition_mode( return self.composition_metadata_mode def plane_axis_for_output_context( - self, - context: AlignedImageSliceContext, + self, context: AlignedImageSliceContext, ) -> RuntimePlaneAxis | None: """Resolve a named output's own inner plane axis before slice projection.""" payloads = tuple( - payload - for payload, declared_context in zip( - self.slices, self.slice_contexts, strict=True - ) + payload for payload, declared_context in zip(self.slices, self.slice_contexts, strict=True) if declared_context == context ) if len(payloads) != 1: @@ -2021,7 +1884,9 @@ def compose_aligned_image_payload( if len(image_payloads) == 1: return ImagePayloadComposition( payload=aligned_payloads[0], - execution_mode=(AlignedStackExecution), + execution_mode=( + AlignedStackExecution + ), ) declared_contexts = tuple( payload.slice_contexts diff --git a/openhcs/core/artifacts.py b/openhcs/core/artifacts.py index fbe89bde93..3310dc79a0 100644 --- a/openhcs/core/artifacts.py +++ b/openhcs/core/artifacts.py @@ -658,8 +658,7 @@ def contextualize_output( tuple( cls.contextualize_output( RuntimeSliceProjection.value_for_slice( - source_payload, - plane_projection.selected_plane(slice_index), + source_payload, plane_projection.selected_plane(slice_index), ), item.data if isinstance(item, RuntimeValue) else item, output_plan, @@ -685,9 +684,7 @@ def contextualize_output( output_metadata.plane_axis is None and output_metadata.source_provenance.source_plane_count == plane_projection.axis_size - and plane_projection.dense_shape_carries_axis( - output_value.geometry.shape - ) + and plane_projection.dense_shape_carries_axis(output_value.geometry.shape) ): output_value = output_metadata.replace_fields( plane_axis=plane_projection.axis, @@ -834,9 +831,7 @@ def compose_runtime_values( from openhcs.core.aligned_image_payload import stack_image_payloads from openhcs.core.memory import detect_memory_type - from openhcs.core.runtime_image_values import ( - ImagePayloadMetadataCompositionMode, - ) + from openhcs.core.runtime_image_values import ImagePayloadMetadataCompositionMode from openhcs.core.runtime_plane_projection import RuntimePlaneAxis components = {value.key.scope.component for value in values} @@ -3655,6 +3650,7 @@ def materialization_metadata( ) -> "ImagePayloadMetadata": """Derive filename metadata from the declared naming source.""" + payload_metadata = self.artifact_type.materialization_image_metadata(value) materialization_source = self.materialization_source() if ( diff --git a/openhcs/core/axes.py b/openhcs/core/axes.py index 0adc4aa06e..d79cccb8dc 100644 --- a/openhcs/core/axes.py +++ b/openhcs/core/axes.py @@ -351,17 +351,8 @@ def _validate_declaration(cls) -> None: if issubclass(cls, GridAddressed): undeclared = [ member - for member in ( - "default_grid", - "grid_coordinates", - "grid_position", - "row_label", - ) - if not any( - member in base.__dict__ - for base in cls.__mro__ - if base is not GridAddressed - ) + for member in ("default_grid", "grid_coordinates", "grid_position", "row_label") + if not any(member in base.__dict__ for base in cls.__mro__ if base is not GridAddressed) ] if undeclared: raise TypeError( @@ -405,6 +396,7 @@ def filename_token(cls, value: object) -> str: return f"{cls.filename_prefix}{text}" + class Ungrouped(GroupingDeclaration): """Explicit absent-grouping declaration: the step does not partition its value.""" @@ -506,28 +498,21 @@ def _validate_declaration(cls) -> None: f"Axis family {cls.__qualname__} must declare exactly one PartitionAxis." ) spatial_rank = cls.__dict__.get("payload_spatial_rank") - if ( - not isinstance(spatial_rank, int) - or isinstance(spatial_rank, bool) - or spatial_rank < 0 - ): + if not isinstance(spatial_rank, int) or isinstance(spatial_rank, bool) or spatial_rank < 0: raise TypeError( f"Axis family {cls.__qualname__} must declare payload_spatial_rank " "as a nonnegative int." ) for axis in declared: if "family" in axis.__dict__: - raise TypeError( - f"Axis {axis.__qualname__} already belongs to a family." - ) + raise TypeError(f"Axis {axis.__qualname__} already belongs to a family.") axis.family = cls cls.axes = declared cls.family_name = f"{cls.__module__}:{cls.__qualname__}" + def __new__(cls, *args: object, **kwargs: object): - raise TypeError( - f"{cls.__qualname__} is a family declaration, not a value type." - ) + raise TypeError(f"{cls.__qualname__} is a family declaration, not a value type.") # -- activation --------------------------------------------------------- @@ -554,9 +539,7 @@ def activate_entry_point(cls) -> None: """ inherited = os.environ.get(ACTIVE_FAMILY_VARIABLE) - ( - cls if inherited is None else AxisFamily.from_family_name(inherited) - ).activate() + (cls if inherited is None else AxisFamily.from_family_name(inherited)).activate() @staticmethod def from_family_name(family_name: str) -> type[AxisFamily]: diff --git a/openhcs/core/callable_contract.py b/openhcs/core/callable_contract.py index d1fdca8dc2..baf16cd4c8 100644 --- a/openhcs/core/callable_contract.py +++ b/openhcs/core/callable_contract.py @@ -274,9 +274,7 @@ def validate_source_metadata( ) -> None: """Validate one exact source after source-binding transformations.""" - if not load_as_monochrome and metadata.pixel_semantics.axes.with_role( - self.role - ): + if not load_as_monochrome and metadata.pixel_semantics.axes.with_role(self.role): return raise ValueError( f"Primary image requires a declared {self.role.__name__} payload axis, " @@ -336,7 +334,8 @@ def _validate_optional_instance( if value is not None and not isinstance(value, kind): raise TypeError( - f"{owner} must be {kind.__name__} or None, got " f"{type(value).__name__}." + f"{owner} must be {kind.__name__} or None, got " + f"{type(value).__name__}." ) @@ -350,7 +349,7 @@ def __init__( parameters=None, *, return_annotation=inspect.Signature.empty, - __validate_parameters__=True, + __validate_parameters__=True ): from openhcs.core.vfs_protocol import DatasetPathDeclaration @@ -418,18 +417,14 @@ class CallableMetadata: payload_axis_transition: PayloadAxisTransition | None = None @staticmethod - def prepared_callable_signature( - func: Callable[..., object], - ) -> inspect.Signature | None: + def prepared_callable_signature(func: Callable[..., object]) -> inspect.Signature | None: """Admit an owned snapshot only for its exact semantic callable boundary.""" raw = getattr(func, FunctionContractAttribute.raw_processing_function, None) if raw is not None and raw is not func: return None signature = getattr(func, FunctionContractAttribute.canonical_signature, None) if signature is not None and not isinstance(signature, inspect.Signature): - raise TypeError( - "Prepared callable signature must be an inspect.Signature or None." - ) + raise TypeError("Prepared callable signature must be an inspect.Signature or None.") return signature @classmethod @@ -472,9 +467,7 @@ def is_request_boundary(candidate: Callable[..., object]) -> bool: ) return resolved - def resolve_signature( - self, raw_callable: Callable[..., object] - ) -> inspect.Signature: + def resolve_signature(self, raw_callable: Callable[..., object]) -> inspect.Signature: """Resolve the semantic ABI once after declaration preparation.""" signature = inspect.signature(raw_callable) annotations = ( @@ -484,9 +477,7 @@ def resolve_signature( ) return signature.replace( parameters=tuple( - parameter.replace( - annotation=annotations.get(name, parameter.annotation) - ) + parameter.replace(annotation=annotations.get(name, parameter.annotation)) for name, parameter in signature.parameters.items() ), return_annotation=annotations.get("return", signature.return_annotation), @@ -507,8 +498,7 @@ def _nominal_argument_types(cls, annotation: object) -> tuple[type, ...]: annotation = resolve_annotated(annotation) if is_union_type(annotation): return tuple( - argument_type - for member in get_args(annotation) + argument_type for member in get_args(annotation) for argument_type in cls._nominal_argument_types(member) ) return ( @@ -517,9 +507,7 @@ def _nominal_argument_types(cls, annotation: object) -> tuple[type, ...]: else () ) - def raw_main_flow_call_argument( - self, source_payload: Any, signature: inspect.Signature - ) -> Any: + def raw_main_flow_call_argument(self, source_payload: Any, signature: inspect.Signature) -> Any: """Pass the main-flow value as the primary parameter declares it. A bare array enters the payload family here; a parameter that does not @@ -529,26 +517,17 @@ def raw_main_flow_call_argument( from openhcs.core.runtime_image_values import ImagePayload, owned_runtime_value primary_name = self.primary_input_name(signature.parameters) - annotation = ( - None - if primary_name is None - else signature.parameters[primary_name].annotation - ) + annotation = None if primary_name is None else signature.parameters[primary_name].annotation if ( annotation is inspect.Parameter.empty and self.image_payload_consumption is ImagePayloadConsumption.COMPOSED ): return source_payload source_payload = owned_runtime_value(source_payload) - return ( - source_payload - if any( - isinstance(source_payload, argument_type) - for argument_type in self._nominal_argument_types(annotation) - ) - or not isinstance(source_payload, ImagePayload) - else source_payload.data - ) + return source_payload if any( + isinstance(source_payload, argument_type) + for argument_type in self._nominal_argument_types(annotation) + ) or not isinstance(source_payload, ImagePayload) else source_payload.data def canonical_signature_for(self, func: Any) -> inspect.Signature: """Use the prepared semantic ABI; unprepared authoring remains live.""" @@ -572,21 +551,14 @@ def raw_runtime_signature_for(self, func: Any) -> inspect.Signature: return self.canonical_signature return self.resolve_signature(self.resolve_raw_runtime_callable(func)) - def with_prepared_signatures( - self, canonical: Callable, runtime: Callable - ) -> "CallableMetadata": + def with_prepared_signatures(self, canonical: Callable, runtime: Callable) -> "CallableMetadata": """Snapshot each actual target independently after declaration preparation.""" return dataclasses.replace( - self, - canonical_signature=self.resolve_signature(canonical), - raw_runtime_signature=( - None if runtime is canonical else self.resolve_signature(runtime) - ), + self, canonical_signature=self.resolve_signature(canonical), + raw_runtime_signature=None if runtime is canonical else self.resolve_signature(runtime), ) - def with_missing_signatures( - self, canonical: Callable[[], Callable], runtime: Callable[[], Callable] - ) -> "CallableMetadata": + def with_missing_signatures(self, canonical: Callable[[], Callable], runtime: Callable[[], Callable]) -> "CallableMetadata": """Reuse library readiness or capture the newly prepared authored targets.""" if self.canonical_signature is not None: return self @@ -594,14 +566,10 @@ def with_missing_signatures( def publish_signatures(self, namespace: MutableMapping[str, Any]) -> None: """Replace the namespace pair as one library preparation epoch.""" - namespace[FunctionContractAttribute.canonical_signature] = ( - self.canonical_signature - ) + namespace[FunctionContractAttribute.canonical_signature] = self.canonical_signature namespace.pop(FunctionContractAttribute.raw_runtime_signature, None) if self.raw_runtime_signature is not None: - namespace[FunctionContractAttribute.raw_runtime_signature] = ( - self.raw_runtime_signature - ) + namespace[FunctionContractAttribute.raw_runtime_signature] = self.raw_runtime_signature @property def artifact_output_policy(self) -> type[ArtifactOutputPolicy]: @@ -616,8 +584,7 @@ def __post_init__(self) -> None: if self.canonical_signature is not None: object.__setattr__( - self, - "canonical_signature", + self, "canonical_signature", CallableSignature.from_signature(self.canonical_signature), ) @@ -698,19 +665,13 @@ def from_projection( if runtime_adapter is not None and callable(projection.func): runtime_adapter.validate_callable_signature(projection.func) declared_artifact_inputs = _artifact_specs_from_namespace( - namespace, - projection.name, - FunctionContractAttribute.artifact_inputs, + namespace, projection.name, FunctionContractAttribute.artifact_inputs, ) signature = ( - cls.callable_signature(projection.func) - if callable(projection.func) - else None + cls.callable_signature(projection.func) if callable(projection.func) else None ) artifact_inputs = _artifact_input_specs_from_projection( - projection, - declared_artifact_inputs, - signature, + projection, declared_artifact_inputs, signature, ) return cls( input_memory_type=reader.optional_string( @@ -805,13 +766,9 @@ def as_namespace(self) -> dict[str, object]: """Project typed metadata into callable declaration keys.""" namespace: dict[str, object] = {} if self.canonical_signature is not None: - namespace[FunctionContractAttribute.canonical_signature] = ( - self.canonical_signature - ) + namespace[FunctionContractAttribute.canonical_signature] = self.canonical_signature if self.raw_runtime_signature is not None: - namespace[FunctionContractAttribute.raw_runtime_signature] = ( - self.raw_runtime_signature - ) + namespace[FunctionContractAttribute.raw_runtime_signature] = self.raw_runtime_signature if self.input_memory_type is not None: MemoryContractAttribute.INPUT.write(namespace, self.input_memory_type) if self.output_memory_type is not None: @@ -1278,10 +1235,7 @@ def lifecycle_anchor_owner_inputs(self) -> ArtifactSpecCollection: undeclared artifact inputs are context for that anchor. A declared group-lineage relation still names its owning inputs. """ - if ( - self.main_flow_supplies_primary_image - and not self.output_group_scope_sources - ): + if self.main_flow_supplies_primary_image and not self.output_group_scope_sources: return ArtifactSpecCollection(()) return self.group_scope_inputs @@ -1404,9 +1358,7 @@ def declared_path_values( signature = self.canonical_signature values: dict[str, tuple["DatasetPathDeclaration", object]] = {} for parameter_name, declaration in self.declared_path_parameters.items(): - value = kwargs.get( - parameter_name, signature.parameters[parameter_name].default - ) + value = kwargs.get(parameter_name, signature.parameters[parameter_name].default) if parameter_name not in kwargs and value is inspect.Parameter.empty: continue if value is not None: @@ -1481,15 +1433,12 @@ def runtime_main_flow_call_argument(self, source_payload: Any) -> Any: def raw_main_flow_call_argument(self, source_payload: Any) -> Any: """Project the metadata-owned raw carrier declaration for this contract.""" - return self.metadata.raw_main_flow_call_argument( - source_payload, self.canonical_signature - ) + return self.metadata.raw_main_flow_call_argument(source_payload, self.canonical_signature) def main_flow_call_argument(self, source_payload: Any) -> Any: """Let the processing declaration retain context needed before raw calls.""" return self.require_processing_contract().main_flow_call_argument( - self, - source_payload, + self, source_payload, ) @property @@ -1656,13 +1605,9 @@ def canonical_signature(self) -> inspect.Signature: def with_prepared_signature(self) -> "CallableContract": """Freeze the prepared declaration into its existing compiled contract.""" - return dataclasses.replace( - self, - metadata=self.metadata.with_prepared_signatures( - self.resolve_canonical_raw_callable(), - self.resolve_raw_runtime_callable(), - ), - ) + return dataclasses.replace(self, metadata=self.metadata.with_prepared_signatures( + self.resolve_canonical_raw_callable(), self.resolve_raw_runtime_callable(), + )) @property def raw_runtime_signature(self) -> inspect.Signature: @@ -1675,13 +1620,9 @@ def from_prepared_callable(cls, func: Callable[..., object]) -> "CallableContrac projection = CallableProjection.from_callable(func) prepared = cls.from_callable(projection.prepared_callable()) return dataclasses.replace( - prepared, - func=func, - function_name=projection.name, - module_name=projection.module_name, + prepared, func=func, function_name=projection.name, module_name=projection.module_name, metadata=prepared.metadata.with_missing_signatures( - prepared.resolve_canonical_raw_callable, - prepared.resolve_raw_runtime_callable, + prepared.resolve_canonical_raw_callable, prepared.resolve_raw_runtime_callable, ), ) @@ -1690,19 +1631,12 @@ def decode_public_kwargs(self, kwargs: Mapping[str, object]) -> dict[str, object decoded = dict(kwargs) for name, annotation in self.canonical_parameter_annotations.items(): - if ( - name not in kwargs - or enum_member_type(resolve_annotated(annotation)) is None - ): + if name not in kwargs or enum_member_type(resolve_annotated(annotation)) is None: continue value = kwargs[name] - decoded[name] = ( - value if value is None else coerce_enum_member(annotation, value) - ) + decoded[name] = value if value is None else coerce_enum_member(annotation, value) validate_annotation_value( - annotation, - decoded[name], - path=f"{self.function_name}.{name}", + annotation, decoded[name], path=f"{self.function_name}.{name}", ) return decoded @@ -1727,10 +1661,7 @@ def bind_runtime_owned(parameter_name: str) -> None: parameter = signature.parameters.get(parameter_name) if parameter is None: return - if ( - parameter_name in call_kwargs - and call_kwargs[parameter_name] is not runtime_owned_value - ): + if parameter_name in call_kwargs and call_kwargs[parameter_name] is not runtime_owned_value: raise TypeError( f"Callable {self.function_name!r} public kwargs cannot set " f"runtime-owned parameter {parameter_name!r}." @@ -1744,22 +1675,13 @@ def bind_runtime_owned(parameter_name: str) -> None: overridable_parameters = self.overridable_runtime_parameter_names for parameter_name in ( - *( - name - for name in self.artifact_input_parameter_names - if name not in call_kwargs or name in runtime_loaded_parameters - ), + *(name for name in self.artifact_input_parameter_names + if name not in call_kwargs or name in runtime_loaded_parameters), self.runtime_context_parameter, - ( - self.runtime_adapter.require_parameter_name() - if self.runtime_adapter is not None - else None - ), - *( - parameter_type.require_parameter_name() - for parameter_type in self.runtime_bound_parameter_types - if parameter_type.require_parameter_name() not in overridable_parameters - ), + self.runtime_adapter.require_parameter_name() if self.runtime_adapter is not None else None, + *(parameter_type.require_parameter_name() + for parameter_type in self.runtime_bound_parameter_types + if parameter_type.require_parameter_name() not in overridable_parameters), *self.config_bound_parameter_names, ): if parameter_name is not None: @@ -2090,12 +2012,8 @@ def attach_callable_contract_metadata( payload_axis_transition: PayloadAxisTransition | None = None, ) -> None: """Attach OpenHCS callable metadata used by compiler/runtime phases.""" - _mutable_callable_namespace(func).pop( - FunctionContractAttribute.canonical_signature, None - ) - _mutable_callable_namespace(func).pop( - FunctionContractAttribute.raw_runtime_signature, None - ) + _mutable_callable_namespace(func).pop(FunctionContractAttribute.canonical_signature, None) + _mutable_callable_namespace(func).pop(FunctionContractAttribute.raw_runtime_signature, None) if processing_contract is not None: if not ( isinstance(processing_contract, type) @@ -2452,11 +2370,7 @@ def _runtime_context_parameter( from openhcs.core.context.processing_context import ProcessingContext parameter_name = ProcessingContext.require_parameter_name() - return ( - parameter_name - if signature is not None and parameter_name in signature.parameters - else None - ) + return parameter_name if signature is not None and parameter_name in signature.parameters else None @dataclass(frozen=True, slots=True) @@ -2468,17 +2382,12 @@ class CallableMetadataReader: def has_prepared_signature(self) -> bool: """Validate the namespace-owned readiness declaration before phase selection.""" - return ( - self.optional_signature(FunctionContractAttribute.canonical_signature) - is not None - ) + return self.optional_signature(FunctionContractAttribute.canonical_signature) is not None def optional_signature(self, field_name: str) -> inspect.Signature | None: value = self.namespace.get(field_name) if value is not None and not isinstance(value, inspect.Signature): - raise TypeError( - f"{self.function_name!r}.{field_name} must be an inspect.Signature or None." - ) + raise TypeError(f"{self.function_name!r}.{field_name} must be an inspect.Signature or None.") return value def optional_string(self, field_name: str) -> str | None: @@ -2571,9 +2480,8 @@ def optional_variable_component_stack_requirement( ) return value - def optional_subclass( - self, field_name: str, family: type[_ValueT] - ) -> type[_ValueT] | None: + + def optional_subclass(self, field_name: str, family: type[_ValueT]) -> type[_ValueT] | None: """Return an optional declared class from one nominal family.""" value = self.namespace.get(field_name) if value is None: diff --git a/openhcs/core/compiled_step_plan.py b/openhcs/core/compiled_step_plan.py index f6286708f0..b53c3096ef 100644 --- a/openhcs/core/compiled_step_plan.py +++ b/openhcs/core/compiled_step_plan.py @@ -318,27 +318,20 @@ class CompiledStepPlan: error: str | None = None def stored_primary_input_edges_for_group( - self, - group: CompiledFunctionGroup, - component_key: str | None, + self, group: CompiledFunctionGroup, component_key: str | None, ) -> tuple[InvocationArtifactInputEdgePlan, ...] | None: """Admit exact producer transport only without path-based transforms.""" if self.input_conversion is not None or self.sequential_filter_plan.enabled: return None edges = group.stored_primary_input_edges_for_component( - self.execution_group_scope, - component_key, + self.execution_group_scope, component_key, source_bindings=self.source_binding_plan, ) if edges and any( - ( - edge.storage_plan.source_step_scope_id == self.step_scope_id - if edge.storage_plan.source_step_scope_id is not None - else edge.storage_plan.source_step_id - in ( - self.step_index, - self.step_scope_id, - ) + edge.storage_plan.source_step_scope_id == self.step_scope_id + if edge.storage_plan.source_step_scope_id is not None + else edge.storage_plan.source_step_id in ( + self.step_index, self.step_scope_id, ) for edge in edges ): @@ -346,8 +339,7 @@ def stored_primary_input_edges_for_group( return edges def requires_main_flow_checkpoint( - self, - plans: Mapping[int, "CompiledStepPlan"], + self, plans: Mapping[int, "CompiledStepPlan"], ) -> bool: """Follow this value's consumers through preserved input lifetimes.""" if not plans or self.step_index not in plans: @@ -379,10 +371,8 @@ def requires_main_flow_checkpoint( if pattern is None or any( (group := pattern.group_for_component(component_key)) is None or consumer.stored_primary_input_edges_for_group( - group, - component_key, - ) - is None + group, component_key, + ) is None for component_key in consumer.execution_group_scope.keys ): return True @@ -562,6 +552,7 @@ def memory_device_scope( return self.device_assignment.scope_for(declaration) + @property def execution_group_value(self) -> str | None: component = self.execution_group_scope.component diff --git a/openhcs/core/component_filters.py b/openhcs/core/component_filters.py index 39301693aa..7a24821549 100644 --- a/openhcs/core/component_filters.py +++ b/openhcs/core/component_filters.py @@ -25,9 +25,7 @@ def __post_init__(self) -> None: if not self.by_axis: return family = AxisFamily.active() - unknown = sorted( - name for name, _values in self.by_axis if name not in family.names() - ) + unknown = sorted(name for name, _values in self.by_axis if name not in family.names()) if unknown: raise ValueError( f"component_filters name unknown axes {unknown}; the " @@ -47,11 +45,7 @@ def from_mapping(cls, mapping: ComponentFilterMapping | None) -> ComponentFilter by_axis=tuple( ( str(name), - ( - (values,) - if isinstance(values, str) - else tuple(str(value) for value in values) - ), + (values,) if isinstance(values, str) else tuple(str(value) for value in values), ) for name, values in mapping.items() ) diff --git a/openhcs/core/config.py b/openhcs/core/config.py index 80b39f4fb7..dc7cc5c551 100644 --- a/openhcs/core/config.py +++ b/openhcs/core/config.py @@ -257,9 +257,7 @@ def slot_for_axis(self, axis: type[Axis]) -> str: def component_modes(self) -> dict[str, str]: """The slot of every axis of the active family, by axis name.""" - return { - axis.name: self.slot_for_axis(axis) for axis in AxisFamily.active().axes - } + return {axis.name: self.slot_for_axis(axis) for axis in AxisFamily.active().axes} def display_payload_extra(self) -> dict[str, ViewerWireValue]: """Stream the axis declarations and this viewer's settings.""" @@ -310,9 +308,7 @@ class NapariDisplayConfig(AxisRoleModeDisplayConfig, NapariDisplaySettings): # Apply the global pipeline config decorator with ui_hidden=True # This config is only inherited by NapariStreamingConfig, so hide it from UI -NapariDisplayConfig = global_pipeline_config(ui_hidden=True, inherit_as_none=False)( - NapariDisplayConfig -) +NapariDisplayConfig = global_pipeline_config(ui_hidden=True, inherit_as_none=False)(NapariDisplayConfig) FijiDimensionMode = FijiSlots.choice_enum("FijiDimensionMode", __name__) @@ -341,9 +337,7 @@ class FijiDisplayConfig(AxisRoleModeDisplayConfig, FijiDisplaySettings): # Apply the global pipeline config decorator with ui_hidden=True # This config is only inherited by FijiStreamingConfig, so hide it from UI -FijiDisplayConfig = global_pipeline_config(ui_hidden=True, inherit_as_none=False)( - FijiDisplayConfig -) +FijiDisplayConfig = global_pipeline_config(ui_hidden=True, inherit_as_none=False)(FijiDisplayConfig) @abbreviation("pfc") @@ -359,9 +353,9 @@ class PartitionFilterConfig(AnnotatedDataclassValidationMixin): show_preview_without_partition_filter: ClassVar[bool] = False - partition_filter: Annotated[ - Optional[Union[List[str], str, int]], abbreviation("") - ] = None + partition_filter: Annotated[Optional[Union[List[str], str, int]], abbreviation("")] = ( + None + ) """Selection matched against the dataset's ordered partition values. Use a list for exact IDs; a string for one ID, comma-separated IDs, @@ -371,9 +365,9 @@ class PartitionFilterConfig(AnnotatedDataclassValidationMixin): mode selects all. """ - partition_filter_mode: Annotated[ - PartitionFilterMode, abbreviation("filter_mode") - ] = PartitionFilterMode.INCLUDE + partition_filter_mode: Annotated[PartitionFilterMode, abbreviation("filter_mode")] = ( + PartitionFilterMode.INCLUDE + ) """Apply ``partition_filter`` as the values to include or the values to exclude. Include mode rejects unknown explicit values. Exclude mode ignores unknown @@ -628,9 +622,7 @@ class StepPartitionFilterConfig(PartitionFilterConfig): @abbreviation("mat") @global_pipeline_config(preview_label="MAT", always_viewable_fields=["sub_dir"]) @dataclass(frozen=True) -class StepMaterializationConfig( - Enableable, StepPartitionFilterConfig, PathPlanningConfig -): +class StepMaterializationConfig(Enableable, StepPartitionFilterConfig, PathPlanningConfig): """ Configuration for persistent copies of a step's ordinary main-flow result. @@ -745,6 +737,11 @@ def port(self) -> TcpPort: """Port for streaming communication. Each streamer type has its own default.""" pass + + + + + @classmethod @abstractmethod def port_from_config(cls, config) -> Optional[int]: @@ -892,6 +889,7 @@ class CompilationDebugConfig(AnnotatedDataclassValidationMixin, Enableable): """ + # Inject all accumulated fields at the end of module loading. # Use the ObjectState owner that registered the pending field declarations. from objectstate.lazy_factory import _inject_all_pending_fields diff --git a/openhcs/core/dataset_sources/choice.py b/openhcs/core/dataset_sources/choice.py index 36af173540..fc772fa8ba 100644 --- a/openhcs/core/dataset_sources/choice.py +++ b/openhcs/core/dataset_sources/choice.py @@ -127,10 +127,7 @@ def source_type_for( filemanager: "FileManager", source_bindings: "SourceBindingsConfig | None", ) -> "type[DatasetSource]": - from openhcs.core.dataset_sources.source import ( - DatasetSource, - DeclaredFileSource, - ) + from openhcs.core.dataset_sources.source import DatasetSource, DeclaredFileSource detected = DatasetSource.detect_source_type(root, filemanager, source_bindings) if detected is not None: diff --git a/openhcs/core/dataset_sources/interfaces.py b/openhcs/core/dataset_sources/interfaces.py index 3d368d79c3..3c75d6a6a9 100644 --- a/openhcs/core/dataset_sources/interfaces.py +++ b/openhcs/core/dataset_sources/interfaces.py @@ -29,6 +29,7 @@ from polystore.virtual_workspace import SourcePixelRef from openhcs.core.axes import Axis, AxisFamily + if TYPE_CHECKING: from openhcs.core.source_projection import SourcePlaneDataset, SourceProjection from openhcs.core.source_workspace_projection import ( @@ -407,7 +408,9 @@ def parse_image_path(self, relative_path: str) -> FilenameParseResult | None: else parsed.with_values(self.image_path_components(path)) ) - def image_path_components(self, path: Path) -> tuple[tuple[type[Axis], int], ...]: + def image_path_components( + self, path: Path + ) -> tuple[tuple[type[Axis], int], ...]: """Identity terminus for cooperative acquisition-coordinate capabilities.""" return () @@ -574,9 +577,7 @@ def get_metadata_pixel_size(self, dataset_root: Union[str, Path]) -> float: """ return self.get_pixel_size(dataset_root) - def source_voxel_spacing( - self, dataset_root: Union[str, Path] - ) -> SourceVoxelSpacing: + def source_voxel_spacing(self, dataset_root: Union[str, Path]) -> SourceVoxelSpacing: """Project this acquisition's physical calibration into source coordinates.""" pixel_size = self.get_pixel_size(dataset_root) diff --git a/openhcs/core/dataset_sources/openhcs_format.py b/openhcs/core/dataset_sources/openhcs_format.py index 4dcafe8eed..6e564a493c 100644 --- a/openhcs/core/dataset_sources/openhcs_format.py +++ b/openhcs/core/dataset_sources/openhcs_format.py @@ -103,11 +103,7 @@ def invalidate_metadata_cache(self) -> None: self._metadata_dict_dataset_root_cache: Optional[Path] = None def _metadata_field( - self, - dataset_root: Union[str, Path], - field: str, - *, - merge: bool = False, + self, dataset_root: Union[str, Path], field: str, *, merge: bool = False, ) -> Any: """Resolve only the requested fact within the declared projection scope. @@ -116,16 +112,14 @@ def _metadata_field( silently choose one projection's geometry or execution input. """ subdirectories = self._metadata_subdirectories( - self.load_metadata_document(dataset_root), - dataset_root, + self.load_metadata_document(dataset_root), dataset_root, ) try: selected = self._main_subdirectory_name(subdirectories, dataset_root) except MetadataNotFoundError: values = {name: data.get(field) for name, data in subdirectories.items()} resolver = ( - self._merge_subdirectory_mapping - if merge + self._merge_subdirectory_mapping if merge else self._consistent_subdirectory_value ) return resolver(values, dataset_root, field) @@ -308,9 +302,7 @@ def get_pixel_size(self, dataset_root: Union[str, Path]) -> float: self._source_voxel_spacings(dataset_root) ) - def source_voxel_spacing( - self, dataset_root: Union[str, Path] - ) -> SourceVoxelSpacing: + def source_voxel_spacing(self, dataset_root: Union[str, Path]) -> SourceVoxelSpacing: """Preserve the stored source frame, including unknown/relative units.""" return SourceVoxelSpacing.common(self._source_voxel_spacings(dataset_root)) @@ -321,8 +313,7 @@ def _source_voxel_spacings( return tuple( SourceVoxelSpacing.from_source_metadata(source) for source in ( - self._metadata_field(dataset_root, FIELDS.SOURCE_METADATA, merge=True) - or {} + self._metadata_field(dataset_root, FIELDS.SOURCE_METADATA, merge=True) or {} ).values() ) @@ -337,9 +328,7 @@ def get_metadata_pixel_size(self, dataset_root: Union[str, Path]) -> float: def get_source_filename_parser_name(self, dataset_root: Union[str, Path]) -> str: """Get source filename parser name from OpenHCS metadata.""" - parser_name = self._metadata_field( - dataset_root, FIELDS.SOURCE_FILENAME_PARSER_NAME - ) + parser_name = self._metadata_field(dataset_root, FIELDS.SOURCE_FILENAME_PARSER_NAME) if not (isinstance(parser_name, str) and parser_name): raise ValueError( f"'{FIELDS.SOURCE_FILENAME_PARSER_NAME}' must be a non-empty string in {self.METADATA_FILENAME}" @@ -408,9 +397,7 @@ def reconciliation_directories( ) document = OpenHCSMetadataSubdirectories.from_path(metadata_path) accepted_entries = { - name: VirtualWorkspaceSourceProjectionEntries.from_subdirectory( - subdirectory - ) + name: VirtualWorkspaceSourceProjectionEntries.from_subdirectory(subdirectory) for name, subdirectory in document.items() } return self.reconciliation_directories_from_document( @@ -441,9 +428,7 @@ def reconciliation_directories_from_document( is not None ) subdirectories = self._metadata_subdirectories( - document.metadata, - dataset_root, - workspace_root=dataset_root, + document.metadata, dataset_root, workspace_root=dataset_root, ) source_projection = None if document.has_workspace_mapping(): @@ -644,13 +629,13 @@ def _metadata_subdirectories( root = ( self.source_workspace_root(dataset_root) - if workspace_root is None - else workspace_root + if workspace_root is None else workspace_root ).absolute() requested = Path(dataset_root).absolute() if requested != root: matches = tuple( - name for name in subdirectories if requested.is_relative_to(root / name) + name for name in subdirectories + if requested.is_relative_to(root / name) ) if not matches: raise MetadataNotFoundError( @@ -747,9 +732,7 @@ def get_plate_acquisition_datetime( self, dataset_root: Union[str, Path] ) -> Optional[str]: """Get plate acquisition datetime if available.""" - return self._get_optional_metadata_str( - dataset_root, FIELDS.ACQUISITION_DATETIME - ) + return self._get_optional_metadata_str(dataset_root, FIELDS.ACQUISITION_DATETIME) def get_plate_name(self, dataset_root: Union[str, Path]) -> Optional[str]: """Get plate name if available.""" @@ -778,14 +761,9 @@ def backend_availability(self, input_dir: Union[str, Path]) -> Dict[str, bool]: Raises: MetadataNotFoundError: If metadata file cannot be found or parsed """ - available_backends = ( - self._metadata_field( - input_dir, - FIELDS.AVAILABLE_BACKENDS, - merge=True, - ) - or {} - ) + available_backends = self._metadata_field( + input_dir, FIELDS.AVAILABLE_BACKENDS, merge=True, + ) or {} if not isinstance(available_backends, dict): logger.warning( @@ -882,9 +860,7 @@ class OpenHCSMetadata: def collection_fields() -> tuple[str, ...]: """Persisted value-label keys, one per declared axis.""" - return tuple( - axis.metadata_collection_field for axis in AxisFamily.active().axes - ) + return tuple(axis.metadata_collection_field for axis in AxisFamily.active().axes) @staticmethod def labels_by_field( @@ -936,10 +912,7 @@ def serialized_values( pixel_size=pixel_size, image_files=image_files, axis_value_labels=cls.labels_by_field( - { - component: serialized_values(component) - for component in AxisFamily.active().axes - } + {component: serialized_values(component) for component in AxisFamily.active().axes} ), available_backends=available_backends, source_diagnostics=source_diagnostics, @@ -963,10 +936,7 @@ def _openhcs_metadata_from_subdirectory( ) -> OpenHCSMetadata: missing_fields = tuple( field - for field in ( - *_OPENHCS_METADATA_REQUIRED_FIELDS, - *OpenHCSMetadata.collection_fields(), - ) + for field in (*_OPENHCS_METADATA_REQUIRED_FIELDS, *OpenHCSMetadata.collection_fields()) if field not in subdirectory_data ) if missing_fields: @@ -1337,23 +1307,18 @@ class OpenHCSDatasetSource(PreparedWorkspaceSource, DatasetSource): @classmethod def create( - cls, - *, - filemanager: FileManager, - pattern_format: Optional[str] = None, + cls, *, filemanager: FileManager, pattern_format: Optional[str] = None, source_bindings_config=None, ) -> "OpenHCSDatasetSource": """Keep the prepared source while applying the declared source bindings.""" from openhcs.core.source_bindings import source_bindings_defaults_to_base handler = super().create( - filemanager=filemanager, - pattern_format=pattern_format, + filemanager=filemanager, pattern_format=pattern_format, source_bindings_config=source_bindings_config, ) handler._source_bindings_config = ( - None - if source_bindings_config is None + None if source_bindings_config is None else source_bindings_defaults_to_base(source_bindings_config) ) return handler @@ -1362,6 +1327,7 @@ def source_bindings_still_required(self): """Expose the original prepared-workspace declaration to runtime readers.""" return self._source_bindings_config + @classmethod def source_selection_guidance(cls) -> str: """Explain when OpenHCS workspace metadata describes the dataset.""" @@ -1489,6 +1455,8 @@ def root_dir(self) -> str: # Return empty string as placeholder (not used for virtual workspace) return "" + + @property def compatible_backends(self) -> List[Backend]: """ @@ -1581,9 +1549,7 @@ def get_primary_backend( # 3. Fall back to first available backend (usually disk) return next(iter(available_backends_dict.keys())) - def initialize_workspace( - self, dataset_root: Path, filemanager: FileManager - ) -> Path: + def initialize_workspace(self, dataset_root: Path, filemanager: FileManager) -> Path: """ OpenHCS format doesn't need workspace - determines the correct input subdirectory from metadata. @@ -1604,9 +1570,7 @@ def initialize_workspace( # The caller's declared projection selects facts; storage stays root-relative. self.dataset_root = dataset_root if self._source_bindings_config is not None: - from openhcs.core.source_workspace_projection import ( - WorkspaceSourceProjections, - ) + from openhcs.core.source_workspace_projection import WorkspaceSourceProjections projection = WorkspaceSourceProjections.from_dataset_metadata( dataset_root=dataset_root, @@ -1614,13 +1578,8 @@ def initialize_workspace( filemanager=filemanager, source_bindings=self._source_bindings_config, ).projection_if_available() - if ( - projection is None - and self._source_bindings_config.source_filter_declarations - ): - raise ValueError( - "Prepared source filtering requires a typed workspace projection." - ) + if projection is None and self._source_bindings_config.source_filter_declarations: + raise ValueError("Prepared source filtering requires a typed workspace projection.") logger.debug("OpenHCSHandler: dataset_root set to %s", self.dataset_root) # Determine the main subdirectory from metadata - fail-loud on errors @@ -1654,9 +1613,7 @@ def _main_subdirectory_metadata( self, dataset_root: Path, ) -> Mapping[str, object]: - metadata = self.metadata_handler.source_workspace_metadata_document( - dataset_root - ) + metadata = self.metadata_handler.source_workspace_metadata_document(dataset_root) if not isinstance(metadata, Mapping): raise ValueError("OpenHCS source-workspace metadata must be a mapping.") subdirectories = metadata.get(FIELDS.SUBDIRECTORIES) @@ -1692,14 +1649,10 @@ def _register_declared_workspace_backends( @classmethod def register_workspace_backends( - cls, - dataset_root: Union[str, Path], - filemanager: FileManager, + cls, dataset_root: Union[str, Path], filemanager: FileManager, ) -> None: """Register storage at its metadata root, not the selected projection child.""" - workspace_root = OpenHCSMetadataHandler(filemanager).source_workspace_root( - dataset_root - ) + workspace_root = OpenHCSMetadataHandler(filemanager).source_workspace_root(dataset_root) super().register_workspace_backends(workspace_root, filemanager) def post_workspace( diff --git a/openhcs/core/dataset_sources/source.py b/openhcs/core/dataset_sources/source.py index 5e5be7665a..a988d67125 100644 --- a/openhcs/core/dataset_sources/source.py +++ b/openhcs/core/dataset_sources/source.py @@ -189,7 +189,8 @@ def source_selection_role(cls) -> type[SourceSelectionRole]: return next( base for base in cls.__mro__ - if isinstance(base, type) and SourceSelectionRole in base.__bases__ + if isinstance(base, type) + and SourceSelectionRole in base.__bases__ ) @classmethod @@ -311,9 +312,7 @@ def compatible_backends(self) -> List[Backend]: """Storage backends this source can read, in priority order.""" @abstractmethod - def initialize_workspace( - self, dataset_root: Path, filemanager: FileManager - ) -> Path: + def initialize_workspace(self, dataset_root: Path, filemanager: FileManager) -> Path: """Prepare the dataset for reading and return its image directory.""" def get_required_backend(self) -> Optional["MaterializationBackend"]: diff --git a/openhcs/core/dataset_sources/source_bindings_source.py b/openhcs/core/dataset_sources/source_bindings_source.py index c2639242b5..ec0d4ceb05 100644 --- a/openhcs/core/dataset_sources/source_bindings_source.py +++ b/openhcs/core/dataset_sources/source_bindings_source.py @@ -31,6 +31,7 @@ def projects_declared_source_bindings(cls) -> bool: return True + @classmethod def source_selection_guidance(cls) -> str: """Explain when declarations, rather than a vendor/store owner, ingest files.""" @@ -95,6 +96,8 @@ def __init__( def root_dir(self) -> str: return FIELDS.DEFAULT_SUBDIRECTORY + + @property def compatible_backends(self) -> list[Backend]: return [Backend.DISK] @@ -130,9 +133,7 @@ def _replay_persisted_workspace( VirtualWorkspaceSourceProjection, ) - metadata = self.metadata_handler.source_workspace_metadata_document( - dataset_root - ) + metadata = self.metadata_handler.source_workspace_metadata_document(dataset_root) workspace_metadata = self.metadata_handler.workspace_mapping_metadata( dataset_root ) diff --git a/openhcs/core/dataset_sources/source_schema.py b/openhcs/core/dataset_sources/source_schema.py index 0c31e7bb8c..5725c18708 100644 --- a/openhcs/core/dataset_sources/source_schema.py +++ b/openhcs/core/dataset_sources/source_schema.py @@ -45,3 +45,4 @@ def construct_filename(self, components: FilenameParseResult) -> str: for component, _value in components.declared_values() ) return address.filename(extension=components.extension) + diff --git a/openhcs/core/debug.py b/openhcs/core/debug.py index 7dc9b1844a..2bbaa8deee 100644 --- a/openhcs/core/debug.py +++ b/openhcs/core/debug.py @@ -275,7 +275,11 @@ def to_dict(self) -> dict[str, Any]: return { "debug_session_id": self.debug_session_id, "state": self.state.value, - "cursor": (None if self.cursor is None else to_jsonable(self.cursor)), + "cursor": ( + None + if self.cursor is None + else to_jsonable(self.cursor) + ), } @classmethod diff --git a/openhcs/core/equivalence/keys.py b/openhcs/core/equivalence/keys.py index b0ae80583c..e0e83f2ef5 100644 --- a/openhcs/core/equivalence/keys.py +++ b/openhcs/core/equivalence/keys.py @@ -674,7 +674,9 @@ def project_uncached(self) -> tuple[str, str | None]: return aggregate_feature parts_projection = parts_projection.strip_category_prefix_for_core() - direct_alias = self.dialect.feature_part_aliases().get(parts_projection.parts) + direct_alias = self.dialect.feature_part_aliases().get( + parts_projection.parts + ) if direct_alias is not None: return "_".join(direct_alias), None diff --git a/openhcs/core/equivalence/measurement_facts.py b/openhcs/core/equivalence/measurement_facts.py index c1ab76c874..e589998224 100644 --- a/openhcs/core/equivalence/measurement_facts.py +++ b/openhcs/core/equivalence/measurement_facts.py @@ -80,7 +80,9 @@ class RuntimeDirectionalPairMeasurementDerivationContract: @property def regression_slope_feature(self) -> str | None: """Return the dialect-declared pair regression-slope family.""" - return self.policy.measurement_dialect.pair_regression_slope_feature_name() + return ( + self.policy.measurement_dialect.pair_regression_slope_feature_name() + ) @property def correlation_feature(self) -> str | None: @@ -367,7 +369,8 @@ def dedupe_observed_alias_records( ) -> RuntimeMeasurementFacts: """Filter unobserved padding groups and collapse same-row aliases.""" return cls.dedupe_alias_facts( - (record.key, record.value) for record in cls.observed_records(records) + (record.key, record.value) + for record in cls.observed_records(records) ) @classmethod diff --git a/openhcs/core/equivalence/measurement_features.py b/openhcs/core/equivalence/measurement_features.py index edb4c1b79e..1ab162b52e 100644 --- a/openhcs/core/equivalence/measurement_features.py +++ b/openhcs/core/equivalence/measurement_features.py @@ -91,7 +91,9 @@ def _for_feature_key_payload( key = RuntimeMeasurementFeatureKey.from_cache_payload(key_payload) context = RuntimeMeasurementFeatureSemanticContext( key, - RuntimeEquivalencePolicy(measurement_dialect=dialect), + RuntimeEquivalencePolicy( + measurement_dialect=dialect + ), ) strategy = cls.for_context( context, @@ -487,7 +489,9 @@ def _object_measurement_feature_matches_marker_cached( dialect: MeasurementDialect, ) -> bool: key = RuntimeMeasurementFeatureKey.from_cache_payload(key_payload) - policy = RuntimeEquivalencePolicy(measurement_dialect=dialect) + policy = RuntimeEquivalencePolicy( + measurement_dialect=dialect + ) provider_marker_types = policy.measurement_dialect.measurement_feature_marker_types( key ) diff --git a/openhcs/core/equivalence/measurement_rows.py b/openhcs/core/equivalence/measurement_rows.py index 8f5a8b023c..73d0d9dfb5 100644 --- a/openhcs/core/equivalence/measurement_rows.py +++ b/openhcs/core/equivalence/measurement_rows.py @@ -72,7 +72,6 @@ ObjectInstanceKey, ) - @lru_cache(maxsize=64) def measurement_identity_fields( contract: RuntimeMeasurementRowIdentityContract, @@ -163,9 +162,7 @@ def measurement_row_sample_identity_key( for field_name, value in row.items() if value is not None and str(value).strip() ) - selected_fields = contract.selected_sample_identity_fields( - normalized_present_fields - ) + selected_fields = contract.selected_sample_identity_fields(normalized_present_fields) identity_values: list[tuple[str, object]] = [] for field_name, value in row.items(): normalized_field_name = normalize_runtime_identifier(field_name) diff --git a/openhcs/core/execution_state.py b/openhcs/core/execution_state.py index 783891af3b..e5824c9b63 100644 --- a/openhcs/core/execution_state.py +++ b/openhcs/core/execution_state.py @@ -31,7 +31,9 @@ def __post_init__(self) -> None: self.auto_add_output_dataset, bool, ): - raise TypeError("auto_add_output_dataset must be a bool or None") + raise TypeError( + "auto_add_output_dataset must be a bool or None" + ) @classmethod def from_results_summary( diff --git a/openhcs/core/function_patterns.py b/openhcs/core/function_patterns.py index 6529849019..373464139e 100644 --- a/openhcs/core/function_patterns.py +++ b/openhcs/core/function_patterns.py @@ -255,10 +255,7 @@ def source_projection( ) -> MainFlowInputProjection | None: """Derive the primary source role independently of its pixel producer.""" main_flow_refs = main_flow_artifacts.ref_set() - if ( - spec.ref() not in main_flow_refs - or spec.ref() not in invocation_sources.ref_set() - ): + if spec.ref() not in main_flow_refs or spec.ref() not in invocation_sources.ref_set(): return None return ( MainFlowInputProjection.COMPLETE_PAYLOAD @@ -280,15 +277,11 @@ def from_source_declarations( """Project one unstored source occurrence from its exact declarations.""" main_flow_projection = cls.source_projection( - spec, - main_flow_artifacts, - invocation_sources, + spec, main_flow_artifacts, invocation_sources, ) edge_type = ( CompiledMetadataArtifactInputEdgePlan - if metadata_available - and main_flow_projection is None - and not source_binding_available + if metadata_available and main_flow_projection is None and not source_binding_available else cls ) return edge_type( @@ -304,8 +297,7 @@ def uses_runtime_storage(self) -> bool: return self.storage_plan is not None def requires_current_image_carrier( - self, - source_bindings: CompiledSourceBindingPlan, + self, source_bindings: CompiledSourceBindingPlan, ) -> bool: """Whether this unstored input consumes declared current source pixels.""" if self.uses_runtime_storage(): @@ -329,16 +321,13 @@ def resolve_unstored_payload( """Admit pre-call input pixels through their exact compiled origin.""" request = ( scope.runtime_adapter_request(primary_source_payload) - if request is None - else request + if request is None else request ) loaded_payload = scope.loaded_source_artifact_payload(request, self) if loaded_payload is not None: return loaded_payload return scope.declared_source_payload( - self.spec.ref(), - primary_source_payload, - loaded_artifact_payloads={}, + self.spec.ref(), primary_source_payload, loaded_artifact_payloads={}, request=request, ) @@ -405,13 +394,9 @@ def artifact_parameter_value(self, spec: ArtifactSpec) -> object: for parameter_name, value in self.kwargs: if parameter_name == spec.parameter_name: if value is None and spec.required: - raise ValueError( - f"Required metadata artifact {spec.ref()!r} has no value." - ) + raise ValueError(f"Required metadata artifact {spec.ref()!r} has no value.") return value - raise ValueError( - f"Metadata artifact {spec.ref()!r} has no compiled parameter value." - ) + raise ValueError(f"Metadata artifact {spec.ref()!r} has no compiled parameter value.") @dataclass(frozen=True, slots=True) @@ -706,18 +691,13 @@ def convert_input(self, payload: object, source_memory_type: str) -> object: source = MemoryType(source_memory_type) if source is self._input_memory_type and not source.is_gpu: mask = payload.mask - if ( - project_image_mask_to_data_domain( - mask, payload, metadata=payload.metadata - ) - is mask - ): + if project_image_mask_to_data_domain( + mask, payload, metadata=payload.metadata + ) is mask: return payload - return payload.with_pixels( - source.convert_to( + return payload.with_pixels(source.convert_to( payload.data, self._input_memory_type, self.input_device_id - ), - ) + ),) def main_flow_call_argument(self, source_payload: object) -> object: """Apply the captured processing declaration to current request pixels.""" @@ -873,9 +853,7 @@ def select_outputs( else compiled_output_plans ) if any(not isinstance(plan, ArtifactOutputPlan) for plan in active_outputs): - raise TypeError( - "Compiled artifact outputs must be ArtifactOutputPlan values." - ) + raise TypeError("Compiled artifact outputs must be ArtifactOutputPlan values.") selected: dict[ArtifactSpecRef, ArtifactOutputPlan] = {} for compiled_plan in active_outputs: runtime_plan = output_plans.get(compiled_plan.ref()) @@ -955,8 +933,7 @@ def from_normalized_item( artifact_graph: "ArtifactGraph | None" = None, ) -> "CompiledFunctionInvocation": contract_plan = ( - invocation_contract_provider(item, step_context) - if artifact_graph is None + invocation_contract_provider(item, step_context) if artifact_graph is None else artifact_graph.invocation_contract_plans[item.key] ) if contract_plan is None: @@ -968,8 +945,7 @@ def from_normalized_item( ) item = replace(item, contract=contract_plan.contract) artifact_selector = ( - declaration_provider(item, step_context) - if artifact_graph is None + declaration_provider(item, step_context) if artifact_graph is None else artifact_graph.invocation_declarations[item.key] ) if artifact_graph is None: @@ -1007,9 +983,7 @@ def from_normalized_item( f"paths {relative_parameters!r} but compilation supplied no " "CompilationPathResolver." ) - runtime_loaded_input_refs = frozenset( - plan.ref() for plan in artifact_input_plans - ) + runtime_loaded_input_refs = frozenset(plan.ref() for plan in artifact_input_plans) validated_kwargs = item.contract.validate_public_kwargs( public_kwargs, runtime_loaded_artifact_parameter_names=( @@ -1044,9 +1018,7 @@ def runtime_domain(self) -> RuntimeInvocationDomain: return RuntimeInvocationDomain.from_invocations(self.invocations) def main_flow_input_refs( - self, - *, - source_bindings: CompiledSourceBindingPlan, + self, *, source_bindings: CompiledSourceBindingPlan, ) -> tuple[ArtifactSpecRef, ...] | None: """Return exact main-flow refs, or None for an implicit image argument.""" @@ -1054,16 +1026,14 @@ def main_flow_input_refs( tuple( (invocation, invocation.artifact_input_edges) for invocation in self.invocations - ), - source_bindings=source_bindings, + ), source_bindings=source_bindings, ) def main_flow_input_refs_for_component( self, execution_scope: ComponentGroupScope, component_key: str | None, - *, - source_bindings: CompiledSourceBindingPlan, + *, source_bindings: CompiledSourceBindingPlan, ) -> tuple[ArtifactSpecRef, ...] | None: """Return main-flow refs active for one compiled component execution.""" @@ -1079,8 +1049,7 @@ def main_flow_input_refs_for_component( is not None ) return self._main_flow_input_refs( - invocations_and_edges, - source_bindings=source_bindings, + invocations_and_edges, source_bindings=source_bindings, ) @staticmethod @@ -1091,8 +1060,7 @@ def _main_flow_input_refs( Sequence[InvocationArtifactInputEdgePlan], ] ], - *, - source_bindings: CompiledSourceBindingPlan, + *, source_bindings: CompiledSourceBindingPlan, ) -> tuple[ArtifactSpecRef, ...] | None: """Return exact main-flow refs for the supplied active invocations.""" @@ -1131,11 +1099,9 @@ def stored_primary_input_edges_for_component( for invocation in self.invocations if ( active_outputs := invocation.output_plans_for_component( - execution_scope, - component_key, + execution_scope, component_key, ) - ) - is not None + ) is not None ) if not active or any( invocation.contract.accepts_implicit_main_flow_input @@ -1164,11 +1130,10 @@ def stored_primary_input_edges_for_component( primary_edges = tuple( edge for invocation, edges in active - for domain_refs in ( - invocation.contract.invocation_domain_inputs.ref_set(), - ) + for domain_refs in (invocation.contract.invocation_domain_inputs.ref_set(),) for edge in edges - if edge.spec.ref() in domain_refs + if edge.spec.ref() + in domain_refs and edge.spec.artifact_type.carries_source_image_context ) if not primary_edges or any( diff --git a/openhcs/core/function_reference.py b/openhcs/core/function_reference.py index 49d01179f4..6fb96799c2 100644 --- a/openhcs/core/function_reference.py +++ b/openhcs/core/function_reference.py @@ -79,13 +79,11 @@ def source_literal(self) -> str: return self.source_expression(self.source_import_identity.function_name) def source_literal_with_names( - self, - name_mappings: Mapping[tuple[str, str], str], + self, name_mappings: Mapping[tuple[str, str], str], ) -> str: identity = self.source_import_identity imported_name = name_mappings.get( - (identity.module_name, identity.function_name), - identity.function_name, + (identity.module_name, identity.function_name), identity.function_name, ) return self.source_expression(imported_name) @@ -178,8 +176,7 @@ def resolve(self) -> Callable: @dataclass(frozen=True, kw_only=True) class ModuleExportRegistryFunctionReference( - ModuleExportFunctionSource, - RegistryFunctionReference, + ModuleExportFunctionSource, RegistryFunctionReference, ): """Direct source import with registry-owned transport and wrapper identity.""" @@ -415,8 +412,7 @@ def _registry_reference( ) identity = function_metadata.import_identity exported = FunctionReferenceTransportAuthority.importable_function( - identity.module_name, - identity.function_name, + identity.module_name, identity.function_name, ) reference_type = ( ModuleExportRegistryFunctionReference diff --git a/openhcs/core/image_file_serialization.py b/openhcs/core/image_file_serialization.py index 41074a2c27..17a897c95b 100644 --- a/openhcs/core/image_file_serialization.py +++ b/openhcs/core/image_file_serialization.py @@ -158,9 +158,7 @@ def require_image_geometry(self) -> tuple[Any, int, int]: """Require complete format-declared dtype and physical image dimensions.""" if self.image_shape_yx is None or self.source_dtype is None: - raise ValueError( - "Image-file header does not declare image dtype and YX geometry." - ) + raise ValueError("Image-file header does not declare image dtype and YX geometry.") height, width = self.image_shape_yx return self.source_dtype, height, width @@ -417,7 +415,9 @@ def persisted_metadata(self, path: Path, payload: Any) -> ImagePayloadMetadata: payload = ImagePayload.of(payload) return header.project_image_metadata( payload.metadata, - values_preserved=self.preserves_pixel_values(payload.data.dtype), + values_preserved=self.preserves_pixel_values( + payload.data.dtype + ), ) def requires_plane_store_decoder(self, path: Path) -> bool: diff --git a/openhcs/core/image_payload_execution_mode.py b/openhcs/core/image_payload_execution_mode.py index 62666f1ac5..859666e5ff 100644 --- a/openhcs/core/image_payload_execution_mode.py +++ b/openhcs/core/image_payload_execution_mode.py @@ -43,8 +43,7 @@ def execute( @classmethod def batch_request( - cls, - request: "RuntimeBatchInvocationRequest", + cls, request: "RuntimeBatchInvocationRequest", ) -> "RuntimeBatchInvocationRequest | None": """Project a request into a batch executor's image domain, or ``None``.""" return request @@ -58,8 +57,7 @@ def executable_payload(cls, payload: Any) -> Any: @classmethod def for_payload_scoped_labels( - cls, - runtime_slice_count: int | None, + cls, runtime_slice_count: int | None, ) -> type[ImagePayloadExecutionMode]: """Return the mode for labels whose domain is the whole payload.""" del runtime_slice_count @@ -67,8 +65,7 @@ def for_payload_scoped_labels( @classmethod def for_plane_scoped_labels( - cls, - projection: "RuntimePlaneAxisValueProjection | None", + cls, projection: "RuntimePlaneAxisValueProjection | None", ) -> type[ImagePayloadExecutionMode]: """Return the mode for labels that still carry an unprojected plane axis.""" del projection @@ -240,11 +237,7 @@ def execute(cls, contract, call, image, kwargs): ) def execute_aligned_stack_slice( - slice_func, - slice_payload, - slice_kwargs, - slice_index, - slice_count, + slice_func, slice_payload, slice_kwargs, slice_index, slice_count, ): return call.invoke( slice_payload, diff --git a/openhcs/core/measurement_dialect.py b/openhcs/core/measurement_dialect.py index 77b65c45c4..3ec0b427cf 100644 --- a/openhcs/core/measurement_dialect.py +++ b/openhcs/core/measurement_dialect.py @@ -71,6 +71,8 @@ def __post_init__(self) -> None: ) + + class RuntimeMeasurementQualifierValueMode(str, Enum): """How a row qualifier value is rendered into a feature suffix.""" @@ -79,6 +81,8 @@ class RuntimeMeasurementQualifierValueMode(str, Enum): FRACTION_OF_COUNT = "fraction_of_count" + + @dataclass(frozen=True, slots=True) class RuntimeMeasurementRowQualifier: """Declarative row fields that qualify measurement feature names.""" @@ -256,6 +260,7 @@ def __post_init__(self) -> None: ) + def _parts(prefix: Iterable[str]) -> FeatureParts: return tuple(part for part in prefix if part) @@ -309,9 +314,7 @@ class MeasurementDialect(ABC, metaclass=AutoRegisterMeta): source_qualifier_suffix_tokens: ClassVar[frozenset[str]] = frozenset() _declaration_generation: ClassVar[int] = 0 - _shared_instances: ClassVar[ - dict[type["MeasurementDialect"], "MeasurementDialect"] - ] = {} + _shared_instances: ClassVar[dict[type["MeasurementDialect"], "MeasurementDialect"]] = {} def __init__(self) -> None: self._cached_values: dict[str, object] = {} @@ -354,9 +357,7 @@ def registered_dialects(cls) -> tuple["MeasurementDialect", ...]: @classmethod def unqualified_sample_names(cls) -> frozenset[str]: """Source names that mean "the sample itself" in any registered dialect.""" - return _unqualified_sample_names( - tuple(MeasurementDialect.__registry__.values()) - ) + return _unqualified_sample_names(tuple(MeasurementDialect.__registry__.values())) @classmethod def for_active_family(cls) -> "MeasurementDialect": @@ -450,9 +451,7 @@ def measurement_feature_marker_declarations( del key return () - def indexed_descriptor_suffix_width( - self, feature_parts: FeatureParts - ) -> int | None: + def indexed_descriptor_suffix_width(self, feature_parts: FeatureParts) -> int | None: """Trailing descriptor-index token width of one feature, if indexed.""" del feature_parts return None @@ -707,21 +706,14 @@ def feature_parts(self, parts: FeatureParts) -> FeatureParts: """Return dialect-normalized feature parts for one lookup token.""" resolved_parts = parts for prefix in self.category_prefixes(): - if ( - len(resolved_parts) > len(prefix) - and resolved_parts[: len(prefix)] == prefix - ): + if len(resolved_parts) > len(prefix) and resolved_parts[: len(prefix)] == prefix: resolved_parts = resolved_parts[len(prefix) :] break return self.feature_part_aliases().get(resolved_parts, resolved_parts) - def alternative_feature_parts( - self, parts: FeatureParts - ) -> tuple[FeatureParts, ...]: + def alternative_feature_parts(self, parts: FeatureParts) -> tuple[FeatureParts, ...]: """Return ordered semantic fallback fields for one lookup token.""" - return self.alternative_feature_part_aliases().get( - self.feature_parts(parts), () - ) + return self.alternative_feature_part_aliases().get(self.feature_parts(parts), ()) def feature_lookup(self, feature_name: str) -> "RuntimeMeasurementFeatureLookup": """Return the lookup identity for one external feature name.""" @@ -936,7 +928,8 @@ def qualifier_sequence_identifies_feature_suffix( return True base_tokens = feature_tokens[: len(feature_tokens) - len(suffix_tokens)] return any( - base_tokens == prefix for prefix in self.scale_qualified_feature_prefixes() + base_tokens == prefix + for prefix in self.scale_qualified_feature_prefixes() ) def feature_suffix_source_insertion_index( @@ -999,6 +992,7 @@ def source_qualified_feature_family( return None + @lru_cache(maxsize=8) def _unqualified_sample_names( dialect_types: tuple[type[MeasurementDialect], ...], diff --git a/openhcs/core/measurement_feature_queries.py b/openhcs/core/measurement_feature_queries.py index 9b672deefc..00d1d7225d 100644 --- a/openhcs/core/measurement_feature_queries.py +++ b/openhcs/core/measurement_feature_queries.py @@ -635,8 +635,8 @@ def _project_feature_indexes( row_indices[effective_mask], object_values, strict=True ): if row_index not in object_labels_by_row: - object_labels_by_row[row_index] = ( - measurement_object_label_value(object_ids[row_index]) + object_labels_by_row[row_index] = measurement_object_label_value( + object_ids[row_index] ) object_label = object_labels_by_row[row_index] if object_label is not None: @@ -693,7 +693,9 @@ class MeasurementFeatureQuery: feature_name: str object_name: str | None = None - dialect: MeasurementDialectLike = EXECUTING_MEASUREMENT_DIALECT + dialect: MeasurementDialectLike = ( + EXECUTING_MEASUREMENT_DIALECT + ) def __post_init__(self) -> None: if not self.feature_name: @@ -905,7 +907,9 @@ class MeasurementObjectFeatureVectorBatchQuery: feature_name: str object_names: tuple[str, ...] - dialect: MeasurementDialectLike = EXECUTING_MEASUREMENT_DIALECT + dialect: MeasurementDialectLike = ( + EXECUTING_MEASUREMENT_DIALECT + ) def __post_init__(self) -> None: if not self.feature_name: @@ -928,9 +932,9 @@ def value_indexes( ) -> MeasurementValueIndexesByObject: """Return current object-feature indexes from one correlated table pass.""" object_names = self.normalized_object_names - lookup = resolve_measurement_dialect(self.dialect).feature_lookup( - self.feature_name - ) + lookup = resolve_measurement_dialect( + self.dialect + ).feature_lookup(self.feature_name) query_objects_by_requested_object = { object_name: lookup.query_object_name(object_name) for object_name in object_names @@ -986,9 +990,9 @@ def value_indexes_by_axis( ) -> dict[int, MeasurementValueIndexesByObject] | None: """Return current feature indexes preserving complete object/axis rows.""" object_names = self.normalized_object_names - lookup = resolve_measurement_dialect(self.dialect).feature_lookup( - self.feature_name - ) + lookup = resolve_measurement_dialect( + self.dialect + ).feature_lookup(self.feature_name) query_objects_by_requested_object = { object_name: lookup.query_object_name(object_name) for object_name in object_names @@ -1087,9 +1091,9 @@ def table_object_names( semantics = ColumnarMeasurementTableSchema.from_table(table) if not semantics.object_names(table): return requested_object_names - feature_lookup = resolve_measurement_dialect(self.dialect).feature_lookup( - self.feature_name - ) + feature_lookup = resolve_measurement_dialect( + self.dialect + ).feature_lookup(self.feature_name) return tuple( object_name for object_name in requested_object_names @@ -1384,11 +1388,15 @@ def normalize_measurement_token(value: object) -> str: def ordered_measurement_feature_candidates( feature_name: str, *, - dialect: MeasurementDialectLike = (EXECUTING_MEASUREMENT_DIALECT), + dialect: MeasurementDialectLike = ( + EXECUTING_MEASUREMENT_DIALECT + ), ) -> tuple[str, ...]: """Return schema-safe feature-field aliases from most specific to least specific.""" return ( - resolve_measurement_dialect(dialect).feature_lookup(feature_name).field_aliases + resolve_measurement_dialect(dialect) + .feature_lookup(feature_name) + .field_aliases ) @@ -1449,7 +1457,9 @@ def measurement_values_for_feature( object_count: int, object_ids: Sequence[int] | None = None, object_name: str | None = None, - dialect: MeasurementDialectLike = (EXECUTING_MEASUREMENT_DIALECT), + dialect: MeasurementDialectLike = ( + EXECUTING_MEASUREMENT_DIALECT + ), ) -> Any: """Return object-indexed measurement values for one feature.""" query = MeasurementFeatureQuery( diff --git a/openhcs/core/measurement_image_alignment.py b/openhcs/core/measurement_image_alignment.py index 5ea13be7ff..1528fc95ea 100644 --- a/openhcs/core/measurement_image_alignment.py +++ b/openhcs/core/measurement_image_alignment.py @@ -403,12 +403,10 @@ def object_label_reference_image(self) -> RuntimeArrayData | AlignedImageStack: "Object-label reference alignment requires an image source domain " "adapter." ) - return request.image.with_pixels( - label_domain_adapter.extract_source_array( + return request.image.with_pixels(label_domain_adapter.extract_source_array( request.image_array, spatial_axes_yx=image_domain_adapter.spatial_axes_yx, - ), - ) + ),) @classmethod def validate_alignment( diff --git a/openhcs/core/measurement_row_materialization.py b/openhcs/core/measurement_row_materialization.py index 6b9a900e7a..8e1ba7b116 100644 --- a/openhcs/core/measurement_row_materialization.py +++ b/openhcs/core/measurement_row_materialization.py @@ -2419,6 +2419,7 @@ def metadata_for_row(row_index: int): if not all(is_structural_missing_measurement_cell(value) for value in values): coordinate_columns[component.name] = values + def source_names_for_row(row_index: int) -> tuple[str, ...]: if slice_values is None: return source_provenance.represented_source_image_names diff --git a/openhcs/core/memory/decorators.py b/openhcs/core/memory/decorators.py index f4141be461..e72b96f42d 100644 --- a/openhcs/core/memory/decorators.py +++ b/openhcs/core/memory/decorators.py @@ -97,9 +97,7 @@ def _admits_plain_image_payload(annotation: Any) -> bool: annotation = resolve_annotated(annotation) if is_union_type(annotation): - return any( - _admits_plain_image_payload(member) for member in get_args(annotation) - ) + return any(_admits_plain_image_payload(member) for member in get_args(annotation)) return annotation is ImagePayload @@ -162,9 +160,7 @@ def image_payload_callable(*args: Any, **kwargs: Any) -> Any: result = wrapped(*args, **kwargs) # Bare pixels in, bare pixels out: a caller that passed an array (an # array library, slice-by-slice stacking) gets arrays back. - if primary in positions.values() and not isinstance( - primary_value, ImagePayload - ): + if primary in positions.values() and not isinstance(primary_value, ImagePayload): return as_bare(result) return result @@ -235,9 +231,7 @@ def _signature_with_resolved_raw_annotations(wrapped: Any) -> inspect.Signature: def _strip_unowned_semantic_controls(wrapped: Any, contract: Any) -> None: - allowed_semantic_control_names = ( - contract.injected_semantic_control_parameter_names() - ) + allowed_semantic_control_names = contract.injected_semantic_control_parameter_names() semantic_control_names = { parameter_type.require_parameter_name() for parameter_type in contract.semantic_control_parameter_types() diff --git a/openhcs/core/metadata_cache.py b/openhcs/core/metadata_cache.py index 2e77372c5d..c5058551c9 100644 --- a/openhcs/core/metadata_cache.py +++ b/openhcs/core/metadata_cache.py @@ -106,3 +106,4 @@ def _is_cache_valid(self) -> bool: ): return False return True + diff --git a/openhcs/core/orchestrator/bundle_execution.py b/openhcs/core/orchestrator/bundle_execution.py index 99f041da39..cedcef5927 100644 --- a/openhcs/core/orchestrator/bundle_execution.py +++ b/openhcs/core/orchestrator/bundle_execution.py @@ -1055,7 +1055,6 @@ def wait_until_visualizers_ready( def not_ready_ports() -> list[int]: running_ports = {v.port for v in visualizers if v.is_running} return [v.port for v in visualizers if v.port not in running_ports] - start_time = time.time() while time.time() - start_time < max_wait: diff --git a/openhcs/core/orchestrator/execution_result.py b/openhcs/core/orchestrator/execution_result.py index 39459d46a7..157ca6db78 100644 --- a/openhcs/core/orchestrator/execution_result.py +++ b/openhcs/core/orchestrator/execution_result.py @@ -60,9 +60,7 @@ class RuntimeContextObservation: context_key: str records: tuple[StoredRuntimeValue, ...] - outputs: StepExecutionObservation = field( - default_factory=StepExecutionObservation.empty - ) + outputs: StepExecutionObservation = field(default_factory=StepExecutionObservation.empty) @classmethod def from_context( @@ -93,17 +91,15 @@ def from_completed_outputs( cls, contexts: Mapping[str, ProcessingContext] ) -> "RuntimeExecutionObservation": """Project saved facts independently of retained runtime pixel records.""" - return cls( - contexts=tuple( - RuntimeContextObservation( - context_key=context_key, - records=(), - outputs=context.completed_step_outputs, - ) - for context_key, context in contexts.items() - if not context.completed_step_outputs.is_empty + return cls(contexts=tuple( + RuntimeContextObservation( + context_key=context_key, + records=(), + outputs=context.completed_step_outputs, ) - ) + for context_key, context in contexts.items() + if not context.completed_step_outputs.is_empty + )) def merge_into(self, execution_contexts: Mapping[str, ProcessingContext]) -> None: """Merge returned runtime records into parent-owned compiled contexts.""" @@ -253,8 +249,7 @@ def error( error_message=error_message, runtime_observation=( RuntimeExecutionObservation() - if runtime_observation is None - else runtime_observation + if runtime_observation is None else runtime_observation ), ) @@ -273,7 +268,6 @@ def cancelled( error_message=error_message, runtime_observation=( RuntimeExecutionObservation() - if runtime_observation is None - else runtime_observation + if runtime_observation is None else runtime_observation ), ) diff --git a/openhcs/core/orchestrator/orchestrator.py b/openhcs/core/orchestrator/orchestrator.py index ac847cfbbe..c46b482582 100644 --- a/openhcs/core/orchestrator/orchestrator.py +++ b/openhcs/core/orchestrator/orchestrator.py @@ -575,9 +575,7 @@ def _ensure_openhcs_metadata(self, resolved_config: GlobalPipelineConfig) -> Non source_role = self.microscope_handler.source_selection_role() if not source_role.requires_local_directory: - logger.debug( - "Skipping local metadata creation for %s", source_role.role_name - ) + logger.debug("Skipping local metadata creation for %s", source_role.role_name) return # For datasets with a virtual workspace, metadata is already created by _build_virtual_mapping() @@ -668,7 +666,9 @@ def create_context( # Extract cached metadata from service and convert to dict format expected by OpenHCSMetadataGenerator metadata_dict = {} for component in AxisFamily.active().axes: - cached_metadata = self.metadata_cache.get_cached_metadata(component) + cached_metadata = self.metadata_cache.get_cached_metadata( + component + ) if cached_metadata: metadata_dict[component] = cached_metadata context.metadata_cache = metadata_dict @@ -903,7 +903,9 @@ def cache_component_keys( if components is None: components = list(AxisFamily.active().axes) - logger.info(f"Caching component keys for: {[comp.name for comp in components]}") + logger.info( + f"Caching component keys for: {[comp.name for comp in components]}" + ) try: axis_values = self.microscope_handler.axis_values( diff --git a/openhcs/core/orchestrator/worker_execution.py b/openhcs/core/orchestrator/worker_execution.py index b30f98c40e..0528dbdb70 100644 --- a/openhcs/core/orchestrator/worker_execution.py +++ b/openhcs/core/orchestrator/worker_execution.py @@ -771,9 +771,7 @@ class WorkerProcess: def send(self, command: WorkerCommand) -> None: self.connection.send(command) - def receive( - self, progress_queue: ProgressQueue | None = None - ) -> WorkerResult | None: + def receive(self, progress_queue: ProgressQueue | None = None) -> WorkerResult | None: """Read one reply; return the result it completes, if any.""" try: @@ -982,9 +980,7 @@ def release(self, workers: list[WorkerProcess], *, retire: bool) -> None: worker.process.join() worker.connection.close() self.workers = [ - owned - for owned in self.workers - if owned.process is not worker.process + owned for owned in self.workers if owned.process is not worker.process ] def close(self) -> None: @@ -1635,13 +1631,12 @@ def execute_worker_lane( ) -> Dict[str, ExecutionResult]: """Execute a deterministic worker lane: its partitions in order, in one slot.""" - with ( - CProfileWorkerProfilingPolicy.from_environment().profile(lane_context), - RuntimeProfileLogger.run( - execution_id=lane_context.execution_id, - worker_slot=lane_context.worker_slot, - owned_partition_values=tuple(lane_context.owned_partition_values), - ), + with CProfileWorkerProfilingPolicy.from_environment().profile( + lane_context + ), RuntimeProfileLogger.run( + execution_id=lane_context.execution_id, + worker_slot=lane_context.worker_slot, + owned_partition_values=tuple(lane_context.owned_partition_values), ): lane_results: Dict[str, ExecutionResult] = {} for axis_id, axis_contexts in lane_axis_contexts: diff --git a/openhcs/core/orchestrator/worker_profiling.py b/openhcs/core/orchestrator/worker_profiling.py index 9f37b01769..e9c7b4dc65 100644 --- a/openhcs/core/orchestrator/worker_profiling.py +++ b/openhcs/core/orchestrator/worker_profiling.py @@ -72,7 +72,12 @@ def profile( yield finally: profiler.disable() - profiler.dump_stats(str(self.output_dir / self.profile_filename(lane))) + profiler.dump_stats( + str( + self.output_dir + / self.profile_filename(lane) + ) + ) @abstractmethod def configure_profile_event_scope(self) -> None: diff --git a/openhcs/core/partition_filter.py b/openhcs/core/partition_filter.py index a535f7247e..1512e69b17 100644 --- a/openhcs/core/partition_filter.py +++ b/openhcs/core/partition_filter.py @@ -66,9 +66,7 @@ def from_value(cls, value: list) -> "PartitionValueList": def matches(self, available_values: Sequence[str], *, strict: bool) -> Set[str]: available_keys = {_identity_key(value) for value in available_values} - unknown = [ - value for value in self.values if _identity_key(value) not in available_keys - ] + unknown = [value for value in self.values if _identity_key(value) not in available_keys] if unknown and strict: raise ValueError( f"Invalid partition values specified: {unknown}. " @@ -77,9 +75,7 @@ def matches(self, available_values: Sequence[str], *, strict: bool) -> Set[str]: if unknown: logger.warning(f"Ignoring unknown partition values in filter: {unknown}") requested = {_identity_key(value) for value in self.values} - return { - value for value in available_values if _identity_key(value) in requested - } + return {value for value in available_values if _identity_key(value) in requested} @dataclass(frozen=True) @@ -140,9 +136,7 @@ def matches(self, available_values: Sequence[str], *, strict: bool) -> Set[str]: if pattern.startswith(self.ROW_PREFIX): return self._row(pattern[len(self.ROW_PREFIX) :].strip(), available_values) if pattern.startswith(self.COL_PREFIX): - return self._column( - pattern[len(self.COL_PREFIX) :].strip(), available_values - ) + return self._column(pattern[len(self.COL_PREFIX) :].strip(), available_values) if self.RANGE_SEPARATOR in pattern: return self._range(pattern, available_values) return {pattern} @@ -154,9 +148,7 @@ def _row(row: str, available_values: Sequence[str]) -> Set[str]: # Opera Phenix format fallback (A → R01C*, B → R02C*) if not result and len(row) == 1 and row.isalpha(): row_pattern = f"R{ord(row.upper()) - ord('A') + 1:02d}C" - result = { - value for value in available_values if value.startswith(row_pattern) - } + result = {value for value in available_values if value.startswith(row_pattern)} return result @staticmethod @@ -172,9 +164,7 @@ def numeric_suffix(value: str) -> int: digits = "".join(char for char in reversed(value) if char.isdigit()) return int(digits[::-1]) if digits else 0 - result = { - value for value in available_values if numeric_suffix(value) in col_range - } + result = {value for value in available_values if numeric_suffix(value) in col_range} # Opera Phenix format fallback (C01, C02, etc.) if not result: patterns = {f"C{col:02d}" for col in col_range} @@ -223,8 +213,7 @@ def resolve_filter_with_mode( result = [ value for value in available_values - if (_identity_key(value) in matched_keys) - is partition_filter_mode.keeps_matches + if (_identity_key(value) in matched_keys) is partition_filter_mode.keeps_matches ] logger.debug( "Partition filter %r (%s) over %s selects %s", diff --git a/openhcs/core/pipeline/compilation_session.py b/openhcs/core/pipeline/compilation_session.py index 01d5840b6b..d3f7296d9b 100644 --- a/openhcs/core/pipeline/compilation_session.py +++ b/openhcs/core/pipeline/compilation_session.py @@ -250,9 +250,9 @@ def _admit_artifact_graphs(self) -> None: step, index, execution_scope=FunctionStepExecutionScope.require_uniform( - () - if graph.pattern is None - else (item.contract for item in graph.pattern.iter_items()) + () if graph.pattern is None else ( + item.contract for item in graph.pattern.iter_items() + ) ), source_bindings=step.source_bindings, context=context, @@ -339,12 +339,9 @@ def validate_source_group_domains( or context.group_by is Ungrouped ): continue - if ( - FunctionStepExecutionScope.require_uniform( - item.contract for item in pattern.iter_items() - ) - is FunctionStepExecutionScope.PLATE - ): + if FunctionStepExecutionScope.require_uniform( + item.contract for item in pattern.iter_items() + ) is FunctionStepExecutionScope.PLATE: continue result = validator.validate_dict_pattern_keys( pattern, diff --git a/openhcs/core/pipeline/compiler.py b/openhcs/core/pipeline/compiler.py index dd06a27294..67f7a5e04c 100644 --- a/openhcs/core/pipeline/compiler.py +++ b/openhcs/core/pipeline/compiler.py @@ -283,7 +283,9 @@ def _configure_input_conversion_if_needed( if vfs_config.materialization_backend != MaterializationBackend.ZARR: return - available_backends = context.microscope_handler.available_backends(dataset_root) + available_backends = context.microscope_handler.available_backends( + dataset_root + ) if Backend.ZARR in available_backends: return @@ -505,8 +507,7 @@ def declare_zarr_stores(session: CompilationSession) -> None: last_step_index = session.step_count - 1 session.plan(last_step_index).zarr_config = { "all_wells": session.orchestrator.get_component_keys( - AxisFamily.active().partition_axis(), - resolved_config=session.global_config, + AxisFamily.active().partition_axis(), resolved_config=session.global_config ), "needs_initialization": True, } @@ -559,8 +560,7 @@ def plan_materialization_flags( f"Missing required keys: {missing_keys}." ) PipelineCompiler._compile_runtime_artifact_materialization_plans( - session, - materialization_planner, + session, materialization_planner, ) @staticmethod @@ -582,8 +582,10 @@ def _compile_runtime_artifact_materialization_plans( persistent_step_indexes.add(step_index) if persistent_step_indexes: - persistent_backend = materialization_planner.resolve_backend( - session.global_config.vfs_config.materialization_backend, + persistent_backend = ( + materialization_planner.resolve_backend( + session.global_config.vfs_config.materialization_backend, + ) ) for step_index, step_plan in session.plans.items(): persistent_enabled = step_index in persistent_step_indexes @@ -677,6 +679,7 @@ def analyze_pipeline_sequential_mode( seq_comp, resolved_config=global_config ) + if not component_values: logger.warning(f"No {seq_comp} values found in orchestrator cache") component_values_lists.append([]) @@ -889,10 +892,7 @@ def _axis_values_to_process( partition_filter_config = ( effective_config.partition_filter_config if effective_config else None ) - if ( - partition_filter_config - and partition_filter_config.partition_filter is not None - ): + if partition_filter_config and partition_filter_config.partition_filter is not None: available_values = orchestrator.get_component_keys( AxisFamily.active().partition_axis(), resolved_config=effective_config ) @@ -1088,8 +1088,7 @@ def _compile_sequential_axis_contexts( ) PipelineCompiler.declare_zarr_stores(session) PipelineCompiler.plan_materialization_flags( - session, - request.materialization_planner, + session, request.materialization_planner, ) PipelineCompiler._run_post_plan_compile_stages( session, @@ -1162,9 +1161,11 @@ def validate_payload_axis_requirements( f"{group.group_key!r}, invocation " f"{invocation.contract.function_name!r}" ) - prefix_proof = group.payload_axis_proof( - requirement, - stop_before=invocation_index, + prefix_proof = ( + group.payload_axis_proof( + requirement, + stop_before=invocation_index, + ) ) prefix_proof.validate_obligation( failure_message=lambda unproved: ( @@ -1248,11 +1249,8 @@ def _validate_payload_axis_source_anchor( and source_binding_plan.primary_plane_bindings ): PipelineCompiler._validate_pipeline_start_payload_axis_requirement( - session=session, - source_binding_plan=source_binding_plan, - owner=owner, - requirement=requirement, - failures=failures, + session=session, source_binding_plan=source_binding_plan, + owner=owner, requirement=requirement, failures=failures, ) return True if plan.requires_terminal_source_projection: @@ -1275,13 +1273,9 @@ def _validate_inherited_payload_axis_requirement( """Validate inheritance until a declaration creates the required payload axis.""" if PipelineCompiler._validate_payload_axis_source_anchor( - plan, - group, - session=session, - owner=owner, + plan, group, session=session, owner=owner, grouped_pattern=plan.compiled_function_pattern.is_grouped, - requirement=requirement, - failures=failures, + requirement=requirement, failures=failures, ): return True try: @@ -1323,7 +1317,9 @@ def _validate_inherited_payload_axis_requirement( f"empty producer group at step {producer.step_index}." ) return True - proof = producer_group.payload_axis_proof(requirement) + proof = producer_group.payload_axis_proof( + requirement + ) if proof.validate_obligation( failure_message=lambda unproved: ( f"{owner}: requirement {requirement.label} is not " @@ -1331,13 +1327,9 @@ def _validate_inherited_payload_axis_requirement( f"{unproved.contract.function_name!r}." ), source_validation=lambda: PipelineCompiler._validate_payload_axis_source_anchor( - producer, - producer_group, - session=session, - owner=owner, + producer, producer_group, session=session, owner=owner, grouped_pattern=producer.compiled_function_pattern.is_grouped, - requirement=requirement, - failures=failures, + requirement=requirement, failures=failures, ), failures=failures, ): @@ -1608,9 +1600,7 @@ def compile_pipelines( ), ) path_resolver = CompilationPathResolver( - dataset_scope=CompilationDatasetScope.from_path( - orchestrator.dataset_root - ), + dataset_scope=CompilationDatasetScope.from_path(orchestrator.dataset_root), filemanager=orchestrator.filemanager, backend=Backend.DISK.value, ) @@ -1637,12 +1627,10 @@ def compile_pipelines( orchestrator, effective_config.vfs_config, ) - global_step_axis_filters = ( - PipelineCompiler._resolve_global_step_axis_filters( - orchestrator, - pipeline_inputs, - effective_config, - ) + global_step_axis_filters = PipelineCompiler._resolve_global_step_axis_filters( + orchestrator, + pipeline_inputs, + effective_config, ) axis_request = AxisCompilationRequest( orchestrator=orchestrator, @@ -1656,8 +1644,7 @@ def compile_pipelines( filemanager=orchestrator.filemanager, input_dir=orchestrator.input_dir, available_axis_values=orchestrator.get_component_keys( - AxisFamily.active().partition_axis(), - resolved_config=effective_config, + AxisFamily.active().partition_axis(), resolved_config=effective_config ), ), source_projections_by_axis=DEFAULT_SOURCE_PROJECTION_CACHE.partition_by_axes( @@ -1766,10 +1753,7 @@ def _resolve_step_axis_filters( for step_index, step in enumerate(pipeline.steps): step_filters: dict[type[PartitionFilterConfig], StepAxisFilterResolution] = {} for field_name, config in vars(step).items(): - if ( - not isinstance(config, PartitionFilterConfig) - or config.partition_filter is None - ): + if not isinstance(config, PartitionFilterConfig) or config.partition_filter is None: continue provenance = pipeline.step_provenance[step_index].get( f"{field_name}.partition_filter" diff --git a/openhcs/core/pipeline/funcstep_contract_validator.py b/openhcs/core/pipeline/funcstep_contract_validator.py index 6198e8ba23..e9b97c74a6 100644 --- a/openhcs/core/pipeline/funcstep_contract_validator.py +++ b/openhcs/core/pipeline/funcstep_contract_validator.py @@ -1247,3 +1247,4 @@ def validate_allowed_group_by( f"group_by axes with role {allowed_names}; resolved " f"{resolved_group_by.name}." ) + diff --git a/openhcs/core/pipeline/function_contracts.py b/openhcs/core/pipeline/function_contracts.py index 6a0f6efc29..c79d486e06 100644 --- a/openhcs/core/pipeline/function_contracts.py +++ b/openhcs/core/pipeline/function_contracts.py @@ -80,16 +80,9 @@ def resolved_callable_type_hints(func: Callable) -> dict[str, Any]: from typing import _strip_annotations return { - **{ - name: _strip_annotations(parameter.annotation) - for name, parameter in signature.parameters.items() - if parameter.annotation is not inspect.Parameter.empty - }, - **( - {} - if signature.return_annotation is inspect.Signature.empty - else {"return": _strip_annotations(signature.return_annotation)} - ), + **{name: _strip_annotations(parameter.annotation) for name, parameter in signature.parameters.items() + if parameter.annotation is not inspect.Parameter.empty}, + **({} if signature.return_annotation is inspect.Signature.empty else {"return": _strip_annotations(signature.return_annotation)}), } @@ -189,7 +182,9 @@ def invocation_kwargs( @classmethod def semantic_label_payload(cls, source_projected_payload, completion_payload): """Return the label payload that represents this measurement domain.""" - raise TypeError(f"{cls.__name__} declares no object-measurement label domain.") + raise TypeError( + f"{cls.__name__} declares no object-measurement label domain." + ) @classmethod def image_execution_mode( diff --git a/openhcs/core/pipeline/path_planner.py b/openhcs/core/pipeline/path_planner.py index 2e65c8b3b0..f81d6a71cc 100644 --- a/openhcs/core/pipeline/path_planner.py +++ b/openhcs/core/pipeline/path_planner.py @@ -286,9 +286,7 @@ def context_producer_scope( output_plan, source_component ) if producer.producer_step_index is not None: - scope = self.planner.plans[ - producer.producer_step_index - ].execution_group_scope + scope = self.planner.plans[producer.producer_step_index].execution_group_scope return PathPlannerGroupScope.from_raw(scope.keys, component=scope.component) if producer.spec.ref() != PIPELINE_INPUT_ARTIFACT.ref(): return PathPlannerGroupScope.from_raw( @@ -303,8 +301,7 @@ def context_producer_scope( ) return ( PathPlannerGroupScope.from_raw(source_keys, component=source_component) - if source_keys - else PathPlannerGroupScope.dynamic(source_component) + if source_keys else PathPlannerGroupScope.dynamic(source_component) ) def get_execution_groups( @@ -434,8 +431,7 @@ def artifact_owned_execution_scope( continue has_source_lineage = ( - contract_source_bindings.binding_for_artifact_ref(spec.ref()) - is not None + contract_source_bindings.binding_for_artifact_ref(spec.ref()) is not None or available_artifacts.by_name_and_artifact_type( spec.name, spec.artifact_type, @@ -509,9 +505,7 @@ def dynamic_execution_scope_for_group_by( group_by: type[GroupingDeclaration] | None, ) -> PathPlannerGroupScope: """Return a typed runtime-discovered scope for a concrete group axis.""" - group_by_component = PathPlannerComponentScopes.component_from_group_by( - group_by - ) + group_by_component = PathPlannerComponentScopes.component_from_group_by(group_by) if group_by_component is None: return PathPlannerGroupScope.ungrouped() source_keys = ( @@ -541,9 +535,7 @@ def source_binding_scope_for_group_by( source_bindings: SourceBindingDeclarationsMixin | None = None, ) -> PathPlannerGroupScope: """Derive execution groups declared by source-binding component identity.""" - group_by_component = PathPlannerComponentScopes.component_from_group_by( - group_by - ) + group_by_component = PathPlannerComponentScopes.component_from_group_by(group_by) if group_by_component is None: return PathPlannerGroupScope.ungrouped() @@ -576,6 +568,7 @@ def execution_component_for_dict_pattern( ) return grouping_axes[0] + @staticmethod def normalized_group_by(step: AbstractStep) -> type[GroupingDeclaration]: """Use the same group_by normalization as compiled execution plans.""" @@ -610,10 +603,9 @@ def inject_metadata( ) -> FunctionPatternSyntax: """Inject values once; compiled metadata edges reference these kwargs.""" for spec in inputs.values(): - if spec.ref().for_plan_type( - ArtifactOutputPlan - ) not in self.planner.declared and self.metadata_artifact_available( - spec.name + if ( + spec.ref().for_plan_type(ArtifactOutputPlan) not in self.planner.declared + and self.metadata_artifact_available(spec.name) ): value = self.planner.ctx.microscope_handler.resolve_metadata_artifact( spec.name, @@ -649,13 +641,11 @@ def source_binding_component_domains( PathPlannerGroupScope.from_raw(values, component=component) for component in AxisFamily.active().axes for values in ( - tuple( - dict.fromkeys( - value - for domains in binding_domains - for value in domains.get(component, ()) - ) - ), + tuple(dict.fromkeys( + value + for domains in binding_domains + for value in domains.get(component, ()) + )), ) if values ) @@ -667,9 +657,7 @@ def compile_plan_maps( declarations: ArtifactGraph, group_scope: PathPlannerGroupScope, execution_scope: FunctionStepExecutionScope = FunctionStepExecutionScope.AXIS, - source_bindings: ( - StepSourceBindingsConfig | CompiledSourceBindingPlan - ) = EMPTY_SOURCE_BINDINGS, + source_bindings: StepSourceBindingsConfig | CompiledSourceBindingPlan = EMPTY_SOURCE_BINDINGS, ) -> ArtifactPlanMaps: """Compile artifact declarations into runtime I/O maps.""" step_name = step.name @@ -1096,9 +1084,7 @@ def compile_invocation_input_edges( ], execution_group_scope: PathPlannerGroupScope, consumer_variable_components: ComponentSet, - source_bindings: ( - StepSourceBindingsConfig | CompiledSourceBindingPlan - ) = EMPTY_SOURCE_BINDINGS, + source_bindings: StepSourceBindingsConfig | CompiledSourceBindingPlan = EMPTY_SOURCE_BINDINGS, available_artifacts: ArtifactSpecCollection = ArtifactSpecCollection(()), main_flow_artifacts: ArtifactSpecCollection = ArtifactSpecCollection(()), metadata_artifact_available: Callable[[str], bool] = lambda _name: False, @@ -1145,9 +1131,8 @@ def compile_invocation_input_edges( adapter is not None and adapter.manages_artifact_inputs ) raw_main_flow_refs = ( - main_flow_artifacts.ref_set().intersection( - invocation.contract.output_group_scope_sources - ) + main_flow_artifacts.ref_set() + .intersection(invocation.contract.output_group_scope_sources) if not owns_artifact_inputs else frozenset() ) @@ -1158,8 +1143,7 @@ def compile_invocation_input_edges( strict=True, ): storage_plan = ( - None - if input_spec.ref() in raw_main_flow_refs + None if input_spec.ref() in raw_main_flow_refs else selected_plans_by_ref.get(input_spec.ref()) ) if storage_plan is None: @@ -1169,12 +1153,8 @@ def compile_invocation_input_edges( spec=input_spec, main_flow_artifacts=main_flow_artifacts, invocation_sources=invocation.contract.group_scope_inputs, - metadata_available=metadata_artifact_available( - input_spec.name - ), - source_binding_available=source_bindings.declares_artifact_ref( - input_spec.ref() - ), + metadata_available=metadata_artifact_available(input_spec.name), + source_binding_available=source_bindings.declares_artifact_ref(input_spec.ref()), ) ) continue @@ -1189,15 +1169,11 @@ def compile_invocation_input_edges( consumer_variable_components=consumer_variable_components, source_bindings=source_bindings, available_artifacts=available_artifacts, - main_flow_projection=( - InvocationArtifactInputEdgePlan.source_projection( - input_spec, - main_flow_artifacts, - invocation.contract.group_scope_inputs, - ) - if owns_artifact_inputs - else None - ), + main_flow_projection=InvocationArtifactInputEdgePlan.source_projection( + input_spec, + main_flow_artifacts, + invocation.contract.group_scope_inputs, + ) if owns_artifact_inputs else None, ) ) invocations.append(invocation.with_artifact_input_edges(tuple(edges))) @@ -1282,15 +1258,13 @@ def invocation_input_edge( invocation=invocation, artifact_ref=artifact_ref, ) - producer_selection_scope = ( - ArtifactInputProjectionPlan.producer_selection_for_invocation( - input_spec=input_spec, - storage_plan=storage_plan, - component_scopes=component_scopes, - consumer_variable_components=consumer_variable_components, - invocation_key=invocation.key, - execution_scope=invocation.contract.execution_scope, - ) + producer_selection_scope = ArtifactInputProjectionPlan.producer_selection_for_invocation( + input_spec=input_spec, + storage_plan=storage_plan, + component_scopes=component_scopes, + consumer_variable_components=consumer_variable_components, + invocation_key=invocation.key, + execution_scope=invocation.contract.execution_scope, ) projection = ArtifactInputProjectionPlan( @@ -1795,9 +1769,7 @@ def materialized_output_dir_for_step( step_index, StepAxisFilterSet.empty(), ) - if not step_axis_filters.allows( - materialization_config, self.planner.ctx.axis_id - ): + if not step_axis_filters.allows(materialization_config, self.planner.ctx.axis_id): logger.debug( "Skipping materialization for step %s, axis %s (filtered out)", step.name, @@ -1884,9 +1856,7 @@ def validate(self) -> None: ): continue if dependency.kind is not StepInputDependencyKind.STEP_OUTPUT: - raise ValueError( - f"Step {curr.name} has unresolved main input dependency." - ) + raise ValueError(f"Step {curr.name} has unresolved main input dependency.") source_step_index = dependency.source_step_index if source_step_index is None: raise ValueError( @@ -1919,8 +1889,7 @@ def validate_materialization_paths(self) -> None: self.planner.paths.build_output_path(step.step_materialization_config), ) for i, step in enumerate(self.planner.session.pipeline.steps) - if step.step_materialization_config - and step.step_materialization_config.enabled + if step.step_materialization_config and step.step_materialization_config.enabled ] path_groups = defaultdict(list) @@ -2114,8 +2083,7 @@ def plan_step(self, step: AbstractStep, step_index: int) -> None: declarations = pipeline.artifact_graphs[step_index] func_pattern = declarations.pattern contracts = ( - () - if func_pattern is None + () if func_pattern is None else tuple(item.contract for item in func_pattern.iter_items()) ) execution_scope = FunctionStepExecutionScope.require_uniform(contracts) @@ -2179,9 +2147,7 @@ def plan_step(self, step: AbstractStep, step_index: int) -> None: self.planner.materialization.apply_materialization_plan( step, step_index, - self.planner.materialization.materialized_output_dir_for_step( - step, step_index - ), + self.planner.materialization.materialized_output_dir_for_step(step, step_index), ) self.planner.materialization.apply_input_conversion_plan( step_index, @@ -2241,9 +2207,7 @@ def update_core_step_plan( step_plan.main_input_dependency = main_input_dependency step_plan.artifact_inputs = artifact_maps.inputs step_plan.artifact_outputs = artifact_maps.outputs - step_plan.future_artifact_inputs = self.planner.future_artifact_inputs[ - step_index - ] + step_plan.future_artifact_inputs = self.planner.future_artifact_inputs[step_index] step_plan.execution_group_scope = artifact_maps.group_scope step_plan.compiled_function_pattern = compiled_function_pattern step_plan.source_binding_plan = artifact_maps.source_binding_plan @@ -2256,16 +2220,11 @@ def main_input_dependency( *, declarations: ArtifactGraph = ArtifactGraph(), execution_scope: FunctionStepExecutionScope = FunctionStepExecutionScope.AXIS, - source_bindings: ( - StepSourceBindingsConfig | CompiledSourceBindingPlan - ) = EMPTY_SOURCE_BINDINGS, + source_bindings: StepSourceBindingsConfig | CompiledSourceBindingPlan = EMPTY_SOURCE_BINDINGS, ) -> StepInputDependency: """Resolve the explicit main-input edge for one step.""" existing_plan = self.planner.plans.get(step_index) - if ( - existing_plan is not None - and existing_plan.main_input_dependency.is_resolved - ): + if existing_plan is not None and existing_plan.main_input_dependency.is_resolved: return existing_plan.main_input_dependency previous = self.planner.plans.get(step_index - 1) @@ -2430,6 +2389,7 @@ def _build_axis_filename( return _cached_axis_filename(axis_id, key, extension, step_index) + @lru_cache(maxsize=65536) def _cached_output_dataset_root( dataset_root: str, diff --git a/openhcs/core/post_execute.py b/openhcs/core/post_execute.py index bbd4594be8..340025b598 100644 --- a/openhcs/core/post_execute.py +++ b/openhcs/core/post_execute.py @@ -93,9 +93,7 @@ def run( # -- the family as the kernel uses it ------------------------------------- @staticmethod - def bind_all( - global_config: "GlobalPipelineConfig", - ) -> tuple["PostExecuteHook", ...]: + def bind_all(global_config: "GlobalPipelineConfig") -> tuple["PostExecuteHook", ...]: return tuple( hook_type.bind(global_config) for hook_type in PostExecuteHook.__registry__.values() diff --git a/openhcs/core/processing_contracts.py b/openhcs/core/processing_contracts.py index 4c01053e89..afcdf2eb35 100644 --- a/openhcs/core/processing_contracts.py +++ b/openhcs/core/processing_contracts.py @@ -150,9 +150,7 @@ def accepted_kwargs( from openhcs.core.callable_contract import CallableMetadata parameters = ( - CallableMetadata.callable_signature(func) - if signature is None - else signature + CallableMetadata.callable_signature(func) if signature is None else signature ).parameters if any( parameter.kind is inspect.Parameter.VAR_KEYWORD @@ -280,7 +278,8 @@ def contextualize_main_image_output(source_image: Any, result: Any) -> Any: return result source_image = owned_runtime_value(source_image) if not isinstance(source_image, ImagePayload) or ( - source_image.mask is None and not source_image.metadata.has_values + source_image.mask is None + and not source_image.metadata.has_values ): return result return source_image.with_pixels(result) @@ -445,9 +444,7 @@ class ProducedImageStackPure2DInputSlicer(ImagePayloadPure2DInputSlicer): value_type = ProducedImageStack - def slice_value( - self, value: ProducedImageStack, memory_type: str - ) -> tuple[Any, ...]: + def slice_value(self, value: ProducedImageStack, memory_type: str) -> tuple[Any, ...]: del memory_type return tuple( RuntimeSliceProjection.value_for_slice( @@ -700,9 +697,7 @@ def aggregate_values( plane_axis: RuntimePlaneAxis, ) -> Any: return ImagePayloadSliceStack.from_output_slices( - values, - memory_type=memory_type, - plane_axis=plane_axis, + values, memory_type=memory_type, plane_axis=plane_axis, ) @@ -768,9 +763,7 @@ def aggregate_values( plane_axis: RuntimePlaneAxis, ) -> Any: return ImagePayloadSliceStack.from_output_slices( - values, - memory_type=memory_type, - plane_axis=plane_axis, + values, memory_type=memory_type, plane_axis=plane_axis, ) @@ -852,6 +845,8 @@ class TuplePure2DAuxiliaryOutputAggregator(FlatSequencePure2DAuxiliaryOutputAggr value_type = tuple + + # --------------------------------------------------------------------------- # The contract family # --------------------------------------------------------------------------- @@ -914,17 +909,13 @@ def invoke_unbound(self, value: Any, kwargs: Mapping[str, Any]) -> Any: """Call once with the value as the stack holds it.""" def full_stack_inputs( - self, - image: Any, - kwargs: Mapping[str, Any], + self, image: Any, kwargs: Mapping[str, Any], ) -> tuple[Any, Mapping[str, Any]]: """Return the image and kwargs a whole-stack call receives.""" return image, kwargs def validate_full_stack_kwargs( - self, - contract: type["ProcessingContract"], - kwargs: Mapping[str, Any], + self, contract: type["ProcessingContract"], kwargs: Mapping[str, Any], ) -> None: """Reject whole-stack kwargs this environment cannot hand ``contract``.""" @@ -956,9 +947,7 @@ def slice_kwargs( @abstractmethod def aggregate_slices( - self, - split: PlaneSplit, - batch: "Pure2DSliceResultBatch", + self, split: PlaneSplit, batch: "Pure2DSliceResultBatch", ) -> Any: """Restack per-plane results.""" @@ -1000,8 +989,7 @@ def for_key(cls, key: str) -> type["ProcessingContract"]: def for_probe(cls, probe: ContractProbe) -> type["ProcessingContract"] | None: """Return the contract whose declaration explains a probe outcome.""" explaining = tuple( - contract - for contract in cls.__registry__.values() + contract for contract in cls.__registry__.values() if contract.explains_probe(probe) ) if len(explaining) > 1: @@ -1031,8 +1019,7 @@ def explains_probe(cls, probe: ContractProbe) -> bool: @classmethod def supports_measurement_image_batch( - cls, - request: RuntimeBatchInvocationRequest, + cls, request: RuntimeBatchInvocationRequest, ) -> bool: """Whether the request already occupies this callable's image domain.""" return True @@ -1076,9 +1063,7 @@ def main_flow_output_source_payload(cls, source_payload: Any) -> Any: @classmethod def main_flow_call_argument( - cls, - callable_contract: "CallableContract", - source_payload: Any, + cls, callable_contract: "CallableContract", source_payload: Any, ) -> Any: """Expose the raw ABI when this contract needs no earlier plane slicing.""" return callable_contract.runtime_main_flow_call_argument(source_payload) @@ -1096,8 +1081,7 @@ def consume_semantic_controls( from openhcs.core.callable_contract import CallableMetadata parameters = ( - {} - if func is None + {} if func is None else CallableMetadata.callable_signature(func).parameters ) values: dict[str, Any] = {} @@ -1128,19 +1112,13 @@ def variable_component_stack_requirement( @classmethod @abstractmethod def execute( - cls, - call: ContractCall, - image: Any, - kwargs: dict[str, Any], + cls, call: ContractCall, image: Any, kwargs: dict[str, Any], ) -> Any: """Execute ``call.func`` on ``image`` under this contract.""" @classmethod def execute_whole_stack( - cls, - call: ContractCall, - image: Any, - kwargs: dict[str, Any], + cls, call: ContractCall, image: Any, kwargs: dict[str, Any], ) -> Any: """Call once on the whole stack.""" projected_image, projected_kwargs = call.full_stack_inputs(image, kwargs) @@ -1184,8 +1162,7 @@ def explains_probe(cls, probe: ContractProbe) -> bool: @classmethod def supports_measurement_image_batch( - cls, - request: RuntimeBatchInvocationRequest, + cls, request: RuntimeBatchInvocationRequest, ) -> bool: """Leave preserved per-plane domains to the plane slicer.""" return ( @@ -1196,9 +1173,7 @@ def supports_measurement_image_batch( @classmethod def main_flow_call_argument( - cls, - callable_contract: "CallableContract", - source_payload: Any, + cls, callable_contract: "CallableContract", source_payload: Any, ) -> Any: """Retain plane context only for the declared contract execution wrapper.""" if callable_contract.raw_processing_function is not None: @@ -1211,10 +1186,7 @@ def execute(cls, call: ContractCall, image: Any, kwargs: dict[str, Any]) -> Any: @classmethod def execute_per_plane( - cls, - call: ContractCall, - image: Any, - kwargs: dict[str, Any], + cls, call: ContractCall, image: Any, kwargs: dict[str, Any], ) -> Any: """Call once per plane and restack the results.""" split = call.plane_split(image, kwargs) @@ -1246,9 +1218,7 @@ def execute_slice( signature=call.signature, ) ) - return call.aggregate_slices( - split, Pure2DSliceResultBatch.from_results(results) - ) + return call.aggregate_slices(split, Pure2DSliceResultBatch.from_results(results)) class FlexibleContract(Pure2DContract, Pure3DContract): @@ -1267,15 +1237,13 @@ def explains_probe(cls, probe: ContractProbe) -> bool: @classmethod def supports_measurement_image_batch( - cls, - request: RuntimeBatchInvocationRequest, + cls, request: RuntimeBatchInvocationRequest, ) -> bool: - controls = cls.consume_semantic_controls( - dict(request.kwargs), func=request.func + controls = cls.consume_semantic_controls(dict(request.kwargs), func=request.func) + return ( + not any(bool(value) for value in controls.values()) + or super().supports_measurement_image_batch(request) ) - return not any( - bool(value) for value in controls.values() - ) or super().supports_measurement_image_batch(request) @classmethod def runtime_parameter_types( @@ -1378,29 +1346,16 @@ def invoke( *, func: Callable[..., Any] | None = None, ) -> Any: - return ( - RuntimeCallablePolicy() - .invocation( - self.func if func is None else func, - ( - self.callable_contract.runtime_main_flow_call_argument(value), - *self.args, - ), - kwargs, - ) - .call() - ) + return RuntimeCallablePolicy().invocation( + self.func if func is None else func, + (self.callable_contract.runtime_main_flow_call_argument(value), *self.args), + kwargs, + ).call() def invoke_unbound(self, value: Any, kwargs: Mapping[str, Any]) -> Any: - return ( - RuntimeCallablePolicy() - .invocation( - self.func, - (value, *self.args), - kwargs, - ) - .call() - ) + return RuntimeCallablePolicy().invocation( + self.func, (value, *self.args), kwargs, + ).call() def whole_output(self, image: Any, result: Any) -> Any: return contextualize_main_image_output(image, result) @@ -1428,8 +1383,7 @@ def _consumes_full_label_stack(self, kwargs: Mapping[str, Any]) -> bool: return False positional = ( dict(zip((p.name for p in self._positional_parameters()), self.args)) - if self.args - else {} + if self.args else {} ) return {**positional, **kwargs}.get("labels") is not None @@ -1464,7 +1418,7 @@ def plane_split(self, image: Any, kwargs: Mapping[str, Any]) -> PlaneSplit | Non ) for parameter, value in zip(positional_parameters, self.args): kwargs.setdefault(parameter.name, value) - self.args = self.args[len(positional_parameters) :] + self.args = self.args[len(positional_parameters):] return PlaneSplit( slices=tuple(slicer.slice_value(image, input_memory_type)), plane_axis=plane_axis, @@ -1496,9 +1450,7 @@ def slice_kwargs( ) def aggregate_slices( - self, - split: PlaneSplit, - batch: "Pure2DSliceResultBatch", + self, split: PlaneSplit, batch: "Pure2DSliceResultBatch", ) -> Any: stacked_main_output = Pure2DAuxiliaryOutputAggregator.aggregate( batch.main_outputs, diff --git a/openhcs/core/progress/projection.py b/openhcs/core/progress/projection.py index 867f0b9b18..c63366af2b 100644 --- a/openhcs/core/progress/projection.py +++ b/openhcs/core/progress/projection.py @@ -184,7 +184,9 @@ def accepts_server_lifecycle_state(cls, state: type[SubjectRuntimeState]) -> boo return state is ExecutingSubjectRuntimeState -class CompleteSubjectRuntimeState(TerminalSubjectRuntimeState, SubjectRuntimeState): +class CompleteSubjectRuntimeState( + TerminalSubjectRuntimeState, SubjectRuntimeState +): state_name = "complete" default_channel = ProgressChannel.PIPELINE status_label = "✅ Complete" @@ -192,7 +194,9 @@ class CompleteSubjectRuntimeState(TerminalSubjectRuntimeState, SubjectRuntimeSta count_sort_order = 40 -class FailedSubjectRuntimeState(TerminalSubjectRuntimeState, SubjectRuntimeState): +class FailedSubjectRuntimeState( + TerminalSubjectRuntimeState, SubjectRuntimeState +): state_name = "failed" status_label = "❌ Failed" count_label = "❌ {count} failed" @@ -382,7 +386,9 @@ def count_for_state(self, state: type[SubjectRuntimeState]) -> int: def count_status_labels(self) -> Tuple[str, ...]: labels: List[str] = [] for declaration in SubjectRuntimeState.counted_declarations(): - label = declaration.count_status_label(self.count_for_state(declaration)) + label = declaration.count_status_label( + self.count_for_state(declaration) + ) if label: labels.append(label) return tuple(labels) @@ -412,9 +418,7 @@ def get_subject( self, subject_id: str, execution_id: Optional[str] = None ) -> Optional[SubjectRuntimeProjection]: if execution_id is not None: - return self.by_identity.get( - SubjectRuntimeIdentity(execution_id, subject_id) - ) + return self.by_identity.get(SubjectRuntimeIdentity(execution_id, subject_id)) return self.by_subject_latest.get(subject_id) def reconcile_server_executions( @@ -482,7 +486,8 @@ def reconcile_server_executions( def recalculate_summary(self) -> None: self.state_counts = { - declaration: 0 for declaration in SubjectRuntimeState.__registry__.values() + declaration: 0 + for declaration in SubjectRuntimeState.__registry__.values() } for subject in self.by_subject_latest.values(): self.state_counts[subject.state] += 1 diff --git a/openhcs/core/progress/runtime_tree.py b/openhcs/core/progress/runtime_tree.py index 0125a0da4c..efb472feb1 100644 --- a/openhcs/core/progress/runtime_tree.py +++ b/openhcs/core/progress/runtime_tree.py @@ -122,9 +122,7 @@ def aggregate_percent( return node_percent -class SubjectProgressTreeNode( - MeanPercentRuntimeTreeNode, RuntimeTreeNodeDeclarationBase -): +class SubjectProgressTreeNode(MeanPercentRuntimeTreeNode, RuntimeTreeNodeDeclarationBase): node_kind = "subject" sort_order = 0 @@ -273,9 +271,7 @@ def from_events( dict[str, tuple[str, ...]], ] = {} step_name_sequences: dict[tuple[str, str], tuple[str, ...]] = {} - events_by_subject: dict[tuple[str, str], list[ProgressEvent]] = defaultdict( - list - ) + events_by_subject: dict[tuple[str, str], list[ProgressEvent]] = defaultdict(list) for execution_id, events in executions.items(): for event in sorted(events, key=lambda candidate: candidate.timestamp): @@ -640,10 +636,7 @@ def finalize_subject_node( elif self.all_leaves_queued(node): node.status = "⏳ Queued" self.apply_node_info_text(node) - if ( - subject_projection is not None - and subject_projection.queue_position is not None - ): + if subject_projection is not None and subject_projection.queue_position is not None: node.info = f"{node.percent:.1f}% (q#{subject_projection.queue_position})" def aggregate_percent_recursive(self, node: RuntimeTreeNode) -> float: @@ -791,9 +784,7 @@ def build_progress_tree( topology: RuntimeExecutionTopology, get_subject_name: Callable[[str, str | None], str], ) -> List[RuntimeTreeNode]: - events_by_subject: Dict[tuple[str, str], List[ProgressEvent]] = defaultdict( - list - ) + events_by_subject: Dict[tuple[str, str], List[ProgressEvent]] = defaultdict(list) for exec_id, events_list in executions.items(): for event in events_list: events_by_subject[(exec_id, event.subject_id)].append(event) @@ -811,8 +802,7 @@ def build_progress_tree( worker_assignments=topology.worker_assignments, ) missing_execution_topology = ( - is_executing - and (exec_id, subject_id) not in topology.worker_assignments + is_executing and (exec_id, subject_id) not in topology.worker_assignments ) if missing_execution_topology: children = [] @@ -862,9 +852,7 @@ def build_progress_tree( subject_node, subject_projection=subject_projection, ) - nodes_by_identity[SubjectRuntimeIdentity(exec_id, subject_id)] = ( - subject_node - ) + nodes_by_identity[SubjectRuntimeIdentity(exec_id, subject_id)] = subject_node visible_nodes: list[RuntimeTreeNode] = [] for subject_projection in runtime_projection.by_subject_latest.values(): @@ -972,9 +960,7 @@ def _build_compilation_children( for event in channels[ProgressChannel.COMPILE] if event.axis_id } - known_axis_ids = list( - known_partition_values.get((execution_id, subject_id), []) - ) + known_axis_ids = list(known_partition_values.get((execution_id, subject_id), [])) axis_ids = known_axis_ids if known_axis_ids else sorted(compile_by_axis.keys()) extra_axis_ids = [ axis_id for axis_id in compile_by_axis if axis_id not in axis_ids diff --git a/openhcs/core/progress/types.py b/openhcs/core/progress/types.py index 8abd3ceae8..e84fa4f8a2 100644 --- a/openhcs/core/progress/types.py +++ b/openhcs/core/progress/types.py @@ -224,33 +224,25 @@ def is_success_terminal_for_status( class InitChannelProgressPhase: """Trait for progress phases carried on the init channel.""" - progress_channel: ClassVar[type[ProgressChannelDeclarationBase]] = ( - InitProgressChannel - ) + progress_channel: ClassVar[type[ProgressChannelDeclarationBase]] = InitProgressChannel class CompileChannelProgressPhase: """Trait for progress phases carried on the compile channel.""" - progress_channel: ClassVar[type[ProgressChannelDeclarationBase]] = ( - CompileProgressChannel - ) + progress_channel: ClassVar[type[ProgressChannelDeclarationBase]] = CompileProgressChannel class PipelineChannelProgressPhase: """Trait for progress phases carried on the pipeline execution channel.""" - progress_channel: ClassVar[type[ProgressChannelDeclarationBase]] = ( - PipelineProgressChannel - ) + progress_channel: ClassVar[type[ProgressChannelDeclarationBase]] = PipelineProgressChannel class StepChannelProgressPhase: """Trait for progress phases carried on the step execution channel.""" - progress_channel: ClassVar[type[ProgressChannelDeclarationBase]] = ( - StepProgressChannel - ) + progress_channel: ClassVar[type[ProgressChannelDeclarationBase]] = StepProgressChannel class InitProgressPhase(InitChannelProgressPhase, ProgressPhaseDeclarationBase): @@ -412,9 +404,7 @@ class QueuedProgressStatus(ProgressStatusDeclarationBase): def phase_channel(phase: ProgressPhase) -> ProgressChannel: """Classify phase to semantic channel.""" - return ProgressPhaseDeclarationBase.for_phase( - phase - ).progress_channel.require_channel() + return ProgressPhaseDeclarationBase.for_phase(phase).progress_channel.require_channel() def progress_channel_role(channel: ProgressChannel) -> ProgressChannelRole: diff --git a/openhcs/core/runtime_adapters.py b/openhcs/core/runtime_adapters.py index 1999040b4c..5954af7d00 100644 --- a/openhcs/core/runtime_adapters.py +++ b/openhcs/core/runtime_adapters.py @@ -262,8 +262,7 @@ class RuntimeAdapterSpec: runtime_callable_factory: RuntimeCallableFactory | None = None def invocation_domain_inputs( - self, - contract: "CallableContract", + self, contract: "CallableContract", ) -> ArtifactSpecCollection: """Use declared context inputs when the callable has no raw image ABI.""" if contract.accepts_implicit_main_flow_input: diff --git a/openhcs/core/runtime_batch_contracts.py b/openhcs/core/runtime_batch_contracts.py index 6f8b2136d9..2dbd7d8cfa 100644 --- a/openhcs/core/runtime_batch_contracts.py +++ b/openhcs/core/runtime_batch_contracts.py @@ -12,11 +12,7 @@ from metaclass_registry import AutoRegisterMeta -from openhcs.core.callable_contract import ( - CallableContract, - CallableMetadata, - KeywordRuntimeParameter, -) +from openhcs.core.callable_contract import CallableContract, CallableMetadata, KeywordRuntimeParameter from openhcs.core.function_reference import FunctionReference from openhcs.core.runtime_adapters import RuntimeImageExecutionContext from openhcs.core.runtime_plane_projection import ( @@ -52,17 +48,11 @@ class RuntimePlaneAxisValueProjectionParameter(KeywordRuntimeParameter): def runtime_callable_defaults( - func: Callable[..., object], - *, - signature: inspect.Signature | None = None, + func: Callable[..., object], *, signature: inspect.Signature | None = None, ) -> Mapping[str, object]: """Return callable defaults visible to runtime batch executors.""" try: - callable_signature = ( - CallableMetadata.callable_signature(func) - if signature is None - else signature - ) + callable_signature = CallableMetadata.callable_signature(func) if signature is None else signature except (TypeError, ValueError) as exc: raise TypeError( f"Runtime batch function {func!r} must expose an inspectable signature." @@ -105,9 +95,7 @@ def __post_init__(self) -> None: ) def batch_executor_request( - self, - *, - processing_contract: "type[ProcessingContract]", + self, *, processing_contract: "type[ProcessingContract]", ) -> "RuntimeBatchInvocationRequest | None": """Return a request projected into the batch executor's image domain. @@ -240,9 +228,7 @@ def execute_one(self, slice_index: int) -> RuntimeSliceResultT: def execute_each(self) -> list[RuntimeSliceResultT]: """Execute every slice of this batch, one invocation per slice.""" - return [ - self.execute_one(slice_index) for slice_index in range(self.slice_count) - ] + return [self.execute_one(slice_index) for slice_index in range(self.slice_count)] def execute_one_with_kwargs( self, @@ -311,7 +297,9 @@ def executors(self) -> Mapping[RuntimeBatchExecutionDomain, Callable]: if isinstance(self.raw_processing_function, FunctionReference) else self.raw_processing_function ) - inherited = runtime_batch_executors_from_callable(raw_callable) + inherited = runtime_batch_executors_from_callable( + raw_callable + ) for domain, executor in inherited.items(): if domain not in batch_executors: batch_executors[domain] = executor @@ -323,15 +311,10 @@ class Pure2DSliceBatchExecutor(RuntimeBatchExecutor): @classmethod def from_executors( - cls, - executors: Mapping[RuntimeBatchExecutionDomain, Callable] | None, + cls, executors: Mapping[RuntimeBatchExecutionDomain, Callable] | None, ) -> Callable: """Honor a declared pure-2D executor before the nominal serial default.""" - declared = ( - None - if executors is None - else executors.get(RuntimeBatchExecutionDomain.PURE_2D_SLICES) - ) + declared = None if executors is None else executors.get(RuntimeBatchExecutionDomain.PURE_2D_SLICES) return declared if callable(declared) else cls.default_executor() @classmethod diff --git a/openhcs/core/runtime_exports.py b/openhcs/core/runtime_exports.py index 75bc60d8f7..c1c88183e5 100644 --- a/openhcs/core/runtime_exports.py +++ b/openhcs/core/runtime_exports.py @@ -112,9 +112,7 @@ class RuntimeExportObservation: table_headers_by_path: Mapping[Path, tuple[str, ...]] table_row_counts_by_path: Mapping[Path, int] output_files: tuple[Path, ...] = () - outputs: StepExecutionObservation = field( - default_factory=StepExecutionObservation.empty - ) + outputs: StepExecutionObservation = field(default_factory=StepExecutionObservation.empty) @classmethod def from_runtime_observations( diff --git a/openhcs/core/runtime_measurements.py b/openhcs/core/runtime_measurements.py index 166ab5d24a..0597660053 100644 --- a/openhcs/core/runtime_measurements.py +++ b/openhcs/core/runtime_measurements.py @@ -1098,8 +1098,6 @@ def object_location_coordinate_arrays( ) coordinates.append((feature.value, values)) return tuple(coordinates) - - @dataclass(frozen=True, slots=True) class ObjectMeasurementValueRow: """Nominal long-form object measurement row.""" @@ -1763,8 +1761,7 @@ def rows(self) -> list[dict[str, float | int]]: value_index = value_indexes.get(object_id) row[feature_name] = ( missing_values[object_id].scalar - if value_index is None - else python_values[value_index] + if value_index is None else python_values[value_index] ) self.complete_row(row) rows.append(row) @@ -1841,9 +1838,7 @@ def complete_row(self, row: dict[str, float | int]) -> None: del row def feature_missing_values_for_objects( - self, - feature_name: str, - object_ids: tuple[int, ...], + self, feature_name: str, object_ids: tuple[int, ...], ) -> Mapping[int, ObjectFeatureMissingValue]: """Project one column's missing-value policy onto its absent objects.""" missing_value = self.feature_missing_values.get( @@ -1919,3 +1914,5 @@ def identity_token(self) -> str: self.id_field or "", ) ) + + diff --git a/openhcs/core/runtime_object_label_aggregation.py b/openhcs/core/runtime_object_label_aggregation.py index b21ac16d24..e3ed2e107d 100644 --- a/openhcs/core/runtime_object_label_aggregation.py +++ b/openhcs/core/runtime_object_label_aggregation.py @@ -182,13 +182,9 @@ def aggregate_values(self) -> ObjectLabelValue: and MemoryType(self.memory_type) is MemoryType.NUMPY ): declared = self.declared_variants() - final_planes = tuple( - self.slice_labels(value, FinalLabels) for value in self.values - ) + final_planes = tuple(self.slice_labels(value, FinalLabels) for value in self.values) variant_planes = { - variant: tuple( - self.slice_labels(value, variant) for value in self.values - ) + variant: tuple(self.slice_labels(value, variant) for value in self.values) for variant in declared } return self.output_value( diff --git a/openhcs/core/runtime_object_label_building.py b/openhcs/core/runtime_object_label_building.py index 125793b807..1095e7fcab 100644 --- a/openhcs/core/runtime_object_label_building.py +++ b/openhcs/core/runtime_object_label_building.py @@ -150,8 +150,7 @@ def plane_semantics(self) -> RuntimePlaneAxisValueProjection | None: return None metadata = self.metadata projection.validate_source_declaration( - metadata.plane_axis, - metadata.source_provenance, + metadata.plane_axis, metadata.source_provenance, ) label_array = np.asarray(self.labels) projection.validate_shape( diff --git a/openhcs/core/runtime_object_labels.py b/openhcs/core/runtime_object_labels.py index 4720f0357f..89f953d0f3 100644 --- a/openhcs/core/runtime_object_labels.py +++ b/openhcs/core/runtime_object_labels.py @@ -136,10 +136,7 @@ def ordered_label_variants( if variant is FinalLabels: raise ValueError("Final labels are carried as labels, not as a variant.") return MappingProxyType( - { - variant: variants[variant] - for variant in sorted(variants, key=lambda v: v.name) - } + {variant: variants[variant] for variant in sorted(variants, key=lambda v: v.name)} ) @@ -155,9 +152,7 @@ def __post_init__(self) -> None: @property def shape(self) -> tuple[int, ...] | None: - return ObjectLabelStorageStrategy.for_value(self.labels).label_shape( - self.labels - ) + return ObjectLabelStorageStrategy.for_value(self.labels).label_shape(self.labels) @property def dtype(self) -> Any: @@ -165,9 +160,7 @@ def dtype(self) -> Any: def validate_plane_count(self, plane_count: int, context: str) -> None: object_label_validate_plane_count( - self.labels, - plane_count=plane_count, - context=context, + self.labels, plane_count=plane_count, context=context, ) @classmethod @@ -332,17 +325,12 @@ def project_plane( ) def project_planes( - self, - plane_indices: tuple[int, ...], - *, - plane_count: int, + self, plane_indices: tuple[int, ...], *, plane_count: int, ) -> "ObjectLabelVariantData": """Project each variant through the same ordered plane selection.""" return self.project( lambda labels: object_label_project_planes( - labels, - plane_indices, - plane_count=plane_count, + labels, plane_indices, plane_count=plane_count, ) ) @@ -368,9 +356,7 @@ class PlaneStackObjectLabelVariantData(ObjectLabelVariantData): __slots__ = ("_planes", "_dense_variants", "_memory_type", "_shape", "_lock") def __init__( - self, - variants: Sequence[ObjectLabelVariantData], - memory_type: str, + self, variants: Sequence[ObjectLabelVariantData], memory_type: str, ) -> None: from openhcs.core.memory import MemoryType, runtime_slice_stack_geometry @@ -410,13 +396,9 @@ def __repr__(self) -> str: def _dense_variant(self, variant: type[ObjectLabelVariant]) -> ObjectLabelData: with self._lock: if variant not in self._dense_variants: - dense = object_label_stack_planes( - self._planes[variant], self._memory_type - ) + dense = object_label_stack_planes(self._planes[variant], self._memory_type) self._dense_variants[variant] = dense - self._planes[variant] = tuple( - dense[index] for index in range(len(dense)) - ) + self._planes[variant] = tuple(dense[index] for index in range(len(dense))) return self._dense_variants[variant] @property @@ -426,15 +408,11 @@ def labels(self) -> ObjectLabelData: @property def variants(self) -> ObjectLabelVariants: return MappingProxyType( - { - variant: self._dense_variant(variant) - for variant in self.present_variants[1:] - } + {variant: self._dense_variant(variant) for variant in self.present_variants[1:]} ) def variant_labels( - self, - variant: type[ObjectLabelVariant], + self, variant: type[ObjectLabelVariant], ) -> ObjectLabelData | None: if variant not in self.present_variants: return None @@ -460,35 +438,22 @@ def validate_plane_count(self, plane_count: int, context: str) -> None: ) def validate_representation( - self, - *, - representation: ObjectLabelRepresentation, - value_label: str, + self, *, representation: ObjectLabelRepresentation, value_label: str, ) -> None: for planes in self._planes.values(): for plane in planes: ObjectLabelStorageStrategy.for_value(plane).validate_representation( - plane, - representation=representation, - value_label=value_label, + plane, representation=representation, value_label=value_label, ) - def in_representation( - self, representation: ObjectLabelRepresentation - ) -> ObjectLabelVariantData: + def in_representation(self, representation: ObjectLabelRepresentation) -> ObjectLabelVariantData: if representation is ObjectLabelRepresentation.DENSE_LABELS: - self.validate_representation( - representation=representation, value_label=type(self).__name__ - ) + self.validate_representation(representation=representation, value_label=type(self).__name__) return self return super().in_representation(representation) def project_plane( - self, - *, - plane_axis: RuntimePlaneAxis, - plane_index: int, - plane_count: int, + self, *, plane_axis: RuntimePlaneAxis, plane_index: int, plane_count: int, ) -> ObjectLabelVariantData: self.validate_plane_count(plane_count, "Object-label data plane projection") if plane_index < 0 or plane_index >= plane_count: @@ -496,10 +461,7 @@ def project_plane( return ObjectLabelPlaneVariantData(self, plane_index) def project_planes( - self, - plane_indices: tuple[int, ...], - *, - plane_count: int, + self, plane_indices: tuple[int, ...], *, plane_count: int, ) -> ObjectLabelVariantData: self.validate_plane_count(plane_count, "Object-label data plane projection") if any(index < 0 or index >= plane_count for index in plane_indices): @@ -510,11 +472,8 @@ def __getstate__(self) -> dict[str, Any]: with self._lock: return { "variants": { - variant: ( - self._dense_variants[variant] - if variant in self._dense_variants - else planes - ) + variant: self._dense_variants[variant] + if variant in self._dense_variants else planes for variant, planes in self._planes.items() }, "memory_type": self._memory_type, @@ -522,27 +481,15 @@ def __getstate__(self) -> dict[str, Any]: } def __setstate__(self, state: dict[str, Any]) -> None: - object.__setattr__( - self, - "_dense_variants", - { - variant: data - for variant, data in state["variants"].items() - if isinstance(data, np.ndarray) - }, - ) - object.__setattr__( - self, - "_planes", - { - variant: ( - tuple(data[index] for index in range(len(data))) - if isinstance(data, np.ndarray) - else data - ) - for variant, data in state["variants"].items() - }, - ) + object.__setattr__(self, "_dense_variants", { + variant: data for variant, data in state["variants"].items() + if isinstance(data, np.ndarray) + }) + object.__setattr__(self, "_planes", { + variant: tuple(data[index] for index in range(len(data))) + if isinstance(data, np.ndarray) else data + for variant, data in state["variants"].items() + }) object.__setattr__(self, "_memory_type", state["memory_type"]) object.__setattr__(self, "_shape", state["shape"]) object.__setattr__(self, "_lock", Lock()) @@ -554,9 +501,7 @@ class ProjectedObjectLabelVariantData(PlaneStackObjectLabelVariantData): __slots__ = ("_source", "_indices") def __init__( - self, - source: PlaneStackObjectLabelVariantData, - indices: tuple[int, ...], + self, source: PlaneStackObjectLabelVariantData, indices: tuple[int, ...], ) -> None: object.__setattr__(self, "_source", source) object.__setattr__(self, "_indices", indices) @@ -581,20 +526,14 @@ def _dense_variant(self, variant: type[ObjectLabelVariant]) -> ObjectLabelData: def _project_variant(self, source: ObjectLabelData) -> ObjectLabelData: return object_label_project_planes( - source, - self._indices, - plane_count=self._source.shape[0], + source, self._indices, plane_count=self._source.shape[0], ) def validate_representation( - self, - *, - representation: ObjectLabelRepresentation, - value_label: str, + self, *, representation: ObjectLabelRepresentation, value_label: str, ) -> None: self._source.validate_representation( - representation=representation, - value_label=value_label, + representation=representation, value_label=value_label, ) def __reduce__(self): @@ -613,9 +552,7 @@ def __init__(self, source: PlaneStackObjectLabelVariantData, index: int) -> None def _project_variant(self, source: ObjectLabelData) -> ObjectLabelData: return object_label_project_plane( - source, - self._indices[0], - plane_count=self._source.shape[0], + source, self._indices[0], plane_count=self._source.shape[0], ) @@ -691,8 +628,7 @@ def __len__(self) -> int: return len(self.labels) def variant_labels( - self, - variant: type[ObjectLabelVariant], + self, variant: type[ObjectLabelVariant], ) -> ObjectLabelData | None: """Return one declared label variant, or None when absent.""" return self.variant_data.variant_labels(variant) @@ -825,8 +761,7 @@ def spatial_adapter( ) return ObjectLabelPayloadSourceSpatialDomainAdapter( - self, - source_shape_override_yx=source_shape_override_yx, + self, source_shape_override_yx=source_shape_override_yx, ) # -- derived image outputs ------------------------------------------------ @@ -857,9 +792,7 @@ def contextualize_image_output( return source_metadata.derive_payload(self, output, plane_projection=None) if plane_projection is None or plane_projection.plane_index is not None: return source_metadata.derive_payload( - self, - output, - plane_projection=plane_projection, + self, output, plane_projection=plane_projection, ) plane_count = source_metadata.source_provenance.source_plane_count if plane_count != plane_projection.axis_size: @@ -876,9 +809,7 @@ def contextualize_image_output( plane_axis=plane_projection.axis, ).attach_to(output) return source_metadata.derive_payload( - self, - contextualized_output, - plane_projection=plane_projection, + self, contextualized_output, plane_projection=plane_projection, ) def object_label_output( @@ -895,8 +826,7 @@ def alignment_slices(self) -> tuple[Any, ...]: if slice_count is None: return (self,) projection = RuntimePlaneAxisValueProjection.preserve( - axis=RuntimePlaneAxis.RUNTIME_SLICE, - axis_size=slice_count, + axis=RuntimePlaneAxis.RUNTIME_SLICE, axis_size=slice_count, ) return tuple( self.value_for_slice(projection.selected_plane(index)) @@ -1066,7 +996,9 @@ def with_measurement_labels(self, labels: ObjectLabelData) -> Self: plane_count=plane_count, context="Object-label measurement replacement", ) - if variants.same_arrays_as(self.variant_data): + if ( + variants.same_arrays_as(self.variant_data) + ): return self return self.with_variants(variants) @@ -1125,8 +1057,7 @@ def with_runtime_slice_projection( ) if label_plane_indices is not None else source_variants.project_runtime_slice( - slice_index=slice_index, - slice_count=slice_count, + slice_index=slice_index, slice_count=slice_count, ) ) projected_domain = self.runtime_slice_domain( @@ -1176,10 +1107,11 @@ def with_plane_projection( ) if normalized_indices == tuple(range(plane_count)): return self - metadata = self.metadata.for_source_planes(normalized_indices) + metadata = self.metadata.for_source_planes( + normalized_indices + ) variants = self.variant_data.project_planes( - normalized_indices, - plane_count=plane_count, + normalized_indices, plane_count=plane_count, ) return self.with_variants( variants, @@ -1483,9 +1415,7 @@ def from_payload( if isinstance(payload, ObjectLabelSet): payload.validate_artifact_name() if payload.source_image_name == "": - raise ValueError( - "ObjectLabelSet.source_image_name cannot be empty." - ) + raise ValueError("ObjectLabelSet.source_image_name cannot be empty.") variants.validate_representation( representation=ObjectLabelRepresentation(payload.representation), value_label=type(payload).__name__, @@ -1584,17 +1514,13 @@ def for_value(cls, labels: object) -> "ObjectLabelStorageStrategy": ) @classmethod - def for_planes( - cls, labels: Sequence[ObjectLabelData] - ) -> "ObjectLabelStorageStrategy": + def for_planes(cls, labels: Sequence[ObjectLabelData]) -> "ObjectLabelStorageStrategy": """Admit one homogeneous semantic plane family before stacking.""" if not labels: raise ValueError("Object-label plane stacking requires values.") authority = cls.for_value(labels[0]) if any(type(value) is not type(labels[0]) for value in labels[1:]): - raise TypeError( - "Object-label plane stacking requires one nominal storage type." - ) + raise TypeError("Object-label plane stacking requires one nominal storage type.") return authority @abstractmethod @@ -1614,15 +1540,10 @@ def dense_data( """Materialize dense labels from this storage.""" def rendering_layers( - self, - labels: object, - *, - source_spatial_shape_yx: tuple[int, int] | None, + self, labels: object, *, source_spatial_shape_yx: tuple[int, int] | None, ) -> tuple[np.ndarray, ...]: """Return nonoverlapping dense layers for image rendering consumers.""" - return ( - self.dense_data(labels, source_spatial_shape_yx=source_spatial_shape_yx), - ) + return (self.dense_data(labels, source_spatial_shape_yx=source_spatial_shape_yx),) @abstractmethod def sparse_ijv_rows(self, labels: object) -> SparseIJVLabelRows: @@ -1798,8 +1719,7 @@ def axis_centers( return super().axis_centers(labels, domain=domain) coordinate_columns = tuple(range(3 - array.ndim, 3)) sums, pixel_counts = _dense_label_coordinate_moments_numba( - coordinate_planes, - coordinate_domain.max_label, + coordinate_planes, coordinate_domain.max_label, ) del coordinate_domain # Like sparse conversion, the moments snapshot precedes domain callbacks. @@ -1902,10 +1822,7 @@ def dense_data( ) def rendering_layers( - self, - labels: object, - *, - source_spatial_shape_yx: tuple[int, int] | None, + self, labels: object, *, source_spatial_shape_yx: tuple[int, int] | None, ) -> tuple[np.ndarray, ...]: return cast(SparseIJVLabelRows, labels).nonoverlapping_dense_layers( source_spatial_shape_yx=source_spatial_shape_yx, @@ -2027,17 +1944,13 @@ def dense_data( ) def rendering_layers( - self, - labels: object, - *, - source_spatial_shape_yx: tuple[int, int] | None, + self, labels: object, *, source_spatial_shape_yx: tuple[int, int] | None, ) -> tuple[np.ndarray, ...]: del source_spatial_shape_yx value = cast(ObjectLabelValue, labels) data = self.label_data(value) return ObjectLabelStorageStrategy.for_value(data).rendering_layers( - data, - source_spatial_shape_yx=value.source_spatial_shape_yx, + data, source_spatial_shape_yx=value.source_spatial_shape_yx, ) def sparse_ijv_rows(self, labels: object) -> SparseIJVLabelRows: @@ -2200,7 +2113,9 @@ def _dense_label_coordinate_moments_numba( @njit(cache=True) -def dense_label_centers_2d_numba(labels: np.ndarray, label_count: int) -> np.ndarray: +def dense_label_centers_2d_numba( + labels: np.ndarray, label_count: int +) -> np.ndarray: """Validate compiled two-dimensional geometry and return its y/x centers.""" height, width = labels.shape sums, counts = _dense_label_coordinate_moments_numba( @@ -2210,8 +2125,7 @@ def dense_label_centers_2d_numba(labels: np.ndarray, label_count: int) -> np.nda for label_id in range(label_count + 1): for axis in range(2): centers[label_id, axis] = ( - np.nan - if counts[label_id] == 0 + np.nan if counts[label_id] == 0 else sums[label_id, axis + 1] / counts[label_id] ) return centers diff --git a/openhcs/core/runtime_relationships.py b/openhcs/core/runtime_relationships.py index f366fe63eb..2d5004b8a2 100644 --- a/openhcs/core/runtime_relationships.py +++ b/openhcs/core/runtime_relationships.py @@ -893,9 +893,7 @@ def __post_init__(self) -> None: def required_slice_index(self) -> int: """Admit this object identity for a comparison requiring an explicit plane.""" if self.slice_index is None: - raise ValueError( - "Object instance requires an explicit image or slice identity." - ) + raise ValueError("Object instance requires an explicit image or slice identity.") return self.slice_index @classmethod diff --git a/openhcs/core/runtime_stores.py b/openhcs/core/runtime_stores.py index 3a0f7c0bda..e044250c3e 100644 --- a/openhcs/core/runtime_stores.py +++ b/openhcs/core/runtime_stores.py @@ -152,7 +152,9 @@ def __post_init__(self) -> None: raise ValueError( "RuntimeArtifactDynamicComponentTarget.backend cannot be empty." ) - object.__setattr__(self, "input_plan", self.input_plan.runtime_query_snapshot()) + object.__setattr__( + self, "input_plan", self.input_plan.runtime_query_snapshot() + ) def matches(self, record: "StoredRuntimeValue") -> bool: scope = record.key.scope @@ -364,9 +366,7 @@ def selects_complete_producer(self) -> bool: producer_scope = storage_plan.producer_group_scope() selection_scope = projection.producer_selection_scope return ( - projection.selects_declared_complete_producer( - self.edge_plan.spec, storage_plan - ) + projection.selects_declared_complete_producer(self.edge_plan.spec, storage_plan) or storage_plan.composes_producer_groups( ComponentSet.of(projection.consumer_variable_components) ) @@ -399,9 +399,7 @@ def candidate_execution_scopes( identity_policy = self._source_context_identity_policy() candidates = tuple( record - for group_key in ( - (None,) if producer_scope.is_dynamic else producer_scope.keys - ) + for group_key in ((None,) if producer_scope.is_dynamic else producer_scope.keys) for record in store.find_matching( RuntimeArtifactQuery.from_input_plan( storage_plan, @@ -416,9 +414,8 @@ def candidate_execution_scopes( record_scope = record.key.scope if record_scope.component is not producer_scope.component: continue - if ( - not selects_complete_producer - and not selection_scope.contains_runtime_key(record_scope.value_text) + if not selects_complete_producer and not selection_scope.contains_runtime_key( + record_scope.value_text ): continue record_coordinates = dict(record_scope.source_component_values) @@ -426,13 +423,10 @@ def candidate_execution_scopes( record_coordinates.pop(producer_scope.component, None) if projected_components: provenance = record.data.metadata.source_provenance - plane_metadata = ( - provenance.source_image_provenance_planes.runtime_component_metadata - ) + plane_metadata = provenance.source_image_provenance_planes.runtime_component_metadata metadata_rows = plane_metadata or ( (provenance.source_component_metadata,) - if provenance.source_component_metadata is not None - else () + if provenance.source_component_metadata is not None else () ) else: metadata_rows = (None,) @@ -444,8 +438,7 @@ def candidate_execution_scopes( component_scope = projection.component_scope(component) if component_scope.is_dynamic: component_value = semantic_source_metadata_value( - metadata, - component.name, + metadata, component.name, ) if component_value is None: break @@ -453,19 +446,12 @@ def candidate_execution_scopes( else: component_value = component_scope.select_runtime_key(None) if not SourceAxisMetadataScope.constraint_matches_metadata( - metadata, - component.name, - component_value, + metadata, component.name, component_value, ): break existing_value = coordinates.get(component) - if ( - existing_value is not None - and not SourceAxisMetadataScope.constraint_matches_metadata( - metadata, - component.name, - existing_value, - ) + if existing_value is not None and not SourceAxisMetadataScope.constraint_matches_metadata( + metadata, component.name, existing_value, ): break coordinates.setdefault(component, component_value) @@ -483,10 +469,7 @@ def candidate_execution_scopes( ): continue fixed_group_value = coordinates.get(execution_scope.component) - if ( - fixed_group_value is not None - and fixed_group_value != component_key - ): + if fixed_group_value is not None and fixed_group_value != component_key: continue fixed_values = tuple( (component, value) @@ -771,7 +754,8 @@ def _matches_execution_scope( ) identity_policy = self._source_context_identity_policy() context_components = ( - context_components.intersection(record_scope.source_components) + context_components + .intersection(record_scope.source_components) .intersection(self.axis_scope.source_components) .intersection(ComponentSet(identity_policy.identity_components())) ) @@ -782,12 +766,10 @@ def _matches_execution_scope( return True return SourceImageSetIdentityCompatibility( record_scope.source_image_set_identity( - identity_policy, - components=context_components, + identity_policy, components=context_components, ), self.axis_scope.source_image_set_identity( - identity_policy, - components=context_components, + identity_policy, components=context_components, ), ).matches() @@ -1006,9 +988,8 @@ def from_dict(cls, data: Mapping[str, Any]) -> "RuntimeArtifactAddress": artifact_type=ArtifactType.coerce(str(key["artifact_type"])), scope=RuntimeExecutionAxisScope.from_raw( str(scope["axis_id"]), - component=( - None if component is None else family.named(str(component)) - ), + component=None if component is None else family.named(str(component)), + value=None if value is None else str(value), fixed_component_values=tuple(fixed_values), ), @@ -1276,9 +1257,7 @@ def values(self) -> tuple[StoredRuntimeValue, ...]: @property def observed_values(self) -> tuple[StoredRuntimeValue, ...]: """Return retained runtime artifact writes in their original order.""" - return tuple( - record for record in self._observation_records if record is not None - ) + return tuple(record for record in self._observation_records if record is not None) def observation_cursor(self) -> RuntimeStoreObservationCursor: """Return a cursor for future observation-delta queries.""" @@ -1299,8 +1278,7 @@ def observed_values_after( f"{len(self._observation_records)}." ) return tuple( - record - for record in self._observation_records[cursor.index :] + record for record in self._observation_records[cursor.index :] if record is not None ) @@ -1318,9 +1296,7 @@ def release_unconsumed( their observation history. Empty stream slots mark retired writes, not fabricated runtime values. """ - retained_addresses = { - (record.key, record.location) for record in retained_records - } + retained_addresses = {(record.key, record.location) for record in retained_records} retained_inputs = { (ref.artifact_type, ref.name) for ref in retained_refs @@ -1328,10 +1304,10 @@ def release_unconsumed( } def retained(record: StoredRuntimeValue) -> bool: - return (record.key.artifact_type, record.key.name) in retained_inputs or ( - record.key, - record.location, - ) in retained_addresses + return ( + (record.key.artifact_type, record.key.name) in retained_inputs + or (record.key, record.location) in retained_addresses + ) retired = tuple( record for record in self.observed_values if not retained(record) @@ -1339,15 +1315,11 @@ def retained(record: StoredRuntimeValue) -> bool: if not retired: return live_locations = { - record.location - for record in self._records_by_location.values() + record.location for record in self._records_by_location.values() if retained(record) } for location in dict.fromkeys(record.location for record in retired): - if ( - location not in live_locations - and location.backend == Backend.MEMORY.value - ): + if location not in live_locations and location.backend == Backend.MEMORY.value: if filemanager.exists(location.path, location.backend): filemanager.delete(location.path, location.backend) for address, record in tuple(self._records_by_location.items()): diff --git a/openhcs/core/source_binding_context.py b/openhcs/core/source_binding_context.py index 8e468fca91..b0ea1f1a06 100644 --- a/openhcs/core/source_binding_context.py +++ b/openhcs/core/source_binding_context.py @@ -26,9 +26,7 @@ class SourceBindingContext: source_backend: str def __post_init__(self) -> None: - object.__setattr__( - self, "display_dataset_root", Path(self.display_dataset_root) - ) + object.__setattr__(self, "display_dataset_root", Path(self.display_dataset_root)) object.__setattr__( self, "execution_dataset_root", Path(self.execution_dataset_root) ) diff --git a/openhcs/core/source_binding_selection.py b/openhcs/core/source_binding_selection.py index 273f770c88..aa74db615a 100644 --- a/openhcs/core/source_binding_selection.py +++ b/openhcs/core/source_binding_selection.py @@ -69,9 +69,7 @@ if TYPE_CHECKING: from openhcs.core.runtime_adapters import RuntimeAdapterRequest - from openhcs.core.runtime_source_binding_cache import ( - RuntimeSourceResolutionSnapshot, - ) + from openhcs.core.runtime_source_binding_cache import RuntimeSourceResolutionSnapshot from polystore.filemanager import FileManager from openhcs.core.context.processing_context import ProcessingContext from openhcs.core.dataset_sources.interfaces import FilenameParser @@ -290,8 +288,7 @@ def _candidate_path_resolution( for path, declared in self.source_projections_by_virtual_path.items() if declared is projection ) - if projection is not None - else (exact_virtual_path,) + if projection is not None else (exact_virtual_path,) ) else: virtual_matches = tuple( @@ -386,8 +383,7 @@ def declared_positions_for_candidates( projection = self.source_projections_by_virtual_path.get(position) positions.append( projection_positions[id(projection)] - if projection is not None - else position + if projection is not None else position ) return tuple(dict.fromkeys(positions)) @@ -562,9 +558,7 @@ def matches( if not selector.components and not selector.metadata: return True - return selector.metadata_candidates_match( - source_context.candidate_metadata(candidate) - ) + return selector.metadata_candidates_match(source_context.candidate_metadata(candidate)) @classmethod def compatible_candidates( @@ -1135,8 +1129,7 @@ def matching_candidates_for_source_identities( if self.resolution_snapshot is not None: return self.resolution_snapshot.matching_candidates_for_source_identities( - identities, - candidates, + identities, candidates, ) candidates = self.declared_positions_for_candidates(candidates) @@ -1239,9 +1232,7 @@ def expand( if not resolution_bindings: # No alias selects files: the candidate matcher keeps every anchor. return SourceBindingCandidateMatcher.compatible_candidates( - anchors, - bindings=(), - source_context=self, + anchors, bindings=(), source_context=self, ) return tuple( dict.fromkeys( @@ -1558,60 +1549,43 @@ def load_images( if self.backend is Backend.MEMORY: return [ ImagePayload.of(value) - for value in filemanager.load_batch( - list(self.files), self.backend.value - ) + for value in filemanager.load_batch(list(self.files), self.backend.value) ] - missing = tuple( - dict.fromkeys( - path - for path in self.files - if not filemanager.exists(path, Backend.MEMORY.value) - ) - ) + missing = tuple(dict.fromkeys( + path for path in self.files + if not filemanager.exists(path, Backend.MEMORY.value) + )) loaded_by_path = {} if missing: pixels = filemanager.load_batch( - list(missing), - self.backend.value, - **( - {"zarr_config": zarr_config} if self.backend is Backend.ZARR else {} - ), + list(missing), self.backend.value, + **({"zarr_config": zarr_config} if self.backend is Backend.ZARR else {}), ) loaded_by_path.update( - ( - path, - ImagePayloadSourceMetadataContext( - SourceImageIdentity(path), - read_backend=self.backend.value, - filemanager=filemanager, - ).payload(image), - ) + (path, ImagePayloadSourceMetadataContext( + SourceImageIdentity(path), + read_backend=self.backend.value, + filemanager=filemanager, + ).payload(image)) for path, image in zip(missing, pixels, strict=True) ) for parent in dict.fromkeys(str(Path(path).parent) for path in missing): filemanager.ensure_directory(parent, Backend.MEMORY.value) filemanager.save_batch( - list(loaded_by_path.values()), - list(loaded_by_path), - Backend.MEMORY.value, + list(loaded_by_path.values()), list(loaded_by_path), Backend.MEMORY.value, ) - retained = tuple( - dict.fromkeys(path for path in self.files if path not in loaded_by_path) - ) + retained = tuple(dict.fromkeys( + path for path in self.files if path not in loaded_by_path + )) if retained: - loaded_by_path.update( - zip( - retained, - ( - ImagePayload.of(value) - for value in filemanager.load_batch( - list(retained), Backend.MEMORY.value - ) - ), - strict=True, - ) - ) + loaded_by_path.update(zip( + retained, + ( + ImagePayload.of(value) + for value in filemanager.load_batch(list(retained), Backend.MEMORY.value) + ), + strict=True, + )) return [loaded_by_path[path] for path in self.files] @@ -1692,9 +1666,7 @@ def from_context( @classmethod def source_artifact_payload( - cls, - request: RuntimeAdapterRequest, - binding: NamedSourceBinding, + cls, request: RuntimeAdapterRequest, binding: NamedSourceBinding, ) -> object: """Resolve original source pixels in this origin's workspace universe.""" @@ -1756,9 +1728,7 @@ def source_artifact_payload( ) projected_payloads = projection.load_binding_payloads( - members, - binding=binding, - filemanager=request.context.filemanager, + members, binding=binding, filemanager=request.context.filemanager, ) payload = ( projected_payloads[0] @@ -1804,8 +1774,7 @@ def admit_source_artifact_cohort( ) metadata = payload.metadata domain = source_binding_plan.source_spatial_domain.with_source_cohort( - metadata.source_spatial_domain, - depth=member_count, + metadata.source_spatial_domain, depth=member_count, ) return metadata.replace_fields(source_spatial_domain=domain).attach_to(payload) @@ -1962,17 +1931,13 @@ class StepInputSourceUniverseRequest(SourceUniverseRequest): @classmethod def source_artifact_payload( - cls, - request: RuntimeAdapterRequest, - binding: NamedSourceBinding, + cls, request: RuntimeAdapterRequest, binding: NamedSourceBinding, ) -> object: """Resolve primary planes from current pixels; companions from source.""" if not binding.requires_current_pixels: return SourceUniverseRequest.source_artifact_payload(request, binding) if request.source_payload is None: - raise ValueError( - f"STEP_INPUT binding {binding.alias!r} requires current pixels." - ) + raise ValueError(f"STEP_INPUT binding {binding.alias!r} requires current pixels.") return binding.project_step_input_payload(request.source_payload) def source_universe(self) -> SourceFileUniverse: diff --git a/openhcs/core/source_binding_workspace.py b/openhcs/core/source_binding_workspace.py index 2feaf8c66e..794963d1bc 100644 --- a/openhcs/core/source_binding_workspace.py +++ b/openhcs/core/source_binding_workspace.py @@ -67,9 +67,7 @@ from openhcs.core.axes import AxisFamily, ColourAxis, PartitionAxis if TYPE_CHECKING: - from openhcs.core.source_workspace_projection import ( - VirtualWorkspaceSourceProjection, - ) + from openhcs.core.source_workspace_projection import VirtualWorkspaceSourceProjection @dataclass(frozen=True, slots=True) @@ -649,39 +647,26 @@ def accept_prepared_projection( VirtualWorkspacePathLookup.from_paths(path, path) ) filter_paths = ( - () - if metadata is None - else SourceMetadataFields.source_filter_paths(metadata) + () if metadata is None else SourceMetadataFields.source_filter_paths(metadata) ) or (ref.backend_address, path) if self.source_bindings.source_path_filters_match(filter_paths): selected_paths.add(path) selected_metadata_paths.update((path, ref.backend_address)) if not selected_paths: - raise ValueError( - "Declared source filters matched no prepared workspace sources." - ) + raise ValueError("Declared source filters matched no prepared workspace sources.") return VirtualWorkspaceSourceProjection( - source_refs_by_virtual_path=MappingProxyType( - { - path: ref - for path, ref in projection.source_refs_by_virtual_path.items() - if path in selected_paths - } - ), - source_metadata_by_path=MappingProxyType( - { - path: metadata - for path, metadata in projection.source_metadata_by_path.items() - if path in selected_metadata_paths - } - ), - source_projections_by_virtual_path=MappingProxyType( - { - path: source - for path, source in projection.source_projections_by_virtual_path.items() - if path in selected_paths - } - ), + source_refs_by_virtual_path=MappingProxyType({ + path: ref for path, ref in projection.source_refs_by_virtual_path.items() + if path in selected_paths + }), + source_metadata_by_path=MappingProxyType({ + path: metadata for path, metadata in projection.source_metadata_by_path.items() + if path in selected_metadata_paths + }), + source_projections_by_virtual_path=MappingProxyType({ + path: source for path, source in projection.source_projections_by_virtual_path.items() + if path in selected_paths + }), workspace_root=projection.workspace_root, ) @@ -1038,14 +1023,21 @@ def candidate_matches_binding( ): return False selector = binding.selector - return all( - (value := semantic_source_metadata_value(candidate.metadata, item.field)) - is not None - and source_metadata_values_equal(value, item.value) - for item in selector.metadata - ) and all( - self._component_is_compatible(candidate.metadata, item) - for item in selector.components + return ( + all( + ( + value := semantic_source_metadata_value( + candidate.metadata, item.field + ) + ) + is not None + and source_metadata_values_equal(value, item.value) + for item in selector.metadata + ) + and all( + self._component_is_compatible(candidate.metadata, item) + for item in selector.components + ) ) @staticmethod @@ -1212,16 +1204,14 @@ def _source_set_projections( address = OpenHCSPlaneAddress( ( axis, - ( - well - if issubclass(axis, PartitionAxis) - else SourceAxisProjection.project_bound( - axis, - set_metadata=source_set.metadata, - set_index=source_set.index, - candidate_metadata=candidate.metadata, - candidate_index=projection_index, - ) + well + if issubclass(axis, PartitionAxis) + else SourceAxisProjection.project_bound( + axis, + set_metadata=source_set.metadata, + set_index=source_set.index, + candidate_metadata=candidate.metadata, + candidate_index=projection_index, ), ) for axis in AxisFamily.active().axes @@ -1268,12 +1258,12 @@ def _source_set_well(self, source_set: _SourceSet) -> str: partition_axis = AxisFamily.active().partition_axis() fields_are_partition_identity = all( - source_metadata_component(field) is partition_axis for field in fields + source_metadata_component(field) + is partition_axis + for field in fields ) if fields_are_partition_identity: - partition_value = SourceMetadataFields.component_value( - group_metadata, partition_axis - ) + partition_value = SourceMetadataFields.component_value(group_metadata, partition_axis) if partition_value is None: raise ValueError( f"Source grouping fields declared {partition_axis.name} identity " @@ -1437,9 +1427,7 @@ def _projected_candidate_components( value for candidate in candidates if ( - value := SourceMetadataFields.component_value( - candidate.metadata, component - ) + value := SourceMetadataFields.component_value(candidate.metadata, component) ) is not None } diff --git a/openhcs/core/source_bindings.py b/openhcs/core/source_bindings.py index b07a08bf36..e688e378b0 100644 --- a/openhcs/core/source_bindings.py +++ b/openhcs/core/source_bindings.py @@ -644,9 +644,7 @@ class SourceSelector: This filters candidates; it does not assign the output identity. """ - metadata: Annotated[ - tuple[MetadataSelector, ...], FieldLabel("Select Metadata") - ] = () + metadata: Annotated[tuple[MetadataSelector, ...], FieldLabel("Select Metadata")] = () """Exact metadata field/value pairs required on selected sources, e.g. Well=A01. This filters candidates; pairing sources into sets uses the match plan. @@ -709,15 +707,12 @@ def metadata_candidates_match( any( source_metadata_values_equal(value, selector.value) for metadata in candidates - for value in source_component_metadata_values( - metadata, selector.component - ) + for value in source_component_metadata_values(metadata, selector.component) ) for selector in self.components ) and all( any( - value is not None - and source_metadata_values_equal(value, selector.value) + value is not None and source_metadata_values_equal(value, selector.value) for metadata in candidates for value in (semantic_source_metadata_value(metadata, selector.field),) ) @@ -728,12 +723,12 @@ def matches_provenance(self, provenance: SourceImageProvenance) -> bool: """Apply the same selectors to identities carried by current pixels.""" identities = provenance.represented_source_identities return ( - not self.filters - or self.path_filters_match( + (not self.filters or self.path_filters_match( tuple(path for identity in identities for path in identity.filter_paths) + )) + and self.metadata_candidates_match( + tuple(identity.component_metadata or {} for identity in identities) ) - ) and self.metadata_candidates_match( - tuple(identity.component_metadata or {} for identity in identities) ) @@ -1147,32 +1142,24 @@ def project_step_input_payload(self, payload: RuntimeArrayData) -> RuntimeArrayD metadata = payload.metadata provenance = metadata.source_provenance if not provenance.has_values: - raise ValueError( - f"STEP_INPUT binding {self.alias!r} requires source provenance." - ) + raise ValueError(f"STEP_INPUT binding {self.alias!r} requires source provenance.") if not provenance.source_plane_count: if not self.selector.matches_provenance(provenance): - raise ValueError( - f"STEP_INPUT binding {self.alias!r} selects no current planes." - ) + raise ValueError(f"STEP_INPUT binding {self.alias!r} selects no current planes.") selected = payload else: named_planes = provenance.source_plane_selection(self.alias) candidate_planes = ( range(provenance.source_plane_count) - if named_planes is None - else named_planes + if named_planes is None else named_planes ) selection = tuple( - index - for index in candidate_planes + index for index in candidate_planes if self.selector.matches_provenance(provenance.for_source_plane(index)) ) if not selection: - raise ValueError( - f"STEP_INPUT binding {self.alias!r} selects no current planes." - ) + raise ValueError(f"STEP_INPUT binding {self.alias!r} selects no current planes.") selected = metadata.project_source_planes(payload, selection) return self.apply_loaded_payload(selected, source_context=None) @@ -1193,10 +1180,11 @@ def _monochrome_source_data( """Convert declared RGB channels using the source intensity domain.""" import numpy as np - normalized = metadata.payload_with(data).normalize_intensity_payload( - dtype=np.float32, - ) - rgb = np.moveaxis(np.asarray(normalized.data), channel_axis, -1)[..., :3] + + normalized = metadata.payload_with(data).normalize_intensity_payload(dtype=np.float32,) + rgb = np.moveaxis( + np.asarray(normalized.data), channel_axis, -1 + )[..., :3] if np.all(rgb == rgb[..., :1]): return np.ascontiguousarray(rgb[..., 0]) @@ -1228,7 +1216,9 @@ def component_values( values.extend(source_component_metadata_values(metadata, component)) return tuple(dict.fromkeys(values)) - def _declared_component_values(self, component: type[Axis]) -> tuple[str, ...]: + def _declared_component_values( + self, component: type[Axis] + ) -> tuple[str, ...]: """Resolve exact identity values before selector-derived coordinates.""" for declarations in (self.component_identity, self.selector.components): values = tuple( @@ -1252,9 +1242,7 @@ def component_domains( for values in (self._declared_component_values(component),) if values } - if realized_source_metadata is None or len(domains) == len( - AxisFamily.active().axes - ): + if realized_source_metadata is None or len(domains) == len(AxisFamily.active().axes): return domains realized: dict[type[Axis], list[str]] = {} for metadata in realized_source_metadata: @@ -2330,8 +2318,7 @@ def from_config( return cls( bindings=( config.binding_declarations - if selected_bindings is None - else selected_bindings + if selected_bindings is None else selected_bindings ), metadata_rules=config.metadata_rule_declarations, match_plan=config.match_plan, @@ -2358,23 +2345,18 @@ def from_contracts( and any(contract.accepts_implicit_main_flow_input for contract in contracts) else () ) - source_specs = tuple( - dict.fromkeys( - ( - *implicit_specs, - *( - spec - for contract in contracts - for spec in contract.artifact_inputs - if config.declares_artifact_ref(spec.ref()) - or available_artifacts.by_name_and_artifact_type( - spec.name, spec.artifact_type - ) - is not None - ), - ) - ) - ) + source_specs = tuple(dict.fromkeys(( + *implicit_specs, + *( + spec + for contract in contracts + for spec in contract.artifact_inputs + if config.declares_artifact_ref(spec.ref()) + or available_artifacts.by_name_and_artifact_type( + spec.name, spec.artifact_type + ) is not None + ), + ))) if not source_specs: return cls.empty() bindings = config.bindings_for_artifact_specs(source_specs, available_artifacts) @@ -2549,6 +2531,7 @@ def _coerce_component(value: Any, field_name: str) -> type[Axis]: raise TypeError(f"{field_name}: {error}") from error + def _require_name(value: str, field_name: str) -> None: if not value: raise ValueError(f"{field_name} cannot be empty.") diff --git a/openhcs/core/source_image_provenance.py b/openhcs/core/source_image_provenance.py index f063cf90fa..3db63f50f2 100644 --- a/openhcs/core/source_image_provenance.py +++ b/openhcs/core/source_image_provenance.py @@ -250,11 +250,7 @@ def from_mapping( raise ValueError(f"Unknown provenance plane fields: {sorted(unknown)!r}.") kind = values.get( "identity_kind", - ( - RuntimeSourceImageProvenancePlane.identity_kind - if identities is None - else None - ), + RuntimeSourceImageProvenancePlane.identity_kind if identities is None else None, ) if not isinstance(kind, str) or kind not in cls.__registry__: raise ValueError(f"Unknown source-image provenance identity kind {kind!r}.") @@ -269,9 +265,7 @@ def from_mapping( raise TypeError("Source provenance contributors must be a sequence.") if identities is None: facts = { - key: values[key] - for key in ("path", "component_metadata") - if key in values + key: values[key] for key in ("path", "component_metadata") if key in values } else: index = values.get("identity") @@ -554,14 +548,11 @@ def from_mapping(cls, values: object) -> "SourceImageProvenancePlanes": else: identities = None planes = values - if not isinstance(planes, Sequence) or isinstance( - planes, (str, bytes, bytearray) - ): + if not isinstance(planes, Sequence) or isinstance(planes, (str, bytes, bytearray)): raise TypeError("Source provenance planes must be a sequence.") return cls( tuple( - SourceImageProvenancePlane.from_mapping(item, identities) - for item in planes + SourceImageProvenancePlane.from_mapping(item, identities) for item in planes ) ) @@ -1088,15 +1079,11 @@ def required_scalar_components( ): continue values = frozenset( - value - for values in contributor_components - for value in values[component] + value for values in contributor_components for value in values[component] ) if len(values) > 1: reduced.add(component.name) - return tuple( - component for component in component_order if component not in reduced - ) + return tuple(component for component in component_order if component not in reduced) def image_set_identities( self, @@ -1121,9 +1108,11 @@ def image_set_plane_identities( ) -> tuple[frozenset[SourceImageSetIdentity], ...]: """Return one image-set identity entry for every runtime source plane.""" return tuple( - self.source_image_provenance_planes.plane(index) - .source_identity.with_missing_from(self.source_identity) - .image_set_identities(policy) + self.source_image_provenance_planes.plane( + index + ).source_identity.with_missing_from( + self.source_identity + ).image_set_identities(policy) for index in range(self.source_plane_count) ) diff --git a/openhcs/core/source_metadata.py b/openhcs/core/source_metadata.py index 91518e78fa..27eb7b08a4 100644 --- a/openhcs/core/source_metadata.py +++ b/openhcs/core/source_metadata.py @@ -218,9 +218,7 @@ def provenance_identity_items( return cls._view( metadata, "provenance_identity_items", - lambda: tuple( - (key, repr(value)) for key, value in cls.identity_items(metadata) - ), + lambda: tuple((key, repr(value)) for key, value in cls.identity_items(metadata)), ) @staticmethod @@ -258,12 +256,10 @@ def literal_field_types( value_types = types_by_name.setdefault(name, set()) if value is not None: value_types.add(type(value)) - return MappingProxyType( - { - name: next(iter(value_types)) if len(value_types) == 1 else None - for name, value_types in types_by_name.items() - } - ) + return MappingProxyType({ + name: next(iter(value_types)) if len(value_types) == 1 else None + for name, value_types in types_by_name.items() + }) @classmethod def component_value( @@ -302,13 +298,11 @@ def component_values( cls, metadata: SourceMetadataMapping, component: type[Axis] ) -> tuple[str, ...]: """Read only the requested component from the current field admission.""" - return tuple( - dict.fromkeys( - str(value) - for owner, value in cls._ordered_component_fields(metadata) - if owner is component - ) - ) + return tuple(dict.fromkeys( + str(value) + for owner, value in cls._ordered_component_fields(metadata) + if owner is component + )) @classmethod def component_domains( @@ -544,8 +538,7 @@ def _derived_view( @property def _accepted_components_complete(self) -> bool: return self._cacheable and all( - self._views.get(component) is not None - for component in AxisFamily.active().axes + self._views.get(component) is not None for component in AxisFamily.active().axes ) def _literal_value(self, key: str) -> SourceMetadataScalar: @@ -863,9 +856,7 @@ def __new__( name: str, physical_projection: Callable[["SourceVoxelSpacing"], float | None], native_unit: str, - physical_coordinates: Callable[ - ["SourceVoxelSpacing"], tuple[float, ...] | None - ], + physical_coordinates: Callable[["SourceVoxelSpacing"], tuple[float, ...] | None], ): member = str.__new__(cls, name) member._value_ = name @@ -877,9 +868,7 @@ def __new__( def physical_pixel_size(self, spacing: "SourceVoxelSpacing") -> float | None: return self._physical_projection(spacing) - def physical_coordinates( - self, spacing: "SourceVoxelSpacing" - ) -> tuple[float, ...] | None: + def physical_coordinates(self, spacing: "SourceVoxelSpacing") -> tuple[float, ...] | None: """Project coordinates only when this declaration establishes physical units.""" return self._physical_coordinates(spacing) @@ -1131,7 +1120,9 @@ def source_metadata_component(field: str) -> type[Axis] | None: @lru_cache(maxsize=1024) -def _axis_for_metadata_field(family: type[AxisFamily], field: str) -> type[Axis] | None: +def _axis_for_metadata_field( + family: type[AxisFamily], field: str +) -> type[Axis] | None: normalized = source_metadata_field_identity(field) owners = tuple( axis diff --git a/openhcs/core/source_workspace_projection.py b/openhcs/core/source_workspace_projection.py index fcc19366bf..b56d16fcd2 100644 --- a/openhcs/core/source_workspace_projection.py +++ b/openhcs/core/source_workspace_projection.py @@ -33,10 +33,7 @@ source_metadata_values_equal, ) from openhcs.core.source_path_identity import source_path_identity_key -from openhcs.core.source_projection import ( - SourceProjection, - SourceProjectionMetadataSerializer, -) +from openhcs.core.source_projection import SourceProjection, SourceProjectionMetadataSerializer from openhcs.core.virtual_workspace_metadata import ( OpenHCSMetadataPayload, OpenHCSMetadataSubdirectories, @@ -443,20 +440,17 @@ def load_binding_payloads( f"Workspace projection for {path!r} does not match compiled " f"source artifact {binding.input_spec().ref()!r}." ) - projected_payloads.append( - binding.apply_loaded_payload( - self.project_payload(lookup, payload), - ImagePayloadSourceMetadataContext( - SourceImageIdentity( - self.logical_path_for(lookup), - self.source_metadata_for(lookup), - ), - projection.ref.backend, - filemanager, - projection.ref.backend_address, + projected_payloads.append(binding.apply_loaded_payload( + self.project_payload(lookup, payload), + ImagePayloadSourceMetadataContext( + SourceImageIdentity( + self.logical_path_for(lookup), self.source_metadata_for(lookup), ), - ) - ) + projection.ref.backend, + filemanager, + projection.ref.backend_address, + ), + )) return tuple(projected_payloads) def validate_runtime_metadata_projection( @@ -531,21 +525,15 @@ def relative_virtual_paths(self) -> tuple[str, ...]: def component_values(self, component) -> tuple[str, ...]: """Derive the execution component domain from admitted source metadata.""" - return tuple( - dict.fromkeys( - value - for path in self.relative_virtual_paths() - for value in source_component_metadata_values( - self.source_metadata_for( - VirtualWorkspacePathLookup.from_paths( - path, self._loadable_virtual_path(path) - ) - ) - or {}, - component, - ) + return tuple(dict.fromkeys( + value + for path in self.relative_virtual_paths() + for value in source_component_metadata_values( + self.source_metadata_for(VirtualWorkspacePathLookup.from_paths( + path, self._loadable_virtual_path(path) + )) or {}, component, ) - ) + )) def filtered_by_axis( self, @@ -574,16 +562,14 @@ def partition_by_axes( ) ) values = ( - () - if metadata is None + () if metadata is None else source_component_metadata_values( metadata, AxisFamily.active().partition_axis() ) ) selected_axes = ( tuple(dict.fromkeys(value for value in values if value in source_refs)) - if values - else tuple(source_refs) + if values else tuple(source_refs) ) if not selected_axes: continue @@ -603,19 +589,17 @@ def partition_by_axes( source_projections[axis_id][virtual_path] = projection source_metadata[axis_id].update(metadata_records) - return MappingProxyType( - { - axis_id: VirtualWorkspaceSourceProjection( - source_refs_by_virtual_path=MappingProxyType(refs), - source_metadata_by_path=MappingProxyType(source_metadata[axis_id]), - source_projections_by_virtual_path=MappingProxyType( - source_projections[axis_id] - ), - workspace_root=self.workspace_root, - ) - for axis_id, refs in source_refs.items() - } - ) + return MappingProxyType({ + axis_id: VirtualWorkspaceSourceProjection( + source_refs_by_virtual_path=MappingProxyType(refs), + source_metadata_by_path=MappingProxyType(source_metadata[axis_id]), + source_projections_by_virtual_path=MappingProxyType( + source_projections[axis_id] + ), + workspace_root=self.workspace_root, + ) + for axis_id, refs in source_refs.items() + }) def _path_belongs_to_axis( self, @@ -684,7 +668,9 @@ def apply(self, payload: RuntimeArrayData) -> RuntimeArrayData: metadata = metadata.replace_fields( source_spatial_domain=metadata.source_spatial_domain.with_native_image_context( current_metadata.source_spatial_domain, - image_shape_yx=current_metadata.spatial_shape_yx(payload.data), + image_shape_yx=current_metadata.spatial_shape_yx( + payload.data + ), ) ) if source_metadata is not None and self.persisted_metadata is None: @@ -718,18 +704,14 @@ class VirtualWorkspaceSourceProjectionCacheEntry: metadata: OpenHCSMetadataPayload projection: VirtualWorkspaceSourceProjection | None axis_filtered_projections: dict[str, VirtualWorkspaceSourceProjection] = field( - default_factory=dict, - compare=False, - repr=False, + default_factory=dict, compare=False, repr=False, ) source_accepted_entries: dict[ - tuple[SourceFilterClause, ...], - VirtualWorkspaceSourceProjectionCacheEntry, + tuple[SourceFilterClause, ...], VirtualWorkspaceSourceProjectionCacheEntry, ] = field(default_factory=dict, compare=False, repr=False) def accepted_for( - self, - source_bindings: SourceBindingsConfig | None, + self, source_bindings: SourceBindingsConfig | None, ) -> VirtualWorkspaceSourceProjectionCacheEntry: """Bind prepared-source admission to its document and declarations.""" if self.projection is None or source_bindings is None: @@ -739,57 +721,46 @@ def accepted_for( return self admitted = self.source_accepted_entries.get(declarations) if admitted is None: - from openhcs.core.source_binding_workspace import ( - SourceBindingWorkspaceProjector, - ) + from openhcs.core.source_binding_workspace import SourceBindingWorkspaceProjector admitted = VirtualWorkspaceSourceProjectionCacheEntry( self.metadata, - SourceBindingWorkspaceProjector( - source_bindings - ).accept_prepared_projection(self.projection), + SourceBindingWorkspaceProjector(source_bindings).accept_prepared_projection( + self.projection + ), ) self.source_accepted_entries[declarations] = admitted return admitted def entry_for_projection( - self, - projection: VirtualWorkspaceSourceProjection, + self, projection: VirtualWorkspaceSourceProjection, ) -> VirtualWorkspaceSourceProjectionCacheEntry | None: """Recognize only projections retained by this admitted document.""" if self.projection is projection: return self - return next( - ( - entry - for entry in self.source_accepted_entries.values() - if entry.projection is projection - ), - None, - ) + return next(( + entry for entry in self.source_accepted_entries.values() + if entry.projection is projection + ), None) def partition_by_axes( - self, - axis_ids: Sequence[str], + self, axis_ids: Sequence[str], ) -> Mapping[str, VirtualWorkspaceSourceProjection]: """Derive missing axis views together from this retained document.""" if self.projection is None: - raise ValueError( - "Axis views require an admitted source workspace projection." - ) + raise ValueError("Axis views require an admitted source workspace projection.") selected = tuple(dict.fromkeys(axis_ids)) missing = tuple( - axis_id - for axis_id in selected + axis_id for axis_id in selected if axis_id not in self.axis_filtered_projections ) if missing: self.axis_filtered_projections.update( self.projection.partition_by_axes(missing) ) - return MappingProxyType( - {axis_id: self.axis_filtered_projections[axis_id] for axis_id in selected} - ) + return MappingProxyType({ + axis_id: self.axis_filtered_projections[axis_id] for axis_id in selected + }) @dataclass(slots=True) @@ -811,11 +782,9 @@ def projection_for( dataset_key = str(dataset_root) cached = self.projections_by_dataset_root.get(dataset_key) if cached is None or cached.metadata is not metadata: - projection = ( - VirtualWorkspaceSourceProjection.from_openhcs_metadata_if_available( - dataset_root, - metadata, - ) + projection = VirtualWorkspaceSourceProjection.from_openhcs_metadata_if_available( + dataset_root, + metadata, ) cached = VirtualWorkspaceSourceProjectionCacheEntry(metadata, projection) self.projections_by_dataset_root[dataset_key] = cached @@ -846,8 +815,7 @@ def partition_by_axes( """Share admitted axis views between compilation and runtime queries.""" document_entry = self.projections_by_dataset_root.get(projection.workspace_root) cached = ( - None - if document_entry is None + None if document_entry is None else document_entry.entry_for_projection(projection) ) if cached is None: @@ -872,10 +840,7 @@ class WorkspaceSourceProjections: cache: VirtualWorkspaceSourceProjectionCache | None = None source_bindings: SourceBindingsConfig | None = None _workspace_metadata_handler: "OpenHCSMetadataHandler | None" = field( - default=None, - init=False, - compare=False, - repr=False, + default=None, init=False, compare=False, repr=False, ) @classmethod @@ -922,10 +887,7 @@ def is_bound_to_context(self, context: "ProcessingContext") -> bool: and self.filemanager is context.filemanager ): return False - return ( - self.source_bindings - == context.microscope_handler.source_bindings_still_required() - ) + return self.source_bindings == context.microscope_handler.source_bindings_still_required() def metadata_handlers(self) -> tuple["MetadataHandler", ...]: """Observe workspace eligibility live while retaining admitted providers.""" @@ -970,32 +932,24 @@ def _projection_for_axis( return self.cache.filtered_by_axis(projection, axis_id=axis_id) def projection_if_available( - self, - *, - axis_id: str | None = None, + self, *, axis_id: str | None = None, ) -> VirtualWorkspaceSourceProjection | None: workspace_root = self.metadata_handler.source_workspace_root(self.dataset_root) for metadata in self.metadata_documents(): if self.cache is None: - projection = ( - VirtualWorkspaceSourceProjection.from_openhcs_metadata_if_available( - workspace_root, - metadata, - ) + projection = VirtualWorkspaceSourceProjection.from_openhcs_metadata_if_available( + workspace_root, + metadata, ) if projection is not None and self.source_bindings is not None: - from openhcs.core.source_binding_workspace import ( - SourceBindingWorkspaceProjector, - ) + from openhcs.core.source_binding_workspace import SourceBindingWorkspaceProjector projection = SourceBindingWorkspaceProjector( self.source_bindings ).accept_prepared_projection(projection) else: projection = self.cache.projection_for( - workspace_root, - metadata, - source_bindings=self.source_bindings, + workspace_root, metadata, source_bindings=self.source_bindings, ) if projection is None: continue @@ -1003,9 +957,7 @@ def projection_if_available( return None def projection_or_empty( - self, - *, - axis_id: str | None = None, + self, *, axis_id: str | None = None, ) -> VirtualWorkspaceSourceProjection: projection = self.projection_if_available(axis_id=axis_id) if projection is not None: @@ -1014,7 +966,9 @@ def projection_or_empty( @dataclass(frozen=True, slots=True) -class RuntimeWorkspaceSourceProjections(WorkspaceSourceProjections): +class RuntimeWorkspaceSourceProjections( + WorkspaceSourceProjections +): """Read completed same-plate outputs from their execution observation owner.""" context: "ProcessingContext" = field(kw_only=True, compare=False, repr=False) @@ -1025,9 +979,7 @@ def is_bound_to_context(self, context: "ProcessingContext") -> bool: ).is_bound_to_context(context) def projection_if_available( - self, - *, - axis_id: str | None = None, + self, *, axis_id: str | None = None, ) -> VirtualWorkspaceSourceProjection | None: projection = super( RuntimeWorkspaceSourceProjections, self @@ -1043,8 +995,7 @@ def projection_if_available( ) if not produced_entries: return ( - None - if projection is None + None if projection is None else self._projection_for_axis(projection, axis_id=axis_id) ) builder = VirtualWorkspaceSourceProjectionBuilder(self.dataset_root) @@ -1058,9 +1009,7 @@ def projection_if_available( ) ) builder.ingest_source_metadata( - VirtualWorkspaceSourceMetadataEntries( - projection.source_metadata_by_path - ) + VirtualWorkspaceSourceMetadataEntries(projection.source_metadata_by_path) ) for entries in produced_entries: fields = SourceProjectionMetadataSerializer.workspace_fields( diff --git a/openhcs/core/steps/function_artifact_materialization.py b/openhcs/core/steps/function_artifact_materialization.py index f53da132ba..1dd7a13372 100644 --- a/openhcs/core/steps/function_artifact_materialization.py +++ b/openhcs/core/steps/function_artifact_materialization.py @@ -177,38 +177,27 @@ def materialize_outputs( pipeline_position=plan.pipeline_position, output_plan=materialization.output_plan, ) - batch = replace( - batch, - context=replace( - batch.context, - backend_kwargs=BackendKwargsAuthority.normalize( - self.backend_kwargs( - materialization=materialization, - outputs=batch.outputs, - persistent_backend_kwargs=persistent_backend_kwargs, - streaming_viewer_surfaces=streaming_viewer_surfaces, - fallback_source_identity=( - materialization.filename_source_identity - if materialization.spec.uses_filename_source_identity( - data - ) - else materialization.source_identity - ), - producer_identity=( - plan.producer_identity_for_artifact( - materialization.output_plan - ) - ), - context=context, - filemanager=filemanager, - images_dir=images_dir, - stream_output_paths=tuple( - output.path for output in batch.outputs - ), - ) + batch = replace(batch, context=replace( + batch.context, + backend_kwargs=BackendKwargsAuthority.normalize(self.backend_kwargs( + materialization=materialization, + outputs=batch.outputs, + persistent_backend_kwargs=persistent_backend_kwargs, + streaming_viewer_surfaces=streaming_viewer_surfaces, + fallback_source_identity=( + materialization.filename_source_identity + if materialization.spec.uses_filename_source_identity(data) + else materialization.source_identity ), - ), - ) + producer_identity=( + plan.producer_identity_for_artifact(materialization.output_plan) + ), + context=context, + filemanager=filemanager, + images_dir=images_dir, + stream_output_paths=tuple(output.path for output in batch.outputs), + )), + )) saved_materializations.append( MaterializedRuntimeArtifact( outputs_by_backend=MappingProxyType( @@ -234,9 +223,7 @@ def streaming_viewer_surfaces( ) return ( { - config.viewer_family.backend.value: config.streaming_viewer_surface( - context - ) + config.viewer_family.backend.value: config.streaming_viewer_surface(context) for config in plan.streaming_configs.values() if step_axis_allows_config( context.step_axis_filters, @@ -320,11 +307,8 @@ def backend_kwargs( for backend, viewer_surface in streamable_viewer_surfaces.items(): backend_instance = filemanager._get_backend(backend) source_metadata_items = materialization.stream_source_metadata_items( - tuple( - output - for output in outputs - if backend_instance.accepts_payload(output.content, output.path) - ), + tuple(output for output in outputs + if backend_instance.accepts_payload(output.content, output.path)), fallback_source_identity, ) stream_backend_kwargs = StreamComponentMessageExtraAuthority.from_context( @@ -1252,7 +1236,8 @@ def observation( return StepExecutionObservation.empty() address = RuntimeArtifactAddress.from_record(self.materialization.record) locations = tuple( - RuntimeArtifactLocation.from_output(output, backend) for output in outputs + RuntimeArtifactLocation.from_output(output, backend) + for output in outputs ) paths = ( tuple(Path(output.path) for output in outputs) diff --git a/openhcs/core/steps/function_execution.py b/openhcs/core/steps/function_execution.py index 23c416408f..652463832b 100644 --- a/openhcs/core/steps/function_execution.py +++ b/openhcs/core/steps/function_execution.py @@ -523,10 +523,8 @@ def apply( def source_pattern_context(self) -> SourcePatternResolutionContext: """Return source-path context used to filter source-bound anchors.""" - projection = ( - self.context.runtime_source_workspace_projections.projection_or_empty( - axis_id=self.plan.axis_id, - ) + projection = self.context.runtime_source_workspace_projections.projection_or_empty( + axis_id=self.plan.axis_id, ) return self.context.runtime_source_binding_context_cache.source_pattern_context( parser=self.context.microscope_handler.parser, @@ -806,9 +804,7 @@ def _detect_patterns(self) -> dict[str, DiscoveredPatternCollection]: plan = self.plan source_files = step_output_manifest(self.context).producer_paths_for(plan) if source_files is None: - source_projection = ( - self.context.runtime_source_workspace_projections.projection_if_available() - ) + source_projection = self.context.runtime_source_workspace_projections.projection_if_available() if ( plan.main_input_dependency.kind is StepInputDependencyKind.PIPELINE_START diff --git a/openhcs/core/steps/function_io.py b/openhcs/core/steps/function_io.py index 3b2b2b5f1e..32e0212ee1 100644 --- a/openhcs/core/steps/function_io.py +++ b/openhcs/core/steps/function_io.py @@ -183,12 +183,12 @@ def project_item_values( "Parsed output identities disagree on component " f"{component.name!r}; missing from item indices {missing_indices!r}" ) - return tuple( - cls.item_value(identity, component) for identity in item_identities - ) + return tuple(cls.item_value(identity, component) for identity in item_identities) @classmethod - def item_value(cls, identity: ZarrBatchItemIdentity, component: type[Axis]) -> str: + def item_value( + cls, identity: ZarrBatchItemIdentity, component: type[Axis] + ) -> str: value = identity.component_values.value_for(component) if value is None: raise ValueError( @@ -216,7 +216,9 @@ class ColourZarrAxisProjection(ZarrComponentAxisProjection): axis_name = "c" @classmethod - def item_value(cls, identity: ZarrBatchItemIdentity, component: type[Axis]) -> str: + def item_value( + cls, identity: ZarrBatchItemIdentity, component: type[Axis] + ) -> str: value = super().item_value(identity, component) qualifier = identity.filename_qualifier return value if qualifier is None else f"{value}:{qualifier}" diff --git a/openhcs/core/steps/function_output_identity.py b/openhcs/core/steps/function_output_identity.py index 2f21e02e95..b0ffc46718 100644 --- a/openhcs/core/steps/function_output_identity.py +++ b/openhcs/core/steps/function_output_identity.py @@ -712,8 +712,7 @@ def _complete_identity_from_paths( identity_cache: FunctionOutputIdentityCache, ) -> FunctionOutputIdentity: if identity.extension is not None and all( - component.name in identity.component_values - for component in AxisFamily.active().axes + component.name in identity.component_values for component in AxisFamily.active().axes ): return identity for path, source in candidates: @@ -930,9 +929,7 @@ def _source_stack_semantic_component_keys( - identity_component_values ) ordered_keys = tuple( - component.name - for component in AxisFamily.active().axes - if component.name in keys + component.name for component in AxisFamily.active().axes if component.name in keys ) extra_keys = tuple(sorted(keys - frozenset(ordered_keys))) return (*ordered_keys, *extra_keys) diff --git a/openhcs/core/steps/function_outputs.py b/openhcs/core/steps/function_outputs.py index 90480d3505..7361f8a384 100644 --- a/openhcs/core/steps/function_outputs.py +++ b/openhcs/core/steps/function_outputs.py @@ -18,10 +18,7 @@ extract_key_from_class_name, ) from polystore.streaming.identity import StreamProducerIdentity -from polystore.streaming.viewer_transport import ( - ViewerStreamProducer, - ViewerStreamSourceMetadata, -) +from polystore.streaming.viewer_transport import ViewerStreamProducer, ViewerStreamSourceMetadata from polystore.virtual_workspace import SourcePixelRef from openhcs.constants.constants import Backend @@ -516,7 +513,9 @@ def partition_by_item_fields( def project_item( request: RuntimeProjectionSourceIdentityRequest, ) -> tuple[RuntimeProjectedPayloadItem, ...]: - return (RequiredSourceComponentMetadata).project_payload_items(request) + return ( + RequiredSourceComponentMetadata + ).project_payload_items(request) @staticmethod def stream_path_for_projected_item( @@ -694,8 +693,7 @@ def for_execution( for declaration in cls.__registry__.values() if (owner := declaration.from_execution(context, plan)) is not None for target in replace( - owner, - artifact_materializations=artifact_materializations, + owner, artifact_materializations=artifact_materializations, create_openhcs_metadata=plan.create_openhcs_metadata, ).production_targets(context, plan) ) @@ -713,9 +711,7 @@ def reconciliation_targets( context: ProcessingContext, *, document: OpenHCSMetadataSubdirectories | None = None, - accepted_entries: ( - Mapping[str, VirtualWorkspaceSourceProjectionEntries] | None - ) = None, + accepted_entries: Mapping[str, VirtualWorkspaceSourceProjectionEntries] | None = None, ) -> tuple[OpenHCSMetadataTarget, ...]: """Resolve destinations after runtime values have been released.""" return (self,) @@ -787,9 +783,7 @@ def write( str(self.output_dir), self.backend ) ) - return ( - metadata_writer or AtomicMetadataWriter() - ).publish_source_projection_metadata( + return (metadata_writer or AtomicMetadataWriter()).publish_source_projection_metadata( METADATA_CONFIG.metadata_path(self.dataset_root), self.sub_dir, projection_entries, @@ -823,8 +817,7 @@ def produced_projection_entries( raise ValueError("OpenHCS metadata requires a file manager.") target = type(self).from_plan(plan) if target is None or self not in replace( - target, - artifact_materializations=self.artifact_materializations, + target, artifact_materializations=self.artifact_materializations, create_openhcs_metadata=self.create_openhcs_metadata, ).production_targets(context, plan): raise ValueError("Produced metadata plan does not own this output target.") @@ -1071,19 +1064,11 @@ def finalize_completed_dataset( for target in cls.for_plan(plan): target_contexts.setdefault(target, context) - produced_by_document: dict[ - Path, - dict[OpenHCSMetadataTarget, list[VirtualWorkspaceSourceProjectionEntries]], - ] = {} + produced_by_document: dict[Path, dict[OpenHCSMetadataTarget, list[VirtualWorkspaceSourceProjectionEntries]]] = {} for observation in runtime_observations: for context_observation in observation.contexts: context = compiled_contexts[context_observation.context_key] - for ( - target, - entries, - ) in ( - context_observation.outputs.source_projection_entries_by_target.items() - ): + for target, entries in context_observation.outputs.source_projection_entries_by_target.items(): produced_by_document.setdefault( METADATA_CONFIG.metadata_path(target.dataset_root), {} ).setdefault(target, []).append(entries) @@ -1097,17 +1082,13 @@ def finalize_completed_dataset( targets_by_document.setdefault( METADATA_CONFIG.metadata_path(owner.dataset_root), {} )[owner] = context - for metadata_path in dict.fromkeys( - (*targets_by_document, *produced_by_document) - ): + for metadata_path in dict.fromkeys((*targets_by_document, *produced_by_document)): AtomicMetadataWriter().reconcile_completed_dataset( metadata_path, targets_by_document.get(metadata_path, {}), produced_entries_by_target={ target: VirtualWorkspaceSourceProjectionEntries.combine(entries) - for target, entries in produced_by_document.get( - metadata_path, {} - ).items() + for target, entries in produced_by_document.get(metadata_path, {}).items() }, ) @@ -1212,22 +1193,17 @@ def reconciliation_targets( context: ProcessingContext, *, document: OpenHCSMetadataSubdirectories | None = None, - accepted_entries: ( - Mapping[str, VirtualWorkspaceSourceProjectionEntries] | None - ) = None, + accepted_entries: Mapping[str, VirtualWorkspaceSourceProjectionEntries] | None = None, ) -> tuple[RuntimeArtifactMetadataTarget, ...]: """Use durable typed projections, without reloading cleaned artifact values.""" from openhcs.core.dataset_sources.openhcs_format import OpenHCSMetadataHandler if (document is None) != (accepted_entries is None): - raise ValueError( - "Reconciliation requires one document and its admitted entries." - ) + raise ValueError("Reconciliation requires one document and its admitted entries.") handler = OpenHCSMetadataHandler(context.filemanager) directories = ( handler.reconciliation_directories(self.dataset_root, self.backend) - if document is None - else handler.reconciliation_directories_from_document( + if document is None else handler.reconciliation_directories_from_document( self.dataset_root, self.backend, document, accepted_entries ) ) diff --git a/openhcs/core/steps/function_runtime.py b/openhcs/core/steps/function_runtime.py index 176e27a46b..cc2c741d64 100644 --- a/openhcs/core/steps/function_runtime.py +++ b/openhcs/core/steps/function_runtime.py @@ -264,10 +264,7 @@ def source_binding_plan(self) -> CompiledSourceBindingPlan: source_refs = self.invocation_source_artifact_refs if source_refs: return declared_plan.for_artifact_refs(source_refs) - if ( - self.compiled_group.runtime_domain - is RuntimeInvocationDomain.ARTIFACT_MANAGED - ): + if self.compiled_group.runtime_domain is RuntimeInvocationDomain.ARTIFACT_MANAGED: # The compiler has already traced stored artifact lineage to its # source bindings. A produced name is not itself a raw alias. return declared_plan @@ -347,9 +344,7 @@ def loaded_fixed_component_values( payload: RuntimeArrayData, ) -> RuntimeFixedComponentValues: """Capture the source-selected cohort's common fixed coordinates.""" - source_provenance = ( - payload.metadata.source_provenance.with_common_scalar_identity_from_planes() - ) + source_provenance = payload.metadata.source_provenance.with_common_scalar_identity_from_planes() common_source_metadata = source_provenance.source_component_metadata or {} variable_components = ComponentSet.of( self.execution_plan.variable_components or () @@ -418,9 +413,7 @@ def _input_relative_path(cls, input_dir: Path, matched_path: str) -> Path: def run(self) -> None: start_time = time.time() if logger.isEnabledFor(logging.DEBUG) else None plan = self.execution_plan - logger.debug( - "Processing pattern %s for axis %s", self.pattern_repr, plan.axis_id - ) + logger.debug("Processing pattern %s for axis %s", self.pattern_repr, plan.axis_id) try: load_started_at = time.perf_counter() @@ -508,9 +501,7 @@ def run(self) -> None: ) except Exception as e: logger.error( - "Error processing pattern group %s: %s", - self.pattern_repr, - e, + "Error processing pattern group %s: %s", self.pattern_repr, e, exc_info=True, ) raise ValueError( @@ -783,6 +774,7 @@ def _filter_matching_files_for_group( parser, matching_files, group_component, + component_value, ) if not filtered: @@ -1082,9 +1074,7 @@ def _project_output_slices( return tuple(zip(output_payloads, slice_contexts, strict=True)) def _unstack_main_output( - self, - processed_data: Any, - expected_count: int, + self, processed_data: Any, expected_count: int, ) -> tuple[Any, ...]: """Split undeclared main-output pixels along their leading runtime-slice axis.""" try: @@ -1245,9 +1235,7 @@ def _save_outputs( output_identity, output_context=output_context, image_metadata=output_metadata, - main_flow_plane_axis=processed_stack.plane_axis_for_output_context( - output_context - ), + main_flow_plane_axis=processed_stack.plane_axis_for_output_context(output_context), ) if output_directory_exists and context.filemanager.exists( @@ -1607,6 +1595,10 @@ def _save_artifact_value( return runtime_value.data + + + + @dataclass(frozen=True, slots=True) class FunctionCoreExecutor: """Execute one scoped callable invocation and route declared artifact I/O.""" @@ -1729,24 +1721,21 @@ def loaded_source_artifact_payload( or request.artifact_inputs.get(edge.key) is not edge or edge.storage_plan is not None or edge.main_flow_projection is None - or self.group_data.main_flow_source_binding_plan.binding_for_artifact_ref( - ref - ) + or self.group_data.main_flow_source_binding_plan.binding_for_artifact_ref(ref) != binding ): return None payload = self.invocation.convert_input( - self.group_data.main_data_stack, - self.source_memory_type, + self.group_data.main_data_stack, self.source_memory_type, ) if edge.main_flow_projection is MainFlowInputProjection.DECLARED_SOURCE_IMAGE: payload = payload.project_declared_source(ref.name) elif edge.main_flow_projection is not MainFlowInputProjection.COMPLETE_PAYLOAD: return None metadata = payload.metadata - if not metadata.persists_whole_image() and metadata.plane_axis not in ( - None, - RuntimePlaneAxis.RUNTIME_SLICE, + if ( + not metadata.persists_whole_image() + and metadata.plane_axis not in (None, RuntimePlaneAxis.RUNTIME_SLICE) ): # A remaining binding axis does not prove this source's ordered # physical members. Resolve it through its declared origin owner. @@ -1758,9 +1747,7 @@ def loaded_source_artifact_payload( member_count=member_count if member_count is not None else 1, ) projection = RuntimePlaneAxisValueProjection.from_projector( - request.plane_projection, - RuntimePlaneAxis.RUNTIME_SLICE, - (), + request.plane_projection, RuntimePlaneAxis.RUNTIME_SLICE, (), ) if projection is not None and projection.plane_index is not None: payload = RuntimeSliceProjection.value_for_slice(payload, projection) @@ -1783,8 +1770,7 @@ def declared_source_payload( stored_payload = loaded_artifact_payloads.get(source_ref) source_binding_plan = ( self.group_data.source_binding_plan - if request is None - else request.source_binding_plan + if request is None else request.source_binding_plan ) source_binding = source_binding_plan.binding_for_artifact_ref(source_ref) main_flow_edges = tuple( @@ -1815,8 +1801,7 @@ def declared_source_payload( RuntimePayload, ( self.runtime_adapter_request(primary_source_payload) - if request is None - else request + if request is None else request ).source_artifact_payload(source_ref), ) if len(main_flow_edges) != 1: diff --git a/openhcs/core/steps/stream_component_semantics.py b/openhcs/core/steps/stream_component_semantics.py index 2d4e5713a6..8df6ace26f 100644 --- a/openhcs/core/steps/stream_component_semantics.py +++ b/openhcs/core/steps/stream_component_semantics.py @@ -98,7 +98,9 @@ def payload_items_for_display( axis=metadata.plane_axis, axis_size=metadata.source_provenance.source_plane_count, ) - source_identity = RequiredSourceComponentMetadata + source_identity = ( + RequiredSourceComponentMetadata + ) return source_identity.project_payload_items( RuntimeProjectionSourceIdentityRequest( value=metadata.attach_to(payload), @@ -182,9 +184,7 @@ def _item_fields( ) colour_axis_position = metadata.axis_position(ColourAxis) if colour_axis_position is not None: - item_fields[ViewerWireField.SOURCE_CHANNEL_AXIS.value] = ( - colour_axis_position - ) + item_fields[ViewerWireField.SOURCE_CHANNEL_AXIS.value] = colour_axis_position if metadata.plane_axis is None: return item_fields @@ -730,6 +730,7 @@ def build_for_component( def supports(cls, component: str) -> bool: return component == AxisFamily.active().partition_axis().name + def domain_metadata_items(self) -> StreamComponentDomainMetadataItems: return tuple({self.component: value} for value in self.execution_axis_values) diff --git a/openhcs/core/viewer_streaming_service.py b/openhcs/core/viewer_streaming_service.py index 572763cf04..8ea8da76f6 100644 --- a/openhcs/core/viewer_streaming_service.py +++ b/openhcs/core/viewer_streaming_service.py @@ -412,7 +412,9 @@ def load_image( source_address=source_address, ).metadata(image) metadata = self.calibrated_metadata(metadata) - return metadata.payload_with(image.data, image.mask) + return metadata.payload_with( + image.data, image.mask + ) def component_metadata_by_path( self, @@ -602,9 +604,7 @@ def _wait_for_viewer_ready( f"{config.viewer_family.display_name} viewer on port {viewer.port} failed to become ready" ) - logger.info( - f"{config.viewer_family.display_name} viewer on port {viewer.port} is ready" - ) + logger.info(f"{config.viewer_family.display_name} viewer on port {viewer.port} is ready") @staticmethod def _require_viewer_settled(request: ViewerStreamingContext) -> None: @@ -843,9 +843,7 @@ def stream_rois( ) archive_metadata.append(metadata) if metadata is not None: - geometric_domain = ROIArchiveSourceMetadata.source_component_domain( - rois, metadata - ) + geometric_domain = ROIArchiveSourceMetadata.source_component_domain(rois, metadata) if geometric_domain is not None: geometric_domains[filename] = geometric_domain data_list.append(rois) @@ -924,9 +922,7 @@ def stream_rois( source_metadata_items = StreamSourceComponentMetadataItems.from_values( component_metadata for path in paths - for component_metadata in geometric_domains.get( - path, (metadata_by_path[path],) - ) + for component_metadata in geometric_domains.get(path, (metadata_by_path[path],)) ) message_authority = StreamComponentMessageExtraAuthority.from_viewer_surface( viewer_surface, @@ -948,17 +944,12 @@ def stream_rois( component_order = message_authority.layout.component_order item_fields_by_archive = tuple( ROIArchiveSourceMetadata.stream_item_fields( - rois, - metadata, - StreamImagePayloadMetadataProjector.item_fields( - metadata, component_order - ), + rois, metadata, + StreamImagePayloadMetadataProjector.item_fields(metadata, component_order), ) for rois, metadata in zip(data_list, payload_metadata, strict=True) ) - for indices in StreamImagePayloadMetadataProjector.partition_item_fields( - item_fields_by_archive - ): + for indices in StreamImagePayloadMetadataProjector.partition_item_fields(item_fields_by_archive): item_fields = item_fields_by_archive[indices[0]] stream_backend_kwargs = message_authority.viewer_backend_kwargs( producer=producer.for_indices( diff --git a/openhcs/core/virtual_workspace_metadata.py b/openhcs/core/virtual_workspace_metadata.py index 6b1a4fc007..21cd8b65cd 100644 --- a/openhcs/core/virtual_workspace_metadata.py +++ b/openhcs/core/virtual_workspace_metadata.py @@ -168,8 +168,7 @@ def update(data): ) published_entries = ( VirtualWorkspaceSourceProjectionEntries(MappingProxyType({})) - if projection_entries is None - else projection_entries + if projection_entries is None else projection_entries ).publish_into_subdirectory( subdirectory, saved_image_paths=saved_paths, @@ -225,22 +224,16 @@ def reconcile_completed_dataset( ] = MappingProxyType({}), ) -> None: """Admit and reconcile all targets from one locked plate document.""" - if ( - not produced_entries_by_target - and not Path(metadata_path).is_file() - and not any( - target.contains_outputs(context) - for target, context in target_contexts.items() - ) + if not produced_entries_by_target and not Path(metadata_path).is_file() and not any( + target.contains_outputs(context) + for target, context in target_contexts.items() ): return def update(data): data = self._ensure_subdirectories_structure(data) subdirectories = data[METADATA_CONFIG.SUBDIRECTORIES_KEY] - updates_by_directory: dict[ - str, list[VirtualWorkspaceSourceProjectionEntries] - ] = {} + updates_by_directory: dict[str, list[VirtualWorkspaceSourceProjectionEntries]] = {} for target, entries in produced_entries_by_target.items(): updates_by_directory.setdefault(target.sub_dir, []).append(entries) admitted = {} @@ -252,11 +245,7 @@ def update(data): document = OpenHCSMetadataSubdirectories(data) for name, subdirectory in document.items(): if name not in admitted: - admitted[name] = ( - VirtualWorkspaceSourceProjectionEntries.from_subdirectory( - subdirectory - ) - ) + admitted[name] = VirtualWorkspaceSourceProjectionEntries.from_subdirectory(subdirectory) for name in updates_by_directory: self._update_projection_geometry( subdirectories[name], admitted[name].entries.values() @@ -275,9 +264,7 @@ def update(data): metadata_document=data, accepted_entries=admitted.get( target.sub_dir, - VirtualWorkspaceSourceProjectionEntries( - MappingProxyType({}) - ), + VirtualWorkspaceSourceProjectionEntries(MappingProxyType({})), ), ) return data @@ -331,6 +318,8 @@ def get_metadata_path(dataset_root: str | Path) -> Path: return METADATA_CONFIG.metadata_path(dataset_root) + + @dataclass(frozen=True) class OpenHCSMetadataFields: """Field identities declared by the OpenHCS metadata contract. @@ -494,7 +483,9 @@ def from_projection_paths( projection_paths: Sequence[tuple[SourceProjection, str]], ) -> "VirtualWorkspaceSourceProjectionEntries": """Admit a producer's typed updates in persisted path order.""" - SourceProjectionSet(tuple(projection for projection, _path in projection_paths)) + SourceProjectionSet( + tuple(projection for projection, _path in projection_paths) + ) return cls( MappingProxyType( {path: projection for projection, path in projection_paths} @@ -531,8 +522,7 @@ def _accept_retained_records( for path, record in records.items(): virtual_path, projection = ( (path, self.entries[path]) - if path in self.entries - else self._projection_record(record) + if path in self.entries else self._projection_record(record) ) if virtual_path in entries: raise RuntimeError( @@ -587,8 +577,7 @@ def publish_into_subdirectory( records = self._records_by_path(subdirectory) admitted = ( self._accept_retained_records(records) - if accepted_entries is None - else accepted_entries + if accepted_entries is None else accepted_entries ) saved_set = frozenset(saved_image_paths) missing = saved_set.difference(admitted.entries) @@ -619,8 +608,7 @@ def publish_into_subdirectory( normalized_path = self._required_text(record, "virtual_path") if normalized_path in retained.entries: retained_records[normalized_path] = ( - record - if path == normalized_path + record if path == normalized_path else {**record, "virtual_path": normalized_path} ) retained_records.update( @@ -701,17 +689,12 @@ def _projection_record( source_metadata = cls._optional_metadata(record, "source_metadata") component_labels = cls._optional_component_labels(record) source_alias = cls._optional_text(record, "source_alias") - image_metadata_value = record.get( - SourcePlaneProjection.image_metadata_wire_field() - ) - if image_metadata_value is not None and not isinstance( - image_metadata_value, Mapping - ): + image_metadata_value = record.get(SourcePlaneProjection.image_metadata_wire_field()) + if image_metadata_value is not None and not isinstance(image_metadata_value, Mapping): raise TypeError("Persisted image metadata must be a mapping.") image_metadata = ( - None - if image_metadata_value is None - else ImagePayloadMetadata.from_mapping(image_metadata_value) + None if image_metadata_value is None else + ImagePayloadMetadata.from_mapping(image_metadata_value) ) if projection_role is SourceProjectionRole.PRIMARY_PLANE: projection: SourceProjection = SourcePlaneProjection( @@ -909,5 +892,6 @@ def from_subdirectory( entries.update({str(key): str(value) for key, value in labels.items()}) return cls(MappingProxyType(entries)) + def label_for(self, channel_value: SourceMetadataScalar) -> str | None: return self.entries.get(str(channel_value)) diff --git a/openhcs/demo/basic_pipeline.py b/openhcs/demo/basic_pipeline.py index 7264e3a100..e7857a4c93 100644 --- a/openhcs/demo/basic_pipeline.py +++ b/openhcs/demo/basic_pipeline.py @@ -45,7 +45,9 @@ step_2 = FunctionStep( func=create_composite, name="create_composite", - processing_config=LazyProcessingConfig(variable_components=[Microscopy.Channel]), + processing_config=LazyProcessingConfig( + variable_components=[Microscopy.Channel] + ), ) pipeline_steps.append(step_2) @@ -53,7 +55,9 @@ step_3 = FunctionStep( func=create_projection, name="Z-Stack Flattening", - processing_config=LazyProcessingConfig(variable_components=[Microscopy.ZIndex]), + processing_config=LazyProcessingConfig( + variable_components=[Microscopy.ZIndex] + ), step_materialization_config=LazyStepMaterializationConfig(), ) pipeline_steps.append(step_3) @@ -80,7 +84,9 @@ step_7 = FunctionStep( func=create_projection, name="Z-Stack Flattening", - processing_config=LazyProcessingConfig(variable_components=[Microscopy.ZIndex]), + processing_config=LazyProcessingConfig( + variable_components=[Microscopy.ZIndex] + ), napari_streaming_config=LazyNapariStreamingConfig(), fiji_streaming_config=LazyFijiStreamingConfig(), ) diff --git a/openhcs/domains/microscopy/analysis_consolidation.py b/openhcs/domains/microscopy/analysis_consolidation.py index 7906e2c9b8..4d4a5c4e0a 100644 --- a/openhcs/domains/microscopy/analysis_consolidation.py +++ b/openhcs/domains/microscopy/analysis_consolidation.py @@ -78,9 +78,7 @@ def from_saved_outputs( return None backend = plan.runtime_artifact_materialization.require_persistent_backend() return cls._from_table_contents( - config, - plan, - saved.materialization, + config, plan, saved.materialization, ( (Path(output.path), output.require_text_content()) for output in saved.outputs_for_backend(backend) @@ -105,9 +103,7 @@ def from_reused_outputs( return None backend = plan.runtime_artifact_materialization.require_persistent_backend() return cls._from_table_contents( - config, - plan, - materialization, + config, plan, materialization, ( (Path(output.path), context.filemanager.load_text(output.path, backend)) for output in outputs @@ -134,12 +130,10 @@ def _from_table_contents( return None backend = plan.runtime_artifact_materialization.require_persistent_backend() destination = RuntimeAnalysisSummaryDestination( - backend=backend, - images_dir=plan.artifact_images_dir, + backend=backend, images_dir=plan.artifact_images_dir, ) output_groups: dict[ - tuple[Path, RuntimeAnalysisSummaryDestination], - list[RuntimeAnalysisTableOutput], + tuple[Path, RuntimeAnalysisSummaryDestination], list[RuntimeAnalysisTableOutput] ] = {} for output_path, content in contents: output_groups.setdefault((output_path.parent, destination), []).append( @@ -162,8 +156,7 @@ def combine( ) -> RuntimeAnalysisConsolidationInputs | None: """Combine actual rendered projections without revisiting their payloads.""" output_groups: dict[ - tuple[Path, RuntimeAnalysisSummaryDestination], - list[RuntimeAnalysisTableOutput], + tuple[Path, RuntimeAnalysisSummaryDestination], list[RuntimeAnalysisTableOutput] ] = {} destinations: set[RuntimeAnalysisSummaryDestination] = set() seen_paths: set[tuple[str, Path]] = set() @@ -177,17 +170,15 @@ def combine( if identity in seen_paths: continue seen_paths.add(identity) - output_groups.setdefault((directory, group.destination), []).append( - output - ) + output_groups.setdefault((directory, group.destination), []).append(output) return cls._from_groups(output_groups, destinations) + @classmethod def _from_groups( cls, output_groups: Mapping[ - tuple[Path, RuntimeAnalysisSummaryDestination], - list[RuntimeAnalysisTableOutput], + tuple[Path, RuntimeAnalysisSummaryDestination], list[RuntimeAnalysisTableOutput] ], destinations: set[RuntimeAnalysisSummaryDestination], ) -> RuntimeAnalysisConsolidationInputs | None: @@ -202,8 +193,7 @@ def _from_groups( return cls( groups={ directory: RuntimeAnalysisDirectoryInputs( - outputs=tuple(outputs), - destination=destination, + outputs=tuple(outputs), destination=destination, ) for (directory, destination), outputs in output_groups.items() }, diff --git a/openhcs/domains/microscopy/axes.py b/openhcs/domains/microscopy/axes.py index 697ef68fa4..1b9c571173 100644 --- a/openhcs/domains/microscopy/axes.py +++ b/openhcs/domains/microscopy/axes.py @@ -86,9 +86,7 @@ def grid_coordinates(cls, value: object) -> tuple[str, str]: match = re.match(r"^([A-Za-z]+)([0-9]+)$", str(value)) if match is None: - raise ValueError( - f"{value!r} is not a well spelled ." - ) + raise ValueError(f"{value!r} is not a well spelled .") return match.group(1), match.group(2) @classmethod @@ -96,9 +94,7 @@ def grid_position(cls, row_label: str, column_label: str) -> tuple[int, int]: """Rows count A=1 … Z=26, AA=27; columns are the column number.""" if not row_label.isalpha() or not column_label.isdecimal(): - raise ValueError( - f"Not a well row and column: {row_label!r}, {column_label!r}." - ) + raise ValueError(f"Not a well row and column: {row_label!r}, {column_label!r}.") row = 0 for letter in row_label.upper(): row = row * 26 + (ord(letter) - ord("A") + 1) diff --git a/openhcs/formats/pattern/pattern_discovery.py b/openhcs/formats/pattern/pattern_discovery.py index 419c70d333..1f4884ba8e 100644 --- a/openhcs/formats/pattern/pattern_discovery.py +++ b/openhcs/formats/pattern/pattern_discovery.py @@ -97,9 +97,7 @@ def path_list_from_pattern( logger.debug("Using pattern template: %s", pattern_str) # Parse pattern template to get expected structure - pattern_metadata = self.pattern_cache.metadata_for_filename( - self.parser, pattern_str - ) + pattern_metadata = self.pattern_cache.metadata_for_filename(self.parser, pattern_str) if not pattern_metadata: logger.error("Failed to parse pattern template: %s", pattern_str) return [] @@ -119,9 +117,7 @@ def path_list_from_pattern( continue # Parse the actual filename - file_metadata = self.pattern_cache.metadata_for_filename( - self.parser, filename - ) + file_metadata = self.pattern_cache.metadata_for_filename(self.parser, filename) if not file_metadata: continue @@ -189,9 +185,7 @@ def group_patterns_by_component( # The has_placeholders() check is only relevant when using patterns as concrete filenames # For pattern discovery and grouping, we WANT patterns with placeholders - metadata = self.pattern_cache.metadata_for_filename( - self.parser, pattern_str - ) + metadata = self.pattern_cache.metadata_for_filename(self.parser, pattern_str) if metadata is None or metadata.value_for(component_declaration) is None: raise ValueError( @@ -214,9 +208,7 @@ def subdivide_patterns_by_components( subdivided = defaultdict(list) component_declarations = tuple(components) for pattern in patterns: - metadata = self.pattern_cache.metadata_for_filename( - self.parser, str(pattern) - ) + metadata = self.pattern_cache.metadata_for_filename(self.parser, str(pattern)) if not metadata: raise ValueError(f"Failed to parse pattern: {pattern}") key = tuple( diff --git a/openhcs/interop/cellprofiler/analyst_export.py b/openhcs/interop/cellprofiler/analyst_export.py index 6c71520d89..e81ec26b33 100644 --- a/openhcs/interop/cellprofiler/analyst_export.py +++ b/openhcs/interop/cellprofiler/analyst_export.py @@ -737,10 +737,7 @@ def _measurement_subject_rows( scope=scope, table=table, ) - default_subject = ( - table.subject.name - or CELLPROFILER_MEASUREMENT_DIALECT.scope_name(table.subject.scope) - ) + default_subject = table.subject.name or CELLPROFILER_MEASUREMENT_DIALECT.scope_name(table.subject.scope) if ( table.subject.scope is MeasurementScope.RUN and measurement_table_row_layout_from_fields(table.rows.fields) @@ -1180,9 +1177,7 @@ def collect_source_bound_image_provenance( "CPA source-bound thumbnails require a runtime context." ) return - workspace = ( - self.context.runtime_source_workspace_projections.projection_if_available() - ) + workspace = self.context.runtime_source_workspace_projections.projection_if_available() if workspace is None: if settings.write_image_thumbnails and any( binding.alias in settings.thumbnail_image_names @@ -1327,9 +1322,7 @@ def _resolved_source_path(self, source_path: str | None) -> Path | None: raise RuntimeError( "Virtual workspace source resolution requires a runtime FileManager." ) - projection = ( - self.context.runtime_source_workspace_projections.projection_if_available() - ) + projection = self.context.runtime_source_workspace_projections.projection_if_available() if projection is None: return Path(source_path).resolve(strict=False) return Path( @@ -1423,10 +1416,7 @@ def build( ), self.context, ) - image_subject = MeasurementSubject( - MeasurementScope.SAMPLE, - CELLPROFILER_MEASUREMENT_DIALECT.scope_name(MeasurementScope.SAMPLE), - ) + image_subject = MeasurementSubject(MeasurementScope.SAMPLE, CELLPROFILER_MEASUREMENT_DIALECT.scope_name(MeasurementScope.SAMPLE)) image_rows_by_number: dict[ int, dict[str, Any], @@ -1767,12 +1757,7 @@ def project_measurement_partition( table_name=projection.dialect.image_table_name(), rows=tuple(image_rows.values()), columns=image_columns, - subject=MeasurementSubject( - MeasurementScope.SAMPLE, - CELLPROFILER_MEASUREMENT_DIALECT.scope_name( - MeasurementScope.SAMPLE - ), - ), + subject=MeasurementSubject(MeasurementScope.SAMPLE, CELLPROFILER_MEASUREMENT_DIALECT.scope_name(MeasurementScope.SAMPLE)), ), experiment_table=CellProfilerProjectedTable( table_name=projection.dialect.object_table_name("Experiment"), diff --git a/openhcs/interop/cellprofiler/database_column_dialect.py b/openhcs/interop/cellprofiler/database_column_dialect.py index 5e17c1c3f6..774ea969cb 100644 --- a/openhcs/interop/cellprofiler/database_column_dialect.py +++ b/openhcs/interop/cellprofiler/database_column_dialect.py @@ -513,9 +513,7 @@ def image_subject( ) -> MeasurementSubject: """Invert an exact CPA image-table declaration into its subject.""" - subject = MeasurementSubject( - MeasurementScope.SAMPLE, CELLPROFILER_SCOPE_NAMES[MeasurementScope.SAMPLE] - ) + subject = MeasurementSubject(MeasurementScope.SAMPLE, CELLPROFILER_SCOPE_NAMES[MeasurementScope.SAMPLE]) expected_table = self.image_table_name() expected_image_id = self.image_id_field().name if ( diff --git a/openhcs/interop/cellprofiler/image_set_numbering.py b/openhcs/interop/cellprofiler/image_set_numbering.py index 4f4a2aa185..dd78bcb137 100644 --- a/openhcs/interop/cellprofiler/image_set_numbering.py +++ b/openhcs/interop/cellprofiler/image_set_numbering.py @@ -270,12 +270,8 @@ def _project_reference_columns( reference_indices.append(index) reference_numbers: dict[int, int] = {} for name in table.rows.columns: - wide_reference = CellProfilerMeasurementDialect.row_identity_contract.is_sample_number_reference( - name - ) - if CellProfilerMeasurementDialect.row_identity_contract.is_aggregate_sample_number_reference( - name - ): + wide_reference = CellProfilerMeasurementDialect.row_identity_contract.is_sample_number_reference(name) + if CellProfilerMeasurementDialect.row_identity_contract.is_aggregate_sample_number_reference(name): continue if not wide_reference and ( name not in value_fields or not reference_indices @@ -327,8 +323,8 @@ def source_slices_for_measurement_table( ) if not projection.has_axisless_rows(MeasurementRowAxisField.SLICE_INDEX): return indices - source_indices = tuple(range(table.source_provenance.source_plane_count)) or ( - 0, + source_indices = ( + tuple(range(table.source_provenance.source_plane_count)) or (0,) ) return tuple(dict.fromkeys((*indices, *source_indices))) diff --git a/openhcs/interop/cellprofiler/measurement_dialect.py b/openhcs/interop/cellprofiler/measurement_dialect.py index 04fe9059a5..d5ade66b94 100644 --- a/openhcs/interop/cellprofiler/measurement_dialect.py +++ b/openhcs/interop/cellprofiler/measurement_dialect.py @@ -223,9 +223,7 @@ def parent_reference_feature_name(self, parent_object_name: str) -> str: ) def parent_reference_object_name(self, feature_name: str) -> str | None: - identity = DirectParentReferenceFeatureDeclaration.from_feature_name( - feature_name - ) + identity = DirectParentReferenceFeatureDeclaration.from_feature_name(feature_name) return None if identity is None else identity.parent_object_name def child_count_feature_name(self, child_object_name: str) -> str: diff --git a/openhcs/interop/cellprofiler/measurement_lookup.py b/openhcs/interop/cellprofiler/measurement_lookup.py index ffdea014ea..d7eb4f6a63 100644 --- a/openhcs/interop/cellprofiler/measurement_lookup.py +++ b/openhcs/interop/cellprofiler/measurement_lookup.py @@ -81,6 +81,7 @@ def feature_name(cls, identity: str) -> str: return f"{cls.prefix}{name}{cls.suffix}" + class CellProfilerMeasurementFeatureKind(Enum): """CellProfiler measurement feature families with structured semantics.""" diff --git a/openhcs/interop/cellprofiler/measurement_scope.py b/openhcs/interop/cellprofiler/measurement_scope.py index f0ce9f25c9..851b5456c7 100644 --- a/openhcs/interop/cellprofiler/measurement_scope.py +++ b/openhcs/interop/cellprofiler/measurement_scope.py @@ -10,6 +10,7 @@ MeasurementScopeSelection, ) + CELLPROFILER_SCOPE_NAMES = MappingProxyType( { MeasurementScope.SAMPLE: "Image", diff --git a/openhcs/interop/cellprofiler/module_artifact_contracts.py b/openhcs/interop/cellprofiler/module_artifact_contracts.py index 53879158dd..804bfe194c 100644 --- a/openhcs/interop/cellprofiler/module_artifact_contracts.py +++ b/openhcs/interop/cellprofiler/module_artifact_contracts.py @@ -222,6 +222,7 @@ def _artifact_context_for_group( ): raise ValueError( f"No source binding declares {grouped_component.name} " + f"group {group_key!r}." ) source_bindings = replace( diff --git a/openhcs/interop/cellprofiler/module_artifact_declarations.py b/openhcs/interop/cellprofiler/module_artifact_declarations.py index 8f60f5cfe0..920638029f 100644 --- a/openhcs/interop/cellprofiler/module_artifact_declarations.py +++ b/openhcs/interop/cellprofiler/module_artifact_declarations.py @@ -653,15 +653,12 @@ def prior_measurement_artifact_inputs( ) group_lineage_refs = source_refs or object_refs for producer in feature_matches: - object_subjects = tuple( - dict.fromkeys( - InputObjectMeasurementSourceRelation(relation.source) - for relation in producer.spec.relations - if relation.source.artifact_type is ObjectLabelsArtifactType - and relation.source.for_plan_type(ArtifactInputPlan) - in object_refs - ) - ) + object_subjects = tuple(dict.fromkeys( + InputObjectMeasurementSourceRelation(relation.source) + for relation in producer.spec.relations + if relation.source.artifact_type is ObjectLabelsArtifactType + and relation.source.for_plan_type(ArtifactInputPlan) in object_refs + )) if object_subjects: measurement_input = object_subjects[0].input_spec_for_output( producer.spec @@ -1235,11 +1232,9 @@ def finalize_artifact_contract_inputs( for spec in inputs.of_artifact_type(ImageArtifactType) ) return tuple( - ( - replace(spec, relations=(*spec.relations, *image_contexts)) - if spec.artifact_type is ObjectLabelsArtifactType and image_contexts - else spec - ) + replace(spec, relations=(*spec.relations, *image_contexts)) + if spec.artifact_type is ObjectLabelsArtifactType and image_contexts + else spec for spec in inputs ) @@ -1257,11 +1252,8 @@ def input_source_for_contract( ) if any( spec.ref() not in produced_refs - and step_context.source_bindings.binding_for_artifact_ref(spec.ref()) - is not None - for spec in callable_contract.artifact_inputs.of_artifact_type( - ImageArtifactType - ) + and step_context.source_bindings.binding_for_artifact_ref(spec.ref()) is not None + for spec in callable_contract.artifact_inputs.of_artifact_type(ImageArtifactType) ): return InputSource.PIPELINE_START return super().input_source_for_contract( @@ -1380,9 +1372,7 @@ def finalize_module_blocks_for_invocation( """Require selection intent before reconstructing multiple object subjects.""" blocks = super().finalize_module_blocks_for_invocation( - blocks, - invocation=invocation, - step_context=step_context, + blocks, invocation=invocation, step_context=step_context, ) binding = cls.object_measurement_binding selector = binding.require_parameter_name() @@ -1400,8 +1390,7 @@ def processing_group_scope_inputs( ) -> tuple[ArtifactSpec, ...]: """Keep all declared measurement subjects in one logical module batch.""" source_refs = frozenset( - spec.ref() - for spec in ( + spec.ref() for spec in ( *super().processing_group_scope_inputs(callable_contract), *callable_contract.group_scope_inputs, ) diff --git a/openhcs/interop/cellprofiler/module_declarations.py b/openhcs/interop/cellprofiler/module_declarations.py index ffa8eb176b..4699376d3e 100644 --- a/openhcs/interop/cellprofiler/module_declarations.py +++ b/openhcs/interop/cellprofiler/module_declarations.py @@ -68,6 +68,7 @@ Ungrouped, ) + _CELLPROFILER_BACKEND_PACKAGE = "openhcs.processing.backends.cellprofiler" _CELLPROFILER_MODULE_REGISTRY = LazyDiscoveryDict(enable_cache=False) if TYPE_CHECKING: @@ -437,12 +438,8 @@ def source_bindings_for_modules( ) modules = tuple(modules) cls.discover_source_declarations( - frozenset( - _module_lookup_key(module.name) for module in modules if module.enabled - ), - ("module_name", "aliases"), - _module_lookup_key, - _declared_lookup_keys, + frozenset(_module_lookup_key(module.name) for module in modules if module.enabled), + ("module_name", "aliases"), _module_lookup_key, _declared_lookup_keys, ) for module in modules: if not module.enabled: @@ -597,8 +594,7 @@ def require_callable( implementation = cls._require_declared_callable(function_name) cls.prepare_declared_callable(implementation) cls._validate_callable_parameter_exclusions( - implementation, - CallableContract.from_callable(implementation), + implementation, CallableContract.from_callable(implementation), ) return implementation @@ -661,9 +657,7 @@ def for_module(cls, module_name: str) -> type["CellProfilerModule"] | None: """Return the registered module class for a canonical name or alias.""" lookup_key = _module_lookup_key(module_name) matches = cls.discover_source_declarations( - frozenset((lookup_key,)), - ("module_name", "aliases"), - _module_lookup_key, + frozenset((lookup_key,)), ("module_name", "aliases"), _module_lookup_key, _declared_lookup_keys, ) return matches[0] if matches else None @@ -955,8 +949,7 @@ def input_source_for_contract( ): return InputSource.PREVIOUS_STEP if any( - step_context.source_bindings.binding_for_artifact_ref(spec.ref()) - is not None + step_context.source_bindings.binding_for_artifact_ref(spec.ref()) is not None for spec in callable_contract.artifact_inputs ): return InputSource.PIPELINE_START @@ -1005,6 +998,7 @@ def processing_config( if grouping_axes and step_context is not None: (grouping_component,) = grouping_axes source_anchor_group_keys = step_context.source_bindings.component_group_keys_for_artifact_specs( + grouping_component, cls.processing_group_scope_inputs(callable_contract), step_context.available_artifacts, diff --git a/openhcs/interop/cellprofiler/pipeline_import.py b/openhcs/interop/cellprofiler/pipeline_import.py index 340b63ba6e..5d30c844d9 100644 --- a/openhcs/interop/cellprofiler/pipeline_import.py +++ b/openhcs/interop/cellprofiler/pipeline_import.py @@ -1061,7 +1061,9 @@ def projected_leaf( if group_key in grouped_items and not preserves_input_main_flow: return None grouped_items.setdefault(group_key, []).append(leaf) - target_units_by_group_key.setdefault(group_key, []).append(unit) + target_units_by_group_key.setdefault(group_key, []).append( + unit + ) public_pattern = { group_key: items[0] if len(items) == 1 else items for group_key, items in grouped_items.items() @@ -1102,8 +1104,7 @@ def target_units_for_invocation( return None invocation_target_contract = ( invocation_target_units[0].contract - if len(invocation_target_units) == 1 - else target_contract + if len(invocation_target_units) == 1 else target_contract ) # Projection may omit a selector only in its original context. # Reuse the binding-owned occurrence relation after earlier calls @@ -1113,8 +1114,7 @@ def target_units_for_invocation( module_type, invocation_target_units, candidate_group_keys=( - () - if invocation.key.group_key == DEFAULT_GROUP_KEY + () if invocation.key.group_key == DEFAULT_GROUP_KEY else (invocation.key.group_key,) ), step_context=group_context, diff --git a/openhcs/interop/cellprofiler/runtime/function_contract_execution.py b/openhcs/interop/cellprofiler/runtime/function_contract_execution.py index 1890135ef5..f1a405b973 100644 --- a/openhcs/interop/cellprofiler/runtime/function_contract_execution.py +++ b/openhcs/interop/cellprofiler/runtime/function_contract_execution.py @@ -129,9 +129,7 @@ def validate_full_stack_kwargs(self, contract, kwargs): def stack_reduction_output(self, result): result_2d = ImagePayload.of(result) stacked = stack_slices( - [result_2d.data], - detect_memory_type(result_2d.data), - 0, + [result_2d.data], detect_memory_type(result_2d.data), 0, ) return result_2d.metadata.payload_with(stacked, mask=result_2d.mask) @@ -292,7 +290,9 @@ def execute( plane_projection, ) execute_started_at = time.perf_counter() - with executing_measurement_dialect(CELLPROFILER_MEASUREMENT_DIALECT): + with executing_measurement_dialect( + CELLPROFILER_MEASUREMENT_DIALECT + ): result = execution_mode.execute( processing_contract, call, image, dict(kwargs) ) diff --git a/openhcs/interop/cellprofiler/runtime/invocation.py b/openhcs/interop/cellprofiler/runtime/invocation.py index 3542c72b6d..25d0030abf 100644 --- a/openhcs/interop/cellprofiler/runtime/invocation.py +++ b/openhcs/interop/cellprofiler/runtime/invocation.py @@ -9,7 +9,7 @@ import numpy as np from openhcs.core.alias_property import AliasProperty -from openhcs.core.aligned_image_payload import AlignedImageStack, ImageOutputBundle +from openhcs.core.aligned_image_payload import (AlignedImageStack, ImageOutputBundle) from openhcs.core.equivalence.keys import RuntimeMeasurementSourcePair from openhcs.core.measurement_image_alignment import ( MeasurementImageAlignmentSource, @@ -199,9 +199,7 @@ def aligned_source_metadata( """ if not sources: raise ValueError("Aligned measurement metadata requires source images.") - source_axes = tuple( - cls.source_provenances(source.payload) for source in sources - ) + source_axes = tuple(cls.source_provenances(source.payload) for source in sources) planes = tuple( SourceImageProvenance.bundle( tuple(source.with_runtime_planes_as_contributors() for source in plane) @@ -230,9 +228,7 @@ def source_provenances( cls.scalar_source_provenance(value) for value in payload.slices ) if isinstance(payload, RuntimeSliceAlignedValueSet): - return tuple( - cls.scalar_source_provenance(value) for value in payload.values - ) + return tuple(cls.scalar_source_provenance(value) for value in payload.values) count = RuntimeSliceProjection.slice_count_from_values((payload,)) metadata = image_metadata_of(payload) provenance = metadata.source_provenance @@ -311,9 +307,7 @@ def __post_init__(self) -> None: if self.primary_input_payloads and ( len(self.primary_input_payloads) != len(self.primary_input_edges) ): - raise ValueError( - "Primary input contexts must preserve exact binding order." - ) + raise ValueError("Primary input contexts must preserve exact binding order.") def consumed_input_value( self, diff --git a/openhcs/interop/cellprofiler/runtime/measurement_recording.py b/openhcs/interop/cellprofiler/runtime/measurement_recording.py index 866a703a93..0af88eb3d8 100644 --- a/openhcs/interop/cellprofiler/runtime/measurement_recording.py +++ b/openhcs/interop/cellprofiler/runtime/measurement_recording.py @@ -149,8 +149,7 @@ def build_measurement_table( if object_name is not None else MeasurementSubject( MeasurementScope.SAMPLE, - source_image_name - or CELLPROFILER_SCOPE_NAMES[MeasurementScope.SAMPLE].casefold(), + source_image_name or CELLPROFILER_SCOPE_NAMES[MeasurementScope.SAMPLE].casefold(), ) ) return MeasurementTable( @@ -713,11 +712,9 @@ def measurement_record_source_metadata( ) -> ImagePayloadMetadata: del rows source_spec = cls.primary_image_output_spec(request) - return ( - request.source.payload + return (request.source.payload if source_spec is None - else request.artifact_output_value(source_spec) - ).metadata + else request.artifact_output_value(source_spec)).metadata class ProducedImagePayloadMeasurementRecordMixin( @@ -762,7 +759,9 @@ def measurement_record_source_metadata( request.adapter.request.source_binding_plan ) identity_axes = tuple( - payload.metadata.source_provenance.image_set_axis(identity_policy) + payload.metadata.source_provenance.image_set_axis( + identity_policy + ) for payload in source_payloads ) missing_identity_specs = tuple( @@ -787,7 +786,9 @@ def measurement_record_source_metadata( ) source_metadata = ImagePayloadMetadata.compose( source_payloads, - source_metadata=tuple(payload.metadata for payload in source_payloads), + source_metadata=tuple( + payload.metadata for payload in source_payloads + ), ).collapse_leading_plane_axis() return source_metadata diff --git a/openhcs/interop/cellprofiler/runtime/module_execution.py b/openhcs/interop/cellprofiler/runtime/module_execution.py index f23a8b9e03..d7fa4dae61 100644 --- a/openhcs/interop/cellprofiler/runtime/module_execution.py +++ b/openhcs/interop/cellprofiler/runtime/module_execution.py @@ -11,11 +11,7 @@ ) from typing import cast -from openhcs.core.aligned_image_payload import ( - AlignedImageSliceContext, - AlignedImageStack, - ImageOutputBundle, -) +from openhcs.core.aligned_image_payload import (AlignedImageSliceContext, AlignedImageStack, ImageOutputBundle) from openhcs.core.image_payload_execution_mode import ImagePayloadExecutionMode from openhcs.core.artifacts import ( ArtifactOutputPlan, @@ -1293,7 +1289,8 @@ def _image_request( if ( runtime_projection is not None and runtime_projection.plane_index is not None - and current_image_payload.metadata.plane_axis is runtime_projection.axis + and current_image_payload.metadata.plane_axis + is runtime_projection.axis ): current_image_payload = cast( RuntimeCallableArgument, @@ -1322,7 +1319,9 @@ def _image_request( image_input_specs = ArtifactSpecCollection(image_inputs) payloads = [] source_names: list[str | None] = [] - image_strategy = CellProfilerOutputRecorder.for_artifact_type(ImageArtifactType) + image_strategy = CellProfilerOutputRecorder.for_artifact_type( + ImageArtifactType + ) input_binding = replace(input_binding, current_image=current_runtime_payload) primary_input_edges = [] for spec in image_inputs: @@ -1331,23 +1330,20 @@ def _image_request( if ( scope is not None and edge.storage_plan is None - and adapter.request.source_binding_plan.declares_artifact_ref( - spec.ref() - ) + and adapter.request.source_binding_plan.declares_artifact_ref(spec.ref()) ): value = edge.resolve_unstored_payload( - scope, - current_runtime_payload, - request=replace( - adapter.request, source_payload=current_runtime_payload - ), + scope, current_runtime_payload, + request=replace(adapter.request, source_payload=current_runtime_payload), ) else: value = input_binding.artifact_value(edge) primary_input_edges.append(edge) payload = image_strategy.runtime_input_value(spec, value) payloads.append(payload) - source_names.append(image_strategy.source_image_name_from_value(payload)) + source_names.append( + image_strategy.source_image_name_from_value(payload) + ) parameter_image_inputs = input_binding.image_inputs broadcast_sources = tuple( sources[0] @@ -1358,26 +1354,24 @@ def _image_request( align_primary_images = len(image_inputs) > 1 and broadcast_sources == tuple( spec.ref() for spec in image_inputs ) - composition = ( - self.callable_contract.image_payload_consumption.compose_image_payload( - f"{module_type.require_module_name()} image inputs " - f"{tuple(spec.name for spec in image_inputs)!r}", - tuple(payloads), - slice_contexts=( - tuple( - AlignedImageSliceContext.main_flow( - output_key=spec.name, - artifact_kind=spec.artifact_type.value, - ) - for spec in image_inputs + composition = self.callable_contract.image_payload_consumption.compose_image_payload( + f"{module_type.require_module_name()} image inputs " + f"{tuple(spec.name for spec in image_inputs)!r}", + tuple(payloads), + slice_contexts=( + tuple( + AlignedImageSliceContext.main_flow( + output_key=spec.name, + artifact_kind=spec.artifact_type.value, ) - if align_primary_images - else () - ), - stack_broadcast_source_indices=( - image_input_specs.stack_broadcast_source_indices() - ), - ) + for spec in image_inputs + ) + if align_primary_images + else () + ), + stack_broadcast_source_indices=( + image_input_specs.stack_broadcast_source_indices() + ), ) return CellProfilerImageRequest( input_binding_request=binding_request, @@ -1526,8 +1520,7 @@ def _invocation_request( invocation_kwargs = object_label_input_execution_mode_from_callable( self.raw_func ).invocation_kwargs( - runtime_kwargs, - execution_mode=execution_mode, + runtime_kwargs, execution_mode=execution_mode, image_projection=image_request.plane_projection, runtime_projection=runtime_projection, semantic_controls=_execution_mode_semantic_control_kwargs( diff --git a/openhcs/interop/cellprofiler/workspace_export.py b/openhcs/interop/cellprofiler/workspace_export.py index 08911537bc..b1976fe70e 100644 --- a/openhcs/interop/cellprofiler/workspace_export.py +++ b/openhcs/interop/cellprofiler/workspace_export.py @@ -70,10 +70,7 @@ class CPAImageWorkspaceAxis(CPAWorkspaceAxis): def field_name(self, dialect): return dialect.measurement_field( - MeasurementSubject( - MeasurementScope.SAMPLE, - CELLPROFILER_SCOPE_NAMES[MeasurementScope.SAMPLE], - ), + MeasurementSubject(MeasurementScope.SAMPLE, CELLPROFILER_SCOPE_NAMES[MeasurementScope.SAMPLE]), FieldSpec(self.measurement, None), ).name diff --git a/openhcs/mcp/context.py b/openhcs/mcp/context.py index a7f442ffac..c47a6938aa 100644 --- a/openhcs/mcp/context.py +++ b/openhcs/mcp/context.py @@ -144,8 +144,7 @@ def endpoint_function_catalog(self) -> "EndpointFunctionCatalogServiceABC": endpoint = load_cached_ui_execution_endpoint_sync() self._endpoint_function_catalog = ZMQFunctionCatalogService( - lambda: endpoint, - path_policy=self.path_policy, + lambda: endpoint, path_policy=self.path_policy, ) return self._endpoint_function_catalog @@ -282,9 +281,7 @@ def runtime_server_service(self) -> "RuntimeServerService": RuntimeServerService, ) - self._runtime_server_service = RuntimeServerService( - path_policy=self.path_policy - ) + self._runtime_server_service = RuntimeServerService(path_policy=self.path_policy) return self._runtime_server_service @property @@ -342,8 +339,7 @@ def create_agent_context() -> OpenHCSAgentContext: endpoint_config = load_cached_ui_execution_endpoint_sync() path_policy = AgentPathPolicy.from_environment() catalog = ZMQFunctionCatalogService( - lambda: endpoint_config, - path_policy=path_policy, + lambda: endpoint_config, path_policy=path_policy, ) return OpenHCSAgentContext( path_policy=path_policy, diff --git a/openhcs/mcp/dev_client_commands/ui.py b/openhcs/mcp/dev_client_commands/ui.py index ab394934b2..368505fb91 100644 --- a/openhcs/mcp/dev_client_commands/ui.py +++ b/openhcs/mcp/dev_client_commands/ui.py @@ -369,9 +369,7 @@ async def _poll_workflow_state( poll_deadline = loop.time() + args.poll_timeout_seconds async def await_events(after_sequence: int) -> int: - connection_seconds = ( - args.timeout_ms or DEFAULT_UI_BRIDGE_TIMEOUT_MS - ) / 1000 + connection_seconds = (args.timeout_ms or DEFAULT_UI_BRIDGE_TIMEOUT_MS) / 1000 wait_seconds = min( max(poll_deadline - loop.time(), 0.0), 0.8 * connection_seconds ) @@ -464,9 +462,7 @@ def render_compact(self, response, args: argparse.Namespace) -> str: lines.append(f"Targets: {', '.join(summary.target_scope_ids)}") if summary.skip_reason is not None: lines.append(f"Skip reason: {summary.skip_reason.value}") - lines.extend( - McpDevOutputRenderer.diagnostic_lines(response.diagnostic_errors()) - ) + lines.extend(McpDevOutputRenderer.diagnostic_lines(response.diagnostic_errors())) final_rows = self._final_state_rows(response) if final_rows: lines.append("Rows:") @@ -542,9 +538,7 @@ def calls_from_args( ) -class ValidateCodeDocumentCommandSpec( - StdinSourceCommandSpec, CapabilityBackedCommandSpec -): +class ValidateCodeDocumentCommandSpec(StdinSourceCommandSpec, CapabilityBackedCommandSpec): capability = agent_capabilities.ui_validate_code_document def configure_parser(self, parser: argparse.ArgumentParser) -> None: diff --git a/openhcs/mcp/dev_client_core.py b/openhcs/mcp/dev_client_core.py index 8d85248149..e975db7572 100644 --- a/openhcs/mcp/dev_client_core.py +++ b/openhcs/mcp/dev_client_core.py @@ -16,16 +16,7 @@ from enum import Enum from functools import singledispatch from pathlib import Path -from typing import ( - TYPE_CHECKING, - ClassVar, - Self, - TextIO, - TypeVar, - cast, - get_args, - get_type_hints, -) +from typing import TYPE_CHECKING, ClassVar, Self, TextIO, TypeVar, cast, get_args, get_type_hints from metaclass_registry import AutoRegisterMeta from python_introspect import ( @@ -1165,9 +1156,7 @@ def _agent_errors(value: object) -> tuple[AgentError, ...]: return tuple( error for declaration in fields(value) - if not isinstance( - field_value := getattr(value, declaration.name), AgentError - ) + if not isinstance(field_value := getattr(value, declaration.name), AgentError) for error in _agent_errors(field_value) ) return () diff --git a/openhcs/mcp/dev_client_renderers/pipeline.py b/openhcs/mcp/dev_client_renderers/pipeline.py index 9b6b33ea92..cb3f07ba00 100644 --- a/openhcs/mcp/dev_client_renderers/pipeline.py +++ b/openhcs/mcp/dev_client_renderers/pipeline.py @@ -132,7 +132,9 @@ def render(cls, response, *, max_source_chars: int = 2_000) -> str: lines.append(f"Ref: uri={created.uri}") if validated is not None and validated.errors: lines.append("Validate errors:") - lines.extend(diagnostic_lines(decoded.diagnostic_errors())) + lines.extend( + diagnostic_lines(decoded.diagnostic_errors()) + ) if validated is not None: if validated.warnings: lines.append("Validate warnings:") @@ -319,7 +321,9 @@ def _materialization_lines( mode = ( "disabled" if materialization.disabled - else "runtime-resolved" if materialization.runtime_resolved else "explicit" + else "runtime-resolved" + if materialization.runtime_resolved + else "explicit" ) parts = [mode, f"persistent={materialization.persistent_enabled}"] parts.extend( diff --git a/openhcs/mcp/dev_client_renderers/plate.py b/openhcs/mcp/dev_client_renderers/plate.py index de82572746..e4a05b8010 100644 --- a/openhcs/mcp/dev_client_renderers/plate.py +++ b/openhcs/mcp/dev_client_renderers/plate.py @@ -188,9 +188,7 @@ def _roi_area_text(preview: PlateInspectionResultFilePreview) -> str: ): return "" mean = ( - "" - if preview.roi_area_mean is None - else f"{preview.roi_area_mean:.3f}" + "" if preview.roi_area_mean is None else f"{preview.roi_area_mean:.3f}" ) return ( f"min={McpDevOutputRenderer.text(preview.roi_area_min)}," @@ -210,9 +208,7 @@ def _csv_lines(cls, preview: PlateInspectionResultFilePreview) -> list[str]: return [] lines = [f" csv columns: {cls._csv_columns_text(columns)}"] visible_rows = rows[: cls.MAX_CSV_PREVIEW_ROWS] - lines.extend( - f" csv row: {cls._csv_row_text(columns, row)}" for row in visible_rows - ) + lines.extend(f" csv row: {cls._csv_row_text(columns, row)}" for row in visible_rows) omitted_row_count = len(rows) - len(visible_rows) if omitted_row_count > 0: lines.append( @@ -592,11 +588,9 @@ def _axis_summary_line( ( f"unknown-{axis.name}" if axis.name not in counts - else ( - f"multi-{axis.name}" - if counts[axis.name] > 1 - else f"single-{axis.name}" - ) + else f"multi-{axis.name}" + if counts[axis.name] > 1 + else f"single-{axis.name}" ) for axis in AxisFamily.active().variable_axes() ) diff --git a/openhcs/mcp/dev_client_renderers/ui_bridge.py b/openhcs/mcp/dev_client_renderers/ui_bridge.py index fead3eb5fa..782e3b6994 100644 --- a/openhcs/mcp/dev_client_renderers/ui_bridge.py +++ b/openhcs/mcp/dev_client_renderers/ui_bridge.py @@ -157,28 +157,20 @@ def render(cls, response) -> str: return "\n".join( ( f"UI smoke: results={len(decoded.results)} mcp_errors={mcp_errors}", - ( - "Health: missing" - if health is None - else f"Health: status={health.status} " - f"restart_required={health.restart_required} " - f"stale_paths={len(health.stale_source_paths)}" - ), - ( - UiBridgeStatusRenderer.unavailable_summary - if status is None - else McpDevOutputRenderer.render_payload_value(status, options) - ), - ( - "Bridges: missing" - if bridges is None - else f"Bridges: live={len(bridges.bridges)} errors={len(bridges.errors)}" - ), - ( - UiWindowCatalogRenderer.unavailable_summary - if windows is None - else McpDevOutputRenderer.render_payload_value(windows, options) - ), + "Health: missing" + if health is None + else f"Health: status={health.status} " + f"restart_required={health.restart_required} " + f"stale_paths={len(health.stale_source_paths)}", + UiBridgeStatusRenderer.unavailable_summary + if status is None + else McpDevOutputRenderer.render_payload_value(status, options), + "Bridges: missing" + if bridges is None + else f"Bridges: live={len(bridges.bridges)} errors={len(bridges.errors)}", + UiWindowCatalogRenderer.unavailable_summary + if windows is None + else McpDevOutputRenderer.render_payload_value(windows, options), *McpDevOutputRenderer.diagnostic_lines(decoded.diagnostic_errors()), ) ) @@ -228,9 +220,7 @@ class UiStateSurfaceStateRenderer(McpDevOutputRenderer): surface_identity: ClassVar[type[UiStateSurfaceIdentityDeclarationBase]] @classmethod - def for_surface_id( - cls, surface_id: str - ) -> type["UiStateSurfaceStateRenderer"] | None: + def for_surface_id(cls, surface_id: str) -> type["UiStateSurfaceStateRenderer"] | None: return next( ( renderer @@ -318,9 +308,7 @@ def _section_lines(cls, section: UiLiveOverviewSection) -> list[str]: ("surface", item.source_surface_id), ("window", item.source_window_id), ): - parts.extend( - cls.optional_lines(value, lambda text: (f"{label}={text}",)) - ) + parts.extend(cls.optional_lines(value, lambda text: (f"{label}={text}",))) lines.append(f"- {item.label}: " + " ".join(parts)) return lines @@ -365,9 +353,7 @@ def row_lines(cls, rows: tuple[DatasetRowState, ...]) -> list[str]: ("output", row.output_root), ("source", row.source_root), ): - parts.extend( - cls.optional_lines(value, lambda text: (f"{label}={text}",)) - ) + parts.extend(cls.optional_lines(value, lambda text: (f"{label}={text}",))) lines.append(f"- {row.name}: " + ", ".join(parts)) return lines @@ -498,9 +484,7 @@ class WidgetTreeOutlineRenderer(McpDevOutputRenderer): output_contract = UiWidgetTreeResult render_options_type = WidgetTreeRenderOptions MAX_LABEL_CHARS = 96 - TECHNICAL_WIDGET_CLASSES = frozenset( - {"QHeaderView", "QScrollBar", "QSplitterHandle"} - ) + TECHNICAL_WIDGET_CLASSES = frozenset({"QHeaderView", "QScrollBar", "QSplitterHandle"}) TECHNICAL_OBJECT_NAME_PREFIX = "qt_scrollarea_" @classmethod @@ -591,7 +575,9 @@ def _node_label( parts = [part for part in (node.class_name,) if part] if node.object_name: parts.append(f"#{node.object_name}") - addressable = node.actionable or (node.visible and any(node.action_kinds)) + addressable = node.actionable or ( + node.visible and any(node.action_kinds) + ) path_id = node.path_id if addressable else "" action = actions[path_id] if path_id in actions else None semantic_parts = [] if action is None else cls._semantic_parts(action) diff --git a/openhcs/mcp/dev_client_session.py b/openhcs/mcp/dev_client_session.py index 3417f13417..2fae6f25bf 100644 --- a/openhcs/mcp/dev_client_session.py +++ b/openhcs/mcp/dev_client_session.py @@ -90,9 +90,7 @@ class SessionJourney: timeout_seconds: float results: list[McpDevToolResult] - async def call( - self, capability, arguments: dict[str, JsonValue] - ) -> McpDevToolResult: + async def call(self, capability, arguments: dict[str, JsonValue]) -> McpDevToolResult: result = await call_mcp_tool( self.session, McpDevToolCall(capability.name, McpToolArguments.from_payload(arguments)), @@ -165,7 +163,7 @@ async def run_source( added = await self.call(agent_capabilities.add_datasets, {"roots": [root]}) if _rejected(added): return - scope_id, *_rest = added.first_decoded_payload().target_scope_ids + (scope_id, *_rest) = added.first_decoded_payload().target_scope_ids piped = await self.call( agent_capabilities.set_dataset_pipeline, {"scope_id": scope_id, "pipeline_source": pipeline_source}, diff --git a/openhcs/mcp/execution.py b/openhcs/mcp/execution.py index a90b3a6428..aff5e5d721 100644 --- a/openhcs/mcp/execution.py +++ b/openhcs/mcp/execution.py @@ -85,9 +85,7 @@ class McpTransportExecutor(AsyncOperationExecutor): def __init__(self) -> None: if threading.current_thread() is not threading.main_thread(): - raise RuntimeError( - "MCP transport must be started on the process main thread." - ) + raise RuntimeError("MCP transport must be started on the process main thread.") existing_application = QCoreApplication.instance() # A process holds one Qt application; one this transport created is # its own and goes with it, so a later GUI can create a QApplication. @@ -126,9 +124,7 @@ def close(self) -> None: leaving it to interpreter finalization is not an MCP lifecycle. """ if threading.current_thread() is not threading.main_thread(): - raise RuntimeError( - "MCP transport must be closed on the process main thread." - ) + raise RuntimeError("MCP transport must be closed on the process main thread.") if self._closed: return try: diff --git a/openhcs/mcp/server.py b/openhcs/mcp/server.py index f93f4e8473..36c3850441 100644 --- a/openhcs/mcp/server.py +++ b/openhcs/mcp/server.py @@ -434,9 +434,7 @@ async def _report_progress_if_available( @contextmanager -def _verbose_blocking_operation_diagnostics( - capability: type[AgentCapabilityDeclaration], -): +def _verbose_blocking_operation_diagnostics(capability: type[AgentCapabilityDeclaration]): """Dump a blocked invocation without affecting normal MCP output.""" heartbeat_seconds = capability.progress_heartbeat_seconds @@ -477,9 +475,7 @@ def _mcp_server_health() -> McpServerHealthResult: server_import_mtime_ns=MCP_SERVER_IMPORT_MTIME_NS, server_current_mtime_ns=current_source_mtime_ns, server_source_changed_since_import=bool(stale_source_paths), - stale_source_paths=tuple( - str(source_path) for source_path in stale_source_paths - ), + stale_source_paths=tuple(str(source_path) for source_path in stale_source_paths), recovery_reason=recovery.reason.value, installation_pointer_path=recovery.installation_pointer_path, installation_pointer_changed_since_import=( @@ -617,18 +613,14 @@ def ui_connection_parameter() -> Parameter: annotation=McpUiBridgeConnectionRequest | None, ) - def ui_connection( - self, arguments: Mapping[str, JsonValue] - ) -> UiBridgeConnectionSpec: + def ui_connection(self, arguments: Mapping[str, JsonValue]) -> UiBridgeConnectionSpec: return UiBridgeConnectionToolArgs.from_request(arguments["connection"]).resolve( self.context ) @staticmethod def viewer_connection_parameters() -> tuple[Parameter, ...]: - return McpViewerConnectionToolFields.signature_parameters( - McpViewerTimeoutPolicy - ) + return McpViewerConnectionToolFields.signature_parameters(McpViewerTimeoutPolicy) @staticmethod def viewer_control( diff --git a/openhcs/microscopes/bioformats.py b/openhcs/microscopes/bioformats.py index c2628a5f43..067ac01a23 100644 --- a/openhcs/microscopes/bioformats.py +++ b/openhcs/microscopes/bioformats.py @@ -134,9 +134,7 @@ def get_metadata_pixel_size(self, dataset_root: Union[str, Path]) -> float: """Retain the dataset's numeric view without asserting physical units.""" return self.source_dataset(dataset_root).pixel_size - def source_voxel_spacing( - self, dataset_root: Union[str, Path] - ) -> SourceVoxelSpacing: + def source_voxel_spacing(self, dataset_root: Union[str, Path]) -> SourceVoxelSpacing: return SourceVoxelSpacing.common(self._source_voxel_spacings(dataset_root)) def _source_voxel_spacings( @@ -238,6 +236,8 @@ def __init__( def root_dir(self) -> str: return FIELDS.DEFAULT_SUBDIRECTORY + + @property def compatible_backends(self) -> List[Backend]: return [Backend.BIOFORMATS] diff --git a/openhcs/microscopes/imagexpress.py b/openhcs/microscopes/imagexpress.py index b74b88999e..caa6019d30 100644 --- a/openhcs/microscopes/imagexpress.py +++ b/openhcs/microscopes/imagexpress.py @@ -35,7 +35,9 @@ class ImageXpressTimePointPaths(FilenameParserCapability): _timepoint_folder_pattern = re.compile(r"TimePoint[_-]?(\d+)", re.IGNORECASE) - def image_path_components(self, path: Path) -> tuple[tuple[type[Axis], int], ...]: + def image_path_components( + self, path: Path + ) -> tuple[tuple[type[Axis], int], ...]: return ( *super().image_path_components(path), *self.indexed_folder_components( @@ -51,7 +53,9 @@ class ImageXpressZStepPaths(FilenameParserCapability): _zstep_folder_pattern = re.compile(r"ZStep[_-]?(\d+)", re.IGNORECASE) - def image_path_components(self, path: Path) -> tuple[tuple[type[Axis], int], ...]: + def image_path_components( + self, path: Path + ) -> tuple[tuple[type[Axis], int], ...]: return ( *super().image_path_components(path), *self.indexed_folder_components( @@ -458,6 +462,8 @@ def root_dir(self) -> str: """ return "." + + @property def compatible_backends(self) -> List[Backend]: """ diff --git a/openhcs/microscopes/omero.py b/openhcs/microscopes/omero.py index 5e5ea70516..7c2ff295d5 100644 --- a/openhcs/microscopes/omero.py +++ b/openhcs/microscopes/omero.py @@ -278,9 +278,7 @@ def get_image_files( all_subdirs: Unused for OMERO (no subdirectories), kept for interface compatibility """ plate_id = ( - dataset_root - if isinstance(dataset_root, int) - else int(Path(dataset_root).name) + dataset_root if isinstance(dataset_root, int) else int(Path(dataset_root).name) ) # Get OMERO backend from registry @@ -345,6 +343,7 @@ def construct_filename(self, components: FilenameParseResult) -> str: extension=components.extension, ).filename() + def extract_component_coordinates(self, component_value: str) -> tuple[str, str]: """Extract coordinates from well identifier (e.g., 'A01' → ('A', '01')).""" @@ -358,6 +357,7 @@ class OMEROHandler(RemoteServiceSource, DatasetSource): source_name = "omero" metadata_handler_class = OMEROMetadataHandler + @classmethod def source_selection_guidance(cls) -> str: """Explain when the OMERO virtual source owner should be selected.""" @@ -391,6 +391,8 @@ def root_dir(self) -> str: """ return "Images" + + @property def compatible_backends(self) -> list[Backend]: """OMERO is only compatible with OMERO_LOCAL backend.""" diff --git a/openhcs/microscopes/opera_phenix.py b/openhcs/microscopes/opera_phenix.py index 979e33d15d..9d5f30eb30 100644 --- a/openhcs/microscopes/opera_phenix.py +++ b/openhcs/microscopes/opera_phenix.py @@ -506,6 +506,8 @@ def root_dir(self) -> str: """ return "Images" + + @property def compatible_backends(self) -> List[Backend]: """ diff --git a/openhcs/microscopes/vendor_layout.py b/openhcs/microscopes/vendor_layout.py index c802a0ed20..6b7544afa2 100644 --- a/openhcs/microscopes/vendor_layout.py +++ b/openhcs/microscopes/vendor_layout.py @@ -26,14 +26,10 @@ class VirtualMappingSource(DatasetSource): """ @abstractmethod - def _build_virtual_mapping( - self, dataset_root: Path, filemanager: FileManager - ) -> Path: + def _build_virtual_mapping(self, dataset_root: Path, filemanager: FileManager) -> Path: """Write this layout's mapping to metadata and return the image directory.""" - def initialize_workspace( - self, dataset_root: Path, filemanager: FileManager - ) -> Path: + def initialize_workspace(self, dataset_root: Path, filemanager: FileManager) -> Path: dataset_root = Path(dataset_root) self.dataset_root = dataset_root self._build_virtual_mapping(dataset_root, filemanager) @@ -105,3 +101,4 @@ def _normalize_mapped_filenames( except Exception as e: logger.error("Error renaming %s to %s: %s", listed_path, new_path, e) return image_dir + diff --git a/openhcs/omero/plugin/omero_openhcs/views.py b/openhcs/omero/plugin/omero_openhcs/views.py index ed76e6b7f4..98211eac7b 100644 --- a/openhcs/omero/plugin/omero_openhcs/views.py +++ b/openhcs/omero/plugin/omero_openhcs/views.py @@ -70,15 +70,11 @@ def ping(self): def get_server_info(self): """Get detailed server info including workers.""" - return self._send_request( - ControlRequestHeader(ControlMessageType.PING).to_dict() - ) + return self._send_request(ControlRequestHeader(ControlMessageType.PING).to_dict()) def get_status(self, execution_id=None): """Get execution status.""" - return self._send_request( - StatusRequest(execution_id=execution_id or None).to_dict() - ) + return self._send_request(StatusRequest(execution_id=execution_id or None).to_dict()) def execute_pipeline(self, plate_id, pipeline_code, config_code): """Execute pipeline on server.""" diff --git a/openhcs/processing/backends/analysis/neurite_outgrowth.py b/openhcs/processing/backends/analysis/neurite_outgrowth.py index fd22c8d424..7f8e638841 100644 --- a/openhcs/processing/backends/analysis/neurite_outgrowth.py +++ b/openhcs/processing/backends/analysis/neurite_outgrowth.py @@ -277,9 +277,7 @@ def analyze( if use_nuclear_stain: nuclear_stain.validate("nuclear_stain") if not 0 <= nuclear_stain.channel_index < image_array.shape[0]: - raise ValueError( - "nuclear_stain.channel_index is outside the input stack" - ) + raise ValueError("nuclear_stain.channel_index is outside the input stack") if nuclear_stain.channel_index == neurite_channel_index: raise ValueError( "nuclear_stain.channel_index must differ from neurite_channel_index" @@ -456,18 +454,14 @@ def analyze( physically_soma_rooted_trace = _physically_soma_rooted_owner_mask( pre_topology_owner_skeleton, cell_body_labels, - maximum_root_distance=_TopologyResult.soma_attachment_radius( - outgrowth_width_px - ), + maximum_root_distance=_TopologyResult.soma_attachment_radius(outgrowth_width_px), ) topology_dropped_trace = ( (pre_topology_owner_skeleton > 0) & (neurite_skeleton == 0) & ~crossing_core_mask ) - topology_added_trace = (neurite_skeleton > 0) & ( - pre_topology_owner_skeleton == 0 - ) + topology_added_trace = (neurite_skeleton > 0) & (pre_topology_owner_skeleton == 0) final_topology_owned_trace_pixels = int(np.count_nonzero(neurite_skeleton)) # A physical crossing core supports two logical paths, but an object-label # raster can store only one identity per pixel. The topology above remains @@ -507,9 +501,7 @@ def analyze( ) secondary_owned_unrooted_trace_pixels = int( np.count_nonzero( - candidate_trace_mask - & ~rooted_trace_mask - & (secondary_owner_regions > 0) + candidate_trace_mask & ~rooted_trace_mask & (secondary_owner_regions > 0) ) ) candidate_mask_pixels = int(np.count_nonzero(candidate_neurite_mask)) @@ -517,9 +509,7 @@ def analyze( np.count_nonzero(candidate_neurite_mask & rooted_mask) ) unrooted_residual_pixels = int(np.count_nonzero(unrooted_residual)) - secondary_owned_residual_pixels = int( - np.count_nonzero(secondary_owned_residual) - ) + secondary_owned_residual_pixels = int(np.count_nonzero(secondary_owned_residual)) cell_results = _build_cell_results( cell_body_labels, @@ -584,14 +574,10 @@ def analyze( final_topology_dropped_unrepresented_trace_pixels=int( np.count_nonzero(topology_dropped_trace & ~topology_path_mask) ), - final_topology_added_trace_pixels=int( - np.count_nonzero(topology_added_trace) - ), + final_topology_added_trace_pixels=int(np.count_nonzero(topology_added_trace)), final_topology_owned_trace_pixels=final_topology_owned_trace_pixels, published_owned_trace_pixels=int(np.count_nonzero(rooted_trace_mask)), - secondary_owned_unrooted_trace_pixels=( - secondary_owned_unrooted_trace_pixels - ), + secondary_owned_unrooted_trace_pixels=(secondary_owned_unrooted_trace_pixels), secondary_unowned_unrooted_trace_pixels=( candidate_trace_pixels - rooted_candidate_trace_pixels @@ -665,14 +651,13 @@ def analyze( ), *admission.planes.selected_outputs(neurite_channel_index), *NeuriteOwnershipPlanes( - initial_owned_trace, - adopted_owned_trace, - pre_topology_owner_skeleton, + initial_owned_trace, adopted_owned_trace, pre_topology_owner_skeleton, ).selected_outputs(neurite_channel_index), neurite_morphology, ) + class PixelMetricCoordinates: """Independent pixel-analysis/export capability composed with the CP recipe.""" @@ -779,8 +764,7 @@ def minimum_area_px(self, coordinate_scale: float) -> float: @staticmethod def _inscribed_disc_support( - radius: np.ndarray, - medial_support: np.ndarray, + radius: np.ndarray, medial_support: np.ndarray, ) -> np.ndarray: """Project original medial discs without adding a new geometric rule.""" support = np.zeros(radius.shape, dtype=bool) @@ -788,12 +772,10 @@ def _inscribed_disc_support( disc_radius = float(radius[row, column]) extent = int(np.ceil(disc_radius)) row_start, row_stop = ( - max(0, row - extent), - min(radius.shape[0], row + extent + 1), + max(0, row - extent), min(radius.shape[0], row + extent + 1), ) column_start, column_stop = ( - max(0, column - extent), - min(radius.shape[1], column + extent + 1), + max(0, column - extent), min(radius.shape[1], column + extent + 1), ) rows, columns = np.ogrid[row_start:row_stop, column_start:column_stop] disc = (rows - row) ** 2 + (columns - column) ** 2 < disc_radius**2 @@ -824,12 +806,9 @@ def separate_terminal_shafts( """ body = np.asarray(body, dtype=bool) radius = ndi.distance_transform_edt(np.pad(body, 1))[1:-1, 1:-1] - skeleton = ( - _raw_processing_leaf(medialaxis)( - np.pad(body, 1).astype(np.float32, copy=False) - )[1:-1, 1:-1] - > 0 - ) + skeleton = _raw_processing_leaf(medialaxis)( + np.pad(body, 1).astype(np.float32, copy=False) + )[1:-1, 1:-1] > 0 distance_to_nucleus = ndi.distance_transform_edt(~nuclear_seed) broad_medial = skeleton & ( (2.0 * radius - 1.0 > maximum_shaft_width_px) @@ -841,12 +820,9 @@ def separate_terminal_shafts( nuclear_regions = np.unique(broad_regions[broad_support & nuclear_seed]) soma_support = broad_support & np.isin(broad_regions, nuclear_regions) protected = skeleton & soma_support - degrees = ( - ndi.convolve( - skeleton.astype(np.int32), connectivity, mode="constant", cval=0 - ) - - skeleton - ) + degrees = ndi.convolve( + skeleton.astype(np.int32), connectivity, mode="constant", cval=0 + ) - skeleton thin_components, count = ndi.label(skeleton & ~protected, connectivity) removed_medial = np.zeros(body.shape, dtype=bool) for component_id in range(1, count + 1): @@ -864,8 +840,7 @@ def separate_terminal_shafts( return body retained_support = self._inscribed_disc_support( - radius, - skeleton & ~removed_medial, + radius, skeleton & ~removed_medial, ) removed_support = self._inscribed_disc_support(radius, removed_medial) retained = body & (~removed_support | retained_support | nuclear_seed) @@ -1254,8 +1229,7 @@ def selected_outputs( ) -> tuple[SelectedDiagnosticPlaneImageOutput, ...]: return tuple( SelectedDiagnosticPlaneImageOutput( - getattr(self, field.name)[None], - (source_index,), + getattr(self, field.name)[None], (source_index,), ) for field in fields(self) ) @@ -1279,6 +1253,7 @@ class NeuriteAdmissionPlanes(NeuriteDiagnosticPlanes): soma_attachment_response: np.ndarray + @dataclass(frozen=True) class NeuriteOwnershipPlanes(NeuriteDiagnosticPlanes): """Original owner labels at each graph stage, before final topology. @@ -1306,9 +1281,7 @@ def response(self) -> np.ndarray: return self.planes.local_response def soma_attachment_labels( - self, - cell_body_labels: np.ndarray, - minimum_response: float, + self, cell_body_labels: np.ndarray, minimum_response: float, ) -> np.ndarray: """Qualify the declared root neighborhood at the soma signal scale. @@ -1317,13 +1290,11 @@ def soma_attachment_labels( neighborhood and nearest accepted soma; it cannot admit remote tracks. """ _, neighborhood = _TopologyResult.soma_neighborhood( - cell_body_labels, - self.outgrowth_width_px, + cell_body_labels, self.outgrowth_width_px, ) return np.where( self.planes.soma_attachment_response >= minimum_response, - neighborhood, - 0, + neighborhood, 0, ).astype(np.int32, copy=False) def skeleton_for(self, cell_body_labels: np.ndarray) -> np.ndarray: @@ -1345,31 +1316,15 @@ def skeleton_for(self, cell_body_labels: np.ndarray) -> np.ndarray: # Both callable ABIs derive diagnostic slots from the same stage declaration. -NeuriteOutgrowthRuntimeTuple = Tuple[ - ( - np.ndarray, - DataclassMeasurementColumnarRows, - DataclassMeasurementColumnarRows, - np.ndarray, - np.ndarray, - np.ndarray, - np.ndarray, - SelectedPlaneImageOutput, - SelectedPlaneImageOutput, - SelectedPlaneImageOutput, - SelectedPlaneImageOutput, - SelectedPlaneImageOutput, - *( - SelectedDiagnosticPlaneImageOutput - for _ in get_type_hints(NeuriteAdmissionPlanes) - ), - *( - SelectedDiagnosticPlaneImageOutput - for _ in get_type_hints(NeuriteOwnershipPlanes) - ), - SpatialGraph, - ) -] +NeuriteOutgrowthRuntimeTuple = Tuple[( + np.ndarray, DataclassMeasurementColumnarRows, DataclassMeasurementColumnarRows, + np.ndarray, np.ndarray, np.ndarray, np.ndarray, + SelectedPlaneImageOutput, SelectedPlaneImageOutput, SelectedPlaneImageOutput, + SelectedPlaneImageOutput, SelectedPlaneImageOutput, + *(SelectedDiagnosticPlaneImageOutput for _ in get_type_hints(NeuriteAdmissionPlanes)), + *(SelectedDiagnosticPlaneImageOutput for _ in get_type_hints(NeuriteOwnershipPlanes)), + SpatialGraph, +)] @dataclass(frozen=True) @@ -1384,12 +1339,9 @@ def supports_owner(self, owner: int, path_owners: np.ndarray) -> bool: return any(int(path_owners[path]) == owner for path in self.arm_paths) def crossover_paths(self, path_owners: np.ndarray) -> tuple[int, ...]: - owners = Counter( - int(path_owners[path]) for path in self.arm_paths if path_owners[path] > 0 - ) - return tuple( - path for path in self.arm_paths if owners[int(path_owners[path])] == 2 - ) + owners = Counter(int(path_owners[path]) for path in self.arm_paths + if path_owners[path] > 0) + return tuple(path for path in self.arm_paths if owners[int(path_owners[path])] == 2) def core_coordinates(self, path_coordinates: tuple[np.ndarray, ...]) -> np.ndarray: if not self.core_paths: @@ -1397,9 +1349,7 @@ def core_coordinates(self, path_coordinates: tuple[np.ndarray, ...]) -> np.ndarr return np.concatenate([path_coordinates[path] for path in self.core_paths]) def regional_core_coordinates( - self, - path_coordinates: tuple[np.ndarray, ...], - shape: tuple[int, int], + self, path_coordinates: tuple[np.ndarray, ...], shape: tuple[int, int], origin: tuple[int, int], ) -> np.ndarray: coordinates = self.core_coordinates(path_coordinates) - origin @@ -1428,26 +1378,20 @@ def soma_attachment_radius(outgrowth_width_px: float) -> int: @classmethod def soma_neighborhood( - cls, - cell_body_labels: np.ndarray, - outgrowth_width_px: float, + cls, cell_body_labels: np.ndarray, outgrowth_width_px: float, ) -> tuple[np.ndarray, np.ndarray]: distance, nearest = ndi.distance_transform_edt( - cell_body_labels == 0, - return_indices=True, + cell_body_labels == 0, return_indices=True, ) labels = np.where( distance <= cls.soma_attachment_radius(outgrowth_width_px), - cell_body_labels[tuple(nearest)], - 0, + cell_body_labels[tuple(nearest)], 0, ) return distance, labels @classmethod def soma_ownership_seeds( - cls, - cell_body_labels: np.ndarray, - outgrowth_width_px: float, + cls, cell_body_labels: np.ndarray, outgrowth_width_px: float, path_coordinates: Sequence[np.ndarray], ) -> dict[int, tuple[int, ...]]: """Near-body evidence seeds ownership, not biological process count.""" @@ -1457,16 +1401,12 @@ def soma_ownership_seeds( labels = neighborhood[tuple(coordinates.T)] labels = labels[labels > 0] if labels.size: - roots[path] = tuple( - label for label, _ in Counter(map(int, labels)).most_common() - ) + roots[path] = tuple(label for label, _ in Counter(map(int, labels)).most_common()) return roots @classmethod def soma_process_roots( - cls, - cell_body_labels: np.ndarray, - outgrowth_width_px: float, + cls, cell_body_labels: np.ndarray, outgrowth_width_px: float, path_coordinates: Sequence[np.ndarray], transitions: Mapping[int, Iterable[int]], path_owners: np.ndarray, @@ -1479,8 +1419,7 @@ def soma_process_roots( distinct exits separated by a more distal junction remain separate. """ distance, neighborhood = cls.soma_neighborhood( - cell_body_labels, - outgrowth_width_px, + cell_body_labels, outgrowth_width_px, ) proximity_by_owner: dict[int, dict[int, float]] = defaultdict(dict) for path, coordinates in enumerate(path_coordinates): @@ -1504,8 +1443,7 @@ def soma_process_roots( while frontier: path = frontier.pop() adjacent = { - neighbor - for neighbor in transitions[path] + neighbor for neighbor in transitions[path] if neighbor in remaining and proximity[neighbor] == proximity[seed] } @@ -1513,9 +1451,9 @@ def soma_process_roots( plateau.update(adjacent) frontier.extend(sorted(adjacent)) if any( - neighbor in proximity and proximity[neighbor] < proximity[seed] - for path in plateau - for neighbor in transitions[path] + neighbor in proximity + and proximity[neighbor] < proximity[seed] + for path in plateau for neighbor in transitions[path] ): continue roots[owner].append(min(plateau)) @@ -1561,15 +1499,12 @@ def classify_owned_endpoints( branch_groups: dict[int, list[int]] = defaultdict(list) for owner, owner_paths in owned_paths.items(): candidates = { - group: paths - for (cell, group), paths in endpoint_paths.items() + group: paths for (cell, group), paths in endpoint_paths.items() if cell == owner and len(paths) >= 3 } if not candidates: continue - coordinates = np.concatenate( - [self.path_coordinates[path] for path in owner_paths] - ) + coordinates = np.concatenate([self.path_coordinates[path] for path in owner_paths]) lower, upper = coordinates.min(axis=0), coordinates.max(axis=0) + 1 body_slice = body_slices[owner - 1] if owner <= len(body_slices) else None if body_slice is not None: @@ -1578,9 +1513,7 @@ def classify_owned_endpoints( margin = self.soma_attachment_radius(outgrowth_width_px) lower = np.maximum(0, lower - margin) upper = np.minimum(cell_body_labels.shape, upper + margin) - region = tuple( - slice(int(start), int(stop)) for start, stop in zip(lower, upper) - ) + region = tuple(slice(int(start), int(stop)) for start, stop in zip(lower, upper)) local_bodies = cell_body_labels[region] own_trace = np.zeros(local_bodies.shape, dtype=np.int32) own_trace[tuple((coordinates - lower).T)] = owner @@ -1593,54 +1526,24 @@ def classify_owned_endpoints( foreground_radius = ndi.distance_transform_edt(foreground) for group, paths in candidates.items(): # Sample physical endpoints, not fractional exported centroids. - centers = np.unique( - np.asarray( - [ - self.path_coordinates[path][0 if endpoint == 0 else -1] - - lower - for path in paths - for endpoint in (0, 1) - if self.path_endpoint_groups[path][endpoint] == group - ] - ), - axis=0, - ) + centers = np.unique(np.asarray([ + self.path_coordinates[path][0 if endpoint == 0 else -1] - lower + for path in paths for endpoint in (0, 1) + if self.path_endpoint_groups[path][endpoint] == group + ]), axis=0) radii = foreground_radius[tuple(centers.T)] neighborhood_radii = radii + outgrowth_width_px - ring_lower = np.maximum( - 0, - np.floor( - (centers - neighborhood_radii[:, None]).min(axis=0) - ).astype(int) - - 2, - ) - ring_upper = np.minimum( - local_bodies.shape, - np.ceil((centers + neighborhood_radii[:, None]).max(axis=0)).astype( - int - ) - + 3, - ) - ring_slice = tuple( - slice(int(start), int(stop)) - for start, stop in zip(ring_lower, ring_upper) - ) - grid = ( - np.moveaxis(np.indices(tuple(ring_upper - ring_lower)), 0, -1) - + ring_lower - ) + ring_lower = np.maximum(0, np.floor((centers - neighborhood_radii[:, None]).min(axis=0)).astype(int) - 2) + ring_upper = np.minimum(local_bodies.shape, np.ceil((centers + neighborhood_radii[:, None]).max(axis=0)).astype(int) + 3) + ring_slice = tuple(slice(int(start), int(stop)) for start, stop in zip(ring_lower, ring_upper)) + grid = np.moveaxis(np.indices(tuple(ring_upper - ring_lower)), 0, -1) + ring_lower distances = np.linalg.norm(grid[:, :, None] - centers, axis=3) for expansion in range(int(np.floor(outgrowth_width_px)) + 1): perimeter_radii = radii + expansion core = np.any(distances <= perimeter_radii, axis=2) - ring = ( - ndi.binary_dilation(core, structure=np.ones((3, 3), dtype=bool)) - & ~core - ) + ring = ndi.binary_dilation(core, structure=np.ones((3, 3), dtype=bool)) & ~core ring &= foreground[ring_slice] & (local_bodies[ring_slice] == 0) - corridors, _ = ndi.label( - ring, structure=np.ones((3, 3), dtype=bool) - ) + corridors, _ = ndi.label(ring, structure=np.ones((3, 3), dtype=bool)) resolved_corridors: set[int] = set() proximal_arm = False for path in paths: @@ -1648,44 +1551,26 @@ def classify_owned_endpoints( for endpoint in (0, 1): if groups[endpoint] != group: continue - arm = ( - self.path_coordinates[path][ - :: 1 if endpoint == 0 else -1 - ] - - lower - ) - inside_core = np.any( - np.linalg.norm(arm[:, None] - centers, axis=2) - <= perimeter_radii, - axis=1, - ) + arm = self.path_coordinates[path][::1 if endpoint == 0 else -1] - lower + inside_core = np.any(np.linalg.norm(arm[:, None] - centers, axis=2) <= perimeter_radii, axis=1) departures = np.flatnonzero(~inside_core) if departures.size: first_exit = arm[departures[0]] - ring_lower - if np.all( - (first_exit >= 0) & (first_exit < corridors.shape) - ): + if np.all((first_exit >= 0) & (first_exit < corridors.shape)): corridor = int(corridors[tuple(first_exit)]) if corridor: resolved_corridors.add(corridor) elif groups[0] != groups[1]: # Only the opposite endpoint can be a short root. neighbors = arm[-1] + body_neighbor_offsets - inside = np.all( - (neighbors >= 0) & (neighbors < local_bodies.shape), - axis=1, - ) - if np.any( - local_bodies[tuple(neighbors[inside].T)] == owner - ): + inside = np.all((neighbors >= 0) & (neighbors < local_bodies.shape), axis=1) + if np.any(local_bodies[tuple(neighbors[inside].T)] == owner): proximal_arm = True if len(resolved_corridors) + int(proximal_arm) >= 3: branch_groups[owner].append(group) break qualified_groups = { - (owner, group) - for owner, groups in branch_groups.items() - for group in groups + (owner, group) for owner, groups in branch_groups.items() for group in groups } branch_types = np.zeros(len(self.path_owners), dtype=np.int32) for path_index, groups in enumerate(self.path_endpoint_groups): @@ -1693,9 +1578,9 @@ def classify_owned_endpoints( if owner <= 0 or path_index in crossing_core_paths: continue branch_types[path_index] = ( - 3 - if groups[0] == groups[1] - else sum((owner, group) in qualified_groups for group in groups) + 3 if groups[0] == groups[1] else sum( + (owner, group) in qualified_groups for group in groups + ) ) coordinates = { group: np.asarray(coordinate) @@ -1712,50 +1597,35 @@ def crossing_nodes(self) -> frozenset[int]: @property def crossing_paths(self) -> frozenset[int]: - return frozenset( - path - for crossing in self.resolved_crossings - for path in crossing.crossover_paths(self.path_owners) - ) + return frozenset(path for crossing in self.resolved_crossings + for path in crossing.crossover_paths(self.path_owners)) @property def crossing_core_paths(self) -> frozenset[int]: - return frozenset( - path for crossing in self.resolved_crossings for path in crossing.core_paths - ) + return frozenset(path for crossing in self.resolved_crossings + for path in crossing.core_paths) def crossing_core_coordinates(self, *, owner: int) -> np.ndarray: - coordinates = [ - crossing.core_coordinates(self.path_coordinates) - for crossing in self.resolved_crossings - if crossing.supports_owner(owner, self.path_owners) - ] - return ( - np.concatenate(coordinates) if coordinates else np.empty((0, 2), dtype=int) - ) + coordinates = [crossing.core_coordinates(self.path_coordinates) + for crossing in self.resolved_crossings + if crossing.supports_owner(owner, self.path_owners)] + return np.concatenate(coordinates) if coordinates else np.empty((0, 2), dtype=int) def crossing_core_mask( - self, - shape: tuple[int, int], - *, + self, shape: tuple[int, int], *, origin: tuple[int, int] = (0, 0), ) -> np.ndarray: """Project the physical cores reserved by resolved logical crossings.""" mask = np.zeros(shape, dtype=bool) for crossing in self.resolved_crossings: coordinates = crossing.regional_core_coordinates( - self.path_coordinates, - shape, - origin, + self.path_coordinates, shape, origin, ) mask[tuple(coordinates.T)] = True return mask def crossing_support_mask( - self, - occupied_owners: np.ndarray, - *, - owner: int, + self, occupied_owners: np.ndarray, *, owner: int, origin: tuple[int, int], ) -> np.ndarray: """Project one logical owner's core without overwriting foreign input.""" @@ -1764,9 +1634,7 @@ def crossing_support_mask( if not crossing.supports_owner(owner, self.path_owners): continue coordinates = crossing.regional_core_coordinates( - self.path_coordinates, - occupied_owners.shape, - origin, + self.path_coordinates, occupied_owners.shape, origin, ) current = occupied_owners[tuple(coordinates.T)] arm_owners = self.path_owners[list(crossing.arm_paths)] @@ -2199,7 +2067,8 @@ def _cell_body_contract_candidates( ) if ( region.area >= minimum_area_px - and maximum_inscribed_diameter_px >= minimum_inscribed_diameter_px + and maximum_inscribed_diameter_px + >= minimum_inscribed_diameter_px and region.axis_minor_length <= maximum_width_px and region_response.size and float(np.mean(region_response)) >= intensity_threshold @@ -2307,20 +2176,14 @@ def _identify_neurites_cellprofiler( ) local_support = response >= settings.intensity_above_local_background soma_attachment_response = local_background_response( - image, - object_width_px=body_width_px, - bright_objects=bright_objects, + image, object_width_px=body_width_px, bright_objects=bright_objects, ) outgrowth_mask = cp_mask & local_support return NeuriteAdmissionResult( outgrowth_mask, NeuriteAdmissionPlanes( - np.asarray(enhanced), - threshold_support, - cp_mask, - response, - local_support, - soma_attachment_response, + np.asarray(enhanced), threshold_support, cp_mask, + response, local_support, soma_attachment_response, ), outgrowth_width_px, ) @@ -2599,13 +2462,7 @@ def _repair_signal_supported_skeleton( shaft_support = np.asarray(admission.mask, dtype=bool) regions = np.asarray(owner_regions, dtype=np.int32) bodies = np.asarray(cell_body_labels, dtype=np.int32) - if ( - not repaired.shape - == response.shape - == shaft_support.shape - == regions.shape - == bodies.shape - ): + if not repaired.shape == response.shape == shaft_support.shape == regions.shape == bodies.shape: raise ValueError( "labels, admission, owner_regions, and cell_body_labels must " "have the same shape" @@ -2651,18 +2508,14 @@ def _repair_signal_supported_skeleton( original_owner = (local_repaired == owner) & ~body_mask origin = (owner_slice[0].start, owner_slice[1].start) local_core = crossing_topology.crossing_core_mask( - local_repaired.shape, - origin=origin, + local_repaired.shape, origin=origin, ) local_shared_core = crossing_topology.crossing_support_mask( - labels[owner_slice], - owner=owner, - origin=origin, + labels[owner_slice], owner=owner, origin=origin, ) original_core_values = local_repaired[local_shared_core].copy() shared_signal_support = ( - local_shared_core - & (local_response >= minimum_response) + local_shared_core & (local_response >= minimum_response) & ((local_bodies == 0) | body_mask) ) local_repaired[(local_repaired == owner) & body_mask] = 0 @@ -2690,16 +2543,9 @@ def _repair_signal_supported_skeleton( signal_support = local_shaft_support & (local_regions == owner) root_support = soma_attachments[owner_slice] == owner allowed = ( - ( - signal_support - | original_owner - | body_mask - | shared_signal_support - | root_support - ) - & ~occupied_by_other_owner - & ~foreign_body - & (~local_core | shared_signal_support | original_owner | body_mask) + signal_support | original_owner | body_mask | shared_signal_support | root_support + ) & ~occupied_by_other_owner & ~foreign_body & ( + ~local_core | shared_signal_support | original_owner | body_mask ) while True: @@ -2987,9 +2833,7 @@ def _analyze_topology( transitions[second].add(first) root_labels_by_path = _TopologyResult.soma_ownership_seeds( - cell_body_labels, - outgrowth_width_px, - path_coordinates, + cell_body_labels, outgrowth_width_px, path_coordinates, ) path_owners, path_distances = _propagate_path_owners( @@ -3049,16 +2893,11 @@ def _analyze_topology( roots_by_cell, ) process_roots = _TopologyResult.soma_process_roots( - cell_body_labels, - outgrowth_width_px, - path_coordinates, - transitions, - path_owners, + cell_body_labels, outgrowth_width_px, path_coordinates, transitions, path_owners, ) resolved_crossings = tuple( - _ResolvedCrossing( - node, crossing_paths_by_node[node], crossing_core_paths_by_node[node] - ) + _ResolvedCrossing(node, crossing_paths_by_node[node], + crossing_core_paths_by_node[node]) for node in sorted(crossing_nodes) ) topology = _TopologyResult( @@ -3076,22 +2915,15 @@ def _analyze_topology( root_paths_by_cell=process_roots, branch_nodes_by_cell={}, resolved_crossings=tuple( - crossing - for crossing in resolved_crossings + crossing for crossing in resolved_crossings if crossing.crossover_paths(path_owners) ), ) path_branch_types, branch_nodes_by_cell = topology.classify_owned_endpoints( - branch_support=branch_support, - cell_body_labels=cell_body_labels, - outgrowth_width_px=outgrowth_width_px, - merge_radius=max(1.0, outgrowth_width_px), - ) - return replace( - topology, - path_branch_types=path_branch_types, - branch_nodes_by_cell=branch_nodes_by_cell, + branch_support=branch_support, cell_body_labels=cell_body_labels, + outgrowth_width_px=outgrowth_width_px, merge_radius=max(1.0, outgrowth_width_px), ) + return replace(topology, path_branch_types=path_branch_types, branch_nodes_by_cell=branch_nodes_by_cell) def _analyze_owned_topology( @@ -3157,8 +2989,7 @@ def _analyze_owned_topology( ) local_owner_mask = owned[owner_slice] == owner local_owner_mask |= crossing_topology.crossing_support_mask( - owned[owner_slice], - owner=owner, + owned[owner_slice], owner=owner, origin=(owner_slice[0].start, owner_slice[1].start), ) local_owned = np.where(local_owner_mask, owner, 0).astype( @@ -3225,11 +3056,9 @@ def _analyze_owned_topology( } ) resolved_crossings.extend( - _ResolvedCrossing( - crossing.node + node_offset, - tuple(path + path_offset for path in crossing.arm_paths), - tuple(path + path_offset for path in crossing.core_paths), - ) + _ResolvedCrossing(crossing.node + node_offset, + tuple(path + path_offset for path in crossing.arm_paths), + tuple(path + path_offset for path in crossing.core_paths)) for crossing in local.resolved_crossings ) @@ -3588,7 +3417,9 @@ def _build_neurite_morphology_graph( root_node = SpatialGraphNode.from_features( node_id=next_node_id, coordinates=root_coordinate, - radius=(soma_radius if root_role == "soma_root" else process_radius), + radius=( + soma_radius if root_role == "soma_root" else process_radius + ), features={ "label": owner, "neuron_label": owner, @@ -3676,7 +3507,9 @@ def enqueue_from(group_id: int) -> None: target_node = nodes_by_group[target_group] coordinates[-1] = target_node.coordinates branch_distance = float(topology.path_lengths[path_index]) - euclidean_distance = float(topology.path_euclidean_lengths[path_index]) + euclidean_distance = float( + topology.path_euclidean_lengths[path_index] + ) graph_edges.append( SpatialGraphEdge.from_features( edge_id=next_edge_id, @@ -4050,7 +3883,9 @@ def _build_summary( number_of_cells=cell_count, coordinate_unit=coordinate_unit, total_outgrowth=total_outgrowth, - mean_outgrowth_per_cell=(total_outgrowth / cell_count if cell_count else 0.0), + mean_outgrowth_per_cell=( + total_outgrowth / cell_count if cell_count else 0.0 + ), total_processes=total_processes, mean_processes_per_cell=(total_processes / cell_count if cell_count else 0.0), total_branches=total_branches, diff --git a/openhcs/processing/backends/cellprofiler/alignment.py b/openhcs/processing/backends/cellprofiler/alignment.py index 16cea63739..071ce95ca6 100644 --- a/openhcs/processing/backends/cellprofiler/alignment.py +++ b/openhcs/processing/backends/cellprofiler/alignment.py @@ -500,6 +500,7 @@ class AlignmentBackendStrategy( ): """Alignment operations keyed by OpenHCS memory type/provider.""" + @abstractmethod def mutual_information_offset( self, @@ -782,7 +783,9 @@ def execute( ) for index in range(plane_count) ) - images = tuple(np.asarray(payload.data) for payload in image_payloads) + images = tuple( + np.asarray(payload.data) for payload in image_payloads + ) first_image, second_image = images[:2] metadata = tuple(payload.metadata for payload in image_payloads) masks = tuple( diff --git a/openhcs/processing/backends/cellprofiler/area_occupied.py b/openhcs/processing/backends/cellprofiler/area_occupied.py index d6c7114097..dfbd07989f 100644 --- a/openhcs/processing/backends/cellprofiler/area_occupied.py +++ b/openhcs/processing/backends/cellprofiler/area_occupied.py @@ -27,13 +27,7 @@ MeasurementSparseColumnarRows, ) from openhcs.core.memory.decorators import numpy -from openhcs.core.pipeline.function_contracts import ( - composed_image_payload, - object_label_input_execution_mode, - resolved_callable_parameter, - runtime_bound_parameters, - special_inputs, -) +from openhcs.core.pipeline.function_contracts import (composed_image_payload, object_label_input_execution_mode, resolved_callable_parameter, runtime_bound_parameters, special_inputs) from openhcs.core.runtime_array_values import RuntimeArrayData from openhcs.core.runtime_batch_contracts import SliceIndexRuntimeParameter from openhcs.core.runtime_measurements import ( @@ -174,15 +168,12 @@ def primary_image_domain_input_binding(cls) -> SettingToKeywordBinding: @classmethod def primary_image_inputs( - cls, - contract: CallableContract, - declared_inputs: tuple[ArtifactSpec, ...], + cls, contract: CallableContract, declared_inputs: tuple[ArtifactSpec, ...], ) -> tuple[ArtifactSpec, ...]: """Binary operands retain their declared main-image domain.""" del cls, contract return tuple( - spec - for spec in declared_inputs + spec for spec in declared_inputs if spec.artifact_type is ImageArtifactType and spec.parameter_name is None and not spec.stack_broadcast_sources() @@ -190,8 +181,7 @@ def primary_image_inputs( @classmethod def primary_image_domain_specs( - cls, - domain_inputs: tuple[ArtifactSpec, ...], + cls, domain_inputs: tuple[ArtifactSpec, ...], ) -> tuple[ArtifactSpec, ...]: """Each object operand owns its independent source-qualified row.""" if not domain_inputs: @@ -200,28 +190,21 @@ def primary_image_domain_specs( @classmethod def invocation_domain_inputs( - cls, - contract: CallableContract, - declared_inputs: tuple[ArtifactSpec, ...], + cls, contract: CallableContract, declared_inputs: tuple[ArtifactSpec, ...], ) -> tuple[ArtifactSpec, ...]: image_inputs = cls.primary_image_inputs(contract, declared_inputs) - return image_inputs or super().invocation_domain_inputs( - contract, declared_inputs - ) + return image_inputs or super().invocation_domain_inputs(contract, declared_inputs) @classmethod def project_invocation_image_request( - cls, - *, - image_request: CellProfilerImageRequest, + cls, *, image_request: CellProfilerImageRequest, runtime_kwargs: RuntimeCallableKwargs, ) -> CellProfilerImageRequest: rows = runtime_kwargs[AreaOccupiedRowsRuntimeParameter.require_parameter_name()] if any(row.operand is OperandChoice.BINARY_IMAGE for row in rows): return image_request return super().project_invocation_image_request( - image_request=image_request, - runtime_kwargs=runtime_kwargs, + image_request=image_request, runtime_kwargs=runtime_kwargs, ) @classmethod @@ -909,8 +892,7 @@ def _binary_images_from_payload( ) return tuple( np.asarray( - ( - RuntimeSliceProjection.value_for_slice( + (RuntimeSliceProjection.value_for_slice( image, RuntimePlaneAxisValueProjection.from_selected_plane( axis=RuntimePlaneAxis.SOURCE_BINDING, @@ -918,8 +900,7 @@ def _binary_images_from_payload( axis_size=projection.axis_size, source_aliases=projection.source_aliases, ), - ) - ).data + )).data ) for index in range(projection.axis_size) ) diff --git a/openhcs/processing/backends/cellprofiler/calculate_statistics.py b/openhcs/processing/backends/cellprofiler/calculate_statistics.py index 62489963e3..16b74ec1da 100644 --- a/openhcs/processing/backends/cellprofiler/calculate_statistics.py +++ b/openhcs/processing/backends/cellprofiler/calculate_statistics.py @@ -419,10 +419,7 @@ def _statistics_feature_series( """Enumerate numeric image and averaged-object features from typed tables.""" series_by_identity: dict[tuple[str, str], dict[int, float]] = {} for table in measurement_tables: - if table.subject.scope not in { - MeasurementScope.SAMPLE, - MeasurementScope.OBJECT, - }: + if table.subject.scope not in {MeasurementScope.SAMPLE, MeasurementScope.OBJECT}: continue semantics = ColumnarMeasurementTableSchema.from_table(table) object_names = ( diff --git a/openhcs/processing/backends/cellprofiler/classification.py b/openhcs/processing/backends/cellprofiler/classification.py index 9d98ac0b56..a1ac8b4ed9 100644 --- a/openhcs/processing/backends/cellprofiler/classification.py +++ b/openhcs/processing/backends/cellprofiler/classification.py @@ -111,9 +111,7 @@ class _ClassificationRuntimeInputParameter(KeywordRuntimeParameter, ABC): @classmethod def bind_runtime_inputs( - cls, - request: RuntimeInputBindingRequest, - object_spec: ArtifactSpec, + cls, request: RuntimeInputBindingRequest, object_spec: ArtifactSpec, labels: ObjectLabelValue, ) -> dict[str, RuntimeCallableArgument]: if not cls.runtime_binding_active(request): @@ -127,9 +125,7 @@ def runtime_binding_active(cls, request: RuntimeInputBindingRequest) -> bool: @classmethod @abstractmethod def runtime_input_values( - cls, - request: RuntimeInputBindingRequest, - object_spec: ArtifactSpec, + cls, request: RuntimeInputBindingRequest, object_spec: ArtifactSpec, labels: ObjectLabelValue, ) -> dict[str, RuntimeCallableArgument]: """Bind this declaration's contribution, cooperatively where composed.""" @@ -153,9 +149,7 @@ class _ClassifiedImageRuleIndicesRuntimeParameter(_ClassificationRuntimeInputPar @classmethod def runtime_input_values( - cls, - request: RuntimeInputBindingRequest, - object_spec: ArtifactSpec, + cls, request: RuntimeInputBindingRequest, object_spec: ArtifactSpec, labels: ObjectLabelValue, ) -> dict[str, RuntimeCallableArgument]: image_outputs = request.adapter.request.require_callable_contract().artifact_outputs.of_artifact_type( @@ -178,9 +172,7 @@ def runtime_input_values( return {cls.require_parameter_name(): tuple(indices)} -class _ClassificationMeasurementVectorRuntimeParameter( - _ClassificationRuntimeInputParameter -): +class _ClassificationMeasurementVectorRuntimeParameter(_ClassificationRuntimeInputParameter): """Bind a declared feature through the original object-vector authority.""" annotation_type = np.ndarray | None @@ -256,9 +248,7 @@ def classification_method(cls) -> ClassificationMethod: @classmethod def runtime_binding_active(cls, request: RuntimeInputBindingRequest) -> bool: - return not _ClassificationRuleValuesRuntimeParameter.runtime_binding_active( - request - ) + return not _ClassificationRuleValuesRuntimeParameter.runtime_binding_active(request) @classmethod def measurement_feature_binding(cls) -> MeasurementFeatureSettingBinding: @@ -325,9 +315,7 @@ def runtime_binding_active(cls, request: RuntimeInputBindingRequest) -> bool: @classmethod def runtime_input_values( - cls, - request: RuntimeInputBindingRequest, - object_spec: ArtifactSpec, + cls, request: RuntimeInputBindingRequest, object_spec: ArtifactSpec, labels: ObjectLabelValue, ) -> dict[str, RuntimeCallableArgument]: return { @@ -336,13 +324,10 @@ def runtime_input_values( request, object_ref=object_spec, feature_name=_classification_rule_measurement_feature( - rule, - request.adapter.request.require_callable_contract().module_name, + rule, request.adapter.request.require_callable_contract().module_name ), labels=labels, - ) - .vector() - .runtime_value + ).vector().runtime_value for rule in cls.rules_for_request(request) ) } @@ -656,9 +641,7 @@ def object_class_rows( def _classification_kwargs( cls, module: "ModuleBlock", binder: "SettingsBinder" ) -> "RuntimeCallableKwargs": - return ClassificationMethod.from_module(cls, module).bind_settings( - cls, module, binder - ) + return ClassificationMethod.from_module(cls, module).bind_settings(cls, module, binder) @classmethod def bind_settings(cls, module, *, binder): @@ -739,9 +722,9 @@ def finalize_module_blocks_for_invocation( invocation=invocation, step_context=step_context, ) - return ClassificationMethod.from_callable_contract( - invocation.contract - ).finalize_module_blocks(cls, blocks, invocation) + return ClassificationMethod.from_callable_contract(invocation.contract).finalize_module_blocks( + cls, blocks, invocation + ) @classmethod def _block_with_scalar_classification_output( @@ -1141,9 +1124,7 @@ def classified_image_outputs( ) -> tuple[ClassifiedImageOutput, ...]: """Return every active repeated classified-image row in rule order.""" - return ClassificationMethod.from_module(cls, module).classified_image_outputs( - cls, module - ) + return ClassificationMethod.from_module(cls, module).classified_image_outputs(cls, module) @classmethod def _single_measurement_classified_image_outputs( @@ -1330,27 +1311,21 @@ class _ClassificationMethodBehavior(ABC): @abstractmethod def bind_settings( - self, - module_type: type[ClassifyObjectsSingleMeasurementModule], - module: ModuleBlock, - binder: SettingsBinder, + self, module_type: type[ClassifyObjectsSingleMeasurementModule], + module: ModuleBlock, binder: SettingsBinder, ) -> RuntimeCallableKwargs: """Bind this mode through the concrete module's existing setting hook.""" @abstractmethod def classified_image_outputs( - self, - module_type: type[ClassifyObjectsSingleMeasurementModule], - module: ModuleBlock, + self, module_type: type[ClassifyObjectsSingleMeasurementModule], module: ModuleBlock, ) -> tuple[ClassifiedImageOutput, ...]: """Project this mode's concrete module image-output hook.""" @abstractmethod def finalize_mode_blocks( - self, - module_type: type[ClassifyObjectsSingleMeasurementModule], - blocks: tuple[ModuleBlock, ...], - invocation: NormalizedFunctionItem, + self, module_type: type[ClassifyObjectsSingleMeasurementModule], + blocks: tuple[ModuleBlock, ...], invocation: NormalizedFunctionItem, ) -> tuple[ModuleBlock, ...]: """Complete this mode after the shared template emits its literal.""" @@ -1368,8 +1343,7 @@ def finalize_mode_blocks(self, module_type, blocks, invocation): rules = invocation.kwargs_dict.get("classification_rules") if rules is not None: if not isinstance(rules, tuple) or any( - not isinstance(rule, SingleMeasurementClassificationRule) - for rule in rules + not isinstance(rule, SingleMeasurementClassificationRule) for rule in rules ): raise TypeError( "ClassifyObjects classification_rules must be a tuple of " @@ -1427,9 +1401,7 @@ class ClassificationMethod(Enum): cellprofiler_literals: tuple[str, ...] def __new__( - cls, - absorbed_value: str, - behavior: _ClassificationMethodBehavior, + cls, absorbed_value: str, behavior: _ClassificationMethodBehavior, *cellprofiler_literals: str, ) -> "ClassificationMethod": member = enum_member_with_payload( @@ -1455,9 +1427,7 @@ def __new__( @classmethod def from_module( - cls, - module_type: type[ClassifyObjectsSingleMeasurementModule], - module: ModuleBlock, + cls, module_type: type[ClassifyObjectsSingleMeasurementModule], module: ModuleBlock, ) -> ClassificationMethod: return coerce_cellprofiler_enum( cls, @@ -1482,18 +1452,14 @@ def from_callable_contract(cls, contract: CallableContract) -> ClassificationMet return next(iter(methods)) def require_callable( - self, - module_type: type[ClassifyObjectsSingleMeasurementModule], + self, module_type: type[ClassifyObjectsSingleMeasurementModule], ) -> Callable[..., object]: matches = tuple( func for name in module_type.declared_function_names() if self.from_callable_contract( - CallableContract.from_callable( - func := module_type.require_callable(name) - ) - ) - is self + CallableContract.from_callable(func := module_type.require_callable(name)) + ) is self ) if len(matches) != 1: raise ValueError( @@ -1502,25 +1468,19 @@ def require_callable( return matches[0] def bind_settings( - self, - module_type: type[ClassifyObjectsSingleMeasurementModule], - module: ModuleBlock, - binder: SettingsBinder, + self, module_type: type[ClassifyObjectsSingleMeasurementModule], + module: ModuleBlock, binder: SettingsBinder, ) -> RuntimeCallableKwargs: return self._behavior.bind_settings(module_type, module, binder) def classified_image_outputs( - self, - module_type: type[ClassifyObjectsSingleMeasurementModule], - module: ModuleBlock, + self, module_type: type[ClassifyObjectsSingleMeasurementModule], module: ModuleBlock, ) -> tuple[ClassifiedImageOutput, ...]: return self._behavior.classified_image_outputs(module_type, module) def finalize_module_blocks( - self, - module_type: type[ClassifyObjectsSingleMeasurementModule], - blocks: tuple[ModuleBlock, ...], - invocation: NormalizedFunctionItem, + self, module_type: type[ClassifyObjectsSingleMeasurementModule], + blocks: tuple[ModuleBlock, ...], invocation: NormalizedFunctionItem, ) -> tuple[ModuleBlock, ...]: blocks = tuple( replace( @@ -1531,11 +1491,9 @@ def finalize_module_blocks( cellprofiler_setting_literal(self.cellprofiler_literals[1]), ), *( - record - for record in block.iter_settings() + record for record in block.iter_settings() if not setting_name_matches( - record.name, - module_type.classification_decision_count_setting, + record.name, module_type.classification_decision_count_setting ) ), ], @@ -1991,6 +1949,7 @@ class ObjectClassificationBackendStrategy( ): """Object classification primitives keyed by memory type/provider.""" + @abstractmethod def positive_label_ids(self, labels: np.ndarray) -> np.ndarray: """Return positive label ids present in ``labels``.""" @@ -2147,10 +2106,8 @@ def classify_objects_single_measurement( measurement_values=rule_values, ).classify(image, labels, backend) classified_images.append( - image.with_pixels( - classification_rgb_image(classified_labels), - metadata=classified_image_metadata, - ) + image.with_pixels(classification_rgb_image(classified_labels), + metadata=classified_image_metadata,) ) results.append(result) configured_output_indices = tuple( @@ -2197,10 +2154,8 @@ def classify_objects_single_measurement( f"{configured_output_indices!r}." ) output = ( - image.with_pixels( - classification_rgb_image(classified_labels), - metadata=classified_image_metadata, - ) + image.with_pixels(classification_rgb_image(classified_labels), + metadata=classified_image_metadata,) if classified_image_rule_indices else labels ) diff --git a/openhcs/processing/backends/cellprofiler/colocalization.py b/openhcs/processing/backends/cellprofiler/colocalization.py index 17fd94d66a..ad162a63f1 100644 --- a/openhcs/processing/backends/cellprofiler/colocalization.py +++ b/openhcs/processing/backends/cellprofiler/colocalization.py @@ -17,10 +17,7 @@ from openhcs.core.axes import ColourAxis, TileAxis from openhcs.constants.constants import MemoryType -from openhcs.core.aligned_image_payload import ( - AlignedImageStack, - pack_aligned_image_outputs, -) +from openhcs.core.aligned_image_payload import (AlignedImageStack, pack_aligned_image_outputs) from openhcs.core.artifacts import ( ArtifactSpecCollection, ArtifactSpecRelation, @@ -39,14 +36,7 @@ MeasurementProjectedColumnarRows, ObjectMeasurementColumnarRows, ) -from openhcs.core.pipeline.function_contracts import ( - composed_image_payload, - object_label_input_execution_mode, - required_axis_roles, - resolved_callable_parameter, - runtime_bound_parameters, - special_inputs, -) +from openhcs.core.pipeline.function_contracts import (composed_image_payload, object_label_input_execution_mode, required_axis_roles, resolved_callable_parameter, runtime_bound_parameters, special_inputs) from python_introspect import public_names_from_objects from openhcs.core.runtime_batch_contracts import ( RuntimeBatchInvocationRequest, @@ -343,11 +333,7 @@ def complete_table_measurement_rows( func = request.callable_contract.resolve_canonical_raw_callable() source_names = request.source.source_aliases channel_indexes = tuple( - int( - request.kwargs.get( - name, resolved_callable_parameter(func, name).default - ) - ) + int(request.kwargs.get(name, resolved_callable_parameter(func, name).default)) for name in ("channel_1", "channel_2") ) first_index, second_index = channel_indexes @@ -1302,6 +1288,7 @@ class ColocalizationCostesBackendStrategy( ): """Costes thresholding primitives keyed by OpenHCS memory/provider.""" + @abstractmethod def linear_costes( self, @@ -2632,7 +2619,9 @@ def measure_colocalization( time.perf_counter() - phase_started_at, function="measure_colocalization", scale_max=options.scale_max, - source_plane_intensity_scales=(image.metadata.source_plane_intensity_scales), + source_plane_intensity_scales=( + image.metadata.source_plane_intensity_scales + ), intensity_scale=image.metadata.intensity_scale, payload_type=type(image).__name__, ) @@ -3015,10 +3004,8 @@ def _colocalization_source_plane_output( "MeasureColocalization source channels require the declared " f"source-binding plane axis, got {metadata.plane_axis.value!r}." ) - source_plane = image.slice_payload( - image_data[source_channel_index], - source_channel_index, - ) + source_plane = image.slice_payload(image_data[source_channel_index], + source_channel_index,) return source_plane.with_pixels(output_data) diff --git a/openhcs/processing/backends/cellprofiler/color.py b/openhcs/processing/backends/cellprofiler/color.py index cfd08f4497..a265279158 100644 --- a/openhcs/processing/backends/cellprofiler/color.py +++ b/openhcs/processing/backends/cellprofiler/color.py @@ -110,10 +110,8 @@ def channel_output( self, image: RuntimeArrayData, *, channel_index: int, output: RuntimeArrayData ) -> RuntimeArrayData: del channel_index - return image.with_pixels( - output, - metadata=image.metadata.without_axis(ColourAxis), - ) + return image.with_pixels(output, + metadata=image.metadata.without_axis(ColourAxis),) class ImageChannelType(Enum): @@ -384,8 +382,7 @@ def _canonical_output_names( return tuple(f"{base_name}Channel{index + 1}" for index in channel_indices) declaration = cls.fixed_image_type(image_type) if any( - index < 0 or index >= len(declaration.fixed_channels) - for index in channel_indices + index < 0 or index >= len(declaration.fixed_channels) for index in channel_indices ): raise ValueError( f"ColorToGray channel indices must address {image_type.value} " @@ -875,8 +872,7 @@ def authoring_default_kwargs(cls, module, *, authored_kwargs): from openhcs.interop.cellprofiler.settings_binder import SettingsBinder inherited = super().authoring_default_kwargs( - module, - authored_kwargs=authored_kwargs, + module, authored_kwargs=authored_kwargs, ) if not any( binding.require_parameter_name() in authored_kwargs @@ -1655,17 +1651,13 @@ def color_to_gray( """ if mode is ColorToGrayMode.COMBINE: output = combine_color_to_gray(image, channel_indices, contributions) - return image.with_pixels( - output, - metadata=color_to_gray_combine_output_metadata(image), - ) + return image.with_pixels(output, + metadata=color_to_gray_combine_output_metadata(image),) outputs = split_color_to_gray(image, image_type, channel_indices) return pack_aligned_image_outputs( tuple( image_type.scalar_output_projection.channel_output( - image, - channel_index=channel_index, - output=output, + image, channel_index=channel_index, output=output, ) for channel_index, output in zip(channel_indices, outputs, strict=True) ) @@ -1744,10 +1736,8 @@ def _invert_for_printing_result( ) if output_mode is OutputMode.COLOR: outputs = ( - image.with_pixels( - np.stack(inverted_channels, axis=-1), - metadata=image.metadata.with_axis(ColourSampleAxisSpec(), -1), - ), + image.with_pixels(np.stack(inverted_channels, axis=-1), + metadata=image.metadata.with_axis(ColourSampleAxisSpec(), -1),), ) output_names = (color_output_name,) else: @@ -1763,10 +1753,8 @@ def _invert_for_printing_result( ) output_metadata = image.metadata.without_axis(ColourAxis) outputs = tuple( - image.with_pixels( - output, - metadata=output_metadata, - ) + image.with_pixels(output, + metadata=output_metadata,) for enabled, output in zip( enabled_outputs, inverted_channels, @@ -1921,25 +1909,22 @@ def combine_color_to_gray( channels = np.asarray(channel_indices, dtype=int) weights = np.asarray(contributions, dtype=float) / float(sum(contributions)) result = _indexed_weighted_projection( - color_stack, - channels, - weights, - axis=-1, + color_stack, channels, weights, axis=-1, ) return restore_color_to_gray_shape(image, result) def color_to_gray_combine_output_metadata(image: RuntimeArrayData): """Return metadata for a color-to-grayscale semantic collapse.""" - return image.metadata.without_unit_interval_intensity_scale().without_axis( - ColourAxis + return ( + image.metadata + .without_unit_interval_intensity_scale() + .without_axis(ColourAxis) ) def split_color_to_gray( - image: RuntimeArrayData, - image_type: ImageChannelType, - channel_indices: tuple[int, ...], + image: RuntimeArrayData, image_type: ImageChannelType, channel_indices: tuple[int, ...] ) -> tuple[np.ndarray, ...]: color_stack = nhwc_color_stack(image).astype(np.float32) source_stack = ( @@ -1996,9 +1981,7 @@ def nhwc_color_stack(image: RuntimeArrayData) -> np.ndarray: return channel_last -def restore_color_to_gray_shape( - original: RuntimeArrayData, stack: np.ndarray -) -> np.ndarray: +def restore_color_to_gray_shape(original: RuntimeArrayData, stack: np.ndarray) -> np.ndarray: metadata = original.metadata if metadata.declares_colour_samples_plane(original): if stack.shape[0] != 1: @@ -2101,10 +2084,8 @@ def unmix_colors( if stain_names: return pack_aligned_image_outputs( tuple( - image.with_pixels( - output, - metadata=output_metadata, - ) + image.with_pixels(output, + metadata=output_metadata,) for output in _unmix_stain_outputs( rgb_image, _stain_definitions(stain_names, custom_absorbances), @@ -2138,10 +2119,8 @@ def unmix_colors( raise ValueError( f"output_stain_index must be in [0, {len(outputs) - 1}], got {output_stain_index}." ) - return image.with_pixels( - outputs[output_stain_index], - metadata=output_metadata, - ) + return image.with_pixels(outputs[output_stain_index], + metadata=output_metadata,) def _stain_definitions( diff --git a/openhcs/processing/backends/cellprofiler/crop.py b/openhcs/processing/backends/cellprofiler/crop.py index f3fc9ea606..85312c48df 100644 --- a/openhcs/processing/backends/cellprofiler/crop.py +++ b/openhcs/processing/backends/cellprofiler/crop.py @@ -375,7 +375,10 @@ def finalize_artifact_contract_inputs( ImageArtifactType, ) (topology_input,) = topology_inputs - if shape is cls.Shape.CROPPING and topology_input.sidecar_role is not CropMask: + if ( + shape is cls.Shape.CROPPING + and topology_input.sidecar_role is not CropMask + ): raise ValueError( f"Crop({module.module_num}) requires one exact crop-mask sidecar, " f"got {topology_input!r}." diff --git a/openhcs/processing/backends/cellprofiler/display_modules.py b/openhcs/processing/backends/cellprofiler/display_modules.py index bcc1a2654b..f2b1e1399b 100644 --- a/openhcs/processing/backends/cellprofiler/display_modules.py +++ b/openhcs/processing/backends/cellprofiler/display_modules.py @@ -562,10 +562,8 @@ def display_data_on_image( ) output = DisplayDataOnImageRenderer(request).render_slice() - return image.with_pixels( - output, - metadata=image.metadata.with_axis(ColourSampleAxisSpec(), -1), - ) + return image.with_pixels(output, + metadata=image.metadata.with_axis(ColourSampleAxisSpec(), -1),) class DensityPlotScaleType(Enum): diff --git a/openhcs/processing/backends/cellprofiler/edge.py b/openhcs/processing/backends/cellprofiler/edge.py index 6a2fb75a6b..ef984bddc5 100644 --- a/openhcs/processing/backends/cellprofiler/edge.py +++ b/openhcs/processing/backends/cellprofiler/edge.py @@ -116,7 +116,9 @@ class EdgeEnhancementStrategyKey: @property def label(self) -> str: - return f"{self.backend_provider.selection_name}:{self.method.value}:{self.direction.value}" + return ( + f"{self.backend_provider.selection_name}:{self.method.value}:{self.direction.value}" + ) @dataclass(frozen=True, slots=True) @@ -551,11 +553,9 @@ def enhance_edges( output = ( EdgeEnhancementStrategy.for_request(request).enhance(request).astype(np.float32) ) - return image.with_pixels( - output, + return image.with_pixels(output, mask=operation_mask if payload_mask is not None else None, - metadata=image.metadata.without_unit_interval_intensity_scale(), - ) + metadata=image.metadata.without_unit_interval_intensity_scale(),) @njit(cache=True) diff --git a/openhcs/processing/backends/cellprofiler/feature_enhancement.py b/openhcs/processing/backends/cellprofiler/feature_enhancement.py index 5938165125..f7285288ba 100644 --- a/openhcs/processing/backends/cellprofiler/feature_enhancement.py +++ b/openhcs/processing/backends/cellprofiler/feature_enhancement.py @@ -111,8 +111,10 @@ def enhance_or_suppress_features( dic_decay=dic_decay, ) ) - return image.metadata.without_unit_interval_intensity_scale().payload_with( - np.asarray(result, dtype=np.float32), image.mask + return ( + image.metadata + .without_unit_interval_intensity_scale() + .payload_with(np.asarray(result, dtype=np.float32), image.mask) ) @@ -310,8 +312,7 @@ def apply(self, request: FeatureEnhancementRequest) -> np.ndarray: enhance_or_suppress_features, backend_provider=OpencvBackendProvider, ).grayscale_opening( - masked, - footprint, + masked, footprint, ) result = masked - opened else: diff --git a/openhcs/processing/backends/cellprofiler/granularity.py b/openhcs/processing/backends/cellprofiler/granularity.py index 6d4c0b9a21..08fd8dbe60 100644 --- a/openhcs/processing/backends/cellprofiler/granularity.py +++ b/openhcs/processing/backends/cellprofiler/granularity.py @@ -712,6 +712,7 @@ class GranularityReconstructionBackendStrategy( ): """Exact grayscale reconstruction backend used by MeasureGranularity.""" + @abstractmethod def reconstruct_radius_one( self, diff --git a/openhcs/processing/backends/cellprofiler/illumination.py b/openhcs/processing/backends/cellprofiler/illumination.py index 2b2f16f562..14c6725019 100644 --- a/openhcs/processing/backends/cellprofiler/illumination.py +++ b/openhcs/processing/backends/cellprofiler/illumination.py @@ -15,7 +15,7 @@ from numba import njit from openhcs.constants.constants import MemoryType -from openhcs.core.aligned_image_payload import AlignedImageStack +from openhcs.core.aligned_image_payload import (AlignedImageStack) from openhcs.core.image_payload_execution_mode import ImagePayloadExecutionMode from openhcs.core.artifacts import ( ArtifactSpec, @@ -1086,8 +1086,12 @@ def correct_illumination_apply( if truncate_high: np.minimum(output_pixels, 1.0, out=output_pixels) mask = image.mask - return image.metadata.without_unit_interval_intensity_scale().payload_with( - output_pixels, None if mask is None else np.asarray(mask, dtype=bool) + return ( + image.metadata + .without_unit_interval_intensity_scale() + .payload_with( + output_pixels, None if mask is None else np.asarray(mask, dtype=bool) + ) ) @@ -1405,6 +1409,7 @@ class ConvexHullSmoothingBackendStrategy( ): """Convex-hull illumination smoothing keyed by OpenHCS memory/provider.""" + @abstractmethod def smooth_background_plane( self, @@ -1422,6 +1427,7 @@ class RankMedianSmoothingBackendStrategy( ): """Rank-median illumination smoothing keyed by OpenHCS memory/provider.""" + @staticmethod def disk_rows(footprint: np.ndarray) -> tuple[np.ndarray, np.ndarray]: """Collapse a dense disk footprint into per-row horizontal radii.""" diff --git a/openhcs/processing/backends/cellprofiler/image_geometry.py b/openhcs/processing/backends/cellprofiler/image_geometry.py index 3a8b68163f..8ea5a1a9d3 100644 --- a/openhcs/processing/backends/cellprofiler/image_geometry.py +++ b/openhcs/processing/backends/cellprofiler/image_geometry.py @@ -1120,9 +1120,9 @@ def mask_image( output_mask = restore_image_mask_planes( image.data, tuple((result[1] for result in masked_plane_results)) ) - return replace(image.metadata, mask_defines_border=True).payload_with( - masked_data, output_mask - ) + return replace( + image.metadata, mask_defines_border=True + ).payload_with(masked_data, output_mask) def masked_image_plane( @@ -1141,8 +1141,7 @@ def masked_image_plane( existing_mask = image.mask if existing_mask is not None: existing_projected_mask = bare_image.mask_for_data( - image_data, - mask=existing_mask, + image_data, mask=existing_mask, ) if existing_projected_mask is not None: binary_mask = binary_mask & np.asarray(existing_projected_mask, dtype=bool) @@ -1397,9 +1396,9 @@ def flip_and_rotate( max_j = max_area_idx[1] + 1 pixel_data = pixel_data[min_i:max_i, min_j:max_j] return ( - image.metadata.with_spatial_resize(pixel_data.shape[:2]).payload_with( - pixel_data.astype(np.float32), None - ), + image.metadata + .with_spatial_resize(pixel_data.shape[:2]) + .payload_with(pixel_data.astype(np.float32), None), DataclassMeasurementColumnarRows( (RotationResult(slice_index=0, rotation_angle=angle),), row_type=RotationResult, diff --git a/openhcs/processing/backends/cellprofiler/image_math.py b/openhcs/processing/backends/cellprofiler/image_math.py index 5036ad721c..2502a28cf9 100644 --- a/openhcs/processing/backends/cellprofiler/image_math.py +++ b/openhcs/processing/backends/cellprofiler/image_math.py @@ -813,7 +813,9 @@ def from_inputs( f"ImageMath {operation_strategy.operation.value!r} requires exactly " f"one declared image operand, got {len(source_payloads)}." ) - operand_pixels = tuple(np.asarray(payload.data) for payload in source_payloads) + operand_pixels = tuple( + np.asarray(payload.data) for payload in source_payloads + ) return cls( source_image=image, source_payloads=source_payloads, diff --git a/openhcs/processing/backends/cellprofiler/image_quality.py b/openhcs/processing/backends/cellprofiler/image_quality.py index d8a208d7b5..c7cca642a4 100644 --- a/openhcs/processing/backends/cellprofiler/image_quality.py +++ b/openhcs/processing/backends/cellprofiler/image_quality.py @@ -772,6 +772,7 @@ class ImageQualityBackendStrategy( ): """Image-quality primitives keyed by OpenHCS memory type/provider.""" + @abstractmethod def haralick_h3(self, image: np.ndarray, *, scale: int) -> float: """Return CP-style Haralick H3 correlation for one image plane.""" diff --git a/openhcs/processing/backends/cellprofiler/intensity.py b/openhcs/processing/backends/cellprofiler/intensity.py index 4162fbdf10..194642350a 100644 --- a/openhcs/processing/backends/cellprofiler/intensity.py +++ b/openhcs/processing/backends/cellprofiler/intensity.py @@ -276,11 +276,7 @@ def active_artifact_bindings( from openhcs.constants.constants import MemoryType from openhcs.core.callable_contract import runtime_image_execution_mode from openhcs.core.memory import numpy as numpy_decorator -from openhcs.core.pipeline.function_contracts import ( - object_label_input_execution_mode, - runtime_bound_parameters, - special_inputs, -) +from openhcs.core.pipeline.function_contracts import (object_label_input_execution_mode, runtime_bound_parameters, special_inputs) from openhcs.core.runtime_batch_contracts import ( RuntimePure2DSliceBatchRequest, SliceIndexRuntimeParameter, @@ -419,7 +415,9 @@ class RescaleIntensityModule(CellProfilerModule): confidence = 1.0 setting_bindings = ( SettingToKeywordBinding.input("Select the input image", ImageArtifactType), - ModuleOnlySettingBinding("Select image to match in maximum intensity"), + ModuleOnlySettingBinding( + "Select image to match in maximum intensity" + ), SettingToKeywordBinding.output("Name the output image", ImageArtifactType), SettingToKeywordBinding( "Rescaling method", @@ -907,7 +905,9 @@ def from_function_arguments( ) -> "ObjectIntensityMeasurementContext": return cls( labels=labels, - backend_provider=CellProfilerBackendSelection.from_input(backend_provider), + backend_provider=CellProfilerBackendSelection.from_input( + backend_provider + ), slice_index=int(slice_index), object_domain=object_domain, object_row_identity=object_row_identity, @@ -1026,6 +1026,7 @@ class ObjectIntensityBackendStrategy( ): """Object-intensity operations keyed by OpenHCS memory type/provider.""" + @abstractmethod def measure( self, image: np.ndarray, labels: ObjectIntensityLabelInput @@ -1455,7 +1456,9 @@ def measure_object_intensity_measurement_image_batch( first_request ).backend_provider ) - images = tuple((np.asarray(request.image.data) for _index, request in group)) + images = tuple( + (np.asarray(request.image.data) for _index, request in group) + ) batch_started_at = time.perf_counter() measurement_batches = backend.measure_prepared_batch(images, prepared_labels) RuntimeProfileLogger.log( diff --git a/openhcs/processing/backends/cellprofiler/intensity_distribution.py b/openhcs/processing/backends/cellprofiler/intensity_distribution.py index 2e5f741df7..423237e5b0 100644 --- a/openhcs/processing/backends/cellprofiler/intensity_distribution.py +++ b/openhcs/processing/backends/cellprofiler/intensity_distribution.py @@ -37,16 +37,10 @@ ObjectMeasurementColumnarRows, ) from openhcs.core.memory.decorators import numpy -from openhcs.core.pipeline.function_contracts import ( - object_label_input_execution_mode, - runtime_bound_parameters, - special_inputs, -) +from openhcs.core.pipeline.function_contracts import (object_label_input_execution_mode, runtime_bound_parameters, special_inputs) from python_introspect import public_names_from_objects from openhcs.core.runtime_identifier import normalize_runtime_identifier -from openhcs.core.measurement_dialect import ( - RuntimeMeasurementQualifierSuffixMatchStrategy, -) +from openhcs.core.measurement_dialect import RuntimeMeasurementQualifierSuffixMatchStrategy from metaclass_registry.strategies import enum_member_with_payload from openhcs.core.runtime_array_values import RuntimeArrayData from openhcs.core.runtime_batch_contracts import SliceIndexRuntimeParameter @@ -965,7 +959,9 @@ def _runtime_heatmap_outputs( @dataclass class RadialLabelGeometryCache( - ProcessLocalBoundedCache["RadialLabelGeometryCacheKey", "RadialLabelGeometry"] + ProcessLocalBoundedCache[ + "RadialLabelGeometryCacheKey", "RadialLabelGeometry" + ] ): """Process-local numerical geometry with shared bounded storage.""" @@ -2636,10 +2632,8 @@ def _intensity_distribution_heatmap( heatmap[rows, columns] = values colormap_name = group.colormap if colormap_name == "gray": - return image.with_pixels( - heatmap, - metadata=image.metadata.without_axis(ColourAxis), - ) + return image.with_pixels(heatmap, + metadata=image.metadata.without_axis(ColourAxis),) if colormap_name == "Default": colormap_name = "viridis" import matplotlib @@ -2649,10 +2643,8 @@ def _intensity_distribution_heatmap( :, :, :3 ] rgb[labels == 0] = 0 - return image.with_pixels( - rgb, - metadata=image.metadata.with_axis(ColourSampleAxisSpec(), -1), - ) + return image.with_pixels(rgb, + metadata=image.metadata.with_axis(ColourSampleAxisSpec(), -1),) def intensity_distribution_object_domain(labels: object) -> tuple[int, ...]: diff --git a/openhcs/processing/backends/cellprofiler/median_filter.py b/openhcs/processing/backends/cellprofiler/median_filter.py index f160506ef0..390768256b 100644 --- a/openhcs/processing/backends/cellprofiler/median_filter.py +++ b/openhcs/processing/backends/cellprofiler/median_filter.py @@ -129,6 +129,7 @@ class MedianFilterBackendStrategy( ): """Median filtering operations keyed by OpenHCS memory type/provider.""" + @abstractmethod def filter( self, @@ -291,10 +292,8 @@ def selection_network_filter( return None radius = int(window_size) // 2 padded = np.pad( - image, - ((radius, radius), (radius, radius), (radius, radius + 7)), - mode="constant", - constant_values=0, + image, ((radius, radius), (radius, radius), (radius, radius + 7)), + mode="constant", constant_values=0, ) depth, height, width = image.shape output_width = ((width + 7) // 8) * 8 diff --git a/openhcs/processing/backends/cellprofiler/morphology.py b/openhcs/processing/backends/cellprofiler/morphology.py index b346e30d1a..c83fc7c575 100644 --- a/openhcs/processing/backends/cellprofiler/morphology.py +++ b/openhcs/processing/backends/cellprofiler/morphology.py @@ -683,10 +683,7 @@ def artifact_output_relations( from openhcs.core.measurement_row_materialization import ( DataclassMeasurementColumnarRows, ) -from openhcs.core.pipeline.function_contracts import ( - object_label_input_execution_mode, - special_inputs, -) +from openhcs.core.pipeline.function_contracts import (object_label_input_execution_mode, special_inputs) from python_introspect import public_names_from_objects from metaclass_registry.strategies import EnumKeyedStrategyMixin from openhcs.processing.backends.cellprofiler.morphology_connected_components_numba import ( @@ -1520,10 +1517,8 @@ def _morph_image_payload( image = ImagePayload.of(image) pixel_data = image.data result = _morph_image_pixels(pixel_data, structuring_element, size, operation) - return image.with_pixels( - result.astype(pixel_data.dtype, copy=False), - metadata=image.metadata.without_unit_interval_intensity_scale(), - ) + return image.with_pixels(result.astype(pixel_data.dtype, copy=False), + metadata=image.metadata.without_unit_interval_intensity_scale(),) @runtime_image_execution_mode(FullStackExecution) @@ -1885,6 +1880,7 @@ class MorphologyBackendStrategy( ): """Nominal morphology operations keyed by OpenHCS memory type.""" + @classmethod def for_memory_type( cls, @@ -2117,9 +2113,7 @@ def grayscale_closing(self, image: np.ndarray, footprint: np.ndarray) -> np.ndar return self._grayscale_morphology(image, footprint, first_pass_is_dilation=True) def grayscale_opening(self, image: np.ndarray, footprint: np.ndarray) -> np.ndarray: - return self._grayscale_morphology( - image, footprint, first_pass_is_dilation=False - ) + return self._grayscale_morphology(image, footprint, first_pass_is_dilation=False) def _grayscale_morphology( self, @@ -2352,8 +2346,7 @@ def prepare_backend(self) -> None: for writable_mask in (False, True): smoothing_mask = ( np.ones(mask.shape, dtype=np.bool_) - if full_mask - else mask.copy() + if full_mask else mask.copy() ) smoothing_mask.flags.writeable = writable_mask self.smooth_image_for_declumping(image, smoothing_mask, 1.0) @@ -3077,7 +3070,10 @@ def _scipy_connected_components( @njit(cache=True, inline="always") def _labeled_hole_node_at_numba(source, background, offset, y, x): background_node = background[y, x] - return source[y, x] if background_node == 0 else background_node + offset + 1 + return ( + source[y, x] + if background_node == 0 else background_node + offset + 1 + ) @njit(cache=True) @@ -3093,15 +3089,11 @@ def _labeled_hole_graph_numba(source, background, offset, node_count): if node != 0 and (y == 0 or y + 1 == height or x == 0 or x + 1 == width): boundary_nodes.add(node) if y + 1 < height: - other = _labeled_hole_node_at_numba( - source, background, offset, y + 1, x - ) + other = _labeled_hole_node_at_numba(source, background, offset, y + 1, x) if node != other: edges.add(min(node, other) * node_count + max(node, other)) if x + 1 < width: - other = _labeled_hole_node_at_numba( - source, background, offset, y, x + 1 - ) + other = _labeled_hole_node_at_numba(source, background, offset, y, x + 1) if node != other: edges.add(min(node, other) * node_count + max(node, other)) encoded = np.empty(len(edges), dtype=np.int64) @@ -3245,16 +3237,11 @@ def _cellprofiler_fill_labeled_holes_2d( np.empty(captured_source.shape)[mask_array] output_mask = ( np.empty((0, 0), dtype=np.bool_) - if mask_array is None - else np.ascontiguousarray(mask_array) + if mask_array is None else np.ascontiguousarray(mask_array) ) output = _write_labeled_holes_numba( - captured_source, - background_labels, - foreground_label_count, - lookup, - output_mask, - mask_array is not None, + captured_source, background_labels, foreground_label_count, lookup, + output_mask, mask_array is not None, ) return output.astype(source.dtype, copy=False) @@ -3671,7 +3658,9 @@ def horizontal_spans(self) -> tuple[tuple[int, int, int], ...]: spans.append((y, x, x)) return tuple(spans) - def grayscale_extremum(self, image: np.ndarray, *, maximum: bool) -> np.ndarray: + def grayscale_extremum( + self, image: np.ndarray, *, maximum: bool + ) -> np.ndarray: """Reduce the exact union of spans, sharing each horizontal window.""" from scipy.ndimage import maximum_filter1d, minimum_filter1d @@ -6621,7 +6610,9 @@ def erode_objects( runtime_profiler.log( "erode_objects_lineage", time.perf_counter() - phase_started_at ) - runtime_profiler.log("erode_objects_total", time.perf_counter() - total_started_at) + runtime_profiler.log( + "erode_objects_total", time.perf_counter() - total_started_at + ) return ( image, DataclassMeasurementColumnarRows((stats,), row_type=ErosionStats), diff --git a/openhcs/processing/backends/cellprofiler/neighbors.py b/openhcs/processing/backends/cellprofiler/neighbors.py index 069c7b57fb..b73c9a65dd 100644 --- a/openhcs/processing/backends/cellprofiler/neighbors.py +++ b/openhcs/processing/backends/cellprofiler/neighbors.py @@ -322,21 +322,17 @@ def retained_images( output_metadata = bare_image.metadata.with_axis(ColourSampleAxisSpec(), -1) if self.retain_neighbor_count_image: retained.append( - bare_image.with_pixels( - self.colored_metric_image( + bare_image.with_pixels(self.colored_metric_image( neighbor_count_image, self.neighbor_count_colormap ), - metadata=output_metadata, - ) + metadata=output_metadata,) ) if self.retain_percent_touching_image: retained.append( - bare_image.with_pixels( - self.colored_metric_image( + bare_image.with_pixels(self.colored_metric_image( percent_touching_image, self.percent_touching_colormap ), - metadata=output_metadata, - ) + metadata=output_metadata,) ) return tuple(retained) @@ -448,6 +444,7 @@ class NeighborTopologyBackendStrategy( ): """Neighbor topology operations keyed by OpenHCS memory type/provider.""" + @abstractmethod def measure_topology( self, diff --git a/openhcs/processing/backends/cellprofiler/object_images.py b/openhcs/processing/backends/cellprofiler/object_images.py index 640abba3c4..251654920e 100644 --- a/openhcs/processing/backends/cellprofiler/object_images.py +++ b/openhcs/processing/backends/cellprofiler/object_images.py @@ -18,10 +18,7 @@ from openhcs.core.measurement_row_materialization import ( DataclassMeasurementColumnarRows, ) -from openhcs.core.pipeline.function_contracts import ( - object_label_input_execution_mode, - special_inputs, -) +from openhcs.core.pipeline.function_contracts import (object_label_input_execution_mode, special_inputs) from python_introspect import public_names_from_objects from openhcs.core.processing_preparation import PersistentNumbaKernelPreparation from metaclass_registry.strategies import EnumKeyedStrategyMixin @@ -385,10 +382,8 @@ def convert_objects_to_image( if image_mode is ImageMode.UINT16 else label_metadata ) - return labels.with_pixels( - rendered, - metadata=output_metadata, - ) + return labels.with_pixels(rendered, + metadata=output_metadata,) class ConvertImageToObjectsModule( diff --git a/openhcs/processing/backends/cellprofiler/object_overlap.py b/openhcs/processing/backends/cellprofiler/object_overlap.py index 667ad0bffd..3a22b935e1 100644 --- a/openhcs/processing/backends/cellprofiler/object_overlap.py +++ b/openhcs/processing/backends/cellprofiler/object_overlap.py @@ -15,10 +15,7 @@ DataclassMeasurementColumnarRows, MeasurementProjectedColumnarRows, ) -from openhcs.core.pipeline.function_contracts import ( - object_label_input_execution_mode, - special_inputs, -) +from openhcs.core.pipeline.function_contracts import (object_label_input_execution_mode, special_inputs) from python_introspect import public_names_from_objects from openhcs.core.runtime_tabular_values import ( FieldSpec, diff --git a/openhcs/processing/backends/cellprofiler/outlines.py b/openhcs/processing/backends/cellprofiler/outlines.py index f2694178ed..bb1429870b 100644 --- a/openhcs/processing/backends/cellprofiler/outlines.py +++ b/openhcs/processing/backends/cellprofiler/outlines.py @@ -37,10 +37,7 @@ from openhcs.core.runtime_array_values import RuntimeArrayData from openhcs.core.steps.function_runtime import RuntimeCallableKwargs from openhcs.core.memory.decorators import numpy -from openhcs.core.pipeline.function_contracts import ( - object_label_input_execution_mode, - special_inputs, -) +from openhcs.core.pipeline.function_contracts import (object_label_input_execution_mode, special_inputs) from python_introspect import public_names_from_objects from openhcs.core.runtime_object_labels import ( ObjectLabelValue, @@ -693,6 +690,7 @@ class ObjectOutlineBackendStrategy( ): """Object outline operations keyed by OpenHCS memory type/provider.""" + @abstractmethod def outline(self, labels: np.ndarray) -> np.ndarray: """Return a labeled inner outline image.""" @@ -906,15 +904,13 @@ def overlay_outlines( image, blank_image=context.blank_image, image_row_count=context.image_row_count ) output = context.render(image_sources) - return image.with_pixels( - output, + return image.with_pixels(output, metadata=replace( image.metadata, axes=PayloadAxes.colour_samples( -1 if context.display_mode is OutlineDisplayMode.COLOR else None ), - ), - ) + ),) @numpy(contract=FlexibleContract) @@ -959,10 +955,8 @@ def overlay_objects( seed=seed, colormap_value=colormap, ) - return image.with_pixels( - overlay, - metadata=image.metadata.with_axis(ColourSampleAxisSpec(), -1), - ) + return image.with_pixels(overlay, + metadata=image.metadata.with_axis(ColourSampleAxisSpec(), -1),) def _runtime_rows( @@ -1066,8 +1060,7 @@ def _draw_object_labels( else: outline_color = outline_intensity for label_plane in ObjectLabelStorageStrategy.for_value(labels).rendering_layers( - labels, - source_spatial_shape_yx=labels.source_spatial_shape_yx, + labels, source_spatial_shape_yx=labels.source_spatial_shape_yx, ): if label_plane.ndim != 2: raise ValueError( @@ -1110,7 +1103,9 @@ def _draw_outline_image( def _outline_image_mask(outline_image: RuntimeArrayData) -> np.ndarray: outline_image = ImagePayload.of(outline_image) mask = np.asarray(outline_image.data) > 0 - channel_axis = outline_image.metadata.axis_index(ColourAxis, outline_image) + channel_axis = outline_image.metadata.axis_index(ColourAxis, + outline_image + ) if channel_axis is not None: return np.any(mask, axis=channel_axis) return mask diff --git a/openhcs/processing/backends/cellprofiler/primary_object_diagnostics.py b/openhcs/processing/backends/cellprofiler/primary_object_diagnostics.py index 54babcdad1..8c59ecf5b3 100644 --- a/openhcs/processing/backends/cellprofiler/primary_object_diagnostics.py +++ b/openhcs/processing/backends/cellprofiler/primary_object_diagnostics.py @@ -19,9 +19,7 @@ ImageArtifactType, SourceStackLineageSourceRelation, ) -from openhcs.core.measurement_row_materialization import ( - DataclassMeasurementColumnarRows, -) +from openhcs.core.measurement_row_materialization import DataclassMeasurementColumnarRows from openhcs.core.projected_image_output import SelectedPlaneImageOutput from openhcs.core.runtime_image_values import ( ImagePayloadMetadata, @@ -38,7 +36,6 @@ MaterializedFilenameIdentity, ) - @dataclass(frozen=True, slots=True) class DiagnosticPlaneSource: """Decode source identity and validity once, sharing one mask across stages.""" @@ -51,11 +48,8 @@ def from_image(cls, image: object) -> "DiagnosticPlaneSource": source_mask = image.mask return cls( image.metadata, - ( - np.ones(np.asarray(image.data).shape, dtype=bool) - if source_mask is None - else np.asarray(source_mask, dtype=bool) - ), + np.ones(np.asarray(image.data).shape, dtype=bool) + if source_mask is None else np.asarray(source_mask, dtype=bool), ) def plane( @@ -144,9 +138,7 @@ def planes( shape = source.validity_mask.shape invalid = np.zeros(shape, dtype=bool) return ( - source.plane( - np.full(shape, np.nan, dtype=np.float32), validity_mask=invalid - ), + source.plane(np.full(shape, np.nan, dtype=np.float32), validity_mask=invalid), source.plane(np.zeros(shape, dtype=bool), validity_mask=invalid), source.plane(np.zeros(shape, dtype=np.int32), validity_mask=invalid), ) diff --git a/openhcs/processing/backends/cellprofiler/primary_objects.py b/openhcs/processing/backends/cellprofiler/primary_objects.py index a9b13ed721..b5db53409d 100644 --- a/openhcs/processing/backends/cellprofiler/primary_objects.py +++ b/openhcs/processing/backends/cellprofiler/primary_objects.py @@ -261,7 +261,9 @@ def identify_primary_objects( else np.asarray(input_mask_payload, dtype=bool) ) input_metadata = ( - image.metadata if respect_source_border_metadata else ImagePayloadMetadata() + image.metadata + if respect_source_border_metadata + else ImagePayloadMetadata() ) raw_image_data = np.asarray(image.data) proven_unit_interval_scale = unit_interval_scale_for_threshold_selection( @@ -664,16 +666,10 @@ def finalize_artifact_contract_outputs( artifact_inputs.specs, ) if len(source) != 1: - raise ValueError( - "IdentifyPrimaryObjects diagnostics require one source image." - ) - objects = ArtifactSpecCollection(outputs).of_artifact_type( - ObjectLabelsArtifactType - ) + raise ValueError("IdentifyPrimaryObjects diagnostics require one source image.") + objects = ArtifactSpecCollection(outputs).of_artifact_type(ObjectLabelsArtifactType) if len(objects) != 1: - raise ValueError( - "IdentifyPrimaryObjects diagnostics require one object output." - ) + raise ValueError("IdentifyPrimaryObjects diagnostics require one object output.") return ( *outputs, *PrimaryObjectDiagnosticPlanes.artifact_specs( diff --git a/openhcs/processing/backends/cellprofiler/relationships.py b/openhcs/processing/backends/cellprofiler/relationships.py index 99a379ab58..85df875ef8 100644 --- a/openhcs/processing/backends/cellprofiler/relationships.py +++ b/openhcs/processing/backends/cellprofiler/relationships.py @@ -881,6 +881,7 @@ class ObjectRelationshipBackendStrategy( ): """Object relationship operations keyed by OpenHCS memory type/provider.""" + @abstractmethod def relate_children_to_parents( self, parent_labels: np.ndarray, child_labels: np.ndarray, child_count: int @@ -1621,9 +1622,7 @@ def upstream_child_feature_indexes( identity_policy = ( self.request.adapter.request.context.source_image_set_identity_policy ) - child_axis = child_labels.metadata.source_provenance.image_set_axis( - identity_policy - ) + child_axis = child_labels.metadata.source_provenance.image_set_axis(identity_policy) if not child_axis: raise ValueError( f"RelateObjects child measurements for {child_spec.name!r} require " diff --git a/openhcs/processing/backends/cellprofiler/secondary.py b/openhcs/processing/backends/cellprofiler/secondary.py index a445b3f059..7db1c7723e 100644 --- a/openhcs/processing/backends/cellprofiler/secondary.py +++ b/openhcs/processing/backends/cellprofiler/secondary.py @@ -535,6 +535,7 @@ class SecondaryDistanceTransformBackendStrategy( ): """Distance transform operations used by secondary segmentation.""" + @abstractmethod def distance_to_foreground(self, labels: np.ndarray) -> np.ndarray: """Return Euclidean distance to the nearest positive label.""" @@ -615,6 +616,7 @@ class SecondaryPropagationBackendStrategy( ): """Regularized label propagation backend for secondary segmentation.""" + @abstractmethod def propagate_result( self, diff --git a/openhcs/processing/backends/cellprofiler/shape.py b/openhcs/processing/backends/cellprofiler/shape.py index c7b963bf06..419691aafb 100644 --- a/openhcs/processing/backends/cellprofiler/shape.py +++ b/openhcs/processing/backends/cellprofiler/shape.py @@ -457,10 +457,7 @@ def postprocess_bound_settings( from numba import njit from openhcs.constants.constants import MemoryType from openhcs.core.memory import numpy as numpy_decorator -from openhcs.core.pipeline.function_contracts import ( - object_label_input_execution_mode, - special_inputs, -) +from openhcs.core.pipeline.function_contracts import (object_label_input_execution_mode, special_inputs) from openhcs.core.runtime_object_label_domains import ( ObjectLabelDomain, dense_object_label_measurement_row_domain, @@ -528,20 +525,20 @@ def feature_array_domain(self, feature_name: str) -> ObjectFeatureArrayDomain: f"{type(self).__name__} feature {feature_name!r} has no declared " "feature-array domain." ) - if ShapeObjectZernikeDescriptorDeclaration.from_feature_name( - feature_name - ) is not None or any( - feature.value == feature_name - and LabelIndexedShapeFeature.matches_feature(feature) - for feature in MeasureObjectSizeShapeModule.MeasurementFeature + if ( + ShapeObjectZernikeDescriptorDeclaration.from_feature_name(feature_name) + is not None + or any( + feature.value == feature_name + and LabelIndexedShapeFeature.matches_feature(feature) + for feature in MeasureObjectSizeShapeModule.MeasurementFeature + ) ): return ObjectFeatureArrayDomain.MEASURED_OBJECT_ID return ObjectFeatureArrayDomain.ROW_ORDINAL def feature_missing_values_for_objects( - self, - feature_name: str, - object_ids: tuple[int, ...], + self, feature_name: str, object_ids: tuple[int, ...], ) -> Mapping[int, ObjectFeatureMissingValue]: """Native radius/Feret vectors contain zeros within their material extent.""" array_domain = self.feature_array_domain(feature_name) @@ -1264,6 +1261,7 @@ class ShapeMeasurementBackendStrategy( ): """Shape-measurement operations keyed by OpenHCS memory type/provider.""" + @abstractmethod def form_factor_values( self, labels: np.ndarray, label_ids: np.ndarray diff --git a/openhcs/processing/backends/cellprofiler/skeleton.py b/openhcs/processing/backends/cellprofiler/skeleton.py index 0a60c676c6..8f081e5e37 100644 --- a/openhcs/processing/backends/cellprofiler/skeleton.py +++ b/openhcs/processing/backends/cellprofiler/skeleton.py @@ -378,10 +378,8 @@ def measure_object_skeleton_with_branchpoint_image( fill_small_holes=fill_small_holes, maximum_hole_size=maximum_hole_size, ) - branchpoint_payload = image.with_pixels( - result.branchpoint_image, - metadata=image.metadata.with_axis(ColourSampleAxisSpec(), -1), - ) + branchpoint_payload = image.with_pixels(result.branchpoint_image, + metadata=image.metadata.with_axis(ColourSampleAxisSpec(), -1),) branchpoint_image = pack_aligned_image_outputs( (branchpoint_payload,), slice_contexts=( diff --git a/openhcs/processing/backends/cellprofiler/smoothing.py b/openhcs/processing/backends/cellprofiler/smoothing.py index 6e2530b82f..ab7d4811c9 100644 --- a/openhcs/processing/backends/cellprofiler/smoothing.py +++ b/openhcs/processing/backends/cellprofiler/smoothing.py @@ -396,9 +396,7 @@ def _normalize_full_support( if width <= kernel.size: np.divide(filtered, weights, out=filtered) else: - np.divide( - filtered[:, :radius], weights[:, :radius], out=filtered[:, :radius] - ) + np.divide(filtered[:, :radius], weights[:, :radius], out=filtered[:, :radius]) np.divide( filtered[:, radius : width - radius], weights[:, radius : radius + 1], @@ -873,8 +871,10 @@ def smooth_image( clip_polynomial=bool(clip_polynomial), ) output = SmoothingStrategy.for_request(request).smooth(request) - return image.metadata.without_unit_interval_intensity_scale().payload_with( - np.asarray(output, dtype=np.float32), mask + return ( + image.metadata + .without_unit_interval_intensity_scale() + .payload_with(np.asarray(output, dtype=np.float32), mask) ) @@ -931,7 +931,10 @@ def smooth_batch(request: RuntimePure2DSliceBatchRequest) -> list[Any]: smoothing_method = kwargs.get("smoothing_method", SmoothingMethod.GAUSSIAN_FILTER) pixel_stack = np.ascontiguousarray( np.stack( - [np.asarray(slice_2d.data, dtype=np.float32) for slice_2d in slices_2d], + [ + np.asarray(slice_2d.data, dtype=np.float32) + for slice_2d in slices_2d + ], axis=0, ) ) @@ -971,9 +974,9 @@ def smooth_batch(request: RuntimePure2DSliceBatchRequest) -> list[Any]: pixel_stack, mask_stack, float(selection_request.sigma) ).astype(np.float32, copy=False) return [ - slice_2d.metadata.without_unit_interval_intensity_scale().payload_with( - output_stack[slice_index], masks[slice_index] - ) + slice_2d.metadata + .without_unit_interval_intensity_scale() + .payload_with(output_stack[slice_index], masks[slice_index]) for slice_index, slice_2d in enumerate(slices_2d) ] diff --git a/openhcs/processing/backends/cellprofiler/spreadsheet_export.py b/openhcs/processing/backends/cellprofiler/spreadsheet_export.py index 53692766a7..2d6dbbf15b 100644 --- a/openhcs/processing/backends/cellprofiler/spreadsheet_export.py +++ b/openhcs/processing/backends/cellprofiler/spreadsheet_export.py @@ -109,10 +109,7 @@ WriteMode, ) -from openhcs.processing.materialization.core import ( - ColumnarCsvOutput, - RenderedColumnarCsvOutput, -) +from openhcs.processing.materialization.core import ColumnarCsvOutput, RenderedColumnarCsvOutput if TYPE_CHECKING: from openhcs.core.function_patterns import FunctionInvocationKey @@ -229,9 +226,7 @@ class SpreadsheetFileSelection: file_name: str relationship_subject_name: ClassVar[str] = "Object relationships" - experiment_subject_name: ClassVar[str] = ( - CELLPROFILER_MEASUREMENT_DIALECT.scope_name(MeasurementScope.RUN) - ) + experiment_subject_name: ClassVar[str] = CELLPROFILER_MEASUREMENT_DIALECT.scope_name(MeasurementScope.RUN) @classmethod def for_subject( @@ -290,9 +285,7 @@ def reference_mean_subjects( references = tuple( name for name in (() if image_rows is None else image_rows.columns) - if CellProfilerMeasurementDialect.row_identity_contract.is_aggregate_sample_number_reference( - name - ) + if CellProfilerMeasurementDialect.row_identity_contract.is_aggregate_sample_number_reference(name) ) return frozenset( subject @@ -414,10 +407,7 @@ def combined_rows( ) def header_rows( - self, - rows: ColumnarRows, - *, - active_subjects: tuple[str, ...], + self, rows: ColumnarRows, *, active_subjects: tuple[str, ...], ) -> tuple[tuple[str, ...], ...]: """Derive native headers in their existing first-present field order.""" @@ -502,9 +492,7 @@ def render_csv( ) -> str: """Realized view of this selection's canonical typed CSV output.""" output = self.prepare_csv( - rows, - active_subjects=active_subjects, - delimiter=delimiter, + rows, active_subjects=active_subjects, delimiter=delimiter, nan_representation=nan_representation, ) return output.options.render(output.content) @@ -520,8 +508,7 @@ class CellProfilerSpreadsheetCsvOptions(CsvOptions): nan_representation: SpreadsheetNanRepresentation def csv_schema( - self, - rows: ColumnarRows, + self, rows: ColumnarRows, ) -> tuple[tuple[str, ...], tuple[tuple[str, ...], ...]]: columns, headers = self.selection.csv_schema( rows, @@ -744,8 +731,7 @@ def prepare_spreadsheet_bundle( def render_spreadsheet_bundle( - artifact_batch: RuntimeArtifactBatch, - **kwargs: object, + artifact_batch: RuntimeArtifactBatch, **kwargs: object, ) -> dict[str, str | bytes]: """Realized view of the canonical schema-bearing spreadsheet bundle.""" return { @@ -1001,8 +987,7 @@ def _source_metadata_measurement_rows( ] = [] dialect = CellProfilerDatabaseColumnDialect() image_subject = MeasurementSubject( - MeasurementScope.SAMPLE, - CELLPROFILER_MEASUREMENT_DIALECT.scope_name(MeasurementScope.SAMPLE), + MeasurementScope.SAMPLE, CELLPROFILER_MEASUREMENT_DIALECT.scope_name(MeasurementScope.SAMPLE) ) for slice_index in image_numbers_by_slice: provenance = table.source_provenance.for_source_plane(slice_index) @@ -1018,11 +1003,8 @@ def _source_metadata_measurement_rows( acquisition.update( dialect.source_metadata_values( None, - ( - Path(provenance.source_path) - if provenance.source_path is not None - else None - ), + Path(provenance.source_path) + if provenance.source_path is not None else None, ) ) file_values: dict[str, str] = {} @@ -1046,12 +1028,7 @@ def _source_metadata_measurement_rows( ) if metadata or acquisition or file_values: rows.append( - ( - image_numbers_by_slice[slice_index], - metadata, - acquisition, - file_values, - ) + (image_numbers_by_slice[slice_index], metadata, acquisition, file_values) ) return tuple(rows) @@ -1198,9 +1175,7 @@ def _with_requested_aggregates( reference_means = frozenset( name for name in (() if source_image is None else source_image.columns) - if CellProfilerMeasurementDialect.row_identity_contract.is_aggregate_sample_number_reference( - name - ) + if CellProfilerMeasurementDialect.row_identity_contract.is_aggregate_sample_number_reference(name) ) if not (mean or median or standard_deviation or reference_means): return tables @@ -1583,10 +1558,8 @@ def _prepare_spreadsheet_partition( ) for path, output in bundle.items() } - return { - path: output.realized_for_composition(partition_fields=request.partition_fields) - for path, output in bundle.items() - } + return {path: output.realized_for_composition(partition_fields=request.partition_fields) + for path, output in bundle.items()} def _partitioned_spreadsheet_export( diff --git a/openhcs/processing/backends/cellprofiler/texture.py b/openhcs/processing/backends/cellprofiler/texture.py index aafa7df2d5..a7e95831a0 100644 --- a/openhcs/processing/backends/cellprofiler/texture.py +++ b/openhcs/processing/backends/cellprofiler/texture.py @@ -230,6 +230,7 @@ class ObjectTextureCropBackendStrategy( ): """Extract masked object intensity crops for texture measurement.""" + @abstractmethod def object_intensity_crops( self, image: np.ndarray, labels: np.ndarray @@ -242,6 +243,7 @@ class HaralickTextureBackendStrategy( ): """Compute CP-compatible 2-D Haralick feature matrices.""" + @abstractmethod def haralick_features( self, pixel_data: np.ndarray, *, scale: int, ignore_zeros: bool diff --git a/openhcs/processing/backends/cellprofiler/thresholding.py b/openhcs/processing/backends/cellprofiler/thresholding.py index f5e41cfad2..2fc2d83fff 100644 --- a/openhcs/processing/backends/cellprofiler/thresholding.py +++ b/openhcs/processing/backends/cellprofiler/thresholding.py @@ -555,10 +555,7 @@ class GlobalThresholdSourceSelection: def normalize_cellprofiler_image(image: RuntimeArrayData) -> np.ndarray: """Return an image in CellProfiler's normalized pixel-data convention.""" - return normalize_cellprofiler_image_payload( - image, - dtype=np.float32, - ).data + return normalize_cellprofiler_image_payload( image, dtype=np.float32, ).data def unit_interval_scale_for_threshold_selection( @@ -758,6 +755,7 @@ class ThresholdSmoothingBackendStrategy( ): """Memory-backend-specific threshold smoothing.""" + @abstractmethod def smooth_threshold_image( self, @@ -866,6 +864,7 @@ class ThresholdDiagnosticsBackendStrategy( ): """Memory-backend-specific threshold diagnostic measurements.""" + def diagnostics( self, image: np.ndarray, @@ -1273,6 +1272,7 @@ class ThresholdPrimitiveBackendStrategy( ): """Small threshold helper primitives supplied by an explicit provider.""" + @abstractmethod def log_transform(self, values: np.ndarray) -> tuple[np.ndarray, object]: """Return CP-compatible log-transformed values and conversion state.""" @@ -2637,9 +2637,9 @@ def threshold( proven_unit_interval_scale=proven_unit_interval_scale, ).calculate() output_image = ( - source_payload.metadata.without_unit_interval_intensity_scale().payload_with( - threshold_result.mask.astype(np.float32), mask - ) + source_payload.metadata + .without_unit_interval_intensity_scale() + .payload_with(threshold_result.mask.astype(np.float32), mask) ) return ( output_image, diff --git a/openhcs/processing/backends/cellprofiler/tracking.py b/openhcs/processing/backends/cellprofiler/tracking.py index 20a41c9ddc..14cc20c861 100644 --- a/openhcs/processing/backends/cellprofiler/tracking.py +++ b/openhcs/processing/backends/cellprofiler/tracking.py @@ -38,11 +38,7 @@ measurement_row_has_object_identity, ) from openhcs.core.memory.decorators import numpy -from openhcs.core.pipeline.function_contracts import ( - object_label_input_execution_mode, - required_axis_roles, - special_inputs, -) +from openhcs.core.pipeline.function_contracts import (object_label_input_execution_mode, required_axis_roles, special_inputs) from python_introspect import public_names_from_objects from metaclass_registry.strategies import EnumKeyedStrategyMixin from openhcs.core.runtime_image_values import RuntimeArrayData @@ -300,9 +296,7 @@ def rows(self) -> MeasurementSparseColumnarRows: return row_policy.annotate_record_rows( rows, object_name=self.object_name, - source_image_name=CELLPROFILER_SCOPE_NAMES[ - MeasurementScope.SAMPLE - ].casefold(), + source_image_name=CELLPROFILER_SCOPE_NAMES[MeasurementScope.SAMPLE].casefold(), ) @classmethod @@ -655,10 +649,8 @@ class CellProfilerRendererAgg(RendererAgg): def _text_bitmap(self, text, properties): pixels, facts = rasterize( - font_manager.findfont(properties), - properties.get_size_in_points(), - self.dpi, - text, + font_manager.findfont(properties), properties.get_size_in_points(), + self.dpi, text, ) return np.frombuffer(pixels, dtype=np.uint8).reshape(facts[1], facts[0]), facts @@ -704,15 +696,11 @@ def render_frame( """Draw one label plane with CellProfiler's stable ID palette.""" if labels.ndim != 2: raise ValueError("Tracked-image rendering requires a 2-D label plane.") - if len(object_numbers) != len(centers[0]) or len(object_numbers) != len( - centers[1] - ): - raise ValueError( - "Tracked-image IDs and centers must share one label domain." - ) + if len(object_numbers) != len(centers[0]) or len(object_numbers) != len(centers[1]): + raise ValueError("Tracked-image IDs and centers must share one label domain.") indexer = np.zeros(len(object_numbers) + 1, dtype=int) indexer[1:] = object_numbers - powers = 2 ** np.mgrid[0:8, 0 : len(indexer)][0] + powers = 2 ** np.mgrid[0:8, 0:len(indexer)][0] bits = (indexer & powers).astype(bool) indexer = np.sum(bits.transpose() * (2 ** np.arange(7, -1, -1)), axis=1) figure = Figure() @@ -926,9 +914,7 @@ def from_frames( ) ) normalized = normalize_cellprofiler_image_payload(rendered) - metadata = normalized.metadata.with_axis( - ColourSampleAxisSpec(), -1 - ).with_source_context_from(labels.metadata) + metadata = normalized.metadata.with_axis(ColourSampleAxisSpec(), -1).with_source_context_from(labels.metadata) output_image = metadata.payload_with(normalized.data, None) return cls( output_image=output_image, @@ -988,6 +974,7 @@ class ObjectTrackingBackendStrategy( ): """TrackObjects primitives keyed by OpenHCS memory type/provider.""" + @abstractmethod def label_centers(self, labels: np.ndarray) -> tuple[np.ndarray, np.ndarray]: """Return y/x centers for dense positive labels.""" @@ -1042,9 +1029,7 @@ def label_centers(self, labels: np.ndarray) -> tuple[np.ndarray, np.ndarray]: label_count = int(labels_array.max()) if labels_array.size else 0 if label_count == 0: return (np.array([], dtype=np.float64), np.array([], dtype=np.float64)) - centers = dense_label_centers_2d_numba( - np.ascontiguousarray(labels_array), label_count - ) + centers = dense_label_centers_2d_numba(np.ascontiguousarray(labels_array), label_count) return (centers[1:, 0], centers[1:, 1]) def track_by_overlap( @@ -1697,9 +1682,7 @@ def _track_by_overlap_numba( def _track_by_distance_numba( frame: TrackingKernelFrame, pixel_radius: int ) -> tuple[np.ndarray, np.ndarray, np.ndarray, int]: - current_centers = dense_label_centers_2d_numba( - frame.current_labels, frame.current_count - ) + current_centers = dense_label_centers_2d_numba(frame.current_labels, frame.current_count) old_centers = dense_label_centers_2d_numba(frame.old_labels, frame.old_count) new_labels = np.zeros(frame.current_count, dtype=np.int64) parent_object_numbers = np.zeros(frame.current_count, dtype=np.int64) diff --git a/openhcs/processing/backends/cellprofiler/watershed.py b/openhcs/processing/backends/cellprofiler/watershed.py index 6502fe6ae3..ff017169bf 100644 --- a/openhcs/processing/backends/cellprofiler/watershed.py +++ b/openhcs/processing/backends/cellprofiler/watershed.py @@ -12,9 +12,7 @@ ObjectLabelsArtifactType, SourceStackLineageSourceRelation, ) -from openhcs.processing.backends.cellprofiler.morphology import ( - MorphologyBackendStrategy, -) +from openhcs.processing.backends.cellprofiler.morphology import MorphologyBackendStrategy from metaclass_registry.strategies import ( EnumKeyedStrategyMixin, ) @@ -1627,9 +1625,7 @@ class NumpyLegacyWatershedBackendStrategy(LegacyWatershedBackendStrategy): prefer_fast = False -class CentrosomeNumpyLegacyWatershedBackendStrategy( - NumpyLegacyWatershedBackendStrategy -): +class CentrosomeNumpyLegacyWatershedBackendStrategy(NumpyLegacyWatershedBackendStrategy): """Centrosome provider using the absorbed CellProfiler reference semantics.""" memory_type = MemoryType.NUMPY @@ -1659,6 +1655,7 @@ class CellProfiler4DistanceMarkerBackendStrategy( ): """Build CellProfiler 4 distance-watershed markers through a typed backend.""" + @abstractmethod def distance_markers( self, diff --git a/openhcs/processing/backends/cellprofiler/worms.py b/openhcs/processing/backends/cellprofiler/worms.py index 8aae3e57ee..6961321277 100644 --- a/openhcs/processing/backends/cellprofiler/worms.py +++ b/openhcs/processing/backends/cellprofiler/worms.py @@ -104,12 +104,7 @@ FlexibleContract, Pure2DContract, ) -from openhcs.core.pipeline.function_contracts import ( - object_label_input_execution_mode, - required_axis_roles, - runtime_bound_parameters, - special_inputs, -) +from openhcs.core.pipeline.function_contracts import (object_label_input_execution_mode, required_axis_roles, runtime_bound_parameters, special_inputs) from metaclass_registry.strategies import EnumKeyedStrategyMixin from openhcs.interop.cellprofiler.settings_binder import coerce_cellprofiler_enum from openhcs.interop.cellprofiler.module_settings import ( @@ -3005,10 +3000,8 @@ def _overlapping_worm_outline( local_mask[label_y - y_start, label_x - x_start] = True local_outline = find_boundaries(local_mask, mode="inner") output[y_start:y_stop, x_start:x_stop][local_outline] = colors[label_id] - return source_image.with_pixels( - output, - metadata=source_image.metadata.with_axis(ColourSampleAxisSpec(), -1), - ) + return source_image.with_pixels(output, + metadata=source_image.metadata.with_axis(ColourSampleAxisSpec(), -1),) def _nonoverlapping_worm_outline( @@ -3023,10 +3016,8 @@ def _nonoverlapping_worm_outline( object_label_dense_array(labels, dtype=np.int32), mode="inner", ) - return source_image.with_pixels( - outline, - metadata=source_image.metadata.without_axis(ColourAxis), - ) + return source_image.with_pixels(outline, + metadata=source_image.metadata.without_axis(ColourAxis),) def _reconstructed_worm_pixels( diff --git a/openhcs/processing/backends/cellprofiler/zernike.py b/openhcs/processing/backends/cellprofiler/zernike.py index efd349cc07..a81eb11e36 100644 --- a/openhcs/processing/backends/cellprofiler/zernike.py +++ b/openhcs/processing/backends/cellprofiler/zernike.py @@ -785,6 +785,7 @@ class ShapeZernikeBackendStrategy( ): """Shape Zernike moment backends keyed by OpenHCS memory type.""" + @abstractmethod def shape_zernike_moments( self, diff --git a/openhcs/processing/backends/enhance/basic_processor_jax.py b/openhcs/processing/backends/enhance/basic_processor_jax.py index 0640f7ec21..148d5fd86e 100644 --- a/openhcs/processing/backends/enhance/basic_processor_jax.py +++ b/openhcs/processing/backends/enhance/basic_processor_jax.py @@ -61,6 +61,7 @@ def _fitted_field_output(name: str) -> ArtifactSpec: @numpy_func(contract=Pure3DContract, dtype_config_default=DtypeConfig()) @allowed_group_by_roles(ColourAxis) @required_axis_roles(FittedIlluminationFieldOutput.observation_role) + @artifact_outputs(CORRECTED_OUTPUT, FLATFIELD_OUTPUT, DARKFIELD_OUTPUT) def basic_flatfield_correction_jax( image: np.ndarray, diff --git a/openhcs/processing/backends/experimental_analysis/analysis.py b/openhcs/processing/backends/experimental_analysis/analysis.py index ac4ed4b98a..ee871bb570 100755 --- a/openhcs/processing/backends/experimental_analysis/analysis.py +++ b/openhcs/processing/backends/experimental_analysis/analysis.py @@ -26,9 +26,7 @@ ExperimentalAnalysisScope, ExperimentalLayoutRowRole, ) -from openhcs.processing.backends.experimental_analysis.result_formats import ( - ExperimentalResultFormatStrategy, -) +from openhcs.processing.backends.experimental_analysis.result_formats import ExperimentalResultFormatStrategy @dataclass(frozen=True, slots=True) diff --git a/openhcs/processing/backends/experimental_analysis/result_formats.py b/openhcs/processing/backends/experimental_analysis/result_formats.py index 884868e75d..d3039f5574 100644 --- a/openhcs/processing/backends/experimental_analysis/result_formats.py +++ b/openhcs/processing/backends/experimental_analysis/result_formats.py @@ -11,9 +11,7 @@ import pandas as pd from metaclass_registry.strategies import EnumKeyedStrategyMixin -from openhcs.processing.backends.experimental_analysis.layout_rows import ( - ExperimentalAnalysisScope, -) +from openhcs.processing.backends.experimental_analysis.layout_rows import ExperimentalAnalysisScope PlateDictionary = dict[str, dict[str, dict[str, Any]]] diff --git a/openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py b/openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py index bfc9ac163c..b1b2487cf7 100644 --- a/openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py +++ b/openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py @@ -4,9 +4,7 @@ from typing import Dict, List, Any, Optional, Tuple from openhcs.domains.microscopy.config import ExperimentalAnalysisConfig -from openhcs.processing.backends.experimental_analysis.layout_rows import ( - ExperimentalAnalysisScope, -) +from openhcs.processing.backends.experimental_analysis.layout_rows import ExperimentalAnalysisScope from openhcs.processing.backends.experimental_analysis.result_formats import ( ExperimentalResultFormatStrategy, ) @@ -19,9 +17,7 @@ class DataProcessingError(RuntimeError): class ExperimentalAnalysisEngine: """Coordinate analysis through the result scope's nominal strategy.""" - def __init__( - self, config: ExperimentalAnalysisConfig = ExperimentalAnalysisConfig() - ): + def __init__(self, config: ExperimentalAnalysisConfig = ExperimentalAnalysisConfig()): """Initialize the engine with a configuration (default: the declared defaults).""" self.config = config @@ -222,9 +218,7 @@ def _read_plate_layout( Returns: Tuple of (scope, plate_layout, conditions, ctrl_positions, excluded_positions, per_well_datapoints) """ - from openhcs.processing.backends.experimental_analysis.analysis import ( - read_plate_layout, - ) + from openhcs.processing.backends.experimental_analysis.analysis import read_plate_layout return read_plate_layout( config_path, @@ -241,9 +235,7 @@ def _load_plate_groups(self, config_path: str) -> Dict: Returns: Plate groups dictionary """ - from openhcs.processing.backends.experimental_analysis.analysis import ( - load_plate_groups, - ) + from openhcs.processing.backends.experimental_analysis.analysis import load_plate_groups return load_plate_groups( config_path, @@ -254,9 +246,7 @@ def _make_experiment_dict_locations( self, plate_groups: Dict, plate_layout: Dict, conditions: List ) -> Dict: """Create experiment location mapping.""" - from openhcs.processing.backends.experimental_analysis.analysis import ( - make_experiment_dict_locations, - ) + from openhcs.processing.backends.experimental_analysis.analysis import make_experiment_dict_locations return make_experiment_dict_locations(plate_groups, plate_layout, conditions) @@ -269,9 +259,7 @@ def _make_experiment_dict_values( per_well_datapoints: bool = False, ) -> Dict: """Map experimental design to measured values.""" - from openhcs.processing.backends.experimental_analysis.analysis import ( - make_experiment_dict_values, - ) + from openhcs.processing.backends.experimental_analysis.analysis import make_experiment_dict_values return make_experiment_dict_values( plates_dict, @@ -311,9 +299,7 @@ def _normalize_experiment( plate_groups: Dict, ) -> Dict: """Apply normalization using control wells.""" - from openhcs.processing.backends.experimental_analysis.analysis import ( - normalize_experiment, - ) + from openhcs.processing.backends.experimental_analysis.analysis import normalize_experiment return normalize_experiment( experiment_dict_values, @@ -331,9 +317,7 @@ def _create_all_feature_tables( per_well_datapoints: bool = False, ) -> Dict: """Create feature tables for export.""" - from openhcs.processing.backends.experimental_analysis.analysis import ( - create_all_feature_tables, - ) + from openhcs.processing.backends.experimental_analysis.analysis import create_all_feature_tables return create_all_feature_tables( experiment_dict_values, features, per_well_datapoints @@ -341,9 +325,7 @@ def _create_all_feature_tables( def _export_results(self, feature_tables: Dict, output_path: str): """Export results to Excel file.""" - from openhcs.processing.backends.experimental_analysis.analysis import ( - feature_tables_to_excel, - ) + from openhcs.processing.backends.experimental_analysis.analysis import feature_tables_to_excel feature_tables_to_excel(feature_tables, output_path) diff --git a/openhcs/processing/backends/lib_registry/openhcs_registry.py b/openhcs/processing/backends/lib_registry/openhcs_registry.py index 9edb85602f..86803ca427 100644 --- a/openhcs/processing/backends/lib_registry/openhcs_registry.py +++ b/openhcs/processing/backends/lib_registry/openhcs_registry.py @@ -605,7 +605,9 @@ def get_modules_to_scan(self) -> List[Tuple[str, Any]]: module_name, ) continue - excluded = _module_distributions_excluded_by_package_metadata(module_name) + excluded = _module_distributions_excluded_by_package_metadata( + module_name + ) if excluded: logger.info( "Skipping OpenHCS module %s - package requirements exclude %s " @@ -942,11 +944,9 @@ def reconstruct_cached_callable( """Reconstruct cached metadata through the declaration's one wrapper.""" return _registered_wrapper( - self, - func, - lambda: super(OpenHCSRegistry, self).reconstruct_cached_callable( + self, func, lambda: super(OpenHCSRegistry, self).reconstruct_cached_callable( func, contract - ), + ) ) def _processing_contract_for_function( diff --git a/openhcs/processing/backends/lib_registry/registry_service.py b/openhcs/processing/backends/lib_registry/registry_service.py index dbe13006f0..b706eff7f5 100644 --- a/openhcs/processing/backends/lib_registry/registry_service.py +++ b/openhcs/processing/backends/lib_registry/registry_service.py @@ -109,13 +109,9 @@ def metadata_for_canonical_key(cls, function_id: str) -> FunctionMetadata: """Ask the original registry declaration to resolve one public identity.""" registry_name, separator, name = function_id.partition(":") - registry_type = ( - LIBRARY_REGISTRIES.get(registry_name) if separator and name else None - ) - claims = ( - () - if registry_type is None - else tuple(registry_type._canonical_metadata_claims(function_id)) + registry_type = LIBRARY_REGISTRIES.get(registry_name) if separator and name else None + claims = () if registry_type is None else tuple( + registry_type._canonical_metadata_claims(function_id) ) for metadata in claims: if metadata.composite_key != function_id: @@ -132,9 +128,7 @@ def metadata_for_canonical_key(cls, function_id: str) -> FunctionMetadata: or inspect.unwrap(metadata.func) is not inspect.unwrap(first.func) for metadata in claims[1:] ): - raise ValueError( - f"Canonical function {function_id!r} has ambiguous owners." - ) + raise ValueError(f"Canonical function {function_id!r} has ambiguous owners.") return first @classmethod @@ -193,15 +187,11 @@ def prepare_in_current_process( cls._metadata_cache = cls._metadata_from_instances( cls._available_registry_instances() ) - callables = tuple( - dict.fromkeys( - target - for metadata in cls._metadata_cache.values() - for target in CallableProjection.from_callable( - metadata.func - ).prepare_targets() - ) - ) + callables = tuple(dict.fromkeys( + target + for metadata in cls._metadata_cache.values() + for target in CallableProjection.from_callable(metadata.func).prepare_targets() + )) emit_status("Preparing registered kernel caches") PreparationCacheBatch.from_callables(callables).populate_child_caches( max_workers=os.cpu_count() or 1, diff --git a/openhcs/processing/backends/lib_registry/unified_registry.py b/openhcs/processing/backends/lib_registry/unified_registry.py index c678271fb1..0e5efdf371 100644 --- a/openhcs/processing/backends/lib_registry/unified_registry.py +++ b/openhcs/processing/backends/lib_registry/unified_registry.py @@ -311,10 +311,7 @@ def metadata_for_declared_callable( @classmethod def _canonical_metadata_claims( - cls, - function_id: str, - *, - prepare_catalog: bool = True, + cls, function_id: str, *, prepare_catalog: bool = True, ) -> Iterator[FunctionMetadata]: """Supply catalog-owned claims; independent capabilities compose via super.""" @@ -322,8 +319,7 @@ def _canonical_metadata_claims( catalog = ( RegistryService.get_all_functions_with_metadata() - if prepare_catalog - else RegistryService.cached_metadata_snapshot() + if prepare_catalog else RegistryService.cached_metadata_snapshot() ) metadata = catalog.get(function_id) if metadata is not None: @@ -554,9 +550,7 @@ def wrapper(image, *args, **kwargs): } return contract.execute( - LibraryContractCall(func, args), - image, - filtered_kwargs, + LibraryContractCall(func, args), image, filtered_kwargs, ) wrapper.__signature__ = public_sig.replace(parameters=new_params) diff --git a/openhcs/processing/backends/processors/cupy_processor.py b/openhcs/processing/backends/processors/cupy_processor.py index 7003761895..7126155a57 100644 --- a/openhcs/processing/backends/processors/cupy_processor.py +++ b/openhcs/processing/backends/processors/cupy_processor.py @@ -229,9 +229,7 @@ def create_weight_mask( Returns: 2D CuPy weight mask of shape (Y, X) """ - return create_linear_weight_mask( - *operations.weight_mask_extent(shape), margin_ratio - ) + return create_linear_weight_mask(*operations.weight_mask_extent(shape), margin_ratio) @cupy_func diff --git a/openhcs/processing/backends/processors/jax_processor.py b/openhcs/processing/backends/processors/jax_processor.py index 76045a8f91..a5cbf5f481 100644 --- a/openhcs/processing/backends/processors/jax_processor.py +++ b/openhcs/processing/backends/processors/jax_processor.py @@ -347,9 +347,7 @@ def create_weight_mask( Returns: 2D JAX weight mask of shape (Y, X) """ - return create_linear_weight_mask( - *operations.weight_mask_extent(shape), margin_ratio - ) + return create_linear_weight_mask(*operations.weight_mask_extent(shape), margin_ratio) @jax_func diff --git a/openhcs/processing/backends/processors/numpy_processor.py b/openhcs/processing/backends/processors/numpy_processor.py index a62d8da5e8..5b41c859de 100644 --- a/openhcs/processing/backends/processors/numpy_processor.py +++ b/openhcs/processing/backends/processors/numpy_processor.py @@ -329,16 +329,14 @@ def _indexed_weighted_projection( preserve_numpy_sum_order and not values.flags.c_contiguous ) if ( - not supported - or pairwise + not supported or pairwise or dtype not in (np.dtype(np.float32), np.dtype(np.float64)) ): if preserve_numpy_sum_order: converted = members.astype(dtype) factors = weights.reshape((len(weights),) + (1,) * (members.ndim - 1)) selected_members = ( - converted - if np.array_equal(selected, np.arange(count)) + converted if np.array_equal(selected, np.arange(count)) else converted[selected] ) return np.sum(selected_members * factors, axis=0) @@ -369,16 +367,12 @@ def execute(self) -> None: for shape in ((3, 2, 2), (3, 2, 2, 2)): values = np.ones(shape, dtype=source_dtype) strided = np.moveaxis( - np.moveaxis(values, 0, -1).copy(), - -1, - 0, + np.moveaxis(values, 0, -1).copy(), -1, 0, ) for members in (values, np.asfortranarray(values), strided): for writable in (True, False): members.flags.writeable = writable - _indexed_weighted_projection( - members, indices, weights, axis=0 - ) + _indexed_weighted_projection(members, indices, weights, axis=0) @numpy_func(contract=VolumetricToSliceContract) @@ -405,11 +399,8 @@ def create_composite( weights_array = np.array(normalized_weights, dtype=np.float32) composite_slice = _indexed_weighted_projection( - stack, - np.arange(n_slices, dtype=np.int64), - weights_array, - axis=0, - preserve_numpy_sum_order=True, + stack, np.arange(n_slices, dtype=np.int64), weights_array, + axis=0, preserve_numpy_sum_order=True, ) # Convert back to original dtype @@ -448,9 +439,7 @@ def create_weight_mask(shape: Tuple[int, int], margin_ratio: float = 0.1) -> np. Returns: 2D NumPy weight mask of shape (Y, X) """ - return create_linear_weight_mask( - *operations.weight_mask_extent(shape), margin_ratio - ) + return create_linear_weight_mask(*operations.weight_mask_extent(shape), margin_ratio) @numpy_func diff --git a/openhcs/processing/backends/processors/pyclesperanto_processor.py b/openhcs/processing/backends/processors/pyclesperanto_processor.py index 3e3bf132d1..deab9b2b81 100644 --- a/openhcs/processing/backends/processors/pyclesperanto_processor.py +++ b/openhcs/processing/backends/processors/pyclesperanto_processor.py @@ -642,6 +642,4 @@ def create_weight_mask(shape: tuple, margin_ratio: float = 0.1) -> "cle.Array": Returns: 2D pyclesperanto Array of shape (Y, X) """ - return create_linear_weight_mask( - *operations.weight_mask_extent(shape), margin_ratio - ) + return create_linear_weight_mask(*operations.weight_mask_extent(shape), margin_ratio) diff --git a/openhcs/processing/backends/processors/tensorflow_processor.py b/openhcs/processing/backends/processors/tensorflow_processor.py index 1ab6342612..f0b88191b3 100644 --- a/openhcs/processing/backends/processors/tensorflow_processor.py +++ b/openhcs/processing/backends/processors/tensorflow_processor.py @@ -367,9 +367,7 @@ def create_weight_mask( Returns: 2D TensorFlow weight mask of shape (Y, X) """ - return create_linear_weight_mask( - *operations.weight_mask_extent(shape), margin_ratio - ) + return create_linear_weight_mask(*operations.weight_mask_extent(shape), margin_ratio) @tensorflow_func diff --git a/openhcs/processing/backends/processors/torch_processor.py b/openhcs/processing/backends/processors/torch_processor.py index be0c90321e..82b9b63273 100644 --- a/openhcs/processing/backends/processors/torch_processor.py +++ b/openhcs/processing/backends/processors/torch_processor.py @@ -406,9 +406,7 @@ def create_weight_mask( Returns: 2D PyTorch weight mask of shape (Y, X) """ - return create_linear_weight_mask( - *operations.weight_mask_extent(shape), margin_ratio - ) + return create_linear_weight_mask(*operations.weight_mask_extent(shape), margin_ratio) @torch_func diff --git a/openhcs/processing/custom_functions/manager.py b/openhcs/processing/custom_functions/manager.py index c607ca8c36..eec75f2639 100644 --- a/openhcs/processing/custom_functions/manager.py +++ b/openhcs/processing/custom_functions/manager.py @@ -113,9 +113,7 @@ def default_storage_directory(cls) -> Path: def source_path_for_name(storage_dir: Path, function_name: str) -> Path: """Own source naming for both admission and actual persistence.""" if not function_name.isidentifier() or function_name.startswith("_"): - raise ValidationError( - "Custom function name must be a public Python identifier." - ) + raise ValidationError("Custom function name must be a public Python identifier.") return storage_dir / f"{function_name}.py" def __init__(self, *, create_storage: bool = True): @@ -156,18 +154,11 @@ def register_from_code( """ if persist and write_admission is not None: if expected_function_name is None: - raise ValueError( - "Write admission requires an explicit custom function name." - ) + raise ValueError("Write admission requires an explicit custom function name.") write_admission(self.storage_dir) - write_admission( - self.source_path_for_name(self.storage_dir, expected_function_name) - ) + write_admission(self.source_path_for_name(self.storage_dir, expected_function_name)) metadata = self._prepare_source(code) - if ( - expected_function_name is not None - and metadata.original_name != expected_function_name - ): + if expected_function_name is not None and metadata.original_name != expected_function_name: raise ValidationError( f"Prepared declaration {metadata.original_name!r} does not match " f"admitted function {expected_function_name!r}." @@ -216,9 +207,7 @@ def source_revision(self) -> CustomFunctionSourceRevision: sources = tuple( CustomFunctionSource( function_name=source_path.stem, - content_sha256=CustomFunctionSource.content_digest( - source_path.read_bytes() - ), + content_sha256=CustomFunctionSource.content_digest(source_path.read_bytes()), ) for source_path in sorted(self.storage_dir.glob("*.py")) ) @@ -500,9 +489,7 @@ def _prepare_source( ) -> "FunctionMetadata": """Validate and project one source without mutating runtime or disk state.""" - from openhcs.processing.backends.lib_registry.openhcs_registry import ( - OpenHCSRegistry, - ) + from openhcs.processing.backends.lib_registry.openhcs_registry import OpenHCSRegistry validation_result = validate_code(code) if not validation_result.is_valid: diff --git a/openhcs/processing/custom_functions/validation.py b/openhcs/processing/custom_functions/validation.py index 679722c21b..25368d9cbe 100644 --- a/openhcs/processing/custom_functions/validation.py +++ b/openhcs/processing/custom_functions/validation.py @@ -188,25 +188,18 @@ def validate_code(code: str) -> ValidationResult: def validate_function(func: Callable) -> ValidationResult: """Validate the selected declaration using its original scope contract.""" - from openhcs.processing.backends.lib_registry.openhcs_registry import ( - OpenHCSRegistry, - ) + from openhcs.processing.backends.lib_registry.openhcs_registry import OpenHCSRegistry errors: list[str] = [] try: contract = OpenHCSRegistry.declared_callable_contract(func) if contract is None: - raise ValueError( - f"Function {func.__name__!r} is not an admitted OpenHCS declaration." - ) + raise ValueError(f"Function {func.__name__!r} is not an admitted OpenHCS declaration.") if contract.execution_scope is FunctionStepExecutionScope.PLATE: - from openhcs.core.pipeline.funcstep_contract_validator import ( - FuncStepContractValidator, - ) + from openhcs.core.pipeline.funcstep_contract_validator import FuncStepContractValidator FuncStepContractValidator.validate_dataset_callable_contracts( - (contract,), - func.__name__, + (contract,), func.__name__, ) else: params = tuple(inspect.signature(func).parameters) @@ -217,8 +210,5 @@ def validate_function(func: Callable) -> ValidationResult: except (TypeError, ValueError) as exc: errors.append(str(exc)) return ValidationResult( - is_valid=not errors, - errors=errors, - warnings=[], - function_names=[func.__name__], + is_valid=not errors, errors=errors, warnings=[], function_names=[func.__name__], ) diff --git a/openhcs/processing/materialization/core.py b/openhcs/processing/materialization/core.py index bb2a47c33a..80a45f9d4b 100644 --- a/openhcs/processing/materialization/core.py +++ b/openhcs/processing/materialization/core.py @@ -15,13 +15,7 @@ import string from abc import ABC, abstractmethod from collections.abc import Callable, Mapping, Sequence, Sized -from dataclasses import ( - dataclass, - field, - fields as dataclass_fields, - is_dataclass, - replace, -) +from dataclasses import dataclass, field, fields as dataclass_fields, is_dataclass, replace from functools import lru_cache, singledispatch from pathlib import Path, PurePosixPath, PureWindowsPath from types import MappingProxyType @@ -347,29 +341,21 @@ def viewer_source_identity(self) -> SourceImageIdentity | None: return self.source_identity def stream_source_metadata_items( - self, - fallback_source_identity: SourceImageIdentity | None, + self, fallback_source_identity: SourceImageIdentity | None, ) -> StreamSourceComponentMetadataItems: """Observe sources from the actual rendered output's declared planes.""" if self.metadata is not None: return StreamSourceComponentMetadataItems.from_image_metadata( - self.metadata, - fallback_source_identity=fallback_source_identity, + self.metadata, fallback_source_identity=fallback_source_identity, ) return StreamSourceComponentMetadataItems.from_values( - ( - ( - fallback_source_identity.component_metadata - if fallback_source_identity is not None - else None - ), - ) + (fallback_source_identity.component_metadata + if fallback_source_identity is not None else None,) ) def viewer_item_fields(self) -> dict[str, ViewerWireValue]: return StreamImagePayloadMetadataProjector.item_fields_for_plane_components( - self.metadata, - self.variable_components, + self.metadata, self.variable_components, ) @property @@ -436,9 +422,7 @@ def source_component_domain(self) -> tuple[SourceComponentMetadata, ...] | None: if self.metadata is None: return None - return ROIArchiveSourceMetadata.source_component_domain( - self.content, self.metadata - ) + return ROIArchiveSourceMetadata.source_component_domain(self.content, self.metadata) @property def viewer_source_identity(self) -> SourceImageIdentity | None: @@ -452,8 +436,7 @@ def viewer_source_identity(self) -> SourceImageIdentity | None: ) def stream_source_metadata_items( - self, - fallback_source_identity: SourceImageIdentity | None, + self, fallback_source_identity: SourceImageIdentity | None, ) -> StreamSourceComponentMetadataItems: domain = self.source_component_domain if domain is None: @@ -464,9 +447,7 @@ def viewer_item_fields(self) -> dict[str, ViewerWireValue]: from openhcs.core.roi_source_metadata import ROIArchiveSourceMetadata return ROIArchiveSourceMetadata.stream_item_fields( - self.content, - self.metadata, - super().viewer_item_fields(), + self.content, self.metadata, super().viewer_item_fields(), ) @@ -485,7 +466,6 @@ def source_component_domain(self) -> tuple[SourceComponentMetadata, ...]: raise ValueError("Point ROI output requires fractional-Z metadata.") return domain - @dataclass(frozen=True) class TextOutput(Output): """Materialization output whose canonical payload is text.""" @@ -505,9 +485,7 @@ class Utf8TextOutput(Output): """Text output stored as UTF-8 bytes for file-bundle compatibility.""" content: bytes - table_shape_info: tuple[tuple[str, ...], int] | None = field( - default=None, compare=False - ) + table_shape_info: tuple[tuple[str, ...], int] | None = field(default=None, compare=False) def table_shape(self) -> tuple[tuple[str, ...], int] | None: return self.table_shape_info @@ -533,7 +511,6 @@ def viewer_stream_requires_source_metadata(self) -> bool: def require_text_content(self) -> str: return self.content.decode("utf-8") - @dataclass(frozen=True) class ColumnarCsvOutput(Output): """CSV whose correlated row domain survives until file materialization. @@ -1987,7 +1964,9 @@ def from_runtime_slice_projected_value( runtime_plane_metadata=item.runtime_plane_metadata, ) for source_payload in source_payloads - for item in (OptionalSourceIdentity).project_payload_items( + for item in ( + OptionalSourceIdentity + ).project_payload_items( RuntimeProjectionSourceIdentityRequest( value=source_payload, source_description="materialization payload", @@ -2031,7 +2010,9 @@ def from_image_value( source_description=item.source_description, runtime_plane_metadata=item.runtime_plane_metadata, ) - for item in (OptionalSourceIdentity).project_payload_items( + for item in ( + OptionalSourceIdentity + ).project_payload_items( RuntimeProjectionSourceIdentityRequest( value=image.value, source_description=image.source_description, @@ -2283,14 +2264,10 @@ def to_filemanager_kwargs( if output_fields is None: return self.values.to_kwargs() component_metadata, item_fields, display_config = output_fields - return ( - self._stream_kwargs(display_config, item_fields) - .with_single_item_source( - component_metadata, - item_fields, - ) - .to_kwargs() - ) + return self._stream_kwargs(display_config, item_fields).with_single_item_source( + component_metadata, + item_fields, + ).to_kwargs() def _stream_kwargs( self, @@ -2299,9 +2276,7 @@ def _stream_kwargs( ) -> ViewerStreamBackendKwargs: """Apply one source-owned display scope to the original typed request.""" request = self.values.with_item_fields(item_fields).stream_request - return ViewerStreamBackendKwargs( - replace(request, display_config=display_config) - ) + return ViewerStreamBackendKwargs(replace(request, display_config=display_config)) def filemanager_batches( self, @@ -2323,11 +2298,8 @@ def filemanager_batches( continue component_metadata, item_fields, display_config = output_fields group = next( - ( - candidate - for candidate in grouped - if candidate[0] == item_fields and candidate[1] == display_config - ), + (candidate for candidate in grouped + if candidate[0] == item_fields and candidate[1] == display_config), None, ) if group is None: @@ -2351,9 +2323,7 @@ def filemanager_batches( raise ValueError( "Viewer materialization batch requires unique output paths." ) - stream_request = self._stream_kwargs( - display_config, item_fields - ).stream_request + stream_request = self._stream_kwargs(display_config, item_fields).stream_request stream_request = replace( stream_request, source=replace( @@ -2398,13 +2368,9 @@ def projected_outputs(self, outputs: Sequence[Output]) -> tuple[Output, ...]: fallback_identity_path=output.path, ) if identity is None: - raise ValueError( - "Projected viewer output requires a filename identity." - ) + raise ValueError("Projected viewer output requires a filename identity.") identity = replace(identity, extension=output_path.suffix) - filename = identity.with_filename_qualifier(output_path.stem).filename( - parser - ) + filename = identity.with_filename_qualifier(output_path.stem).filename(parser) result.append( replace( output, @@ -2418,14 +2384,11 @@ def projected_outputs(self, outputs: Sequence[Output]) -> tuple[Output, ...]: def _output_fields( self, output: Output, - ) -> ( - tuple[ - dict[str, MaterializationValue], - dict[str, ViewerWireValue], - ViewerDisplayConfigABC, - ] - | None - ): + ) -> tuple[ + dict[str, MaterializationValue], + dict[str, ViewerWireValue], + ViewerDisplayConfigABC, + ] | None: source_identity = output.viewer_source_identity if source_identity is not None: source_identity = source_identity.with_parsed_path_components( @@ -2555,9 +2518,7 @@ class MaterializationContext: materialization_spec: MaterializationSpec | None = None def named_source_filename( - self, - metadata: ImagePayloadMetadata, - extension: str, + self, metadata: ImagePayloadMetadata, extension: str, ) -> str | None: """Name a retained image from the actual rendering purpose and role.""" if self.materialization_spec is None: @@ -2565,17 +2526,12 @@ def named_source_filename( qualifier = self.materialization_spec.filename_qualifier(self.output_plan) if qualifier is None: return None - parser = SourceStemAuthoritySelection.from_processing_context( - self.context - ).required_parser() + parser = SourceStemAuthoritySelection.from_processing_context(self.context).required_parser() identity = FunctionOutputIdentity.from_filename_metadata(parser, metadata) if identity is None: - raise ValueError( - "Retained image output has no addressable source filename identity." - ) + raise ValueError("Retained image output has no addressable source filename identity.") identity = self.materialization_spec.filename_identity_for_output( - replace(identity, extension=extension), - self.output_plan, + replace(identity, extension=extension), self.output_plan, ) return identity.filename(parser) @@ -3360,22 +3316,16 @@ def write_image_file( for sequence_index, item in enumerate(projected_items, start=1): if options.relative_path_template is not None: path = _image_relative_output_path( - item.value, - options, - context, - sequence_index=sequence_index, + item.value, options, context, sequence_index=sequence_index, ) elif preserves_planned_path: path = paths.primary_output_path(options) else: - filename = context.named_source_filename( - item.metadata, options.primary_output_suffix - ) + filename = context.named_source_filename(item.metadata, options.primary_output_suffix) if filename is None: filename = ( source_stem_authority.required_source_stem( - item.metadata, - context.artifact_filename_identity, + item.metadata, context.artifact_filename_identity, ) + options.primary_output_suffix ) @@ -4706,6 +4656,7 @@ def metadata_for_slices( component for component in request.variable_components if component.name in selected_variable_components + ), request.artifact_source_identity, ) diff --git a/openhcs/processing/presets/pipelines/czi_brain_axon_cellbody.py b/openhcs/processing/presets/pipelines/czi_brain_axon_cellbody.py index 6a72cd9cbe..c8710ae585 100644 --- a/openhcs/processing/presets/pipelines/czi_brain_axon_cellbody.py +++ b/openhcs/processing/presets/pipelines/czi_brain_axon_cellbody.py @@ -287,7 +287,8 @@ def _declared_crop_candidates( for candidate in dataset.candidates if candidate.component_labels.get(Microscopy.Site.name) == selection.site_label - and candidate.component_labels.get(Microscopy.Channel.name) == channel_label + and candidate.component_labels.get(Microscopy.Channel.name) + == channel_label and candidate.declared_address is not None and candidate.declared_address.value_for(Microscopy.ZIndex) == "1" and candidate.declared_address.value_for(Microscopy.Timepoint) == "1" @@ -475,7 +476,9 @@ def artifact_presentation( output_spec: ArtifactSpec, ) -> StreamProducerIdentity: return StreamProducerIdentity.pipeline_output( - output_kind=(CompiledStepPlan.ARTIFACT_OUTPUT_KIND), + output_kind=( + CompiledStepPlan.ARTIFACT_OUTPUT_KIND + ), output_key=output_spec.name, projection_key=output_spec.name, step_name=step.name, diff --git a/openhcs/processing/presets/pipelines/loose_operaphenix_neurite_outgrowth.py b/openhcs/processing/presets/pipelines/loose_operaphenix_neurite_outgrowth.py index 341a3437f7..f019801cae 100644 --- a/openhcs/processing/presets/pipelines/loose_operaphenix_neurite_outgrowth.py +++ b/openhcs/processing/presets/pipelines/loose_operaphenix_neurite_outgrowth.py @@ -441,7 +441,9 @@ def prepare() -> None: pipeline_config=pipeline_config, pipeline_steps=tuple(pipeline_steps), presentation_identity=StreamProducerIdentity.pipeline_output( - output_kind=(CompiledStepPlan.ARTIFACT_OUTPUT_KIND), + output_kind=( + CompiledStepPlan.ARTIFACT_OUTPUT_KIND + ), output_key=secondary_output_name, projection_key=secondary_output_name, step_name=analysis_step.name, diff --git a/openhcs/processing/presets/pipelines/neuroncyto_ii_crossover_neurite_outgrowth.py b/openhcs/processing/presets/pipelines/neuroncyto_ii_crossover_neurite_outgrowth.py index 1af800129e..05e91f68f0 100644 --- a/openhcs/processing/presets/pipelines/neuroncyto_ii_crossover_neurite_outgrowth.py +++ b/openhcs/processing/presets/pipelines/neuroncyto_ii_crossover_neurite_outgrowth.py @@ -127,9 +127,7 @@ def build_neuroncyto_ii_crossover_demo( ) pipeline_config = PipelineConfig( dataset_source=SourceBindingsSource, - partition_filter_config=LazyPartitionFilterConfig( - partition_filter=inputs.image_id - ), + partition_filter_config=LazyPartitionFilterConfig(partition_filter=inputs.image_id), path_planning_config=LazyPathPlanningConfig( partition_filter=0, global_output_folder=output_root, @@ -297,7 +295,9 @@ def prepare() -> None: pipeline_config=pipeline_config, pipeline_steps=tuple(pipeline_steps), presentation_identity=StreamProducerIdentity.pipeline_output( - output_kind=(CompiledStepPlan.ARTIFACT_OUTPUT_KIND), + output_kind=( + CompiledStepPlan.ARTIFACT_OUTPUT_KIND + ), output_key=NEURITE_MORPHOLOGY_OUTPUT.name, projection_key=NEURITE_MORPHOLOGY_OUTPUT.name, step_name=analysis_step.name, diff --git a/openhcs/pyqt_gui/main.py b/openhcs/pyqt_gui/main.py index 4c56133572..24b602a83a 100644 --- a/openhcs/pyqt_gui/main.py +++ b/openhcs/pyqt_gui/main.py @@ -1311,9 +1311,7 @@ def _on_synthetic_plate_generated(self, output_dir: str, pipeline_path: str): self.show_pipeline_editor() for scope_id in self.session.add_dataset_roots((output_dir,)): self.session.load_pipeline_file(scope_id, pipeline_path) - logger.info( - "Added synthetic dataset and loaded its test pipeline: %s", output_dir - ) + logger.info("Added synthetic dataset and loaded its test pipeline: %s", output_dir) def _on_consolidate_results(self): """Open file dialog to select results directory and consolidate analysis results.""" diff --git a/openhcs/pyqt_gui/services/reactor_providers.py b/openhcs/pyqt_gui/services/reactor_providers.py index 5743d0cbfb..adc685e206 100644 --- a/openhcs/pyqt_gui/services/reactor_providers.py +++ b/openhcs/pyqt_gui/services/reactor_providers.py @@ -231,9 +231,7 @@ def _get_current_orchestrator(self) -> PipelineOrchestrator | None: from openhcs.constants.constants import OrchestratorState from openhcs.core.orchestrator import PipelineOrchestrator - orchestrator = ObjectStateRegistry.get_object( - plate_manager.session.current_scope_id - ) + orchestrator = ObjectStateRegistry.get_object(plate_manager.session.current_scope_id) if not isinstance(orchestrator, PipelineOrchestrator): return None if orchestrator.state is OrchestratorState.CREATED: @@ -281,6 +279,7 @@ def select_components( group_by=group_by, group_label=self.grouping_label(group_by), metadata_lookup=self.get_component_display_name, + parent=parent, ) diff --git a/openhcs/pyqt_gui/services/ui_agent_bridge.py b/openhcs/pyqt_gui/services/ui_agent_bridge.py index f017860c34..913d91547c 100644 --- a/openhcs/pyqt_gui/services/ui_agent_bridge.py +++ b/openhcs/pyqt_gui/services/ui_agent_bridge.py @@ -184,7 +184,6 @@ "for the current template." ) - @dataclass(frozen=True, slots=True) class CodeDocumentExecutionResult(PlateManagerOrchestratorCodePayload): """Parsed and normalized payload from one declarative UI code document.""" @@ -788,7 +787,9 @@ def complete_result( ) -> UiBridgeOperationRef: """Project one typed completion through its registered declaration owner.""" operation = self.get(operation_id) - contract = UiBridgeOperation.for_name(operation.identity.operation_name) + contract = UiBridgeOperation.for_name( + operation.identity.operation_name + ) if not isinstance(result, contract.result_type): raise TypeError( f"{contract.require_name()} requires {contract.result_type.__qualname__}." @@ -810,7 +811,9 @@ def fail_exception( self, operation_id: str, error: Exception ) -> UiBridgeOperationRef: operation = self.get(operation_id) - contract = UiBridgeOperation.for_name(operation.identity.operation_name) + contract = UiBridgeOperation.for_name( + operation.identity.operation_name + ) return self.complete( operation_id, status=UiBridgeOperationStatus.FAILED, diff --git a/openhcs/pyqt_gui/services/ui_bridge_composition.py b/openhcs/pyqt_gui/services/ui_bridge_composition.py index 9cf6763066..3ebbac7430 100644 --- a/openhcs/pyqt_gui/services/ui_bridge_composition.py +++ b/openhcs/pyqt_gui/services/ui_bridge_composition.py @@ -17,6 +17,7 @@ UiBridgeSurfaceRegistry, ) + if TYPE_CHECKING: from openhcs.authoring.session.session import Session diff --git a/openhcs/pyqt_gui/services/ui_bridge_object_state.py b/openhcs/pyqt_gui/services/ui_bridge_object_state.py index c7d117dbcb..74221355d2 100644 --- a/openhcs/pyqt_gui/services/ui_bridge_object_state.py +++ b/openhcs/pyqt_gui/services/ui_bridge_object_state.py @@ -688,7 +688,9 @@ def mutate( self, request: UiObjectStateFieldMutationRequest, ) -> UiObjectStateFieldMutationResult: - state = object_state_for_agent_scope_id(request.object_state_scope_id) + state = object_state_for_agent_scope_id( + request.object_state_scope_id + ) if state is None: return self._error_result( request, diff --git a/openhcs/pyqt_gui/services/ui_bridge_plate_manager.py b/openhcs/pyqt_gui/services/ui_bridge_plate_manager.py index 61bea973f3..ba59cd15a6 100644 --- a/openhcs/pyqt_gui/services/ui_bridge_plate_manager.py +++ b/openhcs/pyqt_gui/services/ui_bridge_plate_manager.py @@ -116,7 +116,6 @@ PlateManagerOrchestratorCodeDocumentIdentity.require_value() ) - class PlateManagerBridgeProviderSet(UiBridgeProviderSetABC): """Register all PlateManager surfaces with a UI bridge registry.""" diff --git a/openhcs/pyqt_gui/services/ui_bridge_session_actions.py b/openhcs/pyqt_gui/services/ui_bridge_session_actions.py index 5a90d8d5ee..d73f4e997b 100644 --- a/openhcs/pyqt_gui/services/ui_bridge_session_actions.py +++ b/openhcs/pyqt_gui/services/ui_bridge_session_actions.py @@ -68,9 +68,7 @@ def summary(self, action_id: str) -> UiActionSummary: ) return UiActionSummary( schema_version=SCHEMA_VERSION, - identity=UiActionIdentity( - widget_id=self.identity.widget_id, action_id=action_id - ), + identity=UiActionIdentity(widget_id=self.identity.widget_id, action_id=action_id), title=operation.button_label(self._session), enabled=error is None, disabled_error=error, @@ -94,11 +92,7 @@ def dispatch(self, request: UiActionInvokeRequest) -> UiActionDispatch: ) if session_request is None: return UiActionDispatch( - errors=( - AgentError( - code="ui_action_cancelled", message="The user cancelled." - ), - ) + errors=(AgentError(code="ui_action_cancelled", message="The user cancelled."),) ) result = self._session.invoke(operation, session_request) return UiActionDispatch( diff --git a/openhcs/pyqt_gui/services/ui_bridge_windows.py b/openhcs/pyqt_gui/services/ui_bridge_windows.py index 1afbc7eeb6..5b4c0b55cd 100644 --- a/openhcs/pyqt_gui/services/ui_bridge_windows.py +++ b/openhcs/pyqt_gui/services/ui_bridge_windows.py @@ -2885,6 +2885,7 @@ def _window_supports_action( return action.is_supported(window.managed_window_action_capabilities()) + @dataclass(frozen=True, slots=True) class MainWindowBridgeProviderSet(UiBridgeProviderSetABC): """Provider set for main-window generic UI projections.""" diff --git a/openhcs/pyqt_gui/session_rendering.py b/openhcs/pyqt_gui/session_rendering.py index 3ce59d0fcc..76b7be501c 100644 --- a/openhcs/pyqt_gui/session_rendering.py +++ b/openhcs/pyqt_gui/session_rendering.py @@ -68,7 +68,9 @@ def selection_scope_ids(self) -> tuple[str, ...]: def request_for(self, operation: type[SessionOperation]) -> object | None: if issubclass(operation, PromptedOperation): return self.session.renderer.prompt(operation) - return operation.request_for_selection(self.session, self.selection_scope_ids()) + return operation.request_for_selection( + self.session, self.selection_scope_ids() + ) def handle_button_action(self, operation_id: str) -> None: commit_focused_widget_edits() diff --git a/openhcs/pyqt_gui/widgets/image_browser.py b/openhcs/pyqt_gui/widgets/image_browser.py index 469ccb5b67..c705ee166f 100644 --- a/openhcs/pyqt_gui/widgets/image_browser.py +++ b/openhcs/pyqt_gui/widgets/image_browser.py @@ -58,6 +58,7 @@ logger = logging.getLogger(__name__) + def _streaming_config_field_names() -> tuple[str, ...]: """Registered streaming-config field names used by image-browser controls.""" return StreamingConfig.supported_config_keys() @@ -1304,9 +1305,7 @@ def _stream_images_to_viewer(self, filenames: list, config_key: str): read_backend=read_backend, ) ) - logger.info( - f"Streaming {len(filenames)} images to {config.viewer_family.display_name}..." - ) + logger.info(f"Streaming {len(filenames)} images to {config.viewer_family.display_name}...") def _show_streaming_error(self, viewer_name: str, error_msg: str): """Show streaming error in UI thread.""" @@ -1513,9 +1512,7 @@ def _update_plate_view(self): grid_axis = require_grid_axis() parser = self.orchestrator.microscope_handler.parser coord_to_well = { - grid_axis.grid_position( - *parser.extract_component_coordinates(well_id) - ): well_id + grid_axis.grid_position(*parser.extract_component_coordinates(well_id)): well_id for well_id in well_ids } plate_dimensions = ( diff --git a/openhcs/pyqt_gui/widgets/pipeline_editor.py b/openhcs/pyqt_gui/widgets/pipeline_editor.py index 4d04674f89..9d482ebc6b 100644 --- a/openhcs/pyqt_gui/widgets/pipeline_editor.py +++ b/openhcs/pyqt_gui/widgets/pipeline_editor.py @@ -468,9 +468,7 @@ def code_document_source(self, clean: bool = True) -> str: pipeline_config=( authored_pipeline_config(scope_id) if scope_id else PipelineConfig() ), - pipeline_steps=( - self.session.pipeline_steps(scope_id) if scope_id else [] - ), + pipeline_steps=self.session.pipeline_steps(scope_id) if scope_id else [], ), clean_mode=clean, ) @@ -525,9 +523,7 @@ def selected_step_scope_ids(self) -> tuple[str, ...]: return () return tuple(self.step_scope_id(step) for step in self.get_selected_items()) - def get_item_insert_index( - self, item: FunctionStep, scope_key: str - ) -> Optional[int]: + def get_item_insert_index(self, item: FunctionStep, scope_key: str) -> Optional[int]: del item token = scope_key.rsplit("::", 1)[-1] parts = token.rsplit("_", 1) @@ -544,9 +540,7 @@ def _get_item_scope_id(self, item: FunctionStep, index: int) -> str: return self.step_scope_id(item) @override - def _format_item_content( - self, item: FunctionStep, index: int, context: None - ) -> str: + def _format_item_content(self, item: FunctionStep, index: int, context: None) -> str: display_text, _ = self.format_item_for_display(item, context, step_index=index) return display_text @@ -585,9 +579,7 @@ def _get_list_placeholder(self) -> tuple[str, None] | None: @override def prepare_list_update(self) -> None: if self.current_plate: - ScopeTokenService.seed_from_objects( - self.current_plate, self.displayed_steps - ) + ScopeTokenService.seed_from_objects(self.current_plate, self.displayed_steps) return None @override @@ -619,9 +611,7 @@ def on_time_travel_complete(self, dirty_states, triggering_scope): == PipelineScopeIdentity.from_plate_scope(self.current_plate).scope_id for scope_id, _state in (dirty_states or ()) ): - GuiEventBusBroadcaster(self.event_bus).pipeline_changed( - self.displayed_steps - ) + GuiEventBusBroadcaster(self.event_bus).pipeline_changed(self.displayed_steps) # -- context for step editors --------------------------------------------- @@ -683,9 +673,7 @@ def open_step_editor( function_invocation_badge_provider=( None if step_index is None - else self.function_presentation.badge_provider( - step, step_index=step_index - ) + else self.function_presentation.badge_provider(step, step_index=step_index) ), compiled_artifact_inspection_provider=self.session.compiled_inspection, before_mutation=( @@ -736,9 +724,7 @@ def _action_paste_steps(self): self.status_message.emit("No dataset selected") return selected_rows = [index.row() for index in self.item_list.selectedIndexes()] - insert_after = ( - max(selected_rows) if selected_rows else len(self.displayed_steps) - 1 - ) + insert_after = max(selected_rows) if selected_rows else len(self.displayed_steps) - 1 pasted = [copy.deepcopy(step) for step in self._clipboard_steps] self.session.insert_steps(self.current_plate, insert_after + 1, pasted) self.status_message.emit( diff --git a/openhcs/pyqt_gui/widgets/shared/services/pipeline_editor_workflows.py b/openhcs/pyqt_gui/widgets/shared/services/pipeline_editor_workflows.py index 932bd9053b..e519650835 100644 --- a/openhcs/pyqt_gui/widgets/shared/services/pipeline_editor_workflows.py +++ b/openhcs/pyqt_gui/widgets/shared/services/pipeline_editor_workflows.py @@ -221,15 +221,11 @@ def run_command( ) -> None: scope_id = self.editor.current_plate if not scope_id: - self.editor.status_message.emit( - "Select a dataset before running debug mode." - ) + self.editor.status_message.emit("Select a dataset before running debug mode.") return session = self.editor.session command_label = command_type.value.replace("_", " ") - self.editor.status_message.emit( - f"Submitting debug {command_label} for {scope_id}" - ) + self.editor.status_message.emit(f"Submitting debug {command_label} for {scope_id}") cursor = self._replay_cursor(command_type) session.debug_terminal_summaries.pop(scope_id, None) session.start( @@ -238,9 +234,7 @@ def run_command( command_type=command_type, pause_step_indices=self.pause_step_indices(), start_step_index=0 if cursor is None else cursor.step_index, - start_after_invocation_key=( - None if cursor is None else cursor.invocation_key - ), + start_after_invocation_key=None if cursor is None else cursor.invocation_key, ) def pause_step_indices(self) -> tuple[int, ...]: @@ -284,9 +278,7 @@ async def inspect() -> None: view_model = await session.debug_runs.inspect_runtime( debug_session_id=active.debug_session_id, ) - session.main_thread.post( - lambda: self._render_runtime_inspection(view_model) - ) + session.main_thread.post(lambda: self._render_runtime_inspection(view_model)) session.start(inspect) diff --git a/openhcs/runtime/fiji_viewer_server.py b/openhcs/runtime/fiji_viewer_server.py index abf30f88e2..3658337c5e 100644 --- a/openhcs/runtime/fiji_viewer_server.py +++ b/openhcs/runtime/fiji_viewer_server.py @@ -462,9 +462,7 @@ def _require_consistent_scalar_metadata( ) -> None: if component not in item.metadata: return - normalized = ViewerComponentMetadataNormalizer( - self.declared_axes - ).normalize_value( + normalized = ViewerComponentMetadataNormalizer(self.declared_axes).normalize_value( component, item.metadata[component], ) @@ -1004,7 +1002,8 @@ def expanded_dimensions(self) -> bool: return False return ( len(self.coordinates.c.values) > len(self.stored.c) - or len(self.coordinates.z.values) > len(self.stored.z) + or len(self.coordinates.z.values) + > len(self.stored.z) or len(self.coordinates.t.values) > len(self.stored.t) ) @@ -1251,9 +1250,7 @@ def batch_message(self) -> "FijiBatchMessage": display_config_payload = fields.required_mapping( ViewerBatchWireField.DISPLAY_CONFIG ) - display_config = FijiDisplaySettings.from_display_payload( - display_config_payload - ) + display_config = FijiDisplaySettings.from_display_payload(display_config_payload) component_axis_semantics = fields.component_axis_semantics( ViewerMappingDisplayConfigInput(display_config_payload), context="Fiji component value domain", @@ -2504,7 +2501,9 @@ def _set_dimension_labels( logger.info( f"🏷️ FIJI SERVER: Setting labels for {len(coordinates.c.values)} channels" ) - for idx, channel_tuple in enumerate(coordinates.c.values, start=1): + for idx, channel_tuple in enumerate( + coordinates.c.values, start=1 + ): label = coordinates.c.label_for_value( channel_tuple, component_names_metadata, @@ -2612,7 +2611,11 @@ def _build_new_hyperstack( logger.info(f"🏷️ FIJI SERVER: Setting dimension labels for {window_key}") # For single-channel images, add channel name to window title since no slider appears - if nChannels == 1 and coordinates.c.components and coordinates.c.values: + if ( + nChannels == 1 + and coordinates.c.components + and coordinates.c.values + ): first_comp = coordinates.c.components[0] first_value_tuple = coordinates.c.values[0] channel_name = component_names_metadata.display_name( @@ -2818,7 +2821,9 @@ def handle(self, request: FijiPayloadHandlerRequest) -> None: logger.info(f"🔬 FIJI SERVER: ROI metadata: {metadata}") logger.info(f"🔬 FIJI SERVER: Channel axis: {request.coordinates.c}") - logger.info(f"🔬 FIJI SERVER: Slice axis: {request.coordinates.z}") + logger.info( + f"🔬 FIJI SERVER: Slice axis: {request.coordinates.z}" + ) logger.info(f"🔬 FIJI SERVER: Frame axis: {request.coordinates.t}") c_value, z_value, t_value = request.coordinates.imagej_position(metadata) diff --git a/openhcs/runtime/viewer_protocol.py b/openhcs/runtime/viewer_protocol.py index f8c06477f0..fddf72edee 100644 --- a/openhcs/runtime/viewer_protocol.py +++ b/openhcs/runtime/viewer_protocol.py @@ -195,8 +195,7 @@ def descend(value): return value return core_schema.no_info_before_validator_function( - descend, - super().__get_pydantic_core_schema__(source_type, handler), + descend, super().__get_pydantic_core_schema__(source_type, handler), ) def __post_init__(self) -> None: @@ -212,8 +211,7 @@ def from_wire_mapping(cls, payload: Mapping[str, object]) -> Self: def to_wire_mapping(self) -> JsonObject: return { - name: value - for name, value in super().to_wire_mapping().items() + name: value for name, value in super().to_wire_mapping().items() if value is not VIEWER_FIELD_ABSENT } | self.wire_overrides() @@ -290,15 +288,9 @@ class ViewerArrayStatistics(ViewerProjectionRecord): nonzero_count: StrictInt | ViewerFieldAbsent = VIEWER_FIELD_ABSENT min: JsonScalar | ViewerFieldAbsent = VIEWER_FIELD_ABSENT max: JsonScalar | ViewerFieldAbsent = VIEWER_FIELD_ABSENT - nonzero_min_coordinate: tuple[StrictInt, ...] | ViewerFieldAbsent = ( - VIEWER_FIELD_ABSENT - ) - nonzero_max_coordinate: tuple[StrictInt, ...] | ViewerFieldAbsent = ( - VIEWER_FIELD_ABSENT - ) - nonzero_example_coordinates: ( - tuple[tuple[StrictInt, ...], ...] | ViewerFieldAbsent - ) = VIEWER_FIELD_ABSENT + nonzero_min_coordinate: tuple[StrictInt, ...] | ViewerFieldAbsent = VIEWER_FIELD_ABSENT + nonzero_max_coordinate: tuple[StrictInt, ...] | ViewerFieldAbsent = VIEWER_FIELD_ABSENT + nonzero_example_coordinates: tuple[tuple[StrictInt, ...], ...] | ViewerFieldAbsent = VIEWER_FIELD_ABSENT nonzero_coordinate_omission_reason: str | ViewerFieldAbsent = VIEWER_FIELD_ABSENT @property @@ -328,45 +320,28 @@ class ViewerShapeCoordinateBounds(ViewerProjectionRecord): class ViewerShapeStatistics(ViewerProjectionRecord): item_count: StrictInt | ViewerFieldAbsent = VIEWER_FIELD_ABSENT shape_payload_count: StrictInt | ViewerFieldAbsent = VIEWER_FIELD_ABSENT - missing_source_spatial_shape_count: StrictInt | ViewerFieldAbsent = ( - VIEWER_FIELD_ABSENT - ) + missing_source_spatial_shape_count: StrictInt | ViewerFieldAbsent = VIEWER_FIELD_ABSENT shape_coordinate_count: StrictInt | ViewerFieldAbsent = VIEWER_FIELD_ABSENT - shape_out_of_source_bounds_count: StrictInt | ViewerFieldAbsent = ( - VIEWER_FIELD_ABSENT - ) - shape_coordinate_bounds_yx: ViewerShapeCoordinateBounds | ViewerFieldAbsent = ( - VIEWER_FIELD_ABSENT - ) + shape_out_of_source_bounds_count: StrictInt | ViewerFieldAbsent = VIEWER_FIELD_ABSENT + shape_coordinate_bounds_yx: ViewerShapeCoordinateBounds | ViewerFieldAbsent = VIEWER_FIELD_ABSENT def returned_member_count(self, returned_count: int) -> int: - return ( - returned_count - if self.shape_payload_count is VIEWER_FIELD_ABSENT - else self.shape_payload_count - ) + return (returned_count if self.shape_payload_count is VIEWER_FIELD_ABSENT + else self.shape_payload_count) def wire_overrides(self) -> JsonObject: overrides = super().wire_overrides() if self.shape_coordinate_bounds_yx is not VIEWER_FIELD_ABSENT: - overrides["shape_coordinate_bounds_yx"] = ( - self.shape_coordinate_bounds_yx.to_wire_mapping() - ) + overrides["shape_coordinate_bounds_yx"] = self.shape_coordinate_bounds_yx.to_wire_mapping() return overrides @dataclass(frozen=True, kw_only=True) class ViewerSourceSpatialSummary(ViewerProjectionRecord): - source_voxel_spacing: SourceVoxelSpacing | None | ViewerFieldAbsent = ( - VIEWER_FIELD_ABSENT - ) + source_voxel_spacing: SourceVoxelSpacing | None | ViewerFieldAbsent = VIEWER_FIELD_ABSENT - spatial_origin_yx: tuple[StrictInt, StrictInt] | None | ViewerFieldAbsent = ( - VIEWER_FIELD_ABSENT - ) - source_spatial_shape_yx: tuple[StrictInt, StrictInt] | None | ViewerFieldAbsent = ( - VIEWER_FIELD_ABSENT - ) + spatial_origin_yx: tuple[StrictInt, StrictInt] | None | ViewerFieldAbsent = VIEWER_FIELD_ABSENT + source_spatial_shape_yx: tuple[StrictInt, StrictInt] | None | ViewerFieldAbsent = VIEWER_FIELD_ABSENT @property def voxel_spacing(self) -> SourceVoxelSpacing: @@ -383,17 +358,12 @@ def source_domain(self) -> ViewerSourceSpatialDomainPayload: def require_uncropped_source_shape(self) -> tuple[int, int]: domain = self.source_domain if domain.origin_yx != (0, 0): - raise ValueError( - "Image record requires explicit uncropped pixel placement." - ) + raise ValueError("Image record requires explicit uncropped pixel placement.") return domain.required_source_shape_yx(source_label="image record") - @dataclass(frozen=True, kw_only=True) class ViewerAggregateComponentSummary(ViewerProjectionRecord): - aggregate_component_values: ( - Mapping[str, tuple[ViewerComponentValue, ...]] | None | ViewerFieldAbsent - ) = VIEWER_FIELD_ABSENT + aggregate_component_values: Mapping[str, tuple[ViewerComponentValue, ...]] | None | ViewerFieldAbsent = VIEWER_FIELD_ABSENT @property def aggregate_values(self) -> Mapping[str, tuple[ViewerComponentValue, ...]]: @@ -404,25 +374,16 @@ def validate_record(self) -> None: if any(not values for values in self.aggregate_values.values()): raise ValueError("Viewer aggregate component domains must not be empty.") - def require_plane_components( - self, - ) -> Mapping[str, tuple[ViewerComponentValue, ...]]: - if ( - self.aggregate_component_values is VIEWER_FIELD_ABSENT - or self.aggregate_component_values is None - ): - raise ValueError( - "Image record requires its explicit plane component domain." - ) + def require_plane_components(self) -> Mapping[str, tuple[ViewerComponentValue, ...]]: + if self.aggregate_component_values is VIEWER_FIELD_ABSENT or self.aggregate_component_values is None: + raise ValueError("Image record requires its explicit plane component domain.") return self.aggregate_component_values @dataclass(frozen=True, kw_only=True) class ViewerPayloadSummary( - ViewerAggregateComponentSummary, - ViewerSourceSpatialSummary, - ViewerShapeStatistics, - ViewerArrayStatistics, + ViewerAggregateComponentSummary, ViewerSourceSpatialSummary, + ViewerShapeStatistics, ViewerArrayStatistics, ): """One original native summary; no agent-side mirror or raw-map carrier.""" @@ -432,15 +393,12 @@ class ViewerPayloadSummary( payload_type: str | ViewerFieldAbsent = VIEWER_FIELD_ABSENT def coordinate_components(self) -> Mapping[str, ViewerComponentValue]: - from openhcs.runtime.viewer_component_system import ( - ViewerComponentMetadataPayload, - ) + from openhcs.runtime.viewer_component_system import ViewerComponentMetadataPayload if self.components is VIEWER_FIELD_ABSENT: raise ValueError("Viewer payload summary missing components.") return ViewerComponentMetadataPayload.component_map( - self.components, - context="viewer payload summary", + self.components, context="viewer payload summary", ) @property @@ -452,13 +410,9 @@ def require_full_image_window(self) -> None: shape = self.require_positive_shape(rank=3) source_shape = self.require_uncropped_source_shape() except ValueError as error: - raise ValueError( - "Image record requires an explicit full three-axis image window." - ) from error + raise ValueError("Image record requires an explicit full three-axis image window.") from error if source_shape != shape[-2:]: - raise ValueError( - "Image record requires an explicit full three-axis image window." - ) + raise ValueError("Image record requires an explicit full three-axis image window.") @property def full_image_plane_count(self) -> int: @@ -474,12 +428,8 @@ class ViewerArrayValueSummary(ViewerArrayStatistics): included: bool | ViewerFieldAbsent = VIEWER_FIELD_ABSENT omitted_reason: str | ViewerFieldAbsent = VIEWER_FIELD_ABSENT max_array_elements: StrictInt | ViewerFieldAbsent = VIEWER_FIELD_ABSENT - slice_ranges: tuple[tuple[StrictInt, StrictInt], ...] | ViewerFieldAbsent = ( - VIEWER_FIELD_ABSENT - ) - requested_slice_ranges: ( - tuple[tuple[StrictInt, StrictInt], ...] | ViewerFieldAbsent - ) = VIEWER_FIELD_ABSENT + slice_ranges: tuple[tuple[StrictInt, StrictInt], ...] | ViewerFieldAbsent = VIEWER_FIELD_ABSENT + requested_slice_ranges: tuple[tuple[StrictInt, StrictInt], ...] | ViewerFieldAbsent = VIEWER_FIELD_ABSENT @property def protocol_supported(self) -> bool: @@ -499,15 +449,10 @@ def shape_element_count(self) -> int | None: count *= value return count - def presentation_omission_reason( - self, *, include_requested: bool | None - ) -> str | None: + def presentation_omission_reason(self, *, include_requested: bool | None) -> str | None: reason = self.optional(self.omitted_reason) - if ( - include_requested is False - and reason == "max_array_elements_exceeded" - and self.max_array_elements == 0 - ): + if (include_requested is False and reason == "max_array_elements_exceeded" + and self.max_array_elements == 0): return "array_values_not_requested" return reason @@ -523,9 +468,7 @@ class ViewerPayloadContent(ViewerProjectionRecord): aggregate_axis_indices: tuple[StrictInt, ...] = () summary: ViewerPayloadSummary = field(default_factory=ViewerPayloadSummary) array_values: tuple[JsonValue, ...] = () - array_value_summary: ViewerArrayValueSummary = field( - default_factory=ViewerArrayValueSummary - ) + array_value_summary: ViewerArrayValueSummary = field(default_factory=ViewerArrayValueSummary) @property def is_image(self) -> bool: @@ -837,9 +780,9 @@ def from_response( payload[ViewerSettleField.ACTIVE_ROUTE_WORK_UNIT_COUNT.value] ), active_route_work_unit_active=active_work_unit_value, - processed_intake_item_count=cast( - int, payload[ViewerSettleField.PROCESSED_INTAKE_ITEM_COUNT.value] - ), + processed_intake_item_count=cast(int, payload[ + ViewerSettleField.PROCESSED_INTAKE_ITEM_COUNT.value + ]), intake_item_active=payload[ViewerSettleField.INTAKE_ITEM_ACTIVE.value], ) @@ -1728,9 +1671,7 @@ def __post_init__(self) -> None: if self.timeout <= 0: raise ValueError("Viewer control timeout must be positive.") - def to_wire_mapping( - self, *, deadline: OperationDeadline | None = None - ) -> dict[str, object]: + def to_wire_mapping(self, *, deadline: OperationDeadline | None = None) -> dict[str, object]: """Project this typed request to primitive wire fields at send.""" request: dict[str, object] = { @@ -1746,13 +1687,8 @@ def to_wire_mapping( if isinstance(payload, ViewerWindowControlRequest): payload = replace(payload, operation_deadline=None) request[ViewerControlResponseField.PAYLOAD.value] = payload - deadline = ( - deadline - or self.operation_deadline - or OperationDeadline.after_milliseconds( - max(1, int(self.timeout * 1000)), - operation="viewer control request", - ) + deadline = deadline or self.operation_deadline or OperationDeadline.after_milliseconds( + max(1, int(self.timeout * 1000)), operation="viewer control request", ) return ControlRequestHeader.with_observation_budget(request, deadline) @@ -1765,8 +1701,7 @@ def send(self) -> ViewerControlResponse: socket = None try: deadline = self.operation_deadline or OperationDeadline.after_milliseconds( - max(1, int(self.timeout * 1000)), - operation="viewer control request", + max(1, int(self.timeout * 1000)), operation="viewer control request", ) context = zmq.Context() socket = context.socket(zmq.REQ) @@ -1893,9 +1828,7 @@ def start_viewer_now(self) -> None: port = self.required_port self.prepare_fresh_viewer_start() if self.lifecycle_state.is_active: - logger.warning( - "%s viewer is already running.", self.viewer_process_label - ) + logger.warning("%s viewer is already running.", self.viewer_process_label) return logger.info( "🔬 VISUALIZER: Starting %s %s viewer (detached) on port %s", @@ -2029,15 +1962,12 @@ def matches_requested_process_launch( """ owns_process = self.owned_viewer_process_is_alive() try: - active = ( - self.process_launch if owns_process else self.active_process_launch() - ) + active = self.process_launch if owns_process else self.active_process_launch() return requested.matches_existing_viewer(active, owns_process=owns_process) except (RuntimeError, TypeError, ValueError, KeyError, zmq.ZMQError) as error: logging.getLogger(type(self).__module__).warning( "%s viewer process-launch check failed: %s", - self.viewer_process_label, - error, + self.viewer_process_label, error, ) return False diff --git a/openhcs/runtime/zmq_compilation.py b/openhcs/runtime/zmq_compilation.py index 514d17b8c8..cf761b8c9e 100644 --- a/openhcs/runtime/zmq_compilation.py +++ b/openhcs/runtime/zmq_compilation.py @@ -169,9 +169,7 @@ def compile_fresh(self) -> ZMQCompilationResult: output_dataset_root=( None if output_dataset_root is None else str(output_dataset_root) ), - auto_add_output_dataset=bool( - self.resolved_config.auto_add_output_dataset - ), + auto_add_output_dataset=bool(self.resolved_config.auto_add_output_dataset), ), ) self.progress_emitter.compiled_init_started( diff --git a/openhcs/runtime/zmq_control.py b/openhcs/runtime/zmq_control.py index 6e359838ae..ca36348501 100644 --- a/openhcs/runtime/zmq_control.py +++ b/openhcs/runtime/zmq_control.py @@ -442,21 +442,16 @@ def handle(self, message: dict, context: ZMQControlRequestContext) -> dict: CustomFunctionRegistrationResult, FunctionCatalogPreparationHandle, ) - try: request = self.request_type.from_control_payload(message) except Exception as error: return self.error_response(error) try: context.require_function_catalog_preparation().observe( - FunctionCatalogPreparationHandle( - request.connection, request.require_server_identity() - ) + FunctionCatalogPreparationHandle(request.connection, request.require_server_identity()) ).require_ready() return CustomFunctionRegistrationControlResponse( - value=context.require_function_catalog().register_custom_function( - request - ), + value=context.require_function_catalog().register_custom_function(request), ).to_control_response() except Exception as error: return CustomFunctionRegistrationControlResponse( @@ -473,16 +468,12 @@ class CustomFunctionRegistrationObservationMessageStrategy(ZMQControlMessageStra registry_key = request_type.message_type.value def handle(self, message: dict, context: ZMQControlRequestContext) -> dict: - from openhcs.agent.dto.functions import ( - CustomFunctionRegistrationObservationControlResponse, - ) + from openhcs.agent.dto.functions import CustomFunctionRegistrationObservationControlResponse try: request = self.request_type.from_control_payload(message) return CustomFunctionRegistrationObservationControlResponse( - value=context.require_function_catalog().observe_custom_function_registration( - request.handle - ), + value=context.require_function_catalog().observe_custom_function_registration(request.handle), ).to_control_response() except Exception as error: return self.error_response(error) diff --git a/openhcs/runtime/zmq_execution_client.py b/openhcs/runtime/zmq_execution_client.py index 0a9092dbf2..9207902662 100644 --- a/openhcs/runtime/zmq_execution_client.py +++ b/openhcs/runtime/zmq_execution_client.py @@ -777,7 +777,6 @@ def custom_function_registration_destination( from openhcs.agent.dto.functions import ( CustomFunctionRegistrationDestinationControlResponse, ) - response = self._send_function_catalog_exchange( request, operation_deadline=operation_deadline, @@ -789,23 +788,17 @@ def custom_function_registration_destination( ) def observe_custom_function_registration( - self, - handle: CustomFunctionRegistrationHandle, + self, handle: CustomFunctionRegistrationHandle, ) -> CustomFunctionRegistrationObservation: """One read-only exchange; never register/load the requested source.""" from openhcs.agent.dto.functions import ( CustomFunctionRegistrationObservationRequest, CustomFunctionRegistrationObservationControlResponse, ) - response = self._send_function_catalog_exchange( CustomFunctionRegistrationObservationRequest(handle), ) - return ( - CustomFunctionRegistrationObservationControlResponse.from_control_response( - response - ).value - ) + return CustomFunctionRegistrationObservationControlResponse.from_control_response(response).value def function_catalog_preparation( self, @@ -1019,9 +1012,7 @@ def serialize_task( task: OpenHCSExecutionSubmission, config=None, ) -> ExecuteRequest: - return self.serialize_prepared_request( - ZMQExecutionRequestBuilder.from_task(task) - ) + return self.serialize_prepared_request(ZMQExecutionRequestBuilder.from_task(task)) def serialize_prepared_request( self, request_builder: ZMQExecutionRequestBuilder diff --git a/openhcs/runtime/zmq_execution_server.py b/openhcs/runtime/zmq_execution_server.py index 445ad5bf2c..84ee87cfae 100644 --- a/openhcs/runtime/zmq_execution_server.py +++ b/openhcs/runtime/zmq_execution_server.py @@ -302,10 +302,7 @@ def _forward_worker_progress(self, worker_queue) -> None: # by a worker lane. They carry topology but never a worker claim. if event.axis_id: self._require_worker_claim(event, assignments) - elif ( - event.worker_slot is not None - or event.owned_partition_values is not None - ): + elif event.worker_slot is not None or event.owned_partition_values is not None: raise ValueError( f"Execution-level progress cannot carry a worker claim: {event}" ) @@ -399,9 +396,7 @@ def _execute_pipeline( execution_id: str, request_payload: ZMQExecutionRequestPayload, ): - logger.info( - "[%s] Starting subject %s", execution_id, request_payload.subject_id - ) + logger.info("[%s] Starting subject %s", execution_id, request_payload.subject_id) request_payload.compile_control.validate() self._cleanup_compiled_artifacts() @@ -650,9 +645,7 @@ def _initialize_orchestrator( if execution_bundle is None: if resolved_config is None: - raise ValueError( - "Fresh initialization requires resolved configuration." - ) + raise ValueError("Fresh initialization requires resolved configuration.") orchestrator = PipelineOrchestrator( dataset_root=Path(dataset_root), pipeline_config=pipeline_config, @@ -703,7 +696,9 @@ def _partition_values_for_execution( execution_bundle.axis_ids if execution_bundle is not None else tuple( - orchestrator.get_component_keys(AxisFamily.active().partition_axis()) + orchestrator.get_component_keys( + AxisFamily.active().partition_axis() + ) ) ) return debug_execution_policy.axis_filter_for_available(available_axis_ids) diff --git a/openhcs/runtime/zmq_execution_signature.py b/openhcs/runtime/zmq_execution_signature.py index d86f1c4491..52ea245922 100644 --- a/openhcs/runtime/zmq_execution_signature.py +++ b/openhcs/runtime/zmq_execution_signature.py @@ -84,9 +84,7 @@ def to_transport(self) -> dict[str, Any]: params: dict[str, Any] = {} if self.partition_values is not None: - params[ZMQAuxiliaryParamField.PARTITION_VALUES.value] = list( - self.partition_values - ) + params[ZMQAuxiliaryParamField.PARTITION_VALUES.value] = list(self.partition_values) if self.debug_execution_config is not None: params.update(self.debug_execution_config.to_config_params()) if self.runtime_observation_export_path is not None: diff --git a/openhcs/runtime/zmq_orchestrator_environment.py b/openhcs/runtime/zmq_orchestrator_environment.py index 76a8695d98..4f2cb40033 100644 --- a/openhcs/runtime/zmq_orchestrator_environment.py +++ b/openhcs/runtime/zmq_orchestrator_environment.py @@ -48,9 +48,7 @@ def prepared_dataset_root(self, storage_registry) -> str: from openhcs.core.dataset_sources.dataset_roots import DatasetRootRule dataset_root = str( - self.subject_id - if self.execution_subject_id is None - else self.execution_subject_id + self.subject_id if self.execution_subject_id is None else self.execution_subject_id ) return DatasetRootRule.for_dataset(dataset_root).prepare_storage( dataset_root, storage_registry diff --git a/openhcs/runtime/zmq_server_hooks.py b/openhcs/runtime/zmq_server_hooks.py index 1811e5079c..831d552785 100644 --- a/openhcs/runtime/zmq_server_hooks.py +++ b/openhcs/runtime/zmq_server_hooks.py @@ -43,7 +43,9 @@ def attach( summary = {} record.results_summary = summary - output_summary = record.get_extra(ExecutionOutputSummary.EXECUTION_RECORD_KEY) + output_summary = record.get_extra( + ExecutionOutputSummary.EXECUTION_RECORD_KEY + ) if output_summary is None: output_summary = ExecutionOutputSummary() elif not isinstance(output_summary, ExecutionOutputSummary): diff --git a/scripts/annotate_wells_with_experimental_metadata.py b/scripts/annotate_wells_with_experimental_metadata.py index 3c7b63b0de..102c30f959 100644 --- a/scripts/annotate_wells_with_experimental_metadata.py +++ b/scripts/annotate_wells_with_experimental_metadata.py @@ -23,10 +23,7 @@ # Add openhcs to path sys.path.insert(0, str(Path(__file__).parent.parent)) -from openhcs.processing.backends.experimental_analysis.analysis import ( - read_plate_layout, - load_plate_groups, -) +from openhcs.processing.backends.experimental_analysis.analysis import read_plate_layout, load_plate_groups from polystore.metadata_writer import AtomicMetadataWriter, get_metadata_path diff --git a/scripts/capture_media_gallery.py b/scripts/capture_media_gallery.py index 2629aec0a8..444196de2b 100755 --- a/scripts/capture_media_gallery.py +++ b/scripts/capture_media_gallery.py @@ -847,9 +847,7 @@ def __init__(self, request: GallerySourceCaptureRequest) -> None: ) def windows(self) -> UiWindowCatalog: - result = self._service.invoke( - UiBridgeListWindowsOperation, connection=self.connection - ) + result = self._service.invoke(UiBridgeListWindowsOperation, connection=self.connection) if result.errors: raise MediaGalleryError(f"UI window catalogue failed: {result.errors!r}") return result @@ -861,9 +859,7 @@ def visible_window_ids(self) -> frozenset[str]: def object_state_scopes(self) -> UiObjectStateScopeCatalog: request = UiObjectStateScopeListRequest.from_fields() - result = self._service.invoke( - UiBridgeListObjectStateScopesOperation, request, self.connection - ) + result = self._service.invoke(UiBridgeListObjectStateScopesOperation, request, self.connection) if result.errors: raise MediaGalleryError( f"ObjectState scope catalogue failed: {result.errors!r}" @@ -880,9 +876,7 @@ def navigate_scope( field_path=role.navigation_field_path, create_if_missing=True, ) - result = self._service.invoke( - UiBridgeNavigateWindowOperation, request, self.connection - ) + result = self._service.invoke(UiBridgeNavigateWindowOperation, request, self.connection) if ( result.errors or not result.focused @@ -894,9 +888,7 @@ def navigate_scope( return result def action_catalog(self) -> UiActionCatalog: - result = self._service.invoke( - UiBridgeListActionsOperation, connection=self.connection - ) + result = self._service.invoke(UiBridgeListActionsOperation, connection=self.connection) if result.errors: raise MediaGalleryError(f"UI action catalogue failed: {result.errors!r}") return result @@ -925,9 +917,7 @@ def invoke_plate_manager_action( observed_selection_revision_token=summary.selection_revision_token, require_confirmation=False, ) - result = self._service.invoke( - UiBridgeInvokeActionOperation, request, self.connection - ) + result = self._service.invoke(UiBridgeInvokeActionOperation, request, self.connection) if result.errors or not result.receipt.accepted: raise MediaGalleryError( f"UI action {target.action.operation_id!r} was rejected: {result!r}" @@ -948,9 +938,7 @@ def widget_tree( max_depth=None, max_nodes=2000, ) - result = self._service.invoke( - UiBridgeWidgetTreeOperation, request, self.connection - ) + result = self._service.invoke(UiBridgeWidgetTreeOperation, request, self.connection) if result.errors or not result.projected: raise MediaGalleryError( f"Widget-tree projection failed for {window_id!r}: {result!r}" @@ -1015,9 +1003,7 @@ def invoke_widget_action( action_kind=action_kind.value, target_index=target_index, ) - result = self._service.invoke( - UiBridgeInvokeWidgetActionOperation, request, self.connection - ) + result = self._service.invoke(UiBridgeInvokeWidgetActionOperation, request, self.connection) if result.errors or not result.receipt.accepted: raise MediaGalleryError( f"Widget action {summary.path_id!r} was rejected: {result!r}" @@ -1133,9 +1119,7 @@ def _poll_until(self, condition: Callable[[], bool], description: str) -> None: def close_window(self, window_id: str) -> None: request = UiWindowCloseRequest.from_fields(window_id=window_id) - result = self._service.invoke( - UiBridgeCloseWindowOperation, request, self.connection - ) + result = self._service.invoke(UiBridgeCloseWindowOperation, request, self.connection) if result.errors or not result.closed: raise MediaGalleryError( f"Could not close UI window {window_id!r}: {result!r}" @@ -1158,9 +1142,7 @@ def capture_window( output_dir_path=str(target_path.parent), create_if_missing=create_if_missing, ) - response = self._service.invoke( - UiBridgeSnapshotWindowOperation, snapshot_request, self.connection - ) + response = self._service.invoke(UiBridgeSnapshotWindowOperation, snapshot_request, self.connection) return retain_native_snapshot_source(self.request, response) diff --git a/scripts/check_basicpy_source.py b/scripts/check_basicpy_source.py index 7eadd4a76c..6af9dfc94c 100644 --- a/scripts/check_basicpy_source.py +++ b/scripts/check_basicpy_source.py @@ -153,24 +153,21 @@ def test_fields_extend_existing_projection_and_metadata_owners(self): for node in ast.walk(owner) if isinstance(node, ast.Call) ] - self.assertIn("source.metadata.collapse_leading_plane_axis", calls) - self.assertIn("source.metadata.require_independent_observation_axis", calls) - metadata_tree = ast.parse( - (ROOT / "openhcs/core/runtime_image_values.py").read_text() + self.assertIn( + "source.metadata.collapse_leading_plane_axis", calls ) + self.assertIn("source.metadata.require_independent_observation_axis", calls) + metadata_tree = ast.parse((ROOT / "openhcs/core/runtime_image_values.py").read_text()) metadata_owner = next( - node - for node in metadata_tree.body + node for node in metadata_tree.body if isinstance(node, ast.ClassDef) and node.name == "ImagePayloadMetadata" ) metadata_calls = { - ast.unparse(node.func) - for node in ast.walk(metadata_owner) + ast.unparse(node.func) for node in ast.walk(metadata_owner) if isinstance(node, ast.Call) } declared_method_names = [ - node.name - for node in metadata_owner.body + node.name for node in metadata_owner.body if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)) ] self.assertEqual(len(declared_method_names), len(set(declared_method_names))) diff --git a/scripts/figures/generate_figure_overlays.py b/scripts/figures/generate_figure_overlays.py index 64f6e20a1f..012cb7e89c 100644 --- a/scripts/figures/generate_figure_overlays.py +++ b/scripts/figures/generate_figure_overlays.py @@ -34,10 +34,7 @@ sys.path.insert(0, str(Path(__file__).parent.parent)) from openhcs.core.roi import load_rois_from_zip, PolygonShape -from openhcs.processing.backends.experimental_analysis.analysis import ( - read_plate_layout, - load_plate_groups, -) +from openhcs.processing.backends.experimental_analysis.analysis import read_plate_layout, load_plate_groups logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s") logger = logging.getLogger(__name__) diff --git a/scripts/master_multi_plate_demo.py b/scripts/master_multi_plate_demo.py index 38ded7030e..723b86db33 100644 --- a/scripts/master_multi_plate_demo.py +++ b/scripts/master_multi_plate_demo.py @@ -152,7 +152,9 @@ def resolve_identity(target: StreamProducerIdentity) -> StreamProducerIdentity: "Declared visual step has no compiled viewer stream." ) - artifact_output_kind = CompiledStepPlan.ARTIFACT_OUTPUT_KIND + artifact_output_kind = ( + CompiledStepPlan.ARTIFACT_OUTPUT_KIND + ) target_outputs = [ output for output in (target_step.get("artifact_outputs") or ()) diff --git a/scripts/mcp_assay_showcase.py b/scripts/mcp_assay_showcase.py index afc1a743de..f534cc61b2 100644 --- a/scripts/mcp_assay_showcase.py +++ b/scripts/mcp_assay_showcase.py @@ -534,7 +534,9 @@ def _primary_object_source(plate_path: Path, output_path: Path) -> str: header = _common_source_header( plate_path, output_path, - processing_imports=("from openhcs.domains.microscopy.axes import Microscopy"), + processing_imports=( + "from openhcs.domains.microscopy.axes import Microscopy" + ), extra_imports=( """from openhcs.processing.backends.analysis.cell_counting_cpu import ( DetectionMethod, diff --git a/scripts/mcp_memory_diagnostic.py b/scripts/mcp_memory_diagnostic.py index 5551b834f9..a2359c9657 100644 --- a/scripts/mcp_memory_diagnostic.py +++ b/scripts/mcp_memory_diagnostic.py @@ -171,8 +171,7 @@ def capture(cls, *, collect: bool) -> ProcessMemoryReceipt: for name, value in (line.split(":", 1),) } full_pressure = next( - line - for line in Path("/proc/pressure/memory").read_text().splitlines() + line for line in Path("/proc/pressure/memory").read_text().splitlines() if line.startswith("full ") ) pressure = dict(part.split("=", 1) for part in full_pressure.split()[1:]) @@ -212,11 +211,11 @@ def capture(cls, *, collect: bool) -> ProcessMemoryReceipt: if memory_type.loaded_module() is not None ), catalog_entries=catalog_entries, - custom_declarations=( - len(custom.CustomFunctionRuntimeRegistry.metadata_by_name()) - if custom - else 0 - ), + custom_declarations=len( + custom.CustomFunctionRuntimeRegistry.metadata_by_name() + ) + if custom + else 0, history_snapshots=len(state.ObjectStateRegistry._snapshots) if state else 0, imported_source_paths=tuple( modules[name].__file__ @@ -271,10 +270,7 @@ class MemoryDiagnosticReport: def __post_init__(self) -> None: if self.rounds < 2: raise ValueError("A retention slope requires at least two repeated rounds.") - if ( - not math.isfinite(self.round_interval_seconds) - or self.round_interval_seconds < 0 - ): + if not math.isfinite(self.round_interval_seconds) or self.round_interval_seconds < 0: raise ValueError("Round interval must be finite and nonnegative.") def write(self, output: Path) -> None: @@ -399,10 +395,8 @@ async def diagnose( ) report = MemoryDiagnosticReport( - rounds=rounds, - request_sequence=sequence, - round_interval_seconds=round_interval_seconds, - gc_at_end=gc_at_end, + rounds=rounds, request_sequence=sequence, + round_interval_seconds=round_interval_seconds, gc_at_end=gc_at_end, ) output.parent.mkdir(parents=True, exist_ok=True) @@ -521,14 +515,14 @@ def main() -> None: return report = asyncio.run( diagnose( - args.rounds, - args.timeout, - args.output, - surface_profile=surface_profile, - scratch_root=args.scratch_root, - sequence_path=args.sequence_json, - round_interval_seconds=args.round_interval_seconds, - gc_at_end=args.gc_at_end, + args.rounds, + args.timeout, + args.output, + surface_profile=surface_profile, + scratch_root=args.scratch_root, + sequence_path=args.sequence_json, + round_interval_seconds=args.round_interval_seconds, + gc_at_end=args.gc_at_end, ) ) print( diff --git a/scripts/mcp_thesis_demo_live.py b/scripts/mcp_thesis_demo_live.py index 7a4dd28e85..80714f7ef3 100755 --- a/scripts/mcp_thesis_demo_live.py +++ b/scripts/mcp_thesis_demo_live.py @@ -7,10 +7,7 @@ from __future__ import annotations from openhcs.authoring.session.operations import SessionOperation -from openhcs.authoring.session.operations.datasets import ( - ShowDatasetImages, - ShowLiveResults, -) +from openhcs.authoring.session.operations.datasets import ShowDatasetImages, ShowLiveResults import argparse import fcntl @@ -971,7 +968,9 @@ def managed_window_action_target( actions = first_payload(response, UiListActionsCapability.name).get("actions") if not isinstance(actions, list): raise RehearsalFailure("Managed-window action catalog is unavailable.") - summaries = (dataclass_from_mapping(UiActionSummary, row) for row in actions) + summaries = ( + dataclass_from_mapping(UiActionSummary, row) for row in actions + ) matches = [ row for row in summaries @@ -1102,7 +1101,8 @@ def exact_config_document_source( if not isinstance(documents, list): raise RehearsalFailure("Code-document catalog is unavailable.") summaries = ( - dataclass_from_mapping(UiCodeDocumentSummary, row) for row in documents + dataclass_from_mapping(UiCodeDocumentSummary, row) + for row in documents ) matches = [ document @@ -2745,7 +2745,8 @@ def visible_window_ids(ctx: RunContext, *, label: str) -> frozenset[str]: def window_ids_from_catalog(windows: list[Any]) -> frozenset[str]: return frozenset( - dataclass_from_mapping(UiWindowSummary, window).window_id for window in windows + dataclass_from_mapping(UiWindowSummary, window).window_id + for window in windows ) @@ -2765,7 +2766,9 @@ def plate_action_summary( actions = first_payload(response, UiListActionsCapability.name).get("actions") if not isinstance(actions, list): raise RehearsalFailure("PlateManager action catalog is unavailable.") - summaries = (dataclass_from_mapping(UiActionSummary, row) for row in actions) + summaries = ( + dataclass_from_mapping(UiActionSummary, row) for row in actions + ) matches = [ row for row in summaries @@ -2777,15 +2780,11 @@ def plate_action_summary( f"Expected one exact PlateManager action {action.operation_id!r}, got {len(matches)}." ) if not matches[0].enabled: - raise RehearsalFailure( - f"PlateManager action {action.operation_id!r} is disabled." - ) + raise RehearsalFailure(f"PlateManager action {action.operation_id!r} is disabled.") return matches[0] -def invoke_plate_action( - ctx: RunContext, action: type[SessionOperation] -) -> dict[str, Any]: +def invoke_plate_action(ctx: RunContext, action: type[SessionOperation]) -> dict[str, Any]: summary = plate_action_summary(ctx, action) response = command_json( ctx, @@ -2863,13 +2862,9 @@ def invoke_action_created_window( *, phase: str, ) -> str: - before = visible_window_ids( - ctx, label=f"{phase}_{action.operation_id}_windows_before" - ) + before = visible_window_ids(ctx, label=f"{phase}_{action.operation_id}_windows_before") invoke_plate_action(ctx, action) - after = visible_window_ids( - ctx, label=f"{phase}_{action.operation_id}_windows_after" - ) + after = visible_window_ids(ctx, label=f"{phase}_{action.operation_id}_windows_after") created = after - before if len(created) != 1: raise RehearsalFailure( @@ -4338,9 +4333,7 @@ def run_one( ctx, BASELINE_SOURCE_BINDING, ) - baseline_init = selected_workflow( - ctx, "initialize_datasets", args.workflow_timeout - ) + baseline_init = selected_workflow(ctx, "initialize_datasets", args.workflow_timeout) baseline_metadata = canonical_component_metadata( ctx, phase="baseline", @@ -4351,9 +4344,7 @@ def run_one( ctx, EDITED_SOURCE_BINDING, ) - edited_init = selected_workflow( - ctx, "initialize_datasets", args.workflow_timeout - ) + edited_init = selected_workflow(ctx, "initialize_datasets", args.workflow_timeout) edited_metadata = canonical_component_metadata( ctx, phase="edited", @@ -4369,9 +4360,7 @@ def run_one( ctx, REVERTED_SOURCE_BINDING, ) - reverted_init = selected_workflow( - ctx, "initialize_datasets", args.workflow_timeout - ) + reverted_init = selected_workflow(ctx, "initialize_datasets", args.workflow_timeout) reverted_metadata = canonical_component_metadata( ctx, phase="reverted", diff --git a/scripts/prototype_flatten_object_state.py b/scripts/prototype_flatten_object_state.py index 322466904a..5c13c6ad92 100644 --- a/scripts/prototype_flatten_object_state.py +++ b/scripts/prototype_flatten_object_state.py @@ -320,8 +320,7 @@ def test_reconstruction(): == original.partition_filter_config.partition_filter ) assert ( - reconstructed.partition_filter_config.enabled - == original.partition_filter_config.enabled + reconstructed.partition_filter_config.enabled == original.partition_filter_config.enabled ) assert ( reconstructed.napari_streaming_config.window_size @@ -466,9 +465,7 @@ def test_none_handling(): print( f" partition_filter_config.partition_filter: {reconstructed.partition_filter_config.partition_filter}" ) - print( - f" partition_filter_config.enabled: {reconstructed.partition_filter_config.enabled}" - ) + print(f" partition_filter_config.enabled: {reconstructed.partition_filter_config.enabled}") # The reconstructed object should have None for partition_filter (not set in filtered_fields) assert reconstructed.partition_filter_config.partition_filter is None diff --git a/scripts/smoke_installed_mcp.py b/scripts/smoke_installed_mcp.py index 06c48c5286..74ef95ebe5 100644 --- a/scripts/smoke_installed_mcp.py +++ b/scripts/smoke_installed_mcp.py @@ -302,7 +302,6 @@ async def _run_measured_execution_protocol_smoke( pipeline_source = PipelineDocumentCodec.render(document) source_file = output_dir / "pipeline.py" source_file.write_text(pipeline_source, encoding="utf-8") - async def call(name: str, arguments: dict, timeout: float = 90) -> dict: return _tool_payload( await asyncio.wait_for(session.call_tool(name, arguments), timeout=timeout) @@ -331,9 +330,7 @@ async def call(name: str, arguments: dict, timeout: float = 90) -> dict: "openhcs_run_datasets", { "runtime_observation_export_path": str( - MeasuredPipelineRunArtifact.RUNTIME_OBSERVATION.path_in( - evidence_dir - ) + MeasuredPipelineRunArtifact.RUNTIME_OBSERVATION.path_in(evidence_dir) ), "runtime_observation_export_scope": "outcomes", }, @@ -343,9 +340,7 @@ async def call(name: str, arguments: dict, timeout: float = 90) -> dict: ): started = await call(tool, {"scope_ids": [scope_id], **arguments}) if started.get("status") != "accepted": - raise AssertionError( - f"Installed MCP rejected {tool}: {started} {connected}" - ) + raise AssertionError(f"Installed MCP rejected {tool}: {started} {connected}") sequence = started["event_sequence"] deadline = time.monotonic() + 180 while True: diff --git a/scripts/test_real_global_config.py b/scripts/test_real_global_config.py index dc2fbda580..172a17fadb 100644 --- a/scripts/test_real_global_config.py +++ b/scripts/test_real_global_config.py @@ -73,13 +73,8 @@ def test_real_global_config(): print(f"Reconstructed successfully: {type(reconstructed) == type(config)}") # Verify a few nested configs are correct - if ( - hasattr(config, "partition_filter_config") - and config.partition_filter_config is not None - ): - assert type(reconstructed.partition_filter_config) == type( - config.partition_filter_config - ) + if hasattr(config, "partition_filter_config") and config.partition_filter_config is not None: + assert type(reconstructed.partition_filter_config) == type(config.partition_filter_config) print(f"✓ partition_filter_config type matches") if ( diff --git a/tests/diagnostics/check_owned_bootstrap_live.py b/tests/diagnostics/check_owned_bootstrap_live.py index bc3304a13f..6e8005b256 100644 --- a/tests/diagnostics/check_owned_bootstrap_live.py +++ b/tests/diagnostics/check_owned_bootstrap_live.py @@ -22,8 +22,8 @@ from openhcs.microscopes.bioformats import BioFormatsHandler SOURCE = Path(__file__).resolve().parents[2] -PYTHON = Path("/home/ts/code/projects/openhcs/.venv/bin/python") -LOCK = Path("/home/ts/wt/openhcs-issue-batch-20260929/validation.lock") +PYTHON = Path('/home/ts/code/projects/openhcs/.venv/bin/python') +LOCK = Path('/home/ts/wt/openhcs-issue-batch-20260929/validation.lock') def fixture_registration_sources(): @@ -35,131 +35,85 @@ def fixture_registration_sources(): from tests.diagnostics import volume_projection_fixture as fixture dependencies = ( - ( - fixture.select_volume_fixture_planes_v2, - "ArrayPayload, numpy, Pure3DContract, artifact_outputs, SELECTED_VOLUME, " - "SelectedPlaneImageOutput, np", - ), - ( - fixture.inspect_volume_fixture_v2, - "ArrayPayload, numpy, Pure3DContract, artifact_outputs, VOLUME_IMAGE, " - "VOLUME_LABELS, VOLUME_ROWS, np, " - "DataclassMeasurementColumnarRows, VolumeProjectionFixtureRow", - ), + (fixture.select_volume_fixture_planes_v2, + 'ArrayPayload, numpy, Pure3DContract, artifact_outputs, SELECTED_VOLUME, ' + 'SelectedPlaneImageOutput, np'), + (fixture.inspect_volume_fixture_v2, + 'ArrayPayload, numpy, Pure3DContract, artifact_outputs, VOLUME_IMAGE, ' + 'VOLUME_LABELS, VOLUME_ROWS, np, ' + 'DataclassMeasurementColumnarRows, VolumeProjectionFixtureRow'), ) return tuple( - ( - function.__name__, - f"from tests.diagnostics.volume_projection_fixture import ({imports})\n\n" - + inspect.getsource(function), - ) + (function.__name__, + f'from tests.diagnostics.volume_projection_fixture import ({imports})\n\n' + + inspect.getsource(function)) for function, imports in dependencies ) def guard(*, projected_disk_gib: float = 0.5, disk_reserve_gib: float = 2) -> dict: result = subprocess.run( - ["/home/ts/bin/agent-resource-check", "--assert-headroom"], - capture_output=True, - text=True, - check=False, + ['/home/ts/bin/agent-resource-check', '--assert-headroom'], + capture_output=True, text=True, check=False, ) resources = json.loads(result.stdout) # Warning text is not an admission policy: in particular, 20 GiB free # is a host notification threshold, not the footprint of this bounded run. - if ( - resources["level"] == "critical" - or resources["ram_available_gib"] < 8 - or resources["free_gib"]["/home"] < projected_disk_gib + disk_reserve_gib - ): - raise RuntimeError(f"Resource gate closed: {resources}") + if (resources['level'] == 'critical' or resources['ram_available_gib'] < 8 + or resources['free_gib']['/home'] < projected_disk_gib + disk_reserve_gib): + raise RuntimeError(f'Resource gate closed: {resources}') return resources def run(args) -> None: assert Path(sys.executable).resolve() == PYTHON.resolve() sha = subprocess.check_output( - ["git", "rev-parse", "HEAD"], - cwd=SOURCE, - text=True, + ['git', 'rev-parse', 'HEAD'], cwd=SOURCE, text=True, ).strip() assert sha == args.expected_sha root = args.receipts.resolve() assert root.is_relative_to(SOURCE) and not root.exists() - with LOCK.open("a+") as lock: + with LOCK.open('a+') as lock: fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB) resources = guard() root.mkdir(parents=True) - owned = root / "owned" + owned = root / 'owned' owned.mkdir() - for key in ( - "OMP_NUM_THREADS", - "OPENBLAS_NUM_THREADS", - "MKL_NUM_THREADS", - "NUMEXPR_NUM_THREADS", - "NUMBA_NUM_THREADS", - "BLIS_NUM_THREADS", - ): - os.environ[key] = "1" + for key in ('OMP_NUM_THREADS', 'OPENBLAS_NUM_THREADS', 'MKL_NUM_THREADS', + 'NUMEXPR_NUM_THREADS', 'NUMBA_NUM_THREADS', 'BLIS_NUM_THREADS'): + os.environ[key] = '1' os.environ.update( - OPENHCS_CPU_ONLY="true", - OPENHCS_HEADLESS="true", - OPENHCS_USE_THREADING="true", - OPENHCS_SUBPROCESS_NO_GPU="1", - POLYSTORE_SUBPROCESS_NO_GPU="1", - CUDA_VISIBLE_DEVICES="", - QT_QPA_PLATFORM="offscreen", - PYTHONDONTWRITEBYTECODE="1", - POLYSTORE_IMAGEJ_ALLOW_DOWNLOAD="false", - POLYSTORE_IMAGEJ_CACHE_ROOT="/home/ts/.cache/polystore/imagej", - XDG_DATA_HOME=str(owned / "data"), - XDG_CACHE_HOME=str(owned / "cache"), - XDG_CONFIG_HOME=str(owned / "config"), - XDG_STATE_HOME=str(owned / "state"), - XDG_RUNTIME_DIR=str(owned / "runtime"), - NUMBA_CACHE_DIR=str(owned / "numba"), - MPLCONFIGDIR=str(owned / "matplotlib"), - OPENHCS_UI_CONFIG_CACHE_FILE=str(owned / "ui_config.config"), - ) - receipt = dict( - accepted=False, - source_sha=sha, - driver_pid=os.getpid(), - source_live_not_installed=True, - resources=resources, - calls=[], - uncertain_dispatch=None, - runtime_terminal=False, + OPENHCS_CPU_ONLY='true', OPENHCS_HEADLESS='true', + OPENHCS_USE_THREADING='true', OPENHCS_SUBPROCESS_NO_GPU='1', + POLYSTORE_SUBPROCESS_NO_GPU='1', CUDA_VISIBLE_DEVICES='', + QT_QPA_PLATFORM='offscreen', PYTHONDONTWRITEBYTECODE='1', + POLYSTORE_IMAGEJ_ALLOW_DOWNLOAD='false', + POLYSTORE_IMAGEJ_CACHE_ROOT='/home/ts/.cache/polystore/imagej', + XDG_DATA_HOME=str(owned / 'data'), XDG_CACHE_HOME=str(owned / 'cache'), + XDG_CONFIG_HOME=str(owned / 'config'), XDG_STATE_HOME=str(owned / 'state'), + XDG_RUNTIME_DIR=str(owned / 'runtime'), + NUMBA_CACHE_DIR=str(owned / 'numba'), MPLCONFIGDIR=str(owned / 'matplotlib'), + OPENHCS_UI_CONFIG_CACHE_FILE=str(owned / 'ui_config.config'), ) + receipt = dict(accepted=False, source_sha=sha, driver_pid=os.getpid(), + source_live_not_installed=True, resources=resources, calls=[], + uncertain_dispatch=None, runtime_terminal=False) def save(): - (root / "journey.json").write_text(json.dumps(receipt, indent=2) + "\n") + (root / 'journey.json').write_text(json.dumps(receipt, indent=2) + '\n') save() # Existing declaration-owned setup builds both recorded native modules. # No package installation, download or shared source mutation. tick = time.monotonic() - with (root / "build.txt").open("w") as log: + with (root / 'build.txt').open('w') as log: build = subprocess.run( - [ - str(PYTHON), - "-B", - "setup.py", - "build_ext", - "--inplace", - "-j", - "1", - "--build-temp", - str(root / "build-temp"), - "--build-lib", - str(root / "build-lib"), - ], - cwd=SOURCE, - stdout=log, - stderr=subprocess.STDOUT, - timeout=60, + [str(PYTHON), '-B', 'setup.py', 'build_ext', '--inplace', '-j', '1', + '--build-temp', str(root / 'build-temp'), + '--build-lib', str(root / 'build-lib')], + cwd=SOURCE, stdout=log, stderr=subprocess.STDOUT, timeout=60, ) - receipt["build"] = dict(exit=build.returncode, seconds=time.monotonic() - tick) + receipt['build'] = dict(exit=build.returncode, seconds=time.monotonic()-tick) save() assert build.returncode == 0 @@ -175,15 +129,11 @@ def save(): from zmqruntime.transport import TransportEndpoint from openhcs.agent.capabilities import agent_capabilities from openhcs.agent.dto.execution import ( - RuntimeBootstrapState, - RuntimeBootstrapCloseResult, - ExecutionJobRef, - ExecutionJobStatus, - OrchestratorSessionRef, + RuntimeBootstrapState, RuntimeBootstrapCloseResult, + ExecutionJobRef, ExecutionJobStatus, OrchestratorSessionRef, ) from openhcs.agent.dto.functions import ( - FunctionCatalogPreparationState, - CustomFunctionRegistrationResult, + FunctionCatalogPreparationState, CustomFunctionRegistrationResult, FunctionCatalogPage, ) from openhcs.mcp.dev_client import McpDevClient @@ -193,130 +143,81 @@ def save(): from openhcs.runtime.zmq_execution_client import ZMQExecutionClient from python_introspect import to_jsonable - modules = ( - openhcs, - polystore, - zmqruntime, - metaclass_registry, - importlib.import_module("openhcs.core._tabular_native"), - importlib.import_module( - "openhcs.processing.backends.cellprofiler._granularity_native" - ), - ) - receipt["imports"] = {module.__name__: module.__file__ for module in modules} - assert all( - Path(module.__file__).resolve().is_relative_to(SOURCE) for module in modules - ) - receipt["submodules"] = subprocess.check_output( - ["git", "submodule", "status"], - cwd=SOURCE, - text=True, + modules = (openhcs, polystore, zmqruntime, metaclass_registry, + importlib.import_module('openhcs.core._tabular_native'), + importlib.import_module('openhcs.processing.backends.cellprofiler._granularity_native')) + receipt['imports'] = {module.__name__: module.__file__ for module in modules} + assert all(Path(module.__file__).resolve().is_relative_to(SOURCE) for module in modules) + receipt['submodules'] = subprocess.check_output( + ['git', 'submodule', 'status'], cwd=SOURCE, text=True, ).splitlines() - assert not any(row.startswith(("-", "+", "U")) for row in receipt["submodules"]) - config = replace( - OPENHCS_ZMQ_CONFIG, - transport_mode=TransportMode.TCP, - client_host="127.0.0.1", - server_host="127.0.0.1", - default_port=args.port, - ) - endpoint = TransportEndpoint("127.0.0.1", args.port, TransportMode.TCP) + assert not any(row.startswith(('-', '+', 'U')) for row in receipt['submodules']) + config = replace(OPENHCS_ZMQ_CONFIG, transport_mode=TransportMode.TCP, + client_host='127.0.0.1', server_host='127.0.0.1', + default_port=args.port) + endpoint = TransportEndpoint('127.0.0.1', args.port, TransportMode.TCP) assert args.port not in (7777, 7790, 5991) assert not endpoint.occupied_ports(config) - launch_plan = ZMQExecutionClient( - port=args.port, - host="127.0.0.1", - transport_mode=TransportMode.TCP, - config=config, - ).runtime_launch_plan() + launch_plan = ZMQExecutionClient(port=args.port, host='127.0.0.1', + transport_mode=TransportMode.TCP, + config=config).runtime_launch_plan() lock_paths = launch_plan.transport_write_paths - assert all( - not path.exists() for path in lock_paths - ), "Preserve existing reservations" - fixture_root = Path( - "/home/ts/wt/openhcs-issue-batch-20260929/issue257-synthetic-ome-inputs-20260930" - ) - fixture_source = SOURCE / "tests/diagnostics/volume_projection_fixture.py" - os.environ["OPENHCS_AGENT_READ_ROOTS"] = os.pathsep.join( - (str(SOURCE), str(fixture_root), *(str(p) for p in lock_paths)) - ) - os.environ["OPENHCS_AGENT_WRITE_ROOTS"] = os.pathsep.join( - (str(owned), *(str(p) for p in lock_paths)) - ) + assert all(not path.exists() for path in lock_paths), 'Preserve existing reservations' + fixture_root = Path('/home/ts/wt/openhcs-issue-batch-20260929/issue257-synthetic-ome-inputs-20260930') + fixture_source = SOURCE / 'tests/diagnostics/volume_projection_fixture.py' + os.environ['OPENHCS_AGENT_READ_ROOTS'] = os.pathsep.join((str(SOURCE), str(fixture_root), *(str(p) for p in lock_paths))) + os.environ['OPENHCS_AGENT_WRITE_ROOTS'] = os.pathsep.join((str(owned), *(str(p) for p in lock_paths))) assert save_ui_config_sync(UIConfig(zmq=config)) - receipt["declared_endpoint_pair"] = list(endpoint.port_pair(config).ports) - receipt["transport_locks"] = [str(p) for p in lock_paths] + receipt['declared_endpoint_pair'] = list(endpoint.port_pair(config).ports) + receipt['transport_locks'] = [str(p) for p in lock_paths] from polystore.zmq_config import POLYSTORE_ZMQ_CONFIG - - receipt["ack_boundary"] = dict( + receipt['ack_boundary'] = dict( native_config_port=config.shared_ack_port, polystore_config_port=POLYSTORE_ZMQ_CONFIG.shared_ack_port, - canonical_owner="zmqruntime.ack_listener.GlobalAckListener", - streaming_caller="polystore.streaming._streaming_backend.StreamingBackend", - bind_host="*", - independent_of_execution_pair=True, - sockets_before=subprocess.check_output(["ss", "-ltnp"], text=True), - foreign_ack_cleanup="never requested", + canonical_owner='zmqruntime.ack_listener.GlobalAckListener', + streaming_caller='polystore.streaming._streaming_backend.StreamingBackend', + bind_host='*', independent_of_execution_pair=True, + sockets_before=subprocess.check_output(['ss', '-ltnp'], text=True), + foreign_ack_cleanup='never requested', ) - original_image = fixture_root / "A01/image.ome.tif" - plate = owned / "plate" + original_image = fixture_root / 'A01/image.ome.tif' + plate = owned / 'plate' plate.mkdir() image_path = plate / original_image.name import shutil - shutil.copyfile(original_image, image_path) import tifffile - input_pixels = tifffile.imread(image_path) assert input_pixels.shape == (3, 8, 9) input_hash = hashlib.sha256(image_path.read_bytes()).hexdigest() - assert ( - input_hash - == "5fc4e6baa8015be4b5ca09356575a2083ed70834f1ff447353e6c116fec248bb" - ) + assert input_hash == '5fc4e6baa8015be4b5ca09356575a2083ed70834f1ff447353e6c116fec248bb' assert image_path.read_bytes() == original_image.read_bytes() fixture_hash = hashlib.sha256(fixture_source.read_bytes()).hexdigest() - assert ( - fixture_hash - == "9b737ed5e669c6f2325cc4af9ebb0b39b706121478149315474f464e3cc30c80" - ) + assert fixture_hash == '9b737ed5e669c6f2325cc4af9ebb0b39b706121478149315474f464e3cc30c80' registration_sources = fixture_registration_sources() from openhcs.processing.custom_functions.manager import CustomFunctionManager - manager = CustomFunctionManager(create_storage=False) for name, code in registration_sources: prepared_source = manager._prepare_source(code) assert prepared_source.original_name == name - (root / f"{name}.py").write_text(code) - receipt["fixture"] = dict( - source=str(fixture_source), - input=str(image_path), - original_input=str(original_image), - input_sha256=input_hash, - byte_identical_copy=True, - source_sha256=fixture_hash, - plane_selections=[[], [2, 0, 1], [0, 1], [1]], - expected_source_planes=[[0, 1, 2], [2, 0, 1], [2, 0], [0]], - registration_source_hashes={ - name: hashlib.sha256(code.encode()).hexdigest() - for name, code in registration_sources - }, - omitted_controls=[], - ) + (root/f'{name}.py').write_text(code) + receipt['fixture'] = dict(source=str(fixture_source), input=str(image_path), + original_input=str(original_image), + input_sha256=input_hash, byte_identical_copy=True, + source_sha256=fixture_hash, + plane_selections=[[], [2, 0, 1], [0, 1], [1]], + expected_source_planes=[[0, 1, 2], [2, 0, 1], [2, 0], [0]], + registration_source_hashes={name: hashlib.sha256(code.encode()).hexdigest() + for name, code in registration_sources}, + omitted_controls=[]) class SourceSpec(McpDevServerSpec): - mcp_environment_keys = ( - *McpDevServerSpec.mcp_environment_keys, - "PYTHONPATH", - "MPLCONFIGDIR", - ) + mcp_environment_keys = (*McpDevServerSpec.mcp_environment_keys, + 'PYTHONPATH', 'MPLCONFIGDIR') - client = McpDevClient( - str(PYTHON), - use_resident_server=False, - server_stderr=(root / "mcp-stderr.txt").open("w"), - initialize_timeout_seconds=10, - ) + client = McpDevClient(str(PYTHON), use_resident_server=False, + server_stderr=(root / 'mcp-stderr.txt').open('w'), + initialize_timeout_seconds=10) client.server_spec = SourceSpec(str(PYTHON)) original_handle = None began = time.monotonic() @@ -324,65 +225,41 @@ class SourceSpec(McpDevServerSpec): def call(name, arguments, result_type=None, *, lifecycle=False): if not lifecycle: current = guard() - size = sum(p.stat().st_size for p in root.rglob("*") if p.is_file()) - assert size < 80 * 1024 * 1024, "80MiB artifact bound" - assert time.monotonic() - began < 240, "Original finite journey bound" - receipt["last_resources"] = current - row = dict( - tool=name, - arguments=arguments, - started_unix=time.time(), - status="dispatched", - ) - receipt["calls"].append(row) - receipt["uncertain_dispatch"] = len(receipt["calls"]) + size = sum(p.stat().st_size for p in root.rglob('*') if p.is_file()) + assert size < 80 * 1024 * 1024, '80MiB artifact bound' + assert time.monotonic()-began < 240, 'Original finite journey bound' + receipt['last_resources'] = current + row = dict(tool=name, arguments=arguments, started_unix=time.time(), status='dispatched') + receipt['calls'].append(row) + receipt['uncertain_dispatch'] = len(receipt['calls']) save() tick = time.monotonic() result = client.execute( - [ - "--allow-error-payloads", - "call", - name, - "--arguments", - json.dumps(arguments), - "--json", - ], - timeout_seconds=10, - ) - row.update( - seconds=time.monotonic() - tick, - response=result.payload, - status="returned", + ['--allow-error-payloads', 'call', name, '--arguments', + json.dumps(arguments), '--json'], timeout_seconds=10, ) + row.update(seconds=time.monotonic()-tick, response=result.payload, status='returned') save() - assert not result.payload["errors"], result.payload - response = result.payload["results"][0] - assert not response["mcp_error"], response - payload = response["payloads"][0] - if ( - payload.get("errors") - and payload["errors"][0]["code"] == "agent_path_policy_rejected" - ): + assert not result.payload['errors'], result.payload + response = result.payload['results'][0] + assert not response['mcp_error'], response + payload = response['payloads'][0] + if payload.get('errors') and payload['errors'][0]['code'] == 'agent_path_policy_rejected': # This specific original path-policy owner rejects before dispatch. # Other errors, especially post-dispatch uncertainty, retain handles. - receipt["uncertain_dispatch"] = None + receipt['uncertain_dispatch'] = None save() - assert not payload.get("errors"), payload - value = ( - dataclass_from_mapping(result_type, payload) if result_type else payload - ) - receipt["uncertain_dispatch"] = None + assert not payload.get('errors'), payload + value = dataclass_from_mapping(result_type, payload) if result_type else payload + receipt['uncertain_dispatch'] = None save() print(f'{name}: {row["seconds"]:.3f}s', flush=True) return value def finish(ref): while True: - state = call( - agent_capabilities.get_execution_status.name, - dict(job_id=ref.job_id, timeout_ms=1000), - ExecutionJobStatus, - ) + state = call(agent_capabilities.get_execution_status.name, + dict(job_id=ref.job_id, timeout_ms=1000), ExecutionJobStatus) if state.is_terminal: assert state.status == ExecutionStatus.COMPLETE.value, state return state @@ -390,258 +267,147 @@ def finish(ref): try: client.start() - health = call("openhcs_health_check", {}) - assert ( - Path(health["server_source_path"]).resolve() - == SOURCE / "openhcs/mcp/server.py" - ) - assert not health["restart_required"] - receipt["mcp_pid"] = health["server_process_id"] - call("openhcs_get_authoring_context", dict(kind="first_use")) - call("openhcs_search_capabilities", dict(query="owned runtime", limit=10)) - startup = call( - agent_capabilities.start_owned_runtime.name, - dict( - port=args.port, - host="127.0.0.1", - transport_mode="tcp", - timeout_ms=500, - ), - RuntimeBootstrapState, - ) + health = call('openhcs_health_check', {}) + assert Path(health['server_source_path']).resolve() == SOURCE/'openhcs/mcp/server.py' + assert not health['restart_required'] + receipt['mcp_pid'] = health['server_process_id'] + call('openhcs_get_authoring_context', dict(kind='first_use')) + call('openhcs_search_capabilities', dict(query='owned runtime', limit=10)) + startup = call(agent_capabilities.start_owned_runtime.name, + dict(port=args.port, host='127.0.0.1', transport_mode='tcp', + timeout_ms=500), RuntimeBootstrapState) original_handle = startup.handle - receipt["bootstrap_handle"] = to_jsonable(original_handle) + receipt['bootstrap_handle'] = to_jsonable(original_handle) save() while not startup.ready: assert startup.process_alive is not False, startup time.sleep(0.5) - startup = call( - agent_capabilities.observe_owned_runtime.name, - dict(handle=to_jsonable(original_handle)), - RuntimeBootstrapState, - ) + startup = call(agent_capabilities.observe_owned_runtime.name, + dict(handle=to_jsonable(original_handle)), RuntimeBootstrapState) assert startup.handle == original_handle - receipt["bootstrap_ready_seconds"] = time.monotonic() - began + receipt['bootstrap_ready_seconds'] = time.monotonic()-began prepare_started = time.monotonic() - prepared = call( - agent_capabilities.start_function_catalog_preparation.name, - dict(port=args.port, host="127.0.0.1", transport_mode="tcp"), - FunctionCatalogPreparationState, - ) + prepared = call(agent_capabilities.start_function_catalog_preparation.name, + dict(port=args.port, host='127.0.0.1', transport_mode='tcp'), + FunctionCatalogPreparationState) preparation_handle = prepared.handle - receipt["preparation_handle"] = to_jsonable(preparation_handle) + receipt['preparation_handle'] = to_jsonable(preparation_handle) while not prepared.outcome.ready: assert not prepared.outcome.terminal, prepared time.sleep(1) - prepared = call( - agent_capabilities.get_function_catalog_preparation_status.name, - to_jsonable(preparation_handle), - FunctionCatalogPreparationState, - ) + prepared = call(agent_capabilities.get_function_catalog_preparation_status.name, + to_jsonable(preparation_handle), FunctionCatalogPreparationState) assert prepared.handle == preparation_handle - receipt["preparation_seconds"] = time.monotonic() - prepare_started - receipt["registrations"] = [] + receipt['preparation_seconds'] = time.monotonic()-prepare_started + receipt['registrations'] = [] for name, code in registration_sources: - registered = call( - "openhcs_register_custom_function", - dict( - source_code=code, - function_name=name, - storage_dir=str(original_handle.launch_plan.storage_dir), - persist=True, - port=args.port, - host="127.0.0.1", - transport_mode="tcp", - ), - CustomFunctionRegistrationResult, - ) + registered = call('openhcs_register_custom_function', dict( + source_code=code, function_name=name, + storage_dir=str(original_handle.launch_plan.storage_dir), persist=True, + port=args.port, host='127.0.0.1', transport_mode='tcp', + ), CustomFunctionRegistrationResult) assert registered.registered_count == 1 assert registered.server_identity == original_handle.process_identity [persisted] = registered.source_file_paths assert Path(persisted).read_text() == code - receipt["registrations"].append(to_jsonable(registered)) - page = call( - "openhcs_search_functions", - dict(query=name, limit=5), - FunctionCatalogPage, - ) - assert registered.functions[0].function_id in { - entry.function_id for entry in page.items - } + receipt['registrations'].append(to_jsonable(registered)) + page = call('openhcs_search_functions', dict(query=name, limit=5), FunctionCatalogPage) + assert registered.functions[0].function_id in {entry.function_id for entry in page.items} save() - from openhcs.core.config import ( - PipelineConfig, - LazyPathPlanningConfig, - LazyVFSConfig, - MaterializationBackend, - LazyStepMaterializationConfig, - LazyProcessingConfig, - ) + from openhcs.core.config import (PipelineConfig, LazyPathPlanningConfig, + LazyVFSConfig, MaterializationBackend, + LazyStepMaterializationConfig, LazyProcessingConfig) from openhcs.core.pipeline_document import PipelineDocumentCodec from openhcs.core.steps.function_step import FunctionStep from openhcs.processing.custom_functions import ( - select_volume_fixture_planes_v2, - inspect_volume_fixture_v2, + select_volume_fixture_planes_v2, inspect_volume_fixture_v2, ) - steps = [] for case, indices in enumerate(((), (2, 0, 1), (0, 1), (1,))): - for phase, function in enumerate( - ( - select_volume_fixture_planes_v2, - inspect_volume_fixture_v2, - inspect_volume_fixture_v2, - ) - ): - steps.append( - FunctionStep( - func=( - (function, dict(plane_indices=indices)) - if phase == 0 - else function - ), - name=f"ProjectionCase{case}Step{phase}", - processing_config=LazyProcessingConfig( - variable_components=[Microscopy.ZIndex] - ), - step_materialization_config=LazyStepMaterializationConfig( - enabled=True - ), - ) - ) + for phase, function in enumerate((select_volume_fixture_planes_v2, + inspect_volume_fixture_v2, inspect_volume_fixture_v2)): + steps.append(FunctionStep( + func=((function, dict(plane_indices=indices)) if phase == 0 else function), + name=f'ProjectionCase{case}Step{phase}', + processing_config=LazyProcessingConfig(variable_components=[Microscopy.ZIndex]), + step_materialization_config=LazyStepMaterializationConfig(enabled=True))) document = PipelineDocumentCodec.from_values( - pipeline_config=PipelineConfig( - num_workers=1, - use_threading=True, + pipeline_config=PipelineConfig(num_workers=1, use_threading=True, dataset_source=BioFormatsHandler, - path_planning_config=LazyPathPlanningConfig( - global_output_folder=owned / "outputs" - ), - vfs_config=LazyVFSConfig( - materialization_backend=MaterializationBackend.DISK - ), - ), + path_planning_config=LazyPathPlanningConfig(global_output_folder=owned/'outputs'), + vfs_config=LazyVFSConfig(materialization_backend=MaterializationBackend.DISK)), pipeline_steps=steps, ) source = PipelineDocumentCodec.render(document) - (root / "pipeline.py").write_text(source) + (root/'pipeline.py').write_text(source) from openhcs.agent.dto.execution import ArtifactPlanInspection - - inspected = call( - "openhcs_inspect_pipeline_source_artifact_plan", - dict(plate_path=str(image_path.parent), pipeline_source=source), - ArtifactPlanInspection, - ) - receipt["artifact_plan"] = to_jsonable(inspected) + inspected = call('openhcs_inspect_pipeline_source_artifact_plan', + dict(plate_path=str(image_path.parent), pipeline_source=source), ArtifactPlanInspection) + receipt['artifact_plan'] = to_jsonable(inspected) assert inspected.step_count == 12 and inspected.axis_count == 1 - session = call( - "openhcs_create_orchestrator_session_from_pipeline_source", - dict( - plate_path=str(image_path.parent), - pipeline_source=source, - port=args.port, - host="127.0.0.1", - transport_mode="tcp", - ), - OrchestratorSessionRef, - ) - for tool, stage in ( - ("openhcs_submit_compile", "compile"), - ("openhcs_submit_pipeline_execution", "execution"), - ): + session = call('openhcs_create_orchestrator_session_from_pipeline_source', + dict(plate_path=str(image_path.parent), pipeline_source=source, + port=args.port, host='127.0.0.1', transport_mode='tcp'), + OrchestratorSessionRef) + for tool, stage in (('openhcs_submit_compile', 'compile'), + ('openhcs_submit_pipeline_execution', 'execution')): tick = time.monotonic() arguments = dict(session_id=session.session_id, wait=False) - if stage == "execution": - arguments["runtime_observation_export_path"] = str( - owned / "observation.pkl.gz" - ) + if stage == 'execution': + arguments['runtime_observation_export_path'] = str(owned/'observation.pkl.gz') job = call(tool, arguments, ExecutionJobRef) receipt[stage] = to_jsonable(finish(job)) - receipt[stage + "_seconds"] = time.monotonic() - tick + receipt[stage+'_seconds'] = time.monotonic()-tick save() - from tests.diagnostics.owned_bootstrap_readback import ( - verify_volume_publication, - ) - - receipt["volume_publication"] = verify_volume_publication( - owned, - image_path, - input_pixels, - inspected, + from tests.diagnostics.owned_bootstrap_readback import verify_volume_publication + receipt['volume_publication'] = verify_volume_publication( + owned, image_path, input_pixels, inspected, ) assert hashlib.sha256(image_path.read_bytes()).hexdigest() == input_hash assert hashlib.sha256(original_image.read_bytes()).hexdigest() == input_hash - receipt.update( - accepted=True, - unchanged_input_sha256=input_hash, - registration_mcp_calls=len(registration_sources), - registrations_per_declaration=1, - no_mutation_replay=True, - ) + receipt.update(accepted=True, unchanged_input_sha256=input_hash, + registration_mcp_calls=len(registration_sources), + registrations_per_declaration=1, no_mutation_replay=True) save() except BaseException as error: - receipt["failure"] = dict( - type=type(error).__name__, - message=str(error), - traceback=traceback.format_exc(), - ) + receipt['failure'] = dict(type=type(error).__name__, message=str(error), + traceback=traceback.format_exc()) save() - print( - "STOPPED: same handle retained; read-only CLI commands or close-owned only.", - flush=True, - ) + print('STOPPED: same handle retained; read-only CLI commands or close-owned only.', flush=True) import shlex - for line in sys.stdin: - if line.strip() == "close-owned": - receipt["operator_disposition"] = ( - "Close exact owned handle; no mutation replay." - ) + if line.strip() == 'close-owned': + receipt['operator_disposition'] = 'Close exact owned handle; no mutation replay.' break observed = client.execute(shlex.split(line), timeout_seconds=10) - receipt.setdefault("same_handle_observations", []).append( - observed.payload - ) + receipt.setdefault('same_handle_observations', []).append(observed.payload) save() print(observed.rendered_output, flush=True) else: - raise RuntimeError("No disposition; retain original handles") from error + raise RuntimeError('No disposition; retain original handles') from error # Cleanup is a distinct authorized lifecycle request, never a source replay. if original_handle is not None: - closed = call( - agent_capabilities.close_owned_runtime.name, - dict( - handle=to_jsonable(original_handle), mode="force", timeout_ms=5000 - ), - RuntimeBootstrapCloseResult, - lifecycle=True, - ) + closed = call(agent_capabilities.close_owned_runtime.name, + dict(handle=to_jsonable(original_handle), mode='force', timeout_ms=5000), + RuntimeBootstrapCloseResult, lifecycle=True) assert closed.handle == original_handle assert closed.outcome.succeeded and closed.outcome.endpoint_terminated assert original_handle.process_identity.is_alive() is False assert not endpoint.occupied_ports(config) - receipt["close"] = to_jsonable(closed) + receipt['close'] = to_jsonable(closed) client.close() - receipt["client_closed"] = True - receipt["runtime_terminal"] = True - receipt["sockets_after"] = subprocess.check_output(["ss", "-ltnp"], text=True) - receipt["total_seconds"] = time.monotonic() - began + receipt['client_closed'] = True + receipt['runtime_terminal'] = True + receipt['sockets_after'] = subprocess.check_output(['ss', '-ltnp'], text=True) + receipt['total_seconds'] = time.monotonic()-began save() - print( - json.dumps( - dict( - accepted=receipt["accepted"], - runtime_terminal=True, - receipt=str(root / "journey.json"), - ) - ), - flush=True, - ) + print(json.dumps(dict(accepted=receipt['accepted'], runtime_terminal=True, + receipt=str(root/'journey.json'))), flush=True) -if __name__ == "__main__": +if __name__ == '__main__': parser = argparse.ArgumentParser() - parser.add_argument("--receipts", type=Path, required=True) - parser.add_argument("--expected-sha", required=True) - parser.add_argument("--port", type=int, default=5963) + parser.add_argument('--receipts', type=Path, required=True) + parser.add_argument('--expected-sha', required=True) + parser.add_argument('--port', type=int, default=5963) run(parser.parse_args()) diff --git a/tests/diagnostics/volume_projection_fixture.py b/tests/diagnostics/volume_projection_fixture.py index 1be8dbf54d..9a318a25a7 100644 --- a/tests/diagnostics/volume_projection_fixture.py +++ b/tests/diagnostics/volume_projection_fixture.py @@ -18,24 +18,15 @@ ObjectLabelsArtifactType, ObjectMeasurementSubjectRelation, ) -from openhcs.core.measurement_row_materialization import ( - DataclassMeasurementColumnarRows, -) +from openhcs.core.measurement_row_materialization import DataclassMeasurementColumnarRows from openhcs.core.memory import numpy from openhcs.core.pipeline.function_contracts import artifact_outputs from openhcs.core.projected_image_output import SelectedPlaneImageOutput -from openhcs.core.runtime_measurements import ( - RuntimeMeasurementFeature, - RuntimeMeasurementFeatureOwner, -) +from openhcs.core.runtime_measurements import RuntimeMeasurementFeature, RuntimeMeasurementFeatureOwner from openhcs.core.processing_contracts import ( Pure3DContract, ) -from openhcs.processing.materialization import ( - CsvOptions, - MaterializationSpec, - ROIOptions, -) +from openhcs.processing.materialization import CsvOptions, MaterializationSpec, ROIOptions class VolumeProjectionFixtureFeature(RuntimeMeasurementFeature): @@ -45,10 +36,7 @@ class VolumeProjectionFixtureFeature(RuntimeMeasurementFeature): class VolumeProjectionFixtureFeatureOwner(RuntimeMeasurementFeatureOwner): @classmethod def owns_measurement_feature_name(cls, feature_name: str) -> bool: - return any( - feature.feature_name == feature_name - for feature in VolumeProjectionFixtureFeature - ) + return any(feature.feature_name == feature_name for feature in VolumeProjectionFixtureFeature) @classmethod def owns_primary_measurement_feature_name(cls, feature_name: str) -> bool: @@ -63,26 +51,17 @@ class VolumeProjectionFixtureRow: SELECTED_VOLUME = MainFlowPlaneProjectionOutputSpec.output( - "selected_volume_fixture_v2", - ImageArtifactType, -) -VOLUME_IMAGE = MainFlowStackOutputSpec.output( - "volume_fixture_image_v2", ImageArtifactType + "selected_volume_fixture_v2", ImageArtifactType, ) +VOLUME_IMAGE = MainFlowStackOutputSpec.output("volume_fixture_image_v2", ImageArtifactType) VOLUME_LABELS = MainFlowStackOutputSpec.output( - "volume_fixture_labels_v2", - ObjectLabelsArtifactType, + "volume_fixture_labels_v2", ObjectLabelsArtifactType, materialization=MaterializationSpec(ROIOptions(min_area=0)), ) VOLUME_ROWS = MainFlowStackOutputSpec.output( - "volume_fixture_rows_v2", - MeasurementsArtifactType, + "volume_fixture_rows_v2", MeasurementsArtifactType, measurement_feature_owner=VolumeProjectionFixtureFeatureOwner, - relations=( - ObjectMeasurementSubjectRelation( - source=VOLUME_LABELS.ref(), id_field="object_label" - ), - ), + relations=(ObjectMeasurementSubjectRelation(source=VOLUME_LABELS.ref(), id_field="object_label"),), materialization=MaterializationSpec(CsvOptions()), ) @@ -115,9 +94,7 @@ def inspect_volume_fixture_v2( metadata = image.metadata labels = pixels.astype(np.int32, copy=True) rows = tuple( - VolumeProjectionFixtureRow( - index, int(label), int(np.count_nonzero(plane == label)) - ) + VolumeProjectionFixtureRow(index, int(label), int(np.count_nonzero(plane == label))) for index, plane in enumerate(labels) for label in np.unique(plane) if label != 0 diff --git a/tests/integration/test_agent_observation_export_zmq.py b/tests/integration/test_agent_observation_export_zmq.py index 0d2280ce6b..9fc672778f 100644 --- a/tests/integration/test_agent_observation_export_zmq.py +++ b/tests/integration/test_agent_observation_export_zmq.py @@ -113,21 +113,14 @@ def _start_run(built, plate: Path, pipeline, export_path: Path) -> tuple[str, in _call( built, "openhcs_set_dataset_pipeline", - { - "scope_id": scope_id, - "pipeline_source": PipelineDocumentCodec.render(pipeline), - }, - ) - _wait_for_row( - built, - _call(built, "openhcs_initialize_datasets", {"scope_ids": [scope_id]}), - lambda row: row["initialized"] and not row["init_pending"], - ) - _wait_for_row( - built, - _call(built, "openhcs_compile_datasets", {"scope_ids": [scope_id]}), - lambda row: row["compiled"] and not row["compile_pending"], - ) + {"scope_id": scope_id, "pipeline_source": PipelineDocumentCodec.render(pipeline)}, + ) + _wait_for_row(built, _call( + built, "openhcs_initialize_datasets", {"scope_ids": [scope_id]} + ), lambda row: row["initialized"] and not row["init_pending"]) + _wait_for_row(built, _call( + built, "openhcs_compile_datasets", {"scope_ids": [scope_id]} + ), lambda row: row["compiled"] and not row["compile_pending"]) started = _call( built, "openhcs_run_datasets", diff --git a/tests/integration/test_cellprofiler_generated_pipeline.py b/tests/integration/test_cellprofiler_generated_pipeline.py index f6f05b5cee..c24a36d5b4 100644 --- a/tests/integration/test_cellprofiler_generated_pipeline.py +++ b/tests/integration/test_cellprofiler_generated_pipeline.py @@ -184,9 +184,7 @@ def _execute_imported_cppipe_via_zmq( if partition_filter is None else LazyPartitionFilterConfig( partition_filter=( - partition_filter - if isinstance(partition_filter, int) - else list(partition_filter) + partition_filter if isinstance(partition_filter, int) else list(partition_filter) ) ) ), diff --git a/tests/integration/test_cellprofiler_official30_zmq.py b/tests/integration/test_cellprofiler_official30_zmq.py index 2b9d024417..7f07bf67a5 100644 --- a/tests/integration/test_cellprofiler_official30_zmq.py +++ b/tests/integration/test_cellprofiler_official30_zmq.py @@ -570,7 +570,8 @@ def test_official30_fiji_variants_project_registered_viewer_configs() -> None: for viewers in _OFFICIAL30_FIJI_VIEWER_VARIANTS: ports_by_viewer = { - viewer: 20_000 + index * 100 for index, viewer in enumerate(viewers) + viewer: 20_000 + index * 100 + for index, viewer in enumerate(viewers) } configs = _registered_streaming_config_kwargs( viewers, diff --git a/tests/integration/test_chained_callable_lineage_journey.py b/tests/integration/test_chained_callable_lineage_journey.py index 28bc1b2b9d..b57f976444 100644 --- a/tests/integration/test_chained_callable_lineage_journey.py +++ b/tests/integration/test_chained_callable_lineage_journey.py @@ -43,9 +43,7 @@ from polystore.roi import load_rois_from_zip # Reuse the public reference's authoritative code block, not a copied callable. -from tests.unit.agent.test_callable_artifact_reference import ( - reference_namespace, -) # noqa: F401 +from tests.unit.agent.test_callable_artifact_reference import reference_namespace # noqa: F401 from openhcs.domains.microscopy.axes import Microscopy from openhcs.core.dataset_sources.source_bindings_source import SourceBindingsSource @@ -63,9 +61,7 @@ def registered_reference_callable(reference_namespace): def test_chained_public_callable_uses_declared_main_flow_not_storage_argument( - tmp_path, - reference_namespace, - registered_reference_callable, + tmp_path, reference_namespace, registered_reference_callable, ): assert reference_namespace["inspect_label_fixture"] is registered_reference_callable plate = tmp_path / "plate" @@ -75,15 +71,13 @@ def test_chained_public_callable_uses_declared_main_flow_not_storage_argument( tifffile.imwrite(plate / "A01_s001_label-fixture.tif", fixture) source = NamedSourceBinding( alias="label-fixture", - selector=SourceSelector( - filters=( - SourceFilterClause( - subject=SourceFilterSubject.FILE, - match_type=SourceFilterMatchType.CONTAINS, - value="label-fixture", - ), - ) - ), + selector=SourceSelector(filters=( + SourceFilterClause( + subject=SourceFilterSubject.FILE, + match_type=SourceFilterMatchType.CONTAINS, + value="label-fixture", + ), + )), component_identity=(ComponentSelector(Microscopy.Channel, "1"),), ) document = PipelineDocumentCodec.from_values( @@ -105,29 +99,24 @@ def test_chained_public_callable_uses_declared_main_flow_not_storage_argument( for index in range(2) ], ) - document = PipelineDocumentCodec.from_source(PipelineDocumentCodec.render(document)) - bundle = ( - InProcessCompileInspectionGateway() - .compile( - CompileInspectionInput( - plate=plate, - pipeline_document=document, - axis_filter=("A01",), - global_pipeline_config=GlobalPipelineConfig( - num_workers=1, use_threading=True - ), - progress_queue=AgentProgressQueue(), - ) - ) - .execution_bundle + document = PipelineDocumentCodec.from_source( + PipelineDocumentCodec.render(document) ) + bundle = InProcessCompileInspectionGateway().compile( + CompileInspectionInput( + plate=plate, + pipeline_document=document, + axis_filter=("A01",), + global_pipeline_config=GlobalPipelineConfig(num_workers=1, use_threading=True), + progress_queue=AgentProgressQueue(), + ) + ).execution_bundle context = bundle.runtime_contexts["A01"] first, second = (context.step_plans[index] for index in range(2)) first_invocation = next(first.compiled_function_pattern.iter_invocations()) second_invocation = next(second.compiled_function_pattern.iter_invocations()) [first_image] = ( - plan - for plan in first_invocation.artifact_output_plans + plan for plan in first_invocation.artifact_output_plans if plan.artifact_type is ImageArtifactType ) incoming_ref = first_image.ref().for_plan_type(ArtifactInputPlan) @@ -143,8 +132,7 @@ def test_chained_public_callable_uses_declared_main_flow_not_storage_argument( assert edge.projection is None orchestrator = PipelineOrchestrator( - plate, - pipeline_config=document.pipeline_config, + plate, pipeline_config=document.pipeline_config, ).initialize() results = orchestrator.execute_compiled_bundle( execution_bundle=bundle, @@ -164,9 +152,7 @@ def test_chained_public_callable_uses_declared_main_flow_not_storage_argument( for plan in (image_plan, labels_plan, rows_plan): [record] = context.runtime_value_store.find_matching( RuntimeArtifactQuery.from_output_plan( - plan, - axis_id="A01", - backend="memory", + plan, axis_id="A01", backend="memory", group_key=plan.group_scope().select_runtime_key( invocation.key.group_key ), @@ -175,9 +161,7 @@ def test_chained_public_callable_uses_declared_main_flow_not_storage_argument( records.append(record) image, labels, rows = (record.data for record in records) np.testing.assert_array_equal(np.squeeze(image.data), fixture) - np.testing.assert_array_equal( - np.squeeze(object_label_dense_array(labels)), fixture - ) + np.testing.assert_array_equal(np.squeeze(object_label_dense_array(labels)), fixture) assert rows.subject.object_name == labels_plan.name assert rows.subject.id_field == "object_label" assert tuple(rows.rows.column_values("object_label")) == (1,) @@ -191,13 +175,9 @@ def test_chained_public_callable_uses_declared_main_flow_not_storage_argument( [persisted] = DictReader(stream) assert persisted["object_label"] == "1" assert persisted["pixel_count"] == "16" - [roi_path] = analysis.glob( - f"*_{labels_plan.name}_step{step_plan.step_index}*.zip" - ) + [roi_path] = analysis.glob(f"*_{labels_plan.name}_step{step_plan.step_index}*.zip") assert len(load_rois_from_zip(roi_path)) == 1 saved_images = tuple(step_plan.materialized_output.output_dir.glob("*.tif")) assert saved_images for image_path in saved_images: - np.testing.assert_array_equal( - np.squeeze(tifffile.imread(image_path)), fixture - ) + np.testing.assert_array_equal(np.squeeze(tifffile.imread(image_path)), fixture) diff --git a/tests/integration/test_custom_source_transport.py b/tests/integration/test_custom_source_transport.py index fdf36c42d5..3d6f005222 100644 --- a/tests/integration/test_custom_source_transport.py +++ b/tests/integration/test_custom_source_transport.py @@ -30,7 +30,6 @@ if dependency_root: sys.path.insert(0, str(WORKTREE)) from openhcs._source_dependencies import ensure_source_checkout_external_paths - ensure_source_checkout_external_paths(Path(dependency_root)) @@ -85,9 +84,9 @@ def _run_child(env, *arguments): while True: remaining = deadline - time.monotonic() assert remaining > 0, f"Child exceeded 30s: {arguments!r}" - assert ( - os.fstat(output_file.fileno()).st_size < OUTPUT_LIMIT - ), f"Child output exceeded {OUTPUT_LIMIT} bytes: {arguments!r}" + assert os.fstat(output_file.fileno()).st_size < OUTPUT_LIMIT, ( + f"Child output exceeded {OUTPUT_LIMIT} bytes: {arguments!r}" + ) try: returncode = process.wait(timeout=min(remaining, 0.1)) except subprocess.TimeoutExpired: @@ -95,9 +94,9 @@ def _run_child(env, *arguments): break output_file.seek(0) captured.extend(output_file.read(OUTPUT_LIMIT)) - assert ( - len(captured) < OUTPUT_LIMIT - ), f"Child output exceeded {OUTPUT_LIMIT} bytes: {arguments!r}" + assert len(captured) < OUTPUT_LIMIT, ( + f"Child output exceeded {OUTPUT_LIMIT} bytes: {arguments!r}" + ) finally: if process.poll() is None: process.kill() @@ -418,13 +417,8 @@ def _spawned_custom_task(payload, execution_id, subject_id): owner = reference.metadata.artifact_outputs[0].measurement_feature_owner results.append(owner.make_row(2.5)) ProgressExecutionContext(execution_id, subject_id).emit( - axis_id="synthetic", - step_name="custom", - phase=ProgressPhase.STEP_COMPLETED, - status=ProgressStatus.SUCCESS, - percent=100, - completed=1, - total=1, + axis_id="synthetic", step_name="custom", phase=ProgressPhase.STEP_COMPLETED, + status=ProgressStatus.SUCCESS, percent=100, completed=1, total=1, ) return os.getpid(), tuple(results) @@ -454,8 +448,7 @@ def _compiled_custom_contexts(payload, runtime): from openhcs.core.compiled_step_plan import CompiledStepPlan from openhcs.core.context.processing_context import ProcessingContext from openhcs.core.function_patterns import ( - compile_function_pattern, - normalize_function_pattern, + compile_function_pattern, normalize_function_pattern, ) from openhcs.core.function_step_transport import FunctionStepTransportAuthority @@ -471,14 +464,11 @@ def _compiled_custom_contexts(payload, runtime): assert callable(invocation.contract.raw_processing_function) prepared = invocation.contract.resolve_runtime_callable() plan = CompiledStepPlan( - step_index=0, - step_name="Synthetic custom", - axis_id=axis_id, - compiled_function_pattern=pattern, + step_index=0, step_name="Synthetic custom", + axis_id=axis_id, compiled_function_pattern=pattern, ) context = ProcessingContext( - axis_id=axis_id, - step_plans={0: plan}, + axis_id=axis_id, step_plans={0: plan}, filemanager=FileManager({Backend.MEMORY.value: MemoryStorageBackend()}), ) context.freeze() @@ -490,15 +480,11 @@ def _compiled_custom_contexts(payload, runtime): assert transported.contract is not invocation.contract assert invocation.contract.resolve_runtime_callable() is prepared assert callable(invocation.contract.raw_processing_function) - assert ( - FunctionStepTransportAuthority.normalize_callable_contract( - transported.contract - ) - is transported.contract - ) + assert FunctionStepTransportAuthority.normalize_callable_contract( + transported.contract + ) is transported.contract return CompiledExecutionBundle.from_runtime_contexts( - pipeline_definition=(), - runtime_contexts=contexts, + pipeline_definition=(), runtime_contexts=contexts, worker_assignments={f"worker_{i}": [axis] for i, axis in enumerate(contexts)}, runtime_environment=runtime, ).transport_contexts @@ -507,8 +493,7 @@ def _compiled_custom_contexts(payload, runtime): def _spawn(path): import multiprocessing from openhcs.core.compiled_execution import ( - CompiledRuntimeEnvironmentPlan, - CompiledWorkerStartPlan, + CompiledRuntimeEnvironmentPlan, CompiledWorkerStartPlan, ) from openhcs.core.config import MultiprocessingStartMethod from openhcs.core.orchestrator.cancellation import ExecutionCancellationSignal @@ -526,25 +511,19 @@ class RichProgressContext(ProgressExecutionContext): payload = _load(path) method = MultiprocessingStartMethod.SPAWN runtime = CompiledRuntimeEnvironmentPlan( - worker_start=CompiledWorkerStartPlan( - method, method, "source fixture", False, False - ), - use_threading=False, - configured_num_workers=2, + worker_start=CompiledWorkerStartPlan(method, method, "source fixture", False, False), + use_threading=False, configured_num_workers=2, ) queue = multiprocessing.get_context(method.value).Queue() resources = WorkerExecutorFactory( - log_file_base=None, - progress_queue=queue, + log_file_base=None, progress_queue=queue, cancellation=ExecutionCancellationSignal(), ).create(runtime_environment=runtime, actual_max_workers=2) contexts = _compiled_custom_contexts(payload, runtime) try: with resources.execution_context(): tasks = tuple( - resources.executor.submit( - _spawned_compiled_custom_task, lane_context, helpers - ) + resources.executor.submit(_spawned_compiled_custom_task, lane_context, helpers) for lane_context, (_reference, helpers) in zip( contexts.values(), payload, strict=True ) @@ -558,10 +537,7 @@ class RichProgressContext(ProgressExecutionContext): assert values == [[0, 1, 2], [3, 4, 5]] assert type(row) is helpers[4] and row == helpers[5] pid, rows = resources.executor.submit( - _spawned_custom_task, - payload, - context.execution_id, - context.subject_id, + _spawned_custom_task, payload, context.execution_id, context.subject_id, ).result(timeout=15) assert pid != os.getpid() for row, (_reference, helpers) in zip(rows, payload, strict=True): @@ -647,20 +623,10 @@ def test_persisted_payload_preserves_nominal_helpers_in_independent_consumer( def test_persisted_custom_revision_executes_in_original_spawn_factory( - tmp_path, - transport_environment, + tmp_path, transport_environment, ): - print( - _run_child( - transport_environment, - "spawn", - str(tmp_path / "spawn.pkl"), - "reference", - "cold", - "unchanged", - ), - end="", - ) + print(_run_child(transport_environment, "spawn", str(tmp_path / "spawn.pkl"), + "reference", "cold", "unchanged"), end="") @pytest.mark.parametrize("mutation", ["changed", "deleted", "renamed"]) diff --git a/tests/integration/test_image_math_publication_620.py b/tests/integration/test_image_math_publication_620.py index 21b86f0924..b849e8e233 100644 --- a/tests/integration/test_image_math_publication_620.py +++ b/tests/integration/test_image_math_publication_620.py @@ -1,5 +1,4 @@ """Default automatic/named publication through actual registered ImageMath.""" - from openhcs.core.progress import ProgressExecutionContext from dataclasses import replace from hashlib import sha256 @@ -12,14 +11,10 @@ import tifffile from openhcs.agent.services.artifact_plan_inspection_service import ( - AgentProgressQueue, - CompileInspectionInput, - InProcessCompileInspectionGateway, + AgentProgressQueue, CompileInspectionInput, InProcessCompileInspectionGateway, ) from openhcs.core.config import ( - GlobalPipelineConfig, - LazyNapariStreamingConfig, - LazyPathPlanningConfig, + GlobalPipelineConfig, LazyNapariStreamingConfig, LazyPathPlanningConfig, LazyProcessingConfig, ) from openhcs.core.orchestrator.execution_result import RuntimeObservationMode @@ -29,8 +24,7 @@ from openhcs.core.source_metadata import SourceVoxelSpacing from openhcs.core.source_workspace_projection import VirtualWorkspaceSourceProjection from openhcs.core.virtual_workspace_metadata import ( - OpenHCSMetadataSubdirectories, - VirtualWorkspaceSourceProjectionEntries, + OpenHCSMetadataSubdirectories, VirtualWorkspaceSourceProjectionEntries, ) from openhcs.interop.cellprofiler.pipeline_import import import_cellprofiler_pipeline from openhcs.domains.microscopy.axes import Microscopy @@ -38,132 +32,70 @@ @pytest.mark.parametrize("rgb", (False, True)) -def test_registered_mixed_image_math_publishes_once_in_both_operand_orders( - tmp_path, rgb -): +def test_registered_mixed_image_math_publishes_once_in_both_operand_orders(tmp_path, rgb): scalar = np.arange(180, dtype=np.uint8).reshape(12, 15) color = ( np.stack((scalar, np.full_like(scalar, 32), np.full_like(scalar, 240)), axis=-1) - if rgb - else 255 - scalar + if rgb else 255 - scalar ) - source = tmp_path / "input" + source = tmp_path / 'input' source.mkdir() - for name, channel, pixels in (("Gray", 1, scalar), ("Color", 2, color)): - tifffile.imwrite(source / f"A01_s1_w{channel}_z1_t1_{name}.tif", pixels) - originals = { - image: sha256(image.read_bytes()).hexdigest() for image in source.glob("*.tif") - } - fixture = ( - Path(__file__).resolve().parents[1] - / "fixtures/pipelines/image_math_mixed_publication.cppipe" - ) + for name, channel, pixels in (('Gray', 1, scalar), ('Color', 2, color)): + tifffile.imwrite(source / f'A01_s1_w{channel}_z1_t1_{name}.tif', pixels) + originals = {image: sha256(image.read_bytes()).hexdigest() for image in source.glob('*.tif')} + fixture = Path(__file__).resolve().parents[1] / 'fixtures/pipelines/image_math_mixed_publication.cppipe' steps, imported = import_cellprofiler_pipeline(fixture) config = replace( - imported, - dataset_source=SourceBindingsSource, + imported, dataset_source=SourceBindingsSource, processing_config=LazyProcessingConfig( - variable_components=[Microscopy.ZIndex], group_by=Microscopy.Channel - ), - path_planning_config=LazyPathPlanningConfig( - global_output_folder=tmp_path / "results" - ), - source_bindings_config=replace( - imported.source_bindings_config, - metadata_rules=( - MetadataExtractionRule( - source=MetadataSource.FILE_NAME, - pattern=r"(?P[A-H]\d{2})_s(?P\d+)_w(?P\d+)_z(?P\d+)_t(?P\d+)_(?:Gray|Color)\.tif", - ), - ), - source_stack_components=(), - source_voxel_spacing=SourceVoxelSpacing((0.5, 0.5)), - ), + variable_components=[Microscopy.ZIndex], group_by=Microscopy.Channel), + path_planning_config=LazyPathPlanningConfig(global_output_folder=tmp_path / 'results'), + source_bindings_config=replace(imported.source_bindings_config, + metadata_rules=(MetadataExtractionRule(source=MetadataSource.FILE_NAME, + pattern=r'(?P[A-H]\d{2})_s(?P\d+)_w(?P\d+)_z(?P\d+)_t(?P\d+)_(?:Gray|Color)\.tif'),), + source_stack_components=(), source_voxel_spacing=SourceVoxelSpacing((0.5, 0.5))), napari_streaming_config=LazyNapariStreamingConfig(enabled=False), ) - document = PipelineDocumentCodec.from_values( - pipeline_config=config, pipeline_steps=steps - ) - bundle = ( - InProcessCompileInspectionGateway() - .compile( - CompileInspectionInput( - plate=source, - pipeline_document=document, - axis_filter=("A01",), - global_pipeline_config=GlobalPipelineConfig( - num_workers=1, use_threading=True - ), - progress_queue=AgentProgressQueue(), - ) - ) - .execution_bundle - ) - outcomes = ( - PipelineOrchestrator(source, pipeline_config=config) - .initialize() - .execute_compiled_bundle( - execution_bundle=bundle, - max_workers=1, - runtime_observation_mode=RuntimeObservationMode.MERGE_INTO_PARENT, - progress_queue=AgentProgressQueue(), - progress_context=ProgressExecutionContext( - execution_id="engineering620-registered", - subject_id=str(source), - ), - ) + document = PipelineDocumentCodec.from_values(pipeline_config=config, pipeline_steps=steps) + bundle = InProcessCompileInspectionGateway().compile(CompileInspectionInput( + plate=source, pipeline_document=document, axis_filter=('A01',), + global_pipeline_config=GlobalPipelineConfig(num_workers=1, use_threading=True), + progress_queue=AgentProgressQueue(), + )).execution_bundle + outcomes = PipelineOrchestrator(source, pipeline_config=config).initialize().execute_compiled_bundle( + execution_bundle=bundle, max_workers=1, + runtime_observation_mode=RuntimeObservationMode.MERGE_INTO_PARENT, + progress_queue=AgentProgressQueue(), + progress_context=ProgressExecutionContext( + execution_id='engineering620-registered', + subject_id=str(source), + ), ) - assert outcomes["A01"].is_success(), outcomes["A01"].error_message + assert outcomes['A01'].is_success(), outcomes['A01'].error_message gray_expected = scalar.astype(np.float32) / 255 - color_expected = ( - rgb2gray(color.astype(np.float32) / 255) - if rgb - else color.astype(np.float32) / 255 - ) - expected = { - "GrayIndependent": gray_expected, - "ColorIndependent": color_expected, - "Forward": (gray_expected + color_expected) / 2, - "Reverse": (gray_expected + color_expected) / 2, - } - plate = tmp_path / "results/input_openhcs" - saved = tuple((plate / "images").glob("*.tif")) + color_expected = rgb2gray(color.astype(np.float32) / 255) if rgb else color.astype(np.float32) / 255 + expected = {'GrayIndependent': gray_expected, 'ColorIndependent': color_expected, + 'Forward': (gray_expected + color_expected) / 2, + 'Reverse': (gray_expected + color_expected) / 2} + plate = tmp_path / 'results/input_openhcs' + saved = tuple((plate / 'images').glob('*.tif')) assert len(saved) == 4 - persisted = json.loads((plate / "openhcs_metadata.json").read_text()) - projection = VirtualWorkspaceSourceProjection.from_openhcs_metadata( - plate, persisted - ) + persisted = json.loads((plate / 'openhcs_metadata.json').read_text()) + projection = VirtualWorkspaceSourceProjection.from_openhcs_metadata(plate, persisted) # The lookup intentionally indexes relative and absolute aliases; count the # original persisted declarations, not the derived lookup's keys. - declared = tuple( - entry - for directory in OpenHCSMetadataSubdirectories(persisted).values() - for entry in VirtualWorkspaceSourceProjectionEntries.from_subdirectory( - directory - ).entries.values() - ) + declared = tuple(entry for directory in OpenHCSMetadataSubdirectories(persisted).values() + for entry in VirtualWorkspaceSourceProjectionEntries.from_subdirectory(directory).entries.values()) assert len(declared) == 4 for name, pixels in expected.items(): - (image,) = ( - image for image in saved if image.name.endswith("_" + name + ".tif") - ) + image, = (image for image in saved if image.name.endswith('_' + name + '.tif')) np.testing.assert_array_equal(tifffile.imread(image), pixels) relative = str(image.relative_to(plate)) - metadata = projection.source_projections_by_virtual_path[ - relative - ].persisted_image_metadata() + metadata = projection.source_projections_by_virtual_path[relative].persisted_image_metadata() assert metadata.source_voxel_spacing == SourceVoxelSpacing((0.5, 0.5)) - assert metadata.source_component_metadata["well"] == "A01" - assert metadata.source_component_metadata["channel"] == ( - "2" if name in ("ColorIndependent", "Reverse") else "1" - ) + assert metadata.source_component_metadata['well'] == 'A01' + assert metadata.source_component_metadata['channel'] == ('2' if name in ('ColorIndependent', 'Reverse') else '1') assert metadata.source_image_names == (name,) for image, digest in originals.items(): assert sha256(image.read_bytes()).hexdigest() == digest - print( - "REGISTERED_DEFAULT_PUBLICATION", - rgb, - len(saved), - "exact elements", - sum(pixels.size for pixels in expected.values()), - ) + print('REGISTERED_DEFAULT_PUBLICATION', rgb, len(saved), 'exact elements', sum(pixels.size for pixels in expected.values())) diff --git a/tests/integration/test_measured_cli_session.py b/tests/integration/test_measured_cli_session.py index 230ff057b5..b81290c951 100644 --- a/tests/integration/test_measured_cli_session.py +++ b/tests/integration/test_measured_cli_session.py @@ -213,13 +213,7 @@ def test_read_only_source_plate_completes_and_stays_byte_identical( port = _port(2) try: completed = subprocess.run( - _command( - plate, - source, - output_dir, - port, - wait_ms=int(RUN_DEADLINE_SECONDS * 1000), - ), + _command(plate, source, output_dir, port, wait_ms=int(RUN_DEADLINE_SECONDS * 1000)), cwd=REPOSITORY_ROOT, env=_environment(tmp_path), capture_output=True, diff --git a/tests/integration/test_measurement_declaration_journey.py b/tests/integration/test_measurement_declaration_journey.py index 91f4458067..0a1538757d 100644 --- a/tests/integration/test_measurement_declaration_journey.py +++ b/tests/integration/test_measurement_declaration_journey.py @@ -40,9 +40,7 @@ ) from openhcs.core.function_step_document import FunctionStepDocumentCodec from openhcs.core.measurement_feature_queries import measurement_values_for_feature -from openhcs.core.measurement_row_materialization import ( - DataclassMeasurementColumnarRows, -) +from openhcs.core.measurement_row_materialization import DataclassMeasurementColumnarRows from openhcs.core.memory import numpy from openhcs.core.orchestrator.execution_result import RuntimeObservationMode from openhcs.core.orchestrator.orchestrator import PipelineOrchestrator @@ -123,8 +121,7 @@ def owns_primary_measurement_feature_name(cls, feature_name): COUNT_IMAGE = MainFlowStackOutputSpec.output("CountedImage", ImageArtifactType) MISSING_SUBJECT_ROWS = MainFlowStackOutputSpec.output( - "PixelCounts", - MeasurementsArtifactType, + "PixelCounts", MeasurementsArtifactType, materialization=MaterializationSpec(CsvOptions()), measurement_feature_owner=CountFeatureOwner, ) @@ -219,14 +216,8 @@ def _document(*, selected=True, two_producers=True, same_source=False): identify_primary_objects, "Primary", { - _binding_name( - IdentifyPrimaryObjectsModule, ArtifactInputPlan, ImageArtifactType - ): "DNA", - _binding_name( - IdentifyPrimaryObjectsModule, - ArtifactOutputPlan, - ObjectLabelsArtifactType, - ): "Nuclei", + _binding_name(IdentifyPrimaryObjectsModule, ArtifactInputPlan, ImageArtifactType): "DNA", + _binding_name(IdentifyPrimaryObjectsModule, ArtifactOutputPlan, ObjectLabelsArtifactType): "Nuclei", "exclude_size": False, "exclude_border_objects": False, "unclump_method": UnclumpMethod.NONE, @@ -240,23 +231,15 @@ def _document(*, selected=True, two_producers=True, same_source=False): identify_secondary_objects, "Secondary", { - _binding_name( - IdentifySecondaryObjectsModule, ArtifactInputPlan, ImageArtifactType - ): measurement_image, - _binding_name( - IdentifySecondaryObjectsModule, - ArtifactInputPlan, - ObjectLabelsArtifactType, - ): "Nuclei", + _binding_name(IdentifySecondaryObjectsModule, ArtifactInputPlan, ImageArtifactType): measurement_image, + _binding_name(IdentifySecondaryObjectsModule, ArtifactInputPlan, ObjectLabelsArtifactType): "Nuclei", IdentifySecondaryObjectsModule.secondary_output_binding.require_parameter_name(): "Cells", "method": SecondaryMethod.DISTANCE_N, "distance_to_dilate": 2, }, ) kwargs = { - MeasureObjectIntensityModule.image_measurement_binding.require_parameter_name(): ( - measurement_image, - ), + MeasureObjectIntensityModule.image_measurement_binding.require_parameter_name(): (measurement_image,), } if selected: kwargs[ @@ -267,19 +250,12 @@ def _document(*, selected=True, two_producers=True, same_source=False): pipeline_config=PipelineConfig( dataset_source=SourceBindingsSource, source_bindings_config=LazySourceBindingsConfig( - bindings=( - (_source("DNA", "1"),) - if same_source - else ( - _source("DNA", "1"), - _source("Actin", "2"), - ) + bindings=(_source("DNA", "1"),) if same_source else ( + _source("DNA", "1"), _source("Actin", "2"), ), ), ), - pipeline_steps=( - [primary, secondary, measure] if two_producers else [primary, measure] - ), + pipeline_steps=[primary, secondary, measure] if two_producers else [primary, measure], ) @@ -292,19 +268,15 @@ def _write_plate(path, *, same_source=False): def _compile(path, document, global_config): - return ( - InProcessCompileInspectionGateway() - .compile( - CompileInspectionInput( - plate=path, - pipeline_document=document, - axis_filter=("A01",), - global_pipeline_config=global_config, - progress_queue=AgentProgressQueue(), - ) + return InProcessCompileInspectionGateway().compile( + CompileInspectionInput( + plate=path, + pipeline_document=document, + axis_filter=("A01",), + global_pipeline_config=global_config, + progress_queue=AgentProgressQueue(), ) - .execution_bundle - ) + ).execution_bundle def _execute(path, document, bundle): @@ -323,30 +295,16 @@ def _execute(path, document, bundle): ) -@pytest.mark.parametrize( - "image_names,current_scale", - [ - (("DNA",), 1), - (("DNA", "Actin"), 1), - (("Actin", "DNA"), 1), - (("DNAHalf",), 1), - (("DNA", "DNAHalf"), 1), - (("DNAHalf", "DNA"), 1), - (("DNA", "Actin"), 0.5), - ], -) +@pytest.mark.parametrize("image_names,current_scale", [ + (("DNA",), 1), (("DNA", "Actin"), 1), (("Actin", "DNA"), 1), + (("DNAHalf",), 1), (("DNA", "DNAHalf"), 1), (("DNAHalf", "DNA"), 1), + (("DNA", "Actin"), 0.5), +]) def test_photometry_carrier_preserves_raw_aliases_and_produced_pixels( - tmp_path, - image_names, - current_scale, - registered_current_transform, + tmp_path, image_names, current_scale, registered_current_transform, ): """A stored label cohort must not substitute label IDs for measured images.""" - from openhcs.processing.backends.cellprofiler.intensity import ( - rescale_intensity, - RescaleMethod, - ) - + from openhcs.processing.backends.cellprofiler.intensity import rescale_intensity, RescaleMethod dna = np.zeros((16, 16), dtype=np.float32) dna[6:8, 6:8] = ((0.6, 0.7), (0.8, 0.9)) actin = (0.1 + np.arange(256).reshape(16, 16) * 0.001).astype(np.float32) @@ -354,111 +312,65 @@ def test_photometry_carrier_preserves_raw_aliases_and_produced_pixels( tifffile.imwrite(tmp_path / "A01_s001_w2_z001_t001.tif", actin) document = _document() if "DNAHalf" in image_names: - document.pipeline_steps.insert( - 0, - _step( - rescale_intensity, - "Produced half intensity", - { - "select_the_input_image": "DNA", - "name_the_output_image": "DNAHalf", - "rescale_method": RescaleMethod.DIVIDE_BY_VALUE, - "divisor_value": 2.0, - }, - ), - ) - document.pipeline_steps[-1] = _step( - measure_object_intensity, - "Actual photometry", - { - "select_images_to_measure": image_names, - "select_object_sets_to_measure": ("Cells",), - }, - ) + document.pipeline_steps.insert(0, _step(rescale_intensity, "Produced half intensity", { + "select_the_input_image": "DNA", "name_the_output_image": "DNAHalf", + "rescale_method": RescaleMethod.DIVIDE_BY_VALUE, "divisor_value": 2.0, + })) + document.pipeline_steps[-1] = _step(measure_object_intensity, "Actual photometry", { + "select_images_to_measure": image_names, "select_object_sets_to_measure": ("Cells",), + }) if current_scale != 1: - document.pipeline_steps.insert( - -1, - _step( - registered_current_transform, - "Transform current image carrier", - {}, - ), - ) + document.pipeline_steps.insert(-1, _step( + registered_current_transform, "Transform current image carrier", {}, + )) measurement = document.pipeline_steps[-1] measurement.processing_config = replace( - measurement.processing_config, - input_source=InputSource.PREVIOUS_STEP, + measurement.processing_config, input_source=InputSource.PREVIOUS_STEP, ) document = PipelineDocumentCodec.from_source(PipelineDocumentCodec.render(document)) - bundle = _compile( - tmp_path, document, GlobalPipelineConfig(num_workers=1, use_threading=True) - ) + bundle = _compile(tmp_path, document, GlobalPipelineConfig(num_workers=1, use_threading=True)) context = bundle.runtime_contexts["A01"] - plan = context.step_plans[len(document.pipeline_steps) - 1] + plan = context.step_plans[len(document.pipeline_steps)-1] group = plan.compiled_function_pattern.default_group selected = plan.stored_primary_input_edges_for_group(group, None) if any(name in ("DNA", "Actin") for name in image_names): - assert ( - selected is None - ) # Raw STEP_INPUT aliases need the current image carrier. + assert selected is None # Raw STEP_INPUT aliases need the current image carrier. results = _execute(tmp_path, document, bundle) - assert results["A01"].is_success(), results["A01"].error_message + assert results['A01'].is_success(), results['A01'].error_message (labels,) = context.runtime_value_store.find(name="Cells", axis_id="A01") foreground = object_label_dense_array(labels.data).squeeze() == 1 (invocation,) = group.invocations (output,) = invocation.artifact_output_plans (measurement,) = context.runtime_value_store.find( - name=output.name, - artifact_type=MeasurementsArtifactType, - axis_id="A01", + name=output.name, artifact_type=MeasurementsArtifactType, axis_id="A01", ) expected_images = { - "DNA": dna * current_scale, - "Actin": actin * current_scale, - "DNAHalf": dna / 2, + "DNA": dna * current_scale, "Actin": actin * current_scale, "DNAHalf": dna / 2, } for image_name in image_names: values = expected_images[image_name][foreground] assert values.size and values.std() > 0 - for feature, expected in ( - ("MeanIntensity", values.mean()), - ("MinIntensity", values.min()), - ("MaxIntensity", values.max()), - ("StdIntensity", values.std()), - ): + for feature, expected in (("MeanIntensity", values.mean()), ("MinIntensity", values.min()), + ("MaxIntensity", values.max()), ("StdIntensity", values.std())): actual = measurement_values_for_feature( - (measurement.data,), - f"Intensity_{feature}_{image_name}", - object_count=1, - object_name="Cells", - dialect=CELLPROFILER_MEASUREMENT_DIALECT, + (measurement.data,), f"Intensity_{feature}_{image_name}", + object_count=1, object_name="Cells", dialect=CELLPROFILER_MEASUREMENT_DIALECT, ) assert tuple(actual) == pytest.approx((expected,), abs=1e-7) def test_label_only_measurement_keeps_stored_cohort(tmp_path): from openhcs.processing.backends.cellprofiler.shape import ( - MeasureObjectSizeShapeModule, - measure_object_size_shape, + MeasureObjectSizeShapeModule, measure_object_size_shape, ) - _write_plate(tmp_path) document = _document() - document.pipeline_steps[-1] = _step( - measure_object_size_shape, - "Label-only area", - { - MeasureObjectSizeShapeModule.object_measurement_binding.require_parameter_name(): ( - "Cells", - ), - "calculate_advanced": False, - "calculate_zernikes": False, - }, - ) + document.pipeline_steps[-1] = _step(measure_object_size_shape, "Label-only area", { + MeasureObjectSizeShapeModule.object_measurement_binding.require_parameter_name(): ("Cells",), + "calculate_advanced": False, "calculate_zernikes": False, + }) document = PipelineDocumentCodec.from_source(PipelineDocumentCodec.render(document)) - bundle = _compile( - tmp_path, document, GlobalPipelineConfig(num_workers=1, use_threading=True) - ) + bundle = _compile(tmp_path, document, GlobalPipelineConfig(num_workers=1, use_threading=True)) context = bundle.runtime_contexts["A01"] plan = context.step_plans[len(document.pipeline_steps) - 1] group = plan.compiled_function_pattern.default_group @@ -471,31 +383,20 @@ def test_label_only_measurement_keeps_stored_cohort(tmp_path): (invocation,) = group.invocations (output,) = invocation.artifact_output_plans (measurement,) = context.runtime_value_store.find( - name=output.name, - artifact_type=MeasurementsArtifactType, - axis_id="A01", + name=output.name, artifact_type=MeasurementsArtifactType, axis_id="A01", ) actual = measurement_values_for_feature( - (measurement.data,), - "AreaShape_Area", - object_count=1, - object_name="Cells", + (measurement.data,), "AreaShape_Area", object_count=1, object_name="Cells", dialect=CELLPROFILER_MEASUREMENT_DIALECT, ) assert tuple(actual) == (np.count_nonzero(object_label_dense_array(labels.data)),) -@pytest.mark.parametrize( - "same_source", [True, False], ids=["single-source", "paired-channels"] -) -def test_exact_secondary_selector_survives_authoring_compile_and_execution( - tmp_path, same_source -): +@pytest.mark.parametrize("same_source", [True, False], ids=["single-source", "paired-channels"]) +def test_exact_secondary_selector_survives_authoring_compile_and_execution(tmp_path, same_source): _write_plate(tmp_path, same_source=same_source) document = _document(same_source=same_source) - selector = ( - MeasureObjectIntensityModule.object_measurement_binding.require_parameter_name() - ) + selector = MeasureObjectIntensityModule.object_measurement_binding.require_parameter_name() source = PipelineDocumentCodec.render(document) reconstructed = PipelineDocumentCodec.from_source(source) measurement = reconstructed.pipeline_steps[-1] @@ -522,16 +423,14 @@ def test_exact_secondary_selector_survives_authoring_compile_and_execution( plan = context.step_plans[2] invocation = next(plan.compiled_function_pattern.iter_invocations()) (edge,) = ( - edge - for edge in invocation.artifact_input_edges + edge for edge in invocation.artifact_input_edges if edge.spec.artifact_type is ObjectLabelsArtifactType ) assert edge.spec.name == "Cells" assert edge.spec.parameter_name == "labels" assert edge.storage_plan.source_step_id == 1 producer_output = next( - output - for output in secondary_invocation.artifact_output_plans + output for output in secondary_invocation.artifact_output_plans if output.ref() == edge.spec.ref().for_plan_type(ArtifactOutputPlan) ) assert edge.storage_plan.relations == producer_output.relations @@ -551,9 +450,7 @@ def test_exact_secondary_selector_survives_authoring_compile_and_execution( secondary_area = np.count_nonzero(object_label_dense_array(secondary.data)) assert secondary_area > primary_area > 0 [output] = invocation.artifact_output_plans - [measurement] = store.find( - name=output.name, artifact_type=MeasurementsArtifactType, axis_id="A01" - ) + [measurement] = store.find(name=output.name, artifact_type=MeasurementsArtifactType, axis_id="A01") assert measurement.data.subject.object_name == "Cells" image_name = "DNA" if same_source else "Actin" values = measurement_values_for_feature( @@ -565,9 +462,7 @@ def test_exact_secondary_selector_survives_authoring_compile_and_execution( ) # The DNA seed is four bright pixels; the grown cell adds dim background. # Measuring primary labels instead would return exactly 4.0. - expected = ( - 4.0 + 0.25 * (secondary_area - 4) if same_source else float(secondary_area) - ) + expected = 4.0 + 0.25 * (secondary_area - 4) if same_source else float(secondary_area) assert tuple(values) == (expected,) @@ -583,8 +478,7 @@ def test_omitted_secondary_selector_still_fails_closed(tmp_path): @pytest.mark.parametrize("producer_group_by", [Ungrouped, Microscopy.Channel]) def test_explicit_measurement_rosters_preserve_compiled_source_groups( - tmp_path, - producer_group_by, + tmp_path, producer_group_by, ): from openhcs.core.steps.function_execution import FunctionStepExecutor @@ -592,21 +486,14 @@ def test_explicit_measurement_rosters_preserve_compiled_source_groups( original = _document() for producer in original.pipeline_steps[:-1]: producer.processing_config = replace( - producer.processing_config, - group_by=producer_group_by, + producer.processing_config, group_by=producer_group_by, ) original.pipeline_steps[-1] = _step( measure_object_intensity, "Measure both source images and object sets", { - MeasureObjectIntensityModule.image_measurement_binding.require_parameter_name(): ( - "DNA", - "Actin", - ), - MeasureObjectIntensityModule.object_measurement_binding.require_parameter_name(): ( - "Nuclei", - "Cells", - ), + MeasureObjectIntensityModule.image_measurement_binding.require_parameter_name(): ("DNA", "Actin"), + MeasureObjectIntensityModule.object_measurement_binding.require_parameter_name(): ("Nuclei", "Cells"), }, ) document = PipelineDocumentCodec.from_values( @@ -614,17 +501,13 @@ def test_explicit_measurement_rosters_preserve_compiled_source_groups( original.pipeline_config, source_bindings_config=LazySourceBindingsConfig( bindings=(_source("DNA", "1"), _source("Actin", "2")), - match_plan=SourceBindingMatchPlan( - method=SourceBindingMatchMethod.ORDER - ), + match_plan=SourceBindingMatchPlan(method=SourceBindingMatchMethod.ORDER), ), ), pipeline_steps=original.pipeline_steps, ) document = PipelineDocumentCodec.from_source(PipelineDocumentCodec.render(document)) - bundle = _compile( - tmp_path, document, GlobalPipelineConfig(num_workers=1, use_threading=True) - ) + bundle = _compile(tmp_path, document, GlobalPipelineConfig(num_workers=1, use_threading=True)) context = bundle.runtime_contexts["A01"] executor = FunctionStepExecutor(context, 2) prepared = executor._prepare_groups(executor._detect_patterns()) @@ -634,23 +517,16 @@ def test_explicit_measurement_rosters_preserve_compiled_source_groups( assert all(len(patterns) == 1 for patterns in prepared.values()) invocation = next(executor.plan.compiled_function_pattern.iter_invocations()) assert tuple( - spec.name - for spec in invocation.contract.artifact_inputs.of_artifact_type( - ImageArtifactType - ) + spec.name for spec in invocation.contract.artifact_inputs.of_artifact_type(ImageArtifactType) ) == ("DNA", "Actin") results = _execute(tmp_path, document, bundle) assert results["A01"].is_success(), results["A01"].error_message (output,) = invocation.artifact_output_plans measurements = context.runtime_value_store.find( - name=output.name, - artifact_type=MeasurementsArtifactType, - axis_id="A01", + name=output.name, artifact_type=MeasurementsArtifactType, axis_id="A01", ) assert len(measurements) == len(expected_keys) - assert ( - tuple(record.key.scope.value_text for record in measurements) == expected_keys - ) + assert tuple(record.key.scope.value_text for record in measurements) == expected_keys for object_name in ("Nuclei", "Cells"): (labels,) = context.runtime_value_store.find(name=object_name, axis_id="A01") area = np.count_nonzero(object_label_dense_array(labels.data)) @@ -680,13 +556,10 @@ def test_omitted_selector_remains_valid_for_one_label_producer(tmp_path): GlobalPipelineConfig(num_workers=1, use_threading=True), ) invocation = next( - bundle.runtime_contexts["A01"] - .step_plans[1] - .compiled_function_pattern.iter_invocations() + bundle.runtime_contexts["A01"].step_plans[1].compiled_function_pattern.iter_invocations() ) (edge,) = ( - edge - for edge in invocation.artifact_input_edges + edge for edge in invocation.artifact_input_edges if edge.spec.artifact_type is ObjectLabelsArtifactType ) assert edge.spec.name == "Nuclei" @@ -694,9 +567,7 @@ def test_omitted_selector_remains_valid_for_one_label_producer(tmp_path): @pytest.mark.parametrize("valid", [False, True]) def test_headless_entrypoint_requires_subject_and_executes_corrected_rows( - tmp_path, - valid, - registered_count_callable, + tmp_path, valid, registered_count_callable, ): _write_plate(tmp_path) document = PipelineDocumentCodec.from_values( @@ -706,21 +577,19 @@ def test_headless_entrypoint_requires_subject_and_executes_corrected_rows( bindings=(_source("DNA", "1"),), ), ), - pipeline_steps=[ - _step( - registered_count_callable, - "Count pixels", - {}, - ) - ], + pipeline_steps=[_step( + registered_count_callable, + "Count pixels", {}, + )], + ) + document = PipelineDocumentCodec.from_source( + PipelineDocumentCodec.render(document) ) - document = PipelineDocumentCodec.from_source(PipelineDocumentCodec.render(document)) invocation = next( normalize_function_pattern(document.pipeline_steps[0].func).iter_items() ) [rows] = ( - spec - for spec in invocation.contract.artifact_specs.specs + spec for spec in invocation.contract.artifact_specs.specs if spec.artifact_type is MeasurementsArtifactType ) assert rows.measurement_feature_owner is CountFeatureOwner @@ -742,9 +611,7 @@ def test_headless_entrypoint_requires_subject_and_executes_corrected_rows( ) is valid global_config = GlobalPipelineConfig(num_workers=1, use_threading=True) if not valid: - with pytest.raises( - ValueError, match="PixelCounts.*no declared measurement subject" - ) as exc: + with pytest.raises(ValueError, match="PixelCounts.*no declared measurement subject") as exc: _compile(tmp_path, document, global_config) assert "ImageMeasurementSubjectRelation" in str(exc.value) return diff --git a/tests/integration/test_neurite_outgrowth_synthetic_plate.py b/tests/integration/test_neurite_outgrowth_synthetic_plate.py index bddccebbff..14e16ffa01 100644 --- a/tests/integration/test_neurite_outgrowth_synthetic_plate.py +++ b/tests/integration/test_neurite_outgrowth_synthetic_plate.py @@ -221,7 +221,9 @@ def observe_dense_output(output, components): progress_events = [] while True: try: - progress_events.append(progress_queue.get_nowait()) + progress_events.append( + progress_queue.get_nowait() + ) except queue.Empty: break live_payloads = tuple( @@ -313,8 +315,7 @@ def observe_dense_output(output, components): # Checkpoints are the declared outputs between the label artifacts and # the trailing morphology table; the callable's output family owns them. checkpoint_names = tuple( - spec.name - for spec in CellProfilerNeuriteEngineProfile.artifact_outputs()[6:-1] + spec.name for spec in CellProfilerNeuriteEngineProfile.artifact_outputs()[6:-1] ) for artifact_name, expected_checkpoint in zip( checkpoint_names, @@ -330,9 +331,7 @@ def observe_dense_output(output, components): retained, np.asarray(expected_checkpoint)[0], ) - assert len({path.name for path in checkpoint_paths}) == 2 * len( - checkpoint_names - ) + assert len({path.name for path in checkpoint_paths}) == 2 * len(checkpoint_names) assert compiled_plan.output_dataset_root is not None output_dataset_root = Path(compiled_plan.output_dataset_root) diff --git a/tests/integration/test_primary_segmentation_diagnostics_journey.py b/tests/integration/test_primary_segmentation_diagnostics_journey.py index 9d45b7ff51..22755da427 100644 --- a/tests/integration/test_primary_segmentation_diagnostics_journey.py +++ b/tests/integration/test_primary_segmentation_diagnostics_journey.py @@ -83,8 +83,7 @@ def test_real_registered_ipo_returns_same_run_stage_pixels(mode): diagnostics.unedited_objects.data, objects.variant_labels(UneditedLabels) ) np.testing.assert_array_equal( - diagnostics.small_removed_objects.data, - objects.variant_labels(SmallRemovedLabels), + diagnostics.small_removed_objects.data, objects.variant_labels(SmallRemovedLabels) ) assert measurements.row_count() > 0 if mode in ("empty", "disabled", "watershed-disabled"): @@ -252,7 +251,9 @@ def step(func, name, kwargs): ), pipeline_steps=[primary, secondary], ) - document = PipelineDocumentCodec.from_source(PipelineDocumentCodec.render(document)) + document = PipelineDocumentCodec.from_source( + PipelineDocumentCodec.render(document) + ) bundle = ( InProcessCompileInspectionGateway() .compile( @@ -353,8 +354,7 @@ def step(func, name, kwargs): persisted.append(record.data) diagnostics = PrimaryObjectDiagnosticPlanes(*persisted) np.testing.assert_array_equal( - diagnostics.unedited_objects.data, - primary_record.data.variant_labels(UneditedLabels), + diagnostics.unedited_objects.data, primary_record.data.variant_labels(UneditedLabels) ) np.testing.assert_array_equal( diagnostics.small_removed_objects.data, diff --git a/tests/integration/test_session_journey.py b/tests/integration/test_session_journey.py index f480cb2fcd..f99d4f83ec 100644 --- a/tests/integration/test_session_journey.py +++ b/tests/integration/test_session_journey.py @@ -145,11 +145,7 @@ def test_headless_mcp_journey_adds_compiles_and_runs_a_dataset(tmp_path: Path) - {"after_sequence": 0, "timeout_seconds": 0.0}, )["events"] } - assert { - "DatasetsChanged", - "CompiledStateChanged", - "DatasetExecutionFinished", - } <= kinds + assert {"DatasetsChanged", "CompiledStateChanged", "DatasetExecutionFinished"} <= kinds finally: context.session.close() @@ -176,9 +172,7 @@ def test_read_only_dataset_initializes_in_a_mirrored_workspace(tmp_path: Path) - workspace_root=workspaces, ) try: - assert session.invoke( - AddDatasets, AddDatasets.request(roots=(str(root),)) - ).accepted + assert session.invoke(AddDatasets, AddDatasets.request(roots=(str(root),))).accepted (scope_id,) = session.dataset_scope_ids() started = session.invoke( InitializeDatasets, InitializeDatasets.request(scope_ids=(scope_id,)) diff --git a/tests/integration/test_source_schema_aggregate_publication.py b/tests/integration/test_source_schema_aggregate_publication.py index 5ea1410efd..37c4614558 100644 --- a/tests/integration/test_source_schema_aggregate_publication.py +++ b/tests/integration/test_source_schema_aggregate_publication.py @@ -20,11 +20,7 @@ InProcessCompileInspectionGateway, ) from openhcs.constants.input_source import InputSource -from openhcs.core.artifacts import ( - ImageArtifactType, - MainFlowStackOutputSpec, - SpecialArtifactType, -) +from openhcs.core.artifacts import ImageArtifactType, MainFlowStackOutputSpec, SpecialArtifactType from openhcs.core.config import ( GlobalPipelineConfig, LazyProcessingConfig, @@ -61,8 +57,7 @@ @artifact_outputs( MainFlowStackOutputSpec.output("PassThrough", ImageArtifactType), MainFlowStackOutputSpec.output( - "VolumeScalar609", - SpecialArtifactType, + "VolumeScalar609", SpecialArtifactType, materialization=MaterializationSpec(CsvOptions()), ), ) @@ -72,9 +67,7 @@ def volume_scalar_609(image: np.ndarray): @pytest.mark.parametrize("group_by", [Ungrouped, Microscopy.Channel]) -def test_real_z_aggregation_persists_scalar_and_unchanged_source_planes( - tmp_path, group_by -): +def test_real_z_aggregation_persists_scalar_and_unchanged_source_planes(tmp_path, group_by): source = tmp_path / "input" source.mkdir() pixels = np.arange(140, dtype=np.uint16).reshape(4, 5, 7) @@ -90,58 +83,35 @@ def test_real_z_aggregation_persists_scalar_and_unchanged_source_planes( pipeline_config=PipelineConfig( dataset_source=SourceBindingsSource, source_bindings_config=LazySourceBindingsConfig( - metadata_rules=( - MetadataExtractionRule( - source=MetadataSource.FILE_NAME, - pattern=r"(?PA[0-9]+)_s(?P[0-9]+)_w(?P[0-9]+)_z(?P[0-9]+)_t(?P[0-9]+)", - ), - ), - bindings=( - NamedSourceBinding( - alias="Fixture", - selector=SourceSelector(), - component_identity=( - ComponentSelector(Microscopy.Channel, "1"), - ), - ), - ), + metadata_rules=(MetadataExtractionRule( + source=MetadataSource.FILE_NAME, + pattern=r"(?PA[0-9]+)_s(?P[0-9]+)_w(?P[0-9]+)_z(?P[0-9]+)_t(?P[0-9]+)", + ),), + bindings=(NamedSourceBinding( + alias="Fixture", selector=SourceSelector(), + component_identity=(ComponentSelector(Microscopy.Channel, "1"),), + ),), ), ), - pipeline_steps=[ - FunctionStep( - func=registered, - name="Actual volume scalar", - processing_config=LazyProcessingConfig( - variable_components=[Microscopy.ZIndex], - group_by=group_by, - input_source=InputSource.PIPELINE_START, - ), - ) - ], - ) - bundle = ( - InProcessCompileInspectionGateway() - .compile( - CompileInspectionInput( - plate=source, - pipeline_document=document, - axis_filter=("A01",), - global_pipeline_config=GlobalPipelineConfig( - num_workers=1, use_threading=True - ), - progress_queue=AgentProgressQueue(), - ) - ) - .execution_bundle + pipeline_steps=[FunctionStep( + func=registered, name="Actual volume scalar", + processing_config=LazyProcessingConfig( + variable_components=[Microscopy.ZIndex], + group_by=group_by, + input_source=InputSource.PIPELINE_START, + ), + )], ) + bundle = InProcessCompileInspectionGateway().compile(CompileInspectionInput( + plate=source, pipeline_document=document, axis_filter=("A01",), + global_pipeline_config=GlobalPipelineConfig(num_workers=1, use_threading=True), + progress_queue=AgentProgressQueue(), + )).execution_bundle context = bundle.runtime_contexts["A01"] assert tuple(context.step_plans[0].variable_components) == (Microscopy.ZIndex,) - orchestrator = PipelineOrchestrator( - source, pipeline_config=document.pipeline_config - ).initialize() + orchestrator = PipelineOrchestrator(source, pipeline_config=document.pipeline_config).initialize() outcomes = orchestrator.execute_compiled_bundle( - execution_bundle=bundle, - max_workers=1, + execution_bundle=bundle, max_workers=1, runtime_observation_mode=RuntimeObservationMode.MERGE_INTO_PARENT, progress_queue=AgentProgressQueue(), progress_context=ProgressExecutionContext( @@ -156,9 +126,7 @@ def test_real_z_aggregation_persists_scalar_and_unchanged_source_planes( assert int(row["volume_sum"]) == int(pixels.sum()) assert int(row["voxel_count"]) == pixels.size assert "_z" not in csv_path.name - saved = tuple( - Path(context.step_plans[0].artifact_images_dir).glob("*PassThrough.tif") - ) + saved = tuple(Path(context.step_plans[0].artifact_images_dir).glob("*PassThrough.tif")) assert len(saved) == 4 for image in saved: z_index = int(image.name.split("_z", 1)[1].split("_", 1)[0]) diff --git a/tests/pyqt_gui/integration/test_end_to_end_workflow_foundation.py b/tests/pyqt_gui/integration/test_end_to_end_workflow_foundation.py index d504043025..a8ed448dc0 100644 --- a/tests/pyqt_gui/integration/test_end_to_end_workflow_foundation.py +++ b/tests/pyqt_gui/integration/test_end_to_end_workflow_foundation.py @@ -232,9 +232,7 @@ def materialization_config( ), ), setup_modifications=( - FieldModificationSpec( - ("step_partition_filter_config", "partition_filter"), 5 - ), + FieldModificationSpec(("step_partition_filter_config", "partition_filter"), 5), ), field_to_test=FieldModificationSpec( ("path_planning_config", "partition_filter"), 99 @@ -247,9 +245,7 @@ def materialization_config( ("napari_streaming_config", "partition_filter"), ("fiji_streaming_config", "partition_filter"), ), - excluded_source_paths=( - ("path_planning_config", "partition_filter"), - ), + excluded_source_paths=(("path_planning_config", "partition_filter"),), ), ), ), diff --git a/tests/pyqt_gui/test_session_gui_journey.py b/tests/pyqt_gui/test_session_gui_journey.py index 76c6b25a92..e5d4ce61a4 100644 --- a/tests/pyqt_gui/test_session_gui_journey.py +++ b/tests/pyqt_gui/test_session_gui_journey.py @@ -56,11 +56,11 @@ def test_gui_journey_runs_the_same_operations_through_the_widgets( main_thread=DispatcherThread(dispatcher), ) services = GuiServiceStub() - manager = PlateManagerWidget(services, session, gui_config=get_default_ui_config()) + manager = PlateManagerWidget( + services, session, gui_config=get_default_ui_config() + ) try: - assert session.invoke( - AddDatasets, AddDatasets.request(roots=(str(root),)) - ).accepted + assert session.invoke(AddDatasets, AddDatasets.request(roots=(str(root),))).accepted (scope_id,) = session.dataset_scope_ids() session.set_pipeline_source(scope_id, source) application.processEvents() @@ -81,8 +81,7 @@ def wait_until(condition, label) -> None: (CompileDatasets, lambda row: row.compiled and not row.compile_pending), ( RunDatasets, - lambda row: row.terminal_status is not None - and not row.execution_active, + lambda row: row.terminal_status is not None and not row.execution_active, ), ): if operation is CompileDatasets: diff --git a/tests/unit/agent/test_agent_services.py b/tests/unit/agent/test_agent_services.py index 43901a6e19..36b94e72c5 100644 --- a/tests/unit/agent/test_agent_services.py +++ b/tests/unit/agent/test_agent_services.py @@ -51,9 +51,7 @@ ) from openhcs.agent.path_policy import AgentPathPolicy, AgentPathPolicyError from openhcs.agent.services import function_catalog_service as function_catalog_module -from openhcs.agent.services import ( - artifact_plan_inspection_service as artifact_plan_module, -) +from openhcs.agent.services import artifact_plan_inspection_service as artifact_plan_module from openhcs.agent.services import viewer_window_service as viewer_window_service_module from openhcs.agent.services.config_service import ConfigService from openhcs.agent.services.artifact_plan_inspection_service import ( @@ -104,10 +102,7 @@ from openhcs.core.pipeline.function_contracts import artifact_inputs, artifact_outputs from openhcs.core.pipeline_document import PipelineDocumentCodec from openhcs.core.progress import ( - ProgressExecutionContext, - ProgressIdentity, - ProgressPhase, - ProgressStatus, + ProgressExecutionContext, ProgressIdentity, ProgressPhase, ProgressStatus, ) from openhcs.core.source_bindings import ( LazySourceBindingsConfig, @@ -317,13 +312,9 @@ def __init__(self) -> None: def emit_progress(request): request.progress_queue.put( ProgressExecutionContext("inspection", str(request.plate)).event( - axis_id="A01", - step_name="compilation", - phase=ProgressPhase.COMPILE, - status=ProgressStatus.RUNNING, - percent=0.0, - completed=0, - total=1, + axis_id="A01", step_name="compilation", + phase=ProgressPhase.COMPILE, status=ProgressStatus.RUNNING, + percent=0.0, completed=0, total=1, ) ) @@ -1293,8 +1284,7 @@ def test_function_catalog_resolves_detail_by_callable_import_path(monkeypatch): selected = _Metadata(tags=[]) unrelated = _Metadata.from_function(sample_large_signature_function, "Unrelated") monkeypatch.setattr( - catalog, - "_all_metadata", + catalog, "_all_metadata", lambda **_kwargs: { "test:aaa_unrelated": unrelated, "test:sample_processing_function": selected, @@ -1619,10 +1609,7 @@ def __repr__(self): == repeated.items[0].signature == "sample_processing_function(sigma=1.0)" ) - assert ( - catalog.search(query="sample", compact_signatures=True).items[0] - is first.items[0] - ) + assert catalog.search(query="sample", compact_signatures=True).items[0] is first.items[0] assert ( catalog.get("test:sample_processing_function").entry.signature == "sample_processing_function(sigma=2.0)" @@ -2925,8 +2912,9 @@ def test_artifact_plan_inspection_inspects_pipeline_source_artifact_plan( plate_path=str(tmp_path), pipeline_source=_pipeline_document_source(), global_config_id=None, - axis_filter=("A01",), - ), + axis_filter=("A01", + ) + ), ) assert compile_gateway.requests[0].plate == tmp_path.resolve() @@ -2941,10 +2929,7 @@ def test_artifact_plan_inspection_inspects_pipeline_source_artifact_plan( assert events[0].phase is ProgressPhase.COMPILE assert events[0].status is ProgressStatus.RUNNING assert events[0].identity == ProgressIdentity( - "inspection", - str(tmp_path.resolve()), - "A01", - "compilation", + "inspection", str(tmp_path.resolve()), "A01", "compilation", ) assert inspection.errors == () assert inspection.axis_count == 1 @@ -2964,9 +2949,9 @@ def test_artifact_plan_inspection_inspects_pipeline_source_artifact_plan( ) assert inspection.steps[0].viewer_streaming[0].viewer_type is ViewerType.NAPARI assert inspection.steps[0].viewer_streaming[0].effective_config["enabled"] is True - assert inspection.steps[0].viewer_streaming[0].effective_config[ - "partition_filter" - ] == ["A01"] + assert inspection.steps[0].viewer_streaming[0].effective_config["partition_filter"] == [ + "A01" + ] assert inspection.steps[0].artifact_inputs[0].name == "positions" assert inspection.steps[0].artifact_inputs[0].kind == "special" assert inspection.steps[0].artifact_inputs[0].source_step_id == 0 @@ -3064,12 +3049,12 @@ def test_compile_inspection_rejects_read_only_plate_before_initialization( with pytest.raises(AgentPathPolicyError, match="Writable path is outside"): service.inspect( - PipelineSourceArtifactPlanInspectionRequest( - plate_path=str(plate), - pipeline_source=_pipeline_document_source(), - global_config_id=None, - ) + PipelineSourceArtifactPlanInspectionRequest( + plate_path=str(plate), + pipeline_source=_pipeline_document_source(), + global_config_id=None, ) + ) assert gateway.requests == [] assert {path.name: path.read_bytes() for path in plate.iterdir()} == before diff --git a/tests/unit/agent/test_authoring_roundtrip_boundaries.py b/tests/unit/agent/test_authoring_roundtrip_boundaries.py index 97aa71635d..6a548fe6ad 100644 --- a/tests/unit/agent/test_authoring_roundtrip_boundaries.py +++ b/tests/unit/agent/test_authoring_roundtrip_boundaries.py @@ -115,7 +115,9 @@ def test_recursive_json_alias_keeps_declaration_namespace_in_new_descendant(): @pytest.fixture def gamma_metadata(monkeypatch): registry = SkimageRegistry() - wrapped = registry.reconstruct_cached_callable(adjust_gamma, FlexibleContract) + wrapped = registry.reconstruct_cached_callable( + adjust_gamma, FlexibleContract + ) metadata = FunctionMetadata( name="exposure.adjust_gamma", func=wrapped, diff --git a/tests/unit/agent/test_callable_artifact_reference.py b/tests/unit/agent/test_callable_artifact_reference.py index 6ae19883a4..668d99e5d9 100644 --- a/tests/unit/agent/test_callable_artifact_reference.py +++ b/tests/unit/agent/test_callable_artifact_reference.py @@ -86,9 +86,7 @@ def reference_namespace(): sys.modules[module.__name__] = module try: for block in REFERENCE_BLOCKS: - exec( - compile(_reference_block(block), DOCUMENT_PATH, "exec"), module.__dict__ - ) + exec(compile(_reference_block(block), DOCUMENT_PATH, "exec"), module.__dict__) yield module.__dict__ finally: sys.modules.pop(module.__name__, None) @@ -188,18 +186,15 @@ def test_diagnostic_reference_uses_one_canonical_slot_and_exact_trailing_artifac returned = function(pixels) assert len(returned) == 3 assert contract.canonical_return_output_specs.names() == ( - "fixture_image", - "fixture_diagnostic", + "fixture_image", "fixture_diagnostic", ) assert contract.trailing_return_output_specs.names() == ( - "fixture_labels", - "fixture_object_rows", + "fixture_labels", "fixture_object_rows", ) matched = contract.resolve_returned_output(returned) np.testing.assert_array_equal(matched[namespace["FIXTURE_IMAGE"].ref()], pixels) np.testing.assert_array_equal( - matched[namespace["FIXTURE_DIAGNOSTIC"].ref()], - pixels > 0, + matched[namespace["FIXTURE_DIAGNOSTIC"].ref()], pixels > 0, ) assert matched[namespace["FIXTURE_ROWS"].ref()].row_mappings() == ( {"slice_index": 0, "object_label": 7, "pixel_count": 16}, @@ -209,8 +204,7 @@ def test_diagnostic_reference_uses_one_canonical_slot_and_exact_trailing_artifac def test_points_declaration_materializes_fractional_geometry_and_full_source_domain( - reference_namespace, - tmp_path, + reference_namespace, tmp_path, ): from openhcs.core.roi_point_metadata import ROIFractionalZ from openhcs.core.roi_source_metadata import ROIArchiveSourceMetadata @@ -222,23 +216,18 @@ def test_points_declaration_materializes_fractional_geometry_and_full_source_dom spec = namespace["CENTRE_ROWS"] labels = namespace["CENTRE_LABELS"] plan = ArtifactOutputPlan( - name=spec.name, - path="/synthetic/centre_rows", - artifact_type=spec.artifact_type, + name=spec.name, path="/synthetic/centre_rows", artifact_type=spec.artifact_type, relations=spec.relations, ) assert plan.object_subject_binding().source == labels.ref() row = namespace["CentreRow"](7, 1.5, 1.25, 2.5) carrier = namespace["DataclassMeasurementColumnarRows"]( - (row,), - row_type=namespace["CentreRow"], + (row,), row_type=namespace["CentreRow"], ) # Direct materialization supplies the same nominal context normally owned # by runtime contextualization, without inventing an archive/CSV writer. table = MeasurementTable( - name=spec.name, - rows=carrier, - subject=plan.measurement_subject(), + name=spec.name, rows=carrier, subject=plan.measurement_subject(), measurement_feature_owner=namespace["CentreFeatureOwner"], source_path="/synthetic/volume.tif", source_image_provenance_planes=SourceImageProvenancePlanes.from_components( @@ -250,23 +239,16 @@ def test_points_declaration_materializes_fractional_geometry_and_full_source_dom ), ) materialize( - spec.materialization, - data=table, - path=str(tmp_path / "centres"), + spec.materialization, data=table, path=str(tmp_path / "centres"), filemanager=FileManager({"disk": DiskStorageBackend()}), - backends=["disk"], - backend_kwargs={}, + backends=["disk"], backend_kwargs={}, ) csv_options = next( - option - for option in spec.materialization.outputs + option for option in spec.materialization.outputs if isinstance(option, namespace["CsvOptions"]) ) - assert ( - tmp_path / ("centres" + csv_options.filename_suffix) - ).read_text().splitlines() == [ - "object_label,center_z,center_y,center_x", - "7,1.5,1.25,2.5", + assert (tmp_path / ("centres" + csv_options.filename_suffix)).read_text().splitlines() == [ + "object_label,center_z,center_y,center_x", "7,1.5,1.25,2.5", ] rois = load_rois_from_zip(tmp_path / "centres_points.roi.zip") assert rois[0].shapes == [PointShape(y=1.25, x=2.5)] @@ -289,8 +271,7 @@ def test_input_reference_compiles_nominal_binding_and_repairs_wrong_annotation( assert contract.artifact_input_parameter_names == ("objects",) contract.validate_artifact_input_parameter_bindings() plan = ArtifactInputPlan( - declaration.name, - "/synthetic/fixture-labels.pkl", + declaration.name, "/synthetic/fixture-labels.pkl", artifact_type=declaration.artifact_type, ) compiled = compile_function_pattern(function, {plan.ref(): plan}, {}) @@ -308,9 +289,7 @@ def test_input_reference_compiles_nominal_binding_and_repairs_wrong_annotation( original_annotation = raw.__annotations__["objects"] try: raw.__annotations__["objects"] = np.ndarray - with pytest.raises( - TypeError, match="does not accept object_labels artifact payloads" - ): + with pytest.raises(TypeError, match="does not accept object_labels artifact payloads"): compile_function_pattern(function, {plan.ref(): plan}, {}) finally: raw.__annotations__["objects"] = original_annotation @@ -349,21 +328,16 @@ def fixture_storage(_name, *, create): def test_input_reference_prepares_in_real_custom_namespace(tmp_path, monkeypatch): monkeypatch.setattr( - custom_manager, - "get_data_file_path", + custom_manager, "get_data_file_path", lambda _name, *, create: tmp_path / "custom_functions", ) manager = custom_manager.CustomFunctionManager(create_storage=False) - metadata = manager._prepare_source( - _reference_block("callable-artifact-input-reference") - ) + metadata = manager._prepare_source(_reference_block("callable-artifact-input-reference")) contract = CallableContract.from_callable(metadata.func) contract.validate_artifact_input_parameter_bindings() assert contract.artifact_inputs.specs[0].parameter_name == "objects" labels = ObjectLabelPayload( - variant_data=ObjectLabelVariantData( - labels=np.asarray([[0, 7]], dtype=np.int32) - ), + variant_data=ObjectLabelVariantData(labels=np.asarray([[0, 7]], dtype=np.int32)), ) image = np.asarray([[3, 5]], dtype=np.uint16) np.testing.assert_array_equal(metadata.func(image, objects=labels), [[0, 5]]) @@ -371,42 +345,29 @@ def test_input_reference_prepares_in_real_custom_namespace(tmp_path, monkeypatch def test_plate_reference_prepares_compiles_and_runs_in_original_parent( - reference_namespace, - tmp_path, - monkeypatch, + reference_namespace, tmp_path, monkeypatch, ): from openhcs.core.callable_contract import FunctionStepExecutionScope - from openhcs.core.pipeline.funcstep_contract_validator import ( - FuncStepContractValidator, - ) + from openhcs.core.pipeline.funcstep_contract_validator import FuncStepContractValidator from openhcs.core.artifacts import SpecialArtifactType - from openhcs.core.measurement_row_materialization import ( - DataclassMeasurementColumnarRows, - ) + from openhcs.core.measurement_row_materialization import DataclassMeasurementColumnarRows from openhcs.core.runtime_measurements import MeasurementTable from openhcs.core.runtime_stores import RuntimeArtifactBatch monkeypatch.setattr( - custom_manager, - "get_data_file_path", + custom_manager, "get_data_file_path", lambda _name, *, create: tmp_path / "custom_functions", ) manager = custom_manager.CustomFunctionManager(create_storage=False) - metadata = manager._prepare_source( - _reference_block("callable-artifact-plate-reference") - ) + metadata = manager._prepare_source(_reference_block("callable-artifact-plate-reference")) contract = CallableContract.from_callable(metadata.func) - FuncStepContractValidator.validate_dataset_callable_contracts( - (contract,), metadata.name - ) + FuncStepContractValidator.validate_dataset_callable_contracts((contract,), metadata.name) assert contract.execution_scope is FunctionStepExecutionScope.PLATE assert contract.runtime_bound_parameter_types == (RuntimeArtifactBatch,) assert contract.input_memory_type is contract.processing_contract is None assert contract.artifact_input_parameter_names == () # Reuse the existing plate execution fixtures, not a replacement dispatcher. - plate = runpy.run_path( - str(ROOT / "tests/unit/test_function_step_execution_scope.py") - ) + plate = runpy.run_path(str(ROOT / "tests/unit/test_function_step_execution_scope.py")) source = contract.artifact_inputs.specs[0] output = contract.artifact_outputs.specs[0] contexts = {} @@ -416,21 +377,13 @@ def test_plate_reference_prepares_compiles_and_runs_in_original_parent( ): path = f"/memory/{axis_id}/rows" plan = plate["_plate_step_plan"]( - axis_id=axis_id, - step_index=0, - func=metadata.func, - artifact_inputs=( - ArtifactInputPlan( - source.name, - path, - artifact_type=source.artifact_type, - ), - ), + axis_id=axis_id, step_index=0, func=metadata.func, + artifact_inputs=(ArtifactInputPlan( + source.name, path, artifact_type=source.artifact_type, + ),), artifact_output=ArtifactOutputPlan( - output.name, - "/memory/plate/summary", - artifact_type=output.artifact_type, - relations=output.relations, + output.name, "/memory/plate/summary", + artifact_type=output.artifact_type, relations=output.relations, ), metadata_writer=axis_id == "axis-a", ) @@ -440,16 +393,12 @@ def test_plate_reference_prepares_compiles_and_runs_in_original_parent( assert isinstance(record.data, MeasurementTable) record.data.rows = reference_namespace["inspect_label_fixture"](fixture)[2] plate["_record_measurements"]( - context, - name="unrelated_rows", - path=f"/memory/{axis_id}/unrelated", - count=99, + context, name="unrelated_rows", path=f"/memory/{axis_id}/unrelated", count=99, ) contexts[axis_id] = context plate["_execute_plate_steps"](contexts) summaries = tuple( - record - for context in contexts.values() + record for context in contexts.values() for record in context.runtime_value_store.values() if record.key.name == output.name ) @@ -462,30 +411,22 @@ def test_plate_reference_prepares_compiles_and_runs_in_original_parent( {"axis_id": "axis-b", "record_count": 1, "measurement_row_count": 0}, ) csv_path = materialize( - output.materialization, - data=table, - path=str(tmp_path / "summary"), + output.materialization, data=table, path=str(tmp_path / "summary"), filemanager=FileManager({"disk": DiskStorageBackend()}), - backends=["disk"], - backend_kwargs={}, + backends=["disk"], backend_kwargs={}, ) assert Path(csv_path).read_text().splitlines() == [ - "axis_id,record_count,measurement_row_count", - "axis-a,1,2", - "axis-b,1,0", + "axis_id,record_count,measurement_row_count", "axis-a,1,2", "axis-b,1,0", ] assert not manager.storage_dir.exists() @pytest.mark.parametrize("plane_count", (None, 1, 2)) def test_reference_retains_plane_domain_until_raw_numpy_invocation( - tmp_path, - monkeypatch, - plane_count, + tmp_path, monkeypatch, plane_count, ): monkeypatch.setattr( - custom_manager, - "get_data_file_path", + custom_manager, "get_data_file_path", lambda _name, *, create: tmp_path / "custom_functions", ) manager = custom_manager.CustomFunctionManager(create_storage=False) @@ -511,18 +452,12 @@ def test_reference_retains_plane_domain_until_raw_numpy_invocation( for plane in range(1 if plane_count is None else plane_count) ) projection = ( - None - if plane_count is None - else RuntimePlaneAxisValueProjection( - axis=RuntimePlaneAxis.RUNTIME_SLICE, - source_aliases=(), - plane_index=None, - axis_size=plane_count, + None if plane_count is None else RuntimePlaneAxisValueProjection( + axis=RuntimePlaneAxis.RUNTIME_SLICE, source_aliases=(), + plane_index=None, axis_size=plane_count, ) ) - label_payload = ObjectLabelsArtifactType.contextualize_output( - stack, labels, None, projection - ) + label_payload = ObjectLabelsArtifactType.contextualize_output(stack, labels, None, projection) assert isinstance(label_payload, ObjectLabelPayload) np.testing.assert_array_equal(label_payload.labels, stack.data) assert label_payload.plane_axis is ( @@ -532,8 +467,7 @@ def test_reference_retains_plane_domain_until_raw_numpy_invocation( assert label_payload.source_spatial_domain.source_shape_yx == (8, 8) assert label_payload.domain.scope is ( ObjectLabelDomainScope.PAYLOAD - if plane_count is None - else ObjectLabelDomainScope.PLANE + if plane_count is None else ObjectLabelDomainScope.PLANE ) if plane_count is not None: assert label_payload.domain.declared_object_id_domains == ((1,),) * plane_count @@ -541,9 +475,7 @@ def test_reference_retains_plane_domain_until_raw_numpy_invocation( plane_axis=RuntimePlaneAxis.RUNTIME_SLICE, ).payload_with(stack.data[:, :-1, :]) with pytest.raises(ValueError, match="Object-label spatial shape"): - ObjectLabelsArtifactType.contextualize_output( - stack, wrong_spatial_shape, None, projection - ) + ObjectLabelsArtifactType.contextualize_output(stack, wrong_spatial_shape, None, projection) assert not manager.storage_dir.exists() @@ -555,8 +487,7 @@ def test_reference_retains_plane_domain_until_raw_numpy_invocation( ), ) def test_reference_binds_all_output_lineage_to_current_image( - reference_namespace, - source, + reference_namespace, source, ): invocation = next( normalize_function_pattern( @@ -577,10 +508,11 @@ def test_reference_binds_all_output_lineage_to_current_image( assert output.source_stack_scope_sources() == (source.ref(),) assert contract.canonical_return_output_specs.names() == ("fixture_image",) assert contract.trailing_return_output_specs.names() == ( - "fixture_labels", - "fixture_object_rows", + "fixture_labels", "fixture_object_rows", + ) + rows = contract.artifact_outputs.by_ref( + reference_namespace["FIXTURE_ROWS"].ref() ) - rows = contract.artifact_outputs.by_ref(reference_namespace["FIXTURE_ROWS"].ref()) assert rows is not None subject = ArtifactOutputPlan( name=rows.name, @@ -642,10 +574,7 @@ def test_packaged_knowledge_projection_retains_executable_reference(tmp_path): ) assert "def inspect_label_fixture(" in document.content assert "from openhcs.core.memory import numpy" in document.content - for query in ( - "PointROIOptions fractional centres", - "aligned diagnostic image returns", - ): + for query in ("PointROIOptions fractional centres", "aligned diagnostic image returns"): hits = service.search(KnowledgeBaseSearchRequest(query=query, limit=10)) assert DOCUMENT_ID in {hit.document.document_id for hit in hits.hits} for block in REFERENCE_BLOCKS: @@ -654,8 +583,7 @@ def test_packaged_knowledge_projection_retains_executable_reference(tmp_path): assert line in document.content route = service.get_document( KnowledgeBaseDocumentRequest.from_fields( - document_id="openhcs_custom_function_workflow", - max_chars=30_000, + document_id="openhcs_custom_function_workflow", max_chars=30_000, ) ) source_path = "packaging/codex/openhcs/skills/use-openhcs/references/custom-function-authoring.md" @@ -665,20 +593,14 @@ def test_packaged_knowledge_projection_retains_executable_reference(tmp_path): "materialize-typed-3d-centres-as-feature-bearing-points", "return-diagnostic-images-without-flattening-the-abi", ): - selected = service.get_document( - KnowledgeBaseDocumentRequest.from_fields( - document_id=DOCUMENT_ID, - section_id=section_id, - max_chars=10_000, - ) - ) + selected = service.get_document(KnowledgeBaseDocumentRequest.from_fields( + document_id=DOCUMENT_ID, section_id=section_id, max_chars=10_000, + )) assert not selected.errors and not selected.truncated assert selected.selected_section_id == section_id original = KnowledgeBaseService(repo_root=ROOT).get_document( KnowledgeBaseDocumentRequest.from_fields( - document_id=DOCUMENT_ID, - section_id=section_id, - max_chars=10_000, + document_id=DOCUMENT_ID, section_id=section_id, max_chars=10_000, ) ) assert selected.content == original.content diff --git a/tests/unit/agent/test_capabilities.py b/tests/unit/agent/test_capabilities.py index 6cdc70a27a..b6db329076 100644 --- a/tests/unit/agent/test_capabilities.py +++ b/tests/unit/agent/test_capabilities.py @@ -121,7 +121,8 @@ def test_capability_transport_defaults_to_local_stdio(): def test_capability_exposition_owns_its_json_projection(): capability = next( - capability for capability in get_capability_registry().capabilities + capability + for capability in get_capability_registry().capabilities ) exposition = capability.exposition @@ -293,15 +294,9 @@ def test_custom_function_registration_is_a_desktop_authoring_mutation(): } registration = full_capabilities["openhcs_register_custom_function"] - assert ( - registration.exposition.workflow_group - is CapabilityWorkflowGroup.FUNCTION_AUTHORING - ) + assert registration.exposition.workflow_group is CapabilityWorkflowGroup.FUNCTION_AUTHORING assert registration.exposition.workflow_stage is CapabilityWorkflowStage.AUTHORING - assert ( - registration.exposition.target_context - is CapabilityTargetContext.FUNCTION_REGISTRY - ) + assert registration.exposition.target_context is CapabilityTargetContext.FUNCTION_REGISTRY assert registration.exposition.visibility is CapabilityVisibility.STANDARD assert registration.exposition.role is CapabilityRole.PRIMARY assert registration.mutating is True @@ -349,12 +344,7 @@ def _declaration_error(**attributes) -> Exception: type( "RejectedDeclaration", (UiBridgeCapability,), - { - "name": name, - "title": "Rejected", - "description": "", - "service": "test", - } + {"name": name, "title": "Rejected", "description": "", "service": "test"} | attributes, ) finally: @@ -417,9 +407,9 @@ def test_similar_mcp_tool_names_are_disambiguated_by_target_context_and_role(): assert capabilities["openhcs_sample_plate_image"].exposition.target_context is ( CapabilityTargetContext.PLATE_PATH ) - assert capabilities[ - "openhcs_ui_sample_selected_plate_image" - ].exposition.target_context is (CapabilityTargetContext.UI_SELECTED_PLATE) + assert capabilities["openhcs_ui_sample_selected_plate_image"].exposition.target_context is ( + CapabilityTargetContext.UI_SELECTED_PLATE + ) assert capabilities["openhcs_ui_sample_selected_plate_image"].exposition.role is ( CapabilityRole.MODE_VARIANT ) @@ -430,25 +420,20 @@ def test_similar_mcp_tool_names_are_disambiguated_by_target_context_and_role(): CapabilityTargetContext.HEADLESS_SESSION ) assert ( - capabilities[ - "openhcs_get_runtime_server_execution_status" - ].exposition.target_context + capabilities["openhcs_get_runtime_server_execution_status"].exposition.target_context is CapabilityTargetContext.RUNTIME_SERVER ) - assert capabilities[ - "openhcs_ui_get_operation_status" - ].exposition.target_context is (CapabilityTargetContext.UI_BRIDGE) - assert capabilities[ - "openhcs_ui_wait_for_operation_receipt" - ].exposition.target_context is (CapabilityTargetContext.UI_BRIDGE) - assert ( - capabilities["openhcs_ui_wait_for_operation_receipt"].exposition.role - is CapabilityRole.PRIMARY + assert capabilities["openhcs_ui_get_operation_status"].exposition.target_context is ( + CapabilityTargetContext.UI_BRIDGE + ) + assert capabilities["openhcs_ui_wait_for_operation_receipt"].exposition.target_context is ( + CapabilityTargetContext.UI_BRIDGE ) assert ( - capabilities["openhcs_ui_invoke_action"].exposition.role + capabilities["openhcs_ui_wait_for_operation_receipt"].exposition.role is CapabilityRole.PRIMARY ) + assert capabilities["openhcs_ui_invoke_action"].exposition.role is CapabilityRole.PRIMARY assert capabilities["openhcs_ui_invoke_widget_action"].exposition.role is ( CapabilityRole.FALLBACK ) @@ -461,9 +446,9 @@ def test_similar_mcp_tool_names_are_disambiguated_by_target_context_and_role(): assert capabilities["openhcs_validate_pipeline"].exposition.workflow_stage is ( CapabilityWorkflowStage.VALIDATION ) - assert capabilities[ - "openhcs_ui_mutate_object_state_field" - ].exposition.visibility is (CapabilityVisibility.EXPERT) + assert capabilities["openhcs_ui_mutate_object_state_field"].exposition.visibility is ( + CapabilityVisibility.EXPERT + ) def test_ui_bridge_capabilities_declare_runtime_security_and_data_exposure(): @@ -561,9 +546,7 @@ def invoke(self, operation, request=None, connection=None): connection = object() invocation = agent_capabilities.ui_bridge_status.invocation - context = SimpleNamespace( - ui_bridge_service=RecordingUiBridgeService(gateway=object()) - ) + context = SimpleNamespace(ui_bridge_service=RecordingUiBridgeService(gateway=object())) assert invocation.execute(context, connection) == "served" assert seen == [(invocation.operation, None, connection)] diff --git a/tests/unit/agent/test_cold_inspection_feedback.py b/tests/unit/agent/test_cold_inspection_feedback.py index 6c70b9fbfc..86e0b113b5 100644 --- a/tests/unit/agent/test_cold_inspection_feedback.py +++ b/tests/unit/agent/test_cold_inspection_feedback.py @@ -8,24 +8,17 @@ from zmqruntime.startup import EndpointStartupStatus from openhcs.agent.capabilities import ( - MainThreadProgressCapability, - QueryPlateFilesCapability, - SamplePlateImageCapability, + MainThreadProgressCapability, QueryPlateFilesCapability, SamplePlateImageCapability, InspectPipelineSourceArtifactPlanCapability, ) from openhcs.agent.path_policy import AgentPathPolicy from openhcs.agent.services.config_service import ConfigService from openhcs.agent.services.artifact_plan_inspection_service import ( - AgentProgressQueue, - CompileInspectionGatewayABC, - ArtifactPlanInspectionService, + AgentProgressQueue, CompileInspectionGatewayABC, ArtifactPlanInspectionService, ) from openhcs.agent.services.plate_inspection_service import PlateInspectionService from openhcs.core.progress import ( - ProgressEvent, - ProgressExecutionContext, - ProgressPhase, - ProgressStatus, + ProgressEvent, ProgressExecutionContext, ProgressPhase, ProgressStatus, ) from openhcs.mcp.execution import McpTransportExecutor from openhcs.mcp.server import build_server @@ -34,9 +27,7 @@ from tests.unit.agent.test_plate_inspection_service import ImageXpressPlateFixture -@pytest.mark.parametrize( - "declaration", (QueryPlateFilesCapability, SamplePlateImageCapability) -) +@pytest.mark.parametrize("declaration", (QueryPlateFilesCapability, SamplePlateImageCapability)) def test_physical_reader_declarations_inherit_original_affine_progress(declaration): assert issubclass(declaration, MainThreadProgressCapability) assert declaration.progress_heartbeat_seconds == 1.0 @@ -47,13 +38,9 @@ def test_physical_reader_declarations_inherit_original_affine_progress(declarati def test_original_progress_codec_is_the_only_event_store_and_rejects_bad_enum(): queue = AgentProgressQueue() event = ProgressExecutionContext("inspection", "/source").event( - axis_id="A01", - step_name="new-declared-step", - phase=ProgressPhase.COMPILE, - status=ProgressStatus.RUNNING, - percent=25, - completed=1, - total=4, + axis_id="A01", step_name="new-declared-step", + phase=ProgressPhase.COMPILE, status=ProgressStatus.RUNNING, percent=25, + completed=1, total=4, ) statuses = [] with EndpointStartupStatus.callback_scope(statuses.append): @@ -70,9 +57,7 @@ def test_original_progress_codec_is_the_only_event_store_and_rejects_bad_enum(): @pytest.mark.parametrize("fails", (False, True)) -def test_new_gateway_hook_cooperates_in_both_mros_before_terminal_result( - tmp_path, fails -): +def test_new_gateway_hook_cooperates_in_both_mros_before_terminal_result(tmp_path, fails): """Independent audit capability needs no edits to service or MCP consumers.""" identity = ContextVar("cold-feedback-request", default="outside") calls, messages = [], [] @@ -92,9 +77,7 @@ def _compile(self, request): assert threading.current_thread() is threading.main_thread() compile_entered.set() self.emit_progress(request) - assert release.wait( - 3 - ), "Actual stage relay was not delivered during held work" + assert release.wait(3), "Actual stage relay was not delivered during held work" if fails: raise error return super()._compile(request) @@ -124,43 +107,30 @@ async def report_progress(self, progress, total=None, message=None): gateway = gateway_type() service = ArtifactPlanInspectionService( path_policy=AgentPathPolicy.with_roots( - readable_roots=(tmp_path,), - writable_roots=(tmp_path,), + readable_roots=(tmp_path,), writable_roots=(tmp_path,), ), - config_service=config, - compile_inspection_gateway=gateway, - ) - built = build_server( - SimpleNamespace(artifact_plan_service=service), - main_thread_dispatcher=executor.dispatcher, - ) - tool = built._tool_manager.get_tool( - InspectPipelineSourceArtifactPlanCapability.name + config_service=config, compile_inspection_gateway=gateway, ) + built = build_server(SimpleNamespace(artifact_plan_service=service), + main_thread_dispatcher=executor.dispatcher) + tool = built._tool_manager.get_tool(InspectPipelineSourceArtifactPlanCapability.name) async def exercise(): token = identity.set("original-request") try: - return await tool.fn( - mcp_context=StageContext(), - plate_path=str(tmp_path), - pipeline_source="pipeline_steps = []", - ) + return await tool.fn(mcp_context=StageContext(), plate_path=str(tmp_path), + pipeline_source="pipeline_steps = []") finally: identity.reset(token) before = len(messages) result = executor.run(exercise) observed = messages[before:] - assert any( - "Preparing source inspection compiler" in item for item in observed - ) + assert any("Preparing source inspection compiler" in item for item in observed) if fails: assert result["errors"][0]["exception_type"] == "ValueError" assert result["errors"][0]["message"] == str(error) - assert any( - "Source inspection compiler failed" in item for item in observed - ) + assert any("Source inspection compiler failed" in item for item in observed) assert not any("projection completed" in item for item in observed) else: assert result["errors"] == [] @@ -175,12 +145,9 @@ async def exercise(): def test_real_inventory_binding_relays_shared_reader_preparation(tmp_path): plate = ImageXpressPlateFixture.write(tmp_path) - service = PlateInspectionService( - AgentPathPolicy.with_roots( - readable_roots=(tmp_path,), - writable_roots=(), - ) - ) + service = PlateInspectionService(AgentPathPolicy.with_roots( + readable_roots=(tmp_path,), writable_roots=(), + )) messages = [] class StageContext: @@ -191,19 +158,13 @@ async def report_progress(self, progress, total=None, message=None): messages.append(message) executor = McpTransportExecutor() - built = build_server( - SimpleNamespace(plate_inspection_service=service), - main_thread_dispatcher=executor.dispatcher, - ) + built = build_server(SimpleNamespace(plate_inspection_service=service), + main_thread_dispatcher=executor.dispatcher) tool = built._tool_manager.get_tool(QueryPlateFilesCapability.name) async def exercise(): - return await tool.fn( - mcp_context=StageContext(), - plate_path=str(plate), - source_format="imagexpress", - include_previews=False, - ) + return await tool.fn(mcp_context=StageContext(), plate_path=str(plate), + source_format="imagexpress", include_previews=False) try: result = executor.run(exercise) diff --git a/tests/unit/agent/test_compile_inspection_declared_callable.py b/tests/unit/agent/test_compile_inspection_declared_callable.py index bd6da4ab18..24c2847bfd 100644 --- a/tests/unit/agent/test_compile_inspection_declared_callable.py +++ b/tests/unit/agent/test_compile_inspection_declared_callable.py @@ -16,61 +16,34 @@ from openhcs.processing.backends.lib_registry.registry_service import RegistryService -def test_real_declared_callable_compiles_without_global_catalog( - monkeypatch, tmp_path: Path -): +def test_real_declared_callable_compiles_without_global_catalog(monkeypatch, tmp_path: Path): def forbidden_catalog(*args, **kwargs): raise AssertionError("A declared callable must not prepare unrelated catalogs") - monkeypatch.setattr( - RegistryService, "get_all_functions_with_metadata", forbidden_catalog - ) + monkeypatch.setattr(RegistryService, "get_all_functions_with_metadata", forbidden_catalog) plate = tmp_path / "plate" SyntheticMicroscopyGenerator( - output_dir=str(plate), - grid_size=(1, 1), - tile_size=(64, 64), - wavelengths=1, - z_stack_levels=1, - wells=["A01"], - num_cells=3, - random_seed=123, - include_all_components=True, + output_dir=str(plate), grid_size=(1, 1), tile_size=(64, 64), + wavelengths=1, z_stack_levels=1, wells=["A01"], num_cells=3, + random_seed=123, include_all_components=True, ).generate_dataset() document = PipelineDocumentCodec.from_values( pipeline_config=PipelineConfig(num_workers=1), - pipeline_steps=[ - FunctionStep( - func=( - reducenoise, - { - "patch_size": 3, - "patch_distance": 3, - "cutoff_distance": 0.1, - }, - ), - name="DeclaredNlm", - ) - ], - ) - compiled = InProcessCompileInspectionGateway().compile( - CompileInspectionInput( - plate=plate, - pipeline_document=document, - axis_filter=(), - global_pipeline_config=GlobalPipelineConfig(num_workers=1), - progress_queue=AgentProgressQueue(), - ) + pipeline_steps=[FunctionStep(func=(reducenoise, { + "patch_size": 3, "patch_distance": 3, "cutoff_distance": 0.1, + }), name="DeclaredNlm")], ) + compiled = InProcessCompileInspectionGateway().compile(CompileInspectionInput( + plate=plate, pipeline_document=document, axis_filter=(), + global_pipeline_config=GlobalPipelineConfig(num_workers=1), + progress_queue=AgentProgressQueue(), + )) assert tuple(compiled.execution_bundle.runtime_contexts) == ("A01",) context = compiled.execution_bundle.runtime_contexts["A01"] assert len(context.step_plans) == 1 assert next(iter(context.step_plans.values())).step_name == "DeclaredNlm" inspection = artifact_plan_inspection_from_compilation( - plate_path=str(plate), - axis_filter=(), - compilation=compiled, - progress_event_count=0, - warnings=(), + plate_path=str(plate), axis_filter=(), compilation=compiled, + progress_event_count=0, warnings=(), ) assert inspection.source_workspace.file_count == 1 diff --git a/tests/unit/agent/test_compile_selector_authoring.py b/tests/unit/agent/test_compile_selector_authoring.py index 54ce7a017c..a937944aa3 100644 --- a/tests/unit/agent/test_compile_selector_authoring.py +++ b/tests/unit/agent/test_compile_selector_authoring.py @@ -302,7 +302,9 @@ class NewAuthoringModule(CellProfilerModule, ExtraOutputSettings): assert len(kwargs) == 2 service, ref = draft(new_authoring_callable, kwargs) assert service.validate(ref).valid - restored = PipelineDocumentCodec.from_source(service.render_source(ref).source) + restored = PipelineDocumentCodec.from_source( + service.render_source(ref).source + ) invocation = next( normalize_function_pattern(restored.pipeline_steps[0].func).iter_items() ) diff --git a/tests/unit/agent/test_declared_callable_documentation.py b/tests/unit/agent/test_declared_callable_documentation.py index f246e9ee5b..e21dd0afc5 100644 --- a/tests/unit/agent/test_declared_callable_documentation.py +++ b/tests/unit/agent/test_declared_callable_documentation.py @@ -57,9 +57,7 @@ def test_installed_witness_declaration_doc_and_parameter_contract_agree(monkeypa @pytest.mark.parametrize("contract", (Pure2DContract, Pure3DContract)) -def test_new_declaration_preserves_authored_prose_without_consumer_edits( - monkeypatch, contract -): +def test_new_declaration_preserves_authored_prose_without_consumer_edits(monkeypatch, contract): @numpy_contract(contract=contract) def independent_doc_probe(image: np.ndarray, *, offset: float = 0.75) -> np.ndarray: """Authored plane/volume operation. @@ -75,50 +73,35 @@ def independent_doc_probe(image: np.ndarray, *, offset: float = 0.75) -> np.ndar """ return image + offset - catalog, metadata = _catalog( - monkeypatch, project_custom_function(independent_doc_probe) - ) + catalog, metadata = _catalog(monkeypatch, project_custom_function(independent_doc_probe)) detail = catalog.get(metadata.composite_key, max_doc_chars=None) assert detail.doc == inspect.getdoc(inspect.unwrap(independent_doc_probe)) assert "Additional Parameters is an authored heading" in detail.doc assert "discussion of slice_by_slice" in detail.doc assert "slice_by_slice" not in {parameter.name for parameter in detail.parameters} - offset = next( - parameter for parameter in detail.parameters if parameter.name == "offset" - ) + offset = next(parameter for parameter in detail.parameters if parameter.name == "offset") assert offset.default_repr == "0.75" assert offset.supplied_by is FunctionParameterSource.AGENT assert offset.description == "Declared intensity offset in original image units." assert catalog.search(query="original image units").total == 1 -def test_flexible_controls_remain_typed_and_existing_mro_behavior_is_unchanged( - monkeypatch, -): +def test_flexible_controls_remain_typed_and_existing_mro_behavior_is_unchanged(monkeypatch): seen = [] @numpy_contract - def independent_flexible_doc_probe( - image: np.ndarray, *, offset: float = 0.5 - ) -> np.ndarray: + def independent_flexible_doc_probe(image: np.ndarray, *, offset: float = 0.5) -> np.ndarray: """Apply an offset with declaration-owned dimensional selection.""" seen.append(image.shape) return image + offset - catalog, metadata = _catalog( - monkeypatch, project_custom_function(independent_flexible_doc_probe) - ) + catalog, metadata = _catalog(monkeypatch, project_custom_function(independent_flexible_doc_probe)) detail = catalog.get(metadata.composite_key, max_doc_chars=None) assert detail.doc == inspect.getdoc(inspect.unwrap(independent_flexible_doc_probe)) parameters = {parameter.name: parameter for parameter in detail.parameters} assert parameters["slice_by_slice"].default_repr == "False" - assert ( - "Process 3D arrays slice-by-slice" in parameters["slice_by_slice"].description - ) - assert ( - parameters["dtype_config"].supplied_by - is FunctionParameterSource.RUNTIME_PARAMETER - ) + assert "Process 3D arrays slice-by-slice" in parameters["slice_by_slice"].description + assert parameters["dtype_config"].supplied_by is FunctionParameterSource.RUNTIME_PARAMETER pixels = np.zeros((2, 4, 6)) result = metadata.func(pixels, slice_by_slice=True) assert seen == [(4, 6), (4, 6)] @@ -135,9 +118,7 @@ class IndependentDocRequest: offset: float = 0.5 -def test_nominal_request_boundary_is_not_unwrapped_to_private_implementation( - monkeypatch, -): +def test_nominal_request_boundary_is_not_unwrapped_to_private_implementation(monkeypatch): @callable_request(IndependentDocRequest) def implementation(request: IndependentDocRequest): """Private implementation prose.""" @@ -159,23 +140,14 @@ def no_doc(image): policy = PARAMETER_DOCUMENTATION_POLICY assert policy.detail_doc(no_doc, None) is None assert policy.detail_doc(no_doc, " ") is None - assert ( - policy.detail_doc(no_doc, "External metadata documentation.") - == "External metadata documentation." - ) + assert policy.detail_doc(no_doc, "External metadata documentation.") == "External metadata documentation." -def test_cellprofiler_setting_help_remains_on_final_typed_parameter_projection( - monkeypatch, -): +def test_cellprofiler_setting_help_remains_on_final_typed_parameter_projection(monkeypatch): from openhcs.interop.cellprofiler.module_declarations import CellProfilerModule - from openhcs.processing.backends.cellprofiler.classification import ( - classify_objects_single_measurement, - ) + from openhcs.processing.backends.cellprofiler.classification import classify_objects_single_measurement - module = CellProfilerModule.for_callable_contract( - CallableContract.from_callable(classify_objects_single_measurement) - ) + module = CellProfilerModule.for_callable_contract(CallableContract.from_callable(classify_objects_single_measurement)) assert module is not None implementation = module.require_callable() catalog, metadata = _catalog( @@ -195,12 +167,6 @@ def test_cellprofiler_setting_help_remains_on_final_typed_parameter_projection( def test_doc_projection_has_no_leaf_control_dispatch_or_text_surgery(): tree = ast.parse(inspect.getsource(type(PARAMETER_DOCUMENTATION_POLICY))) - literals = { - node.value - for node in ast.walk(tree) - if isinstance(node, ast.Constant) and isinstance(node.value, str) - } + literals = {node.value for node in ast.walk(tree) if isinstance(node, ast.Constant) and isinstance(node.value, str)} assert {"slice_by_slice", "PURE_2D", "PURE_3D", "FLEXIBLE"}.isdisjoint(literals) - assert "resolve_raw_runtime_callable" in inspect.getsource( - type(PARAMETER_DOCUMENTATION_POLICY).detail_doc - ) + assert "resolve_raw_runtime_callable" in inspect.getsource(type(PARAMETER_DOCUMENTATION_POLICY).detail_doc) diff --git a/tests/unit/agent/test_function_declaration_help.py b/tests/unit/agent/test_function_declaration_help.py index 3150267600..9d749e2188 100644 --- a/tests/unit/agent/test_function_declaration_help.py +++ b/tests/unit/agent/test_function_declaration_help.py @@ -249,7 +249,8 @@ def denied_preparation(*args, **kwargs): ) assert isinstance(metadata, FunctionMetadata) assert ( - metadata.composite_key == f"openhcs:analysis_neurite_outgrowth_{func.__name__}" + metadata.composite_key + == f"openhcs:analysis_neurite_outgrowth_{func.__name__}" ) assert metadata.func is not func monkeypatch.setattr( @@ -275,14 +276,11 @@ def denied_preparation(*args, **kwargs): docstring_info_for_target(analyzed[parameter.name].param_type) ) assert document.content in parameter.description - assert ( - next( - value["description"] - for value in to_jsonable(detail)["parameters"] - if value["name"] == parameter.name - ) - == parameter.description - ) + assert next( + value["description"] + for value in to_jsonable(detail)["parameters"] + if value["name"] == parameter.name + ) == parameter.description assert not detail.doc_truncated import json diff --git a/tests/unit/agent/test_imagexpress_readonly_paths.py b/tests/unit/agent/test_imagexpress_readonly_paths.py index 3f552e9099..381d36b9d5 100644 --- a/tests/unit/agent/test_imagexpress_readonly_paths.py +++ b/tests/unit/agent/test_imagexpress_readonly_paths.py @@ -67,9 +67,7 @@ def inspect_plate(plate: Path, source_format="imagexpress", **bounds): def inventory_for(plate: Path): ensure_storage_registry() filemanager = FileManager(dict(storage_registry)) - handler = DatasetSourceChoice.named("imagexpress").open( - plate, filemanager=filemanager - ) + handler = DatasetSourceChoice.named("imagexpress").open(plate, filemanager=filemanager) return ( handler, filemanager, @@ -83,10 +81,9 @@ def inventory_for(plate: Path): def identity(record): - return ( - frozenset((c, record.metadata[c.name]) for c in AxisFamily.active().axes), - record.source_path, - ) + return frozenset( + (c, record.metadata[c.name]) for c in AxisFamily.active().axes + ), record.source_path def test_raw_inspection_retains_folder_z_and_never_initializes(tmp_path: Path): @@ -271,9 +268,7 @@ def image_path_components(self, path): assert result.image_files.count == result.parse_summary.parsed_file_count == 4 summaries = {item.component: item for item in result.components} assert tuple(v.key for v in summaries[Microscopy.Site.name].values) == ("7",) - assert tuple(v.key for v in summaries[Microscopy.Timepoint.name].values) == ( - "3", - ) + assert tuple(v.key for v in summaries[Microscopy.Timepoint.name].values) == ("3",) assert summaries[Microscopy.ZIndex.name].count == 2 query = service_for(plate).query_files( PlateFileQueryRequest.from_fields( diff --git a/tests/unit/agent/test_knowledge_base_service.py b/tests/unit/agent/test_knowledge_base_service.py index a9a6a3acd7..0274bb487c 100644 --- a/tests/unit/agent/test_knowledge_base_service.py +++ b/tests/unit/agent/test_knowledge_base_service.py @@ -128,7 +128,8 @@ def test_search_prefers_declared_document_matches_over_equal_body_matches(tmp_pa (docs / "notes.md").write_text( "# Setup notes\n\n" + "\n".join( - f"## Note {index}\n\nThis body mentions a recipe.\n" for index in range(8) + f"## Note {index}\n\nThis body mentions a recipe.\n" + for index in range(8) ), encoding="utf-8", ) diff --git a/tests/unit/agent/test_mcp_affine_inspection_progress.py b/tests/unit/agent/test_mcp_affine_inspection_progress.py index 50ac581510..a1e9a9bf61 100644 --- a/tests/unit/agent/test_mcp_affine_inspection_progress.py +++ b/tests/unit/agent/test_mcp_affine_inspection_progress.py @@ -19,26 +19,19 @@ from openhcs.agent.dto.common import AgentError, SCHEMA_VERSION from openhcs.agent.dto.execution import ArtifactPlanInspection from openhcs.mcp.execution import ( - McpMainThreadDispatcher, - McpQueuedCallNotStartedError, - McpTransportExecutor, + McpMainThreadDispatcher, McpQueuedCallNotStartedError, McpTransportExecutor, ) from pyqt_reactive.services.ui_thread_dispatch import ( - UiThreadDispatchError, - UiThreadDispatcherClosedError, - UiThreadDispatchTimeoutError, + UiThreadDispatchError, UiThreadDispatcherClosedError, UiThreadDispatchTimeoutError, ) from openhcs.mcp.server import build_server from python_introspect import to_jsonable -@pytest.mark.parametrize( - "declaration", - ( - InspectPipelineSourceArtifactPlanCapability, - GetKnowledgeDocumentCapability, - ), -) +@pytest.mark.parametrize("declaration", ( + InspectPipelineSourceArtifactPlanCapability, + GetKnowledgeDocumentCapability, +)) def test_source_leaf_composes_original_progress_and_affinity(declaration): assert issubclass(declaration, MainThreadProgressCapability) spec = declaration @@ -65,9 +58,7 @@ def inspect(self, request): EndpointStartupPhase.PREPARING_CAPABILITIES, "Source inspection on original main thread", ).publish() - assert ( - self.wait_for_completion() - ), "SDK heartbeat was starved by main-affine work" + assert self.wait_for_completion(), "SDK heartbeat was starved by main-affine work" if self.error is not None: raise self.error return ArtifactPlanInspection( @@ -92,9 +83,7 @@ async def report_progress(self, progress, total=None, message=None): self.service.release.set() -@pytest.mark.parametrize( - "error", (None, ValueError("original"), asyncio.CancelledError("original")) -) +@pytest.mark.parametrize("error", (None, ValueError("original"), asyncio.CancelledError("original"))) def test_generated_inspection_main_thread_context_progress_and_terminal(error): identity = ContextVar("affine-inspection-identity", default="outside") service = AffineInspectionService(identity, error=error) @@ -103,9 +92,7 @@ def test_generated_inspection_main_thread_context_progress_and_terminal(error): SimpleNamespace(artifact_plan_service=service), main_thread_dispatcher=executor.dispatcher, ) - tool = built._tool_manager.get_tool( - InspectPipelineSourceArtifactPlanCapability.name - ) + tool = built._tool_manager.get_tool(InspectPipelineSourceArtifactPlanCapability.name) context = InspectionProgressContext(service) async def exercise(): @@ -113,18 +100,10 @@ async def exercise(): try: if isinstance(error, asyncio.CancelledError): with pytest.raises(asyncio.CancelledError) as caught: - await tool.fn( - mcp_context=context, - plate_path="/synthetic", - pipeline_source="pipeline_steps = []", - ) + await tool.fn(mcp_context=context, plate_path="/synthetic", pipeline_source="pipeline_steps = []") assert caught.value is error return None - return await tool.fn( - mcp_context=context, - plate_path="/synthetic", - pipeline_source="pipeline_steps = []", - ) + return await tool.fn(mcp_context=context, plate_path="/synthetic", pipeline_source="pipeline_steps = []") finally: identity.reset(token) @@ -135,24 +114,16 @@ async def exercise(): assert len(service.calls) == 1 assert service.calls[0][1] == "original-request" assert identity.get() == "outside" - assert any( - "Source inspection on original main thread" in message - for message in context.messages - ) + assert any("Source inspection on original main thread" in message for message in context.messages) assert any("still running" in message for message in context.messages) if error is None: assert result["plate_path"] == "/synthetic" assert result["errors"] == [] elif isinstance(error, ValueError): - assert result["errors"] == [ - to_jsonable( - AgentError.from_exception( - "mcp_tool_failed", - error, - hint="The MCP server caught this exception at the tool boundary.", - ) - ) - ] + assert result["errors"] == [to_jsonable(AgentError.from_exception( + "mcp_tool_failed", error, + hint="The MCP server caught this exception at the tool boundary.", + ))] def test_independent_new_leaf_cooperative_hooks_need_no_consumer_edits(): @@ -193,31 +164,20 @@ class NewAfter(InspectPipelineSourceArtifactPlanCapability): identity = ContextVar("new-affine-declaration", default="new-case") service = AffineInspectionService(identity) executor = McpTransportExecutor() - built = build_server( - SimpleNamespace(artifact_plan_service=service), - main_thread_dispatcher=executor.dispatcher, - ) + built = build_server(SimpleNamespace(artifact_plan_service=service), main_thread_dispatcher=executor.dispatcher) async def exercise(): for declaration in (NewBefore, NewAfter): service.release.clear() context = InspectionProgressContext(service) tool = built._tool_manager.get_tool(declaration.name) - result = await tool.fn( - mcp_context=context, - plate_path="/new-case", - pipeline_source="pipeline_steps = []", - ) + result = await tool.fn(mcp_context=context, plate_path="/new-case", pipeline_source="pipeline_steps = []") assert result["plate_path"] == "/new-case" assert any("still running" in message for message in context.messages) try: executor.run(exercise) - assert calls == [ - (stage, declaration.name) - for declaration in (NewBefore, NewAfter) - for stage in ("audit-enter", "audit-exit") - ] + assert calls == [(stage, declaration.name) for declaration in (NewBefore, NewAfter) for stage in ("audit-enter", "audit-exit")] assert len(service.calls) == 2 finally: executor.close() @@ -259,11 +219,7 @@ async def exercise(): assert await asyncio.to_thread(entered.wait, 1) try: with pytest.raises(McpQueuedCallNotStartedError) as cancelled: - await asyncio.to_thread( - executor.dispatcher.call, - lambda: calls.append("forbidden"), - timeout_ms=1, - ) + await asyncio.to_thread(executor.dispatcher.call, lambda: calls.append("forbidden"), timeout_ms=1) assert cancelled.value.to_agent_error().code == "mcp_call_not_started" finally: second_finished.set() @@ -384,11 +340,7 @@ def wait_for_completion(self): return True def inspect(self, request): - self.error = ( - ValueError("Original controlled source error") - if request.pipeline_source == "failed" - else None - ) + self.error = ValueError("Original controlled source error") if request.pipeline_source == "failed" else None return super().inspect(request) @@ -397,30 +349,18 @@ def serve_stdio_inspection_fixture(*, work_seconds=2.4): identity = ContextVar("wire-source-identity", default="wire") with McpStdioTransport.reserve_process_stdio() as transport: - transport.run( - build_server( - SimpleNamespace( - artifact_plan_service=WireInspectionService( - identity, work_seconds=work_seconds - ) - ), - main_thread_dispatcher=transport.execution.dispatcher, - ) - ) + transport.run(build_server( + SimpleNamespace(artifact_plan_service=WireInspectionService(identity, work_seconds=work_seconds)), + main_thread_dispatcher=transport.execution.dispatcher, + )) -@pytest.mark.parametrize( - "work_seconds,idle_seconds,connection_count", - ( - pytest.param(2.4, 1.6, 2, id="original-continuous"), - pytest.param(12.0, 10.0, 1, id="ordinary-ten-second-idle"), - ), -) +@pytest.mark.parametrize("work_seconds,idle_seconds,connection_count", ( + pytest.param(2.4, 1.6, 2, id="original-continuous"), + pytest.param(12.0, 10.0, 1, id="ordinary-ten-second-idle"), +)) def test_resident_continuous_connections_keep_affinity_and_wire_progress( - unix_socket_dir, - work_seconds, - idle_seconds, - connection_count, + unix_socket_dir, work_seconds, idle_seconds, connection_count, ): from pyqt_reactive.services.async_operation_executor import AsyncOperationExecutor from openhcs.mcp.dev_client_core import McpDevServerSpec, McpDevSocketSession @@ -430,13 +370,9 @@ def test_resident_continuous_connections_keep_affinity_and_wire_progress( transport = McpSocketTransport(unix_socket_dir / "affine.sock") service = WireInspectionService( - ContextVar("resident-source-identity", default="resident"), - work_seconds=work_seconds, - ) - built = build_server( - SimpleNamespace(artifact_plan_service=service), - main_thread_dispatcher=transport.execution.dispatcher, + ContextVar("resident-source-identity", default="resident"), work_seconds=work_seconds, ) + built = build_server(SimpleNamespace(artifact_plan_service=service), main_thread_dispatcher=transport.execution.dispatcher) clients = AsyncOperationExecutor(max_workers=1) diagnostics = io.StringIO() @@ -448,14 +384,10 @@ def record_progress_notification(self, notification): progress = [] async def exercise(): - assert await asyncio.to_thread( - wait_for_socket, transport.socket_path, timeout_seconds=10 - ) + assert await asyncio.to_thread(wait_for_socket, transport.socket_path, timeout_seconds=10) try: for connection in range(connection_count): - async with ObservedSession( - McpDevServerSpec(sys.executable), diagnostics, transport.socket_path - ) as session: + async with ObservedSession(McpDevServerSpec(sys.executable), diagnostics, transport.socket_path) as session: await session.initialize(timeout_seconds=10) for source in ("cold", "failed"): started = time.monotonic() @@ -469,24 +401,13 @@ async def exercise(): elapsed = time.monotonic() - started assert elapsed >= work_seconds > idle_seconds events = progress[before:] - assert events and all( - token == request_id for _, token in events - ) + assert events and all(token == request_id for _, token in events) assert events[0][0] - started < 1 assert events[-1][0] - started > idle_seconds - print( - "RESIDENT_AFFINE_TERMINAL", - source, - elapsed, - idle_seconds, - flush=True, - ) + print("RESIDENT_AFFINE_TERMINAL", source, elapsed, idle_seconds, flush=True) payload = result["structuredContent"] if source == "failed": - assert ( - payload["errors"][0]["message"] - == "Original controlled source error" - ) + assert payload["errors"][0]["message"] == "Original controlled source error" else: assert payload["plate_path"] == "/synthetic" assert payload["errors"] == [] diff --git a/tests/unit/agent/test_mcp_affine_real_compile.py b/tests/unit/agent/test_mcp_affine_real_compile.py index 403b225e73..aed75c5e99 100644 --- a/tests/unit/agent/test_mcp_affine_real_compile.py +++ b/tests/unit/agent/test_mcp_affine_real_compile.py @@ -11,8 +11,7 @@ from openhcs.agent.path_policy import AgentPathPolicy from openhcs.agent.services.config_service import ConfigService from openhcs.agent.services.artifact_plan_inspection_service import ( - ArtifactPlanInspectionService, - InProcessCompileInspectionGateway, + ArtifactPlanInspectionService, InProcessCompileInspectionGateway, ) from openhcs.demo.synthetic_data import SyntheticMicroscopyGenerator from openhcs.mcp.execution import McpTransportExecutor @@ -20,26 +19,16 @@ from openhcs.processing.backends.lib_registry.registry_service import RegistryService -def test_real_generated_source_inspection_keeps_original_compiler_on_main( - monkeypatch, tmp_path: Path -): +def test_real_generated_source_inspection_keeps_original_compiler_on_main(monkeypatch, tmp_path: Path): def forbidden_catalog(*args, **kwargs): raise AssertionError("Declared compilation must not prepare unrelated catalogs") - monkeypatch.setattr( - RegistryService, "get_all_functions_with_metadata", forbidden_catalog - ) + monkeypatch.setattr(RegistryService, "get_all_functions_with_metadata", forbidden_catalog) plate = tmp_path / "staged-source" SyntheticMicroscopyGenerator( - output_dir=str(plate), - grid_size=(1, 1), - tile_size=(64, 64), - wavelengths=1, - z_stack_levels=1, - wells=["A01"], - num_cells=3, - random_seed=123, - include_all_components=True, + output_dir=str(plate), grid_size=(1, 1), tile_size=(64, 64), + wavelengths=1, z_stack_levels=1, wells=["A01"], num_cells=3, + random_seed=123, include_all_components=True, ).generate_dataset() # This is a declared-module source fixture, not a catalog-cold/render proof. # The installed cold canonical journey remains parent-owned. @@ -73,29 +62,18 @@ async def report_progress(self, *args, **kwargs): config = ConfigService() service = ArtifactPlanInspectionService( - path_policy=AgentPathPolicy.with_roots( - readable_roots=(tmp_path,), writable_roots=(tmp_path,) - ), + path_policy=AgentPathPolicy.with_roots(readable_roots=(tmp_path,), writable_roots=(tmp_path,)), config_service=config, compile_inspection_gateway=AffineCompileGateway(), ) executor = McpTransportExecutor() - built = build_server( - SimpleNamespace(artifact_plan_service=service), - main_thread_dispatcher=executor.dispatcher, - ) - tool = built._tool_manager.get_tool( - InspectPipelineSourceArtifactPlanCapability.name - ) + built = build_server(SimpleNamespace(artifact_plan_service=service), main_thread_dispatcher=executor.dispatcher) + tool = built._tool_manager.get_tool(InspectPipelineSourceArtifactPlanCapability.name) async def exercise(): token = identity.set("exact-source-request") try: - return await tool.fn( - mcp_context=ProgressContext(), - plate_path=str(plate), - pipeline_source=source, - ) + return await tool.fn(mcp_context=ProgressContext(), plate_path=str(plate), pipeline_source=source) finally: identity.reset(token) @@ -118,10 +96,8 @@ async def exercise(): "Projecting compiled source workspace", "Source inspection compilation and source projection completed", ) - positions = [ - next(index for index, message in enumerate(messages) if stage in message) - for stage in stages - ] + positions = [next(index for index, message in enumerate(messages) if stage in message) + for stage in stages] assert positions == sorted(positions), messages assert len(calls[0].progress_queue.events) == 1 assert calls[0].progress_queue.events[0].axis_id == "A01" diff --git a/tests/unit/agent/test_mcp_dev_client_config_renderer.py b/tests/unit/agent/test_mcp_dev_client_config_renderer.py index 405490e95f..320611f1ad 100644 --- a/tests/unit/agent/test_mcp_dev_client_config_renderer.py +++ b/tests/unit/agent/test_mcp_dev_client_config_renderer.py @@ -67,35 +67,25 @@ def _schema() -> ConfigSchema: enum_values=("channel", "z_index"), ), ), - registries=( - ConfigRegistrySchema(owner_type="PipelineConfig", registered_types=()), - ), - types=( - ConfigTypeSchema( - type_repr="LazyPartitionFilterConfig", description="Wells." - ), - ), + registries=(ConfigRegistrySchema(owner_type="PipelineConfig", registered_types=()),), + types=(ConfigTypeSchema(type_repr="LazyPartitionFilterConfig", description="Wells."),), ) -def test_config_schema_renderer_is_registered_for_the_contract_and_its_subtypes() -> ( - None -): +def test_config_schema_renderer_is_registered_for_the_contract_and_its_subtypes() -> None: @dataclass(frozen=True, slots=True) class AnnotatedConfigSchema(ConfigSchema): annotation: str = "new declaration" - assert ( - McpDevOutputRenderer.for_output_contract(ConfigSchema) is ConfigSchemaRenderer - ) + assert McpDevOutputRenderer.for_output_contract(ConfigSchema) is ConfigSchemaRenderer assert ( McpDevOutputRenderer.for_output_contract(AnnotatedConfigSchema) is ConfigSchemaRenderer ) assert ConfigSchemaRenderer.render_options_type is CatalogRenderOptions - annotated = AnnotatedConfigSchema( - **{name: getattr(_schema(), name) for name in ConfigSchema.__dataclass_fields__} - ) + annotated = AnnotatedConfigSchema(**{ + name: getattr(_schema(), name) for name in ConfigSchema.__dataclass_fields__ + }) assert "flags=optional,lazy,inheritable" in ConfigSchemaRenderer.render( _batch(annotated) ) @@ -192,14 +182,8 @@ def test_config_schema_command_reads_render_options_from_its_renderer() -> None: parser = _build_parser() args = parser.parse_args( ( - "config-schema", - "pipeline", - "--path-prefix", - "fiji_streaming_config", - "--contains", - "lazy", - "--limit", - "1", + "config-schema", "pipeline", "--path-prefix", "fiji_streaming_config", + "--contains", "lazy", "--limit", "1", ) ) call = _calls_from_args(args)[0] diff --git a/tests/unit/agent/test_mcp_dev_client_pipeline_results.py b/tests/unit/agent/test_mcp_dev_client_pipeline_results.py index 776c547b53..bb4bb7f9e2 100644 --- a/tests/unit/agent/test_mcp_dev_client_pipeline_results.py +++ b/tests/unit/agent/test_mcp_dev_client_pipeline_results.py @@ -85,48 +85,25 @@ def batch(capability, value): ) -@pytest.mark.parametrize( - "registered_count, code", - ( - (0, "custom_function_registration_uncertain"), - (1, "custom_function_local_projection_failed"), - ), -) -def test_registration_renderer_retains_typed_error_receipt_and_handle( - registered_count, code -): +@pytest.mark.parametrize("registered_count, code", ( + (0, "custom_function_registration_uncertain"), + (1, "custom_function_local_projection_failed"), +)) +def test_registration_renderer_retains_typed_error_receipt_and_handle(registered_count, code): from openhcs.agent.dto.execution_connection import ExecutionConnectionSpec - from openhcs.agent.dto.functions import ( - CustomFunctionRegistrationHandle, - CustomFunctionRegistrationResult, - ) - from openhcs.mcp.dev_client_renderers.knowledge import ( - CustomFunctionRegistrationRenderer, - ) + from openhcs.agent.dto.functions import CustomFunctionRegistrationHandle, CustomFunctionRegistrationResult + from openhcs.mcp.dev_client_renderers.knowledge import CustomFunctionRegistrationRenderer from zmqruntime.messages import ProcessIdentity handle = CustomFunctionRegistrationHandle( - ExecutionConnectionSpec(port=15993), - ProcessIdentity.current(), - "a" * 64, - "engineering_registration", - False, - None, + ExecutionConnectionSpec(port=15993), ProcessIdentity.current(), + "a" * 64, "engineering_registration", False, None, ) original = CustomFunctionRegistrationResult( - schema_version=SCHEMA_VERSION, - registered_count=registered_count, - connection=handle.connection, - server_identity=handle.server_identity, + schema_version=SCHEMA_VERSION, registered_count=registered_count, + connection=handle.connection, server_identity=handle.server_identity, observation_handle=handle, - errors=( - AgentError( - code=code, - message="original cause", - hint="Do not replay.", - exception_type="ValueError", - ), - ), + errors=(AgentError(code=code, message="original cause", hint="Do not replay.", exception_type="ValueError"),), ) response = batch(agent_capabilities.register_custom_function, original) decoded = McpDevToolBatchResponse.for_rendering(to_jsonable(response)) @@ -138,44 +115,24 @@ def test_registration_renderer_retains_typed_error_receipt_and_handle( assert text.count(f"{code}: original cause") == 1 assert "Do not replay." in text marker = "Read-only observation handle: " - rendered_handle = next( - line.removeprefix(marker) - for line in text.splitlines() - if line.startswith(marker) - ) + rendered_handle = next(line.removeprefix(marker) for line in text.splitlines() if line.startswith(marker)) assert json.loads(rendered_handle) == to_jsonable(handle) def test_registration_renderer_uses_declared_success_entries_without_mapping_reads(): - from openhcs.agent.dto.functions import ( - CustomFunctionRegistrationResult, - FunctionCatalogEntry, - ) - from openhcs.mcp.dev_client_renderers.knowledge import ( - CustomFunctionRegistrationRenderer, - ) + from openhcs.agent.dto.functions import CustomFunctionRegistrationResult, FunctionCatalogEntry + from openhcs.mcp.dev_client_renderers.knowledge import CustomFunctionRegistrationRenderer entry = FunctionCatalogEntry( - function_id="openhcs:engineering_registration", - import_path="openhcs.processing.custom_functions.engineering_registration", - name="engineering_registration", - module="openhcs.processing.custom_functions", - library="openhcs", - signature="engineering_registration(image)", - summary="Original declared summary", - backend_tags=("numpy",), + function_id="openhcs:engineering_registration", import_path="openhcs.processing.custom_functions.engineering_registration", + name="engineering_registration", module="openhcs.processing.custom_functions", library="openhcs", + signature="engineering_registration(image)", summary="Original declared summary", backend_tags=("numpy",), ) original = CustomFunctionRegistrationResult( - schema_version=SCHEMA_VERSION, - registered_count=1, - persisted=False, - storage_dir="/engineering", - source_file_paths=("/engineering/original.py",), - functions=(entry,), - ) - text = CustomFunctionRegistrationRenderer.render( - to_jsonable(batch(agent_capabilities.register_custom_function, original)) + schema_version=SCHEMA_VERSION, registered_count=1, persisted=False, + storage_dir="/engineering", source_file_paths=("/engineering/original.py",), functions=(entry,), ) + text = CustomFunctionRegistrationRenderer.render(to_jsonable(batch(agent_capabilities.register_custom_function, original))) assert "registered=1 persisted=False storage=/engineering" in text assert "Files: /engineering/original.py" in text assert f"{entry.function_id}: {entry.signature} tags=numpy" in text @@ -378,47 +335,29 @@ def test_nested_artifact_contract_and_receipt_survive_compact_limits(): def _dataset_row(**changes) -> DatasetRowState: values = dict( - scope_id="/plate", - name="plate", - root="/plate", - pipeline_path=None, - selected=True, - initialized=False, - compiled=False, - init_pending=False, - compile_pending=False, - execution_active=False, - status_prefix="", - orchestrator_state=None, - execution_id=None, - terminal_status=None, - runtime_state=None, - runtime_percent=None, - queue_position=None, + scope_id="/plate", name="plate", root="/plate", pipeline_path=None, + selected=True, initialized=False, compiled=False, init_pending=False, + compile_pending=False, execution_active=False, status_prefix="", + orchestrator_state=None, execution_id=None, terminal_status=None, + runtime_state=None, runtime_percent=None, queue_position=None, ) values.update(changes) return DatasetRowState(**values) def _datasets(row: DatasetRowState) -> DatasetListState: - return DatasetListState( - rows=(row,), selected_scope_ids=(row.scope_id,), execution_state="idle" - ) + return DatasetListState(rows=(row,), selected_scope_ids=(row.scope_id,), execution_state="idle") @pytest.mark.parametrize("waited", [True, False]) def test_execute_journey_follows_session_stages_and_renders_each_result(waited): def result(operation_id, status, **changes): - return SessionOperationResult( - operation_id=operation_id, status=status, **changes - ) + return SessionOperationResult(operation_id=operation_id, status=status, **changes) initialized = _dataset_row(initialized=True) compiled = _dataset_row(initialized=True, compiled=True) failed = _dataset_row( - initialized=True, - compiled=True, - terminal_status="failed", + initialized=True, compiled=True, terminal_status="failed", orchestrator_state="exec_failed", ) receipts = [ @@ -453,15 +392,11 @@ def result(operation_id, status, **changes): agent_capabilities.initialize_datasets.name, ] assert peer.calls[0][1]["port"] == 15993 - assert peer.calls[2][1] == { - "scope_id": "/plate", - "pipeline_source": "pipeline_steps = []", - } + assert peer.calls[2][1] == {"scope_id": "/plate", "pipeline_source": "pipeline_steps = []"} assert all( arguments == {"scope_ids": ["/plate"]} for name, arguments, _timeout in peer.calls - if name - in ( + if name in ( agent_capabilities.initialize_datasets.name, agent_capabilities.compile_datasets.name, agent_capabilities.run_datasets.name, @@ -568,8 +503,7 @@ class ExtendedInspection(ArtifactPlanInspection): value = McpDevToolResult._decode_payload( to_jsonable( ExtendedInspection(schema_version=SCHEMA_VERSION, plate_path="extended") - ), - (ExtendedInspection,), + ), (ExtendedInspection,) ) assert type(value) is ExtendedInspection and value.extension_fact == "retained" assert "plate=extended" in renderer.render_payload_value( @@ -779,9 +713,7 @@ async def controlled_entry(args): == 0 ) rendered = capsys.readouterr().out - assert ( - "run_datasets" in rendered and "accepted" in rendered and "scope-1" in rendered - ) + assert "run_datasets" in rendered and "accepted" in rendered and "scope-1" in rendered assert json.loads(rendered) == to_jsonable(response) diff --git a/tests/unit/agent/test_mcp_dev_client_plate_renderers.py b/tests/unit/agent/test_mcp_dev_client_plate_renderers.py index 0ef1ffe392..39db8df27b 100644 --- a/tests/unit/agent/test_mcp_dev_client_plate_renderers.py +++ b/tests/unit/agent/test_mcp_dev_client_plate_renderers.py @@ -71,22 +71,11 @@ def render(value) -> str: def row(**changes) -> DatasetRowState: return replace( DatasetRowState( - scope_id="/plates/a", - name="plate-a", - root="/plates/a", - pipeline_path=None, - selected=True, - initialized=True, - compiled=False, - init_pending=False, - compile_pending=False, - execution_active=False, - status_prefix="", - orchestrator_state=None, - execution_id=None, - terminal_status=None, - runtime_state=None, - runtime_percent=None, + scope_id="/plates/a", name="plate-a", root="/plates/a", + pipeline_path=None, selected=True, initialized=True, compiled=False, + init_pending=False, compile_pending=False, execution_active=False, + status_prefix="", orchestrator_state=None, execution_id=None, + terminal_status=None, runtime_state=None, runtime_percent=None, queue_position=None, ), **changes, @@ -107,15 +96,11 @@ def inspection(**changes) -> PlatePathInspectionResult: count=2, sampled_records=( PlateInspectionImageRecordSummary( - "A01_w1.tif", - "/plates/a/A01_w1.tif", - "/src/A01_w1.tif", + "A01_w1.tif", "/plates/a/A01_w1.tif", "/src/A01_w1.tif", {"modified": "2026-01-01", "size": "1 KB"}, ), PlateInspectionImageRecordSummary( - "A01_w2.tif", - "/plates/a/A01_w2.tif", - "A01_w2.tif", + "A01_w2.tif", "/plates/a/A01_w2.tif", "A01_w2.tif", {"modified": "2026-01-02"}, ), ), @@ -125,9 +110,7 @@ def inspection(**changes) -> PlatePathInspectionResult: scanned_file_count=1, sampled_records=( PlateInspectionResultFileRecordSummary( - "results/counts.csv", - "/plates/a/results/counts.csv", - "CSV", + "results/counts.csv", "/plates/a/results/counts.csv", "CSV", preview=PlateInspectionResultFilePreview( csv_columns=("well", "count"), csv_rows=({"well": "A01", "count": "11"},), @@ -137,32 +120,19 @@ def inspection(**changes) -> PlatePathInspectionResult: ), components=( PlateInspectionComponentSummary( - "well", - PlateInspectionValueSource.METADATA, - 1, + "well", PlateInspectionValueSource.METADATA, 1, (PlateInspectionComponentValue("A01"),), ), PlateInspectionComponentSummary( - "channel", - PlateInspectionValueSource.PARSED_FILENAMES, - 2, - ( - PlateInspectionComponentValue("1", "DAPI"), - PlateInspectionComponentValue("2"), - ), + "channel", PlateInspectionValueSource.PARSED_FILENAMES, 2, + (PlateInspectionComponentValue("1", "DAPI"), + PlateInspectionComponentValue("2")), ), ), format_specific_handler_candidates=( PlateInspectionHandlerCandidate( - "opera_phenix", - "OperaPhenixHandler", - "OperaPhenixParser", - "Images", - 3, - 2, - False, - True, - False, + "opera_phenix", "OperaPhenixHandler", "OperaPhenixParser", + "Images", 3, 2, False, True, False, ), ), source_diagnostics=({"diagnostic_type": "scene", "message": "two scenes"},), @@ -179,22 +149,14 @@ def query(**changes) -> PlateFileQueryResult: plate_path="/plates/a", requested_source_format="auto", detected_source_format="imagexpress", - total_count=3, - returned_count=2, - offset=0, - limit=2, - truncated_count=1, + total_count=3, returned_count=2, offset=0, limit=2, truncated_count=1, records=( PlateFileQueryRecordSummary( - PlateFileKind.IMAGE, - "A01_w1.tif", - {"modified": "2026-02-01"}, + PlateFileKind.IMAGE, "A01_w1.tif", {"modified": "2026-02-01"}, source_path="/src/A01_w1.tif", ), PlateFileQueryRecordSummary( - PlateFileKind.RESULT, - "results/summary.csv", - {"modified": "2026-01-01"}, + PlateFileKind.RESULT, "results/summary.csv", {"modified": "2026-01-01"}, relative_path="results/summary.csv", full_path="/plates/a_out/results/summary.csv", file_format="CSV", @@ -220,15 +182,11 @@ def stream(**changes) -> PlateFileStreamResult: requested_source_format="auto", viewer_type=ViewerType.NAPARI, viewer_config_key="napari_streaming_config", - connection=ExecutionConnectionSpec( - port=5555, transport_mode=TransportMode.IPC - ), + connection=ExecutionConnectionSpec(port=5555, transport_mode=TransportMode.IPC), requested_paths=("a.tif", "a.roi.zip"), streamed_image_paths=tuple(f"/src/{index}.tif" for index in range(10)), streamed_roi_paths=("/plates/a_out/a.roi.zip",), - skipped_records=( - PlateFileQueryRecordSummary(PlateFileKind.IMAGE, "missing.tif"), - ), + skipped_records=(PlateFileQueryRecordSummary(PlateFileKind.IMAGE, "missing.tif"),), status_messages=("streamed 11 files",), ), **changes, @@ -239,34 +197,25 @@ def test_every_plate_renderer_presents_its_declared_dto_and_failure(): """One family check: each renderer is registered for its DTO and none reads a dict.""" fixtures = ( SyntheticPlateGenerationResult( - schema_version=SCHEMA_VERSION, - output_dir="/plates/synthetic", + schema_version=SCHEMA_VERSION, output_dir="/plates/synthetic", requested_format=SyntheticPlateFormat.IMAGE_XPRESS, - grid_size=(2, 2), - partition_values=("A01", "B02"), - sampled_image_files=("a.tif",), + grid_size=(2, 2), partition_values=("A01", "B02"), sampled_image_files=("a.tif",), ), inspection(), query(), stream(), SelectedPlateImageInspectionResult( - schema_version=SCHEMA_VERSION, - selected_plate=to_jsonable(row()), + schema_version=SCHEMA_VERSION, selected_plate=to_jsonable(row()), inspection=inspection(), ), SelectedPlateFileQueryResult( - schema_version=SCHEMA_VERSION, - selected_plate=to_jsonable(row()), - query=query(), + schema_version=SCHEMA_VERSION, selected_plate=to_jsonable(row()), query=query(), ), SelectedPlateImageSampleResult( - schema_version=SCHEMA_VERSION, - selected_plate=to_jsonable(row()), + schema_version=SCHEMA_VERSION, selected_plate=to_jsonable(row()), ), SelectedPlateFileStreamResult( - schema_version=SCHEMA_VERSION, - selected_plate=to_jsonable(row()), - stream=stream(), + schema_version=SCHEMA_VERSION, selected_plate=to_jsonable(row()), stream=stream(), ), ) for value in fixtures: @@ -281,10 +230,7 @@ def test_every_plate_renderer_presents_its_declared_dto_and_failure(): McpDevServerSpec(sys.executable), (McpDevToolResult(CAPABILITY[type(value)].name, True, ()),), ) - assert ( - renderer.render(tool_failure).splitlines()[0] - == renderer.unavailable_summary - ) + assert renderer.render(tool_failure).splitlines()[0] == renderer.unavailable_summary def test_inspection_presents_axes_records_previews_and_next_sample(): @@ -292,22 +238,12 @@ def test_inspection_presents_axes_records_previews_and_next_sample(): assert "Status: ok confidence=none microscope=imagexpress" in text assert "Geometry: grid=2x3 pixel_size=" in text assert "Axis sizes: well=1 site= channel=2" in text - assert ( - "profile=unknown-site,multi-channel,unknown-z_index,unknown-timepoint" in text - ) + assert "profile=unknown-site,multi-channel,unknown-z_index,unknown-timepoint" in text assert "Metadata sources: well=metadata, channel=parsed_filenames" in text - assert ( - "- channel: count=2 source=parsed_filenames values=1 (DAPI), 2 truncated=0" - in text - ) - assert ( - " - opera_phenix parser=OperaPhenixParser recognized=2/3 root=Images" in text - ) + assert "- channel: count=2 source=parsed_filenames values=1 (DAPI), 2 truncated=0" in text + assert " - opera_phenix parser=OperaPhenixParser recognized=2/3 root=Images" in text assert "- scene: two scenes" in text - assert ( - "Sampled artifacts modified: mixed latest=2026-01-02 earliest=2026-01-01" - in text - ) + assert "Sampled artifacts modified: mixed latest=2026-01-02 earliest=2026-01-01" in text assert "- A01_w1.tif -> /src/A01_w1.tif modified=2026-01-01 size=1 KB" in text assert "- A01_w2.tif modified=2026-01-02" in text # The most recently modified record is the one suggested for sampling. @@ -315,10 +251,7 @@ def test_inspection_presents_axes_records_previews_and_next_sample(): "Next: sample-plate-image /plates/a A01_w2.tif " "--height 8 --width 8 --no-array-values" ) in text - assert ( - "- results/counts.csv type=CSV\n csv columns: well, count\n csv row: well=A01, count=11" - in text - ) + assert "- results/counts.csv type=CSV\n csv columns: well, count\n csv row: well=A01, count=11" in text assert text.count("plate_parse_failures") == 1 assert text.index("Warnings:") > text.index("Result records:") @@ -327,10 +260,7 @@ def test_query_presents_previews_pages_staleness_roots_and_result_hint(): text = render(query()) assert "Result: returned=2 total=3 offset=0 limit=2 truncated=1" in text assert "- image A01_w1.tif -> /src/A01_w1.tif modified=2026-02-01" in text - assert ( - "- result results/summary.csv type=CSV -> /plates/a_out/results/summary.csv" - in text - ) + assert "- result results/summary.csv type=CSV -> /plates/a_out/results/summary.csv" in text assert " csv row: well=A00; omitted wide cells: notes" in text assert " csv preview: showing 3/5 rows; 2 more in payload" in text assert "Next page: rerun with --offset 2 --limit 2" in text @@ -342,9 +272,7 @@ def test_query_presents_previews_pages_staleness_roots_and_result_hint(): records=(), truncated_count=0, warnings=( - AgentWarning( - PlateInspectionIssueCode.RESULT_FILES_AVAILABLE.value, "results" - ), + AgentWarning(PlateInspectionIssueCode.RESULT_FILES_AVAILABLE.value, "results"), ), ) ) @@ -358,10 +286,7 @@ def test_query_presents_previews_pages_staleness_roots_and_result_hint(): def test_stream_bounds_paths_and_suggests_viewer_commands(): text = render(stream()) - assert ( - "Viewer: napari config=napari_streaming_config host=localhost port=5555 transport=ipc persistent=True" - in text - ) + assert "Viewer: napari config=napari_streaming_config host=localhost port=5555 transport=ipc persistent=True" in text assert "Files: requested=2 resolved=0 images=10 rois=1 skipped=1" in text assert "- ... 2 more" in text assert "Skipped:\n- image missing.tif" in text @@ -377,28 +302,18 @@ def test_stream_bounds_paths_and_suggests_viewer_commands(): @pytest.mark.parametrize( "target,command", [ - ( - SelectedPlateFileQueryTarget.SELECTED, - "Next: selected-plate-sample A01_w2.tif", - ), - ( - SelectedPlateFileQueryTarget.OUTPUT, - "Next: selected-plate-sample --target output A01_w2.tif", - ), + (SelectedPlateFileQueryTarget.SELECTED, "Next: selected-plate-sample A01_w2.tif"), + (SelectedPlateFileQueryTarget.OUTPUT, "Next: selected-plate-sample --target output A01_w2.tif"), ], ) def test_selected_plate_images_render_the_nested_inspection(target, command): text = render( SelectedPlateImageInspectionResult( - schema_version=SCHEMA_VERSION, - selected_plate=to_jsonable(row()), - target=target, - inspection=inspection(), + schema_version=SCHEMA_VERSION, selected_plate=to_jsonable(row()), + target=target, inspection=inspection(), ) ) - assert text.startswith( - f"Selected plate: plate-a root=/plates/a target={target.value}\nPlate: /plates/a" - ) + assert text.startswith(f"Selected plate: plate-a root=/plates/a target={target.value}\nPlate: /plates/a") assert command in text assert "sample-plate-image" not in text assert text.count("plate_parse_failures") == 1 @@ -409,27 +324,18 @@ def test_selected_plate_files_hint_related_output_only_for_an_empty_selected_que empty = query(total_count=0, returned_count=0, truncated_count=0, records=()) text = render( SelectedPlateFileQueryResult( - schema_version=SCHEMA_VERSION, - selected_plate=selected_row, - query=empty, + schema_version=SCHEMA_VERSION, selected_plate=selected_row, query=empty, ) ) assert "Plate file query: /plates/a" in text - assert ( - "Related output: /plates/a_out\nNext: selected-plate-files --target output --kind result" - in text - ) + assert "Related output: /plates/a_out\nNext: selected-plate-files --target output --kind result" in text assert "Related output" not in render( SelectedPlateFileQueryResult( - schema_version=SCHEMA_VERSION, - selected_plate=selected_row, - query=query(), + schema_version=SCHEMA_VERSION, selected_plate=selected_row, query=query(), ) ) assert "Query: " in render( - SelectedPlateFileQueryResult( - schema_version=SCHEMA_VERSION, selected_plate=selected_row - ) + SelectedPlateFileQueryResult(schema_version=SCHEMA_VERSION, selected_plate=selected_row) ) @@ -437,11 +343,8 @@ def test_selected_plate_sample_and_stream_keep_target_context_on_failure(): error = AgentError("ui_selected_plate_count_not_one", "two plates selected") sample_text = render( SelectedPlateImageSampleResult( - schema_version=SCHEMA_VERSION, - selected_plate=to_jsonable(row()), - target=SelectedPlateFileQueryTarget.OUTPUT, - image_path="a.tif", - errors=(error,), + schema_version=SCHEMA_VERSION, selected_plate=to_jsonable(row()), + target=SelectedPlateFileQueryTarget.OUTPUT, image_path="a.tif", errors=(error,), ) ) assert sample_text.startswith( @@ -458,8 +361,7 @@ def test_selected_plate_sample_and_stream_keep_target_context_on_failure(): assert stream_text.count("two plates selected") == 1 nested = render( SelectedPlateFileStreamResult( - schema_version=SCHEMA_VERSION, - selected_plate=to_jsonable(row()), + schema_version=SCHEMA_VERSION, selected_plate=to_jsonable(row()), target=SelectedPlateFileQueryTarget.OUTPUT, stream=stream(warnings=(AgentWarning("viewer_slow", "slow start"),)), ) @@ -471,16 +373,10 @@ def test_selected_plate_sample_and_stream_keep_target_context_on_failure(): def test_synthetic_generation_suggests_inspection_and_query(): text = render( SyntheticPlateGenerationResult( - schema_version=SCHEMA_VERSION, - output_dir="/plates/synthetic", + schema_version=SCHEMA_VERSION, output_dir="/plates/synthetic", requested_format=SyntheticPlateFormat.IMAGE_XPRESS, - grid_size=(2, 2), - tile_size=(64, 64), - overlap_percent=10, - partition_values=("A01", "B02"), - wavelengths=2, - image_count=8, - sampled_image_files=tuple(f"{i}.tif" for i in range(14)), + grid_size=(2, 2), tile_size=(64, 64), overlap_percent=10, partition_values=("A01", "B02"), + wavelengths=2, image_count=8, sampled_image_files=tuple(f"{i}.tif" for i in range(14)), truncated_image_count=0, ) ) diff --git a/tests/unit/agent/test_mcp_dev_client_renderers.py b/tests/unit/agent/test_mcp_dev_client_renderers.py index bd715f63d2..d45e9a5930 100644 --- a/tests/unit/agent/test_mcp_dev_client_renderers.py +++ b/tests/unit/agent/test_mcp_dev_client_renderers.py @@ -53,11 +53,7 @@ def sample_value(hint, depth: int): return tuple(sample_value(args[0], depth + 1) for _ in range(2)) return tuple(sample_value(arg, depth + 1) for arg in args) if origin in (list, typing.get_origin(typing.Sequence[int])): - return ( - [] - if depth >= MAX_DEPTH - else [sample_value(typing.get_args(hint)[0], depth + 1)] - ) + return [] if depth >= MAX_DEPTH else [sample_value(typing.get_args(hint)[0], depth + 1)] if origin in (dict, typing.get_origin(typing.Mapping[str, int])): key_type, value_type = typing.get_args(hint) or (str, str) if depth >= MAX_DEPTH: @@ -76,9 +72,7 @@ def sample_value(hint, depth: int): return "sample" if is_dataclass(hint): return sample_dto(hint, depth + 1) - if isinstance( - hint, typing.TypeAliasType if hasattr(typing, "TypeAliasType") else () - ): + if isinstance(hint, typing.TypeAliasType if hasattr(typing, "TypeAliasType") else ()): return sample_value(hint.__value__, depth) # Dynamic JSON that no class models: an empty object. return {} @@ -129,11 +123,7 @@ def sample_dto(dto_type: type, depth: int = 0): def framed(payload) -> McpDevToolBatchResponse: return McpDevToolBatchResponse( server=SERVER, - results=( - McpDevToolResult( - tool="openhcs_test_tool", mcp_error=False, payloads=(payload,) - ), - ), + results=(McpDevToolResult(tool="openhcs_test_tool", mcp_error=False, payloads=(payload,)),), ) @@ -166,9 +156,7 @@ def test_every_renderer_presents_its_declared_dto(renderer): def test_every_renderer_reports_a_missing_payload_as_unavailable(renderer): failed = McpDevToolBatchResponse( server=SERVER, - results=( - McpDevToolResult(tool="openhcs_test_tool", mcp_error=True, payloads=()), - ), + results=(McpDevToolResult(tool="openhcs_test_tool", mcp_error=True, payloads=()),), ) rendered = renderer.render(failed) @@ -188,9 +176,7 @@ def test_base_appends_payload_warnings_and_declared_errors_once(): schema_version="v1", valid=False, errors=(AgentError(code="bad_source", message="Source is invalid."),), - warnings=( - AgentWarning(code="slow", message="Validation was slow.", hint="wait"), - ), + warnings=(AgentWarning(code="slow", message="Validation was slow.", hint="wait"),), ) rendered = McpDevOutputRenderer.for_output_contract( diff --git a/tests/unit/agent/test_mcp_dev_client_sampling_results.py b/tests/unit/agent/test_mcp_dev_client_sampling_results.py index 49b08ecc47..169aca3cec 100644 --- a/tests/unit/agent/test_mcp_dev_client_sampling_results.py +++ b/tests/unit/agent/test_mcp_dev_client_sampling_results.py @@ -14,28 +14,16 @@ from openhcs.agent.capabilities import SamplePlateImageCapability, agent_capabilities from openhcs.agent.dto.common import AgentError, SCHEMA_VERSION -from openhcs.agent.dto.plate import ( - PlateImageSampleResult, - SelectedPlateImageSampleResult, -) -from openhcs.agent.dto.viewer import ( - ViewerWindowImageSampleRecord, - ViewerWindowImageSampleResult, -) +from openhcs.agent.dto.plate import PlateImageSampleResult, SelectedPlateImageSampleResult +from openhcs.agent.dto.viewer import ViewerWindowImageSampleRecord, ViewerWindowImageSampleResult from openhcs.mcp import dev_client from openhcs.mcp.dev_client_commanding import CapabilityBackedCommandSpec from openhcs.mcp.dev_client_core import ( - McpDevClientPhase, - McpDevPayloadFailure, - McpDevServerSpec, - McpDevToolBatchResponse, - McpDevToolResult, + McpDevClientPhase, McpDevPayloadFailure, McpDevServerSpec, + McpDevToolBatchResponse, McpDevToolResult, ) from openhcs.mcp.dev_client_rendering import McpDevOutputRenderer -from openhcs.mcp.dev_client_renderers.plate import ( - PlateImageSampleRenderer, - SelectedPlateSampleRenderer, -) +from openhcs.mcp.dev_client_renderers.plate import PlateImageSampleRenderer, SelectedPlateSampleRenderer from openhcs.mcp.dev_client_renderers.viewer import ViewerImageSampleRenderer from openhcs.runtime.viewer_protocol import ViewerArrayValueSummary from python_introspect import to_jsonable @@ -45,57 +33,29 @@ def forbid_native_or_provider(monkeypatch): def forbidden(*args, **kwargs): raise AssertionError("Source sampling controls must not spawn a process") - monkeypatch.setattr(subprocess, "Popen", forbidden) monkeypatch.setattr(asyncio, "create_subprocess_exec", forbidden) monkeypatch.setattr(asyncio, "create_subprocess_shell", forbidden) def sample(**kwargs): - return replace( - PlateImageSampleResult( - schema_version=SCHEMA_VERSION, - plate_path="source-plate", - requested_image_path="image.tif", - virtual_path="images/image.tif", - source_path="physical/image.tif", - shape=(1, 2, 2), - resolution_shape=(1, 2, 2), - dtype="uint16", - minimum=0, - maximum=4, - mean=2.5, - selected_resolution_index=0, - resolution_count=1, - downsample_yx=(1.0, 1.0), - statistics_scope="source_resolution", - sample_shape=(1, 2, 2), - sample_included=True, - sample_values=[[[0, 2], [3, 4]]], - ), - **kwargs, - ) + return replace(PlateImageSampleResult( + schema_version=SCHEMA_VERSION, plate_path="source-plate", requested_image_path="image.tif", + virtual_path="images/image.tif", source_path="physical/image.tif", + shape=(1, 2, 2), resolution_shape=(1, 2, 2), dtype="uint16", + minimum=0, maximum=4, mean=2.5, selected_resolution_index=0, + resolution_count=1, downsample_yx=(1.0, 1.0), statistics_scope="source_resolution", + sample_shape=(1, 2, 2), sample_included=True, sample_values=[[[0, 2], [3, 4]]], + ), **kwargs) def selected_row(): return DatasetRowState( - scope_id="selected-root", - name="selection", - root="selected-root", - pipeline_path=None, - selected=True, - initialized=True, - compiled=False, - init_pending=False, - compile_pending=False, - execution_active=False, - status_prefix="", - orchestrator_state=None, - execution_id=None, - terminal_status=None, - runtime_state=None, - runtime_percent=None, - queue_position=None, + scope_id="selected-root", name="selection", root="selected-root", + pipeline_path=None, selected=True, initialized=True, compiled=False, + init_pending=False, compile_pending=False, execution_active=False, + status_prefix="", orchestrator_state=None, execution_id=None, + terminal_status=None, runtime_state=None, runtime_percent=None, queue_position=None, ) @@ -103,53 +63,38 @@ def selected(value): return SelectedPlateImageSampleResult( schema_version=SCHEMA_VERSION, selected_plate=to_jsonable(selected_row()), - image_path="image.tif", - auto_selected_image_path=False, - sample=value, + image_path="image.tif", auto_selected_image_path=False, sample=value, ) def batch(value, capability): receipt = {"isError": False, "structuredContent": to_jsonable(value), "content": []} return McpDevToolBatchResponse.from_results( - McpDevServerSpec(sys.executable), - (McpDevToolResult.from_payload(capability.name, receipt),), + McpDevServerSpec(sys.executable), (McpDevToolResult.from_payload(capability.name, receipt),), ) def command_render(response, argv): args = dev_client._build_parser().parse_args(argv) - return dev_client.McpDevCommandSpec.for_name(args.command).render_result( - response, args - ) + return dev_client.McpDevCommandSpec.for_name(args.command).render_result(response, args) @pytest.mark.parametrize("structured", [False, True]) @pytest.mark.parametrize("selected_case", [False, True]) -def test_original_session_framing_to_generated_sampling_command( - structured, selected_case, monkeypatch -): - from tests.unit.agent.test_mcp_dev_client_pipeline_results import ( - ControlledWireSession, - ) +def test_original_session_framing_to_generated_sampling_command(structured, selected_case, monkeypatch): + from tests.unit.agent.test_mcp_dev_client_pipeline_results import ControlledWireSession import openhcs.mcp.dev_client_core as ingress value = selected(sample()) if selected_case else sample() - argv = ( - ("selected-plate-sample",) - if selected_case - else ("sample-plate-image", "source-plate", "image.tif") - ) + argv = ("selected-plate-sample",) if selected_case else ("sample-plate-image", "source-plate", "image.tif") args = dev_client._build_parser().parse_args(argv) command = dev_client.McpDevCommandSpec.for_name(args.command) peer = ControlledWireSession((value,), structured=structured) decoded = [] original_decode = ingress.dataclass_from_mapping - def observe(contract, receipt): decoded.append(contract) return original_decode(contract, receipt) - monkeypatch.setattr(ingress, "dataclass_from_mapping", observe) response = asyncio.run(command.run_session(peer, args)) assert decoded == [type(value)] @@ -162,14 +107,8 @@ def observe(contract, receipt): @pytest.mark.parametrize("selected_case", [False, True]) @pytest.mark.parametrize("generic_call", [False, True]) -def test_generated_commands_retain_decoded_sampling_facts( - monkeypatch, selected_case, generic_call -): - capability = ( - agent_capabilities.ui_sample_selected_plate_image - if selected_case - else agent_capabilities.sample_plate_image - ) +def test_generated_commands_retain_decoded_sampling_facts(monkeypatch, selected_case, generic_call): + capability = agent_capabilities.ui_sample_selected_plate_image if selected_case else agent_capabilities.sample_plate_image value = selected(sample()) if selected_case else sample() response = batch(value, capability) decoded = response.results[0].first_decoded_payload() @@ -180,17 +119,10 @@ def test_generated_commands_retain_decoded_sampling_facts( def forbidden(value): raise AssertionError("Typed sampling must not flatten an owned result to JSON") - monkeypatch.setattr(dev_client, "to_jsonable", forbidden) monkeypatch.setattr("python_introspect.jsonable.to_jsonable", forbidden) - argv = ( - ("call", capability.name, "--arguments", "{}") - if generic_call - else ( - ("selected-plate-sample",) - if selected_case - else ("sample-plate-image", "source-plate", "image.tif") - ) + argv = ("call", capability.name, "--arguments", "{}") if generic_call else ( + ("selected-plate-sample",) if selected_case else ("sample-plate-image", "source-plate", "image.tif") ) rendered = command_render(response, argv) assert "Image: images/image.tif" in rendered @@ -199,51 +131,23 @@ def forbidden(value): assert "mean=2.500" in rendered assert "Sample values:" in rendered if selected_case: - assert ( - "Selected plate: selection root=selected-root target=selected" in rendered - ) + assert "Selected plate: selection root=selected-root target=selected" in rendered assert "Selected image: image.tif auto=False" in rendered @pytest.mark.parametrize("selected_case", [False, True]) -@pytest.mark.parametrize( - "reason,shape,hint", - [ - ( - "max_array_elements_exceeded", - (1, 8, 8), - "--max-array-elements 64 or smaller --width/--height", - ), - ( - "sample has 256 elements, above max_array_elements=40", - (1, 16, 16), - "--max-array-elements 256", - ), - ( - "array_values_not_requested", - (1, 2, 2), - "--include-array-values --max-array-elements 4", - ), - ("empty", (), "Sample values omitted: empty"), - ("max_array_elements_exceeded", (1, 0, 2), "--max-array-elements 0"), - ], -) +@pytest.mark.parametrize("reason,shape,hint", [ + ("max_array_elements_exceeded", (1, 8, 8), "--max-array-elements 64 or smaller --width/--height"), + ("sample has 256 elements, above max_array_elements=40", (1, 16, 16), "--max-array-elements 256"), + ("array_values_not_requested", (1, 2, 2), "--include-array-values --max-array-elements 4"), + ("empty", (), "Sample values omitted: empty"), + ("max_array_elements_exceeded", (1, 0, 2), "--max-array-elements 0"), +]) def test_omission_and_budget_advice_preserved(selected_case, reason, shape, hint): - value = sample( - sample_included=False, - sample_values=(), - sample_shape=shape, - sample_omitted_reason=reason, - ) - capability = ( - agent_capabilities.ui_sample_selected_plate_image - if selected_case - else agent_capabilities.sample_plate_image - ) + value = sample(sample_included=False, sample_values=(), sample_shape=shape, sample_omitted_reason=reason) + capability = agent_capabilities.ui_sample_selected_plate_image if selected_case else agent_capabilities.sample_plate_image response = batch(selected(value) if selected_case else value, capability) - renderer = ( - SelectedPlateSampleRenderer if selected_case else PlateImageSampleRenderer - ) + renderer = SelectedPlateSampleRenderer if selected_case else PlateImageSampleRenderer assert hint in renderer.render(response) @@ -252,54 +156,28 @@ def test_legitimate_absent_sample_and_false_zero_empty_are_distinct(): rendered = SelectedPlateSampleRenderer.render(response) assert "Sample: " in rendered assert "auto=False" in rendered - zero = sample( - virtual_path="", - dtype="", - mean=0.0, - minimum=False, - maximum=0, - sample_shape=(), - sample_values=[], - sample_included=True, - ) - rendered = PlateImageSampleRenderer.render( - batch(zero, agent_capabilities.sample_plate_image) - ) + zero = sample(virtual_path="", dtype="", mean=0.0, minimum=False, maximum=0, + sample_shape=(), sample_values=[], sample_included=True) + rendered = PlateImageSampleRenderer.render(batch(zero, agent_capabilities.sample_plate_image)) assert "Image: \n" in rendered assert "dtype= min=False max=0 mean=0.000" in rendered assert "shape= included=True" in rendered assert "Sample values:\n[]" in rendered -@pytest.mark.parametrize( - "values,count", - [([False, 0, "", None], 3), (list(range(65)), 65), ({"a": [0, False], "b": ""}, 3)], -) +@pytest.mark.parametrize("values,count", [([False, 0, "", None], 3), (list(range(65)), 65), ({"a": [0, False], "b": ""}, 3)]) def test_shared_preview_policy_and_richer_viewer_owner(values, count): assert McpDevOutputRenderer.json_value_count(values) == count - response = batch( - sample(sample_values=values), agent_capabilities.sample_plate_image - ) + response = batch(sample(sample_values=values), agent_capabilities.sample_plate_image) rendered = PlateImageSampleRenderer.render(response) assert ("65 elements; pass --json" in rendered) == (count > 64) record = ViewerWindowImageSampleRecord( - layer_route_key="layer", - layer_title="image", - payload_route_key="payload", - data_type="image", - path="source", - components={}, - array_values=(values,), - array_value_summary=ViewerArrayValueSummary( - requested=True, included=True, shape=(1, count) - ), - ) - viewer = ViewerWindowImageSampleResult( - schema_version=SCHEMA_VERSION, observed=True, records=(record,) - ) - text = ViewerImageSampleRenderer.render( - batch(viewer, agent_capabilities.sample_viewer_window_image) + layer_route_key="layer", layer_title="image", payload_route_key="payload", + data_type="image", path="source", components={}, array_values=(values,), + array_value_summary=ViewerArrayValueSummary(requested=True, included=True, shape=(1, count)), ) + viewer = ViewerWindowImageSampleResult(schema_version=SCHEMA_VERSION, observed=True, records=(record,)) + text = ViewerImageSampleRenderer.render(batch(viewer, agent_capabilities.sample_viewer_window_image)) assert ("sample values:" in text) == (count <= 64) assert record.array_value_summary.shape_element_count == count @@ -308,44 +186,19 @@ def test_shared_preview_policy_and_richer_viewer_owner(values, count): def test_original_typed_diagnostics_retained_once(selected_case): error = AgentError("sample_failure", "retained cause", "retained hint") value = sample(errors=(error,)) - capability = ( - agent_capabilities.ui_sample_selected_plate_image - if selected_case - else agent_capabilities.sample_plate_image - ) - response = batch( - replace(selected(value), errors=(error,)) if selected_case else value, - capability, - ) - text = ( - SelectedPlateSampleRenderer if selected_case else PlateImageSampleRenderer - ).render(response) + capability = agent_capabilities.ui_sample_selected_plate_image if selected_case else agent_capabilities.sample_plate_image + response = batch(replace(selected(value), errors=(error,)) if selected_case else value, capability) + text = (SelectedPlateSampleRenderer if selected_case else PlateImageSampleRenderer).render(response) assert "failed" in text assert text.count("sample_failure") == 1 assert "retained cause" in text and "retained hint" in text assert response.has_errors() -@pytest.mark.parametrize( - "receipt", - [ - {"alien": 1}, - {"schema_version": SCHEMA_VERSION}, - { - "schema_version": SCHEMA_VERSION, - "plate_path": "p", - "requested_image_path": "i", - "sample_included": "false", - }, - ], -) +@pytest.mark.parametrize("receipt", [{"alien": 1}, {"schema_version": SCHEMA_VERSION}, {"schema_version": SCHEMA_VERSION, "plate_path": "p", "requested_image_path": "i", "sample_included": "false"}]) def test_malformed_receipt_never_becomes_successful_empty_sample(receipt): - result = McpDevToolResult.from_payload( - agent_capabilities.sample_plate_image.name, {"structuredContent": receipt} - ) - response = McpDevToolBatchResponse.from_results( - McpDevServerSpec(sys.executable), (result,) - ) + result = McpDevToolResult.from_payload(agent_capabilities.sample_plate_image.name, {"structuredContent": receipt}) + response = McpDevToolBatchResponse.from_results(McpDevServerSpec(sys.executable), (result,)) assert isinstance(result.payloads[0], McpDevPayloadFailure) assert result.payloads[0].payload == receipt rejection = to_jsonable(response)["results"][0]["payloads"][0] @@ -358,19 +211,12 @@ def test_malformed_receipt_never_becomes_successful_empty_sample(receipt): def test_transport_missing_and_unknown_receipts_keep_original_diagnostics(): server = McpDevServerSpec(sys.executable) - missing = McpDevToolBatchResponse.from_results( - server, - (McpDevToolResult(agent_capabilities.sample_plate_image.name, False, ()),), - ) + missing = McpDevToolBatchResponse.from_results(server, (McpDevToolResult(agent_capabilities.sample_plate_image.name, False, ()),)) assert "mcp_payload_missing" in PlateImageSampleRenderer.render(missing) - response = McpDevToolBatchResponse.from_transport_failure( - server, McpDevClientPhase.CALL_TOOL, RuntimeError("wire lost") - ) + response = McpDevToolBatchResponse.from_transport_failure(server, McpDevClientPhase.CALL_TOOL, RuntimeError("wire lost")) assert "wire lost" in PlateImageSampleRenderer.render(response) receipt = {"opaque": {"false": False, "zero": 0, "empty": ""}} - unknown = McpDevToolResult.from_payload( - "unregistered-external-tool", {"structuredContent": receipt} - ) + unknown = McpDevToolResult.from_payload("unregistered-external-tool", {"structuredContent": receipt}) assert unknown.payloads == (receipt,) assert to_jsonable(unknown)["payloads"] == [receipt] @@ -382,16 +228,11 @@ class NewSample(PlateImageSampleResult): independent_fact: str = "owned" events = [] - class SourceFact(PlateImageSampleRenderer): @classmethod def render_payload(cls, payload, options): events.append("source") - return ( - super().render_payload(payload, options) - + "\n" - + payload.independent_fact - ) + return super().render_payload(payload, options) + "\n" + payload.independent_fact class AuditFact(PlateImageSampleRenderer): @classmethod @@ -401,17 +242,12 @@ def render_payload(cls, payload, options): bases = (AuditFact, SourceFact) if reverse_order else (SourceFact, AuditFact) NewRenderer = type("NewSampleRenderer", bases, {"output_contract": NewSample}) - class NewCapability(SamplePlateImageCapability): name = f"openhcs_s1_new_sample_{int(reverse_order)}" cli_command = f"s1-new-sample-{int(reverse_order)}" output_contract = NewSample - value = NewSample( - schema_version=SCHEMA_VERSION, - plate_path="new-plate", - requested_image_path="new-image", - ) + value = NewSample(schema_version=SCHEMA_VERSION, plate_path="new-plate", requested_image_path="new-image") response = batch(value, NewCapability) assert type(response.results[0].first_decoded_payload()) is NewSample command = CapabilityBackedCommandSpec.for_capability_name(NewCapability.name) @@ -425,7 +261,4 @@ class NewCapability(SamplePlateImageCapability): def test_sampling_renderers_share_the_json_preview_count(): assert "_json_value_count" not in vars(PlateImageSampleRenderer) assert "_json_value_count" not in vars(ViewerImageSampleRenderer) - assert ( - PlateImageSampleRenderer.json_value_count.__func__ - is ViewerImageSampleRenderer.json_value_count.__func__ - ) + assert PlateImageSampleRenderer.json_value_count.__func__ is ViewerImageSampleRenderer.json_value_count.__func__ diff --git a/tests/unit/agent/test_mcp_dev_client_ui_action_results.py b/tests/unit/agent/test_mcp_dev_client_ui_action_results.py index ec3f37be10..f7b4c6d5bc 100644 --- a/tests/unit/agent/test_mcp_dev_client_ui_action_results.py +++ b/tests/unit/agent/test_mcp_dev_client_ui_action_results.py @@ -1,5 +1,4 @@ from openhcs.agent.dto.session import DatasetRowState - """Real CLI/codec action-family checks without a server or UI mutation.""" import sys import json @@ -29,16 +28,12 @@ ) from openhcs.mcp import dev_client from openhcs.mcp.dev_client_core import ( - McpDevServerSpec, - McpDevToolBatchResponse, - McpDevToolResult, - workflow_result_action_status, - workflow_result_target_scope_ids, + McpDevServerSpec, McpDevToolBatchResponse, McpDevToolResult, + workflow_result_action_status, workflow_result_target_scope_ids, workflow_poll_summary_result, ) from openhcs.mcp.dev_client_rendering import ( - McpDevOutputRenderer, - McpDevOutputRenderOptions, + McpDevOutputRenderer, McpDevOutputRenderOptions, ) from openhcs.mcp.dev_client_renderers.ui_bridge import UiActionInvokeRenderer from python_introspect import to_jsonable @@ -55,92 +50,55 @@ def action_fixture(): def action_result(capability, value): - return McpDevToolResult.from_payload( - capability.name, - { - "isError": False, - "structuredContent": to_jsonable(value), - "content": [], - }, - ) + return McpDevToolResult.from_payload(capability.name, { + "isError": False, "structuredContent": to_jsonable(value), "content": [], + }) def state_fixture(compiled, revision): summary = UiStateSurfaceSummary( - SCHEMA_VERSION, - UiStateSurfaceIdentity(surface_id="plate_manager.state"), - "Plate Manager", - True, - widget_id="plate_manager", + SCHEMA_VERSION, UiStateSurfaceIdentity(surface_id="plate_manager.state"), + "Plate Manager", True, widget_id="plate_manager", ) row = DatasetRowState( - scope_id="scope-1", - name="plate-one", - root="/plate-one", - pipeline_path=None, - selected=True, - initialized=True, - compiled=compiled, - init_pending=False, - compile_pending=False, - execution_active=False, + scope_id="scope-1", name="plate-one", root="/plate-one", + pipeline_path=None, selected=True, initialized=True, compiled=compiled, + init_pending=False, compile_pending=False, execution_active=False, status_prefix="Compiled" if compiled else "Created", orchestrator_state="compiled" if compiled else "created", - execution_id=None, - terminal_status=None, - runtime_state=None, - runtime_percent=None, - queue_position=None, + execution_id=None, terminal_status=None, + runtime_state=None, runtime_percent=None, queue_position=None, ) state = UiPlateManagerState( - schema_version=SCHEMA_VERSION, - summary=summary, - object_state_token=revision, - manager_execution_state="idle", - rows=(row,), + schema_version=SCHEMA_VERSION, summary=summary, object_state_token=revision, + manager_execution_state="idle", rows=(row,), current_revision_token=f"rev-{revision}", ) return UiStateSurfaceDocument( - schema_version=SCHEMA_VERSION, - summary=summary, - payload_schema="openhcs.ui.plate_manager_state.v1", - payload=to_jsonable(state), + schema_version=SCHEMA_VERSION, summary=summary, + payload_schema="openhcs.ui.plate_manager_state.v1", payload=to_jsonable(state), current_revision_token=state.current_revision_token, ) @pytest.mark.parametrize("json_output", (False, True)) def test_named_cli_wait_preserves_action_receipt_poll_and_final_rows( - monkeypatch, - capsys, - json_output, + monkeypatch, capsys, json_output, ): """Actual CLI/controller/codec chain, with only the MCP wire controlled.""" native = UiSelectedPlateWorkflowResult( - SCHEMA_VERSION, - UiSelectedPlateWorkflowKind("compile_datasets"), - replace( - action_fixture(), - identity=UiActionIdentity( - widget_id="plate_manager", action_id="compile_datasets" - ), - receipt=UiMutationReceipt(UiMutationRequestToken(), "operation-one", True), - ), + SCHEMA_VERSION, UiSelectedPlateWorkflowKind("compile_datasets"), + replace(action_fixture(), + identity=UiActionIdentity(widget_id="plate_manager", action_id="compile_datasets"), + receipt=UiMutationReceipt(UiMutationRequestToken(), "operation-one", True)), ) receipt = UiBridgeOperationRef( - schema_version=SCHEMA_VERSION, - status="completed", - started_at_unix=1.0, - completed_at_unix=2.0, - outcome="completed", - identity=UiBridgeOperationIdentity( - "operation-one", UiBridgeOperationRoute("invoke_action") - ), + schema_version=SCHEMA_VERSION, status="completed", started_at_unix=1.0, + completed_at_unix=2.0, outcome="completed", + identity=UiBridgeOperationIdentity("operation-one", UiBridgeOperationRoute("invoke_action")), ) calls = [] - wire_values = iter( - (state_fixture(False, 1), native, receipt, state_fixture(True, 2)) - ) + wire_values = iter((state_fixture(False, 1), native, receipt, state_fixture(True, 2))) wire_payloads = tuple(to_jsonable(value) for value in wire_values) class ControlledWireSession: @@ -148,43 +106,27 @@ class ControlledWireSession: async def call_tool(self, name, arguments, *, timeout_seconds): calls.append(name) - return { - "isError": False, - "structuredContent": wire_payloads[len(calls) - 1], - "content": [], - } + return {"isError": False, "structuredContent": wire_payloads[len(calls)-1], "content": []} async def controlled_session(args): return await dev_client.McpDevCommandSpec.for_name(args.command).run_session( - ControlledWireSession(), - args, + ControlledWireSession(), args, ) import python_introspect.jsonable as serialization - monkeypatch.setattr(dev_client, "_run_async", controlled_session) if not json_output: - def forbidden_serialization(value): raise AssertionError("Compact polling must retain its typed summary") - monkeypatch.setattr(serialization, "to_jsonable", forbidden_serialization) - argv = [ - "selected-workflow", - "compile_datasets", - "--wait", - "--wait-interval-seconds", - "0", - ] + argv = ["selected-workflow", "compile_datasets", "--wait", "--wait-interval-seconds", "0"] if json_output: argv.append("--json") assert dev_client.main(argv) == 0 - assert calls == [ - agent_capabilities.ui_get_state_surface.name, - agent_capabilities.ui_selected_plate_workflow.name, - agent_capabilities.ui_wait_for_operation_receipt.name, - agent_capabilities.ui_get_state_surface.name, - ] + assert calls == [agent_capabilities.ui_get_state_surface.name, + agent_capabilities.ui_selected_plate_workflow.name, + agent_capabilities.ui_wait_for_operation_receipt.name, + agent_capabilities.ui_get_state_surface.name] rendered = capsys.readouterr().out if json_output: response = json.loads(rendered) @@ -201,17 +143,13 @@ def forbidden_serialization(value): assert "terminal=, selected=True, root=/plate-one" in rendered -def test_malformed_action_is_nonzero_and_preserves_invalid_wire_payload( - monkeypatch, capsys -): +def test_malformed_action_is_nonzero_and_preserves_invalid_wire_payload(monkeypatch, capsys): malformed = {"workflow": "run_datasets", "action_result": {"status": "accepted"}} result = McpDevToolResult.from_payload( agent_capabilities.ui_selected_plate_workflow.name, {"isError": False, "structuredContent": malformed, "content": []}, ) - response = McpDevToolBatchResponse.from_results( - McpDevServerSpec(sys.executable), (result,) - ) + response = McpDevToolBatchResponse.from_results(McpDevServerSpec(sys.executable), (result,)) async def controlled_wire(args): return response @@ -230,38 +168,23 @@ def test_poll_presentation_rejects_row_missing_declared_state_field(): del state.payload["rows"][0]["initialized"] response = McpDevToolBatchResponse.from_results( McpDevServerSpec(sys.executable), - ( - action_result(agent_capabilities.ui_get_state_surface, state), - workflow_poll_summary_result( - workflow="compile_datasets", - poll_requested=True, - poll_completed=True, - poll_count=1, - action_status="accepted", - ), - ), - ) - args = dev_client._build_parser().parse_args( - ["selected-workflow", "compile_datasets", "--wait"] + (action_result(agent_capabilities.ui_get_state_surface, state), + workflow_poll_summary_result( + workflow="compile_datasets", poll_requested=True, + poll_completed=True, poll_count=1, action_status="accepted", + )), ) + args = dev_client._build_parser().parse_args(["selected-workflow", "compile_datasets", "--wait"]) with pytest.raises(ValueError, match="missing required field.*initialized"): - dev_client.McpDevCommandSpec.for_name("selected-workflow").render_result( - response, args - ) + dev_client.McpDevCommandSpec.for_name("selected-workflow").render_result(response, args) -def test_actual_generic_cli_retains_nested_action_without_json_roundtrip( - monkeypatch, capsys -): +def test_actual_generic_cli_retains_nested_action_without_json_roundtrip(monkeypatch, capsys): native = UiSelectedPlateWorkflowResult( - SCHEMA_VERSION, - UiSelectedPlateWorkflowKind("run_datasets"), - action_fixture(), + SCHEMA_VERSION, UiSelectedPlateWorkflowKind("run_datasets"), action_fixture(), ) result = action_result(agent_capabilities.ui_selected_plate_workflow, native) - response = McpDevToolBatchResponse.from_results( - McpDevServerSpec(sys.executable), (result,) - ) + response = McpDevToolBatchResponse.from_results(McpDevServerSpec(sys.executable), (result,)) async def controlled_wire(args): return response @@ -270,20 +193,12 @@ def forbidden_serialization(value): raise AssertionError("Compact presentation must retain the decoded result") import python_introspect.jsonable as serialization - monkeypatch.setattr(dev_client, "_run_async", controlled_wire) monkeypatch.setattr(serialization, "to_jsonable", forbidden_serialization) - assert ( - dev_client.main( - [ - "call", - agent_capabilities.ui_selected_plate_workflow.name, - "--arguments", - '{"workflow":"run_datasets"}', - ] - ) - == 0 - ) + assert dev_client.main([ + "call", agent_capabilities.ui_selected_plate_workflow.name, + "--arguments", '{"workflow":"run_datasets"}', + ]) == 0 rendered = capsys.readouterr().out assert "action=plate_manager/run_datasets status=accepted" in rendered assert "accepted=True" in rendered and "targets=scope-1" in rendered @@ -291,32 +206,20 @@ def forbidden_serialization(value): assert workflow_result_target_scope_ids(result) == ("scope-1",) -@pytest.mark.parametrize( - "presentation", ((), ("--output", "outline"), ("--output", "json"), ("--json",)) -) +@pytest.mark.parametrize("presentation", ((), ("--output", "outline"), + ("--output", "json"), ("--json",))) def test_actual_widget_tree_cli_uses_its_declared_output_format( - monkeypatch, - capsys, - presentation, + monkeypatch, capsys, presentation, ): """The real CLI must not require a second undeclared args.json flag.""" native = UiWidgetTreeResult( - schema_version=SCHEMA_VERSION, - window_id="main_window", - projected=True, - summary=UiWindowSummary( - SCHEMA_VERSION, - UiWindowIdentity(window_id="main_window"), - "OpenHCS regression window", - "qt_top_level", - True, - True, - ), + schema_version=SCHEMA_VERSION, window_id="main_window", projected=True, + summary=UiWindowSummary(SCHEMA_VERSION, UiWindowIdentity(window_id="main_window"), + "OpenHCS regression window", "qt_top_level", True, True), ) result = action_result(agent_capabilities.ui_get_widget_tree, native) response = McpDevToolBatchResponse.from_results( - McpDevServerSpec(sys.executable), - (result,), + McpDevServerSpec(sys.executable), (result,), ) async def controlled_wire(args): @@ -336,39 +239,21 @@ async def controlled_wire(args): def test_generic_widget_tree_call_uses_the_same_output_declaration(monkeypatch, capsys): native = UiWidgetTreeResult( - schema_version=SCHEMA_VERSION, - window_id="main_window", - projected=True, - summary=UiWindowSummary( - SCHEMA_VERSION, - UiWindowIdentity(window_id="main_window"), - "OpenHCS regression window", - "qt_top_level", - True, - True, - ), + schema_version=SCHEMA_VERSION, window_id="main_window", projected=True, + summary=UiWindowSummary(SCHEMA_VERSION, UiWindowIdentity(window_id="main_window"), + "OpenHCS regression window", "qt_top_level", True, True), ) result = action_result(agent_capabilities.ui_get_widget_tree, native) response = McpDevToolBatchResponse.from_results( - McpDevServerSpec(sys.executable), - (result,), + McpDevServerSpec(sys.executable), (result,), ) async def controlled_wire(args): return response monkeypatch.setattr(dev_client, "_run_async", controlled_wire) - assert ( - dev_client.main( - [ - "call", - agent_capabilities.ui_get_widget_tree.name, - "--arguments", - '{"window_id":"main_window"}', - ] - ) - == 0 - ) + assert dev_client.main(["call", agent_capabilities.ui_get_widget_tree.name, + "--arguments", '{"window_id":"main_window"}']) == 0 assert "Window: OpenHCS regression window" in capsys.readouterr().out @@ -392,35 +277,21 @@ def introduction_lines(cls, payload): class ExtendedAction(UiActionInvokeResult): annotation: str = "new declared fact" - bases = ( - (SecondCapability, FirstCapability) - if reverse - else (FirstCapability, SecondCapability) - ) - extension = type( - "ExtendedActionRenderer", - (*bases, UiActionInvokeRenderer), - {"output_contract": ExtendedAction}, - ) + bases = ((SecondCapability, FirstCapability) if reverse else + (FirstCapability, SecondCapability)) + extension = type("ExtendedActionRenderer", (*bases, UiActionInvokeRenderer), + {"output_contract": ExtendedAction}) try: assert McpDevOutputRenderer.for_output_contract(ExtendedAction) is extension value = ExtendedAction( - SCHEMA_VERSION, - UiActionIdentity(widget_id="plate_manager", action_id="run_datasets"), - "accepted", - UiMutationReceipt(UiMutationRequestToken(), accepted=True), - ) - decoded = McpDevToolResult._decode_payload( - to_jsonable(value), (ExtendedAction,) + SCHEMA_VERSION, UiActionIdentity(widget_id="plate_manager", action_id="run_datasets"), + "accepted", UiMutationReceipt(UiMutationRequestToken(), accepted=True), ) + decoded = McpDevToolResult._decode_payload(to_jsonable(value), (ExtendedAction,)) assert decoded.annotation == "new declared fact" rendered = extension.render_payload_value(decoded, McpDevOutputRenderOptions()) assert events == (["second", "first"] if reverse else ["first", "second"]) - assert ( - rendered.count("first capability") - == rendered.count("second capability") - == 1 - ) + assert rendered.count("first capability") == rendered.count("second capability") == 1 assert rendered.count("UI action invoke:") == 1 finally: del McpDevOutputRenderer.__registry__[ExtendedAction] diff --git a/tests/unit/agent/test_mcp_dev_client_ui_renderers.py b/tests/unit/agent/test_mcp_dev_client_ui_renderers.py index 24353fde25..f7dd2edd0e 100644 --- a/tests/unit/agent/test_mcp_dev_client_ui_renderers.py +++ b/tests/unit/agent/test_mcp_dev_client_ui_renderers.py @@ -71,10 +71,7 @@ def render(payload, options=None) -> str: def snapshot(index: int, label: str = "edit") -> UiSnapshotRef: return replace( - sample_dto(UiSnapshotRef), - index=index, - label=label, - branch="main", + sample_dto(UiSnapshotRef), index=index, label=label, branch="main", snapshot_id=f"snapshot-{index}", ) @@ -115,11 +112,9 @@ def test_bridge_status_and_unavailable_bridge(): server=SERVER, results=(McpDevToolResult(tool="t", mcp_error=True, payloads=()),), ) - assert ( - McpDevOutputRenderer.for_output_contract(UiBridgeStatus) - .render(failed) - .startswith("UI bridge: unavailable\nErrors:") - ) + assert McpDevOutputRenderer.for_output_contract(UiBridgeStatus).render( + failed + ).startswith("UI bridge: unavailable\nErrors:") def window(window_id: str, title: str, kind: str, visible: bool) -> UiWindowSummary: @@ -152,28 +147,21 @@ def test_window_catalog_flags_only_visible_stray_top_level_windows(): 'Attention: 1 visible top-level window(s): qt_top_level:1 title="Error"' in rendered ) - assert "- plate_manager [embedded] visible=True" in rendered + assert '- plate_manager [embedded] visible=True' in rendered def test_ui_smoke_composes_each_tool_payload(): from openhcs.agent.capabilities import agent_capabilities def result(capability, payload): - return McpDevToolResult( - tool=capability.name, mcp_error=False, payloads=(payload,) - ) + return McpDevToolResult(tool=capability.name, mcp_error=False, payloads=(payload,)) health = replace( sample_dto(McpServerHealthResult), - status="ok", - restart_required=False, - stale_source_paths=(), - warnings=(), + status="ok", restart_required=False, stale_source_paths=(), warnings=(), ) bridges = replace(sample_dto(UiBridgeCatalog), warnings=()) - status = replace( - sample_dto(UiBridgeStatus), reachable=True, descriptor_status="ok", warnings=() - ) + status = replace(sample_dto(UiBridgeStatus), reachable=True, descriptor_status="ok", warnings=()) response = McpDevToolBatchResponse( server=SERVER, results=( @@ -223,15 +211,9 @@ def document(document_id): def plate_row(name: str, selected: bool, **values) -> DatasetRowState: return replace( sample_dto(DatasetRowState), - name=name, - selected=selected, - orchestrator_state="compiled", - status_prefix="", - terminal_status=None, - root=f"/{name}", - output_root=None, - source_root=None, - **values, + name=name, selected=selected, orchestrator_state="compiled", + status_prefix="", terminal_status=None, root=f"/{name}", + output_root=None, source_root=None, **values, ) @@ -263,21 +245,15 @@ def test_state_surface_documents_render_through_their_surface_state(): sample_dto(UiPipelineEditorState), summary=surface_summary( PipelineEditorStateSurfaceIdentityDeclaration.require_value(), - current_selection_count=1, - total_scope_count=2, + current_selection_count=1, total_scope_count=2, ), current_snapshot=snapshot(4, "edit pipeline"), selected_scope_ids=("/tmp/plate-a::functionstep_1",), steps=( replace( sample_dto(PipelineStepState), - index=0, - name="normalize", - dirty=True, - default_diff=False, - function_names=("normalize",), - function_ids=(), - debug_pause=False, + index=0, name="normalize", dirty=True, default_diff=False, + function_names=("normalize",), function_ids=(), debug_pause=False, step_scope_id=None, ), ), @@ -286,9 +262,7 @@ def test_state_surface_documents_render_through_their_surface_state(): rendered = render(document_for(editor)) assert 'Snapshot: 4 "edit pipeline"' in rendered assert "Selected scopes: /tmp/plate-a::functionstep_1" in rendered - assert ( - "- 0. [*] normalize: enabled=True, selected=True, funcs=normalize" in rendered - ) + assert "- 0. [*] normalize: enabled=True, selected=True, funcs=normalize" in rendered assert "ids=" not in rendered @@ -314,9 +288,7 @@ def test_unregistered_or_unreadable_surface_points_to_the_catalog(): summary=surface_summary("plate_manager", readable=False), payload={}, warnings=(), - errors=( - AgentError(code="ui_state_surface_unreadable", message="Not readable."), - ), + errors=(AgentError(code="ui_state_surface_unreadable", message="Not readable."),), ) rendered = render(document) assert "Surface: plate_manager" in rendered @@ -327,68 +299,38 @@ def test_unregistered_or_unreadable_surface_points_to_the_catalog(): def tree_node(class_name: str, *children, **values) -> UiWidgetTreeNode: defaults = dict( - object_name="", - text=None, - title=None, - current_text=None, - visible=True, - enabled=True, - clickable=True, - actionable=False, - action_kinds=(), + object_name="", text=None, title=None, current_text=None, visible=True, + enabled=True, clickable=True, actionable=False, action_kinds=(), ) return replace( sample_dto(UiWidgetTreeNode), - class_name=class_name, - children=children, - **(defaults | values), + class_name=class_name, children=children, **(defaults | values), ) def test_widget_tree_outline_shows_addressable_paths_and_hides_technical_nodes(): root = tree_node( "PipelineEditorWidget", - tree_node( - "QPushButton", - text="Edit", - path_id="1.0.2", - action_kinds=("button",), - enabled=False, - clickable=False, - ), - tree_node( - "QPushButton", - text="Generate", - path_id="1.0.3", - action_kinds=("button",), - visible=False, - ), + tree_node("QPushButton", text="Edit", path_id="1.0.2", action_kinds=("button",), + enabled=False, clickable=False), + tree_node("QPushButton", text="Generate", path_id="1.0.3", + action_kinds=("button",), visible=False), tree_node("QScrollBar"), tree_node("QLineEdit", path_id="1.0.4", actionable=True, text="Gamma"), tree_node("QLabel", current_text="long " * 40), ) action = replace( sample_dto(UiWidgetActionSummary), - path_id="1.0.4", - object_state_scope_id="/plate::step", - field_path="gamma", + path_id="1.0.4", object_state_scope_id="/plate::step", field_path="gamma", semantic_markers=("*",), ) result = replace( sample_dto(UiWidgetTreeResult), - root=root, - actionable_widgets=(action,), - tree_truncated=False, - actionable_widgets_truncated=False, - warnings=(), - summary=replace( - sample_dto(UiWindowSummary), - title="Pipeline Editor", - dirty=False, - dirty_field_count=0, - signature_diff=True, - semantic_markers=(), - ), + root=root, actionable_widgets=(action,), tree_truncated=False, + actionable_widgets_truncated=False, warnings=(), + summary=replace(sample_dto(UiWindowSummary), title="Pipeline Editor", + dirty=False, dirty_field_count=0, signature_diff=True, + semantic_markers=()), ) rendered = render(result) assert "Window: Pipeline Editor" in rendered @@ -415,29 +357,22 @@ def test_widget_tree_outline_shows_addressable_paths_and_hides_technical_nodes() def test_code_document_bounds_its_source(): document = replace( - sample_dto(UiCodeDocument), - source="pattern = (some_func," + "x" * 50, - current_snapshot=snapshot(10), - warnings=(), + sample_dto(UiCodeDocument), source="pattern = (some_func," + "x" * 50, + current_snapshot=snapshot(10), warnings=(), ) rendered = render(document, CodeDocumentRenderOptions(max_source_chars=20)) assert "Revision: token=" in rendered and "snapshot=main@10 head=" in rendered assert "Source:\npattern = (some_func" in rendered assert "..." in rendered - assert "Source:" not in render( - document, CodeDocumentRenderOptions(include_source=False) - ) + assert "Source:" not in render(document, CodeDocumentRenderOptions(include_source=False)) def test_code_document_apply_shows_revisions_and_snapshots(): applied = replace( sample_dto(UiCodeDocumentApplyResult), - base_revision_token="rev-123", - current_revision_token="rev-456", - new_revision_token="rev-456", - current_snapshot=snapshot(10), - undo_snapshot=None, - warnings=(), + base_revision_token="rev-123", current_revision_token="rev-456", + new_revision_token="rev-456", current_snapshot=snapshot(10), + undo_snapshot=None, warnings=(), ) rendered = render(applied) assert "Revision: base=rev-123 current=rev-456 new=rev-456" in rendered @@ -458,20 +393,13 @@ def test_action_catalog_filters_actions_and_warnings_by_widget(): sample_dto(UiActionCatalog), actions=( ui_action( - "plate_manager", - "compile_datasets", - enabled=False, - disabled_error=AgentError( - code="no_plate", message="No plate.", hint="Add a plate." - ), + "plate_manager", "compile_datasets", enabled=False, + disabled_error=AgentError(code="no_plate", message="No plate.", hint="Add a plate."), ), ui_action("image_browser", "open_file"), ), warnings=( - AgentWarning( - code="plate_path_setup_uses_code_document", - message="Use plate_manager code.", - ), + AgentWarning(code="plate_path_setup_uses_code_document", message="Use plate_manager code."), AgentWarning(code="unrelated", message="Something else."), ), ) @@ -499,9 +427,7 @@ def test_action_catalog_filters_actions_and_warnings_by_widget(): def test_disabled_actions_do_not_make_the_call_fail(): catalog = replace( sample_dto(UiActionCatalog), - actions=( - ui_action("w", "a", disabled_error=AgentError(code="off", message="Off.")), - ), + actions=(ui_action("w", "a", disabled_error=AgentError(code="off", message="Off.")),), ) result = McpDevToolResult(tool="t", mcp_error=False, payloads=(catalog,)) assert not result.has_errors() @@ -512,12 +438,8 @@ def test_tool_list_filters_groups_and_flattens(): server=SERVER, tool_count=3, tools=( - McpDevToolMetadata( - "openhcs_get_viewer_window_state", "Read bounded viewer state.", {} - ), - McpDevToolMetadata( - "openhcs_validate_viewer_window_state", "Validate viewer.", {} - ), + McpDevToolMetadata("openhcs_get_viewer_window_state", "Read bounded viewer state.", {}), + McpDevToolMetadata("openhcs_validate_viewer_window_state", "Validate viewer.", {}), McpDevToolMetadata("openhcs_health_check", "Health.", {}), ), ) @@ -531,6 +453,4 @@ def test_tool_list_filters_groups_and_flattens(): assert "[Viewer Review]" not in flat projected = ToolListRenderer.project_response(tools, contains="viewer", limit=1) assert projected["matched_tool_count"] == 2 - assert [tool["name"] for tool in projected["tools"]] == [ - "openhcs_get_viewer_window_state" - ] + assert [tool["name"] for tool in projected["tools"]] == ["openhcs_get_viewer_window_state"] diff --git a/tests/unit/agent/test_mcp_dev_client_viewer_renderers.py b/tests/unit/agent/test_mcp_dev_client_viewer_renderers.py index 70520a7af9..7b8469ba12 100644 --- a/tests/unit/agent/test_mcp_dev_client_viewer_renderers.py +++ b/tests/unit/agent/test_mcp_dev_client_viewer_renderers.py @@ -50,57 +50,40 @@ def batch(capability, *payloads) -> McpDevToolBatchResponse: def typed(contract, **values): - return dataclass_from_mapping( - contract, {"schema_version": SCHEMA_VERSION, **values} - ) + return dataclass_from_mapping(contract, {"schema_version": SCHEMA_VERSION, **values}) def render(capability, payload, *command_args: str) -> str: """Render through the capability's command, as the CLI does.""" command = capability.cli_command args = _build_parser().parse_args((command, *command_args)) - return McpDevCommandSpec.for_name(command).render_result( - batch(capability, payload), args - ) + return McpDevCommandSpec.for_name(command).render_result(batch(capability, payload), args) def test_every_viewer_output_contract_has_its_own_typed_renderer(): contracts = ( - RuntimeServerScanResult, - RuntimeServerInfo, - RuntimeExecutionStatus, - ViewerWindowSnapshotResult, - ViewerWindowStateResult, - ViewerWindowPayloadResult, - ViewerWindowImageSampleResult, - ViewerWindowRoiSummaryResult, - ViewerWindowNavigationResult, - ViewerWindowLayerIsolationResult, - ViewerWindowProbeResult, - ViewerWindowValidationSummaryResult, + RuntimeServerScanResult, RuntimeServerInfo, RuntimeExecutionStatus, + ViewerWindowSnapshotResult, ViewerWindowStateResult, ViewerWindowPayloadResult, + ViewerWindowImageSampleResult, ViewerWindowRoiSummaryResult, + ViewerWindowNavigationResult, ViewerWindowLayerIsolationResult, + ViewerWindowProbeResult, ViewerWindowValidationSummaryResult, UiWindowSnapshotResult, ) - renderers = [ - McpDevOutputRenderer.for_output_contract(contract) for contract in contracts - ] + renderers = [McpDevOutputRenderer.for_output_contract(contract) for contract in contracts] assert all(renderer is not None for renderer in renderers) - assert all( - renderer.output_contract is contract - for renderer, contract in zip(renderers, contracts) - ) + assert all(renderer.output_contract is contract + for renderer, contract in zip(renderers, contracts)) assert len(set(renderers)) == len(contracts) def test_missing_payload_renders_unavailable_summary_and_diagnostic_once(): empty = McpDevToolBatchResponse( server=McpDevServerIdentity(command="python", module="openhcs.mcp.server"), - results=( - McpDevToolResult(agent_capabilities.navigate_viewer_window.name, False, ()), - ), + results=(McpDevToolResult( + agent_capabilities.navigate_viewer_window.name, False, ()),), ) rendered = McpDevOutputRenderer.for_output_contract( - ViewerWindowNavigationResult - ).render(empty) + ViewerWindowNavigationResult).render(empty) assert rendered.startswith("Viewer navigation: failed\nErrors:\n") assert "mcp_payload_missing" in rendered @@ -109,25 +92,11 @@ def test_validation_tool_boundary_error_is_the_whole_compact_output(): capability = agent_capabilities.validate_viewer_window_state raw = McpDevToolBatchResponse( server=McpDevServerIdentity(command="python", module="openhcs.mcp.server"), - results=( - McpDevToolResult( - capability.name, - False, - ( - { - "schema_version": SCHEMA_VERSION, - "ok": False, - "tool": capability.name, - "errors": [ - { - "code": "mcp_tool_failed", - "message": "Viewer MCP timeout must not exceed 2000ms.", - } - ], - }, - ), - ), - ), + results=(McpDevToolResult(capability.name, False, ({ + "schema_version": SCHEMA_VERSION, "ok": False, "tool": capability.name, + "errors": [{"code": "mcp_tool_failed", + "message": "Viewer MCP timeout must not exceed 2000ms."}], + },)),), ) args = _build_parser().parse_args(("validate-viewer", "5555")) assert McpDevCommandSpec.for_name("validate-viewer").render_result(raw, args) == ( @@ -139,105 +108,60 @@ def test_validation_tool_boundary_error_is_the_whole_compact_output(): def test_validation_reports_policy_layers_and_axis_component_hint(): payload = typed( ViewerWindowValidationSummaryResult, - valid=False, - observed=True, - layer_count=1, - mounted_layer_count=1, - payload_count=1, - nonzero_payload_count=1, - validation_policy={ - "expected_layer_count": 2, - "required_axis_labels": ["site", "well", "channel"], - }, + valid=False, observed=True, layer_count=1, mounted_layer_count=1, + payload_count=1, nonzero_payload_count=1, + validation_policy={"expected_layer_count": 2, + "required_axis_labels": ["site", "well", "channel"]}, connection={"host": "localhost", "port": 5555, "transport_mode": "ipc"}, - layer_summaries=[ - { - "route_key": "image-layer", - "title": "Image", - "valid": False, - "mounted": True, - "item_count": 1, - "axis_labels": ["channel", "y", "x"], - "component_labels": ["channel", "site", "well"], - "missing_required_axis_labels": ["site", "well"], - "axis_labels_present_as_components": ["site", "well"], - } - ], - warnings=[ - { - "code": "viewer_required_axis_labels_missing", - "message": "Viewer layer 'Image' is missing required axis labels: site, well.", - "hint": "These labels are present as component metadata rather than mounted axes: site, well.", - } - ], + layer_summaries=[{ + "route_key": "image-layer", "title": "Image", "valid": False, + "mounted": True, "item_count": 1, "axis_labels": ["channel", "y", "x"], + "component_labels": ["channel", "site", "well"], + "missing_required_axis_labels": ["site", "well"], + "axis_labels_present_as_components": ["site", "well"], + }], + warnings=[{ + "code": "viewer_required_axis_labels_missing", + "message": "Viewer layer 'Image' is missing required axis labels: site, well.", + "hint": "These labels are present as component metadata rather than mounted axes: site, well.", + }], ) rendered = render(agent_capabilities.validate_viewer_window_state, payload, "5555") assert "Viewer validation: valid=False observed=True layers=1 mounted=1" in rendered - assert ( - "Policy: expected_layers=2 required_axes=site,well,channel required_components=" - in rendered - ) + assert "Policy: expected_layers=2 required_axes=site,well,channel required_components=" in rendered assert "Connection: localhost:5555 transport=ipc" in rendered - assert ( - "- image-layer: valid=False mounted=True items=1 axes=channel,y,x" in rendered - ) - assert ( - "missing_axes=site,well components=channel,site,well missing_components=" - in rendered - ) + assert "- image-layer: valid=False mounted=True items=1 axes=channel,y,x" in rendered + assert "missing_axes=site,well components=channel,site,well missing_components=" in rendered assert "axis_as_components=site,well" in rendered - assert ( - "Warnings:\n- viewer_required_axis_labels_missing: " - "Viewer layer 'Image' is missing required axis labels: site, well. " - 'hint="These labels are present as component metadata rather than ' - 'mounted axes: site, well."' - ) in rendered + assert ('Warnings:\n- viewer_required_axis_labels_missing: ' + "Viewer layer 'Image' is missing required axis labels: site, well. " + 'hint="These labels are present as component metadata rather than ' + 'mounted axes: site, well."') in rendered def test_state_merges_component_values_and_bounds_payload_summaries(): payload = typed( ViewerWindowStateResult, - **PORT, - observed=True, + **PORT, observed=True, viewer={"viewer_type": "napari", "title": "OpenHCS Napari Visualization"}, - layer_count=1, - viewer_ndim=3, - axis_labels=["channel", "y", "x"], - current_step=[0, 47, 47], - active_dimension_label_route="image-layer", - component_group_count=1, - component_item_count=2, - layers=[ - { - "route_key": "image-layer", - "title": "1. Agent invert", - "mounted": True, - "visible": True, - "selected": True, - "item_count": 2, - "data_types": ["image"], - "axis_labels": ["channel", "y", "x"], - "component_values": [ - {"well": "A01", "channel": 1, "site": 1}, - {"well": "A01", "channel": 2, "site": 1}, - ], - "axis_component_values": {"channel": [1, 2]}, - } - ], + layer_count=1, viewer_ndim=3, axis_labels=["channel", "y", "x"], + current_step=[0, 47, 47], active_dimension_label_route="image-layer", + component_group_count=1, component_item_count=2, + layers=[{ + "route_key": "image-layer", "title": "1. Agent invert", "mounted": True, + "visible": True, "selected": True, "item_count": 2, + "data_types": ["image"], "axis_labels": ["channel", "y", "x"], + "component_values": [ + {"well": "A01", "channel": 1, "site": 1}, + {"well": "A01", "channel": 2, "site": 1}, + ], + "axis_component_values": {"channel": [1, 2]}, + }], ) rendered = render(agent_capabilities.get_viewer_window_state, payload, "5555") - assert ( - 'Viewer state: observed=True type=napari title="OpenHCS Napari Visualization"' - in rendered - ) - assert ( - "Window: layers=1 ndim=3 axes=channel,y,x current_step=[0, 47, 47] active_route=image-layer" - in rendered - ) - assert ( - '- image-layer: title="1. Agent invert" visible=True selected=True items=2 types=image' - in rendered - ) + assert 'Viewer state: observed=True type=napari title="OpenHCS Napari Visualization"' in rendered + assert "Window: layers=1 ndim=3 axes=channel,y,x current_step=[0, 47, 47] active_route=image-layer" in rendered + assert '- image-layer: title="1. Agent invert" visible=True selected=True items=2 types=image' in rendered assert " components: channel=1,2, site=1, well=A01" in rendered assert " axis values: channel=1,2" in rendered @@ -246,74 +170,35 @@ def test_payloads_mark_streamed_paths_and_shape_payload_counts(tmp_path): streamed_path = tmp_path / "streamed_A01_w1.tif" payload = typed( ViewerWindowPayloadResult, - **PORT, - observed=True, - layer_count=2, + **PORT, observed=True, layer_count=2, layers=[ - { - "route_key": "image-layer", - "title": "Images", - "mounted": True, - "item_count": 2, - "axis_labels": ["channel", "y", "x"], - "stack_axes": ["channel"], - "payloads": [ - { - "route_key": "image-layer:0", - "data_type": "image", - "axis_indices": [0], - "components": {"well": "A01", "channel": 1}, - "path": str(streamed_path), - "summary": { - "shape": [96, 96], - "dtype": "uint16", - "nonzero_count": 9216, - }, - "array_value_summary": { - "included": False, - "shape": [96, 96], - "omitted_reason": "max_array_elements_exceeded", - }, - } - ], - }, - { - "route_key": "roi-layer", - "title": "ROIs", - "mounted": True, - "item_count": 1, - "axis_labels": ["y", "x"], - "payloads": [ - { - "route_key": "roi-layer:0", - "data_type": "shapes", - "components": {"channel": 2}, - "path": "relative/A01_w2.roi.zip", - "summary": {"shape_payload_count": 7, "nonzero_count": 7}, - "shape_payloads": [ - {"metadata": {"label": 1}}, - {"metadata": {"label": 2}}, - ], - } - ], - }, + {"route_key": "image-layer", "title": "Images", "mounted": True, + "item_count": 2, "axis_labels": ["channel", "y", "x"], + "stack_axes": ["channel"], + "payloads": [{ + "route_key": "image-layer:0", "data_type": "image", "axis_indices": [0], + "components": {"well": "A01", "channel": 1}, + "path": str(streamed_path), + "summary": {"shape": [96, 96], "dtype": "uint16", "nonzero_count": 9216}, + "array_value_summary": {"included": False, "shape": [96, 96], + "omitted_reason": "max_array_elements_exceeded"}, + }]}, + {"route_key": "roi-layer", "title": "ROIs", "mounted": True, + "item_count": 1, "axis_labels": ["y", "x"], + "payloads": [{ + "route_key": "roi-layer:0", "data_type": "shapes", "components": {"channel": 2}, + "path": "relative/A01_w2.roi.zip", + "summary": {"shape_payload_count": 7, "nonzero_count": 7}, + "shape_payloads": [{"metadata": {"label": 1}}, {"metadata": {"label": 2}}], + }]}, ], ) rendered = render(agent_capabilities.get_viewer_window_payloads, payload, "5555") assert "Viewer payloads: observed=True layers=2" in rendered - assert ( - '- image-layer: title="Images" mounted=True items=2 axes=channel,y,x stack=channel payloads=1' - in rendered - ) - assert ( - "payload type=image axis=0 aggregate_axis= components=channel=1, well=A01" - in rendered - ) + assert '- image-layer: title="Images" mounted=True items=2 axes=channel,y,x stack=channel payloads=1' in rendered + assert "payload type=image axis=0 aggregate_axis= components=channel=1, well=A01" in rendered assert "shape=[96, 96] dtype=uint16 nonzero=9216" in rendered - assert ( - "array=included=False sample_shape=[96, 96] reason=max_array_elements_exceeded" - in rendered - ) + assert "array=included=False sample_shape=[96, 96] reason=max_array_elements_exceeded" in rendered assert f"path={streamed_path} (streamed/non-materialized)" in rendered assert "payload type=shapes axis=" in rendered assert "shape_members=7 returned_shapes=2 semantic_rois=use-viewer-rois" in rendered @@ -322,25 +207,15 @@ def test_payloads_mark_streamed_paths_and_shape_payload_counts(tmp_path): ROI_PAYLOAD = { - "layer_route_key": "roi-layer", - "layer_title": "ROIs", - "payload_route_key": "roi-layer:0", - "path": "/tmp/A01.roi.zip", - "components": {"well": "A01", "channel": 1}, - "axis_indices": [0, 1], - "roi_count": 3, - "returned_roi_count": 3, - "roi_count_exact": False, - "roi_member_count": 7, - "returned_roi_member_count": 3, - "roi_duplicate_member_count": 0, - "roi_payloads_truncated": True, + "layer_route_key": "roi-layer", "layer_title": "ROIs", + "payload_route_key": "roi-layer:0", "path": "/tmp/A01.roi.zip", + "components": {"well": "A01", "channel": 1}, "axis_indices": [0, 1], + "roi_count": 3, "returned_roi_count": 3, "roi_count_exact": False, + "roi_member_count": 7, "returned_roi_member_count": 3, + "roi_duplicate_member_count": 0, "roi_payloads_truncated": True, "area": {"min": 10.0, "median": 42.0, "mean": 40.0, "max": 80.0}, - "perimeter": None, - "bounds_yx": None, - "coordinate_count": 128, - "spatial_origin_yx": [0, 0], - "source_spatial_shape_yx": [96, 96], + "perimeter": None, "bounds_yx": None, "coordinate_count": 128, + "spatial_origin_yx": [0, 0], "source_spatial_shape_yx": [96, 96], "out_of_source_bounds_count": 0, "example_rois": [{"label": "cell-1", "area": 42, "centroid_yx": [5, 6]}], } @@ -349,37 +224,21 @@ def test_payloads_mark_streamed_paths_and_shape_payload_counts(tmp_path): def test_rois_render_payload_statistics_and_examples(): payload = typed( ViewerWindowRoiSummaryResult, - observed=True, - route_key="roi-layer", - axis_indices=[0, 1], - layer_count=2, - payload_record_count=1, - payload_type_counts={"shapes": 1}, - roi_payload_count=1, - total_roi_count=3, - returned_roi_count=3, - roi_count_exact=False, - total_roi_member_count=7, - returned_roi_member_count=3, - roi_payloads_truncated=True, - payloads=[ROI_PAYLOAD], - ) - rendered = render( - agent_capabilities.summarize_viewer_window_rois, payload, "5555", "roi-layer" + observed=True, route_key="roi-layer", axis_indices=[0, 1], layer_count=2, + payload_record_count=1, payload_type_counts={"shapes": 1}, roi_payload_count=1, + total_roi_count=3, returned_roi_count=3, roi_count_exact=False, + total_roi_member_count=7, returned_roi_member_count=3, + roi_payloads_truncated=True, payloads=[ROI_PAYLOAD], ) + rendered = render(agent_capabilities.summarize_viewer_window_rois, payload, "5555", "roi-layer") assert "Viewer ROIs: observed=True route=roi-layer axis=0,1" in rendered assert "ROIs: total=3 returned=3 exact=False members=7/3 truncated=True" in rendered assert "Payload types: shapes=1" in rendered - assert ( - '- title="ROIs" layer_route=roi-layer payload_route=roi-layer:0 axis=0,1 ' - "components=channel=1, well=A01 roi_count=3 returned=3 exact=False " - "members=7/3 duplicate_members=0" - ) in rendered + assert ('- title="ROIs" layer_route=roi-layer payload_route=roi-layer:0 axis=0,1 ' + "components=channel=1, well=A01 roi_count=3 returned=3 exact=False " + "members=7/3 duplicate_members=0") in rendered assert "area=min=10.0,median=42.0,mean=40.0,max=80.0 perimeter=" in rendered - assert ( - "coords=128 source_origin=[0, 0] source_shape=[96, 96] out_of_bounds=0" - in rendered - ) + assert "coords=128 source_origin=[0, 0] source_shape=[96, 96] out_of_bounds=0" in rendered assert "example label=cell-1 area=42 centroid=[5, 6]" in rendered @@ -387,23 +246,14 @@ def test_rois_render_payload_statistics_and_examples(): def test_rois_explain_missing_roi_payloads_for_the_requested_scope(route_key): payload = typed( ViewerWindowRoiSummaryResult, - observed=True, - route_key=route_key, - layer_count=7, - payload_record_count=32, - payload_type_counts={"image": 32}, + observed=True, route_key=route_key, layer_count=7, + payload_record_count=32, payload_type_counts={"image": 32}, ) rendered = render(agent_capabilities.summarize_viewer_window_rois, payload, "5555") - assert ( - f"Viewer ROIs: observed=True route={route_key or ''} axis= layers=7 records=32 payloads=0" - in rendered - ) + assert f"Viewer ROIs: observed=True route={route_key or ''} axis= layers=7 records=32 payloads=0" in rendered assert "Payload types: image=32" in rendered suffix = " route." if route_key else "." - assert ( - f"Interpretation: no ROI/shapes payloads were found for the requested viewer{suffix}" - in rendered - ) + assert f"Interpretation: no ROI/shapes payloads were found for the requested viewer{suffix}" in rendered rerun = "- If `viewer-state` shows a shapes layer, rerun `viewer-rois ` for that route." assert (rerun in rendered) is (route_key is None) assert "- If payload types are image-only" in rendered @@ -411,8 +261,7 @@ def test_rois_explain_missing_roi_payloads_for_the_requested_scope(route_key): def test_rois_with_errors_do_not_explain_absence(): payload = typed( - ViewerWindowRoiSummaryResult, - observed=False, + ViewerWindowRoiSummaryResult, observed=False, errors=[{"code": "viewer_window_rois_failed", "message": "timed out"}], ) rendered = render(agent_capabilities.summarize_viewer_window_rois, payload, "5555") @@ -423,69 +272,34 @@ def test_rois_with_errors_do_not_explain_absence(): def image_sample(max_array_elements: int, *, included: bool = False, values=()): return typed( ViewerWindowImageSampleResult, - observed=True, - route_key="image-layer", - axis_indices={"channel": 1}, - array_slices=[[0, 8], [0, 8]], - record_count=1, - returned_record_count=1, - raw_image_record_count=1, - total_payload_record_count=2, + observed=True, route_key="image-layer", axis_indices={"channel": 1}, + array_slices=[[0, 8], [0, 8]], record_count=1, returned_record_count=1, + raw_image_record_count=1, total_payload_record_count=2, sample_protocol_supported=True, - sample_included_count=int(included), - sample_omitted_count=int(not included), - records=[ - { - "layer_title": "image-layer", - "data_type": "image", - "components": {}, - "payload_route_key": "image-layer:0", - "layer_route_key": "image-layer", - "axis_indices": [1], - "path": "virtual/image.tif", - "summary": { - "shape": [96, 96], - "dtype": "uint16", - "min": 23, - "max": 10755, - "nonzero_count": 9216, - }, - "array_value_summary": { - "included": included, - "shape": [8, 8], - **( - {} - if included - else {"omitted_reason": "max_array_elements_exceeded"} - ), - "max_array_elements": max_array_elements, - }, - "array_values": list(values), - } - ], + sample_included_count=int(included), sample_omitted_count=int(not included), + records=[{ + "layer_title": "image-layer", "data_type": "image", "components": {}, + "payload_route_key": "image-layer:0", "layer_route_key": "image-layer", + "axis_indices": [1], "path": "virtual/image.tif", + "summary": {"shape": [96, 96], "dtype": "uint16", "min": 23, + "max": 10755, "nonzero_count": 9216}, + "array_value_summary": { + "included": included, "shape": [8, 8], + **({} if included else {"omitted_reason": "max_array_elements_exceeded"}), + "max_array_elements": max_array_elements, + }, + "array_values": list(values), + }], ) def test_image_sample_includes_small_sample_values(): sample = image_sample(64, included=True, values=([1, 2], [3, 4])) - rendered = render( - agent_capabilities.sample_viewer_window_image, - sample, - "5555", - "image-layer", - "--include-array-values", - ) - assert ( - "Viewer image sample: observed=True route=image-layer axis=channel=1 slices=[[0, 8], [0, 8]]" - in rendered - ) - assert ( - "Records: matched=1 returned=1 truncated=0 image=1 total_payloads=2 sample_supported=True" - in rendered - ) - assert ( - "- image-layer:0: layer=image-layer axis=1 path=virtual/image.tif" in rendered - ) + rendered = render(agent_capabilities.sample_viewer_window_image, sample, + "5555", "image-layer", "--include-array-values") + assert "Viewer image sample: observed=True route=image-layer axis=channel=1 slices=[[0, 8], [0, 8]]" in rendered + assert "Records: matched=1 returned=1 truncated=0 image=1 total_payloads=2 sample_supported=True" in rendered + assert "- image-layer:0: layer=image-layer axis=1 path=virtual/image.tif" in rendered assert "shape=[96, 96] dtype=uint16 min=23 max=10755 nonzero=9216" in rendered assert "included=True sample_shape=[8, 8]" in rendered assert "sample values: [[1, 2], [3, 4]]" in rendered @@ -494,29 +308,15 @@ def test_image_sample_includes_small_sample_values(): @pytest.mark.parametrize( "command_args,budget,expected", ( - ( - (), - 0, - "reason=array_values_not_requested max_elements=0 " - "rerun_with=--include-array-values --max-array-elements 64", - ), - ( - ("--include-array-values", "--max-array-elements", "40"), - 40, - "reason=max_array_elements_exceeded max_elements=40 rerun_max_elements=64", - ), + ((), 0, "reason=array_values_not_requested max_elements=0 " + "rerun_with=--include-array-values --max-array-elements 64"), + (("--include-array-values", "--max-array-elements", "40"), 40, + "reason=max_array_elements_exceeded max_elements=40 rerun_max_elements=64"), ), ) -def test_image_sample_omission_names_the_rerun_that_includes_values( - command_args, budget, expected -): - rendered = render( - agent_capabilities.sample_viewer_window_image, - image_sample(budget), - "5555", - "image-layer", - *command_args, - ) +def test_image_sample_omission_names_the_rerun_that_includes_values(command_args, budget, expected): + rendered = render(agent_capabilities.sample_viewer_window_image, image_sample(budget), + "5555", "image-layer", *command_args) assert "axis={'channel': 1}" not in rendered assert "included=False sample_shape=[8, 8]" in rendered assert expected in rendered @@ -525,292 +325,146 @@ def test_image_sample_omission_names_the_rerun_that_includes_values( def test_navigation_and_isolation_report_position_layers_and_grouped_errors(): navigation = typed( ViewerWindowNavigationResult, - **PORT, - observed=True, - route_key="roi-layer", - visible=True, - selected=True, - axis_labels=["channel", "y", "x"], - current_step=[1, 0, 0], + **PORT, observed=True, route_key="roi-layer", visible=True, selected=True, + axis_labels=["channel", "y", "x"], current_step=[1, 0, 0], active_dimension_label_route="roi-layer", available_layers=[ - { - "route_key": "image-layer", - "title": "Image", - "visible": True, - "selected": False, - }, - { - "route_key": "roi-layer", - "title": "ROIs", - "visible": True, - "selected": True, - }, - ], - warnings=[ - {"code": "viewer_axis_missing", "message": "z_index is not mounted."} + {"route_key": "image-layer", "title": "Image", "visible": True, "selected": False}, + {"route_key": "roi-layer", "title": "ROIs", "visible": True, "selected": True}, ], + warnings=[{"code": "viewer_axis_missing", "message": "z_index is not mounted."}], ) - rendered = render( - agent_capabilities.navigate_viewer_window, navigation, "5555", "roi-layer" - ) - assert ( - "Viewer navigation: observed=True route=roi-layer visible=True selected=True" - in rendered - ) - assert ( - "Position: axes=channel,y,x current_step=[1, 0, 0] active_route=roi-layer" - in rendered - ) - assert ( - 'Available layers:\n- image-layer: visible=True selected=False title="Image"\n' - '- roi-layer: visible=True selected=True title="ROIs"' - ) in rendered + rendered = render(agent_capabilities.navigate_viewer_window, navigation, "5555", "roi-layer") + assert "Viewer navigation: observed=True route=roi-layer visible=True selected=True" in rendered + assert "Position: axes=channel,y,x current_step=[1, 0, 0] active_route=roi-layer" in rendered + assert ('Available layers:\n- image-layer: visible=True selected=False title="Image"\n' + '- roi-layer: visible=True selected=True title="ROIs"') in rendered assert "Warnings:\n- viewer_axis_missing: z_index is not mounted." in rendered isolation = typed( ViewerWindowLayerIsolationResult, - **PORT, - observed=True, - applied=True, - selected_route_key="roi-layer", - changed_route_count=2, - layer_count=2, - axis_labels=["channel", "y", "x"], - current_step=[1, 0, 0], - visible_route_keys=["roi-layer"], - hidden_route_keys=["image-layer"], - missing_route_keys=["gone"], - visible_layers=[ - { - "route_key": "roi-layer", - "title": "ROIs", - "visible": True, - "selected": True, - } - ], - ) - rendered = render( - agent_capabilities.isolate_viewer_window_layers, isolation, "5555", "roi-layer" - ) - assert ( - "Viewer isolation: observed=True applied=True selected=roi-layer changed=2 layers=2" - in rendered - ) + **PORT, observed=True, applied=True, selected_route_key="roi-layer", + changed_route_count=2, layer_count=2, axis_labels=["channel", "y", "x"], + current_step=[1, 0, 0], visible_route_keys=["roi-layer"], + hidden_route_keys=["image-layer"], missing_route_keys=["gone"], + visible_layers=[{"route_key": "roi-layer", "title": "ROIs", + "visible": True, "selected": True}], + ) + rendered = render(agent_capabilities.isolate_viewer_window_layers, isolation, "5555", "roi-layer") + assert "Viewer isolation: observed=True applied=True selected=roi-layer changed=2 layers=2" in rendered assert "Visible: roi-layer\nHidden: image-layer\nMissing routes: gone" in rendered - assert ( - 'Visible layers:\n- roi-layer: visible=True selected=True title="ROIs"' - in rendered - ) + assert 'Visible layers:\n- roi-layer: visible=True selected=True title="ROIs"' in rendered failed = typed( - ViewerWindowLayerIsolationResult, - **PORT, - observed=False, + ViewerWindowLayerIsolationResult, **PORT, observed=False, selected_route_key="roi-layer", errors=[ - { - "code": "viewer_window_navigation_failed", - "message": "Viewer control request timed out.", - }, - { - "code": "viewer_window_navigation_failed", - "message": "Viewer control request timed out.", - }, - { - "code": "viewer_window_state_failed", - "message": "Viewer control request timed out.", - }, + {"code": "viewer_window_navigation_failed", "message": "Viewer control request timed out."}, + {"code": "viewer_window_navigation_failed", "message": "Viewer control request timed out."}, + {"code": "viewer_window_state_failed", "message": "Viewer control request timed out."}, ], ) - rendered = render( - agent_capabilities.isolate_viewer_window_layers, failed, "5555", "roi-layer" - ) - assert ( - "- viewer_window_navigation_failed, viewer_window_state_failed: " - "Viewer control request timed out." - ) in rendered + rendered = render(agent_capabilities.isolate_viewer_window_layers, failed, "5555", "roi-layer") + assert ("- viewer_window_navigation_failed, viewer_window_state_failed: " + "Viewer control request timed out.") in rendered assert rendered.count("Viewer control request timed out.") == 1 def test_snapshots_render_image_and_resource_facts(): - resource = { - "path": "/tmp/snapshots/viewer.png", - "uri": "file:///tmp/snapshots/viewer.png", - "title": "viewer.png", - "mime_type": "image/png", - "size_bytes": 12345, - "sha256": "abc123", - } + resource = {"path": "/tmp/snapshots/viewer.png", "uri": "file:///tmp/snapshots/viewer.png", + "title": "viewer.png", "mime_type": "image/png", "size_bytes": 12345, + "sha256": "abc123"} viewer = typed( - ViewerWindowSnapshotResult, - **PORT, - output_dir_path="/tmp/snapshots", - captured=True, - capture_scope="widget", + ViewerWindowSnapshotResult, **PORT, output_dir_path="/tmp/snapshots", + captured=True, capture_scope="widget", viewer={"viewer_type": "napari", "title": "Viewer"}, - width=640, - height=480, - resource=resource, + width=640, height=480, resource=resource, ) rendered = render(agent_capabilities.viewer_snapshot_window, viewer, "5555") - assert ( - 'Viewer snapshot: captured=True type=napari title="Viewer" scope=widget' - in rendered - ) + assert 'Viewer snapshot: captured=True type=napari title="Viewer" scope=widget' in rendered assert "Image: size=640x480 bytes=12345 mime=image/png" in rendered - assert ( - "Resource: path=/tmp/snapshots/viewer.png " - "uri=file:///tmp/snapshots/viewer.png sha256=abc123" - ) in rendered + assert ("Resource: path=/tmp/snapshots/viewer.png " + "uri=file:///tmp/snapshots/viewer.png sha256=abc123") in rendered window = typed( - UiWindowSnapshotResult, - output_dir_path="/tmp/snapshots", - captured=True, - window_id="global_config", - capture_scope="window", - width=550, - height=600, - resource=resource, + UiWindowSnapshotResult, output_dir_path="/tmp/snapshots", + captured=True, window_id="global_config", capture_scope="window", + width=550, height=600, resource=resource, summary={ - "schema_version": SCHEMA_VERSION, - "identity": {"window_id": "global_config"}, - "title": "Configuration - GlobalPipelineConfig", - "window_kind": "scope", - "visible": True, - "focusable": True, - "signature_diff": True, - "signature_diff_field_count": 3, - "semantic_markers": ["_"], + "schema_version": SCHEMA_VERSION, "identity": {"window_id": "global_config"}, + "title": "Configuration - GlobalPipelineConfig", "window_kind": "scope", + "visible": True, "focusable": True, "signature_diff": True, + "signature_diff_field_count": 3, "semantic_markers": ["_"], "object_state_scope_id": "global_config", "managed_action_ids": ["save_and_close", "save_without_close"], }, ) capability = agent_capabilities.ui_snapshot_window rendered = render(capability, window, "global_config") - assert ( - "Window snapshot: captured=True window=global_config " - 'title="Configuration - GlobalPipelineConfig" kind=scope scope=window' - ) in rendered - assert ( - "Status: visible=True dirty=False dirty_fields=0 default_diff=True " - "default_diff_fields=3 markers=_" - ) in rendered + assert ("Window snapshot: captured=True window=global_config " + 'title="Configuration - GlobalPipelineConfig" kind=scope scope=window') in rendered + assert ("Status: visible=True dirty=False dirty_fields=0 default_diff=True " + "default_diff_fields=3 markers=_") in rendered assert "Image: size=550x600 bytes=12345 mime=image/png" in rendered assert "ObjectState: scope=global_config" in rendered assert "Actions: save_and_close,save_without_close" in rendered call_args = _build_parser().parse_args( - ("call", capability.name, "--arguments", '{"window_id":"global_config"}') - ) - call_rendered = McpDevCommandSpec.for_name("call").render_result( - batch(capability, window), call_args - ) + ("call", capability.name, "--arguments", '{"window_id":"global_config"}')) + call_rendered = McpDevCommandSpec.for_name("call").render_result(batch(capability, window), call_args) assert call_rendered == rendered assert '"payloads"' not in call_rendered def test_runtime_scan_info_and_status_render_server_facts(): - napari = dict( - schema_version=SCHEMA_VERSION, - connection={"port": 5555}, - server="NapariViewer", - reachable=True, - ready=True, - control_port=6555, - log_file_path="/tmp/napari.log", - ) - execution = dict( - schema_version=SCHEMA_VERSION, - connection={"port": 7777}, - server="ZMQExecutionServer", - reachable=True, - ready=True, - control_port=8777, - active_executions=1, - uptime=12.34, - log_file_path="/tmp/exec.log", - ) - scan = typed( - RuntimeServerScanResult, - ports=[5555, 7777], - timeout_ms=300, - servers=[napari, execution], - ) + napari = dict(schema_version=SCHEMA_VERSION, connection={"port": 5555}, + server="NapariViewer", reachable=True, ready=True, control_port=6555, + log_file_path="/tmp/napari.log") + execution = dict(schema_version=SCHEMA_VERSION, connection={"port": 7777}, + server="ZMQExecutionServer", reachable=True, ready=True, + control_port=8777, active_executions=1, uptime=12.34, + log_file_path="/tmp/exec.log") + scan = typed(RuntimeServerScanResult, ports=[5555, 7777], timeout_ms=300, + servers=[napari, execution]) rendered = render(agent_capabilities.scan_runtime_servers, scan, "5555", "7777") assert "Runtime scan: ports=5555,7777 timeout_ms=300 servers=2" in rendered - assert ( - "port=5555 server=NapariViewer reachable=True ready=True control=6555" - in rendered - ) - assert ( - "port=7777 server=ZMQExecutionServer reachable=True ready=True control=8777 " - "active=1 running=0 queued=0 workers=0 uptime=12.3s log=/tmp/exec.log" - ) in rendered + assert "port=5555 server=NapariViewer reachable=True ready=True control=6555" in rendered + assert ("port=7777 server=ZMQExecutionServer reachable=True ready=True control=8777 " + "active=1 running=0 queued=0 workers=0 uptime=12.3s log=/tmp/exec.log") in rendered unreachable = RuntimeServerInfo( - schema_version=SCHEMA_VERSION, - connection=ExecutionConnectionSpec(port=7777), - reachable=False, - errors=(AgentError(code="runtime_unreachable", message="no pong"),), + schema_version=SCHEMA_VERSION, connection=ExecutionConnectionSpec(port=7777), + reachable=False, errors=(AgentError(code="runtime_unreachable", message="no pong"),), ) rendered = McpDevOutputRenderer.for_output_contract(RuntimeServerInfo).render( - batch(agent_capabilities.get_runtime_server_info, unreachable) - ) - assert rendered.startswith( - "Runtime server:\n- port=7777 server= reachable=False" - ) + batch(agent_capabilities.get_runtime_server_info, unreachable)) + assert rendered.startswith("Runtime server:\n- port=7777 server= reachable=False") assert rendered.endswith("Errors:\n- runtime_unreachable: no pong") status = RuntimeExecutionStatus( - schema_version=SCHEMA_VERSION, - connection=ExecutionConnectionSpec(port=7777), - execution_id="run-1", - status="ok", - response={ - "status": "ok", - "active_executions": 1, - "uptime": 12.34, - "executions": ["run-1", "run-2"], - "queued_executions": [], - "running_executions": [ - { - "execution_id": "run-1", - "subject_id": "p", - "start_time": 1.0, - "elapsed": 2.0, - } - ], - }, + schema_version=SCHEMA_VERSION, connection=ExecutionConnectionSpec(port=7777), + execution_id="run-1", status="ok", + response={"status": "ok", "active_executions": 1, "uptime": 12.34, + "executions": ["run-1", "run-2"], "queued_executions": [], + "running_executions": [{"execution_id": "run-1", "subject_id": "p", + "start_time": 1.0, "elapsed": 2.0}]}, ) rendered = RuntimeExecutionStatusRenderer.render( - batch(agent_capabilities.get_runtime_server_execution_status, status) - ) - assert ( - "Runtime execution status: status=ok execution_id=run-1 port=7777" in rendered - ) + batch(agent_capabilities.get_runtime_server_execution_status, status)) + assert "Runtime execution status: status=ok execution_id=run-1 port=7777" in rendered assert "Executions: known=2 active=1 running=1 queued=0 uptime=12.3s" in rendered def test_probe_presents_false_zero_and_empty_facts(): probe = ViewerWindowProbeResult( - schema_version=SCHEMA_VERSION, - connection=ExecutionConnectionSpec(port=5555), - reachable=True, - observed=True, - layer_count=2, - component_group_count=2, + schema_version=SCHEMA_VERSION, connection=ExecutionConnectionSpec(port=5555), + reachable=True, observed=True, layer_count=2, component_group_count=2, component_item_count=4, - viewer=ViewerWindowDescriptor( - ViewerType.NAPARI, "OpenHCS Napari Visualization" - ), + viewer=ViewerWindowDescriptor(ViewerType.NAPARI, "OpenHCS Napari Visualization"), warnings=(AgentWarning(code="notice", message="shown"),), ) rendered = render(agent_capabilities.probe_viewer_window, probe, "5555") - assert ( - "Viewer probe: reachable=True observed=True type=napari " - 'title="OpenHCS Napari Visualization"' - ) in rendered + assert ('Viewer probe: reachable=True observed=True type=napari ' + 'title="OpenHCS Napari Visualization"') in rendered assert "Window: port=5555 layers=2 component_groups=2 component_items=4" in rendered assert rendered.endswith("Warnings:\n- notice: shown") @@ -822,12 +476,7 @@ def test_viewer_renderers_read_no_record_by_key(): import openhcs.mcp.dev_client_renderers.viewer as module tree = ast.parse(inspect.getsource(module)) - calls = [ - node - for node in ast.walk(tree) - if isinstance(node, ast.Call) - and isinstance(node.func, ast.Attribute) - and node.func.attr == "get" - ] + calls = [node for node in ast.walk(tree) if isinstance(node, ast.Call) + and isinstance(node.func, ast.Attribute) and node.func.attr == "get"] assert calls == [] assert issubclass(module.ViewerProbeRenderer, ViewerResultRenderer) diff --git a/tests/unit/agent/test_mcp_plate_stream_progress.py b/tests/unit/agent/test_mcp_plate_stream_progress.py index fb954b6a45..9c71292a29 100644 --- a/tests/unit/agent/test_mcp_plate_stream_progress.py +++ b/tests/unit/agent/test_mcp_plate_stream_progress.py @@ -18,10 +18,8 @@ from polystore.streaming.viewer_transport import ViewerStreamKwarg from openhcs.agent.capabilities import ( - AgentCapabilityDeclaration, - MainThreadProgressCapability, - StreamPlateFilesToViewerCapability, - UiStreamSelectedPlateFilesToViewerCapability, + AgentCapabilityDeclaration, MainThreadProgressCapability, + StreamPlateFilesToViewerCapability, UiStreamSelectedPlateFilesToViewerCapability, ) from openhcs.agent.services.plate_streaming_service import PlateStreamingService from openhcs.core.roi_source_metadata import ROIArchiveSourceMetadata @@ -32,10 +30,7 @@ from openhcs.mcp.server import build_server from openhcs.mcp.socket import McpSocketTransport, wait_for_socket -from tests.unit.agent.test_persisted_output_reopen import ( - NoRuntimeBridge, - declared_output, -) +from tests.unit.agent.test_persisted_output_reopen import NoRuntimeBridge, declared_output from tests.unit.test_streaming_service import FakeViewer @@ -49,26 +44,19 @@ def saved_roi_stream(tmp_path, monkeypatch): archive = result_root / "not-an-acquisition.roi.zip" geometry = [ROI([PointShape(12.5, 8.25)], {"label": 7})] DiskStorageBackend().save( - ROIArchiveSourceMetadata.bind(geometry, projection.image_metadata), - archive, + ROIArchiveSourceMetadata.bind(geometry, projection.image_metadata), archive, ) persisted = ROIArchiveSourceMetadata.decode(DiskStorageBackend().load(archive)) assert persisted.source_provenance == projection.image_metadata.source_provenance - assert ( - persisted.source_voxel_spacing == projection.image_metadata.source_voxel_spacing - ) + assert persisted.source_voxel_spacing == projection.image_metadata.source_voxel_spacing unsigned = result_root / "missing-source.roi.zip" DiskStorageBackend().save(geometry, unsigned) # Reuse the original path authority, explicitly adding this synthetic root. from openhcs.agent.path_policy import AgentPathPolicy from openhcs.agent.services.plate_inspection_service import PlateInspectionService - - inspection = PlateInspectionService( - AgentPathPolicy.with_roots( - readable_roots=(tmp_path,), - writable_roots=(), - ) - ) + inspection = PlateInspectionService(AgentPathPolicy.with_roots( + readable_roots=(tmp_path,), writable_roots=(), + )) viewer = FakeViewer() received = [] timings = [] @@ -87,21 +75,12 @@ def receive(_manager, data, paths, backend, **fields): restored = ImagePayloadMetadata.from_viewer_image_metadata( request.source.item_fields["image_metadata"] ) - assert ( - restored.to_viewer_image_metadata() - == projection.image_metadata.to_viewer_image_metadata() - ) + assert restored.to_viewer_image_metadata() == projection.image_metadata.to_viewer_image_metadata() assert request.source.metadata.metadata_by_path[str(archive)] == dict( persisted.source_provenance.scalar_source_identity.component_metadata, ) - assert ( - restored.source_voxel_spacing - == projection.image_metadata.source_voxel_spacing - ) - assert ( - restored.source_spatial_domain - == projection.image_metadata.source_spatial_domain - ) + assert restored.source_voxel_spacing == projection.image_metadata.source_voxel_spacing + assert restored.source_spatial_domain == projection.image_metadata.source_spatial_domain # The archive travels as saved: its geometry plus the bound source # declaration, which the viewer reads and keeps out of the feature table. assert [ROIArchiveSourceMetadata.geometry(rois) for rois in data] == [geometry] @@ -119,30 +98,22 @@ def settle(): return True monkeypatch.setattr( - "openhcs.agent.services.plate_streaming_service.StreamingViewerLifecycle.get_or_create_visualizer", - acquire, + "openhcs.agent.services.plate_streaming_service.StreamingViewerLifecycle.get_or_create_visualizer", acquire, ) monkeypatch.setattr(FileManager, "save_batch", receive) monkeypatch.setattr(viewer, "settle_viewer_state", settle) service = PlateStreamingService(inspection, NoRuntimeBridge()) arguments = { "plate_path": str(source_path.parent.parent), - "result_directory": str(result_root), - "kind": "result", - "limit": 1, - "file_paths": [archive.name], - "fresh_viewer": False, + "result_directory": str(result_root), "kind": "result", "limit": 1, + "file_paths": [archive.name], "fresh_viewer": False, } return service, arguments, identity, received, timings, unsigned -@pytest.mark.parametrize( - "declaration", - ( - StreamPlateFilesToViewerCapability, - UiStreamSelectedPlateFilesToViewerCapability, - ), -) +@pytest.mark.parametrize("declaration", ( + StreamPlateFilesToViewerCapability, UiStreamSelectedPlateFilesToViewerCapability, +)) def test_stream_declaration_composes_original_progress_and_main_affinity(declaration): assert issubclass(declaration, MainThreadProgressCapability) spec = declaration @@ -153,15 +124,12 @@ def test_stream_declaration_composes_original_progress_and_main_affinity(declara def test_saved_roi_continuous_sdk_receipt_error_reuse_beyond_ordinary_idle( - unix_socket_dir, - saved_roi_stream, + unix_socket_dir, saved_roi_stream, ): service, arguments, identity, received, timings, unsigned = saved_roi_stream transport = McpSocketTransport(unix_socket_dir / "stream.sock") - built = build_server( - OpenHCSAgentContext(plate_streaming_service=service), - main_thread_dispatcher=transport.execution.dispatcher, - ) + built = build_server(OpenHCSAgentContext(plate_streaming_service=service), + main_thread_dispatcher=transport.execution.dispatcher) clients = AsyncOperationExecutor(max_workers=1) diagnostics = io.StringIO() events = [] @@ -170,72 +138,37 @@ class ObservedSession(McpDevSocketSession): def record_progress_notification(self, notification): super().record_progress_notification(notification) assert threading.current_thread() is not threading.main_thread() - events.append( - ( - time.monotonic(), - notification.params.progressToken, - notification.params.message, - ) - ) + events.append((time.monotonic(), notification.params.progressToken, notification.params.message)) async def exercise(): - assert await asyncio.to_thread( - wait_for_socket, transport.socket_path, timeout_seconds=10 - ) + assert await asyncio.to_thread(wait_for_socket, transport.socket_path, timeout_seconds=10) try: - async with ObservedSession( - McpDevServerSpec(sys.executable), diagnostics, transport.socket_path - ) as session: + async with ObservedSession(McpDevServerSpec(sys.executable), diagnostics, transport.socket_path) as session: await session.initialize(timeout_seconds=10) - for index, paths in enumerate( - (arguments["file_paths"], [unsigned.name], arguments["file_paths"]) - ): + for index, paths in enumerate((arguments["file_paths"], [unsigned.name], arguments["file_paths"])): before = len(events) request_id = session.request_id + 1 started = time.monotonic() - result = await session.call_tool( - StreamPlateFilesToViewerCapability.name, - {**arguments, "file_paths": paths}, - timeout_seconds=10, - ) + result = await session.call_tool(StreamPlateFilesToViewerCapability.name, + {**arguments, "file_paths": paths}, timeout_seconds=10) elapsed = time.monotonic() - started progress = events[before:] assert progress and progress[0][0] - started < 1 assert all(event[1] == request_id for event in progress) payload = result["structuredContent"] if index == 1: - assert ( - payload["errors"][0]["code"] == "plate_file_stream_failed" - ) - assert ( - "Native ROI source metadata is required" - in payload["errors"][0]["message"] - ) + assert payload["errors"][0]["code"] == "plate_file_stream_failed" + assert "Native ROI source metadata is required" in payload["errors"][0]["message"] assert len(received) == 1 else: assert payload["errors"] == [] - assert payload["streamed_roi_paths"] == [ - str(Path(arguments["result_directory"]) / paths[0]) - ] + assert payload["streamed_roi_paths"] == [str(Path(arguments["result_directory"]) / paths[0])] assert payload["streamed_image_paths"] == [] if index == 0: assert elapsed >= 12 - assert any( - event[0] - started > 10 and "still running" in event[2] - for event in progress - ) - assert any( - "Streaming 1 ROI file(s)" in event[2] for event in progress - ) - print( - "ROI_STREAM_TERMINAL", - index, - elapsed, - "idle10", - "token", - request_id, - flush=True, - ) + assert any(event[0] - started > 10 and "still running" in event[2] for event in progress) + assert any("Streaming 1 ROI file(s)" in event[2] for event in progress) + print("ROI_STREAM_TERMINAL", index, elapsed, "idle10", "token", request_id, flush=True) finally: transport._stop.set() if transport._listener is not None: @@ -252,9 +185,7 @@ async def exercise(): clients.close() -def test_independent_stream_leaf_cooperative_hook_uses_generated_consumer( - saved_roi_stream, -): +def test_independent_stream_leaf_cooperative_hook_uses_generated_consumer(saved_roi_stream): service, arguments, identity, received, timings, _unsigned = saved_roi_stream timings.append(0.0) # This declaration test does not repeat the slow experiment. hooks = [] @@ -288,18 +219,15 @@ async def report_progress(self, progress, total=None, message=None): hooks.append(message) executor = McpTransportExecutor() - built = build_server( - OpenHCSAgentContext(plate_streaming_service=service), - main_thread_dispatcher=executor.dispatcher, - ) + built = build_server(OpenHCSAgentContext(plate_streaming_service=service), + main_thread_dispatcher=executor.dispatcher) async def exercise(): token = identity.set("stream-request") try: for declaration in (IndependentStream, IndependentStreamAfter): result = await built._tool_manager.get_tool(declaration.name).fn( - mcp_context=ProgressContext(), **arguments - ) + mcp_context=ProgressContext(), **arguments) assert result["errors"] == [] assert received[-1][3] == "stream-request" finally: @@ -310,9 +238,7 @@ async def exercise(): assert len(received) == 2 assert hooks.count("enter") == hooks.count("exit") == 2 assert hooks.index("enter") < hooks.index("exit") - assert any( - "published and viewer settlement completed" in message for message in hooks - ) + assert any("published and viewer settlement completed" in message for message in hooks) finally: executor.close() for declaration in (IndependentStream, IndependentStreamAfter): diff --git a/tests/unit/agent/test_mcp_process_resource_lifetime.py b/tests/unit/agent/test_mcp_process_resource_lifetime.py index d351041c20..775d4723bc 100644 --- a/tests/unit/agent/test_mcp_process_resource_lifetime.py +++ b/tests/unit/agent/test_mcp_process_resource_lifetime.py @@ -35,9 +35,7 @@ def shutdown_jvm(self): context.ij = Gateway() resource_patch.setattr(BioFormatsJavaContext, "_instance", context) resource_patch.setattr(backend_registry, "_cleanup_callbacks", []) - backend_registry.register_cleanup_callback( - BioFormatsJavaContext.shutdown_instance - ) + backend_registry.register_cleanup_callback(BioFormatsJavaContext.shutdown_instance) class AuditCloseCapability: def close(self): @@ -70,10 +68,7 @@ def independent_resource(): assert BioFormatsJavaContext._instance is None assert context.ij is None assert calls[:4] == [ - "cooperative enter", - "gateway disposed", - "JVM stopped", - "independent resource", + "cooperative enter", "gateway disposed", "JVM stopped", "independent resource" ] assert calls.count("cooperative exit") == int(failure is None) executor.close() @@ -84,9 +79,7 @@ def independent_resource(): def test_off_main_close_rejected_before_resource_release(monkeypatch): calls = [] - monkeypatch.setattr( - backend_registry, "_cleanup_callbacks", [lambda: calls.append("closed")] - ) + monkeypatch.setattr(backend_registry, "_cleanup_callbacks", [lambda: calls.append("closed")]) executor = McpTransportExecutor() failures = [] @@ -128,8 +121,7 @@ def test_stdio_eof_retains_exact_normal_child_exit(tmp_path, exit_code): async def journey(): session = McpDevStdioSession( - ScriptServerSpec(python_executable=sys.executable, script=script), - io.StringIO(), + ScriptServerSpec(python_executable=sys.executable, script=script), io.StringIO() ) with (tmp_path / "diagnostic.log").open("w+") as diagnostic: session.server_stderr = diagnostic @@ -153,8 +145,7 @@ def test_stdio_stuck_child_uses_unchanged_forced_reap_budget(tmp_path): async def journey(): with (tmp_path / "diagnostic.log").open("w+") as diagnostic: session = McpDevStdioSession( - ScriptServerSpec(python_executable=sys.executable, script=script), - diagnostic, + ScriptServerSpec(python_executable=sys.executable, script=script), diagnostic ) assert session.teardown_timeout_seconds == 2.0 async with session: @@ -166,9 +157,7 @@ async def journey(): @pytest.mark.parametrize("fail_cleanup", (False, True)) -def test_real_sdk_stdio_declared_resource_closes_before_child_exit( - tmp_path, fail_cleanup -): +def test_real_sdk_stdio_declared_resource_closes_before_child_exit(tmp_path, fail_cleanup): from openhcs.runtime.import_authority import OpenHCSRuntimeImportAuthority source_root = OpenHCSRuntimeImportAuthority.current().import_root @@ -186,11 +175,7 @@ def test_real_sdk_stdio_declared_resource_closes_before_child_exit( "def close_resource():\n" " assert threading.current_thread() is threading.main_thread()\n" f" Path({str(closed)!r}).write_text('main-thread resource closed')\n" - + ( - " raise ValueError('original declared cleanup error')\n" - if fail_cleanup - else "" - ) + + (" raise ValueError('original declared cleanup error')\n" if fail_cleanup else "") + "register_cleanup_callback(close_resource)\n" "with McpStdioTransport.reserve_process_stdio() as transport:\n" " transport.run(server)\n" @@ -199,32 +184,21 @@ def test_real_sdk_stdio_declared_resource_closes_before_child_exit( async def journey(): with (tmp_path / "sdk-diagnostic.log").open("w+") as diagnostic: session = McpDevStdioSession( - ScriptServerSpec(python_executable=sys.executable, script=script), - diagnostic, + ScriptServerSpec(python_executable=sys.executable, script=script), diagnostic ) async with session: process = session.require_process() await session.initialize(timeout_seconds=10) first = await session.call_tool("health", {}, timeout_seconds=10) second = await session.call_tool("health", {}, timeout_seconds=10) - assert ( - first["structuredContent"] - == second["structuredContent"] - == {"ok": True} - ) + assert first["structuredContent"] == second["structuredContent"] == {"ok": True} assert process.returncode == int(fail_cleanup) assert closed.read_text() == "main-thread resource closed" diagnostic.seek(0) text = diagnostic.read() assert "Fatal error" not in text assert ("original declared cleanup error" in text) is fail_cleanup - print( - "SDK_STDIO_DECLARED_RESOURCE", - process.pid, - process.returncode, - fail_cleanup, - flush=True, - ) + print("SDK_STDIO_DECLARED_RESOURCE", process.pid, process.returncode, fail_cleanup, flush=True) asyncio.run(journey()) @@ -254,15 +228,11 @@ def release_resource(): clients = AsyncOperationExecutor(max_workers=1) async def journey(): - assert await asyncio.to_thread( - wait_for_socket, transport.socket_path, timeout_seconds=10 - ) + assert await asyncio.to_thread(wait_for_socket, transport.socket_path, timeout_seconds=10) try: for connection in range(2): async with McpDevSocketSession( - McpDevServerSpec(sys.executable), - io.StringIO(), - transport.socket_path, + McpDevServerSpec(sys.executable), io.StringIO(), transport.socket_path ) as session: await session.initialize(timeout_seconds=10) result = await session.call_tool("health", {}, timeout_seconds=10) diff --git a/tests/unit/agent/test_mcp_server.py b/tests/unit/agent/test_mcp_server.py index d85e82d787..6cf70f5288 100644 --- a/tests/unit/agent/test_mcp_server.py +++ b/tests/unit/agent/test_mcp_server.py @@ -110,29 +110,23 @@ from openhcs.core.streaming_config_declarations import ViewerType from openhcs.mcp.context import OpenHCSAgentContext from openhcs.runtime.import_authority import OpenHCSRuntimeImportAuthority -from openhcs.runtime.viewer_protocol import ( - ViewerPayloadSummary, - ViewerArrayValueSummary, -) +from openhcs.runtime.viewer_protocol import ViewerPayloadSummary, ViewerArrayValueSummary from openhcs.runtime.zmq_config import OPENHCS_ZMQ_CONFIG + class _OperationMethodsBridgeService: """Fake bridge service answering each operation from the method named for it.""" def invoke(self, operation, request=None, connection=None): - name = ( - operation.name - or re.sub( - r"(? str: """Return first text block from FastMCP structured or legacy direct calls.""" if hasattr(result, "content"): @@ -1228,7 +1222,10 @@ def test_mcp_ui_catalog_invocations_preserve_declared_identity( ) assert payload == to_jsonable(catalog) - assert dataclass_from_mapping(capability.output_contract, payload) == catalog + assert ( + dataclass_from_mapping(capability.output_contract, payload) + == catalog + ) def test_mcp_widget_tree_projection_can_return_full_action_fields(): @@ -2183,9 +2180,7 @@ def list_object_state_scopes(self, request, connection): field_summary("napari_streaming_config.enabled"), field_summary("napari_streaming_config.port"), field_summary("partition_filter_config.partition_filter"), - field_summary( - "partition_filter_config.partition_filter_mode" - ), + field_summary("partition_filter_config.partition_filter_mode"), ), ), ), @@ -5623,9 +5618,7 @@ def test_mcp_dev_client_object_state_set_command_projects_tool_arguments(): assert reset_call.name == "openhcs_ui_mutate_object_state_field" assert reset_call.arguments["object_state_scope_id"] == "global_config" - assert ( - reset_call.arguments["field_path"] == "partition_filter_config.partition_filter" - ) + assert reset_call.arguments["field_path"] == "partition_filter_config.partition_filter" assert reset_call.arguments["value"] is None assert reset_call.arguments["reset"] is True assert reset_call.arguments["include_field_values"] is False @@ -5880,29 +5873,15 @@ def test_mcp_dev_client_workflow_poll_terminal_state_policy(): from dataclasses import replace import openhcs.mcp.dev_client as dev_client - + idle = DatasetRowState( - scope_id="scope", - name="plate", - root="/plate", - pipeline_path=None, - selected=True, - initialized=False, - compiled=False, - init_pending=False, - compile_pending=False, - execution_active=False, - status_prefix="", - orchestrator_state=None, - execution_id=None, - terminal_status=None, - runtime_state=None, - runtime_percent=None, - queue_position=None, - ) - init_policy = dev_client.WorkflowStatePollPolicy.from_workflow_text( - "initialize_datasets" + scope_id="scope", name="plate", root="/plate", pipeline_path=None, + selected=True, initialized=False, compiled=False, init_pending=False, + compile_pending=False, execution_active=False, status_prefix="", + orchestrator_state=None, execution_id=None, terminal_status=None, + runtime_state=None, runtime_percent=None, queue_position=None, ) + init_policy = dev_client.WorkflowStatePollPolicy.from_workflow_text("initialize_datasets") compile_policy = dev_client.WorkflowStatePollPolicy.from_workflow_text( "compile_datasets" ) @@ -5925,42 +5904,20 @@ def _complete_plate_manager_state(sparse): from openhcs.agent.dto.ui_bridge import UiPlateManagerState summary = UiStateSurfaceSummary( - SCHEMA_VERSION, - UiStateSurfaceIdentity(surface_id="plate_manager.state"), - "Plate Manager", - True, - widget_id="plate_manager", - ) - row = to_jsonable( - DatasetRowState( - scope_id="scope", - name="plate", - root="/plate", - pipeline_path=None, - selected=False, - initialized=False, - compiled=False, - init_pending=False, - compile_pending=False, - execution_active=False, - status_prefix="", - orchestrator_state=None, - execution_id=None, - terminal_status=None, - runtime_state=None, - runtime_percent=None, - queue_position=None, - ) - ) - state = to_jsonable( - UiPlateManagerState( - schema_version=SCHEMA_VERSION, - summary=summary, - object_state_token=0, - manager_execution_state="idle", - rows=(), - ) - ) + SCHEMA_VERSION, UiStateSurfaceIdentity(surface_id="plate_manager.state"), + "Plate Manager", True, widget_id="plate_manager", + ) + row = to_jsonable(DatasetRowState( + scope_id="scope", name="plate", root="/plate", pipeline_path=None, + selected=False, initialized=False, compiled=False, init_pending=False, + compile_pending=False, execution_active=False, status_prefix="", + orchestrator_state=None, execution_id=None, terminal_status=None, + runtime_state=None, runtime_percent=None, queue_position=None, + )) + state = to_jsonable(UiPlateManagerState( + schema_version=SCHEMA_VERSION, summary=summary, object_state_token=0, + manager_execution_state="idle", rows=(), + )) return { **state, **sparse, @@ -5970,20 +5927,14 @@ def _complete_plate_manager_state(sparse): def _state_surface_dev_wire(dev_client): """Declare the complete wire envelope; each poll test supplies its body.""" - return to_jsonable( - UiStateSurfaceDocument( - schema_version=SCHEMA_VERSION, - summary=UiStateSurfaceSummary( - SCHEMA_VERSION, - UiStateSurfaceIdentity(surface_id="plate_manager.state"), - "Plate Manager", - True, - widget_id="plate_manager", - ), - payload_schema="openhcs.ui.plate_manager_state.v1", - payload={}, - ) - ) + return to_jsonable(UiStateSurfaceDocument( + schema_version=SCHEMA_VERSION, + summary=UiStateSurfaceSummary( + SCHEMA_VERSION, UiStateSurfaceIdentity(surface_id="plate_manager.state"), + "Plate Manager", True, widget_id="plate_manager", + ), + payload_schema="openhcs.ui.plate_manager_state.v1", payload={}, + )) def test_mcp_dev_client_workflow_poll_filters_target_scope_ids(): @@ -5995,23 +5946,21 @@ def test_mcp_dev_client_workflow_poll_filters_target_scope_ids(): payloads=( { **_state_surface_dev_wire(dev_client), - "payload": _complete_plate_manager_state( - { - "manager_execution_state": "idle", - "rows": [ - { - "scope_id": "scope-a", - "compile_pending": False, - "compiled": True, - }, - { - "scope_id": "scope-b", - "compile_pending": True, - "compiled": False, - }, - ], - } - ), + "payload": _complete_plate_manager_state({ + "manager_execution_state": "idle", + "rows": [ + { + "scope_id": "scope-a", + "compile_pending": False, + "compiled": True, + }, + { + "scope_id": "scope-b", + "compile_pending": True, + "compiled": False, + }, + ], + }) }, ), ) @@ -6043,18 +5992,16 @@ def test_mcp_dev_client_workflow_poll_waits_for_manager_finalization(): payloads=( { **_state_surface_dev_wire(dev_client), - "payload": _complete_plate_manager_state( - { - "manager_execution_state": "running", - "rows": [ - { - "scope_id": "scope-a", - "compile_pending": False, - "compiled": True, - } - ], - } - ), + "payload": _complete_plate_manager_state({ + "manager_execution_state": "running", + "rows": [ + { + "scope_id": "scope-a", + "compile_pending": False, + "compiled": True, + } + ], + }) }, ), ) @@ -6088,19 +6035,17 @@ def test_mcp_dev_client_workflow_poll_reports_failed_terminal_rows(): payloads=( { **_state_surface_dev_wire(dev_client), - "payload": _complete_plate_manager_state( - { - "manager_execution_state": "idle", - "rows": [ - { - "scope_id": "scope-a", - "execution_active": False, - "queue_position": None, - "terminal_status": "failed", - } - ], - } - ), + "payload": _complete_plate_manager_state({ + "manager_execution_state": "idle", + "rows": [ + { + "scope_id": "scope-a", + "execution_active": False, + "queue_position": None, + "terminal_status": "failed", + } + ], + }) }, ), ) @@ -6110,19 +6055,17 @@ def test_mcp_dev_client_workflow_poll_reports_failed_terminal_rows(): payloads=( { **_state_surface_dev_wire(dev_client), - "payload": _complete_plate_manager_state( - { - "manager_execution_state": "idle", - "rows": [ - { - "scope_id": "scope-a", - "compile_pending": False, - "compiled": False, - "orchestrator_state": "compile_failed", - } - ], - } - ), + "payload": _complete_plate_manager_state({ + "manager_execution_state": "idle", + "rows": [ + { + "scope_id": "scope-a", + "compile_pending": False, + "compiled": False, + "orchestrator_state": "compile_failed", + } + ], + }) }, ), ) @@ -6179,29 +6122,23 @@ async def fake_call_tool(session, call, timeout_seconds): { **_state_surface_dev_wire(dev_client), "current_revision_token": f"rev-{state_call_count}", - "payload": _complete_plate_manager_state( - { - "manager_execution_state": ( - "idle" if compiled else "running" - ), - "object_state_token": state_call_count, - "rows": [ - { - "scope_id": "scope-a", - "compile_pending": False, - "compiled": compiled, - } - ], - } - ), + "payload": _complete_plate_manager_state({ + "manager_execution_state": ("idle" if compiled else "running"), + "object_state_token": state_call_count, + "rows": [ + { + "scope_id": "scope-a", + "compile_pending": False, + "compiled": compiled, + } + ], + }), }, ), ) monkeypatch.setattr( - ui_commands, - "call_mcp_tool", - _answering_session_events(dev_client, fake_call_tool), + ui_commands, "call_mcp_tool", _answering_session_events(dev_client, fake_call_tool) ) parser = dev_client._build_parser() @@ -6314,17 +6251,13 @@ async def fake_call_tool(session, call, timeout_seconds): { **_state_surface_dev_wire(dev_client), "current_revision_token": "baseline", - "payload": _complete_plate_manager_state( - {"object_state_token": 1, "rows": []} - ), + "payload": _complete_plate_manager_state({"object_state_token": 1, "rows": []}), }, ), ) monkeypatch.setattr( - ui_commands, - "call_mcp_tool", - _answering_session_events(dev_client, fake_call_tool), + ui_commands, "call_mcp_tool", _answering_session_events(dev_client, fake_call_tool) ) args = dev_client._build_parser().parse_args( @@ -6398,17 +6331,13 @@ async def fake_call_tool(session, call, timeout_seconds): { **_state_surface_dev_wire(dev_client), "current_revision_token": "baseline", - "payload": _complete_plate_manager_state( - {"object_state_token": 1, "rows": []} - ), + "payload": _complete_plate_manager_state({"object_state_token": 1, "rows": []}), }, ), ) monkeypatch.setattr( - ui_commands, - "call_mcp_tool", - _answering_session_events(dev_client, fake_call_tool), + ui_commands, "call_mcp_tool", _answering_session_events(dev_client, fake_call_tool) ) args = dev_client._build_parser().parse_args( ("selected-workflow", "compile_datasets", "--wait") @@ -6426,16 +6355,9 @@ async def fake_call_tool(session, call, timeout_seconds): "openhcs_ui_selected_plate_workflow", "openhcs_ui_wait_for_operation_receipt", ] - assert ( - response.results[-1].payloads[0].status - is dev_client.WorkflowPollSummaryStatus.FAILED - ) + assert response.results[-1].payloads[0].status is dev_client.WorkflowPollSummaryStatus.FAILED from openhcs.mcp.dev_client_core import WorkflowPollSkipReason - - assert ( - response.results[-1].payloads[0].skip_reason - is WorkflowPollSkipReason.OPERATION_RECEIPT_FAILED - ) + assert response.results[-1].payloads[0].skip_reason is WorkflowPollSkipReason.OPERATION_RECEIPT_FAILED def test_mcp_dev_client_selected_workflow_wait_rejects_stale_terminal_state( @@ -6455,19 +6377,17 @@ def test_mcp_dev_client_selected_workflow_wait_rejects_stale_terminal_state( { **_state_surface_dev_wire(dev_client), "current_revision_token": "prior-completed-run", - "payload": _complete_plate_manager_state( - { - "manager_execution_state": "idle", - "object_state_token": 1, - "rows": [ - { - "scope_id": "scope-a", - "compile_pending": False, - "compiled": True, - } - ], - } - ), + "payload": _complete_plate_manager_state({ + "manager_execution_state": "idle", + "object_state_token": 1, + "rows": [ + { + "scope_id": "scope-a", + "compile_pending": False, + "compiled": True, + } + ], + }), }, ), ) @@ -6484,9 +6404,7 @@ async def fake_call_tool(session, call, timeout_seconds): return stale_state monkeypatch.setattr( - ui_commands, - "call_mcp_tool", - _answering_session_events(dev_client, fake_call_tool), + ui_commands, "call_mcp_tool", _answering_session_events(dev_client, fake_call_tool) ) args = dev_client._build_parser().parse_args( @@ -6536,19 +6454,17 @@ def test_mcp_dev_client_selected_workflow_accepts_idempotent_init_terminal_state { **_state_surface_dev_wire(dev_client), "current_revision_token": "already-initialized", - "payload": _complete_plate_manager_state( - { - "manager_execution_state": "idle", - "object_state_token": 1, - "rows": [ - { - "scope_id": "scope-a", - "init_pending": False, - "initialized": True, - } - ], - } - ), + "payload": _complete_plate_manager_state({ + "manager_execution_state": "idle", + "object_state_token": 1, + "rows": [ + { + "scope_id": "scope-a", + "init_pending": False, + "initialized": True, + } + ], + }), }, ), ) @@ -6569,9 +6485,7 @@ async def fake_call_tool(session, call, timeout_seconds): return initialized_state monkeypatch.setattr( - ui_commands, - "call_mcp_tool", - _answering_session_events(dev_client, fake_call_tool), + ui_commands, "call_mcp_tool", _answering_session_events(dev_client, fake_call_tool) ) args = dev_client._build_parser().parse_args( @@ -6654,29 +6568,23 @@ async def fake_call_tool(session, call, timeout_seconds): { **_state_surface_dev_wire(dev_client), "current_revision_token": f"rev-{state_call_count}", - "payload": _complete_plate_manager_state( - { - "manager_execution_state": ( - "idle" if compiled else "running" - ), - "object_state_token": state_call_count, - "rows": [ - { - "scope_id": "scope-a", - "compile_pending": False, - "compiled": compiled, - } - ], - } - ), + "payload": _complete_plate_manager_state({ + "manager_execution_state": ("idle" if compiled else "running"), + "object_state_token": state_call_count, + "rows": [ + { + "scope_id": "scope-a", + "compile_pending": False, + "compiled": compiled, + } + ], + }), }, ), ) monkeypatch.setattr( - ui_commands, - "call_mcp_tool", - _answering_session_events(dev_client, fake_call_tool), + ui_commands, "call_mcp_tool", _answering_session_events(dev_client, fake_call_tool) ) parser = dev_client._build_parser() @@ -6770,27 +6678,23 @@ async def fake_call_tool(session, call, timeout_seconds): { **_state_surface_dev_wire(dev_client), "current_revision_token": "terminal", - "payload": _complete_plate_manager_state( - { - "manager_execution_state": "idle", - "object_state_token": 2, - "rows": [ - { - "scope_id": "scope-a", - "compile_pending": False, - "compiled": True, - } - ], - } - ), + "payload": _complete_plate_manager_state({ + "manager_execution_state": "idle", + "object_state_token": 2, + "rows": [ + { + "scope_id": "scope-a", + "compile_pending": False, + "compiled": True, + } + ], + }), }, ), ) monkeypatch.setattr( - ui_commands, - "call_mcp_tool", - _answering_session_events(dev_client, fake_call_tool), + ui_commands, "call_mcp_tool", _answering_session_events(dev_client, fake_call_tool) ) parser = dev_client._build_parser() @@ -6868,9 +6772,7 @@ async def fake_call_tool(session, call, timeout_seconds): { **_state_surface_dev_wire(dev_client), "current_revision_token": "baseline", - "payload": _complete_plate_manager_state( - {"object_state_token": 1, "rows": []} - ), + "payload": _complete_plate_manager_state({"object_state_token": 1, "rows": []}), }, ), ) @@ -6890,9 +6792,7 @@ async def fake_call_tool(session, call, timeout_seconds): ) monkeypatch.setattr( - ui_commands, - "call_mcp_tool", - _answering_session_events(dev_client, fake_call_tool), + ui_commands, "call_mcp_tool", _answering_session_events(dev_client, fake_call_tool) ) parser = dev_client._build_parser() @@ -6965,33 +6865,27 @@ async def fake_call_tool(session, call, timeout_seconds): { **_state_surface_dev_wire(dev_client), "current_revision_token": f"rev-{state_call_count}", - "payload": _complete_plate_manager_state( - { - "manager_execution_state": ( - "idle" if failed else "running" - ), - "object_state_token": state_call_count, - "rows": [ - { - "scope_id": "scope-a", - "execution_active": False, - "queue_position": None, - "terminal_status": ("failed" if failed else None), - "orchestrator_state": ( - "exec_failed" if failed else "compiled" - ), - } - ], - } - ), + "payload": _complete_plate_manager_state({ + "manager_execution_state": ("idle" if failed else "running"), + "object_state_token": state_call_count, + "rows": [ + { + "scope_id": "scope-a", + "execution_active": False, + "queue_position": None, + "terminal_status": ("failed" if failed else None), + "orchestrator_state": ( + "exec_failed" if failed else "compiled" + ), + } + ], + }), }, ), ) monkeypatch.setattr( - ui_commands, - "call_mcp_tool", - _answering_session_events(dev_client, fake_call_tool), + ui_commands, "call_mcp_tool", _answering_session_events(dev_client, fake_call_tool) ) parser = dev_client._build_parser() @@ -7061,12 +6955,10 @@ async def fake_call_tool(session, call, timeout_seconds): { **_state_surface_dev_wire(dev_client), "current_revision_token": "rev-1", - "payload": _complete_plate_manager_state( - { - "object_state_token": 1, - "rows": [], - } - ), + "payload": _complete_plate_manager_state({ + "object_state_token": 1, + "rows": [], + }), }, ), ) @@ -7086,9 +6978,7 @@ async def fake_call_tool(session, call, timeout_seconds): ) monkeypatch.setattr( - ui_commands, - "call_mcp_tool", - _answering_session_events(dev_client, fake_call_tool), + ui_commands, "call_mcp_tool", _answering_session_events(dev_client, fake_call_tool) ) parser = dev_client._build_parser() @@ -7136,33 +7026,22 @@ def test_mcp_dev_client_selected_workflow_poll_summarizes_rejection( import openhcs.mcp.dev_client as dev_client import openhcs.mcp.dev_client_commands.ui as ui_commands from openhcs.agent.dto.ui_bridge import ( - UiPlateManagerState, - UiStateSurfaceDocument, - UiStateSurfaceIdentity, - UiStateSurfaceSummary, + UiPlateManagerState, UiStateSurfaceDocument, + UiStateSurfaceIdentity, UiStateSurfaceSummary, ) surface_summary = UiStateSurfaceSummary( - "test", - UiStateSurfaceIdentity(surface_id="plate_manager.state"), - "Plate Manager", - True, - widget_id="plate_manager", + "test", UiStateSurfaceIdentity(surface_id="plate_manager.state"), + "Plate Manager", True, widget_id="plate_manager", ) baseline = UiPlateManagerState( - schema_version="test", - summary=surface_summary, - object_state_token=1, - manager_execution_state="idle", - rows=(), - current_revision_token="rev-1", + schema_version="test", summary=surface_summary, object_state_token=1, + manager_execution_state="idle", rows=(), current_revision_token="rev-1", ) baseline_document = UiStateSurfaceDocument( - schema_version="test", - summary=surface_summary, + schema_version="test", summary=surface_summary, payload_schema="openhcs.ui.plate_manager_state.v1", - payload=to_jsonable(baseline), - current_revision_token="rev-1", + payload=to_jsonable(baseline), current_revision_token="rev-1", ) calls: list[dev_client.McpDevToolCall] = [] @@ -7181,9 +7060,7 @@ async def fake_call_tool(session, call, timeout_seconds): ) monkeypatch.setattr( - ui_commands, - "call_mcp_tool", - _answering_session_events(dev_client, fake_call_tool), + ui_commands, "call_mcp_tool", _answering_session_events(dev_client, fake_call_tool) ) parser = dev_client._build_parser() @@ -7222,9 +7099,9 @@ async def fake_call_tool(session, call, timeout_seconds): "action_status": "rejected", } - rendered = dev_client.McpDevCommandSpec.for_name("selected-workflow").render_result( - response, args - ) + rendered = dev_client.McpDevCommandSpec.for_name( + "selected-workflow" + ).render_result(response, args) assert "Workflow: compile_datasets" in rendered assert "Skip reason: workflow_not_accepted" in rendered @@ -8070,8 +7947,7 @@ def window_payloads(self, request): path="/tmp/A01_w2_rois.roi.zip", components={"well": "A01", "channel": 2}, summary=ViewerPayloadSummary( - shape_payload_count=3, - shape_coordinate_count=12, + shape_payload_count=3, shape_coordinate_count=12, ), shape_payloads=( { @@ -8169,15 +8045,10 @@ def window_payloads(self, request): components={"well": "A01", "channel": 1}, axis_indices=(index,), summary=ViewerPayloadSummary( - shape=(96, 96), - dtype="uint16", - min=1, - max=4, + shape=(96, 96), dtype="uint16", min=1, max=4, nonzero_count=10, ), - array_value_summary=ViewerArrayValueSummary( - requested=True, included=False - ), + array_value_summary=ViewerArrayValueSummary(requested=True, included=False), ) for index in range(5) ), @@ -8252,9 +8123,7 @@ def window_payloads(self, request): components={"well": "A01"}, axis_indices=(0,), summary=ViewerPayloadSummary(shape=(8, 8)), - array_value_summary=ViewerArrayValueSummary( - requested=True, included=False - ), + array_value_summary=ViewerArrayValueSummary(requested=True, included=False), ), ), ), @@ -8271,9 +8140,7 @@ def window_payloads(self, request): components={"well": "A01"}, axis_indices=(0,), summary=ViewerPayloadSummary(shape=(8, 8)), - array_value_summary=ViewerArrayValueSummary( - requested=True, included=False - ), + array_value_summary=ViewerArrayValueSummary(requested=True, included=False), ), ), ), @@ -8337,9 +8204,7 @@ def window_payloads(self, request): components={"well": "A01"}, axis_indices=(0,), summary=ViewerPayloadSummary(shape=(8, 8)), - array_value_summary=ViewerArrayValueSummary( - requested=True, included=False - ), + array_value_summary=ViewerArrayValueSummary(requested=True, included=False), ), ), ), @@ -8356,9 +8221,7 @@ def window_payloads(self, request): components={"well": "A01"}, axis_indices=(0,), summary=ViewerPayloadSummary(shape=(8, 8)), - array_value_summary=ViewerArrayValueSummary( - requested=True, included=False - ), + array_value_summary=ViewerArrayValueSummary(requested=True, included=False), ), ), ), @@ -8857,10 +8720,7 @@ def test_mcp_bootstrap_wraps_server_run_failure(monkeypatch): class FailingRunServer: def __init__(self, *args, **kwargs): - assert ( - kwargs["main_thread_dispatcher"] - is recording_transport.execution.dispatcher - ) + assert kwargs["main_thread_dispatcher"] is recording_transport.execution.dispatcher def run(self, *, transport: str) -> None: transports.append(transport) @@ -8977,14 +8837,9 @@ def test_mcp_server_exposes_session_tools_and_no_job_tools(): tool_names = {tool.name for tool in tools} for operation_id in ( - "add_datasets", - "connect_server", - "set_dataset_pipeline", - "initialize_datasets", - "compile_datasets", - "run_datasets", - "stop_execution", - "delete_datasets", + "add_datasets", "connect_server", "set_dataset_pipeline", + "initialize_datasets", "compile_datasets", "run_datasets", + "stop_execution", "delete_datasets", ): assert f"openhcs_{operation_id}" in tool_names for state in ("datasets", "pipeline", "events"): @@ -9146,7 +9001,9 @@ def test_mcp_run_datasets_projects_observation_export_request(): tools = ( asyncio.run(listed_tools) if inspect.isawaitable(listed_tools) else listed_tools ) - schema = {tool.name: tool.inputSchema for tool in tools}["openhcs_run_datasets"] + schema = {tool.name: tool.inputSchema for tool in tools}[ + "openhcs_run_datasets" + ] assert "runtime_observation_export_path" in schema["properties"] assert "runtime_observation_export_path" not in schema["required"] diff --git a/tests/unit/agent/test_physical_source_inspection_445.py b/tests/unit/agent/test_physical_source_inspection_445.py index e9173034df..7617b3e176 100644 --- a/tests/unit/agent/test_physical_source_inspection_445.py +++ b/tests/unit/agent/test_physical_source_inspection_445.py @@ -49,10 +49,7 @@ def source_dataset(self, plate_path): return replace( dataset, candidates=tuple( - replace( - candidate, - metadata={**candidate.metadata, "inspection_witness": "declared"}, - ) + replace(candidate, metadata={**candidate.metadata, "inspection_witness": "declared"}) for candidate in dataset.candidates ), ) @@ -65,9 +62,7 @@ def _fixture(root: Path, channels: int): manifest_path = root / "bioformats_spw.json" manifest = json.loads(manifest_path.read_text()) image = manifest["images"][0] - image["channel_names"] = [ - f"engineering-{channel}" for channel in range(1, channels + 1) - ] + image["channel_names"] = [f"engineering-{channel}" for channel in range(1, channels + 1)] image["pixels"]["size_c"] = channels image["pixels"]["planes"] = [ {"c": channel, "z": 1, "t": 1, "index": channel - 1} @@ -111,15 +106,11 @@ def _sample(service, root, image_path, *, source_format="bioformats"): @pytest.mark.parametrize("channels", (3, 4)) -def test_original_physical_candidates_are_sampleable_before_projection( - tmp_path, channels -): +def test_original_physical_candidates_are_sampleable_before_projection(tmp_path, channels): stack, service, _ = _fixture(tmp_path, channels) query = _query(service, tmp_path) assert query.total_count == channels - record = next( - record for record in query.records if int(record.metadata["channel"]) == 3 - ) + record = next(record for record in query.records if int(record.metadata["channel"]) == 3) sample = _sample(service, tmp_path, record.virtual_path) assert int(sample.source_metadata["channel"]) == 3 assert sample.sample_values == stack[0, 0, 2, 2:4, 3:5].tolist() @@ -138,16 +129,14 @@ def test_physical_c3_remains_available_after_c1_only_preparation(tmp_path, chann ) ) filemanager = factory.create() - BioFormatsHandler( - filemanager, source_bindings_config=bindings - ).initialize_workspace(tmp_path, filemanager) + BioFormatsHandler(filemanager, source_bindings_config=bindings).initialize_workspace( + tmp_path, filemanager + ) metadata_path = tmp_path / "openhcs_metadata.json" before = metadata_path.read_bytes() persisted = json.loads(before)["subdirectories"]["."] assert len(persisted["image_files"]) == 1 - assert { - int(value["channel"]) for value in persisted["source_metadata"].values() - } == {1} + assert {int(value["channel"]) for value in persisted["source_metadata"].values()} == {1} try: # Ordinary auto/workspace routing must keep the persisted C1 selection. selected = _query(service, tmp_path, source_format="auto") @@ -160,18 +149,14 @@ def test_physical_c3_remains_available_after_c1_only_preparation(tmp_path, chann assert sample.sample_values == stack[0, 0, 0, 2:4, 3:5].tolist() query = _query(service, tmp_path) - observed_channels = { - int(record.metadata["channel"]) for record in query.records - } + observed_channels = {int(record.metadata["channel"]) for record in query.records} # No invented C3 path, receipt, source-ref override, or projection mutation. assert 3 in observed_channels, ( "Explicit physical-source inspection hides C3 after C1 preparation: " f"observed_channels={sorted(observed_channels)}, total_count={query.total_count}" ) assert observed_channels == set(range(1, channels + 1)) - record = next( - record for record in query.records if int(record.metadata["channel"]) == 3 - ) + record = next(record for record in query.records if int(record.metadata["channel"]) == 3) physical = _sample(service, tmp_path, record.virtual_path) assert int(physical.source_metadata["channel"]) == 3 assert physical.sample_values == stack[0, 0, 2, 2:4, 3:5].tolist() @@ -202,9 +187,7 @@ def test_physical_c3_remains_available_after_c1_only_preparation(tmp_path, chann from openhcs.core.viewer_streaming_service import ViewerStreamingSource context, errors, _ = service.open_context( - PlatePathInspectionRequest( - plate_path=str(tmp_path), source_format="bioformats" - ) + PlatePathInspectionRequest(plate_path=str(tmp_path), source_format="bioformats") ) assert errors == () inventory, warnings = service.file_inventory(context, kind=PlateFileKind.IMAGE) @@ -258,14 +241,7 @@ def test_independent_metadata_hook_flows_through_unchanged_inventory_consumer(tm inventory = service._image_inventory(handler, tmp_path, filemanager, warnings) assert warnings == [] assert len(inventory.records) == 4 - assert {record.metadata["inspection_witness"] for record in inventory.records} == { - "declared" - } - assert {int(record.metadata["channel"]) for record in inventory.records} == { - 1, - 2, - 3, - 4, - } + assert {record.metadata["inspection_witness"] for record in inventory.records} == {"declared"} + assert {int(record.metadata["channel"]) for record in inventory.records} == {1, 2, 3, 4} assert all(record.source_ref is not None for record in inventory.records) assert not (tmp_path / "openhcs_metadata.json").exists() diff --git a/tests/unit/agent/test_plate_inspection_cold_runtime.py b/tests/unit/agent/test_plate_inspection_cold_runtime.py index 770ff78be4..dcebf0fa47 100644 --- a/tests/unit/agent/test_plate_inspection_cold_runtime.py +++ b/tests/unit/agent/test_plate_inspection_cold_runtime.py @@ -87,9 +87,9 @@ def inspect(self, request): built.call_tool("openhcs_list_capabilities", {}), 0.5 ) assert result is not None - assert ( - not pending.done() - ), "Discovery must finish while inspection is still held" + assert not pending.done(), ( + "Discovery must finish while inspection is still held" + ) finally: release.set() await asyncio.wait_for(pending, 1) diff --git a/tests/unit/agent/test_plate_inspection_service.py b/tests/unit/agent/test_plate_inspection_service.py index 60071768c3..66cb9b206f 100644 --- a/tests/unit/agent/test_plate_inspection_service.py +++ b/tests/unit/agent/test_plate_inspection_service.py @@ -430,8 +430,14 @@ def test_registered_handler_selection_roles_are_owned_polymorphically() -> None: from openhcs.microscopes.imagexpress import ImageXpressHandler from openhcs.microscopes.opera_phenix import OperaPhenixHandler - assert BioFormatsHandler.source_selection_role() is BroadStoreSource - assert SourceBindingsSource.source_selection_role() is DeclaredFileSource + assert ( + BioFormatsHandler.source_selection_role() + is BroadStoreSource + ) + assert ( + SourceBindingsSource.source_selection_role() + is DeclaredFileSource + ) assert "structured or rich container" in ( BioFormatsHandler.source_selection_guidance() ) @@ -1248,8 +1254,7 @@ def test_plate_inspection_reads_no_main_openhcs_output_subdirectories( for image_name in image_names ], "source_metadata": _calibrated_source_metadata( - f"checkpoints_step1/{image_name}" - for image_name in image_names + f"checkpoints_step1/{image_name}" for image_name in image_names ), "results_dir": "checkpoints_step1_results", }, @@ -1260,8 +1265,7 @@ def test_plate_inspection_reads_no_main_openhcs_output_subdirectories( for image_name in image_names ], "source_metadata": _calibrated_source_metadata( - f"checkpoints_step2/{image_name}" - for image_name in image_names + f"checkpoints_step2/{image_name}" for image_name in image_names ), "results_dir": "checkpoints_step2_results", }, diff --git a/tests/unit/agent/test_plate_streaming_service.py b/tests/unit/agent/test_plate_streaming_service.py index b73e832abb..8e4b895f86 100644 --- a/tests/unit/agent/test_plate_streaming_service.py +++ b/tests/unit/agent/test_plate_streaming_service.py @@ -51,6 +51,7 @@ ) from openhcs.domains.microscopy.axes import Microscopy + # A metadata owner that anchors workspace-relative addresses at the plate. PLATE_ROOT_METADATA = SimpleNamespace(source_workspace_root=Path) @@ -71,28 +72,19 @@ def test_saved_site_free_image_inventory_and_loading_preserve_original_scope(tmp from openhcs.core.plate_image_inventory import PlateImageInventory from openhcs.core.runtime_image_values import ImagePayloadMetadata from openhcs.core.source_image_provenance import ( - SourceImageIdentity, - SourceImageProvenance, - SourceImageProvenanceContributor, + SourceImageIdentity, SourceImageProvenance, SourceImageProvenanceContributor, ) from openhcs.core.source_projection import SourceArtifactProjection from openhcs.core.source_metadata import SourceVoxelSpacing - from openhcs.core.source_workspace_projection import ( - VirtualWorkspaceSourceProjectionBuilder, - ) - from openhcs.core.virtual_workspace_metadata import ( - VirtualWorkspaceSourceProjectionEntries, - ) + from openhcs.core.source_workspace_projection import VirtualWorkspaceSourceProjectionBuilder + from openhcs.core.virtual_workspace_metadata import VirtualWorkspaceSourceProjectionEntries from openhcs.core.dataset_sources.source_schema import SourceSchemaFilenameParser filename = "A01_s001_w1_z001_t001.tif" - pixels = np.concatenate( - ( - np.full((4, 6), 11, dtype=np.uint16), - np.full((4, 6), 13, dtype=np.uint16), - ), - axis=1, - ) + pixels = np.concatenate(( + np.full((4, 6), 11, dtype=np.uint16), + np.full((4, 6), 13, dtype=np.uint16), + ), axis=1) tifffile.imwrite(tmp_path / filename, pixels) components = {"well": "A01", "channel": "1", "z_index": "1", "timepoint": "1"} contributors = tuple( @@ -101,8 +93,7 @@ def test_saved_site_free_image_inventory_and_loading_preserve_original_scope(tmp str(tmp_path / f"original-s{site}.tif"), {**components, "site": site}, ), - ) - for site in ("1", "3") + ) for site in ("1", "3") ) metadata = ImagePayloadMetadata( source_provenance=SourceImageProvenance( @@ -113,72 +104,45 @@ def test_saved_site_free_image_inventory_and_loading_preserve_original_scope(tmp ) ref = SourcePixelRef(backend="disk", backend_address=filename) source = SourceArtifactProjection( - address=None, - ref=ref, - source_alias="Signal", - artifact_kind=ImageArtifactType, - source_metadata=components, - image_metadata=metadata, + address=None, ref=ref, source_alias="Signal", artifact_kind=ImageArtifactType, + source_metadata=components, image_metadata=metadata, execution_scope=RuntimeExecutionAxisScope.from_raw( - "A01", - component=Microscopy.Channel, - value="1", - fixed_component_values=( - (Microscopy.ZIndex, "1"), - (Microscopy.Timepoint, "1"), - ), + "A01", component=Microscopy.Channel, value="1", + fixed_component_values=((Microscopy.ZIndex, "1"), (Microscopy.Timepoint, "1")), ), ) builder = VirtualWorkspaceSourceProjectionBuilder(tmp_path) builder.record_workspace_source_path(filename, ref) builder.record_source_metadata(filename, components) - builder.ingest_source_projections( - VirtualWorkspaceSourceProjectionEntries({filename: source}) - ) + builder.ingest_source_projections(VirtualWorkspaceSourceProjectionEntries({filename: source})) filemanager = FileManager({"disk": DiskStorageBackend()}) handler = SimpleNamespace( parse_image_path=SourceSchemaFilenameParser().parse_filename, metadata_handler=PLATE_ROOT_METADATA, ) record = PlateImageInventory._record( - plate_path=tmp_path, - image_file=filename, - handler=handler, - projection=builder.projection(), - filemanager=filemanager, - backend="disk", + plate_path=tmp_path, image_file=filename, handler=handler, + projection=builder.projection(), filemanager=filemanager, backend="disk", ) assert "site" not in record.metadata context = PlateInspectionContext( - plate_path=tmp_path, - filemanager=filemanager, - handler=handler, - parser=None, + plate_path=tmp_path, filemanager=filemanager, handler=handler, parser=None, ) projection = PlateStreamingService._inventory_source_projection( - (PlateFileRecord.from_image(record),), - context, + (PlateFileRecord.from_image(record),), context, ) stream_source = ViewerStreamingSource( - plate_path=tmp_path, - filemanager=filemanager, - microscope_handler=handler, + plate_path=tmp_path, filemanager=filemanager, microscope_handler=handler, ) actual = stream_source.load_image( - filename, - "disk", - source_projection=projection, - component_metadata=record.metadata, + filename, "disk", source_projection=projection, component_metadata=record.metadata, ) np.testing.assert_array_equal(actual.data, pixels) actual_metadata = actual.metadata assert "site" not in actual_metadata.source_component_metadata assert actual_metadata.plane_axis is None assert actual_metadata.source_voxel_spacing.values_zyx == (0.25, 0.5) - assert ( - actual_metadata.source_provenance.source_image_provenance_planes - == metadata.source_provenance.source_image_provenance_planes - ) + assert actual_metadata.source_provenance.source_image_provenance_planes == metadata.source_provenance.source_image_provenance_planes class FakeInspectionService: diff --git a/tests/unit/agent/test_progressive_authoring_context.py b/tests/unit/agent/test_progressive_authoring_context.py index 60a7c318a5..a3a6c09580 100644 --- a/tests/unit/agent/test_progressive_authoring_context.py +++ b/tests/unit/agent/test_progressive_authoring_context.py @@ -193,9 +193,7 @@ def test_task_authorization_routes_to_the_existing_canonical_knowledge_owner() - assert document.selected_section_id == target.section_id assert document.document.source_path == "docs/source/architecture/quick_start.rst" for kind in ("first_use", "ui_visible_workflow"): - context = service.get_bounded_authoring_context( - AuthoringContextRequest(kind=kind) - ) + context = service.get_bounded_authoring_context(AuthoringContextRequest(kind=kind)) assert context == service.get_authoring_context(kind) assert target.document_id in context.content assert agent_capabilities.get_knowledge_document.name in context.content @@ -243,10 +241,9 @@ class _CombinedReviewContext( source = Path(__file__).resolve().parents[3] / document.document.source_path lines = tuple(source.read_text(encoding="utf-8").splitlines()) section = target.find_section(document.sections) - assert ( - document.content.strip() - == "\n".join(section.span.line_slice(lines)).strip() - ) + assert document.content.strip() == "\n".join( + section.span.line_slice(lines) + ).strip() for declaration in ( ImageAnalysisWorkflowAuthoringContext, ViewerReviewAuthoringContext, @@ -287,7 +284,10 @@ def test_knowledge_summary_growth_does_not_expand_context_deepening_links() -> N target_id = target.document_id if target.section_id is not None: target_id = f"{target_id}#{target.section_id}" - assert f"{target_id} — Title for {target.document_id}" in delivered.content + assert ( + f"{target_id} — Title for {target.document_id}" + in delivered.content + ) @pytest.mark.parametrize("kind", ("first_use", "pipeline")) @@ -552,10 +552,7 @@ def test_task_contexts_expose_only_the_next_relevant_boundary() -> None: assert "openhcs_get_viewer_window_payloads" in debugging assert "openhcs_summarize_viewer_window_rois" in debugging assert "schema-bearing per-object measurement rows" in debugging - assert ( - "follow the 'show_live_results' action's `related_state_surface_ids`" - in debugging - ) + assert "follow the 'show_live_results' action's `related_state_surface_ids`" in debugging assert "never guess from a title substring" in debugging assert "bounded retained-table authority" in debugging assert "row/object cardinality" in debugging diff --git a/tests/unit/agent/test_public_callable_enum_ingress.py b/tests/unit/agent/test_public_callable_enum_ingress.py index 3a4cafabf9..d9b1515160 100644 --- a/tests/unit/agent/test_public_callable_enum_ingress.py +++ b/tests/unit/agent/test_public_callable_enum_ingress.py @@ -55,14 +55,10 @@ class IndependentProjectionRequest: @numpy @callable_request(IndependentProjectionRequest) def independent_request_projection(request: IndependentProjectionRequest): - return ( - request.image if request.mode is None else request.mode.project(request.image) - ) + return request.image if request.mode is None else request.mode.project(request.image) -@pytest.fixture( - params=(independent_declared_projection, independent_request_projection) -) +@pytest.fixture(params=(independent_declared_projection, independent_request_projection)) def author(monkeypatch, request): # Declarations outside the registry's modules are custom functions. metadata = project_custom_function(request.param) @@ -71,9 +67,7 @@ def author(monkeypatch, request): monkeypatch.setattr(RegistryService, "_metadata_cache", {function_id: metadata}) monkeypatch.setattr(RegistryService, "_resolved_reference_callables", {}) try: - yield PipelineAuthoringService( - function_catalog=SelectedDeclarations() - ), function_id + yield PipelineAuthoringService(function_catalog=SelectedDeclarations()), function_id finally: CustomFunctionRuntimeRegistry.remove(request.param.__name__) @@ -83,21 +77,13 @@ def author(monkeypatch, request): def test_new_callable_json_roundtrip_exercises_cooperative_member(author, clean, mode): service, function_id = author ref = service.create_pipeline() - service.add_function_step_from_request( - FunctionStepAddRequest.from_fields( - pipeline_id=ref.pipeline_id, - function_id=function_id, - kwargs={"mode": mode}, - ) - ) + service.add_function_step_from_request(FunctionStepAddRequest.from_fields( + pipeline_id=ref.pipeline_id, function_id=function_id, kwargs={"mode": mode}, + )) validation = service.validate(ref) assert validation.valid, validation.errors - document = PipelineDocumentCodec.from_source( - service.render_source(ref, clean=clean).source - ) - invocation = next( - normalize_function_pattern(document.pipeline_steps[0].func).iter_items() - ) + document = PipelineDocumentCodec.from_source(service.render_source(ref, clean=clean).source) + invocation = next(normalize_function_pattern(document.pipeline_steps[0].func).iter_items()) expected = None if mode is None else IndependentProjection.DOUBLE_WITH_OFFSET if mode is None: # Clean source legitimately omits a parameter equal to its declared default. @@ -105,38 +91,24 @@ def test_new_callable_json_roundtrip_exercises_cooperative_member(author, clean, else: assert invocation.kwargs_dict == {"mode": expected} contract = invocation.contract - assert contract.validate_public_kwargs(invocation.kwargs_dict) == tuple( - invocation.kwargs_dict.items() - ) + assert contract.validate_public_kwargs(invocation.kwargs_dict) == tuple(invocation.kwargs_dict.items()) # Execute both actual hooks through the decoded original enum member's MRO. assert contract.resolve_raw_runtime_callable()(5, **invocation.kwargs_dict) == ( 5 if mode is None else 13 ) - assert IndependentProjection.__mro__.index( - _OffsetCapability - ) < IndependentProjection.__mro__.index(_ScaleBehavior) - - -@pytest.mark.parametrize( - "kwargs", - ( - {"mode": "not_declared"}, - {"mode": True}, - {"unknown": "double_with_offset"}, - {"dtype_config": {}}, - {"image": 5}, - ), -) + assert IndependentProjection.__mro__.index(_OffsetCapability) < IndependentProjection.__mro__.index(_ScaleBehavior) + + +@pytest.mark.parametrize("kwargs", ( + {"mode": "not_declared"}, {"mode": True}, {"unknown": "double_with_offset"}, + {"dtype_config": {}}, {"image": 5}, +)) def test_public_ingress_rejects_unknown_and_runtime_owned_kwargs(author, kwargs): service, function_id = author ref = service.create_pipeline() - service.add_function_step_from_request( - FunctionStepAddRequest.from_fields( - pipeline_id=ref.pipeline_id, - function_id=function_id, - kwargs=kwargs, - ) - ) + service.add_function_step_from_request(FunctionStepAddRequest.from_fields( + pipeline_id=ref.pipeline_id, function_id=function_id, kwargs=kwargs, + )) assert not service.validate(ref).valid with pytest.raises((TypeError, ValueError)): service.render_source(ref) diff --git a/tests/unit/agent/test_selected_plane_reopen_stream.py b/tests/unit/agent/test_selected_plane_reopen_stream.py index 9f95643b93..f6416bf0ec 100644 --- a/tests/unit/agent/test_selected_plane_reopen_stream.py +++ b/tests/unit/agent/test_selected_plane_reopen_stream.py @@ -39,10 +39,7 @@ ) from openhcs.runtime.napari_viewer_server import NapariBatchPayload, NapariImagePayload -from tests.unit.agent.test_persisted_output_reopen import ( - NoRuntimeBridge, - declared_output, -) +from tests.unit.agent.test_persisted_output_reopen import NoRuntimeBridge, declared_output @pytest.fixture diff --git a/tests/unit/agent/test_u1_g8_guards.py b/tests/unit/agent/test_u1_g8_guards.py index 2566215b7b..daafdb09f2 100644 --- a/tests/unit/agent/test_u1_g8_guards.py +++ b/tests/unit/agent/test_u1_g8_guards.py @@ -34,9 +34,7 @@ def test_agent_dtos_declare_no_well_or_microscope_type_field() -> None: and isinstance(statement.target, ast.Name) and statement.target.id in {"well", "microscope_type"} ): - offenders.append( - f"{path.relative_to(ROOT)}:{node.name}.{statement.target.id}" - ) + offenders.append(f"{path.relative_to(ROOT)}:{node.name}.{statement.target.id}") assert offenders == [] @@ -45,9 +43,10 @@ def test_gui_restates_no_plate_grid_default_or_row_letter_decode() -> None: for path in _modules("pyqt_gui"): tree = ast.parse(path.read_text()) for node in ast.walk(tree): - if isinstance(node, ast.Tuple) and [ - getattr(element, "value", None) for element in node.elts - ] == [8, 12]: + if ( + isinstance(node, ast.Tuple) + and [getattr(element, "value", None) for element in node.elts] == [8, 12] + ): offenders.append(f"{path.relative_to(ROOT)}:{node.lineno} (8, 12)") if ( isinstance(node, ast.BinOp) @@ -57,9 +56,7 @@ def test_gui_restates_no_plate_grid_default_or_row_letter_decode() -> None: for side in (node.left, node.right) ) ): - offenders.append( - f"{path.relative_to(ROOT)}:{node.lineno} base-26 decode" - ) + offenders.append(f"{path.relative_to(ROOT)}:{node.lineno} base-26 decode") assert offenders == [] @@ -79,10 +76,6 @@ def test_grid_placement_is_declared_only_by_grid_addressed_axes() -> None: base.id if isinstance(base, ast.Name) else getattr(base, "attr", "") for base in node.bases } - if ( - declares - and node.name != "GridAddressed" - and "GridAddressed" not in bases - ): + if declares and node.name != "GridAddressed" and "GridAddressed" not in bases: offenders.append(f"{path.relative_to(ROOT)}:{node.name}") assert offenders == [] diff --git a/tests/unit/agent/test_ui_bridge_operation_family.py b/tests/unit/agent/test_ui_bridge_operation_family.py index c61085b5bc..c1b3b75623 100644 --- a/tests/unit/agent/test_ui_bridge_operation_family.py +++ b/tests/unit/agent/test_ui_bridge_operation_family.py @@ -119,8 +119,7 @@ def test_every_wire_operation_is_one_mcp_tool_deriving_its_contracts() -> None: capabilities = tuple( capability for capability in agent_capability_declarations() - if issubclass(capability, UiBridgeCapability) - and "operation" in vars(capability) + if issubclass(capability, UiBridgeCapability) and "operation" in vars(capability) ) named = [capability.operation for capability in capabilities] assert len(named) == len(set(named)) diff --git a/tests/unit/agent/test_ui_bridge_service.py b/tests/unit/agent/test_ui_bridge_service.py index 3b50be84ee..986576bdf2 100644 --- a/tests/unit/agent/test_ui_bridge_service.py +++ b/tests/unit/agent/test_ui_bridge_service.py @@ -1020,9 +1020,7 @@ def test_descriptor_resolver_rejects_dead_process(monkeypatch, tmp_path): staticmethod(lambda _pid: None), ) - status = UiBridgeService(gateway=_FakeUiBridgeGateway()).invoke( - UiBridgeStatusOperation - ) + status = UiBridgeService(gateway=_FakeUiBridgeGateway()).invoke(UiBridgeStatusOperation) assert status.reachable is False assert status.descriptor_status == "stale_ui_bridge_descriptor" @@ -1111,9 +1109,7 @@ def test_status_rechecks_descriptor_process_liveness_after_gateway_call( staticmethod(lambda _pid: next(process_start_times)), ) - status = UiBridgeService(gateway=_FakeUiBridgeGateway()).invoke( - UiBridgeStatusOperation - ) + status = UiBridgeService(gateway=_FakeUiBridgeGateway()).invoke(UiBridgeStatusOperation) assert status.reachable is False assert status.descriptor_status == "stale_ui_bridge_descriptor" @@ -1140,9 +1136,7 @@ def status(self, connection: UiBridgeConnectionSpec) -> UiBridgeStatus: ).write() monkeypatch.setenv("OPENHCS_UI_BRIDGE_DESCRIPTOR", str(descriptor_path)) - status = UiBridgeService(gateway=MismatchedIdentityGateway()).invoke( - UiBridgeStatusOperation - ) + status = UiBridgeService(gateway=MismatchedIdentityGateway()).invoke(UiBridgeStatusOperation) assert status.reachable is False assert status.descriptor_status == "ui_bridge_endpoint_identity_mismatch" @@ -1276,9 +1270,7 @@ def test_configured_descriptor_directory_includes_process_advertised_descriptor( proc_root, ) - status = UiBridgeService(gateway=_FakeUiBridgeGateway()).invoke( - UiBridgeStatusOperation - ) + status = UiBridgeService(gateway=_FakeUiBridgeGateway()).invoke(UiBridgeStatusOperation) assert status.reachable is True assert status.descriptor_status == "ok" @@ -1316,9 +1308,7 @@ def test_descriptor_catalog_unions_default_and_process_advertised_live_bridges( ) catalog = UiBridgeService().list_bridges() - status = UiBridgeService(gateway=_FakeUiBridgeGateway()).invoke( - UiBridgeStatusOperation - ) + status = UiBridgeService(gateway=_FakeUiBridgeGateway()).invoke(UiBridgeStatusOperation) assert [bridge.bridge_instance_id for bridge in catalog.bridges] == [ "default-bridge", @@ -1382,9 +1372,7 @@ def test_service_forwards_fake_gateway_requests(monkeypatch, tmp_path): ) catalog = service.invoke(UiBridgeListDocumentsOperation, connection=connection) - state_catalog = service.invoke( - UiBridgeListStateSurfacesOperation, connection=connection - ) + state_catalog = service.invoke(UiBridgeListStateSurfacesOperation, connection=connection) document = service.invoke( UiBridgeGetDocumentOperation, UiCodeDocumentRequest(document_id=DOCUMENT_ID), @@ -1832,7 +1820,7 @@ def list_object_state_scopes( scope_ids=("global_config",), field_path_contains=("napari_streaming_config",), field_filter="semantic", - ), + ) ) assert result.field_filter == "semantic" @@ -1861,7 +1849,9 @@ def close_window( errors=( AgentError( code="unsupported_ui_bridge_operation", - message=("Unsupported UI bridge operation: close_window"), + message=( + "Unsupported UI bridge operation: close_window" + ), exception_type="UiBridgeUnsupportedOperationError", ), ) @@ -1891,7 +1881,7 @@ def test_restore_request_rejects_multiple_selectors(monkeypatch, tmp_path): result = service.invoke( UiBridgeRestoreSnapshotOperation, - UiSnapshotRestoreRequest(snapshot_id="snap-1", index=0), + UiSnapshotRestoreRequest(snapshot_id="snap-1", index=0) ) assert result.restored is False diff --git a/tests/unit/agent/test_user_facing_help_projection.py b/tests/unit/agent/test_user_facing_help_projection.py index 09e6e86dc0..7804584e2e 100644 --- a/tests/unit/agent/test_user_facing_help_projection.py +++ b/tests/unit/agent/test_user_facing_help_projection.py @@ -296,9 +296,7 @@ def test_preprocessing_guidance_projects_from_callable_owners(monkeypatch) -> No normalized_shared_text = " ".join(shared_text.split()) # Catalog prose is the semantic declaration's, not the memory wrapper's. - raw = CallableContract.from_callable( - metadata.func - ).resolve_raw_runtime_callable() + raw = CallableContract.from_callable(metadata.func).resolve_raw_runtime_callable() assert detail.doc == inspect.getdoc(raw) assert detail.entry.summary == shared_help.summary for phrase in expected_phrases[metadata.func]: diff --git a/tests/unit/benchmark/test_agent_validation_attempt_recorder.py b/tests/unit/benchmark/test_agent_validation_attempt_recorder.py index 05129ddfa8..77037adc67 100644 --- a/tests/unit/benchmark/test_agent_validation_attempt_recorder.py +++ b/tests/unit/benchmark/test_agent_validation_attempt_recorder.py @@ -160,35 +160,20 @@ def _ui_pipeline_document(source: str) -> UiCodeDocument: def _accepted(operation_id: str) -> SessionOperationResult: return SessionOperationResult( - operation_id=operation_id, - status="accepted", - target_scope_ids=("/plate",), + operation_id=operation_id, status="accepted", target_scope_ids=("/plate",), ) def _finished_datasets() -> DatasetListState: row = DatasetRowState( - scope_id="/plate", - name="plate", - root="/plate", - pipeline_path=None, - selected=True, - initialized=True, - compiled=True, - init_pending=False, - compile_pending=False, - execution_active=False, - status_prefix="", - orchestrator_state="completed", - execution_id="execution-1", - terminal_status="complete", - runtime_state=None, - runtime_percent=None, + scope_id="/plate", name="plate", root="/plate", pipeline_path=None, + selected=True, initialized=True, compiled=True, init_pending=False, + compile_pending=False, execution_active=False, status_prefix="", + orchestrator_state="completed", execution_id="execution-1", + terminal_status="complete", runtime_state=None, runtime_percent=None, queue_position=None, ) - return DatasetListState( - rows=(row,), selected_scope_ids=("/plate",), execution_state="idle" - ) + return DatasetListState(rows=(row,), selected_scope_ids=("/plate",), execution_state="idle") def test_attempt_recorder_derives_dsl_evidence_from_mcp_receipts(tmp_path: Path): diff --git a/tests/unit/cellprofiler_runtime_test_support.py b/tests/unit/cellprofiler_runtime_test_support.py index e159ba234f..d80059e9a7 100644 --- a/tests/unit/cellprofiler_runtime_test_support.py +++ b/tests/unit/cellprofiler_runtime_test_support.py @@ -48,7 +48,7 @@ class CellProfilerRuntimeTestContext: filemanager: object | None microscope_handler: object source_image_set_identity_policy: SourceImageSetIdentityPolicy - dataset_root: Path = Path("/plate/Images") + dataset_root: Path = Path('/plate/Images') completed_step_outputs: StepExecutionObservation = field( default_factory=StepExecutionObservation.empty, ) diff --git a/tests/unit/desktop/test_desktop_restart.py b/tests/unit/desktop/test_desktop_restart.py index 95c1bd1f3c..f288587b7a 100644 --- a/tests/unit/desktop/test_desktop_restart.py +++ b/tests/unit/desktop/test_desktop_restart.py @@ -116,9 +116,7 @@ def test_session_restart_admission_uses_existing_declaration_work_guard( window = SimpleNamespace(session=restart_session) assert not DesktopSessionRestart.available(window) - with pytest.raises( - DesktopUpdateError, match="active initialization or compilation" - ): + with pytest.raises(DesktopUpdateError, match="active initialization or compilation"): DesktopRestartSession.require_capture_allowed(window) diff --git a/tests/unit/progress/test_debug_projection.py b/tests/unit/progress/test_debug_projection.py index c13a79fe70..72c87472d9 100644 --- a/tests/unit/progress/test_debug_projection.py +++ b/tests/unit/progress/test_debug_projection.py @@ -78,9 +78,7 @@ def test_debug_progress_record_fails_loudly_for_malformed_debug_context() -> Non with pytest.raises(ValueError, match="missing required field"): DebugProgressRecord.from_progress_event( _event( - context={ - DebugProgressContext.context_key: {"debug_session_id": "debug-1"} - } + context={DebugProgressContext.context_key: {"debug_session_id": "debug-1"}} ) ) diff --git a/tests/unit/pyqt_gui/session_harness.py b/tests/unit/pyqt_gui/session_harness.py index ee75752c5b..a01c385592 100644 --- a/tests/unit/pyqt_gui/session_harness.py +++ b/tests/unit/pyqt_gui/session_harness.py @@ -79,9 +79,7 @@ def caller_session(**kwargs) -> Iterator[Session]: if persisted is not None: write_dataset_scope_ids(list(persisted)) session = Session( - transport_config=OpenHCSZMQConfig( - default_port=session_port(), persistent=False - ), + transport_config=OpenHCSZMQConfig(default_port=session_port(), persistent=False), global_config=global_config, main_thread=kwargs.pop("main_thread", CallerThread()), **kwargs, diff --git a/tests/unit/pyqt_gui/test_concurrent_initialization_progress.py b/tests/unit/pyqt_gui/test_concurrent_initialization_progress.py index c2b2416475..dffca9c157 100644 --- a/tests/unit/pyqt_gui/test_concurrent_initialization_progress.py +++ b/tests/unit/pyqt_gui/test_concurrent_initialization_progress.py @@ -15,9 +15,7 @@ def test_init_signals_and_completion_preserve_running_progress(): app = QApplication.instance() or QApplication([]) projection = ExecutionRuntimeProjection() plate = SubjectRuntimeProjection( - identity=SubjectRuntimeIdentity( - subject_id="/running", execution_id="owned-run" - ), + identity=SubjectRuntimeIdentity(subject_id="/running", execution_id="owned-run"), state=ExecutingSubjectRuntimeState, percent=43, axis_progress=(), diff --git a/tests/unit/pyqt_gui/test_config_preview_formatters.py b/tests/unit/pyqt_gui/test_config_preview_formatters.py index e253bc4ef0..b392867baf 100644 --- a/tests/unit/pyqt_gui/test_config_preview_formatters.py +++ b/tests/unit/pyqt_gui/test_config_preview_formatters.py @@ -24,10 +24,7 @@ def test_partition_filter_preview_label_is_owned_by_config_declaration() -> None def test_plain_partition_filter_preview_requires_a_selection() -> None: - assert ( - format_partition_filter_config(PartitionFilterConfig(), "partition_filter") - is None - ) + assert format_partition_filter_config(PartitionFilterConfig(), "partition_filter") is None assert ( format_partition_filter_config( PartitionFilterConfig( @@ -45,10 +42,7 @@ def test_lazy_plain_partition_filter_preserves_base_preview_policy() -> None: assert resolution is not None assert resolution.owner is PartitionFilterConfig - assert ( - format_partition_filter_config(LazyPartitionFilterConfig(), "partition_filter") - is None - ) + assert format_partition_filter_config(LazyPartitionFilterConfig(), "partition_filter") is None def test_lazy_specialized_config_uses_its_declared_preview_policy() -> None: diff --git a/tests/unit/pyqt_gui/test_debug_toolbar.py b/tests/unit/pyqt_gui/test_debug_toolbar.py index 6844eb17f6..c291aaf739 100644 --- a/tests/unit/pyqt_gui/test_debug_toolbar.py +++ b/tests/unit/pyqt_gui/test_debug_toolbar.py @@ -579,11 +579,9 @@ def test_pipeline_change_marks_the_debug_session_dirty(tmp_path, monkeypatch) -> ).with_cursor(_cursor(1, "segment")) statuses = [] harness.session.subscribe( - lambda record: ( - statuses.append(record.event.text) - if isinstance(record.event, StatusReported) - else None - ) + lambda record: statuses.append(record.event.text) + if isinstance(record.event, StatusReported) + else None ) harness.session.set_pipeline( diff --git a/tests/unit/pyqt_gui/test_main_config_propagation.py b/tests/unit/pyqt_gui/test_main_config_propagation.py index cbdb766b14..db75ee27fb 100644 --- a/tests/unit/pyqt_gui/test_main_config_propagation.py +++ b/tests/unit/pyqt_gui/test_main_config_propagation.py @@ -437,13 +437,11 @@ def test_lifecycle_workflow_adopts_a_saved_global_config_into_the_session( (scope_id,) = add_datasets(session, Path(tempfile.mkdtemp()), "plate") changed = [] session.subscribe( - lambda record: ( - changed.append( - (record.event.scope_id, record.event.effective_config.num_workers) - ) - if isinstance(record.event, DatasetConfigChanged) - else None + lambda record: changed.append( + (record.event.scope_id, record.event.effective_config.num_workers) ) + if isinstance(record.event, DatasetConfigChanged) + else None ) workflow.propagate_config(new_config) diff --git a/tests/unit/pyqt_gui/test_main_status_bar.py b/tests/unit/pyqt_gui/test_main_status_bar.py index 9c7e9805a6..ab8b22056d 100644 --- a/tests/unit/pyqt_gui/test_main_status_bar.py +++ b/tests/unit/pyqt_gui/test_main_status_bar.py @@ -129,10 +129,7 @@ def test_endpoint_and_application_status_have_one_visual_owner( assert window._status_message_label.text() == application_message assert window._zmq_status_indicator._label.text() == "ZMQ: Connected" - assert window.button_refreshes == [ - "plate_manager_widget", - "pipeline_editor_widget", - ] + assert window.button_refreshes == ["plate_manager_widget", "pipeline_editor_widget"] assert ( window._zmq_status_indicator.toolTip() == "Execution endpoint 7777: Connected" diff --git a/tests/unit/pyqt_gui/test_main_window_file_actions.py b/tests/unit/pyqt_gui/test_main_window_file_actions.py index d78a5d182b..aaa1a11b50 100644 --- a/tests/unit/pyqt_gui/test_main_window_file_actions.py +++ b/tests/unit/pyqt_gui/test_main_window_file_actions.py @@ -21,14 +21,8 @@ def show_dataset_code(self, selection_mode) -> None: @pytest.mark.parametrize( "action, mode", ( - ( - OpenHCSMainWindow.load_orchestrator_configuration, - SelectedAllSelectionMode.SELECTED, - ), - ( - OpenHCSMainWindow.save_orchestrator_configuration, - SelectedAllSelectionMode.ALL, - ), + (OpenHCSMainWindow.load_orchestrator_configuration, SelectedAllSelectionMode.SELECTED), + (OpenHCSMainWindow.save_orchestrator_configuration, SelectedAllSelectionMode.ALL), ), ) def test_file_actions_open_the_dataset_code_document(action, mode) -> None: diff --git a/tests/unit/pyqt_gui/test_one_persistence_format.py b/tests/unit/pyqt_gui/test_one_persistence_format.py index 2b4e7a2eaf..233d6f82e0 100644 --- a/tests/unit/pyqt_gui/test_one_persistence_format.py +++ b/tests/unit/pyqt_gui/test_one_persistence_format.py @@ -84,10 +84,7 @@ def test_pipeline_round_trips_through_python_file(tmp_path: Path) -> None: assert (function, {key: kwargs[key] for key in _normalize(2.0)[1]}) == ( _normalize(2.0) ) - assert [step.name for step in editor.displayed_steps] == [ - "Normalize", - "Crop", - ] + assert [step.name for step in editor.displayed_steps] == ["Normalize", "Crop"] assert editor.code_document_source(clean=True) == target.read_text( encoding="utf-8" ) @@ -164,7 +161,8 @@ def test_gui_persistence_modules_do_not_pickle() -> None: for path in sorted(tree.rglob("*.py")): if path in TRANSPORT_MODULES: assert not any( - use.endswith((".dump", ".load")) for use in _pickle_uses(path) + use.endswith((".dump", ".load")) + for use in _pickle_uses(path) ), f"{path} may pickle in memory for ZMQ only" continue offenders.extend(_pickle_uses(path)) @@ -172,7 +170,7 @@ def test_gui_persistence_modules_do_not_pickle() -> None: def test_gui_offers_no_pickle_file_formats() -> None: - retired = ("*.func", "*.pipeline", "*.step", '.func"', '.pipeline"', '.step"') + retired = ("*.func", "*.pipeline", "*.step", ".func\"", ".pipeline\"", ".step\"") offenders = [ f"{path}: {marker}" for tree in PERSISTENCE_TREES diff --git a/tests/unit/pyqt_gui/test_parameter_form_manager_cross_window.py b/tests/unit/pyqt_gui/test_parameter_form_manager_cross_window.py index 555b75014c..33266987cc 100644 --- a/tests/unit/pyqt_gui/test_parameter_form_manager_cross_window.py +++ b/tests/unit/pyqt_gui/test_parameter_form_manager_cross_window.py @@ -147,9 +147,7 @@ def _on_live_context_changed(self): QTimer.singleShot(240, loop.quit) loop.exec() - assert manager.state.updated == [ - ("step_partition_filter_config.partition_filter", "B02") - ] + assert manager.state.updated == [("step_partition_filter_config.partition_filter", "B02")] assert manager.synced == ["partition_filter"] assert manager.flash_requests == [] assert manager.live_refreshes == 1 @@ -293,20 +291,14 @@ def get_resolved_value(self, path): partition_filter.form_tree = form_tree path_planning.form_tree = form_tree - root.chrome_sync.refresh_widgets_for_paths( - {"partition_filter_config.partition_filter"} - ) + root.chrome_sync.refresh_widgets_for_paths({"partition_filter_config.partition_filter"}) - assert widget_service.calls == [ - ("partition_filter_config", "partition_filter", "A01", False) - ] + assert widget_service.calls == [("partition_filter_config", "partition_filter", "A01", False)] assert partition_filter.widgets["partition_filter"].values == ["A01"] assert path_planning.widgets["partition_filter"].values == [] assert placeholder_service.calls == [] - root.chrome_sync.refresh_widgets_for_paths( - {"path_planning_config.partition_filter"} - ) + root.chrome_sync.refresh_widgets_for_paths({"path_planning_config.partition_filter"}) assert widget_service.calls[-1] == ( "path_planning_config", @@ -385,15 +377,9 @@ def _flush_resolved_values_changed(self): manager.deleteLater() qapp.processEvents() - assert manager.chrome_sync.widget_paths == [ - {"partition_filter_config.partition_filter"} - ] - assert manager.chrome_sync.state_paths == [ - {"partition_filter_config.partition_filter"} - ] - assert manager.flash_registrations == [ - ("partition_filter_config.partition_filter", False) - ] + assert manager.chrome_sync.widget_paths == [{"partition_filter_config.partition_filter"}] + assert manager.chrome_sync.state_paths == [{"partition_filter_config.partition_filter"}] + assert manager.flash_registrations == [("partition_filter_config.partition_filter", False)] assert manager.flash_batches == [("partition_filter_config.partition_filter",)] diff --git a/tests/unit/pyqt_gui/test_pipeline_code_document_transactionality.py b/tests/unit/pyqt_gui/test_pipeline_code_document_transactionality.py index 80095dc957..88e6421267 100644 --- a/tests/unit/pyqt_gui/test_pipeline_code_document_transactionality.py +++ b/tests/unit/pyqt_gui/test_pipeline_code_document_transactionality.py @@ -22,11 +22,9 @@ def _editor(tmp_path): gui.settle() changes = [] gui.session.subscribe( - lambda record: ( - changes.append(record.event.scope_id) - if isinstance(record.event, PipelineChanged) - else None - ) + lambda record: changes.append(record.event.scope_id) + if isinstance(record.event, PipelineChanged) + else None ) yield gui, scope_id, gui.pipeline_editor.code_document_driver(), changes diff --git a/tests/unit/pyqt_gui/test_pipeline_editor_widget.py b/tests/unit/pyqt_gui/test_pipeline_editor_widget.py index 9928b253a7..75818291c8 100644 --- a/tests/unit/pyqt_gui/test_pipeline_editor_widget.py +++ b/tests/unit/pyqt_gui/test_pipeline_editor_widget.py @@ -840,11 +840,9 @@ def test_code_document_driver_reads_validates_and_applies(tmp_path) -> None: harness.set_steps([FunctionStep(name="Original")]) changed = [] harness.session.subscribe( - lambda record: ( - changed.append(record.event.scope_id) - if isinstance(record.event, PipelineChanged) - else None - ) + lambda record: changed.append(record.event.scope_id) + if isinstance(record.event, PipelineChanged) + else None ) driver = harness.editor.code_document_driver() assert driver is not None @@ -1025,12 +1023,7 @@ def test_time_travel_broadcasts_only_pipeline_level_restores( assert broadcasts == [] harness.editor.on_time_travel_complete( - [ - ( - PipelineScopeIdentity.from_plate_scope(harness.scope_id).scope_id, - object(), - ) - ], + [(PipelineScopeIdentity.from_plate_scope(harness.scope_id).scope_id, object())], None, ) assert [[step.name for step in steps] for steps in broadcasts] == [ @@ -1261,9 +1254,7 @@ def test_step_partition_filter_live_resolution_is_visible_in_the_row(tmp_path) - ) dataset_state = ObjectStateRegistry.get_by_scope(harness.scope_id) - dataset_state.update_parameter( - "partition_filter_config.partition_filter", "A01" - ) + dataset_state.update_parameter("partition_filter_config.partition_filter", "A01") ObjectStateRegistry._notify_change() (step,) = harness.editor.displayed_steps styled_text, _ = harness.editor.format_item_for_display(step, step_index=0) @@ -1273,9 +1264,7 @@ def test_step_partition_filter_live_resolution_is_visible_in_the_row(tmp_path) - for segment in styled_text.layout.preview_segments if segment.field_path } - assert ( - preview_by_path["step_partition_filter_config.partition_filter"] == ":A01" - ) + assert preview_by_path["step_partition_filter_config.partition_filter"] == ":A01" def test_a_pipeline_config_scope_is_not_a_current_orchestrator() -> None: diff --git a/tests/unit/pyqt_gui/test_plate_compile_connection_action.py b/tests/unit/pyqt_gui/test_plate_compile_connection_action.py index 6621377350..5a959f2e60 100644 --- a/tests/unit/pyqt_gui/test_plate_compile_connection_action.py +++ b/tests/unit/pyqt_gui/test_plate_compile_connection_action.py @@ -59,9 +59,9 @@ def test_endpoint_phase_resolves_the_compile_slot(monkeypatch, phase, expected): def test_connect_needs_no_dataset_and_startup_is_never_available(tmp_path): with caller_session() as session: (scope_id,) = add_datasets(session, tmp_path, "plate") - session.orchestrator(scope_id)._state = session.orchestrator( - scope_id - )._state.READY + session.orchestrator(scope_id)._state = ( + session.orchestrator(scope_id)._state.READY + ) session.set_pipeline(scope_id, [FunctionStep(name="Step")]) assert ConnectServer.available(session, ConnectServer.request()) is None for selection in ((), (scope_id,)): diff --git a/tests/unit/pyqt_gui/test_plate_manager_widget.py b/tests/unit/pyqt_gui/test_plate_manager_widget.py index cf280cb893..d6fd7d8429 100644 --- a/tests/unit/pyqt_gui/test_plate_manager_widget.py +++ b/tests/unit/pyqt_gui/test_plate_manager_widget.py @@ -438,9 +438,7 @@ def test_workspace_import_replaces_steps_keeps_public_functions_and_applies_conf ) with caller_session() as session: session.add_dataset_roots((root,)) - session.set_pipeline( - scope_id, [FunctionStep(func=percentile_normalize, name="Existing")] - ) + session.set_pipeline(scope_id, [FunctionStep(func=percentile_normalize, name="Existing")]) session.import_workspace_pipeline(scope_id, _workspace([crop], pipeline_config)) @@ -535,9 +533,7 @@ def test_workspace_import_never_writes_into_the_displayed_dataset(tmp_path) -> N assert _names(gui.pipeline_editor.displayed_steps) == ["StartOnly"] -@pytest.mark.parametrize( - "state", (OrchestratorState.READY, OrchestratorState.EXECUTING) -) +@pytest.mark.parametrize("state", (OrchestratorState.READY, OrchestratorState.EXECUTING)) def test_source_binding_context_projects_the_current_orchestrator_config( tmp_path, state ) -> None: @@ -591,9 +587,7 @@ def test_source_binding_context_projects_the_current_orchestrator_config( def test_compile_needs_an_initialized_dataset(tmp_path) -> None: with caller_session() as session: (scope_id,) = add_datasets(session, tmp_path, "plate") - session.set_pipeline( - scope_id, [FunctionStep(func=percentile_normalize, name="Defined")] - ) + session.set_pipeline(scope_id, [FunctionStep(func=percentile_normalize, name="Defined")]) error = CompileDatasets.available( session, CompileDatasets.request_for_selection(session, (scope_id,)) @@ -670,9 +664,7 @@ def navigate(item_id, field_path=None): ): result = navigate(item_id, field_path) assert result.focused and not result.navigated - assert ( - result.errors[0].code == "ui_window_navigation_target_unsupported" - ) + assert result.errors[0].code == "ui_window_navigation_target_unsupported" assert gui.session.current_scope_id == paths[0] assert selected == list(reversed(paths)) @@ -711,9 +703,7 @@ def test_restart_restores_a_missing_dataset_without_initializing_it(tmp_path) -> payload = PlateManagerCodeDocumentAuthority.from_values( plate_paths=(scope_id,), global_pipeline_config=GlobalPipelineConfig(), - per_plate_configs={ - scope_id: PipelineConfig(dataset_source=OpenHCSDatasetSource) - }, + per_plate_configs={scope_id: PipelineConfig(dataset_source=OpenHCSDatasetSource)}, pipeline_data={scope_id: []}, ) restart.session_document.write_text( @@ -1036,9 +1026,7 @@ def test_adopting_a_global_config_preserves_a_dirty_dataset_override(tmp_path) - session.adopt_global_config(global_state.to_object(update_delegate=False)) gui.settle() - assert ( - object.__getattribute__(orchestrator.pipeline_config, "num_workers") is None - ) + assert object.__getattribute__(orchestrator.pipeline_config, "num_workers") is None assert dataset_state.parameters["num_workers"] == 3 assert dataset_state._saved_parameters["num_workers"] is None assert dataset_state.get_resolved_value("num_workers") == 3 @@ -1164,9 +1152,7 @@ def test_code_document_keeps_an_authored_dataset_override(tmp_path) -> None: def test_code_document_reads_the_global_config_from_object_state() -> None: - initial = GlobalPipelineConfig( - partition_filter_config=PartitionFilterConfig(partition_filter="A01") - ) + initial = GlobalPipelineConfig(partition_filter_config=PartitionFilterConfig(partition_filter="A01")) with caller_session(global_config=initial) as session: global_state = _global_state(session) global_state.update_parameter("partition_filter_config.partition_filter", "A02") @@ -1183,9 +1169,7 @@ def test_code_document_reads_the_global_config_from_object_state() -> None: == "A02" ) assert ( - object.__getattribute__( - session.global_config.partition_filter_config, "partition_filter" - ) + object.__getattribute__(session.global_config.partition_filter_config, "partition_filter") == "A01" ) @@ -1220,9 +1204,7 @@ def test_a_pipeline_change_clears_stale_compilation_and_terminal_tracking( global_pipeline_config=live_global_config(session), per_plate_configs={scope_id: authored_pipeline_config(scope_id)}, pipeline_data={ - scope_id: [ - FunctionStep(func=percentile_normalize, name="Replacement") - ] + scope_id: [FunctionStep(func=percentile_normalize, name="Replacement")] }, ), AllDatasetDocumentScope(), @@ -1248,9 +1230,7 @@ def _failed(execution_id: str, detail: str) -> ExecutionCompletionPayload: def _executing_dataset(gui, tmp_path) -> str: session = gui.session (scope_id,) = add_datasets(session, tmp_path, "plate") - session.set_pipeline( - scope_id, [FunctionStep(func=percentile_normalize, name="Failing")] - ) + session.set_pipeline(scope_id, [FunctionStep(func=percentile_normalize, name="Failing")]) orchestrator = session.orchestrator(scope_id) orchestrator._initialized = True orchestrator._state = OrchestratorState.EXECUTING @@ -1324,9 +1304,7 @@ def test_a_failure_is_presented_only_after_the_run_finalizes(tmp_path) -> None: replacement = PipelineDocumentCodec.render( PipelineDocumentCodec.from_values( pipeline_config=PipelineConfig(), - pipeline_steps=[ - FunctionStep(func=percentile_normalize, name="Replacement") - ], + pipeline_steps=[FunctionStep(func=percentile_normalize, name="Replacement")], ) ) @@ -1349,19 +1327,14 @@ def apply(source): ) try: - assert apply( - replacement.replace("Replacement", "Edited during run") - ).applied + assert apply(replacement.replace("Replacement", "Edited during run")).applied assert session.batch.is_active(scope_id) assert session.batch.execution_id(scope_id) == "execution-1" assert _names(session.pipeline_steps(scope_id)) == ["Edited during run"] worker = threading.Thread( target=session.finish_dataset_execution, - args=( - _failed("execution-1", "expected asynchronous failure"), - scope_id, - ), + args=(_failed("execution-1", "expected asynchronous failure"), scope_id), ) worker.start() worker.join(timeout=5) @@ -1467,14 +1440,10 @@ def test_a_produced_output_dataset_selects_prepared_replay_once( assert output_root in session.dataset_scope_ids() orchestrator = session.orchestrator(output_root) - assert ( - orchestrator.get_effective_config().dataset_source is OpenHCSDatasetSource - ) + assert orchestrator.get_effective_config().dataset_source is OpenHCSDatasetSource orchestrator.apply_pipeline_config( - replace_raw( - orchestrator.pipeline_config, dataset_source=SourceBindingsSource - ) + replace_raw(orchestrator.pipeline_config, dataset_source=SourceBindingsSource) ) _running(session, source, "produced-again") session.finish_dataset_execution( @@ -1490,9 +1459,7 @@ def test_a_produced_output_dataset_selects_prepared_replay_once( ) assert session.orchestrator(output_root) is orchestrator - assert ( - orchestrator.get_effective_config().dataset_source is SourceBindingsSource - ) + assert orchestrator.get_effective_config().dataset_source is SourceBindingsSource def test_a_standard_run_retires_only_its_targets_debug_summaries( diff --git a/tests/unit/pyqt_gui/test_session_batch_workflows.py b/tests/unit/pyqt_gui/test_session_batch_workflows.py index 67d2fc63d6..016708c015 100644 --- a/tests/unit/pyqt_gui/test_session_batch_workflows.py +++ b/tests/unit/pyqt_gui/test_session_batch_workflows.py @@ -160,11 +160,9 @@ def test_run_and_debug_reserve_before_connect_and_release_on_connection_failure( session.compile_pending.add(unrelated) errors = [] session.subscribe( - lambda record: ( - errors.append(record.event) - if type(record.event).__name__ == "ErrorReported" - else None - ) + lambda record: errors.append(record.event) + if type(record.event).__name__ == "ErrorReported" + else None ) async def connect(): @@ -187,7 +185,9 @@ async def connect(): @pytest.mark.parametrize("fails", (False, True)) -def test_compiling_another_dataset_keeps_the_active_batch(tmp_path, monkeypatch, fails): +def test_compiling_another_dataset_keeps_the_active_batch( + tmp_path, monkeypatch, fails +): with caller_session() as session: running, other = add_datasets(session, tmp_path, "running", "other") _ready(session, running, other) @@ -250,11 +250,9 @@ def test_compile_publishes_its_inspection_and_clears_the_previous_one( _ready(session, scope_id) compiled_states = [] session.subscribe( - lambda record: ( - compiled_states.append(record.event.compiled) - if isinstance(record.event, CompiledStateChanged) - else None - ) + lambda record: compiled_states.append(record.event.compiled) + if isinstance(record.event, CompiledStateChanged) + else None ) def during_compile(_scope_id): @@ -328,11 +326,9 @@ def test_failed_compile_reports_the_dataset(tmp_path, monkeypatch): _ready(session, scope_id) failures = [] session.subscribe( - lambda record: ( - failures.append(record.event) - if isinstance(record.event, CompilationFailed) - else None - ) + lambda record: failures.append(record.event) + if isinstance(record.event, CompilationFailed) + else None ) StandInServer(session, monkeypatch, failing={scope_id}) asyncio.run(session.compile_datasets((scope_id,))) @@ -490,19 +486,16 @@ def _debug_progress_event(*, snapshot_id: str | None, event_type=None): callable_name="IdentifyPrimaryObjects", axis_id="A01", ) - return ( - event, - DebugProgressEventRequest( - debug_session_id="debug-1", - debug_event=event, - progress_context=ProgressExecutionContext("exec-1", "plate-1"), - **( - {} - if snapshot_id is None - else {"snapshot_id": snapshot_id, "snapshot_store_ref": "/tmp/debug"} - ), - ).to_progress_event(), - ) + return event, DebugProgressEventRequest( + debug_session_id="debug-1", + debug_event=event, + progress_context=ProgressExecutionContext("exec-1", "plate-1"), + **( + {} + if snapshot_id is None + else {"snapshot_id": snapshot_id, "snapshot_store_ref": "/tmp/debug"} + ), + ).to_progress_event() def _events_of(session, kind): @@ -594,9 +587,7 @@ def test_runtime_artifact_progress_is_published(): artifact_type=MeasurementsArtifactType, scope=RuntimeExecutionAxisScope(axis_id="A01"), ), - location=RuntimeArtifactLocation( - path="/memory/result.pkl", backend="memory" - ), + location=RuntimeArtifactLocation(path="/memory/result.pkl", backend="memory"), value_type="ndarray", ) event = _progress_event( @@ -620,9 +611,7 @@ def test_progress_the_registry_rejects_publishes_nothing(monkeypatch): artifact_type=MeasurementsArtifactType, scope=RuntimeExecutionAxisScope(axis_id="A01"), ), - location=RuntimeArtifactLocation( - path="/memory/result.pkl", backend="memory" - ), + location=RuntimeArtifactLocation(path="/memory/result.pkl", backend="memory"), ) session.progress.on_progress( _progress_event( @@ -639,6 +628,7 @@ def test_server_queue_snapshot_projects_queued_executions(monkeypatch): from openhcs.core.progress.projection import QueuedSubjectRuntimeState + with caller_session() as session: monkeypatch.setattr( session.progress._server_info_poller, diff --git a/tests/unit/pyqt_gui/test_synthetic_plate_pipeline_document.py b/tests/unit/pyqt_gui/test_synthetic_plate_pipeline_document.py index 331f7bcb9f..e5e00c7f07 100644 --- a/tests/unit/pyqt_gui/test_synthetic_plate_pipeline_document.py +++ b/tests/unit/pyqt_gui/test_synthetic_plate_pipeline_document.py @@ -96,11 +96,9 @@ def test_main_window_loads_emitted_synthetic_pipeline_document( main_window = _MainWindowHarness(session) changed = [] session.subscribe( - lambda record: ( - changed.append(record.event.scope_id) - if isinstance(record.event, PipelineChanged) - else None - ) + lambda record: changed.append(record.event.scope_id) + if isinstance(record.event, PipelineChanged) + else None ) OpenHCSMainWindow._on_synthetic_plate_generated( @@ -111,9 +109,7 @@ def test_main_window_loads_emitted_synthetic_pipeline_document( assert main_window.plate_manager_shown == 1 assert main_window.pipeline_editor_shown == 1 assert session.current_scope_id == scope_id - assert ( - len(session.pipeline_steps(scope_id)) == len(document.pipeline_steps) == 8 - ) + assert len(session.pipeline_steps(scope_id)) == len(document.pipeline_steps) == 8 assert [step.name for step in session.pipeline_steps(scope_id)] == [ step.name for step in document.pipeline_steps ] diff --git a/tests/unit/pyqt_gui/test_ui_agent_bridge.py b/tests/unit/pyqt_gui/test_ui_agent_bridge.py index e4c0e23c27..885f55da94 100644 --- a/tests/unit/pyqt_gui/test_ui_agent_bridge.py +++ b/tests/unit/pyqt_gui/test_ui_agent_bridge.py @@ -2648,6 +2648,8 @@ def test_object_state_exact_field_paths_include_path_type_and_description() -> N ) + + def test_managed_window_save_action_returns_before_deferred_save_runs() -> None: app = QtApplicationAuthority.app() dispatcher = QueuedPostDispatcher() @@ -2695,17 +2697,13 @@ def test_managed_window_save_action_returns_before_deferred_save_runs() -> None: assert len(action.target_scope_ids) == 2 assert len(dispatcher.callbacks) == 1 assert ( - bridge.get_operation_status( - UiBridgeOperationStatusRequest(result.receipt.bridge_operation_id) - ).status + bridge.get_operation_status(UiBridgeOperationStatusRequest(result.receipt.bridge_operation_id)).status == "running" ) dispatcher.run_next() - operation = bridge.get_operation_status( - UiBridgeOperationStatusRequest(result.receipt.bridge_operation_id) - ) + operation = bridge.get_operation_status(UiBridgeOperationStatusRequest(result.receipt.bridge_operation_id)) assert window.save_count == 1 assert window.saved_close_window is False assert operation.status == "completed" @@ -3114,18 +3112,11 @@ def test_apply_document_returns_running_operation_before_queued_ui_apply_runs() assert result.receipt.accepted is True assert result.receipt.bridge_operation_id == result.operation_id assert operations.pre_count == 0 - assert ( - bridge.get_operation_status( - UiBridgeOperationStatusRequest(result.operation_id) - ).status - == "running" - ) + assert bridge.get_operation_status(UiBridgeOperationStatusRequest(result.operation_id)).status == "running" dispatcher.run_next() - operation = bridge.get_operation_status( - UiBridgeOperationStatusRequest(result.operation_id) - ) + operation = bridge.get_operation_status(UiBridgeOperationStatusRequest(result.operation_id)) assert operations.pre_count == 1 assert operations.post_count == 1 assert operation.status == "completed" @@ -3158,18 +3149,11 @@ def test_queued_apply_document_error_updates_operation_status() -> None: assert result.applied is False assert result.operation_id is not None assert result.receipt.accepted is True - assert ( - bridge.get_operation_status( - UiBridgeOperationStatusRequest(result.operation_id) - ).status - == "running" - ) + assert bridge.get_operation_status(UiBridgeOperationStatusRequest(result.operation_id)).status == "running" dispatcher.run_next() - operation = bridge.get_operation_status( - UiBridgeOperationStatusRequest(result.operation_id) - ) + operation = bridge.get_operation_status(UiBridgeOperationStatusRequest(result.operation_id)) assert operation.status == "completed" assert operation.outcome == "not_applied" assert operation.errors @@ -3451,15 +3435,15 @@ def test_two_ipc_ui_bridges_route_each_descriptor_to_its_exact_instance( service = UiBridgeService() first_status = service.invoke( UiBridgeStatusOperation, - connection=service.connection_from_args( + connection= service.connection_from_args( descriptor_file_path=first_binding.descriptor_file_path - ), + ) ) second_status = service.invoke( UiBridgeStatusOperation, - connection=service.connection_from_args( + connection= service.connection_from_args( descriptor_file_path=second_binding.descriptor_file_path - ), + ) ) assert first_status.reachable is True @@ -3910,7 +3894,9 @@ def document(mode): assert newly_selected_document.summary.current_selection_count == 1 assert newly_selected_state.selected_scope_ids == (second,) assert newly_selected_state.summary.current_selection_count == 1 - assert [row["scope_id"] for row in newly_selected_state.payload["rows"]] == [second] + assert [ + row["scope_id"] for row in newly_selected_state.payload["rows"] + ] == [second] assert newly_selected_state.payload["rows"][0]["selected"] is True assert ( newly_selected_state.current_revision_token @@ -4049,8 +4035,7 @@ def test_plate_manager_intermediate_plate_keeps_both_path_relations( tmp_path: Path, reverse_rows: bool ) -> None: roots = tuple( - str(tmp_path / name) - for name in ("raw", "raw_stitched", "raw_stitched_analysis") + str(tmp_path / name) for name in ("raw", "raw_stitched", "raw_stitched_analysis") ) ordered = roots[::-1] if reverse_rows else roots with dataset_gui(*ordered, selected=(1,)) as (gui, _scope_ids): @@ -4224,19 +4209,12 @@ def test_selected_workflow_returns_before_the_queued_operation_runs( ) assert started == [] assert len(dispatcher.callbacks) == 1 - assert ( - bridge.get_operation_status( - UiBridgeOperationStatusRequest(operation_id) - ).status - == "running" - ) + assert bridge.get_operation_status(UiBridgeOperationStatusRequest(operation_id)).status == "running" dispatcher.run_next() assert started == [((DATASET_SCOPE_ID,),)] - operation = bridge.get_operation_status( - UiBridgeOperationStatusRequest(operation_id) - ) + operation = bridge.get_operation_status(UiBridgeOperationStatusRequest(operation_id)) assert operation.status == "completed" assert operation.outcome == "accepted" @@ -4478,7 +4456,9 @@ def function_reference(function): assert state.payload["steps"][0]["function_ids"] == ["test:function_0"] assert state.payload["steps"][1]["function_ids"] == ["test:function_1"] assert selected_state.payload["steps"][0]["name"] == "step_two" - assert selected_state.payload["steps"][0]["function_ids"] == ["test:function_1"] + assert selected_state.payload["steps"][0]["function_ids"] == [ + "test:function_1" + ] assert selected_state.unchanged is False @@ -4486,7 +4466,9 @@ def _debug_actions(bridge): return _actions(bridge, PipelineDebugToolbarWidgetIdentity.require_value()) -def test_pipeline_debug_toolbar_actions_are_exposed_from_toolbar_declarations() -> None: +def test_pipeline_debug_toolbar_actions_are_exposed_from_toolbar_declarations() -> ( + None +): with pipeline_gui() as gui: actions = _debug_actions(_pipeline_editor_bridge(gui.pipeline_editor)) @@ -4574,9 +4556,7 @@ def test_pipeline_debug_toolbar_runtime_values_action_requires_debug_session( ) bridge = _pipeline_editor_bridge(gui.pipeline_editor) widget_id = PipelineDebugToolbarWidgetIdentity.require_value() - action = _debug_actions(bridge)[ - DebugToolbarAuxiliaryAction.RUNTIME_VALUES.value - ] + action = _debug_actions(bridge)[DebugToolbarAuxiliaryAction.RUNTIME_VALUES.value] result = bridge.invoke_action( UiActionInvokeRequest( @@ -4749,6 +4729,5 @@ def test_pipeline_debug_session_terminal_summary_retires_the_inspected_session() assert actions[DebugCommandType.RESTART.value]["enabled"] is False assert actions[DebugCommandType.STOP.value]["enabled"] is False assert ( - actions[DebugToolbarAuxiliaryAction.RUNTIME_VALUES.value]["enabled"] - is False + actions[DebugToolbarAuxiliaryAction.RUNTIME_VALUES.value]["enabled"] is False ) diff --git a/tests/unit/pyqt_gui/test_ui_bridge_live_measurements.py b/tests/unit/pyqt_gui/test_ui_bridge_live_measurements.py index 25d02cdba6..7c44f54be7 100644 --- a/tests/unit/pyqt_gui/test_ui_bridge_live_measurements.py +++ b/tests/unit/pyqt_gui/test_ui_bridge_live_measurements.py @@ -293,9 +293,7 @@ def test_live_measurement_surface_honors_plate_selection_and_populates_overview( "value": "1", } assert section["metrics"][1]["value"] == "3" - assert section["items"][0]["label"] == ( - "Neurite Outgrowth: PerNeuronMeasurements" - ) + assert section["items"][0]["label"] == ("Neurite Outgrowth: PerNeuronMeasurements") manager.selected = [manager.plates[1]] filtered = bridge.get_state_surface( diff --git a/tests/unit/pyqt_gui/test_ui_window_flash_snapshot.py b/tests/unit/pyqt_gui/test_ui_window_flash_snapshot.py index fc5be727a2..a91f70f080 100644 --- a/tests/unit/pyqt_gui/test_ui_window_flash_snapshot.py +++ b/tests/unit/pyqt_gui/test_ui_window_flash_snapshot.py @@ -94,24 +94,24 @@ def test_async_snapshot_uses_existing_operation_and_registered_response_contract bridge._mutation_gate._lock.release() form.update_parameter("number", 8) qtbot.waitUntil( - lambda: bridge.get_operation_status( - UiBridgeOperationStatusRequest(accepted.operation_id) - ).completed_at_unix + lambda: bridge.get_operation_status(UiBridgeOperationStatusRequest(accepted.operation_id)).completed_at_unix is not None, timeout=2000, ) - operation = bridge.get_operation_status( - UiBridgeOperationStatusRequest(accepted.operation_id) - ) + operation = bridge.get_operation_status(UiBridgeOperationStatusRequest(accepted.operation_id)) assert operation.status == "completed" contract = UiBridgeOperation.for_name(operation.identity.operation_name) - result = dataclass_from_mapping(contract.result_type, operation.result_payload) + result = dataclass_from_mapping( + contract.result_type, operation.result_payload + ) assert isinstance(result, UiWindowSnapshotResult) assert result.captured assert result.frame_condition is request.frame_condition assert result.observation.frame.has_maximum_alpha assert result.resource.path.endswith(".png") - wire_operation = dataclass_from_mapping(type(operation), to_jsonable(operation)) + wire_operation = dataclass_from_mapping( + type(operation), to_jsonable(operation) + ) assert wire_operation == operation wire_result = dataclass_from_mapping( contract.result_type, @@ -152,12 +152,7 @@ def test_async_snapshot_invalid_baseline_is_terminal_failed( ) assert not result.captured assert result.errors - assert ( - bridge.get_operation_status( - UiBridgeOperationStatusRequest(result.operation_id) - ).status - == "failed" - ) + assert bridge.get_operation_status(UiBridgeOperationStatusRequest(result.operation_id)).status == "failed" def test_noop_reset_quiet_operation_preserves_actual_interval_receipt( @@ -176,17 +171,15 @@ def test_noop_reset_quiet_operation_preserves_actual_interval_receipt( ) form.reset_buttons["number"].click() qtbot.waitUntil( - lambda: bridge.get_operation_status( - UiBridgeOperationStatusRequest(accepted.operation_id) - ).completed_at_unix + lambda: bridge.get_operation_status(UiBridgeOperationStatusRequest(accepted.operation_id)).completed_at_unix is not None, timeout=2000, ) - operation = bridge.get_operation_status( - UiBridgeOperationStatusRequest(accepted.operation_id) - ) + operation = bridge.get_operation_status(UiBridgeOperationStatusRequest(accepted.operation_id)) contract = UiBridgeOperation.for_name(operation.identity.operation_name) - result = dataclass_from_mapping(contract.result_type, operation.result_payload) + result = dataclass_from_mapping( + contract.result_type, operation.result_payload + ) assert result.captured assert ( result.observation.flash_start_count @@ -219,25 +212,26 @@ def test_failed_observation_frame_and_existing_trace_decode_through_registered_o ) form.update_parameter("number", 8) qtbot.waitUntil( - lambda: bridge.get_operation_status( - UiBridgeOperationStatusRequest(accepted.operation_id) - ).completed_at_unix + lambda: bridge.get_operation_status(UiBridgeOperationStatusRequest(accepted.operation_id)).completed_at_unix is not None, timeout=2000, ) - operation = bridge.get_operation_status( - UiBridgeOperationStatusRequest(accepted.operation_id) - ) + operation = bridge.get_operation_status(UiBridgeOperationStatusRequest(accepted.operation_id)) assert operation.status == "failed" contract = UiBridgeOperation.for_name(operation.identity.operation_name) - result = dataclass_from_mapping(contract.result_type, operation.result_payload) + result = dataclass_from_mapping( + contract.result_type, operation.result_payload + ) assert not result.captured and result.resource is None assert isinstance(result.observation.frame, FlashPaintFrame) assert result.observation.trace and len(result.observation.trace) <= 300 assert all( isinstance(record, FlashTraceRecord) for record in result.observation.trace ) - assert dataclass_from_mapping(type(result), to_jsonable(result)) == result + assert ( + dataclass_from_mapping(type(result), to_jsonable(result)) + == result + ) def test_snapshot_cli_derives_typed_observation_arguments(tmp_path): @@ -309,16 +303,16 @@ def _route_index(self): assert not accepted.navigated assert accepted.target_exposed is None qtbot.waitUntil( - lambda: bridge.get_operation_status( - UiBridgeOperationStatusRequest(accepted.operation_id) - ).completed_at_unix + lambda: bridge.get_operation_status(UiBridgeOperationStatusRequest(accepted.operation_id)).completed_at_unix is not None ) - operation = bridge.get_operation_status( - UiBridgeOperationStatusRequest(accepted.operation_id) + operation = bridge.get_operation_status(UiBridgeOperationStatusRequest(accepted.operation_id)) + contract = UiBridgeOperation.for_name( + operation.identity.operation_name + ) + result = dataclass_from_mapping( + contract.result_type, operation.result_payload ) - contract = UiBridgeOperation.for_name(operation.identity.operation_name) - result = dataclass_from_mapping(contract.result_type, operation.result_payload) assert isinstance(result, UiWindowNavigateResult) assert operation.status == "completed" assert selected == ["number"] @@ -344,12 +338,7 @@ def test_unowned_target_navigation_is_terminal_failed_not_pending( ) assert accepted.errors assert accepted.operation_id - assert ( - bridge.get_operation_status( - UiBridgeOperationStatusRequest(accepted.operation_id) - ).status - == "failed" - ) + assert bridge.get_operation_status(UiBridgeOperationStatusRequest(accepted.operation_id)).status == "failed" def test_tracker_observation_failure_uses_declared_operation_error_owner( diff --git a/tests/unit/pyqt_gui/test_zmq_client_service.py b/tests/unit/pyqt_gui/test_zmq_client_service.py index b62e3d094b..668f86ef54 100644 --- a/tests/unit/pyqt_gui/test_zmq_client_service.py +++ b/tests/unit/pyqt_gui/test_zmq_client_service.py @@ -93,9 +93,7 @@ def endpoint_compatibility(self) -> EndpointApplicationCompatibility: ) -def test_async_connect_cancellation_retires_same_attempt_and_releases_ownership( - monkeypatch, -): +def test_async_connect_cancellation_retires_same_attempt_and_releases_ownership(monkeypatch): import openhcs.runtime.zmq_execution_client as client_module SlowFakeExecutionClient.instances = [] @@ -181,7 +179,6 @@ def new_connection_attempt(self): class BlockingConnectionAttempt(EndpointConnectionAttempt): def __init__(self): pass - def cancel(self) -> None: BlockingExecutionClient.attempt_cancelled.set() diff --git a/tests/unit/remote_sensing_witness_pipeline.py b/tests/unit/remote_sensing_witness_pipeline.py index 68e78185d2..b051cf76c7 100644 --- a/tests/unit/remote_sensing_witness_pipeline.py +++ b/tests/unit/remote_sensing_witness_pipeline.py @@ -94,8 +94,7 @@ def pipeline_steps(napari_streaming_config=None) -> list: ) streaming = ( - {} - if napari_streaming_config is None + {} if napari_streaming_config is None else {"napari_streaming_config": napari_streaming_config} ) return [ diff --git a/tests/unit/test_ack_return_route_journey.py b/tests/unit/test_ack_return_route_journey.py index 2894eb767a..37241227fd 100644 --- a/tests/unit/test_ack_return_route_journey.py +++ b/tests/unit/test_ack_return_route_journey.py @@ -49,15 +49,10 @@ def stream_producer(pipe, port, mode, config): identity=ViewerStreamSourceIdentity(SyntheticMicroscope(), None), metadata=BatchViewerStreamSourceMetadata({"well": "synthetic"}), ), - producer=ViewerStreamProducer.from_identity( - StreamProducerIdentity.pipeline_output( - output_kind="main", - output_key="main", - projection_key="main", - step_name="SyntheticTransport", - pipeline_position=0, - ) - ), + producer=ViewerStreamProducer.from_identity(StreamProducerIdentity.pipeline_output( + output_kind="main", output_key="main", projection_key="main", + step_name="SyntheticTransport", pipeline_position=0, + )), ) try: for _ in range(2): @@ -67,19 +62,10 @@ def stream_producer(pipe, port, mode, config): break assert command == "send" completed.clear() - backend.save_batch( - [np.arange(12, dtype=np.uint16).reshape(3, 4)], - ["synthetic.tif"], - stream_request=request, - ) + backend.save_batch([np.arange(12, dtype=np.uint16).reshape(3, 4)], + ["synthetic.tif"], stream_request=request) tracker = GlobalQueueTrackerRegistry().get_tracker(port) - pipe.send( - ( - listener.return_route, - tracker.get_progress(), - len(backend._shared_memory_blocks), - ) - ) + pipe.send((listener.return_route, tracker.get_progress(), len(backend._shared_memory_blocks))) assert pipe.poll(6), "No controlled ACK check" command = pipe.recv() if command == "close": @@ -102,13 +88,9 @@ def test_two_native_stream_producers_keep_deferred_ack_ownership(mode, unix_sock directory = unix_socket_dir config = ZMQConfig(default_port=48000, ipc_socket_dir=str(directory)) pair = DataControlPortPairAuthority.acquire(config, transport_mode=mode) - server = NapariViewerServer( - NapariViewerServerRequest( - port=pair.data_port, - viewer_title="synthetic ACK transport", - transport_mode=mode, - ) - ) + server = NapariViewerServer(NapariViewerServerRequest( + port=pair.data_port, viewer_title="synthetic ACK transport", transport_mode=mode, + )) server.config = config server._running = True owners = [] @@ -118,17 +100,9 @@ def test_two_native_stream_producers_keep_deferred_ack_ownership(mode, unix_sock parent, child = Pipe() # Separate native interpreters, like the application: multiprocessing # siblings would share one resource-tracker and invalidate this proof. - process = Popen( - [ - sys.executable, - __file__, - str(child.fileno()), - str(pair.data_port), - mode.value, - str(directory), - ], - pass_fds=(child.fileno(),), - ) + process = Popen([sys.executable, __file__, str(child.fileno()), + str(pair.data_port), mode.value, str(directory)], + pass_fds=(child.fileno(),)) child.close() owners.append((process, parent)) routes = [] @@ -141,9 +115,7 @@ def test_two_native_stream_producers_keep_deferred_ack_ownership(mode, unix_sock accepted = server.accepted_stream_batches.get(timeout=3) item = accepted.items[0] assert item.payload.transfer.return_route == route - np.testing.assert_array_equal( - item.data, np.arange(12, dtype=np.uint16).reshape(3, 4) - ) + np.testing.assert_array_equal(item.data, np.arange(12, dtype=np.uint16).reshape(3, 4)) with pytest.raises(FileNotFoundError): SharedMemory(name=item.payload.shm_name) assert server.send_ack(item.payload.transfer) @@ -152,9 +124,7 @@ def test_two_native_stream_producers_keep_deferred_ack_ownership(mode, unix_sock if index == 0: pipe.send("send") route_again, progress, remaining_memory = receive(pipe) - assert ( - route_again == route - ) # One warm process owner, not a port per batch. + assert route_again == route # One warm process owner, not a port per batch. assert progress == (1, 2) second = server.accepted_stream_batches.get(timeout=3).items[0] assert server.send_ack(second.payload.transfer) @@ -187,9 +157,5 @@ def test_two_native_stream_producers_keep_deferred_ack_ownership(mode, unix_sock if __name__ == "__main__": - stream_producer( - Connection(int(sys.argv[1])), - int(sys.argv[2]), - TransportMode(sys.argv[3]), - ZMQConfig(ipc_socket_dir=sys.argv[4]), - ) + stream_producer(Connection(int(sys.argv[1])), int(sys.argv[2]), + TransportMode(sys.argv[3]), ZMQConfig(ipc_socket_dir=sys.argv[4])) diff --git a/tests/unit/test_acquisition_calibration_pipeline_journey.py b/tests/unit/test_acquisition_calibration_pipeline_journey.py index 48a622e510..dd34e649c5 100644 --- a/tests/unit/test_acquisition_calibration_pipeline_journey.py +++ b/tests/unit/test_acquisition_calibration_pipeline_journey.py @@ -28,7 +28,9 @@ def test_htd_spacing_survives_gaussian_compile_execute_and_persist(tmp_path): plate = tmp_path / "plate" image_dir = plate / "TimePoint_1" image_dir.mkdir(parents=True) - (plate / "plate.HTD").write_text('"XSites", 1\n"YSites", 1\n"PixelSizeUM", 0.65') + (plate / "plate.HTD").write_text( + '"XSites", 1\n"YSites", 1\n"PixelSizeUM", 0.65' + ) pixels = np.random.default_rng(31).integers(0, 1000, (96, 96), dtype=np.uint16) filename = "A01_s001_w1_z001_t001.tif" tifffile.imwrite(image_dir / filename, pixels) diff --git a/tests/unit/test_aggregate_source_scope.py b/tests/unit/test_aggregate_source_scope.py index 1b590e100a..318d135176 100644 --- a/tests/unit/test_aggregate_source_scope.py +++ b/tests/unit/test_aggregate_source_scope.py @@ -7,44 +7,26 @@ import numpy as np import pytest -from openhcs.core.artifacts import ( - ArtifactSpec, - ImageArtifactType, - MeasurementsArtifactType, -) +from openhcs.core.artifacts import ArtifactSpec, ImageArtifactType, MeasurementsArtifactType from openhcs.core.component_group_scope import RuntimeExecutionAxisScope from openhcs.core.measurement_row_materialization import MeasurementSparseColumnarRows from openhcs.core.orchestrator.execution_result import ( - RuntimeContextObservation, - RuntimeExecutionObservation, + RuntimeContextObservation, RuntimeExecutionObservation, ) from openhcs.core.projected_image_output import SourceProjectedImageOutput from openhcs.core.runtime_artifact_values import ArtifactKey, RuntimeValue from openhcs.core.runtime_exports import ( - RuntimeExportExpectation, - RuntimeExportObservation, - runtime_export_failures, + RuntimeExportExpectation, RuntimeExportObservation, runtime_export_failures, ) from openhcs.core.runtime_image_values import ImagePayloadMetadata -from openhcs.core.runtime_measurements import ( - MeasurementScope, - MeasurementSubject, - MeasurementTable, -) -from openhcs.core.runtime_plane_projection import ( - RuntimePlaneAxis, - RuntimePlaneAxisValueProjection, -) +from openhcs.core.runtime_measurements import MeasurementScope, MeasurementSubject, MeasurementTable +from openhcs.core.runtime_plane_projection import RuntimePlaneAxis, RuntimePlaneAxisValueProjection from openhcs.core.runtime_stores import ( - RuntimeArtifactAddress, - RuntimeArtifactLocation, - RuntimeValueStore, + RuntimeArtifactAddress, RuntimeArtifactLocation, RuntimeValueStore, ) from openhcs.core.runtime_tabular_values import FieldSpec from openhcs.core.source_image_provenance import ( - RuntimeSourceImageProvenancePlane, - SourceImageIdentity, - SourceImageProvenancePlanes, + RuntimeSourceImageProvenancePlane, SourceImageIdentity, SourceImageProvenancePlanes, ) from openhcs.core.source_metadata import SourceVoxelSpacing from openhcs.core.steps.abstract import StepExecutionObservation @@ -63,19 +45,12 @@ def _source(sites): source_image_provenance_planes=SourceImageProvenancePlanes.from_components( paths=tuple(f"/input/field_{i}.tif" for i in range(len(sites))), component_metadata=tuple( - { - "well": "A01", - "channel": 2, - "z_index": 1, - "timepoint": 1, - **({"site": site} if site is not None else {}), - } + {"well": "A01", "channel": 2, "z_index": 1, "timepoint": 1, + **({"site": site} if site is not None else {})} for site in sites ), ), - ).payload_with( - np.arange(len(sites) * 12, dtype=np.uint16).reshape(len(sites), 3, 4) - ) + ).payload_with(np.arange(len(sites) * 12, dtype=np.uint16).reshape(len(sites), 3, 4)) class ProjectionAudit: @@ -105,12 +80,8 @@ def test_mixed_acquired_and_reduced_batch_uses_distinct_original_display_scopes( source = _source((1, 3)) declaration = DeclaredReducedField(np.full((3, 4), 720, dtype=np.uint16), []) reduced = ImageArtifactType.contextualize_output( - source, - declaration, - None, - RuntimePlaneAxisValueProjection.preserve( - axis=RuntimePlaneAxis.SOURCE_BINDING, axis_size=2 - ), + source, declaration, None, + RuntimePlaneAxisValueProjection.preserve(axis=RuntimePlaneAxis.SOURCE_BINDING, axis_size=2), ) assert declaration.calls == ["before", "after"] acquired_metadata = source.metadata.for_leading_source_plane(0) @@ -124,52 +95,33 @@ def test_mixed_acquired_and_reduced_batch_uses_distinct_original_display_scopes( request = kwargs["stream_request"] single = adapter.to_filemanager_kwargs(output)["stream_request"] assert request.display_config == single.display_config - assert request.source.metadata.component_metadata_for_item( - output.path, 0 - ) == single.source.metadata.component_metadata_for_item(output.path, 0) + assert request.source.metadata.component_metadata_for_item(output.path, 0) == single.source.metadata.component_metadata_for_item(output.path, 0) assert request.producer == adapter.values.stream_request.producer - assert ( - request.viewer_transport == adapter.values.stream_request.viewer_transport - ) - assert ( - request.display_config.component_modes() - == adapter.values.stream_request.display_config.component_modes() - ) + assert request.viewer_transport == adapter.values.stream_request.viewer_transport + assert request.display_config.component_modes() == adapter.values.stream_request.display_config.component_modes() assert output.metadata.source_voxel_spacing == SourceVoxelSpacing((0.5, 0.5)) - acquired_request, reduced_request = ( - kwargs["stream_request"] for _, kwargs in batches - ) + acquired_request, reduced_request = (kwargs["stream_request"] for _, kwargs in batches) assert "site" in acquired_request.display_config.COMPONENT_ORDER assert "site" not in reduced_request.display_config.COMPONENT_ORDER - assert ( - aggregate.metadata.source_provenance.source_image_provenance_planes.contributor_count - == 2 - ) + assert aggregate.metadata.source_provenance.source_image_provenance_planes.contributor_count == 2 np.testing.assert_array_equal(aggregate.content, declaration.data) @pytest.mark.parametrize("sites", ((), (None,), (1,), (1, 1), (1, None), (None, None))) -def test_absence_one_equal_or_incomplete_contributors_cannot_exempt_acquired_plane( - sites, -): +def test_absence_one_equal_or_incomplete_contributors_cannot_exempt_acquired_plane(sites): metadata = _source(sites).metadata.collapse_leading_plane_axis() components = { - "well": "A01", - "channel": 2, - "z_index": 1, - "timepoint": 1, + "well": "A01", "channel": 2, "z_index": 1, "timepoint": 1, } # Explicitly absent scalar identity, not a selected plane with a valid common site. metadata = ImagePayloadMetadata( source_component_metadata=components, - source_image_provenance_planes=SourceImageProvenancePlanes( - ( - RuntimeSourceImageProvenancePlane( - SourceImageIdentity(component_metadata=components), - contributors=metadata.source_image_provenance_planes.contributors, - ), - ) - ), + source_image_provenance_planes=SourceImageProvenancePlanes(( + RuntimeSourceImageProvenancePlane( + SourceImageIdentity(component_metadata=components), + contributors=metadata.source_image_provenance_planes.contributors, + ), + )), ) output = Output("unaddressed.tif", np.zeros((3, 4), dtype=np.uint16), metadata) adapter = _viewer_stream_backend_kwargs() @@ -182,119 +134,64 @@ def test_absence_one_equal_or_incomplete_contributors_cannot_exempt_acquired_pla def test_present_scalar_override_remains_required_despite_varied_contributors(): metadata = (_source((1, 3))).metadata.collapse_leading_plane_axis() - metadata = metadata.replace_fields( - source_component_metadata={ - "well": "A01", - "site": 7, - "channel": 2, - "z_index": 1, - "timepoint": 1, - } - ) + metadata = metadata.replace_fields(source_component_metadata={ + "well": "A01", "site": 7, "channel": 2, "z_index": 1, "timepoint": 1, + }) assert metadata.source_provenance.required_scalar_components(("site",)) == ("site",) output = Output("override.tif", np.zeros((3, 4), dtype=np.uint16), metadata) - request = _viewer_stream_backend_kwargs().to_filemanager_kwargs(output)[ - "stream_request" - ] + request = _viewer_stream_backend_kwargs().to_filemanager_kwargs(output)["stream_request"] assert "site" in request.display_config.COMPONENT_ORDER - assert ( - request.source.metadata.component_metadata_for_item(output.path, 0)["site"] == 7 - ) + assert request.source.metadata.component_metadata_for_item(output.path, 0)["site"] == 7 def _saved_table(tmp_path, *, producer, fields): table = MeasurementTable( - name="Cells", - subject=MeasurementSubject(MeasurementScope.ARTIFACT, "Cells"), + name="Cells", subject=MeasurementSubject(MeasurementScope.ARTIFACT, "Cells"), rows=MeasurementSparseColumnarRows.from_rows( - ({field: 1 for field in fields},), - fields=tuple(FieldSpec(field) for field in fields), + ({field: 1 for field in fields},), fields=tuple(FieldSpec(field) for field in fields), ), ) store = RuntimeValueStore() - record = store.record( - RuntimeValue( - key=ArtifactKey( - name="Cells", - artifact_type=MeasurementsArtifactType, - scope=RuntimeExecutionAxisScope(axis_id="A01"), - ), - data=table, - ), - path=f"/runtime/{producer}/Cells", - backend="memory", - ) + record = store.record(RuntimeValue( + key=ArtifactKey(name="Cells", artifact_type=MeasurementsArtifactType, + scope=RuntimeExecutionAxisScope(axis_id="A01")), + data=table, + ), path=f"/runtime/{producer}/Cells", backend="memory") filemanager = FileManager({"memory": MemoryStorageBackend()}) outputs = materialization_outputs( - MaterializationSpec(CsvOptions(filename_suffix=".csv")), - table, - str(tmp_path / f"Cells_{producer}"), - filemanager, + MaterializationSpec(CsvOptions(filename_suffix=".csv")), table, + str(tmp_path / f"Cells_{producer}"), filemanager, ) for output in outputs: # Persist exactly the original writer's content and declared path. Path(output.path).write_text(output.require_text_content(), encoding="utf-8") observation = StepExecutionObservation( - { - RuntimeArtifactAddress.from_record(record): tuple( - RuntimeArtifactLocation(output.path, "disk") for output in outputs - ) - }, + {RuntimeArtifactAddress.from_record(record): tuple(RuntimeArtifactLocation(output.path, "disk") for output in outputs)}, tuple(Path(output.path) for output in outputs), ) return record, observation def test_same_named_field_and_aggregate_tables_keep_exact_producer_ownership(tmp_path): - acquired, acquired_outputs = _saved_table( - tmp_path, producer="step1", fields=("site", "source_image_name") - ) - aggregate, aggregate_outputs = _saved_table( - tmp_path, producer="step3", fields=("area",) - ) - execution = RuntimeExecutionObservation( - ( - RuntimeContextObservation( - "A01", - (acquired, aggregate), - StepExecutionObservation.combine((acquired_outputs, aggregate_outputs)), - ), - ) - ) + acquired, acquired_outputs = _saved_table(tmp_path, producer="step1", fields=("site", "source_image_name")) + aggregate, aggregate_outputs = _saved_table(tmp_path, producer="step3", fields=("area",)) + execution = RuntimeExecutionObservation(( + RuntimeContextObservation("A01", (acquired, aggregate), StepExecutionObservation.combine((acquired_outputs, aggregate_outputs))), + )) transported = pickle.loads(pickle.dumps(execution)) exports = RuntimeExportObservation.from_runtime_observations((transported,)) - spec = ArtifactSpec.output( - name="Cells", - artifact_type=MeasurementsArtifactType, - materialization=MaterializationSpec(CsvOptions(filename_suffix=".csv")), - ) + spec = ArtifactSpec.output(name="Cells", artifact_type=MeasurementsArtifactType, + materialization=MaterializationSpec(CsvOptions(filename_suffix=".csv"))) expectation = RuntimeExportExpectation.from_output_specs((spec,)) - assert ( - exports.matching_table_outputs(acquired) - == acquired_outputs.runtime_export_paths - ) - assert ( - exports.matching_table_outputs(aggregate) - == aggregate_outputs.runtime_export_paths - ) - assert ( - runtime_export_failures(expectation, exports, {"A01": (acquired, aggregate)}) - == () - ) + assert exports.matching_table_outputs(acquired) == acquired_outputs.runtime_export_paths + assert exports.matching_table_outputs(aggregate) == aggregate_outputs.runtime_export_paths + assert runtime_export_failures(expectation, exports, {"A01": (acquired, aggregate)}) == () # Files and a same-named neighbor cannot replace the missing producer relation. - unowned = RuntimeExportObservation.from_output_paths( - exports.output_files, outputs=aggregate_outputs - ) - failures = runtime_export_failures( - expectation, unowned, {"A01": (acquired, aggregate)} - ) + unowned = RuntimeExportObservation.from_output_paths(exports.output_files, outputs=aggregate_outputs) + failures = runtime_export_failures(expectation, unowned, {"A01": (acquired, aggregate)}) assert len(failures) == 1 and "no matching table output" in failures[0] # A correct address does not waive its own required columns. acquired_outputs.runtime_export_paths[0].write_text("area\n1\n", encoding="utf-8") invalid = RuntimeExportObservation.from_runtime_observations((transported,)) - failures = runtime_export_failures( - expectation, invalid, {"A01": (acquired, aggregate)} - ) - assert any( - "site" in failure and "source_image_name" in failure for failure in failures - ) + failures = runtime_export_failures(expectation, invalid, {"A01": (acquired, aggregate)}) + assert any("site" in failure and "source_image_name" in failure for failure in failures) diff --git a/tests/unit/test_artifact_publication_journey.py b/tests/unit/test_artifact_publication_journey.py index 52f4fca2cb..811d66ac05 100644 --- a/tests/unit/test_artifact_publication_journey.py +++ b/tests/unit/test_artifact_publication_journey.py @@ -18,24 +18,16 @@ from openhcs.core.orchestrator.orchestrator import PipelineOrchestrator from openhcs.core.progress import set_progress_queue from openhcs.core.artifacts import ( - ArtifactSpec, - ObjectLabelsArtifactType, - MeasurementsArtifactType, + ArtifactSpec, ObjectLabelsArtifactType, MeasurementsArtifactType, ArtifactMeasurementSubjectRelation, ) from openhcs.core.function_patterns import compile_function_pattern from openhcs.core.aligned_image_payload import AlignedImageSliceContext from openhcs.core.measurement_row_materialization import MeasurementSparseColumnarRows from openhcs.core.runtime_tabular_values import FieldSpec -from openhcs.core.pipeline.function_contracts import ( - artifact_inputs, - artifact_outputs, - object_label_input_execution_mode, -) +from openhcs.core.pipeline.function_contracts import (artifact_inputs, artifact_outputs, object_label_input_execution_mode) from openhcs.core.memory import numpy as numpy_decorator -from openhcs.core.runtime_object_label_building import ( - SourceImageObjectLabelBuildRequest, -) +from openhcs.core.runtime_object_label_building import SourceImageObjectLabelBuildRequest from openhcs.core.runtime_object_labels import ObjectLabelValue from openhcs.core.processing_contracts import ( Pure2DContract, @@ -67,29 +59,22 @@ @numpy_decorator(contract=Pure2DContract) def tiny_objects(image: np.ndarray) -> ObjectLabelValue: return SourceImageObjectLabelBuildRequest( - image=image, - labels=(np.asarray(image) > 0).astype(np.int32), - declared_object_count=1, - declared_object_ids=(1,), + image=image, labels=(np.asarray(image) > 0).astype(np.int32), + declared_object_count=1, declared_object_ids=(1,), ).payload() -@artifact_inputs( - ArtifactSpec.input("objects", ObjectLabelsArtifactType, parameter_name="labels") -) +@artifact_inputs(ArtifactSpec.input("objects", ObjectLabelsArtifactType, parameter_name="labels")) @numpy_decorator(contract=Pure3DContract) @object_label_input_execution_mode(FullStackLabels) def render_labels(image: np.ndarray, labels: ObjectLabelValue) -> np.ndarray: return convert_objects_to_image(image, labels, image_mode=ImageMode.UINT16) -@artifact_outputs( - ArtifactSpec.output( - "counts", - MeasurementsArtifactType, - relations=(ArtifactMeasurementSubjectRelation(),), - ) -) +@artifact_outputs(ArtifactSpec.output( + "counts", MeasurementsArtifactType, + relations=(ArtifactMeasurementSubjectRelation(),), +)) @numpy_decorator(contract=Pure2DContract) def tiny_counts(image: np.ndarray): return image, MeasurementSparseColumnarRows.from_rows( @@ -114,21 +99,10 @@ def _plate(root: Path, site_count: int = 1) -> Path: for site in range(1, site_count + 1): path = root / f"A01_s{site:03d}_w1_z001_t001.tif" tifffile.imwrite(path, pixels) - address = OpenHCSPlaneAddress( - ( - (Microscopy.Well, "A01"), - (Microscopy.Site, site), - (Microscopy.Channel, 1), - (Microscopy.ZIndex, 1), - (Microscopy.Timepoint, 1), - ) - ) + address = OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, site), (Microscopy.Channel, 1), (Microscopy.ZIndex, 1), (Microscopy.Timepoint, 1))) metadata = ImagePayloadMetadata( source_path=str(path), - source_component_metadata={ - **address.as_component_metadata(), - "extension": ".tif", - }, + source_component_metadata={**address.as_component_metadata(), "extension": ".tif"}, source_voxel_spacing=SourceVoxelSpacing((0.65, 0.65)), ) projection = SourcePlaneProjection( @@ -139,9 +113,7 @@ def _plate(root: Path, site_count: int = 1) -> Path: ) projections.append(projection) projection_paths.append((projection, path.name)) - document = SourceProjectionMetadataSerializer( - SourceSchemaFilenameParser() - ).metadata_dict( + document = SourceProjectionMetadataSerializer(SourceSchemaFilenameParser()).metadata_dict( SourceProjectionSet(tuple(projections)), microscope_handler_name="openhcsdata", source_filename_parser_name="SourceSchemaFilenameParser", @@ -150,9 +122,7 @@ def _plate(root: Path, site_count: int = 1) -> Path: main=True, projection_paths=tuple(projection_paths), ) - (root / "openhcs_metadata.json").write_text( - json.dumps({"subdirectories": {".": document}}) - ) + (root / "openhcs_metadata.json").write_text(json.dumps({"subdirectories": {".": document}})) return root @@ -160,9 +130,7 @@ def _compile(root: Path, results: Path, main_filter=0): plate = _plate(root / "source") config = PipelineConfig( materialization_results_path=results, - path_planning_config=LazyPathPlanningConfig( - output_dir_suffix="_out", partition_filter=main_filter - ), + path_planning_config=LazyPathPlanningConfig(output_dir_suffix="_out", partition_filter=main_filter), processing_config=LazyProcessingConfig( group_by=Microscopy.Channel, variable_components=[Microscopy.Site], @@ -177,9 +145,7 @@ def _compile(root: Path, results: Path, main_filter=0): func=render_labels, name="render", step_materialization_config=LazyStepMaterializationConfig( - enabled=True, - output_dir_suffix="_out", - sub_dir="converted", + enabled=True, output_dir_suffix="_out", sub_dir="converted", ), ), FunctionStep(func=tiny_counts, name="counts"), @@ -188,17 +154,13 @@ def _compile(root: Path, results: Path, main_filter=0): def test_public_planning_rejects_external_results_root(tmp_path): - with pytest.raises( - ValueError, match="materialization_results_path.*output plate root" - ): + with pytest.raises(ValueError, match="materialization_results_path.*output plate root"): _compile(tmp_path, tmp_path / "external-results") @pytest.mark.parametrize("absolute", (False, True)) @pytest.mark.parametrize("main_filter", (0, 1)) -def test_converted_checkpoint_has_typed_address_on_final_reconciliation( - tmp_path, absolute, main_filter -): +def test_converted_checkpoint_has_typed_address_on_final_reconciliation(tmp_path, absolute, main_filter): results = tmp_path / "source_out" / "results" if absolute else Path("results") orchestrator, steps, bundle = _compile(tmp_path, results, main_filter) for context in bundle.runtime_contexts.values(): @@ -221,27 +183,11 @@ def test_converted_checkpoint_has_typed_address_on_final_reconciliation( assert entry["image_files"] == [projection["virtual_path"]] # Reopen the durable typed projection. Zero automatic main-flow output must # not invent a default main when independently saved targets coexist. - reopened_metadata = ( - orchestrator.microscope_handler.metadata_handler.load_metadata_document( - saved.parent - ) - ) - reopened = VirtualWorkspaceSourceProjection.from_openhcs_metadata( - saved.parent, reopened_metadata - ) + reopened_metadata = orchestrator.microscope_handler.metadata_handler.load_metadata_document(saved.parent) + reopened = VirtualWorkspaceSourceProjection.from_openhcs_metadata(saved.parent, reopened_metadata) retained = reopened.source_projections_by_virtual_path[projection["virtual_path"]] - assert retained.address == OpenHCSPlaneAddress( - ( - (Microscopy.Well, "A01"), - (Microscopy.Site, 1), - (Microscopy.Channel, 1), - (Microscopy.ZIndex, 1), - (Microscopy.Timepoint, 1), - ) - ) - assert retained.image_metadata.source_voxel_spacing == SourceVoxelSpacing( - (0.65, 0.65) - ) + assert retained.address == OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, 1), (Microscopy.Channel, 1), (Microscopy.ZIndex, 1), (Microscopy.Timepoint, 1))) + assert retained.image_metadata.source_voxel_spacing == SourceVoxelSpacing((0.65, 0.65)) assert retained.image_metadata.source_path == projected_metadata.source_path if main_filter: reopened_plate = PipelineOrchestrator(saved.parent).initialize() @@ -259,9 +205,7 @@ def test_converted_checkpoint_has_typed_address_on_final_reconciliation( def test_planning_rejects_relative_results_escape(tmp_path): - with pytest.raises( - ValueError, match="materialization_results_path.*output plate root" - ): + with pytest.raises(ValueError, match="materialization_results_path.*output plate root"): _compile(tmp_path, Path("../external-results")) @@ -273,9 +217,7 @@ def test_planning_preserves_symlinked_plate_identity(tmp_path): _orchestrator, _steps, bundle = _compile(alias, Path("results")) for context in bundle.runtime_contexts.values(): for plan in context.step_plans.values(): - assert Path(plan.analysis_results_dir).is_relative_to( - plan.output_dataset_root - ) + assert Path(plan.analysis_results_dir).is_relative_to(plan.output_dataset_root) def test_planning_rejects_symlinked_results_escape(tmp_path): @@ -284,9 +226,7 @@ def test_planning_rejects_symlinked_results_escape(tmp_path): output = tmp_path / "source_out" output.mkdir() (output / "results").symlink_to(outside, target_is_directory=True) - with pytest.raises( - ValueError, match="materialization_results_path.*output plate root" - ): + with pytest.raises(ValueError, match="materialization_results_path.*output plate root"): _compile(tmp_path, Path("results")) @@ -294,13 +234,10 @@ def test_new_unnamed_replacement_and_table_passthrough_contracts(): def another_image(image): return image + 1 - @artifact_outputs( - ArtifactSpec.output( - "counts", - MeasurementsArtifactType, - relations=(ArtifactMeasurementSubjectRelation(),), - ) - ) + @artifact_outputs(ArtifactSpec.output( + "counts", MeasurementsArtifactType, + relations=(ArtifactMeasurementSubjectRelation(),), + )) def table_only(image): return image diff --git a/tests/unit/test_automatic_image_role_materialization_435.py b/tests/unit/test_automatic_image_role_materialization_435.py index 7f89a73318..bb666aa7f8 100644 --- a/tests/unit/test_automatic_image_role_materialization_435.py +++ b/tests/unit/test_automatic_image_role_materialization_435.py @@ -11,25 +11,14 @@ from polystore.filemanager import FileManager from polystore.memory import MemoryStorageBackend -from tests.unit.test_function_outputs import ( - context_stub, - function_step_plan, - record_output_path, -) +from tests.unit.test_function_outputs import context_stub, function_step_plan, record_output_path from openhcs.constants.constants import Backend -from openhcs.core.aligned_image_payload import ( - AlignedImageSliceContext, - ImagePayloadBundleContext, -) +from openhcs.core.aligned_image_payload import AlignedImageSliceContext, ImagePayloadBundleContext from openhcs.core.artifacts import ArtifactOutputPlan, ArtifactSpec, ImageArtifactType from openhcs.core.compiled_step_plan import RuntimeArtifactMaterializationPlan from openhcs.core.pipeline.artifact_planning import ArtifactOutputMaterializationPlanner from openhcs.core.runtime_artifact_values import RuntimeValue -from openhcs.core.runtime_image_values import ( - ImageMetadataPayload, - ImagePayloadMetadata, - ImagePayloadMetadataCompositionMode, -) +from openhcs.core.runtime_image_values import ImageMetadataPayload, ImagePayloadMetadata, ImagePayloadMetadataCompositionMode from openhcs.core.runtime_stores import RuntimeValueStore from openhcs.core.source_metadata import SourceVoxelSpacing from openhcs.core.source_projection import SourceProjectionMetadataSerializer @@ -37,9 +26,7 @@ from openhcs.core.steps.function_output_identity import ( FunctionOutputIdentity, ) -from openhcs.core.steps.function_artifact_materialization import ( - ArtifactMaterializationTargetPlan, -) +from openhcs.core.steps.function_artifact_materialization import ArtifactMaterializationTargetPlan from openhcs.core.steps.function_outputs import ( PrimaryImageMetadataTarget, ) @@ -48,15 +35,11 @@ @pytest.mark.parametrize("plane_count", (1, 2)) -def test_automatic_two_role_images_publish_their_actual_saved_occurrences( - tmp_path, plane_count -): - filemanager = FileManager( - { - Backend.DISK.value: DiskStorageBackend(), - Backend.MEMORY.value: MemoryStorageBackend(), - } - ) +def test_automatic_two_role_images_publish_their_actual_saved_occurrences(tmp_path, plane_count): + filemanager = FileManager({ + Backend.DISK.value: DiskStorageBackend(), + Backend.MEMORY.value: MemoryStorageBackend(), + }) context = context_stub(filemanager, parser=SourceSchemaFilenameParser()) context.microscope_handler.source_name = "openhcsdata" context.runtime_value_store = RuntimeValueStore() @@ -73,25 +56,17 @@ def test_automatic_two_role_images_publish_their_actual_saved_occurrences( plan.sub_dir = "images" plan.analysis_results_dir = str(plan.output_dir) plan.runtime_artifact_materialization = RuntimeArtifactMaterializationPlan( - persistent_enabled=True, - persistent_backend=Backend.DISK.value, + persistent_enabled=True, persistent_backend=Backend.DISK.value, ) context.step_plans = {plan.step_index: plan} plans = tuple( ArtifactOutputPlan( - name=name, - path=f"/memory/{name}.pkl", - artifact_type=ImageArtifactType, + name=name, path=f"/memory/{name}.pkl", artifact_type=ImageArtifactType, producer_step_index=plan.step_index, producer_step_scope_id=plan.step_scope_id, producer_step_name=plan.step_name, materialization=ArtifactOutputMaterializationPlanner.materialization_for( - ArtifactSpec( - name=name, - artifact_type=ImageArtifactType, - plan_type=ArtifactOutputPlan, - ), - (), + ArtifactSpec(name=name, artifact_type=ImageArtifactType, plan_type=ArtifactOutputPlan), (), ), ) for name in ("raw_body_processing_role", "capped_outgrowth_processing_role") @@ -105,12 +80,8 @@ def test_automatic_two_role_images_publish_their_actual_saved_occurrences( (np.arange(20, dtype=np.float32).reshape(4, 5) + index) * factor, ImagePayloadMetadata( source_component_metadata={ - "well": "A01", - "site": "1", - "channel": "2", - "z_index": str(index), - "timepoint": "1", - "extension": ".tif", + "well": "A01", "site": "1", "channel": "2", "z_index": str(index), + "timepoint": "1", "extension": ".tif", }, source_image_names=(output.name,), source_voxel_spacing=SourceVoxelSpacing((1.3556, 1.3556)), @@ -119,195 +90,115 @@ def test_automatic_two_role_images_publish_their_actual_saved_occurrences( for index in range(1, plane_count + 1) ) value = ( - payloads[0] - if plane_count == 1 - else ImagePayloadBundleContext.from_payloads( - payloads, - metadata_mode=ImagePayloadMetadataCompositionMode.STACK, + payloads[0] if plane_count == 1 else ImagePayloadBundleContext.from_payloads( + payloads, metadata_mode=ImagePayloadMetadataCompositionMode.STACK, ).compose() ) context.runtime_value_store.record( RuntimeValue.normalize(output, value, axis_id="A01"), - path=output.path, - backend=Backend.MEMORY.value, + path=output.path, backend=Backend.MEMORY.value, ) expected[output.name] = [] for payload in payloads: identity = FunctionOutputIdentity.from_filename_metadata( - context.microscope_handler.parser, - payload.metadata, + context.microscope_handler.parser, payload.metadata, ).with_filename_qualifier(output_context.output_key) - destination = plan.output_dir / identity.filename( - context.microscope_handler.parser - ) + destination = plan.output_dir / identity.filename(context.microscope_handler.parser) filemanager.ensure_directory(plan.output_dir, Backend.MEMORY.value) filemanager.ensure_directory(plan.output_dir, Backend.DISK.value) filemanager.save(payload, str(destination), Backend.MEMORY.value) filemanager.save(payload.data, str(destination), Backend.DISK.value) record_output_path( - context, - plan, - str(destination), - output_context=output_context, - image_metadata=payload.metadata, - identity=identity, + context, plan, str(destination), output_context=output_context, + image_metadata=payload.metadata, identity=identity, ) expected[output.name].append((destination, payload.data)) saved = ArtifactMaterializationTargetPlan.materialize(context, plan) - target = replace( - PrimaryImageMetadataTarget.from_plan(plan), artifact_materializations=saved - ) + target = replace(PrimaryImageMetadataTarget.from_plan(plan), artifact_materializations=saved) # The original implementation raises its real duplicate projection guard # here: both automatic materializations used the same unqualified path. structured = target.produced_projection_entries(context, plan) structured = SourceProjectionMetadataSerializer.projection_fields( structured.projection_paths ) - assert {item["source_alias"] for item in structured["source_projection"]} == set( - expected - ) + assert {item["source_alias"] for item in structured["source_projection"]} == set(expected) assert len(structured["source_projection"]) == 2 * plane_count for materialized in saved: outputs = materialized.outputs_for_backend(Backend.DISK.value) assert len(outputs) == plane_count - for actual, (path, pixels) in zip( - outputs, - expected[materialized.materialization.output_plan.name], - strict=True, - ): + for actual, (path, pixels) in zip(outputs, expected[materialized.materialization.output_plan.name], strict=True): assert actual.path == str(path) np.testing.assert_array_equal(tifffile.imread(path), pixels) target.write(context, produced_plan=plan) document = json.loads((tmp_path / "openhcs_metadata.json").read_text()) - reopened = VirtualWorkspaceSourceProjection.from_openhcs_metadata( - tmp_path, document - ) + reopened = VirtualWorkspaceSourceProjection.from_openhcs_metadata(tmp_path, document) for name, expected_planes in expected.items(): for path, pixels in expected_planes: - projection = reopened.source_projections_by_virtual_path[ - str(path.relative_to(tmp_path)) - ] + projection = reopened.source_projections_by_virtual_path[str(path.relative_to(tmp_path))] assert projection.source_alias == name assert projection.address.as_component_metadata()["channel"] == "2" - assert projection.image_metadata.source_voxel_spacing == SourceVoxelSpacing( - (1.3556, 1.3556) - ) - np.testing.assert_array_equal( - tifffile.imread(tmp_path / projection.ref.backend_address), pixels - ) + assert projection.image_metadata.source_voxel_spacing == SourceVoxelSpacing((1.3556, 1.3556)) + np.testing.assert_array_equal(tifffile.imread(tmp_path / projection.ref.backend_address), pixels) -@pytest.mark.parametrize( - "purpose", - ( - "explicit_source", - "explicit_template", - "terminal_template", - "explicit_artifact", - "terminal_artifact", - "unnamed_terminal", - "unnamed_explicit", - "explicit_source_declared_terminal", - "terminal_source_declared_explicit", - ), -) -def test_retained_role_policy_preserves_authored_and_unnamed_image_paths( - tmp_path, purpose -): +@pytest.mark.parametrize("purpose", ( + "explicit_source", "explicit_template", "terminal_template", + "explicit_artifact", "terminal_artifact", "unnamed_terminal", "unnamed_explicit", + "explicit_source_declared_terminal", "terminal_source_declared_explicit", +)) +def test_retained_role_policy_preserves_authored_and_unnamed_image_paths(tmp_path, purpose): from openhcs.processing.materialization import ( - ImageFileOptions, - MaterializationSpec, - MaterializedFilenameIdentity, - TerminalMaterializationSpec, - materialization_outputs, + ImageFileOptions, MaterializationSpec, MaterializedFilenameIdentity, + TerminalMaterializationSpec, materialization_outputs, ) options = ImageFileOptions( filename_suffix=".tif", - relative_path_template=( - "authored/plane_{index}.tif" if purpose.endswith("template") else None - ), + relative_path_template=("authored/plane_{index}.tif" if purpose.endswith("template") else None), filename_identity=( - MaterializedFilenameIdentity.ARTIFACT_NAME - if purpose.endswith("artifact") + MaterializedFilenameIdentity.ARTIFACT_NAME if purpose.endswith("artifact") else MaterializedFilenameIdentity.SOURCE_IDENTITY ), ) spec = ( - TerminalMaterializationSpec(options) - if purpose.startswith("terminal") or purpose == "unnamed_terminal" + TerminalMaterializationSpec(options) if purpose.startswith("terminal") or purpose == "unnamed_terminal" else MaterializationSpec(options) ) declared_spec = ( - TerminalMaterializationSpec(options) - if purpose == "explicit_source_declared_terminal" - else ( - MaterializationSpec(options) - if purpose == "terminal_source_declared_explicit" - else spec - ) + TerminalMaterializationSpec(options) if purpose == "explicit_source_declared_terminal" + else MaterializationSpec(options) if purpose == "terminal_source_declared_explicit" + else spec ) - output_plan = ( - None - if purpose in ("unnamed_terminal", "unnamed_explicit") - else ArtifactOutputPlan( - name="DeclaredRole", - path="/memory/DeclaredRole.pkl", - artifact_type=ImageArtifactType, - materialization=declared_spec, - ) + output_plan = None if purpose in ("unnamed_terminal", "unnamed_explicit") else ArtifactOutputPlan( + name="DeclaredRole", path="/memory/DeclaredRole.pkl", + artifact_type=ImageArtifactType, materialization=declared_spec, ) payloads = tuple( - ImageMetadataPayload( - np.ones((4, 5), dtype=np.float32) * index, - ImagePayloadMetadata( - source_component_metadata={ - "well": "A01", - "site": "1", - "channel": "2", - "z_index": str(index), - "timepoint": "1", - "extension": ".tif", - }, - ), - ) + ImageMetadataPayload(np.ones((4, 5), dtype=np.float32) * index, ImagePayloadMetadata( + source_component_metadata={ + "well": "A01", "site": "1", "channel": "2", "z_index": str(index), + "timepoint": "1", "extension": ".tif", + }, + )) for index in (1, 2) ) value = ImagePayloadBundleContext.from_payloads( - payloads, - metadata_mode=ImagePayloadMetadataCompositionMode.STACK, + payloads, metadata_mode=ImagePayloadMetadataCompositionMode.STACK, ).compose() filemanager = FileManager({Backend.MEMORY.value: MemoryStorageBackend()}) context = context_stub(filemanager, parser=SourceSchemaFilenameParser()) outputs = materialization_outputs( - spec, - value, - str(tmp_path / "AuthoredBase"), - filemanager, - context=context, - output_plan=output_plan, + spec, value, str(tmp_path / "AuthoredBase"), filemanager, + context=context, output_plan=output_plan, variable_components=(Microscopy.ZIndex,), ) expected_paths = ( - ("authored/plane_1.tif", "authored/plane_2.tif") - if purpose.endswith("template") - else ( - ("AuthoredBase.tif",) - if purpose.endswith("artifact") - else ( - ( - "A01_s001_w2_z001_t001_DeclaredRole.tif", - "A01_s001_w2_z002_t001_DeclaredRole.tif", - ) - if purpose == "terminal_source_declared_explicit" - else ("A01_s001_w2_z001_t001.tif", "A01_s001_w2_z002_t001.tif") - ) - ) - ) - assert ( - tuple(Path(output.path).relative_to(tmp_path).as_posix() for output in outputs) - == expected_paths + ("authored/plane_1.tif", "authored/plane_2.tif") if purpose.endswith("template") + else ("AuthoredBase.tif",) if purpose.endswith("artifact") + else ("A01_s001_w2_z001_t001_DeclaredRole.tif", "A01_s001_w2_z002_t001_DeclaredRole.tif") if purpose == "terminal_source_declared_explicit" + else ("A01_s001_w2_z001_t001.tif", "A01_s001_w2_z002_t001.tif") ) + assert tuple(Path(output.path).relative_to(tmp_path).as_posix() for output in outputs) == expected_paths if len(outputs) == 2: for actual, expected in zip(outputs, payloads, strict=True): np.testing.assert_array_equal(actual.content, expected.data) @@ -316,130 +207,60 @@ def test_retained_role_policy_preserves_authored_and_unnamed_image_paths( def test_authored_duplicate_template_still_fails_before_any_save(tmp_path): - from openhcs.processing.materialization import ( - ImageFileOptions, - TerminalMaterializationSpec, - materialization_outputs, - ) + from openhcs.processing.materialization import ImageFileOptions, TerminalMaterializationSpec, materialization_outputs - spec = TerminalMaterializationSpec( - ImageFileOptions( - filename_suffix=".tif", - relative_path_template="same.tif", - ) - ) - output_plan = ArtifactOutputPlan( - name="NamedRole", - path="/memory/role.pkl", - artifact_type=ImageArtifactType, - materialization=spec, - ) + spec = TerminalMaterializationSpec(ImageFileOptions( + filename_suffix=".tif", relative_path_template="same.tif", + )) + output_plan = ArtifactOutputPlan(name="NamedRole", path="/memory/role.pkl", artifact_type=ImageArtifactType, materialization=spec) payloads = tuple( - ImageMetadataPayload( - np.ones((4, 5), dtype=np.float32), - ImagePayloadMetadata( - source_component_metadata={ - "well": "A01", - "site": "1", - "channel": "2", - "z_index": str(index), - "timepoint": "1", - "extension": ".tif", - }, - ), - ) - for index in (1, 2) + ImageMetadataPayload(np.ones((4, 5), dtype=np.float32), ImagePayloadMetadata( + source_component_metadata={"well": "A01", "site": "1", "channel": "2", "z_index": str(index), "timepoint": "1", "extension": ".tif"}, + )) for index in (1, 2) ) - value = ImagePayloadBundleContext.from_payloads( - payloads, metadata_mode=ImagePayloadMetadataCompositionMode.STACK - ).compose() + value = ImagePayloadBundleContext.from_payloads(payloads, metadata_mode=ImagePayloadMetadataCompositionMode.STACK).compose() manager = FileManager({Backend.MEMORY.value: MemoryStorageBackend()}) with pytest.raises(ValueError, match="produced duplicate paths"): - materialization_outputs( - spec, - value, - str(tmp_path / "role"), - manager, - context=context_stub(manager, parser=SourceSchemaFilenameParser()), - output_plan=output_plan, - ) + materialization_outputs(spec, value, str(tmp_path / "role"), manager, + context=context_stub(manager, parser=SourceSchemaFilenameParser()), + output_plan=output_plan) assert not (tmp_path / "same.tif").exists() @pytest.mark.parametrize("suffix", (".tif", ".png", ".ome.tif", ".labels.tif")) @pytest.mark.parametrize("plane_count", (1, 2)) -def test_retained_image_qualifier_respects_complete_writer_suffix( - tmp_path, suffix, plane_count -): - from openhcs.core.steps.function_artifact_materialization import ( - RuntimeArtifactMaterialization, - ) - from openhcs.processing.materialization import ( - ImageFileOptions, - TerminalMaterializationSpec, - ) +def test_retained_image_qualifier_respects_complete_writer_suffix(tmp_path, suffix, plane_count): + from openhcs.core.steps.function_artifact_materialization import RuntimeArtifactMaterialization + from openhcs.processing.materialization import ImageFileOptions, TerminalMaterializationSpec manager = FileManager({Backend.MEMORY.value: MemoryStorageBackend()}) context = context_stub(manager, parser=SourceSchemaFilenameParser()) - plan = function_step_plan( - "Retained format role", - variable_components=(Microscopy.ZIndex,) if plane_count > 1 else (), - ) + plan = function_step_plan("Retained format role", variable_components=(Microscopy.ZIndex,) if plane_count > 1 else ()) plan.output_dir = tmp_path / "images" plan.analysis_results_dir = str(plan.output_dir) output_plan = ArtifactOutputPlan( - name="Role Name", - path="/memory/role.pkl", - artifact_type=ImageArtifactType, - materialization=TerminalMaterializationSpec( - ImageFileOptions(filename_suffix=suffix) - ), + name="Role Name", path="/memory/role.pkl", artifact_type=ImageArtifactType, + materialization=TerminalMaterializationSpec(ImageFileOptions(filename_suffix=suffix)), ) payloads = tuple( - ImageMetadataPayload( - np.ones((4, 5), dtype=np.uint8) * index, - ImagePayloadMetadata( - source_component_metadata={ - "well": "A01", - "site": "1", - "channel": "2", - "z_index": str(index), - "timepoint": "1", - "extension": ".tif", - }, - ), - ) - for index in range(1, plane_count + 1) - ) - value = ( - payloads[0] - if plane_count == 1 - else ImagePayloadBundleContext.from_payloads( - payloads, - metadata_mode=ImagePayloadMetadataCompositionMode.STACK, - ).compose() + ImageMetadataPayload(np.ones((4, 5), dtype=np.uint8) * index, ImagePayloadMetadata( + source_component_metadata={"well": "A01", "site": "1", "channel": "2", "z_index": str(index), "timepoint": "1", "extension": ".tif"}, + )) for index in range(1, plane_count + 1) ) + value = payloads[0] if plane_count == 1 else ImagePayloadBundleContext.from_payloads( + payloads, metadata_mode=ImagePayloadMetadataCompositionMode.STACK, + ).compose() store = RuntimeValueStore() - record = store.record( - RuntimeValue.normalize(output_plan, value, axis_id="A01"), - path=output_plan.path, - backend=Backend.MEMORY.value, - ) + record = store.record(RuntimeValue.normalize(output_plan, value, axis_id="A01"), path=output_plan.path, backend=Backend.MEMORY.value) outputs = RuntimeArtifactMaterialization.from_record( - output_plan=output_plan, - record=record, - plan=plan, - context=context, + output_plan=output_plan, record=record, plan=plan, context=context, ).outputs(plan, context) assert len(outputs) == plane_count for index, (output, payload) in enumerate(zip(outputs, payloads, strict=True), 1): identity = FunctionOutputIdentity.from_filename_metadata( - context.microscope_handler.parser, - payload.metadata, + context.microscope_handler.parser, payload.metadata, ).with_filename_qualifier(output_plan.name) - expected = replace(identity, extension=suffix).filename( - context.microscope_handler.parser - ) + expected = replace(identity, extension=suffix).filename(context.microscope_handler.parser) assert Path(output.path).name == expected assert Path(output.path).name.endswith("_Role_Name" + suffix) assert output.metadata.source_component_metadata["channel"] == "2" @@ -448,121 +269,60 @@ def test_retained_image_qualifier_respects_complete_writer_suffix( def test_distinct_aliases_that_normalize_to_one_filename_still_conflict(tmp_path): - from openhcs.core.steps.function_artifact_materialization import ( - RuntimeArtifactMaterialization, - ) - from openhcs.processing.materialization import ( - ImageFileOptions, - TerminalMaterializationSpec, - ) + from openhcs.core.steps.function_artifact_materialization import RuntimeArtifactMaterialization + from openhcs.processing.materialization import ImageFileOptions, TerminalMaterializationSpec manager = FileManager({Backend.MEMORY.value: MemoryStorageBackend()}) context = context_stub(manager, parser=SourceSchemaFilenameParser()) plan = function_step_plan("Ambiguous role spelling") plan.output_dir = tmp_path / "images" plan.analysis_results_dir = str(plan.output_dir) - value = ImageMetadataPayload( - np.ones((4, 5), dtype=np.float32), - ImagePayloadMetadata( - source_component_metadata={ - "well": "A01", - "site": "1", - "channel": "2", - "z_index": "1", - "timepoint": "1", - "extension": ".tif", - }, - ), - ) + value = ImageMetadataPayload(np.ones((4, 5), dtype=np.float32), ImagePayloadMetadata( + source_component_metadata={"well": "A01", "site": "1", "channel": "2", "z_index": "1", "timepoint": "1", "extension": ".tif"}, + )) paths = [] store = RuntimeValueStore() for name in ("a/b", "a b"): - output_plan = ArtifactOutputPlan( - name=name, - path=f"/memory/{name}.pkl", - artifact_type=ImageArtifactType, - materialization=TerminalMaterializationSpec( - ImageFileOptions(filename_suffix=".tif") - ), - ) - record = store.record( - RuntimeValue.normalize(output_plan, value, axis_id="A01"), - path=output_plan.path, - backend=Backend.MEMORY.value, - ) - outputs = RuntimeArtifactMaterialization.from_record( - output_plan=output_plan, record=record, plan=plan, context=context - ).outputs(plan, context) + output_plan = ArtifactOutputPlan(name=name, path=f"/memory/{name}.pkl", artifact_type=ImageArtifactType, + materialization=TerminalMaterializationSpec(ImageFileOptions(filename_suffix=".tif"))) + record = store.record(RuntimeValue.normalize(output_plan, value, axis_id="A01"), path=output_plan.path, backend=Backend.MEMORY.value) + outputs = RuntimeArtifactMaterialization.from_record(output_plan=output_plan, record=record, plan=plan, context=context).outputs(plan, context) paths.append(outputs[0].path) assert paths[0] == paths[1] # Existing main-flow destination authority rejects this declared naming # collision instead of choosing a new suffix or inventing a channel. from openhcs.core.steps.function_output_identity import FunctionOutputIdentity - with pytest.raises(ValueError, match="duplicate"): - FunctionOutputIdentity.validate_output_paths( - paths, - input_paths=[], - step_name=plan.step_name, - pattern_repr="declared roles", - ) + FunctionOutputIdentity.validate_output_paths(paths, input_paths=[], step_name=plan.step_name, pattern_repr="declared roles") -def test_batch_binds_actual_purpose_once_without_rewriting_plan_or_source( - monkeypatch, tmp_path -): - from openhcs.processing.materialization import ( - ImageFileOptions, - MaterializationSpec, - TerminalMaterializationSpec, - ) - from openhcs.processing.materialization.core import ( - MaterializationBatch, - MaterializationContext, - Output, - WriterSpec, - _WRITERS_BY_OPTIONS, - ) +def test_batch_binds_actual_purpose_once_without_rewriting_plan_or_source(monkeypatch, tmp_path): + from openhcs.processing.materialization import ImageFileOptions, MaterializationSpec, TerminalMaterializationSpec + from openhcs.processing.materialization.core import MaterializationBatch, MaterializationContext, Output, WriterSpec, _WRITERS_BY_OPTIONS options = ImageFileOptions(filename_suffix=".tif") declared = TerminalMaterializationSpec(options) explicit = MaterializationSpec(options) - plan = ArtifactOutputPlan( - name="Role", - path="/memory/role", - artifact_type=ImageArtifactType, - materialization=declared, - ) + plan = ArtifactOutputPlan(name="Role", path="/memory/role", artifact_type=ImageArtifactType, materialization=declared) manager = FileManager({Backend.MEMORY.value: MemoryStorageBackend()}) source = {"physical_source": "original"} context = MaterializationContext( - base_path=str(tmp_path / "role"), - backends=[], - backend_kwargs={}, - filemanager=manager, - extra_inputs=source, - output_plan=plan, + base_path=str(tmp_path / "role"), backends=[], backend_kwargs={}, + filemanager=manager, extra_inputs=source, output_plan=plan, materialization_spec=declared, ) seen = [] - def custom_writer(data, actual_options, actual_context): assert actual_options is options assert actual_context.output_plan is plan assert actual_context.extra_inputs is source seen.append(actual_context) return [Output(path=str(tmp_path / "custom.tif"), content=data)] - - monkeypatch.setitem( - _WRITERS_BY_OPTIONS, - ImageFileOptions, - WriterSpec( - options_type=ImageFileOptions, - write=custom_writer, - primary_path=lambda outputs: outputs[0].path, - candidate_paths=lambda _options, base: (base,), - ), - ) + monkeypatch.setitem(_WRITERS_BY_OPTIONS, ImageFileOptions, WriterSpec( + options_type=ImageFileOptions, + write=custom_writer, primary_path=lambda outputs: outputs[0].path, + candidate_paths=lambda _options, base: (base,), + )) pixels = np.ones((4, 5), dtype=np.float32) first = MaterializationBatch.render(explicit, pixels, context) second = MaterializationBatch.render(declared, pixels, first.context) @@ -575,48 +335,20 @@ def custom_writer(data, actual_options, actual_context): assert source == {"physical_source": "original"} -def test_direct_image_writer_without_purpose_binding_retains_legacy_source_path( - tmp_path, -): - from openhcs.processing.materialization import ( - ImageFileOptions, - TerminalMaterializationSpec, - ) - from openhcs.processing.materialization.core import ( - MaterializationContext, - write_image_file, - ) +def test_direct_image_writer_without_purpose_binding_retains_legacy_source_path(tmp_path): + from openhcs.processing.materialization import ImageFileOptions, TerminalMaterializationSpec + from openhcs.processing.materialization.core import MaterializationContext, write_image_file options = ImageFileOptions(filename_suffix=".tif") - plan = ArtifactOutputPlan( - name="Role", - path="/memory/role", - artifact_type=ImageArtifactType, - materialization=TerminalMaterializationSpec(options), - ) + plan = ArtifactOutputPlan(name="Role", path="/memory/role", artifact_type=ImageArtifactType, + materialization=TerminalMaterializationSpec(options)) manager = FileManager({Backend.MEMORY.value: MemoryStorageBackend()}) - value = ImageMetadataPayload( - np.ones((4, 5), dtype=np.float32), - ImagePayloadMetadata( - source_component_metadata={ - "well": "A01", - "site": "1", - "channel": "2", - "z_index": "1", - "timepoint": "1", - "extension": ".tif", - }, - ), - ) - context = MaterializationContext( - base_path=str(tmp_path / "role"), - backends=[], - backend_kwargs={}, - filemanager=manager, - extra_inputs={}, - output_plan=plan, - context=context_stub(manager, parser=SourceSchemaFilenameParser()), - ) + value = ImageMetadataPayload(np.ones((4, 5), dtype=np.float32), ImagePayloadMetadata( + source_component_metadata={"well": "A01", "site": "1", "channel": "2", "z_index": "1", "timepoint": "1", "extension": ".tif"}, + )) + context = MaterializationContext(base_path=str(tmp_path / "role"), backends=[], backend_kwargs={}, + filemanager=manager, extra_inputs={}, output_plan=plan, + context=context_stub(manager, parser=SourceSchemaFilenameParser())) outputs = write_image_file(value, options, context) assert Path(outputs[0].path).name == "A01_s001_w2_z001_t001.tif" assert context.materialization_spec is None @@ -625,50 +357,23 @@ def test_direct_image_writer_without_purpose_binding_retains_legacy_source_path( @pytest.mark.parametrize("purpose", ("named_terminal", "explicit", "unnamed_terminal")) -def test_parser_free_render_requires_parser_only_for_automatic_named_role( - tmp_path, purpose -): - from openhcs.processing.materialization import ( - ImageFileOptions, - MaterializationSpec, - TerminalMaterializationSpec, - materialization_outputs, - ) +def test_parser_free_render_requires_parser_only_for_automatic_named_role(tmp_path, purpose): + from openhcs.processing.materialization import ImageFileOptions, MaterializationSpec, TerminalMaterializationSpec, materialization_outputs options = ImageFileOptions(filename_suffix=".tif") - spec = ( - MaterializationSpec(options) - if purpose == "explicit" - else TerminalMaterializationSpec(options) - ) - plan = ( - None - if purpose == "unnamed_terminal" - else ArtifactOutputPlan( - name="Role", - path="/memory/role", - artifact_type=ImageArtifactType, - materialization=spec, - ) + spec = MaterializationSpec(options) if purpose == "explicit" else TerminalMaterializationSpec(options) + plan = None if purpose == "unnamed_terminal" else ArtifactOutputPlan( + name="Role", path="/memory/role", artifact_type=ImageArtifactType, materialization=spec, ) manager = FileManager({Backend.MEMORY.value: MemoryStorageBackend()}) - value = ImageMetadataPayload( - np.ones((4, 5), dtype=np.float32), - ImagePayloadMetadata( - source_path="/physical/OriginalSource.tif", - ), - ) + value = ImageMetadataPayload(np.ones((4, 5), dtype=np.float32), ImagePayloadMetadata( + source_path="/physical/OriginalSource.tif", + )) if purpose == "named_terminal": - with pytest.raises( - ValueError, match="Parser-backed source-stem authority requires a parser" - ): - materialization_outputs( - spec, value, str(tmp_path / "Role"), manager, output_plan=plan - ) + with pytest.raises(ValueError, match="Parser-backed source-stem authority requires a parser"): + materialization_outputs(spec, value, str(tmp_path / "Role"), manager, output_plan=plan) else: - outputs = materialization_outputs( - spec, value, str(tmp_path / "Role"), manager, output_plan=plan - ) + outputs = materialization_outputs(spec, value, str(tmp_path / "Role"), manager, output_plan=plan) assert Path(outputs[0].path).name == "OriginalSource.tif" np.testing.assert_array_equal(outputs[0].content, value.data) assert value.metadata.source_path == "/physical/OriginalSource.tif" diff --git a/tests/unit/test_axis_family_guards.py b/tests/unit/test_axis_family_guards.py index de6f22ce44..0ca8f4efd1 100644 --- a/tests/unit/test_axis_family_guards.py +++ b/tests/unit/test_axis_family_guards.py @@ -52,9 +52,7 @@ def _python_files(*roots: str) -> list[Path]: for root in roots: base = REPO_ROOT / root files.extend( - path - for path in base.rglob("*.py") - if "benchmark/results" not in path.as_posix() + path for path in base.rglob("*.py") if "benchmark/results" not in path.as_posix() ) return files @@ -79,9 +77,7 @@ def _identifiers(tree: ast.AST) -> set[str]: def test_deleted_component_enums_and_factory_do_not_exist() -> None: offenders = { - _relative(path): sorted( - DELETED_NAMES & _identifiers(ast.parse(path.read_text())) - ) + _relative(path): sorted(DELETED_NAMES & _identifiers(ast.parse(path.read_text()))) for path in _python_files("openhcs", "tests", "benchmark", "scripts") if path.name != Path(__file__).name } @@ -94,10 +90,7 @@ def test_kernel_modules_name_no_microscopy_member() -> None: offenders: dict[str, list[str]] = {} for path in _python_files("openhcs"): relative = _relative(path) - if ( - relative.startswith(DOMAIN_MODULE_PREFIXES) - or relative == DOMAIN_ENTRY_POINT - ): + if relative.startswith(DOMAIN_MODULE_PREFIXES) or relative == DOMAIN_ENTRY_POINT: continue tree = ast.parse(path.read_text()) found: set[str] = set() @@ -153,17 +146,13 @@ def _domain_member_spellings() -> frozenset[str]: ) -def _member_spellings_in( - tree: ast.AST, spellings: frozenset[str] -) -> set[tuple[str, str]]: +def _member_spellings_in(tree: ast.AST, spellings: frozenset[str]) -> set[tuple[str, str]]: """(enclosing class qualname, spelling) for literals, attributes and class fields.""" docstrings = { id(node.body[0].value) for node in ast.walk(tree) - if isinstance( - node, (ast.Module, ast.ClassDef, ast.FunctionDef, ast.AsyncFunctionDef) - ) + if isinstance(node, (ast.Module, ast.ClassDef, ast.FunctionDef, ast.AsyncFunctionDef)) and node.body and isinstance(node.body[0], ast.Expr) and isinstance(node.body[0].value, ast.Constant) @@ -179,11 +168,9 @@ def visit(node: ast.AST, scope: str) -> None: targets = ( [statement.target] if isinstance(statement, ast.AnnAssign) - else ( - statement.targets - if isinstance(statement, ast.Assign) - else [] - ) + else statement.targets + if isinstance(statement, ast.Assign) + else [] ) found.update( (child_scope, f"field {target.id}") @@ -217,9 +204,7 @@ def test_kernel_modules_spell_no_microscopy_member() -> None: continue found.update( (relative, scope, spelling) - for scope, spelling in _member_spellings_in( - ast.parse(path.read_text()), spellings - ) + for scope, spelling in _member_spellings_in(ast.parse(path.read_text()), spellings) ) assert found == EXTERNAL_SPELLINGS @@ -342,10 +327,7 @@ def test_runtime_foreign_names_name_existing_modules() -> None: ("openhcs/core/config.py", "PlateOutputDirectory"), ("openhcs/core/config.py", "PlateOutputFile"), ("openhcs/core/dataset_sources/openhcs_format.py", "PLATE_NAME"), - ( - "openhcs/core/dataset_sources/openhcs_format.py", - "get_plate_acquisition_datetime", - ), + ("openhcs/core/dataset_sources/openhcs_format.py", "get_plate_acquisition_datetime"), ("openhcs/core/dataset_sources/openhcs_format.py", "get_plate_name"), ("openhcs/core/dataset_sources/source.py", "plate_root"), ("openhcs/core/debug_session_projection.py", "NO_PLATE"), @@ -392,23 +374,11 @@ def test_runtime_foreign_names_name_existing_modules() -> None: ("openhcs/core/source_tile_geometry.py", "sites"), ("openhcs/core/steps/stream_component_semantics.py", "plate_path"), ("openhcs/core/streaming_config_factory.py", "plate_path"), - ( - "openhcs/core/synthetic_plate_generation.py", - "SYNTHETIC_PLATE_GENERATION_PROFILE", - ), + ("openhcs/core/synthetic_plate_generation.py", "SYNTHETIC_PLATE_GENERATION_PROFILE"), ("openhcs/core/synthetic_plate_generation.py", "SyntheticPlateFormat"), - ( - "openhcs/core/synthetic_plate_generation.py", - "SyntheticPlateGenerationBounds", - ), - ( - "openhcs/core/synthetic_plate_generation.py", - "SyntheticPlateGenerationParameters", - ), - ( - "openhcs/core/synthetic_plate_generation.py", - "SyntheticPlateGenerationProfile", - ), + ("openhcs/core/synthetic_plate_generation.py", "SyntheticPlateGenerationBounds"), + ("openhcs/core/synthetic_plate_generation.py", "SyntheticPlateGenerationParameters"), + ("openhcs/core/synthetic_plate_generation.py", "SyntheticPlateGenerationProfile"), ("openhcs/core/vfs_protocol.py", "PlateInputDirectory"), ("openhcs/core/vfs_protocol.py", "PlateInputDirectoryDeclaration"), ("openhcs/core/vfs_protocol.py", "PlateInputFile"), @@ -458,9 +428,7 @@ def test_kernel_domain_references_only_shrink() -> None: for name in _names_and_identifier_literals(ast.parse(path.read_text())) if _identifier_words(name) & domain_words } - assert ( - sorted(found - KERNEL_DOMAIN_REFERENCES) == [] - ), "new domain word in the kernel" + assert sorted(found - KERNEL_DOMAIN_REFERENCES) == [], "new domain word in the kernel" assert sorted(KERNEL_DOMAIN_REFERENCES - found) == [], "removed: shrink the pin" assert len(KERNEL_DOMAIN_REFERENCES) <= KERNEL_DOMAIN_REFERENCE_CEILING @@ -499,3 +467,4 @@ def test_renamed_dataset_names_do_not_exist() -> None: if path.name != Path(__file__).name } assert {path: names for path, names in offenders.items() if names} == {} + diff --git a/tests/unit/test_axis_family_witness.py b/tests/unit/test_axis_family_witness.py index f09956c9d7..ad13de24d3 100644 --- a/tests/unit/test_axis_family_witness.py +++ b/tests/unit/test_axis_family_witness.py @@ -66,10 +66,7 @@ def test_child_processes_serve_the_parent_family(remote_sensing) -> None: capture_output=True, text=True, timeout=300, - env={ - **os.environ, - "PYTHONPATH": os.pathsep.join(map(str, (import_root, repo_root))), - }, + env={**os.environ, "PYTHONPATH": os.pathsep.join(map(str, (import_root, repo_root)))}, check=True, ) assert child.stdout.strip().splitlines()[-1] == RemoteSensing.family_name @@ -150,9 +147,9 @@ def test_viewer_decodes_a_stream_by_its_own_declared_axes(remote_sensing) -> Non assert fiji.components_in(FijiSlots.HyperstackChannel) == ("band",) assert fiji.components_in(FijiSlots.HyperstackSlice) == () assert fiji.components_in(FijiSlots.HyperstackFrame) == ("tile", "date", "scene") - assert FijiDisplaySettings.from_display_payload( - fiji_payload - ) == FijiDisplaySettings(lut="Fire") + assert FijiDisplaySettings.from_display_payload(fiji_payload) == FijiDisplaySettings( + lut="Fire" + ) napari = ViewerMappingDisplayConfigInput(napari_payload).layout() assert napari.components_in(NapariSlots.Stack) == ("tile", "band", "date", "scene") @@ -242,12 +239,8 @@ def test_step_validation_and_compiler_axis_resolution(remote_sensing) -> None: ) validator = GenericValidator(AxisFamily.active()) - assert validator.validate_step( - [RemoteSensing.Tile], RemoteSensing.Band, True, "s" - ).is_valid - rejected = validator.validate_step( - [RemoteSensing.Band], RemoteSensing.Band, True, "s" - ) + assert validator.validate_step([RemoteSensing.Tile], RemoteSensing.Band, True, "s").is_valid + rejected = validator.validate_step([RemoteSensing.Band], RemoteSensing.Band, True, "s") assert not rejected.is_valid partition = validator.validate_step([RemoteSensing.Scene], Ungrouped, False, "s") assert "multiprocessing axis: scene" in partition.error_message @@ -341,10 +334,7 @@ def test_pipeline_runs_end_to_end_through_the_kernel( timeout=900, # The kernel this test imports, then the checkout for the witness domain; # processes the kernel spawns inherit both. - env={ - **os.environ, - "PYTHONPATH": os.pathsep.join(map(str, (import_root, repo_root))), - }, + env={**os.environ, "PYTHONPATH": os.pathsep.join(map(str, (import_root, repo_root)))}, check=False, ) assert completed.returncode == 0, completed.stderr[-4000:] diff --git a/tests/unit/test_benchmark_control.py b/tests/unit/test_benchmark_control.py index 38d8973460..5a2587dba2 100644 --- a/tests/unit/test_benchmark_control.py +++ b/tests/unit/test_benchmark_control.py @@ -63,10 +63,7 @@ InstalledDistributionVersion, RuntimeEnvironmentSnapshot, ) -from openhcs.runtime.zmq_execution_client import ( - OpenHCSExecutionSubmission, - ZMQExecutionRequestBuilder, -) +from openhcs.runtime.zmq_execution_client import OpenHCSExecutionSubmission, ZMQExecutionRequestBuilder from openhcs.runtime.zmq_execution_signature import ZMQAuxiliaryExecutionParams diff --git a/tests/unit/test_bioformats_microscope_handler.py b/tests/unit/test_bioformats_microscope_handler.py index 1d5d6b2859..fdc40481c8 100644 --- a/tests/unit/test_bioformats_microscope_handler.py +++ b/tests/unit/test_bioformats_microscope_handler.py @@ -217,9 +217,7 @@ def test_openhcs_replay_registers_declared_source_handler_backends( reopened_filemanager = bioformats_filemanager() assert Backend.BIOFORMATS.value not in reopened_filemanager.registry - handler = DatasetSourceChoice.named("auto").open( - tmp_path, filemanager=reopened_filemanager - ) + handler = DatasetSourceChoice.named("auto").open(tmp_path, filemanager=reopened_filemanager) assert handler.dataset_root == tmp_path handler.initialize_workspace(tmp_path, reopened_filemanager) @@ -306,9 +304,7 @@ def test_openhcs_output_subdirectory_initializes_from_metadata_root( def test_create_microscope_handler_supports_explicit_bioformats(tmp_path: Path) -> None: - handler = DatasetSourceChoice.named("bioformats").open( - tmp_path, filemanager=bioformats_filemanager() - ) + handler = DatasetSourceChoice.named("bioformats").open(tmp_path, filemanager=bioformats_filemanager()) assert isinstance(handler, BioFormatsHandler) diff --git a/tests/unit/test_calibrated_gaussian_spatial_axes.py b/tests/unit/test_calibrated_gaussian_spatial_axes.py index 09c4bb99b8..ba152e6163 100644 --- a/tests/unit/test_calibrated_gaussian_spatial_axes.py +++ b/tests/unit/test_calibrated_gaussian_spatial_axes.py @@ -13,14 +13,9 @@ ) from openhcs.core.runtime_plane_projection import RuntimePlaneAxis from openhcs.core.source_metadata import SourceVoxelSpacing -from openhcs.core.source_spatial_domain import ( - SourceSpatialDomain, - VolumeSourceSpatialDomain, -) +from openhcs.core.source_spatial_domain import SourceSpatialDomain, VolumeSourceSpatialDomain from openhcs.interop.cellprofiler.runtime.adapter import CellProfilerRuntimeAdapter -from openhcs.interop.cellprofiler.runtime.function_contract_execution import ( - CellProfilerFunctionContractExecutor, -) +from openhcs.interop.cellprofiler.runtime.function_contract_execution import CellProfilerFunctionContractExecutor from openhcs.processing.backends.cellprofiler.gaussian_filter import gaussian_filter from openhcs.core.payload_axes import PayloadAxes from openhcs.core.axes import ColourAxis @@ -31,25 +26,16 @@ def _execute(source): contract = CallableContract.from_callable(gaussian_filter).with_prepared_signature() - contract = replace( - contract, - metadata=replace( - contract.metadata, - runtime_adapter=CellProfilerRuntimeAdapter.runtime_adapter_spec(), - ), - ) + contract = replace(contract, metadata=replace( + contract.metadata, runtime_adapter=CellProfilerRuntimeAdapter.runtime_adapter_spec(), + )) return CellProfilerFunctionContractExecutor().execute( - contract, - contract.resolve_canonical_raw_callable(), - source, - {"sigma": 1.5}, + contract, contract.resolve_canonical_raw_callable(), source, {"sigma": 1.5}, execution_mode=FullStackExecution, ) -def _source( - pixels, *, domain=None, spacing=(0.5, 0.75), channel_axis=None, cohort=False -): +def _source(pixels, *, domain=None, spacing=(0.5, 0.75), channel_axis=None, cohort=False): metadata = ImagePayloadMetadata( source_path="/synthetic/original.tif", source_image_names=("Original",), @@ -82,18 +68,12 @@ def test_anisotropic_physical_z_matches_library_without_site_blur(cohort): volume[2, 4, 5] = 1 pixels = np.stack((volume, np.zeros_like(volume))) if cohort else volume source = _source( - pixels, - domain=VolumeSourceSpatialDomain(source_depth=5), - spacing=(2, 0.5, 0.75), - cohort=cohort, + pixels, domain=VolumeSourceSpatialDomain(source_depth=5), + spacing=(2, 0.5, 0.75), cohort=cohort, ) result = _execute(source) expected_volume = gaussian(volume, sigma=(0.75, 3, 2)) - expected = ( - np.stack((expected_volume, np.zeros_like(volume))) - if cohort - else expected_volume - ) + expected = np.stack((expected_volume, np.zeros_like(volume))) if cohort else expected_volume np.testing.assert_allclose(result.data, expected) assert result.metadata == source.metadata assert np.any(result.data[0, 1] if cohort else result.data[1]) @@ -105,9 +85,9 @@ def test_declared_channel_axis_is_not_smoothed(channel_axis): channels[0, 4, 5] = 1 pixels = np.moveaxis(channels, 0, channel_axis) result = _execute(_source(pixels, channel_axis=channel_axis)) - expected = np.moveaxis( - np.stack([gaussian(plane, sigma=(3, 2)) for plane in channels]), 0, channel_axis - ) + expected = np.moveaxis(np.stack([ + gaussian(plane, sigma=(3, 2)) for plane in channels + ]), 0, channel_axis) np.testing.assert_allclose(result.data, expected) @@ -115,11 +95,8 @@ def test_physical_volume_with_channels_and_independent_sites(): pixels = np.zeros((2, 5, 9, 11, 2), dtype=np.float32) pixels[0, 2, 4, 5, 0] = 1 source = _source( - pixels, - domain=VolumeSourceSpatialDomain(source_depth=5), - spacing=(2, 0.5, 0.75), - channel_axis=-1, - cohort=True, + pixels, domain=VolumeSourceSpatialDomain(source_depth=5), + spacing=(2, 0.5, 0.75), channel_axis=-1, cohort=True, ) result = _execute(source) expected = np.zeros_like(pixels) @@ -129,9 +106,7 @@ def test_physical_volume_with_channels_and_independent_sites(): def test_missing_physical_z_spacing_still_rejects(): - source = _source( - np.zeros((3, 9, 11)), domain=VolumeSourceSpatialDomain(source_depth=3) - ) + source = _source(np.zeros((3, 9, 11)), domain=VolumeSourceSpatialDomain(source_depth=3)) with pytest.raises(ValueError, match="Cannot project"): _execute(source) @@ -140,9 +115,7 @@ def test_invalid_channel_and_spatial_rank_still_reject(): with pytest.raises(ValueError, match="invalid for payload rank"): _execute(_source(np.zeros((9, 11)), channel_axis=4)) with pytest.raises(ValueError, match="spatial rank"): - _execute( - _source(np.zeros((9, 11)), domain=VolumeSourceSpatialDomain(source_depth=3)) - ) + _execute(_source(np.zeros((9, 11)), domain=VolumeSourceSpatialDomain(source_depth=3))) @pytest.mark.parametrize( @@ -155,10 +128,7 @@ def test_invalid_channel_and_spatial_rank_still_reject(): ), ) def test_optional_yx_and_strict_intrinsic_share_undeclared_axes( - shape, - channel_axis, - undeclared_axes, - yx, + shape, channel_axis, undeclared_axes, yx, ): metadata = ImagePayloadMetadata(axes=PayloadAxes.colour_samples(channel_axis)) pixels = np.zeros(shape) @@ -181,17 +151,11 @@ def test_yx_does_not_inherit_strict_volume_rank_requirement(): def test_axis_capability_is_inherited_without_metadata_overrides(): for name in ( - "undeclared_axes", - "axis_position", - "axis_index", - "spatial_axes_yx", - "declares_colour_samples_plane", - "declares_colour_samples_stack", + "undeclared_axes", "axis_position", "axis_index", "spatial_axes_yx", + "declares_colour_samples_plane", "declares_colour_samples_stack", ): assert name not in ImagePayloadMetadata.__dict__ - assert getattr(ImagePayloadMetadata, name) is getattr( - ImagePayloadAxisFields, name - ) + assert getattr(ImagePayloadMetadata, name) is getattr(ImagePayloadAxisFields, name) metadata = ImagePayloadMetadata() assert metadata.axis_position(ColourAxis) is None assert metadata.plane_axis is None @@ -218,8 +182,7 @@ def test_original_cohort_composition_keeps_every_source_plane(physical_volume): source_path=f"/synthetic/plane{index}.tif", source_image_names=("Original",), source_component_metadata={ - "well": "A01", - "site": 1 if physical_volume else index + 1, + "well": "A01", "site": 1 if physical_volume else index + 1, "z_index": index if physical_volume else 0, }, source_voxel_spacing=SourceVoxelSpacing( @@ -234,15 +197,13 @@ def test_original_cohort_composition_keeps_every_source_plane(physical_volume): if physical_volume: metadata = metadata.replace_fields( source_spatial_domain=VolumeSourceSpatialDomain().with_source_cohort( - metadata.source_spatial_domain, - depth=len(planes), + metadata.source_spatial_domain, depth=len(planes), ) ) pixels = np.stack([plane.data for plane in planes]) result = _execute(metadata.payload_with(pixels, None)) expected = ( - gaussian(pixels, sigma=(0.75, 3, 2)) - if physical_volume + gaussian(pixels, sigma=(0.75, 3, 2)) if physical_volume else np.stack([gaussian(plane, sigma=(3, 2)) for plane in pixels]) ) np.testing.assert_allclose(result.data, expected) diff --git a/tests/unit/test_callable_abi_preparation.py b/tests/unit/test_callable_abi_preparation.py index 6fc524966d..80b4504618 100644 --- a/tests/unit/test_callable_abi_preparation.py +++ b/tests/unit/test_callable_abi_preparation.py @@ -15,29 +15,20 @@ import openhcs.core.callable_contract as contract_module from openhcs.core.callable_contract import ( - CallableContract, - CallableImportIdentity, - CallableMetadata, - CallableProjection, - attach_callable_contract_metadata, - attach_processing_prepare, + CallableContract, CallableImportIdentity, CallableMetadata, CallableProjection, + attach_callable_contract_metadata, attach_processing_prepare, ) from openhcs.core.function_contract_metadata import FunctionContractAttribute from openhcs.core.function_patterns import NormalizedFunctionGroup from openhcs.core.function_reference import ImportableFunctionReference -from openhcs.core.processing_preparation import ( - CallablePreparation, - PreparationCacheBatch, -) +from openhcs.core.processing_preparation import CallablePreparation, PreparationCacheBatch from openhcs.core.runtime_image_values import ImagePayloadMetadata from openhcs.core.runtime_plane_projection import RuntimePlaneAxis from openhcs.core.pipeline.function_contracts import composed_image_payload from openhcs.processing.backends.lib_registry.registry_service import RegistryService from openhcs.core.runtime_batch_contracts import ( - Pure2DSliceBatchExecutor, - RuntimeBatchExecutionDomain, - SerialPure2DSliceBatchExecutor, - runtime_callable_defaults, + Pure2DSliceBatchExecutor, RuntimeBatchExecutionDomain, + SerialPure2DSliceBatchExecutor, runtime_callable_defaults, ) from openhcs.core.pipeline.function_contracts import resolved_callable_parameter from openhcs.core.processing_contracts import ( @@ -57,26 +48,19 @@ def unannotated_composed_echo(image): @pytest.mark.parametrize( ("composed", "annotation", "retains_payload"), - ( - (True, inspect.Parameter.empty, True), - (False, inspect.Parameter.empty, False), - (True, np.ndarray, False), - (True, Any, False), - (True, ArrayPayload, True), - ), + ((True, inspect.Parameter.empty, True), + (False, inspect.Parameter.empty, False), + (True, np.ndarray, False), + (True, Any, False), + (True, ArrayPayload, True)), ) def test_composition_default_respects_explicit_raw_carrier_boundary( - composed, - annotation, - retains_payload, - fake_preparation, + composed, annotation, retains_payload, fake_preparation, ): received = [] - def raw(image): received.append(image) return image - if annotation is not inspect.Parameter.empty: raw.__annotations__["image"] = annotation if composed: @@ -91,9 +75,7 @@ def raw(image): ).payload_with(pixels, mask) expected = source if retains_payload else pixels - result = ( - RuntimeCallablePolicy().contract_invocation(contract, raw, source, {}).call() - ) + result = RuntimeCallablePolicy().contract_invocation(contract, raw, source, {}).call() assert result is expected assert received[0] is expected @@ -103,23 +85,16 @@ def raw(image): @pytest.mark.parametrize("serializer", (pickle, cloudpickle)) def test_prepared_composition_default_survives_transport_without_introspection( - serializer, - fake_preparation, - monkeypatch, + serializer, fake_preparation, monkeypatch, ): contract = CallableContract.from_prepared_callable(unannotated_composed_echo) reference = ImportableFunctionReference( import_identity=CallableImportIdentity.from_callable(unannotated_composed_echo), - composite_key="test:unannotated_composed_echo", - metadata=contract.metadata, + composite_key="test:unannotated_composed_echo", metadata=contract.metadata, ) - restored = CallableContract.from_callable( - serializer.loads(serializer.dumps(reference)) - ) - + restored = CallableContract.from_callable(serializer.loads(serializer.dumps(reference))) def forbidden(*args, **kwargs): pytest.fail("Prepared composed ABI must not inspect live callable declarations") - monkeypatch.setattr(contract_module, "get_type_hints", forbidden) monkeypatch.setattr(inspect, "signature", forbidden) payload = ImagePayloadMetadata( @@ -134,9 +109,7 @@ def forbidden(*args, **kwargs): def fake_preparation(monkeypatch): prepare = Mock() monkeypatch.setattr(CallablePreparation, "prepare", prepare) - monkeypatch.setattr( - PreparationCacheBatch, "populate_child_caches", lambda *a, **k: None - ) + monkeypatch.setattr(PreparationCacheBatch, "populate_child_caches", lambda *a, **k: None) return prepare @@ -150,14 +123,9 @@ def last(image: np.ndarray): return image first.__dict__[FunctionContractAttribute.raw_processing_function] = last - metadata = { - "first": SimpleNamespace(func=first), - "alias": SimpleNamespace(func=first), - } + metadata = {"first": SimpleNamespace(func=first), "alias": SimpleNamespace(func=first)} monkeypatch.setattr(RegistryService, "_metadata_cache", metadata) - monkeypatch.setattr( - PreparationCacheBatch, "populate_child_caches", lambda *a, **k: None - ) + monkeypatch.setattr(PreparationCacheBatch, "populate_child_caches", lambda *a, **k: None) def prepare(owner): events.append(("prepare", owner.projection.func)) @@ -172,40 +140,17 @@ def hints(func, **kwargs): monkeypatch.setattr(CallablePreparation, "prepare", prepare) monkeypatch.setattr(contract_module, "get_type_hints", hints) - assert ( - RegistryService.prepare_in_current_process(status_callback=events.append) - is metadata - ) - preparations = [ - i - for i, item in enumerate(events) - if isinstance(item, tuple) and item[0] == "prepare" - ] - signatures = [ - i - for i, item in enumerate(events) - if isinstance(item, tuple) and item[0] == "signature" - ] + assert RegistryService.prepare_in_current_process(status_callback=events.append) is metadata + preparations = [i for i, item in enumerate(events) if isinstance(item, tuple) and item[0] == "prepare"] + signatures = [i for i, item in enumerate(events) if isinstance(item, tuple) and item[0] == "signature"] assert len(preparations) == 2 assert max(preparations) < min(signatures) - assert max(signatures) < next( - i - for i, item in enumerate(events) - if isinstance(item, str) and "kernels ready" in item - ) - assert ( - CallableContract.from_callable(first).canonical_parameter_annotations["image"] - is ArrayPayload - ) - assert ( - CallableContract.from_callable(last).canonical_parameter_annotations["image"] - is ArrayPayload - ) + assert max(signatures) < next(i for i, item in enumerate(events) if isinstance(item, str) and "kernels ready" in item) + assert CallableContract.from_callable(first).canonical_parameter_annotations["image"] is ArrayPayload + assert CallableContract.from_callable(last).canonical_parameter_annotations["image"] is ArrayPayload -def test_explicit_library_rewarm_refreshes_new_contract_not_existing_snapshot( - monkeypatch, fake_preparation -): +def test_explicit_library_rewarm_refreshes_new_contract_not_existing_snapshot(monkeypatch, fake_preparation): def raw(image: np.ndarray): return image @@ -214,19 +159,14 @@ def raw(image: np.ndarray): RegistryService.prepare_in_current_process() first = CallableContract.from_callable(raw) raw.__annotations__["image"] = ArrayPayload - assert ( - CallableContract.from_callable(raw).canonical_parameter_annotations["image"] - is np.ndarray - ) + assert CallableContract.from_callable(raw).canonical_parameter_annotations["image"] is np.ndarray RegistryService.prepare_in_current_process() second = CallableContract.from_callable(raw) assert first.canonical_parameter_annotations["image"] is np.ndarray assert second.canonical_parameter_annotations["image"] is ArrayPayload -def test_authored_compilation_prepares_before_signature_and_refreshes_each_compilation( - monkeypatch, -): +def test_authored_compilation_prepares_before_signature_and_refreshes_each_compilation(monkeypatch): def raw(image: np.ndarray, scale=2): return image @@ -246,7 +186,6 @@ def prepare(owner): def test_authored_preparation_precedes_metadata_validation(monkeypatch): def raw(image: np.ndarray): return image - events = [] original = CallableMetadata.from_projection.__func__ @@ -254,33 +193,25 @@ def validate(cls, projection): assert events == ["prepared"] return original(cls, projection) - monkeypatch.setattr( - CallablePreparation, "prepare", lambda self: events.append("prepared") - ) + monkeypatch.setattr(CallablePreparation, "prepare", lambda self: events.append("prepared")) monkeypatch.setattr(CallableMetadata, "from_projection", classmethod(validate)) contract = CallableContract.from_prepared_callable(raw) assert contract.canonical_parameter_annotations["image"] is np.ndarray -def test_unprepared_function_reference_compilation_reads_prepared_live_declaration( - monkeypatch, fake_preparation -): +def test_unprepared_function_reference_compilation_reads_prepared_live_declaration(monkeypatch, fake_preparation): reference = ImportableFunctionReference( import_identity=CallableImportIdentity.from_callable(raw_numeric), composite_key="test:raw_numeric", ) - contract = ( - NormalizedFunctionGroup.from_pattern("group", reference).items[0].contract - ) + contract = NormalizedFunctionGroup.from_pattern("group", reference).items[0].contract assert contract.func is reference assert contract.canonical_parameter_annotations["image"] is np.ndarray assert fake_preparation.call_count == 1 @pytest.mark.parametrize("serializer", [pickle, cloudpickle]) -def test_prepared_signature_metadata_namespace_and_reference_transport( - serializer, fake_preparation -): +def test_prepared_signature_metadata_namespace_and_reference_transport(serializer, fake_preparation): contract = CallableContract.from_prepared_callable(raw_numeric) metadata = contract.metadata @@ -288,14 +219,10 @@ def proxy(image): return image proxy.__dict__.update(metadata.as_namespace()) - assert ( - CallableMetadata.from_callable(proxy).canonical_signature - == metadata.canonical_signature - ) + assert CallableMetadata.from_callable(proxy).canonical_signature == metadata.canonical_signature reference = ImportableFunctionReference( import_identity=CallableImportIdentity.from_callable(raw_numeric), - composite_key="test:raw_numeric", - metadata=metadata, + composite_key="test:raw_numeric", metadata=metadata, ) restored = serializer.loads(serializer.dumps(reference)) assert restored.metadata.canonical_signature == metadata.canonical_signature @@ -305,76 +232,49 @@ def proxy(image): assert restored_contract.canonical_signature.return_annotation is np.ndarray -def test_compiled_raw_projection_never_queries_signature_or_annotations( - monkeypatch, fake_preparation -): +def test_compiled_raw_projection_never_queries_signature_or_annotations(monkeypatch, fake_preparation): contract = CallableContract.from_prepared_callable(raw_numeric) reference = ImportableFunctionReference( import_identity=CallableImportIdentity.from_callable(raw_numeric), - composite_key="test:raw_numeric", - metadata=contract.metadata, + composite_key="test:raw_numeric", metadata=contract.metadata, ) restored = CallableContract.from_callable(pickle.loads(pickle.dumps(reference))) - def forbidden(*args, **kwargs): raise AssertionError("Compiled runtime queried a live signature") - monkeypatch.setattr(contract_module, "get_type_hints", forbidden) monkeypatch.setattr(inspect, "signature", forbidden) - payload = ImagePayloadMetadata(source_image_names=("DNA",)).payload_with( - np.ones((2, 3)), None - ) + payload = ImagePayloadMetadata(source_image_names=("DNA",)).payload_with(np.ones((2, 3)), None) for _ in range(40): assert restored.raw_main_flow_call_argument(payload) is payload.data assert restored.primary_input_parameter_name == "image" assert restored.canonical_parameter_annotations["image"] is np.ndarray -def test_custom_signature_and_mutable_default_are_retained_until_recompilation( - monkeypatch, fake_preparation -): +def test_custom_signature_and_mutable_default_are_retained_until_recompilation(monkeypatch, fake_preparation): default = [] - def raw(image, values=None): return image - - raw.__signature__ = inspect.Signature( - [ - inspect.Parameter( - "image", inspect.Parameter.POSITIONAL_ONLY, annotation=ArrayPayload - ), - inspect.Parameter( - "values", inspect.Parameter.KEYWORD_ONLY, default=default - ), - ] - ) + raw.__signature__ = inspect.Signature([ + inspect.Parameter("image", inspect.Parameter.POSITIONAL_ONLY, annotation=ArrayPayload), + inspect.Parameter("values", inspect.Parameter.KEYWORD_ONLY, default=default), + ]) first = CallableContract.from_prepared_callable(raw) assert first.canonical_signature.parameters["values"].default is default - raw.__signature__ = raw.__signature__.replace( - parameters=[ - inspect.Parameter( - "image", inspect.Parameter.POSITIONAL_ONLY, annotation=np.ndarray - ), - ] - ) + raw.__signature__ = raw.__signature__.replace(parameters=[ + inspect.Parameter("image", inspect.Parameter.POSITIONAL_ONLY, annotation=np.ndarray), + ]) second = CallableContract.from_prepared_callable(raw) assert first.canonical_parameter_annotations["image"] is ArrayPayload assert second.canonical_parameter_annotations["image"] is np.ndarray -@pytest.mark.parametrize( - "mutation", - [ - lambda func: attach_processing_prepare(func, lambda: None), - lambda func: attach_callable_contract_metadata( - func, raw_processing_function=raw_numeric - ), - ], -) +@pytest.mark.parametrize("mutation", [ + lambda func: attach_processing_prepare(func, lambda: None), + lambda func: attach_callable_contract_metadata(func, raw_processing_function=raw_numeric), +]) def test_supported_declaration_changes_invalidate_published_signature(mutation): def raw(image: np.ndarray): return image - CallableProjection.from_callable(raw).warm_canonical_signature() assert FunctionContractAttribute.canonical_signature in vars(raw) mutation(raw) @@ -384,7 +284,6 @@ def raw(image: np.ndarray): def test_malformed_signature_declaration_is_rejected(): def raw(image): return image - raw.__dict__[FunctionContractAttribute.canonical_signature] = "not a signature" with pytest.raises(TypeError, match="inspect.Signature"): CallableMetadata.from_callable(raw) @@ -393,17 +292,11 @@ def raw(image): def test_exact_raw_default_filter_and_type_views_refresh_with_library_snapshot(): def raw(image: np.ndarray, scale: float = 2): return image * scale - CallableProjection.from_callable(raw).warm_canonical_signature() policy = RuntimeCallablePolicy(kwarg_policy=SignatureFilteredKwargs) assert runtime_callable_defaults(raw)["scale"] == 2 assert resolved_callable_parameter(raw, "scale").annotation is float - assert ( - policy.invocation(raw, (np.ones((2, 3)),), {"scale": 3, "unknown": 9}).call()[ - 0, 0 - ] - == 3 - ) + assert policy.invocation(raw, (np.ones((2, 3)),), {"scale":3, "unknown":9}).call()[0,0] == 3 raw.__defaults__ = (4,) raw.__annotations__["scale"] = int assert runtime_callable_defaults(raw)["scale"] == 2 @@ -416,78 +309,48 @@ def raw(image: np.ndarray, scale: float = 2): def test_different_wrapper_signature_remains_live_for_default_filter_and_parameter_queries(): def raw(image: np.ndarray, private: int = 8): return image - def wrapper(image: np.ndarray, public: int = 2): return image * public - wrapper.__dict__[FunctionContractAttribute.raw_processing_function] = raw CallableProjection.from_callable(wrapper).warm_canonical_signature() - assert runtime_callable_defaults(wrapper) == {"public": 2} + assert runtime_callable_defaults(wrapper) == {"public":2} assert resolved_callable_parameter(wrapper, "public").annotation is int policy = RuntimeCallablePolicy(kwarg_policy=SignatureFilteredKwargs) - assert ( - policy.invocation( - wrapper, (np.ones((2, 3)),), {"public": 3, "private": 99} - ).call()[0, 0] - == 3 - ) + assert policy.invocation(wrapper, (np.ones((2, 3)),), {"public":3, "private":99}).call()[0,0] == 3 wrapper.__defaults__ = (5,) - assert runtime_callable_defaults(wrapper) == {"public": 5} - assert ( - policy.invocation(wrapper, (np.ones((2, 3)),), {"private": 99}).call()[0, 0] - == 5 - ) + assert runtime_callable_defaults(wrapper) == {"public":5} + assert policy.invocation(wrapper, (np.ones((2, 3)),), {"private":99}).call()[0,0] == 5 def test_prepared_type_query_preserves_legacy_nested_annotation_stripping(): from typing import Annotated, get_type_hints - def raw(image, labels): return image - - raw.__annotations__ = { - "image": Annotated[np.ndarray, "image"], - "labels": tuple[Annotated[int, "id"], ...], - "return": Annotated[np.ndarray, "result"], - } + raw.__annotations__ = {"image":Annotated[np.ndarray,"image"],"labels":tuple[Annotated[int,"id"],...],"return":Annotated[np.ndarray,"result"]} expected = get_type_hints(raw) CallableProjection.from_callable(raw).warm_canonical_signature() - assert resolved_callable_parameter(raw, "image").annotation == expected["image"] - assert resolved_callable_parameter(raw, "labels").annotation == expected["labels"] - assert ( - CallableContract.from_callable(raw).canonical_parameter_annotations["labels"] - == raw.__annotations__["labels"] - ) + assert resolved_callable_parameter(raw,"image").annotation == expected["image"] + assert resolved_callable_parameter(raw,"labels").annotation == expected["labels"] + assert CallableContract.from_callable(raw).canonical_parameter_annotations["labels"] == raw.__annotations__["labels"] -def test_distinct_authored_runtime_signature_transport_and_new_compilation_refresh( - fake_preparation, -): +def test_distinct_authored_runtime_signature_transport_and_new_compilation_refresh(fake_preparation): def raw(image: np.ndarray, scale: int = 2) -> np.ndarray: return image * scale - @wraps(raw) def wrapper(*args, **kwargs): return raw(*args, **kwargs) - - wrapper.__signature__ = inspect.signature(raw).replace( - parameters=[ - *inspect.signature(raw).parameters.values(), - inspect.Parameter( - "injected", inspect.Parameter.KEYWORD_ONLY, default=False - ), - ] - ) + wrapper.__signature__ = inspect.signature(raw).replace(parameters=[ + *inspect.signature(raw).parameters.values(), + inspect.Parameter("injected",inspect.Parameter.KEYWORD_ONLY,default=False), + ]) first = CallableContract.from_prepared_callable(wrapper) assert "injected" in first.canonical_signature.parameters assert "injected" not in first.raw_runtime_signature.parameters assert first.metadata.raw_runtime_signature is not None restored = pickle.loads(pickle.dumps(first.metadata)) assert restored.raw_runtime_signature == first.raw_runtime_signature - assert ( - restored.as_namespace()[FunctionContractAttribute.raw_runtime_signature] - == first.raw_runtime_signature - ) + assert restored.as_namespace()[FunctionContractAttribute.raw_runtime_signature] == first.raw_runtime_signature assert FunctionContractAttribute.canonical_signature not in vars(raw) assert FunctionContractAttribute.canonical_signature not in vars(wrapper) raw.__defaults__ = (4,) @@ -498,31 +361,23 @@ def wrapper(*args, **kwargs): assert second.raw_runtime_signature.parameters["scale"].annotation is float -def test_distinct_targets_own_signatures_even_when_layouts_match_without_comparing_defaults( - fake_preparation, -): +def test_distinct_targets_own_signatures_even_when_layouts_match_without_comparing_defaults(fake_preparation): class RejectEquality: def __eq__(self, other): raise AssertionError("Signature reuse compared a mutable default") - default = RejectEquality() - def raw(image: np.ndarray, values=default) -> np.ndarray: return image - @wraps(raw) def wrapper(*args, **kwargs): return raw(*args, **kwargs) - contract = CallableContract.from_prepared_callable(wrapper) assert contract.metadata.raw_runtime_signature is not None assert contract.raw_runtime_signature is not contract.canonical_signature assert contract.raw_runtime_signature.parameters["values"].default is default raw.__defaults__ = ("new raw default",) second = CallableContract.from_prepared_callable(wrapper) - assert ( - second.raw_runtime_signature.parameters["values"].default == "new raw default" - ) + assert second.raw_runtime_signature.parameters["values"].default == "new raw default" assert contract.raw_runtime_signature.parameters["values"].default is default @@ -532,34 +387,24 @@ def test_same_actual_target_shares_canonical_signature(fake_preparation): assert contract.raw_runtime_signature is contract.canonical_signature -@pytest.mark.parametrize( - "executors", (None, {}, {RuntimeBatchExecutionDomain.PURE_2D_SLICES: None}) -) +@pytest.mark.parametrize("executors", (None, {}, {RuntimeBatchExecutionDomain.PURE_2D_SLICES: None})) def test_pure_2d_executor_selection_retains_serial_fallback(executors): - assert isinstance( - Pure2DSliceBatchExecutor.from_executors(executors), - SerialPure2DSliceBatchExecutor, - ) + assert isinstance(Pure2DSliceBatchExecutor.from_executors(executors), SerialPure2DSliceBatchExecutor) def test_pure_2d_executor_selection_retains_exact_declared_executor(): def declared(*args, **kwargs): raise AssertionError("Selecting an executor must not execute it") - selected = Pure2DSliceBatchExecutor.from_executors( - { - RuntimeBatchExecutionDomain.PURE_2D_SLICES: declared, - } - ) + selected = Pure2DSliceBatchExecutor.from_executors({ + RuntimeBatchExecutionDomain.PURE_2D_SLICES: declared, + }) assert selected is declared -def test_compiled_input_places_named_bundle_once_before_cpu_identity_shortcut( - monkeypatch, -): +def test_compiled_input_places_named_bundle_once_before_cpu_identity_shortcut(monkeypatch): from openhcs.core.aligned_image_payload import ( - AlignedImageSliceContext, - ImageOutputBundle, + AlignedImageSliceContext, ImageOutputBundle, ) from openhcs.core.function_patterns import compile_function_pattern from openhcs.processing.backends.processors.numpy_processor import gaussian_blur @@ -567,14 +412,10 @@ def test_compiled_input_places_named_bundle_once_before_cpu_identity_shortcut( pixels = tuple(np.full((3, 4), value, dtype=np.float32) for value in (0.25, 0.75)) masks = tuple(np.full((3, 4), value, dtype=bool) for value in (True, False)) source = ImageOutputBundle( - tuple( - ImagePayloadMetadata().payload_with(data, mask) - for data, mask in zip(pixels, masks, strict=True) - ), - tuple( - AlignedImageSliceContext.independent_main_flow(name) - for name in ("DNA", "Membrane") - ), + tuple(ImagePayloadMetadata().payload_with(data, mask) + for data, mask in zip(pixels, masks, strict=True)), + tuple(AlignedImageSliceContext.independent_main_flow(name) + for name in ("DNA", "Membrane")), ) calls = [] original = ImageOutputBundle.compose @@ -584,9 +425,7 @@ def observed(value, **destination): return original(value, **destination) monkeypatch.setattr(ImageOutputBundle, "compose", observed) - invocation = compile_function_pattern( - gaussian_blur, {}, {} - ).default_group.invocations[0] + invocation = compile_function_pattern(gaussian_blur, {}, {}).default_group.invocations[0] placed = invocation.convert_input(source, "numpy") invocation.main_flow_call_argument(placed) placed.data @@ -602,20 +441,16 @@ def observed(value, **destination): def test_compiled_input_reuses_produced_stack_realization_without_copy(): from openhcs.core.aligned_image_payload import ( - AlignedImageSliceContext, - ProducedImageStack, + AlignedImageSliceContext, ProducedImageStack, ) from openhcs.core.function_patterns import compile_function_pattern from openhcs.processing.backends.processors.numpy_processor import gaussian_blur source = ProducedImageStack( tuple(np.full((3, 4), value, dtype=np.float32) for value in (1, 2)), - memory_type="numpy", - plane_axis=RuntimePlaneAxis.RUNTIME_SLICE, + memory_type="numpy", plane_axis=RuntimePlaneAxis.RUNTIME_SLICE, ) - invocation = compile_function_pattern( - gaussian_blur, {}, {} - ).default_group.invocations[0] + invocation = compile_function_pattern(gaussian_blur, {}, {}).default_group.invocations[0] placed = invocation.convert_input(source, "numpy") assert invocation.convert_input(source, "numpy") is placed assert placed.data is source.data diff --git a/tests/unit/test_callable_contract.py b/tests/unit/test_callable_contract.py index 4f4d50e56b..e5017a2346 100644 --- a/tests/unit/test_callable_contract.py +++ b/tests/unit/test_callable_contract.py @@ -147,10 +147,16 @@ def process(image): contract = CallableContract.from_callable(process) - assert contract.payload_axis_requirement == PayloadAxisRequirement(ColourAxis) - assert contract.metadata.as_namespace()[ - FunctionContractAttribute.payload_axis_requirement - ] == PayloadAxisRequirement(ColourAxis) + assert ( + contract.payload_axis_requirement + == PayloadAxisRequirement(ColourAxis) + ) + assert ( + contract.metadata.as_namespace()[ + FunctionContractAttribute.payload_axis_requirement + ] + == PayloadAxisRequirement(ColourAxis) + ) def test_callable_metadata_rejects_string_payload_axis_requirement() -> None: @@ -176,7 +182,9 @@ def test_callable_metadata_reader_returns_declared_requirement_kinds() -> None: FunctionContractAttribute.payload_axis_requirement: ( PayloadAxisRequirement(ColourAxis) ), - FunctionContractAttribute.payload_axis_transition: (PreservedPayloadAxes()), + FunctionContractAttribute.payload_axis_transition: ( + PreservedPayloadAxes() + ), }, "process", ) @@ -201,7 +209,10 @@ def crop_like(image): contract = CallableContract.from_callable(crop_like) - assert contract.payload_axis_transition == PreservedPayloadAxes() + assert ( + contract.payload_axis_transition + == PreservedPayloadAxes() + ) assert ( contract.metadata.as_namespace()[ FunctionContractAttribute.payload_axis_transition @@ -255,15 +266,9 @@ def project(image, method: ProjectionMethod = ProjectionMethod.MAX): with pytest.raises(TypeError, match="project.method must be ProjectionMethod"): contract.validate_public_kwargs({"method": "max"}) - assert contract.decode_public_kwargs({"method": "max"}) == { - "method": ProjectionMethod.MAX - } - assert contract.decode_public_kwargs({"method": "MAX"}) == { - "method": ProjectionMethod.MAX - } - assert contract.decode_public_kwargs({"method": ProjectionMethod.MAX}) == { - "method": ProjectionMethod.MAX - } + assert contract.decode_public_kwargs({"method": "max"}) == {"method": ProjectionMethod.MAX} + assert contract.decode_public_kwargs({"method": "MAX"}) == {"method": ProjectionMethod.MAX} + assert contract.decode_public_kwargs({"method": ProjectionMethod.MAX}) == {"method": ProjectionMethod.MAX} with pytest.raises(TypeError, match="project.method must be ProjectionMethod"): contract.decode_public_kwargs({"method": None}) with pytest.raises(ValueError, match="not a valid"): @@ -293,18 +298,14 @@ def process(image, *, slice_index: int = 0): contract.validate_public_kwargs({"slice_index": 3}) -def test_callable_contract_reads_wrapper_declared_config_parameters( - monkeypatch, -) -> None: +def test_callable_contract_reads_wrapper_declared_config_parameters(monkeypatch) -> None: @numpy(contract=Pure3DContract) def process(image): return image contract = CallableContract.from_callable(process) declared_parameter = contract.canonical_signature.parameters["dtype_config"] - assert isinstance( - signature(process).parameters["dtype_config"].default, LazyDtypeConfig - ) + assert isinstance(signature(process).parameters["dtype_config"].default, LazyDtypeConfig) def forbidden_config_value(*args, **kwargs): raise AssertionError("Config schema discovery constructed a value") @@ -332,29 +333,19 @@ def forbidden_signature_read(owner): assert graph.config_parameters_for_step("process") is parameters -def test_config_parameter_schema_refreshes_from_replaced_pipeline_declaration( - monkeypatch, -): +def test_config_parameter_schema_refreshes_from_replaced_pipeline_declaration(monkeypatch): def forbidden_default(): raise AssertionError("Config field discovery evaluated a default factory") original_default = object() parameter = Parameter( - "custom_config", - Parameter.KEYWORD_ONLY, - annotation=LazyDtypeConfig, - default=original_default, + "custom_config", Parameter.KEYWORD_ONLY, + annotation=LazyDtypeConfig, default=original_default, ) for declared_type, admitted in ((LazyDtypeConfig, True), (int, False)): replacement = make_dataclass( "PipelineConfig", - [ - ( - "custom_config", - declared_type, - field(default_factory=forbidden_default), - ) - ], + [("custom_config", declared_type, field(default_factory=forbidden_default))], ) monkeypatch.setattr(config_module, "PipelineConfig", replacement) resolved = config_module.runtime_config_parameter(parameter) @@ -440,13 +431,16 @@ def test_callable_contract_metadata_preserves_explicit_nominal_processing_contra def process(image): return image - vars(process)[FunctionContractAttribute.processing_contract] = FlexibleContract + vars(process)[ + FunctionContractAttribute.processing_contract + ] = FlexibleContract attach_callable_contract_metadata(process, processing_contract=Pure2DContract) contract = CallableContract.from_callable(process) assert ( - vars(process)[FunctionContractAttribute.processing_contract] is FlexibleContract + vars(process)[FunctionContractAttribute.processing_contract] + is FlexibleContract ) assert contract.processing_contract is FlexibleContract @@ -607,18 +601,14 @@ def test_function_reference_preserves_batching_through_compiler_and_transport( restored = pickle.loads(pickle.dumps(contract)) assert restored.func == reference expected_metadata = reference.metadata.with_prepared_signatures(func, func) - assert ( - restored.metadata.canonical_signature == expected_metadata.canonical_signature - ) + assert restored.metadata.canonical_signature == expected_metadata.canonical_signature assert restored.metadata == expected_metadata - assert ( - restored.runtime_batch_executor(RuntimeBatchExecutionDomain.PURE_2D_SLICES) - is _batch_executor - ) - assert ( - restored.runtime_batch_executor(RuntimeBatchExecutionDomain.MEASUREMENT_IMAGES) - is _measurement_batch_executor - ) + assert restored.runtime_batch_executor( + RuntimeBatchExecutionDomain.PURE_2D_SLICES + ) is _batch_executor + assert restored.runtime_batch_executor( + RuntimeBatchExecutionDomain.MEASUREMENT_IMAGES + ) is _measurement_batch_executor def test_reference_batch_family_preserves_wrapper_precedence_and_raw_inheritance( @@ -637,14 +627,12 @@ def test_reference_batch_family_preserves_wrapper_precedence_and_raw_inheritance ) contract = normalize_function_pattern(reference).groups[0].items[0].contract - assert ( - contract.runtime_batch_executor(RuntimeBatchExecutionDomain.PURE_2D_SLICES) - is _override_batch_executor - ) - assert ( - contract.runtime_batch_executor(RuntimeBatchExecutionDomain.MEASUREMENT_IMAGES) - is _measurement_batch_executor - ) + assert contract.runtime_batch_executor( + RuntimeBatchExecutionDomain.PURE_2D_SLICES + ) is _override_batch_executor + assert contract.runtime_batch_executor( + RuntimeBatchExecutionDomain.MEASUREMENT_IMAGES + ) is _measurement_batch_executor def test_registry_reference_preserves_real_intensity_batch_declarations() -> None: diff --git a/tests/unit/test_canonical_custom_source_lookup.py b/tests/unit/test_canonical_custom_source_lookup.py index 780eb61e91..7067b036b2 100644 --- a/tests/unit/test_canonical_custom_source_lookup.py +++ b/tests/unit/test_canonical_custom_source_lookup.py @@ -30,11 +30,7 @@ def source_owner(monkeypatch, tmp_path): store = tmp_path / "custom_functions" store.mkdir() - monkeypatch.setattr( - CustomFunctionManager, - "default_storage_directory", - classmethod(lambda cls: store), - ) + monkeypatch.setattr(CustomFunctionManager, "default_storage_directory", classmethod(lambda cls: store)) monkeypatch.setattr(CustomFunctionRuntimeRegistry, "_declarations_by_name", {}) monkeypatch.setattr(CustomFunctionRuntimeRegistry, "_published_exports", {}) monkeypatch.setattr(CustomFunctionRuntimeRegistry, "_preparation_outcomes", {}) @@ -53,21 +49,14 @@ def source(name: str, expression: str = "image") -> str: def forbid_catalog(monkeypatch) -> None: def forbidden(cls, **kwargs): pytest.fail("a selected custom source must not prepare the global catalog") - - monkeypatch.setattr( - RegistryService, "get_all_functions_with_metadata", classmethod(forbidden) - ) + monkeypatch.setattr(RegistryService, "get_all_functions_with_metadata", classmethod(forbidden)) @pytest.mark.parametrize("cache", [None, {}]) -def test_persisted_lookup_loads_only_exact_source_and_reuses_original_owner( - source_owner, monkeypatch, cache -): +def test_persisted_lookup_loads_only_exact_source_and_reuses_original_owner(source_owner, monkeypatch, cache): name = "canonical_selected_source" (source_owner / f"{name}.py").write_text(source(name), encoding="utf-8") - (source_owner / "unrelated_invalid.py").write_text( - "invalid source !!!", encoding="utf-8" - ) + (source_owner / "unrelated_invalid.py").write_text("invalid source !!!", encoding="utf-8") monkeypatch.setattr(RegistryService, "_metadata_cache", cache) forbid_catalog(monkeypatch) function = get_function(f"openhcs:{name}") @@ -76,65 +65,41 @@ def test_persisted_lookup_loads_only_exact_source_and_reuses_original_owner( assert get_function(metadata.composite_key) is function assert RegistryService._metadata_cache is cache assert tuple(CustomFunctionRuntimeRegistry.metadata_by_name()) == (name,) - assert ( - metadata.import_identity.import_path - == f"openhcs.processing.custom_functions.{name}" - ) + assert metadata.import_identity.import_path == f"openhcs.processing.custom_functions.{name}" @pytest.mark.parametrize("persist", [False, True]) -def test_published_lookup_does_not_reexecute_declaration_or_warm_catalog( - source_owner, monkeypatch, persist -): - [function] = CustomFunctionManager().register_from_code( - source("canonical_published"), persist=persist - ) +def test_published_lookup_does_not_reexecute_declaration_or_warm_catalog(source_owner, monkeypatch, persist): + [function] = CustomFunctionManager().register_from_code(source("canonical_published"), persist=persist) monkeypatch.setattr(RegistryService, "_metadata_cache", None) - monkeypatch.setattr( - CustomFunctionManager, - "_prepare_source", - lambda *a: pytest.fail("reexecuted source"), - ) + monkeypatch.setattr(CustomFunctionManager, "_prepare_source", lambda *a: pytest.fail("reexecuted source")) forbid_catalog(monkeypatch) assert get_function("openhcs:canonical_published") is function -@pytest.mark.parametrize( - "mutation", - ["source_changed", "source_removed", "export_displaced", "declaration_removed"], -) -def test_cached_custom_claim_never_rescues_stale_source( - source_owner, monkeypatch, mutation -): +@pytest.mark.parametrize("mutation", ["source_changed", "source_removed", "export_displaced", "declaration_removed"]) +def test_cached_custom_claim_never_rescues_stale_source(source_owner, monkeypatch, mutation): manager = CustomFunctionManager() name = "canonical_stale" manager.register_from_code(source(name)) metadata = CustomFunctionRuntimeRegistry.metadata_by_name()[name] - monkeypatch.setattr( - RegistryService, "_metadata_cache", {metadata.composite_key: metadata} - ) + monkeypatch.setattr(RegistryService, "_metadata_cache", {metadata.composite_key: metadata}) with monkeypatch.context() as mutation_patch: if mutation == "source_changed": - (source_owner / f"{name}.py").write_text( - source(name, "image + 1"), encoding="utf-8" - ) + (source_owner / f"{name}.py").write_text(source(name, "image + 1"), encoding="utf-8") elif mutation == "source_removed": (source_owner / f"{name}.py").unlink() elif mutation == "export_displaced": mutation_patch.setattr(custom_functions, name, object()) else: manager.delete_custom_function(name) - mutation_patch.setattr( - RegistryService, "_metadata_cache", {metadata.composite_key: metadata} - ) + mutation_patch.setattr(RegistryService, "_metadata_cache", {metadata.composite_key: metadata}) with pytest.raises(RuntimeError, match="changed; recompile"): get_function(metadata.composite_key) assert RegistryService._metadata_cache[metadata.composite_key] is metadata -@pytest.mark.parametrize( - "name", ["missing_canonical", "bad:name", "_private", "../outside"] -) +@pytest.mark.parametrize("name", ["missing_canonical", "bad:name", "_private", "../outside"]) def test_missing_or_noncanonical_source_is_rejected(source_owner, name): with pytest.raises(KeyError, match="Unknown canonical function ID"): get_function(f"openhcs:{name}") @@ -143,19 +108,13 @@ def test_missing_or_noncanonical_source_is_rejected(source_owner, name): def test_filename_declaration_mismatch_is_not_accepted(source_owner): - (source_owner / "canonical_mismatch.py").write_text( - source("other_declaration"), encoding="utf-8" - ) - with pytest.raises( - ValidationError, match="filename and declaration name must match" - ): + (source_owner / "canonical_mismatch.py").write_text(source("other_declaration"), encoding="utf-8") + with pytest.raises(ValidationError, match="filename and declaration name must match"): get_function("openhcs:canonical_mismatch") assert not CustomFunctionRuntimeRegistry.metadata_by_name() -def test_public_canonical_source_evaluation_and_render_roundtrip_use_selected_owner( - source_owner, monkeypatch -): +def test_public_canonical_source_evaluation_and_render_roundtrip_use_selected_owner(source_owner, monkeypatch): from openhcs.core.pipeline_document import PipelineDocumentCodec from openhcs.core.function_reference import FunctionReferenceTransportAuthority @@ -170,10 +129,7 @@ def test_public_canonical_source_evaluation_and_render_roundtrip_use_selected_ow ) function = CustomFunctionRuntimeRegistry.metadata_by_name()[name].func assert document.pipeline_steps[0].func is function - assert ( - FunctionReferenceTransportAuthority.function_reference(function).resolve() - is function - ) + assert FunctionReferenceTransportAuthority.function_reference(function).resolve() is function rendered = PipelineDocumentCodec.render(document) assert f"from openhcs.processing.custom_functions import {name}" in rendered assert "get_function" not in rendered @@ -181,38 +137,29 @@ def test_public_canonical_source_evaluation_and_render_roundtrip_use_selected_ow assert restored.pipeline_steps[0].func is function reference = FunctionReferenceTransportAuthority.function_reference(function) from openhcs.core.function_reference import ModuleExportRegistryFunctionReference - assert isinstance(reference, ModuleExportRegistryFunctionReference) assert reference.composite_key == f"openhcs:{name}" assert reference.resolve() is function namespace = {} - exec(compile(rendered, "", "exec"), namespace) - assert namespace["pipeline_steps"][0].func is function + exec(compile(rendered, '', 'exec'), namespace) + assert namespace['pipeline_steps'][0].func is function -def test_catalog_key_cannot_contradict_original_metadata_identity( - source_owner, monkeypatch -): +def test_catalog_key_cannot_contradict_original_metadata_identity(source_owner, monkeypatch): @numpy def independent_identity(image): raise AssertionError("lookup must not execute processing") metadata = FunctionMetadata( - name="actual_declared_name", - func=independent_identity, - contract=FlexibleContract, - registry=OpenHCSRegistry(), - ) - monkeypatch.setattr( - RegistryService, "_metadata_cache", {"openhcs:wrong_name": metadata} + name="actual_declared_name", func=independent_identity, + contract=FlexibleContract, registry=OpenHCSRegistry(), ) + monkeypatch.setattr(RegistryService, "_metadata_cache", {"openhcs:wrong_name": metadata}) with pytest.raises(ValueError, match="contradicts declaration"): get_function("openhcs:wrong_name") -def test_cached_native_collision_rejects_ambiguous_current_custom_owner( - source_owner, monkeypatch -): +def test_cached_native_collision_rejects_ambiguous_current_custom_owner(source_owner, monkeypatch): name = "canonical_ambiguous" CustomFunctionManager().register_from_code(source(name), persist=False) custom = CustomFunctionRuntimeRegistry.metadata_by_name()[name] @@ -222,23 +169,15 @@ def independent_native(image): raise AssertionError("lookup must not execute processing") native = FunctionMetadata( - name=name, - func=independent_native, - contract=FlexibleContract, - registry=OpenHCSRegistry(), - original_name="independent_native", - module=__name__, - ) - monkeypatch.setattr( - RegistryService, "_metadata_cache", {custom.composite_key: native} + name=name, func=independent_native, contract=FlexibleContract, + registry=OpenHCSRegistry(), original_name="independent_native", module=__name__, ) + monkeypatch.setattr(RegistryService, "_metadata_cache", {custom.composite_key: native}) with pytest.raises(ValueError, match="ambiguous owners"): get_function(custom.composite_key) -def test_new_nominal_registry_and_independent_capabilities_use_real_cooperative_mro( - source_owner, -): +def test_new_nominal_registry_and_independent_capabilities_use_real_cooperative_mro(source_owner): events = [] class TraceClaims: @@ -270,31 +209,20 @@ def independently_declared(image): registry = IndependentCanonicalRegistry() registry_type = type(registry) registry_type.probe = FunctionMetadata( - name="independently_declared", - func=independently_declared, - contract=FlexibleContract, - registry=registry, - original_name="independently_declared", - module=__name__, + name="independently_declared", func=independently_declared, + contract=FlexibleContract, registry=registry, + original_name="independently_declared", module=__name__, ) assert get_function(registry_type.probe.composite_key) is independently_declared assert events == ["trace-enter", "declaration", "trace-exit"] - assert registry_type.__mro__.index(TraceClaims) < registry_type.__mro__.index( - DeclaredClaims - ) - assert registry_type.__mro__.index( - OpenHCSRegistry - ) < registry_type.__mro__.index(LibraryRegistryBase) + assert registry_type.__mro__.index(TraceClaims) < registry_type.__mro__.index(DeclaredClaims) + assert registry_type.__mro__.index(OpenHCSRegistry) < registry_type.__mro__.index(LibraryRegistryBase) events.clear() with pytest.raises(KeyError, match="Unknown canonical function ID"): get_function(f"{registry.library_name}:missing") assert events == ["trace-enter", "declaration", "trace-exit"] finally: - dict.pop( - LibraryRegistryBase.__registry__, - IndependentCanonicalRegistry._registry_name, - None, - ) + dict.pop(LibraryRegistryBase.__registry__, IndependentCanonicalRegistry._registry_name, None) def test_shared_consumer_has_no_concrete_owner_dispatch(): @@ -304,8 +232,7 @@ def test_shared_consumer_has_no_concrete_owner_dispatch(): syntax = ast.parse(dedent(inspect.getsource(method))) assert not any(isinstance(node, ast.Match) for node in ast.walk(syntax)) assert not any( - isinstance(node, ast.Name) - and node.id in {"OpenHCSRegistry", "CustomFunctionRuntimeRegistry"} + isinstance(node, ast.Name) and node.id in {"OpenHCSRegistry", "CustomFunctionRuntimeRegistry"} for node in ast.walk(syntax) ) assert "get_all_functions_with_metadata" not in inspect.getsource(get_function) diff --git a/tests/unit/test_catalog_current_readiness.py b/tests/unit/test_catalog_current_readiness.py index 640e32f059..6dd712f033 100644 --- a/tests/unit/test_catalog_current_readiness.py +++ b/tests/unit/test_catalog_current_readiness.py @@ -26,9 +26,7 @@ FlexibleContract, ) from openhcs.processing.custom_functions.manager import CustomFunctionManager -from openhcs.processing.custom_functions.runtime_registry import ( - CustomFunctionRuntimeRegistry, -) +from openhcs.processing.custom_functions.runtime_registry import CustomFunctionRuntimeRegistry from openhcs.runtime.function_catalog_preparation import FunctionCatalogPreparation @@ -42,19 +40,13 @@ def catalog(monkeypatch, tmp_path): monkeypatch.setenv("XDG_DATA_HOME", str(tmp_path / "data")) manager = CustomFunctionManager() metadata = FunctionMetadata( - name="declaration", - func=declaration, + name="declaration", func=declaration, contract=FlexibleContract, registry=SimpleNamespace(library_name="probe"), - module=__name__, - doc=declaration.__doc__, - ) - monkeypatch.setattr( - RegistryService, "_metadata_cache", {"probe:declaration": metadata} - ) - monkeypatch.setattr( - CustomFunctionRuntimeRegistry, "_source_revision", manager.source_revision() + module=__name__, doc=declaration.__doc__, ) + monkeypatch.setattr(RegistryService, "_metadata_cache", {"probe:declaration": metadata}) + monkeypatch.setattr(CustomFunctionRuntimeRegistry, "_source_revision", manager.source_revision()) def projected_metadata(self, **kwargs): # Model the original registry/source owners publishing their revision. @@ -64,8 +56,7 @@ def projected_metadata(self, **kwargs): monkeypatch.setattr(FunctionCatalogService, "_all_metadata", projected_metadata) kernel_calls = [] monkeypatch.setattr( - RegistryService, - "prepare_persistent_catalog", + RegistryService, "prepare_persistent_catalog", lambda **kwargs: kernel_calls.append("kernel"), ) return FunctionCatalogService(), manager, kernel_calls @@ -82,10 +73,7 @@ def test_both_public_views_are_warm(catalog, monkeypatch): service, _, _ = catalog calls = [] original = service._entry - from openhcs.agent.services.function_catalog_service import ( - PARAMETER_DOCUMENTATION_POLICY, - ) - + from openhcs.agent.services.function_catalog_service import PARAMETER_DOCUMENTATION_POLICY parameter_calls = [] original_parameters = PARAMETER_DOCUMENTATION_POLICY.parameter_specs @@ -93,9 +81,7 @@ def parameter_specs(*args, **kwargs): parameter_calls.append(args[0]) return original_parameters(*args, **kwargs) - monkeypatch.setattr( - PARAMETER_DOCUMENTATION_POLICY, "parameter_specs", parameter_specs - ) + monkeypatch.setattr(PARAMETER_DOCUMENTATION_POLICY, "parameter_specs", parameter_specs) def entry(*args, **kwargs): calls.append(args[2:4]) @@ -108,10 +94,7 @@ def entry(*args, **kwargs): assert len(calls) == 1 for compact in (False, True, False, True): page = service.catalog(compact_signatures=compact) - assert ( - service.search(query="declaration", compact_signatures=compact).items - == page.items - ) + assert service.search(query="declaration", compact_signatures=compact).items == page.items service.require_revision(page.revision) assert len(calls) == 1 assert parameter_calls == [declaration] @@ -129,21 +112,16 @@ def test_stale_success_refreshes_without_kernel_rewarm(catalog, mutation): assert preparation.observe(handle()).outcome is Outcome.READY old = RegistryService._metadata_cache if mutation == "body_only": - manager.storage_dir.joinpath("source_probe.py").write_text( - "# source revision changed\n" - ) + manager.storage_dir.joinpath("source_probe.py").write_text("# source revision changed\n") elif mutation == "delete": RegistryService._metadata_cache = {} elif mutation == "update": RegistryService._metadata_cache = { - "probe:declaration": replace( - old["probe:declaration"], doc="Updated declaration" - ) + "probe:declaration": replace(old["probe:declaration"], doc="Updated declaration") } else: RegistryService._metadata_cache = { - **old, - "probe:second": replace(old["probe:declaration"], name="second"), + **old, "probe:second": replace(old["probe:declaration"], name="second") } state = preparation.observe(handle()) assert state.outcome is Outcome.NOT_STARTED @@ -238,9 +216,7 @@ def entry(*args, **kwargs): "probe:declaration": replace(old, doc="Current declaration prose") } if source_change: - manager.storage_dir.joinpath("midway.py").write_text( - "# current source\n" - ) + manager.storage_dir.joinpath("midway.py").write_text("# current source\n") changed.set() return result @@ -279,9 +255,7 @@ def cancelling_entry(*args, **kwargs): preparation.cancel_and_join() -def test_overlapping_projection_cannot_commit_old_view_into_new_metadata( - catalog, monkeypatch -): +def test_overlapping_projection_cannot_commit_old_view_into_new_metadata(catalog, monkeypatch): service, _, _ = catalog changed = Event() original = service._entry @@ -305,10 +279,7 @@ def overlapping_entry(*args, **kwargs): assert service.projections_current() assert all(projection.metadata is current for projection in service._projections) for compact in (False, True): - assert ( - service.catalog(compact_signatures=compact).items[0].summary - == declaration.__doc__ - ) + assert service.catalog(compact_signatures=compact).items[0].summary == declaration.__doc__ preparation.cancel_and_join() @@ -340,8 +311,7 @@ def test_startup_uses_original_main_thread_warmup(catalog, monkeypatch): service, _, child_kernels = catalog main_kernels = [] monkeypatch.setattr( - FunctionCatalogPreparation, - "prepare_persistent_catalog", + FunctionCatalogPreparation, "prepare_persistent_catalog", staticmethod(lambda **kwargs: main_kernels.append("main")), ) preparation = FunctionCatalogPreparation(service) @@ -354,40 +324,22 @@ def test_startup_uses_original_main_thread_warmup(catalog, monkeypatch): def test_original_abc_owns_refresh_and_both_implementations_are_concrete(): - from openhcs.agent.services.function_catalog_service import ( - FunctionCatalogServiceABC, - ) - from openhcs.agent.services.endpoint_function_catalog_service import ( - ZMQFunctionCatalogService, - ) + from openhcs.agent.services.function_catalog_service import FunctionCatalogServiceABC + from openhcs.agent.services.endpoint_function_catalog_service import ZMQFunctionCatalogService assert "projections_current" in FunctionCatalogServiceABC.__abstractmethods__ - assert ( - FunctionCatalogService.prepare_projections - is FunctionCatalogServiceABC.prepare_projections - ) - assert ( - ZMQFunctionCatalogService.prepare_projections - is FunctionCatalogServiceABC.prepare_projections - ) + assert FunctionCatalogService.prepare_projections is FunctionCatalogServiceABC.prepare_projections + assert ZMQFunctionCatalogService.prepare_projections is FunctionCatalogServiceABC.prepare_projections assert not FunctionCatalogService.__abstractmethods__ assert not ZMQFunctionCatalogService.__abstractmethods__ -@pytest.mark.parametrize( - "native_outcome", (Outcome.READY, Outcome.PENDING, Outcome.FAILED) -) +@pytest.mark.parametrize("native_outcome", (Outcome.READY, Outcome.PENDING, Outcome.FAILED)) def test_endpoint_current_readiness_uses_bound_native_status(native_outcome): from openhcs.agent.dto.common import SCHEMA_VERSION - from openhcs.agent.dto.functions import ( - FunctionCatalogPreparationState, - FunctionCatalogEntry, - catalog_page, - ) + from openhcs.agent.dto.functions import FunctionCatalogPreparationState, FunctionCatalogEntry, catalog_page from openhcs.agent.services.endpoint_function_catalog_service import ( - FunctionCatalogClientSession, - FunctionCatalogProjection, - ZMQFunctionCatalogService, + FunctionCatalogClientSession, FunctionCatalogProjection, ZMQFunctionCatalogService, ) from openhcs.runtime.zmq_config import OPENHCS_ZMQ_CONFIG from zmqruntime.startup import EndpointStartupPhase, EndpointStartupStatus @@ -398,14 +350,11 @@ def test_endpoint_current_readiness_uses_bound_native_status(native_outcome): def status(request): seen.append(request) return FunctionCatalogPreparationState( - schema_version=SCHEMA_VERSION, - handle=request.handle, + schema_version=SCHEMA_VERSION, handle=request.handle, outcome=native_outcome, progress=EndpointStartupStatus( - sequence=1, - phase=EndpointStartupPhase.PREPARING_CAPABILITIES, - message="Native readiness witness", - timestamp=0, + sequence=1, phase=EndpointStartupPhase.PREPARING_CAPABILITIES, + message="Native readiness witness", timestamp=0, ), ) @@ -413,34 +362,20 @@ def status(request): connected_endpoint=SimpleNamespace(process_identity=ProcessIdentity.current()), function_catalog_preparation=status, ) - service = ZMQFunctionCatalogService( - lambda: endpoint, client_factory=lambda _: client - ) + service = ZMQFunctionCatalogService(lambda: endpoint, client_factory=lambda _: client) assert not service.projections_current() assert not seen entry = FunctionCatalogEntry( - function_id="probe:declaration", - import_path=__name__ + ".declaration", - name="declaration", - module=__name__, - library="probe", - signature="declaration(image)", - summary="Witness", - backend_tags=(), + function_id="probe:declaration", import_path=__name__ + ".declaration", + name="declaration", module=__name__, library="probe", + signature="declaration(image)", summary="Witness", backend_tags=(), ) page = catalog_page( - items=(entry,), - catalog_items=(entry,), - total=1, - limit=1, - query=None, - library=None, + items=(entry,), catalog_items=(entry,), total=1, limit=1, query=None, library=None, ) service._client_session = FunctionCatalogClientSession(endpoint, client) service._endpoint_state = FunctionCatalogProjection.from_page( - endpoint, - page, - compact_signatures=True, + endpoint, page, compact_signatures=True, ) assert service.projections_current() is (native_outcome is Outcome.READY) assert len(seen) == 1 @@ -452,17 +387,11 @@ def test_original_control_family_stays_pending_during_refresh( catalog, monkeypatch, request_kind ): from openhcs.agent.dto.functions import ( - FunctionCatalogControlPayload, - FunctionCatalogControlRequest, - FunctionCatalogPreparationControlResponse, - FunctionDetailControlRequest, - FunctionReferenceControlRequest, - FunctionSearchRequest, - ) - from openhcs.runtime.zmq_control import ( - ZMQControlMessageRouter, - ZMQControlRequestContext, + FunctionCatalogControlPayload, FunctionCatalogControlRequest, + FunctionCatalogPreparationControlResponse, FunctionDetailControlRequest, + FunctionReferenceControlRequest, FunctionSearchRequest, ) + from openhcs.runtime.zmq_control import ZMQControlMessageRouter, ZMQControlRequestContext service, _, _ = catalog preparation = FunctionCatalogPreparation(service) @@ -496,18 +425,13 @@ def delayed(*args, **kwargs): try: assert entered.wait(timeout=2) response = ZMQControlMessageRouter.handle( - FunctionCatalogControlPayload.from_request( - requests[request_kind] - ).to_dict(), + FunctionCatalogControlPayload.from_request(requests[request_kind]).to_dict(), ZMQControlRequestContext( - compiled_artifacts={}, - function_catalog=service, + compiled_artifacts={}, function_catalog=service, function_catalog_preparation=preparation, ), ) - pending = FunctionCatalogPreparationControlResponse.from_control_response( - response - ) + pending = FunctionCatalogPreparationControlResponse.from_control_response(response) assert pending.status is not None assert not future.done() finally: diff --git a/tests/unit/test_cellprofiler_adapter.py b/tests/unit/test_cellprofiler_adapter.py index 70054dde60..90f8da35bb 100644 --- a/tests/unit/test_cellprofiler_adapter.py +++ b/tests/unit/test_cellprofiler_adapter.py @@ -207,8 +207,7 @@ def test_native_cellprofiler_imported_metadata_places_files_by_path_columns( with csv_path.open("a", encoding="utf-8") as handle: handle.write("unselected.tif,20585/\n") universe = NativeCellProfilerSelectedSourceUniverse( - (image_path, csv_path), - placements=placements, + (image_path, csv_path), placements=placements, imported_metadata_paths=(csv_path,), ) staged = universe.materialize_flat_input_dir(tmp_path / "staged") diff --git a/tests/unit/test_cellprofiler_additional_prepared_payload_abi.py b/tests/unit/test_cellprofiler_additional_prepared_payload_abi.py index 849fa9a421..c6439f57d4 100644 --- a/tests/unit/test_cellprofiler_additional_prepared_payload_abi.py +++ b/tests/unit/test_cellprofiler_additional_prepared_payload_abi.py @@ -1,5 +1,4 @@ """Prepared raw arguments retain context consumed by nominal CP request owners.""" - from dataclasses import replace import numpy as np @@ -15,27 +14,16 @@ CellProfilerFunctionContractExecutor, ) from openhcs.processing.backends.cellprofiler.alignment import AlignModule, align -from openhcs.processing.backends.cellprofiler.area_occupied import ( - measure_image_area_occupied, -) +from openhcs.processing.backends.cellprofiler.area_occupied import measure_image_area_occupied from openhcs.processing.backends.cellprofiler.crop import CropModule, crop -from openhcs.processing.backends.cellprofiler.grid import ( - ShapeChoice, - identify_objects_in_grid, -) -from openhcs.core.runtime_object_label_building import ( - SourceImageObjectLabelBuildRequest, -) +from openhcs.processing.backends.cellprofiler.grid import ShapeChoice, identify_objects_in_grid +from openhcs.core.runtime_object_label_building import SourceImageObjectLabelBuildRequest from openhcs.core.runtime_spatial_grid import SpatialGrid from openhcs.core.source_metadata import SourceVoxelSpacing from openhcs.processing.backends.cellprofiler.image_quality import measure_image_quality from openhcs.processing.backends.cellprofiler.intensity import measure_object_intensity -from openhcs.processing.backends.cellprofiler.manual_objects import ( - identify_objects_manually, -) -from openhcs.processing.backends.cellprofiler.object_images import ( - convert_image_to_objects, -) +from openhcs.processing.backends.cellprofiler.manual_objects import identify_objects_manually +from openhcs.processing.backends.cellprofiler.object_images import convert_image_to_objects from openhcs.processing.backends.cellprofiler.thresholding import threshold from openhcs.processing.backends.cellprofiler.worms import ( identify_dead_worms, @@ -104,13 +92,8 @@ def test_prepared_primary_context_consumers_keep_nominal_payload_and_bare_pixels @pytest.mark.parametrize("shape", ((5, 5), (4, 7))) -@pytest.mark.parametrize( - "shape_choice", - (ShapeChoice.RECTANGLE, ShapeChoice.NATURAL, ShapeChoice.CIRCLE_NATURAL), -) -def test_prepared_grid_objects_preserve_rgb_mask_calibration_and_crop_domain( - shape, shape_choice -): +@pytest.mark.parametrize("shape_choice", (ShapeChoice.RECTANGLE, ShapeChoice.NATURAL, ShapeChoice.CIRCLE_NATURAL)) +def test_prepared_grid_objects_preserve_rgb_mask_calibration_and_crop_domain(shape, shape_choice): pixels = np.arange(np.prod(shape) * 3, dtype=np.float32).reshape(*shape, 3) mask = np.ones(shape, dtype=bool) mask[0] = False @@ -119,35 +102,22 @@ def test_prepared_grid_objects_preserve_rgb_mask_calibration_and_crop_domain( axes=PayloadAxes.colour_samples(-1), source_voxel_spacing=SourceVoxelSpacing((2.0, 0.7, 0.9)), source_spatial_domain=SourceSpatialDomain( - origin_yx=(2, 3), - source_shape_yx=(shape[0] + 4, shape[1] + 6), + origin_yx=(2, 3), source_shape_yx=(shape[0] + 4, shape[1] + 6), ), ) source = metadata.payload_with(pixels, mask) grid = SpatialGrid( - name="Grid", - rows=1, - columns=1, - x_spacing=4.0, - y_spacing=4.0, - x_origin=2.0, - y_origin=2.0, - source_spatial_shape_yx=shape, + name="Grid", rows=1, columns=1, x_spacing=4.0, y_spacing=4.0, + x_origin=2.0, y_origin=2.0, source_spatial_shape_yx=shape, ) guide_pixels = np.zeros(shape, dtype=np.int32) guide_pixels[1:3, 1:4] = 1 - guides = SourceImageObjectLabelBuildRequest( - image=source, labels=guide_pixels - ).payload() - topology_inputs = ( - (grid,) if shape_choice is ShapeChoice.RECTANGLE else (grid, guides) - ) + guides = SourceImageObjectLabelBuildRequest(image=source, labels=guide_pixels).payload() + topology_inputs = (grid,) if shape_choice is ShapeChoice.RECTANGLE else (grid, guides) contract = _prepared_contract(identify_objects_in_grid) returned_image, _rows, objects = CellProfilerFunctionContractExecutor().execute( - contract, - contract.resolve_canonical_raw_callable(), - source, + contract, contract.resolve_canonical_raw_callable(), source, {"topology_inputs": topology_inputs, "shape_choice": shape_choice}, execution_mode=FullStackExecution, ) @@ -156,21 +126,14 @@ def test_prepared_grid_objects_preserve_rgb_mask_calibration_and_crop_domain( assert returned_image.mask is mask assert objects.source_provenance == metadata.source_provenance assert objects.source_spatial_domain.origin_yx == (2, 3) - assert ( - objects.source_spatial_domain.source_shape_yx - == metadata.source_spatial_domain.source_shape_yx - ) + assert objects.source_spatial_domain.source_shape_yx == metadata.source_spatial_domain.source_shape_yx assert objects.parent_image_source_voxel_spacing == metadata.source_voxel_spacing assert objects.labels.shape == shape assert objects.domain.declared_object_count == 1 assert objects.domain.declared_object_ids == (1,) - expanded = objects.source_spatial_domain.materialize( - objects.labels, spatial_axes_yx=(0, 1) - ) + expanded = objects.source_spatial_domain.materialize(objects.labels, spatial_axes_yx=(0, 1)) assert expanded.shape == metadata.source_spatial_domain.source_shape_yx - np.testing.assert_array_equal( - expanded[2 : 2 + shape[0], 3 : 3 + shape[1]], objects.labels - ) + np.testing.assert_array_equal(expanded[2:2 + shape[0], 3:3 + shape[1]], objects.labels) @pytest.mark.parametrize("color", (False, True)) @@ -184,10 +147,7 @@ def test_prepared_align_preserves_named_plane_axis_masks_and_source_identity(col source_image_names=("First", "Second"), source_image_provenance_planes=SourceImageProvenancePlanes.from_components( paths=("/input/first.tif", "/input/second.tif"), - component_metadata=( - {"well": "A01", "channel": "1"}, - {"well": "A01", "channel": "2"}, - ), + component_metadata=({"well": "A01", "channel": "1"}, {"well": "A01", "channel": "2"}), ), plane_axis=RuntimePlaneAxis.SOURCE_BINDING, axes=PayloadAxes.colour_samples(3 if color else None), @@ -202,10 +162,8 @@ def test_prepared_align_preserves_named_plane_axis_masks_and_source_identity(col contract, contract.resolve_canonical_raw_callable(), source, - { - "method": AlignModule.Method.NORMALIZED_CROSS_CORRELATION, - "crop_mode": AlignModule.CropMode.KEEP_SIZE, - }, + {"method": AlignModule.Method.NORMALIZED_CROSS_CORRELATION, + "crop_mode": AlignModule.CropMode.KEEP_SIZE}, execution_mode=FullStackExecution, ) @@ -215,9 +173,7 @@ def test_prepared_align_preserves_named_plane_axis_masks_and_source_identity(col np.testing.assert_array_equal(output.data, pixels[index]) np.testing.assert_array_equal(output.mask, masks[index]) output_metadata = output.metadata - assert output_metadata.source_image_paths == ( - ("/input/first.tif", "/input/second.tif")[index], - ) + assert output_metadata.source_image_paths == (("/input/first.tif", "/input/second.tif")[index],) assert output_metadata.source_spatial_domain == metadata.source_spatial_domain assert output_metadata.axis_position(ColourAxis) == (2 if color else None) @@ -239,11 +195,9 @@ def test_prepared_crop_preserves_rgb_spatial_mask_and_calibrated_source_domain() contract, contract.resolve_canonical_raw_callable(), source, - { - "left_right_rectangle_positions": (1, 4), - "top_bottom_rectangle_positions": (0, 3), - "removal_method": CropModule.RemovalMethod.NO, - }, + {"left_right_rectangle_positions": (1, 4), + "top_bottom_rectangle_positions": (0, 3), + "removal_method": CropModule.RemovalMethod.NO}, execution_mode=NaturalExecution, ) @@ -265,13 +219,7 @@ def test_prepared_threshold_keeps_owned_unit_interval_proof_and_mask(): pixels = np.arange(20, dtype=np.uint8).reshape(4, 5) mask = np.ones(pixels.shape, dtype=bool) mask[0] = False - source = ( - ImagePayloadMetadata.for_array(pixels) - .payload_with(pixels, mask) - .normalize_intensity_payload( - dtype=np.float32, - ) - ) + source = ImagePayloadMetadata.for_array(pixels).payload_with(pixels, mask).normalize_intensity_payload(dtype=np.float32,) contract = _prepared_contract(threshold) projected = contract.raw_main_flow_call_argument(source) diff --git a/tests/unit/test_cellprofiler_analyst_export.py b/tests/unit/test_cellprofiler_analyst_export.py index de9047d74e..b91206e86a 100644 --- a/tests/unit/test_cellprofiler_analyst_export.py +++ b/tests/unit/test_cellprofiler_analyst_export.py @@ -142,10 +142,7 @@ def count_projection( ) -> FieldSpec | None: calls.append((table.name, subject, field_name)) return original( - self, - table, - field_name, - subject=subject, + self, table, field_name, subject=subject, project_database_field=project_database_field, ) @@ -171,9 +168,7 @@ def count_projection( subject=subject, ) - first_subject, first_rows, first_fields = next( - projection.measurement_projections(first, scope=None) - ) + first_subject, first_rows, first_fields = next(projection.measurement_projections(first, scope=None)) assert first_subject == subject assert first_fields == (FieldSpec("Count_Nuclei", int),) assert first_rows.row_mappings() == ( @@ -181,9 +176,7 @@ def count_projection( {"Count_Nuclei": 2}, {"Count_Nuclei": 3}, ) - second_subject, second_rows, second_fields = next( - projection.measurement_projections(second, scope=None) - ) + second_subject, second_rows, second_fields = next(projection.measurement_projections(second, scope=None)) assert second_subject == subject assert second_fields == (FieldSpec("Count_Nuclei", float),) assert second_rows.row_mappings() == ( @@ -210,9 +203,7 @@ def count_projection( ).row_mappings()[0] == {"Cells_AreaShape_Area": 1.0} assert projection._project_runtime_rows( object_table, - MeasurementProjectedColumnarRows( - {"AreaShape_Area": (2.0,)}, fields=(FieldSpec("AreaShape_Area", float),) - ), + MeasurementProjectedColumnarRows({"AreaShape_Area": (2.0,)}, fields=(FieldSpec("AreaShape_Area", float),)), subject=nuclei, field_projection_cache=object_field_cache, project_database_field=None, @@ -253,21 +244,16 @@ def test_database_field_projection_refreshes_declarations_and_live_dtype_hooks( table = MeasurementTable( name="EmptyDistances", rows=MeasurementProjectedColumnarRows( - {field.name: ()}, - fields=(field,), + {field.name: ()}, fields=(field,), ), subject=MeasurementSubject(MeasurementScope.RUN), measurement_feature_owner=RelateObjectsModule, ) - _subject, actual_rows, actual_fields = next( - projection.measurement_projections(table, scope=None) - ) + _subject, actual_rows, actual_fields = next(projection.measurement_projections(table, scope=None)) assert actual_rows.row_mappings() == () assert actual_fields == (field,) monkeypatch.undo() - _subject, actual_rows, actual_fields = next( - projection.measurement_projections(table, scope=None) - ) + _subject, actual_rows, actual_fields = next(projection.measurement_projections(table, scope=None)) assert actual_rows.row_mappings() == () assert actual_fields == (replace(field, dtype=int),) @@ -279,9 +265,7 @@ def database_measurement_field_projection(cls): return lambda source_field: replace(source_field, dtype=str) custom_table = replace(table, measurement_feature_owner=CustomFieldOwner) - _subject, actual_rows, actual_fields = next( - projection.measurement_projections(custom_table, scope=None) - ) + _subject, actual_rows, actual_fields = next(projection.measurement_projections(custom_table, scope=None)) assert actual_rows.row_mappings() == () assert actual_fields == (replace(field, dtype=str),) assert CustomFieldOwner.database_measurement_field(field).dtype is str @@ -874,29 +858,13 @@ def _borrowed_source_export_fixture( sources.extend( ( SourcePlaneProjection( - address=OpenHCSPlaneAddress( - ( - (Microscopy.Well, "A01"), - (Microscopy.Site, site), - (Microscopy.Channel, 1), - (Microscopy.ZIndex, 1), - (Microscopy.Timepoint, 1), - ) - ), + address=OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, site), (Microscopy.Channel, 1), (Microscopy.ZIndex, 1), (Microscopy.Timepoint, 1))), ref=SourcePixelRef("disk", str(original)), source_alias="DNA", source_metadata=metadata, ), SourceArtifactProjection( - address=OpenHCSPlaneAddress( - ( - (Microscopy.Well, "A01"), - (Microscopy.Site, site), - (Microscopy.Channel, 2), - (Microscopy.ZIndex, 1), - (Microscopy.Timepoint, 1), - ) - ), + address=OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, site), (Microscopy.Channel, 2), (Microscopy.ZIndex, 1), (Microscopy.Timepoint, 1))), ref=SourcePixelRef( "disk", str(calibration), (site - 1,) if embedded_tiff else () ), @@ -926,7 +894,7 @@ def _borrowed_source_export_fixture( metadata_handler=SimpleNamespace( source_workspace_root=lambda _plate_path: tmp_path, source_workspace_metadata_document=lambda _plate_path: document, - ), + ) ) context.filemanager = SimpleNamespace( exists=lambda *_args: False, @@ -995,10 +963,7 @@ def test_source_only_calibration_occurrences_export_each_executed_site( tmp_path ) assert batch.records_of_type(ImageArtifactType)["A01"] == () - from openhcs.core.source_workspace_projection import ( - VirtualWorkspaceSourceProjection, - ) - + from openhcs.core.source_workspace_projection import VirtualWorkspaceSourceProjection monkeypatch.setattr( VirtualWorkspaceSourceProjection, "load_binding_payloads", @@ -1971,27 +1936,13 @@ def _record_relationship( def _external_rows( table: CellProfilerProjectedTable, ) -> tuple[dict[str, object], ...]: - return tuple( - dict(row) - for row in ( - table.rows.iter_row_mappings() - if isinstance(table.rows, ColumnarRows) - else table.rows - ) - ) + return tuple(dict(row) for row in (table.rows.iter_row_mappings() if isinstance(table.rows, ColumnarRows) else table.rows)) def _field_rows( table: CellProfilerProjectedTable, ) -> tuple[dict[str, object], ...]: - return tuple( - dict(row) - for row in ( - table.rows.iter_row_mappings() - if isinstance(table.rows, ColumnarRows) - else table.rows - ) - ) + return tuple(dict(row) for row in (table.rows.iter_row_mappings() if isinstance(table.rows, ColumnarRows) else table.rows)) def test_database_projection_includes_derived_grid_measurements() -> None: @@ -2039,8 +1990,7 @@ def test_database_projection_includes_derived_grid_measurements() -> None: def test_source_digest_is_shared_across_image_numbers_and_channel_aliases( - tmp_path, - monkeypatch, + tmp_path, monkeypatch, ): from collections import defaultdict from hashlib import md5 @@ -2096,10 +2046,8 @@ def test_source_projection_frame_error_precedes_digest_read(tmp_path, monkeypatc source = tmp_path / "source.tif" tifffile.imwrite( - source, - np.zeros((3, 4, 5), dtype=np.uint16), - photometric="minisblack", - metadata={"axes": "ZYX"}, + source, np.zeros((3, 4, 5), dtype=np.uint16), + photometric="minisblack", metadata={"axes": "ZYX"}, ) def forbidden_read(self): @@ -2109,9 +2057,7 @@ def forbidden_read(self): monkeypatch.setattr(Path, "read_bytes", forbidden_read) with pytest.raises(ValueError, match="Source frame index True"): export._source_image_projection_values( - source, - "DNA", - CellProfilerDatabaseColumnDialect(), + source, "DNA", CellProfilerDatabaseColumnDialect(), source_axis_indices=(True,), ) assert ImageFileRevision.md5_digest.cache_info().currsize == 0 @@ -2125,16 +2071,12 @@ def test_repeated_source_contributors_share_physical_projection(tmp_path, monkey source = tmp_path / "DNA.tif" import tifffile - tifffile.imwrite( - source, - np.arange(192, dtype=np.uint16).reshape(3, 8, 8), - photometric="minisblack", - metadata={"axes": "ZYX"}, + source, np.arange(192, dtype=np.uint16).reshape(3, 8, 8), + photometric="minisblack", metadata={"axes": "ZYX"}, ) provenance = SourceImageProvenance( - source_path=str(source), - source_component_metadata={"well": "A01", "site": "1"}, + source_path=str(source), source_component_metadata={"well": "A01", "site": "1"}, source_image_names=("DNA",), ) owner = CPATableRowProjection( @@ -2144,126 +2086,77 @@ def test_repeated_source_contributors_share_physical_projection(tmp_path, monkey rows, metadata = {}, defaultdict(list) manufacture = export._source_image_projection_values calls = [] - def counted(*args, **kwargs): calls.append(args[0]) return manufacture(*args, **kwargs) - monkeypatch.setattr(export, "_source_image_projection_values", counted) for _ in range(3): owner.collect_image_provenance( - provenance, - scope=RuntimeExecutionAxisScope(AXIS_ID), - source_image_name="DNA", - image_rows_by_number=rows, - source_metadata_by_image_number=metadata, + provenance, scope=RuntimeExecutionAxisScope(AXIS_ID), source_image_name="DNA", + image_rows_by_number=rows, source_metadata_by_image_number=metadata, ) assert calls == [source] assert len(metadata[1]) == 3 # Every contributor's correlation evidence survives. conflicting = provenance.with_source_path(str(tmp_path / "other.tif")) with pytest.raises(ValueError, match="conflicting values"): owner.collect_image_provenance( - conflicting, - scope=RuntimeExecutionAxisScope(AXIS_ID), - source_image_name="DNA", - image_rows_by_number=rows, - source_metadata_by_image_number=metadata, + conflicting, scope=RuntimeExecutionAxisScope(AXIS_ID), source_image_name="DNA", + image_rows_by_number=rows, source_metadata_by_image_number=metadata, ) with pytest.raises(ValueError, match="conflicting values"): owner.collect_image_provenance( - provenance, - scope=RuntimeExecutionAxisScope(AXIS_ID), - source_image_name="DNA", - image_rows_by_number=rows, - source_metadata_by_image_number=metadata, + provenance, scope=RuntimeExecutionAxisScope(AXIS_ID), source_image_name="DNA", + image_rows_by_number=rows, source_metadata_by_image_number=metadata, source_axis_indices=(1,), ) assert calls == [source] # Explicit frame admission still rejects contradiction. -def test_produced_thumbnail_uses_named_pixels_not_measurement_source( - tmp_path, monkeypatch -): +def test_produced_thumbnail_uses_named_pixels_not_measurement_source(tmp_path, monkeypatch): import openhcs.interop.cellprofiler.analyst_export as export - source = tmp_path / "DNA.tif" Image.fromarray(np.arange(64, dtype=np.uint8).reshape(8, 8)).save(source) store = RuntimeValueStore() - dna = _record_image( - store, - name="DNA", - source_path=str(source), - metadata={"well": "A01", "site": "1"}, - ) + dna = _record_image(store, name="DNA", source_path=str(source), metadata={"well": "A01", "site": "1"}) planes = SourceImageProvenancePlanes.from_components( - paths=(str(source),), - component_metadata=({"well": "A01", "site": "1"},), - ) - measurements = tuple( - _record_measurements( - store, - table=MeasurementTable( - name=f"Measurements{index}", - source_image_names=("DNA",), - subject=MeasurementSubject(MeasurementScope.SAMPLE, "Image"), - rows=MeasurementSparseColumnarRows.from_rows( - ({RUNTIME_IMAGE_FIELD: 0, f"Value{index}": float(index)},), - fields=( - FieldSpec(RUNTIME_IMAGE_FIELD, int), - FieldSpec(f"Value{index}", float), - ), - ), - ), - source_image_provenance_planes=planes, - ) - for index in range(3) + paths=(str(source),), component_metadata=({"well": "A01", "site": "1"},), ) + measurements = tuple(_record_measurements( + store, table=MeasurementTable( + name=f"Measurements{index}", source_image_names=("DNA",), + subject=MeasurementSubject(MeasurementScope.SAMPLE, "Image"), + rows=MeasurementSparseColumnarRows.from_rows( + ({RUNTIME_IMAGE_FIELD: 0, f"Value{index}": float(index)},), + fields=(FieldSpec(RUNTIME_IMAGE_FIELD, int), FieldSpec(f"Value{index}", float)), + ), + ), source_image_provenance_planes=planes, + ) for index in range(3)) batch = RuntimeArtifactBatch( - input_specs=(dna, *measurements), - records_by_axis={AXIS_ID: store.values()}, + input_specs=(dna, *measurements), records_by_axis={AXIS_ID: store.values()}, source_image_set_identity_policy=SourceImageSetIdentityPolicy(), ) - settings = replace( - _settings(), - write_image_thumbnails=True, - thumbnail_image_names=("DNA",), - auto_scale_thumbnail_intensities=False, - ) + settings = replace(_settings(), write_image_thumbnails=True, thumbnail_image_names=("DNA",), auto_scale_thumbnail_intensities=False) encode = export._thumbnail_png_base64 pixels = [] - def counted(image, **kwargs): pixels.append(np.asarray(image).copy()) return encode(image, **kwargs) - monkeypatch.setattr(export, "_thumbnail_png_base64", counted) - projection = _projection_builder().build( - batch, - settings, - (CPAImageChannelSpec(alias="DNA", image_name="DNA", channel_color="none"),), - ) + projection = _projection_builder().build(batch, settings, (CPAImageChannelSpec(alias="DNA", image_name="DNA", channel_color="none"),)) assert len(pixels) == 1 assert np.array_equal(pixels[0], np.zeros((8, 8), dtype=np.uint8)) - assert _external_rows(projection.image_table)[0]["Image_Thumbnail_DNA"] == encode( - pixels[0], auto_scale=False - ) + assert _external_rows(projection.image_table)[0]["Image_Thumbnail_DNA"] == encode(pixels[0], auto_scale=False) def test_source_bound_thumbnail_only_ref_preserves_executed_sites(tmp_path): - builder, batch, _channels, calibration, _document = _borrowed_source_export_fixture( - tmp_path - ) + builder, batch, _channels, calibration, _document = _borrowed_source_export_fixture(tmp_path) builder.context.filemanager.load_batch = lambda paths, _backend: [ np.load(calibration) for _path in paths ] - builder.context.filemanager.physical_source_path = ( - lambda address, _backend, *, base_path: address - ) + builder.context.filemanager.physical_source_path = lambda address, _backend, *, base_path: address settings = replace( - _settings(), - write_image_thumbnails=True, - thumbnail_image_names=("IllumDNA",), + _settings(), write_image_thumbnails=True, thumbnail_image_names=("IllumDNA",), auto_scale_thumbnail_intensities=False, ) projection = builder.build(batch, settings, ()) @@ -2271,7 +2164,5 @@ def test_source_bound_thumbnail_only_ref_preserves_executed_sites(tmp_path): assert len(rows) == 2 # The third declared source occurrence was not executed. assert all(row["Image_Thumbnail_IllumDNA"] for row in rows) assert all("Image_FileName_IllumDNA" not in row for row in rows) - disabled = replace( - settings, write_image_thumbnails=False, thumbnail_image_names=("Missing",) - ) + disabled = replace(settings, write_image_thumbnails=False, thumbnail_image_names=("Missing",)) assert len(_external_rows(builder.build(batch, disabled, ()).image_table)) == 2 diff --git a/tests/unit/test_cellprofiler_artifact_declarations.py b/tests/unit/test_cellprofiler_artifact_declarations.py index c3810635bb..29144593ee 100644 --- a/tests/unit/test_cellprofiler_artifact_declarations.py +++ b/tests/unit/test_cellprofiler_artifact_declarations.py @@ -596,35 +596,30 @@ def test_calculate_math_retains_measurement_subjects_across_producer_channels() ), ) contract = _callable_contract( - _module( - 8, - "CalculateMath", - { - "Name the output measurement": "Ratio", - "Operation": "Divide", - "Select the numerator objects": "Nuclei", - "Select the numerator measurement": "Intensity_MeanIntensity_CorrProtein", - "Select the denominator objects": "Cells", - "Select the denominator measurement": "Intensity_MeanIntensity_CorrProtein", - }, - ), + _module(8, "CalculateMath", { + "Name the output measurement": "Ratio", + "Operation": "Divide", + "Select the numerator objects": "Nuclei", + "Select the numerator measurement": "Intensity_MeanIntensity_CorrProtein", + "Select the denominator objects": "Cells", + "Select the denominator measurement": "Intensity_MeanIntensity_CorrProtein", + }), step_index=7, available_artifacts=ArtifactSpecCollection((image, *objects, measurements)), main_flow_artifacts=ArtifactSpecCollection((image,)), available_artifact_producers=artifact_producers_for_outputs( (measurements,), groups=("2", "1"), - invocation_keys=( - FunctionInvocationKey("measure_object_intensity", DEFAULT_GROUP_KEY, 0), - ), + invocation_keys=(FunctionInvocationKey( + "measure_object_intensity", DEFAULT_GROUP_KEY, 0 + ),), ), ) (measurement_input,) = contract.artifact_inputs.of_artifact_type( MeasurementsArtifactType ) assert tuple( - relation.source.name - for relation in measurement_input.relations + relation.source.name for relation in measurement_input.relations if isinstance(relation, InputObjectMeasurementSourceRelation) ) == ("Nuclei", "Cells") storage = ArtifactInputPlan( diff --git a/tests/unit/test_cellprofiler_backend_public_api.py b/tests/unit/test_cellprofiler_backend_public_api.py index 778290f49e..0a6e9298f6 100644 --- a/tests/unit/test_cellprofiler_backend_public_api.py +++ b/tests/unit/test_cellprofiler_backend_public_api.py @@ -284,6 +284,7 @@ def test_callable_ownership_uses_complete_import_identity() -> None: assert CellProfilerModule.for_callable_contract(numpy_contract) is None + @pytest.mark.parametrize("mutation", ("replacement", "unowned_exclusion")) def test_compiled_callable_owner_keeps_current_declaration_admission( monkeypatch: pytest.MonkeyPatch, @@ -312,7 +313,6 @@ def test_compiled_callable_owner_keeps_current_declaration_admission( finally: set_parameter_exclusions(raw, original) - def test_duplicate_function_ownership_fails_during_module_declaration() -> None: existing_owner = CellProfilerModule.require_module("Crop") diff --git a/tests/unit/test_cellprofiler_display_manual_projection_declarations.py b/tests/unit/test_cellprofiler_display_manual_projection_declarations.py index 818484e397..305a42a2ee 100644 --- a/tests/unit/test_cellprofiler_display_manual_projection_declarations.py +++ b/tests/unit/test_cellprofiler_display_manual_projection_declarations.py @@ -550,9 +550,7 @@ def test_display_callables_resolve_values_from_declared_measurement_tables() -> assert density.row_mappings()[0]["num_points"] == 2 assert histogram.row_mappings()[0]["total_count"] == 2 assert scatter.row_mappings()[0]["point_count"] == 2 - assert { - row["well_name"] for row in platemap.row_mappings() if "well_name" in row - } == { + assert {row["well_name"] for row in platemap.row_mappings() if "well_name" in row} == { "A01", "A02", } diff --git a/tests/unit/test_cellprofiler_export_equivalence.py b/tests/unit/test_cellprofiler_export_equivalence.py index 5adb3dc374..a2703aeaa3 100644 --- a/tests/unit/test_cellprofiler_export_equivalence.py +++ b/tests/unit/test_cellprofiler_export_equivalence.py @@ -224,9 +224,7 @@ def test_native_shards_merge_cpa_declared_image_and_object_rows( _write_sharded_native_output(second, (2,)) report = cellprofiler_native_shard_equivalence( - reference, - (first, second), - policy=RuntimeEquivalencePolicy(measurement_dialect=SAVED_OUTPUT_DIALECT), + reference, (first, second), policy=RuntimeEquivalencePolicy(measurement_dialect=SAVED_OUTPUT_DIALECT) ) assert report.is_equivalent @@ -267,9 +265,7 @@ def test_native_shards_merge_declared_relationship_rows_and_view( ) report = cellprofiler_native_shard_equivalence( - reference, - (first, second), - policy=RuntimeEquivalencePolicy(measurement_dialect=SAVED_OUTPUT_DIALECT), + reference, (first, second), policy=RuntimeEquivalencePolicy(measurement_dialect=SAVED_OUTPUT_DIALECT) ) assert report.is_equivalent @@ -284,9 +280,7 @@ def test_native_shards_reject_overlapping_image_sets(tmp_path: Path) -> None: _write_sharded_native_output(second, (1,)) report = cellprofiler_native_shard_equivalence( - reference, - (first, second), - policy=RuntimeEquivalencePolicy(measurement_dialect=SAVED_OUTPUT_DIALECT), + reference, (first, second), policy=RuntimeEquivalencePolicy(measurement_dialect=SAVED_OUTPUT_DIALECT) ) assert not report.is_equivalent @@ -311,9 +305,7 @@ def test_native_shards_reject_nonpartitionable_experiment_aggregate( ) report = cellprofiler_native_shard_equivalence( - reference, - (first, second), - policy=RuntimeEquivalencePolicy(measurement_dialect=SAVED_OUTPUT_DIALECT), + reference, (first, second), policy=RuntimeEquivalencePolicy(measurement_dialect=SAVED_OUTPUT_DIALECT) ) assert any( @@ -332,9 +324,7 @@ def test_native_shards_reject_undeclared_side_writes(tmp_path: Path) -> None: (second / "unexpected.txt").write_text("side write") report = cellprofiler_native_shard_equivalence( - reference, - (first, second), - policy=RuntimeEquivalencePolicy(measurement_dialect=SAVED_OUTPUT_DIALECT), + reference, (first, second), policy=RuntimeEquivalencePolicy(measurement_dialect=SAVED_OUTPUT_DIALECT) ) assert any("unexpected files" in message for message in report.failure_messages()) @@ -644,9 +634,7 @@ def test_database_export_equivalence_applies_numeric_tolerance( report = cellprofiler_database_export_equivalence( reference, RuntimeExportObservation.from_output_root(candidate), - policy=RuntimeEquivalencePolicy( - numeric_abs_tolerance=0.001, measurement_dialect=SAVED_OUTPUT_DIALECT - ), + policy=RuntimeEquivalencePolicy(numeric_abs_tolerance=0.001, measurement_dialect=SAVED_OUTPUT_DIALECT), ) assert report.is_equivalent @@ -904,9 +892,7 @@ def test_database_column_declarations_own_export_and_equivalence_names() -> None ) -def test_database_measurements_keep_complete_relationship_cohort( - tmp_path: Path, -) -> None: +def test_database_measurements_keep_complete_relationship_cohort(tmp_path: Path) -> None: import pytest from benchmark.cellprofiler_export_equivalence import _sqlite_database_differences @@ -915,9 +901,7 @@ def test_database_measurements_keep_complete_relationship_cohort( "Per_Parents": MeasurementSubject(MeasurementScope.OBJECT, "Parents"), "Per_Children": MeasurementSubject(MeasurementScope.OBJECT, "Children"), } - reference, candidate = ( - tmp_path / name for name in ("reference.db", "candidate.db") - ) + reference, candidate = (tmp_path / name for name in ("reference.db", "candidate.db")) for path, value in ((reference, 1.5), (candidate, 1.5 + 1e-13)): with sqlite3.connect(path) as connection: connection.execute( diff --git a/tests/unit/test_cellprofiler_full_stack_materialization.py b/tests/unit/test_cellprofiler_full_stack_materialization.py index 550d4713ae..325733c61f 100644 --- a/tests/unit/test_cellprofiler_full_stack_materialization.py +++ b/tests/unit/test_cellprofiler_full_stack_materialization.py @@ -5,14 +5,7 @@ import numpy as np import pytest -from openhcs.core.aligned_image_payload import ( - AlignedImageSliceContext, - AlignedImageStack, - ImageOutputBundle, - ImagePayloadStackComposition, - ImagePayloadSliceStack, - compose_aligned_image_payload, -) +from openhcs.core.aligned_image_payload import (AlignedImageSliceContext, AlignedImageStack, ImageOutputBundle, ImagePayloadStackComposition, ImagePayloadSliceStack, compose_aligned_image_payload) from openhcs.core.artifacts import ArtifactSpec, ImageArtifactType from openhcs.core.callable_contract import CallableContract, CallableMetadata from openhcs.core.measurement_row_materialization import MeasurementSparseColumnarRows @@ -21,10 +14,7 @@ MeasurementSubject, MeasurementTable, ) -from openhcs.core.runtime_object_labels import ( - ObjectLabelPayload, - ObjectLabelVariantData, -) +from openhcs.core.runtime_object_labels import ObjectLabelPayload, ObjectLabelVariantData from openhcs.core.runtime_image_values import ImagePayloadMetadata from openhcs.core.runtime_plane_projection import ( RuntimePlaneAxis, @@ -41,10 +31,7 @@ CellProfilerFunctionContractExecutor, ) from openhcs.processing.backends.cellprofiler.intensity import rescale_intensity -from openhcs.processing.backends.cellprofiler.morphology import ( - remove_holes, - remove_holes_3d, -) +from openhcs.processing.backends.cellprofiler.morphology import remove_holes, remove_holes_3d from openhcs.core.processing_contracts import ( ProcessingContract, Pure3DContract, @@ -78,9 +65,7 @@ def test_real_rescale_full_stack_materializes_composed_runtime_sources(): @pytest.mark.parametrize("function", (remove_holes, remove_holes_3d)) -def test_literal_volume_keeps_dense_contract_semantics_without_bundle_classification( - function, -): +def test_literal_volume_keeps_dense_contract_semantics_without_bundle_classification(function): pixels = np.ones((60, 6, 8), dtype=np.float32) pixels[20:22, 2:4, 3:5] = 0 mask = np.ones(pixels.shape, dtype=bool) @@ -88,8 +73,7 @@ def test_literal_volume_keeps_dense_contract_semantics_without_bundle_classifica metadata = ImagePayloadMetadata( plane_axis=RuntimePlaneAxis.RUNTIME_SLICE, source_voxel_spacing=SourceVoxelSpacing( - (1.25, 2.5, 3.0), - SourceVoxelSpacingUnit.MICROMETERS, + (1.25, 2.5, 3.0), SourceVoxelSpacingUnit.MICROMETERS, ), source_spatial_domain=SourceSpatialDomain(source_shape_yx=(6, 8)), source_image_provenance_planes=SourceImageProvenancePlanes.from_components( @@ -99,18 +83,12 @@ def test_literal_volume_keeps_dense_contract_semantics_without_bundle_classifica ) dense = metadata.payload_with(pixels, mask) projection = RuntimePlaneAxisValueProjection.preserve( - axis=RuntimePlaneAxis.RUNTIME_SLICE, - axis_size=60, + axis=RuntimePlaneAxis.RUNTIME_SLICE, axis_size=60, ) literal = ImagePayloadSliceStack.from_output_slices( - tuple( - RuntimeSliceProjection.value_for_slice( - dense, projection.selected_plane(index) - ) - for index in range(60) - ), - memory_type="numpy", - plane_axis=RuntimePlaneAxis.RUNTIME_SLICE, + tuple(RuntimeSliceProjection.value_for_slice(dense, projection.selected_plane(index)) + for index in range(60)), + memory_type="numpy", plane_axis=RuntimePlaneAxis.RUNTIME_SLICE, ) assert not isinstance(literal, AlignedImageStack) composition = compose_aligned_image_payload("literal volume", (literal,)) @@ -118,39 +96,24 @@ def test_literal_volume_keeps_dense_contract_semantics_without_bundle_classifica assert composition.plane_axis is RuntimePlaneAxis.RUNTIME_SLICE assert literal._composed_payload is None contract = CallableContract.from_callable(function) - contract = replace( - contract, - metadata=replace( - contract.metadata, - artifact_outputs=(ArtifactSpec.output("Filled", ImageArtifactType),), - ), - ) + contract = replace(contract, metadata=replace( + contract.metadata, artifact_outputs=(ArtifactSpec.output("Filled", ImageArtifactType),), + )) executor = CellProfilerFunctionContractExecutor() outputs = tuple( executor.execute( - contract, - contract.resolve_canonical_raw_callable(), - value, - {}, - execution_mode=compose_aligned_image_payload( - "volume", (value,) - ).execution_mode, + contract, contract.resolve_canonical_raw_callable(), value, {}, + execution_mode=compose_aligned_image_payload("volume", (value,)).execution_mode, plane_projection=projection, ) for value in (dense, literal) ) - np.testing.assert_array_equal( - ImagePayload.of(outputs[1]).data, ImagePayload.of(outputs[0]).data - ) - np.testing.assert_array_equal( - ImagePayload.of(outputs[1]).mask, ImagePayload.of(outputs[0]).mask - ) + np.testing.assert_array_equal(ImagePayload.of(outputs[1]).data, ImagePayload.of(outputs[0]).data) + np.testing.assert_array_equal(ImagePayload.of(outputs[1]).mask, ImagePayload.of(outputs[0]).mask) assert ImagePayload.of(outputs[1]).metadata == ImagePayload.of(outputs[0]).metadata -@pytest.mark.parametrize( - "processing_contract", tuple(ProcessingContract.__registry__.values()) -) +@pytest.mark.parametrize("processing_contract", tuple(ProcessingContract.__registry__.values())) @pytest.mark.parametrize("slice_count", (1, 2)) def test_every_full_stack_processing_family_materializes_image_and_image_kwargs( processing_contract, slice_count @@ -250,7 +213,12 @@ def test_aligned_materialization_retains_masks_calibration_and_exact_contributor } == set(expected_paths) np.testing.assert_array_equal( dense.mask, - np.stack([np.broadcast_to(plane.mask, (2, 3, 4)) for plane in composed.slices]), + np.stack( + [ + np.broadcast_to(plane.mask, (2, 3, 4)) + for plane in composed.slices + ] + ), ) for index in range(2): projected = RuntimeSliceProjection.value_for_slice( @@ -259,7 +227,9 @@ def test_aligned_materialization_retains_masks_calibration_and_exact_contributor axis=RuntimePlaneAxis.RUNTIME_SLICE, plane_index=index, axis_size=2 ), ) - np.testing.assert_array_equal(projected.data, composed.slices[index].data) + np.testing.assert_array_equal( + projected.data, composed.slices[index].data + ) expected = {f"/synthetic/A01_s{index + 1}_w{channel}.tif" for channel in (1, 2)} assert { identity.path @@ -278,10 +248,7 @@ def test_named_output_bundle_materializes_its_own_source_binding_axis(): ) dense = RuntimeSliceProjection.full_stack_value(bundle) assert dense.metadata.plane_axis is RuntimePlaneAxis.SOURCE_BINDING - assert dense.metadata.source_provenance.represented_source_image_names == ( - "red", - "green", - ) + assert dense.metadata.source_provenance.represented_source_image_names == ("red", "green") np.testing.assert_array_equal(dense.data, np.stack(planes)) assert all( slice_payload.data is plane @@ -322,7 +289,9 @@ def test_named_bundle_preserves_outer_aliases_with_real_inner_plane_provenance() axis=RuntimePlaneAxis.SOURCE_BINDING, plane_index=index, axis_size=2 ), ) - np.testing.assert_array_equal(selected.data, payloads[index].data) + np.testing.assert_array_equal( + selected.data, payloads[index].data + ) selected_metadata = selected.metadata assert selected_metadata.source_image_names == (alias,) assert { @@ -338,10 +307,8 @@ class CalibratedAlignedImageStack(AlignedImageStack): """A generated acquisition grid owns one additional metadata fact.""" def composition_payload_metadata(self, metadata): - return ( - super() - .composition_payload_metadata(metadata) - .replace_fields(source_voxel_spacing=spacing) + return super().composition_payload_metadata(metadata).replace_fields( + source_voxel_spacing=spacing ) data = np.arange(24, dtype=np.float32).reshape(2, 3, 4) @@ -417,9 +384,7 @@ def test_full_stack_preserves_nonimage_identity_domains_and_dense_images(): assert materialized[name] is value -@pytest.mark.parametrize( - "processing_contract", tuple(ProcessingContract.__registry__.values()) -) +@pytest.mark.parametrize("processing_contract", tuple(ProcessingContract.__registry__.values())) def test_full_stack_raw_callable_keeps_opaque_nonimage_kwargs(processing_contract): opaque = object() image = np.ones((2, 3, 4), dtype=np.float32) diff --git a/tests/unit/test_cellprofiler_function_contract_execution.py b/tests/unit/test_cellprofiler_function_contract_execution.py index 726e05ca53..26bca67424 100644 --- a/tests/unit/test_cellprofiler_function_contract_execution.py +++ b/tests/unit/test_cellprofiler_function_contract_execution.py @@ -13,11 +13,7 @@ from arraybridge import ArrayPayload import openhcs.core.callable_contract as callable_contract_module -from openhcs.core.aligned_image_payload import ( - AlignedImageStack, - ImageOutputBundle, - compose_aligned_image_payload, -) +from openhcs.core.aligned_image_payload import (AlignedImageStack, ImageOutputBundle, compose_aligned_image_payload) from openhcs.core.image_payload_execution_mode import ImagePayloadExecutionMode from openhcs.core.artifacts import ( ArtifactSpec, @@ -70,9 +66,8 @@ ) -def _execute_one_slice( - contract, raw, projection, value, kwargs, slice_index, slice_count -): + +def _execute_one_slice(contract, raw, projection, value, kwargs, slice_index, slice_count): """Run one plane of a per-plane contract in the CellProfiler environment.""" call = CellProfilerContractCall(contract, raw, projection) return contextualize_main_image_output( @@ -80,7 +75,6 @@ def _execute_one_slice( call.invoke(value, call.slice_kwargs(None, kwargs, slice_index, slice_count)), ) - def _compiled_contract( func: Callable[..., object], processing_contract: ProcessingContract, @@ -118,9 +112,7 @@ def test_morphological_skeleton_executes_one_planar_runtime_image() -> None: assert result.data.shape == image.shape -def test_registered_morph_distance_projects_raw_abi_and_retains_source_context() -> ( - None -): +def test_registered_morph_distance_projects_raw_abi_and_retains_source_context() -> None: contract = CallableContract.from_callable(morph) contract = replace( contract, @@ -152,8 +144,7 @@ def test_registered_morph_distance_projects_raw_abi_and_retains_source_context() {"operation": MorphOperation.DISTANCE}, execution_mode=NaturalExecution, plane_projection=RuntimePlaneAxisValueProjection.preserve( - axis=RuntimePlaneAxis.RUNTIME_SLICE, - axis_size=1, + axis=RuntimePlaneAxis.RUNTIME_SLICE, axis_size=1, ), ) @@ -183,8 +174,7 @@ def payload_raw(image: ArrayPayload, *, labels: np.ndarray) -> ArrayPayload: assert isinstance(image, ArrayPayload) seen.append((image, labels)) return image.metadata.payload_with( - image.data + labels, - image.mask, + image.data + labels, image.mask, ) raw = payload_raw if payload_annotation else array_raw @@ -197,29 +187,19 @@ def payload_raw(image: ArrayPayload, *, labels: np.ndarray) -> ArrayPayload: ), ) source = ImagePayloadMetadata(source_image_names=("DNA",)).payload_with( - np.ones((3, 4), dtype=np.float32), - np.ones((3, 4), dtype=bool), + np.ones((3, 4), dtype=np.float32), np.ones((3, 4), dtype=bool), ) selected_labels = np.full((3, 4), 5, dtype=np.float32) projection = RuntimePlaneAxisValueProjection.preserve( - axis=RuntimePlaneAxis.RUNTIME_SLICE, - axis_size=2, + axis=RuntimePlaneAxis.RUNTIME_SLICE, axis_size=2, ) assert contract.runtime_main_flow_call_argument(source) is source result = _execute_one_slice( - contract, - raw, - projection, - source, - { - "labels": RuntimeSliceAlignedValues( - slices=(np.zeros((3, 4)), selected_labels) - ), - "adapter_only_control": 1, - }, - 1, - 2, + contract, raw, projection, source, + {"labels": RuntimeSliceAlignedValues(slices=(np.zeros((3, 4)), selected_labels)), + "adapter_only_control": 1}, + 1, 2, ) assert seen[0][0] is (source if payload_annotation else source.data) @@ -236,12 +216,10 @@ def bad_shape(image: np.ndarray) -> np.ndarray: contract = _compiled_contract(bad_shape, Pure2DContract) source = ImagePayloadMetadata(source_image_names=("DNA",)).payload_with( - np.ones((3, 4)), - np.ones((3, 4), dtype=bool), + np.ones((3, 4)), np.ones((3, 4), dtype=bool), ) projection = RuntimePlaneAxisValueProjection.preserve( - axis=RuntimePlaneAxis.RUNTIME_SLICE, - axis_size=1, + axis=RuntimePlaneAxis.RUNTIME_SLICE, axis_size=1, ) with pytest.raises(ValueError, match="[Mm]ask.*shape|shape.*[Mm]ask"): _execute_one_slice(contract, bad_shape, projection, source, {}, 0, 1) @@ -285,16 +263,11 @@ def payload_raw(image: ArrayPayload) -> ArrayPayload: ), ) source = ImagePayloadMetadata(source_image_names=("DNA",)).payload_with( - np.ones((3, 4)), - None, + np.ones((3, 4)), None, ) result = CellProfilerFunctionContractExecutor().execute( - contract, - raw, - source, - {}, - execution_mode=execution_mode, + contract, raw, source, {}, execution_mode=execution_mode, ) assert seen == [source if payload_annotation else source.data] @@ -321,8 +294,7 @@ def payload_raw(image: ArrayPayload) -> ArrayPayload: ), ) def test_canonical_argument_projection_uses_declared_nominal_annotation( - annotation: object, - retains_payload: bool, + annotation: object, retains_payload: bool, ) -> None: def raw(image): return image @@ -331,8 +303,7 @@ def raw(image): raw.__annotations__["image"] = annotation contract = _compiled_contract(raw, Pure2DContract) source = ImagePayloadMetadata(source_image_names=("DNA",)).payload_with( - np.ones((3, 4)), - None, + np.ones((3, 4)), None, ) expected = source if retains_payload else source.data @@ -349,9 +320,7 @@ def raw(image): assert adapted.runtime_main_flow_call_argument(source) is source -def test_raw_slice_argument_retains_buffer_identity_for_inplace_array_callable() -> ( - None -): +def test_raw_slice_argument_retains_buffer_identity_for_inplace_array_callable() -> None: def inplace(image: np.ndarray) -> np.ndarray: image[0, 0] = 7 return image @@ -359,12 +328,10 @@ def inplace(image: np.ndarray) -> np.ndarray: contract = _compiled_contract(inplace, Pure2DContract) pixels = np.zeros((3, 4)) source = ImagePayloadMetadata(source_image_names=("DNA",)).payload_with( - pixels, - None, + pixels, None, ) projection = RuntimePlaneAxisValueProjection.preserve( - axis=RuntimePlaneAxis.RUNTIME_SLICE, - axis_size=1, + axis=RuntimePlaneAxis.RUNTIME_SLICE, axis_size=1, ) result = _execute_one_slice(contract, inplace, projection, source, {}, 0, 1) @@ -382,17 +349,14 @@ def raw(image: np.ndarray) -> np.ndarray: return image contract = _compiled_contract( - raw, - Pure2DContract, + raw, Pure2DContract, artifact_outputs=(ArtifactSpec.output("Result", ImageArtifactType),), ) source = ImagePayloadMetadata( - plane_axis=RuntimePlaneAxis.RUNTIME_SLICE, - source_image_names=("DNA",), + plane_axis=RuntimePlaneAxis.RUNTIME_SLICE, source_image_names=("DNA",), ).payload_with(np.ones((12, 3, 4)), None) projection = RuntimePlaneAxisValueProjection.preserve( - axis=RuntimePlaneAxis.RUNTIME_SLICE, - axis_size=12, + axis=RuntimePlaneAxis.RUNTIME_SLICE, axis_size=12, ) executor = CellProfilerFunctionContractExecutor() hints = [] @@ -417,10 +381,7 @@ def counted_signature(func, **kwargs): raw.__annotations__["image"] = annotation contract = contract.with_prepared_signature() result = executor.execute( - contract, - raw, - source, - {}, + contract, raw, source, {}, execution_mode=NaturalExecution, plane_projection=projection, ) @@ -442,31 +403,23 @@ def reject_premature_annotations(self): raise AssertionError("Input guard resolved the raw ABI prematurely") monkeypatch.setattr( - CallableContract, - "canonical_parameter_declarations", + CallableContract, "canonical_parameter_declarations", property(reject_premature_annotations), ) with pytest.raises(TypeError, match="requires AlignedImageStack"): CellProfilerFunctionContractExecutor().execute( - contract, - raw, - np.zeros((2, 3)), - {}, + contract, raw, np.zeros((2, 3)), {}, execution_mode=AlignedStackExecution, ) projection = RuntimePlaneAxisValueProjection.preserve( - axis=RuntimePlaneAxis.RUNTIME_SLICE, - axis_size=2, + axis=RuntimePlaneAxis.RUNTIME_SLICE, axis_size=2, ) mismatched_image = ImagePayloadMetadata( plane_axis=RuntimePlaneAxis.SOURCE_BINDING, ).payload_with(np.zeros((2, 3, 4)), None) with pytest.raises(RuntimeSliceProjectionDeclarationError, match="conflicts"): CellProfilerFunctionContractExecutor().execute( - contract, - raw, - mismatched_image, - {}, + contract, raw, mismatched_image, {}, execution_mode=NaturalExecution, plane_projection=projection, ) @@ -481,25 +434,16 @@ def raw(image: np.ndarray) -> np.ndarray: return image contract = _compiled_contract( - raw, - Pure2DContract, + raw, Pure2DContract, artifact_outputs=(ArtifactSpec.output("Result", ImageArtifactType),), ) - source = ImagePayloadMetadata( - plane_axis=RuntimePlaneAxis.RUNTIME_SLICE - ).payload_with( - np.ones((2, 3, 4)), - None, + source = ImagePayloadMetadata(plane_axis=RuntimePlaneAxis.RUNTIME_SLICE).payload_with( + np.ones((2, 3, 4)), None, ) result = CellProfilerFunctionContractExecutor().execute( - contract, - raw, - source, - {}, - execution_mode=NaturalExecution, + contract, raw, source, {}, execution_mode=NaturalExecution, plane_projection=RuntimePlaneAxisValueProjection.preserve( - axis=RuntimePlaneAxis.RUNTIME_SLICE, - axis_size=2, + axis=RuntimePlaneAxis.RUNTIME_SLICE, axis_size=2, ), ) @@ -512,107 +456,66 @@ def foreign_raw(image: ArrayPayload) -> ArrayPayload: foreign_contract = _compiled_contract(foreign_raw, Pure2DContract) policy = RuntimeCallablePolicy(kwarg_policy=SignatureFilteredKwargs) - assert ( - policy.contract_invocation(foreign_contract, foreign_raw, source, {}).call() - is source - ) + assert policy.contract_invocation(foreign_contract, foreign_raw, source, {}).call() is source -def test_actual_prepared_executor_planes_make_no_signature_or_hint_queries( - monkeypatch, -) -> None: +def test_actual_prepared_executor_planes_make_no_signature_or_hint_queries(monkeypatch) -> None: calls = [] - def raw(image: np.ndarray, scale: int = 2) -> np.ndarray: calls.append(image.shape) return image * scale - CallableProjection.from_callable(raw).warm_canonical_signature() contract = _compiled_contract( - raw, - Pure2DContract, - artifact_outputs=(ArtifactSpec.output("Result", ImageArtifactType),), + raw, Pure2DContract, + artifact_outputs=(ArtifactSpec.output("Result",ImageArtifactType),), ) source = ImagePayloadMetadata( - plane_axis=RuntimePlaneAxis.RUNTIME_SLICE, - source_image_names=("DNA",), - ).payload_with(np.ones((12, 3, 4)), None) - projection = RuntimePlaneAxisValueProjection.preserve( - axis=RuntimePlaneAxis.RUNTIME_SLICE, axis_size=12 - ) - + plane_axis=RuntimePlaneAxis.RUNTIME_SLICE,source_image_names=("DNA",), + ).payload_with(np.ones((12,3,4)),None) + projection = RuntimePlaneAxisValueProjection.preserve(axis=RuntimePlaneAxis.RUNTIME_SLICE,axis_size=12) def forbidden(*args, **kwargs): raise AssertionError("Prepared execution introspected a live callable") - - monkeypatch.setattr(inspect, "signature", forbidden) - monkeypatch.setattr(callable_contract_module, "get_type_hints", forbidden) + monkeypatch.setattr(inspect,"signature",forbidden) + monkeypatch.setattr(callable_contract_module,"get_type_hints",forbidden) result = CellProfilerFunctionContractExecutor().execute( - contract, - raw, - source, - {}, - execution_mode=NaturalExecution, + contract,raw,source,{},execution_mode=NaturalExecution, plane_projection=projection, ) - np.testing.assert_array_equal(result.data, np.full((12, 3, 4), 2)) - assert calls == [(3, 4)] * 12 + np.testing.assert_array_equal(result.data,np.full((12,3,4),2)) + assert calls == [(3,4)]*12 -def test_authored_distinct_raw_abi_batch_has_no_runtime_queries_and_filters_controls( - monkeypatch, -) -> None: +def test_authored_distinct_raw_abi_batch_has_no_runtime_queries_and_filters_controls(monkeypatch) -> None: from openhcs.core.processing_preparation import CallablePreparation - calls = [] - def raw(image: np.ndarray, scale: int = 2) -> np.ndarray: calls.append(image.shape) return image * scale - @wraps(raw) def wrapper(*args, **kwargs): return raw(*args, **kwargs) - - wrapper.__signature__ = inspect.signature(raw).replace( - parameters=[ - *inspect.signature(raw).parameters.values(), - inspect.Parameter( - "injected", inspect.Parameter.KEYWORD_ONLY, default=False - ), - ] - ) - monkeypatch.setattr(CallablePreparation, "prepare", lambda self: None) + wrapper.__signature__ = inspect.signature(raw).replace(parameters=[ + *inspect.signature(raw).parameters.values(), + inspect.Parameter("injected",inspect.Parameter.KEYWORD_ONLY,default=False), + ]) + monkeypatch.setattr(CallablePreparation,"prepare",lambda self:None) contract = CallableContract.from_prepared_callable(wrapper) - contract = replace( - contract, - metadata=replace( - contract.metadata, - processing_contract=Pure2DContract, - artifact_outputs=(ArtifactSpec.output("Result", ImageArtifactType),), - ), - ) - source = ImagePayloadMetadata( - plane_axis=RuntimePlaneAxis.RUNTIME_SLICE, source_image_names=("DNA",) - ).payload_with(np.ones((12, 3, 4)), None) - projection = RuntimePlaneAxisValueProjection.preserve( - axis=RuntimePlaneAxis.RUNTIME_SLICE, axis_size=12 - ) - + contract = replace(contract,metadata=replace( + contract.metadata,processing_contract=Pure2DContract, + artifact_outputs=(ArtifactSpec.output("Result",ImageArtifactType),), + )) + source = ImagePayloadMetadata(plane_axis=RuntimePlaneAxis.RUNTIME_SLICE,source_image_names=("DNA",)).payload_with(np.ones((12,3,4)),None) + projection = RuntimePlaneAxisValueProjection.preserve(axis=RuntimePlaneAxis.RUNTIME_SLICE,axis_size=12) def forbidden(*args, **kwargs): raise AssertionError("Prepared authored raw target queried a live signature") - - monkeypatch.setattr(inspect, "signature", forbidden) - monkeypatch.setattr(callable_contract_module, "get_type_hints", forbidden) + monkeypatch.setattr(inspect,"signature",forbidden) + monkeypatch.setattr(callable_contract_module,"get_type_hints",forbidden) result = CellProfilerFunctionContractExecutor().execute( - contract, - wrapper, - source, - {"injected": True}, - execution_mode=NaturalExecution, - plane_projection=projection, + contract,wrapper,source,{"injected":True}, + execution_mode=NaturalExecution,plane_projection=projection, ) - np.testing.assert_array_equal(result.data, np.full((12, 3, 4), 2)) - assert calls == [(3, 4)] * 12 + np.testing.assert_array_equal(result.data,np.full((12,3,4),2)) + assert calls == [(3,4)]*12 def test_compiled_slice_execution_resolves_raw_target_once_and_preserves_request_binding( @@ -698,11 +601,9 @@ def decorated(*args: object, **kwargs: object) -> object: signature = inspect.signature(raw) decorated.__signature__ = signature.replace( parameters=( - ( - parameter.replace(default=canonical_default) - if parameter.name == "slice_by_slice" - else parameter - ) + parameter.replace(default=canonical_default) + if parameter.name == "slice_by_slice" + else parameter for parameter in signature.parameters.values() ) ) @@ -723,12 +624,14 @@ def decorated(*args: object, **kwargs: object) -> object: ), ) - assert seen == ([((4, 5), True)] * 3 if canonical_default else [((3, 4, 5), False)]) + assert seen == ( + [((4, 5), True)] * 3 + if canonical_default + else [((3, 4, 5), False)] + ) -@pytest.mark.parametrize( - "processing_contract", tuple(ProcessingContract.__registry__.values()) -) +@pytest.mark.parametrize("processing_contract", tuple(ProcessingContract.__registry__.values())) @pytest.mark.parametrize( ("image_mode", "expected_call_count"), ( @@ -768,7 +671,10 @@ def dispatch_probe(image: np.ndarray) -> np.ndarray: ) ) - if image_mode is AlignedStackExecution and processing_contract is Pure3DContract: + if ( + image_mode is AlignedStackExecution + and processing_contract is Pure3DContract + ): with pytest.raises( ValueError, match=( diff --git a/tests/unit/test_cellprofiler_geometry_prepared_payload_abi.py b/tests/unit/test_cellprofiler_geometry_prepared_payload_abi.py index 03801cf2de..4a9e6d8e65 100644 --- a/tests/unit/test_cellprofiler_geometry_prepared_payload_abi.py +++ b/tests/unit/test_cellprofiler_geometry_prepared_payload_abi.py @@ -43,8 +43,7 @@ def _prepared_contract(func): def test_prepared_resize_preserves_typed_primary_and_bare_array(func): pixels = np.arange(24, dtype=np.float32).reshape(4, 6) source = ImagePayloadMetadata( - source_path="/images/input.tif", - source_image_names=("Original",), + source_path="/images/input.tif", source_image_names=("Original",), ).payload_with(pixels, pixels % 2 == 0) contract = _prepared_contract(func) @@ -66,14 +65,9 @@ def test_prepared_full_stack_resize_preserves_existing_metadata_and_mask( ImagePayloadMetadata( source_path=f"/images/z{index}.tif", source_image_names=("Original",), - source_component_metadata={ - "z_index": index, - "timepoint": 1, - "well": "W001", - }, + source_component_metadata={"z_index": index, "timepoint": 1, "well": "W001"}, source_spatial_domain=SourceSpatialDomain( - origin_yx=(2, 3), - source_shape_yx=(8, 10), + origin_yx=(2, 3), source_shape_yx=(8, 10), ), ).payload_with(np.full((4, 6), index, dtype=np.float32), None) for index in (3, 1, 2) @@ -95,22 +89,21 @@ def test_prepared_full_stack_resize_preserves_existing_metadata_and_mask( contract = _prepared_contract(resize_volumetric) canonical = contract.resolve_canonical_raw_callable() kwargs = dict( - resizing_factor_x=0.5, - resizing_factor_y=0.5, - resizing_factor_z=z_factor, + resizing_factor_x=0.5, resizing_factor_y=0.5, resizing_factor_z=z_factor, ) expected = canonical(source, **kwargs) result = CellProfilerFunctionContractExecutor().execute( - contract, - canonical, - source, - kwargs, + contract, canonical, source, kwargs, execution_mode=FullStackExecution, ) - np.testing.assert_array_equal(result.data, expected.data) - np.testing.assert_array_equal(result.mask, expected.mask) + np.testing.assert_array_equal( + result.data, expected.data + ) + np.testing.assert_array_equal( + result.mask, expected.mask + ) assert result.metadata == expected.metadata assert result.metadata.plane_axis is axis assert result.metadata.source_image_paths == metadata.source_image_paths @@ -136,10 +129,7 @@ def test_prepared_resize_bare_pixels_preserve_canonical_return_contract(func): expected = canonical(pixels, **kwargs) result = CellProfilerFunctionContractExecutor().execute( - contract, - canonical, - pixels, - kwargs, + contract, canonical, pixels, kwargs, execution_mode=FullStackExecution, ) diff --git a/tests/unit/test_cellprofiler_identify_secondary_replacement_primary.py b/tests/unit/test_cellprofiler_identify_secondary_replacement_primary.py index ca7552197b..57baed1854 100644 --- a/tests/unit/test_cellprofiler_identify_secondary_replacement_primary.py +++ b/tests/unit/test_cellprofiler_identify_secondary_replacement_primary.py @@ -229,10 +229,7 @@ def test_replacement_primary_runtime_emits_typed_labels_and_relationships() -> N primary_labels = SourceImageObjectLabelBuildRequest( image=image, labels=primary_array, - variants={ - UneditedLabels: primary_array.copy(), - SmallRemovedLabels: primary_array.copy(), - }, + variants={UneditedLabels: primary_array.copy(), SmallRemovedLabels: primary_array.copy()}, declared_object_ids=(1, 2), ).payload() @@ -302,9 +299,7 @@ def test_replacement_primary_runtime_emits_typed_labels_and_relationships() -> N variantless_replacement = variantless_result[-1] assert isinstance(variantless_replacement, ObjectLabelValue) assert variantless_replacement.variant_data.variant_labels(UneditedLabels) is None - assert ( - variantless_replacement.variant_data.variant_labels(SmallRemovedLabels) is None - ) + assert variantless_replacement.variant_data.variant_labels(SmallRemovedLabels) is None assert ( variantless_replacement.source_provenance == primary_without_variants.source_provenance diff --git a/tests/unit/test_cellprofiler_image_quality.py b/tests/unit/test_cellprofiler_image_quality.py index 14eb0df4e3..d5716ae0df 100644 --- a/tests/unit/test_cellprofiler_image_quality.py +++ b/tests/unit/test_cellprofiler_image_quality.py @@ -544,9 +544,7 @@ def test_quality_control_measurements_keep_distinct_site_image_identities() -> N object_name=None, source_image_name=None, source_metadata=ImagePayloadMetadata( - source_provenance=MeasurementTable.joined_source_provenance( - "quality_metrics", tables - ), + source_provenance=MeasurementTable.joined_source_provenance("quality_metrics", tables), ), ) @@ -708,6 +706,8 @@ def test_running_variance_preserves_numpy_accumulation(dtype, reverse): if reverse: values = values[::-1] means = values.cumsum() / np.arange(1, len(values) + 1) - accumulated = ((values[1:] - means[:-1]) * (values[1:] - means[1:])).cumsum() + accumulated = ( + (values[1:] - means[:-1]) * (values[1:] - means[1:]) + ).cumsum() expected = np.hstack(([0], accumulated / np.arange(1, len(values)))) np.testing.assert_array_equal(running_variance_numba(values), expected) diff --git a/tests/unit/test_cellprofiler_library_loading.py b/tests/unit/test_cellprofiler_library_loading.py index 5f6cec73fd..19af8406fa 100644 --- a/tests/unit/test_cellprofiler_library_loading.py +++ b/tests/unit/test_cellprofiler_library_loading.py @@ -11,10 +11,7 @@ import skimage.morphology import skimage.segmentation from python_introspect import declared_enum_type -from openhcs.core.aligned_image_payload import ( - AlignedImageStack, - ImagePayloadBundleContext, -) +from openhcs.core.aligned_image_payload import (AlignedImageStack, ImagePayloadBundleContext) from openhcs.core.callable_contract import CallableContract from openhcs.core.runtime_tabular_values import FieldSpec from openhcs.processing.backends.cellprofiler.alignment import ( @@ -202,6 +199,7 @@ ) + def test_module_registry_resolves_every_declared_function(): for module_type in CellProfilerModule.__registry__.values(): if not module_type.emits_function_step(): @@ -263,14 +261,16 @@ def test_measure_colocalization_declares_composed_stack_execution() -> None: def test_track_objects_callable_requires_timepoint(): track_objects = CellProfilerModule.__registry__["TrackObjects"].require_callable() - assert CallableContract.from_callable(track_objects).required_axis_roles == ( - TimeAxis, - ) + assert CallableContract.from_callable( + track_objects + ).required_axis_roles == (TimeAxis,) def test_correct_illumination_apply_inherits_resolved_stack_grouping() -> None: assert ( - CallableContract.from_callable(correct_illumination_apply).required_axis_roles + CallableContract.from_callable( + correct_illumination_apply + ).required_axis_roles == () ) assert ( @@ -280,7 +280,9 @@ def test_correct_illumination_apply_inherits_resolved_stack_grouping() -> None: def test_mask_objects_has_no_unconditional_image_stack_axis() -> None: - assert CallableContract.from_callable(mask_objects).required_axis_roles == () + assert ( + CallableContract.from_callable(mask_objects).required_axis_roles == () + ) def test_all_cellprofiler_module_aliases_canonicalize(): @@ -900,9 +902,7 @@ def test_edge_direction_round_trips_through_compiled_worker_transport(direction) assert EdgeDirection(direction.value) is direction restored = pickle.loads(pickle.dumps(direction, protocol=pickle.HIGHEST_PROTOCOL)) assert restored is direction - assert ( - restored.includes_horizontal_response == direction.includes_horizontal_response - ) + assert restored.includes_horizontal_response == direction.includes_horizontal_response assert restored.includes_vertical_response == direction.includes_vertical_response @@ -2103,9 +2103,7 @@ def test_watershed_xy_downsample_factors_preserve_leading_axes(): assert watershed_xy_downsample_factors(2, 2) == (2.0, 2.0) assert watershed_xy_downsample_factors(3, 2) == (1.0, 2.0, 2.0) assert watershed_xy_downsample_factors(4, 2) == (1.0, 1.0, 2.0, 2.0) - from openhcs.processing.backends.cellprofiler.morphology import ( - MorphologyBackendStrategy, - ) + from openhcs.processing.backends.cellprofiler.morphology import MorphologyBackendStrategy labels = MorphologyBackendStrategy.for_memory_type().label_equal_values( np.ones((2, 3, 4, 5), dtype=bool) @@ -3708,7 +3706,10 @@ def test_absorbed_processing_contract_metadata_does_not_act_as_validator(): image = np.ones((8, 8), dtype=np.float32) result = correct_illumination_calculate(image, dtype_config=DtypeConfig()) assert result.shape == image.shape - assert correct_illumination_calculate.__processing_contract__ is FlexibleContract + assert ( + correct_illumination_calculate.__processing_contract__ + is FlexibleContract + ) assert opening.__processing_contract__ is FlexibleContract @@ -4362,9 +4363,7 @@ def flexible(image): def test_unmix_colors_returns_one_output_per_stain_row(): image = np.full((8, 9, 3), 0.5, dtype=np.float32) outputs = unmix_colors( - ImagePayloadMetadata(axes=PayloadAxes.colour_samples(-1)).payload_with( - image, None - ), + ImagePayloadMetadata(axes=PayloadAxes.colour_samples(-1)).payload_with(image, None), stain_names=(StainType.HEMATOXYLIN, StainType.EOSIN, StainType.CUSTOM), custom_absorbances=((0.5, 0.5, 0.5), (0.5, 0.5, 0.5), (0.1, 0.2, 0.3)), dtype_config=DtypeConfig(), @@ -4373,7 +4372,8 @@ def test_unmix_colors_returns_one_output_per_stain_row(): assert [output.shape for output in outputs.slices] == [(8, 9), (8, 9), (8, 9)] assert all((output.dtype == np.float32 for output in outputs.slices)) assert all( - output.metadata.axis_position(ColourAxis) is None for output in outputs.slices + output.metadata.axis_position(ColourAxis) is None + for output in outputs.slices ) assert unmix_colors.__processing_contract__ is FlexibleContract @@ -4408,9 +4408,9 @@ def test_binary_mask_output_preserves_declared_color_channel_axis() -> None: def test_crop_preserves_hwc_color_image_domain() -> None: image = np.arange(8 * 9 * 3, dtype=np.uint8).reshape(8, 9, 3) - image_payload = ImagePayloadMetadata( - axes=PayloadAxes.colour_samples(-1) - ).payload_with(image, None) + image_payload = ImagePayloadMetadata(axes=PayloadAxes.colour_samples(-1)).payload_with( + image, None + ) cropped, mask, measurements = crop( image_payload, removal_method=CropModule.RemovalMethod.ALL, @@ -4539,7 +4539,10 @@ def test_measure_image_area_occupied_runs_mixed_rows(): "DNA", "Nuclei", ] - assert measure_image_area_occupied.__processing_contract__ is FlexibleContract + assert ( + measure_image_area_occupied.__processing_contract__ + is FlexibleContract + ) def test_measure_image_area_occupied_uses_runtime_slice_for_every_operand_row(): @@ -5076,10 +5079,7 @@ def test_overlay_outlines_renders_exact_projected_empty_label_plane(): ), ) def test_measurement_batch_admission_uses_declared_processing_domain( - contract, - mode, - slice_by_slice, - admitted, + contract, mode, slice_by_slice, admitted, ) -> None: image = np.stack((np.ones((2, 3)), np.full((2, 3), 2.0))) request = RuntimeBatchInvocationRequest( diff --git a/tests/unit/test_cellprofiler_measurement_rows.py b/tests/unit/test_cellprofiler_measurement_rows.py index 335e731b63..21ff0d08bc 100644 --- a/tests/unit/test_cellprofiler_measurement_rows.py +++ b/tests/unit/test_cellprofiler_measurement_rows.py @@ -695,9 +695,7 @@ def test_concatenated_measurement_rows_preserve_structural_missing_cells() -> No ) -def test_declared_wide_admission_preserves_features_before_absent_object_column() -> ( - None -): +def test_declared_wide_admission_preserves_features_before_absent_object_column() -> None: rows = MeasurementSparseColumnarRows.from_rows( ({"slice_index": 1, "contrast": 0.25, "entropy": 0.75},), fields=( @@ -859,9 +857,7 @@ def test_wide_measurement_projection_uses_row_owned_scope_for_artifact_table() - @pytest.mark.parametrize("layout", ["identity", "wide", "long"]) def test_measurement_row_contract_owns_output_identity_without_renumbering(layout): - from openhcs.interop.cellprofiler.measurement_dialect import ( - CELLPROFILER_MEASUREMENT_DIALECT, - ) + from openhcs.interop.cellprofiler.measurement_dialect import CELLPROFILER_MEASUREMENT_DIALECT columns = {"slice_index": (1, 1), "object_label": (2, 17)} if layout == "wide": @@ -870,10 +866,8 @@ def test_measurement_row_contract_owns_output_identity_without_renumbering(layou columns.update(feature_name=("Area", "Area"), result_value=(3.0, 4.0)) rows = MeasurementSparseColumnarRows( columns, - fields=tuple( - FieldSpec(name, str if name == "feature_name" else float) - for name in columns - ), + fields=tuple(FieldSpec(name, str if name == "feature_name" else float) + for name in columns), ) for contract, output_name in ( (PlainMeasurementDialect.row_identity_contract, "object_label"), @@ -881,15 +875,10 @@ def test_measurement_row_contract_owns_output_identity_without_renumbering(layou ): accumulator = WideMeasurementRowAccumulator(contract) accumulator.add( - rows, - lambda feature, qualifiers: feature, - default_subject="Objects", - default_scope=MeasurementScope.OBJECT, + rows, lambda feature, qualifiers: feature, + default_subject="Objects", default_scope=MeasurementScope.OBJECT, ) projected = accumulator.row_mappings_by_subject()["Objects"] assert tuple(row[output_name] for row in projected) == (2, 17) - assert all( - ("object_label" if output_name == "object_number" else "object_number") - not in row - for row in projected - ) + assert all(("object_label" if output_name == "object_number" else "object_number") + not in row for row in projected) diff --git a/tests/unit/test_cellprofiler_module_execution.py b/tests/unit/test_cellprofiler_module_execution.py index b46bc4a2bd..610c22a693 100644 --- a/tests/unit/test_cellprofiler_module_execution.py +++ b/tests/unit/test_cellprofiler_module_execution.py @@ -20,16 +20,7 @@ DirectParentReferenceMeasurementFeature, ) from openhcs.constants.constants import Backend, MemoryType -from openhcs.core.aligned_image_payload import ( - AlignedImageSliceContext, - AlignedImageStack, - ProducedImageStack, - ImageOutputBundle, - ImagePayloadBundleContext, - aligned_image_stack_kwargs, - compose_aligned_image_payload, - pack_aligned_image_outputs, -) +from openhcs.core.aligned_image_payload import (AlignedImageSliceContext, AlignedImageStack, ProducedImageStack, ImageOutputBundle, ImagePayloadBundleContext, aligned_image_stack_kwargs, compose_aligned_image_payload, pack_aligned_image_outputs) from openhcs.core.image_payload_execution_mode import ImagePayloadExecutionMode from openhcs.core.artifacts import ( ArtifactInputPlan, @@ -75,12 +66,7 @@ DataclassMeasurementColumnarRows, MeasurementSparseColumnarRows, ) -from openhcs.core.pipeline.function_contracts import ( - object_label_input_execution_mode, - object_label_input_execution_mode_from_callable, - runtime_bound_parameters, - special_inputs, -) +from openhcs.core.pipeline.function_contracts import (object_label_input_execution_mode, object_label_input_execution_mode_from_callable, runtime_bound_parameters, special_inputs) from openhcs.core.runtime_artifact_queries import ( MeasurementTableAxisProjection, RuntimeArtifactQueryContext, @@ -481,8 +467,7 @@ def reject_authored_preparation(*args, **kwargs): ) executor = CellProfilerModuleExecutor(raw, contract) image_request = executor._image_request( - image, - adapter, + image, adapter, module_type=executor.module_type(), active_input_specs=contract.artifact_inputs.specs, ) @@ -490,7 +475,6 @@ def reject_authored_preparation(*args, **kwargs): assert image_request.source_aliases == (source_spec.name,) np.testing.assert_array_equal(image_request.payload.data, image) - def test_default_invocation_keeps_compiled_source_bindings_outside_anchor_group() -> ( None ): @@ -1419,7 +1403,9 @@ def test_image_artifact_resolution_uses_declared_artifact_alias( ).payload_with(np.ones((4, 5), dtype=np.float32), None) payload = CellProfilerOutputRecorder.for_artifact_type( ImageArtifactType - ).raw_runtime_input_value(spec=spec, value=source) + ).raw_runtime_input_value( + spec=spec, value=source + ) metadata = payload.metadata assert metadata.source_image_names == ("IllumStain1",) @@ -1479,12 +1465,11 @@ def test_main_flow_image_artifact_selects_declared_alias_plane() -> None: payload = CellProfilerOutputRecorder.for_artifact_type( ImageArtifactType ).raw_runtime_input_value( - first_spec, - RuntimeInputBindingRequest( + first_spec, RuntimeInputBindingRequest( adapter=adapter, kwargs={}, current_image=current_image, - ).artifact_value_for_spec(first_spec), + ).artifact_value_for_spec(first_spec) ) np.testing.assert_array_equal(payload.data, np.zeros((4, 5))) @@ -1520,8 +1505,7 @@ def test_single_main_flow_image_artifact_selects_declared_source_binding_plane() payload = CellProfilerOutputRecorder.for_artifact_type( ImageArtifactType ).raw_runtime_input_value( - spec, - RuntimeInputBindingRequest( + spec, RuntimeInputBindingRequest( adapter=_FakeCellProfilerRuntime( {}, callable_contract=contract, @@ -1534,7 +1518,7 @@ def test_single_main_flow_image_artifact_selects_declared_source_binding_plane() ), kwargs={}, current_image=current_image, - ).artifact_value_for_spec(spec), + ).artifact_value_for_spec(spec) ) np.testing.assert_array_equal(payload.data, np.zeros((4, 5))) @@ -1572,8 +1556,7 @@ def test_main_flow_image_artifact_projects_named_provenance_plane() -> None: payload = CellProfilerOutputRecorder.for_artifact_type( ImageArtifactType ).raw_runtime_input_value( - first_spec, - RuntimeInputBindingRequest( + first_spec, RuntimeInputBindingRequest( adapter=_FakeCellProfilerRuntime( {}, callable_contract=contract, @@ -1586,7 +1569,7 @@ def test_main_flow_image_artifact_projects_named_provenance_plane() -> None: ), kwargs={}, current_image=current_image, - ).artifact_value_for_spec(first_spec), + ).artifact_value_for_spec(first_spec) ) np.testing.assert_array_equal(payload.data, np.zeros((4, 5))) @@ -1618,8 +1601,7 @@ def test_main_flow_image_artifact_preserves_declared_runtime_slice_stack() -> No payload = CellProfilerOutputRecorder.for_artifact_type( ImageArtifactType ).raw_runtime_input_value( - first_spec, - RuntimeInputBindingRequest( + first_spec, RuntimeInputBindingRequest( adapter=_FakeCellProfilerRuntime( {}, callable_contract=contract, @@ -1632,10 +1614,12 @@ def test_main_flow_image_artifact_preserves_declared_runtime_slice_stack() -> No ), kwargs={}, current_image=current_image, - ).artifact_value_for_spec(first_spec), + ).artifact_value_for_spec(first_spec) ) - np.testing.assert_array_equal(payload.data, current_image.data) + np.testing.assert_array_equal( + payload.data, current_image.data + ) assert payload.metadata.source_image_names == ("Stain1",) @@ -1725,9 +1709,7 @@ def record_resolution(self, spec: ArtifactSpec, value): assert runtime_kwargs["illumination_function"].data.shape == (4, 5) -def test_composed_measurement_images_keep_declared_aliases_and_resolved_payload() -> ( - None -): +def test_composed_measurement_images_keep_declared_aliases_and_resolved_payload() -> None: image_spec = ArtifactSpec.input("DeclaredImage", ImageArtifactType) contract = _compiled_callable_contract( CellProfilerModule.require_module("MeasureObjectIntensity").require_callable(), @@ -1768,9 +1750,7 @@ def test_composed_measurement_images_keep_declared_aliases_and_resolved_payload( assert measurement_image.execution_mode is image_request.execution_mode assert measurement_image.plane_projection is image_request.plane_projection assert not measurement_image.align_to_labels - with pytest.raises( - ValueError, match="requires a composed measurement image request" - ): + with pytest.raises(ValueError, match="requires a composed measurement image request"): executor._measurement_image_inputs( runtime, image, @@ -1891,8 +1871,7 @@ def test_single_main_flow_image_preserves_runtime_slice_axis() -> None: payload = CellProfilerOutputRecorder.for_artifact_type( ImageArtifactType ).raw_runtime_input_value( - spec, - RuntimeInputBindingRequest( + spec, RuntimeInputBindingRequest( adapter=_FakeCellProfilerRuntime( {}, callable_contract=contract, @@ -1905,7 +1884,7 @@ def test_single_main_flow_image_preserves_runtime_slice_axis() -> None: ), kwargs={}, current_image=current_image, - ).artifact_value_for_spec(spec), + ).artifact_value_for_spec(spec) ) assert payload.data.shape == (1, 4, 5) @@ -2916,10 +2895,7 @@ def test_watershed_output_source_policy_uses_declared_image_as_parent() -> None: image_payload.data, ) assert source_context.parent_image_payload is source_context.source_payload - assert ( - source_context.source_payload.metadata.source_voxel_spacing - == SourceVoxelSpacing((2.0, 1.0, 1.0)) - ) + assert source_context.source_payload.metadata.source_voxel_spacing == SourceVoxelSpacing((2.0, 1.0, 1.0)) def test_object_label_recorder_suppresses_parent_spacing_when_policy_declares_no_parent_image() -> ( @@ -5015,10 +4991,7 @@ def test_object_row_binding_honors_authoritative_singleton_runtime_projection( assert projection is not None and projection.plane_index is None measurement_values = RuntimeSliceAlignedValues((np.asarray([0.25]),)) projected = RuntimeSliceProjection.kwargs_for_slice( - { - "object_labels": (bound_labels,), - "measurement_values": measurement_values, - }, + {"object_labels": (bound_labels,), "measurement_values": measurement_values}, projection.selected_plane(0), ) assert object_label_dense_array(projected["object_labels"][0]).shape == (2, 2) @@ -6823,27 +6796,20 @@ def test_multi_image_photometry_consumes_each_named_current_source(aliases): ).payload_with(np.stack(tuple(images[name] for name in aliases))) image_specs = tuple(ArtifactSpec.input(name, ImageArtifactType) for name in aliases) contract = _compiled_callable_contract( - MeasureObjectIntensityModule.require_callable(), - artifact_inputs=image_specs, + MeasureObjectIntensityModule.require_callable(), artifact_inputs=image_specs, ) executor = _module_executor(contract) runtime = _FakeCellProfilerRuntime( - {}, - callable_contract=contract, + {}, callable_contract=contract, source_bindings=tuple(NamedSourceBinding(alias=name) for name in aliases), - artifact_input_edges=tuple( - _artifact_input_edge_for_test(spec, stored=False) for spec in image_specs - ), + artifact_input_edges=tuple(_artifact_input_edge_for_test(spec, stored=False) for spec in image_specs), ) measurement_images = executor._resolved_measurement_images( - image_specs, - runtime, - payload, + image_specs, runtime, payload, reference_domain=CellProfilerMeasurementImageDomain.OBJECT_LABELS, ) labels = ObjectLabelSet( - name="Cells", - variant_data=ObjectLabelVariantData(labels=np.ones((4, 5), dtype=np.int32)), + name="Cells", variant_data=ObjectLabelVariantData(labels=np.ones((4, 5), dtype=np.int32)), domain=ObjectLabelDomain(declared_object_count=1), ) for source in measurement_images: @@ -6851,56 +6817,31 @@ def test_multi_image_photometry_consumes_each_named_current_source(aliases): image = prepared.aligned_image.data np.testing.assert_allclose(image, images[source.source_image_name] / 255.0) arrays = object_intensity_backend().measure_prepared( - image, - ObjectIntensityPreparedLabels.from_source( - prepared.completion_payload, prepared.measurement_labels - ), + image, ObjectIntensityPreparedLabels.from_source(prepared.completion_payload, prepared.measurement_labels), ) - expected = {"Process": (221 / 255.0, 1, 2), "Nuclear": (1.0, 2, 3)}[ - source.source_image_name - ] + expected = {"Process": (221 / 255.0, 1, 2), "Nuclear": (1.0, 2, 3)}[source.source_image_name] np.testing.assert_allclose(arrays.max_intensity, (expected[0],)) np.testing.assert_allclose(arrays.max_intensity_y, (expected[1],)) np.testing.assert_allclose(arrays.max_intensity_x, (expected[2],)) -@pytest.mark.parametrize( - "aliases", - ( - ("Process",), - ("Process", "Nuclear"), - ("Nuclear", "Process"), - ("Process", "Nuclear", "Third"), - ), -) +@pytest.mark.parametrize("aliases", (("Process",), ("Process", "Nuclear"), ("Nuclear", "Process"), ("Process", "Nuclear", "Third"))) @pytest.mark.parametrize("slice_count", (1, 2)) -def test_registered_object_photometry_retains_each_source_at_each_slice( - aliases, slice_count -): +def test_registered_object_photometry_retains_each_source_at_each_slice(aliases, slice_count): image_specs = tuple(ArtifactSpec.input(name, ImageArtifactType) for name in aliases) object_spec = ArtifactSpec.input("Cells", ObjectLabelsArtifactType) output_spec = ArtifactSpec.output("Photometry", MeasurementsArtifactType) images = {} for name in aliases: value = {"Process": 100, "Nuclear": 200, "Third": 300}[name] - paths = tuple( - f"/synthetic/{name}_z{index + 1}.tif" for index in range(slice_count) - ) - fields = tuple( - { - "well": "A01", - "site": "1", - "channel": str(value // 100), - "z_index": str(index + 1), - "pixel_size": 0.5, - } - for index in range(slice_count) - ) + paths = tuple(f"/synthetic/{name}_z{index + 1}.tif" for index in range(slice_count)) + fields = tuple({"well": "A01", "site": "1", "channel": str(value // 100), + "z_index": str(index + 1), "pixel_size": 0.5} + for index in range(slice_count)) metadata = ImagePayloadMetadata( source_image_names=(name,), source_image_provenance_planes=SourceImageProvenancePlanes.from_components( - paths=paths, component_metadata=fields - ), + paths=paths, component_metadata=fields), plane_axis=RuntimePlaneAxis.RUNTIME_SLICE if slice_count > 1 else None, ) shape = (slice_count, 4, 5) if slice_count > 1 else (4, 5) @@ -6908,67 +6849,39 @@ def test_registered_object_photometry_retains_each_source_at_each_slice( label_shape = (slice_count, 4, 5) if slice_count > 1 else (4, 5) labels = ObjectLabelSet( name="Cells", - variant_data=ObjectLabelVariantData( - labels=np.ones(label_shape, dtype=np.int32) - ), + variant_data=ObjectLabelVariantData(labels=np.ones(label_shape, dtype=np.int32)), plane_axis=RuntimePlaneAxis.RUNTIME_SLICE if slice_count > 1 else None, domain=ObjectLabelDomain( - scope=( - ObjectLabelDomainScope.PLANE - if slice_count > 1 - else ObjectLabelDomainScope.PAYLOAD - ), + scope=ObjectLabelDomainScope.PLANE if slice_count > 1 else ObjectLabelDomainScope.PAYLOAD, declared_object_id_domains=((1,),) * slice_count if slice_count > 1 else (), declared_object_count=None if slice_count > 1 else 1, ), ) contract = _compiled_callable_contract( MeasureObjectIntensityModule.require_callable(), - artifact_inputs=(*image_specs, object_spec), - artifact_outputs=(output_spec,), + artifact_inputs=(*image_specs, object_spec), artifact_outputs=(output_spec,), ) runtime = _FakeCellProfilerRuntime( - images, - objects={"Cells": labels}, - callable_contract=contract, - artifact_input_edges=tuple( - _artifact_input_edge_for_test(spec) for spec in (*image_specs, object_spec) - ), + images, objects={"Cells": labels}, callable_contract=contract, + artifact_input_edges=tuple(_artifact_input_edge_for_test(spec) for spec in (*image_specs, object_spec)), artifact_output_bindings=((output_spec, _artifact_output_plan(output_spec)),), plane_projection=RuntimePlaneProjection.stack(slice_count), ) - _run_module( - _module_executor(contract), images[aliases[0]], cellprofiler_runtime=runtime - ) + _run_module(_module_executor(contract), images[aliases[0]], cellprofiler_runtime=runtime) assert len(runtime.measurements) == 1 table = runtime.measurements[0] assert table.subject.object_name == "Cells" assert set(table.source_provenance.represented_source_image_names) == set(aliases) - assert {row["slice_index"] for row in table.rows.iter_row_mappings()} == set( - range(slice_count) - ) + assert {row["slice_index"] for row in table.rows.iter_row_mappings()} == set(range(slice_count)) for name in aliases: provenance = table.source_provenance.for_source_image(name) assert provenance.source_image_provenance_planes.paths == tuple( - f"/synthetic/{name}_z{index + 1}.tif" for index in range(slice_count) - ) - assert all( - fields["pixel_size"] == 0.5 - for fields in provenance.source_image_provenance_planes.component_metadata - ) - assert tuple( - fields["z_index"] - for fields in provenance.source_image_provenance_planes.component_metadata - ) == tuple(str(index + 1) for index in range(slice_count)) + f"/synthetic/{name}_z{index + 1}.tif" for index in range(slice_count)) + assert all(fields["pixel_size"] == 0.5 for fields in provenance.source_image_provenance_planes.component_metadata) + assert tuple(fields["z_index"] for fields in provenance.source_image_provenance_planes.component_metadata) == tuple(str(index + 1) for index in range(slice_count)) column = f"Intensity_MaxIntensity_{name}" - values = tuple( - row[column] for row in table.rows.iter_row_mappings() if column in row - ) - np.testing.assert_allclose( - values, - ({"Process": 100, "Nuclear": 200, "Third": 300}[name] / 65535,) - * slice_count, - ) + values = tuple(row[column] for row in table.rows.iter_row_mappings() if column in row) + np.testing.assert_allclose(values, ({"Process": 100, "Nuclear": 200, "Third": 300}[name] / 65535,) * slice_count) def test_aligned_measurement_provenance_uses_cooperative_source_hook(): @@ -6980,35 +6893,22 @@ def source_provenances(cls, payload): calls.append(payload) return super().source_provenances(payload) - sources = tuple( - DeclaredMeasurementSource( - source_image_name=name, - payload=ImagePayloadMetadata( - source_path=f"/{name}.tif", source_image_names=(name,) - ).payload_with(np.ones((2, 3))), - ) - for name in ("First", "Independent") - ) + sources = tuple(DeclaredMeasurementSource( + source_image_name=name, + payload=ImagePayloadMetadata(source_path=f"/{name}.tif", source_image_names=(name,)).payload_with(np.ones((2, 3))), + ) for name in ("First", "Independent")) metadata = DeclaredMeasurementSource.aligned_source_metadata(sources) assert calls == [source.payload for source in sources] assert metadata.source_provenance.source_plane_count == 1 - assert metadata.source_provenance.represented_source_image_names == ( - "First", - "Independent", - ) - assert metadata.source_provenance.for_source_image( - "Independent" - ).source_image_provenance_planes.paths == ("/Independent.tif",) + assert metadata.source_provenance.represented_source_image_names == ("First", "Independent") + assert metadata.source_provenance.for_source_image("Independent").source_image_provenance_planes.paths == ("/Independent.tif",) def test_aligned_measurement_provenance_rejects_unequal_and_empty_axes(): - sources = tuple( - CellProfilerMeasurementImage( - source_image_name="Image", - payload=AlignedImageStack(tuple(np.ones((2, 3)) for _ in range(count))), - ) - for count in (1, 2) - ) + sources = tuple(CellProfilerMeasurementImage( + source_image_name="Image", + payload=AlignedImageStack(tuple(np.ones((2, 3)) for _ in range(count))), + ) for count in (1, 2)) with pytest.raises(ValueError, match="zip"): CellProfilerMeasurementImage.aligned_source_metadata(sources) with pytest.raises(ValueError, match="requires source images"): @@ -8061,9 +7961,7 @@ def test_aligned_payload_slices_runtime_slice_aligned_values() -> None: first = np.zeros((4, 5), dtype=np.int32) second = np.ones((4, 5), dtype=np.int32) - slices = RuntimeSliceProjection.alignment_slices( - RuntimeSliceAlignedValues((first, second)) - ) + slices = RuntimeSliceProjection.alignment_slices(RuntimeSliceAlignedValues((first, second))) assert slices[0] is first assert slices[1] is second @@ -8189,7 +8087,9 @@ def test_cellprofiler_auxiliary_payload_stack_preserves_metadata() -> None: stacked = RuntimeSliceProjection.full_stack_value(stacked) assert isinstance(stacked, ImageMetadataPayload) assert stacked.data.shape == (2, 1, 4, 5) - assert stacked.metadata.for_source_plane(0).intensity_scale == 65535.0 + assert ( + stacked.metadata.for_source_plane(0).intensity_scale == 65535.0 + ) assert stacked.metadata.for_source_plane(1).source_dtype == "uint8" @@ -8201,16 +8101,14 @@ def test_cellprofiler_image_aggregation_uses_nominal_image_payload_type() -> Non aggregated = RuntimeSliceProjection.full_stack_value(aggregated) assert aggregated.data.shape == (1, 4, 5) - assert aggregated.metadata.plane_axis is (RuntimePlaneAxis.RUNTIME_SLICE) + assert aggregated.metadata.plane_axis is ( + RuntimePlaneAxis.RUNTIME_SLICE + ) @pytest.mark.parametrize("owner", (AlignedImageStack, ImageOutputBundle)) -def test_cellprofiler_aligned_main_output_aggregation_transposes_surfaces( - owner, -) -> None: - contexts = tuple( - AlignedImageSliceContext.main_flow(name) for name in ("First", "Second") - ) +def test_cellprofiler_aligned_main_output_aggregation_transposes_surfaces(owner) -> None: + contexts = tuple(AlignedImageSliceContext.main_flow(name) for name in ("First", "Second")) outputs = ( owner( ( @@ -9178,10 +9076,7 @@ def keep_payload(image: np.ndarray): ObjectLabelPayload( variant_data=ObjectLabelVariantData( labels=labels, - variants={ - UneditedLabels: labels + 10, - SmallRemovedLabels: labels + 20, - }, + variants={UneditedLabels: labels + 10, SmallRemovedLabels: labels + 20}, ), domain=ObjectLabelDomain(declared_object_ids=(object_id,)), ), @@ -9311,7 +9206,9 @@ def segment(image: object): def test_aligned_stack_object_label_auxiliary_aggregates_on_runtime_slice_axis(): def segment(image: object): - site_index = int(image.metadata.source_component_metadata["site"]) + site_index = int( + image.metadata.source_component_metadata["site"] + ) labels = np.full( image.data.shape, site_index, @@ -9605,11 +9502,7 @@ def source_payload(alias: str, channel: int) -> AlignedImageStack: (RuntimePlaneAxis.RUNTIME_SLICE, ImagePayloadMetadataCompositionMode.STACK, 2), (RuntimePlaneAxis.RUNTIME_SLICE, ImagePayloadMetadataCompositionMode.BUNDLE, 1), (RuntimePlaneAxis.SOURCE_BINDING, ImagePayloadMetadataCompositionMode.STACK, 1), - ( - RuntimePlaneAxis.SOURCE_BINDING, - ImagePayloadMetadataCompositionMode.BUNDLE, - 2, - ), + (RuntimePlaneAxis.SOURCE_BINDING, ImagePayloadMetadataCompositionMode.BUNDLE, 2), ), ) def test_image_source_composition_preserves_only_compatible_incoming_topology( @@ -9632,10 +9525,7 @@ def test_image_source_composition_preserves_only_compatible_incoming_topology( if expected_planes == 2: assert planes.paths == paths else: - assert ( - tuple(contributor.path for contributor in planes.plane(0).contributors) - == paths - ) + assert tuple(contributor.path for contributor in planes.plane(0).contributors) == paths def test_measurement_source_composition_reads_dense_plane_facts_without_pixel_projection( @@ -9806,12 +9696,8 @@ def _pattern_group_runtime_for_output_memory( context.microscope_handler = SimpleNamespace(parser=SourceSchemaFilenameParser()) return PatternGroupExecutionRequest( context=context, - compiled_group=compile_function_pattern( - lambda image: image, {}, {} - ).default_group, - pattern_group_info="", - component_index=0, - component_count=1, + compiled_group=compile_function_pattern(lambda image: image, {}, {}).default_group, + pattern_group_info="", component_index=0, component_count=1, execution_plan=CompiledStepPlan( step_index=0, step_name="pattern output", @@ -9836,9 +9722,7 @@ def test_pattern_group_runtime_unstacks_aligned_image_stack_output(): output = runtime._project_output_slices(aligned, matching_files) stack_payload = aligned.copy_projected_output_stack( - output, - memory_type="numpy", - device_id=None, + output, memory_type="numpy", device_id=None, ) assert [payload for payload, _context in output] == [dna_payload, rna_payload] assert stack_payload.metadata.source_plane_intensity_scales == (65535, 65535) @@ -9894,9 +9778,12 @@ def test_pattern_group_runtime_does_not_invent_nested_axes_from_image_rank(): assert len(output_slices) == 2 np.testing.assert_array_equal(output_slices[0].data, first_site.data) - np.testing.assert_array_equal(output_slices[1].data, second_site.data) + np.testing.assert_array_equal( + output_slices[1].data, second_site.data + ) assert [ - payload.metadata.source_image_provenance_planes for payload in output_slices + payload.metadata.source_image_provenance_planes + for payload in output_slices ] == [ first_site.metadata.source_image_provenance_planes, second_site.metadata.source_image_provenance_planes, @@ -9915,22 +9802,15 @@ def test_pattern_group_runtime_leaves_variable_shape_aligned_outputs_uncached(): payload.data is pixels for (payload, _context), pixels in zip(output, (first, second), strict=True) ) - assert ( - aligned.copy_projected_output_stack( - output, - memory_type="numpy", - device_id=None, - ) - is None - ) + assert aligned.copy_projected_output_stack( + output, memory_type="numpy", device_id=None, + ) is None def test_invocation_request_keeps_original_aliases_when_labels_change_image_domain( monkeypatch, ): - contract = _compiled_callable_contract( - IdentifyTertiaryObjectsModule.require_callable() - ) + contract = _compiled_callable_contract(IdentifyTertiaryObjectsModule.require_callable()) executor = _module_executor(contract) larger = ObjectLabelSet( name="Larger", @@ -11367,7 +11247,9 @@ def test_enhance_or_suppress_features_tubeness_matches_hessian_reference(): expected = (-roots[..., 0] * (roots[..., 0] < 0) * (smoothing_value**2)).astype( np.float32 ) - np.testing.assert_allclose(result.data, expected, rtol=1e-6, atol=1e-7) + np.testing.assert_allclose( + result.data, expected, rtol=1e-6, atol=1e-7 + ) def test_cellprofiler_contract_executor_preserves_multi_image_stack_payload(): @@ -11459,7 +11341,9 @@ def test_compose_image_payload_broadcasts_only_from_declared_stack_owner(): stack_broadcast_source_indices=(None, 0), ) - assert composition.execution_mode is (AlignedStackExecution) + assert composition.execution_mode is ( + AlignedStackExecution + ) assert isinstance(composition.payload, AlignedImageStack) assert len(composition.payload.slices) == 2 first_pair = composition.payload.slices[0].data @@ -12135,7 +12019,8 @@ def test_track_objects_record_builder_uses_nominal_image_table_ownership() -> No for feature_name in object_feature_names ) assert all( - rows_by_feature[feature_name]["source_image_name"] == "image" + rows_by_feature[feature_name]["source_image_name"] + == "image" and "object_name" not in rows_by_feature[feature_name] for feature_name in image_feature_names | mean_feature_names ) @@ -12211,18 +12096,12 @@ def test_object_label_output_recorder_uses_output_label_domain() -> None: @pytest.mark.parametrize("carrier_axis", tuple(RuntimePlaneAxis)) -@pytest.mark.parametrize( - "module_name", - ( - "ExpandOrShrinkObjects", - "DilateObjects", - "ShrinkToObjectCenters", - "ClassifyObjectsSingleMeasurement", - ), -) +@pytest.mark.parametrize("module_name", ( + "ExpandOrShrinkObjects", "DilateObjects", "ShrinkToObjectCenters", + "ClassifyObjectsSingleMeasurement", +)) def test_object_only_module_uses_label_planes_instead_of_carrier_axis( - carrier_axis, - module_name, + carrier_axis, module_name, ) -> None: labels = np.zeros((2, 7, 7), dtype=np.int32) labels[0, 2, 2] = 1 @@ -12244,70 +12123,46 @@ def test_object_only_module_uses_label_planes_instead_of_carrier_axis( "InputObjects", ObjectLabelsArtifactType, parameter_name="labels" ) runtime = _FakeCellProfilerRuntime( - {}, - objects={"InputObjects": input_payload}, + {}, objects={"InputObjects": input_payload}, artifact_input_edges=(_artifact_input_edge_for_test(input_spec),), plane_projection=RuntimePlaneProjection.stack(2), ) output_spec = _output_from_input("ExpandedObjects", "InputObjects") classification = module_name == "ClassifyObjectsSingleMeasurement" measurement_spec = ArtifactSpec.output( - f"{module_name}_measurements", - MeasurementsArtifactType, - relations=( - ArtifactSpecRelation( - source=input_spec.ref() if classification else output_spec.ref() - ), - ), - ) - executor = _module_executor( - _compiled_callable_contract( - CellProfilerModule.require_module(module_name).require_callable(), - artifact_inputs=(input_spec,), - artifact_outputs=( - (measurement_spec,) - if classification - else (measurement_spec, output_spec) - ), - ) - ) - runtime.request = replace( - runtime.request, - artifact_outputs={ - spec.ref(): _artifact_output_plan(spec) - for spec in executor.callable_contract.artifact_outputs - }, - ) + f"{module_name}_measurements", MeasurementsArtifactType, + relations=(ArtifactSpecRelation( + source=input_spec.ref() if classification else output_spec.ref() + ),), + ) + executor = _module_executor(_compiled_callable_contract( + CellProfilerModule.require_module(module_name).require_callable(), + artifact_inputs=(input_spec,), + artifact_outputs=(measurement_spec,) if classification else (measurement_spec, output_spec), + )) + runtime.request = replace(runtime.request, artifact_outputs={ + spec.ref(): _artifact_output_plan(spec) + for spec in executor.callable_contract.artifact_outputs + }) # The image only carries main flow; its three planes do not define the # independent two-plane object domain. carrier = ImagePayloadMetadata(plane_axis=carrier_axis).payload_with( np.zeros((3, 7, 7), dtype=np.float32) ) result = _run_module( - executor, - carrier, - cellprofiler_runtime=runtime, + executor, carrier, cellprofiler_runtime=runtime, dtype_config=DtypeConfig(), - **( - { - "measurement_values": RuntimeSliceAlignedValues( - slices=( - np.array([0.25]), - np.array([0.75, 0.75, 0.75, 0.75]), - ) - ), - "bin_count": 2, - } - if classification - else {} - ), + **({ + "measurement_values": RuntimeSliceAlignedValues(slices=( + np.array([0.25]), np.array([0.75, 0.75, 0.75, 0.75]), + )), + "bin_count": 2, + } if classification else {}), ) assert result is carrier if classification: rows = tuple(runtime.measurements[0].rows.row_mappings()) - feature = ( - ClassifyObjectsSingleMeasurementModule.MeasurementFeatureTemplate.OBJECT_CLASS - ) + feature = ClassifyObjectsSingleMeasurementModule.MeasurementFeatureTemplate.OBJECT_CLASS first_bin = feature.feature_name(bin_name="Bin_1") second_bin = feature.feature_name(bin_name="Bin_2") classified = tuple( @@ -12327,8 +12182,8 @@ def test_object_only_module_uses_label_planes_instead_of_carrier_axis( expected = labels.copy() if module_name != "ShrinkToObjectCenters": for plane, coordinate, object_id in ((0, 2, 1), (1, 4, 4)): - expected[plane, coordinate - 1 : coordinate + 2, coordinate] = object_id - expected[plane, coordinate, coordinate - 1 : coordinate + 2] = object_id + expected[plane, coordinate - 1:coordinate + 2, coordinate] = object_id + expected[plane, coordinate, coordinate - 1:coordinate + 2] = object_id np.testing.assert_array_equal(recorded.labels, expected) @@ -12890,17 +12745,13 @@ def test_literal_measurement_stack_preserves_pixels_in_object_reference_domain( ) stack = ProducedImageStack( (metadata.payload_with(pixels, mask),), - memory_type="numpy", - plane_axis=RuntimePlaneAxis.RUNTIME_SLICE, + memory_type="numpy", plane_axis=RuntimePlaneAxis.RUNTIME_SLICE, ) labels = ObjectLabelPayload( - variant_data=ObjectLabelVariantData( - labels=np.ones(pixels.shape, dtype=np.int32) - ), + variant_data=ObjectLabelVariantData(labels=np.ones(pixels.shape, dtype=np.int32)), ) carrier = CellProfilerMeasurementImage( - source_image_name="Source", - payload=stack, + source_image_name="Source", payload=stack, reference_domain=CellProfilerMeasurementImageDomain.OBJECT_LABELS, execution_mode=execution_mode, ) @@ -13068,23 +12919,16 @@ def test_filterobjects_binds_selection_measurement_values_to_label_slices( assert filtered_array[1, 3, 3] == 0 -@pytest.mark.parametrize( - "carrier_axis, binary_operand", - ( - (RuntimePlaneAxis.SOURCE_BINDING, False), - (RuntimePlaneAxis.RUNTIME_SLICE, False), - (RuntimePlaneAxis.RUNTIME_SLICE, True), - ), -) -def test_area_occupied_rows_preserve_the_selected_domain( - carrier_axis, binary_operand -) -> None: +@pytest.mark.parametrize("carrier_axis, binary_operand", ( + (RuntimePlaneAxis.SOURCE_BINDING, False), + (RuntimePlaneAxis.RUNTIME_SLICE, False), + (RuntimePlaneAxis.RUNTIME_SLICE, True), +)) +def test_area_occupied_rows_preserve_the_selected_domain(carrier_axis, binary_operand) -> None: from openhcs.processing.backends.cellprofiler.area_occupied import OperandChoice object_specs = tuple( - ArtifactSpec.input( - name, ObjectLabelsArtifactType, parameter_name="object_labels" - ) + ArtifactSpec.input(name, ObjectLabelsArtifactType, parameter_name="object_labels") for name in ("Primary", "Other") ) objects = {} @@ -13102,19 +12946,15 @@ def test_area_occupied_rows_preserve_the_selected_domain( plane_axis=RuntimePlaneAxis.RUNTIME_SLICE, ) measurement_spec = ArtifactSpec.output( - "AreaMeasurements", - MeasurementsArtifactType, + "AreaMeasurements", MeasurementsArtifactType, measurement_feature_owner=MeasureImageAreaOccupiedBinaryModule, - relations=tuple( - ArtifactSpecRelation(source=spec.ref()) for spec in object_specs - ), + relations=tuple(ArtifactSpecRelation(source=spec.ref()) for spec in object_specs), ) binary_spec = ArtifactSpec.input("Binary", ImageArtifactType) input_specs = ((binary_spec,) if binary_operand else ()) + object_specs contract = _compiled_callable_contract( MeasureImageAreaOccupiedBinaryModule.require_callable(), - artifact_inputs=input_specs, - artifact_outputs=(measurement_spec,), + artifact_inputs=input_specs, artifact_outputs=(measurement_spec,), ) assert contract.invocation_domain_inputs.specs == ( (binary_spec,) if binary_operand else object_specs @@ -13128,23 +12968,15 @@ def test_area_occupied_rows_preserve_the_selected_domain( source_image_names=("Binary",) if binary_operand else (), ).payload_with(carrier_pixels) runtime = _FakeCellProfilerRuntime( - {"Binary": carrier} if binary_operand else {}, - objects=objects, - artifact_input_edges=tuple( - _artifact_input_edge_for_test(spec) for spec in input_specs - ), + {"Binary": carrier} if binary_operand else {}, objects=objects, + artifact_input_edges=tuple(_artifact_input_edge_for_test(spec) for spec in input_specs), plane_projection=RuntimePlaneProjection.stack(2), ) - runtime.request = replace( - runtime.request, - artifact_outputs={ - measurement_spec.ref(): _artifact_output_plan(measurement_spec), - }, - ) + runtime.request = replace(runtime.request, artifact_outputs={ + measurement_spec.ref(): _artifact_output_plan(measurement_spec), + }) result = _run_module( - _module_executor(contract), - carrier, - cellprofiler_runtime=runtime, + _module_executor(contract), carrier, cellprofiler_runtime=runtime, operand_choices=((OperandChoice.BINARY_IMAGE,) if binary_operand else ()) + (OperandChoice.OBJECTS, OperandChoice.OBJECTS), dtype_config=DtypeConfig(), @@ -13152,26 +12984,15 @@ def test_area_occupied_rows_preserve_the_selected_domain( assert result is carrier rows = tuple(runtime.measurements[0].rows.row_mappings()) measured = tuple( - ( - row["slice_index"], - row["source_image_name"], - row[f"AreaOccupied_AreaOccupied_{row['source_image_name']}"], - ) + (row["slice_index"], row["source_image_name"], + row[f"AreaOccupied_AreaOccupied_{row['source_image_name']}"]) for row in rows ) - expected = ( - (0, "Primary", 1.0), - (0, "Other", 2.0), - (1, "Primary", 4.0), - (1, "Other", 3.0), - ) + expected = ((0, "Primary", 1.0), (0, "Other", 2.0), + (1, "Primary", 4.0), (1, "Other", 3.0)) if binary_operand: - expected = ( - (0, "Binary", 3.0), - *expected[:2], - (1, "Binary", 6.0), - *expected[2:], - ) + expected = ((0, "Binary", 3.0), *expected[:2], + (1, "Binary", 6.0), *expected[2:]) assert measured == expected @@ -14408,10 +14229,7 @@ def test_relateobjects_parent_means_align_scoped_child_tables_by_source_plane( changed_rows = MeasurementSparseColumnarRows( columns={ **first.rows.columns, - colocalization_feature: ( - 123.0, - *first.rows.column_values(colocalization_feature)[1:], - ), + colocalization_feature: (123.0, *first.rows.column_values(colocalization_feature)[1:]), }, fields=first.rows.fields, ) @@ -14526,8 +14344,7 @@ def test_relateobjects_parent_means_align_scoped_child_tables_by_source_plane( if conflicting_upstream: with pytest.raises(ValueError, match="conflicting values"): projector.parent_mean_upstream_measurement_rows( - parent_spec=parent_spec, - child_spec=child_spec, + parent_spec=parent_spec, child_spec=child_spec, payload=runtime.relationships[0], ) return @@ -14801,10 +14618,9 @@ def test_spatial_grid_output_recorder_accepts_pure_2d_grid_sequence() -> None: recorded = runtime.spatial_grids["Grid"] assert isinstance(recorded, RuntimeSliceAlignedValues) - assert [recorded.value_at(slice_index).x_origin for slice_index in range(2)] == [ - 1.0, - 2.0, - ] + assert [ + recorded.value_at(slice_index).x_origin for slice_index in range(2) + ] == [1.0, 2.0] def test_identify_objects_in_grid_respects_row_primary_ordering() -> None: @@ -17147,12 +16963,11 @@ def test_object_label_payload_preserves_source_metadata_for_measurements() -> No payload = CellProfilerOutputRecorder.for_artifact_type( spec.artifact_type ).raw_runtime_input_value( - spec, - RuntimeInputBindingRequest( + spec, RuntimeInputBindingRequest( adapter=runtime, kwargs={}, current_image=image, - ).artifact_value_for_spec(spec), + ).artifact_value_for_spec(spec) ) np.testing.assert_array_equal(payload.labels, labels.labels) @@ -17206,12 +17021,11 @@ def test_runtime_object_label_payload_ignores_measurement_image_as_selector() -> payload = CellProfilerOutputRecorder.for_artifact_type( spec.artifact_type ).raw_runtime_input_value( - spec, - RuntimeInputBindingRequest( + spec, RuntimeInputBindingRequest( adapter=runtime, kwargs={}, current_image=measurement_image, - ).artifact_value_for_spec(spec), + ).artifact_value_for_spec(spec) ) np.testing.assert_array_equal(payload.labels, labels.labels) @@ -17258,12 +17072,11 @@ def test_full_stack_object_measurement_resolves_complete_label_artifact() -> Non label_payload = CellProfilerOutputRecorder.for_artifact_type( object_spec.artifact_type ).raw_runtime_input_value( - object_spec, - RuntimeInputBindingRequest( + object_spec, RuntimeInputBindingRequest( adapter=runtime, kwargs={}, current_image=image, - ).artifact_value_for_spec(object_spec), + ).artifact_value_for_spec(object_spec) ) _, executable_labels, _, _, _, _, _ = object_measurement_runtime_inputs( object_label_execution=object_label_input_execution_mode_from_callable( @@ -17339,12 +17152,11 @@ def test_object_label_payload_for_measurement_image_projects_source_spatial_crop label_payload = CellProfilerOutputRecorder.for_artifact_type( spec.artifact_type ).raw_runtime_input_value( - spec, - RuntimeInputBindingRequest( + spec, RuntimeInputBindingRequest( adapter=runtime, kwargs={}, current_image=image, - ).artifact_value_for_spec(spec), + ).artifact_value_for_spec(spec) ) payload = ( CellProfilerMeasurementImage( @@ -18430,9 +18242,7 @@ def test_convert_objects_to_image_uses_declared_label_source_for_runtime_plane_d variable_components=(Microscopy.ZIndex,), relations=(GroupLineageSourceRelation(source_spec.ref()),), ) - result = ( - ImageArtifactType if output_plan is None else output_plan.artifact_type - ).contextualize_output_from_projector( + result = (ImageArtifactType if output_plan is None else output_plan.artifact_type).contextualize_output_from_projector( label_payload, raw_result, output_plan, @@ -18468,9 +18278,7 @@ def test_object_label_image_output_rejects_source_plane_count_drift() -> None: ValueError, match="source-plane provenance must match the declared runtime plane axis", ): - ( - ImageArtifactType if output_plan is None else output_plan.artifact_type - ).contextualize_output_from_projector( + (ImageArtifactType if output_plan is None else output_plan.artifact_type).contextualize_output_from_projector( label_payload, ImagePayloadMetadata().payload_with( np.zeros(labels.shape, dtype=np.uint16) @@ -18501,9 +18309,7 @@ def test_object_label_singleton_volume_output_declares_runtime_plane_axis() -> N relations=(GroupLineageSourceRelation(source_spec.ref()),), ) - result = ( - ImageArtifactType if output_plan is None else output_plan.artifact_type - ).contextualize_output_from_projector( + result = (ImageArtifactType if output_plan is None else output_plan.artifact_type).contextualize_output_from_projector( label_payload, raw_result, output_plan, @@ -18535,9 +18341,7 @@ def test_object_label_scalar_image_output_does_not_invent_runtime_plane_axis() - relations=(GroupLineageSourceRelation(source_spec.ref()),), ) - result = ( - ImageArtifactType if output_plan is None else output_plan.artifact_type - ).contextualize_output_from_projector( + result = (ImageArtifactType if output_plan is None else output_plan.artifact_type).contextualize_output_from_projector( label_payload, raw_result, output_plan, @@ -18791,9 +18595,7 @@ def test_measure_object_size_shape_surface_areas_match_marching_cubes_oracle() - expected.append(_surface_area(labels[bounds] == label_id)) features, measured_labels = measure_object_size_shape_feature_arrays( - labels, - calculate_advanced=False, - calculate_zernikes=False, + labels, calculate_advanced=False, calculate_zernikes=False, ) np.testing.assert_array_equal(measured_labels, label_ids) np.testing.assert_allclose( @@ -18868,7 +18670,8 @@ def test_natural_object_measurement_resolves_one_roster_without_eager_compositio images, {"Cells": labels}, artifact_input_edges=tuple( - _artifact_input_edge_for_test(spec) for spec in (*image_specs, object_spec) + _artifact_input_edge_for_test(spec) + for spec in (*image_specs, object_spec) ), artifact_output_bindings=( (measurement_spec, _artifact_output_plan(measurement_spec)), @@ -18914,8 +18717,7 @@ def table_name(self, measurement_images, source_image_name): assert resolutions == ["DNA", "ER"][:image_count] assert fallbacks == [None if image_count == 1 else "dna__er"] table = next( - table - for table in runtime.measurements + table for table in runtime.measurements if table.subject == MeasurementSubject(MeasurementScope.OBJECT, "Cells") ) rows = table.rows.row_mappings() @@ -18930,14 +18732,11 @@ def test_natural_measurement_fallback_preserves_selected_main_flow_plane(monkeyp image_spec = ArtifactSpec.input("DNA", ImageArtifactType) object_spec = ArtifactSpec.input("Cells", ObjectLabelsArtifactType) - output_spec = ArtifactSpec.output( - "SelectedPlaneIntensity", MeasurementsArtifactType - ) + output_spec = ArtifactSpec.output("SelectedPlaneIntensity", MeasurementsArtifactType) planes = tuple( ImagePayloadMetadata(source_image_names=(name,)).payload_with( np.ones((3, 4), dtype=np.float32) - ) - for name in ("DNA", "ER") + ) for name in ("DNA", "ER") ) image = ImagePayloadMetadata.compose( planes, mode=ImagePayloadMetadataCompositionMode.STACK @@ -18952,8 +18751,7 @@ def test_natural_measurement_fallback_preserves_selected_main_flow_plane(monkeyp ), ) runtime = _FakeCellProfilerRuntime( - {}, - {"Cells": labels}, + {}, {"Cells": labels}, artifact_input_edges=( replace( _artifact_input_edge_for_test(image_spec, stored=False), @@ -18964,36 +18762,26 @@ def test_natural_measurement_fallback_preserves_selected_main_flow_plane(monkeyp artifact_output_bindings=((output_spec, _artifact_output_plan(output_spec)),), plane_projection=RuntimePlaneProjection(plane_index=0, plane_count=2), ) - executor = _module_executor( - _compiled_callable_contract( - MeasureObjectIntensityModule.require_callable(), - artifact_inputs=(image_spec, object_spec), - artifact_outputs=(output_spec,), - ) - ) + executor = _module_executor(_compiled_callable_contract( + MeasureObjectIntensityModule.require_callable(), + artifact_inputs=(image_spec, object_spec), artifact_outputs=(output_spec,), + )) _activate_runtime_contract(executor.callable_contract, runtime) original_fallback = executor._image_request( - image, - runtime, - module_type=MeasureObjectIntensityModule, + image, runtime, module_type=MeasureObjectIntensityModule, active_input_specs=(image_spec, object_spec), ).source_image_name assert original_fallback == "DNA" monkeypatch.setattr( - CellProfilerModuleExecutor, - "_image_request", + CellProfilerModuleExecutor, "_image_request", lambda *_args, **_kwargs: pytest.fail("unused composed request"), ) - policy_type = type( - MeasureObjectIntensityModule.runtime_object_measurement_row_policy() - ) + policy_type = type(MeasureObjectIntensityModule.runtime_object_measurement_row_policy()) original_table_name = policy_type.table_source_image_name fallbacks = [] - def table_name(self, measurement_images, source_image_name): fallbacks.append(source_image_name) return original_table_name(self, measurement_images, source_image_name) - monkeypatch.setattr(policy_type, "table_source_image_name", table_name) assert _run_module(executor, image, cellprofiler_runtime=runtime) is image assert fallbacks == [original_fallback] @@ -19050,11 +18838,11 @@ def execute_batch(func, requests, execute_one): contract = replace( contract, - runtime_batch_executors=( - {RuntimeBatchExecutionDomain.MEASUREMENT_IMAGES: execute_batch} - if batched - else {} - ), + runtime_batch_executors={ + RuntimeBatchExecutionDomain.MEASUREMENT_IMAGES: execute_batch + } + if batched + else {}, ) runtime = _FakeCellProfilerRuntime( images, diff --git a/tests/unit/test_cellprofiler_module_execution_semantics.py b/tests/unit/test_cellprofiler_module_execution_semantics.py index 1c869df72e..967be27b01 100644 --- a/tests/unit/test_cellprofiler_module_execution_semantics.py +++ b/tests/unit/test_cellprofiler_module_execution_semantics.py @@ -40,9 +40,7 @@ ) from openhcs.core.runtime_plane_projection import RuntimePlaneAxis from openhcs.core.runtime_stores import RuntimeValueStore -from openhcs.core.pipeline.function_contracts import ( - object_label_input_execution_mode_from_callable, -) +from openhcs.core.pipeline.function_contracts import object_label_input_execution_mode_from_callable from openhcs.interop.cellprofiler.runtime.module_execution import ( CellProfilerModuleExecutor, ) @@ -248,7 +246,8 @@ def test_composed_image_measurement_callable_retains_standard_execution() -> Non raw_func = MeasureColocalizationModule.require_callable() assert ( - object_label_input_execution_mode_from_callable(raw_func) is SliceAlignedLabels + object_label_input_execution_mode_from_callable(raw_func) + is SliceAlignedLabels ) assert not MeasureColocalizationModule.executes_per_image_measurements( raw_func, diff --git a/tests/unit/test_cellprofiler_morphology_public_contracts.py b/tests/unit/test_cellprofiler_morphology_public_contracts.py index 6b74e6a6b8..734f0c6ac4 100644 --- a/tests/unit/test_cellprofiler_morphology_public_contracts.py +++ b/tests/unit/test_cellprofiler_morphology_public_contracts.py @@ -266,10 +266,7 @@ def test_registry_morphology_owners_compile_exact_public_function_step_abi( assert tuple(spec.artifact_type for spec in contract.artifact_outputs) == ( case.output_types ) - assert ( - CallableContract.from_callable(case.function).artifact_input_parameter_names - == case.special_inputs - ) + assert CallableContract.from_callable(case.function).artifact_input_parameter_names == case.special_inputs def test_fill_objects_returns_its_declared_object_label_artifact() -> None: diff --git a/tests/unit/test_cellprofiler_output_record_occurrence.py b/tests/unit/test_cellprofiler_output_record_occurrence.py index 5bb5d55d27..b63cec28aa 100644 --- a/tests/unit/test_cellprofiler_output_record_occurrence.py +++ b/tests/unit/test_cellprofiler_output_record_occurrence.py @@ -269,18 +269,13 @@ def _image_record_request() -> tuple[CellProfilerOutputRecordRequest, ArtifactSp spec = ArtifactSpec.input("Original", ImageArtifactType) request, (edge,) = _output_record_request((spec,), ((0, spec),)) edge = replace(edge, main_flow_projection=MainFlowInputProjection.COMPLETE_PAYLOAD) - adapter = SimpleNamespace( - request=replace(request.adapter.request, artifact_inputs={edge.key: edge}) - ) - return ( - replace( - request, - adapter=adapter, - active_input_edges=(edge,), - current_image=np.zeros((3, 4), dtype=np.float32), - ), - spec, - ) + adapter = SimpleNamespace(request=replace( + request.adapter.request, artifact_inputs={edge.key: edge} + )) + return replace( + request, adapter=adapter, active_input_edges=(edge,), + current_image=np.zeros((3, 4), dtype=np.float32), + ), spec def test_record_binding_keeps_output_admission_before_live_input_validation( @@ -292,9 +287,7 @@ def test_record_binding_keeps_output_admission_before_live_input_validation( def fail_selection(_self): raise error - monkeypatch.setattr( - RuntimeAdapterRequest, "selected_artifact_input_specs", fail_selection - ) + monkeypatch.setattr(RuntimeAdapterRequest, "selected_artifact_input_specs", fail_selection) # Output-only recording does not observe input declaration callbacks. assert replace(request).output_value is request.output_value with pytest.raises(ValueError, match="does not match active output"): @@ -306,23 +299,17 @@ def fail_selection(_self): assert caught.value is error -def test_record_binding_reads_current_pixels_and_mutable_kwargs_without_holder_aliases() -> ( - None -): +def test_record_binding_reads_current_pixels_and_mutable_kwargs_without_holder_aliases() -> None: request, spec = _image_record_request() assert isinstance(request, RuntimeInputBindingRequest) assert "call_kwargs" not in {item.name for item in fields(request)} assert not hasattr(request, "__dict__") with pytest.raises(TypeError): CellProfilerOutputRecordRequest( - request.callable_contract, - request.active_input_edges, - adapter=request.adapter, - kwargs=request.kwargs, - current_image=request.current_image, - spec=request.spec, - output_plan=request.output_plan, - output_value=request.output_value, + request.callable_contract, request.active_input_edges, + adapter=request.adapter, kwargs=request.kwargs, + current_image=request.current_image, spec=request.spec, + output_plan=request.output_plan, output_value=request.output_value, source=request.source, ) with pytest.raises(FrozenInstanceError): @@ -363,7 +350,9 @@ def observe_bind(strategy, input_spec, value): binding_inputs.append(value) return original_bind(strategy, input_spec, value) - monkeypatch.setattr(ImageOutputRecorder, "raw_runtime_input_value", observe_bind) + monkeypatch.setattr( + ImageOutputRecorder, "raw_runtime_input_value", observe_bind + ) for pixel_value in (65535, 32768): pixels[:] = pixel_value binding_inputs.clear() @@ -371,22 +360,18 @@ def observe_bind(strategy, input_spec, value): assert binding_inputs == [payload] assert source.data.dtype == np.float32 np.testing.assert_allclose( - source.data, - np.float32(pixel_value) / 65535, + source.data, np.float32(pixel_value) / 65535, ) assert source.mask is mask metadata = source.metadata assert metadata.source_image_names == (spec.name,) assert metadata.source_image_provenance_planes.paths == ("/input/actual.tif",) assert set(metadata.source_provenance.represented_source_image_names) == { - "Acquired", - spec.name, + "Acquired", spec.name, } -def test_record_source_returns_the_actual_bound_value_without_rebinding( - monkeypatch, -) -> None: +def test_record_source_returns_the_actual_bound_value_without_rebinding(monkeypatch) -> None: request, _spec = _image_record_request() bound = ImagePayloadMetadata( source_image_names=("Original",), @@ -409,9 +394,7 @@ def reject_rebinding(*_args, **_kwargs): assert reads == [(edge, edge.spec.parameter_name)] -def test_record_endpoint_identity_does_not_change_reference_broadcast_selection() -> ( - None -): +def test_record_endpoint_identity_does_not_change_reference_broadcast_selection() -> None: request, spec = _image_record_request() equivalent = replace(spec) with pytest.raises(RuntimeError, match="has no exact compiled input edge"): @@ -442,8 +425,7 @@ def observe_selection(adapter_request): request.kwargs["epoch"] = epoch events.clear() original = RuntimeInputBindingRequest( - adapter=request.adapter, - kwargs=request.kwargs, + adapter=request.adapter, kwargs=request.kwargs, current_image=request.current_image, ).runtime_value(request.active_input_edges[0]) expected_events = tuple(events) @@ -457,9 +439,7 @@ def observe_selection(adapter_request): # in the map cannot excuse a live selector removing its declaration. edge = request.active_input_edges[0] output = ArtifactSpec.output_preserving_source_stack_scope( - request.spec.name, - request.spec.artifact_type, - spec, + request.spec.name, request.spec.artifact_type, spec, ) consumed = CellProfilerImageRequest( source_image_name=spec.name, @@ -472,14 +452,12 @@ def observe_selection(adapter_request): primary_input_edges=(edge,), ) request = replace( - request, - spec=output, + request, spec=output, output_plan=replace(request.output_plan, relations=output.relations), source=consumed, ) monkeypatch.setattr( - RuntimeAdapterRequest, - "selected_artifact_input_specs", + RuntimeAdapterRequest, "selected_artifact_input_specs", lambda _adapter_request: ArtifactSpecCollection(()), ) with pytest.raises(ValueError, match="does not declare artifact input"): @@ -491,9 +469,7 @@ def test_record_explicit_object_subset_preserves_binding_admission() -> None: second = ArtifactSpec.input("Second", ObjectLabelsArtifactType) request, _edges = _output_record_request((first, second), ((0, first), (1, second))) standalone = RuntimeInputBindingRequest( - adapter=request.adapter, - kwargs=request.kwargs, - current_image=request.current_image, + adapter=request.adapter, kwargs=request.kwargs, current_image=request.current_image, ) for subset in ((first,), (replace(second),), ()): actual = request.with_object_inputs(subset) @@ -519,15 +495,11 @@ def test_record_subset_validation_runs_after_output_admission_and_preserves_erro def fail_selection(_self): raise error - monkeypatch.setattr( - RuntimeAdapterRequest, "selected_artifact_input_specs", fail_selection - ) + monkeypatch.setattr(RuntimeAdapterRequest, "selected_artifact_input_specs", fail_selection) for change in ( lambda: RuntimeInputBindingRequest( - adapter=request.adapter, - kwargs=request.kwargs, - current_image=request.current_image, - selected_object_inputs=(), + adapter=request.adapter, kwargs=request.kwargs, + current_image=request.current_image, selected_object_inputs=(), ), lambda: request.with_object_inputs(()), ): @@ -536,8 +508,7 @@ def fail_selection(_self): assert caught.value is error with pytest.raises(ValueError, match="does not match active output"): replace( - request, - selected_object_inputs=(), + request, selected_object_inputs=(), output_plan=replace(request.output_plan, name="Wrong"), ) # The ordinary post-call record still observes no input selector at birth. diff --git a/tests/unit/test_cellprofiler_prepared_payload_abi.py b/tests/unit/test_cellprofiler_prepared_payload_abi.py index 5df50b3e40..43e4b77799 100644 --- a/tests/unit/test_cellprofiler_prepared_payload_abi.py +++ b/tests/unit/test_cellprofiler_prepared_payload_abi.py @@ -7,19 +7,14 @@ from openhcs.core.function_patterns import NormalizedFunctionGroup from openhcs.core.runtime_image_values import ImagePayloadMetadata -from openhcs.core.runtime_object_label_building import ( - SourceImageObjectLabelBuildRequest, -) +from openhcs.core.runtime_object_label_building import SourceImageObjectLabelBuildRequest from openhcs.core.runtime_object_labels import object_label_dense_array from openhcs.core.source_spatial_domain import SourceSpatialDomain from openhcs.interop.cellprofiler.runtime.adapter import CellProfilerRuntimeAdapter from openhcs.interop.cellprofiler.runtime.function_contract_execution import ( CellProfilerFunctionContractExecutor, ) -from openhcs.processing.backends.cellprofiler.color import ( - ColorToGrayMode, - color_to_gray, -) +from openhcs.processing.backends.cellprofiler.color import ColorToGrayMode, color_to_gray from openhcs.processing.backends.cellprofiler.median_filter import medianfilter from openhcs.processing.backends.cellprofiler.secondary import ( SecondaryMethod, @@ -67,18 +62,12 @@ def test_prepared_color_to_gray_preserves_channel_mask_and_spatial_domain(): ).payload_with(pixels, mask) result = _execute_prepared( - color_to_gray, - source, - { - "mode": ColorToGrayMode.COMBINE, - "channel_indices": (0, 2), - "contributions": (1.0, 3.0), - }, + color_to_gray, source, + {"mode": ColorToGrayMode.COMBINE, + "channel_indices": (0, 2), "contributions": (1.0, 3.0)}, ) - np.testing.assert_array_equal( - result.data, (pixels[..., 0] + 3 * pixels[..., 2]) / 4 - ) + np.testing.assert_array_equal(result.data, (pixels[..., 0] + 3 * pixels[..., 2]) / 4) np.testing.assert_array_equal(result.mask, mask) metadata = result.metadata assert metadata.axis_position(ColourAxis) is None @@ -90,19 +79,10 @@ def test_prepared_medianfilter_keeps_original_intensity_scale_and_mask(): pixels = np.arange(25, dtype=np.uint16).reshape(5, 5) mask = np.ones(pixels.shape, dtype=bool) mask[0] = False - source = ( - ImagePayloadMetadata.for_array(pixels) - .payload_with(pixels, mask) - .normalize_intensity_payload( - dtype=np.float32, - ) - ) + source = ImagePayloadMetadata.for_array(pixels).payload_with(pixels, mask).normalize_intensity_payload(dtype=np.float32,) result = _execute_prepared( - medianfilter, - source, - {"window_size": 3}, - FullStackExecution, + medianfilter, source, {"window_size": 3}, FullStackExecution, ) np.testing.assert_array_equal( @@ -118,34 +98,24 @@ def test_prepared_secondary_object_output_keeps_actual_source_spatial_domain(): pixels = np.zeros((9, 9), dtype=np.float32) domain = SourceSpatialDomain(origin_yx=(3, 4), source_shape_yx=(15, 16)) source = ImagePayloadMetadata( - source_image_names=("DNA",), - source_path="/input/dna.tif", + source_image_names=("DNA",), source_path="/input/dna.tif", source_spatial_domain=domain, ).payload_with(pixels, None) seed_pixels = np.zeros(pixels.shape, dtype=np.int32) seed_pixels[4, 4] = 1 seeds = SourceImageObjectLabelBuildRequest( - image=source, - labels=seed_pixels, - declared_object_ids=(1,), + image=source, labels=seed_pixels, declared_object_ids=(1,), ).payload() result = _execute_prepared( - identify_secondary_objects, - source, - { - "primary_labels": seeds, - "method": SecondaryMethod.DISTANCE_N, - "distance_to_dilate": 1, - "fill_holes": False, - }, + identify_secondary_objects, source, + {"primary_labels": seeds, "method": SecondaryMethod.DISTANCE_N, + "distance_to_dilate": 1, "fill_holes": False}, ) _image, _rows, relationship, objects = result expected = ndimage.distance_transform_edt(seed_pixels == 0) <= 1 - np.testing.assert_array_equal( - object_label_dense_array(objects), expected.astype(np.int32) - ) + np.testing.assert_array_equal(object_label_dense_array(objects), expected.astype(np.int32)) assert objects.source_spatial_domain.origin_yx == domain.origin_yx assert objects.source_spatial_domain.source_shape_yx == domain.source_shape_yx assert objects.source_provenance == source.metadata.source_provenance diff --git a/tests/unit/test_cellprofiler_runtime_adapter.py b/tests/unit/test_cellprofiler_runtime_adapter.py index 81ac50f779..3d43c9b698 100644 --- a/tests/unit/test_cellprofiler_runtime_adapter.py +++ b/tests/unit/test_cellprofiler_runtime_adapter.py @@ -49,12 +49,7 @@ MeasurementSparseColumnarRows, measurement_rows, ) -from openhcs.core.pipeline.function_contracts import ( - composed_image_payload, - object_label_input_execution_mode, - runtime_bound_parameters, - special_inputs, -) +from openhcs.core.pipeline.function_contracts import (composed_image_payload, object_label_input_execution_mode, runtime_bound_parameters, special_inputs) from openhcs.core.runtime_artifact_queries import ( MeasurementLabelSliceFeatureQuery, MeasurementTableAxisProjection, @@ -387,6 +382,8 @@ def runtime_axis_scope( ) + + @dataclass(frozen=True) class StaticFilenameParser: metadata: Mapping[str, object] @@ -698,52 +695,44 @@ def _pipeline_start_contains_binding(alias): def _source_bound_image_adapter(output_bindings, images): from polystore.virtual_workspace import SourcePixelRef from openhcs.core.source_projection import ( - OpenHCSPlaneAddress, - SourcePlaneProjection, - SourceProjectionSet, - ) - from openhcs.core.source_workspace_projection import ( - VirtualWorkspaceSourceProjection, + OpenHCSPlaneAddress, SourcePlaneProjection, SourceProjectionSet, ) + from openhcs.core.source_workspace_projection import VirtualWorkspaceSourceProjection from openhcs.core.dataset_sources.source_schema import SourceSchemaFilenameParser filemanager = FileManagerStub() for alias, image in images.items(): filemanager.saved[("memory", f"/src/{alias}.tif")] = image context = ContextStub(filemanager) - projections = SourceProjectionSet( - tuple( - SourcePlaneProjection( - address=OpenHCSPlaneAddress( - ( - (Microscopy.Well, "A01"), - (Microscopy.Site, "1"), - (Microscopy.Channel, str(index)), - (Microscopy.ZIndex, "1"), - (Microscopy.Timepoint, "1"), - ) - ), - ref=SourcePixelRef("memory", f"/src/{alias}.tif"), - source_alias=alias, - ) - for index, alias in enumerate(images, start=1) + projections = SourceProjectionSet(tuple( + SourcePlaneProjection( + address=OpenHCSPlaneAddress( + ( + (Microscopy.Well, 'A01'), + (Microscopy.Site, '1'), + (Microscopy.Channel, str(index)), + (Microscopy.ZIndex, '1'), + (Microscopy.Timepoint, '1'), + ) + ), + ref=SourcePixelRef('memory', f'/src/{alias}.tif'), + source_alias=alias, ) - ) + for index, alias in enumerate(images, start=1) + )) subdirectory = projections.metadata_dict( parser=SourceSchemaFilenameParser(), - microscope_handler_name="SourceBindingsSource", - source_filename_parser_name="SourceSchemaFilenameParser", - grid_dimensions=[1, 1], - pixel_size=1.0, + microscope_handler_name='SourceBindingsSource', + source_filename_parser_name='SourceSchemaFilenameParser', + grid_dimensions=[1, 1], pixel_size=1., ) - document = {"subdirectories": {".": subdirectory}} + document = {'subdirectories': {'.': subdirectory}} context.microscope_handler.metadata_handler = SimpleNamespace( source_workspace_root=lambda _plate: Path(context.input_dir), source_workspace_metadata_document=lambda _plate: document, ) workspace = VirtualWorkspaceSourceProjection.from_openhcs_metadata( - Path(context.input_dir), - document, + Path(context.input_dir), document, ) for path, projection in workspace.source_projections_by_virtual_path.items(): pixels = images[projection.source_alias] @@ -751,14 +740,11 @@ def _source_bound_image_adapter(output_bindings, images): source_path=projection.ref.backend_address, source_image_names=(projection.source_alias,), source_spatial_domain=SourceSpatialDomain( - origin_yx=(0, 0), - source_shape_yx=pixels.shape[-2:], + origin_yx=(0, 0), source_shape_yx=pixels.shape[-2:], ), ).payload_with(pixels) - filemanager.saved[("virtual_workspace", path)] = payload - filemanager.saved[ - ("virtual_workspace", str(Path(context.input_dir) / path)) - ] = payload + filemanager.saved[('virtual_workspace', path)] = payload + filemanager.saved[('virtual_workspace', str(Path(context.input_dir) / path))] = payload return cellprofiler_runtime_adapter_for_test( runtime_value_store=RuntimeValueStore(), axis_scope=runtime_axis_scope(AXIS_ID), @@ -1261,7 +1247,9 @@ def test_cellprofiler_adapter_rejects_unaddressable_source_aligned_label_stack() def test_cellprofiler_adapter_does_not_cache_current_image_object_selection(): store = RuntimeValueStore() outputs = { - NUCLEI: _plan(NUCLEI, ObjectLabelsArtifactType, group_component=Microscopy.Site) + NUCLEI: _plan( + NUCLEI, ObjectLabelsArtifactType, group_component=Microscopy.Site + ) } output_bindings = ( _output_binding( @@ -1427,7 +1415,9 @@ def test_cellprofiler_adapter_does_not_select_relationship_from_current_source_p ("1", "2"), component=Microscopy.Site ), component_scopes=( - ComponentGroupScope.from_raw(("1", "2"), component=Microscopy.Site), + ComponentGroupScope.from_raw( + ("1", "2"), component=Microscopy.Site + ), ), consumer_variable_components=(Microscopy.Site,), ), @@ -1525,7 +1515,9 @@ def test_cellprofiler_adapter_aligns_grouped_relationships_to_runtime_slices(): ("1", "2"), component=Microscopy.Site ), component_scopes=( - ComponentGroupScope.from_raw(("1", "2"), component=Microscopy.Site), + ComponentGroupScope.from_raw( + ("1", "2"), component=Microscopy.Site + ), ), consumer_variable_components=(Microscopy.Site,), ), @@ -1558,7 +1550,9 @@ def test_cellprofiler_adapter_aligns_grouped_relationships_to_runtime_slices(): def test_cellprofiler_adapter_does_not_source_scope_default_image_records(): store = RuntimeValueStore() outputs = { - DNA_IMAGE: _plan(DNA_IMAGE, ImageArtifactType, group_component=Microscopy.Site) + DNA_IMAGE: _plan( + DNA_IMAGE, ImageArtifactType, group_component=Microscopy.Site + ) } output_bindings = ( _output_binding(DNA_IMAGE, ImageArtifactType, plan=outputs[DNA_IMAGE]), @@ -1595,7 +1589,9 @@ def test_cellprofiler_adapter_does_not_source_scope_default_image_records(): group_component=Microscopy.Site, ), invocation_scope=ComponentGroupScope.ungrouped(), - producer_selection_scope=ComponentGroupScope.dynamic(Microscopy.Site), + producer_selection_scope=ComponentGroupScope.dynamic( + Microscopy.Site + ), component_scopes=(ComponentGroupScope.dynamic(Microscopy.Site),), consumer_variable_components=(Microscopy.Site,), ), @@ -1682,9 +1678,13 @@ def test_cellprofiler_adapter_stacks_declared_default_image_input_runtime_groups invocation_scope=ComponentGroupScope.from_raw( (AXIS_ID,), component=Microscopy.Well ), - producer_selection_scope=ComponentGroupScope.dynamic(Microscopy.Site), + producer_selection_scope=ComponentGroupScope.dynamic( + Microscopy.Site + ), component_scopes=( - ComponentGroupScope.from_raw((AXIS_ID,), component=Microscopy.Well), + ComponentGroupScope.from_raw( + (AXIS_ID,), component=Microscopy.Well + ), ComponentGroupScope.dynamic(Microscopy.Site), ), consumer_variable_components=(Microscopy.Site,), @@ -1778,7 +1778,9 @@ def test_cellprofiler_adapter_keeps_multisource_current_image_grouped_when_files group_component=Microscopy.Site, ), invocation_scope=ComponentGroupScope.ungrouped(), - producer_selection_scope=ComponentGroupScope.dynamic(Microscopy.Site), + producer_selection_scope=ComponentGroupScope.dynamic( + Microscopy.Site + ), component_scopes=(ComponentGroupScope.dynamic(Microscopy.Site),), consumer_variable_components=(Microscopy.Site,), ), @@ -1867,7 +1869,9 @@ def test_cellprofiler_adapter_stacks_declared_image_input_for_pattern_group(): group_component=Microscopy.Site, ), invocation_scope=ComponentGroupScope.ungrouped(), - producer_selection_scope=ComponentGroupScope.dynamic(Microscopy.Site), + producer_selection_scope=ComponentGroupScope.dynamic( + Microscopy.Site + ), component_scopes=(ComponentGroupScope.dynamic(Microscopy.Site),), consumer_variable_components=(Microscopy.Site,), ), @@ -2288,8 +2292,12 @@ def test_cellprofiler_adapter_uses_grouped_input_when_consumer_group_is_differen ("1", "2"), component=Microscopy.Site ), component_scopes=( - ComponentGroupScope.from_raw(("1",), component=Microscopy.Channel), - ComponentGroupScope.from_raw(("1", "2"), component=Microscopy.Site), + ComponentGroupScope.from_raw( + ("1",), component=Microscopy.Channel + ), + ComponentGroupScope.from_raw( + ("1", "2"), component=Microscopy.Site + ), ), consumer_variable_components=(Microscopy.Site,), ), @@ -2594,7 +2602,9 @@ def test_cellprofiler_adapter_does_not_select_image_record_from_current_source_s ("1", "2"), component=Microscopy.Site ), component_scopes=( - ComponentGroupScope.from_raw(("1", "2"), component=Microscopy.Site), + ComponentGroupScope.from_raw( + ("1", "2"), component=Microscopy.Site + ), ), consumer_variable_components=(Microscopy.Site,), ), @@ -2654,7 +2664,9 @@ def test_cellprofiler_adapter_does_not_select_image_record_from_current_source_s def test_cellprofiler_adapter_keeps_template_scoped_object_records_grouped(): store = RuntimeValueStore() outputs = { - NUCLEI: _plan(NUCLEI, ObjectLabelsArtifactType, group_component=Microscopy.Site) + NUCLEI: _plan( + NUCLEI, ObjectLabelsArtifactType, group_component=Microscopy.Site + ) } output_bindings = ( _output_binding( @@ -2698,7 +2710,9 @@ def test_cellprofiler_adapter_keeps_template_scoped_object_records_grouped(): group_component=Microscopy.Site, ), invocation_scope=ComponentGroupScope.ungrouped(), - producer_selection_scope=ComponentGroupScope.dynamic(Microscopy.Site), + producer_selection_scope=ComponentGroupScope.dynamic( + Microscopy.Site + ), component_scopes=(ComponentGroupScope.dynamic(Microscopy.Site),), consumer_variable_components=(Microscopy.Site,), ), @@ -3051,7 +3065,9 @@ def test_cellprofiler_adapter_availability_accepts_grouped_runtime_inputs(): ("1", "2"), component=Microscopy.Site ), component_scopes=( - ComponentGroupScope.from_raw(("1", "2"), component=Microscopy.Site), + ComponentGroupScope.from_raw( + ("1", "2"), component=Microscopy.Site + ), ), consumer_variable_components=(Microscopy.Site,), ), @@ -3319,7 +3335,9 @@ def test_cellprofiler_adapter_composes_object_input_across_declared_site_axis(): ("1", "2"), component=Microscopy.Site ), component_scopes=( - ComponentGroupScope.from_raw(("1", "2"), component=Microscopy.Site), + ComponentGroupScope.from_raw( + ("1", "2"), component=Microscopy.Site + ), ), consumer_variable_components=(Microscopy.Site,), ), @@ -3411,7 +3429,9 @@ def test_cellprofiler_adapter_does_not_resolve_object_input_from_source_context( ("1", "2"), component=Microscopy.Site ), component_scopes=( - ComponentGroupScope.from_raw(("1", "2"), component=Microscopy.Site), + ComponentGroupScope.from_raw( + ("1", "2"), component=Microscopy.Site + ), ), consumer_variable_components=(Microscopy.Site,), ), @@ -3495,7 +3515,9 @@ def test_cellprofiler_adapter_preserves_ungrouped_runtime_slice_output_stack(): invocation_scope=ComponentGroupScope.ungrouped(), producer_selection_scope=ComponentGroupScope.ungrouped(), component_scopes=( - ComponentGroupScope.from_raw(("1", "2"), component=Microscopy.Site), + ComponentGroupScope.from_raw( + ("1", "2"), component=Microscopy.Site + ), ), consumer_variable_components=(Microscopy.Site,), ), @@ -3586,7 +3608,9 @@ def test_cellprofiler_adapter_stacks_dynamic_compiled_grouped_images(): paths_by_group=group_paths, ), invocation_scope=ComponentGroupScope.ungrouped(), - producer_selection_scope=ComponentGroupScope.dynamic(Microscopy.Site), + producer_selection_scope=ComponentGroupScope.dynamic( + Microscopy.Site + ), component_scopes=(ComponentGroupScope.dynamic(Microscopy.Site),), consumer_variable_components=(Microscopy.Site,), ), @@ -3987,18 +4011,17 @@ def test_cellprofiler_adapter_adds_and_reads_slice_aligned_spatial_grids(): assert isinstance(stored, RuntimeSliceAlignedValues) assert stored.slice_count == 2 - assert [stored.value_at(index).name for index in range(stored.slice_count)] == [ - "Grid", - "Grid", - ] - assert [stored.value_at(index).x_origin for index in range(stored.slice_count)] == [ - 1.0, - 2.0, - ] + assert [ + stored.value_at(index).name for index in range(stored.slice_count) + ] == ["Grid", "Grid"] + assert [ + stored.value_at(index).x_origin for index in range(stored.slice_count) + ] == [1.0, 2.0] saved_grids = filemanager.saved[("memory", "/memory/Grid.pkl")] assert isinstance(saved_grids, RuntimeSliceAlignedValues) assert [ - saved_grids.value_at(index).x_origin for index in range(saved_grids.slice_count) + saved_grids.value_at(index).x_origin + for index in range(saved_grids.slice_count) ] == [ 1.0, 2.0, @@ -4357,7 +4380,9 @@ def test_cellprofiler_adapter_does_not_select_measurement_record_from_current_so ("1", "2"), component=Microscopy.Site ), component_scopes=( - ComponentGroupScope.from_raw(("1", "2"), component=Microscopy.Site), + ComponentGroupScope.from_raw( + ("1", "2"), component=Microscopy.Site + ), ), consumer_variable_components=(Microscopy.Site,), ), @@ -4824,7 +4849,9 @@ def test_cellprofiler_adapter_aligns_multiplane_measurements_across_groups(): group_component=Microscopy.Site, ), invocation_scope=ComponentGroupScope.ungrouped(), - producer_selection_scope=ComponentGroupScope.dynamic(Microscopy.Site), + producer_selection_scope=ComponentGroupScope.dynamic( + Microscopy.Site + ), component_scopes=(ComponentGroupScope.dynamic(Microscopy.Site),), consumer_variable_components=(Microscopy.Site,), ), @@ -5250,7 +5277,9 @@ def test_cellprofiler_adapter_multiplane_measurement_alignment_is_feature_scoped group_component=Microscopy.Site, ), invocation_scope=ComponentGroupScope.ungrouped(), - producer_selection_scope=ComponentGroupScope.dynamic(Microscopy.Site), + producer_selection_scope=ComponentGroupScope.dynamic( + Microscopy.Site + ), component_scopes=(ComponentGroupScope.dynamic(Microscopy.Site),), consumer_variable_components=(Microscopy.Site,), ), @@ -6080,7 +6109,9 @@ def test_cellprofiler_module_executor_reads_objects_for_measurements( def measure_object_size_shape( image_arg: ImagePayload, *, labels: ObjectLabelValue ) -> tuple[object, DataclassMeasurementColumnarRows]: - np.testing.assert_array_equal(image_arg.data, image.data) + np.testing.assert_array_equal( + image_arg.data, image.data + ) np.testing.assert_array_equal(object_label_dense_array(labels), labels_array) return image_arg, rows @@ -6235,10 +6266,7 @@ def test_cellprofiler_object_only_pure_2d_module_executes_label_runtime_slices( @runtime_bound_parameters(SliceIndexRuntimeParameter) @with_processing_contract(Pure2DContract) def measure_object_size_shape( - image_arg: ImagePayload, - *, - labels: ObjectLabelValue, - slice_index: int | None = None, + image_arg: ImagePayload, *, labels: ObjectLabelValue, slice_index: int | None = None ) -> tuple[object, DataclassMeasurementColumnarRows]: image_pixels = np.asarray(image_arg.data) label_pixels = object_label_dense_array(labels) @@ -6953,7 +6981,9 @@ def test_measurement_lookup_uses_canonical_runtime_identifier_for_numbered_featu def test_cellprofiler_child_count_lookup_uses_parent_row_domain(): - lookup = CELLPROFILER_MEASUREMENT_DIALECT.feature_lookup("Children_PH3_Count") + lookup = CELLPROFILER_MEASUREMENT_DIALECT.feature_lookup( + "Children_PH3_Count" + ) assert lookup.query_object_name(NUCLEI) is None @@ -9655,7 +9685,9 @@ def relate_objects( ), calculate_per_parent_means: bool = False, ): - np.testing.assert_array_equal(image_arg.data, image.data) + np.testing.assert_array_equal( + image_arg.data, image.data + ) np.testing.assert_array_equal(object_label_dense_array(parent_labels), cells) np.testing.assert_array_equal(object_label_dense_array(child_labels), nuclei) return ( diff --git a/tests/unit/test_cellprofiler_runtime_callable_introspection.py b/tests/unit/test_cellprofiler_runtime_callable_introspection.py index 5d7a3db38b..e737aadada 100644 --- a/tests/unit/test_cellprofiler_runtime_callable_introspection.py +++ b/tests/unit/test_cellprofiler_runtime_callable_introspection.py @@ -187,9 +187,7 @@ def test_step_invocation_contract_provider_validates_public_cellprofiler_config( ValueError, match="unknown object_labels artifact 'Objects1'", ): - CellProfilerInvocationContractProviderFactory.provider_for_pipeline( - session.pipeline - ) + CellProfilerInvocationContractProviderFactory.provider_for_pipeline(session.pipeline) def test_public_compile_time_provider_binds_cellprofiler_runtime_from_step_declarations(): diff --git a/tests/unit/test_cellprofiler_save_images.py b/tests/unit/test_cellprofiler_save_images.py index 086c4d3c27..cef5b6aba2 100644 --- a/tests/unit/test_cellprofiler_save_images.py +++ b/tests/unit/test_cellprofiler_save_images.py @@ -217,9 +217,7 @@ def test_save_images_is_an_adapter_free_executable_axis_module() -> None: assert callable_contract.processing_contract is Pure3DContract assert callable_contract.runtime_adapter is None assert callable_contract.resolve_runtime_callable() is save_images - assert CallableContract.from_callable( - save_images - ).artifact_input_parameter_names == ("image_to_save",) + assert CallableContract.from_callable(save_images).artifact_input_parameter_names == ("image_to_save",) def test_save_images_contract_consumes_runtime_image_and_declares_export_only() -> None: @@ -269,9 +267,7 @@ def test_save_images_selected_image_binding_is_runtime_special_identity() -> Non assert bound.kwargs["file_format"] is SaveImagesFileFormat.PNG assert bound.kwargs["bit_depth"] is SaveImagesBitDepth.UINT8 assert bound.kwargs["filename_method"] is SaveImagesFilenameMethod.SINGLE_NAME - assert CallableContract.from_callable( - save_images - ).artifact_input_parameter_names == ("image_to_save",) + assert CallableContract.from_callable(save_images).artifact_input_parameter_names == ("image_to_save",) def test_save_images_public_contract_selects_upstream_runtime_image() -> None: @@ -958,12 +954,8 @@ def test_image_output_context_projects_declared_group_lineage_source() -> None: ), ) - contextualized = ( - ImageArtifactType if output_plan is None else output_plan.artifact_type - ).contextualize_output( - (source_slices).project_declared_source( - (output_plan.source_context_source()).name - ), + contextualized = (ImageArtifactType if output_plan is None else output_plan.artifact_type).contextualize_output( + (source_slices).project_declared_source((output_plan.source_context_source()).name), np.zeros((3, 2, 2, 3), dtype=np.uint8), output_plan, RuntimePlaneAxisValueProjection.preserve( @@ -1037,12 +1029,8 @@ def test_declared_group_lineage_source_overrides_complete_output_identity() -> N ), ) - contextualized = ( - ImageArtifactType if output_plan is None else output_plan.artifact_type - ).contextualize_output( - (source_slices).project_declared_source( - (output_plan.source_context_source()).name - ), + contextualized = (ImageArtifactType if output_plan is None else output_plan.artifact_type).contextualize_output( + (source_slices).project_declared_source((output_plan.source_context_source()).name), complete_output, output_plan, RuntimePlaneAxisValueProjection.preserve( @@ -1084,9 +1072,7 @@ def test_planned_image_output_restores_axis_without_duplicate_provenance() -> No ), ) - contextualized = ( - ImageArtifactType if output_plan is None else output_plan.artifact_type - ).contextualize_output( + contextualized = (ImageArtifactType if output_plan is None else output_plan.artifact_type).contextualize_output( source_payload, source_payload, output_plan, @@ -1134,12 +1120,8 @@ def test_image_output_context_removes_selected_source_runtime_axis() -> None: ), ) - contextualized = ( - ImageArtifactType if output_plan is None else output_plan.artifact_type - ).contextualize_output( - (source_payload).project_declared_source( - (output_plan.source_context_source()).name - ), + contextualized = (ImageArtifactType if output_plan is None else output_plan.artifact_type).contextualize_output( + (source_payload).project_declared_source((output_plan.source_context_source()).name), np.zeros((2, 2, 3), dtype=np.uint8), output_plan, RuntimePlaneAxisValueProjection.preserve( @@ -1175,12 +1157,8 @@ def test_image_output_context_preserves_repeated_declared_source_planes() -> Non ), ) - contextualized = ( - ImageArtifactType if output_plan is None else output_plan.artifact_type - ).contextualize_output( - (source_payload).project_declared_source( - (output_plan.source_context_source()).name - ), + contextualized = (ImageArtifactType if output_plan is None else output_plan.artifact_type).contextualize_output( + (source_payload).project_declared_source((output_plan.source_context_source()).name), np.zeros((2, 2, 2), dtype=np.uint8), output_plan, RuntimePlaneAxisValueProjection.preserve( @@ -1214,9 +1192,7 @@ def test_image_output_context_accepts_exact_loaded_derived_artifact() -> None: relations=(SourceStackLineageSourceRelation(source=selected_artifact.ref()),), ) - contextualized = ( - ImageArtifactType if output_plan is None else output_plan.artifact_type - ).contextualize_output( + contextualized = (ImageArtifactType if output_plan is None else output_plan.artifact_type).contextualize_output( source_payload, np.ones((2, 2), dtype=np.uint8), output_plan, diff --git a/tests/unit/test_cellprofiler_source_bindings.py b/tests/unit/test_cellprofiler_source_bindings.py index bfa19fa2bb..c33bb9c172 100644 --- a/tests/unit/test_cellprofiler_source_bindings.py +++ b/tests/unit/test_cellprofiler_source_bindings.py @@ -1065,28 +1065,18 @@ def test_setup_fold_is_source_ordered_and_uses_module_registry() -> None: ) -@pytest.mark.parametrize( - "dtype,foreground", [(np.uint8, 255), (np.uint16, 65535), (np.float32, 0.25)] -) -def test_declared_binary_source_preserves_foreground_through_cp_normalization( - dtype, foreground -): - from openhcs.interop.cellprofiler.image_normalization import ( - normalize_cellprofiler_image_payload, - ) - from openhcs.processing.backends.cellprofiler.image_geometry import ( - binary_mask_plane, - ) +@pytest.mark.parametrize("dtype,foreground", [(np.uint8, 255), (np.uint16, 65535), (np.float32, 0.25)]) +def test_declared_binary_source_preserves_foreground_through_cp_normalization(dtype, foreground): + from openhcs.interop.cellprofiler.image_normalization import normalize_cellprofiler_image_payload + from openhcs.processing.backends.cellprofiler.image_geometry import binary_mask_plane raw = np.asarray([[0, foreground], [foreground, 0]], dtype=dtype) - context = ImagePayloadSourceMetadataContext(SourceImageIdentity("/input/mask.tif")) - loaded = NamedSourceBinding(alias="Mask", load_as_mask=True).apply_loaded_payload( - raw, context - ) + context = ImagePayloadSourceMetadataContext(SourceImageIdentity('/input/mask.tif')) + loaded = NamedSourceBinding(alias='Mask', load_as_mask=True).apply_loaded_payload(raw, context) source_metadata = loaded.metadata normalized = normalize_cellprofiler_image_payload(loaded) np.testing.assert_array_equal(normalized.data, raw != 0) np.testing.assert_array_equal(binary_mask_plane(normalized), raw != 0) assert source_metadata.unit_interval_intensity.scale == 1 assert normalized.metadata.source_provenance == source_metadata.source_provenance - assert source_metadata.source_path == "/input/mask.tif" + assert source_metadata.source_path == '/input/mask.tif' diff --git a/tests/unit/test_cellprofiler_source_bindings_ingestion.py b/tests/unit/test_cellprofiler_source_bindings_ingestion.py index 2d02e9c5c6..d5efb73a41 100644 --- a/tests/unit/test_cellprofiler_source_bindings_ingestion.py +++ b/tests/unit/test_cellprofiler_source_bindings_ingestion.py @@ -134,12 +134,9 @@ def test_axis_projection_reads_each_axis_through_its_declarations() -> None: assert source_metadata_component(alias) is axis primary = axis.metadata_aliases[0] assert SourceAxisProjection.project(axis, {primary: "B07"}, 0) == "B07" - assert ( - SourceAxisProjection.project( - Microscopy.Well, {"Metadata_Row": "c", "Metadata_Column": "4"}, 0 - ) - == "C04" - ) + assert SourceAxisProjection.project( + Microscopy.Well, {"Metadata_Row": "c", "Metadata_Column": "4"}, 0 + ) == "C04" assert SourceAxisProjection.project(Microscopy.Well, {}, 3) == "A01" assert SourceAxisProjection.project(Microscopy.Channel, {}, 3) == "4" assert SourceAxisProjection.project(Microscopy.ZIndex, {}, 3) == "1" diff --git a/tests/unit/test_cellprofiler_special_io_declarations.py b/tests/unit/test_cellprofiler_special_io_declarations.py index e41ad30ef0..cf2b61bacb 100644 --- a/tests/unit/test_cellprofiler_special_io_declarations.py +++ b/tests/unit/test_cellprofiler_special_io_declarations.py @@ -731,7 +731,10 @@ def test_fixed_return_slots_follow_nominal_contract_output_order() -> None: f"{diagnostic_prefix}__{stage}" for stage in PrimaryObjectDiagnosticPlanes._fields ) - assert all(spec.sidecar_role is QaCheckpoint for spec in diagnostic_outputs) + assert all( + spec.sidecar_role is QaCheckpoint + for spec in diagnostic_outputs + ) def test_contract_order_owns_guided_grid_and_tertiary_input_mapping() -> None: @@ -827,13 +830,7 @@ def test_object_colocalization_rank_provider_is_runtime_bound() -> None: parameter = parameters["rank_provider"] assert parameter.kind is inspect.Parameter.KEYWORD_ONLY - assert CallableContract.from_callable( - measure_colocalization_objects - ).runtime_bound_parameters == ( - "rank_provider", - "threshold_mask_outputs", - "costes_threshold_batch", - ) + assert CallableContract.from_callable(measure_colocalization_objects).runtime_bound_parameters == ("rank_provider", "threshold_mask_outputs", "costes_threshold_batch") contract = CallableContract.from_callable(measure_colocalization_objects) assert contract.config_bound_parameter_names == ("dtype_config",) assert parameters["dtype_config"].annotation is LazyDtypeConfig diff --git a/tests/unit/test_cellprofiler_spreadsheet_export.py b/tests/unit/test_cellprofiler_spreadsheet_export.py index a4c86829f3..85e5679579 100644 --- a/tests/unit/test_cellprofiler_spreadsheet_export.py +++ b/tests/unit/test_cellprofiler_spreadsheet_export.py @@ -108,7 +108,6 @@ from openhcs.core.axes import Axis from openhcs.domains.microscopy.axes import Microscopy - def test_export_to_spreadsheet_declares_exact_plate_callable_abi() -> None: contract = CallableContract.from_callable(export_to_spreadsheet) parameter = inspect.signature(export_to_spreadsheet).parameters["artifact_batch"] @@ -422,176 +421,136 @@ def test_export_to_spreadsheet_renders_only_declared_batch_records() -> None: @pytest.mark.parametrize("add_files", (False, True)) @pytest.mark.parametrize("axisless", (False, True)) def test_spreadsheet_projects_source_identity_without_upstream_image_features( - add_metadata: bool, - add_files: bool, - axisless: bool, + add_metadata: bool, add_files: bool, axisless: bool, ) -> None: - provenance = SourceImageProvenancePlanes( - ( - RuntimeSourceImageProvenancePlane( - SourceImageIdentity( - "/acquisition/body.tif", - { - "well": "A01", - "site": "2", - "channel": "3", - "z_index": "4", - "timepoint": "5", - DECLARED_SOURCE_METADATA_FIELD: {"Treatment": "control"}, - }, - ), - source_image_name="Body", - ), - ) - ) + provenance = SourceImageProvenancePlanes(( + RuntimeSourceImageProvenancePlane( + SourceImageIdentity('/acquisition/body.tif', { + 'well': 'A01', 'site': '2', 'channel': '3', 'z_index': '4', + 'timepoint': '5', DECLARED_SOURCE_METADATA_FIELD: {'Treatment': 'control'}, + }), + source_image_name='Body', + ), + )) cells = _measurement_record( - "cells", - axis_id="A01", - subject=MeasurementSubject(MeasurementScope.OBJECT, "Cells", "object_number"), - rows=( - { - **({} if axisless else {"slice_index": 0}), - "object_number": 7, - "Area": 12.0, - }, - ), + 'cells', axis_id='A01', + subject=MeasurementSubject(MeasurementScope.OBJECT, 'Cells', 'object_number'), + rows=({**({} if axisless else {'slice_index': 0}), 'object_number': 7, 'Area': 12.0},), source_image_provenance_planes=provenance, ) bundle = render_spreadsheet_bundle( artifact_batch=RuntimeArtifactBatch( - input_specs=(ArtifactSpec.input("cells", MeasurementsArtifactType),), - records_by_axis={"A01": (cells,)}, + input_specs=(ArtifactSpec.input('cells', MeasurementsArtifactType),), + records_by_axis={'A01': (cells,)}, source_image_set_identity_policy=SourceImageSetIdentityPolicy(), ), - add_image_metadata=add_metadata, - add_image_file_names=add_files, + add_image_metadata=add_metadata, add_image_file_names=add_files, add_filename_prefix=False, ) - (image,) = csv.DictReader(io.StringIO(bundle["Image.csv"])) - (cell,) = csv.DictReader(io.StringIO(bundle["Cells.csv"])) - assert image["Metadata_Treatment"] == "control" - assert ("Metadata_site" in image) is add_metadata - assert ("FileName_Body" in image) is add_files + image, = csv.DictReader(io.StringIO(bundle['Image.csv'])) + cell, = csv.DictReader(io.StringIO(bundle['Cells.csv'])) + assert image['Metadata_Treatment'] == 'control' + assert ('Metadata_site' in image) is add_metadata + assert ('FileName_Body' in image) is add_files if add_metadata: - assert image["Metadata_well"] == "A01" - assert image["Metadata_site"] == "2" - assert image["Metadata_z_index"] == "4" - assert image["Metadata_timepoint"] == "5" + assert image['Metadata_well'] == 'A01' + assert image['Metadata_site'] == '2' + assert image['Metadata_z_index'] == '4' + assert image['Metadata_timepoint'] == '5' if add_files: - assert image["FileName_Body"] == "body.tif" - assert image["PathName_Body"] == "/acquisition" - assert cell["object_number"] == "7" - assert cell["Area"] == "12.0" - assert ("Metadata_site" in cell) is add_metadata - assert ("Image_FileName_Body" in cell) is add_files + assert image['FileName_Body'] == 'body.tif' + assert image['PathName_Body'] == '/acquisition' + assert cell['object_number'] == '7' + assert cell['Area'] == '12.0' + assert ('Metadata_site' in cell) is add_metadata + assert ('Image_FileName_Body' in cell) is add_files if add_files: - assert cell["Image_FileName_Body"] == "body.tif" - assert cell["Image_PathName_Body"] == "/acquisition" - - -@pytest.mark.parametrize("names", (("Body", "Nuclear"), ("Nuclear", "Body"))) -def test_spreadsheet_preserves_independent_contributor_filenames( - names: tuple[str, ...], -) -> None: - provenance = SourceImageProvenancePlanes( - ( - RuntimeSourceImageProvenancePlane( - SourceImageIdentity(component_metadata={"well": "A01", "site": "2"}), - contributors=tuple( - SourceImageProvenanceContributor( - SourceImageIdentity( - f"/acquisition/{name}.tif", {"well": "A01", "site": "2"} - ), - source_image_name=name, - ) - for name in names - ), + assert cell['Image_FileName_Body'] == 'body.tif' + assert cell['Image_PathName_Body'] == '/acquisition' + + +@pytest.mark.parametrize('names', (('Body', 'Nuclear'), ('Nuclear', 'Body'))) +def test_spreadsheet_preserves_independent_contributor_filenames(names: tuple[str, ...]) -> None: + provenance = SourceImageProvenancePlanes(( + RuntimeSourceImageProvenancePlane( + SourceImageIdentity(component_metadata={'well': 'A01', 'site': '2'}), + contributors=tuple( + SourceImageProvenanceContributor( + SourceImageIdentity(f'/acquisition/{name}.tif', {'well': 'A01', 'site': '2'}), + source_image_name=name, + ) + for name in names ), - ) - ) + ), + )) record = _measurement_record( - "cells", - axis_id="A01", - subject=MeasurementSubject(MeasurementScope.OBJECT, "Cells", "object_number"), - rows=({"slice_index": 0, "object_number": 1, "Area": 12.0},), + 'cells', axis_id='A01', + subject=MeasurementSubject(MeasurementScope.OBJECT, 'Cells', 'object_number'), + rows=({'slice_index': 0, 'object_number': 1, 'Area': 12.0},), source_image_provenance_planes=provenance, ) bundle = render_spreadsheet_bundle( artifact_batch=RuntimeArtifactBatch( - input_specs=(ArtifactSpec.input("cells", MeasurementsArtifactType),), - records_by_axis={"A01": (record,)}, + input_specs=(ArtifactSpec.input('cells', MeasurementsArtifactType),), + records_by_axis={'A01': (record,)}, source_image_set_identity_policy=SourceImageSetIdentityPolicy(), ), - add_image_metadata=True, - add_image_file_names=True, - add_filename_prefix=False, + add_image_metadata=True, add_image_file_names=True, add_filename_prefix=False, ) - (row,) = csv.DictReader(io.StringIO(bundle["Cells.csv"])) + row, = csv.DictReader(io.StringIO(bundle['Cells.csv'])) for name in names: - assert row[f"Image_FileName_{name}"] == f"{name}.tif" - assert row[f"Image_PathName_{name}"] == "/acquisition" - assert "Metadata_channel" not in row - assert "Metadata_z_index" not in row + assert row[f'Image_FileName_{name}'] == f'{name}.tif' + assert row[f'Image_PathName_{name}'] == '/acquisition' + assert 'Metadata_channel' not in row + assert 'Metadata_z_index' not in row def test_requested_source_columns_preserve_original_extraction_path_template() -> None: image = _measurement_record( - "image", - axis_id="A01", - subject=MeasurementSubject(MeasurementScope.SAMPLE, "Image"), - rows=({"slice_index": 0, "Count_Cells": 1},), + 'image', axis_id='A01', + subject=MeasurementSubject(MeasurementScope.SAMPLE, 'Image'), + rows=({'slice_index': 0, 'Count_Cells': 1},), source_image_provenance_planes=SourceImageProvenancePlanes.from_components( - component_metadata=( - {"site": "1", DECLARED_SOURCE_METADATA_FIELD: {"Run": "run1"}}, - ), + component_metadata=({'site': '1', DECLARED_SOURCE_METADATA_FIELD: {'Run': 'run1'}},), ), ) cells = _measurement_record( - "cells", - axis_id="A01", - subject=MeasurementSubject(MeasurementScope.OBJECT, "Cells", "object_number"), - rows=({"slice_index": 0, "object_number": 1, "Area": 12.0},), + 'cells', axis_id='A01', + subject=MeasurementSubject(MeasurementScope.OBJECT, 'Cells', 'object_number'), + rows=({'slice_index': 0, 'object_number': 1, 'Area': 12.0},), ) bundle = render_spreadsheet_bundle( artifact_batch=RuntimeArtifactBatch( - input_specs=tuple( - ArtifactSpec.input(record.key.name, MeasurementsArtifactType) - for record in (image, cells) - ), - records_by_axis={"A01": (image, cells)}, + input_specs=tuple(ArtifactSpec.input(record.key.name, MeasurementsArtifactType) for record in (image, cells)), + records_by_axis={'A01': (image, cells)}, source_image_set_identity_policy=SourceImageSetIdentityPolicy(), ), - add_image_metadata=True, - add_image_file_names=True, - output_directory="{Run}", - add_filename_prefix=False, + add_image_metadata=True, add_image_file_names=True, + output_directory='{Run}', add_filename_prefix=False, ) - assert set(bundle) == {"run1/Image.csv", "run1/Cells.csv"} - (row,) = csv.DictReader(io.StringIO(bundle["run1/Cells.csv"])) - assert row["Metadata_Run"] == "run1" - assert row["Metadata_site"] == "1" + assert set(bundle) == {'run1/Image.csv', 'run1/Cells.csv'} + row, = csv.DictReader(io.StringIO(bundle['run1/Cells.csv'])) + assert row['Metadata_Run'] == 'run1' + assert row['Metadata_site'] == '1' def test_spreadsheet_rejects_existing_filename_conflicting_with_provenance() -> None: record = _measurement_record( - "image", - axis_id="A01", - subject=MeasurementSubject(MeasurementScope.SAMPLE, "Image"), - rows=({"slice_index": 0, "FileName_Body": "other.tif"},), - source_image_provenance_planes=SourceImageProvenancePlanes( - ( - RuntimeSourceImageProvenancePlane( - SourceImageIdentity("/acquisition/body.tif", {"site": "1"}), - source_image_name="Body", - ), - ) - ), + 'image', axis_id='A01', + subject=MeasurementSubject(MeasurementScope.SAMPLE, 'Image'), + rows=({'slice_index': 0, 'FileName_Body': 'other.tif'},), + source_image_provenance_planes=SourceImageProvenancePlanes(( + RuntimeSourceImageProvenancePlane( + SourceImageIdentity('/acquisition/body.tif', {'site': '1'}), + source_image_name='Body', + ), + )), ) - with pytest.raises(ValueError, match="Conflicting sparse measurement values"): + with pytest.raises(ValueError, match='Conflicting sparse measurement values'): render_spreadsheet_bundle( artifact_batch=RuntimeArtifactBatch( - input_specs=(ArtifactSpec.input("image", MeasurementsArtifactType),), - records_by_axis={"A01": (record,)}, + input_specs=(ArtifactSpec.input('image', MeasurementsArtifactType),), + records_by_axis={'A01': (record,)}, source_image_set_identity_policy=SourceImageSetIdentityPolicy(), ), add_image_file_names=True, @@ -623,7 +582,6 @@ def test_export_to_spreadsheet_bundle_uses_generic_file_materialization() -> Non context=context, ) from openhcs.processing.materialization.core import ColumnarCsvOutput - assert all(isinstance(output, ColumnarCsvOutput) for output in bundle.values()) outputs = materialization_outputs( MaterializationSpec(FileBundleOptions()), @@ -1944,11 +1902,11 @@ def _measurement_record( axis_id=axis_id, ) return StoredRuntimeValue( - key=value.key, - data=value.data, - materialization_source_metadata=value.materialization_source_metadata, - location=RuntimeArtifactLocation(path=output_plan.path, backend="memory"), - ) + key=value.key, + data=value.data, + materialization_source_metadata=value.materialization_source_metadata, + location=RuntimeArtifactLocation(path=output_plan.path, backend="memory"), + ) def _fixture_field_dtype( @@ -1995,11 +1953,11 @@ def _relationship_record(name: str, *, axis_id: str) -> StoredRuntimeValue: axis_id=axis_id, ) return StoredRuntimeValue( - key=value.key, - data=value.data, - materialization_source_metadata=value.materialization_source_metadata, - location=RuntimeArtifactLocation(path=output_plan.path, backend="memory"), - ) + key=value.key, + data=value.data, + materialization_source_metadata=value.materialization_source_metadata, + location=RuntimeArtifactLocation(path=output_plan.path, backend="memory"), + ) @pytest.mark.parametrize("cycle_aligned", (False, True)) @@ -2119,26 +2077,19 @@ def test_image_number_references_follow_exact_source_numbering( from openhcs.core.measurement_row_materialization import ConcatenatedColumnarRows fields = tuple( - ( - FieldSpec(name, int) - if name in ("slice_index", "object_number") - else FieldSpec(name, required=False) - ) + FieldSpec(name, int) + if name in ("slice_index", "object_number") + else FieldSpec(name, required=False) for name in columns ) source_rows = ( - ConcatenatedColumnarRows( - tuple( - MeasurementSparseColumnarRows( - { - name: values[start : start + 3] - for name, values in columns.items() - }, - fields=fields, - ) - for start in (0, 3) + ConcatenatedColumnarRows(tuple( + MeasurementSparseColumnarRows( + {name: values[start:start + 3] for name, values in columns.items()}, + fields=fields, ) - ) + for start in (0, 3) + )) if concatenated else MeasurementSparseColumnarRows(columns, fields=fields) ) @@ -2185,7 +2136,9 @@ def columns(**values): name = "Mean_Cells_Tracking_ParentImageNumber_50" source = OrderedDict( Image=columns(image_number=(23,), **{name: (0.5,)}), - Cells=columns(image_number=(23, 23), Tracking_ParentImageNumber_50=(0, 22)), + Cells=columns( + image_number=(23, 23), Tracking_ParentImageNumber_50=(0, 22) + ), ) selected = OrderedDict(Image=source["Image"]) result = _with_requested_aggregates( diff --git a/tests/unit/test_cellprofiler_watershed_provider.py b/tests/unit/test_cellprofiler_watershed_provider.py index c1ac6c0eea..f94ae6e23d 100644 --- a/tests/unit/test_cellprofiler_watershed_provider.py +++ b/tests/unit/test_cellprofiler_watershed_provider.py @@ -6,9 +6,7 @@ import pytest from openhcs.constants.constants import MemoryType -from openhcs.processing.backends.cellprofiler._backend import ( - CellProfilerBackendProvider, -) +from openhcs.processing.backends.cellprofiler._backend import CellProfilerBackendProvider from openhcs.processing.backends.cellprofiler.watershed import ( LegacyWatershedBackendStrategy, NumbaNumpyLegacyWatershedBackendStrategy, @@ -33,9 +31,8 @@ def reject_centrosome(name, *args, **kwargs): MemoryType.NUMPY, backend_provider=provider ) assert strategy.backend_provider is provider.provider - assert ( - provider.provider - in LegacyWatershedBackendStrategy.available_backend_providers(MemoryType.NUMPY) + assert provider.provider in LegacyWatershedBackendStrategy.available_backend_providers( + MemoryType.NUMPY ) image = np.array([[0.0, 1.0, 0.0]]) markers = marker_sign * np.array([[1, 0, 2]], dtype=np.int32) @@ -135,27 +132,25 @@ def observe_request(self, image, *, markers, mask, connectivity=1): ) ).astype(np.float32) original_image = image.copy() - returned_image, measurements, payload, *_diagnostic_planes = ( - identify_primary_objects( - image, - min_diameter=4, - max_diameter=14, - exclude_size=False, - exclude_border_objects=False, - unclump_method=UnclumpMethod.INTENSITY, - watershed_method=WatershedMethod.INTENSITY, - automatic_smoothing=False, - smoothing_filter_size=0, - automatic_suppression=False, - maxima_suppression_size=3, - low_res_maxima=False, - fill_holes=FillHolesOption.NEVER, - threshold_method=CellProfilerThresholdMethod.MANUAL, - manual_threshold=0.2, - threshold_smoothing_scale=0, - morphology_backend_provider=CellProfilerBackendProvider.CENTROSOME, - watershed_backend_provider=provider, - ) + returned_image, measurements, payload, *_diagnostic_planes = identify_primary_objects( + image, + min_diameter=4, + max_diameter=14, + exclude_size=False, + exclude_border_objects=False, + unclump_method=UnclumpMethod.INTENSITY, + watershed_method=WatershedMethod.INTENSITY, + automatic_smoothing=False, + smoothing_filter_size=0, + automatic_suppression=False, + maxima_suppression_size=3, + low_res_maxima=False, + fill_holes=FillHolesOption.NEVER, + threshold_method=CellProfilerThresholdMethod.MANUAL, + manual_threshold=0.2, + threshold_smoothing_scale=0, + morphology_backend_provider=CellProfilerBackendProvider.CENTROSOME, + watershed_backend_provider=provider, ) (selected_backend,) = selected (request,) = requests @@ -166,10 +161,7 @@ def observe_request(self, image, *, markers, mask, connectivity=1): assert labels.dtype == np.int32 assert labels.shape == image.shape assert np.unique(labels).tolist() == [0, 1, 2] - assert payload.domain.require_explicit_id_domain(context="primary journey") == ( - 1, - 2, - ) + assert payload.domain.require_explicit_id_domain(context="primary journey") == (1, 2) np.testing.assert_array_equal(labels > 0, image > 0.2) np.testing.assert_array_equal(returned_image, original_image) np.testing.assert_array_equal(image, original_image) diff --git a/tests/unit/test_cellprofiler_workspace_export.py b/tests/unit/test_cellprofiler_workspace_export.py index b755968869..ad64ca2a3c 100644 --- a/tests/unit/test_cellprofiler_workspace_export.py +++ b/tests/unit/test_cellprofiler_workspace_export.py @@ -1,3 +1,4 @@ + from tests.unit.saved_output_dialect import SAVED_OUTPUT_DIALECT from dataclasses import replace from pathlib import Path @@ -253,9 +254,7 @@ def test_workspace_comparison_coverage_is_physical_and_separate_from_science(tmp for root in (reference, candidate) ) report = cellprofiler_database_export_equivalence( - reference, - exports[1], - policy=RuntimeEquivalencePolicy(measurement_dialect=SAVED_OUTPUT_DIALECT), + reference, exports[1], policy=RuntimeEquivalencePolicy(measurement_dialect=SAVED_OUTPUT_DIALECT) ) assert report.is_equivalent assert report.compared_output_files == frozenset(paths) @@ -272,9 +271,7 @@ def test_workspace_comparison_coverage_is_physical_and_separate_from_science(tmp _require_compared_output_inventory(**kwargs, compared_file_report=report) paths[1].write_text(paths[1].read_text().replace("Image_", "Changed_", 1)) changed = cellprofiler_database_export_equivalence( - reference, - exports[1], - policy=RuntimeEquivalencePolicy(measurement_dialect=SAVED_OUTPUT_DIALECT), + reference, exports[1], policy=RuntimeEquivalencePolicy(measurement_dialect=SAVED_OUTPUT_DIALECT) ) assert not changed.is_equivalent assert changed.compared_output_files == frozenset(paths) @@ -303,9 +300,7 @@ def test_unmatched_workspace_and_reader_failures_never_claim_coverage( extra.write_bytes(fixture.read_bytes()) exports = RuntimeExportObservation.from_output_root(candidate) report = cellprofiler_database_export_equivalence( - reference, - exports, - policy=RuntimeEquivalencePolicy(measurement_dialect=SAVED_OUTPUT_DIALECT), + reference, exports, policy=RuntimeEquivalencePolicy(measurement_dialect=SAVED_OUTPUT_DIALECT) ) assert not report.is_equivalent assert not report.compared_output_files @@ -315,9 +310,7 @@ def test_unmatched_workspace_and_reader_failures_never_claim_coverage( extra.write_text("not a CPA workspace") with pytest.raises(ValueError): cellprofiler_database_export_equivalence( - reference, - exports, - policy=RuntimeEquivalencePolicy(measurement_dialect=SAVED_OUTPUT_DIALECT), + reference, exports, policy=RuntimeEquivalencePolicy(measurement_dialect=SAVED_OUTPUT_DIALECT) ) extra.write_bytes(fixture.read_bytes()) original_read = Path.read_text @@ -331,9 +324,7 @@ def fail_read(path, *args, **kwargs): monkeypatch.setattr(Path, "read_text", fail_read) with pytest.raises(OSError) as raised: cellprofiler_database_export_equivalence( - reference, - exports, - policy=RuntimeEquivalencePolicy(measurement_dialect=SAVED_OUTPUT_DIALECT), + reference, exports, policy=RuntimeEquivalencePolicy(measurement_dialect=SAVED_OUTPUT_DIALECT) ) assert raised.value is failure diff --git a/tests/unit/test_classification_two_vector_authoring.py b/tests/unit/test_classification_two_vector_authoring.py index 5ce03e08a5..9acf7d0341 100644 --- a/tests/unit/test_classification_two_vector_authoring.py +++ b/tests/unit/test_classification_two_vector_authoring.py @@ -11,11 +11,7 @@ import numpy as np import pytest -from openhcs.agent.dto.pipeline import ( - FunctionSpecRef, - FunctionStepAddRequest, - FunctionStepSpec, -) +from openhcs.agent.dto.pipeline import FunctionSpecRef, FunctionStepAddRequest, FunctionStepSpec from openhcs.agent.services.pipeline_authoring_service import PipelineAuthoringService from openhcs.core.function_patterns import normalize_function_pattern from openhcs.core.runtime_image_values import ImagePayloadMetadata @@ -80,9 +76,9 @@ def test_original_mode_projections_have_no_case_dispatch(projection): for node in ast.walk(tree) if isinstance(node, (ast.If, ast.IfExp, ast.Match, ast.Compare, ast.Dict)) ) - assert ( - not forbidden - ), f"{projection} still decides case behavior: {[type(node).__name__ for node in forbidden]}" + assert not forbidden, ( + f"{projection} still decides case behavior: {[type(node).__name__ for node in forbidden]}" + ) member_names = frozenset(ClassificationMethod.__members__) assert not any( isinstance(node, ast.Attribute) and node.attr in member_names @@ -300,9 +296,7 @@ def test_original_parsed_pair_runtime_binds_only_its_declared_signature(): @pytest.mark.parametrize("clean", (False, True)) -def test_original_public_016_json_kwargs_reach_strict_020_callable_abi( - clean, monkeypatch -): +def test_original_public_016_json_kwargs_reach_strict_020_callable_abi(clean, monkeypatch): """Exact live request's kwargs, without plate/native/registration replay.""" function_id, metadata = RegistryService.declared_metadata_for_callable( classify_objects_two_measurements @@ -311,7 +305,7 @@ def test_original_public_016_json_kwargs_reach_strict_020_callable_abi( monkeypatch.setattr(RegistryService, "_resolved_reference_callables", {}) author = PipelineAuthoringService(function_catalog=SelectedDeclarations()) ref = author.create_pipeline() - kwargs = json.loads("""{ + kwargs = json.loads('''{ "select_the_object_to_be_classified": "engineering_cells", "measurement1_feature": "pixel_count", "measurement2_feature": "calibration_um", @@ -319,39 +313,22 @@ def test_original_public_016_json_kwargs_reach_strict_020_callable_abi( "threshold2_method": "custom", "threshold2_value": 2.0, "low_low_name": "low_low", "high_low_name": "high_low", "low_high_name": "low_high", "high_high_name": "high_high" - }""") - author.add_function_step_from_request( - FunctionStepAddRequest.from_fields( - pipeline_id=ref.pipeline_id, - function_id=function_id, - name="engineering_pair_classification", - kwargs=kwargs, - ) - ) + }''') + author.add_function_step_from_request(FunctionStepAddRequest.from_fields( + pipeline_id=ref.pipeline_id, function_id=function_id, + name="engineering_pair_classification", kwargs=kwargs, + )) assert author.validate(ref).valid - document = PipelineDocumentCodec.from_source( - author.render_source(ref, clean=clean).source - ) - invocation = next( - normalize_function_pattern(document.pipeline_steps[0].func).iter_items() - ) + document = PipelineDocumentCodec.from_source(author.render_source(ref, clean=clean).source) + invocation = next(normalize_function_pattern(document.pipeline_steps[0].func).iter_items()) values = invocation.kwargs_dict assert values["threshold1_method"] is ClassificationThresholdMethod.CUSTOM assert values["threshold2_method"] is ClassificationThresholdMethod.CUSTOM assert (values["threshold1_value"], values["threshold2_value"]) == (100.0, 2.0) - assert (values["measurement1_feature"], values["measurement2_feature"]) == ( - "pixel_count", - "calibration_um", - ) + assert (values["measurement1_feature"], values["measurement2_feature"]) == ("pixel_count", "calibration_um") assert values["select_the_object_to_be_classified"] == "engineering_cells" - executable = { - name: value - for name, value in values.items() - if name != "select_the_object_to_be_classified" - } - assert invocation.contract.validate_public_kwargs(executable) == tuple( - executable.items() - ) + executable = {name: value for name, value in values.items() if name != "select_the_object_to_be_classified"} + assert invocation.contract.validate_public_kwargs(executable) == tuple(executable.items()) @pytest.mark.parametrize( @@ -594,12 +571,10 @@ def test_paired_author_validate_render_parse_compile_and_original_runtime( author = PipelineAuthoringService(function_catalog=SelectedDeclarations()) authored_kwargs = paired_kwargs() if json_kwargs: - authored_kwargs = json.loads( - json.dumps( - authored_kwargs, - default=lambda member: member.value, - ) - ) + authored_kwargs = json.loads(json.dumps( + authored_kwargs, + default=lambda member: member.value, + )) ref = author.create_pipeline( steps=( FunctionStepSpec( @@ -614,14 +589,8 @@ def test_paired_author_validate_render_parse_compile_and_original_runtime( invocation = next( normalize_function_pattern(restored.pipeline_steps[0].func).iter_items() ) - assert ( - invocation.kwargs_dict["threshold1_method"] - is ClassificationThresholdMethod.CUSTOM - ) - assert ( - invocation.kwargs_dict["threshold2_method"] - is ClassificationThresholdMethod.CUSTOM - ) + assert invocation.kwargs_dict["threshold1_method"] is ClassificationThresholdMethod.CUSTOM + assert invocation.kwargs_dict["threshold2_method"] is ClassificationThresholdMethod.CUSTOM contract = _public_function_step_contract( ClassifyObjectsSingleMeasurementModule, invocation.contract.func, diff --git a/tests/unit/test_collapsed_persisted_occurrence_435.py b/tests/unit/test_collapsed_persisted_occurrence_435.py index dd4e4070cf..e513f32f86 100644 --- a/tests/unit/test_collapsed_persisted_occurrence_435.py +++ b/tests/unit/test_collapsed_persisted_occurrence_435.py @@ -37,9 +37,7 @@ ) from openhcs.core.source_workspace_projection import VirtualWorkspaceSourceProjection from openhcs.core.steps.function_output_identity import FunctionOutputIdentity -from openhcs.core.steps.function_artifact_materialization import ( - ArtifactMaterializationTargetPlan, -) +from openhcs.core.steps.function_artifact_materialization import ArtifactMaterializationTargetPlan from openhcs.core.steps.function_outputs import ( PrimaryImageMetadataTarget, ) diff --git a/tests/unit/test_colocalization_metric_setting_authority.py b/tests/unit/test_colocalization_metric_setting_authority.py index e5ae55a195..59d99fede6 100644 --- a/tests/unit/test_colocalization_metric_setting_authority.py +++ b/tests/unit/test_colocalization_metric_setting_authority.py @@ -153,7 +153,7 @@ def test_recorded_source_pair_features_exclude_disabled_metrics(disabled, famili for name in enabled.columns if not any( name.startswith( - f"Correlation_{(family if family == 'RWC' else family.title())}_" + f"Correlation_{(family if family == "RWC" else family.title())}_" ) for family in families ) diff --git a/tests/unit/test_colocalization_preparation_readiness.py b/tests/unit/test_colocalization_preparation_readiness.py index 762f6b4a32..0299706bfd 100644 --- a/tests/unit/test_colocalization_preparation_readiness.py +++ b/tests/unit/test_colocalization_preparation_readiness.py @@ -79,16 +79,10 @@ def reject_late_work(*args, **kwargs): """) environment = os.environ.copy() environment.update( - OPENHCS_CPU_ONLY="true", - NUMBA_CACHE_DIR=str(tmp_path / "kernels"), + OPENHCS_CPU_ONLY="true", NUMBA_CACHE_DIR=str(tmp_path / "kernels"), ) result = subprocess.run( - (sys.executable, "-c", script), - cwd=Path(__file__).parents[2], - env=environment, - check=False, - capture_output=True, - text=True, - timeout=90, + (sys.executable, "-c", script), cwd=Path(__file__).parents[2], + env=environment, check=False, capture_output=True, text=True, timeout=90, ) assert result.returncode == 0, result.stdout + result.stderr diff --git a/tests/unit/test_compilation_paths.py b/tests/unit/test_compilation_paths.py index 522ed0f381..67b0949dcf 100644 --- a/tests/unit/test_compilation_paths.py +++ b/tests/unit/test_compilation_paths.py @@ -235,9 +235,7 @@ def test_callable_compilation_resolves_authored_and_default_declared_paths() -> captured = CallableContract.from_callable(_path_callable).with_prepared_signature() restored = pickle.loads(pickle.dumps(captured)) - assert ( - restored.metadata.canonical_signature == captured.metadata.canonical_signature - ) + assert restored.metadata.canonical_signature == captured.metadata.canonical_signature assert restored.declared_path_parameters == captured.declared_path_parameters assert restored.resolve_declared_paths({}, resolver) == {"template_path": expected} @@ -246,28 +244,19 @@ def test_path_declarations_follow_live_or_captured_signature_epoch() -> None: def operation(image, *, path=None): return image - operation.__signature__ = inspect.Signature( - ( - inspect.Parameter("image", inspect.Parameter.POSITIONAL_OR_KEYWORD), - inspect.Parameter( - "path", - inspect.Parameter.KEYWORD_ONLY, - annotation=PlateInputFile, - default=None, - ), - ) - ) + operation.__signature__ = inspect.Signature(( + inspect.Parameter("image", inspect.Parameter.POSITIONAL_OR_KEYWORD), + inspect.Parameter("path", inspect.Parameter.KEYWORD_ONLY, + annotation=PlateInputFile, default=None), + )) live = CallableContract.from_callable(operation) captured = live.with_prepared_signature() layout = captured.declared_path_parameters - operation.__signature__ = operation.__signature__.replace( - parameters=( - inspect.Parameter("image", inspect.Parameter.POSITIONAL_OR_KEYWORD), - inspect.Parameter( - "path", inspect.Parameter.KEYWORD_ONLY, annotation=str, default=None - ), - ) - ) + operation.__signature__ = operation.__signature__.replace(parameters=( + inspect.Parameter("image", inspect.Parameter.POSITIONAL_OR_KEYWORD), + inspect.Parameter("path", inspect.Parameter.KEYWORD_ONLY, + annotation=str, default=None), + )) assert captured.declared_path_parameters is layout assert isinstance(layout["path"], PlateInputFileDeclaration) assert not live.declared_path_parameters diff --git a/tests/unit/test_compilation_session.py b/tests/unit/test_compilation_session.py index be21c1f5fa..4af3639b8a 100644 --- a/tests/unit/test_compilation_session.py +++ b/tests/unit/test_compilation_session.py @@ -126,8 +126,7 @@ def full_plate_keys(component, *, resolved_config): assert plans[0].zarr_config is None if backend is MaterializationBackend.ZARR: assert plans[1].zarr_config == { - "all_wells": ("A01", "A02"), - "needs_initialization": True, + "all_wells": ("A01", "A02"), "needs_initialization": True } assert len(requests) == 1 and requests[0][1] is config else: @@ -313,8 +312,7 @@ def _compile_source_plans_for_contract( ) compiled = ( replace(execution_bindings, metadata_fields=fields) - if execution_bindings.bindings - else execution_bindings + if execution_bindings.bindings else execution_bindings ) return execution_bindings, ( compiled, @@ -359,14 +357,12 @@ def test_compilation_session_shares_resolved_pipeline_and_owns_axis_plans(monkey orchestrator=_orchestrator(), global_config=GlobalPipelineConfig(), pipeline=ResolvedPipelineDefinition( - steps=( - _resolved_step( - step, - source_bindings=StepSourceBindingsConfig( - enabled=True, bindings=(NamedSourceBinding(alias="DNA"),) - ), + steps=(_resolved_step( + step, + source_bindings=StepSourceBindingsConfig( + enabled=True, bindings=(NamedSourceBinding(alias="DNA"),) ), - ), + ),), step_scope_ids={0: step_state.scope_id}, step_provenance={0: {}}, ), @@ -417,9 +413,9 @@ def infer_fields(cls, records): source_workspace_projection=VirtualWorkspaceSourceProjection( source_refs_by_virtual_path={}, source_metadata_by_path={ - "image": DurableSourceMetadata.from_mapping( - {DECLARED_SOURCE_METADATA_FIELD: {"Dose": value}} - ), + "image": DurableSourceMetadata.from_mapping({ + DECLARED_SOURCE_METADATA_FIELD: {"Dose": value} + }), }, ), ) @@ -430,9 +426,7 @@ def infer_fields(cls, records): assert len(calls) == 2 -def test_resolved_declarations_reuse_contracts_but_keep_axis_and_author_epochs( - monkeypatch, -): +def test_resolved_declarations_reuse_contracts_but_keep_axis_and_author_epochs(monkeypatch): from openhcs.core.invocation_artifacts import InvocationContractPlan from openhcs.core.pipeline.artifact_planning import extract_artifact_declarations @@ -502,7 +496,9 @@ def test_compiler_keeps_variable_components_as_stack_source(): orchestrator=_orchestrator(), global_config=GlobalPipelineConfig(), pipeline=ResolvedPipelineDefinition( - steps=(_resolved_step(step, variable_components=(Microscopy.Channel,)),), + steps=( + _resolved_step(step, variable_components=(Microscopy.Channel,)), + ), step_scope_ids={0: "plate::functionstep_0"}, step_provenance={0: {}}, ), @@ -692,9 +688,7 @@ def test_path_planner_activates_declared_source_binding_for_pipeline_start(): assert source_binding_plan.bindings == (binding,) -def test_plate_export_contract_construction_projects_inputs_to_runtime_batch( - monkeypatch, -): +def test_plate_export_contract_construction_projects_inputs_to_runtime_batch(monkeypatch): from openhcs.interop.cellprofiler.compile_time_contracts import ( CellProfilerInvocationContractProviderFactory, ) @@ -735,9 +729,7 @@ def counted_provider(cls, pipeline): classmethod(counted_provider), ) final_pipeline = replace(session.pipeline) - assert ( - calls == [] - ) # Intermediate saved/path-resolved owners do not build providers. + assert calls == [] # Intermediate saved/path-resolved owners do not build providers. session.pipeline = final_pipeline other_context = _context() other_context.axis_id = "B01" diff --git a/tests/unit/test_compiled_execution.py b/tests/unit/test_compiled_execution.py index 4e863650bb..ed2eda228a 100644 --- a/tests/unit/test_compiled_execution.py +++ b/tests/unit/test_compiled_execution.py @@ -248,9 +248,7 @@ def execute_axis(_pipeline, context, _lane, **_kwargs): return ExecutionResult.success(context.axis_id) monkeypatch.setattr(worker_execution, "_execute_single_axis_static", execute_axis) - monkeypatch.setattr( - progress_types, "_progress_queue", SimpleNamespace(put=lambda _event: None) - ) + monkeypatch.setattr(progress_types, "_progress_queue", SimpleNamespace(put=lambda _event: None)) results = worker_execution.execute_worker_lane( pipeline_definition=[object()], @@ -374,9 +372,7 @@ def execute_current_run( "_execute_single_axis_static", execute_current_run, ) - monkeypatch.setattr( - progress_types, "_progress_queue", SimpleNamespace(put=lambda _event: None) - ) + monkeypatch.setattr(progress_types, "_progress_queue", SimpleNamespace(put=lambda _event: None)) context = ProcessingContext( axis_id="A01", step_plans={ @@ -437,9 +433,7 @@ def fail_reset() -> None: ("image",), memory_type="numpy", stack=pixels ) del pixels - monkeypatch.setattr( - progress_types, "_progress_queue", SimpleNamespace(put=lambda _event: None) - ) + monkeypatch.setattr(progress_types, "_progress_queue", SimpleNamespace(put=lambda _event: None)) monkeypatch.setattr( worker_execution, "_execute_single_axis_static", @@ -495,9 +489,7 @@ def execute_axis(_pipeline, context, _lane, **_kwargs): return ExecutionResult.success(context.axis_id) monkeypatch.setattr(worker_execution, "_execute_single_axis_static", execute_axis) - monkeypatch.setattr( - progress_types, "_progress_queue", SimpleNamespace(put=lambda _event: None) - ) + monkeypatch.setattr(progress_types, "_progress_queue", SimpleNamespace(put=lambda _event: None)) results = worker_execution.execute_worker_lane( pipeline_definition=[object()], lane_axis_contexts=[ @@ -548,9 +540,7 @@ def execute_axis(_pipeline, context, _lane, **_kwargs): return ExecutionResult.success(context.axis_id) monkeypatch.setattr(worker_execution, "_execute_single_axis_static", execute_axis) - monkeypatch.setattr( - progress_types, "_progress_queue", SimpleNamespace(put=lambda _event: None) - ) + monkeypatch.setattr(progress_types, "_progress_queue", SimpleNamespace(put=lambda _event: None)) results = worker_execution.execute_worker_lane( pipeline_definition=[object()], lane_axis_contexts=[ diff --git a/tests/unit/test_completed_output_publication_lifecycle.py b/tests/unit/test_completed_output_publication_lifecycle.py index d32331698a..dd9746b8d8 100644 --- a/tests/unit/test_completed_output_publication_lifecycle.py +++ b/tests/unit/test_completed_output_publication_lifecycle.py @@ -1,5 +1,4 @@ """Completed persisted facts survive cleanup, failure and deferred publication.""" - import openhcs.core.progress.types as progress_types from types import SimpleNamespace from pathlib import Path @@ -28,64 +27,39 @@ ) from openhcs.core.steps.abstract import StepExecutionObservation from openhcs.core.steps.function_outputs import PrimaryImageMetadataTarget -from openhcs.core.virtual_workspace_metadata import ( - VirtualWorkspaceSourceProjectionEntries, -) +from openhcs.core.virtual_workspace_metadata import VirtualWorkspaceSourceProjectionEntries from openhcs.core.dataset_sources.source_schema import SourceSchemaFilenameParser from openhcs.domains.microscopy.axes import Microscopy def _facts(plate_root: Path) -> StepExecutionObservation: target = PrimaryImageMetadataTarget( - output_dir=plate_root / "images", - backend="disk", - dataset_root=str(plate_root), - sub_dir="images", - results_dir=None, + output_dir=plate_root / "images", backend="disk", dataset_root=str(plate_root), + sub_dir="images", results_dir=None, ) projection = SourcePlaneProjection( - address=OpenHCSPlaneAddress( - ( - (Microscopy.Well, "A01"), - (Microscopy.Site, 1), - (Microscopy.Channel, 0), - (Microscopy.ZIndex, 1), - (Microscopy.Timepoint, 1), - ) - ), - ref=SourcePixelRef("disk", "images/saved.tif"), - source_alias="saved", + address=OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, 1), (Microscopy.Channel, 0), (Microscopy.ZIndex, 1), (Microscopy.Timepoint, 1))), + ref=SourcePixelRef("disk", "images/saved.tif"), source_alias="saved", image_metadata=ImagePayloadMetadata(source_dtype="uint16"), ) return StepExecutionObservation( - MappingProxyType({}), - source_projection_entries_by_target=MappingProxyType( - { - target: VirtualWorkspaceSourceProjectionEntries.from_projection_paths( - ((projection, "images/saved.tif"),) - ) - } - ), + MappingProxyType({}), source_projection_entries_by_target=MappingProxyType({ + target: VirtualWorkspaceSourceProjectionEntries.from_projection_paths( + ((projection, "images/saved.tif"),) + ) + }), ) def _context(plate_root: Path, count: int) -> ProcessingContext: context = ProcessingContext( - axis_id="A01", - filemanager=SimpleNamespace(exists=lambda *_args: False), - step_plans={ - index: SimpleNamespace( - compiled_function_pattern=None, - execution_scope=FunctionStepExecutionScope.AXIS, - step_name=str(index), - visualize=False, - streaming_configs={}, - artifact_outputs={}, - future_artifact_inputs=None, - requires_main_flow_checkpoint=lambda _plans: False, - ) - for index in range(count) - }, + axis_id="A01", filemanager=SimpleNamespace(exists=lambda *_args: False), + step_plans={index: SimpleNamespace( + compiled_function_pattern=None, execution_scope=FunctionStepExecutionScope.AXIS, + step_name=str(index), visualize=False, streaming_configs={}, + artifact_outputs={}, future_artifact_inputs=None, + requires_main_flow_checkpoint=lambda _plans: False, + ) for index in range(count)}, ) context.dataset_root = plate_root context.microscope_handler = SimpleNamespace( @@ -102,21 +76,15 @@ def _context(plate_root: Path, count: int) -> ProcessingContext: def _lane() -> WorkerLaneExecutionContext: return WorkerLaneExecutionContext( - execution_id="output-facts", - subject_id="plate", - worker_slot="worker_0", + execution_id="output-facts", subject_id="plate", worker_slot="worker_0", debug_execution_policy=NoOpDebugExecutionPolicy(), worker_assignments={"worker_0": ["A01"]}, ) @pytest.mark.parametrize("cancel", [False, True]) -def test_completed_projection_facts_survive_error_or_cancel_and_omit( - tmp_path, monkeypatch, cancel -): - monkeypatch.setattr( - progress_types, "_progress_queue", SimpleNamespace(put=lambda _event: None) - ) +def test_completed_projection_facts_survive_error_or_cancel_and_omit(tmp_path, monkeypatch, cancel): + monkeypatch.setattr(progress_types, "_progress_queue", SimpleNamespace(put=lambda _event: None)) facts = _facts(tmp_path) signal = ExecutionCancellationSignal() pixel_refs = [] @@ -126,9 +94,7 @@ def process(self, context, index): pixels = np.ones((8, 8), dtype=np.float32) pixel_refs.append(weakref.ref(pixels)) context.runtime_image_stack_cache.store( - ("temporary-image",), - memory_type="numpy", - stack=pixels, + ("temporary-image",), memory_type="numpy", stack=pixels, ) if cancel: signal.request() @@ -140,12 +106,8 @@ def process(self, context, index): context = _context(tmp_path, 2) result = worker_execution._execute_axis_with_sequential_combinations( - [Save(), Fail()], - [("A01", context)], - _lane(), - RuntimeObservationMode.OMIT, - cancellation=signal, - release_axis_resources=False, + [Save(), Fail()], [("A01", context)], _lane(), RuntimeObservationMode.OMIT, + cancellation=signal, release_axis_resources=False, ) assert result.is_cancelled() if cancel else result.is_error() assert not context.runtime_value_store.observed_values @@ -153,23 +115,12 @@ def process(self, context, index): assert len(result.runtime_observation.contexts) == 1 observation = result.runtime_observation.contexts[0] assert not observation.records - assert ( - observation.outputs.source_projection_entries_by_target - == facts.source_projection_entries_by_target - ) + assert observation.outputs.source_projection_entries_by_target == facts.source_projection_entries_by_target restored = pickle.loads(pickle.dumps(result)) - assert ( - restored.runtime_observation.contexts[ - 0 - ].outputs.source_projection_entries_by_target - == facts.source_projection_entries_by_target - ) + assert restored.runtime_observation.contexts[0].outputs.source_projection_entries_by_target == facts.source_projection_entries_by_target parent = ProcessingContext(axis_id="A01") restored.runtime_observation.merge_into({"A01": parent}) - assert ( - parent.completed_step_outputs.source_projection_entries_by_target - == facts.source_projection_entries_by_target - ) + assert parent.completed_step_outputs.source_projection_entries_by_target == facts.source_projection_entries_by_target def test_same_plate_reader_uses_completed_facts_before_durable_publication(tmp_path): @@ -179,17 +130,12 @@ def test_same_plate_reader_uses_completed_facts_before_durable_publication(tmp_p assert authority.projection_if_available() is None context.record_completed_step_outputs(_facts(tmp_path)) projection = authority.projection_or_empty() - lookup = VirtualWorkspacePathLookup.from_paths( - "images/saved.tif", str(tmp_path / "images/saved.tif") - ) - assert ( - projection.source_projection_for(lookup).image_metadata.source_dtype == "uint16" - ) + lookup = VirtualWorkspacePathLookup.from_paths("images/saved.tif", str(tmp_path / "images/saved.tif")) + assert projection.source_projection_for(lookup).image_metadata.source_dtype == "uint16" assert projection.source_metadata_for(lookup)["source_alias"] == "saved" # A direct durable reader retains its published-boundary contract. direct = WorkspaceSourceProjections.from_dataset_metadata( - dataset_root=tmp_path, - metadata_handler=context.microscope_handler.metadata_handler, + dataset_root=tmp_path, metadata_handler=context.microscope_handler.metadata_handler, filemanager=context.filemanager, ) assert direct.projection_if_available() is None @@ -209,46 +155,22 @@ def test_runtime_overlay_excludes_another_output_plate_and_other_context(tmp_pat assert not authority.is_bound_to_context(other) -def test_runtime_axis_query_is_explicit_and_full_plate_outputs_remain_available( - tmp_path, -): +def test_runtime_axis_query_is_explicit_and_full_plate_outputs_remain_available(tmp_path): context = _context(tmp_path, 1) first = _facts(tmp_path) target, entries = next(iter(first.source_projection_entries_by_target.items())) projection, _ = entries.projection_paths[0] - second = replace( - projection, - address=OpenHCSPlaneAddress( - ( - (Microscopy.Well, "A02"), - (Microscopy.Site, 1), - (Microscopy.Channel, 0), - (Microscopy.ZIndex, 1), - (Microscopy.Timepoint, 1), - ) - ), - ) - outputs = replace( - first, - source_projection_entries_by_target=MappingProxyType( - { - target: VirtualWorkspaceSourceProjectionEntries.from_projection_paths( - ( - (projection, "images/A01.tif"), - (second, "images/A02.tif"), - ) - ), - } - ), - ) + second = replace(projection, address=OpenHCSPlaneAddress(((Microscopy.Well, "A02"), (Microscopy.Site, 1), (Microscopy.Channel, 0), (Microscopy.ZIndex, 1), (Microscopy.Timepoint, 1)))) + outputs = replace(first, source_projection_entries_by_target=MappingProxyType({ + target: VirtualWorkspaceSourceProjectionEntries.from_projection_paths(( + (projection, "images/A01.tif"), (second, "images/A02.tif"), + )), + })) context.record_completed_step_outputs(outputs) authority = context.runtime_source_workspace_projections full = authority.projection_or_empty() axis = authority.projection_or_empty(axis_id="A01") - assert set(full.pipeline_start_files()) == { - str(tmp_path / "images/A01.tif"), - str(tmp_path / "images/A02.tif"), - } + assert set(full.pipeline_start_files()) == {str(tmp_path / "images/A01.tif"), str(tmp_path / "images/A02.tif")} assert axis.pipeline_start_files() == (str(tmp_path / "images/A01.tif"),) # Another context shares declaration admission, never the completed facts. other = _context(tmp_path, 1) diff --git a/tests/unit/test_custom_function_lifecycle.py b/tests/unit/test_custom_function_lifecycle.py index 2317153959..1a07d75e86 100644 --- a/tests/unit/test_custom_function_lifecycle.py +++ b/tests/unit/test_custom_function_lifecycle.py @@ -30,7 +30,7 @@ def _source(name: str, expression: str = "image") -> str: def _plate_source(name: str) -> str: - return f"""from openhcs.core.artifacts import ArtifactSpec, SpecialArtifactType + return f'''from openhcs.core.artifacts import ArtifactSpec, SpecialArtifactType from openhcs.core.callable_contract import FunctionStepExecutionScope from openhcs.core.pipeline.function_contracts import ( artifact_outputs, execution_scope, runtime_bound_parameters, @@ -42,32 +42,22 @@ def _plate_source(name: str) -> str: @artifact_outputs(ArtifactSpec.output("EngineeringBundle", SpecialArtifactType)) def {name}(*, artifact_batch: RuntimeArtifactBatch): return {{"engineering.txt": b"independent ABI fixture"}} -""" +''' @pytest.mark.parametrize("persist", (True, False)) def test_custom_plate_uses_native_abi_projection_and_source_lifecycle( - isolated_custom_runtime, - persist, + isolated_custom_runtime, persist, ) -> None: - from openhcs.core.callable_contract import ( - CallableContract, - FunctionStepExecutionScope, - ) - from openhcs.core.pipeline.funcstep_contract_validator import ( - FuncStepContractValidator, - ) - from openhcs.processing.custom_functions.runtime_registry import ( - CustomFunctionMetadata, - ) + from openhcs.core.callable_contract import CallableContract, FunctionStepExecutionScope + from openhcs.core.pipeline.funcstep_contract_validator import FuncStepContractValidator + from openhcs.processing.custom_functions.runtime_registry import CustomFunctionMetadata from openhcs.processing.func_registry import get_function source = _plate_source("engineering_plate_probe") manager = CustomFunctionManager() [function] = manager.register_from_code(source, persist=persist) - metadata = CustomFunctionRuntimeRegistry.metadata_by_name()[ - "engineering_plate_probe" - ] + metadata = CustomFunctionRuntimeRegistry.metadata_by_name()["engineering_plate_probe"] assert isinstance(metadata, CustomFunctionMetadata) assert get_function(metadata.composite_key) is function contract = CallableContract.from_callable(function) @@ -75,9 +65,7 @@ def test_custom_plate_uses_native_abi_projection_and_source_lifecycle( assert not contract.declared_memory_types assert contract.processing_contract is None assert metadata.tags == ["openhcs", "custom"] - FuncStepContractValidator.validate_dataset_callable_contracts( - (contract,), "engineering" - ) + FuncStepContractValidator.validate_dataset_callable_contracts((contract,), "engineering") reference = FunctionReferenceTransportAuthority.function_reference(function) assert reference.resolve() is function @@ -91,9 +79,7 @@ def test_custom_plate_uses_native_abi_projection_and_source_lifecycle( assert manager.load_all_custom_functions() == 1 reloaded = get_function(metadata.composite_key) assert CallableContract.from_callable(reloaded).processing_contract is None - manager.update_custom_function( - "engineering_plate_probe", source.replace("fixture", "replacement") - ) + manager.update_custom_function("engineering_plate_probe", source.replace("fixture", "replacement")) with pytest.raises(RuntimeError, match="changed"): reference.resolve() else: @@ -109,8 +95,7 @@ def test_custom_plate_uses_native_abi_projection_and_source_lifecycle( ), ) def test_custom_plate_rejects_invalid_original_batch_abi_without_publication( - isolated_custom_runtime, - replacement, + isolated_custom_runtime, replacement, ) -> None: source = _plate_source("engineering_invalid_plate_probe") source = source.replace("*, artifact_batch: RuntimeArtifactBatch", replacement) @@ -125,8 +110,7 @@ def test_custom_plate_does_not_admit_axis_memory_contract( isolated_custom_runtime, ) -> None: source = _plate_source("engineering_mixed_scope_probe").replace( - "@execution_scope", - "@numpy\n@execution_scope", + "@execution_scope", "@numpy\n@execution_scope", ) with pytest.raises(ValidationError, match="cannot declare axis-local"): CustomFunctionManager().register_from_code(source) @@ -417,9 +401,7 @@ def pause_reconstruction(self, declared, contract): def isolated_custom_runtime(monkeypatch, tmp_path): storage_dir = tmp_path / "custom_functions" storage_dir.mkdir() - monkeypatch.setattr( - manager_module, "get_data_file_path", lambda _name, *, create=True: storage_dir - ) + monkeypatch.setattr(manager_module, "get_data_file_path", lambda _name, *, create=True: storage_dir) monkeypatch.setattr(CustomFunctionRuntimeRegistry, "_declarations_by_name", {}) monkeypatch.setattr(CustomFunctionRuntimeRegistry, "_published_exports", {}) monkeypatch.setattr(CustomFunctionRuntimeRegistry, "_preparation_outcomes", {}) @@ -431,14 +413,11 @@ def isolated_custom_runtime(monkeypatch, tmp_path): @pytest.mark.parametrize("persist", (False, True)) def test_registration_observation_uses_current_owners_without_loading( - isolated_custom_runtime, - monkeypatch, - persist, + isolated_custom_runtime, monkeypatch, persist, ): from dataclasses import replace from openhcs.agent.dto.functions import ( - CustomFunctionRegistrationHandle, - CustomFunctionRegistrationRequest, + CustomFunctionRegistrationHandle, CustomFunctionRegistrationRequest, CustomFunctionRegistrationObservationOutcome, ) from openhcs.agent.path_policy import AgentPathPolicy @@ -449,60 +428,29 @@ def test_registration_observation_uses_current_owners_without_loading( policy = AgentPathPolicy.with_roots(readable_roots=(root,), writable_roots=(root,)) manager = CustomFunctionManager(create_storage=False) request = CustomFunctionRegistrationRequest.from_fields( - source_code=_source("observation_probe"), - function_name="observation_probe", - persist=persist, - storage_dir=str(root), - port=22319, - ) - handle = CustomFunctionRegistrationHandle.from_request( - replace(request, server_identity=ProcessIdentity.current()) + source_code=_source("observation_probe"), function_name="observation_probe", + persist=persist, storage_dir=str(root), port=22319, ) + handle = CustomFunctionRegistrationHandle.from_request(replace(request, server_identity=ProcessIdentity.current())) service = FunctionCatalogService(path_policy=policy) before = service.observe_custom_function_registration(handle) assert before.outcome is CustomFunctionRegistrationObservationOutcome.NOT_OBSERVED - [function] = manager.register_from_code( - request.source_code, persist=persist, clear_caches=False, emit_signal=False - ) - monkeypatch.setattr( - CustomFunctionManager, - "_prepare_source", - lambda *_: pytest.fail("Read-only observation cannot evaluate source"), - ) - monkeypatch.setattr( - CustomFunctionManager, - "load_custom_function", - lambda *_a, **_k: pytest.fail("Read-only observation cannot lazy load"), - ) + [function] = manager.register_from_code(request.source_code, persist=persist, clear_caches=False, emit_signal=False) + monkeypatch.setattr(CustomFunctionManager, "_prepare_source", lambda *_: pytest.fail("Read-only observation cannot evaluate source")) + monkeypatch.setattr(CustomFunctionManager, "load_custom_function", lambda *_a, **_k: pytest.fail("Read-only observation cannot lazy load")) observed = service.observe_custom_function_registration(handle) assert observed.outcome is CustomFunctionRegistrationObservationOutcome.REGISTERED assert observed.published_sources == (handle.require_named_source(),) - assert ( - CustomFunctionRuntimeRegistry.metadata_by_name()["observation_probe"].func - is function - ) - assert observed.persisted_source == ( - handle.require_named_source() if persist else None - ) - changed = service.observe_custom_function_registration( - replace(handle, content_sha256="0" * 64) - ) + assert CustomFunctionRuntimeRegistry.metadata_by_name()["observation_probe"].func is function + assert observed.persisted_source == (handle.require_named_source() if persist else None) + changed = service.observe_custom_function_registration(replace(handle, content_sha256="0" * 64)) assert changed.outcome is CustomFunctionRegistrationObservationOutcome.NOT_OBSERVED assert not changed.published_sources with pytest.raises(RuntimeError, match="stale"): - service.observe_custom_function_registration( - replace( - handle, - server_identity=replace(ProcessIdentity.current(), create_time=0), - ) - ) + service.observe_custom_function_registration(replace(handle, server_identity=replace(ProcessIdentity.current(), create_time=0))) CustomFunctionRuntimeRegistry.remove("observation_probe") after = service.observe_custom_function_registration(handle) - assert after.outcome is ( - CustomFunctionRegistrationObservationOutcome.PERSISTED_ONLY - if persist - else CustomFunctionRegistrationObservationOutcome.NOT_OBSERVED - ) + assert after.outcome is (CustomFunctionRegistrationObservationOutcome.PERSISTED_ONLY if persist else CustomFunctionRegistrationObservationOutcome.NOT_OBSERVED) def test_register_rejects_multi_declaration_source_without_partial_publication( @@ -719,9 +667,7 @@ def load_one(): def test_failed_preparation_keeps_evidence_not_payload_or_reader_frames( isolated_custom_runtime, ) -> None: - from openhcs.processing.custom_functions.source_namespace import ( - CustomFunctionSource, - ) + from openhcs.processing.custom_functions.source_namespace import CustomFunctionSource source = CustomFunctionSource("failed_payload_probe", "failure-revision") references = [] @@ -745,9 +691,7 @@ def prepare(): errors.append(error) assert str(error) == "source failed | Line 7 | Code: raise cause" assert (error.message, error.line_number, error.code_snippet) == ( - "source failed", - 7, - "raise cause", + "source failed", 7, "raise cause", ) assert "ValueError: original source cause" in error.__notes__[0] assert "raise ValidationError" in error.__notes__[0] @@ -1142,17 +1086,11 @@ def nested_helper_transport_probe(image): pickle.dumps(helper) -@pytest.mark.parametrize( - "operation", ["replace", "remove", "reconcile", "clear", "stale_preparation"] -) +@pytest.mark.parametrize("operation", ["replace", "remove", "reconcile", "clear", "stale_preparation"]) def test_source_retirement_thaws_before_dropping_captured_owners( - isolated_custom_runtime, - monkeypatch, - operation, + isolated_custom_runtime, monkeypatch, operation, ): - from openhcs.processing.custom_functions.source_namespace import ( - CustomFunctionSource, - ) + from openhcs.processing.custom_functions.source_namespace import CustomFunctionSource manager = CustomFunctionManager() name = "startup_gc_retirement_probe" @@ -1166,10 +1104,7 @@ def unfreeze(): assert vars(custom_functions)[name] is previous.func calls.append("thaw") - monkeypatch.setattr( - "openhcs.processing.backends.lib_registry.registry_service.gc.unfreeze", - unfreeze, - ) + monkeypatch.setattr("openhcs.processing.backends.lib_registry.registry_service.gc.unfreeze", unfreeze) if operation == "replace": manager.update_custom_function(name, _source(name, "image + 1")) elif operation == "remove": diff --git a/tests/unit/test_czi_brain_axon_cellbody_demo.py b/tests/unit/test_czi_brain_axon_cellbody_demo.py index 8582ce7620..53f02de4c0 100644 --- a/tests/unit/test_czi_brain_axon_cellbody_demo.py +++ b/tests/unit/test_czi_brain_axon_cellbody_demo.py @@ -75,15 +75,7 @@ def _candidate( Microscopy.Site.name: site_label, Microscopy.Channel.name: channel_label, }, - declared_address=OpenHCSPlaneAddress( - ( - (Microscopy.Well, "."), - (Microscopy.Site, site), - (Microscopy.Channel, channel), - (Microscopy.ZIndex, "1"), - (Microscopy.Timepoint, "1"), - ) - ), + declared_address=OpenHCSPlaneAddress(((Microscopy.Well, "."), (Microscopy.Site, site), (Microscopy.Channel, channel), (Microscopy.ZIndex, "1"), (Microscopy.Timepoint, "1"))), dataset_identity=identity, store_identity=SourcePlaneStoreIdentity( container_paths=(source_path,), @@ -173,17 +165,25 @@ def test_czi_demo_declares_bounded_three_channel_analysis_and_artifacts( assert get_core_callable(steps[2].func) is tophat assert get_core_callable(steps[3].func) is skan_axon_skeletonize_and_analyze - assert steps[0].processing_config.variable_components == [Microscopy.Channel] + assert steps[0].processing_config.variable_components == [ + Microscopy.Channel + ] assert steps[0].processing_config.input_source is InputSource.PIPELINE_START - assert steps[1].processing_config.variable_components == [Microscopy.Channel] + assert steps[1].processing_config.variable_components == [ + Microscopy.Channel + ] assert steps[1].processing_config.input_source is InputSource.PREVIOUS_STEP assert steps[2].processing_config.group_by is Ungrouped - assert steps[2].processing_config.variable_components == [Microscopy.Channel] + assert steps[2].processing_config.variable_components == [ + Microscopy.Channel + ] assert steps[2].processing_config.input_source is InputSource.PIPELINE_START assert steps[2].source_bindings.enabled is True assert [binding.alias for binding in steps[2].source_bindings.bindings] == ["Axon"] assert steps[3].processing_config.group_by is Ungrouped - assert steps[3].processing_config.variable_components == [Microscopy.Channel] + assert steps[3].processing_config.variable_components == [ + Microscopy.Channel + ] assert steps[3].processing_config.input_source is InputSource.PREVIOUS_STEP count_kwargs = steps[1].func[1] diff --git a/tests/unit/test_dataset_source_family.py b/tests/unit/test_dataset_source_family.py index 16c18aa2c0..145735000b 100644 --- a/tests/unit/test_dataset_source_family.py +++ b/tests/unit/test_dataset_source_family.py @@ -11,9 +11,7 @@ from openhcs.agent.dto.plate import PlatePathInspectionRequest from openhcs.agent.path_policy import AgentPathPolicy from openhcs.agent.services.config_service import ConfigService -from openhcs.agent.services.artifact_plan_inspection_service import ( - ArtifactPlanInspectionService, -) +from openhcs.agent.services.artifact_plan_inspection_service import ArtifactPlanInspectionService from openhcs.agent.services.plate_inspection_service import PlateInspectionService from openhcs.core.config import LazyPartitionFilterConfig, PipelineConfig from openhcs.core.config_document import ConfigDocumentAuthority @@ -81,10 +79,7 @@ def test_every_registered_source_declares_its_identity_role_and_backends() -> No ) @pytest.mark.parametrize("missing", (False, True)) def test_metadata_detection_uses_each_declared_owner( - monkeypatch, - tmp_path: Path, - handler_type, - missing: bool, + monkeypatch, tmp_path: Path, handler_type, missing: bool, ) -> None: from polystore.exceptions import MetadataNotFoundError @@ -139,16 +134,12 @@ def test_config_schema_patch_and_source_share_opera_handler_identity() -> None: def test_handler_factory_uses_exact_declared_identity(tmp_path: Path) -> None: - handler = DatasetSourceChoice.named("opera_phenix").open( - tmp_path, filemanager=bioformats_filemanager() - ) + handler = DatasetSourceChoice.named("opera_phenix").open(tmp_path, filemanager=bioformats_filemanager()) assert isinstance(handler, OperaPhenixHandler) assert handler.source_name == "opera_phenix" with pytest.raises(ValueError, match="Unknown dataset source 'OperaPhenix'"): - DatasetSourceChoice.named("OperaPhenix").open( - tmp_path, filemanager=bioformats_filemanager() - ) + DatasetSourceChoice.named("OperaPhenix").open(tmp_path, filemanager=bioformats_filemanager()) def test_source_selection_role_owns_local_availability_contract( @@ -159,9 +150,13 @@ def test_source_selection_role_owns_local_availability_contract( file_path.touch() with pytest.raises(FileNotFoundError, match=str(missing_path)): - FormatSpecificSource.require_available_source(missing_path) + FormatSpecificSource.require_available_source( + missing_path + ) with pytest.raises(NotADirectoryError, match=str(file_path)): - FormatSpecificSource.require_available_source(file_path) + FormatSpecificSource.require_available_source( + file_path + ) RemoteServiceSource.require_available_source(missing_path) diff --git a/tests/unit/test_dataset_source_guards.py b/tests/unit/test_dataset_source_guards.py index 963daa61e5..2693a55228 100644 --- a/tests/unit/test_dataset_source_guards.py +++ b/tests/unit/test_dataset_source_guards.py @@ -98,9 +98,7 @@ def test_kernel_imports_no_domain_package() -> None: def test_deleted_source_names_do_not_exist() -> None: offenders = { - _relative(path): sorted( - DELETED_NAMES & _identifiers(ast.parse(path.read_text())) - ) + _relative(path): sorted(DELETED_NAMES & _identifiers(ast.parse(path.read_text()))) for root in ("openhcs", "tests", "benchmark", "scripts") for path in (REPO_ROOT / root).rglob("*.py") if "benchmark/results" not in path.as_posix() @@ -115,7 +113,9 @@ def test_kernel_spells_no_domain_root_or_filename_token() -> None: for path in _kernel_files() for node in ast.walk(ast.parse(path.read_text())) if isinstance(node, (ast.Constant, ast.JoinedStr)) - and any(token in ast.unparse(node) for token in ('"/omero/', "'/omero/", "_w{")) + and any( + token in ast.unparse(node) for token in ('"/omero/', "'/omero/", "_w{") + ) } assert offenders == set() diff --git a/tests/unit/test_debug_runtime.py b/tests/unit/test_debug_runtime.py index 61910405a6..e12751fe23 100644 --- a/tests/unit/test_debug_runtime.py +++ b/tests/unit/test_debug_runtime.py @@ -177,7 +177,6 @@ def lane(cls) -> WorkerLaneExecutionContext: worker_slot="worker_0", worker_assignments={"worker_0": ("A01",)}, ) - SEGMENT_NAME = "segment" @staticmethod @@ -1002,7 +1001,8 @@ def test_debug_step_executes_one_function_pattern_invocation(): np.testing.assert_array_equal(result, np.array([2, 4, 6])) assert [ - DebugProgressContext.from_context(event.context).event_type for event in emitted + DebugProgressContext.from_context(event.context).event_type + for event in emitted ] == [DebugEventType.BEFORE_INVOCATION, DebugEventType.AFTER_INVOCATION] @@ -1038,9 +1038,7 @@ def test_debug_step_can_advance_past_current_function_pattern_invocation(): np.testing.assert_array_equal(result, np.array([2, 3, 4])) assert len(emitted) == 2 - assert DebugProgressContext.from_context( - emitted[0].context - ).cursor.invocation_key.endswith(":1:plus_one") + assert DebugProgressContext.from_context(emitted[0].context).cursor.invocation_key.endswith(":1:plus_one") def test_execute_chain_emits_debug_exception_event(): @@ -1119,7 +1117,10 @@ def test_projection_exception_contains_complete_invocation_context_without_debug assert "declared_cardinality=2" in message assert "selected_runtime_plane=preserved_stack" in message assert "execution_axis_cardinality=3" in message - assert "image_payload_execution_mode=FullStackExecution" in message + assert ( + "image_payload_execution_mode=FullStackExecution" + in message + ) assert "processing_contract=Pure2DContract" in message assert isinstance(exc.__cause__, RuntimeSliceProjectionDeclarationError) assert str(exc.__cause__) == "projection preflight mismatch" @@ -1305,8 +1306,7 @@ def test_progress_debug_event_sink_uses_injected_progress_emitter(): assert emitted[0].status is ProgressStatus.STARTED assert emitted[0].context is not None assert ( - emitted[0].context[DebugProgressContext.context_key]["debug_session_id"] - == DebugRuntimeFixture.DEBUG_SESSION_ID + emitted[0].context[DebugProgressContext.context_key]["debug_session_id"] == DebugRuntimeFixture.DEBUG_SESSION_ID ) assert ( emitted[0].context[DebugProgressContext.context_key]["snapshot_store_ref"] @@ -1484,9 +1484,9 @@ def test_debug_execution_policy_skips_before_start_step_index(): def test_debug_session_marks_current_cursor_dirty_nominally(): cursor = DebugRuntimeFixture.cursor() - session = DebugSession.create( - subject_id=DebugRuntimeFixture.SUBJECT_ID - ).with_cursor(cursor) + session = DebugSession.create(subject_id=DebugRuntimeFixture.SUBJECT_ID).with_cursor( + cursor + ) dirty_session = session.mark_dirty_from_cursor() diff --git a/tests/unit/test_declared_singleton_composition_437.py b/tests/unit/test_declared_singleton_composition_437.py index 23ce5f9f5e..48725ef09e 100644 --- a/tests/unit/test_declared_singleton_composition_437.py +++ b/tests/unit/test_declared_singleton_composition_437.py @@ -5,10 +5,7 @@ import numpy as np import pytest -from openhcs.core.aligned_image_payload import ( - AlignedImageSliceContext, - AlignedImageStack, -) +from openhcs.core.aligned_image_payload import (AlignedImageSliceContext, AlignedImageStack) from openhcs.core.callable_contract import CallableContract, ImagePayloadConsumption from openhcs.core.component_group_scope import RuntimeExecutionAxisScope from openhcs.core.function_patterns import ( @@ -200,7 +197,9 @@ def test_new_declaration_reuses_composition_and_executor(slice_count, aligned): assert metadata.source_image_names == ("FITC",) assert metadata.source_component_metadata["channel"] == "2" assert metadata.source_image_paths == (f"/synthetic/A01_s{index + 1}_w2.tif",) - np.testing.assert_array_equal(plane.data[0], original_pixels[index]) + np.testing.assert_array_equal( + plane.data[0], original_pixels[index] + ) np.testing.assert_array_equal(plane.mask, original_masks[index]) result = CellProfilerFunctionContractExecutor().execute( contract, @@ -215,13 +214,23 @@ def test_new_declaration_reuses_composition_and_executor(slice_count, aligned): result_plane = RuntimeSliceProjection.value_for_slice( result, projection.selected_plane(index) ) - np.testing.assert_array_equal(result_plane.data, original_pixels[index]) - np.testing.assert_array_equal(result_plane.mask, original_masks[index]) + np.testing.assert_array_equal( + result_plane.data, original_pixels[index] + ) + np.testing.assert_array_equal( + result_plane.mask, original_masks[index] + ) metadata = result_plane.metadata assert metadata.plane_axis is None assert metadata.axis_position(ColourAxis) is None - assert metadata.source_voxel_spacing == source.metadata.source_voxel_spacing - assert metadata.source_spatial_domain == source.metadata.source_spatial_domain + assert ( + metadata.source_voxel_spacing + == source.metadata.source_voxel_spacing + ) + assert ( + metadata.source_spatial_domain + == source.metadata.source_spatial_domain + ) np.testing.assert_array_equal(source.data, original_pixels) np.testing.assert_array_equal(source.mask, original_masks) @@ -243,7 +252,10 @@ def test_declared_composed_scalar_introduces_one_source_axis(): (scalar,), ) assert composition.execution_mode is FullStackExecution - assert composition.payload.metadata.plane_axis is RuntimePlaneAxis.SOURCE_BINDING + assert ( + composition.payload.metadata.plane_axis + is RuntimePlaneAxis.SOURCE_BINDING + ) contract = CallableContract.from_callable(independently_declared_source_echo) result = CellProfilerFunctionContractExecutor().execute( contract, @@ -252,7 +264,9 @@ def test_declared_composed_scalar_introduces_one_source_axis(): {}, execution_mode=composition.execution_mode, ) - np.testing.assert_array_equal(result.data, source.data[0]) + np.testing.assert_array_equal( + result.data, source.data[0] + ) @pytest.mark.parametrize("aligned", (False, True)) @@ -266,7 +280,9 @@ def test_natural_declaration_retains_exact_singleton(aligned): ) assert composition.payload is current assert composition.execution_mode is ( - AlignedStackExecution if aligned else NaturalExecution + AlignedStackExecution + if aligned + else NaturalExecution ) if not aligned: result = CellProfilerFunctionContractExecutor().execute( @@ -281,8 +297,12 @@ def test_natural_declaration_retains_exact_singleton(aligned): result.data, source.data * source.mask, ) - np.testing.assert_array_equal(result.mask, source.mask) - assert result.metadata.plane_axis is RuntimePlaneAxis.RUNTIME_SLICE + np.testing.assert_array_equal( + result.mask, source.mask + ) + assert ( + result.metadata.plane_axis is RuntimePlaneAxis.RUNTIME_SLICE + ) def test_kernel_still_rejects_unprojected_runtime_axis(): diff --git a/tests/unit/test_derived_role_publication_435.py b/tests/unit/test_derived_role_publication_435.py index 7ac6a14ded..4acdffdfef 100644 --- a/tests/unit/test_derived_role_publication_435.py +++ b/tests/unit/test_derived_role_publication_435.py @@ -11,102 +11,57 @@ from polystore.memory import MemoryStorageBackend from tests.unit.test_function_outputs import ( - context_stub, - function_step_plan, - record_output_path, + context_stub, function_step_plan, record_output_path, ) from openhcs.core.dataset_sources.source_schema import SourceSchemaFilenameParser from openhcs.constants.constants import Backend -from openhcs.core.aligned_image_payload import ( - AlignedImageSliceContext, - stack_image_payloads, -) -from openhcs.core.artifacts import ( - ArtifactOutputPlan, - ImageArtifactType, - ObjectLabelsArtifactType, -) -from openhcs.core.component_group_scope import ( - ComponentGroupScope, - RuntimeExecutionAxisScope, -) +from openhcs.core.aligned_image_payload import AlignedImageSliceContext, stack_image_payloads +from openhcs.core.artifacts import ArtifactOutputPlan, ImageArtifactType, ObjectLabelsArtifactType +from openhcs.core.component_group_scope import ComponentGroupScope, RuntimeExecutionAxisScope from openhcs.core.compiled_step_plan import RuntimeArtifactMaterializationPlan from openhcs.core.runtime_artifact_values import RuntimeValue from openhcs.core.runtime_image_values import ( - ImageMetadataPayload, - ImagePayloadMetadata, - ImagePayloadMetadataCompositionMode, + ImageMetadataPayload, ImagePayloadMetadata, ImagePayloadMetadataCompositionMode, ) from openhcs.core.runtime_stores import RuntimeValueStore -from openhcs.core.runtime_object_label_building import ( - SourceImageObjectLabelBuildRequest, -) +from openhcs.core.runtime_object_label_building import SourceImageObjectLabelBuildRequest from openhcs.core.source_metadata import SourceVoxelSpacing from openhcs.core.source_image_provenance import ( - SourceImageProvenance, - SourceImageProvenancePlanes, + SourceImageProvenance, SourceImageProvenancePlanes, ) from openhcs.core.source_projection import SourceProjectionMetadataSerializer from openhcs.core.source_workspace_projection import VirtualWorkspaceSourceProjection -from openhcs.core.steps.function_artifact_materialization import ( - ArtifactMaterializationTargetPlan, -) +from openhcs.core.steps.function_artifact_materialization import ArtifactMaterializationTargetPlan from openhcs.core.steps.function_outputs import ( PrimaryImageMetadataTarget, ) from openhcs.core.steps.function_output_identity import FunctionOutputIdentity -from openhcs.core.runtime_plane_projection import ( - RuntimePlaneAxis, - RuntimePlaneAxisValueProjection, -) +from openhcs.core.runtime_plane_projection import RuntimePlaneAxis, RuntimePlaneAxisValueProjection from python_introspect import to_jsonable from openhcs.processing.materialization import ( - ImageFileOptions, - MaterializationSpec, - MaterializedFilenameIdentity, + ImageFileOptions, MaterializationSpec, MaterializedFilenameIdentity, ) from openhcs.domains.microscopy.axes import Microscopy -@pytest.mark.parametrize( - "scenario,aggregate", - ( - *( - (scenario, False) - for scenario in ( - "same_occurrence", - "different_path", - "stale_scope", - "conflicting_metadata", - "different_kind", - ) - ), - *( - (scenario, True) - for scenario in ( - "same_occurrence", - "different_path", - "stale_scope", - "conflicting_metadata", - "different_kind", - ) - ), - ("conflicting_scope", True), - ("mosaic_context", False), - ), -) -def test_saved_roles_publish_once_per_persisted_occurrence( - tmp_path, scenario, aggregate -): +@pytest.mark.parametrize("scenario,aggregate", ( + *((scenario, False) for scenario in ( + "same_occurrence", "different_path", "stale_scope", "conflicting_metadata", "different_kind", + )), + *((scenario, True) for scenario in ( + "same_occurrence", "different_path", "stale_scope", "conflicting_metadata", "different_kind", + )), + ("conflicting_scope", True), + ("mosaic_context", False), +)) +def test_saved_roles_publish_once_per_persisted_occurrence(tmp_path, scenario, aggregate): mosaic = scenario == "mosaic_context" same_persisted_path = scenario != "different_path" - filemanager = FileManager( - { - Backend.DISK.value: DiskStorageBackend(), - Backend.MEMORY.value: MemoryStorageBackend(), - } - ) + filemanager = FileManager({ + Backend.DISK.value: DiskStorageBackend(), + Backend.MEMORY.value: MemoryStorageBackend(), + }) context = context_stub(filemanager, parser=SourceSchemaFilenameParser()) context.microscope_handler.source_name = "openhcsdata" context.runtime_value_store = RuntimeValueStore() @@ -120,29 +75,24 @@ def test_saved_roles_publish_once_per_persisted_occurrence( plan.sub_dir = "images" plan.analysis_results_dir = str(plan.output_dir) plan.runtime_artifact_materialization = RuntimeArtifactMaterializationPlan( - persistent_enabled=True, - persistent_backend=Backend.DISK.value, + persistent_enabled=True, persistent_backend=Backend.DISK.value, ) context.step_plans = {plan.step_index: plan} filemanager.ensure_directory(plan.output_dir, Backend.MEMORY.value) components = { - "well": "A01", - "site": "1", - "channel": "2", - "z_index": "1", - "timepoint": "1", - "extension": ".tif", + "well": "A01", "site": "1", "channel": "2", + "z_index": "1", "timepoint": "1", "extension": ".tif", } roles = ( - (("MosaicRole", 1.0),) if mosaic else (("RawRole", 1.0), ("CappedRole", 0.5)) + (("MosaicRole", 1.0),) if mosaic + else (("RawRole", 1.0), ("CappedRole", 0.5)) ) if aggregate: components.update(channel="1", z_index="0") plan.variable_components = (Microscopy.ZIndex,) plan.group_by = Microscopy.Channel plan.execution_group_scope = ComponentGroupScope.from_raw( - ("1",), - component=Microscopy.Channel, + ("1",), component=Microscopy.Channel, ) source_planes = [] source_dir = tmp_path / "physical_source" @@ -151,16 +101,13 @@ def test_saved_roles_publish_once_per_persisted_occurrence( for z_index, plane in enumerate(pixels): path = source_dir / f"A01_s001_w1_z{z_index:03d}_t001.tif" filemanager.save(plane, str(path), Backend.DISK.value) - source_planes.append( - ImagePayloadMetadata( - source_path=str(path), - source_component_metadata={**components, "z_index": str(z_index)}, - source_voxel_spacing=SourceVoxelSpacing((2.0, 0.65, 0.65)), - ).payload_with(filemanager.load(str(path), Backend.DISK.value)) - ) + source_planes.append(ImagePayloadMetadata( + source_path=str(path), + source_component_metadata={**components, "z_index": str(z_index)}, + source_voxel_spacing=SourceVoxelSpacing((2.0, 0.65, 0.65)), + ).payload_with(filemanager.load(str(path), Backend.DISK.value))) source_stack = stack_image_payloads( - source_planes, - metadata_mode=ImagePayloadMetadataCompositionMode.STACK, + source_planes, metadata_mode=ImagePayloadMetadataCompositionMode.STACK, ) roles = (("fixture_image_494", 1.0),) for role, scale in roles: @@ -170,41 +117,29 @@ def test_saved_roles_publish_once_per_persisted_occurrence( f"_z{int(components['z_index']):03d}_t001_{role}.tif" ) output_plan = ArtifactOutputPlan( - name=role, - path=f"/memory/{filename.removesuffix('.tif')}.pkl", + name=role, path=f"/memory/{filename.removesuffix('.tif')}.pkl", artifact_type=( ObjectLabelsArtifactType - if scenario == "different_kind" and tested_role - else ImageArtifactType + if scenario == "different_kind" and tested_role else ImageArtifactType ), producer_step_index=plan.step_index, producer_step_scope_id=( "different-step-scope" - if scenario == "stale_scope" and tested_role - else plan.step_scope_id + if scenario == "stale_scope" and tested_role else plan.step_scope_id ), producer_step_name=plan.step_name, - materialization=MaterializationSpec( - ImageFileOptions( - filename_suffix=".tif", - filename_identity=( - MaterializedFilenameIdentity.ARTIFACT_NAME - if aggregate - else MaterializedFilenameIdentity.SOURCE_IDENTITY - ), - relative_path_template=( - filename - if same_persisted_path - else filename.replace(".tif", "_independent.tif") - ), - ) - ), + materialization=MaterializationSpec(ImageFileOptions( + filename_suffix=".tif", + filename_identity=( + MaterializedFilenameIdentity.ARTIFACT_NAME if aggregate + else MaterializedFilenameIdentity.SOURCE_IDENTITY + ), + relative_path_template=(filename if same_persisted_path else filename.replace(".tif", "_independent.tif")), + )), ) if aggregate: output_plan = replace( - output_plan, - group_keys=("1",), - group_component=Microscopy.Channel, + output_plan, group_keys=("1",), group_component=Microscopy.Channel, variable_components=(Microscopy.ZIndex,), ) plan.artifact_outputs[output_plan.ref()] = output_plan @@ -237,205 +172,128 @@ def test_saved_roles_publish_once_per_persisted_occurrence( component_metadata=({"site": "1"}, {"site": "2"}), ), ), - ).payload_with( - np.stack( - [ - filemanager.load(str(path), Backend.DISK.value) - for path in contributor_paths - ] - ) - ) + ).payload_with(np.stack([ + filemanager.load(str(path), Backend.DISK.value) + for path in contributor_paths + ])) if aggregate: payload = ImageArtifactType.contextualize_output( - source_stack, - pixels.copy(), - output_plan, - RuntimePlaneAxisValueProjection( - RuntimePlaneAxis.RUNTIME_SLICE, (), None, 4 - ), + source_stack, pixels.copy(), output_plan, + RuntimePlaneAxisValueProjection(RuntimePlaneAxis.RUNTIME_SLICE, (), None, 4), ) if aggregate: execution_scope = RuntimeExecutionAxisScope.from_raw( - "A01", - component=Microscopy.Channel, - value="1", + "A01", component=Microscopy.Channel, value="1", fixed_component_values=( - (Microscopy.Site, "1"), - (Microscopy.Timepoint, "1"), + (Microscopy.Site, "1"), (Microscopy.Timepoint, "1"), ), ) payload = RuntimeValue.normalize_for_execution_scope( - replace(output_plan, artifact_type=ImageArtifactType), - payload, + replace(output_plan, artifact_type=ImageArtifactType), payload, execution_scope=execution_scope, ).data artifact_payload = payload if tested_role and scenario == "conflicting_metadata": conflicting_fields = ( {"source_voxel_spacing": SourceVoxelSpacing((3.0, 0.65, 0.65))} - if aggregate - else {"source_image_names": ("IndependentRole",)} + if aggregate else {"source_image_names": ("IndependentRole",)} ) artifact_payload = payload.metadata.replace_fields( **conflicting_fields ).payload_with(payload.data) if tested_role and scenario == "different_kind": artifact_payload = SourceImageObjectLabelBuildRequest( - image=payload, - labels=np.ones(payload.data.shape, dtype=np.int32), - declared_object_count=1, - declared_object_ids=(1,), + image=payload, labels=np.ones(payload.data.shape, dtype=np.int32), + declared_object_count=1, declared_object_ids=(1,), ).payload() output_identity = ( FunctionOutputIdentity.from_metadata( - context.microscope_handler.parser, - payload.metadata, + context.microscope_handler.parser, payload.metadata, variable_components=plan.variable_components, - ) - if aggregate - else None + ) if aggregate else None ) value = ( RuntimeValue.normalize_for_execution_scope( - output_plan, - artifact_payload, + output_plan, artifact_payload, execution_scope=( RuntimeExecutionAxisScope.from_raw( - "A01", - component=Microscopy.Channel, - value="1", + "A01", component=Microscopy.Channel, value="1", fixed_component_values=( - (Microscopy.Site, "2"), - (Microscopy.Timepoint, "1"), + (Microscopy.Site, "2"), (Microscopy.Timepoint, "1"), ), - ) - if scenario == "conflicting_scope" - else execution_scope + ) if scenario == "conflicting_scope" else execution_scope ), - ) - if aggregate - else ( + ) if aggregate else ( RuntimeValue.normalize_for_execution_scope( - output_plan, - artifact_payload, + output_plan, artifact_payload, execution_scope=RuntimeExecutionAxisScope.from_raw( - "A01", - component=None, - value=None, - fixed_component_values=( - (Microscopy.Channel, "2"), - (Microscopy.ZIndex, "1"), + "A01", component=None, value=None, fixed_component_values=( + (Microscopy.Channel, "2"), (Microscopy.ZIndex, "1"), (Microscopy.Timepoint, "1"), ), ), - ) - if mosaic - else RuntimeValue.normalize( - output_plan, artifact_payload, axis_id="A01" - ) + ) if mosaic else RuntimeValue.normalize(output_plan, artifact_payload, axis_id="A01") ) ) if mosaic: payload = value.data output_identity = FunctionOutputIdentity( component_values={ - key: value - for key, value in components.items() + key: value for key, value in components.items() if key not in ("site", "extension") }, filename_component_values={ - key: value - for key, value in components.items() - if key != "extension" + key: value for key, value in components.items() if key != "extension" }, - extension=".tif", - source="collapsed Mosaic contributors", + extension=".tif", source="collapsed Mosaic contributors", ) context.runtime_value_store.record( value, - path=output_plan.path, - backend=Backend.MEMORY.value, + path=output_plan.path, backend=Backend.MEMORY.value, ) destination = plan.output_dir / filename filemanager.save(payload, str(destination), Backend.MEMORY.value) filemanager.ensure_directory(plan.output_dir, Backend.DISK.value) filemanager.save(payload.data, str(destination), Backend.DISK.value) record_output_path( - context, - plan, - str(destination), - image_metadata=payload.metadata, + context, plan, str(destination), image_metadata=payload.metadata, output_context=AlignedImageSliceContext.main_flow( - output_key=role, - artifact_kind=ImageArtifactType.value, + output_key=role, artifact_kind=ImageArtifactType.value, ), identity=output_identity, ) saved = ArtifactMaterializationTargetPlan.materialize(context, plan) assert len(saved) == len(roles) - target = replace( - PrimaryImageMetadataTarget.from_plan(plan), artifact_materializations=saved - ) + target = replace(PrimaryImageMetadataTarget.from_plan(plan), artifact_materializations=saved) records = target.produced_records(context, plan) artifacts = target.runtime_artifact_projection_paths(context, plan) receipt = { "source_scope": plan.step_scope_id, - "produced": [ - { - "path": str(record.path_under(plan.output_dir)), - "identity": asdict(record.producer_identity), - "address": record.filename_address.as_component_metadata(), - } - for record in records - ], - "materialized": [ - { - "path": output.path, - "role": item.materialization.output_plan.name, - "semantic_source": ( - None - if item.materialization.source_identity is None - else { - "path": item.materialization.source_identity.path, - "component_metadata": dict( - item.materialization.source_identity.component_metadata - or {} - ), - } - ), - "producer_scope": item.materialization.output_plan.producer_step_scope_id, - "stored_key": repr(item.materialization.record.key), - } - for item in saved - for output in item.outputs_for_backend(Backend.DISK.value) - ], - "artifact_projections": [ - { - "path": path, - "role": projection.projection_role.value, - "address": ( - None - if projection.address is None - else projection.address.as_component_metadata() - ), - "alias": projection.source_alias, - "kind": projection.artifact_kind.value, - } - for projection, path in artifacts - ], + "produced": [{"path": str(record.path_under(plan.output_dir)), + "identity": asdict(record.producer_identity), + "address": record.filename_address.as_component_metadata()} for record in records], + "materialized": [{"path": output.path, + "role": item.materialization.output_plan.name, + "semantic_source": None if item.materialization.source_identity is None else { + "path": item.materialization.source_identity.path, + "component_metadata": dict(item.materialization.source_identity.component_metadata or {}), + }, + "producer_scope": item.materialization.output_plan.producer_step_scope_id, + "stored_key": repr(item.materialization.record.key)} + for item in saved for output in item.outputs_for_backend(Backend.DISK.value)], + "artifact_projections": [{"path": path, + "role": projection.projection_role.value, + "address": None if projection.address is None else projection.address.as_component_metadata(), + "alias": projection.source_alias, + "kind": projection.artifact_kind.value} + for projection, path in artifacts], "pre_join_main_flow": SourceProjectionMetadataSerializer.projection_records( - replace(target, artifact_materializations=()) - .produced_projection_entries(context, plan) - .projection_paths, - ), - "pre_join_artifacts": SourceProjectionMetadataSerializer.projection_records( - artifacts + replace(target, artifact_materializations=()).produced_projection_entries(context, plan).projection_paths, ), + "pre_join_artifacts": SourceProjectionMetadataSerializer.projection_records(artifacts), "produced_records": [to_jsonable(record) for record in records], } - (tmp_path / "persisted-occurrence-witness.json").write_text( - json.dumps(receipt, indent=2) - ) + (tmp_path / "persisted-occurrence-witness.json").write_text(json.dumps(receipt, indent=2)) produced_paths = {item["path"] for item in receipt["produced"]} materialized_paths = {item["path"] for item in receipt["materialized"]} if same_persisted_path: @@ -445,44 +303,29 @@ def test_saved_roles_publish_once_per_persisted_occurrence( if mosaic: expected_sums = {780.0} elif aggregate: - expected_sums = ( - {140.0} if scenario == "different_kind" else {float(pixels.sum())} - ) + expected_sums = {140.0} if scenario == "different_kind" else {float(pixels.sum())} else: expected_sums = {20.0} if scenario == "different_kind" else {20.0, 10.0} - assert { - float(tifffile.imread(item["path"]).sum()) for item in receipt["materialized"] - } == expected_sums - if scenario not in ("same_occurrence", "mosaic_context") and not ( - aggregate and scenario == "different_path" - ): + assert {float(tifffile.imread(item["path"]).sum()) for item in receipt["materialized"]} == expected_sums + if scenario not in ("same_occurrence", "mosaic_context") and not (aggregate and scenario == "different_path"): message = ( "Conflicting metadata for persisted image" - if scenario - in ("conflicting_metadata", "conflicting_scope", "mosaic_context") + if scenario in ("conflicting_metadata", "conflicting_scope", "mosaic_context") else "Duplicate source projection address" ) with pytest.raises(ValueError, match=message) as rejection: target.produced_projection_entries(context, plan) receipt["rejection"] = str(rejection.value) - (tmp_path / "persisted-occurrence-witness.json").write_text( - json.dumps(receipt, indent=2) - ) + (tmp_path / "persisted-occurrence-witness.json").write_text(json.dumps(receipt, indent=2)) return if aggregate: (main_flow,) = receipt["pre_join_main_flow"] (artifact,) = receipt["pre_join_artifacts"] assert main_flow["address"] is artifact["address"] is None - assert ( - main_flow["execution_scope"] - == artifact["execution_scope"] - == { - "axis_id": "A01", - "component": "channel", - "value": "1", - "fixed_component_values": [["site", "1"], ["timepoint", "1"]], - } - ) + assert main_flow["execution_scope"] == artifact["execution_scope"] == { + "axis_id": "A01", "component": "channel", "value": "1", + "fixed_component_values": [["site", "1"], ["timepoint", "1"]], + } assert main_flow["image_metadata"] == artifact["image_metadata"] assert main_flow["source_metadata"] == artifact["source_metadata"] assert receipt["produced"][0]["address"]["z_index"] == "0" @@ -490,20 +333,14 @@ def test_saved_roles_publish_once_per_persisted_occurrence( metadata = main_flow["image_metadata"] assert metadata["source_dtype"] == "uint16" assert metadata["source_voxel_spacing"] == { - "values_zyx": [2.0, 0.65, 0.65], - "unit": "micrometers", + "values_zyx": [2.0, 0.65, 0.65], "unit": "micrometers", } provenance = metadata["source_provenance"] assert "z_index" not in provenance["source_component_metadata"] - assert [ - plane["component_metadata"]["z_index"] - for plane in provenance["source_image_provenance_planes"] - ] == ["0", "1", "2", "3"] - assert [ - plane["path"] for plane in provenance["source_image_provenance_planes"] - ] == [ - str(source_dir / f"A01_s001_w1_z{z_index:03d}_t001.tif") - for z_index in range(4) + assert [plane["component_metadata"]["z_index"] for plane in + provenance["source_image_provenance_planes"]] == ["0", "1", "2", "3"] + assert [plane["path"] for plane in provenance["source_image_provenance_planes"]] == [ + str(source_dir / f"A01_s001_w1_z{z_index:03d}_t001.tif") for z_index in range(4) ] assert metadata["source_spatial_domain"]["spatial_dimensions"] == 2 for path in produced_paths | materialized_paths: @@ -519,32 +356,20 @@ def test_saved_roles_publish_once_per_persisted_occurrence( assert main_flow["image_metadata"] == artifact["image_metadata"] assert main_flow["source_metadata"] == artifact["source_metadata"] assert receipt["produced"][0]["address"]["site"] == "1" - assert ( - "site" - not in receipt["materialized"][0]["semantic_source"]["component_metadata"] - ) + assert "site" not in receipt["materialized"][0]["semantic_source"]["component_metadata"] structured = SourceProjectionMetadataSerializer.projection_fields( structured.projection_paths ) occurrence_count = 2 if aggregate and scenario == "different_path" else 1 assert len(structured["source_projection"]) == len(roles) * occurrence_count - assert {item["source_alias"] for item in structured["source_projection"]} == { - role for role, _ in roles - } + assert {item["source_alias"] for item in structured["source_projection"]} == {role for role, _ in roles} target.write(context, produced_plan=plan) document = json.loads((tmp_path / "openhcs_metadata.json").read_text()) - reopened = VirtualWorkspaceSourceProjection.from_openhcs_metadata( - tmp_path, document - ) + reopened = VirtualWorkspaceSourceProjection.from_openhcs_metadata(tmp_path, document) projections = reopened.source_projections_by_virtual_path - relative_paths = { - path.removeprefix(str(tmp_path) + "/") - for path in produced_paths | materialized_paths - } + relative_paths = {path.removeprefix(str(tmp_path) + "/") for path in produced_paths | materialized_paths} assert set(projections) == relative_paths | produced_paths | materialized_paths - assert {projection.source_alias for projection in projections.values()} == { - role for role, _ in roles - } + assert {projection.source_alias for projection in projections.values()} == {role for role, _ in roles} for relative_path in relative_paths: projection = projections[relative_path] if mosaic: @@ -555,26 +380,15 @@ def test_saved_roles_publish_once_per_persisted_occurrence( assert metadata.source_image_provenance_planes.contributor_count == 2 assert metadata.source_voxel_spacing.values_zyx == (0.5, 0.5) assert projection.execution_scope.fixed_component_values == ( - (Microscopy.Channel, "2"), - (Microscopy.ZIndex, "1"), + (Microscopy.Channel, "2"), (Microscopy.ZIndex, "1"), (Microscopy.Timepoint, "1"), ) - np.testing.assert_array_equal( - tifffile.imread(tmp_path / relative_path), mosaic_pixels - ) + np.testing.assert_array_equal(tifffile.imread(tmp_path / relative_path), mosaic_pixels) continue if aggregate: assert projection.address is None - np.testing.assert_array_equal( - tifffile.imread(tmp_path / relative_path), pixels - ) + np.testing.assert_array_equal(tifffile.imread(tmp_path / relative_path), pixels) continue - assert projection.address.as_component_metadata() == { - key: components[key] - for key in ("well", "site", "channel", "z_index", "timepoint") - } + assert projection.address.as_component_metadata() == {key: components[key] for key in ("well", "site", "channel", "z_index", "timepoint")} expected_scale = 1.0 if projection.source_alias == "RawRole" else 0.5 - np.testing.assert_array_equal( - tifffile.imread(tmp_path / relative_path), - np.ones((4, 5), dtype=np.float32) * expected_scale, - ) + np.testing.assert_array_equal(tifffile.imread(tmp_path / relative_path), np.ones((4, 5), dtype=np.float32) * expected_scale) diff --git a/tests/unit/test_documented_dimensionality.py b/tests/unit/test_documented_dimensionality.py index 8ae8c71c0c..5a2e513f1a 100644 --- a/tests/unit/test_documented_dimensionality.py +++ b/tests/unit/test_documented_dimensionality.py @@ -3,9 +3,7 @@ from pathlib import Path from openhcs.core.callable_contract import CallableContract -from openhcs.core.pipeline.function_contracts import ( - object_label_input_execution_mode_from_callable, -) +from openhcs.core.pipeline.function_contracts import object_label_input_execution_mode_from_callable from openhcs.processing.backends.cellprofiler.area_occupied import ( measure_image_volume_occupied_binary, measure_image_volume_occupied_objects, diff --git a/tests/unit/test_experimental_analysis_engine.py b/tests/unit/test_experimental_analysis_engine.py index e199b58504..1109908af6 100644 --- a/tests/unit/test_experimental_analysis_engine.py +++ b/tests/unit/test_experimental_analysis_engine.py @@ -81,9 +81,7 @@ def test_desktop_workflow_does_not_copy_config_filenames_or_use_legacy_runner() def test_engine_projects_declaration_owned_sheet_names(monkeypatch) -> None: - from openhcs.processing.backends.experimental_analysis import ( - analysis as experimental_analysis, - ) + from openhcs.processing.backends.experimental_analysis import analysis as experimental_analysis config = ExperimentalAnalysisConfig( design_sheet_name="design", @@ -112,9 +110,7 @@ def test_engine_projects_declaration_owned_sheet_names(monkeypatch) -> None: def test_engine_delegates_to_declared_normalization_method(monkeypatch) -> None: - from openhcs.processing.backends.experimental_analysis import ( - analysis as experimental_analysis, - ) + from openhcs.processing.backends.experimental_analysis import analysis as experimental_analysis engine = ExperimentalAnalysisEngine( ExperimentalAnalysisConfig(normalization_method=NormalizationMethod.Z_SCORE) diff --git a/tests/unit/test_exported_relationship_equivalence.py b/tests/unit/test_exported_relationship_equivalence.py index 13b1707ce0..10b79f331c 100644 --- a/tests/unit/test_exported_relationship_equivalence.py +++ b/tests/unit/test_exported_relationship_equivalence.py @@ -1,5 +1,5 @@ -from tests.unit.saved_output_dialect import SAVED_OUTPUT_POLICY +from tests.unit.saved_output_dialect import SAVED_OUTPUT_POLICY """Saved directed edges must retain the native object-row correlations.""" from dataclasses import replace @@ -61,17 +61,12 @@ def _edit_table(snapshot: RuntimeOutputSnapshot, stem: str, **changes): return replace( snapshot, tables=tuple( - ( - replace( - table, - **{ - name: change(getattr(table, name)) - for name, change in changes.items() - }, - ) - if table.path.stem == stem - else table + replace( + table, + **{name: change(getattr(table, name)) for name, change in changes.items()}, ) + if table.path.stem == stem + else table for table in snapshot.tables ), ) @@ -90,9 +85,7 @@ def test_saved_relationship_edges_join_native_parent_and_zero_child_counts( candidate = RuntimeMeasurementSnapshot.from_output_snapshot( _saved_outputs(tmp_path / "openhcs", edges=True), policy=SAVED_OUTPUT_POLICY ) - assert runtime_measurement_equivalence( - reference, candidate, policy=SAVED_OUTPUT_POLICY - ).is_equivalent + assert runtime_measurement_equivalence(reference, candidate, policy=SAVED_OUTPUT_POLICY).is_equivalent def test_relationship_endpoint_permutation_cannot_hide_in_equal_value_histograms( @@ -102,12 +95,9 @@ def test_relationship_endpoint_permutation_cannot_hide_in_equal_value_histograms _saved_outputs(tmp_path / "native", edges=False), policy=SAVED_OUTPUT_POLICY ) candidate = RuntimeMeasurementSnapshot.from_output_snapshot( - _saved_outputs(tmp_path / "openhcs", edges=True, swapped=True), - policy=SAVED_OUTPUT_POLICY, + _saved_outputs(tmp_path / "openhcs", edges=True, swapped=True), policy=SAVED_OUTPUT_POLICY ) - assert not runtime_measurement_equivalence( - reference, candidate, policy=SAVED_OUTPUT_POLICY - ).is_equivalent + assert not runtime_measurement_equivalence(reference, candidate, policy=SAVED_OUTPUT_POLICY).is_equivalent def test_relationship_rows_must_agree_with_explicit_parent_measurements( @@ -125,9 +115,7 @@ def test_relationship_rows_must_agree_with_explicit_parent_measurements( ), ) with pytest.raises(ValueError, match="relationship|Relationship"): - RuntimeMeasurementSnapshot.from_output_snapshot( - snapshot, policy=SAVED_OUTPUT_POLICY - ) + RuntimeMeasurementSnapshot.from_output_snapshot(snapshot, policy=SAVED_OUTPUT_POLICY) def test_ordinary_image_row_conflicts_remain_rejected(tmp_path: Path) -> None: @@ -222,9 +210,7 @@ def test_saved_edge_schema_and_coverage_rejects_malformed_inputs( snapshot, "Relationships", header=lambda _: header, rows=lambda _: rows ) with pytest.raises((ValueError, TypeError), match=message): - RuntimeMeasurementSnapshot.from_output_snapshot( - snapshot, policy=SAVED_OUTPUT_POLICY - ) + RuntimeMeasurementSnapshot.from_output_snapshot(snapshot, policy=SAVED_OUTPUT_POLICY) def test_saved_relationship_counts_include_noncontiguous_zero_child_parents( @@ -237,28 +223,19 @@ def test_saved_relationship_counts_include_noncontiguous_zero_child_parents( snapshot, "Cells", rows=lambda rows: tuple( - tuple( - "30" if i == 1 and value == "3" else value - for i, value in enumerate(row) - ) + tuple("30" if i == 1 and value == "3" else value for i, value in enumerate(row)) for row in rows ), ) for snapshot in (native, candidate) ) - a = RuntimeMeasurementSnapshot.from_output_snapshot( - native, policy=SAVED_OUTPUT_POLICY - ) - b = RuntimeMeasurementSnapshot.from_output_snapshot( - candidate, policy=SAVED_OUTPUT_POLICY - ) + a = RuntimeMeasurementSnapshot.from_output_snapshot(native, policy=SAVED_OUTPUT_POLICY) + b = RuntimeMeasurementSnapshot.from_output_snapshot(candidate, policy=SAVED_OUTPUT_POLICY) assert ( a.correlated_relationships is not None and b.correlated_relationships is not None ) - assert runtime_measurement_equivalence( - a, b, policy=SAVED_OUTPUT_POLICY - ).is_equivalent + assert runtime_measurement_equivalence(a, b, policy=SAVED_OUTPUT_POLICY).is_equivalent def test_snapshot_cache_retains_known_correlations_and_unknown_typed_scope( @@ -313,9 +290,7 @@ def test_saved_relationship_omitted_slice_count_preserves_optional_payload_contr header=lambda header: header[:-1], rows=lambda rows: tuple(row[:-1] for row in rows), ) - result = RuntimeMeasurementSnapshot.from_output_snapshot( - snapshot, policy=SAVED_OUTPUT_POLICY - ) + result = RuntimeMeasurementSnapshot.from_output_snapshot(snapshot, policy=SAVED_OUTPUT_POLICY) assert result.correlated_relationships is not None @@ -330,12 +305,8 @@ def test_saved_relationship_row_order_does_not_change_directed_correlation( "Relationships", rows=lambda rows: rows[:4] + rows[4:][::-1], ) - result = RuntimeMeasurementSnapshot.from_output_snapshot( - snapshot, policy=SAVED_OUTPUT_POLICY - ) - assert runtime_measurement_equivalence( - native, result, policy=SAVED_OUTPUT_POLICY - ).is_equivalent + result = RuntimeMeasurementSnapshot.from_output_snapshot(snapshot, policy=SAVED_OUTPUT_POLICY) + assert runtime_measurement_equivalence(native, result, policy=SAVED_OUTPUT_POLICY).is_equivalent def test_unparented_zero_edges_still_validate_actual_child_domain( @@ -351,9 +322,7 @@ def test_unparented_zero_edges_still_validate_actual_child_domain( "Relationships", rows=lambda rows: tuple((*row[:6], "0", *row[7:]) for row in rows), ) - result = RuntimeMeasurementSnapshot.from_output_snapshot( - snapshot, policy=SAVED_OUTPUT_POLICY - ) + result = RuntimeMeasurementSnapshot.from_output_snapshot(snapshot, policy=SAVED_OUTPUT_POLICY) assert result.correlated_relationships is not None assert all( not graph.source_keys for graph in result.correlated_relationships.values() @@ -364,9 +333,7 @@ def test_unparented_zero_edges_still_validate_actual_child_domain( rows=lambda rows: tuple((*row[:7], "999", *row[8:]) for row in rows), ) with pytest.raises(ValueError, match="absent child"): - RuntimeMeasurementSnapshot.from_output_snapshot( - snapshot, policy=SAVED_OUTPUT_POLICY - ) + RuntimeMeasurementSnapshot.from_output_snapshot(snapshot, policy=SAVED_OUTPUT_POLICY) @pytest.mark.parametrize( @@ -381,9 +348,7 @@ def test_unparented_zero_edges_still_validate_actual_child_domain( def test_child_count_parser_and_core_declaration_share_original_grammar( feature: str, identity: str | None ) -> None: - from openhcs.interop.cellprofiler.measurement_lookup import ( - ChildCountFeatureDeclaration, - ) + from openhcs.interop.cellprofiler.measurement_lookup import ChildCountFeatureDeclaration from openhcs.interop.cellprofiler.measurement_lookup import ( CellProfilerChildCountFeatureParser, ) @@ -432,9 +397,7 @@ def test_native_parent_rows_require_real_integral_parent_domain( rows=lambda rows: tuple((*row[:2], parent_value) for row in rows), ) with pytest.raises(ValueError, match=message): - RuntimeMeasurementSnapshot.from_output_snapshot( - snapshot, policy=SAVED_OUTPUT_POLICY - ) + RuntimeMeasurementSnapshot.from_output_snapshot(snapshot, policy=SAVED_OUTPUT_POLICY) def test_native_child_count_permutation_cannot_hide_in_equal_histograms( @@ -443,14 +406,10 @@ def test_native_child_count_permutation_cannot_hide_in_equal_histograms( snapshot = _edit_table( _saved_outputs(tmp_path / "native", edges=False), "Cells", - rows=lambda rows: tuple( - (*row[:2], "0" if row[1] == "1" else "1") for row in rows - ), + rows=lambda rows: tuple((*row[:2], "0" if row[1] == "1" else "1") for row in rows), ) with pytest.raises(ValueError, match="disagree with.*children_nuclei_count"): - RuntimeMeasurementSnapshot.from_output_snapshot( - snapshot, policy=SAVED_OUTPUT_POLICY - ) + RuntimeMeasurementSnapshot.from_output_snapshot(snapshot, policy=SAVED_OUTPUT_POLICY) def test_snapshot_json_cache_retains_correlation_scope(tmp_path: Path) -> None: diff --git a/tests/unit/test_exported_table_axis_snapshot.py b/tests/unit/test_exported_table_axis_snapshot.py index df5369e665..aaf5d67d97 100644 --- a/tests/unit/test_exported_table_axis_snapshot.py +++ b/tests/unit/test_exported_table_axis_snapshot.py @@ -1,10 +1,10 @@ + from tests.unit.saved_output_dialect import SAVED_OUTPUT_POLICY from benchmark.equivalence.outputs import ExportedTableAxis from openhcs.interop.cellprofiler.measurement_dialect import ( CELLPROFILER_MEASUREMENT_DIALECT, ) - """Exporter-owned row domains use physical paths before semantic namespaces.""" from pathlib import Path @@ -56,8 +56,7 @@ def test_prefixed_axis_snapshot_preserves_exact_relationships(tmp_path, prefix, candidate = _exports(tmp_path / "candidate", prefix, (1, 2)) physical_contents = {path: path.read_bytes() for path in candidate.output_files} snapshot = RuntimeOutputSnapshot.from_export_observation( - candidate.for_execution_axis(axis), - execution_axis=ExportedTableAxis(axis, CELLPROFILER_MEASUREMENT_DIALECT), + candidate.for_execution_axis(axis), execution_axis=ExportedTableAxis(axis, CELLPROFILER_MEASUREMENT_DIALECT) ) assert all(len(table.rows) == 1 for table in snapshot.tables) number = "1" if axis == "W001" else "2" @@ -68,12 +67,9 @@ def test_prefixed_axis_snapshot_preserves_exact_relationships(tmp_path, prefix, "Image.csv", } reference = RuntimeMeasurementSnapshot.from_output_snapshot( - RuntimeOutputSnapshot.from_export_observation(native), - policy=SAVED_OUTPUT_POLICY, - ) - actual = RuntimeMeasurementSnapshot.from_output_snapshot( - snapshot, policy=SAVED_OUTPUT_POLICY + RuntimeOutputSnapshot.from_export_observation(native), policy=SAVED_OUTPUT_POLICY ) + actual = RuntimeMeasurementSnapshot.from_output_snapshot(snapshot, policy=SAVED_OUTPUT_POLICY) assert ( actual.required_relationship_correlations() == reference.required_relationship_correlations() @@ -88,13 +84,10 @@ def test_prefixed_axis_snapshot_still_rejects_absent_parent(tmp_path): path = tmp_path / "candidate/experiment_Children.csv" path.write_text("ImageNumber,ObjectNumber,Parent_Parents\n1,11,7\n2,11,8\n") snapshot = RuntimeOutputSnapshot.from_export_observation( - candidate.for_execution_axis("W002"), - execution_axis=ExportedTableAxis("W002", CELLPROFILER_MEASUREMENT_DIALECT), + candidate.for_execution_axis("W002"), execution_axis=ExportedTableAxis("W002", CELLPROFILER_MEASUREMENT_DIALECT) ) with pytest.raises(ValueError, match="absent parent endpoint"): - RuntimeMeasurementSnapshot.from_output_snapshot( - snapshot, policy=SAVED_OUTPUT_POLICY - ) + RuntimeMeasurementSnapshot.from_output_snapshot(snapshot, policy=SAVED_OUTPUT_POLICY) def test_prefixed_axis_snapshot_still_rejects_unowned_image_id(tmp_path): @@ -103,6 +96,5 @@ def test_prefixed_axis_snapshot_still_rejects_unowned_image_id(tmp_path): path.write_text("ImageNumber,ObjectNumber,Parent_Parents\n1,11,7\n3,11,7\n") with pytest.raises(ValueError, match="unowned image identity"): RuntimeOutputSnapshot.from_export_observation( - candidate.for_execution_axis("W002"), - execution_axis=ExportedTableAxis("W002", CELLPROFILER_MEASUREMENT_DIALECT), + candidate.for_execution_axis("W002"), execution_axis=ExportedTableAxis("W002", CELLPROFILER_MEASUREMENT_DIALECT) ) diff --git a/tests/unit/test_fast_speckle_opening.py b/tests/unit/test_fast_speckle_opening.py index 5c617c366f..4464708d5e 100644 --- a/tests/unit/test_fast_speckle_opening.py +++ b/tests/unit/test_fast_speckle_opening.py @@ -25,13 +25,11 @@ def reflected_opening_reference(image, footprint): """Explicit reflect-index oracle, independent of either native filter.""" padding = tuple((size // 2, size // 2) for size in footprint.shape) erode_windows = np.lib.stride_tricks.sliding_window_view( - np.pad(image, padding, mode="symmetric"), - footprint.shape, + np.pad(image, padding, mode="symmetric"), footprint.shape, ) eroded = erode_windows[..., footprint.astype(bool)].min(axis=-1) dilate_windows = np.lib.stride_tricks.sliding_window_view( - np.pad(eroded, padding, mode="symmetric"), - footprint.shape, + np.pad(eroded, padding, mode="symmetric"), footprint.shape, ) return dilate_windows[..., footprint.astype(bool)].max(axis=-1) @@ -45,18 +43,13 @@ def test_declared_fast_speckles_matches_original_disk_opening(shape, dtype, radi image[-1, -1] = -5 footprint = morphology.disk(max(1, round(radius))) expected = image - ndimage.maximum_filter( - ndimage.minimum_filter(image, footprint=footprint), - footprint=footprint, + ndimage.minimum_filter(image, footprint=footprint), footprint=footprint, ) actual = enhance_or_suppress_features( - image, - radius=radius, - speckle_accuracy=SpeckleAccuracy.FAST, + image, radius=radius, speckle_accuracy=SpeckleAccuracy.FAST, dtype_config=DtypeConfig(), ) - np.testing.assert_array_equal( - ImagePayload.of(actual).data, expected.astype(np.float32) - ) + np.testing.assert_array_equal(ImagePayload.of(actual).data, expected.astype(np.float32)) assert ImagePayload.of(actual).data.dtype == np.float32 @@ -73,15 +66,11 @@ def test_speckle_mask_background_and_metadata_remain_owned(accuracy, masked): masked_image = np.where(mask, image, 0) footprint = morphology.disk(5) expected = masked_image - ndimage.maximum_filter( - ndimage.minimum_filter(masked_image, footprint=footprint), - footprint=footprint, + ndimage.minimum_filter(masked_image, footprint=footprint), footprint=footprint, ) expected[~mask] = image[~mask] actual = enhance_or_suppress_features( - original, - radius=5, - speckle_accuracy=accuracy, - dtype_config=DtypeConfig(), + original, radius=5, speckle_accuracy=accuracy, dtype_config=DtypeConfig(), ) np.testing.assert_array_equal(ImagePayload.of(actual).data, expected) np.testing.assert_array_equal(actual.mask, mask) @@ -91,22 +80,15 @@ def test_speckle_mask_background_and_metadata_remain_owned(accuracy, masked): def test_fast_speckles_registered_callable_keeps_independent_planes(): image = np.random.default_rng(616).uniform(0, 1, (2, 17, 23)).astype(np.float32) - expected = np.stack( - [ - plane - - ndimage.maximum_filter( - ndimage.minimum_filter(plane, footprint=morphology.disk(5)), - footprint=morphology.disk(5), - ) - for plane in image - ] - ) + expected = np.stack([ + plane - ndimage.maximum_filter( + ndimage.minimum_filter(plane, footprint=morphology.disk(5)), + footprint=morphology.disk(5), + ) for plane in image + ]) actual = enhance_or_suppress_features(image, radius=5, dtype_config=DtypeConfig()) np.testing.assert_array_equal(ImagePayload.of(actual).data, expected) - assert ( - CallableContract.from_callable(enhance_or_suppress_features).processing_contract - is Pure2DContract - ) + assert CallableContract.from_callable(enhance_or_suppress_features).processing_contract is Pure2DContract @pytest.mark.parametrize("shape,radius", [((1, 9), 150), ((5, 7), 12), ((17, 23), 5)]) @@ -117,12 +99,8 @@ def test_fast_speckles_matches_reflected_opening_math(shape, radius, dtype): image[-1, -1] = -5 before = image.copy() expected = image - reflected_opening_reference(image, morphology.disk(radius)) - actual = enhance_or_suppress_features( - image, radius=radius, dtype_config=DtypeConfig() - ) - np.testing.assert_array_equal( - ImagePayload.of(actual).data, expected.astype(np.float32) - ) + actual = enhance_or_suppress_features(image, radius=radius, dtype_config=DtypeConfig()) + np.testing.assert_array_equal(ImagePayload.of(actual).data, expected.astype(np.float32)) np.testing.assert_array_equal(image, before) @@ -136,8 +114,7 @@ def test_fast_oversized_opening_preserves_constant_negative_plane(): def test_speckle_module_keeps_original_feature_size_binding(): bound = BoundModuleSettings({"feature_size": 11}) actual = EnhanceOrSuppressFeaturesModule.postprocess_bound_settings( - ModuleBlock("EnhanceOrSuppressFeatures", 1), - bound, + ModuleBlock("EnhanceOrSuppressFeatures", 1), bound, ) assert actual.kwargs == {"radius": 5.5} assert actual.unmapped_kwargs == bound.unmapped_kwargs diff --git a/tests/unit/test_fiji_viewer_server.py b/tests/unit/test_fiji_viewer_server.py index fc0e3f0ed2..29f16608a4 100644 --- a/tests/unit/test_fiji_viewer_server.py +++ b/tests/unit/test_fiji_viewer_server.py @@ -61,7 +61,6 @@ ) from tests.unit.viewer_axes_fixture import STREAM_AXES - def fiji_control_server(*, settlement=None, ij=None, process_launch=None): """A Fiji server with only the state its control actions read.""" @@ -90,11 +89,8 @@ def fiji_control_server(*, settlement=None, ij=None, process_launch=None): "artifact_kind": None, } -ACK_ROUTE = AckReturnRoute( - "tcp://127.0.0.1:8111", - "00000000-0000-0000-0000-000000000001", - ProcessIdentity.current(), -) +ACK_ROUTE = AckReturnRoute("tcp://127.0.0.1:8111", "00000000-0000-0000-0000-000000000001", + ProcessIdentity.current()) def _component_value_domain(payload: dict) -> ViewerComponentValueDomainPayload: @@ -227,6 +223,14 @@ def test_fiji_server_stop_closes_transport_before_owned_jvm(monkeypatch) -> None assert server.ij is None + + + + + + + + def test_fiji_settlement_tracks_queued_and_bounded_native_work() -> None: settlement = FijiBatchSettlementState() @@ -322,6 +326,8 @@ def test_fiji_window_registry_closes_replaced_hyperstack() -> None: assert registry.open_hyperstack("A14") is new_image_plus + + def test_fiji_macro_control_executes_inside_managed_imagej_runtime(tmp_path) -> None: macro_path = tmp_path / "copy.ijm" macro_path.write_text("// managed macro", encoding="utf-8") @@ -650,9 +656,7 @@ def test_fiji_payload_local_plane_axis_rejects_malformed_declarations() -> None: } with pytest.raises(ValueError, match="exactly one component declaration"): - FijiPayloadLocalPlaneExpander(STREAM_AXES).expand_item( - FijiWireItem.from_payload(payload) - ) + FijiPayloadLocalPlaneExpander(STREAM_AXES).expand_item(FijiWireItem.from_payload(payload)) payload_without_axis = dict(payload) payload_without_axis["plane_component_values"] = {"site": [1, 2]} diff --git a/tests/unit/test_funcstep_contract_validator.py b/tests/unit/test_funcstep_contract_validator.py index 19e5d3b4dd..22871cd1f4 100644 --- a/tests/unit/test_funcstep_contract_validator.py +++ b/tests/unit/test_funcstep_contract_validator.py @@ -326,8 +326,7 @@ def process(image): contract = CallableContract.from_callable(process) with pytest.raises( - ValueError, - match=r"requires variable_components with role TimeAxis \(timepoint\)", + ValueError, match=r"requires variable_components with role TimeAxis \(timepoint\)" ): FuncStepContractValidator.validate_required_axis_roles( (), @@ -777,7 +776,5 @@ def process(image): ), ) - with pytest.raises( - ValueError, match="requires variable_components with role TimeAxis" - ): + with pytest.raises(ValueError, match="requires variable_components with role TimeAxis"): FuncStepContractValidator.validate_funcstep(step) diff --git a/tests/unit/test_function_artifact_materialization.py b/tests/unit/test_function_artifact_materialization.py index 8d7b4f565b..17096f587f 100644 --- a/tests/unit/test_function_artifact_materialization.py +++ b/tests/unit/test_function_artifact_materialization.py @@ -428,6 +428,7 @@ def _plan( AxisFamily.active().named(group_by_value) if group_by_value is not None else Ungrouped + ), execution_group_scope=execution_group_scope, variable_components=variable_components, @@ -648,9 +649,7 @@ def test_slice_aligned_object_label_arrays_preserve_source_slice_metadata(): ) label_slices = RuntimeSliceAlignedValues(tuple(expected_labels)) - contextualized = ( - ImageArtifactType if output_plan is None else output_plan.artifact_type - ).contextualize_output( + contextualized = (ImageArtifactType if output_plan is None else output_plan.artifact_type).contextualize_output( source, label_slices, output_plan, @@ -729,9 +728,7 @@ def test_image_outputs_merge_source_provenance_when_output_already_has_metadata( metadata=ImagePayloadMetadata(source_dtype="float32"), ) - contextualized = ( - ImageArtifactType if output_plan is None else output_plan.artifact_type - ).contextualize_output( + contextualized = (ImageArtifactType if output_plan is None else output_plan.artifact_type).contextualize_output( source, output, output_plan, @@ -787,9 +784,7 @@ def test_object_label_payload_stack_preserves_source_slice_metadata(): ), ) - contextualized = ( - ImageArtifactType if output_plan is None else output_plan.artifact_type - ).contextualize_output( + contextualized = (ImageArtifactType if output_plan is None else output_plan.artifact_type).contextualize_output( source, labels, output_plan, @@ -814,7 +809,9 @@ def test_object_label_payload_stack_preserves_source_slice_metadata(): assert runtime_value.data.source_spatial_domain == SourceSpatialDomain( value_name="Object-label" ) - assert runtime_value.data.source_provenance == (source.metadata.source_provenance) + assert runtime_value.data.source_provenance == ( + source.metadata.source_provenance + ) assert runtime_value.data.dimensions == () assert runtime_value.data.source_image_name is None assert runtime_value.materialization_payload() is runtime_value.data @@ -854,9 +851,7 @@ def fake_materialize(_spec, data, path, *_args, **_kwargs): _observe_materialization(monkeypatch, fake_materialize) - PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( - filemanager, _plan(output_plan), context - ) + PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs(filemanager, _plan(output_plan), context) assert materialized == [ ({"x": "from-runtime"}, "/analysis/A01_positions_step7.roi.zip") @@ -897,17 +892,13 @@ def fake_materialize(_spec, data, path, *_args, **_kwargs): _observe_materialization(monkeypatch, fake_materialize) - PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( - filemanager, - _plan( + PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs(filemanager, _plan( output_plan, variable_components=( Microscopy.ZIndex, Microscopy.Channel, ), - ), - context, - ) + ), context) assert len(materialized) == 1 data, path = materialized[0] @@ -995,9 +986,7 @@ def fake_materialize(spec, data, path, manager, *_args, **kwargs): == expected_paths ) - PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( - filemanager, plan, context - ) + PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs(filemanager, plan, context) assert output_paths == [str(path) for path in expected_paths] @@ -1013,9 +1002,7 @@ def test_materialize_artifact_outputs_requires_runtime_store_record(): context = _context(filemanager) with pytest.raises(RuntimeError, match="Missing RuntimeValueStore record"): - PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( - filemanager, _plan(output_plan), context - ) + PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs(filemanager, _plan(output_plan), context) def test_materialize_artifact_outputs_does_not_require_vfs_payload_for_store_record( @@ -1040,9 +1027,7 @@ def fake_materialize(_spec, data, path, *_args, **_kwargs): _observe_materialization(monkeypatch, fake_materialize) - PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( - filemanager, _plan(output_plan), context - ) + PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs(filemanager, _plan(output_plan), context) assert materialized == [({"x": 1}, "/analysis/A01_positions_step7.roi.zip")] @@ -1088,15 +1073,10 @@ def fake_materialize(_spec, data, path, *_args, **_kwargs): _observe_materialization(monkeypatch, fake_materialize) - PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( - filemanager, _plan(output_plan), context - ) + PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs(filemanager, _plan(output_plan), context) assert [(data.row_mappings(), path) for data, path in materialized] == [ - ( - ({"object_id": 1, "area": 42},), - "/analysis/A01_channel-1_measurements_step7.roi.zip", - ) + (({"object_id": 1, "area": 42},), "/analysis/A01_channel-1_measurements_step7.roi.zip") ] @@ -1138,9 +1118,7 @@ def fake_materialize(spec, data, path, *_args, **_kwargs): _observe_materialization(monkeypatch, fake_materialize) - PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( - filemanager, _plan(output_plan), context - ) + PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs(filemanager, _plan(output_plan), context) spec, data, path = materialized[0] assert isinstance(spec.outputs[0], CsvOptions) @@ -1221,9 +1199,7 @@ def test_runtime_artifact_plan_materializes_3d_point_measurements(source_z_origi assert [item["z_index"] for item in domain.values] == list( range(source_z_origin, source_z_origin + 4) ) - assert ( - result.viewer_source_identity.component_metadata["z_index"] == source_z_origin - ) + assert result.viewer_source_identity.component_metadata["z_index"] == source_z_origin config = streaming_config_stub() plan = _plan( output_plan, @@ -1231,18 +1207,13 @@ def test_runtime_artifact_plan_materializes_3d_point_measurements(source_z_origi streaming_configs={"napari_stream": config}, ) [saved] = StreamingOnlyArtifactMaterializationTargetPlan().materialize_outputs( - context.filemanager, - plan, - context, + context.filemanager, plan, context, ) [streamed] = saved.outputs_for_backend("napari_stream") assert streamed.content[0].metadata[ROIFractionalZ.FIELD] == 2.375 request = context.filemanager.saved[-1][3]["stream_request"] assert "z_index" in request.display_config.COMPONENT_ORDER - assert ( - request.source.metadata.component_metadata_for_item(streamed.path, 0)["z_index"] - == source_z_origin - ) + assert request.source.metadata.component_metadata_for_item(streamed.path, 0)["z_index"] == source_z_origin z_domain = request.message_extra["component_value_domain"] assert z_domain["z_index"] == list(range(source_z_origin, source_z_origin + 4)) @@ -1323,9 +1294,7 @@ def test_multi_plane_measurement_materialization_uses_aggregate_artifact_name(): ) -@pytest.mark.parametrize( - "parser_type", [ImageXpressFilenameParser, SourceSchemaFilenameParser] -) +@pytest.mark.parametrize("parser_type", [ImageXpressFilenameParser, SourceSchemaFilenameParser]) def test_multi_plane_special_output_uses_aggregate_artifact_name(parser_type): output_plan = ArtifactOutputPlan( name="cell_counts", @@ -1376,9 +1345,7 @@ def test_multi_plane_special_output_uses_aggregate_artifact_name(parser_type): ) -@pytest.mark.parametrize( - "parser_type", [ImageXpressFilenameParser, SourceSchemaFilenameParser] -) +@pytest.mark.parametrize("parser_type", [ImageXpressFilenameParser, SourceSchemaFilenameParser]) def test_scalar_special_output_preserves_complete_source_identity(parser_type): output_plan = ArtifactOutputPlan( name="cell_counts", @@ -1422,9 +1389,7 @@ def test_scalar_special_output_preserves_complete_source_identity(parser_type): ) -@pytest.mark.parametrize( - "parser_type", [ImageXpressFilenameParser, SourceSchemaFilenameParser] -) +@pytest.mark.parametrize("parser_type", [ImageXpressFilenameParser, SourceSchemaFilenameParser]) def test_incomplete_scalar_special_output_keeps_strict_filename_failure(parser_type): output_plan = ArtifactOutputPlan( name="cell_counts", @@ -1462,9 +1427,7 @@ def test_incomplete_scalar_special_output_keeps_strict_filename_failure(parser_t runtime_artifact_materializations(_plan(output_plan), context) -@pytest.mark.parametrize( - "parser_type", [ImageXpressFilenameParser, SourceSchemaFilenameParser] -) +@pytest.mark.parametrize("parser_type", [ImageXpressFilenameParser, SourceSchemaFilenameParser]) def test_grouped_special_output_retains_group_coordinate_in_aggregate_name(parser_type): output_plan = ArtifactOutputPlan( name="cell_counts", @@ -1598,7 +1561,9 @@ def test_multi_plane_roi_aggregate_defers_source_filenames_to_plane_writer(monke ) roi_outputs = materialization.outputs(plan, context) assert tuple( - output.path for output in roi_outputs if output.path.endswith(".roi.zip") + output.path + for output in roi_outputs + if output.path.endswith(".roi.zip") ) == tuple( ( f"/analysis/A01_s{site:03d}_w{channel}_z001_t001" @@ -1619,12 +1584,8 @@ def test_multi_plane_roi_aggregate_defers_source_filenames_to_plane_writer(monke "Consolidation must not reconstruct unrelated ROI geometry" ), ) - hook = AnalysisConsolidationHook( - AnalysisConsolidationConfig(), PlateMetadataConfig() - ) - assert ( - hook.observe_reused_outputs(context, plan, materialization, roi_outputs) is None - ) + hook = AnalysisConsolidationHook(AnalysisConsolidationConfig(), PlateMetadataConfig()) + assert hook.observe_reused_outputs(context, plan, materialization, roi_outputs) is None def test_multi_plane_measurement_aggregate_names_retain_runtime_group_coordinate(): @@ -1774,12 +1735,9 @@ def record_for_site(site: str) -> None: current_execution_records = context.runtime_value_store.observed_values_after( current_execution_cursor ) - saved = PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( - context.filemanager, plan, context - ) + saved = PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs(context.filemanager, plan, context) selected_saved = tuple( - artifact - for artifact in saved + artifact for artifact in saved if artifact.materialization.record.key == current_execution_records[0].key ) assert len(selected_saved) == 1 @@ -1788,12 +1746,7 @@ def record_for_site(site: str) -> None: context.filemanager.save(historical_content, selected_path, "disk") reused = preview_reused_step_outputs(plan, context, current_execution_records) assert reused.hook_observations.get("analysis_consolidation") is not None - assert ( - reused.hook_observations.get("analysis_consolidation") - .outputs_by_directory[Path("/analysis")][0] - .csv_content - == historical_content - ) + assert reused.hook_observations.get("analysis_consolidation").outputs_by_directory[Path("/analysis")][0].csv_content == historical_content assert tuple( Path(location.path) for locations in reused.materialized_locations_by_address.values() @@ -1816,11 +1769,7 @@ def record_for_site(site: str) -> None: ), } context.step_plans = {plan.step_index: plan} - consolidation_inputs = ( - selected_saved[0] - .observation(plan, context) - .hook_observations.get("analysis_consolidation") - ) + consolidation_inputs = selected_saved[0].observation(plan, context).hook_observations.get("analysis_consolidation") assert consolidation_inputs is not None runtime_output = consolidation_inputs.outputs_by_directory[Path("/analysis")][0] assert runtime_output.path == Path( @@ -1837,17 +1786,14 @@ def record_for_site(site: str) -> None: @pytest.mark.parametrize("sites", [("1", "3"), ("3", "1")]) @pytest.mark.parametrize("automatic", [True, False]) -def test_scalar_acquired_tables_keep_fixed_site_addresses_on_shared_runtime_path( - sites, automatic -): +def test_scalar_acquired_tables_keep_fixed_site_addresses_on_shared_runtime_path(sites, automatic): output_plan = ArtifactOutputPlan( name="AcquiredFieldMeasurements", path="/memory/A01_w1_AcquiredFieldMeasurements_step0.pkl", artifact_type=MeasurementsArtifactType, materialization=( AutomaticMeasurementsArtifactOutputMaterializationStrategy().materialization() - if automatic - else csv_only() + if automatic else csv_only() ), variable_components=(Microscopy.ZIndex,), ) @@ -1863,11 +1809,8 @@ def test_scalar_acquired_tables_keep_fixed_site_addresses_on_shared_runtime_path ) for site in sites: metadata = { - "well": "A01", - "site": site, - "channel": "1", - "z_index": "1", - "timepoint": "1", + "well": "A01", "site": site, "channel": "1", + "z_index": "1", "timepoint": "1", } context.runtime_value_store.record( RuntimeValue.normalize_for_execution_scope( @@ -1887,9 +1830,7 @@ def test_scalar_acquired_tables_keep_fixed_site_addresses_on_shared_runtime_path subject=MeasurementSubject(MeasurementScope.ARTIFACT), ), execution_scope=RuntimeExecutionAxisScope.from_raw( - "A01", - component=Microscopy.Channel, - value="1", + "A01", component=Microscopy.Channel, value="1", fixed_component_values=( (Microscopy.Site, site), (Microscopy.Timepoint, "1"), @@ -1904,9 +1845,7 @@ def test_scalar_acquired_tables_keep_fixed_site_addresses_on_shared_runtime_path assert len(records) == 2 assert len({record.location.path for record in records}) == 1 saved = PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( - context.filemanager, - plan, - context, + context.filemanager, plan, context, ) assert len(saved) == 2 saved_paths = set() @@ -1917,8 +1856,7 @@ def test_scalar_acquired_tables_keep_fixed_site_addresses_on_shared_runtime_path paths = observation.paths_for(record) expected_stem = ( f"A01_site-{site}_channel-1_timepoint-1" - if automatic - else f"A01_s{int(site):03d}_w1_z001_t001" + if automatic else f"A01_s{int(site):03d}_w1_z001_t001" ) assert paths == ( Path( @@ -2017,9 +1955,7 @@ def test_consolidation_preserves_export_bundle_and_text_tables(options, payload) persistent_backend="disk", ) context.step_plans = {plan.step_index: plan} - saved = PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( - context.filemanager, plan, context - ) + saved = PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs(context.filemanager, plan, context) consolidation = StepExecutionObservation.combine( artifact.observation(plan, context) for artifact in saved ).hook_observations.get("analysis_consolidation") @@ -2040,44 +1976,33 @@ def test_consolidation_preserves_export_bundle_and_text_tables(options, payload) (RuntimeObservationMode.OMIT, RuntimeObservationMode.MERGE_INTO_PARENT), ) def test_completed_observation_projects_tables_before_worker_payload_release( - observation_mode, - monkeypatch, + observation_mode, monkeypatch, ): image_plan = ArtifactOutputPlan( - name="Corrected", - path="/memory/Corrected.pkl", - artifact_type=ImageArtifactType, + name="Corrected", path="/memory/Corrected.pkl", artifact_type=ImageArtifactType, ) table_plan = ArtifactOutputPlan( - name="cell_counts", - path="/memory/cell_counts.pkl", - artifact_type=MeasurementsArtifactType, - materialization=csv_only(), + name="cell_counts", path="/memory/cell_counts.pkl", + artifact_type=MeasurementsArtifactType, materialization=csv_only(), ) context = _context(FileManagerStub()) image_step = replace(_plan(image_plan), step_index=5) table_step = _plan(table_plan) table_step.runtime_artifact_materialization = RuntimeArtifactMaterializationPlan( - persistent_enabled=True, - persistent_backend="disk", + persistent_enabled=True, persistent_backend="disk", ) - context.step_plans = { - image_step.step_index: image_step, - table_step.step_index: table_step, - } + context.step_plans = {image_step.step_index: image_step, table_step.step_index: table_step} pixels = np.ones((8, 8), dtype=np.float32) pixel_reference = weakref.ref(pixels) context.runtime_value_store.record( RuntimeValue.normalize( image_plan, ImageMetadataPayload( - data=pixels, - metadata=ImagePayloadMetadata(source_dtype="float32"), + data=pixels, metadata=ImagePayloadMetadata(source_dtype="float32"), ), axis_id="A01", ), - path=image_plan.path, - backend="memory", + path=image_plan.path, backend="memory", ) context.runtime_value_store.record( RuntimeValue.normalize( @@ -2085,30 +2010,24 @@ def test_completed_observation_projects_tables_before_worker_payload_release( MeasurementTable( name=table_plan.name, rows=MeasurementSparseColumnarRows.from_rows( - ({"cell_count": 2},), - fields=(FieldSpec("cell_count", int),), + ({"cell_count": 2},), fields=(FieldSpec("cell_count", int),), ), subject=MeasurementSubject(MeasurementScope.ARTIFACT), ), axis_id="A01", ), - path=table_plan.path, - backend="memory", - ) - saved = PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( - context.filemanager, table_step, context + path=table_plan.path, backend="memory", ) + saved = PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs(context.filemanager, table_step, context) monkeypatch.setattr( - RuntimeArtifactMaterialization, - "outputs", + RuntimeArtifactMaterialization, "outputs", lambda *_args, **_kwargs: pytest.fail("Saved tables must not render again"), ) step_observation = StepExecutionObservation.combine( artifact.observation(table_step, context) for artifact in saved ) observation = RuntimeContextObservation.from_context( - context_key="A01", - context=context, + context_key="A01", context=context, records=context.runtime_value_store.observed_values, runtime_observation_mode=observation_mode, outputs=step_observation, @@ -2123,21 +2042,9 @@ def test_completed_observation_projects_tables_before_worker_payload_release( ) assert len(observation.outputs.runtime_export_paths) == 1 assert observation.outputs.runtime_export_paths[0].suffix == ".csv" - assert ( - observation.outputs.hook_observations.get("analysis_consolidation") is not None - ) - assert ( - observation.outputs.hook_observations.get( - "analysis_consolidation" - ).destination.backend - == "disk" - ) - assert ( - observation.outputs.hook_observations.get( - "analysis_consolidation" - ).destination.images_dir - == "/images" - ) + assert observation.outputs.hook_observations.get("analysis_consolidation") is not None + assert observation.outputs.hook_observations.get("analysis_consolidation").destination.backend == "disk" + assert observation.outputs.hook_observations.get("analysis_consolidation").destination.images_dir == "/images" execution_observation = RuntimeExecutionObservation(contexts=(observation,)) transported = pickle.loads(pickle.dumps(execution_observation)) consolidated = PostExecuteHook.combine_all( @@ -2212,9 +2119,7 @@ def fake_materialize(_spec, data, path, *_args, **_kwargs): _observe_materialization(monkeypatch, fake_materialize) - PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( - filemanager, _plan(output_plan), context - ) + PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs(filemanager, _plan(output_plan), context) assert materialized == [ ( @@ -2508,13 +2413,9 @@ def fake_materialize( _observe_materialization(monkeypatch, fake_materialize) - PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( - filemanager, - _plan( + PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs(filemanager, _plan( output_plan, streaming_configs={"napari_stream": streaming_config_stub()} - ), - context, - ) + ), context) spec, data, path, backends, backend_kwargs = materialized[0] assert isinstance(spec.outputs[0], CsvOptions) @@ -2569,9 +2470,7 @@ def fake_materialize(spec, data, path, *_args, **_kwargs): _observe_materialization(monkeypatch, fake_materialize) - PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( - filemanager, _plan(output_plan), context - ) + PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs(filemanager, _plan(output_plan), context) assert len(materialized) == 1 spec, data, path = materialized[0] @@ -2842,9 +2741,7 @@ def fake_materialize(spec, data, path, *_args, **_kwargs): _observe_materialization(monkeypatch, fake_materialize) - PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( - filemanager, _plan(output_plan), context - ) + PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs(filemanager, _plan(output_plan), context) assert [path for _spec, _data, path in materialized] == [ "/analysis/A01_s001_w5_z001_t001_measurements_step7.roi.zip", @@ -2899,9 +2796,7 @@ def fake_materialize(spec, data, path, *_args, **_kwargs): _observe_materialization(monkeypatch, fake_materialize) - PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( - filemanager, _plan(output_plan, group_by_value="channel"), context - ) + PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs(filemanager, _plan(output_plan, group_by_value="channel"), context) assert [path for _spec, _data, path in materialized] == [ "/analysis/A01_channel-2_measurements_step7.roi.zip", @@ -2958,13 +2853,9 @@ def fake_materialize( _observe_materialization(monkeypatch, fake_materialize) - StreamingOnlyArtifactMaterializationTargetPlan().materialize_outputs( - filemanager, - _plan( + StreamingOnlyArtifactMaterializationTargetPlan().materialize_outputs(filemanager, _plan( output_plan, streaming_configs={"napari_stream": streaming_config_stub()} - ), - context, - ) + ), context) _spec, _data, path, _backends, backend_kwargs = materialized[0] assert path == "/analysis/A01_s001_w2_z001_t001_Nuclei_step7.roi.zip" @@ -3025,11 +2916,7 @@ def fake_materialize( streaming_config = streaming_config_stub() - StreamingOnlyArtifactMaterializationTargetPlan().materialize_outputs( - filemanager, - _plan(output_plan, streaming_configs={"napari_stream": streaming_config}), - context, - ) + StreamingOnlyArtifactMaterializationTargetPlan().materialize_outputs(filemanager, _plan(output_plan, streaming_configs={"napari_stream": streaming_config}), context) _spec, _data, path, _backends, backend_kwargs = materialized[0] assert path == "/analysis/A01_Nuclei_step7.roi.zip" @@ -3169,9 +3056,7 @@ def fake_materialize(spec, data, path, *_args, **_kwargs): _observe_materialization(monkeypatch, fake_materialize) - PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( - filemanager, _plan(output_plan), context - ) + PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs(filemanager, _plan(output_plan), context) spec, data, _path = materialized[0] assert isinstance(spec.outputs[0], JsonOptions) @@ -3196,9 +3081,7 @@ def fake_materialize(*args, **kwargs): _observe_materialization(monkeypatch, fake_materialize) - PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( - filemanager, _plan(output_plan), context - ) + PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs(filemanager, _plan(output_plan), context) assert materialized == [] @@ -3254,18 +3137,14 @@ def test_tile_positions_runtime_materializes_native_json_without_changing_payloa analysis_results_dir=str(tmp_path / "results"), ) - PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( - filemanager, plan, context - ) + PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs(filemanager, plan, context) (retained,) = tuple((tmp_path / "results").rglob("*.json")) assert json.loads(retained.read_text()) == [list(pair) for pair in positions] assert positions == [(-1.25, 0.000000001), (921.600000001, -3.5), (1.0, 1843.2)] assert record.key is value.key assert record.data is value.data - assert ( - record.materialization_source_metadata is value.materialization_source_metadata - ) + assert record.materialization_source_metadata is value.materialization_source_metadata def test_materialize_artifact_outputs_skips_explicitly_disabled_artifact_without_record( @@ -3286,9 +3165,7 @@ def fake_materialize(*args, **kwargs): _observe_materialization(monkeypatch, fake_materialize) - PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( - filemanager, _plan(output_plan), context - ) + PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs(filemanager, _plan(output_plan), context) assert materialized == [] @@ -3320,9 +3197,7 @@ def fake_materialize(spec, data, path, *_args, **_kwargs): _observe_materialization(monkeypatch, fake_materialize) - PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( - filemanager, _plan(output_plan), context - ) + PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs(filemanager, _plan(output_plan), context) spec, data, path = materialized[0] assert isinstance(spec.outputs[0], ROIOptions) @@ -3370,11 +3245,7 @@ def fake_materialize( _observe_materialization(monkeypatch, fake_materialize) - StreamingOnlyArtifactMaterializationTargetPlan().materialize_outputs( - filemanager, - _plan(output_plan, streaming_configs={"napari_stream": streaming_config}), - context, - ) + StreamingOnlyArtifactMaterializationTargetPlan().materialize_outputs(filemanager, _plan(output_plan, streaming_configs={"napari_stream": streaming_config}), context) spec, data, path, backends, backend_kwargs = materialized[0] assert isinstance(spec.outputs[0], ROIOptions) @@ -3454,15 +3325,11 @@ def fake_materialize( _observe_materialization(monkeypatch, fake_materialize) - PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( - filemanager, - _plan( + PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs(filemanager, _plan( output_plan, streaming_configs={"napari_stream": streaming_config_stub()}, compiled_function_pattern=_main_flow_output_compiled_pattern(output_plan), - ), - context, - ) + ), context) assert len(materialized) == 1 backends, backend_kwargs = materialized[0] @@ -3514,14 +3381,10 @@ def fake_materialize( _observe_materialization(monkeypatch, fake_materialize) - StreamingOnlyArtifactMaterializationTargetPlan().materialize_outputs( - filemanager, - _plan( + StreamingOnlyArtifactMaterializationTargetPlan().materialize_outputs(filemanager, _plan( output_plan, streaming_configs={"napari_stream": streaming_config}, - ), - context, - ) + ), context) _spec, _data, path, _backends, backend_kwargs = materialized[0] assert path == "/analysis/A01_s002_w3_z001_t001_labels_step7.roi.zip" @@ -3595,9 +3458,7 @@ def fake_materialize( _observe_materialization(monkeypatch, fake_materialize) - StreamingOnlyArtifactMaterializationTargetPlan().materialize_outputs( - filemanager, plan, context - ) + StreamingOnlyArtifactMaterializationTargetPlan().materialize_outputs(filemanager, plan, context) _spec, _data, path, _backends, backend_kwargs = materialized[0] assert path == "/analysis/01_POS002_D_Nuclei_step7.roi.zip" @@ -3650,15 +3511,11 @@ def fake_materialize(_spec, data, path, *_args, **_kwargs): _observe_materialization(monkeypatch, fake_materialize) - PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( - filemanager, - _plan( + PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs(filemanager, _plan( output_plan, group_by_value="timepoint", variable_components=(Microscopy.Timepoint,), - ), - context, - ) + ), context) assert materialized == [ ( @@ -3667,7 +3524,9 @@ def fake_materialize(_spec, data, path, *_args, **_kwargs): ) ] assert materialized[0][0] is runtime_value.data - assert materialized[0][0].metadata.source_image_names == ("AdjacentImage",) + assert materialized[0][0].metadata.source_image_names == ( + "AdjacentImage", + ) def test_materialize_artifact_outputs_merges_parser_axes_into_source_metadata( @@ -3714,11 +3573,7 @@ def fake_materialize( _observe_materialization(monkeypatch, fake_materialize) - StreamingOnlyArtifactMaterializationTargetPlan().materialize_outputs( - filemanager, - _plan(output_plan, streaming_configs={"napari_stream": streaming_config}), - context, - ) + StreamingOnlyArtifactMaterializationTargetPlan().materialize_outputs(filemanager, _plan(output_plan, streaming_configs={"napari_stream": streaming_config}), context) _spec, _data, path, _backends, backend_kwargs = materialized[0] assert path == "/analysis/A01_s002_w3_z001_t001_Nuclei_step7.roi.zip" @@ -3810,15 +3665,11 @@ def fake_materialize( _observe_materialization(monkeypatch, fake_materialize) - StreamingOnlyArtifactMaterializationTargetPlan().materialize_outputs( - filemanager, - _plan( + StreamingOnlyArtifactMaterializationTargetPlan().materialize_outputs(filemanager, _plan( output_plan, streaming_configs={"napari_stream": streaming_config}, variable_components=(Microscopy.ZIndex,), - ), - context, - ) + ), context) _spec, _data, path, _backends, backend_kwargs = materialized[0] assert path == "/analysis/A01_Nuclei_step7.roi.zip" @@ -3878,15 +3729,11 @@ def test_materialize_artifact_outputs_streams_singleton_roi_plane_from_output_pl backend="memory", ) - StreamingOnlyArtifactMaterializationTargetPlan().materialize_outputs( - filemanager, - _plan( + StreamingOnlyArtifactMaterializationTargetPlan().materialize_outputs(filemanager, _plan( output_plan, streaming_configs={"napari_stream": streaming_config_stub()}, variable_components=(Microscopy.ZIndex,), - ), - context, - ) + ), context) roi_saves = [item for item in filemanager.saved if item[1].endswith(".roi.zip")] assert len(roi_saves) == 1 @@ -3976,15 +3823,11 @@ def fake_materialize( _observe_materialization(monkeypatch, fake_materialize) - StreamingOnlyArtifactMaterializationTargetPlan().materialize_outputs( - filemanager, - _plan( + StreamingOnlyArtifactMaterializationTargetPlan().materialize_outputs(filemanager, _plan( output_plan, streaming_configs={"napari_stream": streaming_config}, variable_components=(Microscopy.Channel,), - ), - context, - ) + ), context) _spec, _data, path, _backends, backend_kwargs = materialized[0] assert path == "/analysis/A01_segmentation_masks_step7.roi.zip" @@ -4024,11 +3867,8 @@ def test_materialize_rgb_artifact_streams_filename_channel_identity( # contributors alone do not establish a scalar channel on the result. source_path="/input/A01_s001_w3_z001_t001.TIF", source_component_metadata={ - "well": "A01", - "site": "1", - "channel": "3", - "z_index": "1", - "timepoint": "1", + "well": "A01", "site": "1", "channel": "3", + "z_index": "1", "timepoint": "1", }, source_image_provenance_planes=SourceImageProvenancePlanes.from_components( paths=( @@ -4084,15 +3924,11 @@ def fake_materialize( _observe_materialization(monkeypatch, fake_materialize) - StreamingOnlyArtifactMaterializationTargetPlan().materialize_outputs( - filemanager, - _plan( + StreamingOnlyArtifactMaterializationTargetPlan().materialize_outputs(filemanager, _plan( output_plan, streaming_configs={"napari_stream": streaming_config}, variable_components=(Microscopy.Channel,), - ), - context, - ) + ), context) _spec, _data, path, _backends, backend_kwargs = materialized[0] assert path == "/images/A01_s001_w3_z001_t001.TIF" @@ -4165,11 +4001,7 @@ def fake_materialize(_spec, _data, path, *_args, **_kwargs): _observe_materialization(monkeypatch, fake_materialize) - PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( - filemanager, - _plan(output_plan, variable_components=(Microscopy.Channel,)), - context, - ) + PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs(filemanager, _plan(output_plan, variable_components=(Microscopy.Channel,)), context) assert materialized == ["/images/A01_s001_w1_z001_t001.TIF"] @@ -4253,28 +4085,18 @@ def test_compiled_z_axis_reaches_source_named_image_materialization() -> None: source_image_names=(selected_image.name,), source_path="/derived/A01_s001_w2_z001_t001.tif", source_component_metadata={ - "well": "A01", - "site": "1", - "channel": "2", + "well": "A01", "site": "1", "channel": "2", "timepoint": "1", }, source_image_provenance_planes=SourceImageProvenancePlanes.from_components( paths=tuple(f"/derived/A01_s001_w2_z00{z}_t001.tif" for z in (1, 2)), component_metadata=tuple( - { - "well": "A01", - "site": "1", - "channel": "2", - "z_index": str(z), - "timepoint": "1", - } - for z in (1, 2) + {"well": "A01", "site": "1", "channel": "2", + "z_index": str(z), "timepoint": "1"} for z in (1, 2) ), ), ).payload_with(np.ones((2, 5, 7), dtype=np.uint16), None) - saved_payload = ( - ImageArtifactType if output_plan is None else output_plan.artifact_type - ).contextualize_output_from_projector( + saved_payload = (ImageArtifactType if output_plan is None else output_plan.artifact_type).contextualize_output_from_projector( selected_payload, ImagePayloadMetadata( plane_axis=RuntimePlaneAxis.RUNTIME_SLICE, @@ -4337,7 +4159,9 @@ def test_compiled_z_axis_reaches_source_named_image_materialization() -> None: persistent_backend="disk", ) - assert saved_payload.metadata.plane_axis is (RuntimePlaneAxis.RUNTIME_SLICE) + assert saved_payload.metadata.plane_axis is ( + RuntimePlaneAxis.RUNTIME_SLICE + ) assert tuple( Path(output.path) for materialization in runtime_artifact_materializations(plan, context) @@ -4415,15 +4239,11 @@ def fake_materialize( _observe_materialization(monkeypatch, fake_materialize) - StreamingOnlyArtifactMaterializationTargetPlan().materialize_outputs( - filemanager, - _plan( + StreamingOnlyArtifactMaterializationTargetPlan().materialize_outputs(filemanager, _plan( output_plan, streaming_configs={"napari_stream": streaming_config}, variable_components=(Microscopy.Site,), - ), - context, - ) + ), context) _spec, _data, path, _backends, backend_kwargs = materialized[0] assert path == "/images/A01_s001_w2_z001_t001.TIF" diff --git a/tests/unit/test_function_artifact_outputs.py b/tests/unit/test_function_artifact_outputs.py index 71dbac8717..949a45933c 100644 --- a/tests/unit/test_function_artifact_outputs.py +++ b/tests/unit/test_function_artifact_outputs.py @@ -299,7 +299,9 @@ def assemble(image, **kwargs): assert result.data.shape == (4, 8) assert result.data.dtype == tiles.dtype - assert result.metadata.source_path == ("/input/A01_s001_w1_z001_t001.tif") + assert result.metadata.source_path == ( + "/input/A01_s001_w1_z001_t001.tif" + ) def test_function_core_preserves_output_declared_runtime_slice_axis(): @@ -343,9 +345,7 @@ def test_image_output_context_preserves_stack_after_exact_source_projection(): relations=(GroupLineageSourceRelation(source_spec.ref()),), ) - result = ( - ImageArtifactType if output_plan is None else output_plan.artifact_type - ).contextualize_output( + result = (ImageArtifactType if output_plan is None else output_plan.artifact_type).contextualize_output( source, output, output_plan, @@ -395,9 +395,7 @@ def test_image_output_context_does_not_infer_axis_for_unmarked_payload(): relations=(GroupLineageSourceRelation(source_spec.ref()),), ) - result = ( - ImageArtifactType if output_plan is None else output_plan.artifact_type - ).contextualize_output_from_projector( + result = (ImageArtifactType if output_plan is None else output_plan.artifact_type).contextualize_output_from_projector( source, np.ones((2, 4, 5), dtype=np.uint16), output_plan, @@ -429,9 +427,7 @@ def test_image_output_context_rejects_declared_output_axis_shape_drift(): ValueError, match="declared 'runtime_slice' axis of size 2", ): - ( - ImageArtifactType if output_plan is None else output_plan.artifact_type - ).contextualize_output( + (ImageArtifactType if output_plan is None else output_plan.artifact_type).contextualize_output( np.ones((1, 4, 5), dtype=np.float32), output, output_plan, @@ -456,9 +452,7 @@ def test_image_output_context_preserves_complete_scalar_rgb_identity() -> None: source_image_names=(source_spec.name,), axes=PayloadAxes.colour_samples(2), ).payload_with(np.ones((4, 5, 3), dtype=np.float32), None) - output = source.with_pixels( - np.ones((4, 5, 3), dtype=np.uint8), - ) + output = source.with_pixels(np.ones((4, 5, 3), dtype=np.uint8),) output_plan = ArtifactOutputPlan( name="SavedColorNeighbors", path="/memory/SavedColorNeighbors.png", @@ -467,9 +461,7 @@ def test_image_output_context_preserves_complete_scalar_rgb_identity() -> None: relations=(GroupLineageSourceRelation(source_spec.ref()),), ) - result = ( - ImageArtifactType if output_plan is None else output_plan.artifact_type - ).contextualize_output_from_projector( + result = (ImageArtifactType if output_plan is None else output_plan.artifact_type).contextualize_output_from_projector( source, output, output_plan, @@ -555,9 +547,7 @@ def test_image_output_context_projects_complete_multi_plane_identity() -> None: relations=(GroupLineageSourceRelation(source_spec.ref()),), ) - result = ( - ImageArtifactType if output_plan is None else output_plan.artifact_type - ).contextualize_output_from_projector( + result = (ImageArtifactType if output_plan is None else output_plan.artifact_type).contextualize_output_from_projector( source, output, output_plan, @@ -683,7 +673,9 @@ def compose_channels(_image): def test_crop_mask_sidecar_names_derive_from_core_artifact_role(): - assert CropMask.name_for("CroppedImage") == ("CroppedImage__crop_mask") + assert CropMask.name_for("CroppedImage") == ( + "CroppedImage__crop_mask" + ) def test_unstack_payload_context_slices_volume_stack_mask_with_volume_data(): @@ -779,10 +771,10 @@ def test_unstack_payload_context_preserves_declared_axis_over_default(): default_plane_axis=RuntimePlaneAxis.RUNTIME_SLICE, ) - assert [slice_payload.metadata.source_image_names for slice_payload in slices] == [ - ("DNA",), - ("Actin",), - ] + assert [ + slice_payload.metadata.source_image_names + for slice_payload in slices + ] == [("DNA",), ("Actin",)] def test_stack_payload_context_projects_single_payload_mask_to_stack_domain(): @@ -1393,7 +1385,9 @@ def produce_corrected_image(image): stored = context.runtime_value_store.find(name="CorrectedImage", axis_id="A01") assert len(stored) == 1 np.testing.assert_array_equal(stored[0].data.data, output) - assert stored[0].data.metadata.source_image_names == ("CorrectedImage",) + assert stored[0].data.metadata.source_image_names == ( + "CorrectedImage", + ) def test_saved_canonical_source_projection_survives_main_flow_return(): @@ -1407,37 +1401,22 @@ def test_saved_canonical_source_projection_survives_main_flow_return(): source_image_provenance_planes=SourceImageProvenancePlanes.from_components( paths=paths, component_metadata=tuple( - { - "well": "A01", - "site": site, - "channel": "2", - "z_index": "1", - "timepoint": "1", - } + {"well": "A01", "site": site, "channel": "2", "z_index": "1", "timepoint": "1"} for site in ("1", "2") ), ), ).payload_with(pixels) output = ArtifactSpec.output("Selected", ImageArtifactType) - plan = ArtifactOutputPlan( - name=output.name, path="/memory/selected.pkl", artifact_type=ImageArtifactType - ) + plan = ArtifactOutputPlan(name=output.name, path="/memory/selected.pkl", artifact_type=ImageArtifactType) @artifact_outputs(output) def select_second_plane(image): return SelectedPlaneImageOutput(image[1:2], (1,)) - result = _execute_function_core( - CoreExecutionRequest( - func_callable=select_second_plane, - main_data_arg=source, - base_kwargs={}, - context=context, - artifact_inputs={}, - artifact_outputs={plan.ref(): plan}, - runtime_plane_count=2, - ) - ) + result = _execute_function_core(CoreExecutionRequest( + func_callable=select_second_plane, main_data_arg=source, base_kwargs={}, context=context, + artifact_inputs={}, artifact_outputs={plan.ref(): plan}, runtime_plane_count=2, + )) stored = context.runtime_value_store.find(name=output.name, axis_id="A01") assert len(stored) == 1 for payload in (result, stored[0].data): @@ -1451,9 +1430,7 @@ def select_second_plane(image): @pytest.mark.parametrize("declared_canonical", [False, True]) -def test_unsaved_main_return_reads_source_context_after_artifact_save( - declared_canonical, -): +def test_unsaved_main_return_reads_source_context_after_artifact_save(declared_canonical): context = ContextStub() source = ImagePayloadMetadata( source_path="/source/before.tif", @@ -1461,21 +1438,13 @@ def test_unsaved_main_return_reads_source_context_after_artifact_save( ).payload_with(np.zeros((2, 3), dtype=np.float32)) sidecar = ArtifactSpec.output("Sidecar", MetadataArtifactType) canonical = ArtifactSpec.output("Unselected", ImageArtifactType) - plan = ArtifactOutputPlan( - name=sidecar.name, - path="/memory/sidecar.pkl", - artifact_type=MetadataArtifactType, - ) + plan = ArtifactOutputPlan(name=sidecar.name, path="/memory/sidecar.pkl", artifact_type=MetadataArtifactType) original_save = context.filemanager.save def save_and_change_source(value, path, backend): original_save(value, path, backend) source.metadata.source_path = "/source/after.tif" - source.metadata.source_component_metadata = { - "well": "A01", - "site": "9", - "channel": "2", - } + source.metadata.source_component_metadata = {"well": "A01", "site": "9", "channel": "2"} context.filemanager.save = save_and_change_source @@ -1483,16 +1452,10 @@ def save_and_change_source(value, path, backend): def produce(image): return image + 1, {"note": "saved"} - result = _execute_function_core( - CoreExecutionRequest( - func_callable=produce, - main_data_arg=source, - base_kwargs={}, - context=context, - artifact_inputs={}, - artifact_outputs={plan.ref(): plan}, - ) - ) + result = _execute_function_core(CoreExecutionRequest( + func_callable=produce, main_data_arg=source, base_kwargs={}, context=context, + artifact_inputs={}, artifact_outputs={plan.ref(): plan}, + )) np.testing.assert_array_equal(result.data, np.ones((2, 3))) assert result.metadata.source_path == "/source/after.tif" assert result.metadata.source_component_metadata["site"] == "9" @@ -1558,7 +1521,9 @@ def produce_image(image): output_payload.data, output_slices[index], ) - assert metadata.source_path == source_payload.metadata.source_path + assert ( + metadata.source_path == source_payload.metadata.source_path + ) assert metadata.source_image_names == (output_spec.name,) assert metadata.source_provenance.represented_source_image_names == ( output_spec.name, @@ -1810,10 +1775,8 @@ def rgb_output(image): ) from openhcs.core.aligned_image_payload import ImagePayloadStackComposition - stack_payload = (scalar_output).copied( - memory_type=MEMORY_TYPE_NUMPY, - device_id=None, - ) + stack_payload = (scalar_output).copied(memory_type=MEMORY_TYPE_NUMPY, + device_id=None,) assert stack_payload.data.shape == (2, 3, 3) stack_metadata = stack_payload.metadata assert stack_metadata.plane_axis is None @@ -1848,7 +1811,10 @@ def test_pattern_group_runtime_uses_declared_output_slice_cardinality(): ) assert len(output) == 1 assert output[0][0].data.shape == (4, 5) - assert declared_output.metadata.plane_axis is RuntimePlaneAxis.RUNTIME_SLICE + assert ( + declared_output.metadata.plane_axis + is RuntimePlaneAxis.RUNTIME_SLICE + ) def test_pattern_group_runtime_projects_nominal_object_label_stack(): @@ -2345,7 +2311,9 @@ def segment(image): assert label_set.declared_plane_count() == 2 np.testing.assert_array_equal( label_set.labels, - np.stack(tuple(labels.value_at(index) for index in range(labels.slice_count))), + np.stack( + tuple(labels.value_at(index) for index in range(labels.slice_count)) + ), ) assert label_set.source_provenance.source_image_provenance_planes.paths == tuple( item.metadata.source_path for item in source_slices @@ -2370,15 +2338,11 @@ def test_measurement_subject_is_required_before_schema_rows_execute(boundary): def count(image): raise AssertionError("Invalid measurement declarations must not execute") - with pytest.raises( - ValueError, match="cell_counts.*no declared measurement subject" - ) as exc: + with pytest.raises(ValueError, match="cell_counts.*no declared measurement subject") as exc: if boundary == "contract": from openhcs.core.callable_contract import CallableContract - CallableContract.from_callable( - count - ).validate_artifact_output_declarations() + CallableContract.from_callable(count).validate_artifact_output_declarations() else: compile_function_pattern(count, {}, {}) assert "ImageMeasurementSubjectRelation" in str(exc.value) @@ -2961,9 +2925,7 @@ def declared_source_origin(image, *, image_to_save=None): value=None, ), execution_plan=CompiledStepPlan( - step_index=0, - step_name="source-bound", - axis_id="A01", + step_index=0, step_name="source-bound", axis_id="A01", variable_components=(), ), ), @@ -3173,6 +3135,5 @@ def test_declared_source_payload_rejects_ambiguous_stored_and_source_bound_origi np.zeros((1, 2, 3), dtype=np.uint16) ) with pytest.raises(ValueError, match="must resolve to exactly one .* resolved 2"): - executor.declared_source_payload( - spec.ref(), primary, loaded_artifact_payloads={spec.ref(): primary} - ) + executor.declared_source_payload(spec.ref(), primary, + loaded_artifact_payloads={spec.ref(): primary}) diff --git a/tests/unit/test_function_execution_lifecycle_anchors.py b/tests/unit/test_function_execution_lifecycle_anchors.py index d09ec72dad..00134ac514 100644 --- a/tests/unit/test_function_execution_lifecycle_anchors.py +++ b/tests/unit/test_function_execution_lifecycle_anchors.py @@ -50,8 +50,7 @@ def _anchor_executor( ) executor.context.runtime_source_workspace_projections = ( WorkspaceSourceProjections.from_context( - executor.context, - cache=source_workspace_projection_cache, + executor.context, cache=source_workspace_projection_cache, ) ) if output_manifest is not None: diff --git a/tests/unit/test_function_io.py b/tests/unit/test_function_io.py index 794e11c239..79e391490d 100644 --- a/tests/unit/test_function_io.py +++ b/tests/unit/test_function_io.py @@ -307,8 +307,7 @@ def test_bulk_preload_preserves_nested_virtual_workspace_paths(tmp_path: Path) - assert virtual_paths == expected_paths SourceFileUniverse( - tuple(expected_paths), - Backend.VIRTUAL_WORKSPACE, + tuple(expected_paths), Backend.VIRTUAL_WORKSPACE, ).load_images(filemanager) loaded = filemanager.load_batch(expected_paths, Backend.MEMORY.value) @@ -527,9 +526,7 @@ def test_bulk_preload_loads_matlab_pixels_through_declared_vfs_backend( str(workspace_root / virtual_path) for virtual_path in materialization.plane_mappings ) - SourceFileUniverse(virtual_paths, Backend.VIRTUAL_WORKSPACE).load_images( - filemanager - ) + SourceFileUniverse(virtual_paths, Backend.VIRTUAL_WORKSPACE).load_images(filemanager) loaded = filemanager.load_batch( list(virtual_paths), diff --git a/tests/unit/test_function_outputs.py b/tests/unit/test_function_outputs.py index 6a5cd3782f..d14c6c2f0d 100644 --- a/tests/unit/test_function_outputs.py +++ b/tests/unit/test_function_outputs.py @@ -75,9 +75,7 @@ MaterializedRuntimeArtifact, RuntimeArtifactMaterialization, ) -from openhcs.core.steps.function_artifact_materialization import ( - ArtifactMaterializationTargetPlan, -) +from openhcs.core.steps.function_artifact_materialization import ArtifactMaterializationTargetPlan from openhcs.core.steps.function_outputs import ( MaterializedImageOutputWriter, MemoryOutputWriter, @@ -125,8 +123,7 @@ def _publish_saved_step(context, plan, *, artifact_materializations=()): METADATA_CONFIG.metadata_path(target.dataset_root) for target in facts ): entries = { - target: value - for target, value in facts.items() + target: value for target, value in facts.items() if METADATA_CONFIG.metadata_path(target.dataset_root) == metadata_path } AtomicMetadataWriter().reconcile_completed_dataset( @@ -177,8 +174,7 @@ def test_function_step_metadata_follows_runtime_artifact_persistence( authority, method_name, lambda _context, _plan, event=event, **_kwargs: ( - events.append(event), - {} if event == "metadata" else (), + events.append(event), {} if event == "metadata" else () )[1], ) @@ -309,10 +305,7 @@ def parse_filename(self, name): } ) return FilenameParseResult( - ( - (component, metadata.get(component.name)) - for component in AxisFamily.active().axes - ), + ((component, metadata.get(component.name)) for component in AxisFamily.active().axes), extension=str(metadata["extension"]), ) @@ -517,9 +510,7 @@ def test_memory_output_writer_materializes_preserved_source_at_current_destinati @pytest.mark.parametrize("source_extension", (None, ".czi")) -def test_function_output_identity_preserves_non_axis_source_metadata( - source_extension, -) -> None: +def test_function_output_identity_preserves_non_axis_source_metadata(source_extension) -> None: identity = FunctionOutputIdentity( component_values={"channel": "2", "site": "1"}, extension=".tif", @@ -527,10 +518,10 @@ def test_function_output_identity_preserves_non_axis_source_metadata( ) source_metadata = { - "Run": "Sequence1", - "Specimen": "DrosophilaEmbryo", - "ChannelNumber": "1", - "OpenHCSOriginalSourceMetadata": {"FrameNumber": "0007"}, + "Run": "Sequence1", + "Specimen": "DrosophilaEmbryo", + "ChannelNumber": "1", + "OpenHCSOriginalSourceMetadata": {"FrameNumber": "0007"}, } if source_extension is not None: source_metadata["extension"] = source_extension @@ -545,9 +536,7 @@ def test_function_output_identity_preserves_non_axis_source_metadata( **({"extension": source_extension} if source_extension is not None else {}), } assert identity.filename_component_metadata() == { - "channel": "2", - "site": "1", - "extension": ".tif", + "channel": "2", "site": "1", "extension": ".tif", } assert source_metadata["ChannelNumber"] == "1" assert source_metadata.get("extension") == source_extension @@ -1676,9 +1665,7 @@ def test_produced_projection_metadata_persists_typed_collapsed_semantics( assert len(restored.source_provenance.represented_source_identities) == 2 assert record["address"] is None assert record["execution_scope"]["fixed_component_values"] == [ - ["channel", "1"], - ["z_index", "1"], - ["timepoint", "1"], + ["channel", "1"], ["z_index", "1"], ["timepoint", "1"], ] assert "site" not in record["source_metadata"] assert "site" not in subdirectory[FIELDS.SOURCE_METADATA]["images/A01_s1_w1.tif"] @@ -1957,10 +1944,7 @@ def parse_filename(self, name): } ) return FilenameParseResult( - ( - (component, metadata.get(component.name)) - for component in AxisFamily.active().axes - ), + ((component, metadata.get(component.name)) for component in AxisFamily.active().axes), extension=str(metadata["extension"]), ) @@ -2100,7 +2084,9 @@ def test_produced_address_publication_never_parses_generated_filenames( }, source_voxel_spacing=SourceVoxelSpacing((0.5, 0.5)), ) - identity = FunctionOutputIdentity.from_metadata(parser, metadata) + identity = FunctionOutputIdentity.from_metadata( + parser, metadata + ) assert identity is not None assert identity.extension == extension identity = identity.with_filename_qualifier("centre_dots") @@ -2272,15 +2258,11 @@ def test_completed_plate_metadata_includes_outputs_written_after_owner_axis( follower_plan.create_openhcs_metadata = True filemanager.save(first_pixels, str(later_image), Backend.MEMORY.value) record_output_path(follower_context, follower_plan, str(later_image)) - later_facts = OpenHCSMetadataTarget.observe_for_step( - follower_context, follower_plan - ) + later_facts = OpenHCSMetadataTarget.observe_for_step(follower_context, follower_plan) combined_facts = { target: VirtualWorkspaceSourceProjectionEntries.combine( - ( - first_facts.get(target, VirtualWorkspaceSourceProjectionEntries({})), - later_facts.get(target, VirtualWorkspaceSourceProjectionEntries({})), - ) + (first_facts.get(target, VirtualWorkspaceSourceProjectionEntries({})), + later_facts.get(target, VirtualWorkspaceSourceProjectionEntries({}))) ) for target in dict.fromkeys((*first_facts, *later_facts)) } @@ -2407,9 +2389,7 @@ def test_runtime_image_metadata_target_requires_persisted_images(tmp_path, conte context, plan, artifact_materializations=saved_artifacts ) if contents == "images": - assert selected == ( - replace(target, create_openhcs_metadata=plan.create_openhcs_metadata), - ) + assert selected == (replace(target, create_openhcs_metadata=plan.create_openhcs_metadata),) else: assert selected == () plan.create_openhcs_metadata = True @@ -2479,8 +2459,12 @@ def test_declared_image_destinations_publish_and_reconcile_after_value_cleanup( path=output_plan.path, backend=Backend.MEMORY.value, ) - materializations = ArtifactMaterializationTargetPlan.materialize(context, plan) - _publish_saved_step(context, plan, artifact_materializations=materializations) + materializations = ArtifactMaterializationTargetPlan.materialize( + context, plan + ) + _publish_saved_step( + context, plan, artifact_materializations=materializations + ) context.runtime_value_store.clear() if stray_image: tifffile.imwrite(tmp_path / "images/review/unowned.tif", pixels) @@ -2572,7 +2556,9 @@ def test_actual_array_exports_only_publish_declared_raster_inventory( backend=Backend.MEMORY.value, ) - materializations = ArtifactMaterializationTargetPlan.materialize(context, plan) + materializations = ArtifactMaterializationTargetPlan.materialize( + context, plan + ) saved_outputs = tuple( output for artifact in materializations @@ -2590,15 +2576,15 @@ def test_actual_array_exports_only_publish_declared_raster_inventory( location for artifact in materializations for locations in ( - artifact.observation( - plan, context - ).materialized_locations_by_address.values() + artifact.observation(plan, context).materialized_locations_by_address.values() ) for location in locations ) assert {Path(location.path) for location in observed_locations} == expected_paths - _publish_saved_step(context, plan, artifact_materializations=materializations) + _publish_saved_step( + context, plan, artifact_materializations=materializations + ) context.runtime_value_store.clear() OpenHCSMetadataTarget.finalize_completed_dataset({"A01": context}) metadata_path = tmp_path / "openhcs_metadata.json" @@ -2637,13 +2623,8 @@ def test_actual_array_exports_only_publish_declared_raster_inventory( def _stream_batch_sources(projections): plan = function_step_plan("StreamOwned") - arrays = [ - np.full((2, 3), index, dtype=np.float32) for index in range(len(projections)) - ] - paths = [ - str(plan.output_dir / f"A01_s1_w{index}.tif") - for index in range(len(projections)) - ] + arrays = [np.full((2, 3), index, dtype=np.float32) for index in range(len(projections))] + paths = [str(plan.output_dir / f"A01_s1_w{index}.tif") for index in range(len(projections))] records = tuple( ProducedOutputSemantics.from_output( plan, @@ -2654,8 +2635,7 @@ def _stream_batch_sources(projections): source="stream source", ), output_context=AlignedImageSliceContext.main_flow( - output_key=f"image-{index}", - projection_key=projection, + output_key=f"image-{index}", projection_key=projection, artifact_kind=ImageArtifactType.value, ), ) @@ -2664,12 +2644,8 @@ def _stream_batch_sources(projections): return arrays, paths, records -def test_stream_batch_owns_correlated_projection_order_and_frozen_input_lists( - monkeypatch, -): - arrays, paths, records = _stream_batch_sources( - ("second", "first", "second", "first") - ) +def test_stream_batch_owns_correlated_projection_order_and_frozen_input_lists(monkeypatch): + arrays, paths, records = _stream_batch_sources(("second", "first", "second", "first")) expected_arrays, expected_paths = tuple(arrays), tuple(paths) observed = [] original_project = StreamOutputBatch.project_item @@ -2682,9 +2658,7 @@ def project(request): monkeypatch.setattr(StreamOutputBatch, "project_item", staticmethod(project)) batches = StreamOutputBatch.from_projection_groups( - parser=ImageXpressFilenameParser(), - payloads=arrays, - paths=paths, + parser=ImageXpressFilenameParser(), payloads=arrays, paths=paths, produced_outputs=records, ) assert observed == [expected_paths[index] for index in (0, 2, 1, 3)] @@ -2694,20 +2668,14 @@ def project(request): records[index].producer_identity for index in (0, 2, 1, 3) ] for item, index in zip( - (item for batch in batches for item in batch.items), - (0, 2, 1, 3), - strict=True, + (item for batch in batches for item in batch.items), (0, 2, 1, 3), strict=True, ): assert item.data is expected_arrays[index] assert item.source_component_metadata["channel"] == str(index) -def test_stream_batch_keeps_first_projection_error_and_prior_callback_effects( - monkeypatch, -): - arrays, paths, records = _stream_batch_sources( - ("first", "second", "first", "second") - ) +def test_stream_batch_keeps_first_projection_error_and_prior_callback_effects(monkeypatch): + arrays, paths, records = _stream_batch_sources(("first", "second", "first", "second")) observed = [] failure = ValueError("actual projector failure") original_project = StreamOutputBatch.project_item @@ -2721,9 +2689,7 @@ def project(request): monkeypatch.setattr(StreamOutputBatch, "project_item", staticmethod(project)) with pytest.raises(ValueError) as raised: StreamOutputBatch.from_projection_groups( - parser=ImageXpressFilenameParser(), - payloads=arrays, - paths=paths, + parser=ImageXpressFilenameParser(), payloads=arrays, paths=paths, produced_outputs=records, ) assert raised.value is failure @@ -2736,37 +2702,23 @@ def test_stream_batch_validates_cardinality_and_routes_before_projecting(monkeyp def forbidden_project(request): raise AssertionError("invalid inputs reached projection") - monkeypatch.setattr( - StreamOutputBatch, "project_item", staticmethod(forbidden_project) - ) + monkeypatch.setattr(StreamOutputBatch, "project_item", staticmethod(forbidden_project)) parser = ImageXpressFilenameParser() with pytest.raises(ValueError, match="payload/path cardinality mismatch"): StreamOutputBatch.from_projection_groups( - parser=parser, - payloads=arrays, - paths=paths[:1], - produced_outputs=(), + parser=parser, payloads=arrays, paths=paths[:1], produced_outputs=(), ) with pytest.raises(ValueError, match="payload/output-record cardinality mismatch"): StreamOutputBatch.from_projection_groups( - parser=parser, - payloads=arrays, - paths=paths, - produced_outputs=records[:1], + parser=parser, payloads=arrays, paths=paths, produced_outputs=records[:1], ) with pytest.raises(ValueError, match="at least one produced output record"): StreamOutputBatch.from_projection_groups( - parser=parser, - payloads=(), - paths=(), - produced_outputs=(), + parser=parser, payloads=(), paths=(), produced_outputs=(), ) with pytest.raises(ValueError, match="cannot mix producer projections"): StreamOutputBatch.from_projection( - parser=parser, - payloads=arrays, - paths=paths, - produced_outputs=records, + parser=parser, payloads=arrays, paths=paths, produced_outputs=records, ) @@ -2828,10 +2780,7 @@ def test_whole_volume_checkpoint_publication_retains_exact_producer_scope( ) assert str(contextualized.source_component_metadata["timepoint"]) == "1" assert "z_index" not in contextualized.source_component_metadata - assert ( - contextualized.source_image_provenance_planes - == metadata.source_image_provenance_planes - ) + assert contextualized.source_image_provenance_planes == metadata.source_image_provenance_planes manager = FileManager( {"disk": DiskStorageBackend(), "memory": MemoryStorageBackend()} ) @@ -2901,7 +2850,9 @@ def test_whole_volume_checkpoint_publication_retains_exact_producer_scope( path=output_plan.path, backend=Backend.MEMORY.value, ) - materializations = ArtifactMaterializationTargetPlan.materialize(context, plan) + materializations = ArtifactMaterializationTargetPlan.materialize( + context, plan + ) assert { Path(output.path) for artifact in materializations @@ -2915,14 +2866,19 @@ def test_whole_volume_checkpoint_publication_retains_exact_producer_scope( assert projection.address is None assert projection.source_alias == "DNA" assert ( - projection.execution_scope.value_text_for_component(Microscopy.Channel) == "2" + projection.execution_scope.value_text_for_component(Microscopy.Channel) + == "2" + ) + assert ( + projection.execution_scope.value_text_for_component(Microscopy.Site) == "1" ) - assert projection.execution_scope.value_text_for_component(Microscopy.Site) == "1" assert ( - projection.execution_scope.value_text_for_component(Microscopy.Timepoint) == "1" + projection.execution_scope.value_text_for_component(Microscopy.Timepoint) + == "1" ) assert ( - projection.execution_scope.value_text_for_component(Microscopy.ZIndex) is None + projection.execution_scope.value_text_for_component(Microscopy.ZIndex) + is None ) assert ( projection.image_metadata.source_image_provenance_planes.component_metadata diff --git a/tests/unit/test_function_reference_identity.py b/tests/unit/test_function_reference_identity.py index e7af6b453b..fbf967e1e4 100644 --- a/tests/unit/test_function_reference_identity.py +++ b/tests/unit/test_function_reference_identity.py @@ -99,7 +99,8 @@ def test_cold_external_resolution_retains_registry_classified_contract( resolved = reference.resolve() assert ( - CallableContract.from_callable(resolved).processing_contract is FlexibleContract + CallableContract.from_callable(resolved).processing_contract + is FlexibleContract ) @@ -142,14 +143,11 @@ def test_module_export_source_keeps_exact_registry_transport_identity(monkeypatc metadata = RegistryService.declared_metadata_for_callable( cellprofiler_backend.crop )[1] - monkeypatch.setattr( - RegistryService, "_metadata_cache", {metadata.composite_key: metadata} - ) + monkeypatch.setattr(RegistryService, "_metadata_cache", {metadata.composite_key: metadata}) monkeypatch.setattr(RegistryService, "_resolved_reference_callables", {}) reference = FunctionReferenceTransportAuthority.function_reference(metadata.func) exported = FunctionReferenceTransportAuthority.importable_function( - metadata.import_identity.module_name, - metadata.import_identity.function_name, + metadata.import_identity.module_name, metadata.import_identity.function_name, ) if exported is metadata.func: assert isinstance(reference, ModuleExportRegistryFunctionReference) diff --git a/tests/unit/test_function_runtime_source_projection.py b/tests/unit/test_function_runtime_source_projection.py index 3e52a96930..af776f9a81 100644 --- a/tests/unit/test_function_runtime_source_projection.py +++ b/tests/unit/test_function_runtime_source_projection.py @@ -143,8 +143,7 @@ def _anchor_executor( ) executor.context.runtime_source_workspace_projections = ( WorkspaceSourceProjections.from_context( - executor.context, - cache=source_workspace_projection_cache, + executor.context, cache=source_workspace_projection_cache, ) ) if output_manifest is not None: @@ -296,15 +295,7 @@ def test_workspace_source_files_select_exact_projection_roles( plate_path = tmp_path / "plate" canonical_path = "A01_s001_w1_z001_t001.tif" artifact_path = f"_source/Illumination/{canonical_path}" - address = OpenHCSPlaneAddress( - ( - (Microscopy.Well, "A01"), - (Microscopy.Site, "1"), - (Microscopy.Channel, "1"), - (Microscopy.ZIndex, "1"), - (Microscopy.Timepoint, "1"), - ) - ) + address = OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, "1"), (Microscopy.Channel, "1"), (Microscopy.ZIndex, "1"), (Microscopy.Timepoint, "1"))) plane_projection = SourcePlaneProjection( address=address, ref=SourcePixelRef("disk", "/source/image.tif"), @@ -343,15 +334,7 @@ def test_workspace_logical_identity_requires_paired_projection_declaration( path = "A01_s001_w1_z001_t001.tif" full_path = str(tmp_path / path) first = SourcePlaneProjection( - address=OpenHCSPlaneAddress( - ( - (Microscopy.Well, "A01"), - (Microscopy.Site, 1), - (Microscopy.Channel, 1), - (Microscopy.ZIndex, 1), - (Microscopy.Timepoint, 1), - ) - ), + address=OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, 1), (Microscopy.Channel, 1), (Microscopy.ZIndex, 1), (Microscopy.Timepoint, 1))), ref=SourcePixelRef("disk", "/physical/shared.tif"), source_alias="Original", ) @@ -367,16 +350,7 @@ def test_workspace_logical_identity_requires_paired_projection_declaration( # Sharing a backend reference does not prove a shared logical declaration. declarations[full_path] = replace( - first, - address=OpenHCSPlaneAddress( - ( - (Microscopy.Well, "B01"), - (Microscopy.Site, 1), - (Microscopy.Channel, 1), - (Microscopy.ZIndex, 1), - (Microscopy.Timepoint, 1), - ) - ), + first, address=OpenHCSPlaneAddress(((Microscopy.Well, "B01"), (Microscopy.Site, 1), (Microscopy.Channel, 1), (Microscopy.ZIndex, 1), (Microscopy.Timepoint, 1))) ) assert projection.logical_path_for(lookup) == full_path assert projection.source_path_for(lookup) == "/physical/shared.tif" @@ -393,15 +367,7 @@ def test_workspace_source_projection_carries_exact_aliases_into_stack_provenance projection_set = SourceProjectionSet( tuple( SourcePlaneProjection( - address=OpenHCSPlaneAddress( - ( - (Microscopy.Well, "A01"), - (Microscopy.Site, "1"), - (Microscopy.Channel, str(channel)), - (Microscopy.ZIndex, "1"), - (Microscopy.Timepoint, "1"), - ) - ), + address=OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, "1"), (Microscopy.Channel, str(channel)), (Microscopy.ZIndex, "1"), (Microscopy.Timepoint, "1"))), ref=SourcePixelRef("disk", str(source_path)), source_alias=alias, ) @@ -458,10 +424,12 @@ def test_workspace_source_projection_carries_exact_aliases_into_stack_provenance for virtual_path in subdirectory["image_files"] ) == ("Raw", "Illum") assert tuple( - payload.metadata.source_image_names for payload in projected_payloads + payload.metadata.source_image_names + for payload in projected_payloads ) == (("Raw",), ("Illum",)) assert all( - "source_alias" not in (payload.metadata.source_component_metadata or {}) + "source_alias" + not in (payload.metadata.source_component_metadata or {}) for payload in projected_payloads ) @@ -500,15 +468,7 @@ def test_workspace_replay_preserves_collapsed_semantic_identity( projection_set = SourceProjectionSet( ( SourcePlaneProjection( - address=OpenHCSPlaneAddress( - ( - (Microscopy.Well, "A14"), - (Microscopy.Site, 1), - (Microscopy.Channel, 1), - (Microscopy.ZIndex, 1), - (Microscopy.Timepoint, 1), - ) - ), + address=OpenHCSPlaneAddress(((Microscopy.Well, "A14"), (Microscopy.Site, 1), (Microscopy.Channel, 1), (Microscopy.ZIndex, 1), (Microscopy.Timepoint, 1))), ref=SourcePixelRef("disk", "/outputs/A14_mosaic.tif"), source_alias="neurite", image_metadata=persisted_metadata, @@ -578,15 +538,7 @@ def test_workspace_source_loading_preserves_declared_tiff_intensity_scale( ) virtual_path = "A01_s001_w1_z001_t001.tif" source_plane = SourcePlaneProjection( - address=OpenHCSPlaneAddress( - ( - (Microscopy.Well, "A01"), - (Microscopy.Site, "1"), - (Microscopy.Channel, "1"), - (Microscopy.ZIndex, "1"), - (Microscopy.Timepoint, "1"), - ) - ), + address=OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, "1"), (Microscopy.Channel, "1"), (Microscopy.ZIndex, "1"), (Microscopy.Timepoint, "1"))), ref=SourcePixelRef("disk", str(source_path)), source_alias="OrigBlue", ) @@ -1114,10 +1066,9 @@ def test_step_output_manifest_pattern_lookup_returns_producer_memory_paths( ) index = store.producer_record_index_for(consumer, SourceSchemaFilenameParser()) - assert [ - record.output_path - for record in index.matching_records("A14_s{iii}_w3_z001_t001.tif") - ] == [ + assert [record.output_path for record in index.matching_records( + "A14_s{iii}_w3_z001_t001.tif" + )] == [ str(output_dir / "A14_s001_w3_z001_t001.tif"), str(output_dir / "A14_s002_w3_z001_t001.tif"), ] @@ -2897,11 +2848,7 @@ def main_flow_with_auxiliary_source(image): spec=spec, storage_plan=None, projection=None, - main_flow_projection=( - MainFlowInputProjection.DECLARED_SOURCE_IMAGE - if input_index == 0 - else None - ), + main_flow_projection=(MainFlowInputProjection.DECLARED_SOURCE_IMAGE if input_index == 0 else None), ) for input_index, (edge_key, spec) in enumerate( zip( @@ -3205,6 +3152,7 @@ def measure_neurites(image, pixel_size=1.0): spec=pixel_size_spec, storage_plan=None, projection=None, + ), ) ) @@ -3342,12 +3290,7 @@ def test_pipeline_start_main_flow_survives_prior_producer_image_input( assert mask_edge.storage_plan is not None assert (mask_edge.main_flow_projection is not None) is False assert invocation.contract.accepts_implicit_main_flow_input is True - assert ( - compiled_pattern.default_group.main_flow_input_refs( - source_bindings=CompiledSourceBindingPlan.empty() - ) - is None - ) + assert compiled_pattern.default_group.main_flow_input_refs(source_bindings=CompiledSourceBindingPlan.empty()) is None assert tuple( binding.alias for binding in scope.main_flow_source_binding_plan.binding_declarations @@ -3790,15 +3733,7 @@ def test_alias_only_workspace_filter_excludes_unselected_source_and_orders_stack ) source_planes = tuple( SourcePlaneProjection( - address=OpenHCSPlaneAddress( - ( - (Microscopy.Well, "R04C09"), - (Microscopy.Site, "11"), - (Microscopy.Channel, channel), - (Microscopy.ZIndex, "1"), - (Microscopy.Timepoint, "1"), - ) - ), + address=OpenHCSPlaneAddress(((Microscopy.Well, "R04C09"), (Microscopy.Site, "11"), (Microscopy.Channel, channel), (Microscopy.ZIndex, "1"), (Microscopy.Timepoint, "1"))), ref=SourcePixelRef("disk", f"/source/ch{channel}.tiff"), source_alias=binding.alias, ) @@ -3884,15 +3819,7 @@ def test_unbound_workspace_source_keeps_filename_component_provenance( source_metadata_by_path={virtual_path: source_metadata}, source_projections_by_virtual_path={ virtual_path: SourcePlaneProjection( - address=OpenHCSPlaneAddress( - ( - (Microscopy.Well, "A01"), - (Microscopy.Site, "2"), - (Microscopy.Channel, "1"), - (Microscopy.ZIndex, "3"), - (Microscopy.Timepoint, "4"), - ) - ), + address=OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, "2"), (Microscopy.Channel, "1"), (Microscopy.ZIndex, "3"), (Microscopy.Timepoint, "4"))), ref=source_ref, source_alias="OrigDNA", source_metadata=source_metadata, @@ -3955,9 +3882,7 @@ def resolve_address(backend_address, backend, *, base_path): Microscopy.Well.name: "A01", SourceFilterSubject.EXTENSION.value: ".tif", } - assert projected.metadata.source_provenance.represented_source_image_names == ( - "OrigDNA", - ) + assert projected.metadata.source_provenance.represented_source_image_names == ("OrigDNA",) def test_step_output_load_filter_skips_source_binding_filter() -> None: @@ -4041,15 +3966,7 @@ def test_producer_anchored_pipeline_start_paths_use_exact_source_projection_bund ) source_planes = tuple( SourcePlaneProjection( - address=OpenHCSPlaneAddress( - ( - (Microscopy.Well, "A01"), - (Microscopy.Site, "1"), - (Microscopy.Channel, str(channel)), - (Microscopy.ZIndex, "1"), - (Microscopy.Timepoint, "1"), - ) - ), + address=OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, "1"), (Microscopy.Channel, str(channel)), (Microscopy.ZIndex, "1"), (Microscopy.Timepoint, "1"))), ref=SourcePixelRef("disk", str(source_path)), source_alias=alias, ) @@ -4127,9 +4044,7 @@ def resolve_address(backend_address, backend, *, base_path): "source_workspace_projection_authority", lambda _self: SimpleNamespace( projection_if_available=lambda: projection, - projection_or_empty=lambda *, axis_id=None: projection.filtered_by_axis( - axis_id=axis_id - ), + projection_or_empty=lambda *, axis_id=None: projection.filtered_by_axis(axis_id=axis_id), ), ) @@ -5029,14 +4944,12 @@ def test_function_output_path_uses_payload_identity_over_input_carrier( }, ).payload_with(np.zeros((4, 5), dtype=np.float32), None) - output_path = FunctionOutputIdentity.path_from_request( - FunctionOutputPathRequest( + output_path = FunctionOutputIdentity.path_from_request(FunctionOutputPathRequest( parser=SourceSchemaFilenameParser(), output_dir=tmp_path, output_payload=payload, input_path="A14_s001_w1_z001_t001.tif", - ) - ) + )) assert output_path.name == "A14_s002_w3_z001_t001.tif" @@ -5055,14 +4968,12 @@ def test_function_output_path_uses_payload_identity_without_input_path( }, ).payload_with(np.zeros((4, 5), dtype=np.float32), None) - output_path = FunctionOutputIdentity.path_from_request( - FunctionOutputPathRequest( + output_path = FunctionOutputIdentity.path_from_request(FunctionOutputPathRequest( parser=SourceSchemaFilenameParser(), output_dir=tmp_path, output_payload=payload, input_path=None, - ) - ) + )) assert output_path.name == "A14_s001_w5_z001_t001.tif" @@ -5088,7 +4999,10 @@ def test_function_output_identity_completes_partial_payload_metadata_from_fallba ) assert identity is not None - assert identity.filename(parser) == "Sequence1_s001_w1_z001_t000.tif" + assert ( + identity.filename(parser) + == "Sequence1_s001_w1_z001_t000.tif" + ) def test_function_output_identity_uses_fallback_path_extension_for_payload_identity() -> ( @@ -5112,7 +5026,10 @@ def test_function_output_identity_uses_fallback_path_extension_for_payload_ident ) assert identity is not None - assert identity.filename(parser) == "A01_s001_w2_z001_t001.png" + assert ( + identity.filename(parser) + == "A01_s001_w2_z001_t001.png" + ) @pytest.mark.parametrize( @@ -5131,7 +5048,9 @@ def test_source_stack_output_takes_image_extension_not_artifact_key( metadata = ImagePayloadMetadata( source_provenance=SourceImageProvenance( source_image_provenance_planes=SourceImageProvenancePlanes.from_components( - paths=tuple(f"A01_s001_w0_z{z:03d}_t001.tif" for z in (1, 2, 3)), + paths=tuple( + f"A01_s001_w0_z{z:03d}_t001.tif" for z in (1, 2, 3) + ), component_metadata=tuple( { "well": "A01", @@ -5180,14 +5099,12 @@ def test_function_output_path_uses_input_identity_for_multi_plane_carrier( ), ).payload_with(np.zeros((2, 4, 5), dtype=np.float32), None) - output_path = FunctionOutputIdentity.path_from_request( - FunctionOutputPathRequest( + output_path = FunctionOutputIdentity.path_from_request(FunctionOutputPathRequest( parser=SourceSchemaFilenameParser(), output_dir=tmp_path, output_payload=payload, input_path="A14_s001_w1_z001_t001.tif", - ) - ) + )) assert output_path.name == "A14_s001_w1_z001_t001.tif" @@ -5205,14 +5122,12 @@ def test_function_output_path_rejects_multi_plane_carrier_without_input_path( ).payload_with(np.zeros((2, 4, 5), dtype=np.float32), None) with pytest.raises(ValueError, match="multi-plane source provenance"): - FunctionOutputIdentity.path_from_request( - FunctionOutputPathRequest( + FunctionOutputIdentity.path_from_request(FunctionOutputPathRequest( parser=SourceSchemaFilenameParser(), output_dir=tmp_path, output_payload=payload, input_path=None, - ) - ) + )) def test_function_output_path_uses_variable_component_identity( @@ -5324,7 +5239,10 @@ def test_collapsed_output_identity_uses_retained_source_contributors( assert identity.filename_component_values is not None assert identity.filename_component_values["site"] == 1 assert filename_identity is not None - assert filename_identity.filename(request.parser) == "A14_s001_w1_z001_t001.tif" + assert ( + filename_identity.filename(request.parser) + == "A14_s001_w1_z001_t001.tif" + ) def test_composite_then_z_collapse_uses_current_scalar_identity( @@ -5468,7 +5386,10 @@ def test_function_output_path_rejects_variation_outside_identity_components( ).payload_with(np.zeros((2, 4, 5), dtype=np.float32), None) assert payload.metadata.source_provenance.source_plane_count == 2 - assert payload.metadata.source_image_provenance_planes.contributor_count == 0 + assert ( + payload.metadata.source_image_provenance_planes.contributor_count + == 0 + ) with pytest.raises(ValueError, match="varies outside identity components"): FunctionOutputIdentity.from_request( @@ -5681,14 +5602,15 @@ def save_batch( expected += "_Corrected" assert Path(filemanager.saved_paths[0]).name == expected + ".tif" assert ( - filemanager.saved_payloads[0].metadata.source_component_metadata["site"] == "1" + filemanager.saved_payloads[0].metadata.source_component_metadata[ + "site" + ] + == "1" ) assert records[0].component_values["site"] == 1 if named_topology != "anonymous": assert records[0].output_context == named_context - assert filemanager.saved_payloads[0].metadata.source_image_names == ( - "Corrected", - ) + assert filemanager.saved_payloads[0].metadata.source_image_names == ("Corrected",) assert filemanager.saved_payloads[0].data is payload.data @@ -5722,7 +5644,8 @@ def qualified_producer_manifest(tmp_path): records.append( ProducedOutputSemantics.from_output( producer, - tmp_path / identity.filename(parser), + tmp_path + / identity.filename(parser), identity, output_context=AlignedImageSliceContext.main_flow( output_key=f"Output{plane}", @@ -5748,7 +5671,9 @@ def count_filename(identity, parser): calls.append(identity) return original(identity, parser) - monkeypatch.setattr(FunctionOutputIdentity, "filename", count_filename) + monkeypatch.setattr( + FunctionOutputIdentity, "filename", count_filename + ) paths = ( records[1].output_path, "A01_s001_w2_z001_t001.tif", @@ -5870,9 +5795,7 @@ def test_producer_admission_rederives_live_storage_aliases_each_epoch( ): store, producer, consumer, records, parser = qualified_producer_manifest storage_components = dict(records[0].filename_values) - record = replace( - records[0], filename_component_values=storage_components - ).published() + record = replace(records[0], filename_component_values=storage_components).published() store.record_outputs(producer, (record,)) old_index = store.producer_record_index_for(consumer, parser) old_alias = record.without_filename_qualifier().filename(parser) diff --git a/tests/unit/test_function_step_execution_plan.py b/tests/unit/test_function_step_execution_plan.py index 39134d3cdf..cb7a2bd523 100644 --- a/tests/unit/test_function_step_execution_plan.py +++ b/tests/unit/test_function_step_execution_plan.py @@ -521,8 +521,7 @@ def test_invocation_output_selection_omits_inactive_component_outputs(): {channel_one.ref(): channel_one}, compiled_output_plans=invocation.output_plans_for_component( ComponentGroupScope.from_raw( - ("1", "2"), - component=Microscopy.Channel, + ("1", "2"), component=Microscopy.Channel, ), "1", ), @@ -712,9 +711,7 @@ def publish_derived_channels(image, *, illumination_function=None, runtime): ) selected_outputs = invocation.select_outputs( {active_output.ref(): active_output}, - compiled_output_plans=invocation.output_plans_for_component( - execution_scope, "1" - ), + compiled_output_plans=invocation.output_plans_for_component(execution_scope, "1"), ) assert tuple(edge.spec for edge in selected_inputs.values()) == ( @@ -731,9 +728,7 @@ def publish_derived_channels(image, *, illumination_function=None, runtime): ) second_selected_outputs = invocation.select_outputs( {second_active_output.ref(): second_active_output}, - compiled_output_plans=invocation.output_plans_for_component( - execution_scope, "2" - ), + compiled_output_plans=invocation.output_plans_for_component(execution_scope, "2"), ) assert tuple(edge.spec for edge in second_selected_inputs.values()) == ( @@ -752,8 +747,7 @@ def test_invocation_output_selection_rejects_missing_active_component_output(): {}, compiled_output_plans=invocation.output_plans_for_component( ComponentGroupScope.from_raw( - ("1", "2"), - component=Microscopy.Channel, + ("1", "2"), component=Microscopy.Channel, ), "1", ), @@ -776,8 +770,7 @@ def test_invocation_output_selection_rejects_active_projection_drift(): {drifted_channel_one.ref(): drifted_channel_one}, compiled_output_plans=invocation.output_plans_for_component( ComponentGroupScope.from_raw( - ("1", "2"), - component=Microscopy.Channel, + ("1", "2"), component=Microscopy.Channel, ), "1", ), diff --git a/tests/unit/test_function_step_execution_scope.py b/tests/unit/test_function_step_execution_scope.py index 086e9aa4d9..bb7f3409a3 100644 --- a/tests/unit/test_function_step_execution_scope.py +++ b/tests/unit/test_function_step_execution_scope.py @@ -141,25 +141,17 @@ def test_runtime_axis_scope_returns_none_only_without_declared_component_axis() def test_runtime_axis_scope_joins_correlated_rows_without_inventing_states() -> None: rows = tuple( RuntimeExecutionAxisScope.from_raw( - "A01", - component=None, - value=None, - fixed_component_values=( - (Microscopy.Site, site), - (Microscopy.Timepoint, timepoint), - ), + "A01", component=None, value=None, + fixed_component_values=((Microscopy.Site, site), (Microscopy.Timepoint, timepoint)), ) for site, timepoint in (("1", "3"), ("2", "4")) ) time_three = RuntimeExecutionAxisScope.from_raw( - "A01", - component=None, - value=None, + "A01", component=None, value=None, fixed_component_values=((Microscopy.Timepoint, "3"),), ) joined = tuple( - result - for row in rows + result for row in rows if (result := row.join_execution_cohort(time_three)) is not None ) assert joined == (rows[0],) @@ -169,9 +161,7 @@ def test_runtime_axis_scope_joins_correlated_rows_without_inventing_states() -> def test_runtime_axis_scope_empty_context_broadcasts_to_actual_correlated_row() -> None: empty = RuntimeExecutionAxisScope.from_raw("A01", component=None, value=None) source = RuntimeExecutionAxisScope.from_raw( - "A01", - component=None, - value=None, + "A01", component=None, value=None, fixed_component_values=((Microscopy.Site, "1"), (Microscopy.Timepoint, "3")), ) assert empty.join_execution_cohort(source) == source @@ -179,46 +169,29 @@ def test_runtime_axis_scope_empty_context_broadcasts_to_actual_correlated_row() assert empty.join_execution_cohort(empty) == empty -@pytest.mark.parametrize( - "axis,component,value", - [ - ("B01", Microscopy.Channel, "1"), - ("A01", Microscopy.Site, "1"), - ("A01", Microscopy.Channel, "2"), - ], -) -def test_runtime_axis_scope_cannot_join_other_execution_group( - axis, component, value -) -> None: - own = RuntimeExecutionAxisScope.from_raw( - "A01", component=Microscopy.Channel, value="1" - ) +@pytest.mark.parametrize("axis,component,value", [ + ("B01", Microscopy.Channel, "1"), + ("A01", Microscopy.Site, "1"), + ("A01", Microscopy.Channel, "2"), +]) +def test_runtime_axis_scope_cannot_join_other_execution_group(axis, component, value) -> None: + own = RuntimeExecutionAxisScope.from_raw("A01", component=Microscopy.Channel, value="1") other = RuntimeExecutionAxisScope.from_raw(axis, component=component, value=value) assert own.join_execution_cohort(other) is None def test_runtime_axis_scope_join_derives_partial_fixed_context_union() -> None: left = RuntimeExecutionAxisScope.from_raw( - "A01", - component=None, - value=None, + "A01", component=None, value=None, fixed_component_values=((Microscopy.Site, "1"), (Microscopy.Timepoint, "3")), ) right = RuntimeExecutionAxisScope.from_raw( - "A01", - component=None, - value=None, + "A01", component=None, value=None, fixed_component_values=((Microscopy.Timepoint, "3"), (Microscopy.ZIndex, "2")), ) expected = RuntimeExecutionAxisScope.from_raw( - "A01", - component=None, - value=None, - fixed_component_values=( - (Microscopy.Site, "1"), - (Microscopy.Timepoint, "3"), - (Microscopy.ZIndex, "2"), - ), + "A01", component=None, value=None, + fixed_component_values=((Microscopy.Site, "1"), (Microscopy.Timepoint, "3"), (Microscopy.ZIndex, "2")), ) assert left.join_execution_cohort(right) == expected assert right.join_execution_cohort(left) == expected @@ -939,8 +912,7 @@ def export(*, artifact_batch: RuntimeArtifactBatch): metadata_writer=True, ) plan.runtime_artifact_materialization = RuntimeArtifactMaterializationPlan( - persistent_enabled=True, - persistent_backend=Backend.MEMORY.value, + persistent_enabled=True, persistent_backend=Backend.MEMORY.value, ) plan.output_dataset_root = str(tmp_path) plan.output_dir = tmp_path / "images" @@ -948,8 +920,7 @@ def export(*, artifact_batch: RuntimeArtifactBatch): context = _plate_context("A01", (plan,)) context.metadata_cache = {} context.microscope_handler = SimpleNamespace( - parser=SourceSchemaFilenameParser(), - source_name="test", + parser=SourceSchemaFilenameParser(), source_name="test", ) _record_measurements( context, @@ -972,14 +943,11 @@ def export(*, artifact_batch: RuntimeArtifactBatch): assert {record.key.name for record in observation.contexts[0].records} == { output_spec.name } - analysis_inputs = observation.contexts[0].outputs.hook_observations.get( - "analysis_consolidation" - ) + analysis_inputs = observation.contexts[0].outputs.hook_observations.get("analysis_consolidation") assert analysis_inputs is not None assert analysis_inputs.destination.backend == Backend.MEMORY.value (table,) = tuple( - output - for group in analysis_inputs.outputs_by_directory.values() + output for group in analysis_inputs.outputs_by_directory.values() for output in group ) assert table.csv_content == "ImageNumber,Count\r\n1,7\r\n" diff --git a/tests/unit/test_function_step_transport.py b/tests/unit/test_function_step_transport.py index c52b07b209..498f73a3e8 100644 --- a/tests/unit/test_function_step_transport.py +++ b/tests/unit/test_function_step_transport.py @@ -22,10 +22,7 @@ PipelineConfig, ) from openhcs.core.callable_contract import CallableContract -from openhcs.core.function_patterns import ( - CompiledFunctionInvocation, - FunctionInvocationKey, -) +from openhcs.core.function_patterns import CompiledFunctionInvocation, FunctionInvocationKey from openhcs.core.function_reference import ( FunctionReference, FunctionReferenceTransportAuthority, @@ -315,7 +312,9 @@ def test_transport_preserves_explicit_lazy_processing_defaults() -> None: exec(compile(source, "", "exec"), namespace) restored = FunctionStepTransportAuthority.pipeline_steps_from_namespace(namespace) - assert restored[0].processing_config.variable_components == [Microscopy.Channel] + assert restored[0].processing_config.variable_components == [ + Microscopy.Channel + ] assert restored[0].processing_config.group_by is Microscopy.Site @@ -401,10 +400,7 @@ def test_bundle_transport_preserves_prepared_runtime_contract_identity() -> None ) from openhcs.core.compiled_step_plan import CompiledStepPlan from openhcs.core.context.processing_context import ProcessingContext - from openhcs.core.function_patterns import ( - compile_function_pattern, - normalize_function_pattern, - ) + from openhcs.core.function_patterns import compile_function_pattern, normalize_function_pattern from openhcs.core.orchestrator.execution_result import ( RuntimeExecutionTransportSerialization, ) @@ -451,85 +447,59 @@ def test_bundle_transport_preserves_prepared_runtime_contract_identity() -> None ) shared_items = replace( captured, - groups=( - replace( - captured.groups[0], - items=( - captured_item, - replace(captured_item, key=replace(captured_item.key, position=1)), - ), - ), - ), + groups=(replace( + captured.groups[0], + items=(captured_item, replace(captured_item, key=replace(captured_item.key, position=1))), + ),), ) first_item, second_item = tuple( - FunctionStepTransportAuthority.normalize_function_spec( - shared_items - ).iter_items() + FunctionStepTransportAuthority.normalize_function_spec(shared_items).iter_items() ) assert first_item.contract is second_item.contract repeated_group = replace( pattern.groups[0], - invocations=( - invocation, - replace(invocation, key=replace(invocation.key, position=1)), - ), + invocations=(invocation, replace(invocation, key=replace(invocation.key, position=1))), ) first_invocation, second_invocation = ( - FunctionStepTransportAuthority.normalize_compiled_group( - repeated_group - ).invocations + FunctionStepTransportAuthority.normalize_compiled_group(repeated_group).invocations ) assert first_invocation.contract is second_invocation.contract prepared_callable = invocation.contract.resolve_runtime_callable() plan = CompiledStepPlan( - step_index=0, - step_name="Crop", - axis_id="A01", + step_index=0, step_name="Crop", axis_id="A01", compiled_function_pattern=pattern, ) context = ProcessingContext( - axis_id="A01", - step_plans={0: plan}, + axis_id="A01", step_plans={0: plan}, filemanager=FileManager({Backend.MEMORY.value: MemoryStorageBackend()}), ) context.freeze() second_invocation = replace( - invocation, - kwargs=(("left_right_rectangle_positions", (2, 5)),), + invocation, kwargs=(("left_right_rectangle_positions", (2, 5)),), ) second_pattern = replace( - pattern, - groups=(replace(pattern.groups[0], invocations=(second_invocation,)),), + pattern, groups=(replace(pattern.groups[0], invocations=(second_invocation,)),), ) second_plan = replace(plan, axis_id="A02", compiled_function_pattern=second_pattern) second_context = ProcessingContext( - axis_id="A02", - step_plans={0: second_plan}, - filemanager=context.filemanager, + axis_id="A02", step_plans={0: second_plan}, filemanager=context.filemanager, ) second_context.freeze() contexts = {"A01": context, "A02": second_context} bundle = CompiledExecutionBundle.from_runtime_contexts( - pipeline_definition=(), - runtime_contexts=contexts, + pipeline_definition=(), runtime_contexts=contexts, worker_assignments={"worker_0": ["A01", "A02"]}, runtime_environment=CompiledRuntimeEnvironmentPlan.from_global_config( - GlobalPipelineConfig(), - compiled_contexts=contexts, - server_mode=False, + GlobalPipelineConfig(), compiled_contexts=contexts, server_mode=False, ), ) transport_context = bundle.transport_contexts["A01"] transport_plan = transport_context.step_plans[0] assert transport_context is not context assert transport_plan is not plan - (transport_invocation,) = tuple( - transport_plan.compiled_function_pattern.iter_invocations() - ) + (transport_invocation,) = tuple(transport_plan.compiled_function_pattern.iter_invocations()) second_transport_invocation = next( - bundle.transport_contexts["A02"] - .step_plans[0] - .compiled_function_pattern.iter_invocations() + bundle.transport_contexts["A02"].step_plans[0].compiled_function_pattern.iter_invocations() ) assert transport_invocation.contract is second_transport_invocation.contract assert transport_invocation.func is second_transport_invocation.func @@ -542,9 +512,7 @@ def test_bundle_transport_preserves_prepared_runtime_contract_identity() -> None assert plan.compiled_function_pattern is pattern assert invocation.contract.resolve_runtime_callable() is prepared_callable assert transport_context.runtime_value_store is context.runtime_value_store - assert ( - transport_context.runtime_image_stack_cache is context.runtime_image_stack_cache - ) + assert transport_context.runtime_image_stack_cache is context.runtime_image_stack_cache assert transport_context.filemanager is context.filemanager transport_plan.step_name = "Transport-only name" assert plan.step_name == "Crop" @@ -569,15 +537,10 @@ def test_bundle_transport_preserves_prepared_runtime_contract_identity() -> None is restored_invocation.contract.resolve_runtime_callable() ) second_restored_invocation = next( - restored.runtime_contexts["A02"] - .step_plans[0] - .compiled_function_pattern.iter_invocations() + restored.runtime_contexts["A02"].step_plans[0].compiled_function_pattern.iter_invocations() ) assert restored_invocation.contract is second_restored_invocation.contract - assert ( - restored_invocation.runtime_callable - is second_restored_invocation.runtime_callable - ) + assert restored_invocation.runtime_callable is second_restored_invocation.runtime_callable assert second_restored_invocation.kwargs == second_invocation.kwargs assert restored_invocation.contract is not transport_invocation.contract assert ( @@ -606,9 +569,7 @@ def test_generated_source_resolves_catalog_owned_cellprofiler_callable() -> None restored = FunctionStepTransportAuthority.pipeline_steps_from_namespace(namespace) assert len(restored) == 1 assert restored[0].name == "Crop" - assert restored[0].func is RegistryService.registered_callable( - cellprofiler_backend.crop - ) + assert restored[0].func is RegistryService.registered_callable(cellprofiler_backend.crop) def test_registered_custom_function_is_function_step_picklable() -> None: @@ -659,10 +620,7 @@ def codex_pickle_probe(image): restored_invocation = pickle.loads(pickle.dumps(invocation)) assert restored_invocation.runtime_callable is custom_func assert restored_invocation.contract is not contract - assert ( - restored_invocation.artifact_input_edges - == invocation.artifact_input_edges - ) + assert restored_invocation.artifact_input_edges == invocation.artifact_input_edges assert ( CellProfilerModule.require_module("Crop").require_callable() is canonical_crop @@ -690,9 +648,7 @@ def codex_lazy_import_probe(image): encoding="utf-8", ) vars(custom_functions).pop(func_name, None) - monkeypatch.setattr( - manager_module, "get_data_file_path", lambda _name, *, create=True: storage_dir - ) + monkeypatch.setattr(manager_module, "get_data_file_path", lambda _name, *, create=True: storage_dir) namespace: dict[str, object] = {} try: diff --git a/tests/unit/test_g5_processing_contract_guards.py b/tests/unit/test_g5_processing_contract_guards.py index 67b43b8d8c..e04da345bc 100644 --- a/tests/unit/test_g5_processing_contract_guards.py +++ b/tests/unit/test_g5_processing_contract_guards.py @@ -182,9 +182,7 @@ def _docstring_ids(tree: ast.AST) -> set[int]: ids = set() for node in ast.walk(tree): if ( - isinstance( - node, (ast.Module, ast.ClassDef, ast.FunctionDef, ast.AsyncFunctionDef) - ) + isinstance(node, (ast.Module, ast.ClassDef, ast.FunctionDef, ast.AsyncFunctionDef)) and node.body and isinstance(node.body[0], ast.Expr) and isinstance(node.body[0].value, ast.Constant) diff --git a/tests/unit/test_g7_runtime_protocol_guards.py b/tests/unit/test_g7_runtime_protocol_guards.py index ab2f7d42c8..7c6c5457a1 100644 --- a/tests/unit/test_g7_runtime_protocol_guards.py +++ b/tests/unit/test_g7_runtime_protocol_guards.py @@ -86,19 +86,13 @@ def test_worker_connections_carry_declared_messages() -> None: and isinstance(node.func, ast.Attribute) and node.func.attr == "send" and node.args - and isinstance( - node.args[0], (ast.Tuple, ast.Constant, ast.List, ast.Dict) - ) + and isinstance(node.args[0], (ast.Tuple, ast.Constant, ast.List, ast.Dict)) ): offenders.append(f"{path.name}:{node.lineno}") - if ( - isinstance(node, ast.Compare) - and any( - isinstance(operand, ast.Constant) and isinstance(operand.value, str) - for operand in (node.left, *node.comparators) - ) - and re.search(r"kind|operation|command|message", ast.unparse(node)) - ): + if isinstance(node, ast.Compare) and any( + isinstance(operand, ast.Constant) and isinstance(operand.value, str) + for operand in (node.left, *node.comparators) + ) and re.search(r"kind|operation|command|message", ast.unparse(node)): offenders.append(f"{path.name}:{node.lineno}") assert offenders == [] diff --git a/tests/unit/test_gray_to_color_binding_axis_432.py b/tests/unit/test_gray_to_color_binding_axis_432.py index 370136c258..7e940241bd 100644 --- a/tests/unit/test_gray_to_color_binding_axis_432.py +++ b/tests/unit/test_gray_to_color_binding_axis_432.py @@ -6,10 +6,7 @@ import pytest from openhcs.core.component_group_scope import RuntimeExecutionAxisScope -from openhcs.core.aligned_image_payload import ( - AlignedImageStack, - ImagePayloadBundleContext, -) +from openhcs.core.aligned_image_payload import (AlignedImageStack, ImagePayloadBundleContext) from openhcs.core.callable_contract import CallableContract from openhcs.core.artifacts import ArtifactSpec, ImageArtifactType from openhcs.core.config import StepSourceBindingsConfig @@ -28,22 +25,13 @@ from openhcs.interop.cellprofiler.runtime.function_contract_execution import ( CellProfilerFunctionContractExecutor, ) -from openhcs.interop.cellprofiler.runtime.module_execution import ( - CellProfilerModuleExecutor, -) +from openhcs.interop.cellprofiler.runtime.module_execution import CellProfilerModuleExecutor from openhcs.interop.cellprofiler.runtime.output_recording import ( CellProfilerOutputRecorder, ) -from openhcs.processing.backends.cellprofiler.color import ( - GrayToColorModule, - gray_to_color, -) -from tests.unit.test_cellprofiler_generic_special_input_binding import ( - _compile_public_step, -) -from tests.unit.cellprofiler_runtime_test_support import ( - cellprofiler_runtime_adapter_for_test, -) +from openhcs.processing.backends.cellprofiler.color import GrayToColorModule, gray_to_color +from tests.unit.test_cellprofiler_generic_special_input_binding import _compile_public_step +from tests.unit.cellprofiler_runtime_test_support import cellprofiler_runtime_adapter_for_test from openhcs.domains.microscopy.axes import Microscopy from openhcs.core.axes import ColourAxis from openhcs.core.image_payload_execution_mode import ( @@ -53,9 +41,7 @@ @pytest.mark.parametrize("explicit_selector", [False, True]) -def test_declared_fitc_runtime_plane_retains_values_and_physical_identity( - explicit_selector, -): +def test_declared_fitc_runtime_plane_retains_values_and_physical_identity(explicit_selector): kwargs = { "color_scheme": GrayToColorModule.Scheme.STACK, "rescale_intensity": False, @@ -64,64 +50,40 @@ def test_declared_fitc_runtime_plane_retains_values_and_physical_identity( if explicit_selector: kwargs["image_name"] = ("FITC",) step = FunctionStep( - func=(gray_to_color, kwargs), - name="RetainRawReference", + func=(gray_to_color, kwargs), name="RetainRawReference", source_bindings=StepSourceBindingsConfig( - enabled=True, - bindings=(NamedSourceBinding("FITC"),), + enabled=True, bindings=(NamedSourceBinding("FITC"),), ), ) invocation = _compile_public_step(step) - contract = replace( - invocation.contract, - metadata=replace( - invocation.contract.metadata, - runtime_adapter=CellProfilerRuntimeAdapter.runtime_adapter_spec(), - ), - ) + contract = replace(invocation.contract, metadata=replace( + invocation.contract.metadata, + runtime_adapter=CellProfilerRuntimeAdapter.runtime_adapter_spec(), + )) pixels = np.arange(20, dtype=np.float32).reshape(4, 5) * 1000 - physical = { - "well": "A01", - "site": "1", - "channel": "2", - "z_index": "1", - "timepoint": "1", - } + physical = {"well": "A01", "site": "1", "channel": "2", "z_index": "1", "timepoint": "1"} metadata = ImagePayloadMetadata( plane_axis=RuntimePlaneAxis.RUNTIME_SLICE, source_image_provenance_planes=SourceImageProvenancePlanes.from_components( - paths=("/synthetic/A01_s1_w2_z1_t1.tif",), - component_metadata=(physical,), + paths=("/synthetic/A01_s1_w2_z1_t1.tif",), component_metadata=(physical,), ), source_image_names=("FITC",), source_voxel_spacing=SourceVoxelSpacing((1.0, 1.0)), - source_spatial_domain=SourceSpatialDomain( - origin_yx=(0, 0), source_shape_yx=(4, 5) - ), + source_spatial_domain=SourceSpatialDomain(origin_yx=(0, 0), source_shape_yx=(4, 5)), ) source = metadata.payload_with(pixels[None], None) runtime = cellprofiler_runtime_adapter_for_test( - runtime_value_store=RuntimeValueStore(), - callable_contract=contract, + runtime_value_store=RuntimeValueStore(), callable_contract=contract, variable_components=(Microscopy.Channel,), - axis_scope=RuntimeExecutionAxisScope.from_raw( - "synthetic", component=None, value=None - ), - artifact_inputs={ - edge.key: InvocationArtifactInputEdgePlan.from_source_declarations( - key=edge.key, - spec=edge.spec, - main_flow_artifacts=contract.artifact_inputs, - invocation_sources=contract.artifact_inputs, - ) - for edge in invocation.artifact_input_edges - }, + axis_scope=RuntimeExecutionAxisScope.from_raw("synthetic", component=None, value=None), + artifact_inputs={edge.key: InvocationArtifactInputEdgePlan.from_source_declarations( + key=edge.key, spec=edge.spec, main_flow_artifacts=contract.artifact_inputs, + invocation_sources=contract.artifact_inputs, + ) for edge in invocation.artifact_input_edges}, ) executor = CellProfilerModuleExecutor(gray_to_color, contract) request = executor._image_request( - source, - runtime, - module_type=GrayToColorModule, + source, runtime, module_type=GrayToColorModule, active_input_specs=contract.artifact_inputs.specs, ) assert contract.artifact_inputs.names() == ("FITC",) @@ -133,38 +95,22 @@ def test_declared_fitc_runtime_plane_retains_values_and_physical_identity( assert request.plane_projection.axis is RuntimePlaneAxis.RUNTIME_SLICE assert request.plane_projection.axis_size == 1 assert request.plane_projection.plane_index is None - assert ( - request.payload.slices[0].metadata.plane_axis is RuntimePlaneAxis.SOURCE_BINDING - ) + assert request.payload.slices[0].metadata.plane_axis is RuntimePlaneAxis.SOURCE_BINDING assert request.payload.slices[0].metadata.source_image_names == ("FITC",) execution = executor._invocation_request( - image_request=request, - adapter=runtime, - current_image=source, + image_request=request, adapter=runtime, current_image=source, module_type=GrayToColorModule, - kwargs={ - "color_scheme": GrayToColorModule.Scheme.STACK, - "rescale_intensity": False, - }, + kwargs={"color_scheme": GrayToColorModule.Scheme.STACK, "rescale_intensity": False}, ) result = CellProfilerFunctionContractExecutor().execute( - contract, - contract.resolve_canonical_raw_callable(), - execution.payload, - execution.kwargs, - execution_mode=execution.execution_mode, - plane_projection=execution.plane_projection, + contract, contract.resolve_canonical_raw_callable(), execution.payload, execution.kwargs, + execution_mode=execution.execution_mode, plane_projection=execution.plane_projection, ) # Float32 is the original Stack runner's output contract, not unit normalization. assert result.metadata.plane_axis is RuntimePlaneAxis.RUNTIME_SLICE - plane = RuntimeSliceProjection.value_for_slice( - result, - RuntimePlaneAxisValueProjection.from_selected_plane( - axis=RuntimePlaneAxis.RUNTIME_SLICE, - plane_index=0, - axis_size=1, - ), - ) + plane = RuntimeSliceProjection.value_for_slice(result, RuntimePlaneAxisValueProjection.from_selected_plane( + axis=RuntimePlaneAxis.RUNTIME_SLICE, plane_index=0, axis_size=1, + )) assert plane.data.shape == (4, 5, 1) np.testing.assert_array_equal(plane.data[..., 0], pixels) np.testing.assert_array_equal(source.data[0], pixels) @@ -178,27 +124,17 @@ def test_declared_fitc_runtime_plane_retains_values_and_physical_identity( def _source_plane(pixels, alias): return ImagePayloadMetadata( source_path="/synthetic/A01_s1_w2_z1_t1.tif", - source_component_metadata={ - "well": "A01", - "site": "1", - "channel": "2", - "z_index": "1", - "timepoint": "1", - }, + source_component_metadata={"well": "A01", "site": "1", "channel": "2", "z_index": "1", "timepoint": "1"}, source_image_names=(alias,), source_voxel_spacing=SourceVoxelSpacing((1.0, 1.0)), - source_spatial_domain=SourceSpatialDomain( - origin_yx=(0, 0), source_shape_yx=pixels.shape[:2] - ), + source_spatial_domain=SourceSpatialDomain(origin_yx=(0, 0), source_shape_yx=pixels.shape[:2]), ).payload_with(pixels, None) def _execute_bound_stack(bundle): contract = CallableContract.from_callable(gray_to_color) return CellProfilerFunctionContractExecutor().execute( - contract, - contract.resolve_canonical_raw_callable(), - bundle, + contract, contract.resolve_canonical_raw_callable(), bundle, {"color_scheme": GrayToColorModule.Scheme.STACK, "rescale_intensity": False}, execution_mode=FullStackExecution, plane_projection=RuntimePlaneAxisValueProjection.preserve( @@ -209,17 +145,11 @@ def _execute_bound_stack(bundle): ) -@pytest.mark.parametrize( - "aliases", - [("RawBody", "CappedOutgrowth"), ("IndependentRoleA", "IndependentRoleB")], -) +@pytest.mark.parametrize("aliases", [("RawBody", "CappedOutgrowth"), ("IndependentRoleA", "IndependentRoleB")]) def test_two_processing_roles_share_physical_channel_without_fabrication(aliases): raw = np.arange(20, dtype=np.float32).reshape(4, 5) * 1000 capped = np.clip(raw / 6000, 0, 1) - inputs = tuple( - _source_plane(pixels, alias) - for pixels, alias in zip((raw, capped), aliases, strict=True) - ) + inputs = tuple(_source_plane(pixels, alias) for pixels, alias in zip((raw, capped), aliases, strict=True)) bundle = ImagePayloadBundleContext.from_payloads(inputs).compose() assert bundle.metadata.plane_axis is RuntimePlaneAxis.SOURCE_BINDING result = _execute_bound_stack(bundle) @@ -239,17 +169,10 @@ def test_projected_runtime_plane_needs_a_separate_named_binding_axis(): pixels = np.arange(20, dtype=np.float32).reshape(4, 5) * 1000 scalar = _source_plane(pixels, "FITC") metadata = scalar.metadata - retained = replace( - metadata, plane_axis=RuntimePlaneAxis.RUNTIME_SLICE - ).payload_with(pixels[None], None) - projected = RuntimeSliceProjection.value_for_slice( - retained, - RuntimePlaneAxisValueProjection.from_selected_plane( - axis=RuntimePlaneAxis.RUNTIME_SLICE, - plane_index=0, - axis_size=1, - ), - ) + retained = replace(metadata, plane_axis=RuntimePlaneAxis.RUNTIME_SLICE).payload_with(pixels[None], None) + projected = RuntimeSliceProjection.value_for_slice(retained, RuntimePlaneAxisValueProjection.from_selected_plane( + axis=RuntimePlaneAxis.RUNTIME_SLICE, plane_index=0, axis_size=1, + )) assert projected.metadata.plane_axis is None bundle = ImagePayloadBundleContext.from_payloads((projected,)).compose() assert bundle.metadata.plane_axis is RuntimePlaneAxis.SOURCE_BINDING @@ -261,9 +184,7 @@ def test_projected_runtime_plane_needs_a_separate_named_binding_axis(): def test_stack_color_output_is_not_a_second_scalar_gray_binding(): pixels = np.arange(20, dtype=np.float32).reshape(4, 5) - named = ImagePayloadBundleContext.from_payloads( - (_source_plane(pixels, "FITC"),) - ).compose() + named = ImagePayloadBundleContext.from_payloads((_source_plane(pixels, "FITC"),)).compose() color = _execute_bound_stack(named) # The existing STACK contract emits YXC even for one role. A second grayscale # binding introduces C,Y,X,source-color rather than the required C,Y,X. diff --git a/tests/unit/test_gray_to_color_payload_axis_compile_432.py b/tests/unit/test_gray_to_color_payload_axis_compile_432.py index 219cb58b00..c4d80351e3 100644 --- a/tests/unit/test_gray_to_color_payload_axis_compile_432.py +++ b/tests/unit/test_gray_to_color_payload_axis_compile_432.py @@ -7,40 +7,25 @@ import tifffile from openhcs.core.callable_contract import ( - CallableContract, - CreatedPayloadAxis, - PayloadAxisRequirement, - PreservedPayloadAxes, - creates_payload_axis, - preserves_payload_axes, + CallableContract, CreatedPayloadAxis, PayloadAxisRequirement, PreservedPayloadAxes, + creates_payload_axis, preserves_payload_axes, ) from openhcs.core.compiled_step_plan import CompiledStepPlan from openhcs.core.function_patterns import ( - CompiledFunctionGroup, - CompiledFunctionPattern, - PayloadAxisProof, - InheritedPayloadAxisProof, - CreatedPayloadAxisProof, + CompiledFunctionGroup, CompiledFunctionPattern, PayloadAxisProof, + InheritedPayloadAxisProof, CreatedPayloadAxisProof, UnprovedPayloadAxisProof, ) from openhcs.core.pipeline.compiler import PipelineCompiler from openhcs.core.aligned_image_payload import ImagePayloadBundleContext from openhcs.core.image_payload_execution_mode import NaturalExecution from openhcs.core.step_dependencies import StepInputDependency -from openhcs.interop.cellprofiler.runtime.function_contract_execution import ( - CellProfilerFunctionContractExecutor, -) +from openhcs.interop.cellprofiler.runtime.function_contract_execution import CellProfilerFunctionContractExecutor from openhcs.processing.backends.cellprofiler.color import ( - ColorToGrayMode, - ImageChannelType, - color_to_gray, - gray_to_color, + ColorToGrayMode, ImageChannelType, color_to_gray, gray_to_color, split_color_to_gray, ) -from tests.unit.test_gray_to_color_binding_axis_432 import ( - _execute_bound_stack, - _source_plane, -) +from tests.unit.test_gray_to_color_binding_axis_432 import _execute_bound_stack, _source_plane from tests.unit.test_payload_axis_compile_gate import _compiled_pattern, _session from openhcs.core.payload_axes import ColourSampleAxisSpec from openhcs.core.axes import ColourAxis @@ -51,18 +36,13 @@ def _gray_creator_session(tmp_path): tifffile.imwrite(path, np.zeros((4, 5), dtype=np.uint16)) session = _session(tmp_path, (path,)) producer = replace( - session.plans[0], - step_name="ComposeScalarRoles", - step_scope_id="step-0", + session.plans[0], step_name="ComposeScalarRoles", step_scope_id="step-0", compiled_function_pattern=_compiled_pattern(gray_to_color), ) consumer = CompiledStepPlan( - step_index=1, - step_name="SplitComposedRoles", - axis_id="A01", + step_index=1, step_name="SplitComposedRoles", axis_id="A01", main_input_dependency=StepInputDependency.step_output( - source_step_index=0, - source_step_scope_id="step-0", + source_step_index=0, source_step_scope_id="step-0", ), compiled_function_pattern=_compiled_pattern(color_to_gray), ) @@ -79,23 +59,14 @@ def test_declared_gray_creator_proves_color_consumer_before_execution(tmp_path): def test_declared_gray_creator_proves_same_group_color_consumer(tmp_path): session = _gray_creator_session(tmp_path) - group = CompiledFunctionGroup( - "default", - tuple( - invocation - for plan in session.plans.values() - for invocation in plan.compiled_function_pattern.iter_invocations() - ), - ) - session.plans = { - 0: replace( - session.plans[0], - compiled_function_pattern=CompiledFunctionPattern( - groups=(group,), - is_grouped=False, - ), - ) - } + group = CompiledFunctionGroup("default", tuple( + invocation + for plan in session.plans.values() + for invocation in plan.compiled_function_pattern.iter_invocations() + )) + session.plans = {0: replace(session.plans[0], compiled_function_pattern=CompiledFunctionPattern( + groups=(group,), is_grouped=False, + ))} session.context.step_plans = session.plans PipelineCompiler.validate_payload_axis_requirements(session) @@ -105,23 +76,12 @@ def test_preservation_annotation_cannot_create_missing_source_payload_axis(tmp_p producer = session.plans[0] (invocation,) = tuple(producer.compiled_function_pattern.iter_invocations()) # A local counterexample only: do not mutate production callable/registry. - falsely_preserving = replace( - invocation, - contract=replace( - invocation.contract, - metadata=replace( - invocation.contract.metadata, - payload_axis_transition=PreservedPayloadAxes(), - ), - ), - ) - session.plans[0] = replace( - producer, - compiled_function_pattern=CompiledFunctionPattern( - groups=(CompiledFunctionGroup("default", (falsely_preserving,)),), - is_grouped=False, - ), - ) + falsely_preserving = replace(invocation, contract=replace(invocation.contract, metadata=replace( + invocation.contract.metadata, payload_axis_transition=PreservedPayloadAxes(), + ))) + session.plans[0] = replace(producer, compiled_function_pattern=CompiledFunctionPattern( + groups=(CompiledFunctionGroup("default", (falsely_preserving,)),), is_grouped=False, + )) session.context.step_plans = session.plans with pytest.raises(ValueError, match="requires a declared ColourAxis payload axis"): PipelineCompiler.validate_payload_axis_requirements(session) @@ -130,26 +90,18 @@ def test_preservation_annotation_cannot_create_missing_source_payload_axis(tmp_p def _runtime_scalar_role_split(): raw = np.arange(20, dtype=np.float32).reshape(4, 5) * 1000 / 65535 capped = np.clip((raw - 39 / 65535) / (5961 / 65535), 0, 1) - bundle = ImagePayloadBundleContext.from_payloads( - tuple( - _source_plane(pixels, name) - for pixels, name in ((raw, "RawBody"), (capped, "CappedOutgrowth")) - ) - ).compose() + bundle = ImagePayloadBundleContext.from_payloads(tuple( + _source_plane(pixels, name) + for pixels, name in ((raw, "RawBody"), (capped, "CappedOutgrowth")) + )).compose() composite = _execute_bound_stack(bundle) assert composite.data.shape == (4, 5, 2) assert composite.metadata.axis_position(ColourAxis) == -1 contract = CallableContract.from_callable(color_to_gray) result = CellProfilerFunctionContractExecutor().execute( - contract, - contract.resolve_canonical_raw_callable(), - composite, - { - "mode": ColorToGrayMode.SPLIT, - "image_type": ImageChannelType.CHANNELS, - "channel_indices": (0, 1), - "contributions": (1.0, 1.0), - }, + contract, contract.resolve_canonical_raw_callable(), composite, + {"mode": ColorToGrayMode.SPLIT, "image_type": ImageChannelType.CHANNELS, + "channel_indices": (0, 1), "contributions": (1.0, 1.0)}, execution_mode=NaturalExecution, ) return result, (raw, capped) @@ -162,9 +114,7 @@ def test_original_runtime_split_preserves_both_role_values_and_physical_channel( assert output.data.shape == (4, 5) np.testing.assert_array_equal(output.data, expected) assert output.metadata.source_component_metadata["channel"] == "2" - assert set(output.metadata.source_image_paths) == { - "/synthetic/A01_s1_w2_z1_t1.tif" - } + assert set(output.metadata.source_image_paths) == {"/synthetic/A01_s1_w2_z1_t1.tif"} def test_scalar_role_outputs_consume_the_color_payload_axis(): @@ -175,9 +125,7 @@ def test_scalar_role_outputs_consume_the_color_payload_axis(): @pytest.mark.parametrize("same_group", [False, True]) -def test_independent_creator_declaration_needs_no_consumer_edits( - tmp_path, monkeypatch, same_group -): +def test_independent_creator_declaration_needs_no_consumer_edits(tmp_path, monkeypatch, same_group): @creates_payload_axis(ColourAxis) def independent_lane_creator(image): return np.stack((image, image * 2), axis=-1) @@ -198,30 +146,15 @@ def independent_lane_preserver(image): for invocation in _compiled_pattern(callable_).iter_invocations() ) if same_group: - consumer_invocations = tuple( - session.plans[1].compiled_function_pattern.iter_invocations() - ) - session.plans = { - 0: replace( - session.plans[0], - compiled_function_pattern=CompiledFunctionPattern( - groups=( - CompiledFunctionGroup( - "default", producer_invocations + consumer_invocations - ), - ), - is_grouped=False, - ), - ) - } + consumer_invocations = tuple(session.plans[1].compiled_function_pattern.iter_invocations()) + session.plans = {0: replace(session.plans[0], compiled_function_pattern=CompiledFunctionPattern( + groups=(CompiledFunctionGroup("default", producer_invocations + consumer_invocations),), + is_grouped=False, + ))} else: - session.plans[0] = replace( - session.plans[0], - compiled_function_pattern=CompiledFunctionPattern( - groups=(CompiledFunctionGroup("default", producer_invocations),), - is_grouped=False, - ), - ) + session.plans[0] = replace(session.plans[0], compiled_function_pattern=CompiledFunctionPattern( + groups=(CompiledFunctionGroup("default", producer_invocations),), is_grouped=False, + )) session.context.step_plans = session.plans def reject_source_backtracking(*args, **kwargs): @@ -239,12 +172,9 @@ def test_group_proof_stops_at_creator_but_rejects_later_unknown(unknown_after_cr def unknown(image): return image - functions = ( - (gray_to_color, unknown) if unknown_after_creator else (unknown, gray_to_color) - ) + functions = (gray_to_color, unknown) if unknown_after_creator else (unknown, gray_to_color) invocations = tuple( - invocation - for callable_ in functions + invocation for callable_ in functions for invocation in _compiled_pattern(callable_).iter_invocations() ) proof = CompiledFunctionGroup("default", invocations).payload_axis_proof( @@ -253,9 +183,7 @@ def unknown(image): failures = [] assert proof.validate_obligation( failure_message=lambda invocation: invocation.contract.function_name, - source_validation=lambda: pytest.fail( - "Creation/rejection must stop source validation" - ), + source_validation=lambda: pytest.fail("Creation/rejection must stop source validation"), failures=failures, ) assert failures == (["unknown"] if unknown_after_creator else []) @@ -271,9 +199,7 @@ def validate_source(): complete = InheritedPayloadAxisProof().validate_obligation( source_validation=validate_source, - failure_message=lambda invocation: pytest.fail( - "Inherited proof has no failed invocation" - ), + failure_message=lambda invocation: pytest.fail("Inherited proof has no failed invocation"), failures=failures, ) assert complete is source_complete @@ -282,25 +208,14 @@ def validate_source(): @pytest.mark.parametrize("capability_first", [False, True]) -@pytest.mark.parametrize( - "proof_type,args,expected_visits,expected_failures", - [ - (InheritedPayloadAxisProof, (), ["before", "source", "after"], []), - (CreatedPayloadAxisProof, (), ["before", "after"], []), - ( - UnprovedPayloadAxisProof, - tuple(_compiled_pattern(color_to_gray).iter_invocations()), - ["before", "after"], - ["rejected color_to_gray"], - ), - ], -) +@pytest.mark.parametrize("proof_type,args,expected_visits,expected_failures", [ + (InheritedPayloadAxisProof, (), ["before", "source", "after"], []), + (CreatedPayloadAxisProof, (), ["before", "after"], []), + (UnprovedPayloadAxisProof, + tuple(_compiled_pattern(color_to_gray).iter_invocations()), ["before", "after"], ["rejected color_to_gray"]), +]) def test_independent_same_node_validation_capability_composes_both_mro_orders( - capability_first, - proof_type, - args, - expected_visits, - expected_failures, + capability_first, proof_type, args, expected_visits, expected_failures, ): visits, failures = [], [] @@ -313,11 +228,7 @@ def validate_obligation(self, **kwargs): # Same nominal owner in both bases puts the independent capability before # the shared operation in C3, regardless of its position beside the leaf. - bases = ( - (ValidationVisitCapability, proof_type) - if capability_first - else (proof_type, ValidationVisitCapability) - ) + bases = (ValidationVisitCapability, proof_type) if capability_first else (proof_type, ValidationVisitCapability) declared_proof = type("IndependentlyObservedProof", bases, {})(*args) def validate_source(): @@ -326,8 +237,7 @@ def validate_source(): assert declared_proof.validate_obligation( source_validation=validate_source, - failure_message=lambda invocation: "rejected " - + invocation.contract.function_name, + failure_message=lambda invocation: "rejected " + invocation.contract.function_name, failures=failures, ) assert visits == expected_visits @@ -348,25 +258,18 @@ def reject_source(): assert failures == ["exact source missing its declared payload axis"] -def test_color_to_gray_does_not_preserve_consumed_payload_axis_for_later_consumer( - tmp_path, -): +def test_color_to_gray_does_not_preserve_consumed_payload_axis_for_later_consumer(tmp_path): session = _gray_creator_session(tmp_path) consumer = session.plans[1] session.plans[2] = replace( - consumer, - step_index=2, - step_name="RejectSecondScalarSplit", + consumer, step_index=2, step_name="RejectSecondScalarSplit", main_input_dependency=StepInputDependency.step_output( - source_step_index=1, - source_step_scope_id="step-1", + source_step_index=1, source_step_scope_id="step-1", ), ) session.plans[1] = replace(consumer, step_scope_id="step-1") session.context.step_plans = session.plans - with pytest.raises( - ValueError, match="not preserved by producer step 1 callable 'color_to_gray'" - ): + with pytest.raises(ValueError, match="not preserved by producer step 1 callable 'color_to_gray'"): PipelineCompiler.validate_payload_axis_requirements(session) @@ -380,15 +283,9 @@ def test_scalar_projection_modes_keep_pixels_mask_and_original_source(image_type expected = split_color_to_gray(source, image_type, (0, 1)) contract = CallableContract.from_callable(color_to_gray) result = CellProfilerFunctionContractExecutor().execute( - contract, - contract.resolve_canonical_raw_callable(), - source, - { - "mode": ColorToGrayMode.SPLIT, - "image_type": image_type, - "channel_indices": (0, 1), - "contributions": (1.0, 1.0), - }, + contract, contract.resolve_canonical_raw_callable(), source, + {"mode": ColorToGrayMode.SPLIT, "image_type": image_type, + "channel_indices": (0, 1), "contributions": (1.0, 1.0)}, execution_mode=NaturalExecution, ) for output, expected_pixels in zip(result.slices, expected, strict=True): @@ -397,10 +294,7 @@ def test_scalar_projection_modes_keep_pixels_mask_and_original_source(image_type output_metadata = output.metadata assert output_metadata.axis_position(ColourAxis) is None assert output_metadata.source_path == metadata.source_path - assert ( - output_metadata.source_component_metadata - == metadata.source_component_metadata - ) + assert output_metadata.source_component_metadata == metadata.source_component_metadata assert output_metadata.source_image_names == metadata.source_image_names assert output_metadata.source_voxel_spacing == metadata.source_voxel_spacing assert output_metadata.source_spatial_domain == metadata.source_spatial_domain diff --git a/tests/unit/test_grid_addressed_axis.py b/tests/unit/test_grid_addressed_axis.py index 8db3b47487..678d20f49b 100644 --- a/tests/unit/test_grid_addressed_axis.py +++ b/tests/unit/test_grid_addressed_axis.py @@ -67,10 +67,7 @@ def test_component_filters_validate_against_the_family_and_match_generically() - assert filters.matches({"well": "B02", "channel": 1, "site": 3}) assert not filters.matches({"well": "C03", "channel": 1}) assert not filters.matches({"channel": 1}) - assert ( - ComponentFilters.from_assignments(["well=A01", "well=B02", "channel=1"]) - == filters - ) + assert ComponentFilters.from_assignments(["well=A01", "well=B02", "channel=1"]) == filters assert ComponentFilters().matches({}) with pytest.raises(ValueError, match="unknown axes"): ComponentFilters.from_mapping({"plot": "1"}) diff --git a/tests/unit/test_image_plane_contracts.py b/tests/unit/test_image_plane_contracts.py index ce7e5f4511..033419b84f 100644 --- a/tests/unit/test_image_plane_contracts.py +++ b/tests/unit/test_image_plane_contracts.py @@ -1,14 +1,7 @@ import numpy as np import pytest -from openhcs.core.aligned_image_payload import ( - AlignedImageStack, - AlignedImageSliceContext, - ImageOutputBundle, - compose_aligned_image_payload, - pack_aligned_image_outputs, - stack_image_payloads, -) +from openhcs.core.aligned_image_payload import (AlignedImageStack, AlignedImageSliceContext, ImageOutputBundle, compose_aligned_image_payload, pack_aligned_image_outputs, stack_image_payloads) from openhcs.core.memory import ( MEMORY_TYPE_NUMPY, stack_runtime_slices, @@ -83,12 +76,9 @@ def test_aligned_image_stack_nominally_declares_runtime_alignment() -> None: ) payload = AlignedImageStack(slices) - assert ( - tuple( - member.data for member in RuntimeSliceProjection.alignment_slices(payload) - ) - == slices - ) + assert tuple( + member.data for member in RuntimeSliceProjection.alignment_slices(payload) + ) == slices def test_runtime_plane_projection_requires_matching_payload_axis() -> None: @@ -192,10 +182,9 @@ def test_scalar_mask_projection_requires_metadata_owner_to_declare_plane_axis( else: metadata.payload_with(data, mask).slice_payload(data, 0) - assert ( - ImagePayloadSliceProjector(mask=None, metadata=metadata).mask_for_slice(data, 0) - is None - ) + assert ImagePayloadSliceProjector(mask=None, metadata=metadata).mask_for_slice( + data, 0 + ) is None @pytest.mark.parametrize("batch", (False, True)) @@ -212,9 +201,7 @@ def test_slice_projection_keeps_fresh_source_metadata_across_calls(batch: bool) def project(): if batch: return projector.payloads_for_slices(tuple(data))[1] - return payload.slice_payload( - data[1], 1, plane_axis=RuntimePlaneAxis.RUNTIME_SLICE - ) + return payload.slice_payload(data[1], 1, plane_axis=RuntimePlaneAxis.RUNTIME_SLICE) first = project() metadata.source_component_metadata = {"well": "B02"} @@ -288,7 +275,9 @@ def test_image_bundle_composition_declares_source_binding_axis() -> None: composition = compose_aligned_image_payload("ImageMath", (first, second)) assert composition.execution_mode is FullStackExecution - assert composition.payload.metadata.plane_axis is (RuntimePlaneAxis.SOURCE_BINDING) + assert composition.payload.metadata.plane_axis is ( + RuntimePlaneAxis.SOURCE_BINDING + ) assert composition.payload.data.shape == (2, 4, 5) @@ -301,7 +290,9 @@ def test_single_runtime_slice_payload_is_not_repacked_as_source_binding() -> Non assert composition.execution_mode is NaturalExecution assert composition.payload is payload - assert composition.payload.metadata.plane_axis is (RuntimePlaneAxis.RUNTIME_SLICE) + assert composition.payload.metadata.plane_axis is ( + RuntimePlaneAxis.RUNTIME_SLICE + ) def test_composed_runtime_slice_projection_ignores_outer_file_cardinality() -> None: @@ -373,7 +364,9 @@ def test_single_aligned_payload_is_preserved_without_rebundling() -> None: composition = compose_aligned_image_payload("MeasureImage", (payload,)) - assert composition.execution_mode is (AlignedStackExecution) + assert composition.execution_mode is ( + AlignedStackExecution + ) assert composition.payload is payload @@ -393,7 +386,9 @@ def test_declared_output_contexts_preserve_aligned_multi_image_payloads() -> Non slice_contexts=contexts, ) - assert composition.execution_mode is (AlignedStackExecution) + assert composition.execution_mode is ( + AlignedStackExecution + ) assert isinstance(composition.payload, ImageOutputBundle) assert tuple(member.data for member in composition.payload.slices) == payloads assert composition.payload.slice_contexts == contexts diff --git a/tests/unit/test_implicit_main_flow_compilation.py b/tests/unit/test_implicit_main_flow_compilation.py index c35faa2972..91fb138d72 100644 --- a/tests/unit/test_implicit_main_flow_compilation.py +++ b/tests/unit/test_implicit_main_flow_compilation.py @@ -95,6 +95,6 @@ def test_native_grouped_main_flow_compiles_into_cellprofiler_consumer(tmp_path): assert threshold_plan.compiled_function_pattern is not None invocation = next(threshold_plan.compiled_function_pattern.iter_invocations()) (edge,) = invocation.artifact_input_edges - assert edge.main_flow_projection is not None + assert (edge.main_flow_projection is not None) assert edge.storage_plan is None assert invocation.contract.execution_scope is FunctionStepExecutionScope.AXIS diff --git a/tests/unit/test_installed_mcp_demo.py b/tests/unit/test_installed_mcp_demo.py index 4ba2c1803a..4625f61f71 100644 --- a/tests/unit/test_installed_mcp_demo.py +++ b/tests/unit/test_installed_mcp_demo.py @@ -74,9 +74,7 @@ def _execution(capability, payload, *, returncode: int = 0) -> McpDevCommandExec response = McpDevToolBatchResponse( server=McpDevServerIdentity(command="python", module="openhcs.mcp"), results=( - McpDevToolResult( - tool=capability.name, mcp_error=False, payloads=(payload,) - ), + McpDevToolResult(tool=capability.name, mcp_error=False, payloads=(payload,)), ), ) return McpDevCommandExecution( @@ -337,23 +335,11 @@ def test_command_payload_rejects_a_missing_declared_result() -> None: def _dataset_row(root: Path, terminal_status: str | None) -> DatasetRowState: return DatasetRowState( - scope_id=str(root), - name=root.name, - root=str(root), - pipeline_path=None, - selected=True, - initialized=True, - compiled=True, - init_pending=False, - compile_pending=False, - execution_active=False, - status_prefix="", - orchestrator_state=None, - execution_id=None, - terminal_status=terminal_status, - runtime_state=None, - runtime_percent=None, - queue_position=None, + scope_id=str(root), name=root.name, root=str(root), pipeline_path=None, + selected=True, initialized=True, compiled=True, init_pending=False, + compile_pending=False, execution_active=False, status_prefix="", + orchestrator_state=None, execution_id=None, terminal_status=terminal_status, + runtime_state=None, runtime_percent=None, queue_position=None, ) @@ -373,9 +359,7 @@ def test_execute_pipeline_runs_the_session_journey_and_reads_its_row( def fake_run_mcp(client, argv, *, capability, payload_type, timeout_seconds): calls.append((client, tuple(argv), capability, payload_type, timeout_seconds)) return DatasetListState( - rows=rows, - selected_scope_ids=(str(plate),), - execution_state="idle", + rows=rows, selected_scope_ids=(str(plate),), execution_state="idle", ) monkeypatch.setattr(installed_demo, "_run_mcp", fake_run_mcp) diff --git a/tests/unit/test_invocation_contract_provider.py b/tests/unit/test_invocation_contract_provider.py index 98be7c4e0c..879d721a1b 100644 --- a/tests/unit/test_invocation_contract_provider.py +++ b/tests/unit/test_invocation_contract_provider.py @@ -1164,9 +1164,7 @@ def test_cellprofiler_provider_rejects_under_specified_one_image_align() -> None ValueError, match=r"Align.*cannot reconstruct an exact module block", ): - CellProfilerInvocationContractProviderFactory.provider_for_pipeline( - session.pipeline - ) + CellProfilerInvocationContractProviderFactory.provider_for_pipeline(session.pipeline) def test_cellprofiler_invocation_contract_allows_source_binding_supersets() -> None: diff --git a/tests/unit/test_invocation_input_source_context_identity.py b/tests/unit/test_invocation_input_source_context_identity.py index fe95048984..afee352f86 100644 --- a/tests/unit/test_invocation_input_source_context_identity.py +++ b/tests/unit/test_invocation_input_source_context_identity.py @@ -94,10 +94,7 @@ def test_compiled_input_edges_own_exact_main_flow_membership() -> None: assert tuple(edge.spec for edge in edges) == (plate_template, combined_image) assert tuple(edge.storage_plan for edge in edges) == (None, None) - assert tuple((edge.main_flow_projection is not None) for edge in edges) == ( - False, - True, - ) + assert tuple((edge.main_flow_projection is not None) for edge in edges) == (False, True) assert tuple(edge.main_flow_projection for edge in edges) == ( None, MainFlowInputProjection.COMPLETE_PAYLOAD, @@ -243,8 +240,12 @@ def source_artifact_payload(_request, ref): np.testing.assert_array_equal(source_request.data, 7.0) np.testing.assert_array_equal(main_flow_request.data, 11.0) - assert source_request.metadata.source_image_names == (plate_template.name,) - assert main_flow_request.metadata.source_image_names == (combined_image.name,) + assert source_request.metadata.source_image_names == ( + plate_template.name, + ) + assert main_flow_request.metadata.source_image_names == ( + combined_image.name, + ) def test_storage_backed_input_keeps_exact_runtime_authority() -> None: @@ -293,7 +294,9 @@ def test_storage_backed_input_keeps_exact_runtime_authority() -> None: runtime_request = request.artifact_value(edge) np.testing.assert_array_equal(runtime_request.data, 13.0) - assert runtime_request.metadata.source_image_names == (combined_image.name,) + assert runtime_request.metadata.source_image_names == ( + combined_image.name, + ) def test_compiled_producer_edge_overrides_matching_source_binding() -> None: @@ -345,4 +348,6 @@ def test_compiled_producer_edge_overrides_matching_source_binding() -> None: runtime_request = request.artifact_value(edge) np.testing.assert_array_equal(runtime_request.data, 19.0) - assert runtime_request.metadata.source_image_names == (combined_image.name,) + assert runtime_request.metadata.source_image_names == ( + combined_image.name, + ) diff --git a/tests/unit/test_l3_zmqruntime_vocabulary_guards.py b/tests/unit/test_l3_zmqruntime_vocabulary_guards.py index 4d4dbe5914..cdec3ed7c1 100644 --- a/tests/unit/test_l3_zmqruntime_vocabulary_guards.py +++ b/tests/unit/test_l3_zmqruntime_vocabulary_guards.py @@ -35,20 +35,14 @@ def test_messages_inherit_their_codec_from_the_declared_bases() -> None: for node in tree.body if isinstance(node, ast.ClassDef) and node.name not in bases for member in node.body - if isinstance(member, ast.FunctionDef) - and member.name in {"to_dict", "from_dict"} + if isinstance(member, ast.FunctionDef) and member.name in {"to_dict", "from_dict"} ] assert offenders == [] def test_launch_policy_lives_in_zmqruntime() -> None: assert not ( - REPO - / "external" - / "pyqt-reactive" - / "src" - / "pyqt_reactive" - / "process_launch.py" + REPO / "external" / "pyqt-reactive" / "src" / "pyqt_reactive" / "process_launch.py" ).exists() importers = [ str(path.relative_to(REPO)) diff --git a/tests/unit/test_library_registry_discovery.py b/tests/unit/test_library_registry_discovery.py index 73fc7c4082..a0f3403be1 100644 --- a/tests/unit/test_library_registry_discovery.py +++ b/tests/unit/test_library_registry_discovery.py @@ -109,11 +109,7 @@ def test_native_catalog_excludes_modules_whose_distribution_markers_exclude_here @pytest.mark.parametrize( "platform_system, platform_machine, excluded", - [ - ("Darwin", "x86_64", True), - ("Darwin", "arm64", False), - ("Linux", "x86_64", False), - ], + [("Darwin", "x86_64", True), ("Darwin", "arm64", False), ("Linux", "x86_64", False)], ) def test_basicpy_catalog_module_follows_the_package_basicpy_marker( monkeypatch, platform_system, platform_machine, excluded diff --git a/tests/unit/test_logical_image_output_snapshot.py b/tests/unit/test_logical_image_output_snapshot.py index 1b335b2b62..27c12c578b 100644 --- a/tests/unit/test_logical_image_output_snapshot.py +++ b/tests/unit/test_logical_image_output_snapshot.py @@ -1,5 +1,5 @@ -from tests.unit.saved_output_dialect import SAVED_OUTPUT_DIALECT +from tests.unit.saved_output_dialect import SAVED_OUTPUT_DIALECT """Typed image sets retain physical coverage across volume/plane exports.""" from dataclasses import replace @@ -38,11 +38,7 @@ from openhcs.core.payload_axes import PayloadAxes Z_STACK = SourceImageSetIdentityPolicy(frozenset({Microscopy.ZIndex})) -EXACT = RuntimeEquivalencePolicy( - image_abs_tolerance=0, - image_rel_tolerance=0, - measurement_dialect=SAVED_OUTPUT_DIALECT, -) +EXACT = RuntimeEquivalencePolicy(image_abs_tolerance=0, image_rel_tolerance=0, measurement_dialect=SAVED_OUTPUT_DIALECT) def _write_metadata(root, projections): @@ -77,15 +73,7 @@ def exported_volume(tmp_path): for plane_index, plane in enumerate(volume): path = candidate / f"opaque-{plane_index}.tiff" imageio.imwrite(path, plane) - address = OpenHCSPlaneAddress( - ( - (Microscopy.Well, "A01"), - (Microscopy.Site, "1"), - (Microscopy.Channel, "2"), - (Microscopy.ZIndex, plane_index + 1), - (Microscopy.Timepoint, "1"), - ) - ) + address = OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, "1"), (Microscopy.Channel, "2"), (Microscopy.ZIndex, plane_index + 1), (Microscopy.Timepoint, "1"))) projections.append( SourceArtifactProjection( address=address, diff --git a/tests/unit/test_matched_cellprofiler_batch.py b/tests/unit/test_matched_cellprofiler_batch.py index 647eb158d5..c9c7a8bef2 100644 --- a/tests/unit/test_matched_cellprofiler_batch.py +++ b/tests/unit/test_matched_cellprofiler_batch.py @@ -551,12 +551,9 @@ def reject_read_bytes(_path: Path) -> bytes: monkeypatch.setattr(Path, "read_bytes", reject_read_bytes) expected = hashlib.sha256(b"microscopy pixels").hexdigest() - assert ( - _source_input_inventory( - NativeCellProfilerSelectedSourceUniverse((staged / "image.tif",)) - )[0]["sha256"] - == expected - ) + assert _source_input_inventory( + NativeCellProfilerSelectedSourceUniverse((staged / "image.tif",)) + )[0]["sha256"] == expected assert _output_inventory(tmp_path, frozenset({source}))[0]["sha256"] == expected diff --git a/tests/unit/test_materialization_batch_outcomes.py b/tests/unit/test_materialization_batch_outcomes.py index 1c4b1bee50..16961b7e02 100644 --- a/tests/unit/test_materialization_batch_outcomes.py +++ b/tests/unit/test_materialization_batch_outcomes.py @@ -174,9 +174,7 @@ def test_worker_exports_actual_step_outcomes_after_resource_release( worker_slot="worker-0", worker_assignments={"worker-0": ("A01",)}, ) - monkeypatch.setattr( - progress_types, "_progress_queue", SimpleNamespace(put=lambda _event: None) - ) + monkeypatch.setattr(progress_types, "_progress_queue", SimpleNamespace(put=lambda _event: None)) def forbidden_reconstruction(*_args, **_kwargs): pytest.fail("Ordinary execution must consume actual step outcomes") @@ -197,9 +195,7 @@ def forbidden_reconstruction(*_args, **_kwargs): assert context.released assert transported.is_success() - assert transported.runtime_observation.contexts[0].outputs.runtime_export_paths == ( - path, - ) + assert transported.runtime_observation.contexts[0].outputs.runtime_export_paths == (path,) assert transported.runtime_observation.contexts[0].records == () assert RuntimeExportObservation.from_runtime_observations( (transported.runtime_observation,) @@ -213,18 +209,10 @@ def test_parent_and_worker_exports_share_existing_observation_authority(tmp_path for path in (worker_path, plate_path, unrelated): path.write_text("Area\n42\n") worker = RuntimeExecutionObservation( - contexts=( - RuntimeContextObservation( - "A01", (), StepExecutionObservation({}, (worker_path,)) - ), - ) + contexts=(RuntimeContextObservation("A01", (), StepExecutionObservation({}, (worker_path,))),) ) parent = RuntimeExecutionObservation( - contexts=( - RuntimeContextObservation( - "A01", (), StepExecutionObservation({}, (plate_path,)) - ), - ) + contexts=(RuntimeContextObservation("A01", (), StepExecutionObservation({}, (plate_path,))),) ) results = BundleExecutionResults( {"A01": ExecutionResult.success("A01", runtime_observation=worker)}, diff --git a/tests/unit/test_materialization_flag_planner.py b/tests/unit/test_materialization_flag_planner.py index 297c65ed8d..e2302a2409 100644 --- a/tests/unit/test_materialization_flag_planner.py +++ b/tests/unit/test_materialization_flag_planner.py @@ -249,9 +249,7 @@ def new_submission(): def test_unused_materialization_auto_does_not_query_source(): - config = _pipeline_config( - path_planning_config=PathPlanningConfig(partition_filter=0) - ) + config = _pipeline_config(path_planning_config=PathPlanningConfig(partition_filter=0)) config.vfs_config = VFSConfig( read_backend=Backend.DISK, materialization_backend=MaterializationBackend.AUTO, diff --git a/tests/unit/test_mcp_release_workflow.py b/tests/unit/test_mcp_release_workflow.py index cc1241d6a1..8fb58f3cae 100644 --- a/tests/unit/test_mcp_release_workflow.py +++ b/tests/unit/test_mcp_release_workflow.py @@ -385,7 +385,9 @@ def test_pypi_wheel_smoke_uses_the_canonical_pipeline_document_boundary(): smoke = headless_step["run"] assert "from openhcs.core.config import PipelineConfig" in smoke - assert "from openhcs.core.pipeline_document import PipelineDocumentCodec" in smoke + assert ( + "from openhcs.core.pipeline_document import PipelineDocumentCodec" in smoke + ) assert "from openhcs.core.steps.function_step import FunctionStep" in smoke assert "PipelineDocumentCodec.from_values(" in smoke assert "pipeline_config=PipelineConfig()" in smoke diff --git a/tests/unit/test_measurement_dialect.py b/tests/unit/test_measurement_dialect.py index 4a7c5fa70f..b9f57eb61a 100644 --- a/tests/unit/test_measurement_dialect.py +++ b/tests/unit/test_measurement_dialect.py @@ -64,26 +64,26 @@ def survey() -> Iterator[type[Survey]]: def test_a_family_without_its_own_dialect_writes_kernel_names() -> None: assert type(MeasurementDialect.for_family(Microscopy)) is PlainMeasurementDialect - assert _render_csv_rows( - ({"slice_index": 0, "count": 3},), ("slice_index", "count") - ) == ("slice_index,count\r\n0,3\r\n") + assert _render_csv_rows(({"slice_index": 0, "count": 3},), ("slice_index", "count")) == ( + "slice_index,count\r\n0,3\r\n" + ) def test_the_active_family_names_written_rows_through_its_dialect(survey) -> None: assert type(MeasurementDialect.for_active_family()) is SurveyDialect - assert _render_csv_rows( - ({"slice_index": 0, "count": 3},), ("slice_index", "count") - ) == ("scan_index,count\r\n0,3\r\n") + assert _render_csv_rows(({"slice_index": 0, "count": 3},), ("slice_index", "count")) == ( + "scan_index,count\r\n0,3\r\n" + ) def test_each_dialect_spells_the_unqualified_sample() -> None: names = MeasurementDialect.unqualified_sample_names() assert {"", "sample", "image", "scan"} <= names for name in ("sample", "Image", "scan"): - assert ( - MeasurementSubject(MeasurementScope.SAMPLE, name).source_image_name is None - ) - assert MeasurementSubject(MeasurementScope.SAMPLE, "DNA").source_image_name == "DNA" + assert MeasurementSubject(MeasurementScope.SAMPLE, name).source_image_name is None + assert ( + MeasurementSubject(MeasurementScope.SAMPLE, "DNA").source_image_name == "DNA" + ) def test_cellprofiler_spells_scopes_and_sample_numbers_in_its_own_dialect() -> None: diff --git a/tests/unit/test_measureobjectintensitydistribution.py b/tests/unit/test_measureobjectintensitydistribution.py index d40c3fa025..ee78d01a33 100644 --- a/tests/unit/test_measureobjectintensitydistribution.py +++ b/tests/unit/test_measureobjectintensitydistribution.py @@ -9,9 +9,7 @@ MeasurementFeatureQuery, MeasurementFeatureValueIndex, ) -from openhcs.core.pipeline.function_contracts import ( - object_label_input_execution_mode_from_callable, -) +from openhcs.core.pipeline.function_contracts import object_label_input_execution_mode_from_callable from openhcs.core.measurement_row_materialization import columnar_row_values from openhcs.core.runtime_image_values import ImagePayloadMetadata from openhcs.core.runtime_measurements import ( diff --git a/tests/unit/test_named_checkpoint_publication.py b/tests/unit/test_named_checkpoint_publication.py index fe720be850..5561e66be1 100644 --- a/tests/unit/test_named_checkpoint_publication.py +++ b/tests/unit/test_named_checkpoint_publication.py @@ -13,26 +13,18 @@ from tests.unit.test_artifact_publication_journey import _plate, progress_events from openhcs.core.artifacts import ( - QaCheckpoint, - ImageArtifactType, - MainFlowPlaneProjectionOutputSpec, + QaCheckpoint, ImageArtifactType, MainFlowPlaneProjectionOutputSpec, MainFlowStackOutputSpec, ) from openhcs.core.config import ( - LazyPathPlanningConfig, - LazyProcessingConfig, - LazyStepMaterializationConfig, + LazyPathPlanningConfig, LazyProcessingConfig, LazyStepMaterializationConfig, PipelineConfig, ) from openhcs.core.memory import numpy as numpy_decorator from openhcs.core.orchestrator.orchestrator import PipelineOrchestrator -from openhcs.core.orchestrator.execution_result import ( - RuntimeContextObservation, - RuntimeExecutionObservation, -) +from openhcs.core.orchestrator.execution_result import RuntimeContextObservation, RuntimeExecutionObservation from openhcs.core.pipeline.function_contracts import ( - artifact_outputs, - required_axis_roles, + artifact_outputs, required_axis_roles, ) from openhcs.core.projected_image_output import SourceProjectedImageOutput from openhcs.core.runtime_image_values import ImagePayloadMetadata @@ -46,14 +38,13 @@ Pure3DContract, ) from openhcs.processing.materialization import ( - ImageFileOptions, - MaterializationSpec, - MaterializedFilenameIdentity, + ImageFileOptions, MaterializationSpec, MaterializedFilenameIdentity, ) from openhcs.core.axes import TileAxis from openhcs.domains.microscopy.axes import Microscopy + @dataclass(frozen=True) class SyntheticAggregateField(SourceProjectedImageOutput): data: np.ndarray @@ -64,22 +55,15 @@ def with_data(self, data): def resolve_source_context(self, source, projection): assert projection is not None and projection.plane_index is None assert projection.axis_size == 24 - return source.metadata.collapse_leading_plane_axis().payload_with( - self.data, None - ) + return source.metadata.collapse_leading_plane_axis().payload_with(self.data, None) def _field(name): return MainFlowPlaneProjectionOutputSpec.output( - name, - ImageArtifactType, - sidecar_role=QaCheckpoint, - materialization=MaterializationSpec( - ImageFileOptions( - filename_suffix=".tif", - filename_identity=MaterializedFilenameIdentity.ARTIFACT_NAME, - ) - ), + name, ImageArtifactType, sidecar_role=QaCheckpoint, + materialization=MaterializationSpec(ImageFileOptions( + filename_suffix=".tif", filename_identity=MaterializedFilenameIdentity.ARTIFACT_NAME, + )), ) @@ -87,8 +71,7 @@ def _field(name): @required_axis_roles(TileAxis) @artifact_outputs( MainFlowStackOutputSpec.output("corrected", ImageArtifactType), - _field("flat"), - _field("dark"), + _field("flat"), _field("dark"), ) def synthetic_three_outputs(image): assert image.shape == (24, 8, 8) @@ -100,35 +83,25 @@ def synthetic_three_outputs(image): @pytest.mark.parametrize("main_filter", (0, 1)) -def test_named_and_ordinary_checkpoint_inventory_owns_every_address( - tmp_path, main_filter -): +def test_named_and_ordinary_checkpoint_inventory_owns_every_address(tmp_path, main_filter): plate = _plate(tmp_path / "source", site_count=24) config = PipelineConfig( - path_planning_config=LazyPathPlanningConfig( - output_dir_suffix="_out", partition_filter=main_filter - ), - num_workers=1, - use_threading=True, + path_planning_config=LazyPathPlanningConfig(output_dir_suffix="_out", partition_filter=main_filter), + num_workers=1, use_threading=True, ) orchestrator = PipelineOrchestrator(plate, pipeline_config=config).initialize() step = FunctionStep( func=synthetic_three_outputs, processing_config=LazyProcessingConfig( - group_by=Microscopy.Channel, - variable_components=[Microscopy.Site], + group_by=Microscopy.Channel, variable_components=[Microscopy.Site], ), step_materialization_config=LazyStepMaterializationConfig(enabled=True), ) bundle = orchestrator.compile_pipelines([step]) - observations = RuntimeExecutionObservation( - contexts=tuple( - RuntimeContextObservation( - context_key=key, records=(), outputs=step.process(context, 0) - ) - for key, context in bundle.runtime_contexts.items() - ) - ) + observations = RuntimeExecutionObservation(contexts=tuple( + RuntimeContextObservation(context_key=key, records=(), outputs=step.process(context, 0)) + for key, context in bundle.runtime_contexts.items() + )) OpenHCSMetadataTarget.finalize_completed_dataset( bundle.runtime_contexts, runtime_observations=(observations,) ) @@ -137,14 +110,10 @@ def test_named_and_ordinary_checkpoint_inventory_owns_every_address( reopened = VirtualWorkspaceSourceProjection.from_openhcs_metadata(root, metadata) for target, count in (("checkpoints", 48), ("checkpoints_results", 2)): entry = metadata["subdirectories"][target] - inventory = { - str(path.relative_to(root)) for path in (root / target).glob("*.tif") - } + inventory = {str(path.relative_to(root)) for path in (root / target).glob("*.tif")} assert len(inventory) == count assert set(entry["image_files"]) == inventory - assert { - item["virtual_path"] for item in entry["source_projection"] - } == inventory + assert {item["virtual_path"] for item in entry["source_projection"]} == inventory for item in entry["source_projection"]: retained = ImagePayloadMetadata.from_mapping(item["image_metadata"]) assert retained.source_voxel_spacing == SourceVoxelSpacing((0.65, 0.65)) @@ -154,23 +123,13 @@ def test_named_and_ordinary_checkpoint_inventory_owns_every_address( projection for item in checkpoints if isinstance( - projection := reopened.source_projections_by_virtual_path[ - item["virtual_path"] - ], + projection := reopened.source_projections_by_virtual_path[item["virtual_path"]], SourceArtifactProjection, ) ] assert len(ordinary) == len(checkpoints) - len(ordinary) == 24 assert {projection.address for projection in ordinary} == { - OpenHCSPlaneAddress( - ( - (Microscopy.Well, "A01"), - (Microscopy.Site, site), - (Microscopy.Channel, 1), - (Microscopy.ZIndex, 1), - (Microscopy.Timepoint, 1), - ) - ) + OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, site), (Microscopy.Channel, 1), (Microscopy.ZIndex, 1), (Microscopy.Timepoint, 1))) for site in range(1, 25) } for item in checkpoints: @@ -192,19 +151,9 @@ def test_named_and_ordinary_checkpoint_inventory_owns_every_address( assert all_inventory.issubset(reopened.source_projections_by_virtual_path) if main_filter: assert len(metadata["subdirectories"]["images"]["image_files"]) == 24 - assert ( - PipelineOrchestrator(root) - .initialize() - .microscope_handler.get_pixel_size(root) - == 0.65 - ) + assert PipelineOrchestrator(root).initialize().microscope_handler.get_pixel_size(root) == 0.65 # Keep fail-closed reconciliation: an actually unowned image is still an # error. This is not a missing-address permissive fallback. - tifffile.imwrite( - root / "checkpoints" / "unaddressed.tif", np.zeros((8, 8), dtype=np.uint16) - ) - with pytest.raises( - MetadataWriteError, - match="Saved images lack typed produced addresses.*unaddressed", - ): + tifffile.imwrite(root / "checkpoints" / "unaddressed.tif", np.zeros((8, 8), dtype=np.uint16)) + with pytest.raises(MetadataWriteError, match="Saved images lack typed produced addresses.*unaddressed"): OpenHCSMetadataTarget.finalize_completed_dataset(bundle.runtime_contexts) diff --git a/tests/unit/test_napari_accepted_work_settlement.py b/tests/unit/test_napari_accepted_work_settlement.py index 53d5fd72a8..958d9b25b5 100644 --- a/tests/unit/test_napari_accepted_work_settlement.py +++ b/tests/unit/test_napari_accepted_work_settlement.py @@ -105,9 +105,7 @@ def wire_batch(item): return json.dumps( { "type": "batch", - "images": [ - {**row, "producer_identity": producer.to_payload()} for row in items - ], + "images": [{**row, "producer_identity": producer.to_payload()} for row in items], "display_config": ViewerBatchDisplayPayload( component_modes=config.component_modes(), component_order=config.COMPONENT_ORDER, @@ -184,13 +182,9 @@ def test_saved_roi_reopen_reports_pre_route_failure_and_recovers( # A new accepted batch starts the next existing settlement cycle. The # previous failed cycle must not become a permanent unrelated failure. assert wire_receiver(image_item())["status"] == "success" - admitted = ViewerSettleProgress.from_response( - ViewerControlResponse( - NapariControlAction.for_message_type("settle").transport_thread_response( - receiver, {} - ) - ) - ) + admitted = ViewerSettleProgress.from_response(ViewerControlResponse( + NapariControlAction.for_message_type("settle").transport_thread_response(receiver, {}) + )) assert admitted.phase is ViewerSettlePhase.RUNNING assert admitted.total_update_count == 0 receiver.process_accepted_stream_messages() @@ -211,23 +205,15 @@ def test_previous_complete_cannot_settle_new_accepted_work(receiver): == "success" ) assert not receiver.accepted_stream_batches.empty() - admitted = ViewerSettleProgress.from_response( - ViewerControlResponse( - NapariControlAction.for_message_type("settle").transport_thread_response( - receiver, {} - ) - ) - ) + admitted = ViewerSettleProgress.from_response(ViewerControlResponse( + NapariControlAction.for_message_type("settle").transport_thread_response(receiver, {}) + )) assert admitted.phase is ViewerSettlePhase.RUNNING assert admitted.total_update_count == 0 def test_initial_settlement_is_observable_before_qt_intake_and_completes( - native_opengl, - receiver, - wire_receiver, - qtbot, - tmp_path, + native_opengl, receiver, wire_receiver, qtbot, tmp_path, ): """Use actual transport, native layers and Qt service, without delayed mocks.""" from qtpy.QtCore import QTimer @@ -253,22 +239,11 @@ def test_initial_settlement_is_observable_before_qt_intake_and_completes( items = [] for path in sorted(Path(directory).glob("*_rois.roi.zip")): parts = path.name.split("_") - metadata = { - **image_item()["metadata"], - "well": parts[0], - "site": int(parts[1][1:]), - "channel": int(parts[2][1:]), - } - items.append( - { - "path": str(path), - "data_type": "shapes", - "shapes": NapariROIConverter.rois_to_shapes( - load_rois_from_zip(path) - ), - "metadata": metadata, - } - ) + metadata = {**image_item()["metadata"], "well": parts[0], + "site": int(parts[1][1:]), "channel": int(parts[2][1:])} + items.append({"path": str(path), "data_type": "shapes", + "shapes": NapariROIConverter.rois_to_shapes(load_rois_from_zip(path)), + "metadata": metadata}) assert items item = items assert wire_receiver(item)["status"] == "success" @@ -281,14 +256,10 @@ def test_initial_settlement_is_observable_before_qt_intake_and_completes( control.setsockopt(zmq.RCVTIMEO, 2000) receiver.control_transport_pump.start() control_port = get_control_port(receiver.port, receiver.config) - control.connect( - get_zmq_transport_url( - control_port, - host="localhost", - mode=receiver.transport_mode, - config=receiver.config, - ) - ) + control.connect(get_zmq_transport_url( + control_port, host="localhost", mode=receiver.transport_mode, + config=receiver.config, + )) service = QTimer() service.timeout.connect(receiver.process_accepted_stream_messages) service.timeout.connect(receiver.process_messages) @@ -296,104 +267,69 @@ def test_initial_settlement_is_observable_before_qt_intake_and_completes( filemanager=FileManager({}), runtime_config=StreamingViewerRuntimeConfig( transport_endpoint=ViewerTransportEndpoint( - port=receiver.port, - host="localhost", - transport_mode=receiver.transport_mode, - ), - persistent=False, - viewer_family=NapariViewer, + port=receiver.port, host="localhost", transport_mode=receiver.transport_mode, + ), persistent=False, viewer_family=NapariViewer, ), ) client.lifecycle_state.mark_connected_external() observations = [] client_worker = ThreadPoolExecutor(max_workers=1) - late_observer = ( - os.environ.get("OPENHCS_SETTLEMENT_QUALIFICATION_LATE_OBSERVER") == "true" - ) + late_observer = os.environ.get("OPENHCS_SETTLEMENT_QUALIFICATION_LATE_OBSERVER") == "true" client_started = [] delivery = None control_observation = [] - inspect_pending_control = ( - os.environ.get("OPENHCS_SETTLEMENT_QUALIFICATION_CONTROL") == "true" - ) + inspect_pending_control = os.environ.get("OPENHCS_SETTLEMENT_QUALIFICATION_CONTROL") == "true" def deliver(): if late_observer or inspect_pending_control: deadline = time.monotonic() + 90 while True: progress = receiver.layer_route_state.existing_settlement_progress() - if ( - progress is not None - and progress.active_route is not None - and progress.active_route_work_unit_active - and ( - (inspect_pending_control and receiver.layer_route_state.layers) - or (late_observer and "channel_2" in progress.active_route) - ) - ): + if (progress is not None and progress.active_route is not None + and progress.active_route_work_unit_active + and ((inspect_pending_control and receiver.layer_route_state.layers) + or (late_observer and "channel_2" in progress.active_route))): break if late_observer and time.monotonic() >= deadline: raise AssertionError("No real channel-2 native work was observed.") time.sleep(0.001) if inspect_pending_control: from openhcs.runtime.viewer_controls import ViewerNavigationControlOptions - - route = next( - key for key in receiver.layer_route_state.layers if "channel_1" in key - ) + route = next(key for key in receiver.layer_route_state.layers if "channel_1" in key) native_layer = receiver.layer_route_state.layers[route] started = time.perf_counter() try: reply = ViewerControlMessageRequest( - client.runtime_endpoint, - "navigate", - ViewerNavigationControlOptions(route_key=route, visible=False), - timeout=0.1, + client.runtime_endpoint, "navigate", + ViewerNavigationControlOptions(route_key=route, visible=False), timeout=0.1, ).send() assert not reply.succeeded() except zmq.Again: pass # The caller's original observation budget expired. expired_after = time.perf_counter() - started started = time.perf_counter() - progress = ViewerSettleProgress.from_response( - ViewerControlMessageRequest( - client.runtime_endpoint, - "settle", - timeout=2, - ).send() - ) - control_observation.append( - (native_layer, expired_after, time.perf_counter() - started, progress) - ) + progress = ViewerSettleProgress.from_response(ViewerControlMessageRequest( + client.runtime_endpoint, "settle", timeout=2, + ).send()) + control_observation.append((native_layer, expired_after, time.perf_counter() - started, progress)) assert progress.phase is ViewerSettlePhase.RUNNING client_started.append(time.perf_counter()) return client.settle_viewer_state( - progress_callback=lambda progress: observations.append( - (time.perf_counter(), progress) - ), + progress_callback=lambda progress: observations.append((time.perf_counter(), progress)), ) def observe(): - control.send( - pickle.dumps( - ViewerControlMessageRequest( - client.runtime_endpoint, - "settle", - timeout=2, - ).to_wire_mapping() - ) - ) - return ViewerSettleProgress.from_response( - ViewerControlResponse(pickle.loads(control.recv())) - ) + control.send(pickle.dumps(ViewerControlMessageRequest( + client.runtime_endpoint, "settle", timeout=2, + ).to_wire_mapping())) + return ViewerSettleProgress.from_response(ViewerControlResponse( + pickle.loads(control.recv()) + )) try: started = time.perf_counter() - first = ( - receiver.layer_route_state.existing_settlement_progress() - if late_observer - else observe() - ) + first = (receiver.layer_route_state.existing_settlement_progress() + if late_observer else observe()) elapsed = time.perf_counter() - started assert first.phase is ViewerSettlePhase.RUNNING assert first.completed_update_count == first.total_update_count == 0 @@ -428,46 +364,28 @@ def observe(): assert receiver.accepted_stream_batches.empty() assert receiver.viewer.layers if inspect_pending_control: - native_layer, expired_after, settlement_reply_seconds, pending = ( - control_observation[0] - ) - assert ( - not native_layer.visible - ) # Expiry did not cancel the real Qt mutation. - print( - f"native control expired after {expired_after:.3f}s; subsequent settlement " - f"reply {settlement_reply_seconds:.3f}s; pending={pending}; " - "original navigation applied after observation expiry" - ) - assert ( - final.processed_intake_item_count - == (len(item) if isinstance(item, list) else 1) + 1 - ) - awaiting = [ - (t, p) - for t, p in observations - if p.phase is ViewerSettlePhase.RUNNING and p.total_update_count == 0 - ] + native_layer, expired_after, settlement_reply_seconds, pending = control_observation[0] + assert not native_layer.visible # Expiry did not cancel the real Qt mutation. + print(f"native control expired after {expired_after:.3f}s; subsequent settlement " + f"reply {settlement_reply_seconds:.3f}s; pending={pending}; " + "original navigation applied after observation expiry") + assert final.processed_intake_item_count == (len(item) if isinstance(item, list) else 1) + 1 + awaiting = [(t, p) for t, p in observations + if p.phase is ViewerSettlePhase.RUNNING and p.total_update_count == 0] awaiting_seconds = (awaiting[-1][0] - awaiting[0][0]) if awaiting else 0 receiver.viewer.screenshot(path=str(tmp_path / "native-settlement.png")) - first_reply_seconds = ( - observations[0][0] - client_started[0] if late_observer else elapsed - ) - print( - f"native first settlement {first_reply_seconds:.4f}s; pending intake admitted; " - f"terminal {final.completed_update_count}/{final.total_update_count}; " - f"native layers {len(receiver.viewer.layers)}; archive={archive}; " - f"intake items={final.processed_intake_item_count}; " - f"client delivery={time.perf_counter() - started_delivery:.3f}s; " - f"unbound observations={len(awaiting)} over {awaiting_seconds:.3f}s; " - f"first={awaiting[0][1] if awaiting else None}; " - f"last={awaiting[-1][1] if awaiting else None}" - ) + first_reply_seconds = observations[0][0] - client_started[0] if late_observer else elapsed + print(f"native first settlement {first_reply_seconds:.4f}s; pending intake admitted; " + f"terminal {final.completed_update_count}/{final.total_update_count}; " + f"native layers {len(receiver.viewer.layers)}; archive={archive}; " + f"intake items={final.processed_intake_item_count}; " + f"client delivery={time.perf_counter() - started_delivery:.3f}s; " + f"unbound observations={len(awaiting)} over {awaiting_seconds:.3f}s; " + f"first={awaiting[0][1] if awaiting else None}; " + f"last={awaiting[-1][1] if awaiting else None}") if late_observer: assert observations[0][1].active_route_work_unit_active - assert ( - awaiting_seconds > 30 - ), f"Actual unbound work was only {awaiting_seconds:.3f}s" + assert awaiting_seconds > 30, f"Actual unbound work was only {awaiting_seconds:.3f}s" finally: # Preserve Qt service until the original observation reaches terminal; # never block its callback owner on the observer thread's join. diff --git a/tests/unit/test_napari_roi_manager_integration.py b/tests/unit/test_napari_roi_manager_integration.py index 4a8bff9375..98bea8e64b 100644 --- a/tests/unit/test_napari_roi_manager_integration.py +++ b/tests/unit/test_napari_roi_manager_integration.py @@ -143,9 +143,7 @@ def test_installed_manager_binds_and_selects_native_shapes_without_copying( name="Native ROIs", ) - class ResultSelectionController( - napari_viewer_server.NapariResultSelectionController - ): + class ResultSelectionController(napari_viewer_server.NapariResultSelectionController): @staticmethod def bind(_layer) -> None: return None @@ -191,17 +189,10 @@ def bind(_layer) -> None: @pytest.mark.unit -@pytest.mark.parametrize( - "destroy", - [ - "napari_close", - "manager", - "immediate_manager", - "window", - "manager_then_dock", - "manager_then_window", - ], -) +@pytest.mark.parametrize("destroy", [ + "napari_close", "manager", "immediate_manager", "window", + "manager_then_dock", "manager_then_window", +]) def test_result_surface_native_destruction_releases_and_remounts(qtbot, destroy): napari = pytest.importorskip("napari") module = pytest.importorskip("openhcs.runtime.napari_viewer_server") @@ -221,16 +212,12 @@ def test_result_surface_native_destruction_releases_and_remounts(qtbot, destroy) controller = server.result_selection_controller layer = viewer.add_shapes( [np.array([[0, 0], [0, 4], [4, 4]], dtype=float)], - shape_type="polygon", - features={"name": ["first"]}, - name="First result", + shape_type="polygon", features={"name": ["first"]}, name="First result", ) server.layer_route_state.set_layer("first", layer) server.bind_result_selection_layer(layer) original = server.require_result_selection_surface() - toolbars = viewer.window._qt_window.findChildren( - QToolBar, "openhcs_roi_selection_toolbar" - ) + toolbars = viewer.window._qt_window.findChildren(QToolBar, "openhcs_roi_selection_toolbar") assert len(toolbars) == 1 assert len(controller._selection_observers) == 1 original.manager.hide() @@ -267,9 +254,7 @@ def test_result_surface_native_destruction_releases_and_remounts(qtbot, destroy) next_layer = viewer.add_shapes( [np.array([[8, 8], [8, 12], [12, 12]], dtype=float)], - shape_type="polygon", - features={"name": ["next"]}, - name="Next streamed result", + shape_type="polygon", features={"name": ["next"]}, name="Next streamed result", ) server.layer_route_state.set_layer("next", next_layer) server.bind_result_selection_layer(next_layer) @@ -278,14 +263,7 @@ def test_result_surface_native_destruction_releases_and_remounts(qtbot, destroy) assert replacement.manager._layer is next_layer assert replacement.manager._roilist.rowCount() == 1 assert len(controller._selection_observers) == 1 - assert ( - len( - viewer.window._qt_window.findChildren( - QToolBar, "openhcs_roi_selection_toolbar" - ) - ) - == 1 - ) + assert len(viewer.window._qt_window.findChildren(QToolBar, "openhcs_roi_selection_toolbar")) == 1 get_settings().appearance.highlight.highlight_thickness = 5 controller._notify_selection_observers() viewer.close() diff --git a/tests/unit/test_napari_streaming_handlers.py b/tests/unit/test_napari_streaming_handlers.py index 1dc1c309a6..8b3d124c65 100644 --- a/tests/unit/test_napari_streaming_handlers.py +++ b/tests/unit/test_napari_streaming_handlers.py @@ -108,9 +108,7 @@ def _component_name_metadata(payload, context="test component metadata"): def _component_value_domain(payload, context="test component value domain"): - return ViewerComponentValueDomainPayload.from_wire_mapping( - payload, context=context, declared_axes=STREAM_AXES - ) + return ViewerComponentValueDomainPayload.from_wire_mapping(payload, context=context, declared_axes=STREAM_AXES) def _layer_item( @@ -881,12 +879,7 @@ def add_layer(self, layer): ) def _add_layer(self, layer_type, data, name, kwargs): - layer_attributes = { - "name": name, - "data": data, - "kwargs": kwargs, - "visible": True, - } + layer_attributes = {"name": name, "data": data, "kwargs": kwargs, "visible": True} if layer_type == "shapes": layer_attributes.update( { @@ -1357,12 +1350,7 @@ def test_napari_layer_update_authority_replaces_existing_image_without_global_ax "image", "image-1", "Nuclei", - { - "blending": "additive", - "rgb": False, - "colormap": "gray", - "axis_labels": ("z", "y", "x"), - }, + {"blending": "additive", "rgb": False, "colormap": "gray", "axis_labels": ("z", "y", "x")}, ) assert viewer.calls[1] == ( "image", @@ -1492,9 +1480,7 @@ def test_napari_image_display_stacks_sites_without_rebasing_payload_color_axis() _layer_item( {"site": site}, data=np.full((16, 16, 3), site, dtype=np.uint8), - image_metadata=ImagePayloadMetadata( - axes=PayloadAxes.colour_samples(2) - ), + image_metadata=ImagePayloadMetadata(axes=PayloadAxes.colour_samples(2)), ) for site in (1, 2) ], @@ -1602,10 +1588,7 @@ def test_napari_viewer_model_keeps_declared_plate_axis_sizes_across_routes() -> layout=ViewerComponentLayout.from_parts( component_modes={ **ALL_STACKED, - **{ - component: NapariSlots.Stack.wire_value - for component in full_route_axes - }, + **{component: NapariSlots.Stack.wire_value for component in full_route_axes}, }, component_order=full_route_axes, declared_axes=STREAM_AXES, @@ -1634,10 +1617,7 @@ def test_napari_viewer_model_keeps_declared_plate_axis_sizes_across_routes() -> layout=ViewerComponentLayout.from_parts( component_modes={ **ALL_STACKED, - **{ - component: NapariSlots.Stack.wire_value - for component in reduced_route_axes - }, + **{component: NapariSlots.Stack.wire_value for component in reduced_route_axes}, }, component_order=reduced_route_axes, declared_axes=STREAM_AXES, @@ -1755,8 +1735,7 @@ def test_napari_display_pipeline_rejects_unbound_payload_local_image_axes(): ) assert ( policy.axis_labels( - np.zeros((16, 16, 3)), - ImagePayloadMetadata(axes=PayloadAxes.colour_samples(-1)), + np.zeros((16, 16, 3)), ImagePayloadMetadata(axes=PayloadAxes.colour_samples(-1)) ) == () ) @@ -1806,15 +1785,17 @@ def test_napari_display_pipeline_projects_route_axes_locally(): component_value_domain = _component_value_domain( {"channel": [1, 2, 3, 4, 5], "site": [1, 2]} ) - component_axis_semantics = ViewerComponentAxisSemanticsFactory.from_display_config( - ViewerMappingDisplayConfigInput( - { - "component_modes": {"channel": "stack", "site": "stack"}, - "component_order": ["channel", "site"], - "declared_axes": STREAM_AXES.to_wire(), - } - ), - component_value_domain, + component_axis_semantics = ( + ViewerComponentAxisSemanticsFactory.from_display_config( + ViewerMappingDisplayConfigInput( + { + "component_modes": {"channel": "stack", "site": "stack"}, + "component_order": ["channel", "site"], + "declared_axes": STREAM_AXES.to_wire(), + } + ), + component_value_domain, + ) ) first = pipeline.display_axis_projection( @@ -1855,15 +1836,17 @@ def test_napari_display_pipeline_projects_route_axes_locally(): def test_napari_display_pipeline_uses_declared_domain_independent_of_arrival_order(): napari_viewer_server = pytest.importorskip("openhcs.runtime.napari_viewer_server") pipeline = napari_viewer_server.NapariLayerDisplayPipeline(_FakeNapariServer()) - component_axis_semantics = ViewerComponentAxisSemanticsFactory.from_display_config( - ViewerMappingDisplayConfigInput( - { - "component_modes": {"channel": "stack"}, - "component_order": ["channel"], - "declared_axes": STREAM_AXES.to_wire(), - } - ), - _component_value_domain({"channel": [1, 2, 4]}), + component_axis_semantics = ( + ViewerComponentAxisSemanticsFactory.from_display_config( + ViewerMappingDisplayConfigInput( + { + "component_modes": {"channel": "stack"}, + "component_order": ["channel"], + "declared_axes": STREAM_AXES.to_wire(), + } + ), + _component_value_domain({"channel": [1, 2, 4]}), + ) ) derived = pipeline.display_axis_projection( @@ -2039,77 +2022,51 @@ def test_paired_raw_stack_review_accepts_layer_pipeline_at_same_channel_coordina server.viewer = ViewerModel() pipeline = NapariLayerDisplayPipeline(server) for origin in (("raw", "pipeline") if raw_first else ("pipeline", "raw")): - config = config_type( - **{ - mode_field: ( - NapariDimensionMode.STACK - if origin == stack_origin - else NapariDimensionMode.LAYER - ) - } - ) - producer = ( - StreamProducerIdentity("manual", "image", "raw", "raw") - if origin == "raw" - else None - ) + config = config_type(**{ + mode_field: ( + NapariDimensionMode.STACK if origin == stack_origin + else NapariDimensionMode.LAYER + ) + }) + producer = StreamProducerIdentity("manual", "image", "raw", "raw") if origin == "raw" else None for channel in (1, 2): route = f"{origin}-{channel}" item = _layer_item( - { - "site": 1, - "channel": 1, - "z_index": 1, - "timepoint": 1, - "well": "A01", - component: channel, - }, + {"site": 1, "channel": 1, "z_index": 1, + "timepoint": 1, "well": "A01", component: channel}, data=np.full((4, 4), channel, dtype=np.uint16), producer=producer, ) semantics = ViewerComponentAxisSemantics( - entries=_component_value_domain( - { - component: [value] - for component, value in item.address.components.items() - } - ).entries, + entries=_component_value_domain({ + component: [value] for component, value in item.address.components.items() + }).entries, layout=ViewerObjectDisplayConfigInput(config).layout(), ) server.layer_route_state.set_title(route, route) work = pipeline.display_layer_batch( - layer_key=route, - items=[item], + layer_key=route, items=[item], display_payload=NapariPendingLayerUpdate.from_semantics( - timer=_FakeTimer(), - data_type=StreamingDataType.IMAGE, - semantics=semantics, - display_config=config, + timer=_FakeTimer(), data_type=StreamingDataType.IMAGE, + semantics=semantics, display_config=config, ), component_names_metadata=ViewerComponentNameMetadata.empty(), ) assert work.advance() assert ViewerObjectDisplayConfigInput(config).layout() == semantics.layout for channel in (1, 2): - raw = server.layer_route_state.dimension_state_for( - f"raw-{channel}" - ).presentation - result = server.layer_route_state.dimension_state_for( - f"pipeline-{channel}" - ).presentation + raw = server.layer_route_state.dimension_state_for(f"raw-{channel}").presentation + result = server.layer_route_state.dimension_state_for(f"pipeline-{channel}").presentation axis = raw.display_axis_components.index(component) assert result.display_axis_components == raw.display_axis_components assert raw.viewer_step(0, axis) == result.viewer_step(0, axis) == channel - 1 assert raw.projection.routed_component_values[component] == [channel] assert result.projection.routed_component_values[component] == [channel] assert np.max(server.layer_route_state.layer(f"raw-{channel}").data) == channel - assert ( - np.max(server.layer_route_state.layer(f"pipeline-{channel}").data) - == channel + assert np.max(server.layer_route_state.layer(f"pipeline-{channel}").data) == channel + assert tuple(server.layer_route_state.layer(f"raw-{channel}").translate) == tuple( + server.layer_route_state.layer(f"pipeline-{channel}").translate ) - assert tuple( - server.layer_route_state.layer(f"raw-{channel}").translate - ) == tuple(server.layer_route_state.layer(f"pipeline-{channel}").translate) def test_display_batch_composes_domain_and_names_for_new_declared_component(): @@ -2227,15 +2184,17 @@ def test_napari_display_pipeline_projects_aggregate_payload_axes_into_route_doma component_value_domain = _component_value_domain( {"channel": [1], "z_index": [1, 2]} ) - component_axis_semantics = ViewerComponentAxisSemanticsFactory.from_display_config( - ViewerMappingDisplayConfigInput( - { - "component_modes": {"channel": "stack", "z_index": "stack"}, - "component_order": ["z_index", "channel"], - "declared_axes": STREAM_AXES.to_wire(), - } - ), - component_value_domain, + component_axis_semantics = ( + ViewerComponentAxisSemanticsFactory.from_display_config( + ViewerMappingDisplayConfigInput( + { + "component_modes": {"channel": "stack", "z_index": "stack"}, + "component_order": ["z_index", "channel"], + "declared_axes": STREAM_AXES.to_wire(), + } + ), + component_value_domain, + ) ) projection = pipeline.display_axis_projection( @@ -3448,16 +3407,10 @@ def test_clear_state_preserves_mounted_payload_and_domains_until_native_removal( server.component_groups.items_for("cancelled").append(_layer_item({"well": "A14"})) server.component_values = ViewerRouteComponentValueTracker() server.component_values.update("retained", ["well"], [item]) - server.component_values.declare_component_values( - "retained", ["well"], {"well": ["A01", "A02"]} - ) - server.component_values.update( - "cancelled", ["well"], [_layer_item({"well": "A14"})] - ) + server.component_values.declare_component_values("retained", ["well"], {"well": ["A01", "A02"]}) + server.component_values.update("cancelled", ["well"], [_layer_item({"well": "A14"})]) domain = server.component_values.domain_for("retained", ["well"]) - server.component_name_metadata = _component_name_metadata( - {"well": {"A01": "source well"}} - ) + server.component_name_metadata = _component_name_metadata({"well": {"A01": "source well"}}) server.layer_batch_processor_debounce_policy = NapariLayerBatchDebouncePolicy() server.display_pipeline = NapariLayerDisplayPipeline(server) server.batch_processors = NapariBatchProcessorStore() @@ -3468,9 +3421,7 @@ def test_clear_state_preserves_mounted_payload_and_domains_until_native_removal( assert server.component_groups.existing_items_for("cancelled") is None assert server.component_values.domain_for("retained", ["well"]) is domain assert domain.coordinate_values("well") == {"A01", "A02"} - assert server.component_values.shared_values_for(["well"]) == { - "well": ["A01", "A02"] - } + assert server.component_values.shared_values_for(["well"]) == {"well": ["A01", "A02"]} assert server.component_name_metadata.display_name("well", "A01") == "source well" server.viewer.layers.remove(layer) @@ -3533,8 +3484,7 @@ def require_result_selection_surface(self): ) events.append("result_selection_surface_open") napari_viewer_server._install_result_selection_toolbar( - self.result_selection_surface, - self.result_selection_controller, + self.result_selection_surface, self.result_selection_controller, ) return self.result_selection_surface @@ -4246,9 +4196,7 @@ def test_roi_selection_toolbar_adjusts_native_highlight_setting(qtbot, monkeypat server = type("ResultSelectionServer", (), {"viewer": None})() controller = napari_viewer_server.NapariResultSelectionController(server) - surface = napari_viewer_server.NapariResultSelectionSurface( - result_selection_dock, object() - ) + surface = napari_viewer_server.NapariResultSelectionSurface(result_selection_dock, object()) qt_window.destroyed.connect(surface.release) toolbar = napari_viewer_server._install_result_selection_toolbar( surface, @@ -4315,9 +4263,7 @@ def test_roi_layer_color_button_recolors_every_shape(qtbot, monkeypatch): lambda *_args, **_kwargs: QColor.fromRgbF(0.2, 0.8, 0.3, 1.0), ) - surface = napari_viewer_server.NapariResultSelectionSurface( - result_selection_dock, object() - ) + surface = napari_viewer_server.NapariResultSelectionSurface(result_selection_dock, object()) qt_window.destroyed.connect(surface.release) toolbar = napari_viewer_server._install_result_selection_toolbar( surface, @@ -4387,6 +4333,8 @@ def test_napari_entrypoint_bind_failure_quits_event_loop_and_stops_server(monkey assert events[-1] == "server_stop" + + @pytest.mark.parametrize("message_type", ("shutdown", "force_shutdown")) def test_napari_shutdown_cancels_pending_updates_before_scheduling_viewer_close( monkeypatch, @@ -4460,6 +4408,12 @@ def close(self): assert not any(event.startswith("display:") for event in events) + + + + + + def test_napari_axis_projector_drops_only_globally_singleton_axes(): pytest.importorskip("openhcs.runtime.napari_viewer_server") component_values = { @@ -4583,9 +4537,7 @@ def __init__(self): self.component_groups = NapariComponentGroupStore() self.component_values = ViewerRouteComponentValueTracker() self.batch_processors = napari_viewer_server.NapariBatchProcessorStore() - self.display_pipeline = napari_viewer_server.NapariLayerDisplayPipeline( - self - ) + self.display_pipeline = napari_viewer_server.NapariLayerDisplayPipeline(self) server = _FakeServer() route_key = "deleted-channel-1" @@ -4682,9 +4634,11 @@ def __init__(self): "component_order": ["well"], "declared_axes": STREAM_AXES.to_wire(), } - component_axis_semantics = ViewerComponentAxisSemanticsFactory.from_display_config( - ViewerMappingDisplayConfigInput(display_config), - _component_value_domain({"well": ["A01"]}), + component_axis_semantics = ( + ViewerComponentAxisSemanticsFactory.from_display_config( + ViewerMappingDisplayConfigInput(display_config), + _component_value_domain({"well": ["A01"]}), + ) ) for data, image_metadata in ( @@ -4762,15 +4716,17 @@ def __init__(self): step_name="OverlayOutlines", pipeline_position=7, ) - component_axis_semantics = ViewerComponentAxisSemanticsFactory.from_display_config( - ViewerMappingDisplayConfigInput( - { - "component_modes": {"well": "stack", "channel": "stack"}, - "component_order": ["well", "channel"], - "declared_axes": STREAM_AXES.to_wire(), - } - ), - _component_value_domain({"well": ["A01"], "channel": [2]}), + component_axis_semantics = ( + ViewerComponentAxisSemanticsFactory.from_display_config( + ViewerMappingDisplayConfigInput( + { + "component_modes": {"well": "stack", "channel": "stack"}, + "component_order": ["well", "channel"], + "declared_axes": STREAM_AXES.to_wire(), + } + ), + _component_value_domain({"well": ["A01"], "channel": [2]}), + ) ) coordinator.display( @@ -5437,15 +5393,17 @@ def test_napari_display_projection_derives_sparse_coordinates_from_routed_items( "well": ["A02", "B04", "B12", "B17"], "site": [1, 2, 4, 7], } - component_axis_semantics = ViewerComponentAxisSemanticsFactory.from_display_config( - ViewerMappingDisplayConfigInput( - { - "component_modes": {"well": "stack", "site": "stack"}, - "component_order": ["well", "site"], - "declared_axes": STREAM_AXES.to_wire(), - } - ), - _component_value_domain(component_values), + component_axis_semantics = ( + ViewerComponentAxisSemanticsFactory.from_display_config( + ViewerMappingDisplayConfigInput( + { + "component_modes": {"well": "stack", "site": "stack"}, + "component_order": ["well", "site"], + "declared_axes": STREAM_AXES.to_wire(), + } + ), + _component_value_domain(component_values), + ) ) projection = pipeline.display_axis_projection( @@ -5809,9 +5767,7 @@ def test_napari_shape_feature_columns_fill_sparse_late_columns_in_order(): columns = NapariShapeFeatureColumns() columns.append({"area": 4.0}, label=1, path="first", element_identity="first:0") - columns.append( - {"circularity": 0.8}, label=2, path="second", element_identity="second:0" - ) + columns.append({"circularity": 0.8}, label=2, path="second", element_identity="second:0") columns.append( {"area": 9.0, "circularity": 0.6}, label=3, @@ -5823,11 +5779,7 @@ def test_napari_shape_feature_columns_fill_sparse_late_columns_in_order(): "area": [4.0, None, 9.0], "label": [1, 2, 3], "path": ["first", "second", "third"], - NapariStreamLayerItem.ELEMENT_IDENTITY_FEATURE: [ - "first:0", - "second:0", - "third:0", - ], + NapariStreamLayerItem.ELEMENT_IDENTITY_FEATURE: ["first:0", "second:0", "third:0"], "circularity": [None, 0.8, 0.6], } @@ -5871,15 +5823,17 @@ def test_napari_aggregate_axis_binding_uses_declared_component_not_equal_extent( component_value_domain = _component_value_domain( {"site": [1, 2], "channel": [1, 2]} ) - component_axis_semantics = ViewerComponentAxisSemanticsFactory.from_display_config( - ViewerMappingDisplayConfigInput( - { - "component_modes": {"site": "stack", "channel": "stack"}, - "component_order": ["site", "channel"], - "declared_axes": STREAM_AXES.to_wire(), - } - ), - component_value_domain, + component_axis_semantics = ( + ViewerComponentAxisSemanticsFactory.from_display_config( + ViewerMappingDisplayConfigInput( + { + "component_modes": {"site": "stack", "channel": "stack"}, + "component_order": ["site", "channel"], + "declared_axes": STREAM_AXES.to_wire(), + } + ), + component_value_domain, + ) ) items = [ _layer_item( @@ -5924,15 +5878,17 @@ def test_napari_shape_plane_metadata_requires_declared_component_axis(): component_value_domain = _component_value_domain( {"site": [1, 2], "channel": [1, 2]} ) - component_axis_semantics = ViewerComponentAxisSemanticsFactory.from_display_config( - ViewerMappingDisplayConfigInput( - { - "component_modes": {"site": "stack", "channel": "stack"}, - "component_order": ["site", "channel"], - "declared_axes": STREAM_AXES.to_wire(), - } - ), - component_value_domain, + component_axis_semantics = ( + ViewerComponentAxisSemanticsFactory.from_display_config( + ViewerMappingDisplayConfigInput( + { + "component_modes": {"site": "stack", "channel": "stack"}, + "component_order": ["site", "channel"], + "declared_axes": STREAM_AXES.to_wire(), + } + ), + component_value_domain, + ) ) with pytest.raises(ValueError, match="plane_component_values"): @@ -5962,15 +5918,17 @@ def test_napari_shape_label_rasterizer_rejects_mixed_aggregate_plane_metadata(): component_value_domain = _component_value_domain( {"channel": [1], "z_index": [1, 2]} ) - component_axis_semantics = ViewerComponentAxisSemanticsFactory.from_display_config( - ViewerMappingDisplayConfigInput( - { - "component_modes": {"channel": "stack", "z_index": "stack"}, - "component_order": ["z_index", "channel"], - "declared_axes": STREAM_AXES.to_wire(), - } - ), - component_value_domain, + component_axis_semantics = ( + ViewerComponentAxisSemanticsFactory.from_display_config( + ViewerMappingDisplayConfigInput( + { + "component_modes": {"channel": "stack", "z_index": "stack"}, + "component_order": ["z_index", "channel"], + "declared_axes": STREAM_AXES.to_wire(), + } + ), + component_value_domain, + ) ) with pytest.raises(ValueError, match="mixes plane-indexed and unindexed"): @@ -6208,13 +6166,8 @@ def test_napari_points_layer_display_applies_route_global_axis_translate(): ) item = _layer_item( {"channel": 4}, - [ - { - "type": "points", - "coordinates": [[1, 2]], - "metadata": {"label": 7, "component": 4}, - } - ], + [{"type": "points", "coordinates": [[1, 2]], + "metadata": {"label": 7, "component": 4}}], stream_layer_data_type=StreamingDataType.POINTS, ) @@ -6235,8 +6188,7 @@ def test_napari_points_layer_display_applies_route_global_axis_translate(): assert layer_kwargs["axis_labels"] == ("channel", "y", "x") assert layer_kwargs["translate"] == (3.0, 0.0, 0.0) assert layer_kwargs["properties"] == { - "label": [7], - "component": [4], + "label": [7], "component": [4], NapariStreamLayerItem.ELEMENT_IDENTITY_FEATURE: [item.element_identity(0)], } @@ -6291,30 +6243,14 @@ def test_napari_points_layer_uses_exact_fractional_z_from_native_roi_metadata(): @pytest.mark.parametrize( "values,unit,expected_scale,expected_units", ( - ( - (2.0, 0.65, 0.65), - "micrometers", - (1.0, 2.0, 0.65, 0.65), - ("dimensionless", "micrometer", "micrometer", "micrometer"), - ), - ( - (2.0, 1.0, 1.0), - "relative", - (1.0, 2.0, 1.0, 1.0), - ("dimensionless", "dimensionless", "dimensionless", "dimensionless"), - ), - ( - (0.65, 0.65), - "micrometers", - (1.0, 1.0, 0.65, 0.65), - ("dimensionless", "dimensionless", "micrometer", "micrometer"), - ), - ( - (), - "micrometers", - (1.0, 1.0, 1.0, 1.0), - ("dimensionless", "dimensionless", "pixel", "pixel"), - ), + ((2.0, 0.65, 0.65), "micrometers", (1.0, 2.0, 0.65, 0.65), + ("dimensionless", "micrometer", "micrometer", "micrometer")), + ((2.0, 1.0, 1.0), "relative", (1.0, 2.0, 1.0, 1.0), + ("dimensionless", "dimensionless", "dimensionless", "dimensionless")), + ((0.65, 0.65), "micrometers", (1.0, 1.0, 0.65, 0.65), + ("dimensionless", "dimensionless", "micrometer", "micrometer")), + ((), "micrometers", (1.0, 1.0, 1.0, 1.0), + ("dimensionless", "dimensionless", "pixel", "pixel")), ), ) def test_spatial_presentation_owns_scale_and_world_translation( @@ -6328,26 +6264,19 @@ def test_spatial_presentation_owns_scale_and_world_translation( component_values={"channel": [4], "z_index": [2, 3, 4, 5]}, axis_offsets=(3, 2), ) - item = _layer_item({"channel": 4, "z_index": 2}, np.zeros((5, 7), dtype=np.uint16)) - item = replace( - item, - image_metadata=ImagePayloadMetadata( - source_voxel_spacing=SourceVoxelSpacing( - values, SourceVoxelSpacingUnit(unit) - ) - ), + item = _layer_item( + {"channel": 4, "z_index": 2}, np.zeros((5, 7), dtype=np.uint16) ) + item = replace(item, image_metadata=ImagePayloadMetadata( + source_voxel_spacing=SourceVoxelSpacing(values, SourceVoxelSpacingUnit(unit)) + )) kwargs = presentation.spatial_layer_kwargs([item]) assert kwargs["scale"] == expected_scale assert kwargs["units"] == expected_units assert kwargs["translate"] == (3.0, 2 * expected_scale[1], 0.0, 0.0) payload_kwargs = presentation.spatial_layer_kwargs([item], ("band",)) assert payload_kwargs["scale"] == (*expected_scale[:2], 1.0, *expected_scale[2:]) - assert payload_kwargs["units"] == ( - *expected_units[:2], - "dimensionless", - *expected_units[2:], - ) + assert payload_kwargs["units"] == (*expected_units[:2], "dimensionless", *expected_units[2:]) def test_calibrated_native_image_point_and_semantic_navigation_agree_without_qt(): @@ -6388,17 +6317,12 @@ def test_calibrated_native_image_point_and_semantic_navigation_agree_without_qt( def test_shape_features_exclude_source_transport_without_mutating_payload(): from openhcs.core.roi_source_metadata import ROIArchiveSourceMetadata - full_metadata = { - "label": 7, - "response": 0.25, - ROIArchiveSourceMetadata.FIELD: {"source_provenance": "retained"}, - } + full_metadata = {"label": 7, "response": 0.25, + ROIArchiveSourceMetadata.FIELD: {"source_provenance": "retained"}} features = NapariShapeFeatureColumns() item = _layer_item({}, stream_layer_data_type=StreamingDataType.SHAPES) features.append( - full_metadata, - label=7, - path=item.address.path, + full_metadata, label=7, path=item.address.path, element_identity=item.element_identity(0), ) assert ROIArchiveSourceMetadata.FIELD in full_metadata @@ -6434,9 +6358,7 @@ def test_napari_fractional_z_points_are_relative_to_the_declared_anchor(): ) projection = pipeline.display_axis_projection("centres", semantics, [item]) assert projection.component_values == {"z_index": [2, 3, 4]} - points, _properties = napari_viewer_server._build_nd_points( - [item], projection, ("z_index",) - ) + points, _properties = napari_viewer_server._build_nd_points([item], projection, ("z_index",)) assert tuple(points[0]) == (1.25, 1.25, 3.5) assert projection.axis_offsets == (2,) diff --git a/tests/unit/test_native_viewer_presentation.py b/tests/unit/test_native_viewer_presentation.py index 08923183f7..f384f17f4e 100644 --- a/tests/unit/test_native_viewer_presentation.py +++ b/tests/unit/test_native_viewer_presentation.py @@ -64,9 +64,7 @@ def presentation(): semantics = ViewerComponentAxisSemanticsFactory.empty() return NapariAxisPresentation( entries=semantics.entries, - layout=ViewerComponentLayout.from_parts( - component_modes={}, component_order=(), declared_axes=STREAM_AXES - ), + layout=ViewerComponentLayout.from_parts(component_modes={}, component_order=(), declared_axes=STREAM_AXES), route_key="paired", projection=ViewerLayerAxisProjection( projected_axis_components=(), @@ -310,7 +308,7 @@ def test_native_display_handlers_apply_shared_calibration_after_wire_roundtrip() def test_two_dimensional_calibration_does_not_calibrate_component_z_axis(): - + axis = replace( presentation(), layout=ViewerComponentLayout.from_parts( @@ -581,25 +579,14 @@ def test_selected_file_stream_restores_persisted_crop_calibration_and_native_bat spacing = SourceVoxelSpacing((1.3556, 1.3556)) domain = SourceSpatialDomain((7, 11), (20, 30), 0, "Retained crop") for channel in (1, 2): - address = OpenHCSPlaneAddress( - ( - (Microscopy.Well, well), - (Microscopy.Site, 1), - (Microscopy.Channel, channel), - (Microscopy.ZIndex, 1), - (Microscopy.Timepoint, 1), - ) - ) + address = OpenHCSPlaneAddress(((Microscopy.Well, well), (Microscopy.Site, 1), (Microscopy.Channel, channel), (Microscopy.ZIndex, 1), (Microscopy.Timepoint, 1))) component_metadata = { component.name: value for component, value in address.component_values().items() } contributors = tuple( SourceImageProvenanceContributor( - SourceImageIdentity( - str(tmp_path / f"synthetic-site{site}-channel{channel}.tif"), - {**component_metadata, "site": site}, - ), + SourceImageIdentity(str(tmp_path / f"synthetic-site{site}-channel{channel}.tif"), {**component_metadata, "site": site}), source_image_name="neurite" if channel == 1 else "nucleus", ) for site in range(1, 10) @@ -610,9 +597,7 @@ def test_selected_file_stream_restores_persisted_crop_calibration_and_native_bat source_provenance=SourceImageProvenance( source_component_metadata=component_metadata, source_image_names=("neurite" if channel == 1 else "nucleus",), - source_image_provenance_planes=SourceImageProvenancePlanes( - contributors - ), + source_image_provenance_planes=SourceImageProvenancePlanes(contributors), ), ) path = tmp_path / f"stored-channel{channel}.tif" diff --git a/tests/unit/test_neurite_outgrowth.py b/tests/unit/test_neurite_outgrowth.py index 401719724b..eddc18f647 100644 --- a/tests/unit/test_neurite_outgrowth.py +++ b/tests/unit/test_neurite_outgrowth.py @@ -78,12 +78,7 @@ def _declared_shaft_admission(response, mask=None): return NeuriteAdmissionResult( mask=support, planes=NeuriteAdmissionPlanes( - response, - support, - support, - response, - response > 0, - response, + response, support, support, response, response > 0, response, ), outgrowth_width_px=2.0, ) @@ -105,8 +100,7 @@ def _cell_body_count_implementation(): def test_admission_exports_the_original_independent_gates(seed_factor): image = _draw_fluorescent_neuron(branched=True)[0] settings = MetaXpressOutgrowthSettings( - maximum_width=3, - intensity_above_local_background=100, + maximum_width=3, intensity_above_local_background=100, candidate_hysteresis_seed_correction_factor=seed_factor, ) result = _identify_neurites_cellprofiler( @@ -115,7 +109,9 @@ def test_admission_exports_the_original_independent_gates(seed_factor): assert isinstance(result, NeuriteAdmissionResult) planes = result.planes assert result.response is planes.local_response - np.testing.assert_array_equal(planes.local_support, planes.local_response >= 100) + np.testing.assert_array_equal( + planes.local_support, planes.local_response >= 100 + ) np.testing.assert_array_equal( result.mask, planes.retained_support & planes.local_support ) @@ -145,26 +141,19 @@ def test_pixel_recipe_matches_physical_detection_and_declares_metric_units( image = np.concatenate((body_and_process, nuclei[None]), axis=0) before = image.copy() pixel_body = PixelCellBodySettings( - approximate_max_width=30, - minimum_area=100, + approximate_max_width=30, minimum_area=100, intensity_above_local_background=100, channel_index=0 if with_nuclei else None, ) pixel_growth = PixelOutgrowthSettings( - maximum_width=3, - intensity_above_local_background=100, + maximum_width=3, intensity_above_local_background=100, minimum_cell_growth_to_log_as_significant=20, ) contract = CallableContract.from_callable(neurite_outgrowth_metaxpress_pixels) assert contract.artifact_inputs.names() == () - assert ( - "pixel_size" - not in signature(contract.resolve_raw_runtime_callable()).parameters - ) + assert "pixel_size" not in signature(contract.resolve_raw_runtime_callable()).parameters pixel = contract.resolve_raw_runtime_callable()( - image, - cell_body=pixel_body, - outgrowth=pixel_growth, + image, cell_body=pixel_body, outgrowth=pixel_growth, neurite_channel_index=1 if with_nuclei else 0, use_nuclear_stain=with_nuclei, nuclear_stain=PixelNuclearSettings(channel_index=2), @@ -172,14 +161,12 @@ def test_pixel_recipe_matches_physical_detection_and_declares_metric_units( physical = _implementation()( image, cell_body=MetaXpressCellBodySettings( - approximate_max_width=15, - minimum_area=25, + approximate_max_width=15, minimum_area=25, intensity_above_local_background=100, channel_index=0 if with_nuclei else None, ), outgrowth=MetaXpressOutgrowthSettings( - maximum_width=1.5, - intensity_above_local_background=100, + maximum_width=1.5, intensity_above_local_background=100, minimum_cell_growth_to_log_as_significant=10, ), pixel_size=0.5, @@ -215,12 +202,9 @@ def test_pixel_recipe_matches_physical_detection_and_declares_metric_units( assert tuple(type(option) for option in morphology.materialization.outputs) == ( SpatialGraphROIOptions, ) - assert ( - morphology.relations - == CallableContract.from_callable(neurite_outgrowth_metaxpress) - .artifact_outputs[-1] - .relations - ) + assert morphology.relations == CallableContract.from_callable( + neurite_outgrowth_metaxpress + ).artifact_outputs[-1].relations @pytest.mark.parametrize( @@ -247,13 +231,11 @@ def test_pixel_recipe_rows_csv_and_native_graph_roi_keep_distinct_source_units( ).resolve_raw_runtime_callable()( image, cell_body=PixelCellBodySettings( - approximate_max_width=30, - minimum_area=100, + approximate_max_width=30, minimum_area=100, intensity_above_local_background=100, ), outgrowth=PixelOutgrowthSettings( - maximum_width=3, - intensity_above_local_background=100, + maximum_width=3, intensity_above_local_background=100, minimum_cell_growth_to_log_as_significant=20, ), ) @@ -261,15 +243,10 @@ def test_pixel_recipe_rows_csv_and_native_graph_roi_keep_distinct_source_units( source_voxel_spacing=spacing, source_image_provenance_planes=SourceImageProvenancePlanes.from_components( paths=("/engineering/A01_s1_w2.tif",), - component_metadata=( - { - "well": "A01", - "site": 1, - "channel": 2, - "z_index": 1, - "timepoint": 1, - }, - ), + component_metadata=({ + "well": "A01", "site": 1, "channel": 2, + "z_index": 1, "timepoint": 1, + },), ), ) source = metadata.payload_with(image) @@ -283,12 +260,9 @@ def test_pixel_recipe_rows_csv_and_native_graph_roi_keep_distinct_source_units( neurite_outgrowth_metaxpress_pixels ).artifact_outputs csv_path = materialize( - declarations[1].materialization, - data=result[2], - path=str(tmp_path / "cells.csv"), - filemanager=manager, - backends=["disk"], - backend_kwargs={}, + declarations[1].materialization, data=result[2], + path=str(tmp_path / "cells.csv"), filemanager=manager, + backends=["disk"], backend_kwargs={}, ) with open(csv_path, newline="") as stream: rows = tuple(csv.DictReader(stream)) @@ -296,12 +270,9 @@ def test_pixel_recipe_rows_csv_and_native_graph_roi_keep_distinct_source_units( assert {row["coordinate_unit"] for row in rows} == {"pixels"} assert float(rows[0]["total_outgrowth"]) > 0 zip_path = materialize( - declarations[-1].materialization, - data=graph, - path=str(tmp_path / "morphology.roi.zip"), - filemanager=manager, - backends=["disk"], - backend_kwargs={}, + declarations[-1].materialization, data=graph, + path=str(tmp_path / "morphology.roi.zip"), filemanager=manager, + backends=["disk"], backend_kwargs={}, ) restored = load_rois_from_zip(Path(zip_path)) saved = ROIArchiveSourceMetadata.decode(restored) @@ -312,18 +283,13 @@ def test_pixel_recipe_rows_csv_and_native_graph_roi_keep_distinct_source_units( for roi, edge in zip(geometry, graph.edges, strict=True): np.testing.assert_array_equal(roi.shapes[0].coordinates, edge.coordinates) assert roi.metadata["coordinate_unit"] == "pixels" - assert ( - roi.metadata["branch_distance"] == edge.feature_mapping()["branch_distance"] - ) + assert roi.metadata["branch_distance"] == edge.feature_mapping()["branch_distance"] assert not tuple(tmp_path.glob("*.swc")) with pytest.raises(ValueError, match="explicit micrometer spacing"): materialize( - MaterializationSpec(SWCOptions()), - data=graph, - path=str(tmp_path / "forbidden.swc"), - filemanager=manager, - backends=["disk"], - backend_kwargs={}, + MaterializationSpec(SWCOptions()), data=graph, + path=str(tmp_path / "forbidden.swc"), filemanager=manager, + backends=["disk"], backend_kwargs={}, ) assert not tuple(tmp_path.glob("*.swc")) @@ -366,10 +332,7 @@ class AuditedPixelRecipe(MetricAudit, PixelCellProfilerNeuriteEngineProfile): SpatialGraphROIOptions, ) assert events == [ - "export-before", - "export-after", - "analysis-before", - "analysis-after", + "export-before", "export-after", "analysis-before", "analysis-after" ] @@ -436,9 +399,7 @@ def minimum_area_px(self, pixel_size_um): @dataclass(frozen=True) class DeclaredBody( - ProjectionAudit, - HalfWidthProjection, - QuarterAreaProjection, + ProjectionAudit, HalfWidthProjection, QuarterAreaProjection, MetaXpressCellBodySettings, ): pass @@ -452,21 +413,17 @@ class DeclaredOutgrowth( image = _draw_fluorescent_neuron(branched=True) original = image.copy() expected = _implementation()( - image, - cell_body=_cell_body_settings(), - outgrowth=_outgrowth_settings(), + image, cell_body=_cell_body_settings(), outgrowth=_outgrowth_settings(), pixel_size=1.3556, ) actual = _implementation()( image, cell_body=DeclaredBody( - approximate_max_width=60.0, - minimum_area=400.0, + approximate_max_width=60.0, minimum_area=400.0, intensity_above_local_background=100.0, ), outgrowth=DeclaredOutgrowth( - maximum_width=6.0, - intensity_above_local_background=100.0, + maximum_width=6.0, intensity_above_local_background=100.0, minimum_cell_growth_to_log_as_significant=20.0, ), pixel_size=1.3556, @@ -631,9 +588,7 @@ def test_cell_body_minimum_area_does_not_impose_hidden_roundness(monkeypatch): monkeypatch.setattr( "openhcs.processing.backends.analysis.neurite_outgrowth.identify_primary_objects", lambda *_args, **_kwargs: ( - image, - image, - detected_labels, + image, image, detected_labels, *(object() for _ in PrimaryObjectDiagnosticPlanes._fields), ), ) @@ -668,9 +623,7 @@ def test_nuclear_support_ignores_rejected_nearer_body_candidate(monkeypatch): monkeypatch.setattr( "openhcs.processing.backends.analysis.neurite_outgrowth.identify_primary_objects", lambda *_args, **_kwargs: ( - image, - image, - detected_labels, + image, image, detected_labels, *(object() for _ in PrimaryObjectDiagnosticPlanes._fields), ), ) @@ -815,7 +768,9 @@ def test_topology_metrics_and_significant_threshold_is_scoring_only(): assert low_cell["significant_growth"] is True assert high_cell["significant_growth"] is False assert high_summary["cells_significant_growth"] == 0 - assert high_cell["total_outgrowth"] == pytest.approx(low_cell["total_outgrowth"]) + assert high_cell["total_outgrowth"] == pytest.approx( + low_cell["total_outgrowth"] + ) assert np.array_equal( low_threshold[4], high_threshold[4], @@ -854,11 +809,7 @@ def test_edgeless_skeleton_has_empty_topology(coordinates): skeleton[coordinate] = True original = skeleton.copy() topology = _analyze_topology( - skeleton, - np.zeros(skeleton.shape, np.int32), - 1.3556, - 3.0, - branch_support=skeleton, + skeleton, np.zeros(skeleton.shape, np.int32), 1.3556, 3.0, branch_support=skeleton, ) assert all(len(getattr(topology, field.name)) == 0 for field in fields(topology)) assert np.array_equal(skeleton, original) @@ -1043,12 +994,7 @@ def test_crossing_cluster_counts_three_owned_arms_across_physical_nodes(): bodies[30:35, 3:9] = 1 bodies[7:13, 47:53] = 2 topology = _analyze_topology( - labels > 0, - bodies, - 1.0, - 8.0, - assigned_path_labels=labels, - branch_support=labels > 0, + labels > 0, bodies, 1.0, 8.0, assigned_path_labels=labels, branch_support=labels > 0, ) # Three same-owner arms are now a branch, not a retained two-arm crossover. assert topology.crossing_nodes == frozenset() @@ -1117,33 +1063,23 @@ def test_short_two_junction_crossing_resolves_opposite_rooted_traces(pixel_size_ owned = _render_owned_skeleton(skeleton.shape, topology) response = np.where(skeleton, 150.0, 0.0) repaired = _repair_signal_supported_skeleton( - owned, - _declared_shaft_admission(response), - np.zeros_like(cell_bodies), - cell_bodies, - minimum_response=100.0, - crossing_topology=topology, + owned, _declared_shaft_admission(response), np.zeros_like(cell_bodies), cell_bodies, + minimum_response=100.0, crossing_topology=topology, ) core = topology.crossing_core_mask(skeleton.shape) np.testing.assert_array_equal(repaired[core], owned[core]) # Renumbering reverses physical owner traversal without changing geometry. reversed_topology = replace( - topology, - path_owners=np.where(topology.path_owners > 0, 3 - topology.path_owners, 0), + topology, path_owners=np.where(topology.path_owners > 0, 3 - topology.path_owners, 0), ) reversed_owned = np.where(owned > 0, 3 - owned, 0) reversed_bodies = np.where(cell_bodies > 0, 3 - cell_bodies, 0) reversed_repair = _repair_signal_supported_skeleton( - reversed_owned, - _declared_shaft_admission(response), - np.zeros_like(cell_bodies), - reversed_bodies, - minimum_response=100.0, - crossing_topology=reversed_topology, + reversed_owned, _declared_shaft_admission(response), np.zeros_like(cell_bodies), reversed_bodies, + minimum_response=100.0, crossing_topology=reversed_topology, ) np.testing.assert_array_equal( - reversed_repair, - np.where(repaired > 0, 3 - repaired, 0), + reversed_repair, np.where(repaired > 0, 3 - repaired, 0), ) foreign_owned = owned.copy() foreign_bodies = cell_bodies.copy() @@ -1155,12 +1091,8 @@ def test_short_two_junction_crossing_resolves_opposite_rooted_traces(pixel_size_ foreign_response = response.copy() foreign_response[foreign_rows, foreign_columns] = 150.0 foreign_repair = _repair_signal_supported_skeleton( - foreign_owned, - _declared_shaft_admission(foreign_response), - np.zeros_like(cell_bodies), - foreign_bodies, - minimum_response=100.0, - crossing_topology=topology, + foreign_owned, _declared_shaft_admission(foreign_response), np.zeros_like(cell_bodies), foreign_bodies, + minimum_response=100.0, crossing_topology=topology, ) assert np.all(foreign_repair[core] == 99) foreign_trace = (foreign_owned == 99) & (foreign_bodies == 0) @@ -1168,16 +1100,10 @@ def test_short_two_junction_crossing_resolves_opposite_rooted_traces(pixel_size_ for path_index in topology.crossing_paths: coordinates = topology.path_coordinates[path_index] outside = ~core[tuple(coordinates.T)] & (cell_bodies[tuple(coordinates.T)] == 0) - assert np.all( - repaired[tuple(coordinates[outside].T)] == topology.path_owners[path_index] - ) + assert np.all(repaired[tuple(coordinates[outside].T)] == topology.path_owners[path_index]) repaired[cell_bodies > 0] = 0 # The profile excludes soma pixels from traces. final = _analyze_owned_topology( - repaired, - cell_bodies, - pixel_size_um, - 8.0, - crossing_topology=topology, + repaired, cell_bodies, pixel_size_um, 8.0, crossing_topology=topology, branch_support=response > 0, ) final_terminals = { @@ -1187,8 +1113,7 @@ def test_short_two_junction_crossing_resolves_opposite_rooted_traces(pixel_size_ if tuple(coordinate) in terminals and cell_bodies[tuple(coordinate)] == 0 } assert final_terminals == { - coordinate: label - for coordinate, label in terminal_owners.items() + coordinate: label for coordinate, label in terminal_owners.items() if cell_bodies[coordinate] == 0 } assert final.root_paths_by_cell.keys() == {1, 2} @@ -1196,20 +1121,13 @@ def test_short_two_junction_crossing_resolves_opposite_rooted_traces(pixel_size_ for path_index in topology.crossing_paths: coordinates = topology.path_coordinates[path_index] outside = ~core[tuple(coordinates.T)] & (cell_bodies[tuple(coordinates.T)] == 0) - assert np.all( - final_rendered[tuple(coordinates[outside].T)] - == topology.path_owners[path_index] - ) + assert np.all(final_rendered[tuple(coordinates[outside].T)] == topology.path_owners[path_index]) unsupported_response = response.copy() unsupported_response[core] = 0 unsupported = _repair_signal_supported_skeleton( - owned, - _declared_shaft_admission(unsupported_response), - np.zeros_like(cell_bodies), - cell_bodies, - minimum_response=100.0, - crossing_topology=topology, + owned, _declared_shaft_admission(unsupported_response), np.zeros_like(cell_bodies), cell_bodies, + minimum_response=100.0, crossing_topology=topology, ) unsupported[cell_bodies > 0] = 0 # Merely declaring a crossing does not license a below-threshold bridge. @@ -1273,31 +1191,20 @@ def test_branch_events_require_three_paths_of_the_same_final_owner( def test_branch_qualification_rejects_medial_stars_inside_a_single_cap(): center = np.array([20, 20]) - paths = tuple( - np.array([center, center + offset]) - for offset in ( - [0, 1], - [-1, -1], - [1, -1], - ) - ) + paths = tuple(np.array([center, center + offset]) for offset in ( + [0, 1], [-1, -1], [1, -1], + )) foreground = np.zeros((41, 41), dtype=bool) rows, columns = disk(tuple(center), 3, shape=foreground.shape) foreground[rows, columns] = True topology = replace( - _empty_topology(), - path_owners=np.ones(3, dtype=np.int32), - path_coordinates=paths, - path_endpoint_groups=((0, 1), (0, 2), (0, 3)), - endpoint_group_coordinates={ - 0: tuple(center), - **{i + 1: tuple(path[-1]) for i, path in enumerate(paths)}, - }, + _empty_topology(), path_owners=np.ones(3, dtype=np.int32), + path_coordinates=paths, path_endpoint_groups=((0, 1), (0, 2), (0, 3)), + endpoint_group_coordinates={0: tuple(center), **{i + 1: tuple(path[-1]) for i, path in enumerate(paths)}}, ) branch_types, branches = topology.classify_owned_endpoints( branch_support=foreground, - cell_body_labels=np.zeros(foreground.shape, dtype=np.int32), - merge_radius=3, + cell_body_labels=np.zeros(foreground.shape, dtype=np.int32), merge_radius=3, outgrowth_width_px=3, ) assert branches == {} @@ -1319,25 +1226,16 @@ def test_branch_qualification_keeps_a_short_real_soma_attachment(body_owner): bodies = np.zeros(foreground.shape, dtype=np.int32) bodies[22:27, 19:22] = body_owner topology = replace( - _empty_topology(), - path_owners=np.ones(3, dtype=np.int32), - path_coordinates=paths, - path_endpoint_groups=((0, 1), (0, 2), (0, 3)), - endpoint_group_coordinates={ - 0: (20, 20), - **{i + 1: tuple(path[-1]) for i, path in enumerate(paths)}, - }, + _empty_topology(), path_owners=np.ones(3, dtype=np.int32), + path_coordinates=paths, path_endpoint_groups=((0, 1), (0, 2), (0, 3)), + endpoint_group_coordinates={0: (20, 20), **{i + 1: tuple(path[-1]) for i, path in enumerate(paths)}}, ) branch_types, branches = topology.classify_owned_endpoints( - branch_support=foreground, - cell_body_labels=bodies, - merge_radius=3, + branch_support=foreground, cell_body_labels=bodies, merge_radius=3, outgrowth_width_px=3, ) assert branches == ({1: (0,)} if body_owner == 1 else {}) - np.testing.assert_array_equal( - branch_types, [1, 1, 1] if body_owner == 1 else [0, 0, 0] - ) + np.testing.assert_array_equal(branch_types, [1, 1, 1] if body_owner == 1 else [0, 0, 0]) def test_branch_qualification_resolves_a_short_daughter_beyond_a_thick_junction(): @@ -1354,17 +1252,14 @@ def test_branch_qualification_resolves_a_short_daughter_beyond_a_thick_junction( foreground[11, 8:11] = True foreground[12:14, 2:6] = True topology = replace( - _empty_topology(), - path_owners=np.ones(3, dtype=np.int32), - path_coordinates=paths, - path_endpoint_groups=((1, 0), (0, 2), (0, 3)), + _empty_topology(), path_owners=np.ones(3, dtype=np.int32), + path_coordinates=paths, path_endpoint_groups=((1, 0), (0, 2), (0, 3)), endpoint_group_coordinates={0: (8, 6), 1: (4, 6), 2: (11, 10), 3: (13, 3)}, ) branch_types, branches = topology.classify_owned_endpoints( branch_support=foreground, cell_body_labels=np.zeros(foreground.shape, dtype=np.int32), - merge_radius=2.95, - outgrowth_width_px=2.95, + merge_radius=2.95, outgrowth_width_px=2.95, ) assert branches == {1: (0,)} np.testing.assert_array_equal(branch_types, [1, 1, 1]) @@ -1705,8 +1600,7 @@ def test_explicit_body_nuclear_and_neurite_channels_are_aligned(): assert np.asarray(topology_dropped_trace).shape == (1, *image.shape[1:]) assert np.asarray(topology_added_trace).shape == (1, *image.shape[1:]) assert len(admission_planes) == len( - NeuriteAdmissionPlanes.artifact_specs() - + NeuriteOwnershipPlanes.artifact_specs() + NeuriteAdmissionPlanes.artifact_specs() + NeuriteOwnershipPlanes.artifact_specs() ) for plane in admission_planes: assert plane.source_indices == (2,) @@ -2199,17 +2093,16 @@ def reject_hidden_secondary_threshold(*args, **kwargs): assert neurites[1, 80, 37] > 0 for owner in (1, 2): components, count = ndi.label( - cell_bodies[1] == owner, - structure=np.ones((3, 3), dtype=bool), + cell_bodies[1] == owner, structure=np.ones((3, 3), dtype=bool), ) assert count == 1 assert np.any((components > 0) & (nuclei[0] == owner)) lengths = sorted(row["total_outgrowth"] for row in cell_rows) assert lengths[0] == 0.0 assert lengths[1] > 89.0 - assert lengths[1] == pytest.approx( - sum(edge.feature_mapping()["branch_distance"] for edge in result[-1].edges) - ) + assert lengths[1] == pytest.approx(sum( + edge.feature_mapping()["branch_distance"] for edge in result[-1].edges + )) zero_growth_cell = next( row["cell"] for row in cell_rows if row["total_outgrowth"] == 0.0 ) @@ -2252,8 +2145,7 @@ def test_provisional_owner_cannot_reach_a_shaft_through_unadmitted_background(): support = np.zeros(response.shape, dtype=bool) support[16, 3:29] = True owner_regions = _propagate_neurite_owner_regions( - _declared_shaft_admission(response, support), - bodies, + _declared_shaft_admission(response, support), bodies, minimum_response=100.0, ) assert np.all(owner_regions[16, 3:29] == 1) @@ -2495,9 +2387,7 @@ def test_signal_supported_repair_follows_curved_trace_instead_of_chord(): @pytest.mark.parametrize("attachment_response, retained", [(150.0, True), (0.0, False)]) -def test_signal_supported_repair_qualifies_the_short_soma_transition( - attachment_response, retained -): +def test_signal_supported_repair_qualifies_the_short_soma_transition(attachment_response, retained): labels = np.zeros((32, 48), dtype=np.int32) bodies = np.zeros_like(labels) bodies[13:20, 3:10] = 1 @@ -2507,22 +2397,14 @@ def test_signal_supported_repair_qualifies_the_short_soma_transition( admission = _declared_shaft_admission(response) soma_response = response.copy() soma_response[16, 10:13] = attachment_response - admission = replace( - admission, - planes=replace( - admission.planes, - soma_attachment_response=soma_response, - ), - ) + admission = replace(admission, planes=replace( + admission.planes, soma_attachment_response=soma_response, + )) regions = np.where(response > 0, 1, 0).astype(np.int32) repaired = _repair_signal_supported_skeleton( - labels, - admission, - regions, - bodies, - minimum_response=100.0, - crossing_topology=_empty_topology(), + labels, admission, regions, bodies, + minimum_response=100.0, crossing_topology=_empty_topology(), ) assert bool(np.all(repaired[16, 13:35] == 1)) == retained @@ -2540,12 +2422,8 @@ def test_signal_supported_repair_cannot_use_old_ownership_to_cross_a_foreign_som regions = np.where(response > 0, 1, 0).astype(np.int32) repaired = _repair_signal_supported_skeleton( - labels, - _declared_shaft_admission(response), - regions, - bodies, - minimum_response=100.0, - crossing_topology=_empty_topology(), + labels, _declared_shaft_admission(response), regions, bodies, + minimum_response=100.0, crossing_topology=_empty_topology(), ) assert np.all(repaired[16, 10:20] == 1) @@ -2564,16 +2442,10 @@ def test_process_attachment_groups_daughters_but_preserves_independent_exits(): # All three are ownership evidence, but only the proximal stem starts a # process. Final same-owner transitions determine biological grouping. assert _TopologyResult.soma_ownership_seeds(bodies, 2.0, paths) == { - 0: (1,), - 1: (1,), - 2: (1,), + 0: (1,), 1: (1,), 2: (1,), } assert _TopologyResult.soma_process_roots( - bodies, - 2.0, - paths, - transitions, - np.ones(3, dtype=np.int32), + bodies, 2.0, paths, transitions, np.ones(3, dtype=np.int32), ) == {1: (0,)} exits = ( @@ -2582,10 +2454,7 @@ def test_process_attachment_groups_daughters_but_preserves_independent_exits(): np.array([[12, 16], [18, 16]]), ) assert _TopologyResult.soma_process_roots( - bodies, - 2.0, - exits, - {0: (2,), 1: (2,), 2: (0, 1)}, + bodies, 2.0, exits, {0: (2,), 1: (2,), 2: (0, 1)}, np.ones(3, dtype=np.int32), ) == {1: (0, 1)} @@ -2743,17 +2612,15 @@ def repaired_topology(*, connected: bool): assert np.all(connected_topology.path_owners == 1) -def test_signal_supported_repair_projects_crossings_regionally_on_eight_megapixels( - monkeypatch, -): +def test_signal_supported_repair_projects_crossings_regionally_on_eight_megapixels(monkeypatch): labels = np.zeros((2000, 4000), dtype=np.int32) bodies = np.zeros_like(labels) response = np.zeros(labels.shape, dtype=np.float32) for index in range(1000): y, x = 5 + 40 * (index // 20), 5 + 200 * (index % 20) bodies[y, x] = index + 1 - labels[y, x + 1 : x + 4] = index + 1 - response[y, x : x + 4] = 5 + labels[y, x + 1:x + 4] = index + 1 + response[y, x:x + 4] = 5 projected_shapes = [] original_projection = _TopologyResult.crossing_core_mask @@ -2761,13 +2628,9 @@ def regional_projection(self, shape, **kwargs): projected_shapes.append(shape) return original_projection(self, shape, **kwargs) - monkeypatch.setattr(_TopologyResult, "crossing_core_mask", regional_projection) + monkeypatch.setattr(_TopologyResult, 'crossing_core_mask', regional_projection) repaired = _repair_signal_supported_skeleton( - labels, - _declared_shaft_admission(response), - labels, - bodies, - minimum_response=3, + labels, _declared_shaft_admission(response), labels, bodies, minimum_response=3, crossing_topology=_empty_topology(), ) np.testing.assert_array_equal(repaired, np.where(bodies > 0, bodies, labels)) @@ -2786,21 +2649,14 @@ def test_signal_supported_repair_reserves_core_from_unrelated_earlier_owner(): response[20, 20] = 150.0 regions = np.where(response > 0, 1, 0) topology = replace( - _empty_topology(), - path_owners=np.array([2, 2, 0]), - path_coordinates=( - np.array([[20, x] for x in range(5, 20)]), - np.array([[20, x] for x in range(21, 36)]), - np.array([[20, 19], [20, 20], [20, 21]]), - ), + _empty_topology(), path_owners=np.array([2, 2, 0]), + path_coordinates=(np.array([[20, x] for x in range(5, 20)]), + np.array([[20, x] for x in range(21, 36)]), + np.array([[20, 19], [20, 20], [20, 21]])), resolved_crossings=(_ResolvedCrossing(0, (0, 1), (2,)),), ) repaired = _repair_signal_supported_skeleton( - labels, - _declared_shaft_admission(response), - regions, - bodies, - minimum_response=100.0, + labels, _declared_shaft_admission(response), regions, bodies, minimum_response=100.0, crossing_topology=topology, ) assert np.all(repaired[20, 8:20] == 2) @@ -2810,20 +2666,15 @@ def test_signal_supported_repair_reserves_core_from_unrelated_earlier_owner(): reversed_topology = replace(topology, path_owners=np.array([1, 1, 0])) reverse = lambda image: np.where(image > 0, 3 - image, 0) reversed_repair = _repair_signal_supported_skeleton( - reverse(labels), - _declared_shaft_admission(response), - reverse(regions), - reverse(bodies), - minimum_response=100.0, - crossing_topology=reversed_topology, + reverse(labels), _declared_shaft_admission(response), reverse(regions), reverse(bodies), + minimum_response=100.0, crossing_topology=reversed_topology, ) np.testing.assert_array_equal(reversed_repair, reverse(repaired)) def test_crossing_resolution_keeps_branch_arms_in_shared_support_declaration(): topology = replace( - _empty_topology(), - path_owners=np.array([1, 1, 2, 2, 2, 0]), + _empty_topology(), path_owners=np.array([1, 1, 2, 2, 2, 0]), path_coordinates=tuple(np.array([[4, 5]]) for _ in range(6)), resolved_crossings=(_ResolvedCrossing(0, (0, 1, 2, 3, 4), (5,)),), ) @@ -2983,10 +2834,7 @@ def test_morphology_retains_topology_owners_when_shared_endpoints_collide(): assert rendered[tuple(paths[1].T)].tolist() == [1, 2, 1] graph = _build_neurite_morphology_graph( - topology, - bodies, - coordinate_spacing=SourceVoxelSpacing((1.0, 1.0)), - outgrowth_width_px=1.0, + topology, bodies, coordinate_spacing=SourceVoxelSpacing((1.0, 1.0)), outgrowth_width_px=1.0 ) graph.require_directed_forest() assert len(graph.edges) == 2 diff --git a/tests/unit/test_neuroncyto_ii_neurite_demo.py b/tests/unit/test_neuroncyto_ii_neurite_demo.py index ccb710f768..a93141116e 100644 --- a/tests/unit/test_neuroncyto_ii_neurite_demo.py +++ b/tests/unit/test_neuroncyto_ii_neurite_demo.py @@ -150,14 +150,10 @@ def test_neuroncyto_demo_declares_exact_crossover_channel_semantics( "neurite_secondary_ownership": ArtifactViewerStreaming.ON_DEMAND, "neurite_topology_dropped_trace": ArtifactViewerStreaming.ON_DEMAND, "neurite_topology_added_trace": ArtifactViewerStreaming.ON_DEMAND, - **{ - spec.name: ArtifactViewerStreaming.ON_DEMAND - for spec in NeuriteAdmissionPlanes.artifact_specs() - }, - **{ - spec.name: ArtifactViewerStreaming.ON_DEMAND - for spec in NeuriteOwnershipPlanes.artifact_specs() - }, + **{spec.name: ArtifactViewerStreaming.ON_DEMAND + for spec in NeuriteAdmissionPlanes.artifact_specs()}, + **{spec.name: ArtifactViewerStreaming.ON_DEMAND + for spec in NeuriteOwnershipPlanes.artifact_specs()}, "neurite_morphology": ArtifactViewerStreaming.AUTOMATIC, } @@ -212,14 +208,10 @@ def test_neuroncyto_demo_compiles_exact_loose_tiff_pair(tmp_path: Path) -> None: "neurite_secondary_ownership": ArtifactViewerStreaming.ON_DEMAND, "neurite_topology_dropped_trace": ArtifactViewerStreaming.ON_DEMAND, "neurite_topology_added_trace": ArtifactViewerStreaming.ON_DEMAND, - **{ - spec.name: ArtifactViewerStreaming.ON_DEMAND - for spec in NeuriteAdmissionPlanes.artifact_specs() - }, - **{ - spec.name: ArtifactViewerStreaming.ON_DEMAND - for spec in NeuriteOwnershipPlanes.artifact_specs() - }, + **{spec.name: ArtifactViewerStreaming.ON_DEMAND + for spec in NeuriteAdmissionPlanes.artifact_specs()}, + **{spec.name: ArtifactViewerStreaming.ON_DEMAND + for spec in NeuriteOwnershipPlanes.artifact_specs()}, "neurite_morphology": ArtifactViewerStreaming.AUTOMATIC, } diff --git a/tests/unit/test_nlm_slice_contract.py b/tests/unit/test_nlm_slice_contract.py index bd4d0dd2e2..437e6c3082 100644 --- a/tests/unit/test_nlm_slice_contract.py +++ b/tests/unit/test_nlm_slice_contract.py @@ -21,9 +21,7 @@ def _torch_nlm(): @pytest.mark.parametrize( ("func", "expected_default"), ( - pytest.param( - lambda: jax_nlm_processor.non_local_means_denoise_jax, False, id="jax" - ), + pytest.param(lambda: jax_nlm_processor.non_local_means_denoise_jax, False, id="jax"), pytest.param( lambda: _torch_nlm().non_local_means_denoise_torch, True, diff --git a/tests/unit/test_openhcs_adapter.py b/tests/unit/test_openhcs_adapter.py index 56aabeab8d..c2f94a805c 100644 --- a/tests/unit/test_openhcs_adapter.py +++ b/tests/unit/test_openhcs_adapter.py @@ -389,15 +389,13 @@ def poll_status(self, execution_id): == FakeZMQExecutionClient.submitted[1].pipeline_code ) assert all( - submission.global_config_code - == ConfigDocumentAuthority.render( + submission.global_config_code == ConfigDocumentAuthority.render( global_config, expected_config_type=GlobalPipelineConfig ) for submission in FakeZMQExecutionClient.submitted ) assert [ - submission.compile_control.compile_only - for submission in FakeZMQExecutionClient.submitted + submission.compile_control.compile_only for submission in FakeZMQExecutionClient.submitted ] == [ True, False, diff --git a/tests/unit/test_openhcs_measured_run.py b/tests/unit/test_openhcs_measured_run.py index 00a3464097..b5f47b386c 100644 --- a/tests/unit/test_openhcs_measured_run.py +++ b/tests/unit/test_openhcs_measured_run.py @@ -23,10 +23,7 @@ from openhcs.core.orchestrator.execution_result import ExecutionResult from openhcs.core.pipeline_document import PipelineDocumentCodec from openhcs.runtime.zmq_application import OPENHCS_ENDPOINT_APPLICATION -from openhcs.runtime.zmq_execution_client import ( - OpenHCSExecutionSubmission, - ZMQExecutionRequestBuilder, -) +from openhcs.runtime.zmq_execution_client import OpenHCSExecutionSubmission, ZMQExecutionRequestBuilder from openhcs.runtime.zmq_execution_observation import ZMQRuntimeExecutionOutcomeExport from openhcs.runtime.zmq_execution_signature import ( ZMQAuxiliaryExecutionParams, @@ -240,10 +237,7 @@ def submit_compile(self, submission): } def submit_pipeline(self, submission): - assert ( - submission.compile_control.compile_artifact_id - == f"compile-{self.run_number}" - ) + assert submission.compile_control.compile_artifact_id == f"compile-{self.run_number}" observer({"phase": "axis_started", "timestamp": 11.0}) return { "status": "accepted", diff --git a/tests/unit/test_orchestrator_lane_planning.py b/tests/unit/test_orchestrator_lane_planning.py index e0bd2ab32f..927331ea22 100644 --- a/tests/unit/test_orchestrator_lane_planning.py +++ b/tests/unit/test_orchestrator_lane_planning.py @@ -1125,9 +1125,7 @@ def test_analysis_consolidation_propagates_runtime_failures(monkeypatch): "consolidate_runtime_analysis_table_output_groups", lambda **kwargs: ([], [("results", "invalid CSV")]), ) - hook = AnalysisConsolidationHook( - AnalysisConsolidationConfig(), PlateMetadataConfig() - ) + hook = AnalysisConsolidationHook(AnalysisConsolidationConfig(), PlateMetadataConfig()) with pytest.raises(RuntimeError, match="invalid CSV"): hook.run({"A01": context}, inputs) @@ -1383,7 +1381,9 @@ def force_stop(self): self.running = False progress_queue = SimpleNamespace( - put=lambda raw_event: events.append(raw_event.phase.value) + put=lambda raw_event: events.append( + raw_event.phase.value + ) ) results = _execute_with_visualizer( monkeypatch, diff --git a/tests/unit/test_path_planner_axis_source_scope.py b/tests/unit/test_path_planner_axis_source_scope.py index e9a598e0e1..88532d3d46 100644 --- a/tests/unit/test_path_planner_axis_source_scope.py +++ b/tests/unit/test_path_planner_axis_source_scope.py @@ -46,9 +46,7 @@ def test_path_planner_scopes_source_metadata_to_compilation_axis() -> None: ).source_metadata_by_path.values() ), step_count=0, - pipeline=ResolvedPipelineDefinition( - steps=(), step_scope_ids={}, step_provenance={} - ), + pipeline=ResolvedPipelineDefinition(steps=(), step_scope_ids={}, step_provenance={}), ) planner = PathPlanner(session) diff --git a/tests/unit/test_path_planner_materialization.py b/tests/unit/test_path_planner_materialization.py index 6a44d62239..10fe29a3b3 100644 --- a/tests/unit/test_path_planner_materialization.py +++ b/tests/unit/test_path_planner_materialization.py @@ -196,31 +196,25 @@ def resolve(self, handler, dataset_root): def _prepare_step_declarations(planner, step, step_index): context = replace( - planner.artifact_context, - step_name=step.name, - step_index=step_index, + planner.artifact_context, step_name=step.name, step_index=step_index, ).with_source_binding_scope( source_bindings=step.source_bindings, group_by=PathPlannerExecutionGroups.normalized_group_by(step), input_source=step.processing_config.input_source, ) graph = extract_artifact_declarations( - step.func, - invocation_contract_provider=planner.invocation_contract_provider, + step.func, invocation_contract_provider=planner.invocation_contract_provider, step_context=context, ) contracts = tuple(item.contract for item in graph.pattern.iter_items()) - return ( - graph, - graph.pattern, - FunctionStepExecutionScope.require_uniform(contracts), - contracts, - ) + return graph, graph.pattern, FunctionStepExecutionScope.require_uniform(contracts), contracts def _compile_metadata_pattern(planner, pattern): snapshot = _resolved_step(func=pattern) - declarations, pattern, _, _ = _prepare_step_declarations(planner, snapshot, 3) + declarations, pattern, _, _ = _prepare_step_declarations(planner, + snapshot, 3 + ) input_plans = planner.artifacts.process_artifact_inputs( declarations, 3, @@ -378,8 +372,8 @@ def metadata_consumer(image, grid_dimensions): bindings=(NamedSourceBinding(alias="DNA"),), ), ) - declarations, pattern, _, contracts = _prepare_step_declarations( - planner, snapshot, 3 + declarations, pattern, _, contracts = _prepare_step_declarations(planner, + snapshot, 3 ) execution_bindings = CompiledSourceBindingPlan.from_contracts( snapshot.source_bindings, @@ -437,13 +431,9 @@ def metadata_consumer(image, grid_dimensions): "artifact_name, expected", [("pixel_size", 1.3556), ("engineering_exposure_duration", 17.25)], ) -def test_registered_metadata_provider_value_reaches_runtime_unchanged( - artifact_name, expected -): +def test_registered_metadata_provider_value_reaches_runtime_unchanged(artifact_name, expected): received = [] - spec = ArtifactSpec.input( - artifact_name, SpecialArtifactType, parameter_name="calibration" - ) + spec = ArtifactSpec.input(artifact_name, SpecialArtifactType, parameter_name="calibration") @numpy @artifact_inputs(spec) @@ -455,18 +445,13 @@ def metadata_consumer(image, *, calibration): planner.ctx.microscope_handler = _EngineeringMetadataHandler(filemanager=object()) planner.ctx.dataset_root = planner.dataset_root assert planner.ctx.microscope_handler.can_resolve_metadata_artifact(artifact_name) - compiled, input_plans = _compile_metadata_pattern( - planner, (metadata_consumer, {"calibration": -7.0}) - ) + compiled, input_plans = _compile_metadata_pattern(planner, (metadata_consumer, {"calibration": -7.0})) (invocation,) = compiled.default_group.invocations (edge,) = invocation.artifact_input_edges assert isinstance(edge, CompiledMetadataArtifactInputEdgePlan) assert input_plans == {} assert invocation.kwargs_dict == {"calibration": expected} - assert all( - binding.parameter_name != "calibration" - for binding in invocation.runtime_parameter_bindings - ) + assert all(binding.parameter_name != "calibration" for binding in invocation.runtime_parameter_bindings) _execute_compiled_metadata_pattern(compiled) assert received == [expected] @@ -491,8 +476,7 @@ def resolve(self, handler, dataset_root): return value class CheckedExposureMetadataProvider( - ResolutionReceipt, - PositiveMetadataValue, + ResolutionReceipt, PositiveMetadataValue, _EngineeringExposureMetadataArtifactProvider, ): artifact_name = "engineering_checked_exposure_duration" @@ -560,9 +544,7 @@ def metadata_consumer(image, pixel_size): planner.ctx.dataset_root = planner.dataset_root compiled, _ = _compile_metadata_pattern(planner, metadata_consumer) if value is None: - with pytest.raises( - ValueError, match="Required metadata artifact .* has no value" - ): + with pytest.raises(ValueError, match="Required metadata artifact .* has no value"): _execute_compiled_metadata_pattern(compiled) else: _execute_compiled_metadata_pattern(compiled) @@ -576,9 +558,7 @@ def metadata_consumer(image, unknown_metadata): planner = _artifact_planner_stub() with pytest.raises(MissingArtifactInputError): - _compile_metadata_pattern( - planner, (metadata_consumer, {"unknown_metadata": 1.3556}) - ) + _compile_metadata_pattern(planner, (metadata_consumer, {"unknown_metadata": 1.3556})) def test_exact_producer_takes_precedence_over_metadata_provider(): @@ -591,16 +571,11 @@ def metadata_consumer(image, pixel_size): planner = _artifact_planner_stub() planner.ctx.microscope_handler = _EngineeringMetadataHandler(filemanager=object()) planner.ctx.dataset_root = planner.dataset_root - producer = _record_declared_output( - planner, - ArtifactOutputPlan( - name="pixel_size", - path="/memory/pixel_size.pkl", - artifact_type=SpecialArtifactType, - producer_step_index=2, - producer_step_scope_id="plate::functionstep_2", - ), - ) + producer = _record_declared_output(planner, ArtifactOutputPlan( + name="pixel_size", path="/memory/pixel_size.pkl", + artifact_type=SpecialArtifactType, producer_step_index=2, + producer_step_scope_id="plate::functionstep_2", + )) compiled, input_plans = _compile_metadata_pattern(planner, metadata_consumer) (invocation,) = compiled.default_group.invocations (edge,) = invocation.artifact_input_edges @@ -814,13 +789,12 @@ def __call__(self, invocation, step_context): match="Conflicting compiled invocation output artifact sidecar role", ): compile_function_pattern( - pattern, - {}, - output_plans, + pattern, {}, output_plans, invocation_contract_provider=planner.invocation_contract_provider, ).coalesced_artifact_output_specs() + def test_materialization_collision_updates_results_dir_and_config(): planner = PathPlanner.__new__(PathPlanner) planner.dataset_root = Path("/data/plate1") @@ -1091,7 +1065,9 @@ def test_artifact_output_plan_only_preserves_explicit_source_stack_scope(): ) assert outputs[output_specs[0].ref()].variable_components == () - assert outputs[output_specs[1].ref()].variable_components == (Microscopy.Channel,) + assert outputs[output_specs[1].ref()].variable_components == ( + Microscopy.Channel, + ) def test_artifact_output_source_lookup_combines_repeated_main_flow_inputs(): @@ -1137,8 +1113,7 @@ def test_artifact_output_source_lookup_combines_repeated_main_flow_inputs(): @pytest.mark.parametrize("stored_main_flow", [False, True]) @pytest.mark.parametrize("stored_secondary", [False, True]) def test_compiled_source_edges_only_consume_relation_owned_main_flow( - stored_main_flow, - stored_secondary, + stored_main_flow, stored_secondary, ): source_specs = ( ArtifactSpec.input("DNA", ImageArtifactType), @@ -1188,11 +1163,7 @@ def combine_sources(image, secondary=None): edges = next(compiled.iter_invocations()).artifact_input_edges assert tuple(edge.spec for edge in edges) == source_specs - assert tuple((edge.main_flow_projection is not None) for edge in edges) == ( - True, - False, - False, - ) + assert tuple((edge.main_flow_projection is not None) for edge in edges) == (True, False, False) assert edges[0].storage_plan is None assert edges[0].projection is None assert edges[1].storage_plan is stored_inputs.get(source_specs[1].ref()) @@ -1278,10 +1249,10 @@ def test_implicit_native_main_flow_provenance_drives_artifact_owned_scope(): ) native_pattern = compile_function_pattern(percentile_normalize, {}, {}) native_invocation = next(native_pattern.iter_invocations()) - planner.artifact_context = extract_artifact_declarations( - percentile_normalize - ).advance_declaration_context( - planner.artifact_context, + planner.artifact_context = ( + extract_artifact_declarations(percentile_normalize).advance_declaration_context( + planner.artifact_context, + ) ) cursor_name = unnamed_main_flow_artifact_name(0, native_invocation.key) cursor = ArtifactSpec.input(cursor_name, ImageArtifactType) @@ -1527,7 +1498,9 @@ def test_artifact_output_source_lookup_ignores_shared_input_broadcast_projection execution_scope=FunctionStepExecutionScope.AXIS, ) - assert projected[ambiguous_output.ref()].variable_components == (Microscopy.Site,) + assert projected[ambiguous_output.ref()].variable_components == ( + Microscopy.Site, + ) def test_enabled_source_bindings_preserve_previous_step_main_flow(): @@ -1653,7 +1626,9 @@ def identify(main_image, Stain1): assert maps.group_scope.keys == expected_channel_scope.keys assert maps.group_scope.component is expected_channel_scope.component assert maps.outputs[object_output.ref()].group_keys == ("1",) - assert maps.outputs[object_output.ref()].variable_components == (Microscopy.Site,) + assert maps.outputs[object_output.ref()].variable_components == ( + Microscopy.Site, + ) edge = next(compiled.iter_invocations()).artifact_input_edges[0] assert edge.projection.invocation_scope == expected_channel_scope assert edge.projection.component_scope(Microscopy.Channel) == ( @@ -1891,17 +1866,11 @@ def identify(image): scopes = planner.artifacts.output_groups_from_declared_relations( namespaced, - group_scope=PathPlannerGroupScope.from_raw( - ("1", "2"), component=Microscopy.Channel - ), - relation_source_scopes={}, - consumer_variable_components=ComponentSet(), - step_index=0, - step_name="identify", + group_scope=PathPlannerGroupScope.from_raw(("1", "2"), component=Microscopy.Channel), + relation_source_scopes={}, consumer_variable_components=ComponentSet(), + step_index=0, step_name="identify", ) - assert scopes[ - ArtifactSpec.output("nuclei", ObjectLabelsArtifactType).ref() - ].keys == ("1", "2") + assert scopes[ArtifactSpec.output("nuclei", ObjectLabelsArtifactType).ref()].keys == ("1", "2") def test_artifact_graph_preserves_same_name_outputs_of_different_types(): @@ -1950,13 +1919,9 @@ def declared_artifact_owner(image): assert declarations.output_groups[output_ref] == {None} scopes = planner.artifacts.output_groups_from_declared_relations( namespaced, - group_scope=PathPlannerGroupScope.from_raw( - ("1", "2"), component=Microscopy.Channel - ), - relation_source_scopes={}, - consumer_variable_components=ComponentSet(), - step_index=0, - step_name="correct_illumination", + group_scope=PathPlannerGroupScope.from_raw(("1", "2"), component=Microscopy.Channel), + relation_source_scopes={}, consumer_variable_components=ComponentSet(), + step_index=0, step_name="correct_illumination", ) assert scopes[output_ref].keys == ("1", "2") @@ -2286,11 +2251,7 @@ def filter_objects(main_image, MeasureObjectIntensity_3_measurements): consumer_maps.outputs, consumer_maps.relation_source_scopes, consumer_maps.group_scope, - declarations=extract_artifact_declarations( - filter_objects, - invocation_contract_provider=planner.invocation_contract_provider, - step_context=planner.artifact_context, - ), + declarations=extract_artifact_declarations(filter_objects, invocation_contract_provider=planner.invocation_contract_provider, step_context=planner.artifact_context), ) edge = next(compiled.iter_invocations()).artifact_input_edges[0] @@ -3442,10 +3403,8 @@ def declared_artifact_owner(image): ) pipeline = ResolvedPipelineDefinition( - steps=(snapshot,), - step_scope_ids={0: "plate::identify_cells"}, - step_provenance={0: {}}, - declaration_provider=declarations_for_invocation, + steps=(snapshot,), step_scope_ids={0: "plate::identify_cells"}, + step_provenance={0: {}}, declaration_provider=declarations_for_invocation, ) (declarations,) = pipeline.artifact_graphs func_pattern = declarations.pattern @@ -3511,10 +3470,11 @@ def filter_objects(image, *, runtime): } ) - _declarations, _pattern, execution_scope, _contracts = _prepare_step_declarations( - planner, - snapshot, - 2, + _declarations, _pattern, execution_scope, _contracts = ( + _prepare_step_declarations(planner, + snapshot, + 2, + ) ) execution_groups = planner.execution_groups.get_execution_groups( snapshot, @@ -3531,10 +3491,11 @@ def filter_objects(image, *, runtime): "1": [filter_objects], "2": [filter_objects], } - _declarations, _pattern, execution_scope, _contracts = _prepare_step_declarations( - planner, - snapshot, - 2, + _declarations, _pattern, execution_scope, _contracts = ( + _prepare_step_declarations(planner, + snapshot, + 2, + ) ) execution_groups = planner.execution_groups.get_execution_groups( snapshot, @@ -4988,11 +4949,7 @@ def consume(main_image, image): maps.outputs, maps.relation_source_scopes, maps.group_scope, - declarations=extract_artifact_declarations( - consume, - invocation_contract_provider=planner.invocation_contract_provider, - step_context=planner.artifact_context, - ), + declarations=extract_artifact_declarations(consume, invocation_contract_provider=planner.invocation_contract_provider, step_context=planner.artifact_context), ) @@ -5054,11 +5011,7 @@ def consume(main_image, image): maps.outputs, maps.relation_source_scopes, maps.group_scope, - declarations=extract_artifact_declarations( - consume, - invocation_contract_provider=planner.invocation_contract_provider, - step_context=planner.artifact_context, - ), + declarations=extract_artifact_declarations(consume, invocation_contract_provider=planner.invocation_contract_provider, step_context=planner.artifact_context), ) edge = next(compiled.iter_invocations()).artifact_input_edges[0] @@ -5163,7 +5116,9 @@ def produce(image, source_value): PathPlannerComponentScopes.empty(), source_bindings=source_bindings, ) - declarations = extract_artifact_declarations(produce) + declarations = ( + extract_artifact_declarations(produce) + ) maps = planner.artifacts.compile_plan_maps( snapshot, step_index, diff --git a/tests/unit/test_payload_axis_compile_gate.py b/tests/unit/test_payload_axis_compile_gate.py index 4c443afddb..ea2ba16dc5 100644 --- a/tests/unit/test_payload_axis_compile_gate.py +++ b/tests/unit/test_payload_axis_compile_gate.py @@ -119,15 +119,7 @@ def _session( ) projections = { path.name: SourcePlaneProjection( - address=OpenHCSPlaneAddress( - ( - (Microscopy.Well, "A01"), - (Microscopy.Site, index), - (Microscopy.Channel, 1), - (Microscopy.ZIndex, 1), - (Microscopy.Timepoint, 1), - ) - ), + address=OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, index), (Microscopy.Channel, 1), (Microscopy.ZIndex, 1), (Microscopy.Timepoint, 1))), ref=SourcePixelRef("disk", str(path)), source_alias=binding.alias, ) @@ -210,7 +202,9 @@ def test_all_rgb_sources_prove_color_to_gray_payload_axis_requirement( photometric="rgb", ) - PipelineCompiler.validate_payload_axis_requirements(_session(tmp_path, paths)) + PipelineCompiler.validate_payload_axis_requirements( + _session(tmp_path, paths) + ) def test_grouped_color_requirement_inspects_only_compatible_binding( @@ -254,15 +248,7 @@ def identity(image): ) projections = { path.name: SourcePlaneProjection( - address=OpenHCSPlaneAddress( - ( - (Microscopy.Well, "A01"), - (Microscopy.Site, index), - (Microscopy.Channel, binding.component_identity[0].value), - (Microscopy.ZIndex, 1), - (Microscopy.Timepoint, 1), - ) - ), + address=OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, index), (Microscopy.Channel, binding.component_identity[0].value), (Microscopy.ZIndex, 1), (Microscopy.Timepoint, 1))), ref=SourcePixelRef("disk", str(path)), source_alias=binding.alias, ) @@ -353,7 +339,9 @@ def test_unknown_or_unreadable_exact_source_fails_compile_closed( path.write_bytes(b"not an image header") with pytest.raises(ValueError, match="failed before execution") as error: - PipelineCompiler.validate_payload_axis_requirements(_session(tmp_path, (path,))) + PipelineCompiler.validate_payload_axis_requirements( + _session(tmp_path, (path,)) + ) assert str(path) in str(error.value) or str(path.name) in str(error.value) assert "ColourAxis" in str(error.value) @@ -625,28 +613,12 @@ def test_unknown_routed_group_cannot_fall_back_to_all_primary_sources( ) projections = { grayscale_path.name: SourcePlaneProjection( - address=OpenHCSPlaneAddress( - ( - (Microscopy.Well, "A01"), - (Microscopy.Site, 1), - (Microscopy.Channel, 1), - (Microscopy.ZIndex, 1), - (Microscopy.Timepoint, 1), - ) - ), + address=OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, 1), (Microscopy.Channel, 1), (Microscopy.ZIndex, 1), (Microscopy.Timepoint, 1))), ref=SourcePixelRef("disk", str(grayscale_path)), source_alias=grayscale_binding.alias, ), rgb_path.name: SourcePlaneProjection( - address=OpenHCSPlaneAddress( - ( - (Microscopy.Well, "A01"), - (Microscopy.Site, 2), - (Microscopy.Channel, 2), - (Microscopy.ZIndex, 1), - (Microscopy.Timepoint, 1), - ) - ), + address=OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, 2), (Microscopy.Channel, 2), (Microscopy.ZIndex, 1), (Microscopy.Timepoint, 1))), ref=SourcePixelRef("disk", str(rgb_path)), source_alias=rgb_binding.alias, ), diff --git a/tests/unit/test_persisted_image_metadata.py b/tests/unit/test_persisted_image_metadata.py index b496b4904b..125cadcbf9 100644 --- a/tests/unit/test_persisted_image_metadata.py +++ b/tests/unit/test_persisted_image_metadata.py @@ -163,11 +163,8 @@ def test_saved_two_channel_planes_keep_runtime_axis_not_contributor_axis( def receive_projection_entries(writer, metadata_path, subdirectory, entries): target = RuntimeArtifactMetadataTarget( output_dir=Path(metadata_path).parent / subdirectory, - dataset_root=str(Path(metadata_path).parent), - sub_dir=subdirectory, - backend="disk", - results_dir=None, - create_openhcs_metadata=False, + dataset_root=str(Path(metadata_path).parent), sub_dir=subdirectory, + backend="disk", results_dir=None, create_openhcs_metadata=False, ) writer.reconcile_completed_dataset( metadata_path, {}, produced_entries_by_target={target: entries} @@ -182,20 +179,11 @@ def test_atomic_perwell_projection_merge_preserves_all_records(tmp_path): def merge(well): virtual_path = f"{well}.tif" projection = SourcePlaneProjection( - OpenHCSPlaneAddress( - ( - (Microscopy.Well, well), - (Microscopy.Site, 1), - (Microscopy.Channel, 1), - (Microscopy.ZIndex, 1), - (Microscopy.Timepoint, 1), - ) - ), + OpenHCSPlaneAddress(((Microscopy.Well, well), (Microscopy.Site, 1), (Microscopy.Channel, 1), (Microscopy.ZIndex, 1), (Microscopy.Timepoint, 1))), SourcePixelRef("disk", virtual_path), image_metadata=metadata_fixture(), ) - receive_projection_entries( - writer, + receive_projection_entries(writer, path, ".", VirtualWorkspaceSourceProjectionEntries.from_projection_paths( @@ -217,15 +205,7 @@ def calibrated_projection(well, path, pixel_size): source_metadata = {} spacing.merge_into(source_metadata, path=path) return SourcePlaneProjection( - OpenHCSPlaneAddress( - ( - (Microscopy.Well, well), - (Microscopy.Site, 1), - (Microscopy.Channel, 1), - (Microscopy.ZIndex, 1), - (Microscopy.Timepoint, 1), - ) - ), + OpenHCSPlaneAddress(((Microscopy.Well, well), (Microscopy.Site, 1), (Microscopy.Channel, 1), (Microscopy.ZIndex, 1), (Microscopy.Timepoint, 1))), SourcePixelRef("disk", path), source_metadata=source_metadata, image_metadata=metadata_fixture().replace_fields(source_voxel_spacing=spacing), @@ -267,10 +247,8 @@ def test_atomic_projection_merge_refreshes_geometry_from_current_document( writer.merge_subdirectory_metadata(path, {"images": second_fields}) expected_pixel_size = 2.0 else: - receive_projection_entries( - writer, - path, - "images", + receive_projection_entries(writer, + path, "images", VirtualWorkspaceSourceProjectionEntries.from_projection_paths( ((second, "images/second.tif"),) ), @@ -292,11 +270,9 @@ def test_final_publication_prunes_geometry_and_preserves_other_directories(tmp_p (deleted, "images/deleted.tif"), (elsewhere, "other/retained.tif"), ) - receive_projection_entries( - writer, - path, - "images", - VirtualWorkspaceSourceProjectionEntries.from_projection_paths(projection_paths), + receive_projection_entries(writer, + path, "images", + VirtualWorkspaceSourceProjectionEntries.from_projection_paths(projection_paths) ) before = json.loads(path.read_text())[FIELDS.SUBDIRECTORIES]["images"] assert before[FIELDS.PIXEL_SIZE] == 1.0 @@ -413,8 +389,7 @@ def test_invalid_unreplaced_projection_aborts_typed_transaction(tmp_path, publis path = tmp_path / "openhcs_metadata.json" writer = AtomicMetadataWriter() writer.replace_subdirectory_metadata( - path, - "images", + path, "images", {FIELDS.SOURCE_PROJECTION: [{"virtual_path": "images/broken.tif"}]}, ) replacement = calibrated_projection("A02", "images/new.tif", 0.5) @@ -463,10 +438,9 @@ def test_typed_transaction_retains_wire_fields_and_derives_workspace_views( assert "legacy.tif" not in subdir[FIELDS.WORKSPACE_MAPPING] assert "legacy.tif" not in subdir[FIELDS.SOURCE_METADATA] assert subdir[FIELDS.SOURCE_METADATA]["images/original.tif"]["well"] == "A01" - assert ( - "manual_override" - not in subdir[FIELDS.SOURCE_METADATA]["images/original.tif"] - ) + assert "manual_override" not in subdir[FIELDS.SOURCE_METADATA][ + "images/original.tif" + ] assert "well" not in subdir[FIELDS.SOURCE_PROJECTION][0]["source_metadata"] publish_projection_inventory( writer, path, None, ("images/original.tif", "images/new.tif") @@ -545,14 +519,7 @@ def test_native_header_reload_preserves_declared_crop_and_alias(tmp_path, origin context = ImagePayloadSourceMetadataContext(SourceImageIdentity(str(path), {})) from openhcs.core.source_bindings import NamedSourceBinding - current = ( - NamedSourceBinding(alias="neurite") - .apply_loaded_payload( - reloaded, - context, - ) - .metadata - ) + current = NamedSourceBinding(alias="neurite").apply_loaded_payload( reloaded, context, ).metadata assert current.source_dtype == "float32" assert current.source_spatial_domain == metadata.source_spatial_domain assert current.source_image_names == ("neurite",) @@ -578,12 +545,10 @@ def test_native_mosaic_replaces_incompatible_old_tile_domain_without_losing_sour reloaded = ImageMetadataPayload( image_format.read(path), image_format.persisted_metadata(path, payload) ) - current = ( - NamedSourceBinding(alias="neurite").apply_loaded_payload( + current = (NamedSourceBinding(alias="neurite").apply_loaded_payload( reloaded, ImagePayloadSourceMetadataContext(SourceImageIdentity(str(path), {})), - ) - ).metadata + )).metadata assert current.source_spatial_domain.origin_yx == (0, 0) assert current.source_spatial_domain.source_shape_yx == (10, 16) assert current.source_image_names == ("neurite",) diff --git a/tests/unit/test_persisted_roi_only_publication.py b/tests/unit/test_persisted_roi_only_publication.py index f2bff30df0..36e68e53c6 100644 --- a/tests/unit/test_persisted_roi_only_publication.py +++ b/tests/unit/test_persisted_roi_only_publication.py @@ -36,11 +36,7 @@ ProducedImageMetadataCapability, RuntimeArtifactMetadataTarget, ) -from openhcs.core.virtual_workspace_metadata import ( - AtomicMetadataWriter, - METADATA_CONFIG, - MetadataWriteError, -) +from openhcs.core.virtual_workspace_metadata import AtomicMetadataWriter, METADATA_CONFIG, MetadataWriteError from openhcs.processing.materialization.core import ( MaterializationSpec, _WRITERS_BY_OPTIONS, @@ -59,8 +55,7 @@ def _publish_saved_step(context, plan, *, artifact_materializations=()): METADATA_CONFIG.metadata_path(target.dataset_root) for target in facts ): entries = { - target: value - for target, value in facts.items() + target: value for target, value in facts.items() if METADATA_CONFIG.metadata_path(target.dataset_root) == metadata_path } AtomicMetadataWriter().reconcile_completed_dataset( @@ -108,13 +103,8 @@ def plan_for(plate, backend=Backend.DISK.value): def save_archive_outcome( - context, - plate, - destination, - monkeypatch, - *, - filename="independent-outline.roi.zip", - content=b"synthetic admission only", + context, plate, destination, monkeypatch, + *, filename="independent-outline.roi.zip", content=b"synthetic admission only", backend=Backend.DISK.value, ): """Use the original declared bundle writer, batch and disk save boundary.""" @@ -235,7 +225,9 @@ def test_saved_roi_only_batch_publishes_and_reconciles_without_rendering( artifact_materializations=(outcome,), ) assert tuple(target.output_dir for target in targets) == (plate / destination,) - _publish_saved_step(publication_context, plan, artifact_materializations=(outcome,)) + _publish_saved_step( + publication_context, plan, artifact_materializations=(outcome,) + ) assert result_paths(plate, publication_context) == (output.path,) published = json.loads(METADATA_CONFIG.metadata_path(plate).read_text()) assert published["subdirectories"][destination]["image_files"] == [] @@ -313,91 +305,58 @@ def test_shared_writer_accepts_result_only_but_not_unaddressed_images( def test_measurement_only_step_does_not_reconcile_pending_image_producer( - tmp_path, - publication_context, - monkeypatch, + tmp_path, publication_context, monkeypatch, ): from polystore.virtual_workspace import SourcePixelRef from openhcs.core.artifacts import ImageArtifactType - from openhcs.core.source_projection import ( - OpenHCSPlaneAddress, - SourceArtifactProjection, - ) + from openhcs.core.source_projection import OpenHCSPlaneAddress, SourceArtifactProjection from openhcs.core.virtual_workspace_metadata import ( - AtomicMetadataWriter, - FIELDS, - VirtualWorkspaceSourceProjectionEntries, + AtomicMetadataWriter, FIELDS, VirtualWorkspaceSourceProjectionEntries, ) plate = tmp_path / "plate" destination = "shared_results" outcome, table_output = save_archive_outcome( - publication_context, - plate, - destination, - monkeypatch, - filename="measurements.csv", - content=b"ObjectNumber,Area\n1,6\n", + publication_context, plate, destination, monkeypatch, + filename="measurements.csv", content=b"ObjectNumber,Area\n1,6\n", ) pending = plate / destination / "independent-image.tif" # This lifecycle fixture never reads or derives addresses from the bytes. pending.write_bytes(b"producer pixel save precedes its typed publication") plan = plan_for(plate) (target,) = OpenHCSMetadataTarget.for_execution( - publication_context, - plan, - artifact_materializations=(outcome,), + publication_context, plan, artifact_materializations=(outcome,), ) assert target.produced_projection_entries(publication_context, plan).entries == {} staged = OpenHCSMetadataTarget.observe_for_step( - publication_context, - plan, - artifact_materializations=(outcome,), + publication_context, plan, artifact_materializations=(outcome,), ) metadata_path = METADATA_CONFIG.metadata_path(plate) assert not metadata_path.exists() assert table_output.path == str(plate / destination / "measurements.csv") - assert ( - plate / destination / "measurements.csv" - ).read_bytes() == b"ObjectNumber,Area\n1,6\n" + assert (plate / destination / "measurements.csv").read_bytes() == b"ObjectNumber,Area\n1,6\n" with pytest.raises(MetadataWriteError, match="lack typed produced addresses"): AtomicMetadataWriter().reconcile_completed_dataset( - metadata_path, - {target: publication_context}, + metadata_path, {target: publication_context}, produced_entries_by_target=staged, ) assert not metadata_path.exists() virtual_path = str(pending.relative_to(plate)) projection = SourceArtifactProjection( - address=OpenHCSPlaneAddress( - ( - (Microscopy.Well, "A04"), - (Microscopy.Site, 1), - (Microscopy.Channel, 2), - (Microscopy.ZIndex, 1), - (Microscopy.Timepoint, 1), - ) - ), + address=OpenHCSPlaneAddress(((Microscopy.Well, "A04"), (Microscopy.Site, 1), (Microscopy.Channel, 2), (Microscopy.ZIndex, 1), (Microscopy.Timepoint, 1))), ref=SourcePixelRef(Backend.DISK.value, virtual_path), - source_alias="declared_image", - artifact_kind=ImageArtifactType, + source_alias="declared_image", artifact_kind=ImageArtifactType, ) from openhcs.core.source_projection import SourceProjectionMetadataSerializer - AtomicMetadataWriter().replace_subdirectory_metadata( - metadata_path, - destination, - SourceProjectionMetadataSerializer.projection_fields( - ((projection, virtual_path),) - ), + metadata_path, destination, + SourceProjectionMetadataSerializer.projection_fields(((projection, virtual_path),)), ) publication_context.step_plans = {0: plan} OpenHCSMetadataTarget.finalize_completed_dataset({"A01": publication_context}) - reconciled = json.loads(metadata_path.read_text())[FIELDS.SUBDIRECTORIES][ - destination - ] + reconciled = json.loads(metadata_path.read_text())[FIELDS.SUBDIRECTORIES][destination] assert reconciled[FIELDS.IMAGE_FILES] == [virtual_path] assert len(reconciled[FIELDS.SOURCE_PROJECTION]) == 1 restored = VirtualWorkspaceSourceProjectionEntries.from_subdirectory(reconciled) @@ -420,15 +379,7 @@ def test_reconciliation_keeps_artifact_destination_without_results_field( plate = tmp_path / "plate" plate.mkdir() projection = SourceArtifactProjection( - address=OpenHCSPlaneAddress( - ( - (Microscopy.Well, "A01"), - (Microscopy.Site, "1"), - (Microscopy.Channel, "2"), - (Microscopy.ZIndex, "1"), - (Microscopy.Timepoint, "1"), - ) - ), + address=OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, "1"), (Microscopy.Channel, "2"), (Microscopy.ZIndex, "1"), (Microscopy.Timepoint, "1"))), ref=SourcePixelRef(Backend.DISK.value, "physical/acquisition-source.tif"), source_alias="Declared", artifact_kind=MetadataArtifactType, @@ -443,8 +394,7 @@ def test_reconciliation_keeps_artifact_destination_without_results_field( assert handler.analysis_result_directories(plate) == () owner = RuntimeArtifactMetadataTarget.from_plan(plan_for(plate)) assert plate / "nested" in tuple( - target.output_dir - for target in owner.reconciliation_targets(publication_context) + target.output_dir for target in owner.reconciliation_targets(publication_context) ) assert metadata_path.read_bytes() == before @@ -538,16 +488,12 @@ def from_plan(cls, plan): pending.write_bytes(b"never decoded") target.write(publication_context, produced_plan=plan) assert publication_context.publication_audit == [ - ("before", destination), - ("after", destination), - ("before", destination), - ("after", destination), + ("before", destination), ("after", destination), + ("before", destination), ("after", destination), ] # Independent MI hooks reach the inherited empty-step behavior even # while another image producer has not published its typed address. - assert ( - target.produced_projection_entries(publication_context, plan).entries == {} - ) + assert target.produced_projection_entries(publication_context, plan).entries == {} before = METADATA_CONFIG.metadata_path(plate).read_bytes() with pytest.raises(MetadataWriteError, match="lack typed produced addresses"): target.write(publication_context) diff --git a/tests/unit/test_persisted_semantic_metadata.py b/tests/unit/test_persisted_semantic_metadata.py index 66f37bbb7b..5912a215ea 100644 --- a/tests/unit/test_persisted_semantic_metadata.py +++ b/tests/unit/test_persisted_semantic_metadata.py @@ -31,9 +31,7 @@ VirtualWorkspacePathLookup, VirtualWorkspaceSourceProjection, ) -from openhcs.core.virtual_workspace_metadata import ( - VirtualWorkspaceSourceProjectionEntries, -) +from openhcs.core.virtual_workspace_metadata import VirtualWorkspaceSourceProjectionEntries from openhcs.core.dataset_sources.source_schema import SourceSchemaFilenameParser from python_introspect import to_jsonable from openhcs.domains.microscopy.axes import Microscopy @@ -64,15 +62,7 @@ def _projection( source_metadata: Mapping[str, object] | None = None, ) -> SourcePlaneProjection: return SourcePlaneProjection( - address=OpenHCSPlaneAddress( - ( - (Microscopy.Well, "A01"), - (Microscopy.Site, 1), - (Microscopy.Channel, 1), - (Microscopy.ZIndex, 1), - (Microscopy.Timepoint, 1), - ) - ), + address=OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, 1), (Microscopy.Channel, 1), (Microscopy.ZIndex, 1), (Microscopy.Timepoint, 1))), ref=SourcePixelRef("disk", VIRTUAL_PATH), image_metadata=image_metadata, source_metadata={} if source_metadata is None else source_metadata, @@ -145,9 +135,7 @@ def test_publication_retains_typed_provenance_and_exact_wire_document() -> None: original = json.dumps(subdirectory) typed = VirtualWorkspaceSourceProjectionEntries.from_subdirectory(subdirectory) retained = typed.publish_into_subdirectory( - subdirectory, - saved_image_paths=(VIRTUAL_PATH,), - reconcile_directory=".", + subdirectory, saved_image_paths=(VIRTUAL_PATH,), reconcile_directory=".", accepted_entries=typed, ) @@ -176,15 +164,7 @@ def test_image_artifact_projection_round_trips_typed_pixel_metadata() -> None: source_dtype="uint8", ) projection = SourceArtifactProjection( - address=OpenHCSPlaneAddress( - ( - (Microscopy.Well, "A01"), - (Microscopy.Site, 1), - (Microscopy.Channel, 2), - (Microscopy.ZIndex, 1), - (Microscopy.Timepoint, 1), - ) - ), + address=OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, 1), (Microscopy.Channel, 2), (Microscopy.ZIndex, 1), (Microscopy.Timepoint, 1))), ref=SourcePixelRef("disk", "analysis/A01_candidate.checkpoint.tif"), source_alias="neurite_candidate_mask", artifact_kind=ImageArtifactType, diff --git a/tests/unit/test_plane_nlm_declaration.py b/tests/unit/test_plane_nlm_declaration.py index 3ab67024de..7af11cad80 100644 --- a/tests/unit/test_plane_nlm_declaration.py +++ b/tests/unit/test_plane_nlm_declaration.py @@ -26,9 +26,7 @@ from openhcs.core.source_metadata import SourceVoxelSpacing from openhcs.core.source_spatial_domain import SourceSpatialDomain from openhcs.processing.backends.lib_registry.openhcs_registry import OpenHCSRegistry -from openhcs.processing.backends.lib_registry.scikit_image_registry import ( - SkimageRegistry, -) +from openhcs.processing.backends.lib_registry.scikit_image_registry import SkimageRegistry from openhcs.processing.custom_functions.runtime_registry import project_custom_function from openhcs.core.processing_contracts import ( FlexibleContract, @@ -38,13 +36,10 @@ non_local_means_denoise_planes, ) + NLM_KWARGS = dict( - patch_size=3, - patch_distance=2, - h=0.075, - fast_mode=True, - sigma=0.0, - preserve_range=True, + patch_size=3, patch_distance=2, h=0.075, + fast_mode=True, sigma=0.0, preserve_range=True, ) @@ -52,17 +47,15 @@ def _source(plane_count): rng = np.random.default_rng(458) pixels = rng.uniform(0.1, 0.9, (plane_count, 16, 20)).astype(np.float32) provenance = SourceImageProvenancePlanes.from_components( - paths=tuple( - f"/synthetic/A01_s{site + 1}_w1.tif" for site in range(plane_count) - ), + paths=tuple(f"/synthetic/A01_s{site + 1}_w1.tif" for site in range(plane_count)), component_metadata=tuple( {"well": "A01", "site": str(site + 1), "channel": "1"} for site in range(plane_count) ), ) - provenance = SourceImageProvenancePlanes( - tuple(replace(plane, source_image_name="C1") for plane in provenance.planes) - ) + provenance = SourceImageProvenancePlanes(tuple( + replace(plane, source_image_name="C1") for plane in provenance.planes + )) metadata = ImagePayloadMetadata( plane_axis=RuntimePlaneAxis.RUNTIME_SLICE, source_image_names=("C1",) * plane_count, @@ -70,8 +63,7 @@ def _source(plane_count): source_plane_dtypes=("float32",) * plane_count, source_voxel_spacing=SourceVoxelSpacing((0.5, 0.75)), source_spatial_domain=SourceSpatialDomain( - origin_yx=(0, 0), - source_shape_yx=(16, 20), + origin_yx=(0, 0), source_shape_yx=(16, 20), ), ) return metadata.payload_with(pixels) @@ -91,17 +83,13 @@ def _registered(metadata=None): @pytest.mark.parametrize("plane_count", (1, 3)) @pytest.mark.parametrize("fast_mode", (True, False)) def test_real_nlm_kernel_receives_only_planes_and_restores_context( - monkeypatch, - plane_count, - fast_mode, + monkeypatch, plane_count, fast_mode, ): source = _source(plane_count) before = source.data.copy() original = restoration.denoise_nl_means kwargs = dict(NLM_KWARGS, fast_mode=fast_mode) - expected = np.stack( - [original(plane, channel_axis=None, **kwargs) for plane in before] - ) + expected = np.stack([original(plane, channel_axis=None, **kwargs) for plane in before]) seen = [] def observed_kernel(image, **kernel_kwargs): @@ -110,11 +98,7 @@ def observed_kernel(image, **kernel_kwargs): monkeypatch.setattr(restoration, "denoise_nl_means", observed_kernel) result = _registered().func(source, **kwargs) - assert ( - seen - == [((16, 20), np.dtype("float32"), 1280, dict(kwargs, channel_axis=None))] - * plane_count - ) + assert seen == [((16, 20), np.dtype("float32"), 1280, dict(kwargs, channel_axis=None))] * plane_count np.testing.assert_array_equal(result.data, expected) np.testing.assert_array_equal(source.data, before) assert result.data.dtype == expected.dtype == np.dtype("float32") @@ -124,10 +108,7 @@ def observed_kernel(image, **kernel_kwargs): assert current.plane_axis is prior.plane_axis is RuntimePlaneAxis.RUNTIME_SLICE assert current.source_voxel_spacing == prior.source_voxel_spacing assert current.source_spatial_domain == prior.source_spatial_domain - assert ( - current.source_image_provenance_planes.identity - == prior.source_image_provenance_planes.identity - ) + assert current.source_image_provenance_planes.identity == prior.source_image_provenance_planes.identity assert current.source_image_names == prior.source_image_names @@ -164,10 +145,7 @@ def observed_volume(image, *args, **kwargs): def test_declared_catalog_identity_and_public_kwargs_are_original_projections(): metadata = _registered() function_id = f"{metadata.registry.library_name}:{metadata.name}" - assert ( - function_id - == "openhcs:processors_numpy_processor_non_local_means_denoise_planes" - ) + assert function_id == "openhcs:processors_numpy_processor_non_local_means_denoise_planes" assert metadata.original_name == "non_local_means_denoise_planes" assert "slice_by_slice" not in inspect.signature(metadata.func).parameters _validate_callable_kwargs(function_id, metadata.func, NLM_KWARGS) @@ -181,14 +159,9 @@ def test_nominal_transport_resolves_original_declaration_without_global_catalog( metadata = _registered() reference = FunctionReferenceTransportAuthority.function_reference(metadata.func) assert isinstance(reference, RegistryFunctionReference) - assert ( - reference.composite_key - == "openhcs:processors_numpy_processor_non_local_means_denoise_planes" - ) + assert reference.composite_key == "openhcs:processors_numpy_processor_non_local_means_denoise_planes" resolved = reference.resolve() - assert ( - CallableContract.from_callable(resolved).processing_contract is Pure2DContract - ) + assert CallableContract.from_callable(resolved).processing_contract is Pure2DContract result = resolved(_source(1), **NLM_KWARGS) assert result.data.shape == (1, 16, 20) @@ -197,16 +170,12 @@ def test_new_independent_declaration_executes_same_consumer_without_edits(): seen = [] @numpy_contract(contract=Pure2DContract) - def independent_plane_offset( - image: np.ndarray, *, offset: float = 0.25 - ) -> np.ndarray: + def independent_plane_offset(image: np.ndarray, *, offset: float = 0.25) -> np.ndarray: seen.append(image.shape) return image + offset source = _source(3) - result = _registered(project_custom_function(independent_plane_offset)).func( - source, offset=0.5 - ) + result = _registered(project_custom_function(independent_plane_offset)).func(source, offset=0.5) assert seen == [(16, 20)] * 3 np.testing.assert_array_equal(result.data, source.data + 0.5) assert result.metadata.source_image_provenance_planes.identity == ( diff --git a/tests/unit/test_plane_store_sources.py b/tests/unit/test_plane_store_sources.py index 16f38fff96..0c43812468 100644 --- a/tests/unit/test_plane_store_sources.py +++ b/tests/unit/test_plane_store_sources.py @@ -321,9 +321,7 @@ def test_mixed_plane_stores_bind_and_load_through_virtual_workspace( ) ) filemanager = _filemanager() - handler = DatasetSourceChoice.named("auto").open( - tmp_path, filemanager=filemanager, source_bindings_config=source_bindings - ) + handler = DatasetSourceChoice.named("auto").open(tmp_path, filemanager=filemanager, source_bindings_config=source_bindings) assert isinstance(handler, BioFormatsHandler) assert handler.parser.extract_component_coordinates("plain.tif") == ( "S", @@ -356,9 +354,7 @@ def test_saved_source_bindings_rebuild_canonical_store_projection( replace_config: str, ) -> None: def reject_editor(*args, **kwargs): - raise AssertionError( - "Saved runtime config capture must not construct an editor" - ) + raise AssertionError("Saved runtime config capture must not construct an editor") if replace_config != "delegate": monkeypatch.setattr(ObjectState, "__init__", reject_editor) diff --git a/tests/unit/test_plate_image_inventory.py b/tests/unit/test_plate_image_inventory.py index c4bda9ab52..55cc08a6f4 100644 --- a/tests/unit/test_plate_image_inventory.py +++ b/tests/unit/test_plate_image_inventory.py @@ -76,9 +76,7 @@ def test_image_browser_inventory_uses_declared_virtual_workspace_address( ensure_storage_registry() filemanager = FileManager(dict(storage_registry)) - handler = DatasetSourceChoice.named("imagexpress").open( - plate, filemanager=filemanager - ) + handler = DatasetSourceChoice.named("imagexpress").open(plate, filemanager=filemanager) handler.initialize_workspace(plate, filemanager) assert ( handler.get_primary_backend(plate, filemanager) diff --git a/tests/unit/test_primary_object_diagnostics.py b/tests/unit/test_primary_object_diagnostics.py index 2d0af04c71..56bb53ca71 100644 --- a/tests/unit/test_primary_object_diagnostics.py +++ b/tests/unit/test_primary_object_diagnostics.py @@ -21,9 +21,7 @@ ObjectLabelsArtifactType, ) from openhcs.core.runtime_image_values import ImagePayloadMetadata, MaskedImagePayload -from openhcs.core.runtime_object_label_building import ( - SourceImageObjectLabelBuildRequest, -) +from openhcs.core.runtime_object_label_building import SourceImageObjectLabelBuildRequest from openhcs.core.source_spatial_domain import SourceSpatialDomain from openhcs.processing.backends.cellprofiler.primary_object_diagnostics import ( ExecutedDeclumpingEvidence, @@ -73,8 +71,7 @@ def _objects(source): def test_selected_response_keeps_source_frame_without_acquisition_intensity_scale(): from openhcs.core.runtime_plane_projection import ( - RuntimePlaneAxis, - RuntimePlaneAxisValueProjection, + RuntimePlaneAxis, RuntimePlaneAxisValueProjection, ) source = _source() @@ -91,9 +88,7 @@ def test_selected_response_keeps_source_frame_without_acquisition_intensity_scal np.testing.assert_array_equal(resolved.data, pixels[0]) np.testing.assert_array_equal(resolved.normalize_intensity_payload(), pixels[0]) assert resolved.metadata.source_path == source.metadata.source_path - assert ( - resolved.metadata.source_spatial_domain == source.metadata.source_spatial_domain - ) + assert resolved.metadata.source_spatial_domain == source.metadata.source_spatial_domain assert resolved.metadata.source_dtype == "float32" assert resolved.metadata.intensity_scale is None with pytest.raises(ValueError, match="preserve the source spatial shape"): @@ -111,8 +106,7 @@ def test_planes_preserve_exact_source_coordinates_and_owned_variants(executed): markers = maxima.astype(np.int32) evidence = ( ExecutedDeclumpingEvidence(response, maxima, markers) - if executed - else UnexecutedDeclumpingEvidence() + if executed else UnexecutedDeclumpingEvidence() ) threshold_support = source.data > 15 initial_components = objects.labels.copy() @@ -135,9 +129,7 @@ def test_planes_preserve_exact_source_coordinates_and_owned_variants(executed): assert diagnostics.threshold_support.data is threshold_support assert diagnostics.initial_components.data is initial_components assert diagnostics.unedited_objects.data is objects.variant_labels(UneditedLabels) - assert diagnostics.small_removed_objects.data is objects.variant_labels( - SmallRemovedLabels - ) + assert diagnostics.small_removed_objects.data is objects.variant_labels(SmallRemovedLabels) assert objects.domain.declared_object_count == 1 assert diagnostics.threshold_support.mask is diagnostics.initial_components.mask assert diagnostics.unedited_objects.mask is diagnostics.small_removed_objects.mask @@ -161,9 +153,7 @@ def test_planes_preserve_exact_source_coordinates_and_owned_variants(executed): def test_diagnostic_contract_has_source_producer_stage_identity_not_new_objects(): source = ArtifactSpec.input("DNA", ImageArtifactType) objects = ArtifactSpec.output("Nuclei", ObjectLabelsArtifactType) - specs = PrimaryObjectDiagnosticPlanes.artifact_specs( - source_image=source, objects=objects - ) + specs = PrimaryObjectDiagnosticPlanes.artifact_specs(source_image=source, objects=objects) assert len(specs) == len(PrimaryObjectDiagnosticPlanes._fields) for field, spec in zip(PrimaryObjectDiagnosticPlanes._fields, specs, strict=True): assert spec.name.endswith(f"__{field}") @@ -182,22 +172,19 @@ def test_diagnostic_contract_has_source_producer_stage_identity_not_new_objects( assert spec.source_context_sources() == (source.ref(),) assert ArtifactSidecarSourceRelation(source=objects.ref()) in spec.relations assert len(get_args(PrimaryObjectsRuntimeTuple)) == 3 + len(specs) - assert get_args(PrimaryObjectsRuntimeTuple)[3:] == (MaskedImagePayload,) * len( - specs - ) + assert get_args(PrimaryObjectsRuntimeTuple)[3:] == (MaskedImagePayload,) * len(specs) def test_ordinary_image_contextualization_preserves_unexecuted_masks(): from openhcs.core.artifacts import ArtifactOutputPlan + source = _source() objects = _objects(source) diagnostics = PrimaryObjectDiagnosticPlanes.from_execution( - image=source, - threshold_support=source.data > 15, + image=source, threshold_support=source.data > 15, initial_components=objects.labels, - declumping=UnexecutedDeclumpingEvidence(), - objects=objects, + declumping=UnexecutedDeclumpingEvidence(), objects=objects, ) specs = PrimaryObjectDiagnosticPlanes.artifact_specs( source_image=ArtifactSpec.input("DNA", ImageArtifactType), @@ -205,15 +192,13 @@ def test_ordinary_image_contextualization_preserves_unexecuted_masks(): ) for spec, plane in zip(specs, diagnostics, strict=True): plan = ArtifactOutputPlan( - name=spec.name, - path=f"/memory/{spec.name}.pkl", - artifact_type=spec.artifact_type, - sidecar_role=spec.sidecar_role, + name=spec.name, path=f"/memory/{spec.name}.pkl", + artifact_type=spec.artifact_type, sidecar_role=spec.sidecar_role, relations=spec.relations, ) - value = ( - ImageArtifactType if plan is None else plan.artifact_type - ).contextualize_output(source, plane, plan, None) + value = (ImageArtifactType if plan is None else plan.artifact_type).contextualize_output( + source, plane, plan, None + ) np.testing.assert_array_equal(value.mask, plane.mask) np.testing.assert_array_equal(value.data, plane.data) assert value.metadata.source_provenance == source.metadata.source_provenance @@ -225,24 +210,18 @@ def test_existing_pure2d_aggregation_retains_stage_pixels_masks_and_plane_identi from openhcs.core.processing_contracts import ( Pure2DAuxiliaryOutputAggregator, ) - from openhcs.processing.backends.cellprofiler.primary_object_diagnostics import ( - DiagnosticPlaneSource, - ) + from openhcs.processing.backends.cellprofiler.primary_object_diagnostics import DiagnosticPlaneSource first = _source() second = MaskedImagePayload( - first.data + 10, - ~first.mask, + first.data + 10, ~first.mask, ImagePayloadMetadata( - source_path="/synthetic/A01_s002_DNA.tif", - source_image_names=("DNA",), + source_path="/synthetic/A01_s002_DNA.tif", source_image_names=("DNA",), source_spatial_domain=first.metadata.source_spatial_domain, ), ) - planes = [ - DiagnosticPlaneSource.from_image(source).plane(source.data) - for source in (first, second) - ] + planes = [DiagnosticPlaneSource.from_image(source).plane(source.data) + for source in (first, second)] value = Pure2DAuxiliaryOutputAggregator.aggregate( planes, "numpy", plane_axis=RuntimePlaneAxis.RUNTIME_SLICE ) @@ -271,42 +250,28 @@ def test_existing_pure2d_aggregation_retains_stage_pixels_masks_and_plane_identi @pytest.mark.parametrize("shared_mask", (False, True)) @pytest.mark.parametrize("spatial_mask", (False, True)) -def test_produced_input_snapshot_owns_canonical_pixels_and_masks( - shared_mask, spatial_mask -): +def test_produced_input_snapshot_owns_canonical_pixels_and_masks(shared_mask, spatial_mask): import pickle from openhcs.core.aligned_image_payload import ProducedImageStack from openhcs.core.runtime_plane_projection import RuntimePlaneAxis - pixels = ( - np.full((3, 4), 7, dtype=np.uint16), - np.full((3, 4), 0.25, dtype=np.float32), - ) + pixels = (np.full((3, 4), 7, dtype=np.uint16), + np.full((3, 4), 0.25, dtype=np.float32)) first_mask = np.ones((3, 4), dtype=bool) masks = (first_mask, first_mask if shared_mask else ~first_mask) - metadata = ( - ImagePayloadMetadata( - source_spatial_domain=SourceSpatialDomain(source_shape_yx=(3, 4)) - ) - if spatial_mask - else ImagePayloadMetadata() - ) + metadata = ImagePayloadMetadata( + source_spatial_domain=SourceSpatialDomain(source_shape_yx=(3, 4)) + ) if spatial_mask else ImagePayloadMetadata() source = ProducedImageStack( - tuple( - MaskedImagePayload(data, mask, metadata) - for data, mask in zip(pixels, masks, strict=True) - ), - memory_type="numpy", - plane_axis=( - RuntimePlaneAxis.SOURCE_BINDING - if spatial_mask + tuple(MaskedImagePayload(data, mask, metadata) + for data, mask in zip(pixels, masks, strict=True)), + memory_type="numpy", plane_axis=( + RuntimePlaneAxis.SOURCE_BINDING if spatial_mask else RuntimePlaneAxis.RUNTIME_SLICE ), ) expected = np.stack(pixels) - expected_mask = ( - first_mask.copy() if shared_mask and spatial_mask else np.stack(masks) - ) + expected_mask = first_mask.copy() if shared_mask and spatial_mask else np.stack(masks) copied = source.copied(memory_type="numpy", device_id=None) assert source._composed_payload is None canonical = copied.compose() @@ -334,11 +299,9 @@ def test_typed_stage_payloads_roundtrip_existing_pickle_boundary(): source = _source() diagnostics = PrimaryObjectDiagnosticPlanes.from_execution( - image=source, - threshold_support=source.data > 15, + image=source, threshold_support=source.data > 15, initial_components=_objects(source).labels, - declumping=UnexecutedDeclumpingEvidence(), - objects=_objects(source), + declumping=UnexecutedDeclumpingEvidence(), objects=_objects(source), ) reloaded = pickle.loads(pickle.dumps(diagnostics)) for before, after in zip(diagnostics, reloaded, strict=True): @@ -363,34 +326,27 @@ class ExtendedPlanes(NamedTuple): def test_focused_real_source_antipattern_guards(): root = Path(__file__).resolve().parents[2] - source = ( - root / "openhcs/processing/backends/cellprofiler/primary_object_diagnostics.py" - ) + source = root / "openhcs/processing/backends/cellprofiler/primary_object_diagnostics.py" tree = ast.parse(source.read_text()) # IMPL-1/2/3/7 and BOUND-1/2/7: evidence owners have no kind/type # dispatcher, raw string-key consumers or duck-typed fallback readers. assert not any(isinstance(node, ast.Match) for node in ast.walk(tree)) assert not any( - isinstance(node, ast.Call) - and isinstance(node.func, ast.Name) + isinstance(node, ast.Call) and isinstance(node.func, ast.Name) and node.func.id in {"isinstance", "getattr", "hasattr", "eval", "exec"} for node in ast.walk(tree) ) assert not any( - isinstance(node, ast.Subscript) - and isinstance(node.slice, ast.Constant) + isinstance(node, ast.Subscript) and isinstance(node.slice, ast.Constant) and isinstance(node.slice.value, str) for node in ast.walk(tree) ) projection = next( - node - for node in ast.walk(tree) + node for node in ast.walk(tree) if isinstance(node, ast.FunctionDef) and node.name == "artifact_specs" ) # MEMB-1/3: no hand-maintained roster alongside the typed stage record. - assert not any( - isinstance(node, (ast.List, ast.Set, ast.Dict)) for node in ast.walk(projection) - ) + assert not any(isinstance(node, (ast.List, ast.Set, ast.Dict)) for node in ast.walk(projection)) assert any( isinstance(node, ast.Attribute) and node.attr == "_fields" for node in ast.walk(projection) @@ -402,35 +358,18 @@ def test_processing_body_is_unchanged_except_same_run_evidence_capture(): relative = "openhcs/processing/backends/cellprofiler/primary_objects.py" baseline = subprocess.run( ["git", "show", f"a0263e82a1b3296f34f8c5cdd50ffae5c33ef7dd:{relative}"], - cwd=root, - check=True, - capture_output=True, - text=True, + cwd=root, check=True, capture_output=True, text=True, ).stdout - before = next( - n - for n in ast.parse(baseline).body - if isinstance(n, ast.FunctionDef) and n.name == "identify_primary_objects" - ) - after = next( - n - for n in ast.parse((root / relative).read_text()).body - if isinstance(n, ast.FunctionDef) and n.name == before.name - ) + before = next(n for n in ast.parse(baseline).body if isinstance(n, ast.FunctionDef) + and n.name == "identify_primary_objects") + after = next(n for n in ast.parse((root / relative).read_text()).body + if isinstance(n, ast.FunctionDef) and n.name == before.name) class RemoveEvidence(ast.NodeTransformer): def visit_Assign(self, node): - if any( - isinstance(target, ast.Name) - and target.id - in { - "threshold_support", - "initial_components", - "declumping", - "diagnostics", - } - for target in node.targets - ): + if any(isinstance(target, ast.Name) and target.id in { + "threshold_support", "initial_components", "declumping", "diagnostics" + } for target in node.targets): return None return self.generic_visit(node) @@ -454,19 +393,14 @@ def visit_Call(self, node): and not node.keywords ): return ast.Attribute( - value=node.args[0], - attr=self.ACCESSORS[node.func.id], - ctx=ast.Load(), + value=node.args[0], attr=self.ACCESSORS[node.func.id], ctx=ast.Load() ) return node class DeclaredLabelVariants(ast.NodeTransformer): """The label-variant keywords became one variants mapping (G5).""" - VARIANTS = { - "unedited_labels": "UneditedLabels", - "small_removed_labels": "SmallRemovedLabels", - } + VARIANTS = {"unedited_labels": "UneditedLabels", "small_removed_labels": "SmallRemovedLabels"} def visit_Call(self, node): self.generic_visit(node) @@ -474,9 +408,7 @@ def visit_Call(self, node): if not variant_keywords: return node mapping = ast.Dict( - keys=[ - ast.Name(self.VARIANTS[k.arg], ast.Load()) for k in variant_keywords - ], + keys=[ast.Name(self.VARIANTS[k.arg], ast.Load()) for k in variant_keywords], values=[k.value for k in variant_keywords], ) first = node.keywords.index(variant_keywords[0]) @@ -488,5 +420,4 @@ def visit_Call(self, node): before = DeclaredLabelVariants().visit(PayloadAttributes().visit(before)) # Compare the original algorithm, not docstring, signature or output ABI. assert ast.dump(ast.Module(body=before.body[1:-1], type_ignores=[])) == ast.dump( - ast.Module(body=after.body[1:-1], type_ignores=[]) - ) + ast.Module(body=after.body[1:-1], type_ignores=[])) diff --git a/tests/unit/test_process_local_cache.py b/tests/unit/test_process_local_cache.py index 90d2ff02a8..171a0c957d 100644 --- a/tests/unit/test_process_local_cache.py +++ b/tests/unit/test_process_local_cache.py @@ -40,10 +40,7 @@ def test_process_local_domains_join_family_cleanup_and_store_queries_do_not(): assert len({id(cache) for cache in caches}) == len(process_local) assert all(cache.max_entries == 16 for cache in caches[: len(numerical)]) for cache_type in process_local: - assert ( - ProcessLocalBoundedCache.__registry__[cache_type.process_cache_key] - is cache_type - ) + assert ProcessLocalBoundedCache.__registry__[cache_type.process_cache_key] is cache_type owner = object() for cache in caches[:-1]: diff --git a/tests/unit/test_processor_operations_written_once.py b/tests/unit/test_processor_operations_written_once.py index 5896e30125..3a1f5bd2c4 100644 --- a/tests/unit/test_processor_operations_written_once.py +++ b/tests/unit/test_processor_operations_written_once.py @@ -117,9 +117,7 @@ def test_projection_choices_are_served_by_the_projection_family() -> None: } for backend in BACKENDS: create_projection = _processor_module(backend).create_projection - method_type = type( - inspect.signature(create_projection).parameters["method"].default - ) + method_type = type(inspect.signature(create_projection).parameters["method"].default) assert {member.value for member in method_type} <= served, backend @@ -127,11 +125,7 @@ def test_numpy_registrations_compute_the_shared_operations() -> None: numpy_processor = _processor_module("numpy") stack = np.arange(2 * 4 * 4, dtype=np.uint16).reshape(2, 4, 4) - assert numpy_processor.spatial_bin_2d(stack, 2, SpatialBinMethod.MEAN).shape == ( - 2, - 2, - 2, - ) + assert numpy_processor.spatial_bin_2d(stack, 2, SpatialBinMethod.MEAN).shape == (2, 2, 2) np.testing.assert_array_equal( numpy_processor.spatial_bin_3d(stack, 2, SpatialBinMethod.MAX), stack.reshape(1, 2, 2, 2, 2, 2).max(axis=(1, 3, 5)), @@ -144,15 +138,12 @@ def test_numpy_registrations_compute_the_shared_operations() -> None: numpy_processor.create_projection(stack, method), reduce(stack, axis=0) ) np.testing.assert_array_equal( - numpy_processor.create_projection( - stack, numpy_processor.NumpyStackProjectionMethod.MEAN - ), + numpy_processor.create_projection(stack, numpy_processor.NumpyStackProjectionMethod.MEAN), stack.mean(axis=0).astype(stack.dtype), ) assert ( - CallableContract.from_callable( - numpy_processor.create_projection - ).require_processing_contract() + CallableContract.from_callable(numpy_processor.create_projection) + .require_processing_contract() is VolumetricToSliceContract ) mask = np.array([[1, 0, 1, 0]] * 4, dtype=np.uint16) diff --git a/tests/unit/test_pycodify_formatters.py b/tests/unit/test_pycodify_formatters.py index aeea134b1a..1d8c7bf527 100644 --- a/tests/unit/test_pycodify_formatters.py +++ b/tests/unit/test_pycodify_formatters.py @@ -48,11 +48,7 @@ def selected_declaration_catalog(monkeypatch): """Use existing declaration metadata, not a native catalog preparation.""" metadata = ( RegistryService.declared_metadata_for_callable(func) - for func in ( - count_cells_simple, - measure_colocalization_objects, - measure_object_size_shape, - ) + for func in (count_cells_simple, measure_colocalization_objects, measure_object_size_shape) ) monkeypatch.setattr(RegistryService, "_metadata_cache", dict(metadata)) @@ -127,19 +123,15 @@ def test_function_reference_formats_from_declared_identity_without_resolution( assert "remote_backend.filters" not in source -def test_module_export_registry_reference_uses_direct_import_without_resolution( - monkeypatch, -): +def test_module_export_registry_reference_uses_direct_import_without_resolution(monkeypatch): reference = ModuleExportRegistryFunctionReference( import_identity=CallableImportIdentity( - module_name="declared.functions", - function_name="exported_filter", + module_name="declared.functions", function_name="exported_filter", ), composite_key="registry:exported_filter", ) monkeypatch.setattr( - RegistryFunctionReference, - "resolve", + RegistryFunctionReference, "resolve", lambda self: (_ for _ in ()).throw(AssertionError("must not resolve")), ) source = _source(reference) @@ -175,9 +167,7 @@ def test_clean_pipeline_config_keeps_explicit_nested_lazy_config_values(): def test_clean_pipeline_config_keeps_explicit_empty_nested_lazy_config(): - source = _source( - PipelineConfig(partition_filter_config=LazyPartitionFilterConfig()) - ) + source = _source(PipelineConfig(partition_filter_config=LazyPartitionFilterConfig())) assert "partition_filter_config=LazyPartitionFilterConfig()" in source @@ -207,9 +197,7 @@ def test_clean_function_step_omits_objectstate_reconstructed_empty_lazy_configs( clean_mode=True, ) - reference = FunctionReferenceTransportAuthority.function_reference( - count_cells_simple - ) + reference = FunctionReferenceTransportAuthority.function_reference(count_cells_simple) assert f"func={reference.source_literal()}" in source assert "name='count_cells'" in source assert "dtype_config=" not in source diff --git a/tests/unit/test_registered_configuration_source_readiness.py b/tests/unit/test_registered_configuration_source_readiness.py index f3ff4ee445..9580f922c4 100644 --- a/tests/unit/test_registered_configuration_source_readiness.py +++ b/tests/unit/test_registered_configuration_source_readiness.py @@ -6,10 +6,7 @@ import objectstate.lazy_factory as lazy_factory from python_introspect import SignatureAnalyzer -from openhcs.core.processing_preparation import ( - CallablePreparation, - PreparationCacheBatch, -) +from openhcs.core.processing_preparation import CallablePreparation, PreparationCacheBatch from openhcs.processing.backends.lib_registry.registry_service import RegistryService @@ -44,9 +41,7 @@ def raw(image): monkeypatch.setattr(RegistryService, "_metadata_cache", metadata) monkeypatch.setattr(lazy_factory, "_lazy_type_registry", {}) monkeypatch.setattr(lazy_factory, "_base_to_lazy_registry", {}) - monkeypatch.setattr( - PreparationCacheBatch, "populate_child_caches", lambda *a, **k: None - ) + monkeypatch.setattr(PreparationCacheBatch, "populate_child_caches", lambda *a, **k: None) pairs = iter(((LazyConfig, PublicConfig), (LaterLazyConfig, LaterPublicConfig))) def prepare(owner): @@ -65,27 +60,16 @@ def declaration(config_type): monkeypatch.setattr(SignatureAnalyzer, "_extract_inline_field_docs", declaration) # The registry admits types that are not dataclasses; readiness filters them. lazy_factory.register_lazy_type_mapping(dict, list) - assert ( - RegistryService.prepare_in_current_process(status_callback=events.append) - is metadata - ) + assert RegistryService.prepare_in_current_process(status_callback=events.append) is metadata assert factories == [] - first_ready = next( - i - for i, item in enumerate(events) - if isinstance(item, str) and "kernels ready" in item - ) + first_ready = next(i for i, item in enumerate(events) if isinstance(item, str) and "kernels ready" in item) assert events.index(LazyConfig) < first_ready assert events.index(PublicConfig) < first_ready assert LaterPublicConfig not in events and dict not in events and list not in events events.clear() # An already loaded callable catalog must not freeze the configuration roster. RegistryService.prepare_in_current_process(status_callback=events.append) - second_ready = next( - i - for i, item in enumerate(events) - if isinstance(item, str) and "kernels ready" in item - ) + second_ready = next(i for i, item in enumerate(events) if isinstance(item, str) and "kernels ready" in item) for config_type in (LazyConfig, PublicConfig, LaterLazyConfig, LaterPublicConfig): assert events.count(config_type) == 1 assert events.index(config_type) < second_ready diff --git a/tests/unit/test_rescale_stream_plane_declaration.py b/tests/unit/test_rescale_stream_plane_declaration.py index 7b3a727640..9fc43bdbb5 100644 --- a/tests/unit/test_rescale_stream_plane_declaration.py +++ b/tests/unit/test_rescale_stream_plane_declaration.py @@ -13,13 +13,8 @@ from openhcs.core.runtime_stores import RuntimeValueStore from openhcs.core.component_group_scope import RuntimeExecutionAxisScope from openhcs.core.function_patterns import InvocationArtifactInputEdgePlan -from openhcs.core.runtime_object_label_building import ( - SourceImageObjectLabelBuildRequest, -) -from openhcs.core.runtime_plane_projection import ( - RuntimePlaneAxis, - RuntimePlaneAxisValueProjection, -) +from openhcs.core.runtime_object_label_building import SourceImageObjectLabelBuildRequest +from openhcs.core.runtime_plane_projection import RuntimePlaneAxis, RuntimePlaneAxisValueProjection from openhcs.core.runtime_object_label_domains import ObjectLabelDomainScope from openhcs.core.source_image_provenance import SourceImageProvenancePlanes from openhcs.core.config import StepSourceBindingsConfig @@ -28,69 +23,44 @@ from openhcs.core.source_metadata import SourceVoxelSpacing, SourceVoxelSpacingUnit from openhcs.core.steps.function_outputs import StreamOutputBatch from openhcs.core.steps.function_step import FunctionStep -from openhcs.core.steps.stream_component_semantics import ( - StreamImagePayloadMetadataProjector, -) +from openhcs.core.steps.stream_component_semantics import StreamImagePayloadMetadataProjector from openhcs.processing.backends.cellprofiler.intensity import ( - AutomaticHigh, - AutomaticLow, - RescaleIntensityModule, - RescaleMethod, - rescale_intensity, + AutomaticHigh, AutomaticLow, RescaleIntensityModule, RescaleMethod, rescale_intensity, ) from openhcs.processing.backends.cellprofiler.shape import measure_object_size_shape from openhcs.interop.cellprofiler.runtime.adapter import CellProfilerRuntimeAdapter -from openhcs.interop.cellprofiler.runtime.module_execution import ( - CellProfilerModuleExecutor, -) -from openhcs.interop.cellprofiler.runtime.function_contract_execution import ( - CellProfilerFunctionContractExecutor, -) +from openhcs.interop.cellprofiler.runtime.module_execution import CellProfilerModuleExecutor +from openhcs.interop.cellprofiler.runtime.function_contract_execution import CellProfilerFunctionContractExecutor from openhcs.runtime.napari_streaming_handlers import ( - NapariAggregateAxisBindingBuilder, - NapariStreamLayerAddress, - NapariStreamLayerItem, + NapariAggregateAxisBindingBuilder, NapariStreamLayerAddress, NapariStreamLayerItem, ) from openhcs.runtime.viewer_component_system import ( - ViewerComponentAxisSemanticsFactory, - ViewerComponentValueDomainPayload, + ViewerComponentAxisSemanticsFactory, ViewerComponentValueDomainPayload, ) # Reuse the original provider-free compiler and runtime fixtures, not a second ABI. -from tests.unit.test_cellprofiler_generic_special_input_binding import ( - _compile_public_step, -) +from tests.unit.test_cellprofiler_generic_special_input_binding import _compile_public_step from tests.unit.cellprofiler_runtime_test_support import ( cellprofiler_runtime_adapter_for_test, ) from tests.unit.viewer_axes_fixture import STREAM_AXES -@pytest.mark.parametrize( - "method", [RescaleMethod.DIVIDE_BY_VALUE, RescaleMethod.MANUAL_INPUT_RANGE] -) +@pytest.mark.parametrize("method", [RescaleMethod.DIVIDE_BY_VALUE, RescaleMethod.MANUAL_INPUT_RANGE]) @pytest.mark.parametrize("matching", [None, "Bright", "Other"]) -def test_public_rescale_preserves_one_source_plane_through_stream_binding( - method, matching -): +def test_public_rescale_preserves_one_source_plane_through_stream_binding(method, matching): kwargs = { - "select_the_input_image": "Bright", - "name_the_output_image": "Detection", - "rescale_method": method, - "automatic_low": AutomaticLow.CUSTOM, - "automatic_high": AutomaticHigh.CUSTOM, - "source_low": 0.0, - "source_high": 20.0, - "divisor_value": 2.0, + "select_the_input_image": "Bright", "name_the_output_image": "Detection", + "rescale_method": method, "automatic_low": AutomaticLow.CUSTOM, + "automatic_high": AutomaticHigh.CUSTOM, "source_low": 0.0, + "source_high": 20.0, "divisor_value": 2.0, } if matching is not None: kwargs["select_image_to_match_in_maximum_intensity"] = matching step = FunctionStep( - func=(rescale_intensity, kwargs), - name="SyntheticRescale", + func=(rescale_intensity, kwargs), name="SyntheticRescale", source_bindings=StepSourceBindingsConfig( - enabled=True, - bindings=(NamedSourceBinding("Bright"),), + enabled=True, bindings=(NamedSourceBinding("Bright"),), ), ) invocation = _compile_public_step(step) @@ -104,32 +74,18 @@ def test_public_rescale_preserves_one_source_plane_through_stream_binding( pixels = np.arange(20, dtype=np.float32).reshape(4, 5) metadata = ImagePayloadMetadata( source_path="/synthetic/A01_s1_w1_z1_t1.tif", - source_component_metadata={ - "well": "A01", - "site": "1", - "channel": "1", - "z_index": "1", - "timepoint": "1", - }, + source_component_metadata={"well": "A01", "site": "1", "channel": "1", "z_index": "1", "timepoint": "1"}, source_image_names=("Bright",), - source_voxel_spacing=SourceVoxelSpacing( - (1.0, 1.0, 1.0), SourceVoxelSpacingUnit.RELATIVE - ), - source_spatial_domain=SourceSpatialDomain( - origin_yx=(0, 0), source_shape_yx=(4, 5) - ), + source_voxel_spacing=SourceVoxelSpacing((1.0, 1.0, 1.0), SourceVoxelSpacingUnit.RELATIVE), + source_spatial_domain=SourceSpatialDomain(origin_yx=(0, 0), source_shape_yx=(4, 5)), ) source = metadata.payload_with(pixels, None) runtime = cellprofiler_runtime_adapter_for_test( - runtime_value_store=RuntimeValueStore(), - callable_contract=contract, - axis_scope=RuntimeExecutionAxisScope.from_raw( - "synthetic", component=None, value=None - ), + runtime_value_store=RuntimeValueStore(), callable_contract=contract, + axis_scope=RuntimeExecutionAxisScope.from_raw("synthetic", component=None, value=None), artifact_inputs={ edge.key: InvocationArtifactInputEdgePlan.from_source_declarations( - key=edge.key, - spec=edge.spec, + key=edge.key, spec=edge.spec, main_flow_artifacts=contract.artifact_inputs, invocation_sources=contract.artifact_inputs, ) @@ -138,85 +94,50 @@ def test_public_rescale_preserves_one_source_plane_through_stream_binding( ) executor = CellProfilerModuleExecutor(rescale_intensity, contract) request = executor._image_request( - source, - runtime, - module_type=RescaleIntensityModule, + source, runtime, module_type=RescaleIntensityModule, active_input_specs=contract.artifact_inputs.specs, ) execution = executor._invocation_request( - image_request=request, - adapter=runtime, - current_image=source, + image_request=request, adapter=runtime, current_image=source, module_type=RescaleIntensityModule, - kwargs={ - "rescale_method": method, - "automatic_low": AutomaticLow.CUSTOM, - "automatic_high": AutomaticHigh.CUSTOM, - "source_low": 0.0, - "source_high": 20.0, - "divisor_value": 2.0, - }, + kwargs={"rescale_method": method, "automatic_low": AutomaticLow.CUSTOM, + "automatic_high": AutomaticHigh.CUSTOM, "source_low": 0.0, + "source_high": 20.0, "divisor_value": 2.0}, ) result = CellProfilerFunctionContractExecutor().execute( - contract, - rescale_intensity, - execution.payload, - execution.kwargs, - execution_mode=execution.execution_mode, - plane_projection=execution.plane_projection, - ) - (item,) = tuple( - StreamOutputBatch.project_item( - RuntimeProjectionSourceIdentityRequest( - value=result, - source_description="/synthetic/Detection.tif", - ) - ) - ) - fields = StreamImagePayloadMetadataProjector.item_fields( - item.metadata, ("well", "site", "channel", "z_index", "timepoint") + contract, rescale_intensity, execution.payload, execution.kwargs, + execution_mode=execution.execution_mode, plane_projection=execution.plane_projection, ) + (item,) = tuple(StreamOutputBatch.project_item(RuntimeProjectionSourceIdentityRequest( + value=result, source_description="/synthetic/Detection.tif", + ))) + fields = StreamImagePayloadMetadataProjector.item_fields(item.metadata, ("well", "site", "channel", "z_index", "timepoint")) native_item = NapariStreamLayerItem( data=item.data, producer=StreamProducerIdentity.pipeline_output( - output_kind="main", - output_key="main", - projection_key="main", - step_name="SyntheticRescale", - pipeline_position=0, - step_scope_id="synthetic", + output_kind="main", output_key="main", projection_key="main", + step_name="SyntheticRescale", pipeline_position=0, step_scope_id="synthetic", ), address=NapariStreamLayerAddress( components=dict(item.require_source_component_metadata()), - path="/synthetic/Detection.tif", - stream_layer_data_type=StreamingDataType.IMAGE, + path="/synthetic/Detection.tif", stream_layer_data_type=StreamingDataType.IMAGE, ), image_metadata=item.metadata, plane_component_domain=ViewerComponentValueDomainPayload.from_wire_mapping( - fields.get(ViewerWireField.PLANE_COMPONENT_VALUES.value, {}), - context="synthetic producer fields", + fields.get(ViewerWireField.PLANE_COMPONENT_VALUES.value, {}), context="synthetic producer fields", declared_axes=STREAM_AXES, ), ) # The exact original strict receiver is reached, not replaced by a mock. - NapariAggregateAxisBindingBuilder.bindings( - (native_item,), ViewerComponentAxisSemanticsFactory.empty() - ) + NapariAggregateAxisBindingBuilder.bindings((native_item,), ViewerComponentAxisSemanticsFactory.empty()) assert contract.artifact_inputs.names() == ("Bright",) assert request.image_count == 1 assert result.data.shape == (4, 5) assert result.metadata.source_spatial_domain == metadata.source_spatial_domain assert result.metadata.source_image_paths == metadata.source_image_paths assert result.metadata.source_voxel_spacing == metadata.source_voxel_spacing - assert ( - SourceVoxelSpacing.common_physical_pixel_size( - (result.metadata.source_voxel_spacing,) - ) - is None - ) - assert ( - item.require_source_component_metadata() == metadata.source_component_metadata - ) + assert SourceVoxelSpacing.common_physical_pixel_size((result.metadata.source_voxel_spacing,)) is None + assert item.require_source_component_metadata() == metadata.source_component_metadata expected = pixels / (2.0 if method is RescaleMethod.DIVIDE_BY_VALUE else 20.0) np.testing.assert_array_equal(item.data, expected) @@ -229,17 +150,14 @@ def test_declared_source_plane_has_area_not_volume(retained_site): metadata = ImagePayloadMetadata( source_path="/synthetic/site.tif", source_component_metadata={"site": "7"}, - source_spatial_domain=SourceSpatialDomain( - origin_yx=(0, 0), source_shape_yx=(4, 5) - ), + source_spatial_domain=SourceSpatialDomain(origin_yx=(0, 0), source_shape_yx=(4, 5)), ) if retained_site: pixels = pixels[None] labels = labels[None] metadata = ImagePayloadMetadata( source_image_provenance_planes=SourceImageProvenancePlanes.from_components( - paths=("/synthetic/site.tif",), - component_metadata=({"site": "7"},), + paths=("/synthetic/site.tif",), component_metadata=({"site": "7"},), ), source_spatial_domain=metadata.source_spatial_domain, plane_axis=RuntimePlaneAxis.RUNTIME_SLICE, @@ -251,26 +169,19 @@ def test_declared_source_plane_has_area_not_volume(retained_site): domain_scope=ObjectLabelDomainScope.PLANE if retained_site else None, plane_projection=( RuntimePlaneAxisValueProjection.from_source_declaration( - metadata.plane_axis, - metadata.source_provenance, + metadata.plane_axis, metadata.source_provenance, ) - if retained_site - else None + if retained_site else None ), ).payload() assert objects.plane_axis is ( RuntimePlaneAxis.RUNTIME_SLICE if retained_site else None ) assert objects.object_label_domain().scope is ( - ObjectLabelDomainScope.PLANE - if retained_site - else ObjectLabelDomainScope.PAYLOAD + ObjectLabelDomainScope.PLANE if retained_site else ObjectLabelDomainScope.PAYLOAD ) _, rows = measure_object_size_shape.__wrapped__( - payload, - objects, - calculate_advanced=False, - calculate_zernikes=False, + payload, objects, calculate_advanced=False, calculate_zernikes=False, ) fields = tuple(field.name for field in rows.fields) assert "Area" in fields @@ -283,15 +194,10 @@ def test_real_volume_is_not_squeezed_or_relabelled_as_area(): pixels = np.ones((3, 4, 5), dtype=np.float32) labels = np.zeros_like(pixels, dtype=np.int32) labels[1:, 1:3, 1:4] = 1 - payload = ImagePayloadMetadata(source_path="/synthetic/volume.tif").payload_with( - pixels, None - ) + payload = ImagePayloadMetadata(source_path="/synthetic/volume.tif").payload_with(pixels, None) objects = SourceImageObjectLabelBuildRequest(image=payload, labels=labels).payload() _, rows = measure_object_size_shape.__wrapped__( - payload, - objects, - calculate_advanced=False, - calculate_zernikes=False, + payload, objects, calculate_advanced=False, calculate_zernikes=False, ) fields = tuple(field.name for field in rows.fields) assert "Volume" in fields diff --git a/tests/unit/test_returned_output_abi.py b/tests/unit/test_returned_output_abi.py index 2d3b8ca180..95a5f07151 100644 --- a/tests/unit/test_returned_output_abi.py +++ b/tests/unit/test_returned_output_abi.py @@ -6,11 +6,7 @@ import pytest from openhcs.processing.backends.cellprofiler.crop import CropMask -from openhcs.core.aligned_image_payload import ( - AlignedImageSliceContext, - AlignedImageStack, - ImageOutputBundle, -) +from openhcs.core.aligned_image_payload import (AlignedImageSliceContext, AlignedImageStack, ImageOutputBundle) from openhcs.core.artifacts import ( ArtifactOutputPlan, ArtifactSpec, diff --git a/tests/unit/test_runner_cellprofiler_compatibility.py b/tests/unit/test_runner_cellprofiler_compatibility.py index d4e7190c32..77728f2fdc 100644 --- a/tests/unit/test_runner_cellprofiler_compatibility.py +++ b/tests/unit/test_runner_cellprofiler_compatibility.py @@ -288,9 +288,7 @@ def test_native_reference_lookup_separates_public_partition_filter_scope( ) assert ( - native_cellprofiler_partition_filter_scope_slug( - global_config.partition_filter_config - ) + native_cellprofiler_partition_filter_scope_slug(global_config.partition_filter_config) == "wells_include_first1" ) assert location.output_dir == ( diff --git a/tests/unit/test_runtime_adapter_source_artifacts.py b/tests/unit/test_runtime_adapter_source_artifacts.py index a3d6ed038f..50105b5c0b 100644 --- a/tests/unit/test_runtime_adapter_source_artifacts.py +++ b/tests/unit/test_runtime_adapter_source_artifacts.py @@ -130,9 +130,7 @@ def binding( ) # This opens the persisted projection; it does not ingest workspace files # as a new raw source folder. Physical inputs remain in source_root. - microscope_handler = DatasetSourceChoice.named("auto").open( - workspace_root, filemanager=filemanager, source_bindings_config=source_bindings - ) + microscope_handler = DatasetSourceChoice.named("auto").open(workspace_root, filemanager=filemanager, source_bindings_config=source_bindings) assert isinstance(microscope_handler, OpenHCSDatasetSource) microscope_handler.initialize_workspace(workspace_root, filemanager) projection_cache = VirtualWorkspaceSourceProjectionCache() @@ -146,11 +144,8 @@ def binding( frozenset((Microscopy.Channel,)) ), ) - context.runtime_source_workspace_projections = ( - WorkspaceSourceProjections.from_context( - context, - cache=projection_cache, - ) + context.runtime_source_workspace_projections = WorkspaceSourceProjections.from_context( + context, cache=projection_cache, ) projection = WorkspaceSourceProjections.from_context( context, @@ -234,7 +229,9 @@ def request() -> RuntimeAdapterRequest: labels_payload = labels_request.source_artifact_payload(labels_spec.ref()) label_set = CellProfilerOutputRecorder.for_artifact_type( ObjectLabelsArtifactType - ).raw_runtime_input_value(spec=labels_spec, value=labels_payload) + ).raw_runtime_input_value( + spec=labels_spec, value=labels_payload + ) assert isinstance(label_set, ObjectLabelSet) np.testing.assert_array_equal(label_set.labels, labels[np.newaxis, ...]) @@ -257,7 +254,9 @@ def request() -> RuntimeAdapterRequest: (illumination_payload, live_illumination), (labels_payload, live_labels), ): - np.testing.assert_array_equal(produced.data, live.data) + np.testing.assert_array_equal( + produced.data, live.data + ) produced_metadata = produced.metadata live_metadata = live.metadata assert produced_metadata == live_metadata @@ -275,7 +274,6 @@ def request() -> RuntimeAdapterRequest: def test_workspace_materialization_preserves_declared_source_pixels(tmp_path, mode): from PIL import Image from skimage.color import rgb2gray - source_root, workspace_root = tmp_path / "source", tmp_path / "workspace" source_root.mkdir() path = source_root / "A01_s1_w1_z001_t001.tif" @@ -284,51 +282,26 @@ def test_workspace_materialization_preserves_declared_source_pixels(tmp_path, mo pixels = pixels[..., 0] Image.fromarray(pixels).save(path) binding = NamedSourceBinding( - alias="Raw", - load_as_monochrome=mode == "monochrome", - load_as_mask=mode == "mask", + alias="Raw", load_as_monochrome=mode == "monochrome", load_as_mask=mode == "mask", ) bindings = SourceBindingsConfig(bindings=(binding,)) filemanager = _filemanager() - SourceBindingWorkspaceProjector( - bindings, parser=SourceSchemaFilenameParser() - ).materialize( - source_root, - workspace_root, - filemanager=filemanager, - source_backend=Backend.DISK, - workspace_backend=Backend.DISK, - source_files=(path,), - ) - microscope = DatasetSourceChoice.named("auto").open( - workspace_root, filemanager=filemanager, source_bindings_config=bindings + SourceBindingWorkspaceProjector(bindings, parser=SourceSchemaFilenameParser()).materialize( + source_root, workspace_root, filemanager=filemanager, + source_backend=Backend.DISK, workspace_backend=Backend.DISK, source_files=(path,), ) + microscope = DatasetSourceChoice.named("auto").open(workspace_root, filemanager=filemanager, source_bindings_config=bindings) microscope.initialize_workspace(workspace_root, filemanager) cache = VirtualWorkspaceSourceProjectionCache() context = SimpleNamespace( completed_step_outputs=StepExecutionObservation.empty(), - dataset_root=workspace_root, - filemanager=filemanager, - microscope_handler=microscope, - ) - workspace = WorkspaceSourceProjections.from_context( - context, cache=cache - ).projection_or_empty() - paths = tuple( - path - for path, _projection in workspace.source_occurrences_for_binding( - binding, axis_id="A01" - ) + dataset_root=workspace_root, filemanager=filemanager, microscope_handler=microscope, ) + workspace = WorkspaceSourceProjections.from_context(context, cache=cache).projection_or_empty() + paths = tuple(path for path, _projection in workspace.source_occurrences_for_binding(binding, axis_id="A01")) assert len(paths) == 1 - (payload,) = workspace.load_binding_payloads( - paths, binding=binding, filemanager=filemanager - ) - expected = ( - rgb2gray(pixels.astype(np.float32) / 255) - if mode == "monochrome" - else pixels.astype(bool) if mode == "mask" else pixels - ) + (payload,) = workspace.load_binding_payloads(paths, binding=binding, filemanager=filemanager) + expected = rgb2gray(pixels.astype(np.float32) / 255) if mode == "monochrome" else pixels.astype(bool) if mode == "mask" else pixels assert np.allclose(payload.data, expected) metadata = payload.metadata assert metadata.axis_position(ColourAxis) == (-1 if mode == "rgb" else None) diff --git a/tests/unit/test_runtime_context_declaration.py b/tests/unit/test_runtime_context_declaration.py index bbdeb131be..8095b1a954 100644 --- a/tests/unit/test_runtime_context_declaration.py +++ b/tests/unit/test_runtime_context_declaration.py @@ -54,9 +54,9 @@ def test_absent_declaration_retains_signature_inference(): assert FunctionContractAttribute.runtime_context_parameter not in vars( _inferred_context ) - assert CallableContract.from_callable( - _inferred_context - ).runtime_context_parameter == ("context") + assert CallableContract.from_callable(_inferred_context).runtime_context_parameter == ( + "context" + ) def test_none_declaration_preserves_callable_identity_and_public_signature(): @@ -75,11 +75,8 @@ def process(image, *, context=None): @pytest.mark.parametrize( ("func", "expected"), - [ - (_inferred_context, "context"), - (_without_context, None), - (_named_context, "execution_context"), - ], + [(_inferred_context, "context"), (_without_context, None), + (_named_context, "execution_context")], ) def test_namespace_reconstruction_retains_captured_selection(func, expected): metadata = CallableMetadata.from_callable(func) @@ -99,9 +96,9 @@ def test_reference_and_pickle_reconstruction_retain_selection(func): reference = FunctionReferenceTransportAuthority.function_reference(func) restored_reference = pickle.loads(pickle.dumps(reference)) assert restored_reference.resolve() is func - assert CallableContract.from_callable( - restored_reference - ).runtime_context_parameter == (expected) + assert CallableContract.from_callable(restored_reference).runtime_context_parameter == ( + expected + ) contract = CallableContract.from_callable(restored_reference) assert pickle.loads(pickle.dumps(contract)).runtime_context_parameter == expected @@ -118,12 +115,9 @@ def process(image, *, context=None): assert contract.runtime_context_parameter is None assert "context" not in contract.runtime_owned_parameter_names assert "context" not in parameter_exclusions(process) - assert ( - inspect.signature(contract.resolve_canonical_raw_callable()) - .parameters["context"] - .default - is None - ) + assert inspect.signature(contract.resolve_canonical_raw_callable()).parameters[ + "context" + ].default is None @pytest.mark.parametrize( @@ -174,10 +168,7 @@ def test_save_images_variants_keep_independent_injection_declarations(): for func in (save_images, save_images_with_measurements): reference = FunctionReferenceTransportAuthority.function_reference(func) expected = CallableContract.from_callable(func).runtime_context_parameter - assert ( - CallableContract.from_callable(reference).runtime_context_parameter - == expected - ) + assert CallableContract.from_callable(reference).runtime_context_parameter == expected def test_measurement_variant_keeps_real_axis_scoped_filename_context(): @@ -211,12 +202,8 @@ def test_named_context_declaration_does_not_resolve_unrelated_forward_references assert inspect.signature(process).parameters["image"].annotation == ( "DefinedAfterDeclaration" ) - assert ( - vars(process)[FunctionContractAttribute.runtime_context_parameter] == "context" - ) - assert ( - CallableMetadata.from_callable(process).runtime_context_parameter == "context" - ) + assert vars(process)[FunctionContractAttribute.runtime_context_parameter] == "context" + assert CallableMetadata.from_callable(process).runtime_context_parameter == "context" marker = namespace["DefinedAfterDeclaration"]() assert process(marker, context=marker) == (marker, marker) diff --git a/tests/unit/test_runtime_equivalence.py b/tests/unit/test_runtime_equivalence.py index 195de14507..b4dcce0efd 100644 --- a/tests/unit/test_runtime_equivalence.py +++ b/tests/unit/test_runtime_equivalence.py @@ -1,8 +1,6 @@ from __future__ import annotations -from openhcs.interop.cellprofiler.measurement_dialect import ( - CellProfilerEquivalencePolicy, -) +from openhcs.interop.cellprofiler.measurement_dialect import CellProfilerEquivalencePolicy from openhcs.core.measurement_dialect import PlainMeasurementDialect @@ -1498,9 +1496,7 @@ def test_runtime_execution_equivalence_detects_artifact_count_mismatch( RuntimeExportObservation.from_output_root(tmp_path / "candidate"), ) - report = runtime_artifact_execution_equivalence( - reference, candidate, policy=RuntimeEquivalencePolicy() - ) + report = runtime_artifact_execution_equivalence(reference, candidate, policy=RuntimeEquivalencePolicy()) assert report.failure_messages() == ( "runtime artifact counts differ: " @@ -11457,7 +11453,7 @@ class DeclaredIdentityDialect(CellProfilerMeasurementDialect): snapshot = RuntimeMeasurementSnapshot.from_output_snapshot( RuntimeOutputSnapshot(tables=(parents, children)), policy=policy ) - (relationship,) = snapshot.correlated_relationships.values() + relationship, = snapshot.correlated_relationships.values() assert relationship.source_keys == (ObjectInstanceKey(7, 0),) assert relationship.target_keys == (ObjectInstanceKey(11, 0),) diff --git a/tests/unit/test_runtime_equivalence_package.py b/tests/unit/test_runtime_equivalence_package.py index 6961b884bd..194a3046e6 100644 --- a/tests/unit/test_runtime_equivalence_package.py +++ b/tests/unit/test_runtime_equivalence_package.py @@ -1,5 +1,4 @@ from openhcs.core.measurement_dialect import PlainMeasurementDialect - """Runtime equivalence package boundary tests.""" import ast @@ -46,10 +45,10 @@ PROJECT_ROOT = Path(__file__).parents[2] + + def test_runtime_equivalence_policy_types_have_package_owner() -> None: - policy = RuntimeEquivalencePolicy( - measurement_dialect=PlainMeasurementDialect.shared() - ) + policy = RuntimeEquivalencePolicy(measurement_dialect=PlainMeasurementDialect.shared()) assert policy.measurement_dialect is not None assert normalize_runtime_identifier("MeanIntensity_OrigBlue") == ( @@ -181,6 +180,8 @@ def test_runtime_equivalence_cell_counter_comparison_has_package_owner() -> None ) + + def test_runtime_equivalence_array_payloads_use_canonical_equivalence_surface() -> None: array = np.array([[1, 2], [3, 4]], dtype=np.int16) diff --git a/tests/unit/test_runtime_execution_validation.py b/tests/unit/test_runtime_execution_validation.py index ab8e4652d5..6306a45e9a 100644 --- a/tests/unit/test_runtime_execution_validation.py +++ b/tests/unit/test_runtime_execution_validation.py @@ -519,7 +519,8 @@ def test_runtime_execution_observation_reads_plate_export_from_exact_owner( assert observation.exports.table_outputs == (contracted_output,) assert expectation.artifact_kinds == frozenset((MeasurementsArtifactType,)) assert tuple( - (item.axis_id, item.artifact_kinds) for item in expectation.axis_expectations + (item.axis_id, item.artifact_kinds) + for item in expectation.axis_expectations ) == ( ("A01", frozenset((MeasurementsArtifactType,))), ("A02", frozenset()), diff --git a/tests/unit/test_runtime_image_singleton_alignment.py b/tests/unit/test_runtime_image_singleton_alignment.py index 86cfef04ad..0bdb70631c 100644 --- a/tests/unit/test_runtime_image_singleton_alignment.py +++ b/tests/unit/test_runtime_image_singleton_alignment.py @@ -3,10 +3,7 @@ import numpy as np import pytest -from openhcs.core.aligned_image_payload import ( - AlignedImageStack, - compose_aligned_image_payload, -) +from openhcs.core.aligned_image_payload import (AlignedImageStack, compose_aligned_image_payload) from openhcs.core.runtime_image_values import ( ImagePayloadMetadata, MaskedImagePayload, @@ -66,7 +63,9 @@ def test_scalar_image_aligns_with_nominal_singleton_owner( composition = compose_aligned_image_payload("Measurement consumer", payloads) - assert composition.execution_mode is (AlignedStackExecution) + assert composition.execution_mode is ( + AlignedStackExecution + ) assert isinstance(composition.payload, AlignedImageStack) assert len(composition.payload.slices) == 1 bundle = composition.payload.slices[0] @@ -116,7 +115,9 @@ def test_same_shaped_scalar_images_do_not_invent_runtime_alignment() -> None: assert composition.execution_mode is FullStackExecution assert not isinstance(composition.payload, AlignedImageStack) - assert composition.payload.metadata.plane_axis is (RuntimePlaneAxis.SOURCE_BINDING) + assert composition.payload.metadata.plane_axis is ( + RuntimePlaneAxis.SOURCE_BINDING + ) def test_singleton_runtime_projection_consumes_rgb_image_and_mask_axis_together() -> ( diff --git a/tests/unit/test_runtime_slice_projection.py b/tests/unit/test_runtime_slice_projection.py index 278014c5a9..880f0c79cf 100644 --- a/tests/unit/test_runtime_slice_projection.py +++ b/tests/unit/test_runtime_slice_projection.py @@ -7,12 +7,7 @@ from arraybridge.decorators import PRESERVE_INPUT_DTYPE_CONFIG import openhcs.processing.backends.cellprofiler # noqa: F401 -from openhcs.core.aligned_image_payload import ( - AlignedImageSliceContext, - AlignedImageStack, - ImageOutputBundle, - ImagePayloadComposition, -) +from openhcs.core.aligned_image_payload import (AlignedImageSliceContext, AlignedImageStack, ImageOutputBundle, ImagePayloadComposition) from openhcs.core.callable_contract import KeywordRuntimeParameter from openhcs.core.measurement_row_materialization import ( DataclassMeasurementColumnarRows, @@ -132,9 +127,7 @@ class UndeclaredValue: RuntimeSliceProjection.value_for_slice( UndeclaredValue(), RuntimePlaneAxisValueProjection.from_selected_plane( - axis=RuntimePlaneAxis.RUNTIME_SLICE, - plane_index=0, - axis_size=1, + axis=RuntimePlaneAxis.RUNTIME_SLICE, plane_index=0, axis_size=1, ), ) @@ -207,9 +200,7 @@ def test_object_label_payload_selects_nominal_object_label_projection() -> None: projected = RuntimeSliceProjection.value_for_slice( payload, RuntimePlaneAxisValueProjection.from_selected_plane( - axis=RuntimePlaneAxis.RUNTIME_SLICE, - plane_index=0, - axis_size=1, + axis=RuntimePlaneAxis.RUNTIME_SLICE, plane_index=0, axis_size=1, ), ) assert projected is payload @@ -665,7 +656,9 @@ def test_variable_component_projection_requires_declared_plane_provenance() -> N RuntimeSliceProjectionDeclarationError, match="nominal payload.*RUNTIME_SLICE", ): - OptionalSourceIdentity.project_payload_items(request) + OptionalSourceIdentity.project_payload_items( + request + ) def test_variable_component_projection_validates_declared_cardinality() -> None: @@ -683,7 +676,9 @@ def test_variable_component_projection_validates_declared_cardinality() -> None: ) with pytest.raises(ValueError, match="declared 'runtime_slice' axis of size 2"): - OptionalSourceIdentity.project_payload_items(request) + OptionalSourceIdentity.project_payload_items( + request + ) def test_variable_component_projection_rejects_expanded_source_provenance() -> None: @@ -704,7 +699,9 @@ def test_variable_component_projection_rejects_expanded_source_provenance() -> N ValueError, match="source provenance must exactly match.*9 != 3", ): - OptionalSourceIdentity.project_payload_items(request) + OptionalSourceIdentity.project_payload_items( + request + ) def test_variable_component_projection_uses_declared_plane_axis() -> None: @@ -713,15 +710,17 @@ def test_variable_component_projection_uses_declared_plane_axis() -> None: ) payload = _declared_plane_metadata(2).payload_with(stack, None) - projected = OptionalSourceIdentity.project_payload_items( - RuntimeProjectionSourceIdentityRequest( - value=payload, - source_description="declared image stack", - variable_components=(Microscopy.Site,), - plane_projection=RuntimePlaneAxisValueProjection.preserve( - axis=RuntimePlaneAxis.RUNTIME_SLICE, - axis_size=2, - ), + projected = ( + OptionalSourceIdentity.project_payload_items( + RuntimeProjectionSourceIdentityRequest( + value=payload, + source_description="declared image stack", + variable_components=(Microscopy.Site,), + plane_projection=RuntimePlaneAxisValueProjection.preserve( + axis=RuntimePlaneAxis.RUNTIME_SLICE, + axis_size=2, + ), + ) ) ) @@ -786,11 +785,13 @@ def test_source_identity_request_projects_explicit_source_binding_label_planes() ), ) - projected = OptionalSourceIdentity.project_payload_items( - RuntimeProjectionSourceIdentityRequest( - value=labels, - source_description="source-bound labels", - plane_projection=labels.declared_plane_projection(), + projected = ( + OptionalSourceIdentity.project_payload_items( + RuntimeProjectionSourceIdentityRequest( + value=labels, + source_description="source-bound labels", + plane_projection=labels.declared_plane_projection(), + ) ) ) diff --git a/tests/unit/test_runtime_value_store.py b/tests/unit/test_runtime_value_store.py index 0067a7fe58..3a5a54a771 100644 --- a/tests/unit/test_runtime_value_store.py +++ b/tests/unit/test_runtime_value_store.py @@ -70,16 +70,9 @@ ComponentSelector, NamedSourceBinding, ) -from openhcs.core.source_matching import ( - SourceImageSetIdentityCompatibility, - SourceImageSetIdentityPolicy, -) -from openhcs.interop.cellprofiler.runtime.artifact_binding import ( - RuntimeInputBindingRequest, -) -from tests.unit.cellprofiler_runtime_test_support import ( - cellprofiler_runtime_adapter_for_test, -) +from openhcs.core.source_matching import SourceImageSetIdentityCompatibility, SourceImageSetIdentityPolicy +from openhcs.interop.cellprofiler.runtime.artifact_binding import RuntimeInputBindingRequest +from tests.unit.cellprofiler_runtime_test_support import cellprofiler_runtime_adapter_for_test from openhcs.core.axes import Axis, AxisFamily from openhcs.domains.microscopy.axes import Microscopy @@ -220,7 +213,10 @@ def test_store_transport_excludes_derived_caches_and_retains_records(): restored = pickle.loads(serialized_without_cache) assert restored.revision == store.revision assert len(restored) == len(store) == 1 - assert restored.observed_values[0].data.row_mappings() == value.data.row_mappings() + assert ( + restored.observed_values[0].data.row_mappings() + == value.data.row_mappings() + ) assert ( restored.query_cache(RuntimeObjectLabelMeasurementQueryCache).cached_value( query @@ -933,9 +929,7 @@ def test_runtime_artifact_input_projection_transposes_producer_stack_axis() -> N group_plan, payload, execution_scope=RuntimeExecutionAxisScope.from_raw( - "A01", - component=Microscopy.Channel, - value=channel, + "A01", component=Microscopy.Channel, value=channel, ), ), path=group_plan.path, @@ -992,15 +986,11 @@ def test_artifact_candidate_scopes_preserve_projected_site_time_correlation() -> path = "/memory/processed_pixels.pkl" variables = (Microscopy.Site, Microscopy.Timepoint) output_plan = ArtifactOutputPlan( - name="image", - path=path, - artifact_type=ImageArtifactType, + name="image", path=path, artifact_type=ImageArtifactType, variable_components=variables, ) storage_plan = ArtifactInputPlan( - name="image", - path=path, - artifact_type=ImageArtifactType, + name="image", path=path, artifact_type=ImageArtifactType, variable_components=variables, ) payload = ImagePayloadMetadata( @@ -1017,11 +1007,8 @@ def test_artifact_candidate_scopes_preserve_projected_site_time_correlation() -> np.asarray([[[True, False], [False, True]], [[False, True], [True, False]]]), ) store = RuntimeValueStore() - store.record( - RuntimeValue.normalize(output_plan, payload, axis_id="A01"), - path=path, - backend="memory", - ) + store.record(RuntimeValue.normalize(output_plan, payload, axis_id="A01"), + path=path, backend="memory") edge = _runtime_input_edge( storage_plan, invocation_scope=ComponentGroupScope.dynamic(Microscopy.Site), @@ -1033,15 +1020,11 @@ def test_artifact_candidate_scopes_preserve_projected_site_time_correlation() -> consumer_variable_components=(), ) runtime_input = RuntimeArtifactInput( - edge_plan=edge, - backend="memory", - axis_scope=RuntimeExecutionAxisScope.from_raw( - "A01", component=None, value=None - ), + edge_plan=edge, backend="memory", + axis_scope=RuntimeExecutionAxisScope.from_raw("A01", component=None, value=None), ) candidates = runtime_input.candidate_execution_scopes( - store, - ComponentGroupScope.dynamic(Microscopy.Site), + store, ComponentGroupScope.dynamic(Microscopy.Site), variable_components=ComponentSet(), ) (coordinates,) = tuple(candidates) @@ -1053,51 +1036,30 @@ def test_artifact_candidate_scopes_preserve_projected_site_time_correlation() -> np.testing.assert_array_equal(selected.mask, payload.mask[0]) -def test_artifact_candidate_scopes_keep_actual_producer_group_as_fixed_context() -> ( - None -): +def test_artifact_candidate_scopes_keep_actual_producer_group_as_fixed_context() -> None: path = "/memory/produced_pixels.pkl" output_plan = ArtifactOutputPlan( - name="image", - path=path, - artifact_type=ImageArtifactType, - group_component=Microscopy.Channel, - group_keys=("1",), + name="image", path=path, artifact_type=ImageArtifactType, + group_component=Microscopy.Channel, group_keys=("1",), ) storage_plan = ArtifactInputPlan( - name="image", - path=path, - artifact_type=ImageArtifactType, - group_component=Microscopy.Channel, - group_keys=("1",), + name="image", path=path, artifact_type=ImageArtifactType, + group_component=Microscopy.Channel, group_keys=("1",), ) payload = ImagePayloadMetadata( source_path="/source/old_filename_w9.tif", - source_component_metadata={ - "site": "2", - "timepoint": "3", - "channel": "1", - "z_index": "1", - }, + source_component_metadata={"site": "2", "timepoint": "3", "channel": "1", "z_index": "1"}, ).payload_with(np.full((2, 2), 17.0), None) store = RuntimeValueStore() record = store.record( RuntimeValue.normalize_for_execution_scope( - output_plan, - payload, + output_plan, payload, execution_scope=RuntimeExecutionAxisScope.from_raw( - "A01", - component=Microscopy.Channel, - value="1", - fixed_component_values=( - (Microscopy.Site, "2"), - (Microscopy.Timepoint, "3"), - (Microscopy.ZIndex, "1"), - ), + "A01", component=Microscopy.Channel, value="1", + fixed_component_values=((Microscopy.Site, "2"), (Microscopy.Timepoint, "3"), (Microscopy.ZIndex, "1")), ), ), - path=path, - backend="memory", + path=path, backend="memory", ) runtime_input = RuntimeArtifactInput( edge_plan=_runtime_input_edge( @@ -1107,22 +1069,17 @@ def test_artifact_candidate_scopes_keep_actual_producer_group_as_fixed_context() component_scopes=(ComponentGroupScope.dynamic(Microscopy.Site),), consumer_variable_components=(), ), - axis_scope=RuntimeExecutionAxisScope.from_raw( - "A01", component=None, value=None - ), + axis_scope=RuntimeExecutionAxisScope.from_raw("A01", component=None, value=None), backend="memory", ) candidates = runtime_input.candidate_execution_scopes( - store, - ComponentGroupScope.dynamic(Microscopy.Site), + store, ComponentGroupScope.dynamic(Microscopy.Site), variable_components=ComponentSet(), ) (coordinates,) = tuple(candidates) assert coordinates.value_text == "2" assert dict(coordinates.fixed_component_values) == { - Microscopy.Channel: "1", - Microscopy.Timepoint: "3", - Microscopy.ZIndex: "1", + Microscopy.Channel: "1", Microscopy.Timepoint: "3", Microscopy.ZIndex: "1", } selected = replace(runtime_input, axis_scope=coordinates).records(store) assert len(selected) == 1 @@ -1130,39 +1087,26 @@ def test_artifact_candidate_scopes_keep_actual_producer_group_as_fixed_context() second_payload = ImagePayloadMetadata( source_path="/source/another_original.tif", - source_component_metadata={ - "site": "2", - "timepoint": "3", - "channel": "1", - "z_index": "2", - }, + source_component_metadata={"site": "2", "timepoint": "3", "channel": "1", "z_index": "2"}, ).payload_with(np.full((2, 2), 29.0), None) second_record = store.record( RuntimeValue.normalize_for_execution_scope( - output_plan, - second_payload, + output_plan, second_payload, execution_scope=RuntimeExecutionAxisScope.from_raw( - "A01", - component=Microscopy.Channel, - value="1", - fixed_component_values=( - (Microscopy.Site, "2"), - (Microscopy.Timepoint, "3"), - (Microscopy.ZIndex, "2"), - ), + "A01", component=Microscopy.Channel, value="1", + fixed_component_values=((Microscopy.Site, "2"), (Microscopy.Timepoint, "3"), (Microscopy.ZIndex, "2")), ), ), - path=path, - backend="memory", + path=path, backend="memory", ) candidates = runtime_input.candidate_execution_scopes( - store, - ComponentGroupScope.dynamic(Microscopy.Site), + store, ComponentGroupScope.dynamic(Microscopy.Site), variable_components=ComponentSet(), ) assert len(candidates) == 2 selected_records = tuple( - replace(runtime_input, axis_scope=scope).records(store) for scope in candidates + replace(runtime_input, axis_scope=scope).records(store) + for scope in candidates ) assert selected_records == ((record,), (second_record,)) @@ -1590,7 +1534,9 @@ def test_runtime_artifact_input_projection_uses_compiled_group_for_plane_scope() ) resolved = runtime_input.resolve_value(store) - np.testing.assert_array_equal(resolved.data, payload.data) + np.testing.assert_array_equal( + resolved.data, payload.data + ) def test_runtime_value_store_observation_cursor_returns_delta_only(): @@ -1790,9 +1736,7 @@ def _ungrouped_runtime_artifact_input( ) -def _paired_channel_label_input( - *, producer_axis="A01", producer_values=(), producer_path=None -): +def _paired_channel_label_input(*, producer_axis="A01", producer_values=(), producer_path=None): """One exact DNA producer consumed in its paired actin image-set context.""" fixed_values = { @@ -1817,9 +1761,7 @@ def _paired_channel_label_input( output_plan, ObjectLabelSet( name=storage_plan.name, - variant_data=ObjectLabelVariantData( - labels=np.ones((2, 2), dtype=np.uint16) - ), + variant_data=ObjectLabelVariantData(labels=np.ones((2, 2), dtype=np.uint16)), ), execution_scope=RuntimeExecutionAxisScope.from_raw( producer_axis, @@ -1842,12 +1784,10 @@ def _paired_channel_label_input( ), ) source_bindings = CompiledSourceBindingPlan( - bindings=( - NamedSourceBinding( - alias="Actin", - component_identity=(ComponentSelector(Microscopy.Channel, "2"),), - ), - ), + bindings=(NamedSourceBinding( + alias="Actin", + component_identity=(ComponentSelector(Microscopy.Channel, "2"),), + ),), ) runtime_input = _ungrouped_runtime_artifact_input( storage_plan, @@ -1859,13 +1799,9 @@ def _paired_channel_label_input( source=source_bindings.binding_declarations[0].input_spec().ref() ) ) - return ( - store, - record, - replace( - runtime_input, - edge_plan=replace(runtime_input.edge_plan, spec=declared_spec), - ), + return store, record, replace( + runtime_input, + edge_plan=replace(runtime_input.edge_plan, spec=declared_spec), ) @@ -1899,17 +1835,12 @@ def _grouped_label_input_with_distinct_consumer_channel(): output_plan, ObjectLabelSet( name="Cells", - variant_data=ObjectLabelVariantData( - labels=np.ones((2, 2), dtype=np.uint16) - ), + variant_data=ObjectLabelVariantData(labels=np.ones((2, 2), dtype=np.uint16)), ), execution_scope=RuntimeExecutionAxisScope.from_raw( - "W001", - component=Microscopy.Channel, - value="0", + "W001", component=Microscopy.Channel, value="0", fixed_component_values=( - (Microscopy.Site, "1"), - (Microscopy.Timepoint, "1"), + (Microscopy.Site, "1"), (Microscopy.Timepoint, "1"), ), ), ) @@ -1924,12 +1855,9 @@ def _grouped_label_input_with_distinct_consumer_channel(): consumer_variable_components=(Microscopy.ZIndex,), ), axis_scope=RuntimeExecutionAxisScope.from_raw( - "W001", - component=None, - value=None, + "W001", component=None, value=None, fixed_component_values=( - (Microscopy.Site, "1"), - (Microscopy.Channel, "2"), + (Microscopy.Site, "1"), (Microscopy.Channel, "2"), (Microscopy.Timepoint, "1"), ), ), @@ -1944,32 +1872,27 @@ def test_exact_producer_group_does_not_constrain_consumer_source_channel(): assert runtime_input.records(store) == (record,) assert runtime_input.resolve_value(store) is record.data candidates = runtime_input.candidate_execution_scopes( - store, - ComponentGroupScope.ungrouped(), + store, ComponentGroupScope.ungrouped(), variable_components=ComponentSet((Microscopy.ZIndex,)), ) (scope,) = candidates # Discovery still owns the producer's semantic channel, rather than adopting # a later consumer's independently selected source-image channel. assert dict(scope.fixed_component_values) == { - Microscopy.Site: "1", - Microscopy.Channel: "0", + Microscopy.Site: "1", Microscopy.Channel: "0", Microscopy.Timepoint: "1", } @pytest.mark.parametrize("component", [Microscopy.Site, Microscopy.Timepoint]) def test_exact_producer_group_preserves_shared_fixed_context_constraints(component): - store, _record, runtime_input = ( - _grouped_label_input_with_distinct_consumer_channel() - ) + store, _record, runtime_input = _grouped_label_input_with_distinct_consumer_channel() fixed_values = dict(runtime_input.axis_scope.fixed_component_values) fixed_values[component] = "2" wrong_context = replace( runtime_input, axis_scope=replace( - runtime_input.axis_scope, - fixed_component_values=tuple(fixed_values.items()), + runtime_input.axis_scope, fixed_component_values=tuple(fixed_values.items()), ), ) with pytest.raises(RuntimeError, match="Missing RuntimeValueStore record"): @@ -1978,9 +1901,7 @@ def test_exact_producer_group_preserves_shared_fixed_context_constraints(compone @pytest.mark.parametrize("mismatch", ["group", "well", "path", "fixed-z"]) def test_exact_producer_group_rejects_other_producer_or_fixed_plane(mismatch): - _store, record, runtime_input = ( - _grouped_label_input_with_distinct_consumer_channel() - ) + _store, record, runtime_input = _grouped_label_input_with_distinct_consumer_channel() scope = record.key.scope path = record.location.path if mismatch == "group": @@ -1991,13 +1912,8 @@ def test_exact_producer_group_rejects_other_producer_or_fixed_plane(mismatch): path = "/memory/other/Cells_channel_0.pkl" else: scope = RuntimeExecutionAxisScope.from_raw( - scope.axis_id, - component=scope.component, - value=scope.value, - fixed_component_values=( - *scope.fixed_component_values, - (Microscopy.ZIndex, "2"), - ), + scope.axis_id, component=scope.component, value=scope.value, + fixed_component_values=(*scope.fixed_component_values, (Microscopy.ZIndex, "2")), ) runtime_input = replace( runtime_input, @@ -2014,32 +1930,25 @@ def test_exact_producer_group_rejects_other_producer_or_fixed_plane(mismatch): store = RuntimeValueStore() store.record( replace(record, key=replace(record.key, scope=scope)), - path=path, - backend=record.location.backend, + path=path, backend=record.location.backend, ) with pytest.raises(RuntimeError, match="Missing RuntimeValueStore record"): runtime_input.records(store) def test_different_producer_group_axis_preserves_fixed_source_channel_constraint(): - _store, record, runtime_input = ( - _grouped_label_input_with_distinct_consumer_channel() - ) + _store, record, runtime_input = _grouped_label_input_with_distinct_consumer_channel() # SITE is now the selected producer group. CHANNEL remains a genuine fixed # image context and cannot use the selected-group exemption. scope = RuntimeExecutionAxisScope.from_raw( - "W001", - component=Microscopy.Site, - value="1", + "W001", component=Microscopy.Site, value="1", fixed_component_values=( - (Microscopy.Channel, "0"), - (Microscopy.Timepoint, "1"), + (Microscopy.Channel, "0"), (Microscopy.Timepoint, "1"), ), ) storage_plan = replace( runtime_input.edge_plan.storage_plan, - group_component=Microscopy.Site, - group_keys=("1",), + group_component=Microscopy.Site, group_keys=("1",), paths_by_group={"1": record.location.path}, ) runtime_input = replace( @@ -2056,8 +1965,7 @@ def test_different_producer_group_axis_preserves_fixed_source_channel_constraint store = RuntimeValueStore() store.record( replace(record, key=replace(record.key, scope=scope)), - path=record.location.path, - backend=record.location.backend, + path=record.location.path, backend=record.location.backend, ) with pytest.raises(RuntimeError, match="Missing RuntimeValueStore record"): runtime_input.records(store) @@ -2097,18 +2005,14 @@ def consume_labels(image, labels): assert projected.axis_scope is adapter.request.axis_scope assert projected.source_binding_plan is runtime_input.source_binding_plan assert projected.records(store) == (record,) - assert adapter.artifact_input_records("Nuclei", ObjectLabelsArtifactType) == ( - record, - ) + assert adapter.artifact_input_records("Nuclei", ObjectLabelsArtifactType) == (record,) request = RuntimeInputBindingRequest( adapter=adapter, kwargs={}, current_image=np.zeros((2, 2)) ) assert request.artifact_value(edge) is record.data -@pytest.mark.parametrize( - "component", [Microscopy.Site, Microscopy.ZIndex, Microscopy.Timepoint] -) +@pytest.mark.parametrize("component", [Microscopy.Site, Microscopy.ZIndex, Microscopy.Timepoint]) def test_paired_channel_input_rejects_other_context_coordinate(component): store, _record, runtime_input = _paired_channel_label_input( producer_values=((component, "2"),), @@ -2118,19 +2022,13 @@ def test_paired_channel_input_rejects_other_context_coordinate(component): runtime_input.records(store) -@pytest.mark.parametrize( - "producer_axis,producer_path", - [ - ("B01", None), - ("A01", "/memory/other_producer/Nuclei.pkl"), - ], -) -def test_paired_channel_input_rejects_other_well_or_producer( - producer_axis, producer_path -): +@pytest.mark.parametrize("producer_axis,producer_path", [ + ("B01", None), + ("A01", "/memory/other_producer/Nuclei.pkl"), +]) +def test_paired_channel_input_rejects_other_well_or_producer(producer_axis, producer_path): store, _record, runtime_input = _paired_channel_label_input( - producer_axis=producer_axis, - producer_path=producer_path, + producer_axis=producer_axis, producer_path=producer_path, ) with pytest.raises(RuntimeError, match="Missing RuntimeValueStore record"): @@ -2140,8 +2038,7 @@ def test_paired_channel_input_rejects_other_well_or_producer( def test_cross_channel_input_without_source_declaration_remains_exact(): store, _record, runtime_input = _paired_channel_label_input() strict_input = replace( - runtime_input, - source_binding_plan=CompiledSourceBindingPlan.empty(), + runtime_input, source_binding_plan=CompiledSourceBindingPlan.empty(), ) with pytest.raises(RuntimeError, match="Missing RuntimeValueStore record"): @@ -2170,11 +2067,9 @@ def test_context_relation_to_another_source_does_not_borrow_visible_plane_member runtime_input.edge_plan, spec=replace( runtime_input.edge_plan.spec, - relations=( - InputGroupLineageSourceRelation( - ArtifactSpec.input("UnrelatedImage", ImageArtifactType).ref() - ), - ), + relations=(InputGroupLineageSourceRelation( + ArtifactSpec.input("UnrelatedImage", ImageArtifactType).ref() + ),), ), ), ) @@ -2194,9 +2089,7 @@ def test_stored_producer_context_retains_its_declared_source_membership(): spec=replace(edge.spec, relations=()), storage_plan=replace( edge.storage_plan, - relations=( - SourceStackLineageSourceRelation(binding.input_spec().ref()), - ), + relations=(SourceStackLineageSourceRelation(binding.input_spec().ref()),), ), ), ) @@ -2215,19 +2108,12 @@ def test_explicit_consumer_context_does_not_borrow_stored_producer_membership(): runtime_input, edge_plan=replace( edge, - spec=replace( - edge.spec, - relations=( - InputGroupLineageSourceRelation( - ArtifactSpec.input("UnrelatedImage", ImageArtifactType).ref(), - ), - ), - ), + spec=replace(edge.spec, relations=(InputGroupLineageSourceRelation( + ArtifactSpec.input("UnrelatedImage", ImageArtifactType).ref(), + ),)), storage_plan=replace( edge.storage_plan, - relations=( - SourceStackLineageSourceRelation(binding.input_spec().ref()), - ), + relations=(SourceStackLineageSourceRelation(binding.input_spec().ref()),), ), ), ) @@ -2247,22 +2133,16 @@ def test_paired_channel_projection_rejects_a_different_producer_site_plane(): ), ) store.replace( - replace(record, data=payload), - path=record.location.path, - backend=record.location.backend, + replace(record, data=payload), path=record.location.path, backend=record.location.backend ) projected_input = replace( runtime_input, edge_plan=replace( edge, - storage_plan=replace( - edge.storage_plan, variable_components=(Microscopy.Site,) - ), + storage_plan=replace(edge.storage_plan, variable_components=(Microscopy.Site,)), projection=replace( edge.projection, - component_scopes=( - ComponentGroupScope(("1",), component=Microscopy.Site), - ), + component_scopes=(ComponentGroupScope(("1",), component=Microscopy.Site),), ), ), ) @@ -2276,11 +2156,7 @@ def test_paired_channel_input_rejects_ambiguous_address_matched_contexts(): _other_store, other_record, _other_input = _paired_channel_label_input( producer_values=((Microscopy.Channel, "3"),), ) - store.record( - other_record, - path=other_record.location.path, - backend=other_record.location.backend, - ) + store.record(other_record, path=other_record.location.path, backend=other_record.location.backend) with pytest.raises(RuntimeError, match="Ambiguous RuntimeValueStore records"): runtime_input.records(store) @@ -2294,27 +2170,23 @@ def test_runtime_artifact_input_preserves_same_scope_semantic_partitions(): ) -@pytest.mark.parametrize( - "producer_values,consumer_values", - [ - pytest.param((), (), id="exact-unscoped"), - pytest.param((), ((Microscopy.ZIndex, "1"),), id="consumer-only"), - pytest.param(((Microscopy.Timepoint, "1"),), (), id="producer-only"), - pytest.param( - ((Microscopy.Timepoint, "1"),), - ((Microscopy.ZIndex, "1"),), - id="disjoint-partial", - ), - pytest.param( - ((Microscopy.Timepoint, "1"),), - ((Microscopy.ZIndex, "1"), (Microscopy.Timepoint, "1")), - id="shared-partial", - ), - ], -) +@pytest.mark.parametrize("producer_values,consumer_values", [ + pytest.param((), (), id="exact-unscoped"), + pytest.param((), ((Microscopy.ZIndex, "1"),), id="consumer-only"), + pytest.param(((Microscopy.Timepoint, "1"),), (), id="producer-only"), + pytest.param( + ((Microscopy.Timepoint, "1"),), + ((Microscopy.ZIndex, "1"),), + id="disjoint-partial", + ), + pytest.param( + ((Microscopy.Timepoint, "1"),), + ((Microscopy.ZIndex, "1"), (Microscopy.Timepoint, "1")), + id="shared-partial", + ), +]) def test_runtime_artifact_input_accepts_exact_unscoped_and_partial_coordinates( - producer_values, - consumer_values, + producer_values, consumer_values, ): store = RuntimeValueStore() storage_plan = ArtifactInputPlan( @@ -2358,30 +2230,23 @@ def test_runtime_artifact_input_accepts_exact_unscoped_and_partial_coordinates( ).records(store) == (record,) -@pytest.mark.parametrize( - "producer_values,consumer_values", - [ - pytest.param((), (), id="empty-projected-context"), - pytest.param(((Microscopy.Timepoint, "1"),), (), id="producer-only"), - pytest.param((), ((Microscopy.ZIndex, "1"),), id="consumer-only"), - pytest.param( - ((Microscopy.Timepoint, "1"),), - ((Microscopy.ZIndex, "1"),), - id="disjoint-partial", - ), - ], -) +@pytest.mark.parametrize("producer_values,consumer_values", [ + pytest.param((), (), id="empty-projected-context"), + pytest.param(((Microscopy.Timepoint, "1"),), (), id="producer-only"), + pytest.param((), ((Microscopy.ZIndex, "1"),), id="consumer-only"), + pytest.param( + ((Microscopy.Timepoint, "1"),), + ((Microscopy.ZIndex, "1"),), + id="disjoint-partial", + ), +]) def test_exact_input_admits_no_shared_projected_context_constraints( - producer_values, - consumer_values, + producer_values, consumer_values, ): _store, original, original_input = _paired_channel_label_input() storage_plan = original_input.edge_plan.storage_plan producer_scope = RuntimeExecutionAxisScope.from_raw( - "A01", - component=None, - value=None, - fixed_component_values=producer_values, + "A01", component=None, value=None, fixed_component_values=producer_values, ) store = RuntimeValueStore() record = store.record( @@ -2390,24 +2255,17 @@ def test_exact_input_admits_no_shared_projected_context_constraints( data=original.data, materialization_source_metadata=original.materialization_source_metadata, ), - path=original.location.path, - backend=original.location.backend, + path=original.location.path, backend=original.location.backend, ) source = NamedSourceBinding( alias="Reference", - component_identity=( - ComponentSelector(AxisFamily.active().partition_axis(), "A01"), - ), + component_identity=(ComponentSelector(AxisFamily.active().partition_axis(), "A01"),), ) consumer_scope = RuntimeExecutionAxisScope.from_raw( - "A01", - component=None, - value=None, - fixed_component_values=consumer_values, + "A01", component=None, value=None, fixed_component_values=consumer_values, ) runtime_input = _ungrouped_runtime_artifact_input( - storage_plan, - axis_scope=consumer_scope, + storage_plan, axis_scope=consumer_scope, source_binding_plan=CompiledSourceBindingPlan(bindings=(source,)), ) runtime_input = replace( @@ -2423,12 +2281,8 @@ def test_exact_input_admits_no_shared_projected_context_constraints( # Plane-identity compatibility still requires evidence of a shared identity. # Exact artifact admission instead has a proven producer/address and only # checks the additional coordinates constrained on both sides. - assert runtime_input.edge_plan.spec.source_context_sources() == ( - source.input_spec().ref(), - ) - policy = SourceImageSetIdentityPolicy.from_source_bindings( - runtime_input.source_binding_plan - ) + assert runtime_input.edge_plan.spec.source_context_sources() == (source.input_spec().ref(),) + policy = SourceImageSetIdentityPolicy.from_source_bindings(runtime_input.source_binding_plan) components = ComponentSet.collect( (component for component, _value in producer_scope.source_component_values), (component for component, _value in consumer_scope.source_component_values), @@ -2439,8 +2293,7 @@ def test_exact_input_admits_no_shared_projected_context_constraints( ).matches() assert runtime_input.records(store) == (record,) wrong_well = replace( - runtime_input, - axis_scope=replace(consumer_scope, axis_id="B01"), + runtime_input, axis_scope=replace(consumer_scope, axis_id="B01"), ) wrong_producer = replace( runtime_input, @@ -2557,14 +2410,14 @@ def test_runtime_value_store_merges_observed_records_from_worker_boundary(): def test_runtime_measurement_observation_axis_accepts_table_record_once(): value = _runtime_value() record = StoredRuntimeValue( - key=value.key, - data=value.data, - materialization_source_metadata=value.materialization_source_metadata, - location=RuntimeArtifactLocation( + key=value.key, + data=value.data, + materialization_source_metadata=value.materialization_source_metadata, + location=RuntimeArtifactLocation( path="/memory/measurements.pkl", backend="memory", ), - ) + ) axis = RuntimeMeasurementObservationAxis("A01") axis.accept_measurement_table(record, PlainMeasurementDialect.shared()) @@ -2601,10 +2454,7 @@ def test_dynamic_input_query_distinguishes_compiled_paths_with_same_backend(): path=record.location.path, artifact_type=MeasurementsArtifactType, group_component=Microscopy.Channel, - paths_by_group={ - None: record.location.path, - "DAPI": record.location.path, - }, + paths_by_group={None: record.location.path, "DAPI": record.location.path}, ), axis_id="A01", backend="memory", @@ -2732,7 +2582,9 @@ def test_store_transport_excludes_all_derived_lookup_caches(): {"object_id": (1,)}, fields=(FieldSpec("object_id", int),) ) record = store.record(value, path="/memory/measurements.pkl", backend="memory") - latest = store.replace(value, path="/later/measurements.pkl", backend="memory") + latest = store.replace( + value, path="/later/measurements.pkl", backend="memory" + ) original_transport = pickle.dumps(store, protocol=5) query = RuntimeArtifactQuery.from_input_plan( ArtifactInputPlan( @@ -2774,9 +2626,7 @@ def test_unified_store_cache_keeps_both_query_domains_and_empty_results(): from metaclass_registry.caches import BoundedCache store = RuntimeValueStore() - record = store.record( - _runtime_value(), path="/memory/measurements.pkl", backend="memory" - ) + record = store.record(_runtime_value(), path="/memory/measurements.pkl", backend="memory") query = RuntimeArtifactQuery.from_input_plan( ArtifactInputPlan( name="measurements", @@ -2844,8 +2694,7 @@ def test_changed_stored_measurement_subject_requires_a_transient_value(): store = RuntimeValueStore() record = store.record(value, path="/memory/measurements.pkl", backend="memory") plan = ArtifactOutputPlan( - name=value.name, - artifact_type=value.artifact_type, + name=value.name, artifact_type=value.artifact_type, path="/memory/measurements.pkl", ) record.data.subject = MeasurementSubject(MeasurementScope.SAMPLE, "NextImage") @@ -2858,13 +2707,6 @@ def test_changed_stored_measurement_subject_requires_a_transient_value(): assert store.get(record.key) is record assert record.location.path == "/memory/measurements.pkl" assert admitted.data is record.data - assert ( - type( - StoredRuntimeValue.from_output_plan( - plan, - admitted.data, - execution_scope=record.key.scope, - ) - ) - is RuntimeValue - ) + assert type(StoredRuntimeValue.from_output_plan( + plan, admitted.data, execution_scope=record.key.scope, + )) is RuntimeValue diff --git a/tests/unit/test_runtime_values.py b/tests/unit/test_runtime_values.py index e44d7450a2..ef7e293a2f 100644 --- a/tests/unit/test_runtime_values.py +++ b/tests/unit/test_runtime_values.py @@ -240,9 +240,7 @@ def test_object_label_set_rejects_nested_payload_and_implicit_authority_merge() labels = np.array([[0, 1], [2, 0]], dtype=np.int16) unedited = np.array([[0, 1], [1, 0]], dtype=np.int16) payload = ObjectLabelPayload( - variant_data=ObjectLabelVariantData( - labels=labels, variants={UneditedLabels: unedited} - ), + variant_data=ObjectLabelVariantData(labels=labels, variants={UneditedLabels: unedited}), domain=ObjectLabelDomain( declared_object_ids=(1, 2), ), @@ -398,9 +396,8 @@ def test_object_label_pure_2d_aggregator_preserves_dense_payload_domains() -> No assert aggregated.domain.declared_object_id_domains == ((1,), (2,)) -def test_object_label_aggregation_preserves_variant_plane_writes_and_transport() -> ( - None -): + +def test_object_label_aggregation_preserves_variant_plane_writes_and_transport() -> None: planes = (np.array([[0, 1]], dtype=np.int16), np.array([[0, 2]], dtype=np.int32)) values = tuple( ObjectLabelPayload( @@ -412,9 +409,7 @@ def test_object_label_aggregation_preserves_variant_plane_writes_and_transport() for index, plane in enumerate(planes) ) aggregate = ObjectLabelPure2DSliceAggregator.aggregate( - values, - "numpy", - plane_axis=RuntimePlaneAxis.RUNTIME_SLICE, + values, "numpy", plane_axis=RuntimePlaneAxis.RUNTIME_SLICE, ) assert aggregate.shape == (2, 1, 2) assert aggregate.dtype == np.dtype(np.int32) @@ -944,9 +939,7 @@ def test_object_label_source_plane_projection_preserves_projected_variants() -> projected = source.project_source_plane(1, labels=labels[1]) np.testing.assert_array_equal(projected.labels, labels[1]) - np.testing.assert_array_equal( - projected.variant_labels(SmallRemovedLabels), small_removed[1] - ) + np.testing.assert_array_equal(projected.variant_labels(SmallRemovedLabels), small_removed[1]) assert projected.domain.scope is ObjectLabelDomainScope.PAYLOAD assert projected.domain.declared_object_ids == (2, 3) assert projected.source_image_names == ("rawGFP",) @@ -966,9 +959,7 @@ def test_object_label_replacement_identity_preserves_planar_variants() -> None: projected = source.with_measurement_labels(labels) np.testing.assert_array_equal(projected.labels, labels) - np.testing.assert_array_equal( - projected.variant_labels(SmallRemovedLabels), small_removed - ) + np.testing.assert_array_equal(projected.variant_labels(SmallRemovedLabels), small_removed) def test_object_label_plane_projection_preserves_ordered_domains_and_variants() -> None: @@ -1147,7 +1138,9 @@ def test_empty_sparse_ijv_stack_preserves_runtime_slice_count() -> None: np.zeros((2, *shape), dtype=np.int32), ) - projected_items = (OptionalSourceIdentity).project_payload_items( + projected_items = ( + OptionalSourceIdentity + ).project_payload_items( RuntimeProjectionSourceIdentityRequest( value=aggregated, source_description="empty sparse labels", @@ -1210,9 +1203,7 @@ def test_shape_object_feature_table_rejects_undeclared_dense_feature_domain() -> table.rows() -def test_shape_descriptors_align_with_measured_ids_not_missing_declared_labels() -> ( - None -): +def test_shape_descriptors_align_with_measured_ids_not_missing_declared_labels() -> None: table = ShapeObjectFeatureValueTable.from_feature_arrays( { MeasureObjectSizeShapeModule.MeasurementFeature.MAX_FERET_DIAMETER.value: np.asarray( @@ -1536,9 +1527,11 @@ def test_source_context_preserves_target_payload_channel_axis_domain() -> None: plane_axis=RuntimePlaneAxis.RUNTIME_SLICE, ).payload_with(np.zeros((1, 2, 4, 5, 3), dtype=np.float32), None) - contextualized = target.metadata.with_source_context_from( - source.metadata - ).attach_source_context_to(target) + contextualized = ( + target.metadata + .with_source_context_from(source.metadata) + .attach_source_context_to(target) + ) metadata = contextualized.metadata assert metadata.axis_index(ColourAxis, contextualized) == 4 @@ -2534,8 +2527,7 @@ def test_object_label_payload_with_measurement_labels_preserves_domain_and_varia small_removed = np.full_like(labels, 2) payload = ObjectLabelPayload( variant_data=ObjectLabelVariantData( - labels=labels, - variants={UneditedLabels: unedited, SmallRemovedLabels: small_removed}, + labels=labels, variants={UneditedLabels: unedited, SmallRemovedLabels: small_removed} ), source_spatial_domain=SourceSpatialDomain( origin_yx=(4, 5), @@ -2595,7 +2587,9 @@ def test_runtime_projection_requirement_projects_singleton_object_label_stack() ), ) - (item,) = (OptionalSourceIdentity).project_payload_items( + (item,) = ( + OptionalSourceIdentity + ).project_payload_items( RuntimeProjectionSourceIdentityRequest( value=payload, source_description="singleton labels", @@ -2630,7 +2624,9 @@ def test_runtime_projection_requirement_tracks_multi_plane_runtime_coordinates() ), ) - first, second = (OptionalSourceIdentity).project_payload_items( + first, second = ( + OptionalSourceIdentity + ).project_payload_items( RuntimeProjectionSourceIdentityRequest( value=payload, source_description="labels", @@ -2673,7 +2669,9 @@ def test_runtime_projection_does_not_expand_indexed_scalar_source_metadata() -> }, ).payload_with(np.arange(12, dtype=np.uint16).reshape(3, 2, 2), None) - projected = (RequiredSourceComponentMetadata).project_payload_items( + projected = ( + RequiredSourceComponentMetadata + ).project_payload_items( RuntimeProjectionSourceIdentityRequest( value=payload, source_description=source_path, @@ -2698,7 +2696,9 @@ def test_runtime_projection_keeps_unindexed_scalar_volume_as_one_payload() -> No }, ).payload_with(np.arange(8, dtype=np.uint16).reshape(2, 2, 2), None) - (projected,) = (RequiredSourceComponentMetadata).project_payload_items( + (projected,) = ( + RequiredSourceComponentMetadata + ).project_payload_items( RuntimeProjectionSourceIdentityRequest( value=payload, source_description=source_path, @@ -2945,7 +2945,9 @@ def test_group_scoped_stack_artifact_preserves_declared_payload_axis() -> None: ) assert value.data.data.shape == (2, 4, 5) - assert value.data.metadata.plane_axis is RuntimePlaneAxis.RUNTIME_SLICE + assert ( + value.data.metadata.plane_axis is RuntimePlaneAxis.RUNTIME_SLICE + ) def test_group_scoped_measurements_preserve_runtime_row_axis() -> None: @@ -3319,14 +3321,12 @@ def test_spatial_grid_normalizes_pure_2d_sequence_from_canonical_mappings(): value = RuntimeValue.normalize(output_plan, list(grids), axis_id="A01") assert isinstance(value.data, RuntimeSliceAlignedValues) - assert [value.data.value_at(slice_index).name for slice_index in range(2)] == [ - "Grid", - "Grid", - ] - assert [value.data.value_at(slice_index).x_origin for slice_index in range(2)] == [ - 1.0, - 2.0, - ] + assert [ + value.data.value_at(slice_index).name for slice_index in range(2) + ] == ["Grid", "Grid"] + assert [ + value.data.value_at(slice_index).x_origin for slice_index in range(2) + ] == [1.0, 2.0] assert [mapping["x_origin"] for mapping in value.materialization_payload()] == [ 1.0, 2.0, @@ -3540,14 +3540,16 @@ def test_derived_image_payload_context_preserves_declared_resized_spatial_domain source_shape_yx=(4, 5), ) ).payload_with(np.zeros((4, 5), dtype=np.float32), None) - output = source.metadata.with_spatial_resize((4, 15)).payload_with( - np.ones((4, 15), dtype=np.float32), - None, + output = ( + source.metadata + .with_spatial_resize((4, 15)) + .payload_with( + np.ones((4, 15), dtype=np.float32), + None, + ) ) - result = source.metadata.derive_payload( - ImagePayload.of(source), ImagePayload.of(output) - ) + result = source.metadata.derive_payload(ImagePayload.of(source), ImagePayload.of(output)) assert result.metadata.source_spatial_domain == SourceSpatialDomain( origin_yx=(0, 0), @@ -3578,9 +3580,7 @@ def test_derived_image_payload_context_merges_source_provenance_into_output_meta metadata=ImagePayloadMetadata(source_dtype="float32"), ) - result = source.metadata.derive_payload( - ImagePayload.of(source), ImagePayload.of(output) - ) + result = source.metadata.derive_payload(ImagePayload.of(source), ImagePayload.of(output)) result_metadata = result.metadata assert result_metadata.source_dtype == "float32" @@ -3677,9 +3677,7 @@ def test_derived_image_payload_context_replaces_stale_scalar_source_with_planes( ), ) - result = source.metadata.derive_payload( - ImagePayload.of(source), ImagePayload.of(output) - ) + result = source.metadata.derive_payload(ImagePayload.of(source), ImagePayload.of(output)) metadata = result.metadata assert metadata.source_path is None @@ -3733,9 +3731,7 @@ def test_derived_image_payload_context_preserves_output_name_while_replacing_sta ), ) - result = source.metadata.derive_payload( - ImagePayload.of(source), ImagePayload.of(output) - ) + result = source.metadata.derive_payload(ImagePayload.of(source), ImagePayload.of(output)) metadata = result.metadata assert metadata.source_image_provenance_planes.paths == ( @@ -3772,9 +3768,7 @@ def test_derived_image_payload_context_preserves_output_name_while_replacing_sta ), ) - result = source.metadata.derive_payload( - ImagePayload.of(source), ImagePayload.of(output) - ) + result = source.metadata.derive_payload(ImagePayload.of(source), ImagePayload.of(output)) metadata = result.metadata assert metadata.source_path == "/input/A01_s001_w3_z001_t001.tif" @@ -3823,9 +3817,7 @@ def test_derived_image_payload_context_preserves_output_name_while_replacing_sin ), ) - result = source.metadata.derive_payload( - ImagePayload.of(source), ImagePayload.of(output) - ) + result = source.metadata.derive_payload(ImagePayload.of(source), ImagePayload.of(output)) metadata = result.metadata assert metadata.source_path == "/input/Sequence1_s001_w1_z001_t007.tif" @@ -3866,9 +3858,7 @@ def test_derived_image_payload_context_expands_indexed_scalar_source_metadata_fo ) output = np.ones((3, 4, 5), dtype=np.float32) - result = source.metadata.derive_payload( - ImagePayload.of(source), ImagePayload.of(output) - ) + result = source.metadata.derive_payload(ImagePayload.of(source), ImagePayload.of(output)) metadata = result.metadata assert metadata.source_image_provenance_planes.paths == ( @@ -4007,9 +3997,7 @@ def test_derived_image_payload_context_uses_source_provenance_atomically() -> No ), ) - result = source.metadata.derive_payload( - ImagePayload.of(source), ImagePayload.of(output) - ) + result = source.metadata.derive_payload(ImagePayload.of(source), ImagePayload.of(output)) metadata = result.metadata assert metadata.source_path is None @@ -4482,7 +4470,9 @@ def test_object_label_runtime_slice_projection_projects_source_path() -> None: "channel": "1", } assert projected.source_image_provenance_planes.paths == () - assert projected.metadata.source_path == ("/input/A01_s002_w1_z001_t001.TIF") + assert projected.metadata.source_path == ( + "/input/A01_s002_w1_z001_t001.TIF" + ) def test_runtime_slice_projection_does_not_infer_raw_array_axis() -> None: @@ -4538,7 +4528,9 @@ def test_object_label_set_runtime_slice_projection_projects_source_path() -> Non "channel": "1", } assert projected.source_image_provenance_planes.paths == () - assert projected.metadata.source_path == ("/input/A01_s003_w1_z001_t001.TIF") + assert projected.metadata.source_path == ( + "/input/A01_s003_w1_z001_t001.TIF" + ) def test_measurement_table_runtime_slice_projection_projects_source_identity() -> None: @@ -4679,7 +4671,9 @@ def test_pure_2d_auxiliary_aggregator_preserves_image_payload_metadata() -> None # PURE2D image stacks stay deferred slices (70a79fa38); any ImagePayload. assert isinstance(stacked, ImagePayload) assert stacked.data.shape == (2, 2, 3) - assert stacked.metadata.for_source_plane(0).intensity_scale == 65535.0 + assert ( + stacked.metadata.for_source_plane(0).intensity_scale == 65535.0 + ) assert stacked.metadata.for_source_plane(1).source_dtype == "uint8" @@ -4755,7 +4749,8 @@ def test_pure_2d_auxiliary_aggregator_preserves_slice_aligned_spatial_grids() -> assert isinstance(aggregated, RuntimeSliceAlignedValues) assert tuple( - aggregated.value_at(index).x_origin for index in range(aggregated.slice_count) + aggregated.value_at(index).x_origin + for index in range(aggregated.slice_count) ) == (1.0, 3.0) @@ -4974,10 +4969,7 @@ def test_normalize_object_label_set_preserves_dense_label_variants(): name="Nuclei", variant_data=ObjectLabelVariantData( labels=labels, - variants={ - UneditedLabels: unedited_labels, - SmallRemovedLabels: small_removed_labels, - }, + variants={UneditedLabels: unedited_labels, SmallRemovedLabels: small_removed_labels}, ), ), axis_id="A01", @@ -4990,12 +4982,8 @@ def test_normalize_object_label_set_preserves_dense_label_variants(): UneditedLabels, ) np.testing.assert_array_equal(restored.labels, labels) - np.testing.assert_array_equal( - restored.variant_labels(UneditedLabels), unedited_labels - ) - np.testing.assert_array_equal( - restored.variant_labels(SmallRemovedLabels), small_removed_labels - ) + np.testing.assert_array_equal(restored.variant_labels(UneditedLabels), unedited_labels) + np.testing.assert_array_equal(restored.variant_labels(SmallRemovedLabels), small_removed_labels) def test_normalize_object_label_set_accepts_sparse_ijv_representation(): diff --git a/tests/unit/test_saved_source_label_plane_domain.py b/tests/unit/test_saved_source_label_plane_domain.py index 8c96cb52db..3cf532baf7 100644 --- a/tests/unit/test_saved_source_label_plane_domain.py +++ b/tests/unit/test_saved_source_label_plane_domain.py @@ -9,81 +9,54 @@ from openhcs.core.callable_contract import CallableContract from openhcs.core.runtime_image_values import ImagePayloadMetadata from openhcs.core.runtime_measurements import MeasurementRowAxisField -from openhcs.core.runtime_object_label_building import ( - SourceImageObjectLabelBuildRequest, -) +from openhcs.core.runtime_object_label_building import SourceImageObjectLabelBuildRequest from openhcs.core.runtime_object_label_domains import ObjectLabelDomainScope from openhcs.core.runtime_tabular_values import MeasurementObjectRowIdentity -from openhcs.core.runtime_plane_projection import ( - RuntimePlaneAxis, - RuntimePlaneAxisValueProjection, -) +from openhcs.core.runtime_plane_projection import RuntimePlaneAxis, RuntimePlaneAxisValueProjection from openhcs.core.source_image_provenance import SourceImageProvenancePlanes from openhcs.core.source_metadata import SourceVoxelSpacing from openhcs.interop.cellprofiler.runtime.output_recording import ( CellProfilerOutputRecorder, ) -from openhcs.interop.cellprofiler.runtime.function_contract_execution import ( - CellProfilerFunctionContractExecutor, -) -from openhcs.processing.backends.cellprofiler.shape import ( - MeasureObjectSizeShapeModule, - measure_object_size_shape, -) +from openhcs.interop.cellprofiler.runtime.function_contract_execution import CellProfilerFunctionContractExecutor +from openhcs.processing.backends.cellprofiler.shape import MeasureObjectSizeShapeModule, measure_object_size_shape from openhcs.core.image_payload_execution_mode import ( FullStackExecution, ) -def _source( - labels, - *, - axis=RuntimePlaneAxis.RUNTIME_SLICE, - plane_count=None, - spacing=(1.3556, 1.3556), -): +def _source(labels, *, axis=RuntimePlaneAxis.RUNTIME_SLICE, plane_count=None, spacing=(1.3556, 1.3556)): count = labels.shape[0] if plane_count is None else plane_count return ImagePayloadMetadata( plane_axis=axis, source_image_names=("saved_labels",) * count, source_image_provenance_planes=SourceImageProvenancePlanes.from_components( - paths=tuple( - f"/synthetic/A01_s001_w{index + 1}_labels.tif" for index in range(count) - ), - component_metadata=tuple( - {"well": "A01", "site": "1", "channel": str(index + 1)} - for index in range(count) - ), + paths=tuple(f"/synthetic/A01_s001_w{index + 1}_labels.tif" for index in range(count)), + component_metadata=tuple({"well": "A01", "site": "1", "channel": str(index + 1)} for index in range(count)), ), source_voxel_spacing=SourceVoxelSpacing(spacing), ).payload_with(labels, None) def _saved_label_input(source): - spec = ArtifactSpec.input( - "saved_labels", ObjectLabelsArtifactType, parameter_name="labels" + spec = ArtifactSpec.input("saved_labels", ObjectLabelsArtifactType, parameter_name="labels") + return CellProfilerOutputRecorder.for_artifact_type(spec.artifact_type).runtime_input_value( + spec=spec, value=source ) - return CellProfilerOutputRecorder.for_artifact_type( - spec.artifact_type - ).runtime_input_value(spec=spec, value=source) @pytest.mark.parametrize("axis", tuple(RuntimePlaneAxis)) @pytest.mark.parametrize("plane_count", (1, 2)) def test_explicit_object_projection_preserves_axis_without_acquisition_provenance( - axis, - plane_count, + axis, plane_count, ): labels = np.ones((plane_count, 6, 7), dtype=np.int32) source = ImagePayloadMetadata(plane_axis=axis).payload_with(labels) projection = RuntimePlaneAxisValueProjection.preserve( - axis=axis, - axis_size=plane_count, + axis=axis, axis_size=plane_count, ) payload = SourceImageObjectLabelBuildRequest( - image=source, - labels=labels, - plane_projection=projection, + image=source, labels=labels, plane_projection=projection, ).payload() assert payload.plane_axis is axis @@ -93,11 +66,9 @@ def test_explicit_object_projection_preserves_axis_without_acquisition_provenanc other_axis = next(member for member in RuntimePlaneAxis if member is not axis) with pytest.raises(ValueError, match="conflicts with the source-image axis"): SourceImageObjectLabelBuildRequest( - image=source, - labels=labels, + image=source, labels=labels, plane_projection=RuntimePlaneAxisValueProjection.preserve( - axis=other_axis, - axis_size=plane_count, + axis=other_axis, axis_size=plane_count, ), ).payload() @@ -109,32 +80,23 @@ def test_original_artifact_admission_and_full_stack_shape_keep_2d_planes(axis): labels[1, 2:5, 3:7] = 106 source = _source(labels, axis=axis) source_bytes = labels.tobytes() - spec = ArtifactSpec.input( - "saved_labels", ObjectLabelsArtifactType, parameter_name="labels" + spec = ArtifactSpec.input("saved_labels", ObjectLabelsArtifactType, parameter_name="labels") + value = CellProfilerOutputRecorder.for_artifact_type(spec.artifact_type).runtime_input_value( + spec=spec, value=source ) - value = CellProfilerOutputRecorder.for_artifact_type( - spec.artifact_type - ).runtime_input_value(spec=spec, value=source) assert value.domain.scope is ObjectLabelDomainScope.PLANE assert value.plane_axis is axis assert value.domain.declared_object_id_domains == ((29,), (106,)) - assert ( - value.source_provenance.identity() - == source.metadata.source_provenance.identity() - ) + assert value.source_provenance.identity() == source.metadata.source_provenance.identity() planes = value.measurement_planes() assert tuple(plane.labels.shape for plane in planes) == ((8, 9), (8, 9)) for index, plane in enumerate(planes): np.testing.assert_array_equal(plane.labels, labels[index]) - assert plane.parent_image_source_voxel_spacing == SourceVoxelSpacing( - (1.3556, 1.3556) - ) + assert plane.parent_image_source_voxel_spacing == SourceVoxelSpacing((1.3556, 1.3556)) contract = CallableContract.from_callable(measure_object_size_shape) _image, rows = CellProfilerFunctionContractExecutor().execute( - contract, - contract.resolve_canonical_raw_callable(), - source, + contract, contract.resolve_canonical_raw_callable(), source, {"labels": value, "calculate_advanced": False, "calculate_zernikes": False}, execution_mode=FullStackExecution, ) @@ -165,15 +127,8 @@ def test_singleton_runtime_slice_is_not_a_physical_volume(): def test_declared_runtime_planes_can_each_contain_a_genuine_calibrated_volume(): labels = np.ones((2, 3, 6, 7), dtype=np.int32) value = _saved_label_input(_source(labels, spacing=(2.0, 1.3556, 1.3556))) - assert tuple(plane.labels.shape for plane in value.measurement_planes()) == ( - (3, 6, 7), - (3, 6, 7), - ) - assert value.parent_image_source_voxel_spacing.spacing_for_ndim(3) == ( - 2.0, - 1.3556, - 1.3556, - ) + assert tuple(plane.labels.shape for plane in value.measurement_planes()) == ((3, 6, 7), (3, 6, 7)) + assert value.parent_image_source_voxel_spacing.spacing_for_ndim(3) == (2.0, 1.3556, 1.3556) def test_undeclared_volume_and_explicit_payload_override_are_not_guessed_planar(): @@ -182,15 +137,11 @@ def test_undeclared_volume_and_explicit_payload_override_are_not_guessed_planar( (_source(labels, axis=None, spacing=(2.0, 1.3556, 1.3556)), {}), (_source(labels), {"domain_scope": ObjectLabelDomainScope.PAYLOAD}), ): - value = SourceImageObjectLabelBuildRequest( - image=source, labels=labels, **options - ).payload() + value = SourceImageObjectLabelBuildRequest(image=source, labels=labels, **options).payload() assert value.domain.scope is ObjectLabelDomainScope.PAYLOAD assert value.plane_axis is None assert value.measurement_planes()[0].labels.shape == (2, 6, 7) - with pytest.raises( - ValueError, match="Cannot project 2-D source voxel spacing onto 3-D data" - ): + with pytest.raises(ValueError, match="Cannot project 2-D source voxel spacing onto 3-D data"): SourceVoxelSpacing((1.3556, 1.3556)).spacing_for_ndim(3) @@ -205,20 +156,16 @@ def test_declared_axis_rejects_payload_wide_ids_and_mismatched_label_geometry(): labels = np.ones((2, 6, 7), dtype=np.int32) source = _source(labels) projection = RuntimePlaneAxisValueProjection.from_source_declaration( - source.metadata.plane_axis, - source.metadata.source_provenance, + source.metadata.plane_axis, source.metadata.source_provenance, ) with pytest.raises(ValueError, match="per-plane domains"): SourceImageObjectLabelBuildRequest( - image=source, - labels=labels, - declared_object_ids=(1,), + image=source, labels=labels, declared_object_ids=(1,), plane_projection=projection, ).payload() with pytest.raises(ValueError, match="spatial shape"): SourceImageObjectLabelBuildRequest( - image=source, - labels=np.ones((2, 5, 7), dtype=np.int32), + image=source, labels=np.ones((2, 5, 7), dtype=np.int32), plane_projection=projection, ).payload() @@ -227,11 +174,8 @@ def test_explicit_projection_conflicting_with_source_declaration_is_rejected(): labels = np.ones((2, 6, 7), dtype=np.int32) with pytest.raises(ValueError, match="conflicts"): SourceImageObjectLabelBuildRequest( - image=_source(labels), - labels=labels, - plane_projection=RuntimePlaneAxisValueProjection.preserve( - axis=RuntimePlaneAxis.SOURCE_BINDING, axis_size=2 - ), + image=_source(labels), labels=labels, + plane_projection=RuntimePlaneAxisValueProjection.preserve(axis=RuntimePlaneAxis.SOURCE_BINDING, axis_size=2), ).payload() @@ -239,15 +183,11 @@ class PhysicalCalibrationRequired: """Independent admission capability for a new physical-measurement leaf.""" def payload(self, **kwargs): - SourceVoxelSpacing.require_physical_pixel_size( - (self.metadata.source_voxel_spacing,) - ) + SourceVoxelSpacing.require_physical_pixel_size((self.metadata.source_voxel_spacing,)) return super().payload(**kwargs) -class CalibratedSavedLabels( - PhysicalCalibrationRequired, SourceImageObjectLabelBuildRequest -): +class CalibratedSavedLabels(PhysicalCalibrationRequired, SourceImageObjectLabelBuildRequest): """A new declaration composes calibration admission with the existing builder.""" @@ -255,26 +195,16 @@ def test_new_calibrated_leaf_executes_cooperative_admission_and_plane_building() labels = np.ones((2, 6, 7), dtype=np.int32) source = _source(labels) projection = RuntimePlaneAxisValueProjection.from_source_declaration( - source.metadata.plane_axis, - source.metadata.source_provenance, + source.metadata.plane_axis, source.metadata.source_provenance, ) value = CalibratedSavedLabels( - image=source, - labels=labels, - plane_projection=projection, + image=source, labels=labels, plane_projection=projection, ).payload() assert value.domain.scope is ObjectLabelDomainScope.PLANE assert len(value.measurement_planes()) == 2 with pytest.raises(ValueError, match="Physical scalar pixel size requires"): - CalibratedSavedLabels( - image=_source(labels, spacing=()), - labels=labels, - plane_projection=projection, - ).payload() - assert ( - _saved_label_input(_source(labels, spacing=())).domain.scope - is ObjectLabelDomainScope.PLANE - ) + CalibratedSavedLabels(image=_source(labels, spacing=()), labels=labels, plane_projection=projection).payload() + assert _saved_label_input(_source(labels, spacing=())).domain.scope is ObjectLabelDomainScope.PLANE class PairedSourceAdmission: @@ -293,31 +223,17 @@ class PairedSourceProjection(PairedSourceAdmission, RuntimePlaneAxisValueProject @pytest.mark.parametrize("axis", tuple(RuntimePlaneAxis)) def test_source_constructor_admits_a_new_cooperative_projection_capability(axis): source = _source(np.ones((2, 6, 7), dtype=np.int32), axis=axis) - value = PairedSourceProjection.from_source_declaration( - axis, source.metadata.source_provenance - ) + value = PairedSourceProjection.from_source_declaration(axis, source.metadata.source_provenance) assert isinstance(value, PairedSourceProjection) assert value.axis is axis assert value.axis_size == 2 assert value.selected_plane(1).require_plane_index() == 1 bigger = _source(np.ones((3, 6, 7), dtype=np.int32), axis=axis) with pytest.raises(ValueError, match="at most two source planes"): - PairedSourceProjection.from_source_declaration( - axis, bigger.metadata.source_provenance - ) - assert ( - RuntimePlaneAxisValueProjection.from_source_declaration( - axis, bigger.metadata.source_provenance - ).axis_size - == 3 - ) + PairedSourceProjection.from_source_declaration(axis, bigger.metadata.source_provenance) + assert RuntimePlaneAxisValueProjection.from_source_declaration(axis, bigger.metadata.source_provenance).axis_size == 3 # Absence is a declaration, not inferred from three source records. - assert ( - PairedSourceProjection.from_source_declaration( - None, bigger.metadata.source_provenance - ) - is None - ) + assert PairedSourceProjection.from_source_declaration(None, bigger.metadata.source_provenance) is None @pytest.mark.parametrize("axis", tuple(RuntimePlaneAxis)) @@ -338,13 +254,8 @@ def test_produced_volume_global_object_domain_is_independent_of_source_plane_axi assert value.plane_axis is None assert len(value.measurement_planes()) == 1 np.testing.assert_array_equal(value.measurement_planes()[0].labels, labels) - assert ( - value.source_provenance.identity() - == source.metadata.source_provenance.identity() - ) - assert ( - value.parent_image_source_voxel_spacing == source.metadata.source_voxel_spacing - ) + assert value.source_provenance.identity() == source.metadata.source_provenance.identity() + assert value.parent_image_source_voxel_spacing == source.metadata.source_voxel_spacing @pytest.mark.parametrize("axis", tuple(RuntimePlaneAxis)) @@ -370,21 +281,15 @@ def test_explicit_plane_domain_still_requires_projection_and_rejects_global_coun source = _source(labels, axis=axis) with pytest.raises(ValueError, match="requires an exact plane projection"): SourceImageObjectLabelBuildRequest( - image=source, - labels=labels, - domain_scope=ObjectLabelDomainScope.PLANE, + image=source, labels=labels, domain_scope=ObjectLabelDomainScope.PLANE, ).payload() projection = RuntimePlaneAxisValueProjection.from_source_declaration( - source.metadata.plane_axis, - source.metadata.source_provenance, + source.metadata.plane_axis, source.metadata.source_provenance, ) with pytest.raises(ValueError, match="per-plane domains"): SourceImageObjectLabelBuildRequest( - image=source, - labels=labels, - domain_scope=ObjectLabelDomainScope.PLANE, - plane_projection=projection, - declared_object_count=1, + image=source, labels=labels, domain_scope=ObjectLabelDomainScope.PLANE, + plane_projection=projection, declared_object_count=1, ).payload() @@ -401,8 +306,7 @@ def test_generic_output_context_uses_execution_projection_not_image_storage_axis assert volume.domain.declared_object_ids == (29, 106) np.testing.assert_array_equal(volume.measurement_planes()[0].labels, labels) projection = RuntimePlaneAxisValueProjection.from_source_declaration( - source.metadata.plane_axis, - source.metadata.source_provenance, + source.metadata.plane_axis, source.metadata.source_provenance, ) planes = ImagePayload.of(labels).object_label_output(source, projection) assert planes.domain.scope is ObjectLabelDomainScope.PLANE @@ -417,9 +321,7 @@ def test_saved_ingress_preserves_already_typed_object_domain_and_identity(): labels = np.ones((2, 6, 7), dtype=np.int32) source = _source(labels, spacing=(2.0, 1.3556, 1.3556)) produced = SourceImageObjectLabelBuildRequest( - image=source, - labels=labels, - declared_object_count=1, + image=source, labels=labels, declared_object_count=1, ).label_set(name="saved_labels") admitted = _saved_label_input(produced) assert admitted is produced diff --git a/tests/unit/test_selected_plane_stream_materialization.py b/tests/unit/test_selected_plane_stream_materialization.py index 55fabeb3c8..081bfd6b17 100644 --- a/tests/unit/test_selected_plane_stream_materialization.py +++ b/tests/unit/test_selected_plane_stream_materialization.py @@ -99,9 +99,7 @@ def _source_stack(axis, component=Microscopy.Channel.name): ) def test_public_selected_checkpoint_materializes_and_streams_without_storage_axes( axis, - output_type, - artifact_name, - dtype, + output_type, artifact_name, dtype, tmp_path, monkeypatch, ): @@ -167,10 +165,11 @@ def capture_stream(data_list, paths, backend, **kwargs): request.source.metadata.component_metadata_for_item(path, 0)["channel"] == 2 ) assert payload.metadata.source_image_names == ("FITC",) - assert payload.metadata.source_path == "/input/A01_s001_w2_z001_t001.tif" - assert payload.metadata.source_voxel_spacing == SourceVoxelSpacing( - (1.3556, 1.3556) + assert ( + payload.metadata.source_path + == "/input/A01_s001_w2_z001_t001.tif" ) + assert payload.metadata.source_voxel_spacing == SourceVoxelSpacing((1.3556, 1.3556)) finally: viewer.cleanup() @@ -239,7 +238,9 @@ def with_data(self, data): ) assert calls == ["observe", "reverse", "declaration"] assert payload.metadata.source_image_names == ("FITC", "DAPI") - assert payload.metadata.retained_plane_component_values() == {"channel": ("2", "1")} + assert payload.metadata.retained_plane_component_values() == { + "channel": ("2", "1") + } np.testing.assert_array_equal(np.asarray(payload), selected.data) @@ -261,13 +262,11 @@ class DeclaredResponse(ObservedProjection, SelectedDiagnosticPlaneImageOutput): pass source = _source_stack(RuntimePlaneAxis.SOURCE_BINDING) - pixels = np.full((1, 7, 8), 0.75, dtype=np.float32) + pixels = np.full((1, 7, 8), .75, dtype=np.float32) response = DeclaredResponse(pixels, (1,)) calls.clear() payload = ImageArtifactType.contextualize_output( - source, - response, - None, + source, response, None, RuntimePlaneAxisValueProjection.preserve( axis=RuntimePlaneAxis.SOURCE_BINDING, axis_size=2 ), @@ -292,27 +291,11 @@ def test_all_public_declared_outputs_persist_with_selected_qa_streams( image=source, labels=labels, plane_projection=projection ).payload() summary = NeuriteOutgrowthSummary( - **{ - field.name: 0 - for field in fields(NeuriteOutgrowthSummary) - if field.name != "coordinate_unit" - }, + **{field.name: 0 for field in fields(NeuriteOutgrowthSummary) if field.name != "coordinate_unit"}, coordinate_unit=SourceVoxelSpacingUnit.MICROMETERS, ) cell = NeuriteOutgrowthCellResult( - 1, - 1, - SourceVoxelSpacingUnit.MICROMETERS, - 0.0, - 0, - 0.0, - 0.0, - 0.0, - 0, - 0.0, - 1.0, - 0.0, - False, + 1, 1, SourceVoxelSpacingUnit.MICROMETERS, 0.0, 0, 0.0, 0.0, 0.0, 0, 0.0, 1.0, 0.0, False ) selected = ImageArtifactType.contextualize_output( source, @@ -341,10 +324,8 @@ def test_all_public_declared_outputs_persist_with_selected_qa_streams( ), *NeuriteAdmissionPlanes( np.asarray(source)[1].astype(np.float32), - labels[1] > 0, - labels[1] > 0, - np.asarray(source)[1].astype(np.float32), - labels[1] > 0, + labels[1] > 0, labels[1] > 0, + np.asarray(source)[1].astype(np.float32), labels[1] > 0, np.asarray(source)[1].astype(np.float32), ).selected_outputs(1), *NeuriteOwnershipPlanes(labels[1], labels[1], labels[1]).selected_outputs(1), @@ -381,9 +362,7 @@ def test_all_public_declared_outputs_persist_with_selected_qa_streams( output_dir=tmp_path / "images", ) for output, value in zip(plans, values, strict=True): - value = ( - ImageArtifactType if output is None else output.artifact_type - ).contextualize_output( + value = (ImageArtifactType if output is None else output.artifact_type).contextualize_output( source, value, output, @@ -407,16 +386,13 @@ def capture_stream(data_list, paths, backend, **kwargs): monkeypatch.setattr(filemanager, "save_batch", capture_stream) try: - PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( - filemanager, plan, context - ) + PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs(filemanager, plan, context) materializations = runtime_artifact_materializations(plan, context) assert {item.output_plan.name for item in materializations} == set( specs.names() ) assert len(materializations) == 12 + len( - NeuriteAdmissionPlanes.artifact_specs() - + NeuriteOwnershipPlanes.artifact_specs() + NeuriteAdmissionPlanes.artifact_specs() + NeuriteOwnershipPlanes.artifact_specs() ) for item in materializations: outputs = item.outputs(plan, context) @@ -428,8 +404,7 @@ def capture_stream(data_list, paths, backend, **kwargs): if paths[0].endswith(".checkpoint.tif") ] assert len(checkpoints) == 5 + len( - NeuriteAdmissionPlanes.artifact_specs() - + NeuriteOwnershipPlanes.artifact_specs() + NeuriteAdmissionPlanes.artifact_specs() + NeuriteOwnershipPlanes.artifact_specs() ) for (data,), (path,), request in checkpoints: assert ( diff --git a/tests/unit/test_singleton_rgb_label_invocation_477.py b/tests/unit/test_singleton_rgb_label_invocation_477.py index cfe05f87b9..7ee07aea04 100644 --- a/tests/unit/test_singleton_rgb_label_invocation_477.py +++ b/tests/unit/test_singleton_rgb_label_invocation_477.py @@ -1,32 +1,15 @@ """Scalar RGB invocations consume only an explicitly declared singleton root.""" - import numpy as np import pytest -from openhcs.core.artifacts import ( - ArtifactSpec, - ImageArtifactType, - ObjectLabelsArtifactType, -) +from openhcs.core.artifacts import ArtifactSpec, ImageArtifactType, ObjectLabelsArtifactType from openhcs.core.function_contract_metadata import FunctionContractAttribute from openhcs.core.runtime_image_values import ImageMetadataPayload, ImagePayloadMetadata -from openhcs.core.runtime_object_label_domains import ( - ObjectLabelDomain, - ObjectLabelDomainScope, -) -from openhcs.core.runtime_object_labels import ( - ObjectLabelSet, - ObjectLabelVariantData, - object_label_dense_array, -) -from openhcs.core.runtime_plane_projection import ( - RuntimePlaneAxis, - RuntimePlaneProjection, -) +from openhcs.core.runtime_object_label_domains import ObjectLabelDomain, ObjectLabelDomainScope +from openhcs.core.runtime_object_labels import ObjectLabelSet, ObjectLabelVariantData, object_label_dense_array +from openhcs.core.runtime_plane_projection import RuntimePlaneAxis, RuntimePlaneProjection from openhcs.core.runtime_slice_projection import RuntimeSliceProjectionDeclarationError -from openhcs.interop.cellprofiler.runtime.function_contract_execution import ( - CellProfilerFunctionContractExecutor, -) +from openhcs.interop.cellprofiler.runtime.function_contract_execution import CellProfilerFunctionContractExecutor from openhcs.processing.backends.cellprofiler.outlines import OverlayOutlinesModule from tests.unit.test_cellprofiler_module_execution import ( _FakeCellProfilerRuntime, @@ -46,69 +29,42 @@ def _invocation_case(*, root_count=1, label_count=1): - image_spec = ArtifactSpec.input("GFPandDNA", ImageArtifactType) - labels_spec = ArtifactSpec.input("Nuclei", ObjectLabelsArtifactType) + image_spec = ArtifactSpec.input('GFPandDNA', ImageArtifactType) + labels_spec = ArtifactSpec.input('Nuclei', ObjectLabelsArtifactType) raw = OverlayOutlinesModule.require_callable() - contract = _compiled_callable_contract( - raw, artifact_inputs=(image_spec, labels_spec) - ) + contract = _compiled_callable_contract(raw, artifact_inputs=(image_spec, labels_spec)) data = np.linspace(0, 1, 8 * 9 * 3, dtype=np.float32).reshape(8, 9, 3) - image = ImageMetadataPayload( - data, - ImagePayloadMetadata( - axes=PayloadAxes.colour_samples(2), source_image_names=("GFPandDNA",) - ), - ) + image = ImageMetadataPayload(data, ImagePayloadMetadata(axes=PayloadAxes.colour_samples(2), source_image_names=('GFPandDNA',))) label_data = np.zeros((label_count, 8, 9), dtype=np.int32) label_data[:, 2:5, 3:6] = 1 labels = ObjectLabelSet( - name="Nuclei", - variant_data=ObjectLabelVariantData(labels=label_data), + name='Nuclei', variant_data=ObjectLabelVariantData(labels=label_data), plane_axis=RuntimePlaneAxis.RUNTIME_SLICE, - domain=ObjectLabelDomain( - declared_object_id_domains=((1,),) * label_count, - scope=ObjectLabelDomainScope.PLANE, - ), + domain=ObjectLabelDomain(declared_object_id_domains=((1,),) * label_count, scope=ObjectLabelDomainScope.PLANE), ) adapter = _FakeCellProfilerRuntime( - {"GFPandDNA": image}, - objects={"Nuclei": labels}, - callable_contract=contract, - artifact_input_edges=tuple( - _artifact_input_edge_for_test(spec) for spec in (image_spec, labels_spec) - ), + {'GFPandDNA': image}, objects={'Nuclei': labels}, callable_contract=contract, + artifact_input_edges=tuple(_artifact_input_edge_for_test(spec) for spec in (image_spec, labels_spec)), plane_projection=RuntimePlaneProjection.stack(root_count), ) executor = _module_executor(contract) - request = executor._image_request( - image, - adapter, - module_type=OverlayOutlinesModule, - active_input_specs=(image_spec, labels_spec), - ) + request = executor._image_request(image, adapter, module_type=OverlayOutlinesModule, active_input_specs=(image_spec, labels_spec)) return raw, contract, image, labels, adapter, executor, request def _bind(case): raw, contract, image, labels, adapter, executor, request = case return executor._invocation_request( - image_request=request, - adapter=adapter, - current_image=image, - kwargs={"outline_colors": ("#21FFFF",)}, - module_type=OverlayOutlinesModule, + image_request=request, adapter=adapter, current_image=image, + kwargs={'outline_colors': ('#21FFFF',)}, module_type=OverlayOutlinesModule, ) def _execute(case, invocation): raw, contract, *_ = case return CellProfilerFunctionContractExecutor().execute( - contract, - raw, - invocation.payload, - invocation.kwargs, - execution_mode=invocation.execution_mode, - plane_projection=invocation.plane_projection, + contract, raw, invocation.payload, invocation.kwargs, + execution_mode=invocation.execution_mode, plane_projection=invocation.plane_projection, ) @@ -118,21 +74,15 @@ def test_match_image_labels_consume_declared_singleton_root_for_scalar_rgb(): image_before = image.data.copy() labels_before = object_label_dense_array(labels).copy() invocation = _bind(case) - selected = invocation.kwargs["object_labels"][0] + selected = invocation.kwargs['object_labels'][0] assert invocation.execution_mode is NaturalExecution assert invocation.plane_projection is None assert invocation.payload.metadata.axis_position(ColourAxis) == 2 assert selected.plane_axis is None assert object_label_dense_array(selected).shape == (8, 9) output = _execute(case, invocation) - scalar_labels = ObjectLabelSet( - name="Nuclei", - variant_data=ObjectLabelVariantData(labels=labels_before[0]), - domain=ObjectLabelDomain( - declared_object_ids=(1,), scope=ObjectLabelDomainScope.PAYLOAD - ), - ) - expected = raw(image, object_labels=(scalar_labels,), outline_colors=("#21FFFF",)) + scalar_labels = ObjectLabelSet(name='Nuclei', variant_data=ObjectLabelVariantData(labels=labels_before[0]), domain=ObjectLabelDomain(declared_object_ids=(1,), scope=ObjectLabelDomainScope.PAYLOAD)) + expected = raw(image, object_labels=(scalar_labels,), outline_colors=('#21FFFF',)) np.testing.assert_array_equal(output.data, expected.data) np.testing.assert_array_equal(image.data, image_before) np.testing.assert_array_equal(object_label_dense_array(labels), labels_before) @@ -140,76 +90,49 @@ def test_match_image_labels_consume_declared_singleton_root_for_scalar_rgb(): assert image.metadata.plane_axis is None -@pytest.mark.parametrize("root_count", (None, 2)) +@pytest.mark.parametrize('root_count', (None, 2)) def test_scalar_rgb_cannot_authorize_unknown_or_multiple_root_planes(root_count): case = _invocation_case(root_count=root_count) invocation = _bind(case) - assert ( - invocation.kwargs["object_labels"][0].plane_axis - is RuntimePlaneAxis.RUNTIME_SLICE - ) - with pytest.raises( - RuntimeSliceProjectionDeclarationError, - match="Kwargs cannot create image-axis execution semantics", - ): + assert invocation.kwargs['object_labels'][0].plane_axis is RuntimePlaneAxis.RUNTIME_SLICE + with pytest.raises(RuntimeSliceProjectionDeclarationError, match='Kwargs cannot create image-axis execution semantics'): _execute(case, invocation) -def test_explicit_full_stack_label_contract_preserves_scalar_boundary_refusal( - monkeypatch, -): +def test_explicit_full_stack_label_contract_preserves_scalar_boundary_refusal(monkeypatch): raw = OverlayOutlinesModule.require_callable() - monkeypatch.setattr( - raw, - FunctionContractAttribute.object_label_input_execution_mode, - FullStackLabels, - ) + monkeypatch.setattr(raw, FunctionContractAttribute.object_label_input_execution_mode, FullStackLabels) case = _invocation_case() invocation = _bind(case) - assert ( - invocation.kwargs["object_labels"][0].plane_axis - is RuntimePlaneAxis.RUNTIME_SLICE - ) - with pytest.raises( - RuntimeSliceProjectionDeclarationError, - match="Kwargs cannot create image-axis execution semantics", - ): + assert invocation.kwargs['object_labels'][0].plane_axis is RuntimePlaneAxis.RUNTIME_SLICE + with pytest.raises(RuntimeSliceProjectionDeclarationError, match='Kwargs cannot create image-axis execution semantics'): _execute(case, invocation) def test_declared_singleton_root_refuses_mismatched_label_cardinality(): case = _invocation_case(label_count=2) - with pytest.raises( - ValueError, match="Object-label runtime plane-axis cardinality mismatch" - ): + with pytest.raises(ValueError, match='Object-label runtime plane-axis cardinality mismatch'): _bind(case) def test_final_module_mode_sees_full_labels_before_scalar_projection(monkeypatch): case = _invocation_case() - def mode(cls, default, *, image, kwargs, variable_components): - assert kwargs["object_labels"][0].plane_axis is RuntimePlaneAxis.RUNTIME_SLICE - assert object_label_dense_array(kwargs["object_labels"][0]).shape == (1, 8, 9) + assert kwargs['object_labels'][0].plane_axis is RuntimePlaneAxis.RUNTIME_SLICE + assert object_label_dense_array(kwargs['object_labels'][0]).shape == (1, 8, 9) return FullStackExecution - - monkeypatch.setattr(OverlayOutlinesModule, "execution_mode", classmethod(mode)) + monkeypatch.setattr(OverlayOutlinesModule, 'execution_mode', classmethod(mode)) invocation = _bind(case) assert invocation.execution_mode is FullStackExecution - assert ( - invocation.kwargs["object_labels"][0].plane_axis - is RuntimePlaneAxis.RUNTIME_SLICE - ) + assert invocation.kwargs['object_labels'][0].plane_axis is RuntimePlaneAxis.RUNTIME_SLICE def test_module_mode_failure_precedes_projection_guard(monkeypatch): case = _invocation_case(label_count=2) - error = RuntimeError("module image-domain selection failed") - + error = RuntimeError('module image-domain selection failed') def mode(cls, default, *, image, kwargs, variable_components): raise error - - monkeypatch.setattr(OverlayOutlinesModule, "execution_mode", classmethod(mode)) + monkeypatch.setattr(OverlayOutlinesModule, 'execution_mode', classmethod(mode)) with pytest.raises(RuntimeError) as caught: _bind(case) assert caught.value is error diff --git a/tests/unit/test_smoke_installed_gui.py b/tests/unit/test_smoke_installed_gui.py index 7461331098..359a1dc2e0 100644 --- a/tests/unit/test_smoke_installed_gui.py +++ b/tests/unit/test_smoke_installed_gui.py @@ -174,9 +174,7 @@ def test_gui_smoke_waits_for_gui_endpoint_before_reading_catalog() -> None: class UnreachableFunctionCatalogService: @staticmethod def prepare() -> Future[FunctionCatalogPage]: - raise AssertionError( - "The catalog attaches only after the GUI endpoint starts." - ) + raise AssertionError("The catalog attaches only after the GUI endpoint starts.") with pytest.raises( OperationTimeoutError, diff --git a/tests/unit/test_snapshot_deadline.py b/tests/unit/test_snapshot_deadline.py index 3fc216571f..09460a9580 100644 --- a/tests/unit/test_snapshot_deadline.py +++ b/tests/unit/test_snapshot_deadline.py @@ -56,9 +56,7 @@ def close(self, *args, **kwargs): pass def send(self, data, flags): - accepted = NapariAcceptedControlRequest.from_wire_mapping( - pickle.loads(data) - ) + accepted = NapariAcceptedControlRequest.from_wire_mapping(pickle.loads(data)) self.message = accepted.message server.accepted_control_requests.put(replace(accepted, response=reply)) server.process_messages() @@ -112,38 +110,28 @@ def poll(self, timeout): def test_snapshot_wire_offer_is_admitted_once_and_capture_uses_local_identity(tmp_path): from zmqruntime.messages import MessageFields - from openhcs.runtime.viewer_protocol import ( - ViewerControlMessageRequest, - ViewerRuntimeEndpoint, - ) + from openhcs.runtime.viewer_protocol import ViewerControlMessageRequest, ViewerRuntimeEndpoint from openhcs.runtime.zmq_config import OPENHCS_ZMQ_CONFIG original = ViewerWindowSnapshotRequest.from_fields( - connection=ExecutionConnectionSpec(port=5584), - output_dir_path=str(tmp_path), + connection=ExecutionConnectionSpec(port=5584), output_dir_path=str(tmp_path), ).start_operation() envelope = ViewerControlMessageRequest( endpoint=ViewerRuntimeEndpoint( transport=original.connection.transport_endpoint(OPENHCS_ZMQ_CONFIG), config=OPENHCS_ZMQ_CONFIG, ), - message_type="screenshot", - payload=original, + message_type="screenshot", payload=original, operation_deadline=original.operation_deadline, ) wire = pickle.loads(pickle.dumps(envelope.to_wire_mapping())) assert wire["payload"].operation_deadline is None - assert ( - 0 < wire[MessageFields.OBSERVATION_BUDGET_SECONDS] <= original.timeout_ms / 1000 - ) + assert 0 < wire[MessageFields.OBSERVATION_BUDGET_SECONDS] <= original.timeout_ms / 1000 admitted_at = time.monotonic() accepted = NapariAcceptedControlRequest.from_wire_mapping(wire) assert accepted.observation_deadline is not original.operation_deadline assert accepted.observation_deadline.expires_at >= admitted_at - assert ( - accepted.message["payload"].snapshot_operation_deadline() - is accepted.observation_deadline - ) + assert accepted.message["payload"].snapshot_operation_deadline() is accepted.observation_deadline assert MessageFields.OBSERVATION_BUDGET_SECONDS not in accepted.message # Qt/action consumers receive an admitted object, not a reusable duration. with pytest.raises(KeyError): diff --git a/tests/unit/test_source_binding_workspace.py b/tests/unit/test_source_binding_workspace.py index 20343d21ee..0b07be04d8 100644 --- a/tests/unit/test_source_binding_workspace.py +++ b/tests/unit/test_source_binding_workspace.py @@ -11,9 +11,7 @@ from openhcs.constants.constants import Backend from openhcs.core.artifacts import ImageArtifactType, ObjectLabelsArtifactType -from openhcs.processing.backends.cellprofiler.infrastructure import ( - relative_voxel_spacing, -) +from openhcs.processing.backends.cellprofiler.infrastructure import relative_voxel_spacing from openhcs.core.source_binding_workspace import SourceBindingWorkspaceProjector from openhcs.core.runtime_tabular_values import FieldSpec from openhcs.core.source_metadata import ( @@ -71,46 +69,29 @@ def test_transient_axis_views_observe_each_new_projection_without_reusing_releas cache = VirtualWorkspaceSourceProjectionCache() refs = MappingProxyType({"A01.tif": SourcePixelRef("disk", "source.tif")}) for generation in range(20): - metadata = MappingProxyType( - { - "A01.tif": MappingProxyType( - { - "well": "A01", - "generation": str(generation), - } - ) - } - ) + metadata = MappingProxyType({"A01.tif": MappingProxyType({ + "well": "A01", "generation": str(generation), + })}) projection = VirtualWorkspaceSourceProjection(refs, metadata, "/plate") filtered = cache.filtered_by_axis(projection, axis_id="A01") - assert filtered.source_metadata_by_path["A01.tif"]["generation"] == str( - generation - ) + assert filtered.source_metadata_by_path["A01.tif"]["generation"] == str(generation) del projection -def test_source_projection_document_replacement_preserves_absence_and_schema_rejection( - tmp_path, -): +def test_source_projection_document_replacement_preserves_absence_and_schema_rejection(tmp_path): cache = VirtualWorkspaceSourceProjectionCache() - native_document = { - FIELDS.SUBDIRECTORIES: { - "images": {FIELDS.IMAGE_FILES: ["native.tif"]}, - } - } - empty_workspace = { - FIELDS.SUBDIRECTORIES: { - "images": {FIELDS.WORKSPACE_MAPPING: {}}, - } - } + native_document = {FIELDS.SUBDIRECTORIES: { + "images": {FIELDS.IMAGE_FILES: ["native.tif"]}, + }} + empty_workspace = {FIELDS.SUBDIRECTORIES: { + "images": {FIELDS.WORKSPACE_MAPPING: {}}, + }} for document in (native_document, empty_workspace): assert cache.projection_for(tmp_path, document) is None assert cache.projection_for(tmp_path, document) is None - malformed_document = { - FIELDS.SUBDIRECTORIES: { - "images": {FIELDS.WORKSPACE_MAPPING: []}, - } - } + malformed_document = {FIELDS.SUBDIRECTORIES: { + "images": {FIELDS.WORKSPACE_MAPPING: []}, + }} with pytest.raises(RuntimeError, match="workspace_mapping must be a mapping"): cache.projection_for(tmp_path, malformed_document) assert cache.projection_for(tmp_path, native_document) is None @@ -129,113 +110,68 @@ def _filemanager() -> FileManager: return FileManager(dict(storage_registry)) -@pytest.mark.parametrize( - "selection,expected_sites", ((None, tuple(range(1, 10))), (7, (7,)), (3, (3,))) -) +@pytest.mark.parametrize("selection,expected_sites", ((None, tuple(range(1, 10))), (7, (7,)), (3, (3,)))) def test_prepared_workspace_admits_declared_source_universe_without_rewriting_provenance( - tmp_path, - monkeypatch, - selection, - expected_sites, + tmp_path, monkeypatch, selection, expected_sites, ): source = tmp_path / "source" source.mkdir() - files = tuple( - source / f"A01_s{site:03}_w{channel}.tif" - for site in range(1, 10) - for channel in (1, 2) - ) + files = tuple(source / f"A01_s{site:03}_w{channel}.tif" + for site in range(1, 10) for channel in (1, 2)) for path in files: _write_tiff_stack(path, (11,)) filemanager = _filemanager() workspace = tmp_path / "prepared" materialization = SourceBindingWorkspaceProjector( - SourceBindingsConfig( - metadata_rules=( - MetadataExtractionRule( - source=MetadataSource.FILE_NAME, - pattern=r"(?PA\d+)_s(?P\d+)_w(?P\d+)", - ), - ) - ), + SourceBindingsConfig(metadata_rules=(MetadataExtractionRule( + source=MetadataSource.FILE_NAME, + pattern=r"(?PA\d+)_s(?P\d+)_w(?P\d+)", + ),)), parser=SourceSchemaFilenameParser(), - ).materialize( - source, - workspace, - filemanager=filemanager, - source_backend=Backend.DISK, - workspace_backend=Backend.DISK, - source_files=files, - ) + ).materialize(source, workspace, filemanager=filemanager, + source_backend=Backend.DISK, workspace_backend=Backend.DISK, + source_files=files) (workspace / FIELDS.DEFAULT_SUBDIRECTORY).mkdir(exist_ok=True) original_bytes = materialization.metadata_path.read_bytes() - config = SourceBindingsConfig( - source_filters=( - () - if selection is None - else ( - SourceFilterClause( - subject=SourceFilterSubject.FILE, - match_type=SourceFilterMatchType.CONTAINS, - value=f"_s{selection:03}_", - ), - ) - ) - ) - handler = DatasetSourceChoice.named("auto").open( - workspace, filemanager=filemanager, source_bindings_config=config - ) + config = SourceBindingsConfig(source_filters=() if selection is None else ( + SourceFilterClause(subject=SourceFilterSubject.FILE, + match_type=SourceFilterMatchType.CONTAINS, + value=f"_s{selection:03}_"), + )) + handler = DatasetSourceChoice.named("auto").open(workspace, filemanager=filemanager, source_bindings_config=config) from openhcs.core.dataset_sources.openhcs_format import OpenHCSDatasetSource - assert isinstance(handler, OpenHCSDatasetSource) handler.initialize_workspace(workspace, filemanager) cache = VirtualWorkspaceSourceProjectionCache() full = WorkspaceSourceProjections.from_dataset_metadata( - dataset_root=workspace, - metadata_handler=handler.metadata_handler, - filemanager=filemanager, - cache=cache, + dataset_root=workspace, metadata_handler=handler.metadata_handler, + filemanager=filemanager, cache=cache, ).projection_or_empty() admissions = [] original_admit = SourceBindingWorkspaceProjector.accept_prepared_projection - def observe_admission(projector, projection): admissions.append(projection) return original_admit(projector, projection) - - monkeypatch.setattr( - SourceBindingWorkspaceProjector, "accept_prepared_projection", observe_admission - ) + monkeypatch.setattr(SourceBindingWorkspaceProjector, "accept_prepared_projection", observe_admission) selected = WorkspaceSourceProjections.from_dataset_metadata( - dataset_root=workspace, - metadata_handler=handler.metadata_handler, - filemanager=filemanager, - cache=cache, - source_bindings=config, + dataset_root=workspace, metadata_handler=handler.metadata_handler, + filemanager=filemanager, cache=cache, source_bindings=config, ).projection_or_empty() - assert set(selected.component_values(Microscopy.Site)) == set( - map(str, expected_sites) - ) + assert set(selected.component_values(Microscopy.Site)) == set(map(str, expected_sites)) assert set(selected.component_values(Microscopy.Channel)) == {"1", "2"} assert len(selected.pipeline_start_files()) == len(expected_sites) * 2 for path in selected.pipeline_start_files(): lookup = VirtualWorkspacePathLookup.from_paths(path, path) assert selected.source_ref_for(lookup) is full.source_ref_for(lookup) - assert selected.require_source_projection_for( - lookup - ) is full.require_source_projection_for(lookup) + assert selected.require_source_projection_for(lookup) is full.require_source_projection_for(lookup) assert selected.source_metadata_for(lookup) == full.source_metadata_for(lookup) assert len(full.pipeline_start_files()) == 18 from openhcs.core.context.processing_context import ProcessingContext - context = ProcessingContext(filemanager=filemanager, axis_id="A01") context.dataset_root = workspace context.microscope_handler = handler from openhcs.core import source_workspace_projection - - monkeypatch.setattr( - source_workspace_projection, "DEFAULT_SOURCE_PROJECTION_CACHE", cache - ) + monkeypatch.setattr(source_workspace_projection, "DEFAULT_SOURCE_PROJECTION_CACHE", cache) runtime_projection = WorkspaceSourceProjections.from_context( context, ).projection_or_empty() @@ -244,129 +180,70 @@ def observe_admission(projector, projection): other = ProcessingContext(filemanager=filemanager, axis_id="A01") other.dataset_root = workspace other.microscope_handler = handler - assert ( - context.runtime_source_workspace_projections.cache - is other.runtime_source_workspace_projections.cache - is cache - ) + assert context.runtime_source_workspace_projections.cache is other.runtime_source_workspace_projections.cache is cache assert other.runtime_source_workspace_projections.projection_or_empty() is selected equivalent = WorkspaceSourceProjections.from_dataset_metadata( - dataset_root=workspace, - metadata_handler=handler.metadata_handler, - filemanager=filemanager, - cache=cache, - source_bindings=replace(config), + dataset_root=workspace, metadata_handler=handler.metadata_handler, + filemanager=filemanager, cache=cache, source_bindings=replace(config), ).projection_or_empty() assert equivalent is selected compiled_views = cache.partition_by_axes(selected, axis_ids=("A01",)) - assert ( - context.runtime_source_workspace_projections.projection_or_empty(axis_id="A01") - is compiled_views["A01"] - ) + assert context.runtime_source_workspace_projections.projection_or_empty(axis_id="A01") is compiled_views["A01"] assert len(admissions) == (0 if selection is None else 1) with monkeypatch.context() as admitted_query: - def reject_redecode(_cls, _subdirectory): - raise AssertionError( - "An admitted document must not decode its whole workspace again." - ) - - admitted_query.setattr( - VirtualWorkspaceMapping, "from_subdirectory", classmethod(reject_redecode) - ) - assert ( - context.runtime_source_workspace_projections.projection_or_empty() - is selected - ) - assert ( - context.runtime_source_workspace_projections.projection_or_empty( - axis_id="A01" - ) - is compiled_views["A01"] - ) + raise AssertionError("An admitted document must not decode its whole workspace again.") + admitted_query.setattr(VirtualWorkspaceMapping, "from_subdirectory", classmethod(reject_redecode)) + assert context.runtime_source_workspace_projections.projection_or_empty() is selected + assert context.runtime_source_workspace_projections.projection_or_empty(axis_id="A01") is compiled_views["A01"] prior_authority = context.runtime_source_workspace_projections - handler._source_bindings_config = replace( - config, - source_filters=( - SourceFilterClause( - subject=SourceFilterSubject.FILE, - match_type=SourceFilterMatchType.CONTAINS, - value="_s008_", - ), - ), - ) + handler._source_bindings_config = replace(config, source_filters=( + SourceFilterClause(subject=SourceFilterSubject.FILE, + match_type=SourceFilterMatchType.CONTAINS, value="_s008_"), + )) changed = context.runtime_source_workspace_projections assert changed is not prior_authority assert changed.projection_or_empty().component_values(Microscopy.Site) == ("8",) handler._source_bindings_config = replace(config, source_filters=()) assert context.runtime_source_workspace_projections.projection_or_empty() is full - handler._source_bindings_config = replace( - config, - source_filters=( - SourceFilterClause( - subject=SourceFilterSubject.FILE, - match_type=SourceFilterMatchType.CONTAINS, - value="_s099_", - ), - ), - ) + handler._source_bindings_config = replace(config, source_filters=( + SourceFilterClause(subject=SourceFilterSubject.FILE, + match_type=SourceFilterMatchType.CONTAINS, value="_s099_"), + )) with pytest.raises(ValueError, match="matched no prepared workspace sources"): context.runtime_source_workspace_projections.projection_or_empty() handler._source_bindings_config = config - assert ( - context.runtime_source_workspace_projections.projection_or_empty() is selected - ) + assert context.runtime_source_workspace_projections.projection_or_empty() is selected handler.metadata_handler.invalidate_metadata_cache() - replacement = context.runtime_source_workspace_projections.projection_or_empty( - axis_id="A01" - ) + replacement = context.runtime_source_workspace_projections.projection_or_empty(axis_id="A01") assert replacement is not compiled_views["A01"] - assert ( - replacement.source_metadata_by_path - == compiled_views["A01"].source_metadata_by_path - ) + assert replacement.source_metadata_by_path == compiled_views["A01"].source_metadata_by_path from openhcs.core.config import PipelineConfig, LazySourceBindingsConfig from openhcs.core.orchestrator.orchestrator import PipelineOrchestrator - orchestrator = PipelineOrchestrator( - workspace, - pipeline_config=PipelineConfig( - source_bindings_config=LazySourceBindingsConfig( - source_filters=config.source_filters - ), + workspace, pipeline_config=PipelineConfig( + source_bindings_config=LazySourceBindingsConfig(source_filters=config.source_filters), ), ).initialize() assert orchestrator.source_workspace_files() == selected.pipeline_start_files() - assert set(orchestrator.get_component_keys(Microscopy.Site)) == set( - map(str, expected_sites) - ) - assert orchestrator.get_component_keys( - Microscopy.Site, [str(expected_sites[0])] - ) == [str(expected_sites[0])] + assert set(orchestrator.get_component_keys(Microscopy.Site)) == set(map(str, expected_sites)) + assert orchestrator.get_component_keys(Microscopy.Site, [str(expected_sites[0])]) == [str(expected_sites[0])] assert orchestrator.get_component_keys(Microscopy.Site, ["99"]) == [] from tests.unit.test_completed_output_publication_lifecycle import _facts - context.record_completed_step_outputs(_facts(workspace)) with_output = WorkspaceSourceProjections.from_context(context).projection_or_empty() assert set(with_output.pipeline_start_files()) == { - *selected.pipeline_start_files(), - str(workspace / "images/saved.tif"), + *selected.pipeline_start_files(), str(workspace / "images/saved.tif"), } assert materialization.metadata_path.read_bytes() == original_bytes with pytest.raises(ValueError, match="matched no prepared workspace sources"): WorkspaceSourceProjections.from_dataset_metadata( - dataset_root=workspace, - metadata_handler=handler.metadata_handler, - filemanager=filemanager, - source_bindings=SourceBindingsConfig( - source_filters=( - SourceFilterClause( - subject=SourceFilterSubject.FILE, - match_type=SourceFilterMatchType.CONTAINS, - value="_s099_", - ), - ) - ), + dataset_root=workspace, metadata_handler=handler.metadata_handler, + filemanager=filemanager, source_bindings=SourceBindingsConfig(source_filters=( + SourceFilterClause(subject=SourceFilterSubject.FILE, + match_type=SourceFilterMatchType.CONTAINS, + value="_s099_"), + )), ).projection_if_available() @@ -750,10 +627,7 @@ def binding(alias: str, channel: str) -> NamedSourceBinding: for projection in projection_set.projections } == {("B02", "3")} assert metadata[Microscopy.Well.metadata_collection_field] == {"B02": None} - assert metadata[Microscopy.Channel.metadata_collection_field] == { - "1": "DNA", - "2": "RNA", - } + assert metadata[Microscopy.Channel.metadata_collection_field] == {"1": "DNA", "2": "RNA"} def test_source_binding_workspace_projects_declared_groups_to_wells(tmp_path): @@ -922,13 +796,9 @@ def test_group_address_preserves_distinct_source_well_as_literal_metadata(tmp_pa def test_nonempty_source_bindings_select_their_declared_microscope_handler( tmp_path, ): - handler = DatasetSourceChoice.named("auto").open( - tmp_path, - filemanager=_filemanager(), - source_bindings_config=SourceBindingsConfig( + handler = DatasetSourceChoice.named("auto").open(tmp_path, filemanager=_filemanager(), source_bindings_config=SourceBindingsConfig( bindings=(NamedSourceBinding(alias="DNA"),) - ), - ) + )) assert isinstance(handler, SourceBindingsSource) @@ -936,13 +806,9 @@ def test_nonempty_source_bindings_select_their_declared_microscope_handler( def test_nonempty_source_bindings_override_format_specific_microscope( tmp_path, ): - handler = DatasetSourceChoice.named("imagexpress").open( - tmp_path, - filemanager=_filemanager(), - source_bindings_config=SourceBindingsConfig( + handler = DatasetSourceChoice.named("imagexpress").open(tmp_path, filemanager=_filemanager(), source_bindings_config=SourceBindingsConfig( bindings=(NamedSourceBinding(alias="DNA"),) - ), - ) + )) assert isinstance(handler, SourceBindingsSource) @@ -1088,7 +954,9 @@ def test_order_source_sets_join_imported_metadata_across_aliases(tmp_path): assert {projection.source_alias for projection in projections} == {"DNA", "Actin"} for projection in projections: - original = dict(SourceMetadataFields.declared_items(projection.source_metadata)) + original = dict( + SourceMetadataFields.declared_items(projection.source_metadata) + ) assert original["Compound"] == "DMSO" assert original["Dose"] == "0" @@ -1584,7 +1452,8 @@ def test_source_bindings_reinitialization_resolves_current_imported_metadata(tmp ) ) assert ( - dict(SourceMetadataFields.declared_items(first_metadata))["Compound"] == "First" + dict(SourceMetadataFields.declared_items(first_metadata))["Compound"] + == "First" ) metadata_path = tmp_path / "openhcs_metadata.json" @@ -2155,13 +2024,9 @@ def test_openhcs_metadata_handler_resolves_subdirectory_inputs(tmp_path): def test_source_bindings_handler_is_registry_constructed(): - handler = DatasetSourceChoice.named("source_bindings").open( - Path("."), - filemanager=object(), - source_bindings_config=SourceBindingsConfig( + handler = DatasetSourceChoice.named("source_bindings").open(Path("."), filemanager=object(), source_bindings_config=SourceBindingsConfig( bindings=(NamedSourceBinding(alias="DNA"),), - ), - ) + )) assert isinstance(handler, SourceBindingsSource) diff --git a/tests/unit/test_source_bindings.py b/tests/unit/test_source_bindings.py index 798ad622ce..24dc22a791 100644 --- a/tests/unit/test_source_bindings.py +++ b/tests/unit/test_source_bindings.py @@ -703,8 +703,7 @@ def fake_open(cls, root, **kwargs): def test_admitted_request_config_survives_next_live_global_context( - tmp_path, - monkeypatch, + tmp_path, monkeypatch, ): from openhcs.core.orchestrator.orchestrator import PipelineOrchestrator from openhcs.core.pipeline.compiler import AxisCompilationRequest @@ -733,9 +732,7 @@ def test_admitted_request_config_survives_next_live_global_context( ), ) orchestrator = PipelineOrchestrator( - tmp_path, - pipeline_config=authored, - resolved_config=admitted, + tmp_path, pipeline_config=authored, resolved_config=admitted, ) orchestrator.initialize_microscope_handler(admitted) assert received["source_bindings_config"] is admitted.source_bindings_config @@ -747,10 +744,8 @@ def test_admitted_request_config_survives_next_live_global_context( request = AxisCompilationRequest( orchestrator=orchestrator, global_config=admitted, - pipeline=SimpleNamespace(), - path_resolver=SimpleNamespace(), - global_step_axis_filters={}, - enable_visualizer_override=False, + pipeline=SimpleNamespace(), path_resolver=SimpleNamespace(), + global_step_axis_filters={}, enable_visualizer_override=False, source_projections_by_axis={}, materialization_planner=MaterializationFlagPlanner( pipeline_config=admitted, @@ -766,9 +761,7 @@ def test_admitted_request_config_survives_next_live_global_context( assert context.tiff_config is admitted.tiff_config # The next authored request admits current global and local changes. - orchestrator.pipeline_config = PipelineConfig( - dataset_source=OpenHCSDatasetSource, num_workers=3 - ) + orchestrator.pipeline_config = PipelineConfig(dataset_source=OpenHCSDatasetSource, num_workers=3) next_request = orchestrator.get_effective_config() assert next_request.num_workers == 3 assert next_request.auto_add_output_dataset is False @@ -837,7 +830,9 @@ def test_realized_component_values_are_scoped_by_source_selector(monkeypatch) -> {"channel": 2, "site": 7, "z_index": 9}, ] expected = { - component: binding.component_values(component, realized_source_metadata=records) + component: binding.component_values( + component, realized_source_metadata=records + ) for component in AxisFamily.active().axes } matched = [] @@ -847,13 +842,9 @@ def record_match(owner, metadata): matched.append(metadata) return original_match(owner, metadata) - monkeypatch.setattr( - NamedSourceBinding, "matches_realized_source_metadata", record_match - ) + monkeypatch.setattr(NamedSourceBinding, "matches_realized_source_metadata", record_match) domains = binding.component_domains(realized_source_metadata=iter(records)) - assert { - component: domains.get(component, ()) for component in AxisFamily.active().axes - } == expected + assert {component: domains.get(component, ()) for component in AxisFamily.active().axes} == expected assert matched == records assert domains[Microscopy.Site] == ("3",) assert domains[Microscopy.ZIndex] == ("4", "5") @@ -1008,15 +999,7 @@ def test_alias_only_step_bindings_filter_and_order_workspace_projections() -> No ) source_projections = { virtual_path: SourcePlaneProjection( - address=OpenHCSPlaneAddress( - ( - (Microscopy.Well, "R04C09"), - (Microscopy.Site, "11"), - (Microscopy.Channel, channel), - (Microscopy.ZIndex, "1"), - (Microscopy.Timepoint, "1"), - ) - ), + address=OpenHCSPlaneAddress(((Microscopy.Well, "R04C09"), (Microscopy.Site, "11"), (Microscopy.Channel, channel), (Microscopy.ZIndex, "1"), (Microscopy.Timepoint, "1"))), ref=SourcePixelRef("disk", f"/source/ch{channel}.tiff"), source_alias=binding.alias, ) @@ -1352,9 +1335,7 @@ def test_source_binding_members_canonicalize_declared_physical_spellings(aliases selector=SourceSelector(metadata=(MetadataSelector("channel", channel),)), component_identity=(ComponentSelector(Microscopy.Channel, channel),), ) - for alias, channel in zip( - aliases, map(str, range(1, len(aliases) + 1)), strict=True - ) + for alias, channel in zip(aliases, map(str, range(1, len(aliases) + 1)), strict=True) ) context = SourcePatternResolutionContext.from_sources( parser=SourceSchemaFilenameParser(), source_paths_by_virtual_path=paths @@ -1370,23 +1351,19 @@ def test_source_binding_members_canonicalize_declared_physical_spellings(aliases for binding, virtual_path in zip(bindings[1:], virtual_paths[1:], strict=True): assert matched_set.members_for_binding( binding, - anchor_provenance=SourceImageProvenance( - source_path=paths[virtual_paths[0]] - ), + anchor_provenance=SourceImageProvenance(source_path=paths[virtual_paths[0]]), source_universe=tuple(paths.values()), ) == (paths[virtual_path],) def test_declared_positions_keep_store_planes_and_same_basename_sources_distinct(): positions = ( - "A01_s001_w1_z001_t001.tif", - "A01_s001_w1_z002_t001.tif", + "A01_s001_w1_z001_t001.tif", "A01_s001_w1_z002_t001.tif", "A02_s001_w1_z001_t001.tif", ) paths = dict(zip(positions, ("/a/raw.tif", "/a/raw.tif", "/b/raw.tif"))) context = SourceIdentityResolutionContext.from_sources( - parser=SourceSchemaFilenameParser(), - source_paths_by_virtual_path=paths, + parser=SourceSchemaFilenameParser(), source_paths_by_virtual_path=paths, ) candidates = (*positions, *paths.values()) assert context.virtual_paths_for_sources( @@ -1394,16 +1371,14 @@ def test_declared_positions_keep_store_planes_and_same_basename_sources_distinct ) == (positions[:2], (positions[2],), ()) assert context.declared_positions_for_candidates(candidates) == positions assert context.matching_candidates_for_source_identities( - ( - SourceImageIdentity("/a/raw.tif"), - SourceImageIdentity("/a/raw.tif", {"z_index": "2"}), - SourceImageIdentity("/b/raw.tif"), - SourceImageIdentity("/elsewhere/raw.tif"), - ), + (SourceImageIdentity("/a/raw.tif"), + SourceImageIdentity("/a/raw.tif", {"z_index": "2"}), + SourceImageIdentity("/b/raw.tif"), SourceImageIdentity("/elsewhere/raw.tif")), candidates, ) == (positions[:2], (positions[1],), (positions[2],), ()) + def test_source_binding_members_collapse_lookup_aliases_of_same_projection(): relative = tuple(f"A01_s001_w1_z{z:03d}_t001.tif" for z in (0, 1)) full = tuple(f"/workspace/{path}" for path in relative) @@ -1411,17 +1386,8 @@ def test_source_binding_members_collapse_lookup_aliases_of_same_projection(): binding = NamedSourceBinding(alias="OriginalPhysical") projections = tuple( SourcePlaneProjection( - address=OpenHCSPlaneAddress( - ( - (Microscopy.Well, "A01"), - (Microscopy.Site, "1"), - (Microscopy.Channel, "1"), - (Microscopy.ZIndex, z), - (Microscopy.Timepoint, "1"), - ) - ), - ref=SourcePixelRef("disk", source_path), - source_alias=binding.alias, + address=OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, "1"), (Microscopy.Channel, "1"), (Microscopy.ZIndex, z), (Microscopy.Timepoint, "1"))), + ref=SourcePixelRef("disk", source_path), source_alias=binding.alias, ) for z in (0, 1) ) @@ -1431,20 +1397,15 @@ def test_source_binding_members_collapse_lookup_aliases_of_same_projection(): for spelling in (path, loadable) } workspace = VirtualWorkspaceSourceProjection( - source_refs_by_virtual_path={ - path: projection.ref for path, projection in typed.items() - }, + source_refs_by_virtual_path={path: projection.ref for path, projection in typed.items()}, source_metadata_by_path={full[0]: {"channel": "9"}}, source_projections_by_virtual_path=typed, ) context = SourcePatternResolutionContext.from_projection( - parser=SourceSchemaFilenameParser(), - projection=workspace, + parser=SourceSchemaFilenameParser(), projection=workspace, ) matched = SourceBindingMatchedImageSet.from_plan( - bindings=(binding,), - match_plan=None, - source_context=context, + bindings=(binding,), match_plan=None, source_context=context, identity_policy=SourceImageSetIdentityPolicy(), ) candidates = (*typed, source_path) @@ -1454,18 +1415,12 @@ def test_source_binding_members_collapse_lookup_aliases_of_same_projection(): source_universe=candidates, ) == (relative[0],) assert matched.matching_candidates_for_source_identities( - ( - SourceImageIdentity(source_path), - SourceImageIdentity(full[0]), - SourceImageIdentity(source_path, {"channel": "9"}), - ), - candidates, + (SourceImageIdentity(source_path), SourceImageIdentity(full[0]), + SourceImageIdentity(source_path, {"channel": "9"})), candidates, ) == (relative, (relative[0],), (relative[0],)) # A later declaration replacement is a new position even with equal fields. typed[full[0]] = SourcePlaneProjection( - address=projections[0].address, - ref=projections[0].ref, - source_alias=binding.alias, + address=projections[0].address, ref=projections[0].ref, source_alias=binding.alias, ) with pytest.raises(ValueError, match="one exact declared source-set position"): matched.members_for_binding( @@ -1474,7 +1429,6 @@ def test_source_binding_members_collapse_lookup_aliases_of_same_projection(): source_universe=candidates, ) - def test_source_binding_exact_position_projection_inherits_owning_context(): """Surface policy guard: declaration resolution is not reimplemented in leaves.""" assert ( @@ -1662,15 +1616,7 @@ def test_virtual_workspace_source_matching_uses_declared_filter_identity(): def test_virtual_workspace_source_matching_requires_exact_projection_binding(): - address = OpenHCSPlaneAddress( - ( - (Microscopy.Well, "A01"), - (Microscopy.Site, "1"), - (Microscopy.Channel, "1"), - (Microscopy.ZIndex, "1"), - (Microscopy.Timepoint, "1"), - ) - ) + address = OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, "1"), (Microscopy.Channel, "1"), (Microscopy.ZIndex, "1"), (Microscopy.Timepoint, "1"))) original_path = "A01_s001_w1_z001_t001.tif" illumination_path = f"_source/Illumination/{original_path}" original = SourcePlaneProjection( diff --git a/tests/unit/test_source_bound_matched_image_set_anchors.py b/tests/unit/test_source_bound_matched_image_set_anchors.py index 7db4fb9cc6..0949a7c74e 100644 --- a/tests/unit/test_source_bound_matched_image_set_anchors.py +++ b/tests/unit/test_source_bound_matched_image_set_anchors.py @@ -39,8 +39,7 @@ ) from openhcs.core.source_projection import OpenHCSPlaneAddress, SourcePlaneProjection from openhcs.core.source_workspace_projection import ( - VirtualWorkspaceSourceProjection, - WorkspaceSourceProjections, + VirtualWorkspaceSourceProjection, WorkspaceSourceProjections, ) from openhcs.core.step_dependencies import StepInputDependency from openhcs.core.steps.function_execution import ( @@ -77,8 +76,7 @@ def _anchor_executor( ) executor.context.runtime_source_workspace_projections = ( WorkspaceSourceProjections.from_context( - executor.context, - cache=source_workspace_projection_cache, + executor.context, cache=source_workspace_projection_cache, ) ) if output_manifest is not None: @@ -507,15 +505,7 @@ def identify_bodies(image): aliases = ("Hoechst", "SMI312") projections = { path: SourcePlaneProjection( - address=OpenHCSPlaneAddress( - ( - (Microscopy.Well, "A01"), - (Microscopy.Site, "1"), - (Microscopy.Channel, channel), - (Microscopy.ZIndex, "1"), - (Microscopy.Timepoint, "1"), - ) - ), + address=OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, "1"), (Microscopy.Channel, channel), (Microscopy.ZIndex, "1"), (Microscopy.Timepoint, "1"))), ref=SourcePixelRef("disk", f"/source/{path}"), source_alias=alias, ) @@ -627,15 +617,7 @@ def align_source_set(image): aliases = ("OrigStain1", "OrigStain2") * 2 projections = { path: SourcePlaneProjection( - address=OpenHCSPlaneAddress( - ( - (Microscopy.Well, "A01"), - (Microscopy.Site, str(site)), - (Microscopy.Channel, str(channel)), - (Microscopy.ZIndex, "1"), - (Microscopy.Timepoint, "1"), - ) - ), + address=OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, str(site)), (Microscopy.Channel, str(channel)), (Microscopy.ZIndex, "1"), (Microscopy.Timepoint, "1"))), ref=SourcePixelRef("disk", f"/source/{path}"), source_alias=alias, ) @@ -758,15 +740,7 @@ def align_source_set(image): ) projections = { path: SourcePlaneProjection( - address=OpenHCSPlaneAddress( - ( - (Microscopy.Well, "A01"), - (Microscopy.Site, str(site)), - (Microscopy.Channel, str(channel)), - (Microscopy.ZIndex, "1"), - (Microscopy.Timepoint, "1"), - ) - ), + address=OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, str(site)), (Microscopy.Channel, str(channel)), (Microscopy.ZIndex, "1"), (Microscopy.Timepoint, "1"))), ref=SourcePixelRef("disk", f"/source/{path}"), source_alias=f"OrigStain{channel}", ) diff --git a/tests/unit/test_source_provenance_constructor_defaults.py b/tests/unit/test_source_provenance_constructor_defaults.py index c5765616f7..1436a629b4 100644 --- a/tests/unit/test_source_provenance_constructor_defaults.py +++ b/tests/unit/test_source_provenance_constructor_defaults.py @@ -51,10 +51,7 @@ def _carriers(): subject=MeasurementSubject(MeasurementScope.SAMPLE, "DNA"), ), partial( - SpatialGraph, - name="Graph", - nodes=(), - edges=(), + SpatialGraph, name="Graph", nodes=(), edges=(), coordinate_spacing=SourceVoxelSpacing((1.0, 1.0)), ), partial( diff --git a/tests/unit/test_source_spatial_domain.py b/tests/unit/test_source_spatial_domain.py index 74f3c01a09..5936690655 100644 --- a/tests/unit/test_source_spatial_domain.py +++ b/tests/unit/test_source_spatial_domain.py @@ -133,11 +133,8 @@ def test_xy_alignment_preserves_intrinsic_volume_owner(as_labels): ), ).payload_with(pixels, mask) value = ( - SourceImageObjectLabelBuildRequest(image=source, labels=pixels).label_set( - name="Objects" - ) - if as_labels - else source + SourceImageObjectLabelBuildRequest(image=source, labels=pixels).label_set(name="Objects") + if as_labels else source ) target = ImagePayloadMetadata( source_spatial_domain=SourceSpatialDomain( diff --git a/tests/unit/test_source_tile_geometry.py b/tests/unit/test_source_tile_geometry.py index c914bbde2a..03ef0c1669 100644 --- a/tests/unit/test_source_tile_geometry.py +++ b/tests/unit/test_source_tile_geometry.py @@ -425,16 +425,9 @@ def for_source_plane(self, plane_index): return super().for_source_plane(plane_index) geometries = tuple( - SourceTileGeometry( - column * 6 - 1.25, - row * 4 + 0.5, - row=row, - column=column, - width_pixels=6, - height_pixels=4, - ) - for row in range(3) - for column in range(3) + SourceTileGeometry(column * 6 - 1.25, row * 4 + 0.5, + row=row, column=column, width_pixels=6, height_pixels=4) + for row in range(3) for column in range(3) ) function = OpenHCSRegistry.metadata_for_declared_callable( acquisition_tile_positions @@ -443,10 +436,8 @@ def for_source_plane(self, plane_index): raw_assemble = CallableContract.from_callable( assemble_stack_cpu ).resolve_raw_runtime_callable() - for channel, order in ( - ("1", (8, 0, 4, 3, 1, 7, 2, 6, 5)), - ("2", (5, 2, 6, 1, 3, 4, 7, 0, 8)), - ): + for channel, order in (("1", (8, 0, 4, 3, 1, 7, 2, 6, 5)), + ("2", (5, 2, 6, 1, 3, 4, 7, 0, 8))): original = _metadata(geometries, order, channel) metadata = ProjectionAuditMetadata( source_provenance=original.source_provenance, @@ -454,20 +445,16 @@ def for_source_plane(self, plane_index): source_voxel_spacing=SourceVoxelSpacing((0.25, 0.5)), ) pixels = np.stack([np.full((4, 6), site + 1, np.uint16) for site in order]) - result, positions = function( - ImageMetadataPayload(pixels, metadata), source_metadata=metadata - ) + result, positions = function(ImageMetadataPayload(pixels, metadata), + source_metadata=metadata) np.testing.assert_array_equal(result, pixels) - assert positions == [ - (geometries[site].x_pixels, geometries[site].y_pixels) for site in order - ] + assert positions == [(geometries[site].x_pixels, geometries[site].y_pixels) + for site in order] assert result.metadata.source_voxel_spacing == metadata.source_voxel_spacing assert result.metadata.source_provenance == metadata.source_provenance assert calls == list(range(9)) calls.clear() - assembled.append( - raw_assemble(pixels, positions, blend_method=TileBlendMethod.NONE) - ) + assembled.append(raw_assemble(pixels, positions, blend_method=TileBlendMethod.NONE)) assert assembled[0].shape == assembled[1].shape == (13, 19) @@ -614,7 +601,9 @@ def test_synthetic_pipeline_compiles_and_executes_positions_artifact_then_paired assert any( edge.spec.name == "positions" for edge in consumer.artifact_input_edges ) - assert context.step_plans[2].variable_components == [Microscopy.Channel] + assert context.step_plans[2].variable_components == [ + Microscopy.Channel + ] result = orchestrator.execute_compiled_bundle( execution_bundle=compilation, progress_queue=progress, @@ -673,7 +662,9 @@ def test_synthetic_pipeline_compiles_and_executes_positions_artifact_then_paired (0.25, 0.5) ) assert metadata.plane_axis is None - replay_config = PreparedWorkspaceSource.pipeline_config_for_source(config) + replay_config = PreparedWorkspaceSource.pipeline_config_for_source( + config + ) reopened = PipelineOrchestrator( output_dataset, pipeline_config=replay_config ).initialize() diff --git a/tests/unit/test_stream_component_semantics.py b/tests/unit/test_stream_component_semantics.py index 76d8b53712..3def580922 100644 --- a/tests/unit/test_stream_component_semantics.py +++ b/tests/unit/test_stream_component_semantics.py @@ -42,9 +42,7 @@ def test_streaming_domain_spans_worker_lanes_without_mirroring_worker_identity() context.bind_execution_runtime(lane) assert context.execution_runtime is lane assert lane.worker_assignments is assignments.worker_assignments - assert lane.owned_partition_values == tuple( - assignments.worker_assignments[slot] - ) + assert lane.owned_partition_values == tuple(assignments.worker_assignments[slot]) provider = StreamExecutionAxisDomainProvider.build_for_component( context=context, component=AxisFamily.active().partition_axis().name, @@ -52,7 +50,9 @@ def test_streaming_domain_spans_worker_lanes_without_mirroring_worker_identity() ) domains.append(provider.domain_metadata_items()) assert domains[0] == domains[1] - assert {item[AxisFamily.active().partition_axis().name] for item in domains[0]} == { + assert { + item[AxisFamily.active().partition_axis().name] for item in domains[0] + } == { "R02C05", "R04C04", "R04C02", @@ -215,21 +215,17 @@ def test_retained_stream_domain_includes_exact_planes_not_scalar_address(compone ) scalar = SourceImageIdentity(component_metadata={"well": "A01"}) observations = StreamSourceComponentMetadataItems.from_image_metadata( - metadata, - fallback_source_identity=scalar, + metadata, fallback_source_identity=scalar, ) assert observations.complete_component_order(("well", component, "timepoint")) == ( - "well", - component, + "well", component, ) assert observations.domain_metadata_items(("well", component)) == ( - {"well": "A01", component: 3}, - {"well": "A01", component: 1}, + {"well": "A01", component: 3}, {"well": "A01", component: 1}, ) fields = StreamImagePayloadMetadataProjector.item_fields(metadata, (component,)) route_metadata = StreamViewerComponentMetadataProjector.for_item_fields( - ("well", component), - fields, + ("well", component), fields, ).indexed_source_metadata((scalar.component_metadata,)) # Two plane observations still describe ONE streamed image/route address. assert route_metadata.metadata_by_index == ({"well": "A01"},) @@ -237,12 +233,10 @@ def test_retained_stream_domain_includes_exact_planes_not_scalar_address(compone def test_collapsed_contributors_do_not_restore_a_stream_pixel_axis(): metadata = _plane_metadata( - RuntimePlaneAxis.RUNTIME_SLICE, - ({"site": "3"}, {"site": "1"}), + RuntimePlaneAxis.RUNTIME_SLICE, ({"site": "3"}, {"site": "1"}), ).collapse_leading_plane_axis() observations = StreamSourceComponentMetadataItems.from_image_metadata( - metadata, - fallback_source_identity=SourceImageIdentity( + metadata, fallback_source_identity=SourceImageIdentity( component_metadata={"well": "A01"}, ), ) diff --git a/tests/unit/test_streaming_service.py b/tests/unit/test_streaming_service.py index ee33547c0b..4acc8a7537 100644 --- a/tests/unit/test_streaming_service.py +++ b/tests/unit/test_streaming_service.py @@ -700,10 +700,7 @@ def source_path( "A49", component=None, value=None, - fixed_component_values=( - (Microscopy.ZIndex, "1"), - (Microscopy.Timepoint, "1"), - ), + fixed_component_values=((Microscopy.ZIndex, "1"), (Microscopy.Timepoint, "1")), ), ) workspace_projection = VirtualWorkspaceSourceProjection( @@ -1273,13 +1270,11 @@ def lifecycle_manager(monkeypatch): ManagedViewerLifecycleMixin, "wait_for_ready", lambda self, timeout: True ) monkeypatch.setattr( - ManagedViewerLifecycleMixin, - "force_stop", + ManagedViewerLifecycleMixin, "force_stop", lambda self: self.lifecycle_state.mark_stopped(), ) monkeypatch.setattr( - ManagedViewerLifecycleMixin, - "start", + ManagedViewerLifecycleMixin, "start", lambda self: (_ for _ in ()).throw(AssertionError("must not restart")), ) manager = ViewerStateManager.get_instance() @@ -1288,8 +1283,7 @@ def lifecycle_manager(monkeypatch): def test_streaming_viewer_lifecycle_attaches_existing_viewer_without_restart( - monkeypatch, - lifecycle_manager, + monkeypatch, lifecycle_manager, ) -> None: monkeypatch.setattr( NapariStreamVisualizer, @@ -1402,25 +1396,19 @@ def fail_after_factory(*, factory, **_kwargs): def test_streaming_viewer_lifecycle_admits_new_launch_inside_managed_acquisition( monkeypatch, lifecycle_manager, config_type, owns_process, requested_host, reusable ) -> None: - active_config = config_type( - enabled=True, port=5563, persistent=True, listen_host="*" - ) + active_config = config_type(enabled=True, port=5563, persistent=True, listen_host="*") existing_viewer = active_config.create_visualizer(FakeFileManager()) existing_viewer.lifecycle_state.mark_connected_external() monkeypatch.setattr( - ManagedViewerLifecycleMixin, - "owned_viewer_process_is_alive", + ManagedViewerLifecycleMixin, "owned_viewer_process_is_alive", lambda self: owns_process, ) monkeypatch.setattr( - ViewerControlMessageRequest, - "send", - lambda self: ViewerControlResponse( - { - "status": "success", - "process_launch": active_config.viewer_process_launch_config().to_wire_mapping(), - } - ), + ViewerControlMessageRequest, "send", + lambda self: ViewerControlResponse({ + "status": "success", + "process_launch": active_config.viewer_process_launch_config().to_wire_mapping(), + }), ) lifecycle_manager.get_or_create_viewer( active_config.viewer_family.wire_value, 5563, lambda: existing_viewer @@ -1437,24 +1425,14 @@ def test_streaming_viewer_lifecycle_admits_new_launch_inside_managed_acquisition enabled=True, port=5563, persistent=True, listen_host=requested_host ) if reusable: - assert ( - StreamingViewerLifecycle.get_or_create_visualizer( - filemanager=FakeFileManager(), - config=requested, - fresh=False, - ) - is existing_viewer - ) + assert StreamingViewerLifecycle.get_or_create_visualizer( + filemanager=FakeFileManager(), config=requested, fresh=False, + ) is existing_viewer else: - with pytest.raises( - RuntimeError, match="does not match the requested process launch" - ): + with pytest.raises(RuntimeError, match="does not match the requested process launch"): StreamingViewerLifecycle.get_or_create_visualizer( - filemanager=FakeFileManager(), - config=requested, - fresh=False, + filemanager=FakeFileManager(), config=requested, fresh=False, ) - assert ( - lifecycle_manager.get_viewer(active_config.viewer_family.wire_value, 5563) - is existing_viewer - ) + assert lifecycle_manager.get_viewer( + active_config.viewer_family.wire_value, 5563 + ) is existing_viewer diff --git a/tests/unit/test_u1_session_guards.py b/tests/unit/test_u1_session_guards.py index e6418f545a..cdcbdbdf3f 100644 --- a/tests/unit/test_u1_session_guards.py +++ b/tests/unit/test_u1_session_guards.py @@ -56,11 +56,9 @@ def test_no_widget_holds_session_state() -> None: targets = ( node.targets if isinstance(node, ast.Assign) - else ( - [node.target] - if isinstance(node, (ast.AnnAssign, ast.AugAssign)) - else [] - ) + else [node.target] + if isinstance(node, (ast.AnnAssign, ast.AugAssign)) + else [] ) for target in targets: if ( @@ -69,9 +67,7 @@ def test_no_widget_holds_session_state() -> None: and target.value.id == "self" and target.attr in SESSION_STATE_ATTRIBUTES ): - offenders.append( - f"{path.relative_to(PACKAGE)}:{node.lineno} self.{target.attr}" - ) + offenders.append(f"{path.relative_to(PACKAGE)}:{node.lineno} self.{target.attr}") assert offenders == [] @@ -82,13 +78,13 @@ def test_session_package_imports_no_qt_and_no_gui() -> None: names = ( [alias.name for alias in node.names] if isinstance(node, ast.Import) - else [node.module or ""] if isinstance(node, ast.ImportFrom) else [] + else [node.module or ""] + if isinstance(node, ast.ImportFrom) + else [] ) for name in names: if name.split(".")[0] == "PyQt6" or name.startswith("openhcs.pyqt_gui"): - offenders.append( - f"{path.relative_to(PACKAGE)}:{node.lineno} {name}" - ) + offenders.append(f"{path.relative_to(PACKAGE)}:{node.lineno} {name}") assert offenders == [] @@ -119,9 +115,7 @@ def test_actions_are_operation_classes_not_enum_rosters() -> None: if isinstance(node, ast.Lambda) and [ argument.arg for argument in node.args.args ] in (["widget"], ["window"], ["manager"]): - offenders.append( - f"{path.relative_to(PACKAGE)}:{node.lineno} lambda" - ) + offenders.append(f"{path.relative_to(PACKAGE)}:{node.lineno} lambda") for path, tree in _modules(SESSION_PACKAGE): for node in ast.walk(tree): if isinstance(node, ast.ClassDef) and any( @@ -135,19 +129,12 @@ def test_actions_are_operation_classes_not_enum_rosters() -> None: def test_every_operation_declares_itself_and_headless_ones_are_tools() -> None: from openhcs.agent.capabilities import get_capability_registry - tools = { - capability.name: capability - for capability in get_capability_registry().capabilities - } + tools = {capability.name: capability for capability in get_capability_registry().capabilities} for operation in SessionOperation.all(): - assert ( - operation.label and operation.tooltip and operation.description - ), operation + assert operation.label and operation.tooltip and operation.description, operation assert isinstance(operation.request, type), operation tool_name = f"openhcs_{operation.operation_id}" - assert (tool_name in tools) is issubclass( - operation, HeadlessOperation - ), operation + assert (tool_name in tools) is issubclass(operation, HeadlessOperation), operation if issubclass(operation, RendererOperation): assert not issubclass(operation, HeadlessOperation), operation if tool_name in tools: @@ -160,7 +147,7 @@ def test_pipeline_formats_are_importers_not_suffix_branches() -> None: for root in (PACKAGE / "pyqt_gui", SESSION_PACKAGE): for path in sorted(root.rglob("*.py")): text = path.read_text(encoding="utf-8") - for marker in ('== ".cppipe"', '!= ".cppipe"', "#openhcs-cppipe="): + for marker in ('== ".cppipe"', '!= ".cppipe"', '#openhcs-cppipe='): if marker in text: offenders.append(f"{path.relative_to(PACKAGE)} {marker}") assert offenders == [] diff --git a/tests/unit/test_unified_registry_runtime_payloads.py b/tests/unit/test_unified_registry_runtime_payloads.py index 2869b9a344..c403f5ca63 100644 --- a/tests/unit/test_unified_registry_runtime_payloads.py +++ b/tests/unit/test_unified_registry_runtime_payloads.py @@ -233,9 +233,7 @@ def test_native_flexible_callable_receives_declared_array_abi_and_keeps_slice_co ) -> None: stack = np.arange(40, dtype=np.float32).reshape(2, 4, 5) mask = np.ones_like(stack, dtype=bool) - payload = ImagePayloadMetadata( - plane_axis=RuntimePlaneAxis.RUNTIME_SLICE - ).payload_with(stack, mask) + payload = ImagePayloadMetadata(plane_axis=RuntimePlaneAxis.RUNTIME_SLICE).payload_with(stack, mask) seen_shapes = [] @numpy @@ -245,13 +243,10 @@ def native_image_operation(image: np.ndarray) -> np.ndarray: return image + 1 invocation = compile_function_pattern( - (native_image_operation, {"slice_by_slice": slice_by_slice}), - {}, - {}, + (native_image_operation, {"slice_by_slice": slice_by_slice}), {}, {}, ).default_group.invocations[0] result = invocation.runtime_callable( - invocation.main_flow_call_argument(payload), - **dict(invocation.runtime_kwargs), + invocation.main_flow_call_argument(payload), **dict(invocation.runtime_kwargs), ) assert seen_shapes == ([(4, 5), (4, 5)] if slice_by_slice else [(2, 4, 5)]) np.testing.assert_array_equal(result, stack + 1) @@ -259,17 +254,13 @@ def native_image_operation(image: np.ndarray) -> np.ndarray: def test_native_flexible_callable_preserves_explicitly_declared_carrier_abi() -> None: - payload = ImagePayloadMetadata().payload_with( - np.ones((2, 4, 5)), np.ones((2, 4, 5), dtype=bool) - ) + payload = ImagePayloadMetadata().payload_with(np.ones((2, 4, 5)), np.ones((2, 4, 5), dtype=bool)) @numpy def carrier_operation(image: RuntimeArrayPayload) -> RuntimeArrayPayload: return image - invocation = compile_function_pattern( - carrier_operation, {}, {} - ).default_group.invocations[0] + invocation = compile_function_pattern(carrier_operation, {}, {}).default_group.invocations[0] assert invocation.main_flow_call_argument(payload) is payload @@ -296,8 +287,7 @@ def add_one(image: np.ndarray) -> np.ndarray: return image + 1 wrapped = MinimalRegistry("minimal").apply_contract_wrapper( - add_one, - FlexibleContract, + add_one, FlexibleContract, ) contract = CallableContract.from_callable(wrapped) call_argument = contract.main_flow_call_argument(payload) @@ -307,8 +297,7 @@ def add_one(image: np.ndarray) -> np.ndarray: np.testing.assert_array_equal(result.data, stack + 1) assert result.metadata.plane_axis is RuntimePlaneAxis.RUNTIME_SLICE assert result.metadata.source_image_provenance_planes.paths == ( - "z0.tif", - "z1.tif", + "z0.tif", "z1.tif", ) @@ -338,9 +327,7 @@ def apply_mask( seen_shapes.append(mask_data.shape) true_y, true_x = np.argwhere(mask_data)[0] seen_true_indices.append((int(true_y), int(true_x))) - return image.with_pixels( - np.where(mask_data, image_data + 100, image_data), - ) + return image.with_pixels(np.where(mask_data, image_data + 100, image_data),) apply_mask.output_memory_type = MEMORY_TYPE_NUMPY @@ -469,9 +456,7 @@ def test_with_pixels_rejects_implicit_channel_mask_collapse() -> None: ) with pytest.raises(ValueError, match="does not match declared image mask domain"): - source.with_pixels( - np.ones((4, 5), dtype=np.float32), - ) + source.with_pixels(np.ones((4, 5), dtype=np.float32),) def test_runtime_callable_invocation_can_call_raw_signature_filtered_callable() -> None: @@ -481,7 +466,9 @@ def test_runtime_callable_invocation_can_call_raw_signature_filtered_callable() def raw(image: RuntimeArrayPayload, *, scale: int) -> RuntimeArrayPayload: assert isinstance(image, RuntimeArrayPayload) - return image.metadata.payload_with(image.data * scale, image.mask) + return image.metadata.payload_with( + image.data * scale, image.mask + ) @wraps(raw) def decorated(image: Any, **kwargs: Any) -> np.ndarray: @@ -497,7 +484,9 @@ def decorated(image: Any, **kwargs: Any) -> np.ndarray: assert isinstance(result, RuntimeArrayPayload) np.testing.assert_array_equal(result.data, np.full((4, 5), 3.0)) - np.testing.assert_array_equal(result.mask, np.ones((4, 5), dtype=bool)) + np.testing.assert_array_equal( + result.mask, np.ones((4, 5), dtype=bool) + ) def test_raw_runtime_callable_preserves_request_binding_semantics() -> None: diff --git a/tests/unit/test_viewer_receipt_replay.py b/tests/unit/test_viewer_receipt_replay.py index d26a67a5e8..4b0efd499b 100644 --- a/tests/unit/test_viewer_receipt_replay.py +++ b/tests/unit/test_viewer_receipt_replay.py @@ -49,18 +49,16 @@ from openhcs.core.axes import ColourAxis + def _plate_context(plate_path, filemanager) -> PlateInspectionContext: """A plate whose metadata owner anchors workspace addresses at the plate.""" return PlateInspectionContext( plate_path=plate_path, filemanager=filemanager, - handler=SimpleNamespace( - metadata_handler=SimpleNamespace(source_workspace_root=Path) - ), + handler=SimpleNamespace(metadata_handler=SimpleNamespace(source_workspace_root=Path)), parser=None, ) - def receipt_state(path, *, summary=None): fixed = { component.name: 1 @@ -68,22 +66,16 @@ def receipt_state(path, *, summary=None): if component is not Microscopy.Channel } fixed[Microscopy.Well.name] = "1" - summary = ( - summary - if summary is not None - else ViewerPayloadSummary.from_wire_mapping( - dict( - path=str(path), - data_type="image", - components=fixed, - aggregate_component_values={"channel": [2, 1]}, - shape=[2, 8, 9], - source_spatial_shape_yx=[8, 9], - spatial_origin_yx=[0, 0], - dtype="int32", - ) - ) - ) + summary = summary if summary is not None else ViewerPayloadSummary.from_wire_mapping(dict( + path=str(path), + data_type="image", + components=fixed, + aggregate_component_values={"channel": [2, 1]}, + shape=[2, 8, 9], + source_spatial_shape_yx=[8, 9], + spatial_origin_yx=[0, 0], + dtype="int32", + )) layer = ViewerWindowLayerState( route_key="native_historical_labels", title="Labels", @@ -129,7 +121,9 @@ def test_receipt_codec_and_exact_path_admission(tmp_path): ) -@pytest.mark.parametrize("change", ["transform", "truncated", "window", "producer"]) +@pytest.mark.parametrize( + "change", ["transform", "truncated", "window", "producer"] +) def test_receipt_incomplete_or_conflicting_facts_refuse(tmp_path, change): state = receipt_state(tmp_path / "aggregate.labels.tif") layer = state.layers[0] @@ -148,16 +142,10 @@ def test_receipt_incomplete_or_conflicting_facts_refuse(tmp_path, change): ) -@pytest.mark.parametrize( - "field,value", - [ - ("spatial_origin_yx", [False, 0]), - ("shape", [True, 8, 9]), - ], -) -def test_receipt_boolean_spatial_facts_refuse_at_original_typed_ingress( - tmp_path, field, value -): +@pytest.mark.parametrize("field,value", [ + ("spatial_origin_yx", [False, 0]), ("shape", [True, 8, 9]), +]) +def test_receipt_boolean_spatial_facts_refuse_at_original_typed_ingress(tmp_path, field, value): wire = to_jsonable(receipt_state(tmp_path / "aggregate.labels.tif")) wire["layers"][0]["payload_summaries"][0][field] = value with pytest.raises(TypeError): @@ -186,34 +174,28 @@ def test_observed_and_declared_wire_domains_have_distinct_order_contracts(): ) observed = ViewerComponentValueDomainPayload.from_wire_mapping( - {"channel": ["2", 1, 2]}, - context="observed", + {"channel": ["2", 1, 2]}, context="observed", declared_axes=STREAM_AXES, ) assert observed.to_wire_mapping() == {"channel": [1, 2]} declared = ViewerComponentValueDomainPayload.from_ordered_wire_mapping( - {"channel": ["2", 1]}, - context="declared", + {"channel": ["2", 1]}, context="declared", declared_axes=STREAM_AXES, ) assert declared.to_wire_mapping() == {"channel": [2, 1]} with pytest.raises(ValueError, match="unique after normalization"): ViewerComponentValueDomainPayload.from_ordered_wire_mapping( - {"channel": ["01", 1]}, - context="malformed declared", + {"channel": ["01", 1]}, context="malformed declared", declared_axes=STREAM_AXES, ) @pytest.mark.parametrize("historical_dtype", [None, "int32"]) -@pytest.mark.parametrize( - "spacing", - ( - SourceVoxelSpacing(), - SourceVoxelSpacing((2.0, 0.65, 0.65)), - SourceVoxelSpacing((2.0, 1.0, 0.75), SourceVoxelSpacingUnit.RELATIVE), - ), -) +@pytest.mark.parametrize("spacing", ( + SourceVoxelSpacing(), + SourceVoxelSpacing((2.0, 0.65, 0.65)), + SourceVoxelSpacing((2.0, 1.0, 0.75), SourceVoxelSpacingUnit.RELATIVE), +)) def test_native_persisted_aggregate_source_projection_preserves_order( tmp_path, historical_dtype, spacing ): @@ -224,14 +206,11 @@ def test_native_persisted_aggregate_source_projection_preserves_order( from openhcs.core.runtime_image_values import ImagePayloadMetadata from openhcs.core.source_spatial_domain import SourceSpatialDomain from openhcs.runtime.napari_streaming_handlers import ( - NapariAggregateAxisBindingBuilder, - NapariStreamLayerAddress, - NapariStreamLayerItem, + NapariAggregateAxisBindingBuilder, NapariStreamLayerAddress, NapariStreamLayerItem, ) from openhcs.runtime.napari_viewer_server import NapariViewerStateProjection from openhcs.runtime.viewer_component_system import ( - ViewerComponentAxisSemanticsFactory, - ViewerComponentValueDomainPayload, + ViewerComponentAxisSemanticsFactory, ViewerComponentValueDomainPayload, ViewerMappingDisplayConfigInput, ) @@ -241,43 +220,32 @@ def test_native_persisted_aggregate_source_projection_preserves_order( producer=fixture.layers[0].producer_identities[0], address=NapariStreamLayerAddress( components=fixture.layers[0].payload_summaries[0].components, - path=str(path), - stream_layer_data_type=StreamingDataType.IMAGE, + path=str(path), stream_layer_data_type=StreamingDataType.IMAGE, ), image_metadata=ImagePayloadMetadata( source_voxel_spacing=spacing, source_spatial_domain=SourceSpatialDomain( - origin_yx=(0, 0), - source_shape_yx=(8, 9), + origin_yx=(0, 0), source_shape_yx=(8, 9), ), ), plane_component_domain=ViewerComponentValueDomainPayload.from_ordered_wire_mapping( - {"channel": [2, 1]}, - context="synthetic native receipt", + {"channel": [2, 1]}, context="synthetic native receipt", declared_axes=STREAM_AXES, ), ) native_semantics = ViewerComponentAxisSemanticsFactory.from_display_config( ViewerMappingDisplayConfigInput( - { - "component_modes": {"channel": "stack"}, - "component_order": ["channel"], - "declared_axes": STREAM_AXES.to_wire(), - } + {"component_modes": {"channel": "stack"}, "component_order": ["channel"], "declared_axes": STREAM_AXES.to_wire()} ), ViewerComponentValueDomainPayload.from_wire_mapping( - {"channel": [1, 2]}, - context="synthetic observed receipt domain", + {"channel": [1, 2]}, context="synthetic observed receipt domain", declared_axes=STREAM_AXES, ), ) summary = NapariViewerStateProjection.payload_summary( - native_item, - native_item.address.components, - expected, + native_item, native_item.address.components, expected, aggregate_axis_bindings=NapariAggregateAxisBindingBuilder.bindings( - [native_item], - native_semantics, + [native_item], native_semantics, ), ) assert summary.full_image_plane_count == 2 @@ -304,16 +272,14 @@ def test_native_persisted_aggregate_source_projection_preserves_order( state = replace( state, viewer_ndim=len(presentation.axis_labels), - layers=( - replace( - state.layers[0], - axis_labels=presentation.axis_labels, - native_transform=ViewerNativeLayerTransform( - scale=coordinate_kwargs["scale"], - translate=coordinate_kwargs["translate"], - ), + layers=(replace( + state.layers[0], + axis_labels=presentation.axis_labels, + native_transform=ViewerNativeLayerTransform( + scale=coordinate_kwargs["scale"], + translate=coordinate_kwargs["translate"], ), - ), + ),), ) assert summary.voxel_spacing == spacing data = json.dumps(to_jsonable(state), sort_keys=True).encode() @@ -369,29 +335,19 @@ def test_native_persisted_aggregate_source_projection_preserves_order( assert metadata.source_dtype == "int64" from openhcs.agent.capabilities import GetViewerWindowStateCapability from openhcs.mcp.dev_client_core import ( - McpDevServerIdentity, - McpDevToolBatchResponse, - McpDevToolResult, + McpDevServerIdentity, McpDevToolBatchResponse, McpDevToolResult, ) from openhcs.mcp.dev_client_rendering import McpDevOutputRenderer - public_wire = to_jsonable( - McpDevToolBatchResponse( - server=McpDevServerIdentity(command="source-only", module="openhcs.mcp"), - results=( - McpDevToolResult(GetViewerWindowStateCapability.name, False, (state,)), - ), - ) - ) + public_wire = to_jsonable(McpDevToolBatchResponse( + server=McpDevServerIdentity(command="source-only", module="openhcs.mcp"), + results=(McpDevToolResult(GetViewerWindowStateCapability.name, False, (state,)),), + )) decoded = McpDevToolBatchResponse.for_rendering(public_wire) received = decoded.payload_for(GetViewerWindowStateCapability) assert isinstance(received.layers[0].payload_summaries[0], ViewerPayloadSummary) - assert received.layers[0].payload_summaries[0].require_plane_components() == { - "channel": (2, 1) - } - renderer = McpDevOutputRenderer.for_output_contract( - GetViewerWindowStateCapability.output_contract - ) + assert received.layers[0].payload_summaries[0].require_plane_components() == {"channel": (2, 1)} + renderer = McpDevOutputRenderer.for_output_contract(GetViewerWindowStateCapability.output_contract) compact = renderer.render(decoded) assert '"aggregate_component_values": {"channel": [2, 1]}' in compact assert '"source_spatial_shape_yx": [8, 9]' in compact @@ -400,8 +356,7 @@ def test_native_persisted_aggregate_source_projection_preserves_order( ) domain = ViewerComponentValueDomainPayload.from_ordered_wire_mapping( - metadata.retained_plane_component_values(), - context="native receipt domain", + metadata.retained_plane_component_values(), context="native receipt domain", declared_axes=STREAM_AXES, ) assert domain.to_wire_mapping() == {"channel": [2, 1]} @@ -433,17 +388,12 @@ def test_native_persisted_aggregate_source_projection_preserves_order( from zmqruntime.viewer_protocol import ViewerWirePayload observed = ViewerComponentValueDomainPayload.from_wire_mapping( - {"channel": [1, 2]}, - context="observed stack domain", + {"channel": [1, 2]}, context="observed stack domain", declared_axes=STREAM_AXES, ) semantics = ViewerComponentAxisSemanticsFactory.from_display_config( ViewerMappingDisplayConfigInput( - { - "component_modes": {"channel": "stack"}, - "component_order": ["channel"], - "declared_axes": STREAM_AXES.to_wire(), - } + {"component_modes": {"channel": "stack"}, "component_order": ["channel"], "declared_axes": STREAM_AXES.to_wire()} ), observed, ) @@ -509,13 +459,8 @@ def test_native_persisted_aggregate_source_projection_preserves_order( ) -@pytest.mark.parametrize( - "refusal", - ["unowned", "missing_domain", "null_domain", "empty_axis", "ambiguous_domain"], -) -def test_receipt_projection_refuses_unowned_or_invalid_declared_source( - tmp_path, refusal -): +@pytest.mark.parametrize("refusal", ["unowned", "missing_domain", "null_domain", "empty_axis", "ambiguous_domain"]) +def test_receipt_projection_refuses_unowned_or_invalid_declared_source(tmp_path, refusal): path = tmp_path / "aggregate.labels.tif" state = receipt_state(path) wire = to_jsonable(state) @@ -545,13 +490,8 @@ def test_receipt_projection_refuses_unowned_or_invalid_declared_source( ) service = PlateStreamingService(inspection, object()) context = _plate_context(tmp_path, FileManager({"disk": DiskStorageBackend()})) - source_ref = ( - None - if refusal == "unowned" - else SourcePixelRef( - backend="disk", - backend_address=str(path), - ) + source_ref = None if refusal == "unowned" else SourcePixelRef( + backend="disk", backend_address=str(path), ) expected_message = { "unowned": "inventory source reference", @@ -563,11 +503,7 @@ def test_receipt_projection_refuses_unowned_or_invalid_declared_source( with pytest.raises(ValueError, match=expected_message): service._receipt_source_projection( PlateFileStreamRequest(plate_path=str(tmp_path), source_receipt=resource), - ( - PlateFileRecord( - kind=PlateFileKind.IMAGE, key=str(path), source_ref=source_ref - ), - ), + (PlateFileRecord(kind=PlateFileKind.IMAGE, key=str(path), source_ref=source_ref),), context, ) @@ -622,15 +558,11 @@ def stream_files(self, request, *, ui_bridge_connection): assert admitted_connection.descriptor_file_path == str(tmp_path / "exact-ui.json") -@pytest.mark.parametrize( - "change", ("scale", "missing_axes", "duplicate_axes", "translation", "crop") -) +@pytest.mark.parametrize("change", ("scale", "missing_axes", "duplicate_axes", "translation", "crop")) def test_calibrated_receipt_rejects_conflicting_native_placement(tmp_path, change): spacing = SourceVoxelSpacing((2.0, 0.65, 0.65)) state = receipt_state(tmp_path / "aggregate.labels.tif") - summary = replace( - state.layers[0].payload_summaries[0], source_voxel_spacing=spacing - ) + summary = replace(state.layers[0].payload_summaries[0], source_voxel_spacing=spacing) axes = ("channel", "z_index", "timepoint", "site", "well", "y", "x") transform = ViewerNativeLayerTransform( scale=spacing.layer_coordinate_kwargs(axes)["scale"], translate=(0,) * 7 @@ -645,16 +577,9 @@ def test_calibrated_receipt_rejects_conflicting_native_placement(tmp_path, chang axes = ("z_index", "z_index", "timepoint", "site", "well", "y", "x") else: summary = replace(summary, spatial_origin_yx=(1, 0)) - state = replace( - state, - layers=( - replace( - state.layers[0], - axis_labels=axes, - payload_summaries=(summary,), - native_transform=transform, - ), - ), - ) + state = replace(state, layers=(replace( + state.layers[0], axis_labels=axes, + payload_summaries=(summary,), native_transform=transform, + ),)) with pytest.raises(ValueError): state.image_payload_binding_for(str(tmp_path / "aggregate.labels.tif")) diff --git a/tests/unit/test_viewer_result_element_selection.py b/tests/unit/test_viewer_result_element_selection.py index 1c92a04679..37bb3b8707 100644 --- a/tests/unit/test_viewer_result_element_selection.py +++ b/tests/unit/test_viewer_result_element_selection.py @@ -174,17 +174,13 @@ def _viewer_server(viewer, layer, route_key: str = "result-rois", group_binding= NapariStreamLayerItem( data=layer.data, producer=StreamProducerIdentity( - origin="manual", - output_kind="fixture", - output_key=route_key, + origin="manual", output_kind="fixture", output_key=route_key, projection_key=route_key, ), address=NapariStreamLayerAddress( - components={}, - path=f"fixture/{route_key}", + components={}, path=f"fixture/{route_key}", stream_layer_data_type=( - StreamingDataType.POINTS - if isinstance(layer, Points) + StreamingDataType.POINTS if isinstance(layer, Points) else StreamingDataType.SHAPES ), ), @@ -246,9 +242,7 @@ def test_linked_selection_projects_off_slice_members_without_changing_subject(qt paths = [np.array([[0, i, i], [0, i + 1, i + 1]], dtype=float) for i in range(2)] layer = viewer.add_shapes(paths, shape_type="path", features={"owner": [8, 8]}) linked_paths = [coordinates + [1, 0, 0] for coordinates in paths] - linked = viewer.add_shapes( - linked_paths, shape_type="path", features={"owner": [8, 8]} - ) + linked = viewer.add_shapes(linked_paths, shape_type="path", features={"owner": [8, 8]}) viewer.dims.current_step = (0, 0, 0) binding = NapariResultSelectionGroupBinding("native-test-subject", "owner") server, _, _, _ = _viewer_server(viewer, layer, group_binding=binding) @@ -260,8 +254,7 @@ def test_linked_selection_projects_off_slice_members_without_changing_subject(qt assert state.selected_data_indices == (0, 1) assert linked.selected_data == set() assert server.result_selection_controller._linked_group_members(layer, 1) == ( - (layer, (0, 1)), - (linked, (0, 1)), + (layer, (0, 1)), (linked, (0, 1)), ) assert viewer.dims.current_step == (0, 0, 0) for before, after in zip(original_data, linked.data, strict=True): @@ -275,12 +268,10 @@ def test_linked_points_use_the_same_native_slice_projection(qtbot): viewer.add_image(np.zeros((2, 8, 8), dtype=np.uint8)) layer = viewer.add_shapes( [np.array([[0, 0, 0], [0, 1, 1]], dtype=float)], - shape_type="path", - features={"owner": [8]}, + shape_type="path", features={"owner": [8]}, ) linked = viewer.add_points( - np.array([[0, 2, 2], [1, 3, 3]], dtype=float), - features={"owner": [8, 8]}, + np.array([[0, 2, 2], [1, 3, 3]], dtype=float), features={"owner": [8, 8]}, ) viewer.dims.current_step = (0, 0, 0) binding = NapariResultSelectionGroupBinding("native-test-subject", "owner") @@ -292,8 +283,7 @@ def test_linked_points_use_the_same_native_slice_projection(qtbot): assert state.selected_data_indices == (0,) assert linked.selected_data == {0} assert server.result_selection_controller._linked_group_members(layer, 0) == ( - (layer, (0,)), - (linked, (0, 1)), + (layer, (0,)), (linked, (0, 1)), ) @@ -691,10 +681,8 @@ def test_point_coordinate_owner_preserves_strict_acquisition_admission(coordinat def test_shapes_coordinate_owner_stays_strict_for_hidden_spatial_axes(displayed): with pytest.raises(ValueError, match="integral slice"): ViewerResultElementCoordinates.axis_indices( - coordinates=((1.5, 2.5, 3.5),), - axis_labels=("z_index", "y", "x"), - displayed_axis_indices=displayed, - spatial_axis_labels=("z_index", "y", "x"), + coordinates=((1.5, 2.5, 3.5),), axis_labels=("z_index", "y", "x"), + displayed_axis_indices=displayed, spatial_axis_labels=("z_index", "y", "x"), ) diff --git a/tests/unit/test_viewer_spatial_sample.py b/tests/unit/test_viewer_spatial_sample.py index e589928945..903e7d7b6c 100644 --- a/tests/unit/test_viewer_spatial_sample.py +++ b/tests/unit/test_viewer_spatial_sample.py @@ -97,9 +97,7 @@ def test_raw_array_slices_keep_trailing_dimension_contract_in_same_native_projec rgb = np.stack((gray, gray + 1000, gray + 2000), axis=-1) items = server.component_groups.items_for("source") items[0] = replace( - items[0], - data=rgb, - image_metadata=ImagePayloadMetadata(axes=PayloadAxes.colour_samples(2)), + items[0], data=rgb, image_metadata=ImagePayloadMetadata(axes=PayloadAxes.colour_samples(2)) ) projection = NapariViewerPayloadProjection( server=server, diff --git a/tests/unit/test_well_throughput_scaling.py b/tests/unit/test_well_throughput_scaling.py index d900972949..644cf7b1c8 100644 --- a/tests/unit/test_well_throughput_scaling.py +++ b/tests/unit/test_well_throughput_scaling.py @@ -605,15 +605,7 @@ def test_repeated_wells_keep_all_declared_projection_fields_coherent( ) -> None: parser = SourceSchemaFilenameParser() original = SourcePlaneProjection( - address=OpenHCSPlaneAddress( - ( - (Microscopy.Well, "A01"), - (Microscopy.Site, 1), - (Microscopy.Channel, 1), - (Microscopy.ZIndex, 1), - (Microscopy.Timepoint, 1), - ) - ), + address=OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, 1), (Microscopy.Channel, 1), (Microscopy.ZIndex, 1), (Microscopy.Timepoint, 1))), ref=SourcePixelRef("disk", "/source/blue.tif"), source_alias="Blue", source_metadata={"Well": "A01", "site": "1", "channel": "1"}, @@ -667,15 +659,7 @@ def test_repeated_wells_use_manifest_source_well_scope( serializer = SourceProjectionMetadataSerializer(parser=parser) sources = tuple( SourcePlaneProjection( - address=OpenHCSPlaneAddress( - ( - (Microscopy.Well, well), - (Microscopy.Site, 1), - (Microscopy.Channel, 1), - (Microscopy.ZIndex, 1), - (Microscopy.Timepoint, 1), - ) - ), + address=OpenHCSPlaneAddress(((Microscopy.Well, well), (Microscopy.Site, 1), (Microscopy.Channel, 1), (Microscopy.ZIndex, 1), (Microscopy.Timepoint, 1))), ref=SourcePixelRef("disk", f"/source/{well}.tif"), source_metadata={"Well": well}, ) diff --git a/tests/unit/test_zarr_input_conversion_subdirectory.py b/tests/unit/test_zarr_input_conversion_subdirectory.py index 5f8e1f9ffd..c391a6e00f 100644 --- a/tests/unit/test_zarr_input_conversion_subdirectory.py +++ b/tests/unit/test_zarr_input_conversion_subdirectory.py @@ -62,9 +62,7 @@ def test_zarr_conversion_keys_store_outputs_by_plate_relative_virtual_path(tmp_p orchestrator = PipelineOrchestrator( plate, pipeline_config=PipelineConfig( - vfs_config=LazyVFSConfig( - materialization_backend=MaterializationBackend.ZARR - ) + vfs_config=LazyVFSConfig(materialization_backend=MaterializationBackend.ZARR) ), ).initialize() wells = orchestrator.get_component_keys(AxisFamily.active().partition_axis()) @@ -98,6 +96,6 @@ def test_zarr_conversion_keys_store_outputs_by_plate_relative_virtual_path(tmp_p } assert {Path(path).name for path in converted["image_files"]} == source_files assert all(path.startswith("zarr/") for path in converted["image_files"]) - assert {record["ref"]["backend"] for record in converted["source_projection"]} == { - Backend.ZARR.value - } + assert { + record["ref"]["backend"] for record in converted["source_projection"] + } == {Backend.ZARR.value} diff --git a/tests/unit/test_zmq_compiled_pipeline_run.py b/tests/unit/test_zmq_compiled_pipeline_run.py index 77e0de8cc8..27dac3a465 100644 --- a/tests/unit/test_zmq_compiled_pipeline_run.py +++ b/tests/unit/test_zmq_compiled_pipeline_run.py @@ -57,11 +57,7 @@ def wait_for_completion(self, execution_id, poll_interval=0.5): "execution": { "execution_id": execution_id, "subject_id": "/tmp/plate", - "status": ( - self.compile_status - if execution_id == "compile-1" - else self.execute_status - ), + "status": self.compile_status if execution_id == "compile-1" else self.execute_status, "start_time": 10.0, "end_time": 12.0, }, @@ -110,9 +106,7 @@ def observe(phase): "wait:execute-1", ] assert client.execution_submission.pipeline_code == result.request.pipeline_code - assert ( - client.execution_submission.compile_control.compile_artifact_id == "compile-1" - ) + assert client.execution_submission.compile_control.compile_artifact_id == "compile-1" assert observed_phases == [ (event, phase) for phase in ZMQPipelineRunPhase for event in ("start", "end") ] @@ -123,9 +117,7 @@ class ClientWithResults(FakeExecutionClient): def wait_for_completion(self, execution_id, poll_interval=0.5): response = super().wait_for_completion(execution_id, poll_interval) if execution_id == "execute-1": - response["execution"]["results_summary"] = { - "output_dataset_root": "/output/plate" - } + response["execution"]["results_summary"] = {"output_dataset_root": "/output/plate"} return response run = run_compiled_pipeline(ClientWithResults(), _submission()) @@ -139,9 +131,7 @@ class ClientWithLegacyResults(FakeExecutionClient): def wait_for_completion(self, execution_id, poll_interval=0.5): response = super().wait_for_completion(execution_id, poll_interval) if execution_id == "execute-1": - response["execution"]["results_summary"] = { - "output_dataset_root": "/legacy/plate" - } + response["execution"]["results_summary"] = {"output_dataset_root": "/legacy/plate"} return response run = run_compiled_pipeline(ClientWithLegacyResults(), _submission()) @@ -239,9 +229,7 @@ def test_compiled_pipeline_run_does_not_report_failed_execution_as_complete(): assert client.events[-1] == "wait:execute-1" -def test_compiled_run_retains_its_admitted_wire_and_standalone_request_stays_fresh( - monkeypatch, -): +def test_compiled_run_retains_its_admitted_wire_and_standalone_request_stays_fresh(monkeypatch): submission = _submission() rendered = [] original_render = PipelineDocumentCodec.render @@ -268,9 +256,7 @@ def submit_compile(self, request): assert len(rendered) == 1 assert run.request.pipeline_code == client.compile_submission.pipeline_code - assert ( - run.request.global_config_code == client.compile_submission.global_config_code - ) + assert run.request.global_config_code == client.compile_submission.global_config_code fresh = ZMQExecutionRequestBuilder.from_task(submission) assert fresh.pipeline_code != run.request.pipeline_code assert fresh.global_config_code != run.request.global_config_code diff --git a/tests/unit/test_zmq_execution_server.py b/tests/unit/test_zmq_execution_server.py index e89e81b9cd..bdb9ccb019 100644 --- a/tests/unit/test_zmq_execution_server.py +++ b/tests/unit/test_zmq_execution_server.py @@ -161,9 +161,7 @@ def test_axis_outcomes_qualify_server_terminal_after_export(monkeypatch, failed) if failed: assert "A02: error: producer lineage missing" in terminal[MessageFields.ERROR] else: - assert terminal[MessageFields.RESULTS_SUMMARY][ - MessageFields.PARTITION_VALUES - ] == [ + assert terminal[MessageFields.RESULTS_SUMMARY][MessageFields.PARTITION_VALUES] == [ "A01", "A02", ] @@ -335,9 +333,7 @@ def test_server_exports_outcomes_without_projecting_compiled_values( RuntimeContextObservation( "context", (), - outputs=StepExecutionObservation( - {}, (declared_output,) - ), + outputs=StepExecutionObservation({}, (declared_output,)), ), ) ), @@ -541,9 +537,7 @@ def test_zmq_server_admits_all_progress_through_generic_terminal_watermark() -> step_name="ExportToDatabase", phase=ProgressPhase.RUNNING, status=ProgressStatus.RUNNING, - completed=32, - total=33, - percent=(32 / 33) * 100.0, + completed=32, total=33, percent=(32 / 33) * 100.0, ) server._enqueue_progress(parent) worker_queue.put(parent) @@ -563,11 +557,7 @@ def test_zmq_server_admits_all_progress_through_generic_terminal_watermark() -> subscriber = ProgressStreamSubscriber(lambda: None, client._record_progress) for payload in published: assert subscriber._dispatch_message(json.dumps(payload)) - assert [payload[MessageFields.PROGRESS_SEQUENCE] for payload in received] == [ - 1, - 2, - 3, - ] + assert [payload[MessageFields.PROGRESS_SEQUENCE] for payload in received] == [1, 2, 3] assert client.progress_observation(record.execution_id).sequence == 3 assert record.to_dict()[MessageFields.PROGRESS_SEQUENCE] == 3 assert record.progress_event == published[-1] @@ -610,7 +600,8 @@ def test_zmq_server_records_the_compilation_output_plate_value_without_rebuildin ExecutionOutputSummary.EXECUTION_RECORD_KEY: output_dataset } assert ( - record.get_extra(ExecutionOutputSummary.EXECUTION_RECORD_KEY) is output_dataset + record.get_extra(ExecutionOutputSummary.EXECUTION_RECORD_KEY) + is output_dataset ) @@ -645,8 +636,7 @@ def fail_transport_stop(_server) -> None: def test_compiled_source_adoption_owns_fresh_runtime_services_and_live_source_gate( - tmp_path, - monkeypatch, + tmp_path, monkeypatch, ): import json from polystore.base import reset_memory_backend @@ -716,9 +706,7 @@ def test_compiled_source_adoption_owns_fresh_runtime_services_and_live_source_ga def reject_configuration_work(*args, **kwargs): raise AssertionError("Compiled adoption must use the admitted domain") - monkeypatch.setattr( - PipelineOrchestrator, "get_effective_config", reject_configuration_work - ) + monkeypatch.setattr(PipelineOrchestrator, "get_effective_config", reject_configuration_work) monkeypatch.setattr( "openhcs.core.orchestrator.orchestrator.ZarrStorageBackend", reject_configuration_work, @@ -782,9 +770,7 @@ def reject_source_discovery(*args, **kwargs): ) == ["A02"] -def test_compiled_request_keeps_config_scope_without_resolving_configuration( - monkeypatch, -): +def test_compiled_request_keeps_config_scope_without_resolving_configuration(monkeypatch): server = ZMQExecutionServer(port=5555) global_config = GlobalPipelineConfig( processing_config=ProcessingConfig(group_by=Microscopy.Channel) @@ -814,45 +800,35 @@ def test_compiled_request_keeps_config_scope_without_resolving_configuration( monkeypatch.setattr( zmq_execution_server_module, "ZMQOrchestratorEnvironmentRequest", - lambda **kwargs: SimpleNamespace( - prepare=lambda: SimpleNamespace( - debug_execution_policy=policy, - debug_execution_config=None, - dataset_root=context.subject_id, - ) - ), + lambda **kwargs: SimpleNamespace(prepare=lambda: SimpleNamespace( + debug_execution_policy=policy, + debug_execution_config=None, + dataset_root=context.subject_id, + )), ) def reject_resolution(*args, **kwargs): raise AssertionError("Compiled request must not resolve the saved config again") monkeypatch.setattr( - zmq_execution_server_module.ObjectState, - "resolve_saved_object", - reject_resolution, + zmq_execution_server_module.ObjectState, "resolve_saved_object", reject_resolution ) initialized = [] monkeypatch.setattr( - server, - "_initialize_orchestrator", + server, "_initialize_orchestrator", lambda *args, **kwargs: initialized.append(kwargs) or object(), ) compilations = [] monkeypatch.setattr( - server, - "_resolve_compilation", + server, "_resolve_compilation", lambda **kwargs: compilations.append(kwargs) or object(), ) monkeypatch.setattr(server, "_record_compilation_outputs", lambda *args: None) marker = object() - monkeypatch.setattr( - server, "_finish_compilation_or_execute", lambda **kwargs: marker - ) + monkeypatch.setattr(server, "_finish_compilation_or_execute", lambda **kwargs: marker) assert server._execute_with_orchestrator(context) is marker assert initialized[0]["execution_bundle"] is bundle assert initialized[0]["resolved_config"] is None assert compilations[0]["resolved_config"] is None assert compilations[0]["partition_values"] == ["A01"] - assert ( - get_current_global_config(GlobalPipelineConfig, use_live=False) is global_config - ) + assert get_current_global_config(GlobalPipelineConfig, use_live=False) is global_config diff --git a/tests/unit/test_zmq_progress.py b/tests/unit/test_zmq_progress.py index 9a356541bd..8e3a21d3f9 100644 --- a/tests/unit/test_zmq_progress.py +++ b/tests/unit/test_zmq_progress.py @@ -65,20 +65,17 @@ def test_compile_heartbeat_emits_running_pipeline_compile_event(): _emitter(emitted).compile_heartbeat(step_count=18) assert len(emitted) == 1 - assert ( - emitted[0].identity - == ProgressExecutionContext("exec-1", "/tmp/plate") - .event( - axis_id="", - step_name="pipeline", - phase=ProgressPhase.COMPILE, - status=ProgressStatus.RUNNING, - percent=0.0, - completed=0, - total=18, - ) - .identity - ) + assert emitted[0].identity == ProgressExecutionContext( + "exec-1", "/tmp/plate" + ).event( + axis_id="", + step_name="pipeline", + phase=ProgressPhase.COMPILE, + status=ProgressStatus.RUNNING, + percent=0.0, + completed=0, + total=18, + ).identity assert emitted[0].phase is ProgressPhase.COMPILE assert emitted[0].status is ProgressStatus.RUNNING assert emitted[0].total == 18 diff --git a/tests/unit/test_zmq_submission_timeout.py b/tests/unit/test_zmq_submission_timeout.py index aabd9ac9c6..5f2cdaa16f 100644 --- a/tests/unit/test_zmq_submission_timeout.py +++ b/tests/unit/test_zmq_submission_timeout.py @@ -51,9 +51,7 @@ def ensure_progress_subscription(self, *, timeout_ms: int): observed.append(("progress", timeout_ms)) def serialize_task(self, _task, _config=None): - return ExecuteRequest( - subject_id="subject", pipeline_code="pass", config_code="" - ) + return ExecuteRequest(subject_id="subject", pipeline_code="pass", config_code="") def send_control_request(self, request, *, timeout_ms: int): observed.append((request[MessageFields.TYPE], timeout_ms)) @@ -69,9 +67,7 @@ def send_control_request(self, request, *, timeout_ms: int): client, ) - client._submit_submission( - lambda: client.serialize_task(_Submission().to_task()), timeout_ms=15000 - ) + client._submit_submission(lambda: client.serialize_task(_Submission().to_task()), timeout_ms=15000) assert [phase for phase, _timeout_ms in observed] == [ "progress", @@ -121,9 +117,7 @@ def connect( client.connect = MethodType(connect, client) try: - client._submit_submission( - lambda: client.serialize_task(_Submission().to_task()), timeout_ms=15000 - ) + client._submit_submission(lambda: client.serialize_task(_Submission().to_task()), timeout_ms=15000) except RuntimeError as error: assert str(error) == "Failed to connect to execution server" else: @@ -170,9 +164,7 @@ def send_control_request(self, request, *, timeout_ms: int): ExecutionSubmissionPreparationTimeoutError, match="no execute request was sent", ): - client._submit_submission( - lambda: client.serialize_task(_Submission().to_task()), timeout_ms=10 - ) + client._submit_submission(lambda: client.serialize_task(_Submission().to_task()), timeout_ms=10) assert observed == ["connect"] diff --git a/validation/public_two_vector_388.py b/validation/public_two_vector_388.py index 25c4d63c4b..f0271c40ab 100644 --- a/validation/public_two_vector_388.py +++ b/validation/public_two_vector_388.py @@ -20,9 +20,7 @@ import time ROOT = Path("/home/ts/wt/openhcs-issue-batch-20260929") -FIXTURE = ( - ROOT / "calibration371-classification372-installed-20261001/engineering_fixture.py" -) +FIXTURE = ROOT / "calibration371-classification372-installed-20261001/engineering_fixture.py" FIXTURE_SHA = "3157e246de9348b0522f8e4d2712b84df06db71185d8c09ce868b6f74c7349ce" CLASSIFICATION_SHA = "3ca76fb8a479c2701e367bce1db1935771b818bc4be9662a958ab4e4f7912b74" PLATE = ROOT / "classification382-installed-20261001/synthetic-plate" @@ -38,40 +36,20 @@ def require(condition, message): def expected_labels(): - return [ - [ - ( - 1 - if 8 <= y < 16 and 8 <= x < 16 - else 2 if 32 <= y < 48 and 32 <= x < 48 else 0 - ) - for x in range(64) - ] - for y in range(64) - ] + return [[1 if 8 <= y < 16 and 8 <= x < 16 else + 2 if 32 <= y < 48 and 32 <= x < 48 else 0 + for x in range(64)] for y in range(64)] def assert_label_identity(values, shape): require(tuple(shape) == (64, 64), f"Labels are not a 64x64 plane: {shape!r}") - require( - len(values) == 64 and all(len(row) == 64 for row in values), - "Incomplete pixels (or RGB/stack substituted for labels)", - ) - require( - all(type(value) is int for row in values for value in row), - "Label payload must retain integral object IDs", - ) - require( - [list(row) for row in values] == expected_labels(), - "Full 4096-value object-label identity differs", - ) - return { - "shape": [64, 64], - "pixels_checked": 4096, - "label_0_pixels": 3776, - "label_1_pixels": 64, - "label_2_pixels": 256, - } + require(len(values) == 64 and all(len(row) == 64 for row in values), + "Incomplete pixels (or RGB/stack substituted for labels)") + require(all(type(value) is int for row in values for value in row), + "Label payload must retain integral object IDs") + require([list(row) for row in values] == expected_labels(), "Full 4096-value object-label identity differs") + return {"shape": [64, 64], "pixels_checked": 4096, + "label_0_pixels": 3776, "label_1_pixels": 64, "label_2_pixels": 256} def assert_measurement_rows(objects, classes): @@ -86,69 +64,39 @@ def assert_measurement_rows(objects, classes): require(len(classes) == 3, "Expected one aggregate + two per-object class rows") aggregate = [row for row in classes if row["object_label"] == ""] require(len(aggregate) == 1, "Missing/duplicate aggregate row") - per_object = { - int(row["object_label"]): row for row in classes if row["object_label"] != "" - } + per_object = {int(row["object_label"]): row for row in classes + if row["object_label"] != ""} require(set(per_object) == {1, 2}, "Classification subject differs/duplicate") require(int(aggregate[0]["slice_index"]) == 0, "Aggregate slice changed") for name in QUADRANTS: expected_count = int(name in ("low_low", "high_low")) - require( - int(aggregate[0][f"Classify_{name}_NumObjectsPerBin"]) == expected_count, - f"Wrong quadrant count: {name}", - ) - require( - float(aggregate[0][f"Classify_{name}_PctObjectsPerBin"]) - == 50 * expected_count, - f"Wrong quadrant percentage: {name}", - ) + require(int(aggregate[0][f"Classify_{name}_NumObjectsPerBin"]) == expected_count, + f"Wrong quadrant count: {name}") + require(float(aggregate[0][f"Classify_{name}_PctObjectsPerBin"]) == 50 * expected_count, + f"Wrong quadrant percentage: {name}") for label, expected_name in ((1, "low_low"), (2, "high_low")): row = per_object[label] - require( - row["object_name"] == "engineering_cells", - "Classification object subject changed", - ) + require(row["object_name"] == "engineering_cells", "Classification object subject changed") require(int(row["slice_index"]) == 0, "Classification slice changed") - require( - int(row[f"Classify_{name}"]) == int(name == expected_name), - f"Wrong per-object quadrant: label={label}, quadrant={name}", - ) - return { - "label_1": "low_low", - "label_2": "high_low", - "counts": dict(zip(QUADRANTS, (1, 1, 0, 0))), - "pixel_count": [64, 256], - "calibration_um": [1.3556, 1.3556], - "custom_splits": [100.0, 2.0], - } + require(int(row[f"Classify_{name}"]) == int(name == expected_name), + f"Wrong per-object quadrant: label={label}, quadrant={name}") + return {"label_1": "low_low", "label_2": "high_low", + "counts": dict(zip(QUADRANTS, (1, 1, 0, 0))), + "pixel_count": [64, 256], "calibration_um": [1.3556, 1.3556], + "custom_splits": [100.0, 2.0]} def pipeline_patch(suffix): return { - "config_type": "pipeline", - "values": { - "num_workers": 1, - "use_threading": True, - "microscope": "source_bindings", - "materialize_runtime_artifacts": True, + "config_type": "pipeline", "values": { + "num_workers": 1, "use_threading": True, + "microscope": "source_bindings", "materialize_runtime_artifacts": True, "well_filter_config": {"well_filter": ["A01"]}, "source_bindings_config": { - "bindings": [ - { - "alias": "engineering_raw", - "selector": { - "components": [{"component": "channel", "value": "1"}], - "filters": [ - { - "subject": "file", - "match_type": "contains", - "value": "_w1_", - } - ], - }, - } - ], - "source_voxel_spacing": {"values_zyx": [1.3556, 1.3556]}, + "bindings": [{"alias": "engineering_raw", "selector": { + "components": [{"component": "channel", "value": "1"}], + "filters": [{"subject": "file", "match_type": "contains", "value": "_w1_"}], + }}], "source_voxel_spacing": {"values_zyx": [1.3556, 1.3556]}, }, "path_planning_config": {"output_dir_suffix": suffix}, }, @@ -159,13 +107,9 @@ def step_overrides(viewer_port, *, select_source=False): values = { "processing_config": {"variable_components": ["channel"], "group_by": "NONE"}, "step_materialization_config": {"enabled": True}, - "napari_streaming_config": { - "enabled": True, - "persistent": True, - "host": "127.0.0.1", - "port": viewer_port, - "transport_mode": "tcp", - }, + "napari_streaming_config": {"enabled": True, "persistent": True, + "host": "127.0.0.1", "port": viewer_port, + "transport_mode": "tcp"}, } if select_source: values["source_bindings"] = {"enabled": True} @@ -174,19 +118,12 @@ def step_overrides(viewer_port, *, select_source=False): def paired_kwargs(): - return { - "select_the_object_to_be_classified": "engineering_cells", - "measurement1_feature": "pixel_count", - "measurement2_feature": "calibration_um", - "threshold1_method": "custom", - "threshold1_value": 100.0, - "threshold2_method": "custom", - "threshold2_value": 2.0, - "low_low_name": "low_low", - "high_low_name": "high_low", - "low_high_name": "low_high", - "high_high_name": "high_high", - } + return {"select_the_object_to_be_classified": "engineering_cells", + "measurement1_feature": "pixel_count", "measurement2_feature": "calibration_um", + "threshold1_method": "custom", "threshold1_value": 100.0, + "threshold2_method": "custom", "threshold2_value": 2.0, + "low_low_name": "low_low", "high_low_name": "high_low", + "low_high_name": "low_high", "high_high_name": "high_high"} class PublicJourney: @@ -199,7 +136,6 @@ def __init__(self, client, receipt_dir): def save(self, filename, value): from python_introspect import to_jsonable - with (self.root / filename).open("x") as stream: stream.write(json.dumps(to_jsonable(value), indent=2) + "\n") @@ -207,35 +143,21 @@ def call(self, name, arguments): from openhcs.agent.capabilities import get_agent_capability from openhcs.mcp.dev_client_core import McpDevToolBatchResponse from python_introspect import to_jsonable - capability = get_agent_capability(name) self.sequence += 1 stem = f"{self.sequence:03d}-{name}" self.save(stem + "-request.json", arguments) - result = self.client.execute( - [ - "--timeout-seconds", - "10", - "call", - capability.name, - "--arguments", - json.dumps(to_jsonable(arguments)), - "--json", - ] - ) + result = self.client.execute([ + "--timeout-seconds", "10", "call", capability.name, + "--arguments", json.dumps(to_jsonable(arguments)), "--json", + ]) self.save(stem + "-response.json", result.payload) - self.save( - stem + "-transport.json", - { - "returncode": result.returncode, - "server_stderr_tail": result.server_stderr_tail, - }, - ) + self.save(stem + "-transport.json", { + "returncode": result.returncode, "server_stderr_tail": result.server_stderr_tail, + }) response = McpDevToolBatchResponse.for_rendering(result.payload) - require( - result.returncode == 0 and not response.has_errors(), - f"{name} failed/uncertain; retained response; NEVER replay", - ) + require(result.returncode == 0 and not response.has_errors(), + f"{name} failed/uncertain; retained response; NEVER replay") payload = response.payload_for(capability) require(payload is not None, f"Missing declared output for {name}") return payload @@ -245,39 +167,22 @@ def execute_source(self, source_request, connection): from openhcs.agent.capabilities import agent_capabilities from openhcs.agent.dto.session import DatasetListState from openhcs.mcp.dev_client_core import McpDevToolBatchResponse - - result = self.client.execute( - [ - "--timeout-seconds", - "1800", - "execute-source", - source_request["plate_path"], - "--source-text", - source_request["pipeline_source"], - "--host", - connection["host"], - "--port", - str(connection["port"]), - "--transport-mode", - connection["transport_mode"], - "--wait", - "--json", - ] - ) + result = self.client.execute([ + "--timeout-seconds", "1800", "execute-source", source_request["plate_path"], + "--source-text", source_request["pipeline_source"], + "--host", connection["host"], "--port", str(connection["port"]), + "--transport-mode", connection["transport_mode"], "--wait", "--json", + ]) self.sequence += 1 self.save(f"{self.sequence:03d}-execute-source-response.json", result.payload) response = McpDevToolBatchResponse.for_rendering(result.payload) - require( - result.returncode == 0 and not response.has_errors(), - "execute-source failed/uncertain; retained response; NEVER replay", - ) + require(result.returncode == 0 and not response.has_errors(), + "execute-source failed/uncertain; retained response; NEVER replay") state = response.payload_for(agent_capabilities.session_datasets) require(isinstance(state, DatasetListState), "Missing final dataset list") return state - def observe_until( - self, name, arguments, ready, terminal, *, seconds=60, on_observation=None - ): + def observe_until(self, name, arguments, ready, terminal, *, seconds=60, on_observation=None): deadline = None if seconds is None else time.monotonic() + seconds while True: value = self.call(name, arguments) @@ -287,10 +192,7 @@ def observe_until( return value require(not terminal(value), f"{name}: original attempt became terminal") if deadline is not None: - require( - time.monotonic() < deadline, - f"{name}: pending handle retained; no replay", - ) + require(time.monotonic() < deadline, f"{name}: pending handle retained; no replay") time.sleep(0.5) def readiness_observer(self, bootstrap_handle, phase, validate_handle): @@ -302,7 +204,6 @@ def readiness_observer(self, bootstrap_handle, phase, validate_handle): """ import psutil from python_introspect import to_jsonable - started = time.monotonic() last_progress = None last_report = float("-inf") @@ -311,10 +212,7 @@ def observed(value): nonlocal last_progress, last_report validate_handle(value) identity = bootstrap_handle.process_identity - require( - identity.is_alive() is True, - "Original runtime is terminal/unknown; stop and retain handle", - ) + require(identity.is_alive() is True, "Original runtime is terminal/unknown; stop and retain handle") progress = to_jsonable(value.progress) elapsed = time.monotonic() - started if progress != last_progress or elapsed - last_report >= 5: @@ -322,7 +220,6 @@ def observed(value): native_processes = (native, *native.children(recursive=True)) driver = psutil.Process() driver_processes = (driver, *driver.children(recursive=True)) - # Ephemeral OS observation, no mirrored native authority/cache. def rss(processes): total = 0 @@ -332,608 +229,274 @@ def rss(processes): except psutil.NoSuchProcess: continue return round(total / 1024**2, 2) - - require( - identity.is_alive() is True, - "Runtime incarnation changed during resource observation", - ) - print( - json.dumps( - { - "phase": phase, - "elapsed_seconds": round(elapsed, 2), - "progress": progress, - "original_process_identity": to_jsonable(identity), - "native_tree_rss_mib": rss(native_processes), - "driver_tree_rss_mib": rss(driver_processes), - "resource_scope": "trees may overlap; parent owns combined live budget", - "ordinary_mcp_timeout_seconds": 10, - } - ), - flush=True, - ) + require(identity.is_alive() is True, "Runtime incarnation changed during resource observation") + print(json.dumps({"phase": phase, + "elapsed_seconds": round(elapsed, 2), "progress": progress, + "original_process_identity": to_jsonable(identity), + "native_tree_rss_mib": rss(native_processes), + "driver_tree_rss_mib": rss(driver_processes), + "resource_scope": "trees may overlap; parent owns combined live budget", + "ordinary_mcp_timeout_seconds": 10}), flush=True) last_progress, last_report = progress, elapsed - return observed def csv_preview(query, suffix): require(query.truncated_count == 0, "File inventory truncated") - matches = [ - record - for record in query.records - if record.relative_path is not None and record.relative_path.endswith(suffix) - ] + matches = [record for record in query.records + if record.relative_path is not None and record.relative_path.endswith(suffix)] require(len(matches) == 1, f"Expected exactly one persisted {suffix}: {matches!r}") preview = matches[0].preview - require( - preview is not None and not preview.truncated and not preview.omitted_reason, - "CSV preview incomplete; no count-only acceptance", - ) + require(preview is not None and not preview.truncated and not preview.omitted_reason, + "CSV preview incomplete; no count-only acceptance") return matches[0].full_path, preview.csv_rows def run(args): - from openhcs.agent.dto.execution import ( - RuntimeBootstrapCloseRequest, - RuntimeBootstrapHandle, - ) + from openhcs.agent.dto.execution import RuntimeBootstrapCloseRequest, RuntimeBootstrapHandle from openhcs.mcp.dev_client import McpDevClient from openhcs.processing.custom_functions.manager import CustomFunctionManager from python_introspect import to_jsonable from python_introspect import dataclass_from_mapping - require( - args.parent_serialized_slot_authorized, - "Parent must authorize the released serialized slot", - ) - require( - args.receipt_dir is not None and args.receipt_dir.is_absolute(), - "Explicit persistent receipt-dir required", - ) - require( - args.receipt_dir.resolve().is_relative_to(Path("/home/ts/wt")), - "Receipts must be under persistent /home/ts/wt", - ) + require(args.parent_serialized_slot_authorized, "Parent must authorize the released serialized slot") + require(args.receipt_dir is not None and args.receipt_dir.is_absolute(), "Explicit persistent receipt-dir required") + require(args.receipt_dir.resolve().is_relative_to(Path("/home/ts/wt")), "Receipts must be under persistent /home/ts/wt") require(not args.receipt_dir.exists(), "Refuse to overwrite original receipts") - require( - args.output_suffix.startswith("_classification388_") - and all(char.isalnum() or char == "_" for char in args.output_suffix), - "Invalid unique output suffix", - ) + require(args.output_suffix.startswith("_classification388_") and + all(char.isalnum() or char == "_" for char in args.output_suffix), "Invalid unique output suffix") output_plate = PLATE.with_name(PLATE.name + args.output_suffix) - require( - not output_plate.exists(), "Refuse to reuse/overwrite an earlier output plate" - ) - require( - args.runtime_port != args.viewer_port - and 1 <= args.runtime_port <= 64535 - and 1 <= args.viewer_port <= 64535, - "Invalid/disjoint runtime and viewer endpoint ports", - ) - require( - hashlib.sha256(FIXTURE.read_bytes()).hexdigest() == FIXTURE_SHA, - "Original fixture changed", - ) + require(not output_plate.exists(), "Refuse to reuse/overwrite an earlier output plate") + require(args.runtime_port != args.viewer_port and + 1 <= args.runtime_port <= 64535 and 1 <= args.viewer_port <= 64535, + "Invalid/disjoint runtime and viewer endpoint ports") + require(hashlib.sha256(FIXTURE.read_bytes()).hexdigest() == FIXTURE_SHA, "Original fixture changed") require(PLATE.is_dir(), "Original reviewed synthetic plate is unavailable") - classification_spec = importlib.util.find_spec( - "openhcs.processing.backends.cellprofiler.classification" - ) - require( - classification_spec is not None - and classification_spec.origin is not None - and hashlib.sha256(Path(classification_spec.origin).read_bytes()).hexdigest() - == CLASSIFICATION_SHA, - "Installed classification bytes are not reviewed PR388", - ) - require( - os.environ.get("XDG_DATA_HOME") - and os.environ.get("OPENHCS_UI_CONFIG_CACHE_FILE"), - "Parent must supply the exact controlled launch data/cache selectors", - ) + classification_spec = importlib.util.find_spec("openhcs.processing.backends.cellprofiler.classification") + require(classification_spec is not None and classification_spec.origin is not None and + hashlib.sha256(Path(classification_spec.origin).read_bytes()).hexdigest() == CLASSIFICATION_SHA, + "Installed classification bytes are not reviewed PR388") + require(os.environ.get("XDG_DATA_HOME") and os.environ.get("OPENHCS_UI_CONFIG_CACHE_FILE"), + "Parent must supply the exact controlled launch data/cache selectors") intended_store = CustomFunctionManager.default_storage_directory() original_handle = None if args.runtime_handle is not None: - original_handle = dataclass_from_mapping( - RuntimeBootstrapHandle, json.loads(args.runtime_handle.read_text()) - ) - require( - original_handle.connection.port == args.runtime_port - and original_handle.connection.host == "127.0.0.1" - and original_handle.launch_plan.storage_dir == intended_store, - "Same-context handle disagrees with parent endpoint/store", - ) + original_handle = dataclass_from_mapping(RuntimeBootstrapHandle, json.loads(args.runtime_handle.read_text())) + require(original_handle.connection.port == args.runtime_port and + original_handle.connection.host == "127.0.0.1" and + original_handle.launch_plan.storage_dir == intended_store, + "Same-context handle disagrees with parent endpoint/store") else: # Read-only endpoint check; NEVER take over Dalton's or another listener. - for port in ( - args.runtime_port, - args.runtime_port + 1000, - args.viewer_port, - args.viewer_port + 1000, - ): + for port in (args.runtime_port, args.runtime_port + 1000, args.viewer_port, args.viewer_port + 1000): with socket.socket() as probe: probe.settimeout(0.2) - require( - probe.connect_ex(("127.0.0.1", port)) != 0, - f"Endpoint {port} occupied: stop", - ) - persisted_fixture = CustomFunctionManager.source_path_for_name( - intended_store, FIXTURE_NAME - ) + require(probe.connect_ex(("127.0.0.1", port)) != 0, f"Endpoint {port} occupied: stop") + persisted_fixture = CustomFunctionManager.source_path_for_name(intended_store, FIXTURE_NAME) if args.fixture_function_id is None: - require( - not persisted_fixture.exists(), - "Fixture already persisted: reconcile/reuse explicitly; NEVER register again", - ) + require(not persisted_fixture.exists(), "Fixture already persisted: reconcile/reuse explicitly; NEVER register again") else: - require( - persisted_fixture.is_file() - and hashlib.sha256(persisted_fixture.read_bytes()).hexdigest() - == FIXTURE_SHA, - "Explicit reused fixture is not original reviewed source", - ) + require(persisted_fixture.is_file() and hashlib.sha256(persisted_fixture.read_bytes()).hexdigest() == FIXTURE_SHA, + "Explicit reused fixture is not original reviewed source") args.receipt_dir.mkdir() with (args.receipt_dir / "mcp-server.stderr").open("x") as stderr: - with McpDevClient( - python_executable=sys.executable, - use_resident_server=False, - server_stderr=stderr, - ) as client: + with McpDevClient(python_executable=sys.executable, use_resident_server=False, + server_stderr=stderr) as client: journey = PublicJourney(client, args.receipt_dir) - journey.save( - "engineering-input-selection.json", - { - "fixture": str(FIXTURE), - "sha256": FIXTURE_SHA, - "plate": str(PLATE), - "output_plate": str(output_plate), - "paired_kwargs": paired_kwargs(), - "scope": "reviewed engineering fixture only; not biology or frozen-blind replay", - "installed_python": sys.executable, - "classification_source": classification_spec.origin, - "classification_sha256": CLASSIFICATION_SHA, - "runtime_ownership": ( - "parent retained same-context handle" - if original_handle - else "driver bootstrapped exact handle" - ), - }, - ) + journey.save("engineering-input-selection.json", { + "fixture": str(FIXTURE), "sha256": FIXTURE_SHA, "plate": str(PLATE), + "output_plate": str(output_plate), "paired_kwargs": paired_kwargs(), + "scope": "reviewed engineering fixture only; not biology or frozen-blind replay", + "installed_python": sys.executable, "classification_source": classification_spec.origin, + "classification_sha256": CLASSIFICATION_SHA, + "runtime_ownership": "parent retained same-context handle" if original_handle else "driver bootstrapped exact handle", + }) try: journey.call("openhcs_health_check", {}) journey.call("openhcs_get_authoring_context", {"kind": "first_use"}) - journey.call( - "openhcs_get_authoring_context", {"kind": "custom_function"} - ) + journey.call("openhcs_get_authoring_context", {"kind": "custom_function"}) journey.call("openhcs_search_capabilities", {"text": "owned runtime"}) - connection = { - "host": "127.0.0.1", - "port": args.runtime_port, - "transport_mode": "tcp", - "persistent": True, - } + connection = {"host": "127.0.0.1", "port": args.runtime_port, + "transport_mode": "tcp", "persistent": True} if original_handle is None: - startup = journey.call( - "openhcs_start_owned_runtime", {**connection, "timeout_ms": 500} - ) + startup = journey.call("openhcs_start_owned_runtime", {**connection, "timeout_ms": 500}) original_handle = startup.handle journey.save("original-runtime-handle.json", original_handle) handle = to_jsonable(original_handle) - observed = journey.observe_until( - "openhcs_observe_owned_runtime", - {"handle": handle}, - lambda value: value.ready, - lambda value: value.process_alive is False, - seconds=None, - on_observation=journey.readiness_observer( - original_handle, - "startup", - lambda value: require( - value.handle == original_handle, - "Startup observation changed exact handle", - ), - ), - ) - require( - observed.handle == original_handle, "Runtime incarnation changed" - ) - journey.call( - "openhcs_search_capabilities", {"text": "catalog preparation"} - ) - preparation = journey.call( - "openhcs_start_function_catalog_preparation", connection - ) + observed = journey.observe_until("openhcs_observe_owned_runtime", {"handle": handle}, + lambda value: value.ready, lambda value: value.process_alive is False, + seconds=None, on_observation=journey.readiness_observer( + original_handle, "startup", lambda value: require( + value.handle == original_handle, "Startup observation changed exact handle"))) + require(observed.handle == original_handle, "Runtime incarnation changed") + journey.call("openhcs_search_capabilities", {"text": "catalog preparation"}) + preparation = journey.call("openhcs_start_function_catalog_preparation", connection) journey.save("original-catalog-handle.json", preparation.handle) - prepared = journey.observe_until( - "openhcs_get_function_catalog_preparation_status", + prepared = journey.observe_until("openhcs_get_function_catalog_preparation_status", to_jsonable(preparation.handle), - lambda value: value.outcome.ready, - lambda value: value.outcome.terminal, - seconds=None, - on_observation=journey.readiness_observer( - original_handle, - "catalog", - lambda value: value.require_handle(preparation.handle), - ), - ) + lambda value: value.outcome.ready, lambda value: value.outcome.terminal, + seconds=None, on_observation=journey.readiness_observer( + original_handle, "catalog", lambda value: value.require_handle(preparation.handle))) prepared.require_handle(preparation.handle) - require( - prepared.handle.server_identity == original_handle.process_identity, - "Catalog warmed a different native owner", - ) + require(prepared.handle.server_identity == original_handle.process_identity, + "Catalog warmed a different native owner") fixture_function_id = args.fixture_function_id if fixture_function_id is None: - registered = journey.call( - "openhcs_register_custom_function", - { - **connection, - "source_code": FIXTURE.read_text(), - "persist": True, - "function_name": FIXTURE_NAME, - "storage_dir": original_handle.launch_plan.storage_dir, - }, - ) - require( - registered.server_identity == original_handle.process_identity, - "Fixture registered on a different native owner", - ) - functions = [ - item - for item in registered.functions - if item.name == FIXTURE_NAME - ] - require( - registered.registered_count == 1 and len(functions) == 1, - "Fixture registration differs", - ) + registered = journey.call("openhcs_register_custom_function", { + **connection, "source_code": FIXTURE.read_text(), "persist": True, + "function_name": FIXTURE_NAME, "storage_dir": original_handle.launch_plan.storage_dir, + }) + require(registered.server_identity == original_handle.process_identity, + "Fixture registered on a different native owner") + functions = [item for item in registered.functions if item.name == FIXTURE_NAME] + require(registered.registered_count == 1 and len(functions) == 1, "Fixture registration differs") fixture_function_id = functions[0].function_id - fixture_detail = journey.call( - "openhcs_describe_function", {"function_id": fixture_function_id} - ) - require( - fixture_detail.entry.name == FIXTURE_NAME, - "Reused registry identity names a different fixture", - ) + fixture_detail = journey.call("openhcs_describe_function", {"function_id": fixture_function_id}) + require(fixture_detail.entry.name == FIXTURE_NAME, "Reused registry identity names a different fixture") journey.call("openhcs_describe_function", {"function_id": PAIR_ID}) - journey.call( - "openhcs_describe_config_schema", {"config_type": "pipeline"} - ) + journey.call("openhcs_describe_config_schema", {"config_type": "pipeline"}) journey.call("openhcs_describe_config_schema", {"config_type": "step"}) patch = pipeline_patch(args.output_suffix) validated_config = journey.call("openhcs_validate_config_patch", patch) - require( - validated_config.valid and validated_config.config_ref is not None, - "Invalid complete pipeline config", - ) - pipeline = journey.call( - "openhcs_create_pipeline", - {"pipeline_config_id": validated_config.config_ref.config_id}, - ) - journey.call( - "openhcs_add_function_step", - { - "pipeline_id": pipeline.pipeline_id, - "function_id": fixture_function_id, - "name": "engineering_fixture", - "kwargs": {}, - "step_config_overrides": step_overrides( - args.viewer_port, select_source=True - ), - }, - ) - authored = journey.call( - "openhcs_add_function_step", - { - "pipeline_id": pipeline.pipeline_id, - "function_id": PAIR_ID, - "name": PAIR_STEP, - "kwargs": paired_kwargs(), - "step_config_overrides": step_overrides(args.viewer_port), - }, - ) - require( - len(authored.steps) == 2 - and authored.steps[1].functions[0].kwargs == paired_kwargs(), - "Public authoring lost selectors/settings", - ) - validated = journey.call( - "openhcs_validate_pipeline", {"pipeline_id": pipeline.pipeline_id} - ) + require(validated_config.valid and validated_config.config_ref is not None, "Invalid complete pipeline config") + pipeline = journey.call("openhcs_create_pipeline", {"pipeline_config_id": validated_config.config_ref.config_id}) + journey.call("openhcs_add_function_step", { + "pipeline_id": pipeline.pipeline_id, "function_id": fixture_function_id, + "name": "engineering_fixture", "kwargs": {}, + "step_config_overrides": step_overrides(args.viewer_port, select_source=True), + }) + authored = journey.call("openhcs_add_function_step", { + "pipeline_id": pipeline.pipeline_id, "function_id": PAIR_ID, + "name": PAIR_STEP, "kwargs": paired_kwargs(), + "step_config_overrides": step_overrides(args.viewer_port), + }) + require(len(authored.steps) == 2 and authored.steps[1].functions[0].kwargs == paired_kwargs(), + "Public authoring lost selectors/settings") + validated = journey.call("openhcs_validate_pipeline", {"pipeline_id": pipeline.pipeline_id}) require(validated.valid, "Public complete-document validation failed") - clean = journey.call( - "openhcs_render_pipeline_source", - {"pipeline_id": pipeline.pipeline_id, "clean": True}, - ) - resolved = journey.call( - "openhcs_render_pipeline_source", - {"pipeline_id": pipeline.pipeline_id, "clean": False}, - ) - for name, rendered in ( - ("public-clean.py", clean), - ("public-resolved.py", resolved), - ): + clean = journey.call("openhcs_render_pipeline_source", {"pipeline_id": pipeline.pipeline_id, "clean": True}) + resolved = journey.call("openhcs_render_pipeline_source", {"pipeline_id": pipeline.pipeline_id, "clean": False}) + for name, rendered in (("public-clean.py", clean), ("public-resolved.py", resolved)): with (args.receipt_dir / name).open("x") as source: source.write(rendered.source) - require( - "pipeline_config" in rendered.source - and "pipeline_steps" in rendered.source, - "Renderer returned a steps-only fragment", - ) + require("pipeline_config" in rendered.source and "pipeline_steps" in rendered.source, + "Renderer returned a steps-only fragment") # Native plan/source reconstruction receives EXACT public clean source, not edited source. - source_request = { - "plate_path": str(PLATE), - "pipeline_source": clean.source, - } - plan = journey.call( - "openhcs_inspect_pipeline_source_artifact_plan", source_request - ) - require( - plan.axis_count == 1 - and plan.step_count == 2 - and len(plan.steps) == 2, - "Unexpected/truncated axes or steps", - ) - require( - not plan.truncated_axis_count and not plan.truncated_step_count, - "Plan is truncated", - ) + source_request = {"plate_path": str(PLATE), "pipeline_source": clean.source} + plan = journey.call("openhcs_inspect_pipeline_source_artifact_plan", source_request) + require(plan.axis_count == 1 and plan.step_count == 2 and len(plan.steps) == 2, + "Unexpected/truncated axes or steps") + require(not plan.truncated_axis_count and not plan.truncated_step_count, + "Plan is truncated") final = next(step for step in plan.steps if step.step_name == PAIR_STEP) - require( - final.axis_id == "A01" - and final.step_index == 1 - and not final.truncated_artifact_input_count - and not final.truncated_artifact_output_count, - "Paired compiled axis/order/artifact coverage changed", - ) - require( - {"engineering_cells", "engineering_object_rows"} - <= {artifact.name for artifact in final.artifact_inputs}, - "Compiled pair lost original object subject/prior measurement producer", - ) - require( - final.main_flow_axis_persistence_enabled - and final.main_flow_materialization.plate_root == str(output_plate), - "Final unchanged-label persistence disabled/wrong destination", - ) - require( - len(final.artifact_outputs) == 1 - and final.artifact_outputs[0].name == PAIR_STEP + "_1_measurements" - and final.artifact_outputs[0].materialization is not None - and final.artifact_outputs[0].materialization.persistent_enabled, - "Pair must declare ONLY classification rows (no retained RGB output)", - ) - journey.save( - "exact-output-slots.json", - { - "callable_return_0": "unchanged engineering_cells dense object labels, int32[64,64]", - "callable_return_1": final.artifact_outputs[0], - "fixture_rows": "engineering_object_rows", - "final_main_flow": final.main_flow_materialization, - }, - ) + require(final.axis_id == "A01" and final.step_index == 1 and + not final.truncated_artifact_input_count and not final.truncated_artifact_output_count, + "Paired compiled axis/order/artifact coverage changed") + require({"engineering_cells", "engineering_object_rows"} <= + {artifact.name for artifact in final.artifact_inputs}, + "Compiled pair lost original object subject/prior measurement producer") + require(final.main_flow_axis_persistence_enabled and + final.main_flow_materialization.plate_root == str(output_plate), + "Final unchanged-label persistence disabled/wrong destination") + require(len(final.artifact_outputs) == 1 and + final.artifact_outputs[0].name == PAIR_STEP + "_1_measurements" and + final.artifact_outputs[0].materialization is not None and + final.artifact_outputs[0].materialization.persistent_enabled, + "Pair must declare ONLY classification rows (no retained RGB output)") + journey.save("exact-output-slots.json", { + "callable_return_0": "unchanged engineering_cells dense object labels, int32[64,64]", + "callable_return_1": final.artifact_outputs[0], + "fixture_rows": "engineering_object_rows", "final_main_flow": final.main_flow_materialization, + }) run = journey.execute_source(source_request, connection) journey.save("original-session-run.json", run) (row,) = (row for row in run.rows if row.root == str(PLATE)) - require( - row.compiled and row.terminal_status == "complete", - "Original native compile or execution failed", - ) - results = journey.call( - "openhcs_query_plate_files", - { - "plate_path": str(output_plate), - "kind": "result", - "limit": 50, - "include_previews": True, - "max_preview_lines": 10, - }, - ) - object_file, object_rows = csv_preview( - results, "engineering_object_rows_step0_details.csv" - ) - class_file, class_rows = csv_preview( - results, PAIR_STEP + "_1_measurements_step1_details.csv" - ) - declared_csvs = { - path - for group in final.artifact_outputs[0].materialization.paths - for path in group.candidate_paths - } - require( - class_file in declared_csvs, - "Classification CSV is not the current declared output slot", - ) + require(row.compiled and row.terminal_status == "complete", + "Original native compile or execution failed") + results = journey.call("openhcs_query_plate_files", { + "plate_path": str(output_plate), "kind": "result", "limit": 50, + "include_previews": True, "max_preview_lines": 10, + }) + object_file, object_rows = csv_preview(results, "engineering_object_rows_step0_details.csv") + class_file, class_rows = csv_preview(results, PAIR_STEP + "_1_measurements_step1_details.csv") + declared_csvs = {path for group in final.artifact_outputs[0].materialization.paths + for path in group.candidate_paths} + require(class_file in declared_csvs, "Classification CSV is not the current declared output slot") rows_evidence = assert_measurement_rows(object_rows, class_rows) # Semantic producer identity, NOT a hardcoded route/submission address. - viewer_args = { - "host": "127.0.0.1", - "port": args.viewer_port, - "transport_mode": "tcp", - } - state = journey.call( - "openhcs_get_viewer_window_state", - {**viewer_args, "include_response": False}, - ) - layers = [ - layer - for layer in state.layers - if any( - owner.step_name == PAIR_STEP - and owner.pipeline_position == 1 - and owner.output_kind == "main" - for owner in layer.producer_identities - ) - ] - require( - state.observed and len(layers) == 1, - "Missing/ambiguous paired main-flow viewer layer", - ) - payloads = journey.call( - "openhcs_get_viewer_window_payloads", - { - **viewer_args, - "route_key": layers[0].route_key, - "axis_indices": [], - "include_array_values": True, - "max_array_elements": 4096, - "include_response": False, - }, - ) - require( - payloads.observed - and len(payloads.layers) == 1 - and len(payloads.layers[0].payloads) == 1, - "Missing/ambiguous full paired viewer pixels", - ) + viewer_args = {"host": "127.0.0.1", "port": args.viewer_port, "transport_mode": "tcp"} + state = journey.call("openhcs_get_viewer_window_state", {**viewer_args, "include_response": False}) + layers = [layer for layer in state.layers if any( + owner.step_name == PAIR_STEP and owner.pipeline_position == 1 and owner.output_kind == "main" + for owner in layer.producer_identities)] + require(state.observed and len(layers) == 1, "Missing/ambiguous paired main-flow viewer layer") + payloads = journey.call("openhcs_get_viewer_window_payloads", { + **viewer_args, "route_key": layers[0].route_key, "axis_indices": [], + "include_array_values": True, "max_array_elements": 4096, "include_response": False, + }) + require(payloads.observed and len(payloads.layers) == 1 and len(payloads.layers[0].payloads) == 1, + "Missing/ambiguous full paired viewer pixels") pixels = payloads.layers[0].payloads[0] - require( - pixels.array_value_summary["included"] is True - and pixels.array_value_summary["size"] == 4096, - "Viewer pixels truncated", - ) - pixel_evidence = assert_label_identity( - pixels.array_values, pixels.array_value_summary["shape"] - ) + require(pixels.array_value_summary["included"] is True and + pixels.array_value_summary["size"] == 4096, "Viewer pixels truncated") + pixel_evidence = assert_label_identity(pixels.array_values, pixels.array_value_summary["shape"]) # Persistent file identity derives from the actual compiled final checkpoint. - image_query = journey.call( - "openhcs_query_plate_files", - { - "plate_path": str(output_plate), - "kind": "image", - "limit": 50, - "include_previews": False, - }, - ) - require( - image_query.truncated_count == 0, - "Persisted image inventory truncated", - ) + image_query = journey.call("openhcs_query_plate_files", { + "plate_path": str(output_plate), "kind": "image", "limit": 50, "include_previews": False, + }) + require(image_query.truncated_count == 0, "Persisted image inventory truncated") # The last step publishes the plate's final main image under its # compiled output_dir; checkpoint directories can include other # artifact projections and are not the plate image inventory. final_images = Path(final.output_dir) - images = [ - record - for record in image_query.records - if record.source_path is not None - and Path(record.source_path).is_relative_to(final_images) - ] - require( - len(images) == 1, "Missing/ambiguous persisted final label image" - ) - saved = journey.call( - "openhcs_sample_plate_image", - { - "plate_path": str(output_plate), - "image_path": images[0].virtual_path, - "height": 64, - "width": 64, - "include_array_values": True, - "max_array_elements": 4096, - }, - ) - require( - saved.sample_included and saved.sample_origin_yx == (0, 0), - "Persisted pixels omitted/cropped", - ) - saved_evidence = assert_label_identity( - saved.sample_values, saved.sample_shape - ) - require( - to_jsonable(saved.sample_values) - == to_jsonable(pixels.array_values), - "Streamed/saved label pixels differ", - ) - journey.save( - "assertions-before-close.json", - { - "rows": rows_evidence, - "streamed_labels": pixel_evidence, - "saved_labels": saved_evidence, - "saved_image": images[0].source_path, - "fixture_csv": object_file, - "classification_csv": class_file, - "source_sha256": hashlib.sha256( - clean.source.encode() - ).hexdigest(), - }, - ) + images = [record for record in image_query.records if record.source_path is not None and + Path(record.source_path).is_relative_to(final_images)] + require(len(images) == 1, "Missing/ambiguous persisted final label image") + saved = journey.call("openhcs_sample_plate_image", { + "plate_path": str(output_plate), "image_path": images[0].virtual_path, + "height": 64, "width": 64, "include_array_values": True, "max_array_elements": 4096, + }) + require(saved.sample_included and saved.sample_origin_yx == (0, 0), "Persisted pixels omitted/cropped") + saved_evidence = assert_label_identity(saved.sample_values, saved.sample_shape) + require(to_jsonable(saved.sample_values) == to_jsonable(pixels.array_values), "Streamed/saved label pixels differ") + journey.save("assertions-before-close.json", { + "rows": rows_evidence, "streamed_labels": pixel_evidence, + "saved_labels": saved_evidence, "saved_image": images[0].source_path, + "fixture_csv": object_file, "classification_csv": class_file, + "source_sha256": hashlib.sha256(clean.source.encode()).hexdigest(), + }) closed_by_driver = args.runtime_handle is None if closed_by_driver: - closed_viewer = journey.call( - "openhcs_close_viewer_window", - {**viewer_args, "confirmed": True}, - ) - require( - closed_viewer.process_exited is True, - "Original viewer process exit unproven", - ) - closed_runtime = journey.call( - "openhcs_close_owned_runtime", - to_jsonable( - RuntimeBootstrapCloseRequest(handle=original_handle) - ), - ) - require( - closed_runtime.handle == original_handle - and closed_runtime.outcome.process_exited is True, - "Original runtime process exit unproven", - ) - journey.save( - "ACCEPTANCE.json", - { - "engineering_journey_passed": True, - "native_compiled": row.compiled, - "native_execution_id": row.finished_execution_id, - "native_execution_status": row.terminal_status, - "scope": "public paired engineering only; not installed biology/global FULL", - "processes_exited": True if closed_by_driver else None, - "lifecycle_owner": ( - "driver exact owned close" - if closed_by_driver - else "parent retains handed-off runtime/viewer; no close attempted" - ), - }, - ) + closed_viewer = journey.call("openhcs_close_viewer_window", {**viewer_args, "confirmed": True}) + require(closed_viewer.process_exited is True, "Original viewer process exit unproven") + closed_runtime = journey.call("openhcs_close_owned_runtime", to_jsonable(RuntimeBootstrapCloseRequest(handle=original_handle))) + require(closed_runtime.handle == original_handle and closed_runtime.outcome.process_exited is True, + "Original runtime process exit unproven") + journey.save("ACCEPTANCE.json", {"engineering_journey_passed": True, + "native_compiled": row.compiled, + "native_execution_id": row.finished_execution_id, + "native_execution_status": row.terminal_status, + "scope": "public paired engineering only; not installed biology/global FULL", + "processes_exited": True if closed_by_driver else None, + "lifecycle_owner": "driver exact owned close" if closed_by_driver else "parent retains handed-off runtime/viewer; no close attempted"}) except BaseException as error: - journey.save( - "FAILED-OR-UNCERTAIN.json", - { - "exception": type(error).__name__, - "message": str(error), - "disposition": "STOP: no replay; parent retains original handles and cleanup ownership", - }, - ) + journey.save("FAILED-OR-UNCERTAIN.json", {"exception": type(error).__name__, + "message": str(error), "disposition": "STOP: no replay; parent retains original handles and cleanup ownership"}) raise def offline_check(): labels = expected_labels() assert_label_identity(labels, (64, 64)) - objects = [ - { - "slice_index": "0", - "object_label": str(label), - "pixel_count": str(area), - "calibration_um": "1.3556", - } - for label, area in ((1, 64), (2, 256)) - ] + objects = [{"slice_index": "0", "object_label": str(label), "pixel_count": str(area), + "calibration_um": "1.3556"} for label, area in ((1, 64), (2, 256))] aggregate = {"object_label": "", "object_name": "", "slice_index": "0"} for name in QUADRANTS: aggregate[f"Classify_{name}_NumObjectsPerBin"] = str(int(name in QUADRANTS[:2])) - aggregate[f"Classify_{name}_PctObjectsPerBin"] = str( - 50 * int(name in QUADRANTS[:2]) - ) - classes = [aggregate] + [ - { - "slice_index": "0", - "object_name": "engineering_cells", - "object_label": str(label), - **{f"Classify_{name}": str(int(name == selected)) for name in QUADRANTS}, - } - for label, selected in ((1, "low_low"), (2, "high_low")) - ] + aggregate[f"Classify_{name}_PctObjectsPerBin"] = str(50 * int(name in QUADRANTS[:2])) + classes = [aggregate] + [{"slice_index": "0", "object_name": "engineering_cells", "object_label": str(label), + **{f"Classify_{name}": str(int(name == selected)) for name in QUADRANTS}} + for label, selected in ((1, "low_low"), (2, "high_low"))] assert_measurement_rows(objects, classes) bad_labels = copy.deepcopy(labels) bad_labels[8][8] = 2 @@ -942,23 +505,19 @@ def offline_check(): bad_classes[1]["Classify_low_high"] = "1" wrong_calibration = copy.deepcopy(objects) wrong_calibration[1]["calibration_um"] = "256" - probes = [ - lambda: assert_label_identity(bad_labels, (64, 64)), - lambda: assert_label_identity(labels, (64, 64, 3)), - lambda: assert_label_identity(labels[:32], (64, 64)), - lambda: assert_measurement_rows(objects, bad_classes), - lambda: assert_measurement_rows(wrong_calibration, classes), - lambda: assert_measurement_rows(objects, classes[:2]), - ] + probes = [lambda: assert_label_identity(bad_labels, (64, 64)), + lambda: assert_label_identity(labels, (64, 64, 3)), + lambda: assert_label_identity(labels[:32], (64, 64)), + lambda: assert_measurement_rows(objects, bad_classes), + lambda: assert_measurement_rows(wrong_calibration, classes), + lambda: assert_measurement_rows(objects, classes[:2])] for probe in probes: try: probe() except AssertionError: continue raise AssertionError("Oracle accepted a counterexample") - print( - "OFFLINE ONLY: label/row positives + 6 rejection oracles; no MCP/native/fixture invocation" - ) + print("OFFLINE ONLY: label/row positives + 6 rejection oracles; no MCP/native/fixture invocation") def main(): @@ -969,15 +528,9 @@ def main(): parser.add_argument("--output-suffix", default="_classification388_public_pair") parser.add_argument("--runtime-port", type=int, default=5993) parser.add_argument("--viewer-port", type=int, default=5992) - parser.add_argument( - "--runtime-handle", - type=Path, - help="Parent's exact retained bootstrap handle JSON; observe/reuse, never startup or close that owner's processes", - ) - parser.add_argument( - "--fixture-function-id", - help="Explicit original registered fixture identity; verify original bytes and never register again", - ) + parser.add_argument("--runtime-handle", type=Path, + help="Parent's exact retained bootstrap handle JSON; observe/reuse, never startup or close that owner's processes") + parser.add_argument("--fixture-function-id", help="Explicit original registered fixture identity; verify original bytes and never register again") args = parser.parse_args() if args.run: run(args) From e8229adfa73479061c6f7ba77651247e38517f48 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 04:15:44 -0400 Subject: [PATCH 39/84] B6: colocalization test parses on Python 3.11 (no outer-quote reuse in f-strings) Co-Authored-By: Claude Opus 5.5 --- tests/unit/test_colocalization_metric_setting_authority.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/unit/test_colocalization_metric_setting_authority.py b/tests/unit/test_colocalization_metric_setting_authority.py index 59d99fede6..e5ae55a195 100644 --- a/tests/unit/test_colocalization_metric_setting_authority.py +++ b/tests/unit/test_colocalization_metric_setting_authority.py @@ -153,7 +153,7 @@ def test_recorded_source_pair_features_exclude_disabled_metrics(disabled, famili for name in enabled.columns if not any( name.startswith( - f"Correlation_{(family if family == "RWC" else family.title())}_" + f"Correlation_{(family if family == 'RWC' else family.title())}_" ) for family in families ) From f2ae23c97e9a40b90b5b5f77ea487bedea0ad70c Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 04:15:50 -0400 Subject: [PATCH 40/84] B6: black over the Python files this PR changes Co-Authored-By: Claude Opus 5.5 --- benchmark/openhcs_measured_run.py | 4 +- openhcs/agent/capabilities.py | 37 +- openhcs/core/source_bindings.py | 87 +-- openhcs/core/source_image_provenance.py | 33 +- openhcs/core/steps/function_runtime.py | 53 +- openhcs/mcp/execution.py | 8 +- .../backends/enhance/basic_processor_jax.py | 1 - .../backends/lib_registry/openhcs_registry.py | 10 +- .../processing/custom_functions/manager.py | 25 +- openhcs/pyqt_gui/main.py | 4 +- scripts/smoke_installed_mcp.py | 9 +- .../test_cellprofiler_official30_zmq.py | 3 +- .../test_custom_source_transport.py | 84 ++- .../integration/test_measured_cli_session.py | 8 +- .../test_neurite_outgrowth_synthetic_plate.py | 11 +- tests/integration/test_session_journey.py | 10 +- .../test_end_to_end_workflow_foundation.py | 8 +- tests/pyqt_gui/test_session_gui_journey.py | 11 +- tests/unit/agent/test_capabilities.py | 57 +- .../test_mcp_dev_client_sampling_results.py | 289 +++++++-- .../agent/test_mcp_dev_client_ui_renderers.py | 164 ++++-- .../agent/test_mcp_plate_stream_progress.py | 148 +++-- .../test_mcp_process_resource_lifetime.py | 54 +- .../test_selected_plane_reopen_stream.py | 5 +- tests/unit/test_aggregate_source_scope.py | 211 +++++-- ...utomatic_image_role_materialization_435.py | 555 ++++++++++++++---- .../unit/test_derived_role_publication_435.py | 434 ++++++++++---- .../test_gray_to_color_binding_axis_432.py | 159 +++-- ..._gray_to_color_payload_axis_compile_432.py | 230 ++++++-- tests/unit/test_library_registry_discovery.py | 6 +- .../unit/test_named_checkpoint_publication.py | 105 +++- tests/unit/test_openhcs_adapter.py | 6 +- .../test_rescale_stream_plane_declaration.py | 192 ++++-- ...t_selected_plane_stream_materialization.py | 61 +- tests/unit/test_smoke_installed_gui.py | 4 +- tests/unit/test_snapshot_deadline.py | 24 +- tests/unit/test_viewer_spatial_sample.py | 4 +- 37 files changed, 2286 insertions(+), 828 deletions(-) diff --git a/benchmark/openhcs_measured_run.py b/benchmark/openhcs_measured_run.py index d13287a6d7..c472c90ded 100644 --- a/benchmark/openhcs_measured_run.py +++ b/benchmark/openhcs_measured_run.py @@ -370,9 +370,7 @@ def retain_measured_openhcs_completion( if expected_axis_count is not None and expected_axis_count < 1: raise ValueError("Expected axis count must be positive when declared.") - auxiliary_params = ZMQAuxiliaryExecutionParams.from_transport( - request.config_params - ) + auxiliary_params = ZMQAuxiliaryExecutionParams.from_transport(request.config_params) observation_export_path = auxiliary_params.runtime_observation_export_path if observation_export_path is None: raise ValueError("Measured OpenHCS runs require runtime observation export.") diff --git a/openhcs/agent/capabilities.py b/openhcs/agent/capabilities.py index 18482e2405..5cf4ac9586 100644 --- a/openhcs/agent/capabilities.py +++ b/openhcs/agent/capabilities.py @@ -461,7 +461,8 @@ class NonExpertCapabilitySurfaceMixin: def includes(self, capability: type["AgentCapabilityDeclaration"]) -> bool: return ( capability.exposition.visibility is not CapabilityVisibility.EXPERT - and capability.exposition.role not in (CapabilityRole.EXPERT, CapabilityRole.FALLBACK) + and capability.exposition.role + not in (CapabilityRole.EXPERT, CapabilityRole.FALLBACK) and super().includes(capability) ) @@ -472,8 +473,9 @@ class WorkflowGroupCapabilitySurfaceMixin: workflow_groups: ClassVar[frozenset[CapabilityWorkflowGroup]] def includes(self, capability: type["AgentCapabilityDeclaration"]) -> bool: - return capability.exposition.workflow_group in self.workflow_groups and super().includes( - capability + return ( + capability.exposition.workflow_group in self.workflow_groups + and super().includes(capability) ) @@ -654,7 +656,9 @@ def require_agent_type_contract(contract: AgentContract | None) -> type: def _keyword_parameter(name: str, annotation: object, default: object) -> Parameter: - return Parameter(name, Parameter.KEYWORD_ONLY, default=default, annotation=annotation) + return Parameter( + name, Parameter.KEYWORD_ONLY, default=default, annotation=annotation + ) class AgentCapabilityInvocation(ABC): @@ -786,7 +790,6 @@ class AgentResourceInvocationMixin: kind: ClassVar[CapabilityKind] = CapabilityKind.RESOURCE - def bind_mcp( self, declaration: type[AgentCapabilityDeclaration], @@ -1036,8 +1039,12 @@ def tool_parameters(self, declaration, binder): ) def invoke(self, declaration, binder, arguments): - return super().invoke(declaration, binder, arguments).as_jsonable( - compact_actions=bool(arguments["compact_actions"]), + return ( + super() + .invoke(declaration, binder, arguments) + .as_jsonable( + compact_actions=bool(arguments["compact_actions"]), + ) ) @@ -1152,8 +1159,6 @@ class AgentConfigPatchServiceInvocation( __slots__ = () - - class AgentConnectionRequestServiceInvocation( AgentUiConnectionMixin, AgentFromFieldsInputMixin, @@ -1162,7 +1167,6 @@ class AgentConnectionRequestServiceInvocation( __slots__ = () - class AgentViewerWindowConnectionServiceInvocation( AgentViewerWindowConnectionMixin, AgentServiceInvocation, @@ -1195,6 +1199,7 @@ def invoke(self, declaration, binder, arguments): *self.call_arguments(declaration, binder, arguments), ) + @dataclass(frozen=True, slots=True) class AgentCapabilitySearchRequest: """Bounded registry query over declaration-owned capability metadata.""" @@ -2162,7 +2167,9 @@ class ObserveCustomFunctionRegistrationCapability(FunctionCatalogCapability): output_contract = CustomFunctionRegistrationObservation invocation = AgentDataclassRequestServiceInvocation( service=lambda context: context.function_catalog, - method=lambda service, request: service.observe_custom_function_registration(request), + method=lambda service, request: service.observe_custom_function_registration( + request + ), ) @@ -2381,7 +2388,9 @@ class SamplePlateImageCapability(MainThreadProgressCapability, PlatePathCapabili ) -class StreamPlateFilesToViewerCapability(MainThreadProgressCapability, PlatePathCapability): +class StreamPlateFilesToViewerCapability( + MainThreadProgressCapability, PlatePathCapability +): name = "openhcs_stream_plate_files_to_viewer" cli_command = "stream-plate-files" title = "Stream plate files to viewer" @@ -2510,7 +2519,9 @@ class UiSampleSelectedPlateImageCapability(UiSelectedPlateCapability): ) -class UiStreamSelectedPlateFilesToViewerCapability(MainThreadProgressCapability, UiSelectedPlateCapability): +class UiStreamSelectedPlateFilesToViewerCapability( + MainThreadProgressCapability, UiSelectedPlateCapability +): name = "openhcs_ui_stream_selected_plate_files_to_viewer" cli_command = "selected-plate-stream" title = "Stream selected plate files to viewer" diff --git a/openhcs/core/source_bindings.py b/openhcs/core/source_bindings.py index e688e378b0..b07a08bf36 100644 --- a/openhcs/core/source_bindings.py +++ b/openhcs/core/source_bindings.py @@ -644,7 +644,9 @@ class SourceSelector: This filters candidates; it does not assign the output identity. """ - metadata: Annotated[tuple[MetadataSelector, ...], FieldLabel("Select Metadata")] = () + metadata: Annotated[ + tuple[MetadataSelector, ...], FieldLabel("Select Metadata") + ] = () """Exact metadata field/value pairs required on selected sources, e.g. Well=A01. This filters candidates; pairing sources into sets uses the match plan. @@ -707,12 +709,15 @@ def metadata_candidates_match( any( source_metadata_values_equal(value, selector.value) for metadata in candidates - for value in source_component_metadata_values(metadata, selector.component) + for value in source_component_metadata_values( + metadata, selector.component + ) ) for selector in self.components ) and all( any( - value is not None and source_metadata_values_equal(value, selector.value) + value is not None + and source_metadata_values_equal(value, selector.value) for metadata in candidates for value in (semantic_source_metadata_value(metadata, selector.field),) ) @@ -723,12 +728,12 @@ def matches_provenance(self, provenance: SourceImageProvenance) -> bool: """Apply the same selectors to identities carried by current pixels.""" identities = provenance.represented_source_identities return ( - (not self.filters or self.path_filters_match( + not self.filters + or self.path_filters_match( tuple(path for identity in identities for path in identity.filter_paths) - )) - and self.metadata_candidates_match( - tuple(identity.component_metadata or {} for identity in identities) ) + ) and self.metadata_candidates_match( + tuple(identity.component_metadata or {} for identity in identities) ) @@ -1142,24 +1147,32 @@ def project_step_input_payload(self, payload: RuntimeArrayData) -> RuntimeArrayD metadata = payload.metadata provenance = metadata.source_provenance if not provenance.has_values: - raise ValueError(f"STEP_INPUT binding {self.alias!r} requires source provenance.") + raise ValueError( + f"STEP_INPUT binding {self.alias!r} requires source provenance." + ) if not provenance.source_plane_count: if not self.selector.matches_provenance(provenance): - raise ValueError(f"STEP_INPUT binding {self.alias!r} selects no current planes.") + raise ValueError( + f"STEP_INPUT binding {self.alias!r} selects no current planes." + ) selected = payload else: named_planes = provenance.source_plane_selection(self.alias) candidate_planes = ( range(provenance.source_plane_count) - if named_planes is None else named_planes + if named_planes is None + else named_planes ) selection = tuple( - index for index in candidate_planes + index + for index in candidate_planes if self.selector.matches_provenance(provenance.for_source_plane(index)) ) if not selection: - raise ValueError(f"STEP_INPUT binding {self.alias!r} selects no current planes.") + raise ValueError( + f"STEP_INPUT binding {self.alias!r} selects no current planes." + ) selected = metadata.project_source_planes(payload, selection) return self.apply_loaded_payload(selected, source_context=None) @@ -1180,11 +1193,10 @@ def _monochrome_source_data( """Convert declared RGB channels using the source intensity domain.""" import numpy as np - - normalized = metadata.payload_with(data).normalize_intensity_payload(dtype=np.float32,) - rgb = np.moveaxis( - np.asarray(normalized.data), channel_axis, -1 - )[..., :3] + normalized = metadata.payload_with(data).normalize_intensity_payload( + dtype=np.float32, + ) + rgb = np.moveaxis(np.asarray(normalized.data), channel_axis, -1)[..., :3] if np.all(rgb == rgb[..., :1]): return np.ascontiguousarray(rgb[..., 0]) @@ -1216,9 +1228,7 @@ def component_values( values.extend(source_component_metadata_values(metadata, component)) return tuple(dict.fromkeys(values)) - def _declared_component_values( - self, component: type[Axis] - ) -> tuple[str, ...]: + def _declared_component_values(self, component: type[Axis]) -> tuple[str, ...]: """Resolve exact identity values before selector-derived coordinates.""" for declarations in (self.component_identity, self.selector.components): values = tuple( @@ -1242,7 +1252,9 @@ def component_domains( for values in (self._declared_component_values(component),) if values } - if realized_source_metadata is None or len(domains) == len(AxisFamily.active().axes): + if realized_source_metadata is None or len(domains) == len( + AxisFamily.active().axes + ): return domains realized: dict[type[Axis], list[str]] = {} for metadata in realized_source_metadata: @@ -2318,7 +2330,8 @@ def from_config( return cls( bindings=( config.binding_declarations - if selected_bindings is None else selected_bindings + if selected_bindings is None + else selected_bindings ), metadata_rules=config.metadata_rule_declarations, match_plan=config.match_plan, @@ -2345,18 +2358,23 @@ def from_contracts( and any(contract.accepts_implicit_main_flow_input for contract in contracts) else () ) - source_specs = tuple(dict.fromkeys(( - *implicit_specs, - *( - spec - for contract in contracts - for spec in contract.artifact_inputs - if config.declares_artifact_ref(spec.ref()) - or available_artifacts.by_name_and_artifact_type( - spec.name, spec.artifact_type - ) is not None - ), - ))) + source_specs = tuple( + dict.fromkeys( + ( + *implicit_specs, + *( + spec + for contract in contracts + for spec in contract.artifact_inputs + if config.declares_artifact_ref(spec.ref()) + or available_artifacts.by_name_and_artifact_type( + spec.name, spec.artifact_type + ) + is not None + ), + ) + ) + ) if not source_specs: return cls.empty() bindings = config.bindings_for_artifact_specs(source_specs, available_artifacts) @@ -2531,7 +2549,6 @@ def _coerce_component(value: Any, field_name: str) -> type[Axis]: raise TypeError(f"{field_name}: {error}") from error - def _require_name(value: str, field_name: str) -> None: if not value: raise ValueError(f"{field_name} cannot be empty.") diff --git a/openhcs/core/source_image_provenance.py b/openhcs/core/source_image_provenance.py index 3db63f50f2..f063cf90fa 100644 --- a/openhcs/core/source_image_provenance.py +++ b/openhcs/core/source_image_provenance.py @@ -250,7 +250,11 @@ def from_mapping( raise ValueError(f"Unknown provenance plane fields: {sorted(unknown)!r}.") kind = values.get( "identity_kind", - RuntimeSourceImageProvenancePlane.identity_kind if identities is None else None, + ( + RuntimeSourceImageProvenancePlane.identity_kind + if identities is None + else None + ), ) if not isinstance(kind, str) or kind not in cls.__registry__: raise ValueError(f"Unknown source-image provenance identity kind {kind!r}.") @@ -265,7 +269,9 @@ def from_mapping( raise TypeError("Source provenance contributors must be a sequence.") if identities is None: facts = { - key: values[key] for key in ("path", "component_metadata") if key in values + key: values[key] + for key in ("path", "component_metadata") + if key in values } else: index = values.get("identity") @@ -548,11 +554,14 @@ def from_mapping(cls, values: object) -> "SourceImageProvenancePlanes": else: identities = None planes = values - if not isinstance(planes, Sequence) or isinstance(planes, (str, bytes, bytearray)): + if not isinstance(planes, Sequence) or isinstance( + planes, (str, bytes, bytearray) + ): raise TypeError("Source provenance planes must be a sequence.") return cls( tuple( - SourceImageProvenancePlane.from_mapping(item, identities) for item in planes + SourceImageProvenancePlane.from_mapping(item, identities) + for item in planes ) ) @@ -1079,11 +1088,15 @@ def required_scalar_components( ): continue values = frozenset( - value for values in contributor_components for value in values[component] + value + for values in contributor_components + for value in values[component] ) if len(values) > 1: reduced.add(component.name) - return tuple(component for component in component_order if component not in reduced) + return tuple( + component for component in component_order if component not in reduced + ) def image_set_identities( self, @@ -1108,11 +1121,9 @@ def image_set_plane_identities( ) -> tuple[frozenset[SourceImageSetIdentity], ...]: """Return one image-set identity entry for every runtime source plane.""" return tuple( - self.source_image_provenance_planes.plane( - index - ).source_identity.with_missing_from( - self.source_identity - ).image_set_identities(policy) + self.source_image_provenance_planes.plane(index) + .source_identity.with_missing_from(self.source_identity) + .image_set_identities(policy) for index in range(self.source_plane_count) ) diff --git a/openhcs/core/steps/function_runtime.py b/openhcs/core/steps/function_runtime.py index cc2c741d64..176e27a46b 100644 --- a/openhcs/core/steps/function_runtime.py +++ b/openhcs/core/steps/function_runtime.py @@ -264,7 +264,10 @@ def source_binding_plan(self) -> CompiledSourceBindingPlan: source_refs = self.invocation_source_artifact_refs if source_refs: return declared_plan.for_artifact_refs(source_refs) - if self.compiled_group.runtime_domain is RuntimeInvocationDomain.ARTIFACT_MANAGED: + if ( + self.compiled_group.runtime_domain + is RuntimeInvocationDomain.ARTIFACT_MANAGED + ): # The compiler has already traced stored artifact lineage to its # source bindings. A produced name is not itself a raw alias. return declared_plan @@ -344,7 +347,9 @@ def loaded_fixed_component_values( payload: RuntimeArrayData, ) -> RuntimeFixedComponentValues: """Capture the source-selected cohort's common fixed coordinates.""" - source_provenance = payload.metadata.source_provenance.with_common_scalar_identity_from_planes() + source_provenance = ( + payload.metadata.source_provenance.with_common_scalar_identity_from_planes() + ) common_source_metadata = source_provenance.source_component_metadata or {} variable_components = ComponentSet.of( self.execution_plan.variable_components or () @@ -413,7 +418,9 @@ def _input_relative_path(cls, input_dir: Path, matched_path: str) -> Path: def run(self) -> None: start_time = time.time() if logger.isEnabledFor(logging.DEBUG) else None plan = self.execution_plan - logger.debug("Processing pattern %s for axis %s", self.pattern_repr, plan.axis_id) + logger.debug( + "Processing pattern %s for axis %s", self.pattern_repr, plan.axis_id + ) try: load_started_at = time.perf_counter() @@ -501,7 +508,9 @@ def run(self) -> None: ) except Exception as e: logger.error( - "Error processing pattern group %s: %s", self.pattern_repr, e, + "Error processing pattern group %s: %s", + self.pattern_repr, + e, exc_info=True, ) raise ValueError( @@ -774,7 +783,6 @@ def _filter_matching_files_for_group( parser, matching_files, group_component, - component_value, ) if not filtered: @@ -1074,7 +1082,9 @@ def _project_output_slices( return tuple(zip(output_payloads, slice_contexts, strict=True)) def _unstack_main_output( - self, processed_data: Any, expected_count: int, + self, + processed_data: Any, + expected_count: int, ) -> tuple[Any, ...]: """Split undeclared main-output pixels along their leading runtime-slice axis.""" try: @@ -1235,7 +1245,9 @@ def _save_outputs( output_identity, output_context=output_context, image_metadata=output_metadata, - main_flow_plane_axis=processed_stack.plane_axis_for_output_context(output_context), + main_flow_plane_axis=processed_stack.plane_axis_for_output_context( + output_context + ), ) if output_directory_exists and context.filemanager.exists( @@ -1595,10 +1607,6 @@ def _save_artifact_value( return runtime_value.data - - - - @dataclass(frozen=True, slots=True) class FunctionCoreExecutor: """Execute one scoped callable invocation and route declared artifact I/O.""" @@ -1721,21 +1729,24 @@ def loaded_source_artifact_payload( or request.artifact_inputs.get(edge.key) is not edge or edge.storage_plan is not None or edge.main_flow_projection is None - or self.group_data.main_flow_source_binding_plan.binding_for_artifact_ref(ref) + or self.group_data.main_flow_source_binding_plan.binding_for_artifact_ref( + ref + ) != binding ): return None payload = self.invocation.convert_input( - self.group_data.main_data_stack, self.source_memory_type, + self.group_data.main_data_stack, + self.source_memory_type, ) if edge.main_flow_projection is MainFlowInputProjection.DECLARED_SOURCE_IMAGE: payload = payload.project_declared_source(ref.name) elif edge.main_flow_projection is not MainFlowInputProjection.COMPLETE_PAYLOAD: return None metadata = payload.metadata - if ( - not metadata.persists_whole_image() - and metadata.plane_axis not in (None, RuntimePlaneAxis.RUNTIME_SLICE) + if not metadata.persists_whole_image() and metadata.plane_axis not in ( + None, + RuntimePlaneAxis.RUNTIME_SLICE, ): # A remaining binding axis does not prove this source's ordered # physical members. Resolve it through its declared origin owner. @@ -1747,7 +1758,9 @@ def loaded_source_artifact_payload( member_count=member_count if member_count is not None else 1, ) projection = RuntimePlaneAxisValueProjection.from_projector( - request.plane_projection, RuntimePlaneAxis.RUNTIME_SLICE, (), + request.plane_projection, + RuntimePlaneAxis.RUNTIME_SLICE, + (), ) if projection is not None and projection.plane_index is not None: payload = RuntimeSliceProjection.value_for_slice(payload, projection) @@ -1770,7 +1783,8 @@ def declared_source_payload( stored_payload = loaded_artifact_payloads.get(source_ref) source_binding_plan = ( self.group_data.source_binding_plan - if request is None else request.source_binding_plan + if request is None + else request.source_binding_plan ) source_binding = source_binding_plan.binding_for_artifact_ref(source_ref) main_flow_edges = tuple( @@ -1801,7 +1815,8 @@ def declared_source_payload( RuntimePayload, ( self.runtime_adapter_request(primary_source_payload) - if request is None else request + if request is None + else request ).source_artifact_payload(source_ref), ) if len(main_flow_edges) != 1: diff --git a/openhcs/mcp/execution.py b/openhcs/mcp/execution.py index aff5e5d721..a90b3a6428 100644 --- a/openhcs/mcp/execution.py +++ b/openhcs/mcp/execution.py @@ -85,7 +85,9 @@ class McpTransportExecutor(AsyncOperationExecutor): def __init__(self) -> None: if threading.current_thread() is not threading.main_thread(): - raise RuntimeError("MCP transport must be started on the process main thread.") + raise RuntimeError( + "MCP transport must be started on the process main thread." + ) existing_application = QCoreApplication.instance() # A process holds one Qt application; one this transport created is # its own and goes with it, so a later GUI can create a QApplication. @@ -124,7 +126,9 @@ def close(self) -> None: leaving it to interpreter finalization is not an MCP lifecycle. """ if threading.current_thread() is not threading.main_thread(): - raise RuntimeError("MCP transport must be closed on the process main thread.") + raise RuntimeError( + "MCP transport must be closed on the process main thread." + ) if self._closed: return try: diff --git a/openhcs/processing/backends/enhance/basic_processor_jax.py b/openhcs/processing/backends/enhance/basic_processor_jax.py index 148d5fd86e..0640f7ec21 100644 --- a/openhcs/processing/backends/enhance/basic_processor_jax.py +++ b/openhcs/processing/backends/enhance/basic_processor_jax.py @@ -61,7 +61,6 @@ def _fitted_field_output(name: str) -> ArtifactSpec: @numpy_func(contract=Pure3DContract, dtype_config_default=DtypeConfig()) @allowed_group_by_roles(ColourAxis) @required_axis_roles(FittedIlluminationFieldOutput.observation_role) - @artifact_outputs(CORRECTED_OUTPUT, FLATFIELD_OUTPUT, DARKFIELD_OUTPUT) def basic_flatfield_correction_jax( image: np.ndarray, diff --git a/openhcs/processing/backends/lib_registry/openhcs_registry.py b/openhcs/processing/backends/lib_registry/openhcs_registry.py index 86803ca427..9edb85602f 100644 --- a/openhcs/processing/backends/lib_registry/openhcs_registry.py +++ b/openhcs/processing/backends/lib_registry/openhcs_registry.py @@ -605,9 +605,7 @@ def get_modules_to_scan(self) -> List[Tuple[str, Any]]: module_name, ) continue - excluded = _module_distributions_excluded_by_package_metadata( - module_name - ) + excluded = _module_distributions_excluded_by_package_metadata(module_name) if excluded: logger.info( "Skipping OpenHCS module %s - package requirements exclude %s " @@ -944,9 +942,11 @@ def reconstruct_cached_callable( """Reconstruct cached metadata through the declaration's one wrapper.""" return _registered_wrapper( - self, func, lambda: super(OpenHCSRegistry, self).reconstruct_cached_callable( + self, + func, + lambda: super(OpenHCSRegistry, self).reconstruct_cached_callable( func, contract - ) + ), ) def _processing_contract_for_function( diff --git a/openhcs/processing/custom_functions/manager.py b/openhcs/processing/custom_functions/manager.py index eec75f2639..c607ca8c36 100644 --- a/openhcs/processing/custom_functions/manager.py +++ b/openhcs/processing/custom_functions/manager.py @@ -113,7 +113,9 @@ def default_storage_directory(cls) -> Path: def source_path_for_name(storage_dir: Path, function_name: str) -> Path: """Own source naming for both admission and actual persistence.""" if not function_name.isidentifier() or function_name.startswith("_"): - raise ValidationError("Custom function name must be a public Python identifier.") + raise ValidationError( + "Custom function name must be a public Python identifier." + ) return storage_dir / f"{function_name}.py" def __init__(self, *, create_storage: bool = True): @@ -154,11 +156,18 @@ def register_from_code( """ if persist and write_admission is not None: if expected_function_name is None: - raise ValueError("Write admission requires an explicit custom function name.") + raise ValueError( + "Write admission requires an explicit custom function name." + ) write_admission(self.storage_dir) - write_admission(self.source_path_for_name(self.storage_dir, expected_function_name)) + write_admission( + self.source_path_for_name(self.storage_dir, expected_function_name) + ) metadata = self._prepare_source(code) - if expected_function_name is not None and metadata.original_name != expected_function_name: + if ( + expected_function_name is not None + and metadata.original_name != expected_function_name + ): raise ValidationError( f"Prepared declaration {metadata.original_name!r} does not match " f"admitted function {expected_function_name!r}." @@ -207,7 +216,9 @@ def source_revision(self) -> CustomFunctionSourceRevision: sources = tuple( CustomFunctionSource( function_name=source_path.stem, - content_sha256=CustomFunctionSource.content_digest(source_path.read_bytes()), + content_sha256=CustomFunctionSource.content_digest( + source_path.read_bytes() + ), ) for source_path in sorted(self.storage_dir.glob("*.py")) ) @@ -489,7 +500,9 @@ def _prepare_source( ) -> "FunctionMetadata": """Validate and project one source without mutating runtime or disk state.""" - from openhcs.processing.backends.lib_registry.openhcs_registry import OpenHCSRegistry + from openhcs.processing.backends.lib_registry.openhcs_registry import ( + OpenHCSRegistry, + ) validation_result = validate_code(code) if not validation_result.is_valid: diff --git a/openhcs/pyqt_gui/main.py b/openhcs/pyqt_gui/main.py index 24b602a83a..4c56133572 100644 --- a/openhcs/pyqt_gui/main.py +++ b/openhcs/pyqt_gui/main.py @@ -1311,7 +1311,9 @@ def _on_synthetic_plate_generated(self, output_dir: str, pipeline_path: str): self.show_pipeline_editor() for scope_id in self.session.add_dataset_roots((output_dir,)): self.session.load_pipeline_file(scope_id, pipeline_path) - logger.info("Added synthetic dataset and loaded its test pipeline: %s", output_dir) + logger.info( + "Added synthetic dataset and loaded its test pipeline: %s", output_dir + ) def _on_consolidate_results(self): """Open file dialog to select results directory and consolidate analysis results.""" diff --git a/scripts/smoke_installed_mcp.py b/scripts/smoke_installed_mcp.py index 74ef95ebe5..06c48c5286 100644 --- a/scripts/smoke_installed_mcp.py +++ b/scripts/smoke_installed_mcp.py @@ -302,6 +302,7 @@ async def _run_measured_execution_protocol_smoke( pipeline_source = PipelineDocumentCodec.render(document) source_file = output_dir / "pipeline.py" source_file.write_text(pipeline_source, encoding="utf-8") + async def call(name: str, arguments: dict, timeout: float = 90) -> dict: return _tool_payload( await asyncio.wait_for(session.call_tool(name, arguments), timeout=timeout) @@ -330,7 +331,9 @@ async def call(name: str, arguments: dict, timeout: float = 90) -> dict: "openhcs_run_datasets", { "runtime_observation_export_path": str( - MeasuredPipelineRunArtifact.RUNTIME_OBSERVATION.path_in(evidence_dir) + MeasuredPipelineRunArtifact.RUNTIME_OBSERVATION.path_in( + evidence_dir + ) ), "runtime_observation_export_scope": "outcomes", }, @@ -340,7 +343,9 @@ async def call(name: str, arguments: dict, timeout: float = 90) -> dict: ): started = await call(tool, {"scope_ids": [scope_id], **arguments}) if started.get("status") != "accepted": - raise AssertionError(f"Installed MCP rejected {tool}: {started} {connected}") + raise AssertionError( + f"Installed MCP rejected {tool}: {started} {connected}" + ) sequence = started["event_sequence"] deadline = time.monotonic() + 180 while True: diff --git a/tests/integration/test_cellprofiler_official30_zmq.py b/tests/integration/test_cellprofiler_official30_zmq.py index 7f07bf67a5..2b9d024417 100644 --- a/tests/integration/test_cellprofiler_official30_zmq.py +++ b/tests/integration/test_cellprofiler_official30_zmq.py @@ -570,8 +570,7 @@ def test_official30_fiji_variants_project_registered_viewer_configs() -> None: for viewers in _OFFICIAL30_FIJI_VIEWER_VARIANTS: ports_by_viewer = { - viewer: 20_000 + index * 100 - for index, viewer in enumerate(viewers) + viewer: 20_000 + index * 100 for index, viewer in enumerate(viewers) } configs = _registered_streaming_config_kwargs( viewers, diff --git a/tests/integration/test_custom_source_transport.py b/tests/integration/test_custom_source_transport.py index 3d6f005222..fdf36c42d5 100644 --- a/tests/integration/test_custom_source_transport.py +++ b/tests/integration/test_custom_source_transport.py @@ -30,6 +30,7 @@ if dependency_root: sys.path.insert(0, str(WORKTREE)) from openhcs._source_dependencies import ensure_source_checkout_external_paths + ensure_source_checkout_external_paths(Path(dependency_root)) @@ -84,9 +85,9 @@ def _run_child(env, *arguments): while True: remaining = deadline - time.monotonic() assert remaining > 0, f"Child exceeded 30s: {arguments!r}" - assert os.fstat(output_file.fileno()).st_size < OUTPUT_LIMIT, ( - f"Child output exceeded {OUTPUT_LIMIT} bytes: {arguments!r}" - ) + assert ( + os.fstat(output_file.fileno()).st_size < OUTPUT_LIMIT + ), f"Child output exceeded {OUTPUT_LIMIT} bytes: {arguments!r}" try: returncode = process.wait(timeout=min(remaining, 0.1)) except subprocess.TimeoutExpired: @@ -94,9 +95,9 @@ def _run_child(env, *arguments): break output_file.seek(0) captured.extend(output_file.read(OUTPUT_LIMIT)) - assert len(captured) < OUTPUT_LIMIT, ( - f"Child output exceeded {OUTPUT_LIMIT} bytes: {arguments!r}" - ) + assert ( + len(captured) < OUTPUT_LIMIT + ), f"Child output exceeded {OUTPUT_LIMIT} bytes: {arguments!r}" finally: if process.poll() is None: process.kill() @@ -417,8 +418,13 @@ def _spawned_custom_task(payload, execution_id, subject_id): owner = reference.metadata.artifact_outputs[0].measurement_feature_owner results.append(owner.make_row(2.5)) ProgressExecutionContext(execution_id, subject_id).emit( - axis_id="synthetic", step_name="custom", phase=ProgressPhase.STEP_COMPLETED, - status=ProgressStatus.SUCCESS, percent=100, completed=1, total=1, + axis_id="synthetic", + step_name="custom", + phase=ProgressPhase.STEP_COMPLETED, + status=ProgressStatus.SUCCESS, + percent=100, + completed=1, + total=1, ) return os.getpid(), tuple(results) @@ -448,7 +454,8 @@ def _compiled_custom_contexts(payload, runtime): from openhcs.core.compiled_step_plan import CompiledStepPlan from openhcs.core.context.processing_context import ProcessingContext from openhcs.core.function_patterns import ( - compile_function_pattern, normalize_function_pattern, + compile_function_pattern, + normalize_function_pattern, ) from openhcs.core.function_step_transport import FunctionStepTransportAuthority @@ -464,11 +471,14 @@ def _compiled_custom_contexts(payload, runtime): assert callable(invocation.contract.raw_processing_function) prepared = invocation.contract.resolve_runtime_callable() plan = CompiledStepPlan( - step_index=0, step_name="Synthetic custom", - axis_id=axis_id, compiled_function_pattern=pattern, + step_index=0, + step_name="Synthetic custom", + axis_id=axis_id, + compiled_function_pattern=pattern, ) context = ProcessingContext( - axis_id=axis_id, step_plans={0: plan}, + axis_id=axis_id, + step_plans={0: plan}, filemanager=FileManager({Backend.MEMORY.value: MemoryStorageBackend()}), ) context.freeze() @@ -480,11 +490,15 @@ def _compiled_custom_contexts(payload, runtime): assert transported.contract is not invocation.contract assert invocation.contract.resolve_runtime_callable() is prepared assert callable(invocation.contract.raw_processing_function) - assert FunctionStepTransportAuthority.normalize_callable_contract( - transported.contract - ) is transported.contract + assert ( + FunctionStepTransportAuthority.normalize_callable_contract( + transported.contract + ) + is transported.contract + ) return CompiledExecutionBundle.from_runtime_contexts( - pipeline_definition=(), runtime_contexts=contexts, + pipeline_definition=(), + runtime_contexts=contexts, worker_assignments={f"worker_{i}": [axis] for i, axis in enumerate(contexts)}, runtime_environment=runtime, ).transport_contexts @@ -493,7 +507,8 @@ def _compiled_custom_contexts(payload, runtime): def _spawn(path): import multiprocessing from openhcs.core.compiled_execution import ( - CompiledRuntimeEnvironmentPlan, CompiledWorkerStartPlan, + CompiledRuntimeEnvironmentPlan, + CompiledWorkerStartPlan, ) from openhcs.core.config import MultiprocessingStartMethod from openhcs.core.orchestrator.cancellation import ExecutionCancellationSignal @@ -511,19 +526,25 @@ class RichProgressContext(ProgressExecutionContext): payload = _load(path) method = MultiprocessingStartMethod.SPAWN runtime = CompiledRuntimeEnvironmentPlan( - worker_start=CompiledWorkerStartPlan(method, method, "source fixture", False, False), - use_threading=False, configured_num_workers=2, + worker_start=CompiledWorkerStartPlan( + method, method, "source fixture", False, False + ), + use_threading=False, + configured_num_workers=2, ) queue = multiprocessing.get_context(method.value).Queue() resources = WorkerExecutorFactory( - log_file_base=None, progress_queue=queue, + log_file_base=None, + progress_queue=queue, cancellation=ExecutionCancellationSignal(), ).create(runtime_environment=runtime, actual_max_workers=2) contexts = _compiled_custom_contexts(payload, runtime) try: with resources.execution_context(): tasks = tuple( - resources.executor.submit(_spawned_compiled_custom_task, lane_context, helpers) + resources.executor.submit( + _spawned_compiled_custom_task, lane_context, helpers + ) for lane_context, (_reference, helpers) in zip( contexts.values(), payload, strict=True ) @@ -537,7 +558,10 @@ class RichProgressContext(ProgressExecutionContext): assert values == [[0, 1, 2], [3, 4, 5]] assert type(row) is helpers[4] and row == helpers[5] pid, rows = resources.executor.submit( - _spawned_custom_task, payload, context.execution_id, context.subject_id, + _spawned_custom_task, + payload, + context.execution_id, + context.subject_id, ).result(timeout=15) assert pid != os.getpid() for row, (_reference, helpers) in zip(rows, payload, strict=True): @@ -623,10 +647,20 @@ def test_persisted_payload_preserves_nominal_helpers_in_independent_consumer( def test_persisted_custom_revision_executes_in_original_spawn_factory( - tmp_path, transport_environment, + tmp_path, + transport_environment, ): - print(_run_child(transport_environment, "spawn", str(tmp_path / "spawn.pkl"), - "reference", "cold", "unchanged"), end="") + print( + _run_child( + transport_environment, + "spawn", + str(tmp_path / "spawn.pkl"), + "reference", + "cold", + "unchanged", + ), + end="", + ) @pytest.mark.parametrize("mutation", ["changed", "deleted", "renamed"]) diff --git a/tests/integration/test_measured_cli_session.py b/tests/integration/test_measured_cli_session.py index b81290c951..230ff057b5 100644 --- a/tests/integration/test_measured_cli_session.py +++ b/tests/integration/test_measured_cli_session.py @@ -213,7 +213,13 @@ def test_read_only_source_plate_completes_and_stays_byte_identical( port = _port(2) try: completed = subprocess.run( - _command(plate, source, output_dir, port, wait_ms=int(RUN_DEADLINE_SECONDS * 1000)), + _command( + plate, + source, + output_dir, + port, + wait_ms=int(RUN_DEADLINE_SECONDS * 1000), + ), cwd=REPOSITORY_ROOT, env=_environment(tmp_path), capture_output=True, diff --git a/tests/integration/test_neurite_outgrowth_synthetic_plate.py b/tests/integration/test_neurite_outgrowth_synthetic_plate.py index 14e16ffa01..bddccebbff 100644 --- a/tests/integration/test_neurite_outgrowth_synthetic_plate.py +++ b/tests/integration/test_neurite_outgrowth_synthetic_plate.py @@ -221,9 +221,7 @@ def observe_dense_output(output, components): progress_events = [] while True: try: - progress_events.append( - progress_queue.get_nowait() - ) + progress_events.append(progress_queue.get_nowait()) except queue.Empty: break live_payloads = tuple( @@ -315,7 +313,8 @@ def observe_dense_output(output, components): # Checkpoints are the declared outputs between the label artifacts and # the trailing morphology table; the callable's output family owns them. checkpoint_names = tuple( - spec.name for spec in CellProfilerNeuriteEngineProfile.artifact_outputs()[6:-1] + spec.name + for spec in CellProfilerNeuriteEngineProfile.artifact_outputs()[6:-1] ) for artifact_name, expected_checkpoint in zip( checkpoint_names, @@ -331,7 +330,9 @@ def observe_dense_output(output, components): retained, np.asarray(expected_checkpoint)[0], ) - assert len({path.name for path in checkpoint_paths}) == 2 * len(checkpoint_names) + assert len({path.name for path in checkpoint_paths}) == 2 * len( + checkpoint_names + ) assert compiled_plan.output_dataset_root is not None output_dataset_root = Path(compiled_plan.output_dataset_root) diff --git a/tests/integration/test_session_journey.py b/tests/integration/test_session_journey.py index f99d4f83ec..f480cb2fcd 100644 --- a/tests/integration/test_session_journey.py +++ b/tests/integration/test_session_journey.py @@ -145,7 +145,11 @@ def test_headless_mcp_journey_adds_compiles_and_runs_a_dataset(tmp_path: Path) - {"after_sequence": 0, "timeout_seconds": 0.0}, )["events"] } - assert {"DatasetsChanged", "CompiledStateChanged", "DatasetExecutionFinished"} <= kinds + assert { + "DatasetsChanged", + "CompiledStateChanged", + "DatasetExecutionFinished", + } <= kinds finally: context.session.close() @@ -172,7 +176,9 @@ def test_read_only_dataset_initializes_in_a_mirrored_workspace(tmp_path: Path) - workspace_root=workspaces, ) try: - assert session.invoke(AddDatasets, AddDatasets.request(roots=(str(root),))).accepted + assert session.invoke( + AddDatasets, AddDatasets.request(roots=(str(root),)) + ).accepted (scope_id,) = session.dataset_scope_ids() started = session.invoke( InitializeDatasets, InitializeDatasets.request(scope_ids=(scope_id,)) diff --git a/tests/pyqt_gui/integration/test_end_to_end_workflow_foundation.py b/tests/pyqt_gui/integration/test_end_to_end_workflow_foundation.py index a8ed448dc0..d504043025 100644 --- a/tests/pyqt_gui/integration/test_end_to_end_workflow_foundation.py +++ b/tests/pyqt_gui/integration/test_end_to_end_workflow_foundation.py @@ -232,7 +232,9 @@ def materialization_config( ), ), setup_modifications=( - FieldModificationSpec(("step_partition_filter_config", "partition_filter"), 5), + FieldModificationSpec( + ("step_partition_filter_config", "partition_filter"), 5 + ), ), field_to_test=FieldModificationSpec( ("path_planning_config", "partition_filter"), 99 @@ -245,7 +247,9 @@ def materialization_config( ("napari_streaming_config", "partition_filter"), ("fiji_streaming_config", "partition_filter"), ), - excluded_source_paths=(("path_planning_config", "partition_filter"),), + excluded_source_paths=( + ("path_planning_config", "partition_filter"), + ), ), ), ), diff --git a/tests/pyqt_gui/test_session_gui_journey.py b/tests/pyqt_gui/test_session_gui_journey.py index e5d4ce61a4..76c6b25a92 100644 --- a/tests/pyqt_gui/test_session_gui_journey.py +++ b/tests/pyqt_gui/test_session_gui_journey.py @@ -56,11 +56,11 @@ def test_gui_journey_runs_the_same_operations_through_the_widgets( main_thread=DispatcherThread(dispatcher), ) services = GuiServiceStub() - manager = PlateManagerWidget( - services, session, gui_config=get_default_ui_config() - ) + manager = PlateManagerWidget(services, session, gui_config=get_default_ui_config()) try: - assert session.invoke(AddDatasets, AddDatasets.request(roots=(str(root),))).accepted + assert session.invoke( + AddDatasets, AddDatasets.request(roots=(str(root),)) + ).accepted (scope_id,) = session.dataset_scope_ids() session.set_pipeline_source(scope_id, source) application.processEvents() @@ -81,7 +81,8 @@ def wait_until(condition, label) -> None: (CompileDatasets, lambda row: row.compiled and not row.compile_pending), ( RunDatasets, - lambda row: row.terminal_status is not None and not row.execution_active, + lambda row: row.terminal_status is not None + and not row.execution_active, ), ): if operation is CompileDatasets: diff --git a/tests/unit/agent/test_capabilities.py b/tests/unit/agent/test_capabilities.py index b6db329076..6cdc70a27a 100644 --- a/tests/unit/agent/test_capabilities.py +++ b/tests/unit/agent/test_capabilities.py @@ -121,8 +121,7 @@ def test_capability_transport_defaults_to_local_stdio(): def test_capability_exposition_owns_its_json_projection(): capability = next( - capability - for capability in get_capability_registry().capabilities + capability for capability in get_capability_registry().capabilities ) exposition = capability.exposition @@ -294,9 +293,15 @@ def test_custom_function_registration_is_a_desktop_authoring_mutation(): } registration = full_capabilities["openhcs_register_custom_function"] - assert registration.exposition.workflow_group is CapabilityWorkflowGroup.FUNCTION_AUTHORING + assert ( + registration.exposition.workflow_group + is CapabilityWorkflowGroup.FUNCTION_AUTHORING + ) assert registration.exposition.workflow_stage is CapabilityWorkflowStage.AUTHORING - assert registration.exposition.target_context is CapabilityTargetContext.FUNCTION_REGISTRY + assert ( + registration.exposition.target_context + is CapabilityTargetContext.FUNCTION_REGISTRY + ) assert registration.exposition.visibility is CapabilityVisibility.STANDARD assert registration.exposition.role is CapabilityRole.PRIMARY assert registration.mutating is True @@ -344,7 +349,12 @@ def _declaration_error(**attributes) -> Exception: type( "RejectedDeclaration", (UiBridgeCapability,), - {"name": name, "title": "Rejected", "description": "", "service": "test"} + { + "name": name, + "title": "Rejected", + "description": "", + "service": "test", + } | attributes, ) finally: @@ -407,9 +417,9 @@ def test_similar_mcp_tool_names_are_disambiguated_by_target_context_and_role(): assert capabilities["openhcs_sample_plate_image"].exposition.target_context is ( CapabilityTargetContext.PLATE_PATH ) - assert capabilities["openhcs_ui_sample_selected_plate_image"].exposition.target_context is ( - CapabilityTargetContext.UI_SELECTED_PLATE - ) + assert capabilities[ + "openhcs_ui_sample_selected_plate_image" + ].exposition.target_context is (CapabilityTargetContext.UI_SELECTED_PLATE) assert capabilities["openhcs_ui_sample_selected_plate_image"].exposition.role is ( CapabilityRole.MODE_VARIANT ) @@ -420,20 +430,25 @@ def test_similar_mcp_tool_names_are_disambiguated_by_target_context_and_role(): CapabilityTargetContext.HEADLESS_SESSION ) assert ( - capabilities["openhcs_get_runtime_server_execution_status"].exposition.target_context + capabilities[ + "openhcs_get_runtime_server_execution_status" + ].exposition.target_context is CapabilityTargetContext.RUNTIME_SERVER ) - assert capabilities["openhcs_ui_get_operation_status"].exposition.target_context is ( - CapabilityTargetContext.UI_BRIDGE - ) - assert capabilities["openhcs_ui_wait_for_operation_receipt"].exposition.target_context is ( - CapabilityTargetContext.UI_BRIDGE - ) + assert capabilities[ + "openhcs_ui_get_operation_status" + ].exposition.target_context is (CapabilityTargetContext.UI_BRIDGE) + assert capabilities[ + "openhcs_ui_wait_for_operation_receipt" + ].exposition.target_context is (CapabilityTargetContext.UI_BRIDGE) assert ( capabilities["openhcs_ui_wait_for_operation_receipt"].exposition.role is CapabilityRole.PRIMARY ) - assert capabilities["openhcs_ui_invoke_action"].exposition.role is CapabilityRole.PRIMARY + assert ( + capabilities["openhcs_ui_invoke_action"].exposition.role + is CapabilityRole.PRIMARY + ) assert capabilities["openhcs_ui_invoke_widget_action"].exposition.role is ( CapabilityRole.FALLBACK ) @@ -446,9 +461,9 @@ def test_similar_mcp_tool_names_are_disambiguated_by_target_context_and_role(): assert capabilities["openhcs_validate_pipeline"].exposition.workflow_stage is ( CapabilityWorkflowStage.VALIDATION ) - assert capabilities["openhcs_ui_mutate_object_state_field"].exposition.visibility is ( - CapabilityVisibility.EXPERT - ) + assert capabilities[ + "openhcs_ui_mutate_object_state_field" + ].exposition.visibility is (CapabilityVisibility.EXPERT) def test_ui_bridge_capabilities_declare_runtime_security_and_data_exposure(): @@ -546,7 +561,9 @@ def invoke(self, operation, request=None, connection=None): connection = object() invocation = agent_capabilities.ui_bridge_status.invocation - context = SimpleNamespace(ui_bridge_service=RecordingUiBridgeService(gateway=object())) + context = SimpleNamespace( + ui_bridge_service=RecordingUiBridgeService(gateway=object()) + ) assert invocation.execute(context, connection) == "served" assert seen == [(invocation.operation, None, connection)] diff --git a/tests/unit/agent/test_mcp_dev_client_sampling_results.py b/tests/unit/agent/test_mcp_dev_client_sampling_results.py index 169aca3cec..49b08ecc47 100644 --- a/tests/unit/agent/test_mcp_dev_client_sampling_results.py +++ b/tests/unit/agent/test_mcp_dev_client_sampling_results.py @@ -14,16 +14,28 @@ from openhcs.agent.capabilities import SamplePlateImageCapability, agent_capabilities from openhcs.agent.dto.common import AgentError, SCHEMA_VERSION -from openhcs.agent.dto.plate import PlateImageSampleResult, SelectedPlateImageSampleResult -from openhcs.agent.dto.viewer import ViewerWindowImageSampleRecord, ViewerWindowImageSampleResult +from openhcs.agent.dto.plate import ( + PlateImageSampleResult, + SelectedPlateImageSampleResult, +) +from openhcs.agent.dto.viewer import ( + ViewerWindowImageSampleRecord, + ViewerWindowImageSampleResult, +) from openhcs.mcp import dev_client from openhcs.mcp.dev_client_commanding import CapabilityBackedCommandSpec from openhcs.mcp.dev_client_core import ( - McpDevClientPhase, McpDevPayloadFailure, McpDevServerSpec, - McpDevToolBatchResponse, McpDevToolResult, + McpDevClientPhase, + McpDevPayloadFailure, + McpDevServerSpec, + McpDevToolBatchResponse, + McpDevToolResult, ) from openhcs.mcp.dev_client_rendering import McpDevOutputRenderer -from openhcs.mcp.dev_client_renderers.plate import PlateImageSampleRenderer, SelectedPlateSampleRenderer +from openhcs.mcp.dev_client_renderers.plate import ( + PlateImageSampleRenderer, + SelectedPlateSampleRenderer, +) from openhcs.mcp.dev_client_renderers.viewer import ViewerImageSampleRenderer from openhcs.runtime.viewer_protocol import ViewerArrayValueSummary from python_introspect import to_jsonable @@ -33,29 +45,57 @@ def forbid_native_or_provider(monkeypatch): def forbidden(*args, **kwargs): raise AssertionError("Source sampling controls must not spawn a process") + monkeypatch.setattr(subprocess, "Popen", forbidden) monkeypatch.setattr(asyncio, "create_subprocess_exec", forbidden) monkeypatch.setattr(asyncio, "create_subprocess_shell", forbidden) def sample(**kwargs): - return replace(PlateImageSampleResult( - schema_version=SCHEMA_VERSION, plate_path="source-plate", requested_image_path="image.tif", - virtual_path="images/image.tif", source_path="physical/image.tif", - shape=(1, 2, 2), resolution_shape=(1, 2, 2), dtype="uint16", - minimum=0, maximum=4, mean=2.5, selected_resolution_index=0, - resolution_count=1, downsample_yx=(1.0, 1.0), statistics_scope="source_resolution", - sample_shape=(1, 2, 2), sample_included=True, sample_values=[[[0, 2], [3, 4]]], - ), **kwargs) + return replace( + PlateImageSampleResult( + schema_version=SCHEMA_VERSION, + plate_path="source-plate", + requested_image_path="image.tif", + virtual_path="images/image.tif", + source_path="physical/image.tif", + shape=(1, 2, 2), + resolution_shape=(1, 2, 2), + dtype="uint16", + minimum=0, + maximum=4, + mean=2.5, + selected_resolution_index=0, + resolution_count=1, + downsample_yx=(1.0, 1.0), + statistics_scope="source_resolution", + sample_shape=(1, 2, 2), + sample_included=True, + sample_values=[[[0, 2], [3, 4]]], + ), + **kwargs, + ) def selected_row(): return DatasetRowState( - scope_id="selected-root", name="selection", root="selected-root", - pipeline_path=None, selected=True, initialized=True, compiled=False, - init_pending=False, compile_pending=False, execution_active=False, - status_prefix="", orchestrator_state=None, execution_id=None, - terminal_status=None, runtime_state=None, runtime_percent=None, queue_position=None, + scope_id="selected-root", + name="selection", + root="selected-root", + pipeline_path=None, + selected=True, + initialized=True, + compiled=False, + init_pending=False, + compile_pending=False, + execution_active=False, + status_prefix="", + orchestrator_state=None, + execution_id=None, + terminal_status=None, + runtime_state=None, + runtime_percent=None, + queue_position=None, ) @@ -63,38 +103,53 @@ def selected(value): return SelectedPlateImageSampleResult( schema_version=SCHEMA_VERSION, selected_plate=to_jsonable(selected_row()), - image_path="image.tif", auto_selected_image_path=False, sample=value, + image_path="image.tif", + auto_selected_image_path=False, + sample=value, ) def batch(value, capability): receipt = {"isError": False, "structuredContent": to_jsonable(value), "content": []} return McpDevToolBatchResponse.from_results( - McpDevServerSpec(sys.executable), (McpDevToolResult.from_payload(capability.name, receipt),), + McpDevServerSpec(sys.executable), + (McpDevToolResult.from_payload(capability.name, receipt),), ) def command_render(response, argv): args = dev_client._build_parser().parse_args(argv) - return dev_client.McpDevCommandSpec.for_name(args.command).render_result(response, args) + return dev_client.McpDevCommandSpec.for_name(args.command).render_result( + response, args + ) @pytest.mark.parametrize("structured", [False, True]) @pytest.mark.parametrize("selected_case", [False, True]) -def test_original_session_framing_to_generated_sampling_command(structured, selected_case, monkeypatch): - from tests.unit.agent.test_mcp_dev_client_pipeline_results import ControlledWireSession +def test_original_session_framing_to_generated_sampling_command( + structured, selected_case, monkeypatch +): + from tests.unit.agent.test_mcp_dev_client_pipeline_results import ( + ControlledWireSession, + ) import openhcs.mcp.dev_client_core as ingress value = selected(sample()) if selected_case else sample() - argv = ("selected-plate-sample",) if selected_case else ("sample-plate-image", "source-plate", "image.tif") + argv = ( + ("selected-plate-sample",) + if selected_case + else ("sample-plate-image", "source-plate", "image.tif") + ) args = dev_client._build_parser().parse_args(argv) command = dev_client.McpDevCommandSpec.for_name(args.command) peer = ControlledWireSession((value,), structured=structured) decoded = [] original_decode = ingress.dataclass_from_mapping + def observe(contract, receipt): decoded.append(contract) return original_decode(contract, receipt) + monkeypatch.setattr(ingress, "dataclass_from_mapping", observe) response = asyncio.run(command.run_session(peer, args)) assert decoded == [type(value)] @@ -107,8 +162,14 @@ def observe(contract, receipt): @pytest.mark.parametrize("selected_case", [False, True]) @pytest.mark.parametrize("generic_call", [False, True]) -def test_generated_commands_retain_decoded_sampling_facts(monkeypatch, selected_case, generic_call): - capability = agent_capabilities.ui_sample_selected_plate_image if selected_case else agent_capabilities.sample_plate_image +def test_generated_commands_retain_decoded_sampling_facts( + monkeypatch, selected_case, generic_call +): + capability = ( + agent_capabilities.ui_sample_selected_plate_image + if selected_case + else agent_capabilities.sample_plate_image + ) value = selected(sample()) if selected_case else sample() response = batch(value, capability) decoded = response.results[0].first_decoded_payload() @@ -119,10 +180,17 @@ def test_generated_commands_retain_decoded_sampling_facts(monkeypatch, selected_ def forbidden(value): raise AssertionError("Typed sampling must not flatten an owned result to JSON") + monkeypatch.setattr(dev_client, "to_jsonable", forbidden) monkeypatch.setattr("python_introspect.jsonable.to_jsonable", forbidden) - argv = ("call", capability.name, "--arguments", "{}") if generic_call else ( - ("selected-plate-sample",) if selected_case else ("sample-plate-image", "source-plate", "image.tif") + argv = ( + ("call", capability.name, "--arguments", "{}") + if generic_call + else ( + ("selected-plate-sample",) + if selected_case + else ("sample-plate-image", "source-plate", "image.tif") + ) ) rendered = command_render(response, argv) assert "Image: images/image.tif" in rendered @@ -131,23 +199,51 @@ def forbidden(value): assert "mean=2.500" in rendered assert "Sample values:" in rendered if selected_case: - assert "Selected plate: selection root=selected-root target=selected" in rendered + assert ( + "Selected plate: selection root=selected-root target=selected" in rendered + ) assert "Selected image: image.tif auto=False" in rendered @pytest.mark.parametrize("selected_case", [False, True]) -@pytest.mark.parametrize("reason,shape,hint", [ - ("max_array_elements_exceeded", (1, 8, 8), "--max-array-elements 64 or smaller --width/--height"), - ("sample has 256 elements, above max_array_elements=40", (1, 16, 16), "--max-array-elements 256"), - ("array_values_not_requested", (1, 2, 2), "--include-array-values --max-array-elements 4"), - ("empty", (), "Sample values omitted: empty"), - ("max_array_elements_exceeded", (1, 0, 2), "--max-array-elements 0"), -]) +@pytest.mark.parametrize( + "reason,shape,hint", + [ + ( + "max_array_elements_exceeded", + (1, 8, 8), + "--max-array-elements 64 or smaller --width/--height", + ), + ( + "sample has 256 elements, above max_array_elements=40", + (1, 16, 16), + "--max-array-elements 256", + ), + ( + "array_values_not_requested", + (1, 2, 2), + "--include-array-values --max-array-elements 4", + ), + ("empty", (), "Sample values omitted: empty"), + ("max_array_elements_exceeded", (1, 0, 2), "--max-array-elements 0"), + ], +) def test_omission_and_budget_advice_preserved(selected_case, reason, shape, hint): - value = sample(sample_included=False, sample_values=(), sample_shape=shape, sample_omitted_reason=reason) - capability = agent_capabilities.ui_sample_selected_plate_image if selected_case else agent_capabilities.sample_plate_image + value = sample( + sample_included=False, + sample_values=(), + sample_shape=shape, + sample_omitted_reason=reason, + ) + capability = ( + agent_capabilities.ui_sample_selected_plate_image + if selected_case + else agent_capabilities.sample_plate_image + ) response = batch(selected(value) if selected_case else value, capability) - renderer = SelectedPlateSampleRenderer if selected_case else PlateImageSampleRenderer + renderer = ( + SelectedPlateSampleRenderer if selected_case else PlateImageSampleRenderer + ) assert hint in renderer.render(response) @@ -156,28 +252,54 @@ def test_legitimate_absent_sample_and_false_zero_empty_are_distinct(): rendered = SelectedPlateSampleRenderer.render(response) assert "Sample: " in rendered assert "auto=False" in rendered - zero = sample(virtual_path="", dtype="", mean=0.0, minimum=False, maximum=0, - sample_shape=(), sample_values=[], sample_included=True) - rendered = PlateImageSampleRenderer.render(batch(zero, agent_capabilities.sample_plate_image)) + zero = sample( + virtual_path="", + dtype="", + mean=0.0, + minimum=False, + maximum=0, + sample_shape=(), + sample_values=[], + sample_included=True, + ) + rendered = PlateImageSampleRenderer.render( + batch(zero, agent_capabilities.sample_plate_image) + ) assert "Image: \n" in rendered assert "dtype= min=False max=0 mean=0.000" in rendered assert "shape= included=True" in rendered assert "Sample values:\n[]" in rendered -@pytest.mark.parametrize("values,count", [([False, 0, "", None], 3), (list(range(65)), 65), ({"a": [0, False], "b": ""}, 3)]) +@pytest.mark.parametrize( + "values,count", + [([False, 0, "", None], 3), (list(range(65)), 65), ({"a": [0, False], "b": ""}, 3)], +) def test_shared_preview_policy_and_richer_viewer_owner(values, count): assert McpDevOutputRenderer.json_value_count(values) == count - response = batch(sample(sample_values=values), agent_capabilities.sample_plate_image) + response = batch( + sample(sample_values=values), agent_capabilities.sample_plate_image + ) rendered = PlateImageSampleRenderer.render(response) assert ("65 elements; pass --json" in rendered) == (count > 64) record = ViewerWindowImageSampleRecord( - layer_route_key="layer", layer_title="image", payload_route_key="payload", - data_type="image", path="source", components={}, array_values=(values,), - array_value_summary=ViewerArrayValueSummary(requested=True, included=True, shape=(1, count)), + layer_route_key="layer", + layer_title="image", + payload_route_key="payload", + data_type="image", + path="source", + components={}, + array_values=(values,), + array_value_summary=ViewerArrayValueSummary( + requested=True, included=True, shape=(1, count) + ), + ) + viewer = ViewerWindowImageSampleResult( + schema_version=SCHEMA_VERSION, observed=True, records=(record,) + ) + text = ViewerImageSampleRenderer.render( + batch(viewer, agent_capabilities.sample_viewer_window_image) ) - viewer = ViewerWindowImageSampleResult(schema_version=SCHEMA_VERSION, observed=True, records=(record,)) - text = ViewerImageSampleRenderer.render(batch(viewer, agent_capabilities.sample_viewer_window_image)) assert ("sample values:" in text) == (count <= 64) assert record.array_value_summary.shape_element_count == count @@ -186,19 +308,44 @@ def test_shared_preview_policy_and_richer_viewer_owner(values, count): def test_original_typed_diagnostics_retained_once(selected_case): error = AgentError("sample_failure", "retained cause", "retained hint") value = sample(errors=(error,)) - capability = agent_capabilities.ui_sample_selected_plate_image if selected_case else agent_capabilities.sample_plate_image - response = batch(replace(selected(value), errors=(error,)) if selected_case else value, capability) - text = (SelectedPlateSampleRenderer if selected_case else PlateImageSampleRenderer).render(response) + capability = ( + agent_capabilities.ui_sample_selected_plate_image + if selected_case + else agent_capabilities.sample_plate_image + ) + response = batch( + replace(selected(value), errors=(error,)) if selected_case else value, + capability, + ) + text = ( + SelectedPlateSampleRenderer if selected_case else PlateImageSampleRenderer + ).render(response) assert "failed" in text assert text.count("sample_failure") == 1 assert "retained cause" in text and "retained hint" in text assert response.has_errors() -@pytest.mark.parametrize("receipt", [{"alien": 1}, {"schema_version": SCHEMA_VERSION}, {"schema_version": SCHEMA_VERSION, "plate_path": "p", "requested_image_path": "i", "sample_included": "false"}]) +@pytest.mark.parametrize( + "receipt", + [ + {"alien": 1}, + {"schema_version": SCHEMA_VERSION}, + { + "schema_version": SCHEMA_VERSION, + "plate_path": "p", + "requested_image_path": "i", + "sample_included": "false", + }, + ], +) def test_malformed_receipt_never_becomes_successful_empty_sample(receipt): - result = McpDevToolResult.from_payload(agent_capabilities.sample_plate_image.name, {"structuredContent": receipt}) - response = McpDevToolBatchResponse.from_results(McpDevServerSpec(sys.executable), (result,)) + result = McpDevToolResult.from_payload( + agent_capabilities.sample_plate_image.name, {"structuredContent": receipt} + ) + response = McpDevToolBatchResponse.from_results( + McpDevServerSpec(sys.executable), (result,) + ) assert isinstance(result.payloads[0], McpDevPayloadFailure) assert result.payloads[0].payload == receipt rejection = to_jsonable(response)["results"][0]["payloads"][0] @@ -211,12 +358,19 @@ def test_malformed_receipt_never_becomes_successful_empty_sample(receipt): def test_transport_missing_and_unknown_receipts_keep_original_diagnostics(): server = McpDevServerSpec(sys.executable) - missing = McpDevToolBatchResponse.from_results(server, (McpDevToolResult(agent_capabilities.sample_plate_image.name, False, ()),)) + missing = McpDevToolBatchResponse.from_results( + server, + (McpDevToolResult(agent_capabilities.sample_plate_image.name, False, ()),), + ) assert "mcp_payload_missing" in PlateImageSampleRenderer.render(missing) - response = McpDevToolBatchResponse.from_transport_failure(server, McpDevClientPhase.CALL_TOOL, RuntimeError("wire lost")) + response = McpDevToolBatchResponse.from_transport_failure( + server, McpDevClientPhase.CALL_TOOL, RuntimeError("wire lost") + ) assert "wire lost" in PlateImageSampleRenderer.render(response) receipt = {"opaque": {"false": False, "zero": 0, "empty": ""}} - unknown = McpDevToolResult.from_payload("unregistered-external-tool", {"structuredContent": receipt}) + unknown = McpDevToolResult.from_payload( + "unregistered-external-tool", {"structuredContent": receipt} + ) assert unknown.payloads == (receipt,) assert to_jsonable(unknown)["payloads"] == [receipt] @@ -228,11 +382,16 @@ class NewSample(PlateImageSampleResult): independent_fact: str = "owned" events = [] + class SourceFact(PlateImageSampleRenderer): @classmethod def render_payload(cls, payload, options): events.append("source") - return super().render_payload(payload, options) + "\n" + payload.independent_fact + return ( + super().render_payload(payload, options) + + "\n" + + payload.independent_fact + ) class AuditFact(PlateImageSampleRenderer): @classmethod @@ -242,12 +401,17 @@ def render_payload(cls, payload, options): bases = (AuditFact, SourceFact) if reverse_order else (SourceFact, AuditFact) NewRenderer = type("NewSampleRenderer", bases, {"output_contract": NewSample}) + class NewCapability(SamplePlateImageCapability): name = f"openhcs_s1_new_sample_{int(reverse_order)}" cli_command = f"s1-new-sample-{int(reverse_order)}" output_contract = NewSample - value = NewSample(schema_version=SCHEMA_VERSION, plate_path="new-plate", requested_image_path="new-image") + value = NewSample( + schema_version=SCHEMA_VERSION, + plate_path="new-plate", + requested_image_path="new-image", + ) response = batch(value, NewCapability) assert type(response.results[0].first_decoded_payload()) is NewSample command = CapabilityBackedCommandSpec.for_capability_name(NewCapability.name) @@ -261,4 +425,7 @@ class NewCapability(SamplePlateImageCapability): def test_sampling_renderers_share_the_json_preview_count(): assert "_json_value_count" not in vars(PlateImageSampleRenderer) assert "_json_value_count" not in vars(ViewerImageSampleRenderer) - assert PlateImageSampleRenderer.json_value_count.__func__ is ViewerImageSampleRenderer.json_value_count.__func__ + assert ( + PlateImageSampleRenderer.json_value_count.__func__ + is ViewerImageSampleRenderer.json_value_count.__func__ + ) diff --git a/tests/unit/agent/test_mcp_dev_client_ui_renderers.py b/tests/unit/agent/test_mcp_dev_client_ui_renderers.py index f7dd2edd0e..24353fde25 100644 --- a/tests/unit/agent/test_mcp_dev_client_ui_renderers.py +++ b/tests/unit/agent/test_mcp_dev_client_ui_renderers.py @@ -71,7 +71,10 @@ def render(payload, options=None) -> str: def snapshot(index: int, label: str = "edit") -> UiSnapshotRef: return replace( - sample_dto(UiSnapshotRef), index=index, label=label, branch="main", + sample_dto(UiSnapshotRef), + index=index, + label=label, + branch="main", snapshot_id=f"snapshot-{index}", ) @@ -112,9 +115,11 @@ def test_bridge_status_and_unavailable_bridge(): server=SERVER, results=(McpDevToolResult(tool="t", mcp_error=True, payloads=()),), ) - assert McpDevOutputRenderer.for_output_contract(UiBridgeStatus).render( - failed - ).startswith("UI bridge: unavailable\nErrors:") + assert ( + McpDevOutputRenderer.for_output_contract(UiBridgeStatus) + .render(failed) + .startswith("UI bridge: unavailable\nErrors:") + ) def window(window_id: str, title: str, kind: str, visible: bool) -> UiWindowSummary: @@ -147,21 +152,28 @@ def test_window_catalog_flags_only_visible_stray_top_level_windows(): 'Attention: 1 visible top-level window(s): qt_top_level:1 title="Error"' in rendered ) - assert '- plate_manager [embedded] visible=True' in rendered + assert "- plate_manager [embedded] visible=True" in rendered def test_ui_smoke_composes_each_tool_payload(): from openhcs.agent.capabilities import agent_capabilities def result(capability, payload): - return McpDevToolResult(tool=capability.name, mcp_error=False, payloads=(payload,)) + return McpDevToolResult( + tool=capability.name, mcp_error=False, payloads=(payload,) + ) health = replace( sample_dto(McpServerHealthResult), - status="ok", restart_required=False, stale_source_paths=(), warnings=(), + status="ok", + restart_required=False, + stale_source_paths=(), + warnings=(), ) bridges = replace(sample_dto(UiBridgeCatalog), warnings=()) - status = replace(sample_dto(UiBridgeStatus), reachable=True, descriptor_status="ok", warnings=()) + status = replace( + sample_dto(UiBridgeStatus), reachable=True, descriptor_status="ok", warnings=() + ) response = McpDevToolBatchResponse( server=SERVER, results=( @@ -211,9 +223,15 @@ def document(document_id): def plate_row(name: str, selected: bool, **values) -> DatasetRowState: return replace( sample_dto(DatasetRowState), - name=name, selected=selected, orchestrator_state="compiled", - status_prefix="", terminal_status=None, root=f"/{name}", - output_root=None, source_root=None, **values, + name=name, + selected=selected, + orchestrator_state="compiled", + status_prefix="", + terminal_status=None, + root=f"/{name}", + output_root=None, + source_root=None, + **values, ) @@ -245,15 +263,21 @@ def test_state_surface_documents_render_through_their_surface_state(): sample_dto(UiPipelineEditorState), summary=surface_summary( PipelineEditorStateSurfaceIdentityDeclaration.require_value(), - current_selection_count=1, total_scope_count=2, + current_selection_count=1, + total_scope_count=2, ), current_snapshot=snapshot(4, "edit pipeline"), selected_scope_ids=("/tmp/plate-a::functionstep_1",), steps=( replace( sample_dto(PipelineStepState), - index=0, name="normalize", dirty=True, default_diff=False, - function_names=("normalize",), function_ids=(), debug_pause=False, + index=0, + name="normalize", + dirty=True, + default_diff=False, + function_names=("normalize",), + function_ids=(), + debug_pause=False, step_scope_id=None, ), ), @@ -262,7 +286,9 @@ def test_state_surface_documents_render_through_their_surface_state(): rendered = render(document_for(editor)) assert 'Snapshot: 4 "edit pipeline"' in rendered assert "Selected scopes: /tmp/plate-a::functionstep_1" in rendered - assert "- 0. [*] normalize: enabled=True, selected=True, funcs=normalize" in rendered + assert ( + "- 0. [*] normalize: enabled=True, selected=True, funcs=normalize" in rendered + ) assert "ids=" not in rendered @@ -288,7 +314,9 @@ def test_unregistered_or_unreadable_surface_points_to_the_catalog(): summary=surface_summary("plate_manager", readable=False), payload={}, warnings=(), - errors=(AgentError(code="ui_state_surface_unreadable", message="Not readable."),), + errors=( + AgentError(code="ui_state_surface_unreadable", message="Not readable."), + ), ) rendered = render(document) assert "Surface: plate_manager" in rendered @@ -299,38 +327,68 @@ def test_unregistered_or_unreadable_surface_points_to_the_catalog(): def tree_node(class_name: str, *children, **values) -> UiWidgetTreeNode: defaults = dict( - object_name="", text=None, title=None, current_text=None, visible=True, - enabled=True, clickable=True, actionable=False, action_kinds=(), + object_name="", + text=None, + title=None, + current_text=None, + visible=True, + enabled=True, + clickable=True, + actionable=False, + action_kinds=(), ) return replace( sample_dto(UiWidgetTreeNode), - class_name=class_name, children=children, **(defaults | values), + class_name=class_name, + children=children, + **(defaults | values), ) def test_widget_tree_outline_shows_addressable_paths_and_hides_technical_nodes(): root = tree_node( "PipelineEditorWidget", - tree_node("QPushButton", text="Edit", path_id="1.0.2", action_kinds=("button",), - enabled=False, clickable=False), - tree_node("QPushButton", text="Generate", path_id="1.0.3", - action_kinds=("button",), visible=False), + tree_node( + "QPushButton", + text="Edit", + path_id="1.0.2", + action_kinds=("button",), + enabled=False, + clickable=False, + ), + tree_node( + "QPushButton", + text="Generate", + path_id="1.0.3", + action_kinds=("button",), + visible=False, + ), tree_node("QScrollBar"), tree_node("QLineEdit", path_id="1.0.4", actionable=True, text="Gamma"), tree_node("QLabel", current_text="long " * 40), ) action = replace( sample_dto(UiWidgetActionSummary), - path_id="1.0.4", object_state_scope_id="/plate::step", field_path="gamma", + path_id="1.0.4", + object_state_scope_id="/plate::step", + field_path="gamma", semantic_markers=("*",), ) result = replace( sample_dto(UiWidgetTreeResult), - root=root, actionable_widgets=(action,), tree_truncated=False, - actionable_widgets_truncated=False, warnings=(), - summary=replace(sample_dto(UiWindowSummary), title="Pipeline Editor", - dirty=False, dirty_field_count=0, signature_diff=True, - semantic_markers=()), + root=root, + actionable_widgets=(action,), + tree_truncated=False, + actionable_widgets_truncated=False, + warnings=(), + summary=replace( + sample_dto(UiWindowSummary), + title="Pipeline Editor", + dirty=False, + dirty_field_count=0, + signature_diff=True, + semantic_markers=(), + ), ) rendered = render(result) assert "Window: Pipeline Editor" in rendered @@ -357,22 +415,29 @@ def test_widget_tree_outline_shows_addressable_paths_and_hides_technical_nodes() def test_code_document_bounds_its_source(): document = replace( - sample_dto(UiCodeDocument), source="pattern = (some_func," + "x" * 50, - current_snapshot=snapshot(10), warnings=(), + sample_dto(UiCodeDocument), + source="pattern = (some_func," + "x" * 50, + current_snapshot=snapshot(10), + warnings=(), ) rendered = render(document, CodeDocumentRenderOptions(max_source_chars=20)) assert "Revision: token=" in rendered and "snapshot=main@10 head=" in rendered assert "Source:\npattern = (some_func" in rendered assert "..." in rendered - assert "Source:" not in render(document, CodeDocumentRenderOptions(include_source=False)) + assert "Source:" not in render( + document, CodeDocumentRenderOptions(include_source=False) + ) def test_code_document_apply_shows_revisions_and_snapshots(): applied = replace( sample_dto(UiCodeDocumentApplyResult), - base_revision_token="rev-123", current_revision_token="rev-456", - new_revision_token="rev-456", current_snapshot=snapshot(10), - undo_snapshot=None, warnings=(), + base_revision_token="rev-123", + current_revision_token="rev-456", + new_revision_token="rev-456", + current_snapshot=snapshot(10), + undo_snapshot=None, + warnings=(), ) rendered = render(applied) assert "Revision: base=rev-123 current=rev-456 new=rev-456" in rendered @@ -393,13 +458,20 @@ def test_action_catalog_filters_actions_and_warnings_by_widget(): sample_dto(UiActionCatalog), actions=( ui_action( - "plate_manager", "compile_datasets", enabled=False, - disabled_error=AgentError(code="no_plate", message="No plate.", hint="Add a plate."), + "plate_manager", + "compile_datasets", + enabled=False, + disabled_error=AgentError( + code="no_plate", message="No plate.", hint="Add a plate." + ), ), ui_action("image_browser", "open_file"), ), warnings=( - AgentWarning(code="plate_path_setup_uses_code_document", message="Use plate_manager code."), + AgentWarning( + code="plate_path_setup_uses_code_document", + message="Use plate_manager code.", + ), AgentWarning(code="unrelated", message="Something else."), ), ) @@ -427,7 +499,9 @@ def test_action_catalog_filters_actions_and_warnings_by_widget(): def test_disabled_actions_do_not_make_the_call_fail(): catalog = replace( sample_dto(UiActionCatalog), - actions=(ui_action("w", "a", disabled_error=AgentError(code="off", message="Off.")),), + actions=( + ui_action("w", "a", disabled_error=AgentError(code="off", message="Off.")), + ), ) result = McpDevToolResult(tool="t", mcp_error=False, payloads=(catalog,)) assert not result.has_errors() @@ -438,8 +512,12 @@ def test_tool_list_filters_groups_and_flattens(): server=SERVER, tool_count=3, tools=( - McpDevToolMetadata("openhcs_get_viewer_window_state", "Read bounded viewer state.", {}), - McpDevToolMetadata("openhcs_validate_viewer_window_state", "Validate viewer.", {}), + McpDevToolMetadata( + "openhcs_get_viewer_window_state", "Read bounded viewer state.", {} + ), + McpDevToolMetadata( + "openhcs_validate_viewer_window_state", "Validate viewer.", {} + ), McpDevToolMetadata("openhcs_health_check", "Health.", {}), ), ) @@ -453,4 +531,6 @@ def test_tool_list_filters_groups_and_flattens(): assert "[Viewer Review]" not in flat projected = ToolListRenderer.project_response(tools, contains="viewer", limit=1) assert projected["matched_tool_count"] == 2 - assert [tool["name"] for tool in projected["tools"]] == ["openhcs_get_viewer_window_state"] + assert [tool["name"] for tool in projected["tools"]] == [ + "openhcs_get_viewer_window_state" + ] diff --git a/tests/unit/agent/test_mcp_plate_stream_progress.py b/tests/unit/agent/test_mcp_plate_stream_progress.py index 9c71292a29..fb954b6a45 100644 --- a/tests/unit/agent/test_mcp_plate_stream_progress.py +++ b/tests/unit/agent/test_mcp_plate_stream_progress.py @@ -18,8 +18,10 @@ from polystore.streaming.viewer_transport import ViewerStreamKwarg from openhcs.agent.capabilities import ( - AgentCapabilityDeclaration, MainThreadProgressCapability, - StreamPlateFilesToViewerCapability, UiStreamSelectedPlateFilesToViewerCapability, + AgentCapabilityDeclaration, + MainThreadProgressCapability, + StreamPlateFilesToViewerCapability, + UiStreamSelectedPlateFilesToViewerCapability, ) from openhcs.agent.services.plate_streaming_service import PlateStreamingService from openhcs.core.roi_source_metadata import ROIArchiveSourceMetadata @@ -30,7 +32,10 @@ from openhcs.mcp.server import build_server from openhcs.mcp.socket import McpSocketTransport, wait_for_socket -from tests.unit.agent.test_persisted_output_reopen import NoRuntimeBridge, declared_output +from tests.unit.agent.test_persisted_output_reopen import ( + NoRuntimeBridge, + declared_output, +) from tests.unit.test_streaming_service import FakeViewer @@ -44,19 +49,26 @@ def saved_roi_stream(tmp_path, monkeypatch): archive = result_root / "not-an-acquisition.roi.zip" geometry = [ROI([PointShape(12.5, 8.25)], {"label": 7})] DiskStorageBackend().save( - ROIArchiveSourceMetadata.bind(geometry, projection.image_metadata), archive, + ROIArchiveSourceMetadata.bind(geometry, projection.image_metadata), + archive, ) persisted = ROIArchiveSourceMetadata.decode(DiskStorageBackend().load(archive)) assert persisted.source_provenance == projection.image_metadata.source_provenance - assert persisted.source_voxel_spacing == projection.image_metadata.source_voxel_spacing + assert ( + persisted.source_voxel_spacing == projection.image_metadata.source_voxel_spacing + ) unsigned = result_root / "missing-source.roi.zip" DiskStorageBackend().save(geometry, unsigned) # Reuse the original path authority, explicitly adding this synthetic root. from openhcs.agent.path_policy import AgentPathPolicy from openhcs.agent.services.plate_inspection_service import PlateInspectionService - inspection = PlateInspectionService(AgentPathPolicy.with_roots( - readable_roots=(tmp_path,), writable_roots=(), - )) + + inspection = PlateInspectionService( + AgentPathPolicy.with_roots( + readable_roots=(tmp_path,), + writable_roots=(), + ) + ) viewer = FakeViewer() received = [] timings = [] @@ -75,12 +87,21 @@ def receive(_manager, data, paths, backend, **fields): restored = ImagePayloadMetadata.from_viewer_image_metadata( request.source.item_fields["image_metadata"] ) - assert restored.to_viewer_image_metadata() == projection.image_metadata.to_viewer_image_metadata() + assert ( + restored.to_viewer_image_metadata() + == projection.image_metadata.to_viewer_image_metadata() + ) assert request.source.metadata.metadata_by_path[str(archive)] == dict( persisted.source_provenance.scalar_source_identity.component_metadata, ) - assert restored.source_voxel_spacing == projection.image_metadata.source_voxel_spacing - assert restored.source_spatial_domain == projection.image_metadata.source_spatial_domain + assert ( + restored.source_voxel_spacing + == projection.image_metadata.source_voxel_spacing + ) + assert ( + restored.source_spatial_domain + == projection.image_metadata.source_spatial_domain + ) # The archive travels as saved: its geometry plus the bound source # declaration, which the viewer reads and keeps out of the feature table. assert [ROIArchiveSourceMetadata.geometry(rois) for rois in data] == [geometry] @@ -98,22 +119,30 @@ def settle(): return True monkeypatch.setattr( - "openhcs.agent.services.plate_streaming_service.StreamingViewerLifecycle.get_or_create_visualizer", acquire, + "openhcs.agent.services.plate_streaming_service.StreamingViewerLifecycle.get_or_create_visualizer", + acquire, ) monkeypatch.setattr(FileManager, "save_batch", receive) monkeypatch.setattr(viewer, "settle_viewer_state", settle) service = PlateStreamingService(inspection, NoRuntimeBridge()) arguments = { "plate_path": str(source_path.parent.parent), - "result_directory": str(result_root), "kind": "result", "limit": 1, - "file_paths": [archive.name], "fresh_viewer": False, + "result_directory": str(result_root), + "kind": "result", + "limit": 1, + "file_paths": [archive.name], + "fresh_viewer": False, } return service, arguments, identity, received, timings, unsigned -@pytest.mark.parametrize("declaration", ( - StreamPlateFilesToViewerCapability, UiStreamSelectedPlateFilesToViewerCapability, -)) +@pytest.mark.parametrize( + "declaration", + ( + StreamPlateFilesToViewerCapability, + UiStreamSelectedPlateFilesToViewerCapability, + ), +) def test_stream_declaration_composes_original_progress_and_main_affinity(declaration): assert issubclass(declaration, MainThreadProgressCapability) spec = declaration @@ -124,12 +153,15 @@ def test_stream_declaration_composes_original_progress_and_main_affinity(declara def test_saved_roi_continuous_sdk_receipt_error_reuse_beyond_ordinary_idle( - unix_socket_dir, saved_roi_stream, + unix_socket_dir, + saved_roi_stream, ): service, arguments, identity, received, timings, unsigned = saved_roi_stream transport = McpSocketTransport(unix_socket_dir / "stream.sock") - built = build_server(OpenHCSAgentContext(plate_streaming_service=service), - main_thread_dispatcher=transport.execution.dispatcher) + built = build_server( + OpenHCSAgentContext(plate_streaming_service=service), + main_thread_dispatcher=transport.execution.dispatcher, + ) clients = AsyncOperationExecutor(max_workers=1) diagnostics = io.StringIO() events = [] @@ -138,37 +170,72 @@ class ObservedSession(McpDevSocketSession): def record_progress_notification(self, notification): super().record_progress_notification(notification) assert threading.current_thread() is not threading.main_thread() - events.append((time.monotonic(), notification.params.progressToken, notification.params.message)) + events.append( + ( + time.monotonic(), + notification.params.progressToken, + notification.params.message, + ) + ) async def exercise(): - assert await asyncio.to_thread(wait_for_socket, transport.socket_path, timeout_seconds=10) + assert await asyncio.to_thread( + wait_for_socket, transport.socket_path, timeout_seconds=10 + ) try: - async with ObservedSession(McpDevServerSpec(sys.executable), diagnostics, transport.socket_path) as session: + async with ObservedSession( + McpDevServerSpec(sys.executable), diagnostics, transport.socket_path + ) as session: await session.initialize(timeout_seconds=10) - for index, paths in enumerate((arguments["file_paths"], [unsigned.name], arguments["file_paths"])): + for index, paths in enumerate( + (arguments["file_paths"], [unsigned.name], arguments["file_paths"]) + ): before = len(events) request_id = session.request_id + 1 started = time.monotonic() - result = await session.call_tool(StreamPlateFilesToViewerCapability.name, - {**arguments, "file_paths": paths}, timeout_seconds=10) + result = await session.call_tool( + StreamPlateFilesToViewerCapability.name, + {**arguments, "file_paths": paths}, + timeout_seconds=10, + ) elapsed = time.monotonic() - started progress = events[before:] assert progress and progress[0][0] - started < 1 assert all(event[1] == request_id for event in progress) payload = result["structuredContent"] if index == 1: - assert payload["errors"][0]["code"] == "plate_file_stream_failed" - assert "Native ROI source metadata is required" in payload["errors"][0]["message"] + assert ( + payload["errors"][0]["code"] == "plate_file_stream_failed" + ) + assert ( + "Native ROI source metadata is required" + in payload["errors"][0]["message"] + ) assert len(received) == 1 else: assert payload["errors"] == [] - assert payload["streamed_roi_paths"] == [str(Path(arguments["result_directory"]) / paths[0])] + assert payload["streamed_roi_paths"] == [ + str(Path(arguments["result_directory"]) / paths[0]) + ] assert payload["streamed_image_paths"] == [] if index == 0: assert elapsed >= 12 - assert any(event[0] - started > 10 and "still running" in event[2] for event in progress) - assert any("Streaming 1 ROI file(s)" in event[2] for event in progress) - print("ROI_STREAM_TERMINAL", index, elapsed, "idle10", "token", request_id, flush=True) + assert any( + event[0] - started > 10 and "still running" in event[2] + for event in progress + ) + assert any( + "Streaming 1 ROI file(s)" in event[2] for event in progress + ) + print( + "ROI_STREAM_TERMINAL", + index, + elapsed, + "idle10", + "token", + request_id, + flush=True, + ) finally: transport._stop.set() if transport._listener is not None: @@ -185,7 +252,9 @@ async def exercise(): clients.close() -def test_independent_stream_leaf_cooperative_hook_uses_generated_consumer(saved_roi_stream): +def test_independent_stream_leaf_cooperative_hook_uses_generated_consumer( + saved_roi_stream, +): service, arguments, identity, received, timings, _unsigned = saved_roi_stream timings.append(0.0) # This declaration test does not repeat the slow experiment. hooks = [] @@ -219,15 +288,18 @@ async def report_progress(self, progress, total=None, message=None): hooks.append(message) executor = McpTransportExecutor() - built = build_server(OpenHCSAgentContext(plate_streaming_service=service), - main_thread_dispatcher=executor.dispatcher) + built = build_server( + OpenHCSAgentContext(plate_streaming_service=service), + main_thread_dispatcher=executor.dispatcher, + ) async def exercise(): token = identity.set("stream-request") try: for declaration in (IndependentStream, IndependentStreamAfter): result = await built._tool_manager.get_tool(declaration.name).fn( - mcp_context=ProgressContext(), **arguments) + mcp_context=ProgressContext(), **arguments + ) assert result["errors"] == [] assert received[-1][3] == "stream-request" finally: @@ -238,7 +310,9 @@ async def exercise(): assert len(received) == 2 assert hooks.count("enter") == hooks.count("exit") == 2 assert hooks.index("enter") < hooks.index("exit") - assert any("published and viewer settlement completed" in message for message in hooks) + assert any( + "published and viewer settlement completed" in message for message in hooks + ) finally: executor.close() for declaration in (IndependentStream, IndependentStreamAfter): diff --git a/tests/unit/agent/test_mcp_process_resource_lifetime.py b/tests/unit/agent/test_mcp_process_resource_lifetime.py index 775d4723bc..d351041c20 100644 --- a/tests/unit/agent/test_mcp_process_resource_lifetime.py +++ b/tests/unit/agent/test_mcp_process_resource_lifetime.py @@ -35,7 +35,9 @@ def shutdown_jvm(self): context.ij = Gateway() resource_patch.setattr(BioFormatsJavaContext, "_instance", context) resource_patch.setattr(backend_registry, "_cleanup_callbacks", []) - backend_registry.register_cleanup_callback(BioFormatsJavaContext.shutdown_instance) + backend_registry.register_cleanup_callback( + BioFormatsJavaContext.shutdown_instance + ) class AuditCloseCapability: def close(self): @@ -68,7 +70,10 @@ def independent_resource(): assert BioFormatsJavaContext._instance is None assert context.ij is None assert calls[:4] == [ - "cooperative enter", "gateway disposed", "JVM stopped", "independent resource" + "cooperative enter", + "gateway disposed", + "JVM stopped", + "independent resource", ] assert calls.count("cooperative exit") == int(failure is None) executor.close() @@ -79,7 +84,9 @@ def independent_resource(): def test_off_main_close_rejected_before_resource_release(monkeypatch): calls = [] - monkeypatch.setattr(backend_registry, "_cleanup_callbacks", [lambda: calls.append("closed")]) + monkeypatch.setattr( + backend_registry, "_cleanup_callbacks", [lambda: calls.append("closed")] + ) executor = McpTransportExecutor() failures = [] @@ -121,7 +128,8 @@ def test_stdio_eof_retains_exact_normal_child_exit(tmp_path, exit_code): async def journey(): session = McpDevStdioSession( - ScriptServerSpec(python_executable=sys.executable, script=script), io.StringIO() + ScriptServerSpec(python_executable=sys.executable, script=script), + io.StringIO(), ) with (tmp_path / "diagnostic.log").open("w+") as diagnostic: session.server_stderr = diagnostic @@ -145,7 +153,8 @@ def test_stdio_stuck_child_uses_unchanged_forced_reap_budget(tmp_path): async def journey(): with (tmp_path / "diagnostic.log").open("w+") as diagnostic: session = McpDevStdioSession( - ScriptServerSpec(python_executable=sys.executable, script=script), diagnostic + ScriptServerSpec(python_executable=sys.executable, script=script), + diagnostic, ) assert session.teardown_timeout_seconds == 2.0 async with session: @@ -157,7 +166,9 @@ async def journey(): @pytest.mark.parametrize("fail_cleanup", (False, True)) -def test_real_sdk_stdio_declared_resource_closes_before_child_exit(tmp_path, fail_cleanup): +def test_real_sdk_stdio_declared_resource_closes_before_child_exit( + tmp_path, fail_cleanup +): from openhcs.runtime.import_authority import OpenHCSRuntimeImportAuthority source_root = OpenHCSRuntimeImportAuthority.current().import_root @@ -175,7 +186,11 @@ def test_real_sdk_stdio_declared_resource_closes_before_child_exit(tmp_path, fai "def close_resource():\n" " assert threading.current_thread() is threading.main_thread()\n" f" Path({str(closed)!r}).write_text('main-thread resource closed')\n" - + (" raise ValueError('original declared cleanup error')\n" if fail_cleanup else "") + + ( + " raise ValueError('original declared cleanup error')\n" + if fail_cleanup + else "" + ) + "register_cleanup_callback(close_resource)\n" "with McpStdioTransport.reserve_process_stdio() as transport:\n" " transport.run(server)\n" @@ -184,21 +199,32 @@ def test_real_sdk_stdio_declared_resource_closes_before_child_exit(tmp_path, fai async def journey(): with (tmp_path / "sdk-diagnostic.log").open("w+") as diagnostic: session = McpDevStdioSession( - ScriptServerSpec(python_executable=sys.executable, script=script), diagnostic + ScriptServerSpec(python_executable=sys.executable, script=script), + diagnostic, ) async with session: process = session.require_process() await session.initialize(timeout_seconds=10) first = await session.call_tool("health", {}, timeout_seconds=10) second = await session.call_tool("health", {}, timeout_seconds=10) - assert first["structuredContent"] == second["structuredContent"] == {"ok": True} + assert ( + first["structuredContent"] + == second["structuredContent"] + == {"ok": True} + ) assert process.returncode == int(fail_cleanup) assert closed.read_text() == "main-thread resource closed" diagnostic.seek(0) text = diagnostic.read() assert "Fatal error" not in text assert ("original declared cleanup error" in text) is fail_cleanup - print("SDK_STDIO_DECLARED_RESOURCE", process.pid, process.returncode, fail_cleanup, flush=True) + print( + "SDK_STDIO_DECLARED_RESOURCE", + process.pid, + process.returncode, + fail_cleanup, + flush=True, + ) asyncio.run(journey()) @@ -228,11 +254,15 @@ def release_resource(): clients = AsyncOperationExecutor(max_workers=1) async def journey(): - assert await asyncio.to_thread(wait_for_socket, transport.socket_path, timeout_seconds=10) + assert await asyncio.to_thread( + wait_for_socket, transport.socket_path, timeout_seconds=10 + ) try: for connection in range(2): async with McpDevSocketSession( - McpDevServerSpec(sys.executable), io.StringIO(), transport.socket_path + McpDevServerSpec(sys.executable), + io.StringIO(), + transport.socket_path, ) as session: await session.initialize(timeout_seconds=10) result = await session.call_tool("health", {}, timeout_seconds=10) diff --git a/tests/unit/agent/test_selected_plane_reopen_stream.py b/tests/unit/agent/test_selected_plane_reopen_stream.py index f6416bf0ec..9f95643b93 100644 --- a/tests/unit/agent/test_selected_plane_reopen_stream.py +++ b/tests/unit/agent/test_selected_plane_reopen_stream.py @@ -39,7 +39,10 @@ ) from openhcs.runtime.napari_viewer_server import NapariBatchPayload, NapariImagePayload -from tests.unit.agent.test_persisted_output_reopen import NoRuntimeBridge, declared_output +from tests.unit.agent.test_persisted_output_reopen import ( + NoRuntimeBridge, + declared_output, +) @pytest.fixture diff --git a/tests/unit/test_aggregate_source_scope.py b/tests/unit/test_aggregate_source_scope.py index 318d135176..1b590e100a 100644 --- a/tests/unit/test_aggregate_source_scope.py +++ b/tests/unit/test_aggregate_source_scope.py @@ -7,26 +7,44 @@ import numpy as np import pytest -from openhcs.core.artifacts import ArtifactSpec, ImageArtifactType, MeasurementsArtifactType +from openhcs.core.artifacts import ( + ArtifactSpec, + ImageArtifactType, + MeasurementsArtifactType, +) from openhcs.core.component_group_scope import RuntimeExecutionAxisScope from openhcs.core.measurement_row_materialization import MeasurementSparseColumnarRows from openhcs.core.orchestrator.execution_result import ( - RuntimeContextObservation, RuntimeExecutionObservation, + RuntimeContextObservation, + RuntimeExecutionObservation, ) from openhcs.core.projected_image_output import SourceProjectedImageOutput from openhcs.core.runtime_artifact_values import ArtifactKey, RuntimeValue from openhcs.core.runtime_exports import ( - RuntimeExportExpectation, RuntimeExportObservation, runtime_export_failures, + RuntimeExportExpectation, + RuntimeExportObservation, + runtime_export_failures, ) from openhcs.core.runtime_image_values import ImagePayloadMetadata -from openhcs.core.runtime_measurements import MeasurementScope, MeasurementSubject, MeasurementTable -from openhcs.core.runtime_plane_projection import RuntimePlaneAxis, RuntimePlaneAxisValueProjection +from openhcs.core.runtime_measurements import ( + MeasurementScope, + MeasurementSubject, + MeasurementTable, +) +from openhcs.core.runtime_plane_projection import ( + RuntimePlaneAxis, + RuntimePlaneAxisValueProjection, +) from openhcs.core.runtime_stores import ( - RuntimeArtifactAddress, RuntimeArtifactLocation, RuntimeValueStore, + RuntimeArtifactAddress, + RuntimeArtifactLocation, + RuntimeValueStore, ) from openhcs.core.runtime_tabular_values import FieldSpec from openhcs.core.source_image_provenance import ( - RuntimeSourceImageProvenancePlane, SourceImageIdentity, SourceImageProvenancePlanes, + RuntimeSourceImageProvenancePlane, + SourceImageIdentity, + SourceImageProvenancePlanes, ) from openhcs.core.source_metadata import SourceVoxelSpacing from openhcs.core.steps.abstract import StepExecutionObservation @@ -45,12 +63,19 @@ def _source(sites): source_image_provenance_planes=SourceImageProvenancePlanes.from_components( paths=tuple(f"/input/field_{i}.tif" for i in range(len(sites))), component_metadata=tuple( - {"well": "A01", "channel": 2, "z_index": 1, "timepoint": 1, - **({"site": site} if site is not None else {})} + { + "well": "A01", + "channel": 2, + "z_index": 1, + "timepoint": 1, + **({"site": site} if site is not None else {}), + } for site in sites ), ), - ).payload_with(np.arange(len(sites) * 12, dtype=np.uint16).reshape(len(sites), 3, 4)) + ).payload_with( + np.arange(len(sites) * 12, dtype=np.uint16).reshape(len(sites), 3, 4) + ) class ProjectionAudit: @@ -80,8 +105,12 @@ def test_mixed_acquired_and_reduced_batch_uses_distinct_original_display_scopes( source = _source((1, 3)) declaration = DeclaredReducedField(np.full((3, 4), 720, dtype=np.uint16), []) reduced = ImageArtifactType.contextualize_output( - source, declaration, None, - RuntimePlaneAxisValueProjection.preserve(axis=RuntimePlaneAxis.SOURCE_BINDING, axis_size=2), + source, + declaration, + None, + RuntimePlaneAxisValueProjection.preserve( + axis=RuntimePlaneAxis.SOURCE_BINDING, axis_size=2 + ), ) assert declaration.calls == ["before", "after"] acquired_metadata = source.metadata.for_leading_source_plane(0) @@ -95,33 +124,52 @@ def test_mixed_acquired_and_reduced_batch_uses_distinct_original_display_scopes( request = kwargs["stream_request"] single = adapter.to_filemanager_kwargs(output)["stream_request"] assert request.display_config == single.display_config - assert request.source.metadata.component_metadata_for_item(output.path, 0) == single.source.metadata.component_metadata_for_item(output.path, 0) + assert request.source.metadata.component_metadata_for_item( + output.path, 0 + ) == single.source.metadata.component_metadata_for_item(output.path, 0) assert request.producer == adapter.values.stream_request.producer - assert request.viewer_transport == adapter.values.stream_request.viewer_transport - assert request.display_config.component_modes() == adapter.values.stream_request.display_config.component_modes() + assert ( + request.viewer_transport == adapter.values.stream_request.viewer_transport + ) + assert ( + request.display_config.component_modes() + == adapter.values.stream_request.display_config.component_modes() + ) assert output.metadata.source_voxel_spacing == SourceVoxelSpacing((0.5, 0.5)) - acquired_request, reduced_request = (kwargs["stream_request"] for _, kwargs in batches) + acquired_request, reduced_request = ( + kwargs["stream_request"] for _, kwargs in batches + ) assert "site" in acquired_request.display_config.COMPONENT_ORDER assert "site" not in reduced_request.display_config.COMPONENT_ORDER - assert aggregate.metadata.source_provenance.source_image_provenance_planes.contributor_count == 2 + assert ( + aggregate.metadata.source_provenance.source_image_provenance_planes.contributor_count + == 2 + ) np.testing.assert_array_equal(aggregate.content, declaration.data) @pytest.mark.parametrize("sites", ((), (None,), (1,), (1, 1), (1, None), (None, None))) -def test_absence_one_equal_or_incomplete_contributors_cannot_exempt_acquired_plane(sites): +def test_absence_one_equal_or_incomplete_contributors_cannot_exempt_acquired_plane( + sites, +): metadata = _source(sites).metadata.collapse_leading_plane_axis() components = { - "well": "A01", "channel": 2, "z_index": 1, "timepoint": 1, + "well": "A01", + "channel": 2, + "z_index": 1, + "timepoint": 1, } # Explicitly absent scalar identity, not a selected plane with a valid common site. metadata = ImagePayloadMetadata( source_component_metadata=components, - source_image_provenance_planes=SourceImageProvenancePlanes(( - RuntimeSourceImageProvenancePlane( - SourceImageIdentity(component_metadata=components), - contributors=metadata.source_image_provenance_planes.contributors, - ), - )), + source_image_provenance_planes=SourceImageProvenancePlanes( + ( + RuntimeSourceImageProvenancePlane( + SourceImageIdentity(component_metadata=components), + contributors=metadata.source_image_provenance_planes.contributors, + ), + ) + ), ) output = Output("unaddressed.tif", np.zeros((3, 4), dtype=np.uint16), metadata) adapter = _viewer_stream_backend_kwargs() @@ -134,64 +182,119 @@ def test_absence_one_equal_or_incomplete_contributors_cannot_exempt_acquired_pla def test_present_scalar_override_remains_required_despite_varied_contributors(): metadata = (_source((1, 3))).metadata.collapse_leading_plane_axis() - metadata = metadata.replace_fields(source_component_metadata={ - "well": "A01", "site": 7, "channel": 2, "z_index": 1, "timepoint": 1, - }) + metadata = metadata.replace_fields( + source_component_metadata={ + "well": "A01", + "site": 7, + "channel": 2, + "z_index": 1, + "timepoint": 1, + } + ) assert metadata.source_provenance.required_scalar_components(("site",)) == ("site",) output = Output("override.tif", np.zeros((3, 4), dtype=np.uint16), metadata) - request = _viewer_stream_backend_kwargs().to_filemanager_kwargs(output)["stream_request"] + request = _viewer_stream_backend_kwargs().to_filemanager_kwargs(output)[ + "stream_request" + ] assert "site" in request.display_config.COMPONENT_ORDER - assert request.source.metadata.component_metadata_for_item(output.path, 0)["site"] == 7 + assert ( + request.source.metadata.component_metadata_for_item(output.path, 0)["site"] == 7 + ) def _saved_table(tmp_path, *, producer, fields): table = MeasurementTable( - name="Cells", subject=MeasurementSubject(MeasurementScope.ARTIFACT, "Cells"), + name="Cells", + subject=MeasurementSubject(MeasurementScope.ARTIFACT, "Cells"), rows=MeasurementSparseColumnarRows.from_rows( - ({field: 1 for field in fields},), fields=tuple(FieldSpec(field) for field in fields), + ({field: 1 for field in fields},), + fields=tuple(FieldSpec(field) for field in fields), ), ) store = RuntimeValueStore() - record = store.record(RuntimeValue( - key=ArtifactKey(name="Cells", artifact_type=MeasurementsArtifactType, - scope=RuntimeExecutionAxisScope(axis_id="A01")), - data=table, - ), path=f"/runtime/{producer}/Cells", backend="memory") + record = store.record( + RuntimeValue( + key=ArtifactKey( + name="Cells", + artifact_type=MeasurementsArtifactType, + scope=RuntimeExecutionAxisScope(axis_id="A01"), + ), + data=table, + ), + path=f"/runtime/{producer}/Cells", + backend="memory", + ) filemanager = FileManager({"memory": MemoryStorageBackend()}) outputs = materialization_outputs( - MaterializationSpec(CsvOptions(filename_suffix=".csv")), table, - str(tmp_path / f"Cells_{producer}"), filemanager, + MaterializationSpec(CsvOptions(filename_suffix=".csv")), + table, + str(tmp_path / f"Cells_{producer}"), + filemanager, ) for output in outputs: # Persist exactly the original writer's content and declared path. Path(output.path).write_text(output.require_text_content(), encoding="utf-8") observation = StepExecutionObservation( - {RuntimeArtifactAddress.from_record(record): tuple(RuntimeArtifactLocation(output.path, "disk") for output in outputs)}, + { + RuntimeArtifactAddress.from_record(record): tuple( + RuntimeArtifactLocation(output.path, "disk") for output in outputs + ) + }, tuple(Path(output.path) for output in outputs), ) return record, observation def test_same_named_field_and_aggregate_tables_keep_exact_producer_ownership(tmp_path): - acquired, acquired_outputs = _saved_table(tmp_path, producer="step1", fields=("site", "source_image_name")) - aggregate, aggregate_outputs = _saved_table(tmp_path, producer="step3", fields=("area",)) - execution = RuntimeExecutionObservation(( - RuntimeContextObservation("A01", (acquired, aggregate), StepExecutionObservation.combine((acquired_outputs, aggregate_outputs))), - )) + acquired, acquired_outputs = _saved_table( + tmp_path, producer="step1", fields=("site", "source_image_name") + ) + aggregate, aggregate_outputs = _saved_table( + tmp_path, producer="step3", fields=("area",) + ) + execution = RuntimeExecutionObservation( + ( + RuntimeContextObservation( + "A01", + (acquired, aggregate), + StepExecutionObservation.combine((acquired_outputs, aggregate_outputs)), + ), + ) + ) transported = pickle.loads(pickle.dumps(execution)) exports = RuntimeExportObservation.from_runtime_observations((transported,)) - spec = ArtifactSpec.output(name="Cells", artifact_type=MeasurementsArtifactType, - materialization=MaterializationSpec(CsvOptions(filename_suffix=".csv"))) + spec = ArtifactSpec.output( + name="Cells", + artifact_type=MeasurementsArtifactType, + materialization=MaterializationSpec(CsvOptions(filename_suffix=".csv")), + ) expectation = RuntimeExportExpectation.from_output_specs((spec,)) - assert exports.matching_table_outputs(acquired) == acquired_outputs.runtime_export_paths - assert exports.matching_table_outputs(aggregate) == aggregate_outputs.runtime_export_paths - assert runtime_export_failures(expectation, exports, {"A01": (acquired, aggregate)}) == () + assert ( + exports.matching_table_outputs(acquired) + == acquired_outputs.runtime_export_paths + ) + assert ( + exports.matching_table_outputs(aggregate) + == aggregate_outputs.runtime_export_paths + ) + assert ( + runtime_export_failures(expectation, exports, {"A01": (acquired, aggregate)}) + == () + ) # Files and a same-named neighbor cannot replace the missing producer relation. - unowned = RuntimeExportObservation.from_output_paths(exports.output_files, outputs=aggregate_outputs) - failures = runtime_export_failures(expectation, unowned, {"A01": (acquired, aggregate)}) + unowned = RuntimeExportObservation.from_output_paths( + exports.output_files, outputs=aggregate_outputs + ) + failures = runtime_export_failures( + expectation, unowned, {"A01": (acquired, aggregate)} + ) assert len(failures) == 1 and "no matching table output" in failures[0] # A correct address does not waive its own required columns. acquired_outputs.runtime_export_paths[0].write_text("area\n1\n", encoding="utf-8") invalid = RuntimeExportObservation.from_runtime_observations((transported,)) - failures = runtime_export_failures(expectation, invalid, {"A01": (acquired, aggregate)}) - assert any("site" in failure and "source_image_name" in failure for failure in failures) + failures = runtime_export_failures( + expectation, invalid, {"A01": (acquired, aggregate)} + ) + assert any( + "site" in failure and "source_image_name" in failure for failure in failures + ) diff --git a/tests/unit/test_automatic_image_role_materialization_435.py b/tests/unit/test_automatic_image_role_materialization_435.py index bb666aa7f8..7f89a73318 100644 --- a/tests/unit/test_automatic_image_role_materialization_435.py +++ b/tests/unit/test_automatic_image_role_materialization_435.py @@ -11,14 +11,25 @@ from polystore.filemanager import FileManager from polystore.memory import MemoryStorageBackend -from tests.unit.test_function_outputs import context_stub, function_step_plan, record_output_path +from tests.unit.test_function_outputs import ( + context_stub, + function_step_plan, + record_output_path, +) from openhcs.constants.constants import Backend -from openhcs.core.aligned_image_payload import AlignedImageSliceContext, ImagePayloadBundleContext +from openhcs.core.aligned_image_payload import ( + AlignedImageSliceContext, + ImagePayloadBundleContext, +) from openhcs.core.artifacts import ArtifactOutputPlan, ArtifactSpec, ImageArtifactType from openhcs.core.compiled_step_plan import RuntimeArtifactMaterializationPlan from openhcs.core.pipeline.artifact_planning import ArtifactOutputMaterializationPlanner from openhcs.core.runtime_artifact_values import RuntimeValue -from openhcs.core.runtime_image_values import ImageMetadataPayload, ImagePayloadMetadata, ImagePayloadMetadataCompositionMode +from openhcs.core.runtime_image_values import ( + ImageMetadataPayload, + ImagePayloadMetadata, + ImagePayloadMetadataCompositionMode, +) from openhcs.core.runtime_stores import RuntimeValueStore from openhcs.core.source_metadata import SourceVoxelSpacing from openhcs.core.source_projection import SourceProjectionMetadataSerializer @@ -26,7 +37,9 @@ from openhcs.core.steps.function_output_identity import ( FunctionOutputIdentity, ) -from openhcs.core.steps.function_artifact_materialization import ArtifactMaterializationTargetPlan +from openhcs.core.steps.function_artifact_materialization import ( + ArtifactMaterializationTargetPlan, +) from openhcs.core.steps.function_outputs import ( PrimaryImageMetadataTarget, ) @@ -35,11 +48,15 @@ @pytest.mark.parametrize("plane_count", (1, 2)) -def test_automatic_two_role_images_publish_their_actual_saved_occurrences(tmp_path, plane_count): - filemanager = FileManager({ - Backend.DISK.value: DiskStorageBackend(), - Backend.MEMORY.value: MemoryStorageBackend(), - }) +def test_automatic_two_role_images_publish_their_actual_saved_occurrences( + tmp_path, plane_count +): + filemanager = FileManager( + { + Backend.DISK.value: DiskStorageBackend(), + Backend.MEMORY.value: MemoryStorageBackend(), + } + ) context = context_stub(filemanager, parser=SourceSchemaFilenameParser()) context.microscope_handler.source_name = "openhcsdata" context.runtime_value_store = RuntimeValueStore() @@ -56,17 +73,25 @@ def test_automatic_two_role_images_publish_their_actual_saved_occurrences(tmp_pa plan.sub_dir = "images" plan.analysis_results_dir = str(plan.output_dir) plan.runtime_artifact_materialization = RuntimeArtifactMaterializationPlan( - persistent_enabled=True, persistent_backend=Backend.DISK.value, + persistent_enabled=True, + persistent_backend=Backend.DISK.value, ) context.step_plans = {plan.step_index: plan} plans = tuple( ArtifactOutputPlan( - name=name, path=f"/memory/{name}.pkl", artifact_type=ImageArtifactType, + name=name, + path=f"/memory/{name}.pkl", + artifact_type=ImageArtifactType, producer_step_index=plan.step_index, producer_step_scope_id=plan.step_scope_id, producer_step_name=plan.step_name, materialization=ArtifactOutputMaterializationPlanner.materialization_for( - ArtifactSpec(name=name, artifact_type=ImageArtifactType, plan_type=ArtifactOutputPlan), (), + ArtifactSpec( + name=name, + artifact_type=ImageArtifactType, + plan_type=ArtifactOutputPlan, + ), + (), ), ) for name in ("raw_body_processing_role", "capped_outgrowth_processing_role") @@ -80,8 +105,12 @@ def test_automatic_two_role_images_publish_their_actual_saved_occurrences(tmp_pa (np.arange(20, dtype=np.float32).reshape(4, 5) + index) * factor, ImagePayloadMetadata( source_component_metadata={ - "well": "A01", "site": "1", "channel": "2", "z_index": str(index), - "timepoint": "1", "extension": ".tif", + "well": "A01", + "site": "1", + "channel": "2", + "z_index": str(index), + "timepoint": "1", + "extension": ".tif", }, source_image_names=(output.name,), source_voxel_spacing=SourceVoxelSpacing((1.3556, 1.3556)), @@ -90,115 +119,195 @@ def test_automatic_two_role_images_publish_their_actual_saved_occurrences(tmp_pa for index in range(1, plane_count + 1) ) value = ( - payloads[0] if plane_count == 1 else ImagePayloadBundleContext.from_payloads( - payloads, metadata_mode=ImagePayloadMetadataCompositionMode.STACK, + payloads[0] + if plane_count == 1 + else ImagePayloadBundleContext.from_payloads( + payloads, + metadata_mode=ImagePayloadMetadataCompositionMode.STACK, ).compose() ) context.runtime_value_store.record( RuntimeValue.normalize(output, value, axis_id="A01"), - path=output.path, backend=Backend.MEMORY.value, + path=output.path, + backend=Backend.MEMORY.value, ) expected[output.name] = [] for payload in payloads: identity = FunctionOutputIdentity.from_filename_metadata( - context.microscope_handler.parser, payload.metadata, + context.microscope_handler.parser, + payload.metadata, ).with_filename_qualifier(output_context.output_key) - destination = plan.output_dir / identity.filename(context.microscope_handler.parser) + destination = plan.output_dir / identity.filename( + context.microscope_handler.parser + ) filemanager.ensure_directory(plan.output_dir, Backend.MEMORY.value) filemanager.ensure_directory(plan.output_dir, Backend.DISK.value) filemanager.save(payload, str(destination), Backend.MEMORY.value) filemanager.save(payload.data, str(destination), Backend.DISK.value) record_output_path( - context, plan, str(destination), output_context=output_context, - image_metadata=payload.metadata, identity=identity, + context, + plan, + str(destination), + output_context=output_context, + image_metadata=payload.metadata, + identity=identity, ) expected[output.name].append((destination, payload.data)) saved = ArtifactMaterializationTargetPlan.materialize(context, plan) - target = replace(PrimaryImageMetadataTarget.from_plan(plan), artifact_materializations=saved) + target = replace( + PrimaryImageMetadataTarget.from_plan(plan), artifact_materializations=saved + ) # The original implementation raises its real duplicate projection guard # here: both automatic materializations used the same unqualified path. structured = target.produced_projection_entries(context, plan) structured = SourceProjectionMetadataSerializer.projection_fields( structured.projection_paths ) - assert {item["source_alias"] for item in structured["source_projection"]} == set(expected) + assert {item["source_alias"] for item in structured["source_projection"]} == set( + expected + ) assert len(structured["source_projection"]) == 2 * plane_count for materialized in saved: outputs = materialized.outputs_for_backend(Backend.DISK.value) assert len(outputs) == plane_count - for actual, (path, pixels) in zip(outputs, expected[materialized.materialization.output_plan.name], strict=True): + for actual, (path, pixels) in zip( + outputs, + expected[materialized.materialization.output_plan.name], + strict=True, + ): assert actual.path == str(path) np.testing.assert_array_equal(tifffile.imread(path), pixels) target.write(context, produced_plan=plan) document = json.loads((tmp_path / "openhcs_metadata.json").read_text()) - reopened = VirtualWorkspaceSourceProjection.from_openhcs_metadata(tmp_path, document) + reopened = VirtualWorkspaceSourceProjection.from_openhcs_metadata( + tmp_path, document + ) for name, expected_planes in expected.items(): for path, pixels in expected_planes: - projection = reopened.source_projections_by_virtual_path[str(path.relative_to(tmp_path))] + projection = reopened.source_projections_by_virtual_path[ + str(path.relative_to(tmp_path)) + ] assert projection.source_alias == name assert projection.address.as_component_metadata()["channel"] == "2" - assert projection.image_metadata.source_voxel_spacing == SourceVoxelSpacing((1.3556, 1.3556)) - np.testing.assert_array_equal(tifffile.imread(tmp_path / projection.ref.backend_address), pixels) + assert projection.image_metadata.source_voxel_spacing == SourceVoxelSpacing( + (1.3556, 1.3556) + ) + np.testing.assert_array_equal( + tifffile.imread(tmp_path / projection.ref.backend_address), pixels + ) -@pytest.mark.parametrize("purpose", ( - "explicit_source", "explicit_template", "terminal_template", - "explicit_artifact", "terminal_artifact", "unnamed_terminal", "unnamed_explicit", - "explicit_source_declared_terminal", "terminal_source_declared_explicit", -)) -def test_retained_role_policy_preserves_authored_and_unnamed_image_paths(tmp_path, purpose): +@pytest.mark.parametrize( + "purpose", + ( + "explicit_source", + "explicit_template", + "terminal_template", + "explicit_artifact", + "terminal_artifact", + "unnamed_terminal", + "unnamed_explicit", + "explicit_source_declared_terminal", + "terminal_source_declared_explicit", + ), +) +def test_retained_role_policy_preserves_authored_and_unnamed_image_paths( + tmp_path, purpose +): from openhcs.processing.materialization import ( - ImageFileOptions, MaterializationSpec, MaterializedFilenameIdentity, - TerminalMaterializationSpec, materialization_outputs, + ImageFileOptions, + MaterializationSpec, + MaterializedFilenameIdentity, + TerminalMaterializationSpec, + materialization_outputs, ) options = ImageFileOptions( filename_suffix=".tif", - relative_path_template=("authored/plane_{index}.tif" if purpose.endswith("template") else None), + relative_path_template=( + "authored/plane_{index}.tif" if purpose.endswith("template") else None + ), filename_identity=( - MaterializedFilenameIdentity.ARTIFACT_NAME if purpose.endswith("artifact") + MaterializedFilenameIdentity.ARTIFACT_NAME + if purpose.endswith("artifact") else MaterializedFilenameIdentity.SOURCE_IDENTITY ), ) spec = ( - TerminalMaterializationSpec(options) if purpose.startswith("terminal") or purpose == "unnamed_terminal" + TerminalMaterializationSpec(options) + if purpose.startswith("terminal") or purpose == "unnamed_terminal" else MaterializationSpec(options) ) declared_spec = ( - TerminalMaterializationSpec(options) if purpose == "explicit_source_declared_terminal" - else MaterializationSpec(options) if purpose == "terminal_source_declared_explicit" - else spec + TerminalMaterializationSpec(options) + if purpose == "explicit_source_declared_terminal" + else ( + MaterializationSpec(options) + if purpose == "terminal_source_declared_explicit" + else spec + ) ) - output_plan = None if purpose in ("unnamed_terminal", "unnamed_explicit") else ArtifactOutputPlan( - name="DeclaredRole", path="/memory/DeclaredRole.pkl", - artifact_type=ImageArtifactType, materialization=declared_spec, + output_plan = ( + None + if purpose in ("unnamed_terminal", "unnamed_explicit") + else ArtifactOutputPlan( + name="DeclaredRole", + path="/memory/DeclaredRole.pkl", + artifact_type=ImageArtifactType, + materialization=declared_spec, + ) ) payloads = tuple( - ImageMetadataPayload(np.ones((4, 5), dtype=np.float32) * index, ImagePayloadMetadata( - source_component_metadata={ - "well": "A01", "site": "1", "channel": "2", "z_index": str(index), - "timepoint": "1", "extension": ".tif", - }, - )) + ImageMetadataPayload( + np.ones((4, 5), dtype=np.float32) * index, + ImagePayloadMetadata( + source_component_metadata={ + "well": "A01", + "site": "1", + "channel": "2", + "z_index": str(index), + "timepoint": "1", + "extension": ".tif", + }, + ), + ) for index in (1, 2) ) value = ImagePayloadBundleContext.from_payloads( - payloads, metadata_mode=ImagePayloadMetadataCompositionMode.STACK, + payloads, + metadata_mode=ImagePayloadMetadataCompositionMode.STACK, ).compose() filemanager = FileManager({Backend.MEMORY.value: MemoryStorageBackend()}) context = context_stub(filemanager, parser=SourceSchemaFilenameParser()) outputs = materialization_outputs( - spec, value, str(tmp_path / "AuthoredBase"), filemanager, - context=context, output_plan=output_plan, + spec, + value, + str(tmp_path / "AuthoredBase"), + filemanager, + context=context, + output_plan=output_plan, variable_components=(Microscopy.ZIndex,), ) expected_paths = ( - ("authored/plane_1.tif", "authored/plane_2.tif") if purpose.endswith("template") - else ("AuthoredBase.tif",) if purpose.endswith("artifact") - else ("A01_s001_w2_z001_t001_DeclaredRole.tif", "A01_s001_w2_z002_t001_DeclaredRole.tif") if purpose == "terminal_source_declared_explicit" - else ("A01_s001_w2_z001_t001.tif", "A01_s001_w2_z002_t001.tif") + ("authored/plane_1.tif", "authored/plane_2.tif") + if purpose.endswith("template") + else ( + ("AuthoredBase.tif",) + if purpose.endswith("artifact") + else ( + ( + "A01_s001_w2_z001_t001_DeclaredRole.tif", + "A01_s001_w2_z002_t001_DeclaredRole.tif", + ) + if purpose == "terminal_source_declared_explicit" + else ("A01_s001_w2_z001_t001.tif", "A01_s001_w2_z002_t001.tif") + ) + ) + ) + assert ( + tuple(Path(output.path).relative_to(tmp_path).as_posix() for output in outputs) + == expected_paths ) - assert tuple(Path(output.path).relative_to(tmp_path).as_posix() for output in outputs) == expected_paths if len(outputs) == 2: for actual, expected in zip(outputs, payloads, strict=True): np.testing.assert_array_equal(actual.content, expected.data) @@ -207,60 +316,130 @@ def test_retained_role_policy_preserves_authored_and_unnamed_image_paths(tmp_pat def test_authored_duplicate_template_still_fails_before_any_save(tmp_path): - from openhcs.processing.materialization import ImageFileOptions, TerminalMaterializationSpec, materialization_outputs + from openhcs.processing.materialization import ( + ImageFileOptions, + TerminalMaterializationSpec, + materialization_outputs, + ) - spec = TerminalMaterializationSpec(ImageFileOptions( - filename_suffix=".tif", relative_path_template="same.tif", - )) - output_plan = ArtifactOutputPlan(name="NamedRole", path="/memory/role.pkl", artifact_type=ImageArtifactType, materialization=spec) + spec = TerminalMaterializationSpec( + ImageFileOptions( + filename_suffix=".tif", + relative_path_template="same.tif", + ) + ) + output_plan = ArtifactOutputPlan( + name="NamedRole", + path="/memory/role.pkl", + artifact_type=ImageArtifactType, + materialization=spec, + ) payloads = tuple( - ImageMetadataPayload(np.ones((4, 5), dtype=np.float32), ImagePayloadMetadata( - source_component_metadata={"well": "A01", "site": "1", "channel": "2", "z_index": str(index), "timepoint": "1", "extension": ".tif"}, - )) for index in (1, 2) + ImageMetadataPayload( + np.ones((4, 5), dtype=np.float32), + ImagePayloadMetadata( + source_component_metadata={ + "well": "A01", + "site": "1", + "channel": "2", + "z_index": str(index), + "timepoint": "1", + "extension": ".tif", + }, + ), + ) + for index in (1, 2) ) - value = ImagePayloadBundleContext.from_payloads(payloads, metadata_mode=ImagePayloadMetadataCompositionMode.STACK).compose() + value = ImagePayloadBundleContext.from_payloads( + payloads, metadata_mode=ImagePayloadMetadataCompositionMode.STACK + ).compose() manager = FileManager({Backend.MEMORY.value: MemoryStorageBackend()}) with pytest.raises(ValueError, match="produced duplicate paths"): - materialization_outputs(spec, value, str(tmp_path / "role"), manager, - context=context_stub(manager, parser=SourceSchemaFilenameParser()), - output_plan=output_plan) + materialization_outputs( + spec, + value, + str(tmp_path / "role"), + manager, + context=context_stub(manager, parser=SourceSchemaFilenameParser()), + output_plan=output_plan, + ) assert not (tmp_path / "same.tif").exists() @pytest.mark.parametrize("suffix", (".tif", ".png", ".ome.tif", ".labels.tif")) @pytest.mark.parametrize("plane_count", (1, 2)) -def test_retained_image_qualifier_respects_complete_writer_suffix(tmp_path, suffix, plane_count): - from openhcs.core.steps.function_artifact_materialization import RuntimeArtifactMaterialization - from openhcs.processing.materialization import ImageFileOptions, TerminalMaterializationSpec +def test_retained_image_qualifier_respects_complete_writer_suffix( + tmp_path, suffix, plane_count +): + from openhcs.core.steps.function_artifact_materialization import ( + RuntimeArtifactMaterialization, + ) + from openhcs.processing.materialization import ( + ImageFileOptions, + TerminalMaterializationSpec, + ) manager = FileManager({Backend.MEMORY.value: MemoryStorageBackend()}) context = context_stub(manager, parser=SourceSchemaFilenameParser()) - plan = function_step_plan("Retained format role", variable_components=(Microscopy.ZIndex,) if plane_count > 1 else ()) + plan = function_step_plan( + "Retained format role", + variable_components=(Microscopy.ZIndex,) if plane_count > 1 else (), + ) plan.output_dir = tmp_path / "images" plan.analysis_results_dir = str(plan.output_dir) output_plan = ArtifactOutputPlan( - name="Role Name", path="/memory/role.pkl", artifact_type=ImageArtifactType, - materialization=TerminalMaterializationSpec(ImageFileOptions(filename_suffix=suffix)), + name="Role Name", + path="/memory/role.pkl", + artifact_type=ImageArtifactType, + materialization=TerminalMaterializationSpec( + ImageFileOptions(filename_suffix=suffix) + ), ) payloads = tuple( - ImageMetadataPayload(np.ones((4, 5), dtype=np.uint8) * index, ImagePayloadMetadata( - source_component_metadata={"well": "A01", "site": "1", "channel": "2", "z_index": str(index), "timepoint": "1", "extension": ".tif"}, - )) for index in range(1, plane_count + 1) + ImageMetadataPayload( + np.ones((4, 5), dtype=np.uint8) * index, + ImagePayloadMetadata( + source_component_metadata={ + "well": "A01", + "site": "1", + "channel": "2", + "z_index": str(index), + "timepoint": "1", + "extension": ".tif", + }, + ), + ) + for index in range(1, plane_count + 1) + ) + value = ( + payloads[0] + if plane_count == 1 + else ImagePayloadBundleContext.from_payloads( + payloads, + metadata_mode=ImagePayloadMetadataCompositionMode.STACK, + ).compose() ) - value = payloads[0] if plane_count == 1 else ImagePayloadBundleContext.from_payloads( - payloads, metadata_mode=ImagePayloadMetadataCompositionMode.STACK, - ).compose() store = RuntimeValueStore() - record = store.record(RuntimeValue.normalize(output_plan, value, axis_id="A01"), path=output_plan.path, backend=Backend.MEMORY.value) + record = store.record( + RuntimeValue.normalize(output_plan, value, axis_id="A01"), + path=output_plan.path, + backend=Backend.MEMORY.value, + ) outputs = RuntimeArtifactMaterialization.from_record( - output_plan=output_plan, record=record, plan=plan, context=context, + output_plan=output_plan, + record=record, + plan=plan, + context=context, ).outputs(plan, context) assert len(outputs) == plane_count for index, (output, payload) in enumerate(zip(outputs, payloads, strict=True), 1): identity = FunctionOutputIdentity.from_filename_metadata( - context.microscope_handler.parser, payload.metadata, + context.microscope_handler.parser, + payload.metadata, ).with_filename_qualifier(output_plan.name) - expected = replace(identity, extension=suffix).filename(context.microscope_handler.parser) + expected = replace(identity, extension=suffix).filename( + context.microscope_handler.parser + ) assert Path(output.path).name == expected assert Path(output.path).name.endswith("_Role_Name" + suffix) assert output.metadata.source_component_metadata["channel"] == "2" @@ -269,60 +448,121 @@ def test_retained_image_qualifier_respects_complete_writer_suffix(tmp_path, suff def test_distinct_aliases_that_normalize_to_one_filename_still_conflict(tmp_path): - from openhcs.core.steps.function_artifact_materialization import RuntimeArtifactMaterialization - from openhcs.processing.materialization import ImageFileOptions, TerminalMaterializationSpec + from openhcs.core.steps.function_artifact_materialization import ( + RuntimeArtifactMaterialization, + ) + from openhcs.processing.materialization import ( + ImageFileOptions, + TerminalMaterializationSpec, + ) manager = FileManager({Backend.MEMORY.value: MemoryStorageBackend()}) context = context_stub(manager, parser=SourceSchemaFilenameParser()) plan = function_step_plan("Ambiguous role spelling") plan.output_dir = tmp_path / "images" plan.analysis_results_dir = str(plan.output_dir) - value = ImageMetadataPayload(np.ones((4, 5), dtype=np.float32), ImagePayloadMetadata( - source_component_metadata={"well": "A01", "site": "1", "channel": "2", "z_index": "1", "timepoint": "1", "extension": ".tif"}, - )) + value = ImageMetadataPayload( + np.ones((4, 5), dtype=np.float32), + ImagePayloadMetadata( + source_component_metadata={ + "well": "A01", + "site": "1", + "channel": "2", + "z_index": "1", + "timepoint": "1", + "extension": ".tif", + }, + ), + ) paths = [] store = RuntimeValueStore() for name in ("a/b", "a b"): - output_plan = ArtifactOutputPlan(name=name, path=f"/memory/{name}.pkl", artifact_type=ImageArtifactType, - materialization=TerminalMaterializationSpec(ImageFileOptions(filename_suffix=".tif"))) - record = store.record(RuntimeValue.normalize(output_plan, value, axis_id="A01"), path=output_plan.path, backend=Backend.MEMORY.value) - outputs = RuntimeArtifactMaterialization.from_record(output_plan=output_plan, record=record, plan=plan, context=context).outputs(plan, context) + output_plan = ArtifactOutputPlan( + name=name, + path=f"/memory/{name}.pkl", + artifact_type=ImageArtifactType, + materialization=TerminalMaterializationSpec( + ImageFileOptions(filename_suffix=".tif") + ), + ) + record = store.record( + RuntimeValue.normalize(output_plan, value, axis_id="A01"), + path=output_plan.path, + backend=Backend.MEMORY.value, + ) + outputs = RuntimeArtifactMaterialization.from_record( + output_plan=output_plan, record=record, plan=plan, context=context + ).outputs(plan, context) paths.append(outputs[0].path) assert paths[0] == paths[1] # Existing main-flow destination authority rejects this declared naming # collision instead of choosing a new suffix or inventing a channel. from openhcs.core.steps.function_output_identity import FunctionOutputIdentity + with pytest.raises(ValueError, match="duplicate"): - FunctionOutputIdentity.validate_output_paths(paths, input_paths=[], step_name=plan.step_name, pattern_repr="declared roles") + FunctionOutputIdentity.validate_output_paths( + paths, + input_paths=[], + step_name=plan.step_name, + pattern_repr="declared roles", + ) -def test_batch_binds_actual_purpose_once_without_rewriting_plan_or_source(monkeypatch, tmp_path): - from openhcs.processing.materialization import ImageFileOptions, MaterializationSpec, TerminalMaterializationSpec - from openhcs.processing.materialization.core import MaterializationBatch, MaterializationContext, Output, WriterSpec, _WRITERS_BY_OPTIONS +def test_batch_binds_actual_purpose_once_without_rewriting_plan_or_source( + monkeypatch, tmp_path +): + from openhcs.processing.materialization import ( + ImageFileOptions, + MaterializationSpec, + TerminalMaterializationSpec, + ) + from openhcs.processing.materialization.core import ( + MaterializationBatch, + MaterializationContext, + Output, + WriterSpec, + _WRITERS_BY_OPTIONS, + ) options = ImageFileOptions(filename_suffix=".tif") declared = TerminalMaterializationSpec(options) explicit = MaterializationSpec(options) - plan = ArtifactOutputPlan(name="Role", path="/memory/role", artifact_type=ImageArtifactType, materialization=declared) + plan = ArtifactOutputPlan( + name="Role", + path="/memory/role", + artifact_type=ImageArtifactType, + materialization=declared, + ) manager = FileManager({Backend.MEMORY.value: MemoryStorageBackend()}) source = {"physical_source": "original"} context = MaterializationContext( - base_path=str(tmp_path / "role"), backends=[], backend_kwargs={}, - filemanager=manager, extra_inputs=source, output_plan=plan, + base_path=str(tmp_path / "role"), + backends=[], + backend_kwargs={}, + filemanager=manager, + extra_inputs=source, + output_plan=plan, materialization_spec=declared, ) seen = [] + def custom_writer(data, actual_options, actual_context): assert actual_options is options assert actual_context.output_plan is plan assert actual_context.extra_inputs is source seen.append(actual_context) return [Output(path=str(tmp_path / "custom.tif"), content=data)] - monkeypatch.setitem(_WRITERS_BY_OPTIONS, ImageFileOptions, WriterSpec( - options_type=ImageFileOptions, - write=custom_writer, primary_path=lambda outputs: outputs[0].path, - candidate_paths=lambda _options, base: (base,), - )) + + monkeypatch.setitem( + _WRITERS_BY_OPTIONS, + ImageFileOptions, + WriterSpec( + options_type=ImageFileOptions, + write=custom_writer, + primary_path=lambda outputs: outputs[0].path, + candidate_paths=lambda _options, base: (base,), + ), + ) pixels = np.ones((4, 5), dtype=np.float32) first = MaterializationBatch.render(explicit, pixels, context) second = MaterializationBatch.render(declared, pixels, first.context) @@ -335,20 +575,48 @@ def custom_writer(data, actual_options, actual_context): assert source == {"physical_source": "original"} -def test_direct_image_writer_without_purpose_binding_retains_legacy_source_path(tmp_path): - from openhcs.processing.materialization import ImageFileOptions, TerminalMaterializationSpec - from openhcs.processing.materialization.core import MaterializationContext, write_image_file +def test_direct_image_writer_without_purpose_binding_retains_legacy_source_path( + tmp_path, +): + from openhcs.processing.materialization import ( + ImageFileOptions, + TerminalMaterializationSpec, + ) + from openhcs.processing.materialization.core import ( + MaterializationContext, + write_image_file, + ) options = ImageFileOptions(filename_suffix=".tif") - plan = ArtifactOutputPlan(name="Role", path="/memory/role", artifact_type=ImageArtifactType, - materialization=TerminalMaterializationSpec(options)) + plan = ArtifactOutputPlan( + name="Role", + path="/memory/role", + artifact_type=ImageArtifactType, + materialization=TerminalMaterializationSpec(options), + ) manager = FileManager({Backend.MEMORY.value: MemoryStorageBackend()}) - value = ImageMetadataPayload(np.ones((4, 5), dtype=np.float32), ImagePayloadMetadata( - source_component_metadata={"well": "A01", "site": "1", "channel": "2", "z_index": "1", "timepoint": "1", "extension": ".tif"}, - )) - context = MaterializationContext(base_path=str(tmp_path / "role"), backends=[], backend_kwargs={}, - filemanager=manager, extra_inputs={}, output_plan=plan, - context=context_stub(manager, parser=SourceSchemaFilenameParser())) + value = ImageMetadataPayload( + np.ones((4, 5), dtype=np.float32), + ImagePayloadMetadata( + source_component_metadata={ + "well": "A01", + "site": "1", + "channel": "2", + "z_index": "1", + "timepoint": "1", + "extension": ".tif", + }, + ), + ) + context = MaterializationContext( + base_path=str(tmp_path / "role"), + backends=[], + backend_kwargs={}, + filemanager=manager, + extra_inputs={}, + output_plan=plan, + context=context_stub(manager, parser=SourceSchemaFilenameParser()), + ) outputs = write_image_file(value, options, context) assert Path(outputs[0].path).name == "A01_s001_w2_z001_t001.tif" assert context.materialization_spec is None @@ -357,23 +625,50 @@ def test_direct_image_writer_without_purpose_binding_retains_legacy_source_path( @pytest.mark.parametrize("purpose", ("named_terminal", "explicit", "unnamed_terminal")) -def test_parser_free_render_requires_parser_only_for_automatic_named_role(tmp_path, purpose): - from openhcs.processing.materialization import ImageFileOptions, MaterializationSpec, TerminalMaterializationSpec, materialization_outputs +def test_parser_free_render_requires_parser_only_for_automatic_named_role( + tmp_path, purpose +): + from openhcs.processing.materialization import ( + ImageFileOptions, + MaterializationSpec, + TerminalMaterializationSpec, + materialization_outputs, + ) options = ImageFileOptions(filename_suffix=".tif") - spec = MaterializationSpec(options) if purpose == "explicit" else TerminalMaterializationSpec(options) - plan = None if purpose == "unnamed_terminal" else ArtifactOutputPlan( - name="Role", path="/memory/role", artifact_type=ImageArtifactType, materialization=spec, + spec = ( + MaterializationSpec(options) + if purpose == "explicit" + else TerminalMaterializationSpec(options) + ) + plan = ( + None + if purpose == "unnamed_terminal" + else ArtifactOutputPlan( + name="Role", + path="/memory/role", + artifact_type=ImageArtifactType, + materialization=spec, + ) ) manager = FileManager({Backend.MEMORY.value: MemoryStorageBackend()}) - value = ImageMetadataPayload(np.ones((4, 5), dtype=np.float32), ImagePayloadMetadata( - source_path="/physical/OriginalSource.tif", - )) + value = ImageMetadataPayload( + np.ones((4, 5), dtype=np.float32), + ImagePayloadMetadata( + source_path="/physical/OriginalSource.tif", + ), + ) if purpose == "named_terminal": - with pytest.raises(ValueError, match="Parser-backed source-stem authority requires a parser"): - materialization_outputs(spec, value, str(tmp_path / "Role"), manager, output_plan=plan) + with pytest.raises( + ValueError, match="Parser-backed source-stem authority requires a parser" + ): + materialization_outputs( + spec, value, str(tmp_path / "Role"), manager, output_plan=plan + ) else: - outputs = materialization_outputs(spec, value, str(tmp_path / "Role"), manager, output_plan=plan) + outputs = materialization_outputs( + spec, value, str(tmp_path / "Role"), manager, output_plan=plan + ) assert Path(outputs[0].path).name == "OriginalSource.tif" np.testing.assert_array_equal(outputs[0].content, value.data) assert value.metadata.source_path == "/physical/OriginalSource.tif" diff --git a/tests/unit/test_derived_role_publication_435.py b/tests/unit/test_derived_role_publication_435.py index 4acdffdfef..7ac6a14ded 100644 --- a/tests/unit/test_derived_role_publication_435.py +++ b/tests/unit/test_derived_role_publication_435.py @@ -11,57 +11,102 @@ from polystore.memory import MemoryStorageBackend from tests.unit.test_function_outputs import ( - context_stub, function_step_plan, record_output_path, + context_stub, + function_step_plan, + record_output_path, ) from openhcs.core.dataset_sources.source_schema import SourceSchemaFilenameParser from openhcs.constants.constants import Backend -from openhcs.core.aligned_image_payload import AlignedImageSliceContext, stack_image_payloads -from openhcs.core.artifacts import ArtifactOutputPlan, ImageArtifactType, ObjectLabelsArtifactType -from openhcs.core.component_group_scope import ComponentGroupScope, RuntimeExecutionAxisScope +from openhcs.core.aligned_image_payload import ( + AlignedImageSliceContext, + stack_image_payloads, +) +from openhcs.core.artifacts import ( + ArtifactOutputPlan, + ImageArtifactType, + ObjectLabelsArtifactType, +) +from openhcs.core.component_group_scope import ( + ComponentGroupScope, + RuntimeExecutionAxisScope, +) from openhcs.core.compiled_step_plan import RuntimeArtifactMaterializationPlan from openhcs.core.runtime_artifact_values import RuntimeValue from openhcs.core.runtime_image_values import ( - ImageMetadataPayload, ImagePayloadMetadata, ImagePayloadMetadataCompositionMode, + ImageMetadataPayload, + ImagePayloadMetadata, + ImagePayloadMetadataCompositionMode, ) from openhcs.core.runtime_stores import RuntimeValueStore -from openhcs.core.runtime_object_label_building import SourceImageObjectLabelBuildRequest +from openhcs.core.runtime_object_label_building import ( + SourceImageObjectLabelBuildRequest, +) from openhcs.core.source_metadata import SourceVoxelSpacing from openhcs.core.source_image_provenance import ( - SourceImageProvenance, SourceImageProvenancePlanes, + SourceImageProvenance, + SourceImageProvenancePlanes, ) from openhcs.core.source_projection import SourceProjectionMetadataSerializer from openhcs.core.source_workspace_projection import VirtualWorkspaceSourceProjection -from openhcs.core.steps.function_artifact_materialization import ArtifactMaterializationTargetPlan +from openhcs.core.steps.function_artifact_materialization import ( + ArtifactMaterializationTargetPlan, +) from openhcs.core.steps.function_outputs import ( PrimaryImageMetadataTarget, ) from openhcs.core.steps.function_output_identity import FunctionOutputIdentity -from openhcs.core.runtime_plane_projection import RuntimePlaneAxis, RuntimePlaneAxisValueProjection +from openhcs.core.runtime_plane_projection import ( + RuntimePlaneAxis, + RuntimePlaneAxisValueProjection, +) from python_introspect import to_jsonable from openhcs.processing.materialization import ( - ImageFileOptions, MaterializationSpec, MaterializedFilenameIdentity, + ImageFileOptions, + MaterializationSpec, + MaterializedFilenameIdentity, ) from openhcs.domains.microscopy.axes import Microscopy -@pytest.mark.parametrize("scenario,aggregate", ( - *((scenario, False) for scenario in ( - "same_occurrence", "different_path", "stale_scope", "conflicting_metadata", "different_kind", - )), - *((scenario, True) for scenario in ( - "same_occurrence", "different_path", "stale_scope", "conflicting_metadata", "different_kind", - )), - ("conflicting_scope", True), - ("mosaic_context", False), -)) -def test_saved_roles_publish_once_per_persisted_occurrence(tmp_path, scenario, aggregate): +@pytest.mark.parametrize( + "scenario,aggregate", + ( + *( + (scenario, False) + for scenario in ( + "same_occurrence", + "different_path", + "stale_scope", + "conflicting_metadata", + "different_kind", + ) + ), + *( + (scenario, True) + for scenario in ( + "same_occurrence", + "different_path", + "stale_scope", + "conflicting_metadata", + "different_kind", + ) + ), + ("conflicting_scope", True), + ("mosaic_context", False), + ), +) +def test_saved_roles_publish_once_per_persisted_occurrence( + tmp_path, scenario, aggregate +): mosaic = scenario == "mosaic_context" same_persisted_path = scenario != "different_path" - filemanager = FileManager({ - Backend.DISK.value: DiskStorageBackend(), - Backend.MEMORY.value: MemoryStorageBackend(), - }) + filemanager = FileManager( + { + Backend.DISK.value: DiskStorageBackend(), + Backend.MEMORY.value: MemoryStorageBackend(), + } + ) context = context_stub(filemanager, parser=SourceSchemaFilenameParser()) context.microscope_handler.source_name = "openhcsdata" context.runtime_value_store = RuntimeValueStore() @@ -75,24 +120,29 @@ def test_saved_roles_publish_once_per_persisted_occurrence(tmp_path, scenario, a plan.sub_dir = "images" plan.analysis_results_dir = str(plan.output_dir) plan.runtime_artifact_materialization = RuntimeArtifactMaterializationPlan( - persistent_enabled=True, persistent_backend=Backend.DISK.value, + persistent_enabled=True, + persistent_backend=Backend.DISK.value, ) context.step_plans = {plan.step_index: plan} filemanager.ensure_directory(plan.output_dir, Backend.MEMORY.value) components = { - "well": "A01", "site": "1", "channel": "2", - "z_index": "1", "timepoint": "1", "extension": ".tif", + "well": "A01", + "site": "1", + "channel": "2", + "z_index": "1", + "timepoint": "1", + "extension": ".tif", } roles = ( - (("MosaicRole", 1.0),) if mosaic - else (("RawRole", 1.0), ("CappedRole", 0.5)) + (("MosaicRole", 1.0),) if mosaic else (("RawRole", 1.0), ("CappedRole", 0.5)) ) if aggregate: components.update(channel="1", z_index="0") plan.variable_components = (Microscopy.ZIndex,) plan.group_by = Microscopy.Channel plan.execution_group_scope = ComponentGroupScope.from_raw( - ("1",), component=Microscopy.Channel, + ("1",), + component=Microscopy.Channel, ) source_planes = [] source_dir = tmp_path / "physical_source" @@ -101,13 +151,16 @@ def test_saved_roles_publish_once_per_persisted_occurrence(tmp_path, scenario, a for z_index, plane in enumerate(pixels): path = source_dir / f"A01_s001_w1_z{z_index:03d}_t001.tif" filemanager.save(plane, str(path), Backend.DISK.value) - source_planes.append(ImagePayloadMetadata( - source_path=str(path), - source_component_metadata={**components, "z_index": str(z_index)}, - source_voxel_spacing=SourceVoxelSpacing((2.0, 0.65, 0.65)), - ).payload_with(filemanager.load(str(path), Backend.DISK.value))) + source_planes.append( + ImagePayloadMetadata( + source_path=str(path), + source_component_metadata={**components, "z_index": str(z_index)}, + source_voxel_spacing=SourceVoxelSpacing((2.0, 0.65, 0.65)), + ).payload_with(filemanager.load(str(path), Backend.DISK.value)) + ) source_stack = stack_image_payloads( - source_planes, metadata_mode=ImagePayloadMetadataCompositionMode.STACK, + source_planes, + metadata_mode=ImagePayloadMetadataCompositionMode.STACK, ) roles = (("fixture_image_494", 1.0),) for role, scale in roles: @@ -117,29 +170,41 @@ def test_saved_roles_publish_once_per_persisted_occurrence(tmp_path, scenario, a f"_z{int(components['z_index']):03d}_t001_{role}.tif" ) output_plan = ArtifactOutputPlan( - name=role, path=f"/memory/{filename.removesuffix('.tif')}.pkl", + name=role, + path=f"/memory/{filename.removesuffix('.tif')}.pkl", artifact_type=( ObjectLabelsArtifactType - if scenario == "different_kind" and tested_role else ImageArtifactType + if scenario == "different_kind" and tested_role + else ImageArtifactType ), producer_step_index=plan.step_index, producer_step_scope_id=( "different-step-scope" - if scenario == "stale_scope" and tested_role else plan.step_scope_id + if scenario == "stale_scope" and tested_role + else plan.step_scope_id ), producer_step_name=plan.step_name, - materialization=MaterializationSpec(ImageFileOptions( - filename_suffix=".tif", - filename_identity=( - MaterializedFilenameIdentity.ARTIFACT_NAME if aggregate - else MaterializedFilenameIdentity.SOURCE_IDENTITY - ), - relative_path_template=(filename if same_persisted_path else filename.replace(".tif", "_independent.tif")), - )), + materialization=MaterializationSpec( + ImageFileOptions( + filename_suffix=".tif", + filename_identity=( + MaterializedFilenameIdentity.ARTIFACT_NAME + if aggregate + else MaterializedFilenameIdentity.SOURCE_IDENTITY + ), + relative_path_template=( + filename + if same_persisted_path + else filename.replace(".tif", "_independent.tif") + ), + ) + ), ) if aggregate: output_plan = replace( - output_plan, group_keys=("1",), group_component=Microscopy.Channel, + output_plan, + group_keys=("1",), + group_component=Microscopy.Channel, variable_components=(Microscopy.ZIndex,), ) plan.artifact_outputs[output_plan.ref()] = output_plan @@ -172,128 +237,205 @@ def test_saved_roles_publish_once_per_persisted_occurrence(tmp_path, scenario, a component_metadata=({"site": "1"}, {"site": "2"}), ), ), - ).payload_with(np.stack([ - filemanager.load(str(path), Backend.DISK.value) - for path in contributor_paths - ])) + ).payload_with( + np.stack( + [ + filemanager.load(str(path), Backend.DISK.value) + for path in contributor_paths + ] + ) + ) if aggregate: payload = ImageArtifactType.contextualize_output( - source_stack, pixels.copy(), output_plan, - RuntimePlaneAxisValueProjection(RuntimePlaneAxis.RUNTIME_SLICE, (), None, 4), + source_stack, + pixels.copy(), + output_plan, + RuntimePlaneAxisValueProjection( + RuntimePlaneAxis.RUNTIME_SLICE, (), None, 4 + ), ) if aggregate: execution_scope = RuntimeExecutionAxisScope.from_raw( - "A01", component=Microscopy.Channel, value="1", + "A01", + component=Microscopy.Channel, + value="1", fixed_component_values=( - (Microscopy.Site, "1"), (Microscopy.Timepoint, "1"), + (Microscopy.Site, "1"), + (Microscopy.Timepoint, "1"), ), ) payload = RuntimeValue.normalize_for_execution_scope( - replace(output_plan, artifact_type=ImageArtifactType), payload, + replace(output_plan, artifact_type=ImageArtifactType), + payload, execution_scope=execution_scope, ).data artifact_payload = payload if tested_role and scenario == "conflicting_metadata": conflicting_fields = ( {"source_voxel_spacing": SourceVoxelSpacing((3.0, 0.65, 0.65))} - if aggregate else {"source_image_names": ("IndependentRole",)} + if aggregate + else {"source_image_names": ("IndependentRole",)} ) artifact_payload = payload.metadata.replace_fields( **conflicting_fields ).payload_with(payload.data) if tested_role and scenario == "different_kind": artifact_payload = SourceImageObjectLabelBuildRequest( - image=payload, labels=np.ones(payload.data.shape, dtype=np.int32), - declared_object_count=1, declared_object_ids=(1,), + image=payload, + labels=np.ones(payload.data.shape, dtype=np.int32), + declared_object_count=1, + declared_object_ids=(1,), ).payload() output_identity = ( FunctionOutputIdentity.from_metadata( - context.microscope_handler.parser, payload.metadata, + context.microscope_handler.parser, + payload.metadata, variable_components=plan.variable_components, - ) if aggregate else None + ) + if aggregate + else None ) value = ( RuntimeValue.normalize_for_execution_scope( - output_plan, artifact_payload, + output_plan, + artifact_payload, execution_scope=( RuntimeExecutionAxisScope.from_raw( - "A01", component=Microscopy.Channel, value="1", + "A01", + component=Microscopy.Channel, + value="1", fixed_component_values=( - (Microscopy.Site, "2"), (Microscopy.Timepoint, "1"), + (Microscopy.Site, "2"), + (Microscopy.Timepoint, "1"), ), - ) if scenario == "conflicting_scope" else execution_scope + ) + if scenario == "conflicting_scope" + else execution_scope ), - ) if aggregate else ( + ) + if aggregate + else ( RuntimeValue.normalize_for_execution_scope( - output_plan, artifact_payload, + output_plan, + artifact_payload, execution_scope=RuntimeExecutionAxisScope.from_raw( - "A01", component=None, value=None, fixed_component_values=( - (Microscopy.Channel, "2"), (Microscopy.ZIndex, "1"), + "A01", + component=None, + value=None, + fixed_component_values=( + (Microscopy.Channel, "2"), + (Microscopy.ZIndex, "1"), (Microscopy.Timepoint, "1"), ), ), - ) if mosaic else RuntimeValue.normalize(output_plan, artifact_payload, axis_id="A01") + ) + if mosaic + else RuntimeValue.normalize( + output_plan, artifact_payload, axis_id="A01" + ) ) ) if mosaic: payload = value.data output_identity = FunctionOutputIdentity( component_values={ - key: value for key, value in components.items() + key: value + for key, value in components.items() if key not in ("site", "extension") }, filename_component_values={ - key: value for key, value in components.items() if key != "extension" + key: value + for key, value in components.items() + if key != "extension" }, - extension=".tif", source="collapsed Mosaic contributors", + extension=".tif", + source="collapsed Mosaic contributors", ) context.runtime_value_store.record( value, - path=output_plan.path, backend=Backend.MEMORY.value, + path=output_plan.path, + backend=Backend.MEMORY.value, ) destination = plan.output_dir / filename filemanager.save(payload, str(destination), Backend.MEMORY.value) filemanager.ensure_directory(plan.output_dir, Backend.DISK.value) filemanager.save(payload.data, str(destination), Backend.DISK.value) record_output_path( - context, plan, str(destination), image_metadata=payload.metadata, + context, + plan, + str(destination), + image_metadata=payload.metadata, output_context=AlignedImageSliceContext.main_flow( - output_key=role, artifact_kind=ImageArtifactType.value, + output_key=role, + artifact_kind=ImageArtifactType.value, ), identity=output_identity, ) saved = ArtifactMaterializationTargetPlan.materialize(context, plan) assert len(saved) == len(roles) - target = replace(PrimaryImageMetadataTarget.from_plan(plan), artifact_materializations=saved) + target = replace( + PrimaryImageMetadataTarget.from_plan(plan), artifact_materializations=saved + ) records = target.produced_records(context, plan) artifacts = target.runtime_artifact_projection_paths(context, plan) receipt = { "source_scope": plan.step_scope_id, - "produced": [{"path": str(record.path_under(plan.output_dir)), - "identity": asdict(record.producer_identity), - "address": record.filename_address.as_component_metadata()} for record in records], - "materialized": [{"path": output.path, - "role": item.materialization.output_plan.name, - "semantic_source": None if item.materialization.source_identity is None else { - "path": item.materialization.source_identity.path, - "component_metadata": dict(item.materialization.source_identity.component_metadata or {}), - }, - "producer_scope": item.materialization.output_plan.producer_step_scope_id, - "stored_key": repr(item.materialization.record.key)} - for item in saved for output in item.outputs_for_backend(Backend.DISK.value)], - "artifact_projections": [{"path": path, - "role": projection.projection_role.value, - "address": None if projection.address is None else projection.address.as_component_metadata(), - "alias": projection.source_alias, - "kind": projection.artifact_kind.value} - for projection, path in artifacts], + "produced": [ + { + "path": str(record.path_under(plan.output_dir)), + "identity": asdict(record.producer_identity), + "address": record.filename_address.as_component_metadata(), + } + for record in records + ], + "materialized": [ + { + "path": output.path, + "role": item.materialization.output_plan.name, + "semantic_source": ( + None + if item.materialization.source_identity is None + else { + "path": item.materialization.source_identity.path, + "component_metadata": dict( + item.materialization.source_identity.component_metadata + or {} + ), + } + ), + "producer_scope": item.materialization.output_plan.producer_step_scope_id, + "stored_key": repr(item.materialization.record.key), + } + for item in saved + for output in item.outputs_for_backend(Backend.DISK.value) + ], + "artifact_projections": [ + { + "path": path, + "role": projection.projection_role.value, + "address": ( + None + if projection.address is None + else projection.address.as_component_metadata() + ), + "alias": projection.source_alias, + "kind": projection.artifact_kind.value, + } + for projection, path in artifacts + ], "pre_join_main_flow": SourceProjectionMetadataSerializer.projection_records( - replace(target, artifact_materializations=()).produced_projection_entries(context, plan).projection_paths, + replace(target, artifact_materializations=()) + .produced_projection_entries(context, plan) + .projection_paths, + ), + "pre_join_artifacts": SourceProjectionMetadataSerializer.projection_records( + artifacts ), - "pre_join_artifacts": SourceProjectionMetadataSerializer.projection_records(artifacts), "produced_records": [to_jsonable(record) for record in records], } - (tmp_path / "persisted-occurrence-witness.json").write_text(json.dumps(receipt, indent=2)) + (tmp_path / "persisted-occurrence-witness.json").write_text( + json.dumps(receipt, indent=2) + ) produced_paths = {item["path"] for item in receipt["produced"]} materialized_paths = {item["path"] for item in receipt["materialized"]} if same_persisted_path: @@ -303,29 +445,44 @@ def test_saved_roles_publish_once_per_persisted_occurrence(tmp_path, scenario, a if mosaic: expected_sums = {780.0} elif aggregate: - expected_sums = {140.0} if scenario == "different_kind" else {float(pixels.sum())} + expected_sums = ( + {140.0} if scenario == "different_kind" else {float(pixels.sum())} + ) else: expected_sums = {20.0} if scenario == "different_kind" else {20.0, 10.0} - assert {float(tifffile.imread(item["path"]).sum()) for item in receipt["materialized"]} == expected_sums - if scenario not in ("same_occurrence", "mosaic_context") and not (aggregate and scenario == "different_path"): + assert { + float(tifffile.imread(item["path"]).sum()) for item in receipt["materialized"] + } == expected_sums + if scenario not in ("same_occurrence", "mosaic_context") and not ( + aggregate and scenario == "different_path" + ): message = ( "Conflicting metadata for persisted image" - if scenario in ("conflicting_metadata", "conflicting_scope", "mosaic_context") + if scenario + in ("conflicting_metadata", "conflicting_scope", "mosaic_context") else "Duplicate source projection address" ) with pytest.raises(ValueError, match=message) as rejection: target.produced_projection_entries(context, plan) receipt["rejection"] = str(rejection.value) - (tmp_path / "persisted-occurrence-witness.json").write_text(json.dumps(receipt, indent=2)) + (tmp_path / "persisted-occurrence-witness.json").write_text( + json.dumps(receipt, indent=2) + ) return if aggregate: (main_flow,) = receipt["pre_join_main_flow"] (artifact,) = receipt["pre_join_artifacts"] assert main_flow["address"] is artifact["address"] is None - assert main_flow["execution_scope"] == artifact["execution_scope"] == { - "axis_id": "A01", "component": "channel", "value": "1", - "fixed_component_values": [["site", "1"], ["timepoint", "1"]], - } + assert ( + main_flow["execution_scope"] + == artifact["execution_scope"] + == { + "axis_id": "A01", + "component": "channel", + "value": "1", + "fixed_component_values": [["site", "1"], ["timepoint", "1"]], + } + ) assert main_flow["image_metadata"] == artifact["image_metadata"] assert main_flow["source_metadata"] == artifact["source_metadata"] assert receipt["produced"][0]["address"]["z_index"] == "0" @@ -333,14 +490,20 @@ def test_saved_roles_publish_once_per_persisted_occurrence(tmp_path, scenario, a metadata = main_flow["image_metadata"] assert metadata["source_dtype"] == "uint16" assert metadata["source_voxel_spacing"] == { - "values_zyx": [2.0, 0.65, 0.65], "unit": "micrometers", + "values_zyx": [2.0, 0.65, 0.65], + "unit": "micrometers", } provenance = metadata["source_provenance"] assert "z_index" not in provenance["source_component_metadata"] - assert [plane["component_metadata"]["z_index"] for plane in - provenance["source_image_provenance_planes"]] == ["0", "1", "2", "3"] - assert [plane["path"] for plane in provenance["source_image_provenance_planes"]] == [ - str(source_dir / f"A01_s001_w1_z{z_index:03d}_t001.tif") for z_index in range(4) + assert [ + plane["component_metadata"]["z_index"] + for plane in provenance["source_image_provenance_planes"] + ] == ["0", "1", "2", "3"] + assert [ + plane["path"] for plane in provenance["source_image_provenance_planes"] + ] == [ + str(source_dir / f"A01_s001_w1_z{z_index:03d}_t001.tif") + for z_index in range(4) ] assert metadata["source_spatial_domain"]["spatial_dimensions"] == 2 for path in produced_paths | materialized_paths: @@ -356,20 +519,32 @@ def test_saved_roles_publish_once_per_persisted_occurrence(tmp_path, scenario, a assert main_flow["image_metadata"] == artifact["image_metadata"] assert main_flow["source_metadata"] == artifact["source_metadata"] assert receipt["produced"][0]["address"]["site"] == "1" - assert "site" not in receipt["materialized"][0]["semantic_source"]["component_metadata"] + assert ( + "site" + not in receipt["materialized"][0]["semantic_source"]["component_metadata"] + ) structured = SourceProjectionMetadataSerializer.projection_fields( structured.projection_paths ) occurrence_count = 2 if aggregate and scenario == "different_path" else 1 assert len(structured["source_projection"]) == len(roles) * occurrence_count - assert {item["source_alias"] for item in structured["source_projection"]} == {role for role, _ in roles} + assert {item["source_alias"] for item in structured["source_projection"]} == { + role for role, _ in roles + } target.write(context, produced_plan=plan) document = json.loads((tmp_path / "openhcs_metadata.json").read_text()) - reopened = VirtualWorkspaceSourceProjection.from_openhcs_metadata(tmp_path, document) + reopened = VirtualWorkspaceSourceProjection.from_openhcs_metadata( + tmp_path, document + ) projections = reopened.source_projections_by_virtual_path - relative_paths = {path.removeprefix(str(tmp_path) + "/") for path in produced_paths | materialized_paths} + relative_paths = { + path.removeprefix(str(tmp_path) + "/") + for path in produced_paths | materialized_paths + } assert set(projections) == relative_paths | produced_paths | materialized_paths - assert {projection.source_alias for projection in projections.values()} == {role for role, _ in roles} + assert {projection.source_alias for projection in projections.values()} == { + role for role, _ in roles + } for relative_path in relative_paths: projection = projections[relative_path] if mosaic: @@ -380,15 +555,26 @@ def test_saved_roles_publish_once_per_persisted_occurrence(tmp_path, scenario, a assert metadata.source_image_provenance_planes.contributor_count == 2 assert metadata.source_voxel_spacing.values_zyx == (0.5, 0.5) assert projection.execution_scope.fixed_component_values == ( - (Microscopy.Channel, "2"), (Microscopy.ZIndex, "1"), + (Microscopy.Channel, "2"), + (Microscopy.ZIndex, "1"), (Microscopy.Timepoint, "1"), ) - np.testing.assert_array_equal(tifffile.imread(tmp_path / relative_path), mosaic_pixels) + np.testing.assert_array_equal( + tifffile.imread(tmp_path / relative_path), mosaic_pixels + ) continue if aggregate: assert projection.address is None - np.testing.assert_array_equal(tifffile.imread(tmp_path / relative_path), pixels) + np.testing.assert_array_equal( + tifffile.imread(tmp_path / relative_path), pixels + ) continue - assert projection.address.as_component_metadata() == {key: components[key] for key in ("well", "site", "channel", "z_index", "timepoint")} + assert projection.address.as_component_metadata() == { + key: components[key] + for key in ("well", "site", "channel", "z_index", "timepoint") + } expected_scale = 1.0 if projection.source_alias == "RawRole" else 0.5 - np.testing.assert_array_equal(tifffile.imread(tmp_path / relative_path), np.ones((4, 5), dtype=np.float32) * expected_scale) + np.testing.assert_array_equal( + tifffile.imread(tmp_path / relative_path), + np.ones((4, 5), dtype=np.float32) * expected_scale, + ) diff --git a/tests/unit/test_gray_to_color_binding_axis_432.py b/tests/unit/test_gray_to_color_binding_axis_432.py index 7e940241bd..370136c258 100644 --- a/tests/unit/test_gray_to_color_binding_axis_432.py +++ b/tests/unit/test_gray_to_color_binding_axis_432.py @@ -6,7 +6,10 @@ import pytest from openhcs.core.component_group_scope import RuntimeExecutionAxisScope -from openhcs.core.aligned_image_payload import (AlignedImageStack, ImagePayloadBundleContext) +from openhcs.core.aligned_image_payload import ( + AlignedImageStack, + ImagePayloadBundleContext, +) from openhcs.core.callable_contract import CallableContract from openhcs.core.artifacts import ArtifactSpec, ImageArtifactType from openhcs.core.config import StepSourceBindingsConfig @@ -25,13 +28,22 @@ from openhcs.interop.cellprofiler.runtime.function_contract_execution import ( CellProfilerFunctionContractExecutor, ) -from openhcs.interop.cellprofiler.runtime.module_execution import CellProfilerModuleExecutor +from openhcs.interop.cellprofiler.runtime.module_execution import ( + CellProfilerModuleExecutor, +) from openhcs.interop.cellprofiler.runtime.output_recording import ( CellProfilerOutputRecorder, ) -from openhcs.processing.backends.cellprofiler.color import GrayToColorModule, gray_to_color -from tests.unit.test_cellprofiler_generic_special_input_binding import _compile_public_step -from tests.unit.cellprofiler_runtime_test_support import cellprofiler_runtime_adapter_for_test +from openhcs.processing.backends.cellprofiler.color import ( + GrayToColorModule, + gray_to_color, +) +from tests.unit.test_cellprofiler_generic_special_input_binding import ( + _compile_public_step, +) +from tests.unit.cellprofiler_runtime_test_support import ( + cellprofiler_runtime_adapter_for_test, +) from openhcs.domains.microscopy.axes import Microscopy from openhcs.core.axes import ColourAxis from openhcs.core.image_payload_execution_mode import ( @@ -41,7 +53,9 @@ @pytest.mark.parametrize("explicit_selector", [False, True]) -def test_declared_fitc_runtime_plane_retains_values_and_physical_identity(explicit_selector): +def test_declared_fitc_runtime_plane_retains_values_and_physical_identity( + explicit_selector, +): kwargs = { "color_scheme": GrayToColorModule.Scheme.STACK, "rescale_intensity": False, @@ -50,40 +64,64 @@ def test_declared_fitc_runtime_plane_retains_values_and_physical_identity(explic if explicit_selector: kwargs["image_name"] = ("FITC",) step = FunctionStep( - func=(gray_to_color, kwargs), name="RetainRawReference", + func=(gray_to_color, kwargs), + name="RetainRawReference", source_bindings=StepSourceBindingsConfig( - enabled=True, bindings=(NamedSourceBinding("FITC"),), + enabled=True, + bindings=(NamedSourceBinding("FITC"),), ), ) invocation = _compile_public_step(step) - contract = replace(invocation.contract, metadata=replace( - invocation.contract.metadata, - runtime_adapter=CellProfilerRuntimeAdapter.runtime_adapter_spec(), - )) + contract = replace( + invocation.contract, + metadata=replace( + invocation.contract.metadata, + runtime_adapter=CellProfilerRuntimeAdapter.runtime_adapter_spec(), + ), + ) pixels = np.arange(20, dtype=np.float32).reshape(4, 5) * 1000 - physical = {"well": "A01", "site": "1", "channel": "2", "z_index": "1", "timepoint": "1"} + physical = { + "well": "A01", + "site": "1", + "channel": "2", + "z_index": "1", + "timepoint": "1", + } metadata = ImagePayloadMetadata( plane_axis=RuntimePlaneAxis.RUNTIME_SLICE, source_image_provenance_planes=SourceImageProvenancePlanes.from_components( - paths=("/synthetic/A01_s1_w2_z1_t1.tif",), component_metadata=(physical,), + paths=("/synthetic/A01_s1_w2_z1_t1.tif",), + component_metadata=(physical,), ), source_image_names=("FITC",), source_voxel_spacing=SourceVoxelSpacing((1.0, 1.0)), - source_spatial_domain=SourceSpatialDomain(origin_yx=(0, 0), source_shape_yx=(4, 5)), + source_spatial_domain=SourceSpatialDomain( + origin_yx=(0, 0), source_shape_yx=(4, 5) + ), ) source = metadata.payload_with(pixels[None], None) runtime = cellprofiler_runtime_adapter_for_test( - runtime_value_store=RuntimeValueStore(), callable_contract=contract, + runtime_value_store=RuntimeValueStore(), + callable_contract=contract, variable_components=(Microscopy.Channel,), - axis_scope=RuntimeExecutionAxisScope.from_raw("synthetic", component=None, value=None), - artifact_inputs={edge.key: InvocationArtifactInputEdgePlan.from_source_declarations( - key=edge.key, spec=edge.spec, main_flow_artifacts=contract.artifact_inputs, - invocation_sources=contract.artifact_inputs, - ) for edge in invocation.artifact_input_edges}, + axis_scope=RuntimeExecutionAxisScope.from_raw( + "synthetic", component=None, value=None + ), + artifact_inputs={ + edge.key: InvocationArtifactInputEdgePlan.from_source_declarations( + key=edge.key, + spec=edge.spec, + main_flow_artifacts=contract.artifact_inputs, + invocation_sources=contract.artifact_inputs, + ) + for edge in invocation.artifact_input_edges + }, ) executor = CellProfilerModuleExecutor(gray_to_color, contract) request = executor._image_request( - source, runtime, module_type=GrayToColorModule, + source, + runtime, + module_type=GrayToColorModule, active_input_specs=contract.artifact_inputs.specs, ) assert contract.artifact_inputs.names() == ("FITC",) @@ -95,22 +133,38 @@ def test_declared_fitc_runtime_plane_retains_values_and_physical_identity(explic assert request.plane_projection.axis is RuntimePlaneAxis.RUNTIME_SLICE assert request.plane_projection.axis_size == 1 assert request.plane_projection.plane_index is None - assert request.payload.slices[0].metadata.plane_axis is RuntimePlaneAxis.SOURCE_BINDING + assert ( + request.payload.slices[0].metadata.plane_axis is RuntimePlaneAxis.SOURCE_BINDING + ) assert request.payload.slices[0].metadata.source_image_names == ("FITC",) execution = executor._invocation_request( - image_request=request, adapter=runtime, current_image=source, + image_request=request, + adapter=runtime, + current_image=source, module_type=GrayToColorModule, - kwargs={"color_scheme": GrayToColorModule.Scheme.STACK, "rescale_intensity": False}, + kwargs={ + "color_scheme": GrayToColorModule.Scheme.STACK, + "rescale_intensity": False, + }, ) result = CellProfilerFunctionContractExecutor().execute( - contract, contract.resolve_canonical_raw_callable(), execution.payload, execution.kwargs, - execution_mode=execution.execution_mode, plane_projection=execution.plane_projection, + contract, + contract.resolve_canonical_raw_callable(), + execution.payload, + execution.kwargs, + execution_mode=execution.execution_mode, + plane_projection=execution.plane_projection, ) # Float32 is the original Stack runner's output contract, not unit normalization. assert result.metadata.plane_axis is RuntimePlaneAxis.RUNTIME_SLICE - plane = RuntimeSliceProjection.value_for_slice(result, RuntimePlaneAxisValueProjection.from_selected_plane( - axis=RuntimePlaneAxis.RUNTIME_SLICE, plane_index=0, axis_size=1, - )) + plane = RuntimeSliceProjection.value_for_slice( + result, + RuntimePlaneAxisValueProjection.from_selected_plane( + axis=RuntimePlaneAxis.RUNTIME_SLICE, + plane_index=0, + axis_size=1, + ), + ) assert plane.data.shape == (4, 5, 1) np.testing.assert_array_equal(plane.data[..., 0], pixels) np.testing.assert_array_equal(source.data[0], pixels) @@ -124,17 +178,27 @@ def test_declared_fitc_runtime_plane_retains_values_and_physical_identity(explic def _source_plane(pixels, alias): return ImagePayloadMetadata( source_path="/synthetic/A01_s1_w2_z1_t1.tif", - source_component_metadata={"well": "A01", "site": "1", "channel": "2", "z_index": "1", "timepoint": "1"}, + source_component_metadata={ + "well": "A01", + "site": "1", + "channel": "2", + "z_index": "1", + "timepoint": "1", + }, source_image_names=(alias,), source_voxel_spacing=SourceVoxelSpacing((1.0, 1.0)), - source_spatial_domain=SourceSpatialDomain(origin_yx=(0, 0), source_shape_yx=pixels.shape[:2]), + source_spatial_domain=SourceSpatialDomain( + origin_yx=(0, 0), source_shape_yx=pixels.shape[:2] + ), ).payload_with(pixels, None) def _execute_bound_stack(bundle): contract = CallableContract.from_callable(gray_to_color) return CellProfilerFunctionContractExecutor().execute( - contract, contract.resolve_canonical_raw_callable(), bundle, + contract, + contract.resolve_canonical_raw_callable(), + bundle, {"color_scheme": GrayToColorModule.Scheme.STACK, "rescale_intensity": False}, execution_mode=FullStackExecution, plane_projection=RuntimePlaneAxisValueProjection.preserve( @@ -145,11 +209,17 @@ def _execute_bound_stack(bundle): ) -@pytest.mark.parametrize("aliases", [("RawBody", "CappedOutgrowth"), ("IndependentRoleA", "IndependentRoleB")]) +@pytest.mark.parametrize( + "aliases", + [("RawBody", "CappedOutgrowth"), ("IndependentRoleA", "IndependentRoleB")], +) def test_two_processing_roles_share_physical_channel_without_fabrication(aliases): raw = np.arange(20, dtype=np.float32).reshape(4, 5) * 1000 capped = np.clip(raw / 6000, 0, 1) - inputs = tuple(_source_plane(pixels, alias) for pixels, alias in zip((raw, capped), aliases, strict=True)) + inputs = tuple( + _source_plane(pixels, alias) + for pixels, alias in zip((raw, capped), aliases, strict=True) + ) bundle = ImagePayloadBundleContext.from_payloads(inputs).compose() assert bundle.metadata.plane_axis is RuntimePlaneAxis.SOURCE_BINDING result = _execute_bound_stack(bundle) @@ -169,10 +239,17 @@ def test_projected_runtime_plane_needs_a_separate_named_binding_axis(): pixels = np.arange(20, dtype=np.float32).reshape(4, 5) * 1000 scalar = _source_plane(pixels, "FITC") metadata = scalar.metadata - retained = replace(metadata, plane_axis=RuntimePlaneAxis.RUNTIME_SLICE).payload_with(pixels[None], None) - projected = RuntimeSliceProjection.value_for_slice(retained, RuntimePlaneAxisValueProjection.from_selected_plane( - axis=RuntimePlaneAxis.RUNTIME_SLICE, plane_index=0, axis_size=1, - )) + retained = replace( + metadata, plane_axis=RuntimePlaneAxis.RUNTIME_SLICE + ).payload_with(pixels[None], None) + projected = RuntimeSliceProjection.value_for_slice( + retained, + RuntimePlaneAxisValueProjection.from_selected_plane( + axis=RuntimePlaneAxis.RUNTIME_SLICE, + plane_index=0, + axis_size=1, + ), + ) assert projected.metadata.plane_axis is None bundle = ImagePayloadBundleContext.from_payloads((projected,)).compose() assert bundle.metadata.plane_axis is RuntimePlaneAxis.SOURCE_BINDING @@ -184,7 +261,9 @@ def test_projected_runtime_plane_needs_a_separate_named_binding_axis(): def test_stack_color_output_is_not_a_second_scalar_gray_binding(): pixels = np.arange(20, dtype=np.float32).reshape(4, 5) - named = ImagePayloadBundleContext.from_payloads((_source_plane(pixels, "FITC"),)).compose() + named = ImagePayloadBundleContext.from_payloads( + (_source_plane(pixels, "FITC"),) + ).compose() color = _execute_bound_stack(named) # The existing STACK contract emits YXC even for one role. A second grayscale # binding introduces C,Y,X,source-color rather than the required C,Y,X. diff --git a/tests/unit/test_gray_to_color_payload_axis_compile_432.py b/tests/unit/test_gray_to_color_payload_axis_compile_432.py index c4d80351e3..219cb58b00 100644 --- a/tests/unit/test_gray_to_color_payload_axis_compile_432.py +++ b/tests/unit/test_gray_to_color_payload_axis_compile_432.py @@ -7,25 +7,40 @@ import tifffile from openhcs.core.callable_contract import ( - CallableContract, CreatedPayloadAxis, PayloadAxisRequirement, PreservedPayloadAxes, - creates_payload_axis, preserves_payload_axes, + CallableContract, + CreatedPayloadAxis, + PayloadAxisRequirement, + PreservedPayloadAxes, + creates_payload_axis, + preserves_payload_axes, ) from openhcs.core.compiled_step_plan import CompiledStepPlan from openhcs.core.function_patterns import ( - CompiledFunctionGroup, CompiledFunctionPattern, PayloadAxisProof, - InheritedPayloadAxisProof, CreatedPayloadAxisProof, + CompiledFunctionGroup, + CompiledFunctionPattern, + PayloadAxisProof, + InheritedPayloadAxisProof, + CreatedPayloadAxisProof, UnprovedPayloadAxisProof, ) from openhcs.core.pipeline.compiler import PipelineCompiler from openhcs.core.aligned_image_payload import ImagePayloadBundleContext from openhcs.core.image_payload_execution_mode import NaturalExecution from openhcs.core.step_dependencies import StepInputDependency -from openhcs.interop.cellprofiler.runtime.function_contract_execution import CellProfilerFunctionContractExecutor +from openhcs.interop.cellprofiler.runtime.function_contract_execution import ( + CellProfilerFunctionContractExecutor, +) from openhcs.processing.backends.cellprofiler.color import ( - ColorToGrayMode, ImageChannelType, color_to_gray, gray_to_color, + ColorToGrayMode, + ImageChannelType, + color_to_gray, + gray_to_color, split_color_to_gray, ) -from tests.unit.test_gray_to_color_binding_axis_432 import _execute_bound_stack, _source_plane +from tests.unit.test_gray_to_color_binding_axis_432 import ( + _execute_bound_stack, + _source_plane, +) from tests.unit.test_payload_axis_compile_gate import _compiled_pattern, _session from openhcs.core.payload_axes import ColourSampleAxisSpec from openhcs.core.axes import ColourAxis @@ -36,13 +51,18 @@ def _gray_creator_session(tmp_path): tifffile.imwrite(path, np.zeros((4, 5), dtype=np.uint16)) session = _session(tmp_path, (path,)) producer = replace( - session.plans[0], step_name="ComposeScalarRoles", step_scope_id="step-0", + session.plans[0], + step_name="ComposeScalarRoles", + step_scope_id="step-0", compiled_function_pattern=_compiled_pattern(gray_to_color), ) consumer = CompiledStepPlan( - step_index=1, step_name="SplitComposedRoles", axis_id="A01", + step_index=1, + step_name="SplitComposedRoles", + axis_id="A01", main_input_dependency=StepInputDependency.step_output( - source_step_index=0, source_step_scope_id="step-0", + source_step_index=0, + source_step_scope_id="step-0", ), compiled_function_pattern=_compiled_pattern(color_to_gray), ) @@ -59,14 +79,23 @@ def test_declared_gray_creator_proves_color_consumer_before_execution(tmp_path): def test_declared_gray_creator_proves_same_group_color_consumer(tmp_path): session = _gray_creator_session(tmp_path) - group = CompiledFunctionGroup("default", tuple( - invocation - for plan in session.plans.values() - for invocation in plan.compiled_function_pattern.iter_invocations() - )) - session.plans = {0: replace(session.plans[0], compiled_function_pattern=CompiledFunctionPattern( - groups=(group,), is_grouped=False, - ))} + group = CompiledFunctionGroup( + "default", + tuple( + invocation + for plan in session.plans.values() + for invocation in plan.compiled_function_pattern.iter_invocations() + ), + ) + session.plans = { + 0: replace( + session.plans[0], + compiled_function_pattern=CompiledFunctionPattern( + groups=(group,), + is_grouped=False, + ), + ) + } session.context.step_plans = session.plans PipelineCompiler.validate_payload_axis_requirements(session) @@ -76,12 +105,23 @@ def test_preservation_annotation_cannot_create_missing_source_payload_axis(tmp_p producer = session.plans[0] (invocation,) = tuple(producer.compiled_function_pattern.iter_invocations()) # A local counterexample only: do not mutate production callable/registry. - falsely_preserving = replace(invocation, contract=replace(invocation.contract, metadata=replace( - invocation.contract.metadata, payload_axis_transition=PreservedPayloadAxes(), - ))) - session.plans[0] = replace(producer, compiled_function_pattern=CompiledFunctionPattern( - groups=(CompiledFunctionGroup("default", (falsely_preserving,)),), is_grouped=False, - )) + falsely_preserving = replace( + invocation, + contract=replace( + invocation.contract, + metadata=replace( + invocation.contract.metadata, + payload_axis_transition=PreservedPayloadAxes(), + ), + ), + ) + session.plans[0] = replace( + producer, + compiled_function_pattern=CompiledFunctionPattern( + groups=(CompiledFunctionGroup("default", (falsely_preserving,)),), + is_grouped=False, + ), + ) session.context.step_plans = session.plans with pytest.raises(ValueError, match="requires a declared ColourAxis payload axis"): PipelineCompiler.validate_payload_axis_requirements(session) @@ -90,18 +130,26 @@ def test_preservation_annotation_cannot_create_missing_source_payload_axis(tmp_p def _runtime_scalar_role_split(): raw = np.arange(20, dtype=np.float32).reshape(4, 5) * 1000 / 65535 capped = np.clip((raw - 39 / 65535) / (5961 / 65535), 0, 1) - bundle = ImagePayloadBundleContext.from_payloads(tuple( - _source_plane(pixels, name) - for pixels, name in ((raw, "RawBody"), (capped, "CappedOutgrowth")) - )).compose() + bundle = ImagePayloadBundleContext.from_payloads( + tuple( + _source_plane(pixels, name) + for pixels, name in ((raw, "RawBody"), (capped, "CappedOutgrowth")) + ) + ).compose() composite = _execute_bound_stack(bundle) assert composite.data.shape == (4, 5, 2) assert composite.metadata.axis_position(ColourAxis) == -1 contract = CallableContract.from_callable(color_to_gray) result = CellProfilerFunctionContractExecutor().execute( - contract, contract.resolve_canonical_raw_callable(), composite, - {"mode": ColorToGrayMode.SPLIT, "image_type": ImageChannelType.CHANNELS, - "channel_indices": (0, 1), "contributions": (1.0, 1.0)}, + contract, + contract.resolve_canonical_raw_callable(), + composite, + { + "mode": ColorToGrayMode.SPLIT, + "image_type": ImageChannelType.CHANNELS, + "channel_indices": (0, 1), + "contributions": (1.0, 1.0), + }, execution_mode=NaturalExecution, ) return result, (raw, capped) @@ -114,7 +162,9 @@ def test_original_runtime_split_preserves_both_role_values_and_physical_channel( assert output.data.shape == (4, 5) np.testing.assert_array_equal(output.data, expected) assert output.metadata.source_component_metadata["channel"] == "2" - assert set(output.metadata.source_image_paths) == {"/synthetic/A01_s1_w2_z1_t1.tif"} + assert set(output.metadata.source_image_paths) == { + "/synthetic/A01_s1_w2_z1_t1.tif" + } def test_scalar_role_outputs_consume_the_color_payload_axis(): @@ -125,7 +175,9 @@ def test_scalar_role_outputs_consume_the_color_payload_axis(): @pytest.mark.parametrize("same_group", [False, True]) -def test_independent_creator_declaration_needs_no_consumer_edits(tmp_path, monkeypatch, same_group): +def test_independent_creator_declaration_needs_no_consumer_edits( + tmp_path, monkeypatch, same_group +): @creates_payload_axis(ColourAxis) def independent_lane_creator(image): return np.stack((image, image * 2), axis=-1) @@ -146,15 +198,30 @@ def independent_lane_preserver(image): for invocation in _compiled_pattern(callable_).iter_invocations() ) if same_group: - consumer_invocations = tuple(session.plans[1].compiled_function_pattern.iter_invocations()) - session.plans = {0: replace(session.plans[0], compiled_function_pattern=CompiledFunctionPattern( - groups=(CompiledFunctionGroup("default", producer_invocations + consumer_invocations),), - is_grouped=False, - ))} + consumer_invocations = tuple( + session.plans[1].compiled_function_pattern.iter_invocations() + ) + session.plans = { + 0: replace( + session.plans[0], + compiled_function_pattern=CompiledFunctionPattern( + groups=( + CompiledFunctionGroup( + "default", producer_invocations + consumer_invocations + ), + ), + is_grouped=False, + ), + ) + } else: - session.plans[0] = replace(session.plans[0], compiled_function_pattern=CompiledFunctionPattern( - groups=(CompiledFunctionGroup("default", producer_invocations),), is_grouped=False, - )) + session.plans[0] = replace( + session.plans[0], + compiled_function_pattern=CompiledFunctionPattern( + groups=(CompiledFunctionGroup("default", producer_invocations),), + is_grouped=False, + ), + ) session.context.step_plans = session.plans def reject_source_backtracking(*args, **kwargs): @@ -172,9 +239,12 @@ def test_group_proof_stops_at_creator_but_rejects_later_unknown(unknown_after_cr def unknown(image): return image - functions = (gray_to_color, unknown) if unknown_after_creator else (unknown, gray_to_color) + functions = ( + (gray_to_color, unknown) if unknown_after_creator else (unknown, gray_to_color) + ) invocations = tuple( - invocation for callable_ in functions + invocation + for callable_ in functions for invocation in _compiled_pattern(callable_).iter_invocations() ) proof = CompiledFunctionGroup("default", invocations).payload_axis_proof( @@ -183,7 +253,9 @@ def unknown(image): failures = [] assert proof.validate_obligation( failure_message=lambda invocation: invocation.contract.function_name, - source_validation=lambda: pytest.fail("Creation/rejection must stop source validation"), + source_validation=lambda: pytest.fail( + "Creation/rejection must stop source validation" + ), failures=failures, ) assert failures == (["unknown"] if unknown_after_creator else []) @@ -199,7 +271,9 @@ def validate_source(): complete = InheritedPayloadAxisProof().validate_obligation( source_validation=validate_source, - failure_message=lambda invocation: pytest.fail("Inherited proof has no failed invocation"), + failure_message=lambda invocation: pytest.fail( + "Inherited proof has no failed invocation" + ), failures=failures, ) assert complete is source_complete @@ -208,14 +282,25 @@ def validate_source(): @pytest.mark.parametrize("capability_first", [False, True]) -@pytest.mark.parametrize("proof_type,args,expected_visits,expected_failures", [ - (InheritedPayloadAxisProof, (), ["before", "source", "after"], []), - (CreatedPayloadAxisProof, (), ["before", "after"], []), - (UnprovedPayloadAxisProof, - tuple(_compiled_pattern(color_to_gray).iter_invocations()), ["before", "after"], ["rejected color_to_gray"]), -]) +@pytest.mark.parametrize( + "proof_type,args,expected_visits,expected_failures", + [ + (InheritedPayloadAxisProof, (), ["before", "source", "after"], []), + (CreatedPayloadAxisProof, (), ["before", "after"], []), + ( + UnprovedPayloadAxisProof, + tuple(_compiled_pattern(color_to_gray).iter_invocations()), + ["before", "after"], + ["rejected color_to_gray"], + ), + ], +) def test_independent_same_node_validation_capability_composes_both_mro_orders( - capability_first, proof_type, args, expected_visits, expected_failures, + capability_first, + proof_type, + args, + expected_visits, + expected_failures, ): visits, failures = [], [] @@ -228,7 +313,11 @@ def validate_obligation(self, **kwargs): # Same nominal owner in both bases puts the independent capability before # the shared operation in C3, regardless of its position beside the leaf. - bases = (ValidationVisitCapability, proof_type) if capability_first else (proof_type, ValidationVisitCapability) + bases = ( + (ValidationVisitCapability, proof_type) + if capability_first + else (proof_type, ValidationVisitCapability) + ) declared_proof = type("IndependentlyObservedProof", bases, {})(*args) def validate_source(): @@ -237,7 +326,8 @@ def validate_source(): assert declared_proof.validate_obligation( source_validation=validate_source, - failure_message=lambda invocation: "rejected " + invocation.contract.function_name, + failure_message=lambda invocation: "rejected " + + invocation.contract.function_name, failures=failures, ) assert visits == expected_visits @@ -258,18 +348,25 @@ def reject_source(): assert failures == ["exact source missing its declared payload axis"] -def test_color_to_gray_does_not_preserve_consumed_payload_axis_for_later_consumer(tmp_path): +def test_color_to_gray_does_not_preserve_consumed_payload_axis_for_later_consumer( + tmp_path, +): session = _gray_creator_session(tmp_path) consumer = session.plans[1] session.plans[2] = replace( - consumer, step_index=2, step_name="RejectSecondScalarSplit", + consumer, + step_index=2, + step_name="RejectSecondScalarSplit", main_input_dependency=StepInputDependency.step_output( - source_step_index=1, source_step_scope_id="step-1", + source_step_index=1, + source_step_scope_id="step-1", ), ) session.plans[1] = replace(consumer, step_scope_id="step-1") session.context.step_plans = session.plans - with pytest.raises(ValueError, match="not preserved by producer step 1 callable 'color_to_gray'"): + with pytest.raises( + ValueError, match="not preserved by producer step 1 callable 'color_to_gray'" + ): PipelineCompiler.validate_payload_axis_requirements(session) @@ -283,9 +380,15 @@ def test_scalar_projection_modes_keep_pixels_mask_and_original_source(image_type expected = split_color_to_gray(source, image_type, (0, 1)) contract = CallableContract.from_callable(color_to_gray) result = CellProfilerFunctionContractExecutor().execute( - contract, contract.resolve_canonical_raw_callable(), source, - {"mode": ColorToGrayMode.SPLIT, "image_type": image_type, - "channel_indices": (0, 1), "contributions": (1.0, 1.0)}, + contract, + contract.resolve_canonical_raw_callable(), + source, + { + "mode": ColorToGrayMode.SPLIT, + "image_type": image_type, + "channel_indices": (0, 1), + "contributions": (1.0, 1.0), + }, execution_mode=NaturalExecution, ) for output, expected_pixels in zip(result.slices, expected, strict=True): @@ -294,7 +397,10 @@ def test_scalar_projection_modes_keep_pixels_mask_and_original_source(image_type output_metadata = output.metadata assert output_metadata.axis_position(ColourAxis) is None assert output_metadata.source_path == metadata.source_path - assert output_metadata.source_component_metadata == metadata.source_component_metadata + assert ( + output_metadata.source_component_metadata + == metadata.source_component_metadata + ) assert output_metadata.source_image_names == metadata.source_image_names assert output_metadata.source_voxel_spacing == metadata.source_voxel_spacing assert output_metadata.source_spatial_domain == metadata.source_spatial_domain diff --git a/tests/unit/test_library_registry_discovery.py b/tests/unit/test_library_registry_discovery.py index a0f3403be1..73fc7c4082 100644 --- a/tests/unit/test_library_registry_discovery.py +++ b/tests/unit/test_library_registry_discovery.py @@ -109,7 +109,11 @@ def test_native_catalog_excludes_modules_whose_distribution_markers_exclude_here @pytest.mark.parametrize( "platform_system, platform_machine, excluded", - [("Darwin", "x86_64", True), ("Darwin", "arm64", False), ("Linux", "x86_64", False)], + [ + ("Darwin", "x86_64", True), + ("Darwin", "arm64", False), + ("Linux", "x86_64", False), + ], ) def test_basicpy_catalog_module_follows_the_package_basicpy_marker( monkeypatch, platform_system, platform_machine, excluded diff --git a/tests/unit/test_named_checkpoint_publication.py b/tests/unit/test_named_checkpoint_publication.py index 5561e66be1..fe720be850 100644 --- a/tests/unit/test_named_checkpoint_publication.py +++ b/tests/unit/test_named_checkpoint_publication.py @@ -13,18 +13,26 @@ from tests.unit.test_artifact_publication_journey import _plate, progress_events from openhcs.core.artifacts import ( - QaCheckpoint, ImageArtifactType, MainFlowPlaneProjectionOutputSpec, + QaCheckpoint, + ImageArtifactType, + MainFlowPlaneProjectionOutputSpec, MainFlowStackOutputSpec, ) from openhcs.core.config import ( - LazyPathPlanningConfig, LazyProcessingConfig, LazyStepMaterializationConfig, + LazyPathPlanningConfig, + LazyProcessingConfig, + LazyStepMaterializationConfig, PipelineConfig, ) from openhcs.core.memory import numpy as numpy_decorator from openhcs.core.orchestrator.orchestrator import PipelineOrchestrator -from openhcs.core.orchestrator.execution_result import RuntimeContextObservation, RuntimeExecutionObservation +from openhcs.core.orchestrator.execution_result import ( + RuntimeContextObservation, + RuntimeExecutionObservation, +) from openhcs.core.pipeline.function_contracts import ( - artifact_outputs, required_axis_roles, + artifact_outputs, + required_axis_roles, ) from openhcs.core.projected_image_output import SourceProjectedImageOutput from openhcs.core.runtime_image_values import ImagePayloadMetadata @@ -38,13 +46,14 @@ Pure3DContract, ) from openhcs.processing.materialization import ( - ImageFileOptions, MaterializationSpec, MaterializedFilenameIdentity, + ImageFileOptions, + MaterializationSpec, + MaterializedFilenameIdentity, ) from openhcs.core.axes import TileAxis from openhcs.domains.microscopy.axes import Microscopy - @dataclass(frozen=True) class SyntheticAggregateField(SourceProjectedImageOutput): data: np.ndarray @@ -55,15 +64,22 @@ def with_data(self, data): def resolve_source_context(self, source, projection): assert projection is not None and projection.plane_index is None assert projection.axis_size == 24 - return source.metadata.collapse_leading_plane_axis().payload_with(self.data, None) + return source.metadata.collapse_leading_plane_axis().payload_with( + self.data, None + ) def _field(name): return MainFlowPlaneProjectionOutputSpec.output( - name, ImageArtifactType, sidecar_role=QaCheckpoint, - materialization=MaterializationSpec(ImageFileOptions( - filename_suffix=".tif", filename_identity=MaterializedFilenameIdentity.ARTIFACT_NAME, - )), + name, + ImageArtifactType, + sidecar_role=QaCheckpoint, + materialization=MaterializationSpec( + ImageFileOptions( + filename_suffix=".tif", + filename_identity=MaterializedFilenameIdentity.ARTIFACT_NAME, + ) + ), ) @@ -71,7 +87,8 @@ def _field(name): @required_axis_roles(TileAxis) @artifact_outputs( MainFlowStackOutputSpec.output("corrected", ImageArtifactType), - _field("flat"), _field("dark"), + _field("flat"), + _field("dark"), ) def synthetic_three_outputs(image): assert image.shape == (24, 8, 8) @@ -83,25 +100,35 @@ def synthetic_three_outputs(image): @pytest.mark.parametrize("main_filter", (0, 1)) -def test_named_and_ordinary_checkpoint_inventory_owns_every_address(tmp_path, main_filter): +def test_named_and_ordinary_checkpoint_inventory_owns_every_address( + tmp_path, main_filter +): plate = _plate(tmp_path / "source", site_count=24) config = PipelineConfig( - path_planning_config=LazyPathPlanningConfig(output_dir_suffix="_out", partition_filter=main_filter), - num_workers=1, use_threading=True, + path_planning_config=LazyPathPlanningConfig( + output_dir_suffix="_out", partition_filter=main_filter + ), + num_workers=1, + use_threading=True, ) orchestrator = PipelineOrchestrator(plate, pipeline_config=config).initialize() step = FunctionStep( func=synthetic_three_outputs, processing_config=LazyProcessingConfig( - group_by=Microscopy.Channel, variable_components=[Microscopy.Site], + group_by=Microscopy.Channel, + variable_components=[Microscopy.Site], ), step_materialization_config=LazyStepMaterializationConfig(enabled=True), ) bundle = orchestrator.compile_pipelines([step]) - observations = RuntimeExecutionObservation(contexts=tuple( - RuntimeContextObservation(context_key=key, records=(), outputs=step.process(context, 0)) - for key, context in bundle.runtime_contexts.items() - )) + observations = RuntimeExecutionObservation( + contexts=tuple( + RuntimeContextObservation( + context_key=key, records=(), outputs=step.process(context, 0) + ) + for key, context in bundle.runtime_contexts.items() + ) + ) OpenHCSMetadataTarget.finalize_completed_dataset( bundle.runtime_contexts, runtime_observations=(observations,) ) @@ -110,10 +137,14 @@ def test_named_and_ordinary_checkpoint_inventory_owns_every_address(tmp_path, ma reopened = VirtualWorkspaceSourceProjection.from_openhcs_metadata(root, metadata) for target, count in (("checkpoints", 48), ("checkpoints_results", 2)): entry = metadata["subdirectories"][target] - inventory = {str(path.relative_to(root)) for path in (root / target).glob("*.tif")} + inventory = { + str(path.relative_to(root)) for path in (root / target).glob("*.tif") + } assert len(inventory) == count assert set(entry["image_files"]) == inventory - assert {item["virtual_path"] for item in entry["source_projection"]} == inventory + assert { + item["virtual_path"] for item in entry["source_projection"] + } == inventory for item in entry["source_projection"]: retained = ImagePayloadMetadata.from_mapping(item["image_metadata"]) assert retained.source_voxel_spacing == SourceVoxelSpacing((0.65, 0.65)) @@ -123,13 +154,23 @@ def test_named_and_ordinary_checkpoint_inventory_owns_every_address(tmp_path, ma projection for item in checkpoints if isinstance( - projection := reopened.source_projections_by_virtual_path[item["virtual_path"]], + projection := reopened.source_projections_by_virtual_path[ + item["virtual_path"] + ], SourceArtifactProjection, ) ] assert len(ordinary) == len(checkpoints) - len(ordinary) == 24 assert {projection.address for projection in ordinary} == { - OpenHCSPlaneAddress(((Microscopy.Well, "A01"), (Microscopy.Site, site), (Microscopy.Channel, 1), (Microscopy.ZIndex, 1), (Microscopy.Timepoint, 1))) + OpenHCSPlaneAddress( + ( + (Microscopy.Well, "A01"), + (Microscopy.Site, site), + (Microscopy.Channel, 1), + (Microscopy.ZIndex, 1), + (Microscopy.Timepoint, 1), + ) + ) for site in range(1, 25) } for item in checkpoints: @@ -151,9 +192,19 @@ def test_named_and_ordinary_checkpoint_inventory_owns_every_address(tmp_path, ma assert all_inventory.issubset(reopened.source_projections_by_virtual_path) if main_filter: assert len(metadata["subdirectories"]["images"]["image_files"]) == 24 - assert PipelineOrchestrator(root).initialize().microscope_handler.get_pixel_size(root) == 0.65 + assert ( + PipelineOrchestrator(root) + .initialize() + .microscope_handler.get_pixel_size(root) + == 0.65 + ) # Keep fail-closed reconciliation: an actually unowned image is still an # error. This is not a missing-address permissive fallback. - tifffile.imwrite(root / "checkpoints" / "unaddressed.tif", np.zeros((8, 8), dtype=np.uint16)) - with pytest.raises(MetadataWriteError, match="Saved images lack typed produced addresses.*unaddressed"): + tifffile.imwrite( + root / "checkpoints" / "unaddressed.tif", np.zeros((8, 8), dtype=np.uint16) + ) + with pytest.raises( + MetadataWriteError, + match="Saved images lack typed produced addresses.*unaddressed", + ): OpenHCSMetadataTarget.finalize_completed_dataset(bundle.runtime_contexts) diff --git a/tests/unit/test_openhcs_adapter.py b/tests/unit/test_openhcs_adapter.py index c2f94a805c..56aabeab8d 100644 --- a/tests/unit/test_openhcs_adapter.py +++ b/tests/unit/test_openhcs_adapter.py @@ -389,13 +389,15 @@ def poll_status(self, execution_id): == FakeZMQExecutionClient.submitted[1].pipeline_code ) assert all( - submission.global_config_code == ConfigDocumentAuthority.render( + submission.global_config_code + == ConfigDocumentAuthority.render( global_config, expected_config_type=GlobalPipelineConfig ) for submission in FakeZMQExecutionClient.submitted ) assert [ - submission.compile_control.compile_only for submission in FakeZMQExecutionClient.submitted + submission.compile_control.compile_only + for submission in FakeZMQExecutionClient.submitted ] == [ True, False, diff --git a/tests/unit/test_rescale_stream_plane_declaration.py b/tests/unit/test_rescale_stream_plane_declaration.py index 9fc43bdbb5..7b3a727640 100644 --- a/tests/unit/test_rescale_stream_plane_declaration.py +++ b/tests/unit/test_rescale_stream_plane_declaration.py @@ -13,8 +13,13 @@ from openhcs.core.runtime_stores import RuntimeValueStore from openhcs.core.component_group_scope import RuntimeExecutionAxisScope from openhcs.core.function_patterns import InvocationArtifactInputEdgePlan -from openhcs.core.runtime_object_label_building import SourceImageObjectLabelBuildRequest -from openhcs.core.runtime_plane_projection import RuntimePlaneAxis, RuntimePlaneAxisValueProjection +from openhcs.core.runtime_object_label_building import ( + SourceImageObjectLabelBuildRequest, +) +from openhcs.core.runtime_plane_projection import ( + RuntimePlaneAxis, + RuntimePlaneAxisValueProjection, +) from openhcs.core.runtime_object_label_domains import ObjectLabelDomainScope from openhcs.core.source_image_provenance import SourceImageProvenancePlanes from openhcs.core.config import StepSourceBindingsConfig @@ -23,44 +28,69 @@ from openhcs.core.source_metadata import SourceVoxelSpacing, SourceVoxelSpacingUnit from openhcs.core.steps.function_outputs import StreamOutputBatch from openhcs.core.steps.function_step import FunctionStep -from openhcs.core.steps.stream_component_semantics import StreamImagePayloadMetadataProjector +from openhcs.core.steps.stream_component_semantics import ( + StreamImagePayloadMetadataProjector, +) from openhcs.processing.backends.cellprofiler.intensity import ( - AutomaticHigh, AutomaticLow, RescaleIntensityModule, RescaleMethod, rescale_intensity, + AutomaticHigh, + AutomaticLow, + RescaleIntensityModule, + RescaleMethod, + rescale_intensity, ) from openhcs.processing.backends.cellprofiler.shape import measure_object_size_shape from openhcs.interop.cellprofiler.runtime.adapter import CellProfilerRuntimeAdapter -from openhcs.interop.cellprofiler.runtime.module_execution import CellProfilerModuleExecutor -from openhcs.interop.cellprofiler.runtime.function_contract_execution import CellProfilerFunctionContractExecutor +from openhcs.interop.cellprofiler.runtime.module_execution import ( + CellProfilerModuleExecutor, +) +from openhcs.interop.cellprofiler.runtime.function_contract_execution import ( + CellProfilerFunctionContractExecutor, +) from openhcs.runtime.napari_streaming_handlers import ( - NapariAggregateAxisBindingBuilder, NapariStreamLayerAddress, NapariStreamLayerItem, + NapariAggregateAxisBindingBuilder, + NapariStreamLayerAddress, + NapariStreamLayerItem, ) from openhcs.runtime.viewer_component_system import ( - ViewerComponentAxisSemanticsFactory, ViewerComponentValueDomainPayload, + ViewerComponentAxisSemanticsFactory, + ViewerComponentValueDomainPayload, ) # Reuse the original provider-free compiler and runtime fixtures, not a second ABI. -from tests.unit.test_cellprofiler_generic_special_input_binding import _compile_public_step +from tests.unit.test_cellprofiler_generic_special_input_binding import ( + _compile_public_step, +) from tests.unit.cellprofiler_runtime_test_support import ( cellprofiler_runtime_adapter_for_test, ) from tests.unit.viewer_axes_fixture import STREAM_AXES -@pytest.mark.parametrize("method", [RescaleMethod.DIVIDE_BY_VALUE, RescaleMethod.MANUAL_INPUT_RANGE]) +@pytest.mark.parametrize( + "method", [RescaleMethod.DIVIDE_BY_VALUE, RescaleMethod.MANUAL_INPUT_RANGE] +) @pytest.mark.parametrize("matching", [None, "Bright", "Other"]) -def test_public_rescale_preserves_one_source_plane_through_stream_binding(method, matching): +def test_public_rescale_preserves_one_source_plane_through_stream_binding( + method, matching +): kwargs = { - "select_the_input_image": "Bright", "name_the_output_image": "Detection", - "rescale_method": method, "automatic_low": AutomaticLow.CUSTOM, - "automatic_high": AutomaticHigh.CUSTOM, "source_low": 0.0, - "source_high": 20.0, "divisor_value": 2.0, + "select_the_input_image": "Bright", + "name_the_output_image": "Detection", + "rescale_method": method, + "automatic_low": AutomaticLow.CUSTOM, + "automatic_high": AutomaticHigh.CUSTOM, + "source_low": 0.0, + "source_high": 20.0, + "divisor_value": 2.0, } if matching is not None: kwargs["select_image_to_match_in_maximum_intensity"] = matching step = FunctionStep( - func=(rescale_intensity, kwargs), name="SyntheticRescale", + func=(rescale_intensity, kwargs), + name="SyntheticRescale", source_bindings=StepSourceBindingsConfig( - enabled=True, bindings=(NamedSourceBinding("Bright"),), + enabled=True, + bindings=(NamedSourceBinding("Bright"),), ), ) invocation = _compile_public_step(step) @@ -74,18 +104,32 @@ def test_public_rescale_preserves_one_source_plane_through_stream_binding(method pixels = np.arange(20, dtype=np.float32).reshape(4, 5) metadata = ImagePayloadMetadata( source_path="/synthetic/A01_s1_w1_z1_t1.tif", - source_component_metadata={"well": "A01", "site": "1", "channel": "1", "z_index": "1", "timepoint": "1"}, + source_component_metadata={ + "well": "A01", + "site": "1", + "channel": "1", + "z_index": "1", + "timepoint": "1", + }, source_image_names=("Bright",), - source_voxel_spacing=SourceVoxelSpacing((1.0, 1.0, 1.0), SourceVoxelSpacingUnit.RELATIVE), - source_spatial_domain=SourceSpatialDomain(origin_yx=(0, 0), source_shape_yx=(4, 5)), + source_voxel_spacing=SourceVoxelSpacing( + (1.0, 1.0, 1.0), SourceVoxelSpacingUnit.RELATIVE + ), + source_spatial_domain=SourceSpatialDomain( + origin_yx=(0, 0), source_shape_yx=(4, 5) + ), ) source = metadata.payload_with(pixels, None) runtime = cellprofiler_runtime_adapter_for_test( - runtime_value_store=RuntimeValueStore(), callable_contract=contract, - axis_scope=RuntimeExecutionAxisScope.from_raw("synthetic", component=None, value=None), + runtime_value_store=RuntimeValueStore(), + callable_contract=contract, + axis_scope=RuntimeExecutionAxisScope.from_raw( + "synthetic", component=None, value=None + ), artifact_inputs={ edge.key: InvocationArtifactInputEdgePlan.from_source_declarations( - key=edge.key, spec=edge.spec, + key=edge.key, + spec=edge.spec, main_flow_artifacts=contract.artifact_inputs, invocation_sources=contract.artifact_inputs, ) @@ -94,50 +138,85 @@ def test_public_rescale_preserves_one_source_plane_through_stream_binding(method ) executor = CellProfilerModuleExecutor(rescale_intensity, contract) request = executor._image_request( - source, runtime, module_type=RescaleIntensityModule, + source, + runtime, + module_type=RescaleIntensityModule, active_input_specs=contract.artifact_inputs.specs, ) execution = executor._invocation_request( - image_request=request, adapter=runtime, current_image=source, + image_request=request, + adapter=runtime, + current_image=source, module_type=RescaleIntensityModule, - kwargs={"rescale_method": method, "automatic_low": AutomaticLow.CUSTOM, - "automatic_high": AutomaticHigh.CUSTOM, "source_low": 0.0, - "source_high": 20.0, "divisor_value": 2.0}, + kwargs={ + "rescale_method": method, + "automatic_low": AutomaticLow.CUSTOM, + "automatic_high": AutomaticHigh.CUSTOM, + "source_low": 0.0, + "source_high": 20.0, + "divisor_value": 2.0, + }, ) result = CellProfilerFunctionContractExecutor().execute( - contract, rescale_intensity, execution.payload, execution.kwargs, - execution_mode=execution.execution_mode, plane_projection=execution.plane_projection, + contract, + rescale_intensity, + execution.payload, + execution.kwargs, + execution_mode=execution.execution_mode, + plane_projection=execution.plane_projection, + ) + (item,) = tuple( + StreamOutputBatch.project_item( + RuntimeProjectionSourceIdentityRequest( + value=result, + source_description="/synthetic/Detection.tif", + ) + ) + ) + fields = StreamImagePayloadMetadataProjector.item_fields( + item.metadata, ("well", "site", "channel", "z_index", "timepoint") ) - (item,) = tuple(StreamOutputBatch.project_item(RuntimeProjectionSourceIdentityRequest( - value=result, source_description="/synthetic/Detection.tif", - ))) - fields = StreamImagePayloadMetadataProjector.item_fields(item.metadata, ("well", "site", "channel", "z_index", "timepoint")) native_item = NapariStreamLayerItem( data=item.data, producer=StreamProducerIdentity.pipeline_output( - output_kind="main", output_key="main", projection_key="main", - step_name="SyntheticRescale", pipeline_position=0, step_scope_id="synthetic", + output_kind="main", + output_key="main", + projection_key="main", + step_name="SyntheticRescale", + pipeline_position=0, + step_scope_id="synthetic", ), address=NapariStreamLayerAddress( components=dict(item.require_source_component_metadata()), - path="/synthetic/Detection.tif", stream_layer_data_type=StreamingDataType.IMAGE, + path="/synthetic/Detection.tif", + stream_layer_data_type=StreamingDataType.IMAGE, ), image_metadata=item.metadata, plane_component_domain=ViewerComponentValueDomainPayload.from_wire_mapping( - fields.get(ViewerWireField.PLANE_COMPONENT_VALUES.value, {}), context="synthetic producer fields", + fields.get(ViewerWireField.PLANE_COMPONENT_VALUES.value, {}), + context="synthetic producer fields", declared_axes=STREAM_AXES, ), ) # The exact original strict receiver is reached, not replaced by a mock. - NapariAggregateAxisBindingBuilder.bindings((native_item,), ViewerComponentAxisSemanticsFactory.empty()) + NapariAggregateAxisBindingBuilder.bindings( + (native_item,), ViewerComponentAxisSemanticsFactory.empty() + ) assert contract.artifact_inputs.names() == ("Bright",) assert request.image_count == 1 assert result.data.shape == (4, 5) assert result.metadata.source_spatial_domain == metadata.source_spatial_domain assert result.metadata.source_image_paths == metadata.source_image_paths assert result.metadata.source_voxel_spacing == metadata.source_voxel_spacing - assert SourceVoxelSpacing.common_physical_pixel_size((result.metadata.source_voxel_spacing,)) is None - assert item.require_source_component_metadata() == metadata.source_component_metadata + assert ( + SourceVoxelSpacing.common_physical_pixel_size( + (result.metadata.source_voxel_spacing,) + ) + is None + ) + assert ( + item.require_source_component_metadata() == metadata.source_component_metadata + ) expected = pixels / (2.0 if method is RescaleMethod.DIVIDE_BY_VALUE else 20.0) np.testing.assert_array_equal(item.data, expected) @@ -150,14 +229,17 @@ def test_declared_source_plane_has_area_not_volume(retained_site): metadata = ImagePayloadMetadata( source_path="/synthetic/site.tif", source_component_metadata={"site": "7"}, - source_spatial_domain=SourceSpatialDomain(origin_yx=(0, 0), source_shape_yx=(4, 5)), + source_spatial_domain=SourceSpatialDomain( + origin_yx=(0, 0), source_shape_yx=(4, 5) + ), ) if retained_site: pixels = pixels[None] labels = labels[None] metadata = ImagePayloadMetadata( source_image_provenance_planes=SourceImageProvenancePlanes.from_components( - paths=("/synthetic/site.tif",), component_metadata=({"site": "7"},), + paths=("/synthetic/site.tif",), + component_metadata=({"site": "7"},), ), source_spatial_domain=metadata.source_spatial_domain, plane_axis=RuntimePlaneAxis.RUNTIME_SLICE, @@ -169,19 +251,26 @@ def test_declared_source_plane_has_area_not_volume(retained_site): domain_scope=ObjectLabelDomainScope.PLANE if retained_site else None, plane_projection=( RuntimePlaneAxisValueProjection.from_source_declaration( - metadata.plane_axis, metadata.source_provenance, + metadata.plane_axis, + metadata.source_provenance, ) - if retained_site else None + if retained_site + else None ), ).payload() assert objects.plane_axis is ( RuntimePlaneAxis.RUNTIME_SLICE if retained_site else None ) assert objects.object_label_domain().scope is ( - ObjectLabelDomainScope.PLANE if retained_site else ObjectLabelDomainScope.PAYLOAD + ObjectLabelDomainScope.PLANE + if retained_site + else ObjectLabelDomainScope.PAYLOAD ) _, rows = measure_object_size_shape.__wrapped__( - payload, objects, calculate_advanced=False, calculate_zernikes=False, + payload, + objects, + calculate_advanced=False, + calculate_zernikes=False, ) fields = tuple(field.name for field in rows.fields) assert "Area" in fields @@ -194,10 +283,15 @@ def test_real_volume_is_not_squeezed_or_relabelled_as_area(): pixels = np.ones((3, 4, 5), dtype=np.float32) labels = np.zeros_like(pixels, dtype=np.int32) labels[1:, 1:3, 1:4] = 1 - payload = ImagePayloadMetadata(source_path="/synthetic/volume.tif").payload_with(pixels, None) + payload = ImagePayloadMetadata(source_path="/synthetic/volume.tif").payload_with( + pixels, None + ) objects = SourceImageObjectLabelBuildRequest(image=payload, labels=labels).payload() _, rows = measure_object_size_shape.__wrapped__( - payload, objects, calculate_advanced=False, calculate_zernikes=False, + payload, + objects, + calculate_advanced=False, + calculate_zernikes=False, ) fields = tuple(field.name for field in rows.fields) assert "Volume" in fields diff --git a/tests/unit/test_selected_plane_stream_materialization.py b/tests/unit/test_selected_plane_stream_materialization.py index 081bfd6b17..55fabeb3c8 100644 --- a/tests/unit/test_selected_plane_stream_materialization.py +++ b/tests/unit/test_selected_plane_stream_materialization.py @@ -99,7 +99,9 @@ def _source_stack(axis, component=Microscopy.Channel.name): ) def test_public_selected_checkpoint_materializes_and_streams_without_storage_axes( axis, - output_type, artifact_name, dtype, + output_type, + artifact_name, + dtype, tmp_path, monkeypatch, ): @@ -165,11 +167,10 @@ def capture_stream(data_list, paths, backend, **kwargs): request.source.metadata.component_metadata_for_item(path, 0)["channel"] == 2 ) assert payload.metadata.source_image_names == ("FITC",) - assert ( - payload.metadata.source_path - == "/input/A01_s001_w2_z001_t001.tif" + assert payload.metadata.source_path == "/input/A01_s001_w2_z001_t001.tif" + assert payload.metadata.source_voxel_spacing == SourceVoxelSpacing( + (1.3556, 1.3556) ) - assert payload.metadata.source_voxel_spacing == SourceVoxelSpacing((1.3556, 1.3556)) finally: viewer.cleanup() @@ -238,9 +239,7 @@ def with_data(self, data): ) assert calls == ["observe", "reverse", "declaration"] assert payload.metadata.source_image_names == ("FITC", "DAPI") - assert payload.metadata.retained_plane_component_values() == { - "channel": ("2", "1") - } + assert payload.metadata.retained_plane_component_values() == {"channel": ("2", "1")} np.testing.assert_array_equal(np.asarray(payload), selected.data) @@ -262,11 +261,13 @@ class DeclaredResponse(ObservedProjection, SelectedDiagnosticPlaneImageOutput): pass source = _source_stack(RuntimePlaneAxis.SOURCE_BINDING) - pixels = np.full((1, 7, 8), .75, dtype=np.float32) + pixels = np.full((1, 7, 8), 0.75, dtype=np.float32) response = DeclaredResponse(pixels, (1,)) calls.clear() payload = ImageArtifactType.contextualize_output( - source, response, None, + source, + response, + None, RuntimePlaneAxisValueProjection.preserve( axis=RuntimePlaneAxis.SOURCE_BINDING, axis_size=2 ), @@ -291,11 +292,27 @@ def test_all_public_declared_outputs_persist_with_selected_qa_streams( image=source, labels=labels, plane_projection=projection ).payload() summary = NeuriteOutgrowthSummary( - **{field.name: 0 for field in fields(NeuriteOutgrowthSummary) if field.name != "coordinate_unit"}, + **{ + field.name: 0 + for field in fields(NeuriteOutgrowthSummary) + if field.name != "coordinate_unit" + }, coordinate_unit=SourceVoxelSpacingUnit.MICROMETERS, ) cell = NeuriteOutgrowthCellResult( - 1, 1, SourceVoxelSpacingUnit.MICROMETERS, 0.0, 0, 0.0, 0.0, 0.0, 0, 0.0, 1.0, 0.0, False + 1, + 1, + SourceVoxelSpacingUnit.MICROMETERS, + 0.0, + 0, + 0.0, + 0.0, + 0.0, + 0, + 0.0, + 1.0, + 0.0, + False, ) selected = ImageArtifactType.contextualize_output( source, @@ -324,8 +341,10 @@ def test_all_public_declared_outputs_persist_with_selected_qa_streams( ), *NeuriteAdmissionPlanes( np.asarray(source)[1].astype(np.float32), - labels[1] > 0, labels[1] > 0, - np.asarray(source)[1].astype(np.float32), labels[1] > 0, + labels[1] > 0, + labels[1] > 0, + np.asarray(source)[1].astype(np.float32), + labels[1] > 0, np.asarray(source)[1].astype(np.float32), ).selected_outputs(1), *NeuriteOwnershipPlanes(labels[1], labels[1], labels[1]).selected_outputs(1), @@ -362,7 +381,9 @@ def test_all_public_declared_outputs_persist_with_selected_qa_streams( output_dir=tmp_path / "images", ) for output, value in zip(plans, values, strict=True): - value = (ImageArtifactType if output is None else output.artifact_type).contextualize_output( + value = ( + ImageArtifactType if output is None else output.artifact_type + ).contextualize_output( source, value, output, @@ -386,13 +407,16 @@ def capture_stream(data_list, paths, backend, **kwargs): monkeypatch.setattr(filemanager, "save_batch", capture_stream) try: - PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs(filemanager, plan, context) + PersistentArtifactMaterializationTargetPlan("disk").materialize_outputs( + filemanager, plan, context + ) materializations = runtime_artifact_materializations(plan, context) assert {item.output_plan.name for item in materializations} == set( specs.names() ) assert len(materializations) == 12 + len( - NeuriteAdmissionPlanes.artifact_specs() + NeuriteOwnershipPlanes.artifact_specs() + NeuriteAdmissionPlanes.artifact_specs() + + NeuriteOwnershipPlanes.artifact_specs() ) for item in materializations: outputs = item.outputs(plan, context) @@ -404,7 +428,8 @@ def capture_stream(data_list, paths, backend, **kwargs): if paths[0].endswith(".checkpoint.tif") ] assert len(checkpoints) == 5 + len( - NeuriteAdmissionPlanes.artifact_specs() + NeuriteOwnershipPlanes.artifact_specs() + NeuriteAdmissionPlanes.artifact_specs() + + NeuriteOwnershipPlanes.artifact_specs() ) for (data,), (path,), request in checkpoints: assert ( diff --git a/tests/unit/test_smoke_installed_gui.py b/tests/unit/test_smoke_installed_gui.py index 359a1dc2e0..7461331098 100644 --- a/tests/unit/test_smoke_installed_gui.py +++ b/tests/unit/test_smoke_installed_gui.py @@ -174,7 +174,9 @@ def test_gui_smoke_waits_for_gui_endpoint_before_reading_catalog() -> None: class UnreachableFunctionCatalogService: @staticmethod def prepare() -> Future[FunctionCatalogPage]: - raise AssertionError("The catalog attaches only after the GUI endpoint starts.") + raise AssertionError( + "The catalog attaches only after the GUI endpoint starts." + ) with pytest.raises( OperationTimeoutError, diff --git a/tests/unit/test_snapshot_deadline.py b/tests/unit/test_snapshot_deadline.py index 09460a9580..3fc216571f 100644 --- a/tests/unit/test_snapshot_deadline.py +++ b/tests/unit/test_snapshot_deadline.py @@ -56,7 +56,9 @@ def close(self, *args, **kwargs): pass def send(self, data, flags): - accepted = NapariAcceptedControlRequest.from_wire_mapping(pickle.loads(data)) + accepted = NapariAcceptedControlRequest.from_wire_mapping( + pickle.loads(data) + ) self.message = accepted.message server.accepted_control_requests.put(replace(accepted, response=reply)) server.process_messages() @@ -110,28 +112,38 @@ def poll(self, timeout): def test_snapshot_wire_offer_is_admitted_once_and_capture_uses_local_identity(tmp_path): from zmqruntime.messages import MessageFields - from openhcs.runtime.viewer_protocol import ViewerControlMessageRequest, ViewerRuntimeEndpoint + from openhcs.runtime.viewer_protocol import ( + ViewerControlMessageRequest, + ViewerRuntimeEndpoint, + ) from openhcs.runtime.zmq_config import OPENHCS_ZMQ_CONFIG original = ViewerWindowSnapshotRequest.from_fields( - connection=ExecutionConnectionSpec(port=5584), output_dir_path=str(tmp_path), + connection=ExecutionConnectionSpec(port=5584), + output_dir_path=str(tmp_path), ).start_operation() envelope = ViewerControlMessageRequest( endpoint=ViewerRuntimeEndpoint( transport=original.connection.transport_endpoint(OPENHCS_ZMQ_CONFIG), config=OPENHCS_ZMQ_CONFIG, ), - message_type="screenshot", payload=original, + message_type="screenshot", + payload=original, operation_deadline=original.operation_deadline, ) wire = pickle.loads(pickle.dumps(envelope.to_wire_mapping())) assert wire["payload"].operation_deadline is None - assert 0 < wire[MessageFields.OBSERVATION_BUDGET_SECONDS] <= original.timeout_ms / 1000 + assert ( + 0 < wire[MessageFields.OBSERVATION_BUDGET_SECONDS] <= original.timeout_ms / 1000 + ) admitted_at = time.monotonic() accepted = NapariAcceptedControlRequest.from_wire_mapping(wire) assert accepted.observation_deadline is not original.operation_deadline assert accepted.observation_deadline.expires_at >= admitted_at - assert accepted.message["payload"].snapshot_operation_deadline() is accepted.observation_deadline + assert ( + accepted.message["payload"].snapshot_operation_deadline() + is accepted.observation_deadline + ) assert MessageFields.OBSERVATION_BUDGET_SECONDS not in accepted.message # Qt/action consumers receive an admitted object, not a reusable duration. with pytest.raises(KeyError): diff --git a/tests/unit/test_viewer_spatial_sample.py b/tests/unit/test_viewer_spatial_sample.py index 903e7d7b6c..e589928945 100644 --- a/tests/unit/test_viewer_spatial_sample.py +++ b/tests/unit/test_viewer_spatial_sample.py @@ -97,7 +97,9 @@ def test_raw_array_slices_keep_trailing_dimension_contract_in_same_native_projec rgb = np.stack((gray, gray + 1000, gray + 2000), axis=-1) items = server.component_groups.items_for("source") items[0] = replace( - items[0], data=rgb, image_metadata=ImagePayloadMetadata(axes=PayloadAxes.colour_samples(2)) + items[0], + data=rgb, + image_metadata=ImagePayloadMetadata(axes=PayloadAxes.colour_samples(2)), ) projection = NapariViewerPayloadProjection( server=server, From 239953c4c02b4f1b99fe0f53d16d4736c318ba49 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 04:22:27 -0400 Subject: [PATCH 41/84] B6: keep the structural ratchet: move cohesive method groups into mixins, delete dead theme methods B6's fixes and black's wrapping grew six classes past their ratchet budget. Output-slice projection, function-core debug access and custom-function source files move into mixins the classes inherit (implementation by ancestry); catalog policy moves to module functions; the main window's four theme methods and SourceImageProvenance.source_component_identity had no caller (unused) and are deleted. Co-Authored-By: Claude Opus 5.5 --- openhcs/core/source_image_provenance.py | 4 - openhcs/core/steps/function_runtime.py | 360 +++++++++--------- .../backends/lib_registry/openhcs_registry.py | 95 ++--- .../processing/custom_functions/manager.py | 228 +++++------ openhcs/pyqt_gui/main.py | 54 --- 5 files changed, 355 insertions(+), 386 deletions(-) diff --git a/openhcs/core/source_image_provenance.py b/openhcs/core/source_image_provenance.py index f063cf90fa..d2f3d3d634 100644 --- a/openhcs/core/source_image_provenance.py +++ b/openhcs/core/source_image_provenance.py @@ -1600,10 +1600,6 @@ def with_common_scalar_identity_from_planes(self) -> Self: source_image_names=self.source_image_names, ) - @property - def source_component_identity(self) -> tuple[tuple[str, str], ...] | None: - return self.source_identity.identity[1] - @dataclass(frozen=True, slots=True) class SourcePlaneIndexedMetadata: diff --git a/openhcs/core/steps/function_runtime.py b/openhcs/core/steps/function_runtime.py index 176e27a46b..701e3031fa 100644 --- a/openhcs/core/steps/function_runtime.py +++ b/openhcs/core/steps/function_runtime.py @@ -323,8 +323,139 @@ def selected_artifact_output_plans(self) -> ArtifactOutputPlans: ) +class MainOutputSliceProjectionMixin: + """Project a function's main output into published slices and their producer contexts.""" + + __slots__ = () + + def _producer_output_contexts( + self, + matching_files: Sequence[str], + ) -> tuple[AlignedImageSliceContext, ...]: + """Resolve exact producer contexts for the loaded main-flow paths.""" + + return step_output_manifest(self.context).producer_output_contexts_for_paths( + self.execution_plan, + matching_files, + self.context.microscope_handler.parser, + ) + + def _producer_filename_qualifiers( + self, + matching_files: Sequence[str], + ) -> tuple[str | None, ...]: + """Resolve the filename qualifier each producer gave the loaded paths.""" + + records = step_output_manifest(self.context).producer_output_records_for_paths( + self.execution_plan, + matching_files, + self.context.microscope_handler.parser, + ) + if records is None: + return (None,) * len(matching_files) + return tuple(record.filename_qualifier for record in records) + + def _passes_producer_outputs_through( + self, + processed_stack: ImagePayload, + output_count: int, + input_count: int, + ) -> bool: + """Whether each published slice is the producer's aligned output, passed on. + + A declared passthrough chain names no output of its own, so its slices + keep the exact producer declaration (context and filename qualifier) + of the input file they are aligned to. + """ + if processed_stack.owns_output_surfaces: + return False + if self._unwrapped_main_flow_output_context() is not None: + return False + return output_count == input_count + + def _project_output_slices( + self, + processed_stack: ImagePayload, + matching_files: Sequence[str], + ) -> tuple[tuple[ImagePayload, AlignedImageSliceContext | None], ...]: + """Split the function's main output into its published slices.""" + if processed_stack.declares_whole_image_output: + output_context = self._unwrapped_main_flow_output_context() + if output_context is None and self._passes_producer_outputs_through( + processed_stack, 1, len(matching_files) + ): + output_context = self._producer_output_contexts(matching_files)[0] + if output_context is None: + output_context = AlignedImageSliceContext.anonymous_main_flow() + return ((processed_stack, output_context),) + unstack_started_at = time.perf_counter() + projected = processed_stack.main_output_slices( + lambda data: self._unstack_main_output(data, len(matching_files)) + ) + RuntimeProfileLogger.log( + logger, + "pattern_source_unstack", + time.perf_counter() - unstack_started_at, + step=self.execution_plan.step_index, + step_name=self.execution_plan.step_name, + slices=len(projected), + ) + if processed_stack.owns_output_surfaces: + return projected + output_payloads = tuple(payload for payload, _context in projected) + if self._passes_producer_outputs_through( + processed_stack, len(output_payloads), len(matching_files) + ): + slice_contexts = self._producer_output_contexts(matching_files) + else: + output_context = self._unwrapped_main_flow_output_context() + if output_context is None: + output_context = AlignedImageSliceContext.anonymous_main_flow() + slice_contexts = (output_context,) * len(output_payloads) + return tuple(zip(output_payloads, slice_contexts, strict=True)) + + def _unstack_main_output( + self, + processed_data: Any, + expected_count: int, + ) -> tuple[Any, ...]: + """Split undeclared main-output pixels along their leading runtime-slice axis.""" + try: + return unstack_runtime_slices( + processed_data, + self.execution_plan.output_memory_type, + self.execution_plan.device_id_for( + self.execution_plan.output_memory_type + ), + expected_count=expected_count, + ) + except ValueError as exc: + output_shape = np.shape(processed_data) + logger.error("Function output is not an OpenHCS image stack.") + logger.error("Output shape: %s", output_shape) + logger.error("Output ndim: %s", np.ndim(processed_data)) + raise ValueError( + "Main processing must result in an image stack shaped " + f"(N, H, W) or (N, H, W, C), got " + f"{output_shape}" + ) from exc + + def _unwrapped_main_flow_output_context( + self, + ) -> AlignedImageSliceContext | None: + ArtifactInputPlan.require_exact_map( + self.execution_plan.artifact_inputs, + boundary="Component artifact input", + ) + return self.compiled_group.unwrapped_main_flow_output_context( + self.selected_artifact_output_plans() + ) + + @dataclass(frozen=True, slots=True, kw_only=True) -class PatternGroupExecutionRequest(PatternGroupExecutionScope): +class PatternGroupExecutionRequest( + MainOutputSliceProjectionMixin, PatternGroupExecutionScope +): """All runtime data needed to process one pattern group.""" pattern_group_info: JsonValue @@ -540,33 +671,6 @@ def _record_main_flow_passthrough(self, matching_files: Sequence[str]) -> None: records = tuple(record.passed_through(plan) for record in producer_records) manifest.record_outputs(plan, records) - def _producer_output_contexts( - self, - matching_files: Sequence[str], - ) -> tuple[AlignedImageSliceContext, ...]: - """Resolve exact producer contexts for the loaded main-flow paths.""" - - return step_output_manifest(self.context).producer_output_contexts_for_paths( - self.execution_plan, - matching_files, - self.context.microscope_handler.parser, - ) - - def _producer_filename_qualifiers( - self, - matching_files: Sequence[str], - ) -> tuple[str | None, ...]: - """Resolve the filename qualifier each producer gave the loaded paths.""" - - records = step_output_manifest(self.context).producer_output_records_for_paths( - self.execution_plan, - matching_files, - self.context.microscope_handler.parser, - ) - if records is None: - return (None,) * len(matching_files) - return tuple(record.filename_qualifier for record in records) - def load_input_stack( self, ) -> tuple[list[str], RuntimeArrayData]: @@ -1022,102 +1126,6 @@ def _apply_source_binding_payload( ) return binding.apply_loaded_payload(payload, source_context) - def _passes_producer_outputs_through( - self, - processed_stack: ImagePayload, - output_count: int, - input_count: int, - ) -> bool: - """Whether each published slice is the producer's aligned output, passed on. - - A declared passthrough chain names no output of its own, so its slices - keep the exact producer declaration (context and filename qualifier) - of the input file they are aligned to. - """ - if processed_stack.owns_output_surfaces: - return False - if self._unwrapped_main_flow_output_context() is not None: - return False - return output_count == input_count - - def _project_output_slices( - self, - processed_stack: ImagePayload, - matching_files: Sequence[str], - ) -> tuple[tuple[ImagePayload, AlignedImageSliceContext | None], ...]: - """Split the function's main output into its published slices.""" - if processed_stack.declares_whole_image_output: - output_context = self._unwrapped_main_flow_output_context() - if output_context is None and self._passes_producer_outputs_through( - processed_stack, 1, len(matching_files) - ): - output_context = self._producer_output_contexts(matching_files)[0] - if output_context is None: - output_context = AlignedImageSliceContext.anonymous_main_flow() - return ((processed_stack, output_context),) - unstack_started_at = time.perf_counter() - projected = processed_stack.main_output_slices( - lambda data: self._unstack_main_output(data, len(matching_files)) - ) - RuntimeProfileLogger.log( - logger, - "pattern_source_unstack", - time.perf_counter() - unstack_started_at, - step=self.execution_plan.step_index, - step_name=self.execution_plan.step_name, - slices=len(projected), - ) - if processed_stack.owns_output_surfaces: - return projected - output_payloads = tuple(payload for payload, _context in projected) - if self._passes_producer_outputs_through( - processed_stack, len(output_payloads), len(matching_files) - ): - slice_contexts = self._producer_output_contexts(matching_files) - else: - output_context = self._unwrapped_main_flow_output_context() - if output_context is None: - output_context = AlignedImageSliceContext.anonymous_main_flow() - slice_contexts = (output_context,) * len(output_payloads) - return tuple(zip(output_payloads, slice_contexts, strict=True)) - - def _unstack_main_output( - self, - processed_data: Any, - expected_count: int, - ) -> tuple[Any, ...]: - """Split undeclared main-output pixels along their leading runtime-slice axis.""" - try: - return unstack_runtime_slices( - processed_data, - self.execution_plan.output_memory_type, - self.execution_plan.device_id_for( - self.execution_plan.output_memory_type - ), - expected_count=expected_count, - ) - except ValueError as exc: - output_shape = np.shape(processed_data) - logger.error("Function output is not an OpenHCS image stack.") - logger.error("Output shape: %s", output_shape) - logger.error("Output ndim: %s", np.ndim(processed_data)) - raise ValueError( - "Main processing must result in an image stack shaped " - f"(N, H, W) or (N, H, W, C), got " - f"{output_shape}" - ) from exc - - def _unwrapped_main_flow_output_context( - self, - ) -> AlignedImageSliceContext | None: - ArtifactInputPlan.require_exact_map( - self.execution_plan.artifact_inputs, - boundary="Component artifact input", - ) - return self.compiled_group.unwrapped_main_flow_output_context( - self.selected_artifact_output_plans() - ) - def _save_outputs( self, processed_stack: RuntimeArrayData, @@ -1607,8 +1615,62 @@ def _save_artifact_value( return runtime_value.data +class FunctionCoreDebugMixin: + """Expose one invocation's debug cursor, artifacts and events.""" + + __slots__ = () + + def debug_cursor(self) -> DebugCursor: + return DebugCursor.from_invocation( + step_index=self.group_data.execution_plan.step_index, + step_scope_id=self.group_data.execution_plan.step_scope_id, + invocation=self.invocation, + pattern_group_identity=str(self.group_data.runtime_plane_index), + ) + + def debug_artifacts( + self, + artifact_plans: ArtifactInputPlans | ArtifactOutputPlans, + artifact_values: Mapping[ArtifactSpecRef, object] | None = None, + ) -> DebugArtifactRefProjection: + return DebugArtifactRefProjection.from_artifact_plans( + artifact_plans=artifact_plans, + cursor=self.debug_cursor(), + artifact_values=artifact_values, + ) + + def debug_event( + self, + event_type: DebugEventType, + *, + exception: Exception | None = None, + timing_seconds: float | None = None, + invocation_parameters: tuple[DebugInvocationParameter, ...] = (), + input_artifact_values: Mapping[ArtifactSpecRef, object] | None = None, + ) -> DebugEvent: + return DebugEvent.for_invocation( + event_type=event_type, + cursor=self.debug_cursor(), + step_name=self.group_data.execution_plan.step_name, + callable_name=self.invocation.key.function_name, + axis_id=self.group_data.execution_plan.axis_id, + input_artifacts=self.debug_artifacts( + { + edge.storage_plan.ref(): edge.storage_plan + for edge in self.artifact_inputs.values() + if edge.storage_plan is not None + }, + input_artifact_values, + ), + output_artifacts=self.debug_artifacts(self.artifact_outputs), + exception=exception, + timing_seconds=timing_seconds, + invocation_parameters=invocation_parameters, + ) + + @dataclass(frozen=True, slots=True) -class FunctionCoreExecutor: +class FunctionCoreExecutor(FunctionCoreDebugMixin): """Execute one scoped callable invocation and route declared artifact I/O.""" group_data: PatternGroupData @@ -1927,54 +1989,6 @@ def load_artifact_input( ) return loaded_values - def debug_cursor(self) -> DebugCursor: - return DebugCursor.from_invocation( - step_index=self.group_data.execution_plan.step_index, - step_scope_id=self.group_data.execution_plan.step_scope_id, - invocation=self.invocation, - pattern_group_identity=str(self.group_data.runtime_plane_index), - ) - - def debug_artifacts( - self, - artifact_plans: ArtifactInputPlans | ArtifactOutputPlans, - artifact_values: Mapping[ArtifactSpecRef, object] | None = None, - ) -> DebugArtifactRefProjection: - return DebugArtifactRefProjection.from_artifact_plans( - artifact_plans=artifact_plans, - cursor=self.debug_cursor(), - artifact_values=artifact_values, - ) - - def debug_event( - self, - event_type: DebugEventType, - *, - exception: Exception | None = None, - timing_seconds: float | None = None, - invocation_parameters: tuple[DebugInvocationParameter, ...] = (), - input_artifact_values: Mapping[ArtifactSpecRef, object] | None = None, - ) -> DebugEvent: - return DebugEvent.for_invocation( - event_type=event_type, - cursor=self.debug_cursor(), - step_name=self.group_data.execution_plan.step_name, - callable_name=self.invocation.key.function_name, - axis_id=self.group_data.execution_plan.axis_id, - input_artifacts=self.debug_artifacts( - { - edge.storage_plan.ref(): edge.storage_plan - for edge in self.artifact_inputs.values() - if edge.storage_plan is not None - }, - input_artifact_values, - ), - output_artifacts=self.debug_artifacts(self.artifact_outputs), - exception=exception, - timing_seconds=timing_seconds, - invocation_parameters=invocation_parameters, - ) - def execute( self, *, diff --git a/openhcs/processing/backends/lib_registry/openhcs_registry.py b/openhcs/processing/backends/lib_registry/openhcs_registry.py index 9edb85602f..20e5b6308d 100644 --- a/openhcs/processing/backends/lib_registry/openhcs_registry.py +++ b/openhcs/processing/backends/lib_registry/openhcs_registry.py @@ -353,8 +353,8 @@ def _distribution_excluded_by_package_metadata(name: str) -> bool: "requirements; a module cannot depend on an undeclared distribution." ) if any( - requirement.marker is None or requirement.marker.evaluate({"extra": ""}) - for requirement in requirements + marker is None or marker.evaluate({"extra": ""}) + for marker in (requirement.marker for requirement in requirements) ): return False try: @@ -392,6 +392,46 @@ def _source_required_distributions(source: str, origin: str) -> tuple[str, ...]: return () +def _module_in_this_catalog( + module_name: str, + allowed_memory_types: frozenset[str], +) -> bool: + """Memory-type eligibility and the package requirement markers decide. + + A module whose ``REQUIRED_DISTRIBUTIONS`` names a distribution those + markers leave out of this environment is not in this installation's catalog. + """ + + if not _module_declares_allowed_memory_type(module_name, allowed_memory_types): + logger.debug( + "Skipping OpenHCS module %s - no allowed memory decorators", module_name + ) + return False + excluded = _module_distributions_excluded_by_package_metadata(module_name) + if excluded: + logger.info( + "Skipping OpenHCS module %s - package requirements exclude %s here", + module_name, + ", ".join(excluded), + ) + return not excluded + + +def _catalog_policy_context() -> dict[str, Any]: + """The declaration-import policy as catalogue cache identity.""" + + return { + "allowed_memory_types": sorted(_allowed_openhcs_memory_types()), + "distributions_excluded_by_package_metadata": sorted( + { + canonicalize_name(requirement.name) + for requirement in _openhcs_requirements() + if _distribution_excluded_by_package_metadata(requirement.name) + } + ), + } + + def _module_distributions_excluded_by_package_metadata( module_name: str, ) -> tuple[str, ...]: @@ -582,41 +622,19 @@ def _get_openhcs_modules(self) -> List[str]: return modules def get_modules_to_scan(self) -> List[Tuple[str, Any]]: - """Import every admitted native declaration before catalog publication. - - Memory-type eligibility and the OpenHCS package requirement markers - own exclusions: a module whose ``REQUIRED_DISTRIBUTIONS`` names a - distribution those markers leave out of this environment is not part - of this installation's catalog. Any other eligible OpenHCS module that - cannot import is a broken installation, not an optional function - inventory; its failure must prevent apparent catalog readiness. + """Import every native declaration in this installation's catalog. + + ``_module_in_this_catalog`` owns exclusions; any other OpenHCS module + that cannot import is a broken installation and fails catalog readiness. """ self._ensure_module_inventory() assert self.MODULES_TO_SCAN is not None - modules = [] allowed_memory_types = _allowed_openhcs_memory_types() - for module_name in self.MODULES_TO_SCAN: - if not _module_declares_allowed_memory_type( - module_name, - allowed_memory_types, - ): - logger.debug( - "Skipping OpenHCS module %s - no allowed memory decorators", - module_name, - ) - continue - excluded = _module_distributions_excluded_by_package_metadata(module_name) - if excluded: - logger.info( - "Skipping OpenHCS module %s - package requirements exclude %s " - "from this environment", - module_name, - ", ".join(excluded), - ) - continue - module = import_module_preserving_root_logging(module_name) - modules.append((module_name, module)) - return modules + return [ + (module_name, import_module_preserving_root_logging(module_name)) + for module_name in self.MODULES_TO_SCAN + if _module_in_this_catalog(module_name, allowed_memory_types) + ] # ===== ESSENTIAL ABC METHODS ===== def get_library_version(self) -> str: @@ -647,16 +665,7 @@ def cache_source_mtimes(self) -> Dict[str, float]: def cache_discovery_context(self) -> dict[str, Any]: """Project the declaration-import policy into catalogue cache identity.""" - return { - "allowed_memory_types": sorted(_allowed_openhcs_memory_types()), - "distributions_excluded_by_package_metadata": sorted( - { - canonicalize_name(requirement.name) - for requirement in _openhcs_requirements() - if _distribution_excluded_by_package_metadata(requirement.name) - } - ), - } + return _catalog_policy_context() def is_library_available(self) -> bool: """OpenHCS is always available.""" diff --git a/openhcs/processing/custom_functions/manager.py b/openhcs/processing/custom_functions/manager.py index c607ca8c36..cd426fcb4b 100644 --- a/openhcs/processing/custom_functions/manager.py +++ b/openhcs/processing/custom_functions/manager.py @@ -80,7 +80,122 @@ def backend_label(self) -> str: return self.memory_type or self.contract.execution_scope.value -class CustomFunctionManager: +class CustomFunctionSourceFilesMixin: + """Persist custom-function sources as exact bytes and verify their revision snapshots.""" + + def _save_function_code(self, func_name: str, code: str) -> None: + """ + Save function code to storage directory. + + Args: + func_name: Name of function (used as filename) + code: Python source code + """ + file_path = self.source_path_for_name(self.storage_dir, func_name) + temp_path = self._write_temporary_source(code) + try: + os.replace(temp_path, file_path) + finally: + temp_path.unlink(missing_ok=True) + logger.info(f"Saved custom function to: {file_path}") + + def _snapshot_source( + self, + source_path: Path, + ) -> CustomFunctionSourceSnapshot | None: + """Read one exact persisted source without executing it.""" + + try: + source_bytes = source_path.read_bytes() + except FileNotFoundError: + return None + try: + code = source_bytes.decode("utf-8") + except UnicodeDecodeError as exc: + raise ValidationError( + f"Persisted custom function '{source_path.name}' is not UTF-8." + ) from exc + return CustomFunctionSourceSnapshot( + source=CustomFunctionSource( + function_name=source_path.stem, + content_sha256=CustomFunctionSource.content_digest(source_bytes), + ), + code=code, + ) + + def _snapshot_all_sources( + self, + ) -> tuple[ + CustomFunctionSourceRevision, + tuple[CustomFunctionSourceSnapshot, ...], + ]: + """Read the persisted source set that one reconciliation will prepare.""" + + snapshots = [] + for source_path in sorted(self.storage_dir.glob("*.py")): + snapshot = self._snapshot_source(source_path) + if snapshot is None: + raise ValidationError( + "Persisted custom-function sources changed while being read; " + "retry reconciliation." + ) + snapshots.append(snapshot) + revision = CustomFunctionSourceRevision( + sources=tuple(snapshot.source for snapshot in snapshots) + ) + return revision, tuple(snapshots) + + def _require_snapshot( + self, + source_path: Path, + expected: CustomFunctionSourceSnapshot, + ) -> None: + """Fail loudly when one prepared source no longer owns the same bytes.""" + + current = self._snapshot_source(source_path) + if current is None or current.source != expected.source: + raise ValidationError( + f"Persisted custom function '{source_path.name}' changed during " + "preparation; retry the lifecycle operation." + ) + + def _require_revision(self, expected: CustomFunctionSourceRevision) -> None: + """Fail loudly unless the prepared persisted source set is still exact.""" + + try: + current = self.source_revision() + except FileNotFoundError as exc: + raise ValidationError( + "Persisted custom-function sources changed during preparation; " + "retry reconciliation." + ) from exc + if current != expected: + raise ValidationError( + "Persisted custom-function sources changed during preparation; " + "retry reconciliation." + ) + + def _write_temporary_source(self, code: str) -> Path: + """Write source beside its destination for an atomic replacement.""" + + file_descriptor, raw_path = tempfile.mkstemp( + suffix=".py", + dir=self.storage_dir, + ) + try: + # Persist the exact bytes the revision digest was computed from; + # text-mode newline translation would change them on Windows. + with os.fdopen(file_descriptor, "wb") as stream: + stream.write(code.encode("utf-8")) + stream.flush() + os.fsync(stream.fileno()) + except BaseException: + Path(raw_path).unlink(missing_ok=True) + raise + return Path(raw_path) + + +class CustomFunctionManager(CustomFunctionSourceFilesMixin): """ Manager for custom function lifecycle operations. @@ -574,117 +689,6 @@ def _check_name_collision( "OpenHCS function. Please choose a different name." ) - def _save_function_code(self, func_name: str, code: str) -> None: - """ - Save function code to storage directory. - - Args: - func_name: Name of function (used as filename) - code: Python source code - """ - file_path = self.source_path_for_name(self.storage_dir, func_name) - temp_path = self._write_temporary_source(code) - try: - os.replace(temp_path, file_path) - finally: - temp_path.unlink(missing_ok=True) - logger.info(f"Saved custom function to: {file_path}") - - def _snapshot_source( - self, - source_path: Path, - ) -> CustomFunctionSourceSnapshot | None: - """Read one exact persisted source without executing it.""" - - try: - source_bytes = source_path.read_bytes() - except FileNotFoundError: - return None - try: - code = source_bytes.decode("utf-8") - except UnicodeDecodeError as exc: - raise ValidationError( - f"Persisted custom function '{source_path.name}' is not UTF-8." - ) from exc - return CustomFunctionSourceSnapshot( - source=CustomFunctionSource( - function_name=source_path.stem, - content_sha256=CustomFunctionSource.content_digest(source_bytes), - ), - code=code, - ) - - def _snapshot_all_sources( - self, - ) -> tuple[ - CustomFunctionSourceRevision, - tuple[CustomFunctionSourceSnapshot, ...], - ]: - """Read the persisted source set that one reconciliation will prepare.""" - - snapshots = [] - for source_path in sorted(self.storage_dir.glob("*.py")): - snapshot = self._snapshot_source(source_path) - if snapshot is None: - raise ValidationError( - "Persisted custom-function sources changed while being read; " - "retry reconciliation." - ) - snapshots.append(snapshot) - revision = CustomFunctionSourceRevision( - sources=tuple(snapshot.source for snapshot in snapshots) - ) - return revision, tuple(snapshots) - - def _require_snapshot( - self, - source_path: Path, - expected: CustomFunctionSourceSnapshot, - ) -> None: - """Fail loudly when one prepared source no longer owns the same bytes.""" - - current = self._snapshot_source(source_path) - if current is None or current.source != expected.source: - raise ValidationError( - f"Persisted custom function '{source_path.name}' changed during " - "preparation; retry the lifecycle operation." - ) - - def _require_revision(self, expected: CustomFunctionSourceRevision) -> None: - """Fail loudly unless the prepared persisted source set is still exact.""" - - try: - current = self.source_revision() - except FileNotFoundError as exc: - raise ValidationError( - "Persisted custom-function sources changed during preparation; " - "retry reconciliation." - ) from exc - if current != expected: - raise ValidationError( - "Persisted custom-function sources changed during preparation; " - "retry reconciliation." - ) - - def _write_temporary_source(self, code: str) -> Path: - """Write source beside its destination for an atomic replacement.""" - - file_descriptor, raw_path = tempfile.mkstemp( - suffix=".py", - dir=self.storage_dir, - ) - try: - # Persist the exact bytes the revision digest was computed from; - # text-mode newline translation would change them on Windows. - with os.fdopen(file_descriptor, "wb") as stream: - stream.write(code.encode("utf-8")) - stream.flush() - os.fsync(stream.fileno()) - except BaseException: - Path(raw_path).unlink(missing_ok=True) - raise - return Path(raw_path) - def _clear_caches(self) -> None: """ Clear function registry metadata caches. diff --git a/openhcs/pyqt_gui/main.py b/openhcs/pyqt_gui/main.py index 4c56133572..82c64d7383 100644 --- a/openhcs/pyqt_gui/main.py +++ b/openhcs/pyqt_gui/main.py @@ -20,7 +20,6 @@ from PyQt6.QtWidgets import ( QApplication, QDialog, - QFileDialog, QLabel, QMainWindow, QMessageBox, @@ -1728,59 +1727,6 @@ def _close_object_state_subscriptions(self) -> None: self._object_state_unregister_subscription.release() self.time_travel_widget.close() - # ========== THEME MANAGEMENT METHODS ========== - - def switch_to_dark_theme(self): - """Switch to dark theme variant.""" - self.service_adapter.theme_manager.switch_to_dark_theme() - self.status_message.emit("Switched to dark theme") - - def switch_to_light_theme(self): - """Switch to light theme variant.""" - self.service_adapter.theme_manager.switch_to_light_theme() - self.status_message.emit("Switched to light theme") - - def load_theme_from_file(self): - """Load theme from JSON configuration file.""" - file_path, _ = QFileDialog.getOpenFileName( - self, "Load Theme Configuration", "", "JSON Files (*.json);;All Files (*)" - ) - - if file_path: - theme_loaded = self.service_adapter.theme_manager.load_theme_from_config( - file_path - ) - if theme_loaded: - self.status_message.emit(f"Loaded theme from {Path(file_path).name}") - else: - QMessageBox.warning( - self, - "Theme Load Error", - f"Failed to load theme from {Path(file_path).name}", - ) - - def save_theme_to_file(self): - """Save current theme to JSON configuration file.""" - file_path, _ = QFileDialog.getSaveFileName( - self, - "Save Theme Configuration", - "pyqt6_color_scheme.json", - "JSON Files (*.json);;All Files (*)", - ) - - if file_path: - theme_saved = self.service_adapter.theme_manager.save_current_theme( - file_path - ) - if theme_saved: - self.status_message.emit(f"Saved theme to {Path(file_path).name}") - else: - QMessageBox.warning( - self, - "Theme Save Error", - f"Failed to save theme to {Path(file_path).name}", - ) - @singledispatchmethod def on_session_event(self, event: SessionEvent) -> None: del event From 22a68569de513e32d161e2a31f6813cda32aa73f Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 04:24:00 -0400 Subject: [PATCH 42/84] B6: re-pin documentation authorities after the formatting and ratchet commits Co-Authored-By: Claude Opus 5.5 --- docs/audits/architecture_a_m.json | 242 +++++---- docs/audits/architecture_n_z_development.json | 339 +++++++------ docs/audits/project_surface.json | 21 +- docs/audits/user_docs.json | 480 ++++++++++-------- 4 files changed, 601 insertions(+), 481 deletions(-) diff --git a/docs/audits/architecture_a_m.json b/docs/audits/architecture_a_m.json index ff6bc9e8e0..dc900a0c43 100644 --- a/docs/audits/architecture_a_m.json +++ b/docs/audits/architecture_a_m.json @@ -81,32 +81,32 @@ }, { "path": "openhcs/interop/cellprofiler/module_declarations.py", - "sha256": "493efb9c749b10ba54713d0c16ee7118f07ff35afbafc9ff156b7100dbd63b19", + "sha256": "6bbf41c158496a02439be2ee5aef12d7712fb632419c226a64c9259a55438b95", "role": "Defines CellProfilerModule leaves and require_module as owners of module identity, lowering, headless-import validation and artifact semantics." }, { "path": "openhcs/interop/cellprofiler/module_artifact_declarations.py", - "sha256": "286638babca312c85ee0b8319fd44ba86be5868a3c1a20b74b1b48a0bfdbc652", + "sha256": "f70b52114fb5d5e2c2fc49cd9a2b46986bde48ce02d82f1a34a67f072ca2ab20", "role": "Defines the nominal interactive-module family that rejects headless import." }, { "path": "openhcs/core/callable_contract.py", - "sha256": "2060f856bcb0cc8ef57b92fc17a1e558a676e4033075950b1a4e41dc8656d330", + "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", "role": "Declares CallableContract, the declaration-owned callable metadata at the nominal boundary." }, { "path": "openhcs/core/artifacts.py", - "sha256": "907551ae0ce4bf42a557554f5df0252b0a88fc9b7c1b7ac19705aade66b6656f", + "sha256": "65eb0ec5c903366243ca8bd5ea07412e8d34bdb81e6015fa76b3c4c59838733f", "role": "Declares the ArtifactSpecRelation family for cross-artifact semantics." }, { "path": "openhcs/core/dataset_sources/source.py", - "sha256": "f736728d2f715b72cd5479fcc79520911b037d3410a42ebfc9febef2e51ffda6", + "sha256": "097fde4d2b74031eacf53a88e79f499417e8c1c97ba0753e8a9df831b307efa6", "role": "Defines the DatasetSource registry and its layout detection and workspace projection boundary." }, { "path": "openhcs/core/dataset_sources/source_bindings_source.py", - "sha256": "7d2ac740ebb081869ad98573032849732f9b64cbbfb4fc7bd9778becc328c78c", + "sha256": "fccda877f39e7b2f99d6cc95143480e5955186264e316e832ca9a69c2ae90b8c", "role": "Defines SourceBindingsSource, which composes a dataset source with SourceBindingsConfig declarations." }, { @@ -136,6 +136,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/abstraction_lattices.rst against B6's later changes to openhcs/interop/cellprofiler/module_declarations.py, openhcs/interop/cellprofiler/module_artifact_declarations.py, openhcs/core/callable_contract.py, openhcs/core/artifacts.py, openhcs/core/dataset_sources/source.py, openhcs/core/dataset_sources/source_bindings_source.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/abstraction_lattices.rst against the later B6 changes to openhcs/interop/cellprofiler/module_declarations.py, openhcs/interop/cellprofiler/module_artifact_declarations.py, openhcs/core/callable_contract.py, openhcs/core/artifacts.py, openhcs/core/dataset_sources/source.py, openhcs/core/dataset_sources/source_bindings_source.py, openhcs/agent/capabilities.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/abstraction_lattices.rst in full and compared it against the changed authorities: openhcs/core/registry_strategies.py (deleted; mixins now in metaclass_registry.strategies), openhcs/microscopes/microscope_base.py (deleted; DatasetSource family in openhcs/core/dataset_sources), module_declarations.py, module_artifact_declarations.py, callable_contract.py, openhcs/agent/capabilities.py and the two CellProfiler tests; replaced the microscope-handler owner and SourceBindingsHandler with DatasetSource and SourceBindingsSource, attributed the strategy mixins to metaclass-registry, and confirmed CellProfilerModule.require_module, ArtifactSpecRelation, RuntimePlaneAxisStrategy, for_enum_member and CapabilityTransport against current source. Historical commit references are left as history.", "Re-read docs/source/architecture/abstraction_lattices.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -174,12 +175,12 @@ "authority": [ { "path": "openhcs/domains/microscopy/analysis_consolidation.py", - "sha256": "1084a6761a379417f86a042d635d4d43fbe55c4ac497998e48c94fe56807393a", + "sha256": "292f7138a00dc22dba5b8d8af5de6140d247b955723cc807cc982ffec7a7dd2d", "role": "Defines AnalysisConsolidationHook, its saved/reused output observations, the per-directory destinations and the FileManager summary writer." }, { "path": "openhcs/core/post_execute.py", - "sha256": "2d925b58e1fe242acf705f16fe4ac0ef34119da9827eca00c126c4194eed2919", + "sha256": "42183491eae6a5a17fd68c179970b1682343f1f73f355291f2ba4b02c784905d", "role": "Defines the PostExecuteHook family: binding from the global config, observation combine, and running after every partition succeeds." }, { @@ -199,7 +200,7 @@ }, { "path": "tests/unit/test_function_artifact_materialization.py", - "sha256": "1f6c1aafba18111da085153bbbba8b8141096b94e626647ca717be172cb63a74", + "sha256": "e92bd682c105778d8bde9cb0f63ff1baed9f4c082a05d2c7f8f04bf539bd699f", "role": "Proves the hook consumes saved CSV content, uses the compiled backend destination, skips a disabled config, and propagates failures." }, { @@ -214,6 +215,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/analysis_consolidation_system.rst against B6's later changes to openhcs/domains/microscopy/analysis_consolidation.py, openhcs/core/post_execute.py, tests/unit/test_function_artifact_materialization.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/analysis_consolidation_system.rst against the later B6 changes to openhcs/domains/microscopy/analysis_consolidation.py, openhcs/core/post_execute.py, tests/unit/test_function_artifact_materialization.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/analysis_consolidation_system.rst in full and compared it against the changed authorities: openhcs/core/orchestrator/analysis_consolidation.py (moved to openhcs/domains/microscopy/analysis_consolidation.py as a post-execute hook), consolidate_analysis_results.py, openhcs/core/config.py (consolidation config moved to openhcs/domains/microscopy/config.py), the three tests and openhcs/processing/materialization/core.py; rewrote the flow paragraph for AnalysisConsolidationHook and PostExecuteHook, removed the obsolete output-selection/skip-rendering paragraph, and confirmed the writer, DataSink save context, output-root destination and failure propagation against current source. Dropped test_orchestrator_lane_planning.py and materialization/core.py, which no longer own a stated claim.", "Re-read docs/source/architecture/analysis_consolidation_system.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -243,37 +245,37 @@ "authority": [ { "path": "openhcs/core/artifacts.py", - "sha256": "907551ae0ce4bf42a557554f5df0252b0a88fc9b7c1b7ac19705aade66b6656f", + "sha256": "65eb0ec5c903366243ca8bd5ea07412e8d34bdb81e6015fa76b3c4c59838733f", "role": "Defines the ArtifactType families, ArtifactSpec/ArtifactSpecRef, ArtifactSpecRelation leaves and materialisation declarations." }, { "path": "openhcs/core/callable_contract.py", - "sha256": "2060f856bcb0cc8ef57b92fc17a1e558a676e4033075950b1a4e41dc8656d330", + "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", "role": "Defines CallableContract and its ordered artifact input and output collections." }, { "path": "openhcs/core/function_patterns.py", - "sha256": "3dbdad68a4c7c765cc0dae056079929e5e4632f39bdbe2e746bea3acc171562d", + "sha256": "95b9056c6980e4f7943f909aeae8262252c9aa175247a534befc8d4cc459fdd2", "role": "Defines InvocationArtifactInputEdgePlan, its main_flow_projection and storage plan, and MainFlowInputProjection." }, { "path": "openhcs/core/pipeline/function_contracts.py", - "sha256": "883d22780ab41f122d6ed651c9f59fea540120f8fcf50d30e348ac295bd4fca9", + "sha256": "13bcaf7c7a62654f512d2cc739edccd2deeff35d99970ca493f3e827afbda91a", "role": "Defines the special_inputs, artifact_inputs and artifact_outputs decorators." }, { "path": "openhcs/core/pipeline/path_planner.py", - "sha256": "bc1380d505dae574c6dc1255afcba59b0513d933716017c05217baabed104d24", + "sha256": "677d106969aae4d818745bff29b6f84e487ec79a044c8eaf26bea1916236fc6c", "role": "Implements artifact satisfaction and exact compiled path planning." }, { "path": "tests/unit/test_implicit_main_flow_compilation.py", - "sha256": "1feb8d922c8551500b7d38db1ca1d1aa43475af1bc9a3552443980c2662bf4e4", + "sha256": "9f8f3c44f096ce0318612df7d51affbb21a7bbc837c9846d599dbe20c0a8998b", "role": "Compiles grouped native output into a CellProfiler Threshold consumer and asserts the main-flow edge, group scope and execution scope." }, { "path": "tests/unit/test_callable_contract.py", - "sha256": "b0ce0cba2e396e53f8cb9e95533ff918788e8c6dd60037a66e0aeda603bc64c2", + "sha256": "cc220f53107a08b0406b6e0dd49896e3e4c3c0af6e527b0ef00d4fdaba8f0b2c", "role": "Verifies callable contract construction, validation, and artifact semantics." }, { @@ -288,6 +290,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/artifact_contract_system.rst against B6's later changes to openhcs/core/artifacts.py, openhcs/core/callable_contract.py, openhcs/core/function_patterns.py, openhcs/core/pipeline/function_contracts.py, openhcs/core/pipeline/path_planner.py, tests/unit/test_implicit_main_flow_compilation.py, tests/unit/test_callable_contract.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/artifact_contract_system.rst against the later B6 changes to openhcs/core/artifacts.py, openhcs/core/callable_contract.py, openhcs/core/function_patterns.py, openhcs/core/pipeline/function_contracts.py, openhcs/core/pipeline/path_planner.py, tests/unit/test_implicit_main_flow_compilation.py, tests/unit/test_callable_contract.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/artifact_contract_system.rst in full and compared it against the changed authorities: openhcs/core/artifacts.py, openhcs/core/callable_contract.py, openhcs/core/pipeline/path_planner.py, tests/unit/test_callable_contract.py, tests/unit/test_artifact_input_projection_plan.py; replaced the removed consumes_main_flow flag with main_flow_projection in the occurrence and implicit main-flow sections and the regression description, tidied the special_inputs sentence, and confirmed every other named declaration (ArtifactType families, relation leaves, SourceImageProvenance, ArtifactPlanKeySelector, SelectedPlaneImageOutput, RuntimeValueStore, StepMaterializationConfig) exists with the stated role.", "Re-read docs/source/architecture/artifact_contract_system.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -315,7 +318,7 @@ "authority": [ { "path": "openhcs/authoring/session/operations/datasets.py", - "sha256": "43d9edfed59abe6b88e242d39f1ddc8f5078559e04b3875da1da07b993295993", + "sha256": "208274effc2e790775d90528211fa4f640eb01f7ca3fdd9c927c3bf71e39b632", "role": "Declares CompileDatasets and its initialised, idle and non-empty-pipeline availability checks." }, { @@ -325,7 +328,7 @@ }, { "path": "openhcs/authoring/session/compile_batch.py", - "sha256": "d539bcf8df15581d23bc69701c5d30d489c3fe5f600f19236b28ec35c1c2d92f", + "sha256": "0cbe5a6b86a0fb5e9b92c4fc45f858c979031ac93594b317b9761c28f5aaf9ab", "role": "Runs compile-only batches and compile-before-execution through BatchSubmitWaitEngine." }, { @@ -335,22 +338,22 @@ }, { "path": "openhcs/authoring/session/session.py", - "sha256": "458d71fe96ab75153977c554fb748e78f1773cd7794b625d7ec233c9b22e4453", + "sha256": "587047b63972df89ec46a01ed2c92ff3ef9577335f58a980297db02094e4e8e0", "role": "Implements run_datasets (reserve, reset, connect, compile all, submit each with compile_artifact_id) and the endpoint status binding." }, { "path": "openhcs/authoring/session/submission.py", - "sha256": "e04fdb017c3905903885b21007ebb4ceb530ec16267ea26577b3c9609b82e2ce", + "sha256": "710ac218d1db01f231b67bf0fe229dc03fb428bc6655b23671f3a026be91c673", "role": "Submits each execution with its compile_artifact_id and auxiliary observation-export parameters." }, { "path": "openhcs/authoring/session/progress.py", - "sha256": "eef42f1c12c9649c36e7d1a86d664f527e6c1aa28868eb5b348d1ebdb5d548bd", + "sha256": "cd34f68461b50506600da7854a3bf57646f2f0ff7288d0627bc40cb852a16825", "role": "Defines ExecutionProgress: one registry-mutation subscription, dirty marking and interval rebuilds on the session main thread." }, { "path": "tests/unit/test_u1_session_guards.py", - "sha256": "54e03867fc65858f21384d4c98215dc9d9accec1c0965a85d563b8410085b8f9", + "sha256": "98de1f6239607e92d2aa35bd5e95f3e1c1c42887cc238e1291ffc830ca2cd4a5", "role": "Guards that session state lives in the Qt-free session package and its operations are declared classes." } ], @@ -359,6 +362,7 @@ ], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/batch_workflow_service.rst against B6's later changes to openhcs/authoring/session/operations/datasets.py, openhcs/authoring/session/compile_batch.py, openhcs/authoring/session/session.py, openhcs/authoring/session/submission.py, openhcs/authoring/session/progress.py, tests/unit/test_u1_session_guards.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/batch_workflow_service.rst against the later B6 changes to openhcs/authoring/session/operations/datasets.py, openhcs/authoring/session/compile_batch.py, openhcs/authoring/session/session.py, openhcs/authoring/session/submission.py, openhcs/authoring/session/progress.py, tests/unit/test_u1_session_guards.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/batch_workflow_service.rst in full and compared it against the changed authorities: the deleted openhcs/pyqt_gui batch workflow, compile batch, components, client service and execution-state modules and their tests (replaced by openhcs/authoring/session), openhcs/pyqt_gui/main.py, test_main_zmq_window_routing.py, endpoint_function_catalog_service.py, zmqruntime client.py and test_concurrent_initialization_progress.py; checked CompileDatasets availability, DatasetPipelineRequest/CompiledDataset, BatchSubmitWaitEngine, run_datasets ordering and compile_artifact_id submission, ExecutionProgress listener and interval rebuild, and the endpoint status binding against the session package. The page text is accurate; the record now points at the session owners.", "Read the complete page and challenged accuracy, freshness, duplication, boundary, flow, and navigation against the listed current authorities.", @@ -395,17 +399,17 @@ "authority": [ { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "5d3646e17f7de843a5b36164f7cef7505da7841636423ae4fb4e00d194a87e9c", + "sha256": "3439d96f3fbfcb104a53019dc150049021d9a8132bd27096f1ccf510eefbe37b", "role": "Implements .cppipe parsing, declaration-owned headless validation, and lowering into PipelineConfig and FunctionStep declarations." }, { "path": "openhcs/interop/cellprofiler/module_declarations.py", - "sha256": "493efb9c749b10ba54713d0c16ee7118f07ff35afbafc9ff156b7100dbd63b19", + "sha256": "6bbf41c158496a02439be2ee5aef12d7712fb632419c226a64c9259a55438b95", "role": "Owns module-specific callable, setting, headless-import, scope, artifact, and function-catalogue declarations." }, { "path": "openhcs/interop/cellprofiler/module_artifact_declarations.py", - "sha256": "286638babca312c85ee0b8319fd44ba86be5868a3c1a20b74b1b48a0bfdbc652", + "sha256": "f70b52114fb5d5e2c2fc49cd9a2b46986bde48ce02d82f1a34a67f072ca2ab20", "role": "Defines the nominal interactive-module family that rejects headless import." }, { @@ -415,7 +419,7 @@ }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "90d75f6287c5f21f93d82960729199261f7906e0f4912b7728527260b9723a66", + "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", "role": "Declares ResolvedPipelineDefinition, the resolved step sequence the provider walks." }, { @@ -425,7 +429,7 @@ }, { "path": "tests/unit/test_cellprofiler_artifact_declarations.py", - "sha256": "5792727b3fdb1a7bffffe43fa63e7d5944a6c538ab35e90dc86e1dcfa908b189", + "sha256": "fe894cd2b592d0755c106be3363eeaf078bc26d0b698a7ccfe6662ef0c0c475b", "role": "Verifies typed image, object, measurement, relationship, table, grid, graph, and special artifact declarations." }, { @@ -454,6 +458,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/cellprofiler_interop.rst against B6's later changes to openhcs/interop/cellprofiler/pipeline_import.py, openhcs/interop/cellprofiler/module_declarations.py, openhcs/interop/cellprofiler/module_artifact_declarations.py, openhcs/core/pipeline/compilation_session.py, tests/unit/test_cellprofiler_artifact_declarations.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/cellprofiler_interop.rst against the later B6 changes to openhcs/interop/cellprofiler/pipeline_import.py, openhcs/interop/cellprofiler/module_declarations.py, openhcs/interop/cellprofiler/module_artifact_declarations.py, openhcs/core/pipeline/compilation_session.py, tests/unit/test_cellprofiler_artifact_declarations.py, tests/integration/test_cellprofiler_official30_zmq.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/cellprofiler_interop.rst in full and compared it against the changed authorities: pipeline_import.py, module_declarations.py, module_artifact_declarations.py, test_cellprofiler_artifact_declarations.py, the Official30 integration test, test_cellprofiler_pipeline_import.py and benchmark/cellprofiler_reference_exports.py; replaced the removed StepSnapshot with ResolvedPipelineDefinition and confirmed the import flow, interactive-module rejection, InputSource members, FunctionStepExecutionScope.PLATE, the provider's named methods, the CPA SQLite/properties route and CellProfilerReferenceExportPlan against current source.", "Re-read docs/source/architecture/cellprofiler_interop.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -485,12 +490,12 @@ }, { "path": "openhcs/pyqt_gui/services/ui_bridge_object_state.py", - "sha256": "81b5ffbbd19952f85430962cd1bb61918ac3abe4dd9ec843b8f67e4590ffd06b", + "sha256": "e548b367507b3fb6830ce9ecb844e3b756243af96fe38e0d35de5da0b974c1fd", "role": "Projects live ObjectState scopes, fields, revisions, document capabilities, and mutation receipts through the UI bridge." }, { "path": "openhcs/agent/dto/ui_bridge.py", - "sha256": "5f5df140f0a32fa7e0a169981b7f346f7335a2a517676eefc7b29bd8fdc52a8f", + "sha256": "918b30f34e7d096b19ce04017d9a991d0a10e4088c310797e497fb2444fc37c0", "role": "Carries generic endpoint application and operation-lifecycle declarations through UI bridge descriptors and request/response envelopes." }, { @@ -515,7 +520,7 @@ }, { "path": "openhcs/pyqt_gui/services/ui_agent_bridge.py", - "sha256": "5a03e683a0677166361440c6c78803ec2dadf753342e69996153005bccab5473", + "sha256": "ce650c52475764657bbc0221109879594f9cf1482de8245d6c141f00474d5d23", "role": "Projects ObjectState branch and snapshot restoration through the attached-agent surface." }, { @@ -525,7 +530,7 @@ }, { "path": "tests/unit/pyqt_gui/test_ui_agent_bridge.py", - "sha256": "b03744b1fa201f77de26d2e6050422b6422ebce78443c478c5e8c5f55e0a3e24", + "sha256": "de6758859d9743e826369162855327e4a84056cafdd7ddfdc33868d2a3093353", "role": "Verifies read-only rejection, semantic no-op application, and branch restoration through the UI bridge." }, { @@ -535,7 +540,7 @@ }, { "path": "tests/unit/pyqt_gui/test_pipeline_code_document_transactionality.py", - "sha256": "139725010d983c7e8a98731b4e462339c4188b8d9c44383677191810ad7bf24f", + "sha256": "0bb9b43d867a4e26f13b51531a2b7a847c6c3685841f8e002e209200a37366b6", "role": "Verifies revision checks, validation, snapshots, rollback, and transactional application." }, { @@ -570,7 +575,7 @@ }, { "path": "tests/unit/test_pycodify_formatters.py", - "sha256": "1dd67d1a782d04dc24ac2ba9ec1e457769c1c6c9191c992f0a1cc0feb78487c5", + "sha256": "2f0c6683a5255d01442ae33f676f601d5e3f1e3e04d1b2c2fc945b3c564796ce", "role": "Verifies clean/full parameter order and semantic default handling." }, { @@ -582,6 +587,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/code_ui_interconversion.rst against B6's later changes to openhcs/pyqt_gui/services/ui_bridge_object_state.py, openhcs/agent/dto/ui_bridge.py, openhcs/pyqt_gui/services/ui_agent_bridge.py, tests/unit/pyqt_gui/test_ui_agent_bridge.py, tests/unit/pyqt_gui/test_pipeline_code_document_transactionality.py, tests/unit/test_pycodify_formatters.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/code_ui_interconversion.rst against the later B6 changes to openhcs/pyqt_gui/services/ui_bridge_object_state.py, openhcs/agent/dto/ui_bridge.py, openhcs/pyqt_gui/services/ui_agent_bridge.py, tests/unit/pyqt_gui/test_ui_agent_bridge.py, tests/unit/pyqt_gui/test_pipeline_code_document_transactionality.py, tests/unit/test_pycodify_formatters.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/code_ui_interconversion.rst in full and compared it against the changed authorities: openhcs/core/pipeline_document.py (PipelineDocumentAuthority renamed PipelineDocumentCodec; PipelineDocument unchanged), ui_bridge_object_state.py, openhcs/agent/dto/ui_bridge.py, ui_bridge_transport.py, ui_bridge_server.py, zmqruntime messages.py, python-introspect annotation_types.py, ui_agent_bridge.py and its test, function_step_code_document.py (now FunctionStepDocumentCodec), the code-document transactionality test, step_parameter_editor.py (the pickled .step settings files are gone, which the page never described), pycodify_formatters.py and its test; checked the schema/bridge-protocol/application identity checks on both bridge ends, the field-mutation request token without base revision, enum values in field help, code-document revision apply, and Plate Manager naming. Every claim the page makes still holds; no text change.", "Read the complete page and challenged accuracy, freshness, duplication, boundary, flow, and navigation against the listed current authorities.", @@ -614,12 +620,12 @@ "authority": [ { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "d1216010dec1804c51aec9c774f298513638ca016d500fe6328d76a9a9bfb4d6", + "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", "role": "Lowers declarations into typed step plans, including declaration-derived framework-device assignments." }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "90d75f6287c5f21f93d82960729199261f7906e0f4912b7728527260b9723a66", + "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", "role": "Defines the per-context CompilationSession and ResolvedPipelineDefinition." }, { @@ -633,6 +639,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/compilation_service.rst against B6's later changes to openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/compilation_session.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/compilation_service.rst against the later B6 changes to openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/compilation_session.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/compilation_service.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/compiled_execution.py; the page only names the session, typed-plan and execution-bundle model and redirects to pipeline_compilation_system, all of which still exist (CompilationSession, CompiledStepPlan, CompiledExecutionBundle), so the redirect stays unchanged.", "Re-read docs/source/architecture/compilation_service.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -654,17 +661,17 @@ "authority": [ { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "d1216010dec1804c51aec9c774f298513638ca016d500fe6328d76a9a9bfb4d6", + "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", "role": "Implements the current compiler stages, including declaration-derived framework-device assignment." }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "90d75f6287c5f21f93d82960729199261f7906e0f4912b7728527260b9723a66", + "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", "role": "Defines ResolvedPipelineDefinition, which replaced resolved step snapshots, and CompilationSession." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "f4a851d7de2ea82e1d548f37e6be2511fd20d799580193496a3746e5484906ba", + "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", "role": "Defines typed step-plan fields, exact framework footprints, and framework-local device bindings." }, { @@ -678,6 +685,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/compilation_system_detailed.rst against B6's later changes to openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/compiled_step_plan.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/compilation_system_detailed.rst against the later B6 changes to openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/compiled_step_plan.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/compilation_system_detailed.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/step_snapshot.py (deleted in 0adb68108; resolved steps now live on ResolvedPipelineDefinition in compilation_session.py), openhcs/core/compiled_step_plan.py; replaced StepSnapshot with ResolvedPipelineDefinition in the redirect text and confirmed the redirect target page exists.", "Re-read docs/source/architecture/compilation_system_detailed.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -704,7 +712,7 @@ }, { "path": "openhcs/core/config.py", - "sha256": "7b5622b3d82b1e1d672eaedb14e054be81baef417a736adb06bea34fb6708b28", + "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", "role": "Declares the kernel OpenHCS configuration dataclasses that consume ObjectState mechanics." }, { @@ -718,6 +726,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/component_configuration_framework.rst against B6's later changes to openhcs/core/config.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/component_configuration_framework.rst against the later B6 changes to external/ObjectState/src/objectstate/lazy_factory.py, openhcs/core/config.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/component_configuration_framework.rst in full and compared it against the changed authorities: external/ObjectState/src/objectstate/lazy_factory.py, openhcs/core/config.py (domain sections such as analysis consolidation moved to openhcs/domains/microscopy/config.py, now added as an authority); the page only states that ObjectState owns lazy configuration and that OpenHCS documents its domain declarations, and redirects to component_system_integration and external_foundations, both of which exist, so the redirect stays unchanged.", "Re-read docs/source/architecture/component_configuration_framework.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -754,7 +763,7 @@ }, { "path": "openhcs/domains/microscopy/axes.py", - "sha256": "28cc16a296d4580351cd9abec553155e59c4d56a6ea20d52cbd6abb2879b75c6", + "sha256": "531695809259324032e0b88632813323b4fe6a9a176ad6f636aeef8dec332399", "role": "Declares the microscopy axis family, the OpenHCS domain declaration registered through metaclass-registry." } ], @@ -763,6 +772,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/component_processor_metaprogramming.rst against B6's later changes to openhcs/domains/microscopy/axes.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/component_processor_metaprogramming.rst against the later B6 changes to openhcs/domains/microscopy/axes.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/component_processor_metaprogramming.rst in full and compared it against the changed authorities: signature_analyzer.py, metaclass_registry/core.py, openhcs/constants/constants.py; confirmed signature analysis, callable-declaration projection and AutoRegisterMeta registration remain in the external packages, and replaced the deleted component-enum authority with the microscopy axis family declaration; no page edit was needed.", "Read the complete page and challenged accuracy, freshness, duplication, boundary, flow, and navigation against the listed current authorities.", @@ -787,12 +797,12 @@ "authority": [ { "path": "openhcs/core/axes.py", - "sha256": "03e5a61d808a0221507b2be93f3672cc5f4421e53eb72ce4b683309cc673213b", + "sha256": "8d93133639c576050f089fb8aab99937c8982aff07e742fea44f8a12e62bb7b2", "role": "Declares Axis, AxisFamily and the PartitionAxis role whose single member partitions compiled contexts." }, { "path": "openhcs/core/config.py", - "sha256": "7b5622b3d82b1e1d672eaedb14e054be81baef417a736adb06bea34fb6708b28", + "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", "role": "Declares the step-level variable_components and group_by fields and the pipeline-level sequential_components field." }, { @@ -807,12 +817,12 @@ }, { "path": "openhcs/core/dataset_sources/interfaces.py", - "sha256": "31e3f5789c4a1603de12a0f0f95ceb529ff3190d45f2315e2674ef81a9177047", + "sha256": "dd88254bfd5da8b0fdf0a038e358f36b49db5b8cf0b93213d56dc928af965f00", "role": "Defines MetadataComponentValueSet, the metadata-handler projection of complete axis values." }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "d1216010dec1804c51aec9c774f298513638ca016d500fe6328d76a9a9bfb4d6", + "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", "role": "Consumes the axis declarations to construct execution-axis contexts, stacks, grouping, and typed plans." } ], @@ -822,6 +832,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/component_system_integration.rst against B6's later changes to openhcs/core/axes.py, openhcs/core/config.py, openhcs/core/dataset_sources/interfaces.py, openhcs/core/pipeline/compiler.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/component_system_integration.rst against the later B6 changes to openhcs/core/axes.py, openhcs/core/config.py, openhcs/core/dataset_sources/interfaces.py, openhcs/core/pipeline/compiler.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/component_system_integration.rst in full and compared it against the changed authorities: constants.py, component_set.py, component_values.py, parser_metaprogramming.py, compiler.py; verified AxisValues, FilenameParseResult, MetadataComponentValueSet, variable_components/group_by/sequential_components and AxisFamily.partition_axis in current source, and replaced the nonexistent multiprocessing_axis declaration with the family's PartitionAxis.", "Re-read docs/source/architecture/component_system_integration.rst in full against repository base 32ca17c17 and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -845,7 +856,7 @@ "authority": [ { "path": "openhcs/core/axes.py", - "sha256": "03e5a61d808a0221507b2be93f3672cc5f4421e53eb72ce4b683309cc673213b", + "sha256": "8d93133639c576050f089fb8aab99937c8982aff07e742fea44f8a12e62bb7b2", "role": "Defines AxisFamily.named decoding, variable_axes as every axis but the partition axis, and the Ungrouped declaration." }, { @@ -855,17 +866,17 @@ }, { "path": "openhcs/core/pipeline/funcstep_contract_validator.py", - "sha256": "56968226fda4d3909aa581604e12e170d1b9b13ea1cd34536f6be32d6f4566a1", + "sha256": "4a167f620be9ff6e6524101652b6dfa25984b3a389b19da3f5ce78c14d5947d4", "role": "Validates allowed group_by roles, required callable axis roles, and declared memory contracts per callable." }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "d1216010dec1804c51aec9c774f298513638ca016d500fe6328d76a9a9bfb4d6", + "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", "role": "Rejects pipeline sequential axes that are also a step's variable axes." }, { "path": "tests/unit/test_funcstep_contract_validator.py", - "sha256": "e40f4cd60e418dcafcea5d8b6ddcd61e222478e14da3bd3d8a5d5f5263af9cc0", + "sha256": "a208ca59a60b2d5bbcc9955e99592fd480368995336f6ef86a0a06f8f6991747", "role": "Covers accepted and rejected grouping, required-role, and callable-contract combinations." } ], @@ -876,6 +887,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/component_validation_system.rst against B6's later changes to openhcs/core/axes.py, openhcs/core/pipeline/funcstep_contract_validator.py, openhcs/core/pipeline/compiler.py, tests/unit/test_funcstep_contract_validator.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/component_validation_system.rst against the later B6 changes to openhcs/core/axes.py, openhcs/core/pipeline/funcstep_contract_validator.py, openhcs/core/pipeline/compiler.py, tests/unit/test_funcstep_contract_validator.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/component_validation_system.rst in full and compared it against the changed authorities: validation.py, funcstep_contract_validator.py, test_funcstep_contract_validator.py; traced GenericValidator.validate_step, validate_dict_pattern_keys, validate_allowed_group_by, validate_required_axis_roles and the compiler's sequential-components check, and rewrote the check list and consumer guidance to match.", "Re-read docs/source/architecture/component_validation_system.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -906,7 +918,7 @@ }, { "path": "openhcs/core/orchestrator/bundle_execution.py", - "sha256": "7665bd4ab8b9ce73d3dcb4fb9679e836b23bffb81d70541baef2e114ee54b0d9", + "sha256": "f5ba09a2ec7c688b1ca08167ab398cc976a9e2c39b301ae98f3c5fd560a25c85", "role": "Runs one compiled bundle, then the parent-owned dataset-scoped steps and post-execute hooks." }, { @@ -916,7 +928,7 @@ }, { "path": "openhcs/core/orchestrator/worker_execution.py", - "sha256": "1d71883f545f7f8702c64710f968b4138d0748fa0c58b6e1b87e06a7045e5ebf", + "sha256": "23a3ef2a70378dd1bb5c413a56e36bbb3fa013142741c9b45407bfc47de8c4a4", "role": "Owns executor construction, the WorkerCommand family served by process workers, result collection, cancellation, and cleanup." }, { @@ -931,7 +943,7 @@ }, { "path": "tests/unit/test_orchestrator_lane_planning.py", - "sha256": "4a83337fa35c68e4f641760ebbf86e4cc30f6624da55ccebd09be636a4b56bd0", + "sha256": "3d5c2f3662b8d4c330582af0c4a649b8f4d9582e064d9cf3159798c8934a2bbb", "role": "Verifies that worker-lane grouping remains correct when context mapping keys do not encode axis identity." }, { @@ -948,6 +960,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/concurrency_model.rst against B6's later changes to openhcs/core/orchestrator/bundle_execution.py, openhcs/core/orchestrator/worker_execution.py, tests/unit/test_orchestrator_lane_planning.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/concurrency_model.rst against the later B6 changes to openhcs/core/orchestrator/bundle_execution.py, openhcs/core/orchestrator/worker_execution.py, tests/unit/test_orchestrator_lane_planning.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/concurrency_model.rst in full and compared it against the changed authorities: compiled_execution.py, processing_context.py, worker_lanes.py, test_orchestrator_lane_planning.py, worker_execution.py, test_worker_start_policy.py; confirmed CompiledExecutionBundle, WorkerLaneExecutionContext, resolve_worker_start_context, the cancellation signal and AXIS_COMPLETED, renamed compiled_plate_execution to bundle_execution, and corrected the lane, worker-command, progress and post-execute claims.", "Re-read docs/source/architecture/concurrency_model.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -981,7 +994,7 @@ }, { "path": "openhcs/core/config.py", - "sha256": "7b5622b3d82b1e1d672eaedb14e054be81baef417a736adb06bea34fb6708b28", + "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", "role": "Declares GlobalPipelineConfig and the step and standalone configuration dataclasses OpenHCS composes over ObjectState." } ], @@ -990,6 +1003,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/configuration_framework.rst against B6's later changes to openhcs/core/config.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/configuration_framework.rst against the later B6 changes to external/ObjectState/src/objectstate/lazy_factory.py, openhcs/core/config.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/configuration_framework.rst in full and compared it against the changed authorities: lazy_factory.py, context_manager.py, openhcs/core/config.py; confirmed ObjectState still owns lazy configuration and context resolution, the linked system_overview and external_foundations pages exist, and no page edit was needed.", "Re-read docs/source/architecture/configuration_framework.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -1016,7 +1030,7 @@ }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "90d75f6287c5f21f93d82960729199261f7906e0f4912b7728527260b9723a66", + "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", "role": "Consumes an already resolved GlobalPipelineConfig in a scoped compiler session." } ], @@ -1025,6 +1039,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/context_system.rst against B6's later changes to openhcs/core/pipeline/compilation_session.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/context_system.rst against the later B6 changes to openhcs/core/pipeline/compilation_session.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/context_system.rst in full and compared it against the changed authorities: context_manager.py, compilation_session.py; confirmed ObjectState owns context management and the compiler session consumes a resolved GlobalPipelineConfig; no page edit was needed.", "Read the complete page and challenged accuracy, freshness, duplication, boundary, flow, and navigation against the listed current authorities.", @@ -1072,7 +1087,7 @@ "authority": [ { "path": "openhcs/processing/custom_functions/manager.py", - "sha256": "5cd7e955087f0bd9c527fc6307dff48f4aa959c607a3ec776b8379abac76fb36", + "sha256": "0ffa6fc2a18566600dcff33d4b27466d8a5684662fa1c696cb5ea52ea459a1e4", "role": "Owns persisted custom-function lifecycle and projects validated declarations through the custom runtime registry." }, { @@ -1086,6 +1101,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/custom_function_registration_system.rst against B6's later changes to openhcs/processing/custom_functions/manager.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/custom_function_registration_system.rst against the later B6 changes to openhcs/processing/custom_functions/manager.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/custom_function_registration_system.rst in full and compared it against the changed authority manager.py; confirmed the user_guide/custom_function_management target exists and CustomFunctionManager still owns persisted custom functions through the runtime registry; no page edit was needed.", "Re-read docs/source/architecture/custom_function_registration_system.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -1121,7 +1137,7 @@ }, { "path": "openhcs/authoring/session/views.py", - "sha256": "2963a2c48856704ea8291075316c470ccc8eeed161b5a798181a519a2ceaef98", + "sha256": "8ce6cac27293ac9105f4d2ebc76d10eebeba1ef90e0e8954c0a117910e17e3b3", "role": "Declares the SessionView family, each view's title, and the operations its window binds." }, { @@ -1140,6 +1156,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/declarative_window_system.rst against B6's later changes to openhcs/authoring/session/views.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/declarative_window_system.rst against the later B6 changes to openhcs/authoring/session/views.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/declarative_window_system.rst in full and compared it against the changed authorities: ui_bridge_identities.py, test_build_website.py; confirmed the pyqt-reactive window-manager and scope-window-factory pages and sources exist, traced UiSessionViewWidgetIdentityDeclaration to SessionView.rendered_in, and revised the OpenHCS-retained ownership sentence.", "Read the complete page and challenged accuracy, freshness, duplication, boundary, flow, and navigation against the listed current authorities.", @@ -1162,7 +1179,7 @@ "authority": [ { "path": "openhcs/core/function_patterns.py", - "sha256": "3dbdad68a4c7c765cc0dae056079929e5e4632f39bdbe2e746bea3acc171562d", + "sha256": "95b9056c6980e4f7943f909aeae8262252c9aa175247a534befc8d4cc459fdd2", "role": "Defines callable, sequential-chain, and dictionary-fan-out patterns and their grouped source keys." }, { @@ -1176,6 +1193,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/dict_pattern_case_study.rst against B6's later changes to openhcs/core/function_patterns.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/dict_pattern_case_study.rst against the later B6 changes to openhcs/core/function_patterns.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/dict_pattern_case_study.rst in full and compared it against the changed authorities: function_patterns.py, test_function_patterns.py; confirmed NormalizedFunctionPattern still carries grouped dictionary keys and the redirect target exists; no page edit was needed.", "Re-read docs/source/architecture/dict_pattern_case_study.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -1233,27 +1251,27 @@ }, { "path": "openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py", - "sha256": "bd43ec00c265279585f6bcc9eb0a6b6a2eeedaacad6003a33a60ab181cd13a4d", + "sha256": "2705d459204557c4f7092df3fd21ed687dcf56449ed361485382b1c0ce196444", "role": "Coordinates the workbook-declared format strategy, replicate-local normalization, exports, and the configuration-projected run_directory boundary." }, { "path": "openhcs/processing/backends/experimental_analysis/result_formats.py", - "sha256": "b0790fa7050855c772f6817b3d538c86bbb161d3d0446020d808ada95351a3c0", + "sha256": "6951f41a2ff2bdac1945c7e56584a578c111c078cac1d684030c00f95a23f528", "role": "Defines the scope-keyed CX5 and MetaXpress ExperimentalResultFormatStrategy leaves for result reading and plate population." }, { "path": "openhcs/processing/backends/experimental_analysis/analysis.py", - "sha256": "2fe9bcfba30e739cbf0b74112bfec6ce8714f173bdbd08b6dde1e3b6abe78d6e", + "sha256": "17a701e0740eac3e11319fe73f10302bbcab603218349bfba4e07014aabc27bb", "role": "Owns layout parsing, exclusion projection, replicate averaging, and table/plate-grid export." }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "00ff3bfe7a032bd3de59274697a69d12c9b898177cd8e839f9da3b130becaedf", + "sha256": "01dcb82b5850f3cdb1c300a2aa360ba330e7f15dffb5a24e37504311095090f9", "role": "Routes the desktop analysis action through ExperimentalAnalysisEngine.run_directory with the default configuration." }, { "path": "tests/unit/test_experimental_analysis_engine.py", - "sha256": "c79887f97615197c25c340dd277189ebc331bce1f3e9bfec7466c57c92ebd6cf", + "sha256": "641fadecf8c940cc2316a06771c74db5f3f2db9dafdfbfebdc535760d6d776ec", "role": "Verifies configured path projection, exclusions, normalization-method delegation, and an end-to-end MetaXpress directory run." } ], @@ -1263,6 +1281,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/experimental_analysis_system.rst against B6's later changes to openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py, openhcs/processing/backends/experimental_analysis/result_formats.py, openhcs/processing/backends/experimental_analysis/analysis.py, openhcs/pyqt_gui/main.py, tests/unit/test_experimental_analysis_engine.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/experimental_analysis_system.rst against the later B6 changes to openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py, openhcs/processing/backends/experimental_analysis/result_formats.py, openhcs/processing/backends/experimental_analysis/analysis.py, openhcs/pyqt_gui/main.py, tests/unit/test_experimental_analysis_engine.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/experimental_analysis_system.rst in full and compared it against the changed authorities: core/config.py, unified_analysis_engine.py, the moved formats/experimental_result_formats.py, pyqt_gui/main.py, test_experimental_analysis_engine.py; traced ExperimentalAnalysisConfig and NormalizationMethod to domains/microscopy/config.py, the result-format strategies to result_formats.py and layout parsing to analysis.py, confirmed the desktop action calls ExperimentalAnalysisEngine().run_directory, and removed the stale deprecated-wrapper clause.", "Re-read docs/source/architecture/experimental_analysis_system.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -1453,7 +1472,7 @@ "authority": [ { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "5d3646e17f7de843a5b36164f7cef7505da7841636423ae4fb4e00d194a87e9c", + "sha256": "3439d96f3fbfcb104a53019dc150049021d9a8132bd27096f1ccf510eefbe37b", "role": "Implements CellProfiler lowering to FunctionStep and PipelineConfig declarations with declaration-owned headless validation." }, { @@ -1468,17 +1487,17 @@ }, { "path": "openhcs/interop/cellprofiler/module_declarations.py", - "sha256": "493efb9c749b10ba54713d0c16ee7118f07ff35afbafc9ff156b7100dbd63b19", + "sha256": "6bbf41c158496a02439be2ee5aef12d7712fb632419c226a64c9259a55438b95", "role": "Defines the nominal module validation hook queried by the importer." }, { "path": "openhcs/interop/cellprofiler/module_artifact_declarations.py", - "sha256": "286638babca312c85ee0b8319fd44ba86be5868a3c1a20b74b1b48a0bfdbc652", + "sha256": "f70b52114fb5d5e2c2fc49cd9a2b46986bde48ce02d82f1a34a67f072ca2ab20", "role": "Defines the interactive-module family that rejects headless import." }, { "path": "openhcs/core/dataset_sources/source.py", - "sha256": "f736728d2f715b72cd5479fcc79520911b037d3410a42ebfc9febef2e51ffda6", + "sha256": "097fde4d2b74031eacf53a88e79f499417e8c1c97ba0753e8a9df831b307efa6", "role": "Defines the registered DatasetSource family that microscope handlers implement." }, { @@ -1503,7 +1522,7 @@ }, { "path": "openhcs/runtime/zmq_server_hooks.py", - "sha256": "daf16cc68c8696f6c3680bd230c809be276ead00cfa84488954d328288f51f42", + "sha256": "c3027f3495c5ec188a0f9176db7fd0fcbb22451892248c9fd41d73347ac236dd", "role": "Maps generic targeted cancellation onto the exact OpenHCS orchestrator and its owned workers." }, { @@ -1518,6 +1537,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/external_integrations_overview.rst against B6's later changes to openhcs/interop/cellprofiler/pipeline_import.py, openhcs/interop/cellprofiler/module_declarations.py, openhcs/interop/cellprofiler/module_artifact_declarations.py, openhcs/core/dataset_sources/source.py, openhcs/runtime/zmq_server_hooks.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/external_integrations_overview.rst against the later B6 changes to openhcs/interop/cellprofiler/pipeline_import.py, openhcs/interop/cellprofiler/module_declarations.py, openhcs/interop/cellprofiler/module_artifact_declarations.py, openhcs/core/dataset_sources/source.py, openhcs/runtime/zmq_server_hooks.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/external_integrations_overview.rst in full and compared it against the changed authorities: pipeline_import.py, module_declarations.py, module_artifact_declarations.py, the deleted microscope_base.py, polystore/base.py, zmqruntime execution/server.py, zmq_server_hooks.py, test_cellprofiler_pipeline_import.py; replaced MicroscopeHandler with the DatasetSource family, named the PipelineImporter registration, and confirmed the CellProfiler, OMERO and cancellation claims and every linked page.", "Read the complete page and challenged accuracy, freshness, duplication, boundary, flow, and navigation against the listed current authorities.", @@ -1645,7 +1665,7 @@ "authority": [ { "path": "openhcs/core/function_patterns.py", - "sha256": "3dbdad68a4c7c765cc0dae056079929e5e4632f39bdbe2e746bea3acc171562d", + "sha256": "95b9056c6980e4f7943f909aeae8262252c9aa175247a534befc8d4cc459fdd2", "role": "Normalizes callable, (callable, kwargs), list-chain, and dict patterns into named invocations and compiles them with their memory and artifact plans." }, { @@ -1659,6 +1679,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/function_pattern_system.rst against B6's later changes to openhcs/core/function_patterns.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/function_pattern_system.rst against the later B6 changes to openhcs/core/function_patterns.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/function_pattern_system.rst in full and compared it against the changed authorities: openhcs/core/function_patterns.py and tests/unit/test_function_patterns.py; confirmed the module still accepts callable, (callable, kwargs), list and dict patterns and that docs/source/concepts/function_patterns.rst exists; the page text needed no change.", "Re-read docs/source/architecture/function_pattern_system.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -1689,12 +1710,12 @@ }, { "path": "openhcs/core/callable_contract.py", - "sha256": "2060f856bcb0cc8ef57b92fc17a1e558a676e4033075950b1a4e41dc8656d330", + "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", "role": "Adds OpenHCS processing, runtime, artifact, scope, and ArrayBridge memory semantics on top of python-introspect annotation helpers." }, { "path": "tests/unit/test_callable_contract.py", - "sha256": "b0ce0cba2e396e53f8cb9e95533ff918788e8c6dd60037a66e0aeda603bc64c2", + "sha256": "cc220f53107a08b0406b6e0dd49896e3e4c3c0af6e527b0ef00d4fdaba8f0b2c", "role": "Verifies OpenHCS callable contracts projected from declaration-owned metadata, including execution memory and runtime batching." } ], @@ -1703,6 +1724,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/function_reference_pattern.rst against B6's later changes to openhcs/core/callable_contract.py, tests/unit/test_callable_contract.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/function_reference_pattern.rst against the later B6 changes to openhcs/core/callable_contract.py, tests/unit/test_callable_contract.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/function_reference_pattern.rst in full and compared it against the changed authorities: external/python-introspect/src/python_introspect/signature_analyzer.py, openhcs/core/callable_contract.py and tests/unit/test_callable_contract.py; grepped OpenHCS imports of python_introspect and confirmed docs/source/architecture/processing_semantics.rst exists; the page text needed no change.", "Re-read docs/source/architecture/function_reference_pattern.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -1772,7 +1794,7 @@ }, { "path": "openhcs/core/callable_contract.py", - "sha256": "2060f856bcb0cc8ef57b92fc17a1e558a676e4033075950b1a4e41dc8656d330", + "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", "role": "Projects declaration-owned input, output, and execution memory into immutable callable contracts." }, { @@ -1787,17 +1809,17 @@ }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "d1216010dec1804c51aec9c774f298513638ca016d500fe6328d76a9a9bfb4d6", + "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", "role": "Calls assign_framework_devices on each context's step plans before bundle assembly." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "f4a851d7de2ea82e1d548f37e6be2511fd20d799580193496a3746e5484906ba", + "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", "role": "Owns FrameworkDeviceAssignment bindings, merged cleanup footprints, and the requires_gpu projection." }, { "path": "openhcs/core/orchestrator/worker_execution.py", - "sha256": "1d71883f545f7f8702c64710f968b4138d0748fa0c58b6e1b87e06a7045e5ebf", + "sha256": "23a3ef2a70378dd1bb5c413a56e36bbb3fa013142741c9b45407bfc47de8c4a4", "role": "Releases each axis's compiled footprint after process and inline contexts and the merged footprint only after thread-backed lanes join." }, { @@ -1832,6 +1854,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/gpu_resource_management.rst against B6's later changes to openhcs/core/callable_contract.py, openhcs/core/pipeline/compiler.py, openhcs/core/compiled_step_plan.py, openhcs/core/orchestrator/worker_execution.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/gpu_resource_management.rst against the later B6 changes to openhcs/core/callable_contract.py, openhcs/core/pipeline/compiler.py, openhcs/core/compiled_step_plan.py, openhcs/core/orchestrator/worker_execution.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/gpu_resource_management.rst in full and compared it against the changed authorities: arraybridge decorators.py and types.py, openhcs/core/callable_contract.py, openhcs/core/pipeline/framework_device_assignment.py, openhcs/core/pipeline/compiler.py, openhcs/core/compiled_step_plan.py, openhcs/core/orchestrator/worker_execution.py and the worker-start and execution-scope tests; traced assign_framework_devices from the compiler, FrameworkDeviceResolver's gpu_imports_disabled check, MemoryType.subprocess_environment() callers, _release_runtime_resources per axis and after threaded lanes, and resolve_worker_start_context; replaced one jargon phrase and refreshed evidence roles.", "Re-read docs/source/architecture/gpu_resource_management.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -1896,22 +1919,22 @@ }, { "path": "openhcs/core/config.py", - "sha256": "7b5622b3d82b1e1d672eaedb14e054be81baef417a736adb06bea34fb6708b28", + "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", "role": "Declares GlobalPipelineConfig and the step variable_components and group_by fields, with group_by defaulting from the active AxisFamily." }, { "path": "openhcs/core/processing_contracts.py", - "sha256": "9b999e9210baf292f8fbf5bb3758c41a95e74401bfb1e1c127bae2dba625b0ee", + "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", "role": "Defines the ProcessingContract family named as an independent declaration in the invariants." }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "90d75f6287c5f21f93d82960729199261f7906e0f4912b7728527260b9723a66", + "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", "role": "Defines CompilationSession, the scoped compiler state named by the invariants." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "f4a851d7de2ea82e1d548f37e6be2511fd20d799580193496a3746e5484906ba", + "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", "role": "Defines typed CompiledStepPlan fields, including framework footprints and device bindings." }, { @@ -1923,6 +1946,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/index.rst against B6's later changes to openhcs/core/config.py, openhcs/core/processing_contracts.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/compiled_step_plan.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/index.rst against the later B6 changes to openhcs/core/config.py, openhcs/core/processing_contracts.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/compiled_step_plan.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/index.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compilation_session.py, openhcs/core/compiled_step_plan.py and openhcs/core/compiled_execution.py; verified each architecture invariant against the current declarations and that every toctree target file exists; the page text needed no change.", "Re-read docs/source/architecture/index.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -1953,7 +1977,7 @@ }, { "path": "openhcs/core/config.py", - "sha256": "7b5622b3d82b1e1d672eaedb14e054be81baef417a736adb06bea34fb6708b28", + "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", "role": "Declares PartitionFilterConfig's show_preview_without_partition_filter policy and PartitionFilterMode members' preview_prefix operators." }, { @@ -1963,7 +1987,7 @@ }, { "path": "openhcs/pyqt_gui/services/reactor_providers.py", - "sha256": "43634ee5d4ac6c65d681288c575ee0c9570140d592cb3856942ec69f47c59d90", + "sha256": "aecf1c77636095408486cab53a95cb0fc044fcf2ead0d7b72ac674842cf69abe", "role": "Registers format_partition_filter_config for PartitionFilterConfig with the generic preview service." }, { @@ -1973,12 +1997,12 @@ }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "00ff3bfe7a032bd3de59274697a69d12c9b898177cd8e839f9da3b130becaedf", + "sha256": "01dcb82b5850f3cdb1c300a2aa360ba330e7f15dffb5a24e37504311095090f9", "role": "_apply_list_preview_config applies list_previews to Plate Manager and Pipeline Editor at startup and on UI config change." }, { "path": "tests/unit/pyqt_gui/test_main_config_propagation.py", - "sha256": "499b05f1249ffc522bf18e97de05d1bf2000ab159daa0685958bd37d9dc62cc4", + "sha256": "303d90b43bee7bd531a1c7664fbf9707a2e7be1696a5147614f93c40e5c7021f", "role": "Verifies the list preview config reaches both real lists." }, { @@ -2018,6 +2042,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/list_item_preview_system.rst against B6's later changes to openhcs/core/config.py, openhcs/pyqt_gui/services/reactor_providers.py, openhcs/pyqt_gui/main.py, tests/unit/pyqt_gui/test_main_config_propagation.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/list_item_preview_system.rst against the later B6 changes to openhcs/core/config.py, openhcs/pyqt_gui/services/reactor_providers.py, openhcs/pyqt_gui/main.py, tests/unit/pyqt_gui/test_main_config_propagation.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/list_item_preview_system.rst in full and compared it against the changed authorities: openhcs/core/config.py, openhcs/pyqt_gui/widgets/config_preview_formatters.py, openhcs/pyqt_gui/services/reactor_providers.py, openhcs/pyqt_gui/main.py and tests/unit/pyqt_gui/test_main_config_propagation.py; confirmed PartitionFilterConfig/PartitionFilterMode preview declarations, the single formatter registration, UIConfig.list_previews, _apply_list_preview_config for both managers, and TogglePreviewAction; corrected one stale well-selection phrase.", "Read the complete page and challenged accuracy, freshness, duplication, boundary, flow, and navigation against the listed current authorities.", @@ -2048,7 +2073,7 @@ }, { "path": "openhcs/mcp/server.py", - "sha256": "927e64594a0fc174759eeb58496fed0ccf95e5d8d29bc248cde9a033efb69fbc", + "sha256": "e1f236011df715da3d17574c0b80cbf50fcae45e3f82491c22fb57e98c6bb662", "role": "Binds the selected declaration registry into FastMCP with outputContract metadata and structured output, runs non-worker-safe capabilities through the main-thread dispatcher, and binds the session to the main thread." }, { @@ -2073,12 +2098,12 @@ }, { "path": "openhcs/mcp/dev_client_core.py", - "sha256": "b5b1aea8a5a9352d828efd408408cb0ce867db4c5dc4f91e1fa9fbb7947013b2", + "sha256": "41739b744b0e4287ca7d14cd133371d669e9bff12f4b622313996ae9bb5597da", "role": "Resolves tool identity and compatible profiles from capability declarations and renews the command deadline on each response or progress notification." }, { "path": "openhcs/mcp/dev_client_session.py", - "sha256": "0486edcefae169b51ee2baf4b6498d93411f8dd93d94bcfdfbf46ee21896c73d", + "sha256": "2dceca7937087d05f72af1d037122ea393721d61d690803628a7b14ca70fd980", "role": "Implements execute-source as add-datasets, set-pipeline, then initialize, compile and run stages followed through session events." }, { @@ -2088,12 +2113,12 @@ }, { "path": "openhcs/mcp/context.py", - "sha256": "af9ec43138e05762fc97b34825b96e15f183aa56781d697c601eb79f14a1bc42", + "sha256": "1292c9bb7173e6afe5aa6237b861c5f39bc2e854f364fd722c75a3c6b76f534e", "role": "Composes local stdio authoring and the headless session with the saved desktop execution endpoint while keeping hosted HTTP self-contained." }, { "path": "openhcs/authoring/session/session.py", - "sha256": "458d71fe96ab75153977c554fb748e78f1773cd7794b625d7ec233c9b22e4453", + "sha256": "587047b63972df89ec46a01ed2c92ff3ef9577335f58a980297db02094e4e8e0", "role": "Defines the headless Session whose datasets and pipelines live in the MCP process's ObjectState and whose events clients follow." }, { @@ -2153,7 +2178,7 @@ }, { "path": "openhcs/pyqt_gui/services/ui_bridge_windows.py", - "sha256": "7a10b37aba9c5ea5cdbe46a4c9703b345e83a64554d6abacbfa1ee41cd3c4020", + "sha256": "e76298f78e3cbd949e8426a2b42cd6768cc54770a5f45c9dd6c9f788b257785e", "role": "Resolves authorised OpenHCS windows, projects snapshot results, and distinguishes focus from accepted target navigation." }, { @@ -2198,12 +2223,12 @@ }, { "path": "benchmark/control_service.py", - "sha256": "f894cc090597ca04c273090c99f26fd0e3cfeffd1b7d44bbb94858892e17813c", + "sha256": "a1256b349ffdc8d04cab927326d97821462b61f57b29e564b61cc627302be49c", "role": "Finalises measured-run evidence from the session's finished execution." }, { "path": "benchmark/cellprofiler_benchmark_cli.py", - "sha256": "edc32916215a88b62702621dea9b692586572284caf9b781394b5784f62897ee", + "sha256": "4025c591644e29f3df61e476ecae2143690e014336ea9f889aadeff4e692445e", "role": "Owns the installed openhcs-benchmark execution, report and list-cases command surface." } ], @@ -2218,6 +2243,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/mcp_distribution.rst against B6's later changes to openhcs/mcp/server.py, openhcs/mcp/dev_client_core.py, openhcs/mcp/dev_client_session.py, openhcs/mcp/context.py, openhcs/authoring/session/session.py, openhcs/pyqt_gui/services/ui_bridge_windows.py, benchmark/control_service.py, benchmark/cellprofiler_benchmark_cli.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/mcp_distribution.rst against the later B6 changes to openhcs/agent/capabilities.py, openhcs/mcp/server.py, openhcs/mcp/execution.py, openhcs/mcp/dev_client_core.py, openhcs/mcp/dev_client_session.py, openhcs/mcp/context.py, openhcs/authoring/session/session.py, openhcs/pyqt_gui/services/ui_bridge_windows.py, benchmark/control_service.py, benchmark/cellprofiler_benchmark_cli.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/mcp_distribution.rst in full and compared it against the changed authorities: openhcs/agent/capabilities.py, openhcs/mcp/server.py, stdio.py, context.py, http.py, client_registration.py, dev_client*.py, the deleted openhcs/desktop_deployment.py and desktop_installation.py (moved to openhcs.desktop in 73a2aece2), ui_bridge_windows.py, synthetic-plate and benchmark control owners and tests; verified every named symbol by grep, traced McpTransportExecutor, the binder's dispatcher path, SessionJourney.run_source and skill synchronisation; corrected the PyQt/threading, session-execution, desktop-deployment, client-registration and finalisation passages.", "Read the complete page and challenged accuracy, freshness, duplication, boundary, flow, and navigation against the listed current authorities.", @@ -2287,12 +2313,12 @@ }, { "path": "openhcs/core/equivalence/keys.py", - "sha256": "1f409e1a834afd15f8bff206ebaefc8ef904c1ac7a93d43222d19e1230b7a4cb", + "sha256": "0ae79e4cc4e70804f9d2e948078ade7c271e142473821731b38eeccaef064b5d", "role": "Defines RuntimeMeasurementFeatureKey and the semantic row, source, and aggregate identities." }, { "path": "openhcs/core/equivalence/measurement_features.py", - "sha256": "2899f25b73c04a6f845a68d5e174088388978c4b21cb1fd8127ff7f100a1a0bd", + "sha256": "18ff75cdd4dad0f88870b90b9a91f2ea352372773bcc9a74cee288d4c110f05b", "role": "Defines the RuntimeMeasurementFeatureSemanticProfile most-derived strategy family." }, { @@ -2302,7 +2328,7 @@ }, { "path": "openhcs/core/measurement_dialect.py", - "sha256": "ab20f9e1edd45021e6881497b342489e312f18bef9b7b749ea4359b4cb811375", + "sha256": "96cac83e09e22eeaa02dbd14da2f9553c1f19f51962e83731936ba33429c4578", "role": "Defines the registered MeasurementDialect family and PlainMeasurementDialect." }, { @@ -2337,7 +2363,7 @@ }, { "path": "benchmark/control_service.py", - "sha256": "f894cc090597ca04c273090c99f26fd0e3cfeffd1b7d44bbb94858892e17813c", + "sha256": "a1256b349ffdc8d04cab927326d97821462b61f57b29e564b61cc627302be49c", "role": "Finalises benchmark evidence from the OpenHCS session's finished execution: its request, endpoint handshake, and completion record." }, { @@ -2391,6 +2417,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/measurement_equivalence_system.rst against B6's later changes to openhcs/core/equivalence/keys.py, openhcs/core/equivalence/measurement_features.py, openhcs/core/measurement_dialect.py, benchmark/control_service.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/measurement_equivalence_system.rst against the later B6 changes to openhcs/core/equivalence/keys.py, openhcs/core/equivalence/measurement_features.py, openhcs/core/measurement_dialect.py, benchmark/adapters/openhcs.py, benchmark/openhcs_measured_run.py, benchmark/control_service.py, tests/integration/test_cellprofiler_official30_zmq.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/measurement_equivalence_system.rst in full and compared it against the changed authorities: openhcs/core/equivalence/policy.py and keys.py, the deleted openhcs/core/equivalence/report.py (moved to benchmark/equivalence in f16177ed7), benchmark/adapters/openhcs.py, openhcs_measured_run.py, timing.py, control_service.py, openhcs/runtime/zmq_execution_observation.py, measured_run_receipt.py, the Official30 manifest and integration test, reference exports and their tests; corrected the dialect name, the comparison-package boundary, archived-export readability, the session-based finalisation path and the additive-total rule.", "Read the complete page and challenged accuracy, freshness, duplication, boundary, flow, and navigation against the listed current authorities.", @@ -2445,12 +2472,12 @@ }, { "path": "openhcs/core/callable_contract.py", - "sha256": "2060f856bcb0cc8ef57b92fc17a1e558a676e4033075950b1a4e41dc8656d330", + "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", "role": "Reads ArrayBridge metadata through MemoryContractAttribute into immutable callable input, output, and execution memory contracts." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "f4a851d7de2ea82e1d548f37e6be2511fd20d799580193496a3746e5484906ba", + "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", "role": "Owns FrameworkDeviceAssignment framework-local device bindings and footprint cleanup." }, { @@ -2464,6 +2491,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/memory_type_system.rst against B6's later changes to openhcs/core/callable_contract.py, openhcs/core/compiled_step_plan.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/memory_type_system.rst against the later B6 changes to openhcs/core/callable_contract.py, openhcs/core/compiled_step_plan.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/memory_type_system.rst in full and compared it against the changed authorities: external/arraybridge/src/arraybridge/decorators.py, types.py, openhcs/core/callable_contract.py and openhcs/core/compiled_step_plan.py; confirmed MemoryContractAttribute's INPUT/OUTPUT/EXECUTION keys, FrameworkDeviceAssignment, openhcs/core/runtime_adapters.py and the gpu_resource_management and external_foundations targets; the page text needed no change.", "Re-read docs/source/architecture/memory_type_system.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -2491,12 +2519,12 @@ "authority": [ { "path": "openhcs/core/dataset_sources/source.py", - "sha256": "f736728d2f715b72cd5479fcc79520911b037d3410a42ebfc9febef2e51ffda6", + "sha256": "097fde4d2b74031eacf53a88e79f499417e8c1c97ba0753e8a9df831b307efa6", "role": "Defines the DatasetSource family keyed by source_name, its selection-role mixins, detection_order, create hook, workspace backend registration, and virtual-workspace metadata recording." }, { "path": "openhcs/core/dataset_sources/choice.py", - "sha256": "ece43af801b4b02c302d9ac4bf6e5d07c0d9ebee5027433d12a4b548a19d11f0", + "sha256": "7fb3c7ce701f0d629f288b79090044f43ac8ce1b51ca4f50937534407f226f1a", "role": "Defines DatasetSourceChoice, AutoDetectedSource, and open(), which selects the source class, applies declared-file selection, calls create, and binds dataset_root." }, { @@ -2506,32 +2534,32 @@ }, { "path": "openhcs/core/dataset_sources/interfaces.py", - "sha256": "31e3f5789c4a1603de12a0f0f95ceb529ff3190d45f2315e2674ef81a9177047", + "sha256": "dd88254bfd5da8b0fdf0a038e358f36b49db5b8cf0b93213d56dc928af965f00", "role": "Defines MetadataHandler.component_value_set and MetadataComponentValueSet keyed by the active family's axes." }, { "path": "openhcs/core/dataset_sources/openhcs_format.py", - "sha256": "f4976001de8dcfd598fd08ebe8cb0e2c8a412e03956210d81a392d327a11ce3e", + "sha256": "a8fa6a597104075d888a5b14cc4b24a9676d1a3918c30b5df2bad4a9abb26078", "role": "Reopens a persisted workspace by looking up the recorded source_name and re-registering that source's workspace backends." }, { "path": "openhcs/core/config.py", - "sha256": "7b5622b3d82b1e1d672eaedb14e054be81baef417a736adb06bea34fb6708b28", + "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", "role": "Declares PipelineConfig.dataset_source as a DatasetSourceChoice class defaulting to AutoDetectedSource." }, { "path": "openhcs/microscopes/bioformats.py", - "sha256": "478cac79bb2512a16f1419b6f7c899d06e6e4c0d1046bd13ac031fbc9cd21470", + "sha256": "bda05411746852ea12ffe11b49e5298cb475e2137cd1a09cf9659b52a89f40e4", "role": "Shows a decoder-owned source metadata handler replaced by the workspace metadata handler after the workspace is written." }, { "path": "tests/unit/test_dataset_source_family.py", - "sha256": "e9e4decb541543619d6f6abc1dceffcfec590e916045b7a1396baf51c9bea3be", + "sha256": "4cb651b475e5e033ab94824567b8a35b0ee02f313254082e03e35cd96585d829", "role": "Verifies every registered source declares identity, selection role and backends, and that metadata detection and the factory use declared identity." }, { "path": "tests/unit/test_dataset_source_guards.py", - "sha256": "89b8cfc1492c909477e8f102d8e82baf5bb6f2d324a2866e319f90d7c97ce214", + "sha256": "adfeac23f6196a0c5393dc15b71df04fda0ec34e5ae23bfdaa22e0a165f68170", "role": "Guards that the kernel imports no domain package and that the deleted microscope-handler names do not return." } ], @@ -2543,6 +2571,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/microscope_handler_integration.rst against B6's later changes to openhcs/core/dataset_sources/source.py, openhcs/core/dataset_sources/choice.py, openhcs/core/dataset_sources/interfaces.py, openhcs/core/dataset_sources/openhcs_format.py, openhcs/core/config.py, openhcs/microscopes/bioformats.py, tests/unit/test_dataset_source_family.py, tests/unit/test_dataset_source_guards.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/microscope_handler_integration.rst against the later B6 changes to openhcs/core/dataset_sources/source.py, openhcs/core/dataset_sources/choice.py, openhcs/core/dataset_sources/interfaces.py, openhcs/core/dataset_sources/openhcs_format.py, openhcs/core/config.py, openhcs/microscopes/bioformats.py, tests/unit/test_dataset_source_family.py, tests/unit/test_dataset_source_guards.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/microscope_handler_integration.rst in full and compared it against the changed authorities: the deleted openhcs/microscopes/microscope_base.py, openhcs.py and microscope_interfaces.py (moved to openhcs/core/dataset_sources in c6d9a52ce), openhcs/core/source_bindings.py and the Bio-Formats and plate-inspection tests; traced DatasetSource registration and discovery, DatasetSourceChoice.open, detection_order, PipelineConfig.dataset_source, component_value_set, and workspace replay through the recorded source name; rewrote the page in DatasetSource vocabulary.", "Re-read docs/source/architecture/microscope_handler_integration.rst in full against repository base 32ca17c17 and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -2568,12 +2597,12 @@ "authority": [ { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "d1216010dec1804c51aec9c774f298513638ca016d500fe6328d76a9a9bfb4d6", + "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", "role": "Discovers partition-axis values through the active AxisFamily and compiles one frozen ProcessingContext per value." }, { "path": "openhcs/core/orchestrator/bundle_execution.py", - "sha256": "7665bd4ab8b9ce73d3dcb4fb9679e836b23bffb81d70541baef2e114ee54b0d9", + "sha256": "f5ba09a2ec7c688b1ca08167ab398cc976a9e2c39b301ae98f3c5fd560a25c85", "role": "Validates compiled contexts and worker-safe steps before worker setup and builds the worker-lane execution plan." }, { @@ -2583,7 +2612,7 @@ }, { "path": "openhcs/core/orchestrator/worker_execution.py", - "sha256": "1d71883f545f7f8702c64710f968b4138d0748fa0c58b6e1b87e06a7045e5ebf", + "sha256": "23a3ef2a70378dd1bb5c413a56e36bbb3fa013142741c9b45407bfc47de8c4a4", "role": "Submits one task per active lane, collects per-axis ExecutionResult values, ships or inherits the bundle, and terminates owned process workers on cancellation." }, { @@ -2610,6 +2639,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/multiprocessing_coordination_system.rst against B6's later changes to openhcs/core/pipeline/compiler.py, openhcs/core/orchestrator/bundle_execution.py, openhcs/core/orchestrator/worker_execution.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/multiprocessing_coordination_system.rst against the later B6 changes to openhcs/core/pipeline/compiler.py, openhcs/core/orchestrator/bundle_execution.py, openhcs/core/orchestrator/worker_execution.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/multiprocessing_coordination_system.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compiler.py, openhcs/core/compiled_execution.py and openhcs/core/orchestrator/worker_execution.py; traced _axis_values_to_process through AxisFamily.active().partition_axis(), validate_bundle_execution, CompiledContextLanePlanner, PooledWorkerLaneRunner._submit_lanes, ExecuteShippedLane and ProcessWorkerExecutorResources.cancel_execution; corrected the task granularity, axis vocabulary, bundle installation and cancellation wording.", "Re-read docs/source/architecture/multiprocessing_coordination_system.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", diff --git a/docs/audits/architecture_n_z_development.json b/docs/audits/architecture_n_z_development.json index 3d22254c16..8d9aadee84 100644 --- a/docs/audits/architecture_n_z_development.json +++ b/docs/audits/architecture_n_z_development.json @@ -48,7 +48,7 @@ "authority": [ { "path": "openhcs/core/viewer_streaming_service.py", - "sha256": "b85d6d9b0677bdbd99a657672e19e3a27ef418bfcddfb5550524a83e7598ec4b", + "sha256": "c96eca530f767002cfcc9aca4fe75c93d7a5848a7f42df9e5d336bf6712ed51e", "role": "Compiled output streaming parametrized by viewer type, viewer settlement, and lifecycle cleanup." }, { @@ -67,6 +67,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/napari_streaming_system.rst against B6's later changes to openhcs/core/viewer_streaming_service.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/napari_streaming_system.rst against the later B6 changes to openhcs/core/viewer_streaming_service.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/napari_streaming_system.rst in full and compared it against the changed authorities: openhcs/core/viewer_streaming_service.py and openhcs/runtime/napari_stream_visualizer.py; confirmed both redirect targets exist and the modules remain the OpenHCS streaming service and Napari adapter; the page text needed no change.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", @@ -101,17 +102,17 @@ }, { "path": "openhcs/core/processing_contracts.py", - "sha256": "9b999e9210baf292f8fbf5bb3758c41a95e74401bfb1e1c127bae2dba625b0ee", + "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", "role": "ProcessingContract subclass family owning per-callable execution over the plane axis." }, { "path": "openhcs/processing/backends/lib_registry/openhcs_registry.py", - "sha256": "7872cf8b59a954472871a72614d67bb2658cc2f4ce67e08dba1f8088589ed760", + "sha256": "2a1de7f8af9ade2cba450cbbcc1abd6510a8e3edb58217c6c08fa067bde21de2", "role": "Native callable catalogue derived from each callable's nominal catalogue declaration and the public module surface." }, { "path": "openhcs/interop/cellprofiler/module_declarations.py", - "sha256": "493efb9c749b10ba54713d0c16ee7118f07ff35afbafc9ff156b7100dbd63b19", + "sha256": "6bbf41c158496a02439be2ee5aef12d7712fb632419c226a64c9259a55438b95", "role": "CellProfilerModule declarations owning callable names, catalogue placement, and headless-import eligibility." }, { @@ -126,6 +127,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/nominal_ownership.rst against B6's later changes to openhcs/core/processing_contracts.py, openhcs/processing/backends/lib_registry/openhcs_registry.py, openhcs/interop/cellprofiler/module_declarations.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/nominal_ownership.rst against the later B6 changes to openhcs/core/processing_contracts.py, openhcs/processing/backends/lib_registry/openhcs_registry.py, openhcs/interop/cellprofiler/module_declarations.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/nominal_ownership.rst in full and compared it against the changed authorities: deleted openhcs/core/registry_strategies.py (now metaclass_registry/strategies.py), metaclass_registry/core.py, tests/unit/test_registry_owner_iteration.py, lib_registry/openhcs_registry.py, interop/cellprofiler/module_declarations.py; verified ArtifactType, ProcessingContract, CellProfilerModule, the three mixins and the equivalence MostDerivedContextStrategyMixin users exist; replaced the deleted RuntimeSliceProjectionStrategy example and named the mixins' new home.", "Re-read docs/source/architecture/nominal_ownership.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -225,12 +227,12 @@ "authority": [ { "path": "openhcs/core/orchestrator/bundle_execution.py", - "sha256": "7665bd4ab8b9ce73d3dcb4fb9679e836b23bffb81d70541baef2e114ee54b0d9", + "sha256": "f5ba09a2ec7c688b1ca08167ab398cc976a9e2c39b301ae98f3c5fd560a25c85", "role": "Parent compiled-bundle execution: executor shutdown, BrokenProcessPool handling, parent resource release, reverse-order viewer bootstrap rollback, and viewer settlement ordering." }, { "path": "openhcs/core/orchestrator/worker_execution.py", - "sha256": "1d71883f545f7f8702c64710f968b4138d0748fa0c58b6e1b87e06a7045e5ebf", + "sha256": "23a3ef2a70378dd1bb5c413a56e36bbb3fa013142741c9b45407bfc47de8c4a4", "role": "Inline, process and thread lane resources: per-axis image-cache release, process-local runtime/framework cleanup, and thread-lane deferral to the parent." }, { @@ -240,12 +242,12 @@ }, { "path": "openhcs/runtime/viewer_protocol.py", - "sha256": "f126aaf4f5c0dcc9429b7aae8ce67985260241ec7d2f3d5a948573658cf079a3", + "sha256": "c4bdae4bae6dee412f0d27a9cf75a95db53b3bd95c03cee7d046be16b52fe5b8", "role": "rollback_failed_bootstrap distinguishing connected external viewers from OpenHCS-owned viewer processes." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "f4a851d7de2ea82e1d548f37e6be2511fd20d799580193496a3746e5484906ba", + "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", "role": "Framework/device assignment merge and cleanup_loaded used for the compiled framework/device footprint." }, { @@ -260,7 +262,7 @@ }, { "path": "tests/unit/test_orchestrator_lane_planning.py", - "sha256": "4a83337fa35c68e4f641760ebbf86e4cc30f6624da55ccebd09be636a4b56bd0", + "sha256": "3d5c2f3662b8d4c330582af0c4a649b8f4d9582e064d9cf3159798c8934a2bbb", "role": "Proves partial viewer acquisition and readiness failures roll back every acquired viewer." }, { @@ -274,6 +276,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/orchestrator_cleanup_guarantees.rst against B6's later changes to openhcs/core/orchestrator/bundle_execution.py, openhcs/core/orchestrator/worker_execution.py, openhcs/runtime/viewer_protocol.py, openhcs/core/compiled_step_plan.py, tests/unit/test_orchestrator_lane_planning.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/orchestrator_cleanup_guarantees.rst against the later B6 changes to openhcs/core/orchestrator/bundle_execution.py, openhcs/core/orchestrator/worker_execution.py, openhcs/runtime/viewer_protocol.py, openhcs/core/compiled_step_plan.py, tests/unit/test_orchestrator_lane_planning.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/orchestrator_cleanup_guarantees.rst in full and compared it against the changed authorities: compiled_plate_execution.py (renamed to bundle_execution.py, 4cd901c9f), worker_execution.py, orchestrator.py, viewer_protocol.py, arraybridge types.py, test_orchestrator_lane_planning.py, processing_context.py, test_runtime_stack_cache_lifetime.py; checked executor shutdown and BrokenProcessPool handling, reverse-order viewer rollback with external-client handling, thread-lane deferral of runtime release, per-axis image-cache release and cancellation-scope finish; renamed the boundary in the opening sentence. Dropped orchestrator.py and test_framework_device_resolver.py as authorities; added compiled_step_plan.py for the device-footprint cleanup.", "Re-read docs/source/architecture/orchestrator_cleanup_guarantees.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -300,17 +303,17 @@ "authority": [ { "path": "openhcs/core/orchestrator/orchestrator.py", - "sha256": "4e55309de74e770b2e37b53f6dbedc19ffac6988ab74cdfe4b3dfc51082e8c1a", + "sha256": "a61658b107f170c192eea505966c268ab6826826f03586bd64562a44800e1ccc", "role": "PipelineOrchestrator: __objectstate_delegate__ to pipeline_config, dataset_root identity, and initialize enforcing the source role's availability check." }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "d1216010dec1804c51aec9c774f298513638ca016d500fe6328d76a9a9bfb4d6", + "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", "role": "Compiler consuming ObjectState-resolved steps and the resolved global configuration once per compilation." }, { "path": "openhcs/core/dataset_sources/source.py", - "sha256": "f736728d2f715b72cd5479fcc79520911b037d3410a42ebfc9febef2e51ffda6", + "sha256": "097fde4d2b74031eacf53a88e79f499417e8c1c97ba0753e8a9df831b307efa6", "role": "DatasetSource family and SourceSelectionRole.require_available_source, which owns the local-directory availability contract." }, { @@ -324,6 +327,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/orchestrator_configuration_management.rst against B6's later changes to openhcs/core/orchestrator/orchestrator.py, openhcs/core/pipeline/compiler.py, openhcs/core/dataset_sources/source.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/orchestrator_configuration_management.rst against the later B6 changes to openhcs/core/orchestrator/orchestrator.py, openhcs/core/pipeline/compiler.py, openhcs/core/dataset_sources/source.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/orchestrator_configuration_management.rst in full and compared it against the changed authorities: orchestrator.py, pipeline/compiler.py, objectstate/object_state.py; checked the pipeline_config ObjectState delegation, once-per-compilation resolution, and the initialize availability check (now dataset_source.open plus SourceSelectionRole.require_available_source); corrected the availability paragraph's handler/plate-path vocabulary and added dataset_sources/source.py as authority.", "Re-read docs/source/architecture/orchestrator_configuration_management.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -463,7 +467,7 @@ }, { "path": "openhcs/processing/backends/lib_registry/registry_service.py", - "sha256": "46ad34dea67bb0f7ba54e1f9f79736654df60ceefe1ef4dd523a226d3222f289", + "sha256": "daef8925338fddf8504a9faa066af42456a13fe4602976bf64be06547a45fcc8", "role": "OpenHCS consumer: imports SignatureAnalyzer from python-introspect for callable catalogue preparation instead of keeping its own signature parser." } ], @@ -472,6 +476,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/parser_metaprogramming_system.rst against B6's later changes to openhcs/processing/backends/lib_registry/registry_service.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/parser_metaprogramming_system.rst against the later B6 changes to openhcs/processing/backends/lib_registry/registry_service.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/parser_metaprogramming_system.rst in full and compared it against the changed authorities: python_introspect/signature_analyzer.py, python-introspect docs index, lib_registry/registry_service.py; confirmed python-introspect still owns signature analysis and declaration projection, registry_service.py now imports SignatureAnalyzer from it, and the external_foundations target exists; no prose change needed. Disposition corrected from revise to redirect, since the page is only a route.", "Re-read docs/source/architecture/parser_metaprogramming_system.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -502,12 +507,12 @@ }, { "path": "openhcs/core/source_binding_selection.py", - "sha256": "74061a9dd7d474c908cad4588f59c2bdd1211006b6f2cfe8a5c150d03a96424b", + "sha256": "f97a4475aa26e465679b89ff9a1fe96690ab7631649d71650608fc7a8aa4bf5a", "role": "Source candidate matching, anchor-pattern policies, and source-universe requests that replaced pattern detection." }, { "path": "openhcs/core/dataset_sources/source_bindings_source.py", - "sha256": "7d2ac740ebb081869ad98573032849732f9b64cbbfb4fc7bd9778becc328c78c", + "sha256": "fccda877f39e7b2f99d6cc95143480e5955186264e316e832ca9a69c2ae90b8c", "role": "SourceBindingsSource: the dataset source that ingests arbitrary image folders through source-binding declarations." } ], @@ -516,6 +521,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/pattern_detection_system.rst against B6's later changes to openhcs/core/source_binding_selection.py, openhcs/core/dataset_sources/source_bindings_source.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/pattern_detection_system.rst against the later B6 changes to openhcs/core/source_bindings.py, openhcs/core/source_binding_selection.py, openhcs/core/dataset_sources/source_bindings_source.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/pattern_detection_system.rst in full and compared it against the changed authorities: source_bindings.py, source_binding_selection.py, and the deleted microscopes/source_bindings_handler.py (G4, c6d9a52ce); confirmed typed source matching still owns the model and the source_model target page exists; replaced the deleted handler authority with dataset_sources/source_bindings_source.py; no prose change needed.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", @@ -533,12 +539,12 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "7b5622b3d82b1e1d672eaedb14e054be81baef417a736adb06bea34fb6708b28", + "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", "role": "ProcessingConfig field docstrings: variable_components is the stack-axis authority and group_by partitions already assembled values, used for dictionary-pattern branch selection." }, { "path": "openhcs/core/processing_contracts.py", - "sha256": "9b999e9210baf292f8fbf5bb3758c41a95e74401bfb1e1c127bae2dba625b0ee", + "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", "role": "ProcessingContract family, which moved here from the library registry module (G5)." }, { @@ -548,7 +554,7 @@ }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "628a27fea26ed971cfdc4ab22d11df5491f21eb49b6b90397749b46e73e8145b", + "sha256": "b6a7231bc2964d472070731018f6a4935e5f23cf8f238a232045c32244e6545a", "role": "RuntimeValueStore and runtime artifact queries behind the runtime-value route." } ], @@ -557,6 +563,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/pattern_grouping_and_special_outputs.rst against B6's later changes to openhcs/core/config.py, openhcs/core/processing_contracts.py, openhcs/core/runtime_stores.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/pattern_grouping_and_special_outputs.rst against the later B6 changes to openhcs/core/config.py, openhcs/core/processing_contracts.py, openhcs/core/runtime_stores.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/pattern_grouping_and_special_outputs.rst in full and compared it against the changed authorities: core/config.py, pipeline/artifact_planning.py, core/runtime_stores.py, lib_registry/unified_registry.py; checked the group_by/variable_components claims against the ProcessingConfig docstrings, confirmed ProcessingContract still exists (now in core/processing_contracts.py) and that all four target pages exist; replaced unified_registry.py with processing_contracts.py as authority; no prose change needed.", "Re-read docs/source/architecture/pattern_grouping_and_special_outputs.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -580,17 +587,17 @@ "authority": [ { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "d1216010dec1804c51aec9c774f298513638ca016d500fe6328d76a9a9bfb4d6", + "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", "role": "_resolve_pipeline_once ObjectState resolution, AxisCompilationRequest context projection, stage order, worker assignment, and bundle construction." }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "90d75f6287c5f21f93d82960729199261f7906e0f4912b7728527260b9723a66", + "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", "role": "ResolvedPipelineDefinition scope/provenance validation and CompilationSession fields and missing-plan errors." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "f4a851d7de2ea82e1d548f37e6be2511fd20d799580193496a3746e5484906ba", + "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", "role": "CompiledStepPlan typed field inventory and per-step framework-device assignment." }, { @@ -600,32 +607,32 @@ }, { "path": "openhcs/core/callable_contract.py", - "sha256": "2060f856bcb0cc8ef57b92fc17a1e558a676e4033075950b1a4e41dc8656d330", + "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", "role": "CallableContract.from_prepared_callable, process-local resolve_runtime_callable cache, and AXIS/PLATE FunctionStepExecutionScope." }, { "path": "openhcs/core/function_patterns.py", - "sha256": "3dbdad68a4c7c765cc0dae056079929e5e4632f39bdbe2e746bea3acc171562d", + "sha256": "95b9056c6980e4f7943f909aeae8262252c9aa175247a534befc8d4cc459fdd2", "role": "CompiledFunctionInvocation binds its runtime callable at construction and rebuilds it after process transport." }, { "path": "openhcs/processing/backends/lib_registry/registry_service.py", - "sha256": "46ad34dea67bb0f7ba54e1f9f79736654df60ceefe1ef4dd523a226d3222f289", + "sha256": "daef8925338fddf8504a9faa066af42456a13fe4602976bf64be06547a45fcc8", "role": "prepare_in_current_process endpoint catalogue preparation and registered_callable owner projection." }, { "path": "openhcs/core/orchestrator/bundle_execution.py", - "sha256": "7665bd4ab8b9ce73d3dcb4fb9679e836b23bffb81d70541baef2e114ee54b0d9", + "sha256": "f5ba09a2ec7c688b1ca08167ab398cc976a9e2c39b301ae98f3c5fd560a25c85", "role": "Parent ordering of plate-scoped steps, completed-dataset metadata finalization, and viewer settlement." }, { "path": "openhcs/core/steps/function_outputs.py", - "sha256": "074f5c16c64956616d86353106c7d89849d448a2a80813ddb17d93d347faa595", + "sha256": "89405feaa9092caa7ac3699de649eca5516c7bc54a757a24ce881264c21af262", "role": "OpenHCSMetadataTarget declarations, typed projection persistence, and completed-plate deduplication across axes." }, { "path": "tests/unit/test_compilation_session.py", - "sha256": "044f6224de217df92c553e216be50c4ac023d69b2a92b835d59a3a781ed432a2", + "sha256": "381adf3a87729928a225fd1db5a35f8c802340fb2a1e583501884dc5500f6b9d", "role": "Regression proof that axis sessions share one resolved pipeline and own their axis plans." } ], @@ -638,6 +645,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/pipeline_compilation_system.rst against B6's later changes to openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/compiled_step_plan.py, openhcs/core/callable_contract.py, openhcs/core/function_patterns.py, openhcs/processing/backends/lib_registry/registry_service.py, openhcs/core/orchestrator/bundle_execution.py, openhcs/core/steps/function_outputs.py, tests/unit/test_compilation_session.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/pipeline_compilation_system.rst against the later B6 changes to openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/compiled_step_plan.py, openhcs/core/callable_contract.py, openhcs/core/function_patterns.py, openhcs/processing/backends/lib_registry/registry_service.py, openhcs/core/orchestrator/bundle_execution.py, openhcs/core/steps/function_outputs.py, tests/unit/test_compilation_session.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/pipeline_compilation_system.rst in full and compared it against the changed authorities: pipeline/compiler.py, compilation_session.py, compiled_execution.py, context/processing_context.py, test_compiled_execution.py, compiled_plate_execution.py (renamed bundle_execution.py), steps/function_outputs.py, test_compilation_session.py, callable_contract.py, compiled_step_plan.py, framework_device_assignment.py; verified each named class/function exists (FunctionStepTransportAuthority, RegistryService.registered_callable, AxisCompilationRequest, CompiledRuntimeEnvironmentPlan, RuntimeArtifactBatch, OpenHCSMetadataTarget, FunctionStepExecutionScope AXIS/PLATE); rewrote the resolve-once, preparation, session and plan-inventory passages that named deleted StepSnapshot, prepare_compiled_context_callables and source load plans. Dropped the framework-device resolver/assignment files and test_framework_device_resolver.py as authorities since the page states only that bindings stay on step plans.", "Re-read docs/source/architecture/pipeline_compilation_system.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -666,7 +674,7 @@ "authority": [ { "path": "openhcs/authoring/session/session.py", - "sha256": "458d71fe96ab75153977c554fb748e78f1773cd7794b625d7ec233c9b22e4453", + "sha256": "587047b63972df89ec46a01ed2c92ff3ef9577335f58a980297db02094e4e8e0", "role": "Session class, MainThread/DispatcherThread, DatasetAccess and Renderer ports, dataset add/delete/split and work/mutation checks." }, { @@ -676,22 +684,22 @@ }, { "path": "openhcs/authoring/session/views.py", - "sha256": "2963a2c48856704ea8291075316c470ccc8eeed161b5a798181a519a2ceaef98", + "sha256": "8ce6cac27293ac9105f4d2ebc76d10eebeba1ef90e0e8954c0a117910e17e3b3", "role": "SessionView family with DatasetListView and PipelineStepsView." }, { "path": "openhcs/authoring/session/events.py", - "sha256": "4bd9aa333019a770fd64dc4a42617148eb4e4a3081e488310ca860b68c11265c", + "sha256": "6f0b27e3c6d43de6f301de4c9bb80b713b433c7e10c351eda5b35c57d91ece98", "role": "SessionEvent family and event log with subscribe and events-after reads." }, { "path": "openhcs/authoring/session/compile_batch.py", - "sha256": "d539bcf8df15581d23bc69701c5d30d489c3fe5f600f19236b28ec35c1c2d92f", + "sha256": "0cbe5a6b86a0fb5e9b92c4fc45f858c979031ac93594b317b9761c28f5aaf9ab", "role": "Compile batches that check work is allowed and mark datasets compile-pending before connecting." }, { "path": "openhcs/authoring/session/dataset_document.py", - "sha256": "ca695d82a60dbee76ff660ae3710ac5033d2e55f51a99373ed8c0f0dccdbc16c", + "sha256": "305da3c0bf7bb15a0f251a101696b721f9151d8620450a6b3e12413b79218574", "role": "DatasetDocumentScope family and payload scope-id check for selected code documents." }, { @@ -716,12 +724,12 @@ }, { "path": "openhcs/pyqt_gui/session_rendering.py", - "sha256": "8059792f44b1b394ae0e2c68a5f614e55c9a7cc8c59e2e7c0ca7353f09f10cf4", + "sha256": "deee8c93c5bfd41c5ee25abe959938426ed17120275b3b6ee9ccc25b2dd0d690", "role": "QtSessionEventRelay GUI subscriber." }, { "path": "openhcs/mcp/server.py", - "sha256": "927e64594a0fc174759eeb58496fed0ccf95e5d8d29bc248cde9a033efb69fbc", + "sha256": "e1f236011df715da3d17574c0b80cbf50fcae45e3f82491c22fb57e98c6bb662", "role": "MCP binds the session to DispatcherThread over McpMainThreadDispatcher." } ], @@ -731,6 +739,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/plate_manager_services.rst against B6's later changes to openhcs/authoring/session/session.py, openhcs/authoring/session/views.py, openhcs/authoring/session/events.py, openhcs/authoring/session/compile_batch.py, openhcs/authoring/session/dataset_document.py, openhcs/pyqt_gui/session_rendering.py, openhcs/mcp/server.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/plate_manager_services.rst against the later B6 changes to openhcs/authoring/session/session.py, openhcs/authoring/session/views.py, openhcs/authoring/session/events.py, openhcs/authoring/session/compile_batch.py, openhcs/authoring/session/dataset_document.py, openhcs/agent/capabilities.py, openhcs/pyqt_gui/session_rendering.py, openhcs/mcp/server.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/plate_manager_services.rst in full and compared it against the changed authorities: the page itself (rewritten in U1), deleted plate_manager_batch_workflow.py, batch_workflow_components.py, pipeline_object_state_binding.py and plate_manager_state_projection.py, changed plate_manager_workflows.py, pipeline_editor_workflows.py, endpoint_function_catalog_service.py, pyqt_gui/main.py, the Plate Manager and Pipeline Editor widget tests, and demo/synthetic_plate_pipeline.py; verified every named module in openhcs/authoring/session, SessionOperation/SessionView/SessionEvent families, DatasetListView, PipelineStepsView, DatasetDocumentScope, PipelineImporter keyed by suffix, the CellProfiler .cppipe dataset scope, QtSessionEventRelay, openhcs_session_events, DispatcherThread use in the GUI and MCP, compile-artifact ids on submission and compile-pending reservation; replaced all authorities with session-package owners.", "Re-read docs/source/architecture/plate_manager_services.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -836,17 +845,17 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "7b5622b3d82b1e1d672eaedb14e054be81baef417a736adb06bea34fb6708b28", + "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", "role": "ProcessingConfig variable_components, group_by and input_source field semantics." }, { "path": "openhcs/core/processing_contracts.py", - "sha256": "9b999e9210baf292f8fbf5bb3758c41a95e74401bfb1e1c127bae2dba625b0ee", + "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", "role": "ProcessingContract family: Pure2DContract, Pure3DContract, FlexibleContract, VolumetricToSliceContract and collapses_input_plane_axis." }, { "path": "openhcs/core/callable_contract.py", - "sha256": "2060f856bcb0cc8ef57b92fc17a1e558a676e4033075950b1a4e41dc8656d330", + "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", "role": "CallableContract memory types, allowed group_by roles, variable-component requirement, execution scope, runtime adapter, image mode and ImagePayloadConsumption." }, { @@ -866,7 +875,7 @@ }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "f4a851d7de2ea82e1d548f37e6be2511fd20d799580193496a3746e5484906ba", + "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", "role": "CompiledStepPlan storage of resolved callable facts and framework-local device bindings." } ], @@ -878,6 +887,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/processing_semantics.rst against B6's later changes to openhcs/core/config.py, openhcs/core/processing_contracts.py, openhcs/core/callable_contract.py, openhcs/core/compiled_step_plan.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/processing_semantics.rst against the later B6 changes to openhcs/core/config.py, openhcs/core/processing_contracts.py, openhcs/core/callable_contract.py, openhcs/core/compiled_step_plan.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/processing_semantics.rst in full and compared it against the changed authorities: core/config.py, lib_registry/unified_registry.py, callable_contract.py, runtime_slice_projection.py, compiled_step_plan.py, framework_device_assignment.py; verified the ProcessingConfig field semantics, the four ProcessingContract subclasses and keys, contract=Pure2DContract decorator use, CallableContract fields, FunctionStepExecutionScope and ImagePayloadConsumption; rewrote the payload-rank intro, contract names and the runtime-projection paragraph. Replaced unified_registry.py and framework_device_assignment.py with processing_contracts.py, payload_axes.py and runtime_plane_projection.py as authorities.", "Re-read docs/source/architecture/processing_semantics.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -903,17 +913,17 @@ "authority": [ { "path": "openhcs/core/progress/projection.py", - "sha256": "88851cf9e6704ad812208a3fb58e17897363ab46662fe0cb8f0dec6ecc685f2e", + "sha256": "0c931a096020b5622bcfbf92c856c7cb3ead193805b7276f5fb89bfb5613f8cb", "role": "SubjectRuntimeState family (labels, terminal and server-lifecycle policy), SubjectRuntimeIdentity, state_counts and ExecutionRuntimeProjection heartbeat reconciliation." }, { "path": "openhcs/core/progress/runtime_tree.py", - "sha256": "27689de8557c87bcff5be713fa85ec0024171228c80b0cac6e86039114a18865", + "sha256": "016279f4135e2c9b10cb841e57ba0297c4197a195623f8a746ed90ab06533ce5", "role": "Runtime tree node kinds subject, worker, partition, step and compilation." }, { "path": "openhcs/core/progress/types.py", - "sha256": "b5f2ca1a053665a12cca78a6a6676f1ad7e157afafcc5e2161655fddf8bf34e5", + "sha256": "beabbfeb8a6ed727b4bb09f6155de60c4e8b89e71e7ed053bb85574aa1675419", "role": "ProgressEvent, set_progress_queue and the PATTERN_GROUP, AXIS_COMPLETED and SUCCESS phases." }, { @@ -923,7 +933,7 @@ }, { "path": "openhcs/core/debug.py", - "sha256": "4f02b36c242b7db3c4e0fa2060a852e11ded2ab497abde47494cba41ae7afdca", + "sha256": "95d365afefc46ee50da5abb8d5d246b3980feb32b85839f91e874cb7e689caaa", "role": "Debug command, boundary-event, cursor, session, snapshot, artifact identity, snapshot store, warm-replay and paused-worker declarations." }, { @@ -933,7 +943,7 @@ }, { "path": "openhcs/authoring/session/progress.py", - "sha256": "eef42f1c12c9649c36e7d1a86d664f527e6c1aa28868eb5b348d1ebdb5d548bd", + "sha256": "cd34f68461b50506600da7854a3bf57646f2f0ff7288d0627bc40cb852a16825", "role": "ExecutionProgress: dirty-marking registry subscription and interval rebuild on the session main thread, published as session events." }, { @@ -943,7 +953,7 @@ }, { "path": "openhcs/core/orchestrator/bundle_execution.py", - "sha256": "7665bd4ab8b9ce73d3dcb4fb9679e836b23bffb81d70541baef2e114ee54b0d9", + "sha256": "f5ba09a2ec7c688b1ca08167ab398cc976a9e2c39b301ae98f3c5fd560a25c85", "role": "execute_dataset_scoped_steps and the parent ordering before the pipeline SUCCESS event." }, { @@ -965,6 +975,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/progress_runtime_projection_system.rst against B6's later changes to openhcs/core/progress/projection.py, openhcs/core/progress/runtime_tree.py, openhcs/core/progress/types.py, openhcs/core/debug.py, openhcs/authoring/session/progress.py, openhcs/core/orchestrator/bundle_execution.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/progress_runtime_projection_system.rst against the later B6 changes to openhcs/core/progress/projection.py, openhcs/core/progress/runtime_tree.py, openhcs/core/progress/types.py, openhcs/core/debug.py, openhcs/authoring/session/progress.py, openhcs/agent/capabilities.py, openhcs/core/orchestrator/bundle_execution.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/progress_runtime_projection_system.rst in full and compared it against the changed authorities: the page itself (edited in U1), core/progress/projection.py, core/debug.py, zmqruntime progress/projection.py and execution/progress_stream.py, deleted agent/services/execution_session_service.py, agent/dto/execution.py, tests/unit/agent/test_agent_services.py; verified every named class/function (all debug declarations, RuntimeProjectionBuilder, ExecutionProgress, LiveMeasurementProgressPayload, ZMQ progress emitters, phases) and corrected the plate-to-subject vocabulary, tree path, Qt-timer coalescing description and the dataset-scoped step function name.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", @@ -1002,7 +1013,7 @@ }, { "path": "openhcs/core/processing_contracts.py", - "sha256": "9b999e9210baf292f8fbf5bb3758c41a95e74401bfb1e1c127bae2dba625b0ee", + "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", "role": "Pure2DContract and Pure3DContract named in the minimal example and the locality summary." }, { @@ -1012,12 +1023,12 @@ }, { "path": "openhcs/core/dataset_sources/source_bindings_source.py", - "sha256": "7d2ac740ebb081869ad98573032849732f9b64cbbfb4fc7bd9778becc328c78c", + "sha256": "fccda877f39e7b2f99d6cc95143480e5955186264e316e832ca9a69c2ae90b8c", "role": "SourceBindingsSource, selected when a non-empty source-binding declaration is present." }, { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "5d3646e17f7de843a5b36164f7cef7505da7841636423ae4fb4e00d194a87e9c", + "sha256": "3439d96f3fbfcb104a53019dc150049021d9a8132bd27096f1ccf510eefbe37b", "role": "import_cellprofiler_pipeline signature and headless-import rejection of interactive modules." }, { @@ -1032,7 +1043,7 @@ }, { "path": "openhcs/authoring/session/session.py", - "sha256": "458d71fe96ab75153977c554fb748e78f1773cd7794b625d7ec233c9b22e4453", + "sha256": "587047b63972df89ec46a01ed2c92ff3ef9577335f58a980297db02094e4e8e0", "role": "Session shared by the desktop application and MCP for progress, cancellation and results." }, { @@ -1055,6 +1066,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/quick_start.rst against B6's later changes to openhcs/core/processing_contracts.py, openhcs/core/dataset_sources/source_bindings_source.py, openhcs/interop/cellprofiler/pipeline_import.py, openhcs/authoring/session/session.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/quick_start.rst against the later B6 changes to openhcs/core/processing_contracts.py, openhcs/core/source_bindings.py, openhcs/core/dataset_sources/source_bindings_source.py, openhcs/interop/cellprofiler/pipeline_import.py, openhcs/agent/capabilities.py, openhcs/authoring/session/session.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/quick_start.rst in full and compared it against the changed authorities: the page itself (edited in G5/G1), pipeline_document.py, agent/authoring_contexts.py, cellprofiler/pipeline_import.py, test_cellprofiler_pipeline_import.py, test_pipeline_document.py, agent/services/llm_context_service.py, test_progressive_authoring_context.py; verified every MCP tool name in agent/capabilities.py, the analysis-strategy section id in the knowledge manifest, the code-example imports (Microscopy.Channel, stack_percentile_normalize, Lazy*Config, PipelineDocumentCodec) and the MFD preset paths; corrected StepSnapshot, PURE_2D/PURE_3D, SourceBindingsHandler, the missing InputSource import and the progress owner. scripts/validate_docs.py reports no code-block or import finding for the page.", "Reviewed the complete page against PipelineDocumentAuthority and the public pipeline declarations after the compatibility change.", @@ -1082,7 +1094,7 @@ }, { "path": "openhcs/processing/materialization/core.py", - "sha256": "1e408b7b7c4ac406f5bce8a1fda8a8774cb049dbc2a646832b8c4d008ae2b5c3", + "sha256": "744573f1bafa67b7a81be5af425012ada47741292519863e250b79f1c778b265", "role": "ROI output writers, polyline graph-edge writer with its 2D-only rejection, and projected filename resolution." }, { @@ -1092,7 +1104,7 @@ }, { "path": "openhcs/core/artifacts.py", - "sha256": "907551ae0ce4bf42a557554f5df0252b0a88fc9b7c1b7ac19705aade66b6656f", + "sha256": "65eb0ec5c903366243ca8bd5ea07412e8d34bdb81e6015fa76b3c4c59838733f", "role": "ArtifactOutputPlan, ArtifactSpecRelation and ObjectArtifactSubjectBinding." }, { @@ -1102,7 +1114,7 @@ }, { "path": "tests/unit/test_napari_roi_manager_integration.py", - "sha256": "10d7079b3e10088bac81a8814b4d0f77854d20b7818f2c2a9d121589b295ca15", + "sha256": "a4b7dee556f0ceccaa212751e29eb8160d3d0c08c741760715ee8372b0c0e927", "role": "Regression proof for the native Shapes-layer ROI manager seam." } ], @@ -1111,6 +1123,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/roi_system.rst against B6's later changes to openhcs/processing/materialization/core.py, openhcs/core/artifacts.py, tests/unit/test_napari_roi_manager_integration.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/roi_system.rst against the later B6 changes to openhcs/processing/materialization/core.py, openhcs/core/artifacts.py, tests/unit/test_napari_roi_manager_integration.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/roi_system.rst in full and compared it against the changed authorities: napari_roi_manager/widgets/_roi_manager.py (observer release on destroy), processing/materialization/core.py (new ROIOutput/PointROIOutput writers), polystore/roi.py, test_napari_roi_manager_integration.py; verified SpatialGraphROIOptions, SWCOptions, SpatialGraph, ArtifactOutputPlan, ArtifactSpecRelation, ObjectArtifactSubjectBinding and connect_layer exist and that the polyline writer rejects 3D graphs; reordered one misplaced paragraph. Added options.py and core/artifacts.py as authorities for the named declarations.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", @@ -1138,7 +1151,7 @@ }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "628a27fea26ed971cfdc4ab22d11df5491f21eb49b6b90397749b46e73e8145b", + "sha256": "b6a7231bc2964d472070731018f6a4935e5f23cf8f238a232045c32244e6545a", "role": "RuntimeValueStore recording, replacement, RuntimeArtifactQuery resolution, and observation semantics." }, { @@ -1153,7 +1166,7 @@ }, { "path": "openhcs/core/artifacts.py", - "sha256": "907551ae0ce4bf42a557554f5df0252b0a88fc9b7c1b7ac19705aade66b6656f", + "sha256": "65eb0ec5c903366243ca8bd5ea07412e8d34bdb81e6015fa76b3c4c59838733f", "role": "ArtifactType.contextualize_output root (unchanged) and its image, object-label, measurement, spatial-grid and spatial-graph overrides." }, { @@ -1163,7 +1176,7 @@ }, { "path": "openhcs/core/runtime_object_labels.py", - "sha256": "c9581e4f9ba69cf40d2717745d82770b4b9da06ee59cef6bf60a68b72e576d23", + "sha256": "a8ca5cc37f7049bb0ed06f1600afab9943d515e5ccb43dac0fb39f8f1bc7fbf7", "role": "ObjectLabelValue.contextualize_image_output plane-axis rule for label-rendered images." }, { @@ -1173,7 +1186,7 @@ }, { "path": "openhcs/core/callable_contract.py", - "sha256": "2060f856bcb0cc8ef57b92fc17a1e558a676e4033075950b1a4e41dc8656d330", + "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", "role": "CallableContract.resolve_returned_plan_values binding returned positions to exact output plan refs." }, { @@ -1183,12 +1196,12 @@ }, { "path": "tests/unit/test_function_artifact_outputs.py", - "sha256": "b5f103bd9e64f5685c593213edb8bb763b9502f899420eed75bec76b44fcb3d5", + "sha256": "4c2b42cd9f31cc3cd63682039bb691cdd0af5bec3ca5b6f23f2ebea0fdbf0fac", "role": "Proves image-output contextualization and that named artifacts bypass unrelated source-axis projection." }, { "path": "tests/unit/test_runtime_slice_projection.py", - "sha256": "80a39c9c110cfd54a3db6f0715a916601871fea2202cc505e9f7c27f209278d2", + "sha256": "d47cad11b66f1bb3a7e77796bf5cca83599ef8ef4f876ac685f06e002dc9552a", "role": "Proves undeclared value types are rejected and declared slices project without implicit device-to-host conversion." } ], @@ -1200,6 +1213,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/runtime_value_system.rst against B6's later changes to openhcs/core/runtime_stores.py, openhcs/core/artifacts.py, openhcs/core/runtime_object_labels.py, openhcs/core/callable_contract.py, tests/unit/test_function_artifact_outputs.py, tests/unit/test_runtime_slice_projection.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/runtime_value_system.rst against the later B6 changes to openhcs/core/runtime_stores.py, openhcs/core/artifacts.py, openhcs/core/runtime_object_labels.py, openhcs/core/callable_contract.py, tests/unit/test_function_artifact_outputs.py, tests/unit/test_runtime_slice_projection.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/runtime_value_system.rst in full and compared it against the changed authorities: runtime_artifact_values.py, runtime_stores.py, runtime_slice_projection.py, runtime_image_values.py, steps/function_runtime.py and the two test files; verified every named class/function exists, replaced the deleted output matcher, slice-projection strategy registry and function-output context strategies with their current owners (resolve_returned_plan_values, RuntimeSliceProjectableValue, ArtifactType.contextualize_output and contextualize_image_output), and confirmed the persisted-metadata reload fallback in source_workspace_projection.py still holds.", "Re-read docs/source/architecture/runtime_value_system.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -1295,7 +1309,7 @@ }, { "path": "openhcs/core/image_file_serialization.py", - "sha256": "8640772ec0a5b011c525851f0a64204a23e02d4e5c753852d009ac839fea9235", + "sha256": "644a01f8cccdf8e3d663202e3207d8750b4cf2f03f42c5bcc82dcaffb2c3fb5d", "role": "ImageFileFormat registry: suffix matching, host preparation, read/write, source metadata, intensity scale, and declared pixel-band axes." }, { @@ -1309,6 +1323,7 @@ ], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/serialization_boundaries.rst against B6's later changes to openhcs/core/image_file_serialization.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/serialization_boundaries.rst against the later B6 changes to openhcs/core/image_file_serialization.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/serialization_boundaries.rst in full and compared it against the changed authorities: openhcs/serialization/json.py (deleted, L2) and openhcs/core/image_file_serialization.py; checked every to_jsonable dispatch case in python_introspect/jsonable.py, the dataclass_from_mapping decoder, OpenHCS to_jsonable.register users, and ImageFileFormat suffix/prepare/read/write/pixel-semantics members; no prose change was needed.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", @@ -1389,17 +1404,17 @@ }, { "path": "openhcs/core/source_binding_workspace.py", - "sha256": "33911fd2973176817f155eae78260b81085b4464c07907d28a678eb46de42b15", + "sha256": "4d28066a9f73b283fc637662e05fb2632c6fa18af164476c2899fce58cd647fe", "role": "Source-binding workspace projector and materialization of the virtual workspace." }, { "path": "openhcs/core/dataset_sources/source_bindings_source.py", - "sha256": "7d2ac740ebb081869ad98573032849732f9b64cbbfb4fc7bd9778becc328c78c", + "sha256": "fccda877f39e7b2f99d6cc95143480e5955186264e316e832ca9a69c2ae90b8c", "role": "SourceBindingsSource: recursive folder inventory and declared source-binding projection." }, { "path": "openhcs/core/dataset_sources/choice.py", - "sha256": "ece43af801b4b02c302d9ac4bf6e5d07c0d9ebee5027433d12a4b548a19d11f0", + "sha256": "7fb3c7ce701f0d629f288b79090044f43ac8ce1b51ca4f50937534407f226f1a", "role": "AutoDetectedSource falls back to the registered declared-file source only when detection fails and bindings are non-empty." }, { @@ -1418,6 +1433,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/source_model.rst against B6's later changes to openhcs/core/source_binding_workspace.py, openhcs/core/dataset_sources/source_bindings_source.py, openhcs/core/dataset_sources/choice.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/source_model.rst against the later B6 changes to openhcs/core/source_bindings.py, openhcs/core/source_binding_workspace.py, openhcs/core/dataset_sources/source_bindings_source.py, openhcs/core/dataset_sources/choice.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/source_model.rst in full and compared it against the changed authorities: openhcs/core/source_bindings.py, openhcs/core/source_binding_workspace.py, openhcs/microscopes/source_bindings_handler.py (deleted, G4) and PolyStore filemanager.py; checked every named declaration, compiled plan, projection, diagnostic and PolyStore method exists, corrected the arbitrary-folder fallback to AutoDetectedSource/SourceBindingsSource, and confirmed CompiledStepPlan carries no separate source-binding enabled flag.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", @@ -1436,12 +1452,12 @@ "authority": [ { "path": "openhcs/core/callable_contract.py", - "sha256": "2060f856bcb0cc8ef57b92fc17a1e558a676e4033075950b1a4e41dc8656d330", + "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", "role": "CallableContract input/output declarations and runtime-owned parameter projection." }, { "path": "openhcs/core/pipeline/function_contracts.py", - "sha256": "883d22780ab41f122d6ed651c9f59fea540120f8fcf50d30e348ac295bd4fca9", + "sha256": "13bcaf7c7a62654f512d2cc739edccd2deeff35d99970ca493f3e827afbda91a", "role": "special_inputs decorator: records parameter names as runtime-managed callable metadata and excludes them from user parameters; no special_outputs decorator exists." }, { @@ -1451,12 +1467,12 @@ }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "628a27fea26ed971cfdc4ab22d11df5491f21eb49b6b90397749b46e73e8145b", + "sha256": "b6a7231bc2964d472070731018f6a4935e5f23cf8f238a232045c32244e6545a", "role": "RuntimeValueStore recording, replacement, query, and observation during execution." }, { "path": "openhcs/interop/cellprofiler/module_artifact_contracts.py", - "sha256": "c9f1210146b23991dd5433ae2acac43c972b95cc95cf9bd785fac19f9d810928", + "sha256": "982680cd36409ea6d90a07d4e60893b162847f5525ed516891e995877dc32fd1", "role": "CellProfilerModuleArtifactContracts: per-invocation artifact bindings from CellProfiler setting declarations." } ], @@ -1466,6 +1482,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/special_io_system.rst against B6's later changes to openhcs/core/callable_contract.py, openhcs/core/pipeline/function_contracts.py, openhcs/core/runtime_stores.py, openhcs/interop/cellprofiler/module_artifact_contracts.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/special_io_system.rst against the later B6 changes to openhcs/core/callable_contract.py, openhcs/core/pipeline/function_contracts.py, openhcs/core/runtime_stores.py, openhcs/interop/cellprofiler/module_artifact_contracts.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/special_io_system.rst in full, including its unreviewed special_outputs removal, and compared it against the changed authorities: callable_contract.py, pipeline/artifact_planning.py, runtime_stores.py, interop/cellprofiler/module_artifact_contracts.py; confirmed each named class and the special_inputs decorator exist with the roles the page assigns and that no special_outputs declaration remains; fixed the edited sentence.", "Re-read docs/source/architecture/special_io_system.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -1588,7 +1605,7 @@ "authority": [ { "path": "openhcs/core/viewer_streaming_service.py", - "sha256": "b85d6d9b0677bdbd99a657672e19e3a27ef418bfcddfb5550524a83e7598ec4b", + "sha256": "c96eca530f767002cfcc9aca4fe75c93d7a5848a7f42df9e5d336bf6712ed51e", "role": "Compiled output streaming through StreamComponentMessageExtraAuthority and viewer settlement." }, { @@ -1598,22 +1615,22 @@ }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "f4a851d7de2ea82e1d548f37e6be2511fd20d799580193496a3746e5484906ba", + "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", "role": "CompiledStepPlan.producer_identity_for_main_flow and producer_identity_for_artifact." }, { "path": "openhcs/core/runtime_object_labels.py", - "sha256": "c9581e4f9ba69cf40d2717745d82770b4b9da06ee59cef6bf60a68b72e576d23", + "sha256": "a8ca5cc37f7049bb0ed06f1600afab9943d515e5ccb43dac0fb39f8f1bc7fbf7", "role": "ObjectLabelValue.contextualize_image_output plane-axis and provenance rule for label-rendered images." }, { "path": "openhcs/core/orchestrator/bundle_execution.py", - "sha256": "7665bd4ab8b9ce73d3dcb4fb9679e836b23bffb81d70541baef2e114ee54b0d9", + "sha256": "f5ba09a2ec7c688b1ca08167ab398cc976a9e2c39b301ae98f3c5fd560a25c85", "role": "Viewer bootstrap with rollback of the acquired set, settle-then-read-state order, and stopping of execution-owned viewers." }, { "path": "openhcs/runtime/viewer_protocol.py", - "sha256": "f126aaf4f5c0dcc9429b7aae8ce67985260241ec7d2f3d5a948573658cf079a3", + "sha256": "c4bdae4bae6dee412f0d27a9cf75a95db53b3bd95c03cee7d046be16b52fe5b8", "role": "ManagedViewerLifecycleMixin, ViewerProcessPlatform Linux policy, ViewerSettleProgress phases, and the execution-owned EndpointProcessGroup." }, { @@ -1638,7 +1655,7 @@ }, { "path": "openhcs/runtime/zmq_execution_server.py", - "sha256": "0f26779f2d31a4478b9bf9f5d8e4a5b02d8d2e346861f404e7cc35628e64e25e", + "sha256": "e24bdcb3b086cc60197ceb63f29a6f6b639a93cad175218941148a207eb285f2", "role": "Releases execution-session process resources when the execution server stops." }, { @@ -1658,22 +1675,22 @@ }, { "path": "tests/unit/test_viewer_result_element_selection.py", - "sha256": "a7b348fab106da4c89e58ed32225495aa27792132ef7488657f29821dfd6da92", + "sha256": "b1dae580f0291f53be9453df6f6c6df5458046bd3ff947b550f7ddea94b29f96", "role": "Proves bulk isolation validates every route and navigation before any mutation." }, { "path": "tests/unit/test_napari_streaming_handlers.py", - "sha256": "860e234a39f36f143491e6f1edc9515c1d9cde88590759252378f8d23c0bab12", + "sha256": "93244d4d09c5cd6d7a2645fca2482578ad52646f3bb9e6de5ebe505b157fbfea", "role": "Proves one global ROI-member budget and metadata-only shape projection." }, { "path": "tests/unit/test_napari_accepted_work_settlement.py", - "sha256": "86b37ee03cd1f3ca16d912a47365b4b2a0152263e4bac1f4f1ba08b8e55cfb08", + "sha256": "50cde8ceede31c3ef07db0c0aed69865d058617fd77f34d834bb48252a47fcf1", "role": "Tests accepted display failures, settlement reporting and recovery." }, { "path": "tests/unit/test_orchestrator_lane_planning.py", - "sha256": "4a83337fa35c68e4f641760ebbf86e4cc30f6624da55ccebd09be636a4b56bd0", + "sha256": "3d5c2f3662b8d4c330582af0c4a649b8f4d9582e064d9cf3159798c8934a2bbb", "role": "Proves multi-viewer bootstrap rollback after partial acquisition and readiness failure." } ], @@ -1685,6 +1702,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/streaming_boundary_and_wrappers.rst against B6's later changes to openhcs/core/viewer_streaming_service.py, openhcs/core/compiled_step_plan.py, openhcs/core/runtime_object_labels.py, openhcs/core/orchestrator/bundle_execution.py, openhcs/runtime/viewer_protocol.py, openhcs/runtime/zmq_execution_server.py, tests/unit/test_viewer_result_element_selection.py, tests/unit/test_napari_streaming_handlers.py, tests/unit/test_napari_accepted_work_settlement.py, tests/unit/test_orchestrator_lane_planning.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/streaming_boundary_and_wrappers.rst against the later B6 changes to openhcs/core/viewer_streaming_service.py, openhcs/core/compiled_step_plan.py, openhcs/core/runtime_object_labels.py, openhcs/core/orchestrator/bundle_execution.py, openhcs/runtime/viewer_protocol.py, openhcs/runtime/zmq_execution_server.py, tests/unit/test_viewer_result_element_selection.py, tests/unit/test_napari_streaming_handlers.py, tests/unit/test_napari_accepted_work_settlement.py, tests/unit/test_orchestrator_lane_planning.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/streaming_boundary_and_wrappers.rst in full and compared it against the changed authorities: viewer_streaming_service.py, streaming_config_declarations.py, viewer_protocol.py, viewer_controls.py, napari_viewer_server.py, viewer_window_service.py, zmq_execution_server.py, compiled_plate_execution.py (renamed bundle_execution.py, G7), zmqruntime client.py and the two viewer tests; checked every named class exists, corrected the producer-identity and label-image context owners, and confirmed the bootstrap rollback, settle/read-state/stop order, Linux XCB/MIT-SHM/vblank policy and settle phases in current code.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", @@ -1729,7 +1747,7 @@ }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "d1216010dec1804c51aec9c774f298513638ca016d500fe6328d76a9a9bfb4d6", + "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", "role": "PipelineCompiler phases from declaration resolution to framework device assignment." }, { @@ -1743,6 +1761,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/system_integration.rst against B6's later changes to openhcs/core/pipeline/compiler.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/system_integration.rst against the later B6 changes to openhcs/core/pipeline/compiler.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/system_integration.rst in full and compared it against the changed authorities: openhcs/core/pipeline_document.py (PipelineDocumentAuthority renamed PipelineDocumentCodec), openhcs/core/pipeline/compiler.py and openhcs/core/compiled_execution.py; the redirect states no API detail and its target system_overview.rst exists; no prose change was needed.", "Re-read docs/source/architecture/system_integration.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -1769,12 +1788,12 @@ }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "90d75f6287c5f21f93d82960729199261f7906e0f4912b7728527260b9723a66", + "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", "role": "ResolvedPipelineDefinition (resolved steps, scope ids, field provenance) and the per-axis CompilationSession." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "f4a851d7de2ea82e1d548f37e6be2511fd20d799580193496a3746e5484906ba", + "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", "role": "CompiledStepPlan typed fields, including framework-local device assignment." }, { @@ -1789,7 +1808,7 @@ }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "628a27fea26ed971cfdc4ab22d11df5491f21eb49b6b90397749b46e73e8145b", + "sha256": "b6a7231bc2964d472070731018f6a4935e5f23cf8f238a232045c32244e6545a", "role": "RuntimeValueStore recording, replacement, query, and observation semantics." }, { @@ -1825,6 +1844,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/system_overview.rst against B6's later changes to openhcs/core/pipeline/compilation_session.py, openhcs/core/compiled_step_plan.py, openhcs/core/runtime_stores.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/system_overview.rst against the later B6 changes to openhcs/core/pipeline/compilation_session.py, openhcs/core/compiled_step_plan.py, openhcs/core/runtime_stores.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/system_overview.rst in full and compared it against the changed authorities: pipeline_document.py, pipeline/compilation_session.py, compiled_execution.py, runtime_stores.py, compiled_step_plan.py, framework_device_assignment.py, pyproject.toml and the python-introspect/arraybridge/metaclass-registry owners; replaced StepSnapshot with ResolvedPipelineDefinition, restated the session invariant to what CompilationSession enforces, moved slice projection to value families, added the python-introspect JSON codec, and confirmed the CompiledStepPlan fields and bundle environment facts.", "Re-read docs/source/architecture/system_overview.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -1868,7 +1888,7 @@ }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "00ff3bfe7a032bd3de59274697a69d12c9b898177cd8e839f9da3b130becaedf", + "sha256": "01dcb82b5850f3cdb1c300a2aa360ba330e7f15dffb5a24e37504311095090f9", "role": "PyQt desktop main window composition." }, { @@ -1882,6 +1902,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/tui_system.rst against B6's later changes to openhcs/pyqt_gui/main.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/tui_system.rst against the later B6 changes to pyproject.toml, openhcs/pyqt_gui/main.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/tui_system.rst in full and compared it against the changed authorities: pyproject.toml, openhcs/pyqt_gui/main.py and tests/unit/pyqt_gui/test_ui_config_lifecycle.py; confirmed no Textual dependency, package or entry point exists, openhcs-gui still launches openhcs.pyqt_gui, the textual_tui guard test remains and docs/archive still exists; no prose change was needed.", "Re-read docs/source/architecture/tui_system.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -1971,7 +1992,7 @@ }, { "path": "openhcs/pyqt_gui/services/ui_bridge_session_actions.py", - "sha256": "0d06eb67771e683beeb5c2ca93d60d4dfcd132e992371f9261bca15acf415675", + "sha256": "033aee9296f6f1c75f01d1fc3a39e5aeb0e7c13c15a7e4396872d3d014673123", "role": "SessionOperationActionProvider projects session operation classes as UI-bridge actions." }, { @@ -1981,7 +2002,7 @@ }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "00ff3bfe7a032bd3de59274697a69d12c9b898177cd8e839f9da3b130becaedf", + "sha256": "01dcb82b5850f3cdb1c300a2aa360ba330e7f15dffb5a24e37504311095090f9", "role": "OpenHCSMainWindow.set_ui_config consumer reconciliation and rollback, and preview-policy propagation to both managers." }, { @@ -1991,7 +2012,7 @@ }, { "path": "tests/unit/pyqt_gui/test_main_config_propagation.py", - "sha256": "499b05f1249ffc522bf18e97de05d1bf2000ab159daa0685958bd37d9dc62cc4", + "sha256": "303d90b43bee7bd531a1c7664fbf9707a2e7be1696a5147614f93c40e5c7021f", "role": "Verifies live preview policy propagation through the main configuration lifecycle." }, { @@ -2048,6 +2069,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/ui_services_architecture.rst against B6's later changes to openhcs/pyqt_gui/services/ui_bridge_session_actions.py, openhcs/pyqt_gui/main.py, tests/unit/pyqt_gui/test_main_config_propagation.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/ui_services_architecture.rst against the later B6 changes to openhcs/pyqt_gui/services/ui_bridge_session_actions.py, openhcs/pyqt_gui/main.py, tests/unit/pyqt_gui/test_main_config_propagation.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/ui_services_architecture.rst in full and compared it against the changed authorities: runtime/zmq_config.py, services/window_handlers.py, pyqt_gui/main.py, ui_thread_dispatch.py, the two UI config tests, agent/ui_bridge_actions.py (deleted, U1) and services/desktop_restart.py (deleted); verified every named class exists, replaced MainWindowAction with session operations, moved DesktopRestartSession to openhcs/desktop/update.py, corrected the OpenHCSZMQConfig base, and restated Plate Manager as a renderer of Session state; UIConfig fields, set_ui_config ordering and application shutdown were confirmed in code.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", @@ -2114,7 +2136,7 @@ }, { "path": "openhcs/runtime/zmq_execution_server.py", - "sha256": "0f26779f2d31a4478b9bf9f5d8e4a5b02d8d2e346861f404e7cc35628e64e25e", + "sha256": "e24bdcb3b086cc60197ceb63f29a6f6b639a93cad175218941148a207eb285f2", "role": "OpenHCS compiled-execution adapter over the generic execution server." }, { @@ -2164,7 +2186,7 @@ }, { "path": "openhcs/runtime/zmq_execution_client.py", - "sha256": "202619246805f8d2dfbfd5ca5427af34716e424f551ebc800d8516f7b49e07e5", + "sha256": "147fb663d3ea60b1348063440e05b77d54b40634d1ae81cf4fb4f3b5457fbc99", "role": "Spawns the execution server with the ArrayBridge-projected child environment and carries typed execution transport." }, { @@ -2174,12 +2196,12 @@ }, { "path": "openhcs/authoring/session/submission.py", - "sha256": "e04fdb017c3905903885b21007ebb4ceb530ec16267ea26577b3c9609b82e2ce", + "sha256": "710ac218d1db01f231b67bf0fe229dc03fb428bc6655b23671f3a026be91c673", "role": "ExecutionSubmission records each accepted submission with its request and endpoint and follows status to terminal." }, { "path": "openhcs/authoring/session/session.py", - "sha256": "458d71fe96ab75153977c554fb748e78f1773cd7794b625d7ec233c9b22e4453", + "sha256": "587047b63972df89ec46a01ed2c92ff3ef9577335f58a980297db02094e4e8e0", "role": "Session owns the one execution client and keeps FinishedExecution (request, endpoint, ExecutionRecord) per execution." }, { @@ -2189,7 +2211,7 @@ }, { "path": "openhcs/runtime/zmq_orchestrator_environment.py", - "sha256": "04857f72c1e82381d7fab8c2cb1e881ff693803d93310cd39a34df442b023d72", + "sha256": "5301811d7a37fa500f4c511ce9e57ac3620b172331f0ffc76a29e3a63797bf85", "role": "Opens execution_subject_id when declared, otherwise subject_id." }, { @@ -2199,7 +2221,7 @@ }, { "path": "tests/unit/test_zmq_submission_timeout.py", - "sha256": "1d6508d6aecdde924bc9ca4b9aed53cba740af14c44e712b84e1abc048e73da0", + "sha256": "acea5dcb72c820ed8c1f225dc1960dfd615ad3f002ef6aec8e78b75bf8e50b3e", "role": "Proves a total submit deadline prevents execute dispatch after slow startup and preserves the declared connection inactivity timeout." }, { @@ -2219,17 +2241,17 @@ }, { "path": "openhcs/processing/backends/lib_registry/registry_service.py", - "sha256": "46ad34dea67bb0f7ba54e1f9f79736654df60ceefe1ef4dd523a226d3222f289", + "sha256": "daef8925338fddf8504a9faa066af42456a13fe4602976bf64be06547a45fcc8", "role": "Prepares the canonical catalogue in a child environment projected by ArrayBridge declarations." }, { "path": "openhcs/processing/backends/lib_registry/unified_registry.py", - "sha256": "f2bf83e045d40391446ca7629c91f32fccad9a102f0d57cbb0157f780d630cbc", + "sha256": "b1e9253021b9c3ab14a070b7d0f10909f641ba07784c794ad95d2ce36041b1d3", "role": "Processing-contract execution, registry-owned catalogue admission and cache semantics, and external-signature authoring exclusions." }, { "path": "openhcs/processing/backends/lib_registry/openhcs_registry.py", - "sha256": "7872cf8b59a954472871a72614d67bb2658cc2f4ce67e08dba1f8088589ed760", + "sha256": "2a1de7f8af9ade2cba450cbbcc1abd6510a8e3edb58217c6c08fa067bde21de2", "role": "Derives the public native callable catalogue while retaining explicit reconstruction from each callable declaration." }, { @@ -2264,7 +2286,7 @@ }, { "path": "tests/unit/desktop/test_desktop_restart.py", - "sha256": "6181c22b42f7c9aa2f72b0a5a1d66bf9e5c7dfd8cba6af657851f27e5ac02af4", + "sha256": "50425f39e481a6fff384b9732bc0f2cd68f223d3ac980234a9142b6db981b48f", "role": "Proves the themed mismatch decision and state-preserving version-restart transaction." }, { @@ -2323,6 +2345,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/zmq_execution_service_extracted.rst against B6's later changes to openhcs/runtime/zmq_execution_server.py, openhcs/runtime/zmq_execution_client.py, openhcs/authoring/session/submission.py, openhcs/authoring/session/session.py, openhcs/runtime/zmq_orchestrator_environment.py, tests/unit/test_zmq_submission_timeout.py, openhcs/processing/backends/lib_registry/registry_service.py, openhcs/processing/backends/lib_registry/unified_registry.py, openhcs/processing/backends/lib_registry/openhcs_registry.py, tests/unit/desktop/test_desktop_restart.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/zmq_execution_service_extracted.rst against the later B6 changes to openhcs/runtime/zmq_execution_server.py, external/zmqruntime/src/zmqruntime/transport_modes.py, openhcs/runtime/zmq_execution_client.py, openhcs/authoring/session/submission.py, openhcs/authoring/session/session.py, openhcs/runtime/zmq_orchestrator_environment.py, tests/unit/test_zmq_submission_timeout.py, openhcs/processing/backends/lib_registry/registry_service.py, openhcs/processing/backends/lib_registry/unified_registry.py, openhcs/processing/backends/lib_registry/openhcs_registry.py, tests/unit/test_library_registry_discovery.py, tests/unit/desktop/test_desktop_restart.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/zmq_execution_service_extracted.rst in full and compared it against the changed authorities: zmq_execution_server.py, zmq_execution_observation.py, zmq_execution_client.py, compiled_execution.py, function_catalog_preparation.py, the zmqruntime execution/messages/client/transport/timeout owners, tests/unit/test_zmq_submission_timeout.py and agent/services/execution_session_service.py (deleted, U1); verified every named class and the --prepare-capabilities mode exist, rewrote the headless job, cancellation and execution-subject paragraphs for Session/ExecutionSubmission and subject_id/execution_subject_id, moved the submission budget to ExecutionTransportConfig, and confirmed the compatibility check before submit and the benchmark execution-ID check.", "Re-read docs/source/architecture/zmq_execution_service_extracted.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -2401,7 +2424,7 @@ }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "00ff3bfe7a032bd3de59274697a69d12c9b898177cd8e839f9da3b130becaedf", + "sha256": "01dcb82b5850f3cdb1c300a2aa360ba330e7f15dffb5a24e37504311095090f9", "role": "Projects the configured endpoint snapshot into its dedicated right-hand indicator, and routes endpoint termination to the client lifecycle owner." }, { @@ -2431,7 +2454,7 @@ }, { "path": "openhcs/core/progress/runtime_tree.py", - "sha256": "27689de8557c87bcff5be713fa85ec0024171228c80b0cac6e86039114a18865", + "sha256": "016279f4135e2c9b10cb841e57ba0297c4197a195623f8a746ed90ab06533ce5", "role": "RuntimeExecutionTopology, RuntimeTreeProjectionBuilder and the subject/worker/partition/step/compilation node kinds with mean or explicit percent policies." } ], @@ -2441,6 +2464,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/zmq_server_browser_system.rst against B6's later changes to openhcs/pyqt_gui/main.py, openhcs/core/progress/runtime_tree.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/zmq_server_browser_system.rst against the later B6 changes to openhcs/pyqt_gui/main.py, openhcs/core/progress/runtime_tree.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/zmq_server_browser_system.rst in full and compared it against the changed authorities: server_browser/progress_projection.py, server_browser/live_tree_sync.py, pyqt_gui/main.py, tests/unit/pyqt_gui/test_main_zmq_window_routing.py, zmqruntime client.py and services/zmq_client_service.py (moved, U1); verified every named class and module path exists, corrected the progress subscriber owner and the subject/worker/partition/step tree path and aggregation kinds from core/progress/runtime_tree.py, and confirmed the main window still projects the endpoint snapshot into its indicator and manager button states.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", @@ -2495,7 +2519,7 @@ }, { "path": "tests/unit/test_installed_mcp_demo.py", - "sha256": "5f62d65c0add0936583e8e78bf65bf407ed304d5be15b38be4963332a5fe7914", + "sha256": "8babc443f7ec59cebebd3772633dde9ae9c49b125066cd5e7c7d527dc74c2a09", "role": "Proves the installed demo runs the session journey and validates settled viewer state." }, { @@ -2507,6 +2531,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/agent_workflow_validation.rst against B6's later changes to tests/unit/test_installed_mcp_demo.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/development/agent_workflow_validation.rst against the later B6 changes to tests/unit/test_installed_mcp_demo.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/agent_workflow_validation.rst in full and compared it against the changed authorities: openhcs/mcp/client_registration.py, synthetic_plate_service.py, tests/unit/test_installed_mcp_demo.py, openhcs/demo/synthetic_data.py, openhcs/mcp/installed_demo.py; the client skill sync and session-based execute path do not touch the protocol, and the fixture request values, the registry the page names, the time-derived HTD identifier, the ZMQ Server Manager and Live Results windows and the retained baseline record all still match; updated the test's role because its old progress-polling watchdog tests were replaced by the session journey.", "Re-read docs/source/development/agent_workflow_validation.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -2562,42 +2587,42 @@ "authority": [ { "path": "openhcs/core/callable_contract.py", - "sha256": "2060f856bcb0cc8ef57b92fc17a1e558a676e4033075950b1a4e41dc8656d330", + "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", "role": "Declares CallableContract fields (required_axis_roles, allowed_group_by_roles, memory roles), resolve_returned_output, the return partition and the artifact-parameter binding errors quoted by the page." }, { "path": "openhcs/core/artifacts.py", - "sha256": "907551ae0ce4bf42a557554f5df0252b0a88fc9b7c1b7ac19705aade66b6656f", + "sha256": "65eb0ec5c903366243ca8bd5ea07412e8d34bdb81e6015fa76b3c4c59838733f", "role": "Declares ArtifactSpec, MainFlowStackOutputSpec, the artifact types, ObjectMeasurementSubjectRelation and MaterializationSourceIdentityRelation." }, { "path": "openhcs/core/processing_contracts.py", - "sha256": "9b999e9210baf292f8fbf5bb3758c41a95e74401bfb1e1c127bae2dba625b0ee", + "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", "role": "Declares Pure2DContract and Pure3DContract used in the examples." }, { "path": "openhcs/core/pipeline/function_contracts.py", - "sha256": "883d22780ab41f122d6ed651c9f59fea540120f8fcf50d30e348ac295bd4fca9", + "sha256": "13bcaf7c7a62654f512d2cc739edccd2deeff35d99970ca493f3e827afbda91a", "role": "Declares artifact_inputs, artifact_outputs, special_inputs, execution_scope and runtime_bound_parameters." }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "628a27fea26ed971cfdc4ab22d11df5491f21eb49b6b90397749b46e73e8145b", + "sha256": "b6a7231bc2964d472070731018f6a4935e5f23cf8f238a232045c32244e6545a", "role": "Declares RuntimeArtifactBatch.records and StoredRuntimeValue for plate-scope callables." }, { "path": "openhcs/core/orchestrator/bundle_execution.py", - "sha256": "7665bd4ab8b9ce73d3dcb4fb9679e836b23bffb81d70541baef2e114ee54b0d9", + "sha256": "f5ba09a2ec7c688b1ca08167ab398cc976a9e2c39b301ae98f3c5fd560a25c85", "role": "Requires a plate-scope callable to declare exactly one SpecialArtifactType output." }, { "path": "openhcs/processing/materialization/core.py", - "sha256": "1e408b7b7c4ac406f5bce8a1fda8a8774cb049dbc2a646832b8c4d008ae2b5c3", + "sha256": "744573f1bafa67b7a81be5af425012ada47741292519863e250b79f1c778b265", "role": "Point ROI writer: rejects missing fields, duplicate labels, non-finite coordinates and an empty archive." }, { "path": "openhcs/interop/cellprofiler/module_artifact_contracts.py", - "sha256": "c9f1210146b23991dd5433ae2acac43c972b95cc95cf9bd785fac19f9d810928", + "sha256": "982680cd36409ea6d90a07d4e60893b162847f5525ed516891e995877dc32fd1", "role": "Declares CellProfilerModuleArtifactContracts resolving setting-backed specs into the invocation CallableContract." }, { @@ -2612,7 +2637,7 @@ }, { "path": "tests/unit/test_callable_contract.py", - "sha256": "b0ce0cba2e396e53f8cb9e95533ff918788e8c6dd60037a66e0aeda603bc64c2", + "sha256": "cc220f53107a08b0406b6e0dd49896e3e4c3c0af6e527b0ef00d4fdaba8f0b2c", "role": "Proves exact enum validation and contract-derived runtime-owned parameter exclusions." } ], @@ -2624,6 +2649,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/callable_artifact_authoring.rst against B6's later changes to openhcs/core/callable_contract.py, openhcs/core/artifacts.py, openhcs/core/processing_contracts.py, openhcs/core/pipeline/function_contracts.py, openhcs/core/runtime_stores.py, openhcs/core/orchestrator/bundle_execution.py, openhcs/processing/materialization/core.py, openhcs/interop/cellprofiler/module_artifact_contracts.py, tests/unit/test_callable_contract.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/development/callable_artifact_authoring.rst against the later B6 changes to openhcs/core/callable_contract.py, openhcs/core/artifacts.py, openhcs/core/processing_contracts.py, openhcs/core/pipeline/function_contracts.py, openhcs/core/runtime_stores.py, openhcs/core/orchestrator/bundle_execution.py, openhcs/processing/materialization/core.py, openhcs/interop/cellprofiler/module_artifact_contracts.py, tests/unit/test_callable_contract.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/callable_artifact_authoring.rst in full and compared it against the changed authorities: the page, openhcs/core/callable_contract.py, openhcs/core/artifacts.py, module_artifact_contracts.py, tests/unit/test_artifact_spec_collection.py, the ArrayBridge types and decorators, tests/unit/test_callable_contract.py; grepped every API the page names (SettingToKeywordBinding, SelectedPlaneImageOutput, canonical/trailing return specs, RuntimeArtifactBatch.records, PointROIOptions features, the custom-function capability and guide names) and the quoted error texts, confirmed the plate executor's single SpecialArtifactType rule and the empty-Points rejection, corrected the axis-role and processing-contract names, and added processing_contracts.py, function_contracts.py, runtime_stores.py, bundle_execution.py and materialization/core.py as owners while dropping arraybridge/types.py.", "Re-read docs/source/development/callable_artifact_authoring.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -2651,17 +2677,17 @@ "authority": [ { "path": "openhcs/interop/cellprofiler/module_declarations.py", - "sha256": "493efb9c749b10ba54713d0c16ee7118f07ff35afbafc9ff156b7100dbd63b19", + "sha256": "6bbf41c158496a02439be2ee5aef12d7712fb632419c226a64c9259a55438b95", "role": "CellProfiler module registry, catalogue placement, and headless-import eligibility." }, { "path": "openhcs/interop/cellprofiler/module_artifact_contracts.py", - "sha256": "c9f1210146b23991dd5433ae2acac43c972b95cc95cf9bd785fac19f9d810928", + "sha256": "982680cd36409ea6d90a07d4e60893b162847f5525ed516891e995877dc32fd1", "role": "Invocation-specific CellProfiler artifact contract derivation." }, { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "5d3646e17f7de843a5b36164f7cef7505da7841636423ae4fb4e00d194a87e9c", + "sha256": "3439d96f3fbfcb104a53019dc150049021d9a8132bd27096f1ccf510eefbe37b", "role": "Public CellProfiler pipeline lowering and declaration-owned headless-import validation." }, { @@ -2676,7 +2702,7 @@ }, { "path": "openhcs/processing/backends/lib_registry/openhcs_registry.py", - "sha256": "7872cf8b59a954472871a72614d67bb2658cc2f4ce67e08dba1f8088589ed760", + "sha256": "2a1de7f8af9ade2cba450cbbcc1abd6510a8e3edb58217c6c08fa067bde21de2", "role": "Derives the public native callable catalogue while retaining explicit reconstruction from each callable declaration." }, { @@ -2688,6 +2714,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/cellprofiler_module_authoring.rst against B6's later changes to openhcs/interop/cellprofiler/module_declarations.py, openhcs/interop/cellprofiler/module_artifact_contracts.py, openhcs/interop/cellprofiler/pipeline_import.py, openhcs/processing/backends/lib_registry/openhcs_registry.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/development/cellprofiler_module_authoring.rst against the later B6 changes to openhcs/interop/cellprofiler/module_declarations.py, openhcs/interop/cellprofiler/module_artifact_contracts.py, openhcs/interop/cellprofiler/pipeline_import.py, openhcs/processing/backends/lib_registry/openhcs_registry.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/cellprofiler_module_authoring.rst in full and compared it against the changed authorities: module_declarations.py (axis-family imports, MeasurementDialect, lazy registry preparation, private _require_declared_callable with object-identity check, prepare_declared_callable), module_artifact_contracts.py (target_units_for_invocation replaces target_contract_for_invocation), pipeline_import.py, tests/unit/test_cellprofiler_pipeline_import.py and openhcs_registry.py (LazyDiscoveryDict declaration discovery, declared_callable_contract). Confirmed every name the page relies on still exists at its owner: AutoRegisterMeta, CellProfilerModule.__registry__, for_callable_contract (import identity then object identity), declared_function_names and the lazy CellProfilerBackendModule projection, validate_pipeline_import called by pipeline_import before lowering, CellProfilerModuleArtifactContracts, SettingToKeywordBinding.input/output, module_blocks_for_invocation, SourceBindingsConfig, ModuleBlock and CallableContract; no page change was needed.", "Re-read docs/source/development/cellprofiler_module_authoring.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -2712,22 +2739,22 @@ "authority": [ { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "90d75f6287c5f21f93d82960729199261f7906e0f4912b7728527260b9723a66", + "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", "role": "ResolvedPipelineDefinition and CompilationSession fields, one-time artifact admission before axis fan-out, and the missing-plan failure of plan(index)." }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "d1216010dec1804c51aec9c774f298513638ca016d500fe6328d76a9a9bfb4d6", + "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", "role": "Compilation order, declaration resolution, callable warm-up, framework-local device assignment, and bundle construction." }, { "path": "openhcs/core/pipeline/path_planner.py", - "sha256": "bc1380d505dae574c6dc1255afcba59b0513d933716017c05217baabed104d24", + "sha256": "677d106969aae4d818745bff29b6f84e487ec79a044c8eaf26bea1916236fc6c", "role": "Compiled source satisfaction, artifact edge, path, grouping, and plan decisions." }, { "path": "tests/unit/test_compilation_session.py", - "sha256": "044f6224de217df92c553e216be50c4ac023d69b2a92b835d59a3a781ed432a2", + "sha256": "381adf3a87729928a225fd1db5a35f8c802340fb2a1e583501884dc5500f6b9d", "role": "Regression proof that axis sessions share one resolved pipeline definition while owning their own plans." }, { @@ -2737,7 +2764,7 @@ }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "f4a851d7de2ea82e1d548f37e6be2511fd20d799580193496a3746e5484906ba", + "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", "role": "Typed per-step framework-device binding authority." }, { @@ -2761,6 +2788,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/compiler_extension.rst against B6's later changes to openhcs/core/pipeline/compilation_session.py, openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/path_planner.py, tests/unit/test_compilation_session.py, openhcs/core/compiled_step_plan.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/development/compiler_extension.rst against the later B6 changes to openhcs/core/pipeline/compilation_session.py, openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/path_planner.py, tests/unit/test_compilation_session.py, openhcs/core/compiled_step_plan.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/compiler_extension.rst in full and compared it against the changed authorities: compilation_session.py (StepSnapshot and snapshot validation deleted; ResolvedPipelineDefinition and CompilationDatasetScope added), compiler.py, path_planner.py, tests/unit/test_compilation_session.py, arraybridge types.py, compiled_step_plan.py and framework_device_assignment.py. Rewrote the Invariants section for the current session model; confirmed CompiledStepPlan, ArtifactSpecRef, assign_framework_devices, FrameworkDeviceResolver and the absence of a global GPU scheduler.", "Re-read docs/source/development/compiler_extension.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -2861,7 +2889,7 @@ }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "90d75f6287c5f21f93d82960729199261f7906e0f4912b7728527260b9723a66", + "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", "role": "ResolvedPipelineDefinition and CompilationSession consumed by compiler phases." }, { @@ -2871,12 +2899,12 @@ }, { "path": "openhcs/core/callable_contract.py", - "sha256": "2060f856bcb0cc8ef57b92fc17a1e558a676e4033075950b1a4e41dc8656d330", + "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", "role": "OpenHCS callable declaration projection for input, output, and execution frameworks." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "f4a851d7de2ea82e1d548f37e6be2511fd20d799580193496a3746e5484906ba", + "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", "role": "Typed per-step framework-device binding authority." }, { @@ -2886,7 +2914,7 @@ }, { "path": "benchmark/cellprofiler_benchmark_cli.py", - "sha256": "edc32916215a88b62702621dea9b692586572284caf9b781394b5784f62897ee", + "sha256": "4025c591644e29f3df61e476ecae2143690e014336ea9f889aadeff4e692445e", "role": "Owns registered benchmark command declarations and the derived installed parser." }, { @@ -2896,7 +2924,7 @@ }, { "path": "tests/unit/test_benchmark_control.py", - "sha256": "2798cf80df1222c80bb3aa9955b5c20c0c90e415a1795017779d22460f442bf2", + "sha256": "093b584eeb37cbb12089d5af082ad45acd779e9d2bdd5b3feaea08234fc22183", "role": "Proves registered CLI derivation and declaration-derived receipt construction." } ], @@ -2907,6 +2935,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/extension_workflows.rst against B6's later changes to openhcs/core/pipeline/compilation_session.py, openhcs/core/callable_contract.py, openhcs/core/compiled_step_plan.py, benchmark/cellprofiler_benchmark_cli.py, tests/unit/test_benchmark_control.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/development/extension_workflows.rst against the later B6 changes to pyproject.toml, openhcs/core/pipeline/compilation_session.py, openhcs/core/callable_contract.py, openhcs/core/compiled_step_plan.py, benchmark/cellprofiler_benchmark_cli.py, tests/unit/test_benchmark_control.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/extension_workflows.rst in full and compared it against the changed authorities: deleted openhcs/core/registry_strategies.py (strategy mixins moved to metaclass-registry, b221f8491), deleted openhcs/serialization/json.py (python-introspect codec, c0a53a1e0), compilation_session.py (StepSnapshot removed), arraybridge types.py, callable_contract.py, compiled_step_plan.py, framework_device_assignment.py, the benchmark CLI and run-receipt modules, and pyproject.toml. Corrected the StepSnapshot reference and the to_jsonable owner; confirmed ImageFileFormat, BenchmarkCliCommand, ComparisonSuiteRunDeclaration, ComparisonSuiteRunReceipt, MeasuredPipelineRunReceipt, ArtifactKey, CompiledExecutionBundle and MemoryType.import_environment exist at their owners.", "Re-read docs/source/development/extension_workflows.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -3058,7 +3087,7 @@ "authority": [ { "path": "scripts/capture_media_gallery.py", - "sha256": "00f9463a892bdf12845887641053dc72715a8964c90e2aa2e410deb42314efe7", + "sha256": "f4808f79d960a5cfcdb40edd37780120189f1c21e833e51f5a4878af36fec9f1", "role": "Owns the typed capture manifest, host-tool discovery, trusted UI-bridge session, declaration-selected contextual fixture preparation, X11 capture, deterministic derivative construction, validation, and command-line surface." }, { @@ -3085,6 +3114,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/media_gallery_capture.rst against B6's later changes to scripts/capture_media_gallery.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/development/media_gallery_capture.rst against the later B6 changes to scripts/capture_media_gallery.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/media_gallery_capture.rst in full and compared it against the changed authorities: scripts/capture_media_gallery.py (UI-bridge calls now go through UiBridgeService.invoke with typed operation classes, to_jsonable moved to python_introspect), scripts/gallery_catalog.py (contextual actions are SessionOperation classes such as ShowDatasetImages and ShowLiveResults instead of PlateManagerAction values; StepEditorScope moved to openhcs.authoring.session.step_scopes) and tests/unit/test_capture_media_gallery.py (native-frame snapshot requests). None of these internals are named by the page; re-checked every subcommand and option (doctor, catalog, plan, build, validate, capture-still, capture-scenario-still, capture-ui-window-references, capture-ui-context-references with --fixture-root, record-window with --no-mouse), manifest schema_version 2, the 30-second trim limit, the documentation-fixture preparation path, the UI-bridge environment variables, the multi-plate-overview and lazy-inheritance scenario ids, and gallery_catalog --check; all remain accurate.", "Re-read the complete how-to against scripts/capture_media_gallery.py, scripts/gallery_catalog.py, scripts/website_gallery_projection.py, and tests/unit/test_capture_media_gallery.py; every capture, transformation, ownership, validation, and no-overwrite claim matches its current owner.", @@ -3120,7 +3150,7 @@ }, { "path": "openhcs/mcp/dev_client_core.py", - "sha256": "b5b1aea8a5a9352d828efd408408cb0ce867db4c5dc4f91e1fa9fbb7947013b2", + "sha256": "41739b744b0e4287ca7d14cd133371d669e9bff12f4b622313996ae9bb5597da", "role": "MCP dev-client session, selected-workflow argument projection, revision-token terminal-state criteria with idempotent initialization, and transport timeout." }, { @@ -3150,12 +3180,12 @@ }, { "path": "openhcs/agent/services/ui_bridge_service.py", - "sha256": "dc2c4823cc9a611d384bba2785eb6e1b75338f0ec4f731292515bfded2218d63", + "sha256": "c7bdf8b5caec1c21c986cdc2224524839fadfa8764dafcd05125d90c72351899", "role": "Declares the default UI bridge connection spec whose timeout bounds UI development commands." }, { "path": "openhcs/mcp/dev_client_commands/ui.py", - "sha256": "4ee8f3a24dd24ed49a3489d7c195a0c620fc2daa7100ec04ce2c4e9abafd30aa", + "sha256": "be3bc9096e4a711cc6583667596ba25a60428711f2883248e82d5ac66d6dd28b", "role": "Owns declaration-derived UI development commands, including selected-workflow ordering and the typed output-directory projection for window snapshots." }, { @@ -3170,7 +3200,7 @@ }, { "path": "openhcs/authoring/session/session.py", - "sha256": "458d71fe96ab75153977c554fb748e78f1773cd7794b625d7ec233c9b22e4453", + "sha256": "587047b63972df89ec46a01ed2c92ff3ef9577335f58a980297db02094e4e8e0", "role": "Session submission budget and runtime-observation export validation (writable root, no existing path, outcomes require a path)." }, { @@ -3180,7 +3210,7 @@ }, { "path": "tests/unit/agent/test_mcp_server.py", - "sha256": "6a03cf2de856060f39d221127f41189a0a8094c65856cb1c4cfa765210099b67", + "sha256": "4e6f6b1761b2c0316c99984baad6ea2c97e5c76c3f155111b8e498d7358e3175", "role": "Regression proof for selected-surface preflight, stale-row rejection, idempotent initialization, and every declared or malformed bridge-receipt continuation outcome." }, { @@ -3205,12 +3235,12 @@ }, { "path": "openhcs/runtime/zmq_execution_client.py", - "sha256": "202619246805f8d2dfbfd5ca5427af34716e424f551ebc800d8516f7b49e07e5", + "sha256": "147fb663d3ea60b1348063440e05b77d54b40634d1ae81cf4fb4f3b5457fbc99", "role": "Owns the total headless submission budget and the pre-dispatch timeout boundary documented for MCP developers." }, { "path": "tests/unit/test_zmq_submission_timeout.py", - "sha256": "1d6508d6aecdde924bc9ca4b9aed53cba740af14c44e712b84e1abc048e73da0", + "sha256": "acea5dcb72c820ed8c1f225dc1960dfd615ad3f002ef6aec8e78b75bf8e50b3e", "role": "Regression proof that deadline expiry after slow connection prevents execute dispatch." }, { @@ -3225,12 +3255,12 @@ }, { "path": "tests/unit/pyqt_gui/test_ui_agent_bridge.py", - "sha256": "b03744b1fa201f77de26d2e6050422b6422ebce78443c478c5e8c5f55e0a3e24", + "sha256": "de6758859d9743e826369162855327e4a84056cafdd7ddfdc33868d2a3093353", "role": "Proves stale protocol and application versions fail before UI operation dispatch." }, { "path": "openhcs/mcp/server.py", - "sha256": "927e64594a0fc174759eeb58496fed0ccf95e5d8d29bc248cde9a033efb69fbc", + "sha256": "e1f236011df715da3d17574c0b80cbf50fcae45e3f82491c22fb57e98c6bb662", "role": "Owns the mcp_server_stale guard, declaration-derived progress execution, and verbose delayed thread-stack diagnostics across the full invocation." }, { @@ -3246,6 +3276,7 @@ ], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/mcp_development.rst against B6's later changes to openhcs/mcp/dev_client_core.py, openhcs/agent/services/ui_bridge_service.py, openhcs/mcp/dev_client_commands/ui.py, openhcs/authoring/session/session.py, tests/unit/agent/test_mcp_server.py, openhcs/runtime/zmq_execution_client.py, tests/unit/test_zmq_submission_timeout.py, tests/unit/pyqt_gui/test_ui_agent_bridge.py, openhcs/mcp/server.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/development/mcp_development.rst against the later B6 changes to openhcs/mcp/dev_client_core.py, openhcs/agent/services/ui_bridge_service.py, openhcs/mcp/dev_client_commands/ui.py, openhcs/agent/capabilities.py, openhcs/authoring/session/session.py, tests/unit/agent/test_mcp_server.py, openhcs/runtime/zmq_execution_client.py, tests/unit/test_zmq_submission_timeout.py, tests/unit/pyqt_gui/test_ui_agent_bridge.py, openhcs/mcp/server.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/mcp_development.rst in full and compared it against the changed authorities: the page's own U1 update (session-operation tools replacing the job-status paragraph; selected-workflow initialize_datasets), dev_client.py, dev_client_commanding.py, dev_client_core.py, zmq_config.py, knowledge_pipeline.py, agent/dto/ui_bridge.py, agent/dto/viewer.py, dev_client_commands/ui.py and viewer.py, bootstrap.py, stdio.py, zmq_execution_client.py, ui_bridge_transport.py, ui_bridge_server.py, progress_stream.py, agent/dto/execution.py, server.py, the deleted execution_session_service.py and their tests. Confirmed every dev-client command and option in the examples, the full/desktop/core/authoring surface names, initialize_datasets as a UiSelectedPlateWorkflowKind value, the revision-token and idempotent-initialization wait criteria, the 5000 ms UI and viewer control bounds, the 30000 ms submission default and pre-dispatch timeout error, the openhcs_ session tools, event_sequence, and the runtime-observation export checks; replaced the deleted and no-longer-relevant authorities.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", @@ -3315,7 +3346,7 @@ }, { "path": "tests/unit/agent/test_knowledge_base_service.py", - "sha256": "67a326b8473d18cc0596aa275ad2fa1e91261a24d5083ed332bc32335aeefaee", + "sha256": "ce4636580893a8736fd4607c6ed0f431d25c97e0c44e2b95912af3fc36b1d750", "role": "Regression proof for document IDs, search, bounded reads, and section selection." }, { @@ -3327,6 +3358,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/mcp_knowledge_base.rst against B6's later changes to tests/unit/agent/test_knowledge_base_service.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/development/mcp_knowledge_base.rst against the later B6 changes to openhcs/agent/capabilities.py, tests/unit/agent/test_knowledge_base_service.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/mcp_knowledge_base.rst in full and compared it against the changed authorities: knowledge_base_service.py (comparison-manifest parsing moved to openhcs.agent.knowledge_manifest_schema.ComparisonManifestSnapshot; per-case module inventories are now cached by source mtime), knowledge_manifest.py (comparison_manifest_type_for_root) and both test files. Confirmed the generated Python conversion is still recomputed per request and never stored (the new cache holds only module-name inventories), the 12,000 default bound, the four MCP surface names and dev-client commands with --limit, --max-chars and --section-id, the four document ids in mcp_knowledge_base_manifest.json, benchmark/manifests/official30_portable_axis1.json, the first-use capability names, and every cited documentation path; no page change was needed. Added openhcs/agent/capabilities.py for the surface names.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", @@ -3402,7 +3434,7 @@ }, { "path": "tests/unit/test_mcp_release_workflow.py", - "sha256": "d4d67e9591e7652fe00558337627d18bc9547a43af2a946e8ac671b583c90dd4", + "sha256": "5433d76ab87c00f0047e43b65c7a590aab7a8e397a0a1fd7cf6dc3b01e062854", "role": "Proves immutable tag resolution, exact-SHA checkout, publication ordering, recovery, Registry policy, and publication-independent native candidate ownership." }, { @@ -3507,6 +3539,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/mcp_release.rst against B6's later changes to tests/unit/test_mcp_release_workflow.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/development/mcp_release.rst against the later B6 changes to tests/unit/test_mcp_release_workflow.py, scripts/smoke_installed_gui.py, tests/unit/test_smoke_installed_gui.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/mcp_release.rst in full and compared it against the changed authorities: openhcs/__init__.py (version 0.8.7, microscopy axis-family activation; the version-authority claim holds), .github/workflows/publish.yml and tests/unit/test_mcp_release_workflow.py (new build-native-wheels job, sdist built separately, per-wheel validation, Linux x86_64 smoke selection), .github/workflows/integration-tests.yml (parity-scope gate, Official30 parity on Python 3.12 Linux and 3.14 on all three OSes, run_installed_tests.py coverage), scripts/resolve_successful_job_baseline.py (branch filter removed; ancestor rule unchanged), scripts/smoke_installed_gui.py (typed MCP payloads, ShowDatasetCode), tests/installer/test_windows_simple_installer.py, tests/installer/test_installer_signing.py, tests/unit/test_validate_local_release_floors.py and tests/unit/test_ci_package_boundaries.py. Corrected the single-wheel publication and recovery descriptions and the GUI smoke action name; confirmed the Python 3.14 wheel-candidate lane still excludes the Centrosome oracle and the remaining gate, recovery and signing claims.", "Re-read docs/source/development/mcp_release.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -3747,7 +3780,7 @@ "authority": [ { "path": "openhcs/core/debug.py", - "sha256": "4f02b36c242b7db3c4e0fa2060a852e11ded2ab497abde47494cba41ae7afdca", + "sha256": "95d365afefc46ee50da5abb8d5d246b3980feb32b85839f91e874cb7e689caaa", "role": "Nominal debug command, boundary, snapshot, replay, and paused-worker declarations." }, { @@ -3762,13 +3795,14 @@ }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "d1216010dec1804c51aec9c774f298513638ca016d500fe6328d76a9a9bfb4d6", + "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", "role": "Compilation order, declaration resolution, callable warm-up, framework-local device assignment, and bundle construction." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/pipeline_debugging_guide.rst against B6's later changes to openhcs/core/debug.py, openhcs/core/pipeline/compiler.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/development/pipeline_debugging_guide.rst against the later B6 changes to openhcs/core/debug.py, openhcs/core/pipeline/compiler.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/pipeline_debugging_guide.rst in full and compared it against the changed authorities: openhcs/core/debug.py (DebugJsonCodec removed, DebugProgressContext now a declared progress-context payload, plate_id renamed subject_id), openhcs/agent/services/runtime_server_service.py and openhcs/core/pipeline/compiler.py (CompilationPlateScope and plate_path renamed to dataset scope and dataset_root). The page names none of the renamed fields; confirmed openhcs_inspect_pipeline_source_artifact_plan, openhcs_inspect_debug_runtime_values and its runtime-debug-values command, debug_session_id, pipeline_debug_toolbar, FunctionStep.debug_pause, the Run to Pause, Step and Inspect Runtime actions, StepMaterializationConfig, RuntimeValueStore, CompiledStepPlan, CompilationSession and CompiledExecutionBundle still exist with the described roles.", "Re-read docs/source/development/pipeline_debugging_guide.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -3800,7 +3834,7 @@ }, { "path": "openhcs/processing/backends/processors/pyclesperanto_processor.py", - "sha256": "d9c2d68eae058664321622ffa79113ee741cdb22a5eec6045a1ca537a95f622b", + "sha256": "ae7b91db142b898f47831fcb3e44c67758e9e2f8c2d77ada3b347dadbc34af47", "role": "OpenHCS processing callables using the pyclesperanto integration." }, { @@ -3815,17 +3849,17 @@ }, { "path": "openhcs/core/memory/decorators.py", - "sha256": "2740c030304107079f1fcb08c43f40950d0b752cbe10d8412d519b140c7c1b1c", + "sha256": "76f234785f4865257fd19c935e0aace82d1f5586198737fec619ee243b79cbf6", "role": "OpenHCS metadata-preserving wrapper around ArrayBridge memory decorators, accepting a ProcessingContract class as contract." }, { "path": "openhcs/core/processing_contracts.py", - "sha256": "9b999e9210baf292f8fbf5bb3758c41a95e74401bfb1e1c127bae2dba625b0ee", + "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", "role": "ProcessingContract class family including Pure3DContract used in the example." }, { "path": "openhcs/core/callable_contract.py", - "sha256": "2060f856bcb0cc8ef57b92fc17a1e558a676e4033075950b1a4e41dc8656d330", + "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", "role": "Compiler-visible projection of all declared callable memory roles." }, { @@ -3835,7 +3869,7 @@ }, { "path": "openhcs/core/steps/function_runtime.py", - "sha256": "6b523a5b649c8f58e1b7ea242bcfc45c2524166b6ed0211fc51f54b178722820", + "sha256": "b40ebb069565573c340e4fd95ce220474d27f200396ec6d9c797bf716b2c0893", "role": "Runtime conversion and invocation inside the compiled execution-framework device scope." }, { @@ -3845,13 +3879,14 @@ }, { "path": "tests/unit/test_unified_registry_runtime_payloads.py", - "sha256": "935b3b19272e5a2f52b9fce8c60f9331b3b0adef2c24f8a3c20bd3f675c9a731", + "sha256": "76e02cc26b40f5af984d31e874eab1b18d334259fb5cd44bc9fe5945c474b139", "role": "Proves pyclesperanto output buffers and device handles remain runtime-owned while algorithm parameters stay authored." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/pyclesperanto_simple_implementation.rst against B6's later changes to openhcs/processing/backends/processors/pyclesperanto_processor.py, openhcs/core/memory/decorators.py, openhcs/core/processing_contracts.py, openhcs/core/callable_contract.py, openhcs/core/steps/function_runtime.py, tests/unit/test_unified_registry_runtime_payloads.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/development/pyclesperanto_simple_implementation.rst against the later B6 changes to openhcs/processing/backends/processors/pyclesperanto_processor.py, openhcs/core/memory/decorators.py, openhcs/core/processing_contracts.py, openhcs/core/callable_contract.py, openhcs/core/steps/function_runtime.py, tests/unit/test_unified_registry_runtime_payloads.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/pyclesperanto_simple_implementation.rst in full and compared it against the changed authorities: the page itself (example already moved from ProcessingContract.PURE_3D to the Pure3DContract class, G5), pyclesperanto_processor.py (shared operations moved to image_stack_operations), arraybridge types.py and decorators.py, openhcs/core/memory/decorators.py, callable_contract.py, function_runtime.py and test_unified_registry_runtime_payloads.py. Confirmed the example imports (openhcs.core.memory.pyclesperanto, openhcs.core.processing_contracts.Pure3DContract), that the OpenHCS decorator accepts a ProcessingContract subclass as contract, and the three runtime-owned PyclesperantoRegistry parameters; added processing_contracts.py as the owner of the example contract.", "Re-read docs/source/development/pyclesperanto_simple_implementation.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -3940,12 +3975,12 @@ }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "90d75f6287c5f21f93d82960729199261f7906e0f4912b7728527260b9723a66", + "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", "role": "Resolved step/session identity and typed plan assembly invariants." }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "628a27fea26ed971cfdc4ab22d11df5491f21eb49b6b90397749b46e73e8145b", + "sha256": "b6a7231bc2964d472070731018f6a4935e5f23cf8f238a232045c32244e6545a", "role": "RuntimeValueStore recording, replacement, query, and observation semantics." }, { @@ -3959,6 +3994,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/respecting_codebase_architecture.rst against B6's later changes to openhcs/core/pipeline/compilation_session.py, openhcs/core/runtime_stores.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/development/respecting_codebase_architecture.rst against the later B6 changes to openhcs/core/pipeline/compilation_session.py, openhcs/core/runtime_stores.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/respecting_codebase_architecture.rst in full and compared it against the changed authorities: openhcs/core/registry_strategies.py (deleted in b221f8491), openhcs/core/pipeline/compilation_session.py, openhcs/core/runtime_stores.py, tests/unit/test_configuration_mirror_deletion.py; confirmed the three strategy mixins and the registry types named in the search command exist in metaclass-registry, replaced the deleted openhcs.core.registry_strategies reference with metaclass_registry.strategies, and found the typed-plan, RuntimeValueStore and mirror-deletion claims still accurate.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", @@ -3979,7 +4015,7 @@ "authority": [ { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "90d75f6287c5f21f93d82960729199261f7906e0f4912b7728527260b9723a66", + "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", "role": "Declares ResolvedPipelineDefinition and CompilationSession, the resolved declaration view and per-context compiler boundary named in the assembly order." }, { @@ -3989,7 +4025,7 @@ }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "628a27fea26ed971cfdc4ab22d11df5491f21eb49b6b90397749b46e73e8145b", + "sha256": "b6a7231bc2964d472070731018f6a4935e5f23cf8f238a232045c32244e6545a", "role": "RuntimeValueStore recording, replacement, query, and observation semantics." }, { @@ -3999,12 +4035,12 @@ }, { "path": "openhcs/core/callable_contract.py", - "sha256": "2060f856bcb0cc8ef57b92fc17a1e558a676e4033075950b1a4e41dc8656d330", + "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", "role": "Callable ABI and input/output/execution memory-type declaration authority." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "f4a851d7de2ea82e1d548f37e6be2511fd20d799580193496a3746e5484906ba", + "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", "role": "Typed per-step source-binding plan and framework-device decisions." }, { @@ -4020,6 +4056,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/runtime_system_assembly_rules.rst against B6's later changes to openhcs/core/pipeline/compilation_session.py, openhcs/core/runtime_stores.py, openhcs/core/callable_contract.py, openhcs/core/compiled_step_plan.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/development/runtime_system_assembly_rules.rst against the later B6 changes to openhcs/core/pipeline/compilation_session.py, openhcs/core/runtime_stores.py, openhcs/core/callable_contract.py, openhcs/core/compiled_step_plan.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/runtime_system_assembly_rules.rst in full and compared it against the changed authorities: compilation_session.py, compiled_execution.py, runtime_stores.py, runtime_slice_projection.py, callable_contract.py, compiled_step_plan.py, framework_device_assignment.py; replaced the deleted StepSnapshot with ResolvedPipelineDefinition, corrected the CompiledExecutionBundle field summary and the RuntimeSliceProjection description, and confirmed CallableContract memory roles, CellProfilerModuleArtifactContracts, SourceBindingsConfig, CompiledSourceUniversePlan, ArtifactSpecRef and ArtifactKey still exist as described.", "Re-read docs/source/development/runtime_system_assembly_rules.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -4057,12 +4094,12 @@ }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "628a27fea26ed971cfdc4ab22d11df5491f21eb49b6b90397749b46e73e8145b", + "sha256": "b6a7231bc2964d472070731018f6a4935e5f23cf8f238a232045c32244e6545a", "role": "RuntimeValueStore recording, replacement, query, and observation semantics." }, { "path": "tests/unit/test_runtime_value_store.py", - "sha256": "844ff48fced863824fa73bd704d7348b2de79960364d0242eb5e57a9fb2591a7", + "sha256": "4ea189341b761bda993eaa2b6e24b83917568d047f4834d9f44410f7545d1452", "role": "Regression proof for typed runtime recording, replacement, queries, and failure behaviour." } ], @@ -4072,6 +4109,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/runtime_value_extension.rst against B6's later changes to openhcs/core/runtime_stores.py, tests/unit/test_runtime_value_store.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/development/runtime_value_extension.rst against the later B6 changes to openhcs/core/runtime_stores.py, tests/unit/test_runtime_value_store.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/runtime_value_extension.rst in full and compared it against the changed authorities: runtime_artifact_values.py, runtime_stores.py, runtime_slice_projection.py, tests/unit/test_runtime_value_store.py; replaced the deleted RuntimeSliceProjectionStrategy registration step with the RuntimeSliceProjectableValue base classes from runtime_plane_projection.py, removed the MRO-selection and aggregation claims, and confirmed RuntimeValue, ArtifactKey, separate RuntimeArtifactLocation and fail-closed store queries remain as described.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", @@ -4095,7 +4133,7 @@ }, { "path": "openhcs/core/source_binding_selection.py", - "sha256": "74061a9dd7d474c908cad4588f59c2bdd1211006b6f2cfe8a5c150d03a96424b", + "sha256": "f97a4475aa26e465679b89ff9a1fe96690ab7631649d71650608fc7a8aa4bf5a", "role": "SourceUniverseRequest family that selects source universes." }, { @@ -4105,12 +4143,12 @@ }, { "path": "openhcs/core/source_workspace_projection.py", - "sha256": "edf67cc0c9f6d28ed8f6bd3a808c2bab704db212e432a0c9cd1490cc3b1c9856", + "sha256": "d833421b963550aa5fe428c68c0286216437aec97834e4204b0b88def011f98b", "role": "VirtualWorkspaceSourceProjection mapping virtual paths to SourcePixelRef and source metadata." }, { "path": "openhcs/core/source_binding_workspace.py", - "sha256": "33911fd2973176817f155eae78260b81085b4464c07907d28a678eb46de42b15", + "sha256": "4d28066a9f73b283fc637662e05fb2632c6fa18af164476c2899fce58cd647fe", "role": "Compiled virtual source workspace and binding projection." }, { @@ -4125,6 +4163,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/source_binding_extension.rst against B6's later changes to openhcs/core/source_binding_selection.py, openhcs/core/source_workspace_projection.py, openhcs/core/source_binding_workspace.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/development/source_binding_extension.rst against the later B6 changes to openhcs/core/source_bindings.py, openhcs/core/source_binding_selection.py, openhcs/core/source_workspace_projection.py, openhcs/core/source_binding_workspace.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/source_binding_extension.rst in full and compared it against the changed authorities: source_bindings.py (runtime-context and path-lookup classes removed), source_binding_workspace.py, pipeline/path_planner.py, PolyStore filemanager.py; confirmed every named declaration and compiled plan class still exists, rewrote the strategy-selection paragraph around the current SourceUniverseRequest / SourceFilterMatcher families, changed microscope metadata to dataset source metadata, and moved the PolyStore authority to virtual_workspace.py where SourcePixelRef is declared; path_planner.py and filemanager.py were dropped as the page states no fact they own.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", diff --git a/docs/audits/project_surface.json b/docs/audits/project_surface.json index 9c868fe80e..dbd4d2b422 100644 --- a/docs/audits/project_surface.json +++ b/docs/audits/project_surface.json @@ -32,37 +32,37 @@ }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "d1216010dec1804c51aec9c774f298513638ca016d500fe6328d76a9a9bfb4d6", + "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", "role": "Owns compile-time resolution across selected axes, compile progress emission, and construction of the execution bundle." }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "90d75f6287c5f21f93d82960729199261f7906e0f4912b7728527260b9723a66", + "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", "role": "Declares ResolvedPipelineDefinition and CompilationSession named in the compilation diagram." }, { "path": "openhcs/core/progress/types.py", - "sha256": "b5f2ca1a053665a12cca78a6a6676f1ad7e157afafcc5e2161655fddf8bf34e5", + "sha256": "beabbfeb8a6ed727b4bb09f6155de60c4e8b89e71e7ed053bb85574aa1675419", "role": "Declares set_progress_queue, which compilation requires before it emits progress events." }, { "path": "openhcs/core/artifacts.py", - "sha256": "907551ae0ce4bf42a557554f5df0252b0a88fc9b7c1b7ac19705aade66b6656f", + "sha256": "65eb0ec5c903366243ca8bd5ea07412e8d34bdb81e6015fa76b3c4c59838733f", "role": "Declares image, object, measurement, relationship, table, and file artifact semantics used by materialization." }, { "path": "openhcs/core/dataset_sources/source.py", - "sha256": "f736728d2f715b72cd5479fcc79520911b037d3410a42ebfc9febef2e51ffda6", + "sha256": "097fde4d2b74031eacf53a88e79f499417e8c1c97ba0753e8a9df831b307efa6", "role": "Declares the auto-registered DatasetSource family that the ImageXpress, Opera Phenix, Bio-Formats, OMERO and OpenHCS-format sources join." }, { "path": "openhcs/processing/backends/lib_registry/unified_registry.py", - "sha256": "f2bf83e045d40391446ca7629c91f32fccad9a102f0d57cbb0157f780d630cbc", + "sha256": "b1e9253021b9c3ab14a070b7d0f10909f641ba07784c794ad95d2ce36041b1d3", "role": "Owns declaration-based callable discovery, execution memory contracts, processing contracts, and registry caching." }, { "path": "openhcs/core/viewer_streaming_service.py", - "sha256": "b85d6d9b0677bdbd99a657672e19e3a27ef418bfcddfb5550524a83e7598ec4b", + "sha256": "c96eca530f767002cfcc9aca4fe75c93d7a5848a7f42df9e5d336bf6712ed51e", "role": "Owns manual and orchestrated image and ROI streaming through registered viewer declarations." }, { @@ -77,7 +77,7 @@ }, { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "5d3646e17f7de843a5b36164f7cef7505da7841636423ae4fb4e00d194a87e9c", + "sha256": "3439d96f3fbfcb104a53019dc150049021d9a8132bd27096f1ccf510eefbe37b", "role": "Declares import_cellprofiler_pipeline and its returned FunctionStep list and PipelineConfig." }, { @@ -87,7 +87,7 @@ }, { "path": "benchmark/cellprofiler_benchmark_cli.py", - "sha256": "edc32916215a88b62702621dea9b692586572284caf9b781394b5784f62897ee", + "sha256": "4025c591644e29f3df61e476ecae2143690e014336ea9f889aadeff4e692445e", "role": "Declares the openhcs-benchmark subcommands and options, including run-measured through the OpenHCS session." }, { @@ -107,7 +107,7 @@ }, { "path": "tests/unit/test_benchmark_control.py", - "sha256": "2798cf80df1222c80bb3aa9955b5c20c0c90e415a1795017779d22460f442bf2", + "sha256": "093b584eeb37cbb12089d5af082ad45acd779e9d2bdd5b3feaea08234fc22183", "role": "Proves the installed command derivation, typed receipts, MCP projection, path policy, and inspect-only import boundary." } ], @@ -121,6 +121,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared README.md against B6's later changes to openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/progress/types.py, openhcs/core/artifacts.py, openhcs/core/dataset_sources/source.py, openhcs/processing/backends/lib_registry/unified_registry.py, openhcs/core/viewer_streaming_service.py, openhcs/interop/cellprofiler/pipeline_import.py, benchmark/cellprofiler_benchmark_cli.py, tests/unit/test_benchmark_control.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared README.md against the later B6 changes to pyproject.toml, openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/progress/types.py, openhcs/core/artifacts.py, openhcs/core/dataset_sources/source.py, openhcs/processing/backends/lib_registry/unified_registry.py, openhcs/core/viewer_streaming_service.py, openhcs/agent/capabilities.py, openhcs/interop/cellprofiler/pipeline_import.py, tests/integration/test_cellprofiler_official30_zmq.py, benchmark/cellprofiler_benchmark_cli.py, tests/unit/test_benchmark_control.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read README.md in full and compared it against the changed authorities: pyproject.toml, openhcs/desktop_installation.py (moved to openhcs/desktop/installation.py), openhcs/core/pipeline_document.py, openhcs/core/pipeline/compiler.py, openhcs/core/artifacts.py, openhcs/constants/constants.py, unified_registry.py, viewer_streaming_service.py, openhcs/agent/capabilities.py, openhcs/mcp/client_registration.py, pipeline_import.py, the Official30 integration test and the benchmark CLI/receipt/control modules; installed a progress queue in the CellProfiler example and renamed its root to dataset_root, rewrote the run-measured and finalizer paragraphs for the session-backed run, replaced microscope-handler wording with dataset sources, replaced the deleted StepSnapshot with ResolvedPipelineDefinition in the validation card and compilation diagram, and confirmed extras, console scripts, benchmark subcommands/options, MCP tool names, the full profile, client targets, and the eight submodules against current source.", "Re-read README.md in full and challenged its explanation role, audience, capability claims, installation routes, package boundaries, flow, and links against the hash-pinned declaration owners.", diff --git a/docs/audits/user_docs.json b/docs/audits/user_docs.json index a6d881c7c8..ea93833a69 100644 --- a/docs/audits/user_docs.json +++ b/docs/audits/user_docs.json @@ -11,7 +11,7 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "7b5622b3d82b1e1d672eaedb14e054be81baef417a736adb06bea34fb6708b28", + "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", "role": "Declares PipelineConfig, GlobalPipelineConfig, ProcessingConfig and the lazy processing configuration used by the step example." }, { @@ -21,17 +21,17 @@ }, { "path": "openhcs/domains/microscopy/axes.py", - "sha256": "28cc16a296d4580351cd9abec553155e59c4d56a6ea20d52cbd6abb2879b75c6", + "sha256": "531695809259324032e0b88632813323b4fe6a9a176ad6f636aeef8dec332399", "role": "Declares the microscopy axis family members such as Microscopy.Site used as variable components." }, { "path": "openhcs/core/orchestrator/orchestrator.py", - "sha256": "4e55309de74e770b2e37b53f6dbedc19ffac6988ab74cdfe4b3dfc51082e8c1a", + "sha256": "a61658b107f170c192eea505966c268ab6826826f03586bd64562a44800e1ccc", "role": "Declares the PipelineOrchestrator constructor, compile_pipelines, progress_context and execute_compiled_bundle signatures." }, { "path": "openhcs/core/progress/types.py", - "sha256": "b5f2ca1a053665a12cca78a6a6676f1ad7e157afafcc5e2161655fddf8bf34e5", + "sha256": "beabbfeb8a6ed727b4bb09f6155de60c4e8b89e71e7ed053bb85574aa1675419", "role": "Declares ProgressQueue, set_progress_queue and ProgressExecutionContext, which compilation and execution emit through." }, { @@ -41,7 +41,7 @@ }, { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "5d3646e17f7de843a5b36164f7cef7505da7841636423ae4fb4e00d194a87e9c", + "sha256": "3439d96f3fbfcb104a53019dc150049021d9a8132bd27096f1ccf510eefbe37b", "role": "Declares import_cellprofiler_pipeline and its returned steps and PipelineConfig." }, { @@ -56,6 +56,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/api/index.rst against B6's later changes to openhcs/core/config.py, openhcs/domains/microscopy/axes.py, openhcs/core/orchestrator/orchestrator.py, openhcs/core/progress/types.py, openhcs/interop/cellprofiler/pipeline_import.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/api/index.rst against the later B6 changes to openhcs/core/config.py, openhcs/domains/microscopy/axes.py, openhcs/core/orchestrator/orchestrator.py, openhcs/core/progress/types.py, openhcs/interop/cellprofiler/pipeline_import.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/api/index.rst in full and compared it against the changed authorities: openhcs/core/config.py, openhcs/core/orchestrator/orchestrator.py, openhcs/core/compiled_execution.py, tests/unit/test_compiled_execution.py, openhcs/interop/cellprofiler/pipeline_import.py, tests/unit/test_cellprofiler_pipeline_import.py; rewrote the compile/execute example to install a progress queue, call execute_compiled_bundle with orchestrator.progress_context(), and name the constructor argument dataset_root, matching tests/unit/remote_sensing_witness_pipeline.py; confirmed the FunctionStep/ProcessingConfig example, the import_cellprofiler_pipeline signature and the interactive-module rejection against current source.", "Re-read docs/source/api/index.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -81,12 +82,12 @@ "authority": [ { "path": "openhcs/core/artifacts.py", - "sha256": "907551ae0ce4bf42a557554f5df0252b0a88fc9b7c1b7ac19705aade66b6656f", + "sha256": "65eb0ec5c903366243ca8bd5ea07412e8d34bdb81e6015fa76b3c4c59838733f", "role": "Declares the nominal artefact types exchanged by compiled steps and runtime consumers." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "f4a851d7de2ea82e1d548f37e6be2511fd20d799580193496a3746e5484906ba", + "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", "role": "Declares the per-step compiled plan read by runtime code." }, { @@ -96,12 +97,12 @@ }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "90d75f6287c5f21f93d82960729199261f7906e0f4912b7728527260b9723a66", + "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", "role": "Owns compilation-scoped source, artifact and plan state." }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "628a27fea26ed971cfdc4ab22d11df5491f21eb49b6b90397749b46e73e8145b", + "sha256": "b6a7231bc2964d472070731018f6a4935e5f23cf8f238a232045c32244e6545a", "role": "Implements scoped typed runtime-value storage and lookup for compiled execution." }, { @@ -111,22 +112,22 @@ }, { "path": "openhcs/core/callable_contract.py", - "sha256": "2060f856bcb0cc8ef57b92fc17a1e558a676e4033075950b1a4e41dc8656d330", + "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", "role": "Declares the compiler-visible callable metadata defined by the glossary." }, { "path": "openhcs/core/processing_contracts.py", - "sha256": "9b999e9210baf292f8fbf5bb3758c41a95e74401bfb1e1c127bae2dba625b0ee", + "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", "role": "Declares the ProcessingContract family with Pure2DContract and Pure3DContract." }, { "path": "openhcs/core/axes.py", - "sha256": "03e5a61d808a0221507b2be93f3672cc5f4421e53eb72ce4b683309cc673213b", + "sha256": "8d93133639c576050f089fb8aab99937c8982aff07e742fea44f8a12e62bb7b2", "role": "Declares the PartitionAxis role and the axis family that names the partition axis." }, { "path": "openhcs/core/dataset_sources/source.py", - "sha256": "f736728d2f715b72cd5479fcc79520911b037d3410a42ebfc9febef2e51ffda6", + "sha256": "097fde4d2b74031eacf53a88e79f499417e8c1c97ba0753e8a9df831b307efa6", "role": "Declares DatasetSource with detection, parsing, metadata, virtual-workspace and backend behaviour." } ], @@ -137,6 +138,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/appendices/glossary.rst against B6's later changes to openhcs/core/artifacts.py, openhcs/core/compiled_step_plan.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/runtime_stores.py, openhcs/core/callable_contract.py, openhcs/core/processing_contracts.py, openhcs/core/axes.py, openhcs/core/dataset_sources/source.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/appendices/glossary.rst against the later B6 changes to openhcs/core/artifacts.py, openhcs/core/compiled_step_plan.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/runtime_stores.py, openhcs/core/source_bindings.py, openhcs/core/callable_contract.py, openhcs/core/processing_contracts.py, openhcs/core/axes.py, openhcs/core/dataset_sources/source.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/appendices/glossary.rst in full and compared it against the changed authorities: openhcs/core/artifacts.py, compiled_step_plan.py, pipeline/compilation_session.py, runtime_stores.py, source_bindings.py, callable_contract.py, openhcs/processing/backends/lib_registry/unified_registry.py, plus the G1/G4/G5/G7 owners; rewrote the Processing contract, Dataset source (was Microscope handler), Partition axis (was Execution axis), Compiled step plan, Compiled execution bundle and Source workspace projection entries, and confirmed the remaining terms (FunctionStep, CompiledExecutionBundle, SourceUniverse, RuntimeAdapterSpec, CellProfilerModuleArtifactContracts, variable_components, group_by) against current source.", "On 2026-08-27, re-read docs/source/appendices/glossary.rst in full against callable-owned form exclusions and registry-owned non-authorable controls; challenged its reference definitions for accuracy, completeness, owner boundaries, and navigation, and found no prose drift.", @@ -203,7 +205,7 @@ }, { "path": "benchmark/cellprofiler_benchmark_cli.py", - "sha256": "edc32916215a88b62702621dea9b692586572284caf9b781394b5784f62897ee", + "sha256": "4025c591644e29f3df61e476ecae2143690e014336ea9f889aadeff4e692445e", "role": "Owns installed benchmark execution and report commands." }, { @@ -218,13 +220,14 @@ }, { "path": "tests/unit/test_benchmark_control.py", - "sha256": "2798cf80df1222c80bb3aa9955b5c20c0c90e415a1795017779d22460f442bf2", + "sha256": "093b584eeb37cbb12089d5af082ad45acd779e9d2bdd5b3feaea08234fc22183", "role": "Proves receipt validation and the inspect-only MCP projection." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/appendices/research_impact.rst against B6's later changes to benchmark/cellprofiler_benchmark_cli.py, tests/unit/test_benchmark_control.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/appendices/research_impact.rst against the later B6 changes to pyproject.toml, benchmark/cellprofiler_benchmark_cli.py, tests/unit/test_benchmark_control.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/appendices/research_impact.rst in full and compared it against the changed authorities: pyproject.toml, benchmark/cellprofiler_benchmark_cli.py, benchmark/contracts/run_receipt.py, benchmark/control.py, tests/unit/test_benchmark_control.py; the CLI now runs through the OpenHCS session and receipts name a subject_id, but the openhcs-benchmark entry point, typed receipts, read-only inspect/report service and the archive path the page names all still exist, so no claim changed. Dropped benchmark/contracts/pipeline.py, which the page does not rely on.", "On 2026-08-27, re-read docs/source/appendices/research_impact.rst in full after the python-introspect dependency floor advanced to 0.1.12; challenged its explanation boundary and confirmed that retiring unsupported impact claims remains accurate.", @@ -265,7 +268,7 @@ "authority": [ { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "d1216010dec1804c51aec9c774f298513638ca016d500fe6328d76a9a9bfb4d6", + "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", "role": "Compiles pipeline declarations into validated step plans and execution bundles." }, { @@ -275,17 +278,17 @@ }, { "path": "openhcs/core/dataset_sources/source.py", - "sha256": "f736728d2f715b72cd5479fcc79520911b037d3410a42ebfc9febef2e51ffda6", + "sha256": "097fde4d2b74031eacf53a88e79f499417e8c1c97ba0753e8a9df831b307efa6", "role": "DatasetSource family that owns recognized dataset layouts." }, { "path": "openhcs/core/processing_contracts.py", - "sha256": "9b999e9210baf292f8fbf5bb3758c41a95e74401bfb1e1c127bae2dba625b0ee", + "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", "role": "ProcessingContract family deciding per-plane versus whole-stack execution." }, { "path": "openhcs/core/artifacts.py", - "sha256": "907551ae0ce4bf42a557554f5df0252b0a88fc9b7c1b7ac19705aade66b6656f", + "sha256": "65eb0ec5c903366243ca8bd5ea07412e8d34bdb81e6015fa76b3c4c59838733f", "role": "Nominal artifact types exchanged by compiled steps and runtime consumers." }, { @@ -300,6 +303,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/concepts/building_intuition.rst against B6's later changes to openhcs/core/pipeline/compiler.py, openhcs/core/dataset_sources/source.py, openhcs/core/processing_contracts.py, openhcs/core/artifacts.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/concepts/building_intuition.rst against the later B6 changes to openhcs/core/pipeline/compiler.py, openhcs/core/source_bindings.py, openhcs/core/dataset_sources/source.py, openhcs/core/processing_contracts.py, openhcs/core/artifacts.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/concepts/building_intuition.rst in full and compared it against the changed authorities: pipeline/compiler.py, source_bindings.py, artifacts.py, lib_registry/unified_registry.py and external/arraybridge/README.md; confirmed FunctionStep, variable_components, group_by and ProcessingContract exist, that force_disk_output is absent, and renamed the source-integration owner to DatasetSource.", "On 2026-08-27, re-read docs/source/concepts/building_intuition.rst in full against the full-catalogue editor and owner-declared exclusion changes; challenged its explanation flow and ownership model and found no prose drift.", @@ -326,12 +330,12 @@ "authority": [ { "path": "openhcs/core/axes.py", - "sha256": "03e5a61d808a0221507b2be93f3672cc5f4421e53eb72ce4b683309cc673213b", + "sha256": "8d93133639c576050f089fb8aab99937c8982aff07e742fea44f8a12e62bb7b2", "role": "Declares Axis, AxisFamily, the axis roles, Ungrouped, AxisFamily.active() and with_role()." }, { "path": "openhcs/domains/microscopy/axes.py", - "sha256": "28cc16a296d4580351cd9abec553155e59c4d56a6ea20d52cbd6abb2879b75c6", + "sha256": "531695809259324032e0b88632813323b4fe6a9a176ad6f636aeef8dec332399", "role": "Declares the Microscopy family with Site, Channel, ZIndex, Timepoint and the well partition axis." }, { @@ -341,7 +345,7 @@ }, { "path": "tests/unit/test_axis_family_guards.py", - "sha256": "1ee1fd528b9e95758ad787a791088bc56a3889cab92aefac2a1539db53b94a56", + "sha256": "a81f12f9057e2ed7393e6daeb30f1c086f5c867222289a2bb4af3cf297b59c5c", "role": "Proves the component enums are deleted and kernel modules name no microscopy member." } ], @@ -352,6 +356,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/concepts/component_system_concepts.rst against B6's later changes to openhcs/core/axes.py, openhcs/domains/microscopy/axes.py, tests/unit/test_axis_family_guards.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/concepts/component_system_concepts.rst against the later B6 changes to openhcs/core/axes.py, openhcs/domains/microscopy/axes.py, tests/unit/test_axis_family_guards.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/concepts/component_system_concepts.rst in full and compared it against the changed authorities: openhcs/constants/constants.py, openhcs/core/component_group_scope.py, tests/unit/test_configuration_mirror_deletion.py; the page had already been rewritten for the G1 axis family, so checked every named class, role, module path and the ProcessingConfig/LazyProcessingConfig.from_config example against openhcs/core/axes.py, openhcs/domains/microscopy/axes.py, openhcs/core/config.py and the ObjectState lazy factory, corrected the Site sentence, and moved the authorities to the axis owners.", "Re-read docs/source/concepts/component_system_concepts.rst in full against repository base 32ca17c17 and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -378,12 +383,12 @@ }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "d1216010dec1804c51aec9c774f298513638ca016d500fe6328d76a9a9bfb4d6", + "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", "role": "Creates one CompilationSession per partition-axis context and emits the compiled bundle after validation." }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "90d75f6287c5f21f93d82960729199261f7906e0f4912b7728527260b9723a66", + "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", "role": "Declares CompilationSession with the ResolvedPipelineDefinition, source workspace projection and CompiledStepPlan map." }, { @@ -393,22 +398,22 @@ }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "628a27fea26ed971cfdc4ab22d11df5491f21eb49b6b90397749b46e73e8145b", + "sha256": "b6a7231bc2964d472070731018f6a4935e5f23cf8f238a232045c32244e6545a", "role": "Declares RuntimeValueStore keyed by typed artifact keys and execution scope." }, { "path": "openhcs/core/callable_contract.py", - "sha256": "2060f856bcb0cc8ef57b92fc17a1e558a676e4033075950b1a4e41dc8656d330", + "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", "role": "Declares the CallableContract fields: memory types, required axis roles, allowed group-by roles, processing contract, execution scope, runtime adapter and artifact specs." }, { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "5d3646e17f7de843a5b36164f7cef7505da7841636423ae4fb4e00d194a87e9c", + "sha256": "3439d96f3fbfcb104a53019dc150049021d9a8132bd27096f1ccf510eefbe37b", "role": "Lowers parsed .cppipe modules into ordinary source bindings and FunctionStep declarations." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "f4a851d7de2ea82e1d548f37e6be2511fd20d799580193496a3746e5484906ba", + "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", "role": "Declares the source, artifact, memory, materialisation, streaming and worker fields of CompiledStepPlan." } ], @@ -419,6 +424,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/concepts/core_model.rst against B6's later changes to openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/runtime_stores.py, openhcs/core/callable_contract.py, openhcs/interop/cellprofiler/pipeline_import.py, openhcs/core/compiled_step_plan.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/concepts/core_model.rst against the later B6 changes to openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/runtime_stores.py, openhcs/core/callable_contract.py, openhcs/interop/cellprofiler/pipeline_import.py, openhcs/core/compiled_step_plan.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/concepts/core_model.rst in full and compared it against the changed authorities: openhcs/core/pipeline_document.py, openhcs/core/pipeline/compiler.py, openhcs/core/compiled_execution.py, openhcs/core/runtime_stores.py, openhcs/interop/cellprofiler/pipeline_import.py, openhcs/core/compiled_step_plan.py; grepped every named class (CompilationSession, CompiledExecutionBundle, RuntimeValueStore, CellProfilerModule, SourceBindingsConfig, ArtifactSpec, CompiledStepPlan, ProcessingContract), replaced the deleted StepSnapshot, corrected session scope and contract axis wording, and added compilation_session.py and callable_contract.py as owners.", "On 2026-08-27, re-read docs/source/concepts/core_model.rst in full against the explicit CellProfiler import-validation hook; challenged its declaration-to-runtime explanation and confirmed that its supported lowering model remains accurate.", @@ -443,12 +449,12 @@ "authority": [ { "path": "openhcs/core/axes.py", - "sha256": "03e5a61d808a0221507b2be93f3672cc5f4421e53eb72ce4b683309cc673213b", + "sha256": "8d93133639c576050f089fb8aab99937c8982aff07e742fea44f8a12e62bb7b2", "role": "Declares AxisFamily, its single PartitionAxis and the variable axes a step may stack along." }, { "path": "openhcs/core/processing_contracts.py", - "sha256": "9b999e9210baf292f8fbf5bb3758c41a95e74401bfb1e1c127bae2dba625b0ee", + "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", "role": "Declares the Pure2DContract, Pure3DContract, FlexibleContract and VolumetricToSliceContract subclasses of ProcessingContract." }, { @@ -463,22 +469,22 @@ }, { "path": "openhcs/processing/backends/cellprofiler/watershed.py", - "sha256": "4b8b3e9360c090ba9bbb1584ddb7749e3ff3ecb62d609ea66eb27d8a4b2762d2", + "sha256": "41c8cb9cb139f8b4041a25e1a7a2d19aa85cf0427422fc87b9551c8141069625", "role": "Declares Watershed with FullStackExecution so it consumes the complete payload." }, { "path": "openhcs/processing/backends/cellprofiler/primary_objects.py", - "sha256": "825d2ed2dfe7f0248af0f4b840a80a3f053227fab4cf06896311e89839dfd6d8", + "sha256": "b5b8a9b98dd9166b34a8f5cebf884e8af6da85f02fe878ef1227d6656d31ae4c", "role": "Declares IdentifyPrimaryObjects with Pure2DContract." }, { "path": "openhcs/processing/backends/cellprofiler/secondary.py", - "sha256": "b0773976512067ec267df254111d305dbdec9694e8359c380cf4e28d327bc805", + "sha256": "fc9d9b1bbc3baff7216c6abe3a7135ffddfc9f7e1212d4a391fa8534038a354c", "role": "Declares IdentifySecondaryObjects and IdentifyTertiaryObjects with Pure2DContract." }, { "path": "tests/unit/test_documented_dimensionality.py", - "sha256": "0b83e470095a2c75996c58d26f8ed4b908ee7830b9546bfb7cc40bbd9cc2de52", + "sha256": "45762e74bf5cecbbada509f378ddbfa47c20fa6a9f5bd668d05d003e59bb38e1", "role": "Binds the published dimensionality boundary to the current callable declarations." } ], @@ -489,6 +495,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/concepts/data_dimensions.rst against B6's later changes to openhcs/core/axes.py, openhcs/core/processing_contracts.py, openhcs/processing/backends/cellprofiler/watershed.py, openhcs/processing/backends/cellprofiler/primary_objects.py, openhcs/processing/backends/cellprofiler/secondary.py, tests/unit/test_documented_dimensionality.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/concepts/data_dimensions.rst against the later B6 changes to openhcs/core/axes.py, openhcs/core/processing_contracts.py, openhcs/processing/backends/cellprofiler/watershed.py, openhcs/processing/backends/cellprofiler/primary_objects.py, openhcs/processing/backends/cellprofiler/secondary.py, tests/unit/test_documented_dimensionality.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/concepts/data_dimensions.rst in full and compared it against the changed authorities: openhcs/constants/constants.py, openhcs/core/component_group_scope.py, openhcs/processing/backends/lib_registry/unified_registry.py, openhcs/core/runtime_object_label_domains.py, the three CellProfiler backends, tests/unit/test_callable_contract.py and tests/unit/test_documented_dimensionality.py; confirmed Watershed still declares FullStackExecution and the Identify modules Pure2DContract, the PAYLOAD/PLANE scopes are unchanged, and corrected the processing-contract names and partition-axis wording.", "On 2026-08-27, re-read docs/source/concepts/data_dimensions.rst in full against registry form exclusions, explicit secondary-backend default selection, and expanded contract tests; challenged its dimensional explanation and found no semantic drift.", @@ -512,12 +519,12 @@ "authority": [ { "path": "openhcs/core/dataset_sources/source.py", - "sha256": "f736728d2f715b72cd5479fcc79520911b037d3410a42ebfc9febef2e51ffda6", + "sha256": "097fde4d2b74031eacf53a88e79f499417e8c1c97ba0753e8a9df831b307efa6", "role": "Declares the registered DatasetSource family that projects each microscope format into axis values, backends and metadata." }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "d1216010dec1804c51aec9c774f298513638ca016d500fe6328d76a9a9bfb4d6", + "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", "role": "Resolves inputs, axes, locality, memory conversion and artifact dependencies before emitting compiled plans." }, { @@ -532,6 +539,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/concepts/domain_fundamentals.rst against B6's later changes to openhcs/core/dataset_sources/source.py, openhcs/core/pipeline/compiler.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/concepts/domain_fundamentals.rst against the later B6 changes to openhcs/core/dataset_sources/source.py, openhcs/core/pipeline/compiler.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/concepts/domain_fundamentals.rst in full and compared it against the changed authorities: openhcs/microscopes/microscope_interfaces.py (deleted in G4), openhcs/core/pipeline/compiler.py, openhcs/core/runtime_artifact_values.py; replaced the deleted source interface with openhcs/core/dataset_sources/source.py, updated the dataset-source wording, and confirmed the compilation and typed-result claims still hold.", "Re-read docs/source/concepts/domain_fundamentals.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -554,7 +562,7 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "7b5622b3d82b1e1d672eaedb14e054be81baef417a736adb06bea34fb6708b28", + "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", "role": "Declares DtypeConfig as a DtypeConversionConfig subclass with NATIVE_OUTPUT as its default_dtype_conversion." }, { @@ -564,7 +572,7 @@ }, { "path": "openhcs/core/memory/decorators.py", - "sha256": "2740c030304107079f1fcb08c43f40950d0b752cbe10d8412d519b140c7c1b1c", + "sha256": "76f234785f4865257fd19c935e0aace82d1f5586198737fec619ee243b79cbf6", "role": "Exports the OpenHCS memory decorators that delegate dtype conversion to ArrayBridge." }, { @@ -574,13 +582,14 @@ }, { "path": "tests/unit/test_callable_contract.py", - "sha256": "b0ce0cba2e396e53f8cb9e95533ff918788e8c6dd60037a66e0aeda603bc64c2", + "sha256": "cc220f53107a08b0406b6e0dd49896e3e4c3c0af6e527b0ef00d4fdaba8f0b2c", "role": "Proves dtype_config is the overridable runtime-bound parameter on a decorated callable's contract." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/concepts/dtype_config_system.rst against B6's later changes to openhcs/core/config.py, openhcs/core/memory/decorators.py, tests/unit/test_callable_contract.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/concepts/dtype_config_system.rst against the later B6 changes to openhcs/core/config.py, openhcs/core/memory/decorators.py, tests/unit/test_callable_contract.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/concepts/dtype_config_system.rst in full and compared it against the changed authorities: openhcs/core/config.py, openhcs/core/memory/decorators.py, external/arraybridge/src/arraybridge/decorators.py, tests/unit/test_callable_contract.py; checked the DtypeConversion members, the NATIVE_OUTPUT default, the import paths in the example, the step-level dtype_config field (now owned by openhcs/core/steps/abstract.py, added as an authority) and the runtime-bound dtype_config contract parameter; no claim changed.", "On 2026-08-27, re-read docs/source/concepts/dtype_config_system.rst in full after callable contracts began projecting runtime-owned inputs into generic form exclusions; challenged the dtype ownership boundary and found no prose drift.", @@ -605,32 +614,32 @@ "authority": [ { "path": "openhcs/core/callable_contract.py", - "sha256": "2060f856bcb0cc8ef57b92fc17a1e558a676e4033075950b1a4e41dc8656d330", + "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", "role": "Declares the CallableContract fields the page lists: memory types, artifact specs, runtime-bound parameters, required_axis_roles, allowed_group_by_roles, processing contract, runtime adapter, execution scope and prepare hook." }, { "path": "openhcs/core/processing_contracts.py", - "sha256": "9b999e9210baf292f8fbf5bb3758c41a95e74401bfb1e1c127bae2dba625b0ee", + "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", "role": "Declares the ProcessingContract class family selecting per-slice, full-stack, flexible or volumetric-to-slice execution." }, { "path": "openhcs/processing/backends/lib_registry/unified_registry.py", - "sha256": "f2bf83e045d40391446ca7629c91f32fccad9a102f0d57cbb0157f780d630cbc", + "sha256": "b1e9253021b9c3ab14a070b7d0f10909f641ba07784c794ad95d2ce36041b1d3", "role": "Derives catalogue metadata and form exclusions from library registry declarations." }, { "path": "openhcs/processing/backends/lib_registry/registry_service.py", - "sha256": "46ad34dea67bb0f7ba54e1f9f79736654df60ceefe1ef4dd523a226d3222f289", + "sha256": "daef8925338fddf8504a9faa066af42456a13fe4602976bf64be06547a45fcc8", "role": "Owns backend discovery, availability filtering and the process catalogue projection." }, { "path": "openhcs/processing/backends/lib_registry/openhcs_registry.py", - "sha256": "7872cf8b59a954472871a72614d67bb2658cc2f4ce67e08dba1f8088589ed760", + "sha256": "2a1de7f8af9ade2cba450cbbcc1abd6510a8e3edb58217c6c08fa067bde21de2", "role": "Derives the public native callable catalogue and excludes private decorated helpers." }, { "path": "openhcs/agent/services/function_catalog_service.py", - "sha256": "5e6357e805f6506e0d26d675fb319bf377227baabc4fc735b04f88dad32bd340", + "sha256": "211c81f3747dcbc5cad2bde36191bf7ac90fab007f44c960c729ac33c2737df1", "role": "Serves catalogue search and callable detail to MCP and GUI authoring from the execution endpoint." }, { @@ -650,7 +659,7 @@ }, { "path": "tests/unit/test_unified_registry_runtime_payloads.py", - "sha256": "935b3b19272e5a2f52b9fce8c60f9331b3b0adef2c24f8a3c20bd3f675c9a731", + "sha256": "76e02cc26b40f5af984d31e874eab1b18d334259fb5cd44bc9fe5945c474b139", "role": "Proves external library runtime controls stay out of authored function parameters." }, { @@ -665,6 +674,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/concepts/function_library.rst against B6's later changes to openhcs/core/callable_contract.py, openhcs/core/processing_contracts.py, openhcs/processing/backends/lib_registry/unified_registry.py, openhcs/processing/backends/lib_registry/registry_service.py, openhcs/processing/backends/lib_registry/openhcs_registry.py, openhcs/agent/services/function_catalog_service.py, tests/unit/test_unified_registry_runtime_payloads.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/concepts/function_library.rst against the later B6 changes to openhcs/core/callable_contract.py, openhcs/core/processing_contracts.py, openhcs/processing/backends/lib_registry/unified_registry.py, openhcs/processing/backends/lib_registry/registry_service.py, openhcs/processing/backends/lib_registry/openhcs_registry.py, openhcs/agent/services/function_catalog_service.py, tests/unit/test_library_registry_discovery.py, tests/unit/test_unified_registry_runtime_payloads.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/concepts/function_library.rst in full and compared it against the changed authorities: unified_registry.py, registry_service.py, openhcs/core/callable_contract.py, function_catalog_service.py, openhcs_registry.py, the ArrayBridge types and decorators, and the three registry/contract tests; the G5 processing-contract move and catalogue-service signature/doc projection changes leave the page's catalogue, exclusion and framework claims intact, corrected the contract field names, and added openhcs/core/processing_contracts.py as an owner.", "Re-read docs/source/concepts/function_library.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -699,12 +709,12 @@ }, { "path": "openhcs/core/function_patterns.py", - "sha256": "3dbdad68a4c7c765cc0dae056079929e5e4632f39bdbe2e746bea3acc171562d", + "sha256": "95b9056c6980e4f7943f909aeae8262252c9aa175247a534befc8d4cc459fdd2", "role": "Normalises callable, chain and dictionary patterns into CompiledFunctionPattern." }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "d1216010dec1804c51aec9c774f298513638ca016d500fe6328d76a9a9bfb4d6", + "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", "role": "Validates per-invocation callable contracts and dictionary keys against compiled grouping." }, { @@ -718,6 +728,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/concepts/function_patterns.rst against B6's later changes to openhcs/core/function_patterns.py, openhcs/core/pipeline/compiler.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/concepts/function_patterns.rst against the later B6 changes to openhcs/core/function_patterns.py, openhcs/core/pipeline/compiler.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/concepts/function_patterns.rst in full and compared it against the changed authorities: the page itself (example already moved to Microscopy axes), openhcs/core/pipeline/compiler.py, tests/unit/test_function_patterns.py; confirmed the example imports, CompiledFunctionPattern, and the grouping-versus-stacking claims, corrected the axis-role wording, and added openhcs/core/function_patterns.py as the pattern owner.", "Re-read docs/source/concepts/function_patterns.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -746,18 +757,19 @@ }, { "path": "openhcs/core/config.py", - "sha256": "7b5622b3d82b1e1d672eaedb14e054be81baef417a736adb06bea34fb6708b28", + "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", "role": "Declares the processing, dtype and storage configuration the processing-and-storage pages explain." }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "d1216010dec1804c51aec9c774f298513638ca016d500fe6328d76a9a9bfb4d6", + "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", "role": "Compiles declarations into plans, the step the core-model and dimensions pages describe." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/concepts/index.rst against B6's later changes to openhcs/core/config.py, openhcs/core/pipeline/compiler.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/concepts/index.rst against the later B6 changes to openhcs/core/config.py, openhcs/core/pipeline/compiler.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/concepts/index.rst in full and compared it against the changed authorities: openhcs/core/pipeline_document.py, openhcs/core/config.py, openhcs/core/pipeline/compiler.py; the page is a reading-order toctree whose every entry still exists under docs/source/concepts and makes no product claim affected by the G1/G4/G5 renames; no change.", "Re-read docs/source/concepts/index.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -806,7 +818,7 @@ }, { "path": "openhcs/microscopes/omero.py", - "sha256": "11f486719055f05744b25076274361e332520af02cfa96bc37ec02d49e783149", + "sha256": "a3ab5a257490f729e96ba413c3f11d12f10f68dae44a728206f81b850325da31", "role": "Declares OMEROFilenameParser projecting PolyStore OMERO addresses into the parser family." }, { @@ -858,6 +870,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/concepts/module_structure.rst against B6's later changes to openhcs/microscopes/omero.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/concepts/module_structure.rst against the later B6 changes to pyproject.toml, openhcs/microscopes/omero.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/concepts/module_structure.rst in full and compared it against the changed authorities: pyproject.toml, external/python-introspect/README.md, openhcs/core/config.py, unified_registry.py, pipeline_import.py, openhcs/runtime/viewer_protocol.py, external/zmqruntime client, ArrayBridge README and types, metaclass-registry discovery, benchmark CLI, run_receipt.py and control.py; listed the current openhcs/ top-level packages, confirmed the OMERO address and parser classes, the eight foundation distributions and the benchmark MCP capabilities (now session-backed), and added the domains and authoring areas.", "On 2026-08-27, re-read docs/source/concepts/module_structure.rst in full against the python-introspect 0.1.12 floor, declaration-owned catalogue inputs, and CellProfiler import validation; challenged the owner map and found no prose drift.", @@ -913,12 +926,12 @@ }, { "path": "openhcs/core/config.py", - "sha256": "7b5622b3d82b1e1d672eaedb14e054be81baef417a736adb06bea34fb6708b28", + "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", "role": "Declares ProcessingConfig (variable_components, group_by, input_source) and PipelineConfig." }, { "path": "openhcs/core/orchestrator/orchestrator.py", - "sha256": "4e55309de74e770b2e37b53f6dbedc19ffac6988ab74cdfe4b3dfc51082e8c1a", + "sha256": "a61658b107f170c192eea505966c268ab6826826f03586bd64562a44800e1ccc", "role": "Declares PipelineOrchestrator.compile_pipelines over selected partition values, returning CompiledExecutionBundle." }, { @@ -928,12 +941,12 @@ }, { "path": "openhcs/authoring/session/compile_batch.py", - "sha256": "d539bcf8df15581d23bc69701c5d30d489c3fe5f600f19236b28ec35c1c2d92f", + "sha256": "0cbe5a6b86a0fb5e9b92c4fc45f858c979031ac93594b317b9761c28f5aaf9ab", "role": "Compiles the selected datasets through the session before execution for GUI and MCP clients." }, { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "5d3646e17f7de843a5b36164f7cef7505da7841636423ae4fb4e00d194a87e9c", + "sha256": "3439d96f3fbfcb104a53019dc150049021d9a8132bd27096f1ccf510eefbe37b", "role": "import_cellprofiler_pipeline returns the lowered steps with a populated PipelineConfig." }, { @@ -949,6 +962,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/concepts/pipelines_and_steps.rst against B6's later changes to openhcs/core/config.py, openhcs/core/orchestrator/orchestrator.py, openhcs/authoring/session/compile_batch.py, openhcs/interop/cellprofiler/pipeline_import.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/concepts/pipelines_and_steps.rst against the later B6 changes to openhcs/core/config.py, openhcs/core/orchestrator/orchestrator.py, openhcs/authoring/session/compile_batch.py, openhcs/interop/cellprofiler/pipeline_import.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/concepts/pipelines_and_steps.rst in full and compared it against the changed authorities: the page (example already on Microscopy axes), openhcs/core/pipeline_document.py, openhcs/core/config.py, openhcs/core/pipeline/compiler.py, tests/unit/test_pipeline_document.py; checked the example imports, compile_pipelines signature, CompiledExecutionBundle fields, the session compile path and the CellProfiler import return, and corrected the compile section and axis wording.", "Re-read docs/source/concepts/pipelines_and_steps.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -972,22 +986,22 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "7b5622b3d82b1e1d672eaedb14e054be81baef417a736adb06bea34fb6708b28", + "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", "role": "Declares VFSConfig read/intermediate/materialization backends and the fieldless OpenHCS ZarrConfig and TiffConfig subtypes." }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "628a27fea26ed971cfdc4ab22d11df5491f21eb49b6b90397749b46e73e8145b", + "sha256": "b6a7231bc2964d472070731018f6a4935e5f23cf8f238a232045c32244e6545a", "role": "Declares RuntimeValueStore with explicit replace and retained observation history." }, { "path": "openhcs/core/steps/function_artifact_materialization.py", - "sha256": "4cbd772568286146214828cc83127866b33e3707aacf79a3daa5678773fe6700", + "sha256": "2375f8e784d4d0a5116882ae588c0681940696289b252ec6bd4b53e503c8b20b", "role": "Materialises planned function outputs and forms filenames without changing artifact metadata." }, { "path": "openhcs/core/pipeline/function_contracts.py", - "sha256": "883d22780ab41f122d6ed651c9f59fea540120f8fcf50d30e348ac295bd4fca9", + "sha256": "13bcaf7c7a62654f512d2cc739edccd2deeff35d99970ca493f3e827afbda91a", "role": "Declares the @artifact_outputs decorator named by the page." }, { @@ -1012,7 +1026,7 @@ }, { "path": "tests/unit/test_runtime_value_store.py", - "sha256": "844ff48fced863824fa73bd704d7348b2de79960364d0242eb5e57a9fb2591a7", + "sha256": "4ea189341b761bda993eaa2b6e24b83917568d047f4834d9f44410f7545d1452", "role": "Proves scoped typed runtime-value insertion, replacement and lookup." }, { @@ -1026,6 +1040,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/concepts/storage_system.rst against B6's later changes to openhcs/core/config.py, openhcs/core/runtime_stores.py, openhcs/core/steps/function_artifact_materialization.py, openhcs/core/pipeline/function_contracts.py, tests/unit/test_runtime_value_store.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/concepts/storage_system.rst against the later B6 changes to openhcs/core/config.py, openhcs/core/runtime_stores.py, openhcs/core/steps/function_artifact_materialization.py, openhcs/core/pipeline/function_contracts.py, tests/unit/test_runtime_value_store.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/concepts/storage_system.rst in full and compared it against the changed authorities: the page, openhcs/core/config.py, openhcs/core/runtime_stores.py, function_artifact_materialization.py, PolyStore base/config/disk, ArrayBridge decorators, and the runtime-store and TIFF compression tests; confirmed the three VFSConfig backends and AUTO default, the fieldless ZarrConfig/TiffConfig subtypes, the TIFF NONE/DEFLATE codecs, the @artifact_outputs decorator and RuntimeValueStore replace/observation behaviour, and added openhcs/core/pipeline/function_contracts.py as the decorator owner.", "Re-read docs/source/concepts/storage_system.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -1129,7 +1144,7 @@ }, { "path": "tests/unit/test_installed_mcp_demo.py", - "sha256": "5f62d65c0add0936583e8e78bf65bf407ed304d5be15b38be4963332a5fe7914", + "sha256": "8babc443f7ec59cebebd3772633dde9ae9c49b125066cd5e7c7d527dc74c2a09", "role": "Proves the documented installed MCP verification flow." } ], @@ -1139,6 +1154,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/getting_started/getting_started.rst against B6's later changes to tests/unit/test_installed_mcp_demo.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/getting_started/getting_started.rst against the later B6 changes to pyproject.toml, tests/unit/test_installed_mcp_demo.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/getting_started/getting_started.rst in full and compared it against the changed authorities: pyproject.toml, openhcs/cli.py, openhcs/mcp/installed_demo.py, render_installer_contract.py, both native installers, smoke_installed_desktop.py, installer tests, integration-tests.yml, test_installed_mcp_demo.py, and the desktop modules moved into openhcs/desktop by 73a2aece2; confirmed the extras, entry points (openhcs, openhcs-mcp-demo), --log-level handling in pyqt_gui/launch.py, the demo's generated plate / neurite preset / nonzero Napari payload check / output_root report, the desktop profile extras and pip/uv override suppression, and sibling-environment update with a preserved previous environment; removed the false Python 3.13 limit on cellprofiler-compat and replaced the moved authorities.", "On 2026-08-27, re-read docs/source/getting_started/getting_started.rst in full after the python-introspect dependency floor advanced to 0.1.12; challenged the installation how-to flow and confirmed that the supported procedure is unchanged.", @@ -1220,7 +1236,7 @@ "authority": [ { "path": "openhcs/pyqt_gui/main.py", - "sha256": "00ff3bfe7a032bd3de59274697a69d12c9b898177cd8e839f9da3b130becaedf", + "sha256": "01dcb82b5850f3cdb1c300a2aa360ba330e7f15dffb5a24e37504311095090f9", "role": "Dock layout (system monitor above, Datasets and ZMQ server manager left, Pipeline Editor right), View/Tools/Help menus, and preview-policy application to both panes." }, { @@ -1230,17 +1246,17 @@ }, { "path": "openhcs/pyqt_gui/widgets/pipeline_editor.py", - "sha256": "abca0b56094615407fd81df039761ef9d48ddb57ddd19f31e87e804974f25b22", + "sha256": "8e418631fd9ba4f8ac3d6b257f9af127f9e8711281608516ea2617d7855d6f36", "role": "Implements step editing, pipeline compilation controls, and declaration feedback in the main UI." }, { "path": "openhcs/agent/services/ui_bridge_service.py", - "sha256": "dc2c4823cc9a611d384bba2785eb6e1b75338f0ec4f731292515bfded2218d63", + "sha256": "c7bdf8b5caec1c21c986cdc2224524839fadfa8764dafcd05125d90c72351899", "role": "Owns agent-to-UI declaration inspection and mutation, including code and form synchronisation." }, { "path": "tests/unit/pyqt_gui/test_ui_agent_bridge.py", - "sha256": "b03744b1fa201f77de26d2e6050422b6422ebce78443c478c5e8c5f55e0a3e24", + "sha256": "de6758859d9743e826369162855327e4a84056cafdd7ddfdc33868d2a3093353", "role": "Proves bidirectional UI-bridge inspection and mutation of live declarations, plus truthful rejection of unsupported window-navigation targets." }, { @@ -1250,12 +1266,12 @@ }, { "path": "tests/unit/pyqt_gui/test_pipeline_code_document_transactionality.py", - "sha256": "139725010d983c7e8a98731b4e462339c4188b8d9c44383677191810ad7bf24f", + "sha256": "0bb9b43d867a4e26f13b51531a2b7a847c6c3685841f8e002e209200a37366b6", "role": "Proves Pipeline Editor code validates and applies pipeline_steps without an explicit pipeline_config." }, { "path": "openhcs/pyqt_gui/widgets/shared/services/pipeline_editor_workflows.py", - "sha256": "d77abd94b6db02b55486326d0bbd6c565dbea8dfedb41cd0ee03dcba6a4540b1", + "sha256": "c704ef7e6c5f8a5745c741e923bc4e620a321fdce069e1cdcdf521c0176e4a4f", "role": "Applies Pipeline Editor documents and commits the reconciled editor state graph." }, { @@ -1265,7 +1281,7 @@ }, { "path": "tests/unit/pyqt_gui/test_pipeline_editor_widget.py", - "sha256": "10afc4ce49c91731a8dc7e52726c6a77198747a6ec3a765e7e88ceaf40967e3a", + "sha256": "0ce88271aa7f5f55a8fc4daec8279d732844e03dfd437c11e6e6265ca196b4a0", "role": "Proves Pipeline Editor code apply leaves the reconciled root, step, and nested function states saved." }, { @@ -1275,7 +1291,7 @@ }, { "path": "openhcs/pyqt_gui/services/ui_bridge_windows.py", - "sha256": "7a10b37aba9c5ea5cdbe46a4c9703b345e83a64554d6abacbfa1ee41cd3c4020", + "sha256": "e76298f78e3cbd949e8426a2b42cd6768cc54770a5f45c9dd6c9f788b257785e", "role": "Projects embedded manager navigation through the generic declared driver." }, { @@ -1290,7 +1306,7 @@ }, { "path": "tests/unit/pyqt_gui/test_main_config_propagation.py", - "sha256": "499b05f1249ffc522bf18e97de05d1bf2000ab159daa0685958bd37d9dc62cc4", + "sha256": "303d90b43bee7bd531a1c7664fbf9707a2e7be1696a5147614f93c40e5c7021f", "role": "Verifies live preview policy propagation through the main configuration lifecycle." }, { @@ -1325,12 +1341,12 @@ }, { "path": "openhcs/authoring/session/views.py", - "sha256": "2963a2c48856704ea8291075316c470ccc8eeed161b5a798181a519a2ceaef98", + "sha256": "8ce6cac27293ac9105f4d2ebc76d10eebeba1ef90e0e8954c0a117910e17e3b3", "role": "DatasetListView and PipelineStepsView titles and the operations each pane binds." }, { "path": "openhcs/authoring/session/operations/datasets.py", - "sha256": "43d9edfed59abe6b88e242d39f1ddc8f5078559e04b3875da1da07b993295993", + "sha256": "208274effc2e790775d90528211fa4f640eb01f7ca3fdd9c927c3bf71e39b632", "role": "Compile/Connect/Connecting slot, Run availability (compiled, endpoint accepting requests) and Stop while a batch runs." }, { @@ -1345,6 +1361,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/basic_interface.rst against B6's later changes to openhcs/pyqt_gui/main.py, openhcs/pyqt_gui/widgets/pipeline_editor.py, openhcs/agent/services/ui_bridge_service.py, tests/unit/pyqt_gui/test_ui_agent_bridge.py, tests/unit/pyqt_gui/test_pipeline_code_document_transactionality.py, openhcs/pyqt_gui/widgets/shared/services/pipeline_editor_workflows.py, tests/unit/pyqt_gui/test_pipeline_editor_widget.py, openhcs/pyqt_gui/services/ui_bridge_windows.py, tests/unit/pyqt_gui/test_main_config_propagation.py, openhcs/authoring/session/views.py, openhcs/authoring/session/operations/datasets.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/basic_interface.rst against the later B6 changes to openhcs/pyqt_gui/main.py, openhcs/pyqt_gui/widgets/pipeline_editor.py, openhcs/agent/services/ui_bridge_service.py, tests/unit/pyqt_gui/test_ui_agent_bridge.py, tests/unit/pyqt_gui/test_pipeline_code_document_transactionality.py, openhcs/pyqt_gui/widgets/shared/services/pipeline_editor_workflows.py, tests/unit/pyqt_gui/test_pipeline_editor_widget.py, openhcs/pyqt_gui/services/ui_bridge_windows.py, tests/unit/pyqt_gui/test_main_config_propagation.py, openhcs/authoring/session/views.py, openhcs/authoring/session/operations/datasets.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/basic_interface.rst in full and compared it against the changed authorities: pyqt_gui/main.py, plate_manager.py, pipeline_editor.py, ui_bridge_service.py, ui_bridge_windows.py, pipeline_document.py, pipeline_editor_workflows.py, endpoint_function_catalog_service.py, gallery_catalog.py and their tests; confirmed the dock layout, menus (Image Browser, Log Viewer, Global Configuration, Custom Functions, Analysis Consolidation), Connect/Connecting/Compile/Run/Stop rules in the session operations, selected/all dataset code documents, steps-only pipeline documents, background catalogue preparation, the ui-function-selector gallery scenario and unsupported-navigation reporting; renamed the Plate Manager pane to its current Datasets title and pointed agent step 6 at session events.", "Re-read docs/source/guide_for_biologists/basic_interface.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -1389,7 +1406,7 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "7b5622b3d82b1e1d672eaedb14e054be81baef417a736adb06bea34fb6708b28", + "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", "role": "Declares the supported OpenHCS configuration dataclasses, nested scopes, defaults, and field help metadata." }, { @@ -1409,12 +1426,12 @@ }, { "path": "openhcs/pyqt_gui/services/ui_bridge_object_state.py", - "sha256": "81b5ffbbd19952f85430962cd1bb61918ac3abe4dd9ec843b8f67e4590ffd06b", + "sha256": "e548b367507b3fb6830ce9ecb844e3b756243af96fe38e0d35de5da0b974c1fd", "role": "Projects declaration-derived accepted enum inputs through live ObjectState field help." }, { "path": "tests/unit/pyqt_gui/test_ui_agent_bridge.py", - "sha256": "b03744b1fa201f77de26d2e6050422b6422ebce78443c478c5e8c5f55e0a3e24", + "sha256": "de6758859d9743e826369162855327e4a84056cafdd7ddfdc33868d2a3093353", "role": "Proves live field help reports the exact accepted inputs for enum-backed configuration fields." }, { @@ -1455,6 +1472,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/configuration_reference.rst against B6's later changes to openhcs/core/config.py, openhcs/pyqt_gui/services/ui_bridge_object_state.py, tests/unit/pyqt_gui/test_ui_agent_bridge.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/configuration_reference.rst against the later B6 changes to openhcs/core/config.py, openhcs/pyqt_gui/services/ui_bridge_object_state.py, tests/unit/pyqt_gui/test_ui_agent_bridge.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/configuration_reference.rst in full and compared it against the changed authorities: the page's own unreviewed WellFilter-to-PartitionFilter rename, openhcs/core/config.py, ObjectState object_state.py, config_service.py, python-introspect annotation_types.py, ui_bridge_object_state.py, pipeline_document.py, gallery_catalog.py and their tests; imported GlobalPipelineConfig/PipelineConfig and AbstractStep to confirm every named config field and the step-overridable families, checked the PartitionFilterConfig inheritance tree, InputSource values, main_flow_axis_persistence_enabled, openhcs_describe_config_schema and the four gallery scenarios; replaced the step-snapshot wording, corrected the consolidation field summary and realigned the diagram.", "Re-read docs/source/guide_for_biologists/configuration_reference.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -1496,7 +1514,7 @@ }, { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "5d3646e17f7de843a5b36164f7cef7505da7841636423ae4fb4e00d194a87e9c", + "sha256": "3439d96f3fbfcb104a53019dc150049021d9a8132bd27096f1ccf510eefbe37b", "role": "Imports CellProfiler modules into ordinary OpenHCS pipeline declarations through the interop boundary." }, { @@ -1511,7 +1529,7 @@ }, { "path": "benchmark/cellprofiler_benchmark_cli.py", - "sha256": "edc32916215a88b62702621dea9b692586572284caf9b781394b5784f62897ee", + "sha256": "4025c591644e29f3df61e476ecae2143690e014336ea9f889aadeff4e692445e", "role": "Owns packaged comparison execution and reporting commands." }, { @@ -1526,7 +1544,7 @@ }, { "path": "tests/unit/test_benchmark_control.py", - "sha256": "2798cf80df1222c80bb3aa9955b5c20c0c90e415a1795017779d22460f442bf2", + "sha256": "093b584eeb37cbb12089d5af082ad45acd779e9d2bdd5b3feaea08234fc22183", "role": "Proves exact receipt, progress, artifact, path-policy, and MCP projection behavior." }, { @@ -1536,12 +1554,12 @@ }, { "path": "openhcs/interop/cellprofiler/module_artifact_declarations.py", - "sha256": "286638babca312c85ee0b8319fd44ba86be5868a3c1a20b74b1b48a0bfdbc652", + "sha256": "f70b52114fb5d5e2c2fc49cd9a2b46986bde48ce02d82f1a34a67f072ca2ab20", "role": "Raises the import error for CellProfiler modules that require interactive desktop input." }, { "path": "openhcs/authoring/session/views.py", - "sha256": "2963a2c48856704ea8291075316c470ccc8eeed161b5a798181a519a2ceaef98", + "sha256": "8ce6cac27293ac9105f4d2ebc76d10eebeba1ef90e0e8954c0a117910e17e3b3", "role": "DatasetListView title for the desktop pane that UI-owned work populates." } ], @@ -1550,6 +1568,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/domain_expert_onboarding.rst against B6's later changes to openhcs/interop/cellprofiler/pipeline_import.py, benchmark/cellprofiler_benchmark_cli.py, tests/unit/test_benchmark_control.py, openhcs/interop/cellprofiler/module_artifact_declarations.py, openhcs/authoring/session/views.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/domain_expert_onboarding.rst against the later B6 changes to openhcs/agent/capabilities.py, openhcs/interop/cellprofiler/pipeline_import.py, benchmark/cellprofiler_benchmark_cli.py, tests/unit/test_benchmark_control.py, openhcs/interop/cellprofiler/module_artifact_declarations.py, openhcs/authoring/session/views.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/domain_expert_onboarding.rst in full and compared it against the changed authorities: agent/capabilities.py, knowledge_base_service.py, interop/cellprofiler/pipeline_import.py, test_architecture_projection_service.py, test_cellprofiler_pipeline_import.py, benchmark/cellprofiler_benchmark_cli.py, benchmark/contracts/run_receipt.py, benchmark/control.py, test_benchmark_control.py; confirmed openhcs_explain_architecture with topic cellprofiler_translation, the 50,000-character knowledge document bound, the interactive-module import rejection, the full MCP profile, the openhcs-benchmark entry point and read-only receipt/artifact inspection in benchmark/control.py; renamed the Plate Manager reference to the Datasets pane.", "Reviewed docs/source/guide_for_biologists/domain_expert_onboarding.rst in full; the recorded findings and disposition are the resulting page-specific editorial decision.", @@ -1586,23 +1605,24 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "7b5622b3d82b1e1d672eaedb14e054be81baef417a736adb06bea34fb6708b28", + "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", "role": "ProcessingConfig declares variable_components as the stack-axis authority and group_by as post-assembly partitioning." }, { "path": "openhcs/core/processing_contracts.py", - "sha256": "9b999e9210baf292f8fbf5bb3758c41a95e74401bfb1e1c127bae2dba625b0ee", + "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", "role": "ProcessingContract family (for example Pure2DContract and WholeStackInputContract) declares whether a callable runs per plane or on the whole stack." }, { "path": "openhcs/core/axes.py", - "sha256": "03e5a61d808a0221507b2be93f3672cc5f4421e53eb72ce4b683309cc673213b", + "sha256": "8d93133639c576050f089fb8aab99937c8982aff07e742fea44f8a12e62bb7b2", "role": "AxisFamily and Axis declarations that variable_components and group_by take their values from." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/glossary.rst against B6's later changes to openhcs/core/config.py, openhcs/core/processing_contracts.py, openhcs/core/axes.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/glossary.rst against the later B6 changes to openhcs/core/config.py, openhcs/core/processing_contracts.py, openhcs/core/axes.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/glossary.rst in full and compared it against the changed authorities: openhcs/constants/constants.py (component enums removed by the axis-family refactor), lib_registry/unified_registry.py, openhcs/core/config.py; confirmed variable_components and group_by docstrings in ProcessingConfig and that ProcessingContract still exists as the per-plane versus whole-stack family in processing_contracts.py; the three distinctions remain accurate. Replaced constants.py and unified_registry.py, which no longer own these facts, with processing_contracts.py and core/axes.py.", "On 2026-08-27, re-read docs/source/guide_for_biologists/glossary.rst in full against the declaration-owned catalogue input changes; challenged its concise reference boundary and confirmed that the three axis-processing distinctions remain accurate.", @@ -1632,22 +1652,22 @@ }, { "path": "openhcs/core/source_workspace_projection.py", - "sha256": "edf67cc0c9f6d28ed8f6bd3a808c2bab704db212e432a0c9cd1490cc3b1c9856", + "sha256": "d833421b963550aa5fe428c68c0286216437aec97834e4204b0b88def011f98b", "role": "Projects source bindings and persisted semantic metadata into the concrete workspace identities compiled for execution." }, { "path": "tests/unit/test_source_bindings.py", - "sha256": "926b6a8480791ac9f431caaaf039f0cc3a9ff0f32ca0480c5eefdbc866637eb8", + "sha256": "3d2bca343054aa8aa2d66ce1136ad23d1805d2c800134ee5682fac91feabaadd", "role": "Proves source-binding validation and projection into concrete workspace identities." }, { "path": "openhcs/core/source_binding_workspace.py", - "sha256": "33911fd2973176817f155eae78260b81085b4464c07907d28a678eb46de42b15", + "sha256": "4d28066a9f73b283fc637662e05fb2632c6fa18af164476c2899fce58cd647fe", "role": "Selects raw candidates and projects binding-owned component identity." }, { "path": "tests/unit/test_source_binding_workspace.py", - "sha256": "45e3c25cd271cc6d0e06116d8c4e41473a1ca2ae09a96fd8ff3161aee675f2b9", + "sha256": "2d3582e109bfd2541a1c217cea4209674e7603dd0443f06733644d35ed80a55a", "role": "Proves raw selector coordinates are retained as original metadata while the projected address uses semantic component identity." }, { @@ -1682,7 +1702,7 @@ }, { "path": "tests/unit/test_plane_store_sources.py", - "sha256": "ed55a14cfaefc1938a8c73f88df0e5689f9c219e8133553304368cfe59ded00b", + "sha256": "f0f795651e4e3d48da30a6352088234d3ac8dcdce3a90c93e86c420491d02c1f", "role": "Proves both physical NGFF formats retain axis cardinalities, pixels, source identities, bindings, materialisation, and reinitialisation through OpenHCS." }, { @@ -1702,12 +1722,12 @@ }, { "path": "openhcs/core/artifacts.py", - "sha256": "907551ae0ce4bf42a557554f5df0252b0a88fc9b7c1b7ac19705aade66b6656f", + "sha256": "65eb0ec5c903366243ca8bd5ea07412e8d34bdb81e6015fa76b3c4c59838733f", "role": "Owns InputImageSetContextSourceRelation and its exact input-image and context-bearing target constraints." }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "628a27fea26ed971cfdc4ab22d11df5491f21eb49b6b90397749b46e73e8145b", + "sha256": "b6a7231bc2964d472070731018f6a4935e5f23cf8f238a232045c32244e6545a", "role": "RuntimeArtifactInput resolves exact artifact addresses and checks execution context through the declared image-set relation; it does not select arbitrary same-shape labels." }, { @@ -1717,32 +1737,32 @@ }, { "path": "openhcs/interop/cellprofiler/module_artifact_declarations.py", - "sha256": "286638babca312c85ee0b8319fd44ba86be5868a3c1a20b74b1b48a0bfdbc652", + "sha256": "f70b52114fb5d5e2c2fc49cd9a2b46986bde48ce02d82f1a34a67f072ca2ab20", "role": "The existing object-measurement declaration ancestor derives label input image-set relations from its selected image specs." }, { "path": "openhcs/core/dataset_sources/source.py", - "sha256": "f736728d2f715b72cd5479fcc79520911b037d3410a42ebfc9febef2e51ffda6", + "sha256": "097fde4d2b74031eacf53a88e79f499417e8c1c97ba0753e8a9df831b307efa6", "role": "DatasetSource family with format-specific, broad structured-store and declared-file selection roles and their detection order." }, { "path": "openhcs/core/dataset_sources/source_bindings_source.py", - "sha256": "7d2ac740ebb081869ad98573032849732f9b64cbbfb4fc7bd9778becc328c78c", + "sha256": "fccda877f39e7b2f99d6cc95143480e5955186264e316e832ca9a69c2ae90b8c", "role": "SourceBindingsSource, the declared-file fallback for unrecognized folders driven by source_bindings_config." }, { "path": "openhcs/agent/dto/plate.py", - "sha256": "11c92712caa3eb8dc7d19f3704ed691a6352a14f1733c7127a3e8e6a5fbca6c8", + "sha256": "689e7e720190f7abc2e0229373b88c5c60b2059efa4f7a7de14aa2d74dea3e69", "role": "PlatePathInspectionRequest source_format field and format_specific_handler_candidates result field." }, { "path": "openhcs/domains/microscopy/axes.py", - "sha256": "28cc16a296d4580351cd9abec553155e59c4d56a6ea20d52cbd6abb2879b75c6", + "sha256": "531695809259324032e0b88632813323b4fe6a9a176ad6f636aeef8dec332399", "role": "Microscopy axis family (Well, Site, Channel, ZIndex, Timepoint) used for component selectors, variable_components and group_by." }, { "path": "openhcs/core/axes.py", - "sha256": "03e5a61d808a0221507b2be93f3672cc5f4421e53eb72ce4b683309cc673213b", + "sha256": "8d93133639c576050f089fb8aab99937c8982aff07e742fea44f8a12e62bb7b2", "role": "Ungrouped grouping declaration that overlapping non-dict group_by normalizes to." } ], @@ -1754,6 +1774,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/image_sources.rst against B6's later changes to openhcs/core/source_workspace_projection.py, tests/unit/test_source_bindings.py, openhcs/core/source_binding_workspace.py, tests/unit/test_source_binding_workspace.py, tests/unit/test_plane_store_sources.py, openhcs/core/artifacts.py, openhcs/core/runtime_stores.py, openhcs/interop/cellprofiler/module_artifact_declarations.py, openhcs/core/dataset_sources/source.py, openhcs/core/dataset_sources/source_bindings_source.py, openhcs/agent/dto/plate.py, openhcs/domains/microscopy/axes.py, openhcs/core/axes.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/image_sources.rst against the later B6 changes to openhcs/core/source_bindings.py, openhcs/core/source_workspace_projection.py, tests/unit/test_source_bindings.py, openhcs/core/source_binding_workspace.py, tests/unit/test_source_binding_workspace.py, tests/unit/test_plane_store_sources.py, openhcs/core/artifacts.py, openhcs/core/runtime_stores.py, openhcs/interop/cellprofiler/module_artifact_declarations.py, openhcs/core/dataset_sources/source.py, openhcs/core/dataset_sources/source_bindings_source.py, openhcs/agent/dto/plate.py, openhcs/domains/microscopy/axes.py, openhcs/core/axes.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/image_sources.rst in full and compared it against the changed authorities: the page's own unreviewed axis-family edit, microscope_interfaces.py and source_bindings_handler.py (deleted in c6d9a52ce), source_bindings.py, source_workspace_projection.py, pipeline/compiler.py, source_binding_workspace.py, bioformats_adapter.py, PolyStore zarr.py and config.py, artifacts.py, runtime_stores.py, source_matching.py, module_artifact_declarations.py and their tests; replaced the deleted handler classes with DatasetSource / SourceBindingsSource, microscope_type with source_format, deleted component enum spellings with Microscopy axes and Ungrouped, and Plate Manager with the Datasets pane; confirmed source_stack_components, ImportedMetadataTable, InputImageSetContextSourceRelation, SourceFilterSubject/MatchType, viewer route keys and both code blocks' imports still exist.", "Re-read docs/source/guide_for_biologists/image_sources.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -1783,7 +1804,7 @@ "authority": [ { "path": "openhcs/pyqt_gui/main.py", - "sha256": "00ff3bfe7a032bd3de59274697a69d12c9b898177cd8e839f9da3b130becaedf", + "sha256": "01dcb82b5850f3cdb1c300a2aa360ba330e7f15dffb5a24e37504311095090f9", "role": "Composes the main desktop window, menus, task windows, and registered UI services." }, { @@ -1800,6 +1821,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/index.rst against B6's later changes to openhcs/pyqt_gui/main.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/index.rst against the later B6 changes to openhcs/pyqt_gui/main.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/index.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/main.py, openhcs/core/pipeline_document.py, openhcs/microscopes/microscope_interfaces.py (deleted in c6d9a52ce); the page states no fact owned by the deleted microscope interface or by the pipeline document, so both were replaced by the synthetic-plate generator window and the eight-step synthetic_plate_pipeline.py that back its only product claim; the toctree and links remain correct.", "Reviewed docs/source/guide_for_biologists/index.rst in full; the recorded findings and disposition are the resulting page-specific editorial decision.", @@ -1890,17 +1912,17 @@ "authority": [ { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "d1216010dec1804c51aec9c774f298513638ca016d500fe6328d76a9a9bfb4d6", + "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", "role": "Compiles pipeline declarations into validated step plans and execution bundles." }, { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "5d3646e17f7de843a5b36164f7cef7505da7841636423ae4fb4e00d194a87e9c", + "sha256": "3439d96f3fbfcb104a53019dc150049021d9a8132bd27096f1ccf510eefbe37b", "role": "Imports CellProfiler modules into ordinary OpenHCS pipeline declarations through the interop boundary." }, { "path": "openhcs/processing/backends/lib_registry/registry_service.py", - "sha256": "46ad34dea67bb0f7ba54e1f9f79736654df60ceefe1ef4dd523a226d3222f289", + "sha256": "daef8925338fddf8504a9faa066af42456a13fe4602976bf64be06547a45fcc8", "role": "Owns backend discovery, availability filtering, and the canonical process catalogue projection." }, { @@ -1910,7 +1932,7 @@ }, { "path": "openhcs/core/dataset_sources/choice.py", - "sha256": "ece43af801b4b02c302d9ac4bf6e5d07c0d9ebee5027433d12a4b548a19d11f0", + "sha256": "7fb3c7ce701f0d629f288b79090044f43ac8ce1b51ca4f50937534407f226f1a", "role": "DatasetSourceChoices projects the registered dataset sources, plus auto, as the dataset_source field's choices." } ], @@ -1919,6 +1941,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/intro.rst against B6's later changes to openhcs/core/pipeline/compiler.py, openhcs/interop/cellprofiler/pipeline_import.py, openhcs/processing/backends/lib_registry/registry_service.py, openhcs/core/dataset_sources/choice.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/intro.rst against the later B6 changes to openhcs/core/pipeline/compiler.py, openhcs/interop/cellprofiler/pipeline_import.py, openhcs/processing/backends/lib_registry/registry_service.py, openhcs/core/dataset_sources/choice.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/intro.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compiler.py, openhcs/interop/cellprofiler/pipeline_import.py, lib_registry/registry_service.py, pyqt_gui/widgets/plate_manager.py; the fit list, CellProfiler import claim and compile-before-run flow remain accurate (Run requires a compiled selection in the session operations); replaced the Plate Manager format-catalogue pointer with the dataset_source choices declared in openhcs/core/config.py and dataset_sources/choice.py.", "On 2026-08-27, re-read docs/source/guide_for_biologists/intro.rst in full against the explicit CellProfiler import-validation hook; challenged its explanation of supported imports and found no prose drift.", @@ -1961,12 +1984,12 @@ }, { "path": "openhcs/authoring/session/operations/datasets.py", - "sha256": "43d9edfed59abe6b88e242d39f1ddc8f5078559e04b3875da1da07b993295993", + "sha256": "208274effc2e790775d90528211fa4f640eb01f7ca3fdd9c927c3bf71e39b632", "role": "Declares the Init, Compile, Run, Viewer, and Results dataset operations and their button labels named in the tutorial steps." }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "00ff3bfe7a032bd3de59274697a69d12c9b898177cd8e839f9da3b130becaedf", + "sha256": "01dcb82b5850f3cdb1c300a2aa360ba330e7f15dffb5a24e37504311095090f9", "role": "Places Generate Synthetic Plate in the View menu." }, { @@ -1981,6 +2004,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/intro_stitching.rst against B6's later changes to openhcs/authoring/session/operations/datasets.py, openhcs/pyqt_gui/main.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/intro_stitching.rst against the later B6 changes to openhcs/authoring/session/operations/datasets.py, openhcs/pyqt_gui/main.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/intro_stitching.rst in full and compared it against the changed authorities: openhcs/demo/synthetic_data.py, openhcs/demo/synthetic_plate_pipeline.py, tests/pyqt_gui/integration/test_end_to_end_workflow_foundation.py, scripts/gallery_catalog.py; confirmed the eight step names, the View menu entry, the generator dialog labels, the Init/Compile/Run/Results labels and the four gallery scenario ids, and replaced the unnamed metadata viewer with the Viewer button and Metadata tab.", "Re-read docs/source/guide_for_biologists/intro_stitching.rst in full against repository base bb4218fae and the current shared worktree; challenged its tutorial user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -2005,7 +2029,7 @@ "authority": [ { "path": "openhcs/authoring/session/operations/datasets.py", - "sha256": "43d9edfed59abe6b88e242d39f1ddc8f5078559e04b3875da1da07b993295993", + "sha256": "208274effc2e790775d90528211fa4f640eb01f7ca3fdd9c927c3bf71e39b632", "role": "Declares the Init, Compile, Run, Connect, and Code dataset operations and the submit_timeout_ms request field that the recovery steps name." }, { @@ -2015,7 +2039,7 @@ }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "d1216010dec1804c51aec9c774f298513638ca016d500fe6328d76a9a9bfb4d6", + "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", "role": "Stops before execution on unsatisfied declarations and applies materialize_runtime_artifacts to automatic terminal outputs." }, { @@ -2030,12 +2054,12 @@ }, { "path": "openhcs/runtime/viewer_protocol.py", - "sha256": "f126aaf4f5c0dcc9429b7aae8ce67985260241ec7d2f3d5a948573658cf079a3", + "sha256": "c4bdae4bae6dee412f0d27a9cf75a95db53b3bd95c03cee7d046be16b52fe5b8", "role": "Declares the viewer server contract whose readiness and dependency failures the viewer section diagnoses." }, { "path": "openhcs/runtime/zmq_execution_client.py", - "sha256": "202619246805f8d2dfbfd5ca5427af34716e424f551ebc800d8516f7b49e07e5", + "sha256": "147fb663d3ea60b1348063440e05b77d54b40634d1ae81cf4fb4f3b5457fbc99", "role": "Implements execution-server connection and retains the .startup.jsonl journal for a server that fails before readiness." }, { @@ -2045,7 +2069,7 @@ }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "00ff3bfe7a032bd3de59274697a69d12c9b898177cd8e839f9da3b130becaedf", + "sha256": "01dcb82b5850f3cdb1c300a2aa360ba330e7f15dffb5a24e37504311095090f9", "role": "Wires endpoint compatibility to the version-restart workflow and gates the restart operation until initialisation, compilation and execution have finished." }, { @@ -2070,7 +2094,7 @@ }, { "path": "tests/unit/desktop/test_desktop_restart.py", - "sha256": "6181c22b42f7c9aa2f72b0a5a1d66bf9e5c7dfd8cba6af657851f27e5ac02af4", + "sha256": "50425f39e481a6fff384b9732bc0f2cd68f223d3ac980234a9142b6db981b48f", "role": "Proves the mismatch dialog, saved session, and version-specific restart behavior." }, { @@ -2107,6 +2131,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/troubleshooting_FAQ.rst against B6's later changes to openhcs/authoring/session/operations/datasets.py, openhcs/core/pipeline/compiler.py, openhcs/runtime/viewer_protocol.py, openhcs/runtime/zmq_execution_client.py, openhcs/pyqt_gui/main.py, tests/unit/desktop/test_desktop_restart.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/troubleshooting_FAQ.rst against the later B6 changes to openhcs/authoring/session/operations/datasets.py, openhcs/core/pipeline/compiler.py, openhcs/runtime/viewer_protocol.py, openhcs/runtime/zmq_execution_client.py, openhcs/pyqt_gui/main.py, tests/unit/desktop/test_desktop_restart.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/troubleshooting_FAQ.rst in full and compared it against the changed authorities: plate_manager.py, compiler.py, environment.py, viewer_protocol.py, zmq_execution_client.py, pyqt_gui/main.py, the moved desktop restart modules and tests, smoke_installed_desktop.py, the catalogue service and UI bridge transport/server; confirmed every named option, environment variable, journal file, button and action label in current source, replaced the receipt wording of the restart paragraph, and repointed restart authorities to openhcs/desktop.", "Re-read docs/source/guide_for_biologists/troubleshooting_FAQ.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -2164,7 +2189,7 @@ }, { "path": "openhcs/core/processing_contracts.py", - "sha256": "9b999e9210baf292f8fbf5bb3758c41a95e74401bfb1e1c127bae2dba625b0ee", + "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", "role": "Declares Pure2DContract used in the minimal declaration example." }, { @@ -2174,27 +2199,27 @@ }, { "path": "openhcs/processing/presets/pipelines/loose_operaphenix_neurite_outgrowth.py", - "sha256": "d006e526d467b6bbcf686e5d2ea41980ddb88a0c866341e96a97fe45fc8e940e", + "sha256": "1997c15cf71b8c65c8b49f238fced11b2d7a6c8bd5725ea75bdcd8cf7ab14314", "role": "Declares the example_inputs fields, including source_voxel_spacing, and the UnifiedNeurons result of the loose Opera Phenix example." }, { "path": "openhcs/microscopes/opera_phenix.py", - "sha256": "736f376bf8c1d06519a3f49ebc888dcea6b2e46d2f52e341e6dcba1445d59f07", + "sha256": "0af8f549988b0fb919d7b58121a3c260a47249e6cca6bf6d7e1a854171f2de38", "role": "Declares OperaPhenixHandler, which requires Index.xml for native plate semantics." }, { "path": "scripts/master_multi_plate_demo.py", - "sha256": "e1d2a72c8bc24e6ac4d48a6c3517d9f607bfd90c3af15081af9bc52b358eb819", + "sha256": "82f8f6e074625118ec9a96b0f67b5aa6ad39fcc95aed2b7e877d8f442770abe3", "role": "Implements the master showcase's command-line options, base port 5900, contributor loading and output root." }, { "path": "scripts/mcp_assay_showcase.py", - "sha256": "1c908d0156ac9fbfc1192aee4ef22aacfce2c3666299ab949c663aa9e818c0de", + "sha256": "a1ea7a9b6ffdec553931fbe5e9f2b755e63a98b92561ff9edee13b296d926ff6", "role": "Declares the seven built-in scenario blueprints the master showcase composes." }, { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "5d3646e17f7de843a5b36164f7cef7505da7841636423ae4fb4e00d194a87e9c", + "sha256": "3439d96f3fbfcb104a53019dc150049021d9a8132bd27096f1ccf510eefbe37b", "role": "Defines import_cellprofiler_pipeline, which lowers a .cppipe into steps and a pipeline config." } ], @@ -2205,6 +2230,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/guides/complete_examples.rst against B6's later changes to openhcs/core/processing_contracts.py, openhcs/processing/presets/pipelines/loose_operaphenix_neurite_outgrowth.py, openhcs/microscopes/opera_phenix.py, scripts/master_multi_plate_demo.py, scripts/mcp_assay_showcase.py, openhcs/interop/cellprofiler/pipeline_import.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guides/complete_examples.rst against the later B6 changes to openhcs/core/processing_contracts.py, openhcs/processing/presets/pipelines/loose_operaphenix_neurite_outgrowth.py, openhcs/microscopes/opera_phenix.py, scripts/master_multi_plate_demo.py, scripts/mcp_assay_showcase.py, openhcs/interop/cellprofiler/pipeline_import.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guides/complete_examples.rst in full and compared it against the changed authorities: knowledge_base_service.py, mfd_specs.py, loose_operaphenix_neurite_outgrowth.py, master_multi_plate_demo.py, test_official30_knowledge_examples.py; confirmed the code block's imports, the Official30 document and section ids, the MFD owner names and template path, the master demo flags, port, blueprint count and contributor callables, replaced Microscope.OPERAPHENIX with OperaPhenixHandler and added the source_voxel_spacing input.", "Reviewed docs/source/guides/complete_examples.rst in full; the recorded findings and disposition are the resulting page-specific editorial decision.", @@ -2238,7 +2264,7 @@ }, { "path": "openhcs/processing/presets/pipelines/loose_operaphenix_neurite_outgrowth.py", - "sha256": "d006e526d467b6bbcf686e5d2ea41980ddb88a0c866341e96a97fe45fc8e940e", + "sha256": "1997c15cf71b8c65c8b49f238fced11b2d7a6c8bd5725ea75bdcd8cf7ab14314", "role": "Declares the loose Opera Phenix example's exact Hoechst, MAP2 and SMI312 sources, one-well bound and unified neuron result." }, { @@ -2253,7 +2279,7 @@ }, { "path": "tests/unit/agent/test_knowledge_base_service.py", - "sha256": "67a326b8473d18cc0596aa275ad2fa1e91261a24d5083ed332bc32335aeefaee", + "sha256": "ce4636580893a8736fd4607c6ed0f431d25c97e0c44e2b95912af3fc36b1d750", "role": "Proves discovery and retrieval of the example corpus map and other maintained knowledge documents." } ], @@ -2262,6 +2288,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/guides/example_corpus_map.rst against B6's later changes to openhcs/processing/presets/pipelines/loose_operaphenix_neurite_outgrowth.py, tests/unit/agent/test_knowledge_base_service.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guides/example_corpus_map.rst against the later B6 changes to openhcs/processing/presets/pipelines/loose_operaphenix_neurite_outgrowth.py, openhcs/agent/capabilities.py, tests/unit/agent/test_knowledge_base_service.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guides/example_corpus_map.rst in full and compared it against the changed authorities: knowledge_base_service.py, mfd_specs.py, benchmark/datasets/manifest.py, test_knowledge_base_service.py; confirmed every repository path, the thirty scoped native reference rows, the four MFD wrapper modules, the cellprofiler_translation topic, the authoring-context and plate inspection tool names and the Native Example Source Index; replaced 'parity receipts' with 'parity evidence'.", "Reviewed docs/source/guides/example_corpus_map.rst in full; the recorded findings and disposition are the resulting page-specific editorial decision.", @@ -2284,7 +2311,7 @@ "authority": [ { "path": "openhcs/runtime/fiji_viewer_server.py", - "sha256": "d3c2114d16f71111562474b17a5ef4a54a7391275293c4678259e4410d11d15f", + "sha256": "ee246d32ed828bbf4665cc8a411e807da29bd43c2475012243f27e93cb550410", "role": "Initializes Fiji through FIJI_IMAGEJ_RUNTIME, tracks debounced display-batch settlement, registers the run-macro action and delegates gateway shutdown." }, { @@ -2294,7 +2321,7 @@ }, { "path": "openhcs/runtime/viewer_protocol.py", - "sha256": "f126aaf4f5c0dcc9429b7aae8ce67985260241ec7d2f3d5a948573658cf079a3", + "sha256": "c4bdae4bae6dee412f0d27a9cf75a95db53b3bd95c03cee7d046be16b52fe5b8", "role": "Declares ViewerSettleProgress and the viewer control message types named on the page." }, { @@ -2309,7 +2336,7 @@ }, { "path": "tests/unit/test_fiji_viewer_server.py", - "sha256": "bed61ba6a34937e7827b5965a63ca1afe3874d6ecc27b3c8a32721628e8b4e36", + "sha256": "da575dfb02895ecc831b34d6bdf6b58112b986556459444e9701cbf43221ea22", "role": "Proves Fiji endpoint lifecycle, streamed image updates, settlement and control handling." } ], @@ -2319,6 +2346,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/guides/fiji_viewer_management.rst against B6's later changes to openhcs/runtime/fiji_viewer_server.py, openhcs/runtime/viewer_protocol.py, tests/unit/test_fiji_viewer_server.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guides/fiji_viewer_management.rst against the later B6 changes to openhcs/runtime/fiji_viewer_server.py, openhcs/runtime/viewer_protocol.py, pyproject.toml, tests/unit/test_fiji_viewer_server.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guides/fiji_viewer_management.rst in full and compared it against the changed authorities: fiji_viewer_server.py, viewer_protocol.py, zmqruntime __init__.py, test_fiji_viewer_server.py, plus the new shared viewer_control_actions.py; rewrote the settlement section for batch-tracked settle progress and the unsupported STATE/PAYLOADS answer, and confirmed the extras, FIJI_IMAGEJ_RUNTIME initialization and shutdown delegation.", "Reviewed docs/source/guides/fiji_viewer_management.rst in full; the recorded findings and disposition are the resulting page-specific editorial decision.", @@ -2343,12 +2371,12 @@ "authority": [ { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "d1216010dec1804c51aec9c774f298513638ca016d500fe6328d76a9a9bfb4d6", + "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", "role": "Compiles pipeline declarations into validated step plans and execution bundles." }, { "path": "openhcs/core/memory/decorators.py", - "sha256": "2740c030304107079f1fcb08c43f40950d0b752cbe10d8412d519b140c7c1b1c", + "sha256": "76f234785f4865257fd19c935e0aace82d1f5586198737fec619ee243b79cbf6", "role": "Exports OpenHCS memory and dtype decorators while delegating conversion semantics to ArrayBridge declarations." }, { @@ -2363,7 +2391,7 @@ }, { "path": "openhcs/runtime/viewer_protocol.py", - "sha256": "f126aaf4f5c0dcc9429b7aae8ce67985260241ec7d2f3d5a948573658cf079a3", + "sha256": "c4bdae4bae6dee412f0d27a9cf75a95db53b3bd95c03cee7d046be16b52fe5b8", "role": "Declares the transport-independent viewer messages, stream configuration, and server contract." }, { @@ -2375,6 +2403,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/guides/index.rst against B6's later changes to openhcs/core/pipeline/compiler.py, openhcs/core/memory/decorators.py, openhcs/runtime/viewer_protocol.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guides/index.rst against the later B6 changes to openhcs/core/pipeline/compiler.py, openhcs/core/memory/decorators.py, openhcs/runtime/viewer_protocol.py, external/PolyStore/src/polystore/omero_local.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guides/index.rst in full and compared it against the changed authorities: compiler.py, core/memory/decorators.py, viewer_protocol.py, arraybridge types.py; the five toctree targets exist, ArrayBridge still declares input, output and execution memory attributes, framework device assignment still plans worker scope, and the viewer and OMERO routes name current owners; no correction was needed.", "Re-read docs/source/guides/index.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -2416,22 +2445,22 @@ }, { "path": "openhcs/core/memory/decorators.py", - "sha256": "2740c030304107079f1fcb08c43f40950d0b752cbe10d8412d519b140c7c1b1c", + "sha256": "76f234785f4865257fd19c935e0aace82d1f5586198737fec619ee243b79cbf6", "role": "Exports the OpenHCS memory decorators used in the example while delegating conversion semantics to ArrayBridge." }, { "path": "openhcs/core/processing_contracts.py", - "sha256": "9b999e9210baf292f8fbf5bb3758c41a95e74401bfb1e1c127bae2dba625b0ee", + "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", "role": "Declares the ProcessingContract family and Pure2DContract used in the example." }, { "path": "openhcs/core/callable_contract.py", - "sha256": "2060f856bcb0cc8ef57b92fc17a1e558a676e4033075950b1a4e41dc8656d330", + "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", "role": "Reads input, output and execution memory types through MemoryContractAttribute." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "f4a851d7de2ea82e1d548f37e6be2511fd20d799580193496a3746e5484906ba", + "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", "role": "Declares FrameworkDeviceAssignment and the typed per-step conversion decisions recorded by compilation." }, { @@ -2446,17 +2475,17 @@ }, { "path": "openhcs/core/steps/function_runtime.py", - "sha256": "6b523a5b649c8f58e1b7ea242bcfc45c2524166b6ed0211fc51f54b178722820", + "sha256": "b40ebb069565573c340e4fd95ce220474d27f200396ec6d9c797bf716b2c0893", "role": "Enters the invocation's execution device scope around the function body." }, { "path": "openhcs/core/orchestrator/worker_execution.py", - "sha256": "1d71883f545f7f8702c64710f968b4138d0748fa0c58b6e1b87e06a7045e5ebf", + "sha256": "23a3ef2a70378dd1bb5c413a56e36bbb3fa013142741c9b45407bfc47de8c4a4", "role": "Cleans the exact compiled framework-device footprint during worker teardown." }, { "path": "openhcs/core/config.py", - "sha256": "7b5622b3d82b1e1d672eaedb14e054be81baef417a736adb06bea34fb6708b28", + "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", "role": "Declares variable_components and group_by on the step processing configuration." }, { @@ -2475,6 +2504,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/guides/memory_type_integration.rst against B6's later changes to openhcs/core/memory/decorators.py, openhcs/core/processing_contracts.py, openhcs/core/callable_contract.py, openhcs/core/compiled_step_plan.py, openhcs/core/steps/function_runtime.py, openhcs/core/orchestrator/worker_execution.py, openhcs/core/config.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guides/memory_type_integration.rst against the later B6 changes to openhcs/core/memory/decorators.py, openhcs/core/processing_contracts.py, openhcs/core/callable_contract.py, openhcs/core/compiled_step_plan.py, openhcs/core/steps/function_runtime.py, openhcs/core/orchestrator/worker_execution.py, openhcs/core/config.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guides/memory_type_integration.rst in full and compared it against the changed authorities: arraybridge types.py and decorators.py, constants.py, core/memory/__init__.py and decorators.py, callable_contract.py, compiled_step_plan.py, test_callable_contract.py, framework_device_assignment.py, function_runtime.py, worker_execution.py, test_framework_execution_scope.py; confirmed the MemoryType re-exports, MemoryContractAttribute reads, FrameworkDeviceAssignment, execution device scope, subprocess_environment consumers and variable_components/group_by, and reworded the processing-contract paragraph for the subclass family.", "On 2026-08-27, re-read docs/source/guides/memory_type_integration.rst in full after callable contracts began projecting runtime-owned inputs into generic form exclusions; challenged the ArrayBridge/OpenHCS ownership boundary and found no prose drift.", @@ -2531,17 +2561,17 @@ }, { "path": "openhcs/domains/microscopy/analysis_consolidation.py", - "sha256": "1084a6761a379417f86a042d635d4d43fbe55c4ac497998e48c94fe56807393a", + "sha256": "292f7138a00dc22dba5b8d8af5de6140d247b955723cc807cc982ffec7a7dd2d", "role": "Consolidates execution-ledger CSV content and writes summaries through FileManager and the compiled backend." }, { "path": "openhcs/core/dataset_sources/source.py", - "sha256": "f736728d2f715b72cd5479fcc79520911b037d3410a42ebfc9febef2e51ffda6", + "sha256": "097fde4d2b74031eacf53a88e79f499417e8c1c97ba0753e8a9df831b307efa6", "role": "Declares the DatasetSource interface the OMERO source implements for workspace and metadata access." }, { "path": "openhcs/microscopes/omero.py", - "sha256": "11f486719055f05744b25076274361e332520af02cfa96bc37ec02d49e783149", + "sha256": "a3ab5a257490f729e96ba413c3f11d12f10f68dae44a728206f81b850325da31", "role": "Declares OMEROHandler and OMEROFilenameParser, which map PolyStore's concrete and symbolic OMERO filenames into FilenameParseResult." }, { @@ -2591,6 +2621,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/guides/omero_integration.rst against B6's later changes to openhcs/domains/microscopy/analysis_consolidation.py, openhcs/core/dataset_sources/source.py, openhcs/microscopes/omero.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guides/omero_integration.rst against the later B6 changes to external/PolyStore/src/polystore/omero_local.py, openhcs/domains/microscopy/analysis_consolidation.py, openhcs/core/dataset_sources/source.py, openhcs/microscopes/omero.py, pyproject.toml: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guides/omero_integration.rst in full and compared it against the changed authorities: PolyStore filemanager.py and base.py, microscopes/omero.py, test_omero_microscope_metadata.py, pyproject.toml, tests/integration/test_main.py, ui_bridge_identities.py, plus the moved analysis consolidation and dataset-source owners; confirmed OMEROFilenameParser, FilenameParseResult, the PolyStore address types, DataSink contextual saves, ledger-based consolidation through FileManager and OMEROInstanceManager, and dropped 'canonical'.", "On 2026-08-27, re-read docs/source/guides/omero_integration.rst in full after the python-introspect dependency floor advanced to 0.1.12; challenged its OMERO ownership explanation and found no affected claim.", @@ -2639,22 +2670,22 @@ "authority": [ { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "d1216010dec1804c51aec9c774f298513638ca016d500fe6328d76a9a9bfb4d6", + "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", "role": "Defines compile_pipelines over an orchestrator and list[AbstractStep] and returns a CompiledExecutionBundle." }, { "path": "openhcs/core/orchestrator/orchestrator.py", - "sha256": "4e55309de74e770b2e37b53f6dbedc19ffac6988ab74cdfe4b3dfc51082e8c1a", + "sha256": "a61658b107f170c192eea505966c268ab6826826f03586bd64562a44800e1ccc", "role": "Supplies the initialized orchestrator, its DatasetSource workspace and selected partition values to compilation." }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "90d75f6287c5f21f93d82960729199261f7906e0f4912b7728527260b9723a66", + "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", "role": "Declares CompilationSession, which owns cross-step compile state." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "f4a851d7de2ea82e1d548f37e6be2511fd20d799580193496a3746e5484906ba", + "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", "role": "Declares CompiledStepPlan and its typed per-step memory, conversion, device and runtime fields." }, { @@ -2669,7 +2700,7 @@ }, { "path": "tests/unit/test_compiled_execution.py", - "sha256": "153a7ce595d55b3fde66c7ece1cdc26526fd9d91dba78a5a9bf51c91e870df2f", + "sha256": "cb6c7034e9534470daf99a471b27a9ae662b5d2828f01af8613534412e8eea81", "role": "Proves compilation produces immutable step plans and execution bundles with the expected resolved values." } ], @@ -2678,6 +2709,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/guides/pipeline_compilation_workflow.rst against B6's later changes to openhcs/core/pipeline/compiler.py, openhcs/core/orchestrator/orchestrator.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/compiled_step_plan.py, tests/unit/test_compiled_execution.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guides/pipeline_compilation_workflow.rst against the later B6 changes to openhcs/core/pipeline/compiler.py, openhcs/core/orchestrator/orchestrator.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/compiled_step_plan.py, tests/unit/test_compiled_execution.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guides/pipeline_compilation_workflow.rst in full and compared it against the changed authorities: compiler.py, compilation_session.py, compiled_step_plan.py, compiled_execution.py, test_compiled_execution.py, framework_device_assignment.py; confirmed CompilationSession, CompiledStepPlan, CompiledExecutionBundle and the compile_pipelines inputs, and renamed the microscope/source workspace to the dataset-source workspace.", "Re-read docs/source/guides/pipeline_compilation_workflow.rst in full against repository base bb4218fae and the current shared worktree; challenged its explanation user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -2924,7 +2956,7 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "7b5622b3d82b1e1d672eaedb14e054be81baef417a736adb06bea34fb6708b28", + "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", "role": "Declares StreamingConfig, NapariStreamingConfig, FijiStreamingConfig and StepMaterializationConfig, including the partition filter inherited by streaming configuration." }, { @@ -2939,7 +2971,7 @@ }, { "path": "openhcs/runtime/viewer_protocol.py", - "sha256": "f126aaf4f5c0dcc9429b7aae8ce67985260241ec7d2f3d5a948573658cf079a3", + "sha256": "c4bdae4bae6dee412f0d27a9cf75a95db53b3bd95c03cee7d046be16b52fe5b8", "role": "Declares the SHAPES streaming data type and the settle progress fields." }, { @@ -2974,7 +3006,7 @@ }, { "path": "openhcs/core/artifacts.py", - "sha256": "907551ae0ce4bf42a557554f5df0252b0a88fc9b7c1b7ac19705aade66b6656f", + "sha256": "65eb0ec5c903366243ca8bd5ea07412e8d34bdb81e6015fa76b3c4c59838733f", "role": "Declares SpatialGraphArtifactType." }, { @@ -2989,12 +3021,12 @@ }, { "path": "tests/unit/test_napari_roi_manager_integration.py", - "sha256": "10d7079b3e10088bac81a8814b4d0f77854d20b7818f2c2a9d121589b295ca15", + "sha256": "a4b7dee556f0ceccaa212751e29eb8160d3d0c08c741760715ee8372b0c0e927", "role": "Proves the packaged Napari ROI manager is discovered and integrated with the viewer contract." }, { "path": "tests/unit/test_viewer_result_element_selection.py", - "sha256": "a7b348fab106da4c89e58ed32225495aa27792132ef7488657f29821dfd6da92", + "sha256": "b1dae580f0291f53be9453df6f6c6df5458046bd3ff947b550f7ddea94b29f96", "role": "Proves isolation preflights every requested route and selected-route axis before changing live Napari state." } ], @@ -3004,6 +3036,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/guides/viewer_management.rst against B6's later changes to openhcs/core/config.py, openhcs/runtime/viewer_protocol.py, openhcs/core/artifacts.py, tests/unit/test_napari_roi_manager_integration.py, tests/unit/test_viewer_result_element_selection.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guides/viewer_management.rst against the later B6 changes to openhcs/core/config.py, openhcs/runtime/viewer_protocol.py, openhcs/core/artifacts.py, tests/unit/test_napari_roi_manager_integration.py, tests/unit/test_viewer_result_element_selection.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guides/viewer_management.rst in full and compared it against the changed authorities: napari_viewer_server.py, viewer_protocol.py, viewer_controls.py, viewer_window_service.py, napari.yaml, the ROI manager, streaming handler and element-selection tests, and napari_streaming_handlers.py; confirmed the streaming and materialization config classes, SHAPES boundary, ROI toolbar labels, plugin menu entries, license file, spatial-graph options, max_rois, layer isolation and transport-thread settle, renamed well selection to partition selection, and replaced 'custody'.", "Reviewed docs/source/guides/viewer_management.rst in full; the recorded findings and disposition are the resulting page-specific editorial decision.", @@ -3037,12 +3070,12 @@ }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "00ff3bfe7a032bd3de59274697a69d12c9b898177cd8e839f9da3b130becaedf", + "sha256": "01dcb82b5850f3cdb1c300a2aa360ba330e7f15dffb5a24e37504311095090f9", "role": "Composes the main desktop window, menus, task windows, and registered UI services." }, { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "5d3646e17f7de843a5b36164f7cef7505da7841636423ae4fb4e00d194a87e9c", + "sha256": "3439d96f3fbfcb104a53019dc150049021d9a8132bd27096f1ccf510eefbe37b", "role": "Imports CellProfiler modules into ordinary OpenHCS pipeline declarations through the interop boundary." }, { @@ -3059,6 +3092,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/index.rst against B6's later changes to openhcs/pyqt_gui/main.py, openhcs/interop/cellprofiler/pipeline_import.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/index.rst against the later B6 changes to openhcs/pyqt_gui/main.py, openhcs/interop/cellprofiler/pipeline_import.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/index.rst in full and compared it against the changed authorities: pipeline_document.py (PipelineDocumentAuthority renamed PipelineDocumentCodec), pyqt_gui/main.py, cellprofiler pipeline_import.py, mcp/bootstrap.py, gallery_catalog.py; the page names none of the renamed symbols, the multi-plate-overview scenario still exists, and the routing claims and toctree remain accurate; no correction was needed.", "On 2026-08-27, re-read docs/source/index.rst in full against the explicit CellProfiler import-validation hook; challenged its Diataxis routing and supported-import overview and found no prose drift.", @@ -3155,7 +3189,7 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "7b5622b3d82b1e1d672eaedb14e054be81baef417a736adb06bea34fb6708b28", + "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", "role": "Declares GlobalPipelineConfig, PipelineConfig and the step configuration families with their defaults and field help." }, { @@ -3197,6 +3231,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/reference/configuration.rst against B6's later changes to openhcs/core/config.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/reference/configuration.rst against the later B6 changes to openhcs/core/config.py, openhcs/agent/capabilities.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/reference/configuration.rst in full and compared it against the changed authorities: core/config.py, test_config_reference_service.py, test_user_facing_help_surfaces.py; the four directive names still resolve through config_service aliases, the named config classes, openhcs_describe_config_schema and ConfigPatch exist, and the UIConfig overview matches its fields; no correction was needed. The generated field tables carry the G1/G4 renames without page edits.", "Re-read docs/source/reference/configuration.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -3223,12 +3258,12 @@ "authority": [ { "path": "openhcs/core/processing_contracts.py", - "sha256": "9b999e9210baf292f8fbf5bb3758c41a95e74401bfb1e1c127bae2dba625b0ee", + "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", "role": "Declares the ProcessingContract family: Pure2D, Pure3D, Flexible, and VolumetricToSlice contracts." }, { "path": "openhcs/core/image_payload_execution_mode.py", - "sha256": "7be989333a4a58aff1184173a7b08b2d9cd821761af497575b65d9effb24ed38", + "sha256": "7657995da0722cebfd0448c0cdce4cada1b00e0cff419f62e17dcc3654666ce0", "role": "Declares the image payload execution modes, including FullStackExecution, that preserve a nested volume under an outer plane axis." }, { @@ -3238,32 +3273,32 @@ }, { "path": "openhcs/processing/backends/cellprofiler/watershed.py", - "sha256": "4b8b3e9360c090ba9bbb1584ddb7749e3ff3ecb62d609ea66eb27d8a4b2762d2", + "sha256": "41c8cb9cb139f8b4041a25e1a7a2d19aa85cf0427422fc87b9551c8141069625", "role": "Declares the full-stack payload route for CellProfiler-compatible Watershed." }, { "path": "openhcs/processing/backends/cellprofiler/primary_objects.py", - "sha256": "825d2ed2dfe7f0248af0f4b840a80a3f053227fab4cf06896311e89839dfd6d8", + "sha256": "b5b8a9b98dd9166b34a8f5cebf884e8af6da85f02fe878ef1227d6656d31ae4c", "role": "Declares the current plane-local IdentifyPrimaryObjects route." }, { "path": "openhcs/processing/backends/cellprofiler/secondary.py", - "sha256": "b0773976512067ec267df254111d305dbdec9694e8359c380cf4e28d327bc805", + "sha256": "fc9d9b1bbc3baff7216c6abe3a7135ffddfc9f7e1212d4a391fa8534038a354c", "role": "Declares the current plane-local IdentifySecondaryObjects and IdentifyTertiaryObjects routes." }, { "path": "openhcs/processing/backends/cellprofiler/intensity.py", - "sha256": "3086bf9a2cee76843fd2be8445799d29ec04f0f284570c52c20b3d55f61b62ab", + "sha256": "43bfa771bb26453f864b633a48371c0e68cf5aae0e175ed6d7a1eabb38ec5dbe", "role": "Declares volumetric object-intensity execution and its Z measurement features." }, { "path": "openhcs/processing/backends/cellprofiler/shape.py", - "sha256": "df062abfbdd8c69c9c9a309842cc2ab8917da02dd01a2e330e565515f0ed8695", + "sha256": "69e82de4faa46adc1b5ea543e352edf9a253c4b3bdedec61e0e5e3a7d95b3019", "role": "Declares full-stack object-shape input and the exact 3D feature schema." }, { "path": "openhcs/processing/backends/cellprofiler/area_occupied.py", - "sha256": "4f1b8ce875b3c8f67dad382868919375ee574952993772c7259e92892a882646", + "sha256": "8316d44a5ea2f88f855aeef1c3d4fd979fb01a95a6dc9f6d1bf005629f25b18b", "role": "Declares 3D occupied-volume measurements and physical voxel-spacing input." }, { @@ -3273,7 +3308,7 @@ }, { "path": "tests/unit/test_documented_dimensionality.py", - "sha256": "0b83e470095a2c75996c58d26f8ed4b908ee7830b9546bfb7cc40bbd9cc2de52", + "sha256": "45762e74bf5cecbbada509f378ddbfa47c20fa6a9f5bd668d05d003e59bb38e1", "role": "Binds the published dimensionality claims to the current callable and feature declarations." }, { @@ -3283,12 +3318,12 @@ }, { "path": "tests/unit/test_cellprofiler_library_loading.py", - "sha256": "b0b5451d67ae9329f49e9108d7b834bc69415cb68d2a089bc0e4d6bfc6d9f0ff", + "sha256": "316732ba044a62b80711a0466d8ce5812cb882ff91fbf0417ecae90afa3ff610", "role": "Proves object intensity measures a 3D object as one volume and emits Z features." }, { "path": "tests/unit/test_cellprofiler_module_execution.py", - "sha256": "39b15a579acf2097b0229d3394d19870e200523f92326b96b7d979bfd7cfd2f2", + "sha256": "f5d678e29026578687a68f6ce7abc3184eec5223a7e4e2337038fb2079a98ea2", "role": "Proves payload-scoped size-and-shape rows use the 3D volume schema." }, { @@ -3302,6 +3337,7 @@ ], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/reference/dimensionality_and_measurements.rst against B6's later changes to openhcs/core/processing_contracts.py, openhcs/core/image_payload_execution_mode.py, openhcs/processing/backends/cellprofiler/watershed.py, openhcs/processing/backends/cellprofiler/primary_objects.py, openhcs/processing/backends/cellprofiler/secondary.py, openhcs/processing/backends/cellprofiler/intensity.py, openhcs/processing/backends/cellprofiler/shape.py, openhcs/processing/backends/cellprofiler/area_occupied.py, tests/unit/test_documented_dimensionality.py, tests/unit/test_cellprofiler_library_loading.py, tests/unit/test_cellprofiler_module_execution.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/reference/dimensionality_and_measurements.rst against the later B6 changes to openhcs/core/processing_contracts.py, openhcs/core/image_payload_execution_mode.py, openhcs/processing/backends/cellprofiler/watershed.py, openhcs/processing/backends/cellprofiler/primary_objects.py, openhcs/processing/backends/cellprofiler/secondary.py, openhcs/processing/backends/cellprofiler/intensity.py, openhcs/processing/backends/cellprofiler/shape.py, openhcs/processing/backends/cellprofiler/area_occupied.py, tests/unit/test_documented_dimensionality.py, tests/unit/test_cellprofiler_library_loading.py, tests/unit/test_cellprofiler_module_execution.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/reference/dimensionality_and_measurements.rst in full and compared it against the changed authorities: aligned_image_payload.py, runtime_object_label_domains.py, unified_registry.py, the six CellProfiler backends, the Official30 axis1 manifest, and the four tests; confirmed primary/secondary/tertiary still declare Pure2DContract, Watershed and intensity use FullStackExecution, shape and area-occupied use FullStackLabels, occupied-volume callables use Pure3DContract, PAYLOAD/PLANE scopes remain, and every listed measurement family has a backend module; no page change was needed.", "On 2026-08-27, re-read docs/source/reference/dimensionality_and_measurements.rst in full against registry form exclusions, explicit secondary-backend default selection, and expanded import tests; challenged every dimensional capability claim and found no semantic drift.", @@ -3334,12 +3370,12 @@ }, { "path": "openhcs/processing/backends/experimental_analysis/analysis.py", - "sha256": "2fe9bcfba30e739cbf0b74112bfec6ce8714f173bdbd08b6dde1e3b6abe78d6e", + "sha256": "17a701e0740eac3e11319fe73f10302bbcab603218349bfba4e07014aabc27bb", "role": "Parses the drug_curve_map and plate_groups sheets, applies positional dose/well/plate-group mapping, controls, exclusions, and Per Well Datapoints." }, { "path": "openhcs/processing/backends/experimental_analysis/result_formats.py", - "sha256": "b0790fa7050855c772f6817b3d538c86bbb161d3d0446020d808ada95351a3c0", + "sha256": "6951f41a2ff2bdac1945c7e56584a578c111c078cac1d684030c00f95a23f528", "role": "Owns the CX5 workbook and MetaXpress CSV result readers selected by the workbook scope." }, { @@ -3349,7 +3385,7 @@ }, { "path": "openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py", - "sha256": "bd43ec00c265279585f6bcc9eb0a6b6a2eeedaacad6003a33a60ab181cd13a4d", + "sha256": "2705d459204557c4f7092df3fd21ed687dcf56449ed361485382b1c0ce196444", "role": "Exports ExperimentalAnalysisEngine, whose run_directory projects config filenames and returns the documented result dictionary." }, { @@ -3359,7 +3395,7 @@ }, { "path": "tests/unit/test_experimental_analysis_engine.py", - "sha256": "c79887f97615197c25c340dd277189ebc331bce1f3e9bfec7466c57c92ebd6cf", + "sha256": "641fadecf8c940cc2316a06771c74db5f3f2db9dafdfbfebdc535760d6d776ec", "role": "Exercises declaration-owned paths and a complete MetaXpress directory workflow with all three outputs." } ], @@ -3369,6 +3405,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/reference/experimental_config_syntax.rst against B6's later changes to openhcs/processing/backends/experimental_analysis/analysis.py, openhcs/processing/backends/experimental_analysis/result_formats.py, openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py, tests/unit/test_experimental_analysis_engine.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/reference/experimental_config_syntax.rst against the later B6 changes to openhcs/processing/backends/experimental_analysis/analysis.py, openhcs/processing/backends/experimental_analysis/result_formats.py, openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py, tests/unit/test_experimental_analysis_engine.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/reference/experimental_config_syntax.rst in full and compared it against the changed authorities: the deleted openhcs/formats/experimental_{layout_rows,analysis,result_formats}.py (moved to processing/backends/experimental_analysis/), openhcs/core/config.py, unified_analysis_engine.py, and both experimental-analysis tests; checked the row roles, scope values, engine export and run_directory signature against the moved modules, and corrected the ExperimentalAnalysisConfig import to openhcs.domains.microscopy.config.", "Re-read docs/source/reference/experimental_config_syntax.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -3390,12 +3427,12 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "7b5622b3d82b1e1d672eaedb14e054be81baef417a736adb06bea34fb6708b28", + "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", "role": "Declares the global, pipeline, step, and desktop configuration dataclasses that the generated configuration reference lists." }, { "path": "openhcs/processing/backends/experimental_analysis/analysis.py", - "sha256": "2fe9bcfba30e739cbf0b74112bfec6ce8714f173bdbd08b6dde1e3b6abe78d6e", + "sha256": "17a701e0740eac3e11319fe73f10302bbcab603218349bfba4e07014aabc27bb", "role": "Parses the Excel experimental-design workbook that the syntax reference documents." }, { @@ -3405,7 +3442,7 @@ }, { "path": "openhcs/core/artifacts.py", - "sha256": "907551ae0ce4bf42a557554f5df0252b0a88fc9b7c1b7ac19705aade66b6656f", + "sha256": "65eb0ec5c903366243ca8bd5ea07412e8d34bdb81e6015fa76b3c4c59838733f", "role": "Declares the artifact types named by the shared terminology the index points to." } ], @@ -3414,6 +3451,7 @@ ], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/reference/index.rst against B6's later changes to openhcs/core/config.py, openhcs/processing/backends/experimental_analysis/analysis.py, openhcs/core/artifacts.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/reference/index.rst against the later B6 changes to openhcs/core/config.py, openhcs/processing/backends/experimental_analysis/analysis.py, openhcs/core/artifacts.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/reference/index.rst in full and compared it against the changed authorities: openhcs/core/config.py, the deleted openhcs/formats/experimental_analysis.py, openhcs/agent/services/knowledge_base_service.py, and openhcs/core/artifacts.py; checked that every toctree target exists and each one-line description still matches its page and owner; no page change was needed.", "Re-read docs/source/reference/index.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -3446,12 +3484,12 @@ }, { "path": "openhcs/authoring/session/views.py", - "sha256": "2963a2c48856704ea8291075316c470ccc8eeed161b5a798181a519a2ceaef98", + "sha256": "8ce6cac27293ac9105f4d2ebc76d10eebeba1ef90e0e8954c0a117910e17e3b3", "role": "Owns the session view titles that the plate-manager and pipeline-editor window identities present." }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "00ff3bfe7a032bd3de59274697a69d12c9b898177cd8e839f9da3b130becaedf", + "sha256": "01dcb82b5850f3cdb1c300a2aa360ba330e7f15dffb5a24e37504311095090f9", "role": "Composes the main desktop window and its stable docked surfaces from the registered identities." }, { @@ -3483,6 +3521,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/reference/ui_windows.rst against B6's later changes to openhcs/authoring/session/views.py, openhcs/pyqt_gui/main.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/reference/ui_windows.rst against the later B6 changes to openhcs/authoring/session/views.py, openhcs/pyqt_gui/main.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/reference/ui_windows.rst in full and compared it against the changed authorities: openhcs/agent/ui_bridge_identities.py, openhcs/pyqt_gui/main.py, openhcs/pyqt_gui/services/main_window_workflows.py, scripts/gallery_catalog.py, tests/unit/test_build_website.py, and tests/unit/test_capture_media_gallery.py; the U1 session refactor moved two window titles onto session views but the inventory, titles, and capture targets are still projected from the registered identity declarations, no OMERO window is registered, and all linked pages exist; added openhcs/authoring/session/views.py as the title owner; no page change was needed.", "On 2026-08-28, re-read docs/source/reference/ui_windows.rst in full and challenged its reference boundary, completeness, duplication, navigation, and current desktop claims against the registered identity and projection owners.", @@ -3506,7 +3545,7 @@ "authority": [ { "path": "openhcs/pyqt_gui/main.py", - "sha256": "00ff3bfe7a032bd3de59274697a69d12c9b898177cd8e839f9da3b130becaedf", + "sha256": "01dcb82b5850f3cdb1c300a2aa360ba330e7f15dffb5a24e37504311095090f9", "role": "Declares the four Tools > Analysis Consolidation actions and the experimental-analysis directory check." }, { @@ -3516,17 +3555,17 @@ }, { "path": "openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py", - "sha256": "bd43ec00c265279585f6bcc9eb0a6b6a2eeedaacad6003a33a60ab181cd13a4d", + "sha256": "2705d459204557c4f7092df3fd21ed687dcf56449ed361485382b1c0ce196444", "role": "Projects the five configured filenames and writes normalized, raw, and heatmap outputs." }, { "path": "tests/unit/test_experimental_analysis_engine.py", - "sha256": "c79887f97615197c25c340dd277189ebc331bce1f3e9bfec7466c57c92ebd6cf", + "sha256": "641fadecf8c940cc2316a06771c74db5f3f2db9dafdfbfebdc535760d6d776ec", "role": "Proves the documented directory contract and three output files end to end." }, { "path": "openhcs/domains/microscopy/analysis_consolidation.py", - "sha256": "1084a6761a379417f86a042d635d4d43fbe55c4ac497998e48c94fe56807393a", + "sha256": "292f7138a00dc22dba5b8d8af5de6140d247b955723cc807cc982ffec7a7dd2d", "role": "Owns the post-execute hook that consolidates execution-saved tables per directory and writes the merged summary beneath the compiled output plate root." } ], @@ -3536,6 +3575,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/user_guide/analysis_consolidation.rst against B6's later changes to openhcs/pyqt_gui/main.py, openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py, tests/unit/test_experimental_analysis_engine.py, openhcs/domains/microscopy/analysis_consolidation.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/analysis_consolidation.rst against the later B6 changes to openhcs/pyqt_gui/main.py, openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py, tests/unit/test_experimental_analysis_engine.py, openhcs/domains/microscopy/analysis_consolidation.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/user_guide/analysis_consolidation.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/main.py, openhcs/core/config.py, the deleted openhcs/formats/experimental_analysis.py and openhcs/core/orchestrator/analysis_consolidation.py (now openhcs/domains/microscopy/analysis_consolidation.py), unified_analysis_engine.py, and the engine test; checked the menu actions, directory filenames, consolidation config fields, and per-directory plus merged summary behaviour; corrected the config description.", "Re-read docs/source/user_guide/analysis_consolidation.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -3568,7 +3608,7 @@ }, { "path": "openhcs/authoring/session/dataset_document.py", - "sha256": "ca695d82a60dbee76ff660ae3710ac5033d2e55f51a99373ed8c0f0dccdbc16c", + "sha256": "305da3c0bf7bb15a0f251a101696b721f9151d8620450a6b3e12413b79218574", "role": "Owns the selected and all-datasets document scopes, selected-scope ID enforcement, and the global-draft and per-dataset change checks on apply." }, { @@ -3578,12 +3618,12 @@ }, { "path": "openhcs/authoring/session/views.py", - "sha256": "2963a2c48856704ea8291075316c470ccc8eeed161b5a798181a519a2ceaef98", + "sha256": "8ce6cac27293ac9105f4d2ebc76d10eebeba1ef90e0e8954c0a117910e17e3b3", "role": "Declares the Datasets and Pipeline Editor view titles shown for the two code-document owners." }, { "path": "openhcs/pyqt_gui/widgets/pipeline_editor.py", - "sha256": "abca0b56094615407fd81df039761ef9d48ddb57ddd19f31e87e804974f25b22", + "sha256": "8e418631fd9ba4f8ac3d6b257f9af127f9e8711281608516ea2617d7855d6f36", "role": "Makes the Pipeline Editor document writable only while a dataset is selected." }, { @@ -3603,17 +3643,17 @@ }, { "path": "openhcs/pyqt_gui/services/ui_bridge_object_state.py", - "sha256": "81b5ffbbd19952f85430962cd1bb61918ac3abe4dd9ec843b8f67e4590ffd06b", + "sha256": "e548b367507b3fb6830ce9ecb844e3b756243af96fe38e0d35de5da0b974c1fd", "role": "Enforces driver writability, returns unchanged for equivalent documents, and applies documents against revision tokens." }, { "path": "openhcs/pyqt_gui/services/ui_agent_bridge.py", - "sha256": "5a03e683a0677166361440c6c78803ec2dadf753342e69996153005bccab5473", + "sha256": "ce650c52475764657bbc0221109879594f9cf1482de8245d6c141f00474d5d23", "role": "Rejects wildcard imports and resolves catalogue-listed external functions in submitted documents." }, { "path": "openhcs/agent/services/ui_bridge_service.py", - "sha256": "dc2c4823cc9a611d384bba2785eb6e1b75338f0ec4f731292515bfded2218d63", + "sha256": "c7bdf8b5caec1c21c986cdc2224524839fadfa8764dafcd05125d90c72351899", "role": "Owns agent-to-UI document inspection and mutation and returns mutation receipts." }, { @@ -3643,7 +3683,7 @@ }, { "path": "openhcs/pyqt_gui/widgets/shared/services/pipeline_editor_workflows.py", - "sha256": "d77abd94b6db02b55486326d0bbd6c565dbea8dfedb41cd0ee03dcba6a4540b1", + "sha256": "c704ef7e6c5f8a5745c741e923bc4e620a321fdce069e1cdcdf521c0176e4a4f", "role": "Applies a complete Pipeline Editor document and commits the reconciled live state graph." }, { @@ -3658,12 +3698,12 @@ }, { "path": "tests/unit/pyqt_gui/test_ui_agent_bridge.py", - "sha256": "b03744b1fa201f77de26d2e6050422b6422ebce78443c478c5e8c5f55e0a3e24", + "sha256": "de6758859d9743e826369162855327e4a84056cafdd7ddfdc33868d2a3093353", "role": "Proves bidirectional UI-bridge inspection, capability enforcement, semantic no-op handling, and mutation of live declarations." }, { "path": "tests/unit/pyqt_gui/test_plate_manager_widget.py", - "sha256": "5e66ccf63d5b422e3a241e8f5eb0e65cc5067edacb9656466dfb5f0994e3f07b", + "sha256": "4daa91c0de965dc2dd9cd77ea4969f4fa0aff90fc5b69cbcadaba3bd2a1cfc6b", "role": "Proves selected code-document application preserves unselected dataset graphs." }, { @@ -3673,7 +3713,7 @@ }, { "path": "tests/unit/pyqt_gui/test_pipeline_editor_widget.py", - "sha256": "10afc4ce49c91731a8dc7e52726c6a77198747a6ec3a765e7e88ceaf40967e3a", + "sha256": "0ce88271aa7f5f55a8fc4daec8279d732844e03dfd437c11e6e6265ca196b4a0", "role": "Proves selected-dataset writability and that code apply advances saved baselines for the editor, step, and nested function states." }, { @@ -3683,7 +3723,7 @@ }, { "path": "tests/unit/test_pycodify_formatters.py", - "sha256": "1dd67d1a782d04dc24ac2ba9ec1e457769c1c6c9191c992f0a1cc0feb78487c5", + "sha256": "2f0c6683a5255d01442ae33f676f601d5e3f1e3e04d1b2c2fc945b3c564796ce", "role": "Verifies clean/full parameter order and semantic default handling." }, { @@ -3699,6 +3739,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/user_guide/code_ui_editing.rst against B6's later changes to openhcs/authoring/session/dataset_document.py, openhcs/authoring/session/views.py, openhcs/pyqt_gui/widgets/pipeline_editor.py, openhcs/pyqt_gui/services/ui_bridge_object_state.py, openhcs/pyqt_gui/services/ui_agent_bridge.py, openhcs/agent/services/ui_bridge_service.py, openhcs/pyqt_gui/widgets/shared/services/pipeline_editor_workflows.py, tests/unit/pyqt_gui/test_ui_agent_bridge.py, tests/unit/pyqt_gui/test_plate_manager_widget.py, tests/unit/pyqt_gui/test_pipeline_editor_widget.py, tests/unit/test_pycodify_formatters.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/code_ui_editing.rst against the later B6 changes to openhcs/authoring/session/dataset_document.py, openhcs/authoring/session/views.py, openhcs/pyqt_gui/widgets/pipeline_editor.py, openhcs/pyqt_gui/services/ui_bridge_object_state.py, openhcs/pyqt_gui/services/ui_agent_bridge.py, openhcs/agent/services/ui_bridge_service.py, openhcs/pyqt_gui/widgets/shared/services/pipeline_editor_workflows.py, tests/unit/pyqt_gui/test_ui_agent_bridge.py, tests/unit/pyqt_gui/test_plate_manager_widget.py, tests/unit/pyqt_gui/test_pipeline_editor_widget.py, tests/unit/test_pycodify_formatters.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/user_guide/code_ui_editing.rst in full and compared it against the changed authorities: pipeline_document.py, ui_bridge_object_state.py, ui_bridge_service.py, plate_manager_workflows.py and pipeline_editor_workflows.py, the deleted pipeline_object_state_binding.py (now authoring/session/pipelines.py), step_parameter_editor.py, pycodify_formatters.py, gallery_catalog.py, and the changed tests; checked selected/all document scopes, selected-dataset writability, global-draft protection while work runs, catalogue imports and wildcard rejection, unchanged no-op results, and saved-baseline commit; renamed Plate Manager to the Datasets window.", "Re-read docs/source/user_guide/code_ui_editing.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -3741,7 +3782,7 @@ }, { "path": "openhcs/processing/backends/lib_registry/registry_service.py", - "sha256": "46ad34dea67bb0f7ba54e1f9f79736654df60ceefe1ef4dd523a226d3222f289", + "sha256": "daef8925338fddf8504a9faa066af42456a13fe4602976bf64be06547a45fcc8", "role": "Skips registries that do not support CPU-only mode during function discovery." }, { @@ -3758,6 +3799,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/user_guide/cpu_only_mode.rst against B6's later changes to openhcs/processing/backends/lib_registry/registry_service.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/cpu_only_mode.rst against the later B6 changes to openhcs/processing/backends/lib_registry/registry_service.py, tests/unit/test_library_registry_discovery.py, pyproject.toml: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/user_guide/cpu_only_mode.rst in full and compared it against the changed authorities: openhcs/utils/environment.py, registry_service.py, test_library_registry_discovery.py, and pyproject.toml; confirmed --no-gpu and OPENHCS_CPU_ONLY still select the same mode, discovery still filters registries by CPU-only support, and the gpu extra still exists; the new import-policy defaults (JAX platform, NumPy huge pages, worker profiling) do not contradict the page; no page change was needed.", "On 2026-08-27, re-read docs/source/user_guide/cpu_only_mode.rst in full against declaration-owned catalogue inputs and the python-introspect 0.1.12 floor; challenged the CPU-only workflow and found no behaviour change.", @@ -3796,7 +3838,7 @@ "authority": [ { "path": "openhcs/processing/custom_functions/manager.py", - "sha256": "5cd7e955087f0bd9c527fc6307dff48f4aa959c607a3ec776b8379abac76fb36", + "sha256": "0ffa6fc2a18566600dcff33d4b27466d8a5684662fa1c696cb5ea52ea459a1e4", "role": "Owns persisted source under the data directory, requires one decorated processing function per source, rejects collisions, and coordinates create, load, replace, and delete." }, { @@ -3816,7 +3858,7 @@ }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "00ff3bfe7a032bd3de59274697a69d12c9b898177cd8e839f9da3b130becaedf", + "sha256": "01dcb82b5850f3cdb1c300a2aa360ba330e7f15dffb5a24e37504311095090f9", "role": "Declares the Tools > Custom Functions > Manage Functions action." }, { @@ -3836,7 +3878,7 @@ }, { "path": "openhcs/mcp/context.py", - "sha256": "af9ec43138e05762fc97b34825b96e15f183aa56781d697c601eb79f14a1bc42", + "sha256": "1292c9bb7173e6afe5aa6237b861c5f39bc2e854f364fd722c75a3c6b76f534e", "role": "Supplies the local MCP tools with the session's endpoint function catalogue." }, { @@ -3856,7 +3898,7 @@ }, { "path": "tests/unit/test_custom_function_lifecycle.py", - "sha256": "e333004437e29a3e44aeb93699f5fb0b61092bb9c48068e3bc0011781adbc885", + "sha256": "70c87f0dbefd2c1a204d060c2211ec57f4f84023f7c6931d2cb7d81eb82e9252", "role": "Proves transactional create, load, replace, delete, collision, and source-revision behaviour." }, { @@ -3876,6 +3918,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/user_guide/custom_function_management.rst against B6's later changes to openhcs/processing/custom_functions/manager.py, openhcs/pyqt_gui/main.py, openhcs/mcp/context.py, tests/unit/test_custom_function_lifecycle.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/custom_function_management.rst against the later B6 changes to openhcs/processing/custom_functions/manager.py, openhcs/pyqt_gui/main.py, openhcs/mcp/context.py, tests/unit/test_custom_function_lifecycle.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/user_guide/custom_function_management.rst in full and compared it against the changed authorities: custom_functions/manager.py and runtime_registry.py, custom_function_manager_dialog.py, openhcs/pyqt_gui/main.py, endpoint_function_catalog_service.py, openhcs/mcp/context.py, gallery_catalog.py, and the catalogue and lifecycle tests; checked the menu path, register_from_code signature and package export, source validation rules, ephemeral retention on reload, and catalogue invalidation; corrected the decorator requirement.", "Re-read docs/source/user_guide/custom_function_management.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -3911,12 +3954,12 @@ "authority": [ { "path": "openhcs/core/memory/decorators.py", - "sha256": "2740c030304107079f1fcb08c43f40950d0b752cbe10d8412d519b140c7c1b1c", + "sha256": "76f234785f4865257fd19c935e0aace82d1f5586198737fec619ee243b79cbf6", "role": "Exports one OpenHCS memory decorator per MemoryType, including numpy, accepting a contract argument." }, { "path": "openhcs/core/processing_contracts.py", - "sha256": "9b999e9210baf292f8fbf5bb3758c41a95e74401bfb1e1c127bae2dba625b0ee", + "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", "role": "Declares the ProcessingContract classes Pure2DContract, Pure3DContract, VolumetricToSliceContract, and FlexibleContract." }, { @@ -3926,7 +3969,7 @@ }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "00ff3bfe7a032bd3de59274697a69d12c9b898177cd8e839f9da3b130becaedf", + "sha256": "01dcb82b5850f3cdb1c300a2aa360ba330e7f15dffb5a24e37504311095090f9", "role": "Declares the Tools > Custom Functions > Create New Function action." }, { @@ -3936,7 +3979,7 @@ }, { "path": "openhcs/processing/custom_functions/manager.py", - "sha256": "5cd7e955087f0bd9c527fc6307dff48f4aa959c607a3ec776b8379abac76fb36", + "sha256": "0ffa6fc2a18566600dcff33d4b27466d8a5684662fa1c696cb5ea52ea459a1e4", "role": "Validates and registers saved custom-function source into the function catalogue." }, { @@ -3946,7 +3989,7 @@ }, { "path": "tests/unit/test_custom_function_lifecycle.py", - "sha256": "e333004437e29a3e44aeb93699f5fb0b61092bb9c48068e3bc0011781adbc885", + "sha256": "70c87f0dbefd2c1a204d060c2211ec57f4f84023f7c6931d2cb7d81eb82e9252", "role": "Proves validated custom declarations publish atomically without mirrored catalogue state." }, { @@ -3961,6 +4004,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/user_guide/custom_functions.rst against B6's later changes to openhcs/core/memory/decorators.py, openhcs/core/processing_contracts.py, openhcs/pyqt_gui/main.py, openhcs/processing/custom_functions/manager.py, tests/unit/test_custom_function_lifecycle.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/custom_functions.rst against the later B6 changes to openhcs/core/memory/decorators.py, openhcs/core/processing_contracts.py, openhcs/pyqt_gui/main.py, openhcs/processing/custom_functions/manager.py, tests/unit/test_custom_function_lifecycle.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/user_guide/custom_functions.rst in full and compared it against the changed authorities: the page's own earlier contract-import edit, openhcs/core/memory/decorators.py, unified_registry.py, custom_functions/manager.py and runtime_registry.py, external/arraybridge decorators.py, and the catalogue and lifecycle tests; checked the menu path, the decorator export, and the four contract classes in openhcs/core/processing_contracts.py; corrected the contract names in the list.", "On 2026-08-27, re-read docs/source/user_guide/custom_functions.rst in full against callable- and registry-owned form exclusions; challenged the authoring steps and confirmed that custom runtime inputs remain declaration-owned.", @@ -3986,12 +4030,12 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "7b5622b3d82b1e1d672eaedb14e054be81baef417a736adb06bea34fb6708b28", + "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", "role": "Declares DtypeConfig with default_dtype_conversion (NATIVE_OUTPUT default) and StepMaterializationConfig, from which the lazy step forms are generated." }, { "path": "openhcs/core/memory/decorators.py", - "sha256": "2740c030304107079f1fcb08c43f40950d0b752cbe10d8412d519b140c7c1b1c", + "sha256": "76f234785f4865257fd19c935e0aace82d1f5586198737fec619ee243b79cbf6", "role": "Exports OpenHCS memory decorators while delegating conversion semantics to ArrayBridge declarations." }, { @@ -4006,7 +4050,7 @@ }, { "path": "openhcs/processing/backends/processors/numpy_processor.py", - "sha256": "9c3c8cc1d28188897e1f9d95941f1ef53db5daffb4247933816e7e596520f9b5", + "sha256": "58df7297d1021decca547d9f5959b8d8e563544b4644ac4977452030ed8b30da", "role": "Declares the decorated create_projection function used in the 8-bit materialisation example." }, { @@ -4016,7 +4060,7 @@ }, { "path": "tests/unit/test_callable_contract.py", - "sha256": "b0ce0cba2e396e53f8cb9e95533ff918788e8c6dd60037a66e0aeda603bc64c2", + "sha256": "cc220f53107a08b0406b6e0dd49896e3e4c3c0af6e527b0ef00d4fdaba8f0b2c", "role": "Proves callable contract derivation for dimensions, memory types, dtype policy, and outputs." }, { @@ -4028,6 +4072,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/user_guide/dtype_conversion.rst against B6's later changes to openhcs/core/config.py, openhcs/core/memory/decorators.py, openhcs/processing/backends/processors/numpy_processor.py, tests/unit/test_callable_contract.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/dtype_conversion.rst against the later B6 changes to openhcs/core/config.py, openhcs/core/memory/decorators.py, openhcs/processing/backends/processors/numpy_processor.py, tests/unit/test_callable_contract.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/user_guide/dtype_conversion.rst in full and compared it against the changed authorities: openhcs/core/config.py, openhcs/core/memory/decorators.py, external/arraybridge decorators.py and array_operations.py, numpy_processor.py, external/PolyStore disk.py, test_callable_contract.py, and openhcs/core/steps/abstract.py; confirmed DtypeConfig.default_dtype_conversion and its NATIVE_OUTPUT default, the PRESERVE_INPUT and UINT8 members, LazyDtypeConfig and LazyStepMaterializationConfig imports from openhcs.core.config, create_projection, and step dtype_config/step_materialization_config parameters; no page change was needed.", "On 2026-08-27, re-read docs/source/user_guide/dtype_conversion.rst in full after callable contracts began projecting runtime-owned inputs into generic form exclusions; challenged the dtype task flow and found no behaviour change.", @@ -4052,12 +4097,12 @@ "authority": [ { "path": "openhcs/pyqt_gui/main.py", - "sha256": "00ff3bfe7a032bd3de59274697a69d12c9b898177cd8e839f9da3b130becaedf", + "sha256": "01dcb82b5850f3cdb1c300a2aa360ba330e7f15dffb5a24e37504311095090f9", "role": "Declares the Tools > Analysis Consolidation > Run Experimental Analysis action and its directory check for the config and results files." }, { "path": "openhcs/processing/backends/experimental_analysis/analysis.py", - "sha256": "2fe9bcfba30e739cbf0b74112bfec6ce8714f173bdbd08b6dde1e3b6abe78d6e", + "sha256": "17a701e0740eac3e11319fe73f10302bbcab603218349bfba4e07014aabc27bb", "role": "Parses the drug_curve_map and plate_groups sheets and applies replicate-local normalization and heatmap export." }, { @@ -4067,7 +4112,7 @@ }, { "path": "openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py", - "sha256": "bd43ec00c265279585f6bcc9eb0a6b6a2eeedaacad6003a33a60ab181cd13a4d", + "sha256": "2705d459204557c4f7092df3fd21ed687dcf56449ed361485382b1c0ce196444", "role": "Runs the directory workflow and writes the normalized, raw, and heatmap workbooks." }, { @@ -4077,7 +4122,7 @@ }, { "path": "tests/unit/test_experimental_analysis_engine.py", - "sha256": "c79887f97615197c25c340dd277189ebc331bce1f3e9bfec7466c57c92ebd6cf", + "sha256": "641fadecf8c940cc2316a06771c74db5f3f2db9dafdfbfebdc535760d6d776ec", "role": "Runs the documented MetaXpress directory task end to end." } ], @@ -4086,6 +4131,7 @@ ], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/user_guide/experimental_layouts.rst against B6's later changes to openhcs/pyqt_gui/main.py, openhcs/processing/backends/experimental_analysis/analysis.py, openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py, tests/unit/test_experimental_analysis_engine.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/experimental_layouts.rst against the later B6 changes to openhcs/pyqt_gui/main.py, openhcs/processing/backends/experimental_analysis/analysis.py, openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py, tests/unit/test_experimental_analysis_engine.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/user_guide/experimental_layouts.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/main.py, the deleted openhcs/formats/experimental_analysis.py (moved to processing/backends/experimental_analysis/analysis.py), openhcs/core/config.py, unified_analysis_engine.py, and both experimental-analysis tests; checked the menu path, the five default filenames, normalization_method, and that variable_components and group_by remain step-config fields; no page change was needed.", "Re-read docs/source/user_guide/experimental_layouts.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -4112,7 +4158,7 @@ "authority": [ { "path": "openhcs/pyqt_gui/main.py", - "sha256": "00ff3bfe7a032bd3de59274697a69d12c9b898177cd8e839f9da3b130becaedf", + "sha256": "01dcb82b5850f3cdb1c300a2aa360ba330e7f15dffb5a24e37504311095090f9", "role": "Composes the PyQt desktop main window that the openhcs command launches." }, { @@ -4127,18 +4173,19 @@ }, { "path": "openhcs/core/config.py", - "sha256": "7b5622b3d82b1e1d672eaedb14e054be81baef417a736adb06bea34fb6708b28", + "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", "role": "Declares the nested processing, materialization, and streaming configuration dataclasses on PipelineConfig and steps." }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "d1216010dec1804c51aec9c774f298513638ca016d500fe6328d76a9a9bfb4d6", + "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", "role": "Compiles pipeline declarations into typed step plans and a CompiledExecutionBundle before execution." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/user_guide/index.rst against B6's later changes to openhcs/pyqt_gui/main.py, openhcs/core/config.py, openhcs/core/pipeline/compiler.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/index.rst against the later B6 changes to openhcs/pyqt_gui/main.py, openhcs/core/config.py, openhcs/core/pipeline/compiler.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/user_guide/index.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/main.py, test_ui_config_lifecycle.py, pipeline_document.py, and openhcs/core/pipeline/compiler.py; confirmed the Textual interface is still absent, the declaration is pipeline_steps plus PipelineConfig, nested step configuration still owns processing, input source, materialization and streaming options, and compilation still returns a CompiledExecutionBundle; all toctree targets exist; added openhcs/core/config.py for the nested-configuration claim; no page change was needed.", "Re-read docs/source/user_guide/index.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -4177,7 +4224,7 @@ }, { "path": "openhcs/pyqt_gui/services/reactor_providers.py", - "sha256": "43634ee5d4ac6c65d681288c575ee0c9570140d592cb3856942ec69f47c59d90", + "sha256": "aecf1c77636095408486cab53a95cb0fc044fcf2ead0d7b72ac674842cf69abe", "role": "Registers the log, process, status, and execution reactors consumed by desktop widgets." }, { @@ -4187,7 +4234,7 @@ }, { "path": "openhcs/runtime/zmq_execution_client.py", - "sha256": "202619246805f8d2dfbfd5ca5427af34716e424f551ebc800d8516f7b49e07e5", + "sha256": "147fb663d3ea60b1348063440e05b77d54b40634d1ae81cf4fb4f3b5457fbc99", "role": "Spawns the execution server with its own per-port log file and startup status file and reports connection state." }, { @@ -4199,6 +4246,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/user_guide/log_viewer.rst against B6's later changes to openhcs/pyqt_gui/services/reactor_providers.py, openhcs/runtime/zmq_execution_client.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/log_viewer.rst against the later B6 changes to openhcs/pyqt_gui/services/reactor_providers.py, openhcs/runtime/zmq_execution_client.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/user_guide/log_viewer.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/services/reactor_providers.py, openhcs/runtime/zmq_execution_client.py, and scripts/gallery_catalog.py; the reactor changes (axis-family grouping, session scope) and the execution server's per-port log and startup-status files are consistent with the page's claims that subprocesses write dedicated logs reported by the launching process and that readiness is recorded separately; no page change was needed.", "Reviewed docs/source/user_guide/log_viewer.rst in full; the recorded findings and disposition are the resulting page-specific editorial decision.", @@ -4239,7 +4287,7 @@ }, { "path": "openhcs/authoring/session/operations/datasets.py", - "sha256": "43d9edfed59abe6b88e242d39f1ddc8f5078559e04b3875da1da07b993295993", + "sha256": "208274effc2e790775d90528211fa4f640eb01f7ca3fdd9c927c3bf71e39b632", "role": "Declares the add, set-pipeline, initialize, compile, run, and stop dataset operations behind the session tools." }, { @@ -4269,7 +4317,7 @@ }, { "path": "benchmark/cellprofiler_benchmark_cli.py", - "sha256": "edc32916215a88b62702621dea9b692586572284caf9b781394b5784f62897ee", + "sha256": "4025c591644e29f3df61e476ecae2143690e014336ea9f889aadeff4e692445e", "role": "Declares the list-cases, inspect-run, inspect-measured, and run-measured commands, including the required --wait-timeout-ms." }, { @@ -4289,7 +4337,7 @@ }, { "path": "benchmark/control_service.py", - "sha256": "f894cc090597ca04c273090c99f26fd0e3cfeffd1b7d44bbb94858892e17813c", + "sha256": "a1256b349ffdc8d04cab927326d97821462b61f57b29e564b61cc627302be49c", "role": "Requires the manifest and every declared benchmark source location to satisfy the agent read policy." }, { @@ -4304,12 +4352,12 @@ }, { "path": "tests/unit/agent/test_mcp_server.py", - "sha256": "6a03cf2de856060f39d221127f41189a0a8094c65856cb1c4cfa765210099b67", + "sha256": "4e6f6b1761b2c0316c99984baad6ea2c97e5c76c3f155111b8e498d7358e3175", "role": "Proves MCP server tool registration, request handling, and lifecycle behaviour." }, { "path": "tests/unit/test_benchmark_control.py", - "sha256": "2798cf80df1222c80bb3aa9955b5c20c0c90e415a1795017779d22460f442bf2", + "sha256": "093b584eeb37cbb12089d5af082ad45acd779e9d2bdd5b3feaea08234fc22183", "role": "Proves benchmark capability binding, path policy, progress, artifacts, and lightweight imports." } ], @@ -4320,6 +4368,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/user_guide/mcp_clients.rst against B6's later changes to openhcs/authoring/session/operations/datasets.py, benchmark/cellprofiler_benchmark_cli.py, benchmark/control_service.py, tests/unit/agent/test_mcp_server.py, tests/unit/test_benchmark_control.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/mcp_clients.rst against the later B6 changes to openhcs/agent/capabilities.py, openhcs/authoring/session/operations/datasets.py, benchmark/cellprofiler_benchmark_cli.py, benchmark/control_service.py, tests/unit/agent/test_mcp_server.py, tests/unit/test_benchmark_control.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/user_guide/mcp_clients.rst in full and compared it against the changed authorities: the page's own session-tool edit, openhcs/mcp/bootstrap.py, client_registration.py, openhcs/agent/capabilities.py, the mcpb manifest, test_mcp_server.py, gallery_catalog.py, the benchmark CLI, contracts, control and control_service, openhcs/agent/path_policy.py, and test_benchmark_control.py; confirmed every named session tool derives from a declared headless operation, the four surface names and desktop default, the benchmark subcommands and flags, the path-policy variables, and the packaging paths; restored the --wait-timeout-ms requirement and renamed the window.", "Reviewed docs/source/user_guide/mcp_clients.rst in full; the recorded findings and disposition are the resulting page-specific editorial decision.", @@ -4374,7 +4423,7 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "7b5622b3d82b1e1d672eaedb14e054be81baef417a736adb06bea34fb6708b28", + "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", "role": "Declares StreamingDefaults (persistent, host, transport_mode, enabled), partition_filter, and the Napari and Fiji streaming configs with their ports." }, { @@ -4394,7 +4443,7 @@ }, { "path": "openhcs/runtime/viewer_protocol.py", - "sha256": "f126aaf4f5c0dcc9429b7aae8ce67985260241ec7d2f3d5a948573658cf079a3", + "sha256": "c4bdae4bae6dee412f0d27a9cf75a95db53b3bd95c03cee7d046be16b52fe5b8", "role": "Declares the transport-independent viewer messages and incremental settlement-progress fields." }, { @@ -4424,22 +4473,22 @@ }, { "path": "openhcs/runtime/fiji_viewer_server.py", - "sha256": "d3c2114d16f71111562474b17a5ef4a54a7391275293c4678259e4410d11d15f", + "sha256": "ee246d32ed828bbf4665cc8a411e807da29bd43c2475012243f27e93cb550410", "role": "Implements Fiji stream reception, image-stack updates, ROI handling, and endpoint lifecycle." }, { "path": "tests/unit/test_napari_streaming_handlers.py", - "sha256": "860e234a39f36f143491e6f1edc9515c1d9cde88590759252378f8d23c0bab12", + "sha256": "93244d4d09c5cd6d7a2645fca2482578ad52646f3bb9e6de5ebe505b157fbfea", "role": "Proves Napari streaming updates layers and carries ROI navigation through the declared protocol." }, { "path": "tests/unit/test_viewer_result_element_selection.py", - "sha256": "a7b348fab106da4c89e58ed32225495aa27792132ef7488657f29821dfd6da92", + "sha256": "b1dae580f0291f53be9453df6f6c6df5458046bd3ff947b550f7ddea94b29f96", "role": "Proves focused layer isolation is rejected before mutation when any route or axis is invalid." }, { "path": "tests/unit/test_fiji_viewer_server.py", - "sha256": "bed61ba6a34937e7827b5965a63ca1afe3874d6ecc27b3c8a32721628e8b4e36", + "sha256": "da575dfb02895ecc831b34d6bdf6b58112b986556459444e9701cbf43221ea22", "role": "Proves Fiji endpoint lifecycle, streamed image updates, and ROI message handling." }, { @@ -4454,6 +4503,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/user_guide/real_time_visualization.rst against B6's later changes to openhcs/core/config.py, openhcs/runtime/viewer_protocol.py, openhcs/runtime/fiji_viewer_server.py, tests/unit/test_napari_streaming_handlers.py, tests/unit/test_viewer_result_element_selection.py, tests/unit/test_fiji_viewer_server.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/real_time_visualization.rst against the later B6 changes to openhcs/core/config.py, pyproject.toml, openhcs/runtime/viewer_protocol.py, openhcs/runtime/fiji_viewer_server.py, tests/unit/test_napari_streaming_handlers.py, tests/unit/test_viewer_result_element_selection.py, tests/unit/test_fiji_viewer_server.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/user_guide/real_time_visualization.rst in full and compared it against the changed authorities: the page's own partition_filter edit, openhcs/core/config.py, viewer_protocol.py, viewer_controls.py, viewer_window_service.py, napari_viewer_server.py, fiji_viewer_server.py, napari_streaming_handlers.py, the three viewer tests, and gallery_catalog.py; confirmed the extras, the per-step streaming fields, persistent and partition_filter, max_rois, the selected-layer overlay controller, and forward-progress settlement; corrected the step-3 option list.", "Re-read docs/source/user_guide/real_time_visualization.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", From 35833f3a5dcc234372d87c0e873d5372074d83b3 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 04:28:55 -0400 Subject: [PATCH 43/84] B6: place packaged native recipes by case name within the Windows path budget The installed knowledge projection placed each native .cppipe under its upstream root/path layout, producing installed paths past MAX_PATH (285 chars under the installer CI root) that Windows without long paths cannot open, so installed MCP health reported packaged resources missing. Placement is now .sources/.cppipe (case names are manifest-unique and validated as portable file names), and the build rejects any resource path over PACKAGED_KNOWLEDGE_PATH_BUDGET. A guard test checks every packaged resource under the installer CI root stays below 260 characters. Co-Authored-By: Claude Opus 5.5 --- openhcs/agent/knowledge_manifest_schema.py | 33 ++++++++----- scripts/build_mcp_knowledge_assets.py | 42 +++++++++++----- tests/unit/test_build_mcp_knowledge_assets.py | 48 ++++++++++++++++++- 3 files changed, 98 insertions(+), 25 deletions(-) diff --git a/openhcs/agent/knowledge_manifest_schema.py b/openhcs/agent/knowledge_manifest_schema.py index 9dd6751aaf..845b93794d 100644 --- a/openhcs/agent/knowledge_manifest_schema.py +++ b/openhcs/agent/knowledge_manifest_schema.py @@ -4,6 +4,7 @@ import json import os +import re from collections.abc import Mapping from dataclasses import dataclass from enum import Enum @@ -13,6 +14,12 @@ "docs/source/development/mcp_knowledge_base_manifest.json" ) PACKAGED_KNOWLEDGE_BASE_ROOT = Path("resources/knowledge") +# Windows without long-path support rejects installed paths of 260 characters +# or more. The installer places the knowledge root at +# /env-XXXXXXXX/Lib/site-packages/openhcs/agent/resources/knowledge, +# so every resource path relative to that root must stay within this budget. +PACKAGED_KNOWLEDGE_PATH_BUDGET = 120 +_PORTABLE_CASE_NAME = re.compile(r"[A-Za-z0-9_](?:[A-Za-z0-9_ .-]*[A-Za-z0-9_])?") class KnowledgeBaseManifestField(str, Enum): @@ -145,21 +152,21 @@ def load(cls, path: Path, *, source_root: Path) -> ComparisonManifestSnapshot: return manifest def native_source_relative_path(self, case: Mapping[str, object]) -> Path: - """Derive one placement from its original manifest/root/path identity.""" + """Place one native recipe by its manifest-unique case name. + + Upstream root/path layouts are deep; keying by case name keeps every + installed resource within ``PACKAGED_KNOWLEDGE_PATH_BUDGET``. + """ path = Path(str(case["cppipe_path"])) - root = Path(str(case.get("cppipe_path_root", "unrooted"))) - if ( - path.is_absolute() - or root.is_absolute() - or ".." in path.parts - or ".." in root.parts - or path.suffix != ".cppipe" - ): + name = str(case["name"]) + if path.is_absolute() or ".." in path.parts or path.suffix != ".cppipe": raise ValueError("Native knowledge source must be a relative .cppipe path.") - return ( - self.path.relative_to(self.source_root).with_suffix(".sources") - / root - / path + if not _PORTABLE_CASE_NAME.fullmatch(name): + raise ValueError( + f"Native knowledge case name {name!r} is not a portable file name." + ) + return self.path.relative_to(self.source_root).with_suffix(".sources") / ( + f"{name}.cppipe" ) def native_source_path(self, case: Mapping[str, object]) -> Path: diff --git a/scripts/build_mcp_knowledge_assets.py b/scripts/build_mcp_knowledge_assets.py index 8ed94107b1..59f8660cb7 100644 --- a/scripts/build_mcp_knowledge_assets.py +++ b/scripts/build_mcp_knowledge_assets.py @@ -32,6 +32,7 @@ def _build_module(relative_path: str) -> SimpleNamespace: ManifestRootAcquisitionKind = _UPSTREAM_SOURCES.ManifestRootAcquisitionKind ManifestRootAcquisitionSpec = _UPSTREAM_SOURCES.ManifestRootAcquisitionSpec KNOWLEDGE_MANIFEST_RELATIVE_PATH = _MANIFEST_SCHEMA.DEFAULT_KNOWLEDGE_BASE_MANIFEST_PATH +PACKAGED_KNOWLEDGE_PATH_BUDGET = _MANIFEST_SCHEMA.PACKAGED_KNOWLEDGE_PATH_BUDGET PACKAGED_KNOWLEDGE_ROOT_RELATIVE_PATH = ( Path("openhcs/agent") / _MANIFEST_SCHEMA.PACKAGED_KNOWLEDGE_BASE_ROOT ) @@ -191,6 +192,35 @@ def project_knowledge_assets( ) +def knowledge_asset_projections( + project_root: Path, + recipe_type: type[ComparisonManifestSnapshot] = ComparisonManifestSnapshot, +) -> dict[Path, Path]: + """Packaged resource path -> canonical source, within the path budget.""" + projections = knowledge_source_projections( + project_root / KNOWLEDGE_MANIFEST_RELATIVE_PATH, + source_root=project_root, + recipe_type=recipe_type, + ) + plugin_manifest = project_root / AGENT_PLUGIN_MANIFEST_PATH + if plugin_manifest.is_file(): + skill_sources = AgentSkillBundle.from_manifest(plugin_manifest).source_paths() + projections.update( + {source.relative_to(project_root): source for source in skill_sources} + ) + over_budget = tuple( + relative.as_posix() + for relative in projections + if len(relative.as_posix()) > PACKAGED_KNOWLEDGE_PATH_BUDGET + ) + if over_budget: + raise ValueError( + f"MCP knowledge resource paths exceed {PACKAGED_KNOWLEDGE_PATH_BUDGET} " + f"characters: {over_budget}" + ) + return projections + + def _project_knowledge_assets( project_root: Path, destination_root: Path, @@ -207,17 +237,7 @@ def _project_knowledge_assets( raise ValueError( f"MCP knowledge destination must not own the project root: {destination}" ) - projections = knowledge_source_projections( - root / KNOWLEDGE_MANIFEST_RELATIVE_PATH, - source_root=root, - recipe_type=native_sources.recipe_type(), - ) - plugin_manifest = root / AGENT_PLUGIN_MANIFEST_PATH - if plugin_manifest.is_file(): - skill_sources = AgentSkillBundle.from_manifest(plugin_manifest).source_paths() - projections.update( - {source.relative_to(root): source for source in skill_sources} - ) + projections = knowledge_asset_projections(root, native_sources.recipe_type()) native_sources.materialize() missing = tuple(source for source in projections.values() if not source.is_file()) if missing: diff --git a/tests/unit/test_build_mcp_knowledge_assets.py b/tests/unit/test_build_mcp_knowledge_assets.py index aa05965926..07adb739b0 100644 --- a/tests/unit/test_build_mcp_knowledge_assets.py +++ b/tests/unit/test_build_mcp_knowledge_assets.py @@ -3,13 +3,15 @@ import glob import json import tomllib -from pathlib import Path +from pathlib import Path, PureWindowsPath import pytest from scripts.build_mcp_knowledge_assets import ( KNOWLEDGE_MANIFEST_RELATIVE_PATH, + PACKAGED_KNOWLEDGE_PATH_BUDGET, PACKAGED_KNOWLEDGE_ROOT_RELATIVE_PATH, + knowledge_asset_projections, project_knowledge_assets, ) from openhcs.agent.knowledge_manifest_schema import ( @@ -294,3 +296,47 @@ def test_wheel_globs_include_the_projected_hidden_plugin_manifest(tmp_path): assert plugin_manifest in included plugin_root = plugin_manifest.parent.parent assert {path for path in projected if path.is_relative_to(plugin_root)} <= included + + +PROJECT_ROOT = Path(__file__).resolve().parents[2] +# The installer CI proof root under the runner profile, with long paths disabled. +WINDOWS_INSTALLED_KNOWLEDGE_ROOT = PureWindowsPath( + r"C:\Users\runneradmin\AppData\Local\OpenHCS Installer Smoke", + "env-00000000", + "Lib", + "site-packages", + *PACKAGED_KNOWLEDGE_ROOT_RELATIVE_PATH.parts, +) +WINDOWS_MAX_PATH = 260 + + +def test_every_packaged_resource_fits_the_windows_installed_path_budget(): + workflow = PROJECT_ROOT / ".github" / "workflows" / "integration-tests.yml" + assert '"OpenHCS Installer Smoke"' in workflow.read_text(encoding="utf-8") + root_length = len(str(WINDOWS_INSTALLED_KNOWLEDGE_ROOT)) + 1 + assert root_length + PACKAGED_KNOWLEDGE_PATH_BUDGET < WINDOWS_MAX_PATH + projections = knowledge_asset_projections(PROJECT_ROOT) + assert any(path.suffix == ".cppipe" for path in projections) + too_long = [ + str(WINDOWS_INSTALLED_KNOWLEDGE_ROOT / relative) + for relative in projections + if len(str(WINDOWS_INSTALLED_KNOWLEDGE_ROOT / relative)) >= WINDOWS_MAX_PATH + ] + assert too_long == [] + + +@pytest.mark.parametrize( + "case_name", ["x" * PACKAGED_KNOWLEDGE_PATH_BUDGET, "nested/name", "name:stream"] +) +def test_native_placement_rejects_unbudgeted_case_names(tmp_path, case_name): + project_root, manifest_path, _ = _project_with_document(tmp_path) + recipe_path = project_root / "recipe.json" + recipe_path.write_text( + json.dumps({"cases": [{"name": case_name, "cppipe_path": "a.cppipe"}]}), + encoding="utf-8", + ) + catalog = json.loads(manifest_path.read_text(encoding="utf-8")) + catalog["documents"].append({"source_path": "recipe.json"}) + manifest_path.write_text(json.dumps(catalog), encoding="utf-8") + with pytest.raises(ValueError): + knowledge_asset_projections(project_root) From ded24279ef7354540a844d518d7df97b20588e40 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 04:43:27 -0400 Subject: [PATCH 44/84] B6: macOS kernel cache preparation uses its online CPUs e55e2ec0b restored compiling every registered kernel before the execution server reports ready, but ffba8426c admits child cache workers only through os.sched_getaffinity, which macOS lacks, so macOS compiled every kernel serially in the server. On the macos-15-intel runner the cold catalog then took past the 300 s installed-GUI acceptance deadline (discovery 113 s, serial kernel compilation still running at 290 s). macOS has no affinity API: every online CPU is schedulable, so its capacity is os.cpu_count(); Linux keeps its affinity set and Windows (no fork) is unchanged. Co-Authored-By: Claude Opus 5.5 --- openhcs/core/processing_preparation.py | 21 +++++++++++++-------- tests/unit/test_preparation_admission.py | 8 +++++--- 2 files changed, 18 insertions(+), 11 deletions(-) diff --git a/openhcs/core/processing_preparation.py b/openhcs/core/processing_preparation.py index 2e377b736e..5bf135cc2f 100644 --- a/openhcs/core/processing_preparation.py +++ b/openhcs/core/processing_preparation.py @@ -310,6 +310,18 @@ def close(self) -> None: self.result_connection.close() +def _schedulable_cpu_count() -> int: + """CPUs this process may run on. + + Linux exposes the scheduler's affinity set. macOS has no affinity API: + every online CPU is schedulable there. + """ + try: + return len(os.sched_getaffinity(0)) + except AttributeError: + return os.cpu_count() or 1 + + @dataclass(frozen=True, slots=True) class PreparationCacheBatch: """Schedule derived cache operations without interpreting backend families.""" @@ -338,14 +350,7 @@ def populate_child_caches( raise ValueError("Preparation worker budget must be positive.") if "fork" not in multiprocessing.get_all_start_methods(): return - try: - worker_capacity = min(max_workers, len(os.sched_getaffinity(0))) - except AttributeError: - # Without an affinity witness there is no admitted parallel cache - # work. The existing parent preparation still owns readiness. - return - if worker_capacity < 1: - return + worker_capacity = min(max_workers, _schedulable_cpu_count()) operations = { operation.identity: operation for preparation in self.preparations diff --git a/tests/unit/test_preparation_admission.py b/tests/unit/test_preparation_admission.py index a5ee6049aa..076f3a9b1a 100644 --- a/tests/unit/test_preparation_admission.py +++ b/tests/unit/test_preparation_admission.py @@ -242,14 +242,16 @@ def prepare_selected(): assert events == ["selected", "unselected", "dynamic"] -def test_unavailable_affinity_does_not_admit_speculative_parallelism(monkeypatch): +def test_platform_without_affinity_admits_its_online_cpus(monkeypatch): events = [] - started, _, _ = simulated_workers(monkeypatch, 8) + started, peak, _ = simulated_workers(monkeypatch, 8) monkeypatch.delattr(os, "sched_getaffinity") + monkeypatch.setattr(os, "cpu_count", lambda: 2) PreparationCacheBatch( tuple(DeclaredCache(str(i), events) for i in range(3)) ).populate_child_caches(max_workers=8) - assert not started and not events + assert started == [("0",), ("1",), ("2",)] + assert max(peak) == 2 def test_affinity_query_failure_is_not_misreported_as_success(monkeypatch): From e1bd3435a39a6489817f7157706979db4207f156 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 04:46:48 -0400 Subject: [PATCH 45/84] B6: image-source guide test reads the axis-family spellings the guide now uses Co-Authored-By: Claude Opus 5.5 --- .../unit/agent/test_knowledge_base_service.py | 25 +++++++++++-------- 1 file changed, 15 insertions(+), 10 deletions(-) diff --git a/tests/unit/agent/test_knowledge_base_service.py b/tests/unit/agent/test_knowledge_base_service.py index 0274bb487c..c05ec9cdee 100644 --- a/tests/unit/agent/test_knowledge_base_service.py +++ b/tests/unit/agent/test_knowledge_base_service.py @@ -128,8 +128,7 @@ def test_search_prefers_declared_document_matches_over_equal_body_matches(tmp_pa (docs / "notes.md").write_text( "# Setup notes\n\n" + "\n".join( - f"## Note {index}\n\nThis body mentions a recipe.\n" - for index in range(8) + f"## Note {index}\n\nThis body mentions a recipe.\n" for index in range(8) ), encoding="utf-8", ) @@ -314,14 +313,19 @@ def test_knowledge_base_retrieves_canonical_image_source_guide(): assert "CZI" in document.content assert "OME-TIFF" in document.content assert "OME-Zarr" in document.content - assert "WELL" in document.content - assert "SITE" in document.content - assert "CHANNEL" in document.content - assert "Z_INDEX" in document.content - assert "TIMEPOINT" in document.content + from openhcs.domains.microscopy.axes import Microscopy + + for axis in ( + Microscopy.Well, + Microscopy.Site, + Microscopy.Channel, + Microscopy.ZIndex, + Microscopy.Timepoint, + ): + assert f"Microscopy.{axis.__name__}" in document.content assert "Binding positions are not microscope channel values" in document.content assert "full named physical" in document.content - assert "inputs consumed by the nominal handler projection" in document.content + assert "inputs consumed by the dataset source projection" in document.content assert "Seeing all three in the source workspace inventory is valid" in ( document.content ) @@ -348,8 +352,9 @@ def test_knowledge_base_retrieves_canonical_image_source_guide(): assert "``neurite_channel_index=0``" in document.content assert "``nuclear_stain.channel_index=1``" in document.content assert "not equivalent to MAP2-seeded analysis" in document.content - assert "``variable_components=[CHANNEL]`` assembles the channel stack" in ( - document.content + assert ( + "``variable_components=[Microscopy.Channel]`` assembles the channel stack" + in (document.content) ) assert "compiler normalizes it to" in document.content assert "``Ungrouped``" in document.content From 956326e3e7fd7873cde2a7e33fdc3f55bb1e9fee Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 04:54:51 -0400 Subject: [PATCH 46/84] B6: metadata-document targets admit only backends with a local output root The OpenHCS metadata document is written with a local file lock beside the output dataset root. Image targets excluded OMERO by a backend-name tuple, but the runtime-artifact target (added in cba1c90df) admitted any persistent backend, so OMERO-persisted results reached the disk writer at /omero/_outputs once publication moved to the receiving boundary. All three targets now ask the backend's own declaration: a polystore VirtualBackend (OMERO) names remote objects and has no local root. Co-Authored-By: Claude Opus 5.5 --- openhcs/core/steps/function_outputs.py | 81 ++++++--- tests/unit/test_function_outputs.py | 221 ++++++++++++++++++------- 2 files changed, 221 insertions(+), 81 deletions(-) diff --git a/openhcs/core/steps/function_outputs.py b/openhcs/core/steps/function_outputs.py index 7361f8a384..de15a2d680 100644 --- a/openhcs/core/steps/function_outputs.py +++ b/openhcs/core/steps/function_outputs.py @@ -17,8 +17,13 @@ RegistryFamily, extract_key_from_class_name, ) +from polystore.backend_registry import STORAGE_BACKENDS +from polystore.base import VirtualBackend from polystore.streaming.identity import StreamProducerIdentity -from polystore.streaming.viewer_transport import ViewerStreamProducer, ViewerStreamSourceMetadata +from polystore.streaming.viewer_transport import ( + ViewerStreamProducer, + ViewerStreamSourceMetadata, +) from polystore.virtual_workspace import SourcePixelRef from openhcs.constants.constants import Backend @@ -513,9 +518,7 @@ def partition_by_item_fields( def project_item( request: RuntimeProjectionSourceIdentityRequest, ) -> tuple[RuntimeProjectedPayloadItem, ...]: - return ( - RequiredSourceComponentMetadata - ).project_payload_items(request) + return (RequiredSourceComponentMetadata).project_payload_items(request) @staticmethod def stream_path_for_projected_item( @@ -664,6 +667,15 @@ class OpenHCSMetadataTarget(ABC, metaclass=AutoRegisterMeta): def from_plan(cls, plan: CompiledStepPlan) -> OpenHCSMetadataTarget | None: """Resolve this declaration's independently compiled storage target.""" + @staticmethod + def hosts_metadata_document(backend: str) -> bool: + """The metadata document lives in the output root's local directory. + + A virtual backend (for example OMERO) declares that its paths name + remote objects, so its output root has no directory to hold it. + """ + return not issubclass(STORAGE_BACKENDS[backend], VirtualBackend) + @classmethod def for_plan(cls, plan: CompiledStepPlan) -> tuple[OpenHCSMetadataTarget, ...]: """Derive targets from their declarations, without a consumer roster.""" @@ -693,7 +705,8 @@ def for_execution( for declaration in cls.__registry__.values() if (owner := declaration.from_execution(context, plan)) is not None for target in replace( - owner, artifact_materializations=artifact_materializations, + owner, + artifact_materializations=artifact_materializations, create_openhcs_metadata=plan.create_openhcs_metadata, ).production_targets(context, plan) ) @@ -711,7 +724,9 @@ def reconciliation_targets( context: ProcessingContext, *, document: OpenHCSMetadataSubdirectories | None = None, - accepted_entries: Mapping[str, VirtualWorkspaceSourceProjectionEntries] | None = None, + accepted_entries: ( + Mapping[str, VirtualWorkspaceSourceProjectionEntries] | None + ) = None, ) -> tuple[OpenHCSMetadataTarget, ...]: """Resolve destinations after runtime values have been released.""" return (self,) @@ -783,7 +798,9 @@ def write( str(self.output_dir), self.backend ) ) - return (metadata_writer or AtomicMetadataWriter()).publish_source_projection_metadata( + return ( + metadata_writer or AtomicMetadataWriter() + ).publish_source_projection_metadata( METADATA_CONFIG.metadata_path(self.dataset_root), self.sub_dir, projection_entries, @@ -817,7 +834,8 @@ def produced_projection_entries( raise ValueError("OpenHCS metadata requires a file manager.") target = type(self).from_plan(plan) if target is None or self not in replace( - target, artifact_materializations=self.artifact_materializations, + target, + artifact_materializations=self.artifact_materializations, create_openhcs_metadata=self.create_openhcs_metadata, ).production_targets(context, plan): raise ValueError("Produced metadata plan does not own this output target.") @@ -1064,11 +1082,19 @@ def finalize_completed_dataset( for target in cls.for_plan(plan): target_contexts.setdefault(target, context) - produced_by_document: dict[Path, dict[OpenHCSMetadataTarget, list[VirtualWorkspaceSourceProjectionEntries]]] = {} + produced_by_document: dict[ + Path, + dict[OpenHCSMetadataTarget, list[VirtualWorkspaceSourceProjectionEntries]], + ] = {} for observation in runtime_observations: for context_observation in observation.contexts: context = compiled_contexts[context_observation.context_key] - for target, entries in context_observation.outputs.source_projection_entries_by_target.items(): + for ( + target, + entries, + ) in ( + context_observation.outputs.source_projection_entries_by_target.items() + ): produced_by_document.setdefault( METADATA_CONFIG.metadata_path(target.dataset_root), {} ).setdefault(target, []).append(entries) @@ -1082,13 +1108,17 @@ def finalize_completed_dataset( targets_by_document.setdefault( METADATA_CONFIG.metadata_path(owner.dataset_root), {} )[owner] = context - for metadata_path in dict.fromkeys((*targets_by_document, *produced_by_document)): + for metadata_path in dict.fromkeys( + (*targets_by_document, *produced_by_document) + ): AtomicMetadataWriter().reconcile_completed_dataset( metadata_path, targets_by_document.get(metadata_path, {}), produced_entries_by_target={ target: VirtualWorkspaceSourceProjectionEntries.combine(entries) - for target, entries in produced_by_document.get(metadata_path, {}).items() + for target, entries in produced_by_document.get( + metadata_path, {} + ).items() }, ) @@ -1119,12 +1149,14 @@ def from_execution( @classmethod def from_plan(cls, plan: CompiledStepPlan) -> PrimaryImageMetadataTarget | None: - if plan.write_backend in (Backend.OMERO_LOCAL.value, Backend.MEMORY.value): + if plan.write_backend == Backend.MEMORY.value: return None if plan.write_backend is None: raise ValueError( f"Step {plan.step_index} ({plan.step_name}) has no write backend." ) + if not cls.hosts_metadata_document(plan.write_backend): + return None if plan.output_dir is None: raise ValueError( f"Step {plan.step_index} ({plan.step_name}) has no output directory." @@ -1153,10 +1185,7 @@ def from_plan( cls, plan: CompiledStepPlan ) -> MaterializedImageMetadataTarget | None: output = plan.materialized_output - if output is None or output.backend in ( - Backend.OMERO_LOCAL.value, - Backend.MEMORY.value, - ): + if output is None or not cls.hosts_metadata_document(output.backend): return None return cls( output_dir=output.output_dir, @@ -1193,17 +1222,22 @@ def reconciliation_targets( context: ProcessingContext, *, document: OpenHCSMetadataSubdirectories | None = None, - accepted_entries: Mapping[str, VirtualWorkspaceSourceProjectionEntries] | None = None, + accepted_entries: ( + Mapping[str, VirtualWorkspaceSourceProjectionEntries] | None + ) = None, ) -> tuple[RuntimeArtifactMetadataTarget, ...]: """Use durable typed projections, without reloading cleaned artifact values.""" from openhcs.core.dataset_sources.openhcs_format import OpenHCSMetadataHandler if (document is None) != (accepted_entries is None): - raise ValueError("Reconciliation requires one document and its admitted entries.") + raise ValueError( + "Reconciliation requires one document and its admitted entries." + ) handler = OpenHCSMetadataHandler(context.filemanager) directories = ( handler.reconciliation_directories(self.dataset_root, self.backend) - if document is None else handler.reconciliation_directories_from_document( + if document is None + else handler.reconciliation_directories_from_document( self.dataset_root, self.backend, document, accepted_entries ) ) @@ -1227,7 +1261,12 @@ def for_directory(self, directory: Path) -> RuntimeArtifactMetadataTarget: @classmethod def from_plan(cls, plan: CompiledStepPlan) -> RuntimeArtifactMetadataTarget | None: materialization = plan.runtime_artifact_materialization - if not materialization.has_persistent_target: + if not ( + materialization.has_persistent_target + and cls.hosts_metadata_document( + materialization.require_persistent_backend() + ) + ): return None output_dir = plan.artifact_analysis_output_dir dataset_root = plan.artifact_output_dataset_root diff --git a/tests/unit/test_function_outputs.py b/tests/unit/test_function_outputs.py index d14c6c2f0d..779ef3824b 100644 --- a/tests/unit/test_function_outputs.py +++ b/tests/unit/test_function_outputs.py @@ -75,12 +75,15 @@ MaterializedRuntimeArtifact, RuntimeArtifactMaterialization, ) -from openhcs.core.steps.function_artifact_materialization import ArtifactMaterializationTargetPlan +from openhcs.core.steps.function_artifact_materialization import ( + ArtifactMaterializationTargetPlan, +) from openhcs.core.steps.function_outputs import ( MaterializedImageOutputWriter, MemoryOutputWriter, OpenHCSMetadataTarget, MaterializedImageMetadataTarget, + PrimaryImageMetadataTarget, RuntimeArtifactMetadataTarget, StreamOutputsAuthority, StreamOutputBatch, @@ -123,7 +126,8 @@ def _publish_saved_step(context, plan, *, artifact_materializations=()): METADATA_CONFIG.metadata_path(target.dataset_root) for target in facts ): entries = { - target: value for target, value in facts.items() + target: value + for target, value in facts.items() if METADATA_CONFIG.metadata_path(target.dataset_root) == metadata_path } AtomicMetadataWriter().reconcile_completed_dataset( @@ -174,7 +178,8 @@ def test_function_step_metadata_follows_runtime_artifact_persistence( authority, method_name, lambda _context, _plan, event=event, **_kwargs: ( - events.append(event), {} if event == "metadata" else () + events.append(event), + {} if event == "metadata" else (), )[1], ) @@ -189,6 +194,62 @@ def test_function_step_metadata_follows_runtime_artifact_persistence( ] +@pytest.mark.parametrize( + ("backend", "expected_targets"), + [ + ( + Backend.DISK.value, + { + PrimaryImageMetadataTarget, + MaterializedImageMetadataTarget, + RuntimeArtifactMetadataTarget, + }, + ), + (Backend.OMERO_LOCAL.value, set()), + ], +) +def test_metadata_document_targets_follow_backend_declaration( + monkeypatch, backend, expected_targets +) -> None: + """A virtual backend's output root holds no local metadata document.""" + root = "/omero/plate_1_outputs" + plan = CompiledStepPlan( + step_index=0, + step_name="count", + axis_id="A01", + output_dir=Path(root) / "images", + output_dataset_root=root, + sub_dir="images", + analysis_results_dir=f"{root}/results", + write_backend=backend, + materialized_output=MaterializedOutputPlan( + output_dir=Path(root) / "checkpoint", + backend=backend, + dataset_root=root, + sub_dir="checkpoint", + analysis_results_dir=f"{root}/results", + ), + runtime_artifact_materialization=RuntimeArtifactMaterializationPlan( + persistent_enabled=True, persistent_backend=backend + ), + create_openhcs_metadata=True, + ) + assert {type(t) for t in OpenHCSMetadataTarget.for_plan(plan)} == expected_targets + + documents = [] + monkeypatch.setattr( + AtomicMetadataWriter, + "reconcile_completed_dataset", + lambda _self, metadata_path, *_args, **_kwargs: documents.append(metadata_path), + ) + OpenHCSMetadataTarget.finalize_completed_dataset( + {"A01": SimpleNamespace(step_plans={0: plan})} + ) + assert documents == ( + [METADATA_CONFIG.metadata_path(root)] if expected_targets else [] + ) + + def test_memory_output_writer_rejects_payload_path_cardinality_mismatch(): with pytest.raises(ValueError, match="1 payloads for 0 paths"): MemoryOutputWriter.payloads( @@ -305,7 +366,10 @@ def parse_filename(self, name): } ) return FilenameParseResult( - ((component, metadata.get(component.name)) for component in AxisFamily.active().axes), + ( + (component, metadata.get(component.name)) + for component in AxisFamily.active().axes + ), extension=str(metadata["extension"]), ) @@ -510,7 +574,9 @@ def test_memory_output_writer_materializes_preserved_source_at_current_destinati @pytest.mark.parametrize("source_extension", (None, ".czi")) -def test_function_output_identity_preserves_non_axis_source_metadata(source_extension) -> None: +def test_function_output_identity_preserves_non_axis_source_metadata( + source_extension, +) -> None: identity = FunctionOutputIdentity( component_values={"channel": "2", "site": "1"}, extension=".tif", @@ -518,10 +584,10 @@ def test_function_output_identity_preserves_non_axis_source_metadata(source_exte ) source_metadata = { - "Run": "Sequence1", - "Specimen": "DrosophilaEmbryo", - "ChannelNumber": "1", - "OpenHCSOriginalSourceMetadata": {"FrameNumber": "0007"}, + "Run": "Sequence1", + "Specimen": "DrosophilaEmbryo", + "ChannelNumber": "1", + "OpenHCSOriginalSourceMetadata": {"FrameNumber": "0007"}, } if source_extension is not None: source_metadata["extension"] = source_extension @@ -536,7 +602,9 @@ def test_function_output_identity_preserves_non_axis_source_metadata(source_exte **({"extension": source_extension} if source_extension is not None else {}), } assert identity.filename_component_metadata() == { - "channel": "2", "site": "1", "extension": ".tif", + "channel": "2", + "site": "1", + "extension": ".tif", } assert source_metadata["ChannelNumber"] == "1" assert source_metadata.get("extension") == source_extension @@ -1665,7 +1733,9 @@ def test_produced_projection_metadata_persists_typed_collapsed_semantics( assert len(restored.source_provenance.represented_source_identities) == 2 assert record["address"] is None assert record["execution_scope"]["fixed_component_values"] == [ - ["channel", "1"], ["z_index", "1"], ["timepoint", "1"], + ["channel", "1"], + ["z_index", "1"], + ["timepoint", "1"], ] assert "site" not in record["source_metadata"] assert "site" not in subdirectory[FIELDS.SOURCE_METADATA]["images/A01_s1_w1.tif"] @@ -1944,7 +2014,10 @@ def parse_filename(self, name): } ) return FilenameParseResult( - ((component, metadata.get(component.name)) for component in AxisFamily.active().axes), + ( + (component, metadata.get(component.name)) + for component in AxisFamily.active().axes + ), extension=str(metadata["extension"]), ) @@ -2084,9 +2157,7 @@ def test_produced_address_publication_never_parses_generated_filenames( }, source_voxel_spacing=SourceVoxelSpacing((0.5, 0.5)), ) - identity = FunctionOutputIdentity.from_metadata( - parser, metadata - ) + identity = FunctionOutputIdentity.from_metadata(parser, metadata) assert identity is not None assert identity.extension == extension identity = identity.with_filename_qualifier("centre_dots") @@ -2258,11 +2329,15 @@ def test_completed_plate_metadata_includes_outputs_written_after_owner_axis( follower_plan.create_openhcs_metadata = True filemanager.save(first_pixels, str(later_image), Backend.MEMORY.value) record_output_path(follower_context, follower_plan, str(later_image)) - later_facts = OpenHCSMetadataTarget.observe_for_step(follower_context, follower_plan) + later_facts = OpenHCSMetadataTarget.observe_for_step( + follower_context, follower_plan + ) combined_facts = { target: VirtualWorkspaceSourceProjectionEntries.combine( - (first_facts.get(target, VirtualWorkspaceSourceProjectionEntries({})), - later_facts.get(target, VirtualWorkspaceSourceProjectionEntries({}))) + ( + first_facts.get(target, VirtualWorkspaceSourceProjectionEntries({})), + later_facts.get(target, VirtualWorkspaceSourceProjectionEntries({})), + ) ) for target in dict.fromkeys((*first_facts, *later_facts)) } @@ -2389,7 +2464,9 @@ def test_runtime_image_metadata_target_requires_persisted_images(tmp_path, conte context, plan, artifact_materializations=saved_artifacts ) if contents == "images": - assert selected == (replace(target, create_openhcs_metadata=plan.create_openhcs_metadata),) + assert selected == ( + replace(target, create_openhcs_metadata=plan.create_openhcs_metadata), + ) else: assert selected == () plan.create_openhcs_metadata = True @@ -2459,12 +2536,8 @@ def test_declared_image_destinations_publish_and_reconcile_after_value_cleanup( path=output_plan.path, backend=Backend.MEMORY.value, ) - materializations = ArtifactMaterializationTargetPlan.materialize( - context, plan - ) - _publish_saved_step( - context, plan, artifact_materializations=materializations - ) + materializations = ArtifactMaterializationTargetPlan.materialize(context, plan) + _publish_saved_step(context, plan, artifact_materializations=materializations) context.runtime_value_store.clear() if stray_image: tifffile.imwrite(tmp_path / "images/review/unowned.tif", pixels) @@ -2556,9 +2629,7 @@ def test_actual_array_exports_only_publish_declared_raster_inventory( backend=Backend.MEMORY.value, ) - materializations = ArtifactMaterializationTargetPlan.materialize( - context, plan - ) + materializations = ArtifactMaterializationTargetPlan.materialize(context, plan) saved_outputs = tuple( output for artifact in materializations @@ -2576,15 +2647,15 @@ def test_actual_array_exports_only_publish_declared_raster_inventory( location for artifact in materializations for locations in ( - artifact.observation(plan, context).materialized_locations_by_address.values() + artifact.observation( + plan, context + ).materialized_locations_by_address.values() ) for location in locations ) assert {Path(location.path) for location in observed_locations} == expected_paths - _publish_saved_step( - context, plan, artifact_materializations=materializations - ) + _publish_saved_step(context, plan, artifact_materializations=materializations) context.runtime_value_store.clear() OpenHCSMetadataTarget.finalize_completed_dataset({"A01": context}) metadata_path = tmp_path / "openhcs_metadata.json" @@ -2623,8 +2694,13 @@ def test_actual_array_exports_only_publish_declared_raster_inventory( def _stream_batch_sources(projections): plan = function_step_plan("StreamOwned") - arrays = [np.full((2, 3), index, dtype=np.float32) for index in range(len(projections))] - paths = [str(plan.output_dir / f"A01_s1_w{index}.tif") for index in range(len(projections))] + arrays = [ + np.full((2, 3), index, dtype=np.float32) for index in range(len(projections)) + ] + paths = [ + str(plan.output_dir / f"A01_s1_w{index}.tif") + for index in range(len(projections)) + ] records = tuple( ProducedOutputSemantics.from_output( plan, @@ -2635,7 +2711,8 @@ def _stream_batch_sources(projections): source="stream source", ), output_context=AlignedImageSliceContext.main_flow( - output_key=f"image-{index}", projection_key=projection, + output_key=f"image-{index}", + projection_key=projection, artifact_kind=ImageArtifactType.value, ), ) @@ -2644,8 +2721,12 @@ def _stream_batch_sources(projections): return arrays, paths, records -def test_stream_batch_owns_correlated_projection_order_and_frozen_input_lists(monkeypatch): - arrays, paths, records = _stream_batch_sources(("second", "first", "second", "first")) +def test_stream_batch_owns_correlated_projection_order_and_frozen_input_lists( + monkeypatch, +): + arrays, paths, records = _stream_batch_sources( + ("second", "first", "second", "first") + ) expected_arrays, expected_paths = tuple(arrays), tuple(paths) observed = [] original_project = StreamOutputBatch.project_item @@ -2658,7 +2739,9 @@ def project(request): monkeypatch.setattr(StreamOutputBatch, "project_item", staticmethod(project)) batches = StreamOutputBatch.from_projection_groups( - parser=ImageXpressFilenameParser(), payloads=arrays, paths=paths, + parser=ImageXpressFilenameParser(), + payloads=arrays, + paths=paths, produced_outputs=records, ) assert observed == [expected_paths[index] for index in (0, 2, 1, 3)] @@ -2668,14 +2751,20 @@ def project(request): records[index].producer_identity for index in (0, 2, 1, 3) ] for item, index in zip( - (item for batch in batches for item in batch.items), (0, 2, 1, 3), strict=True, + (item for batch in batches for item in batch.items), + (0, 2, 1, 3), + strict=True, ): assert item.data is expected_arrays[index] assert item.source_component_metadata["channel"] == str(index) -def test_stream_batch_keeps_first_projection_error_and_prior_callback_effects(monkeypatch): - arrays, paths, records = _stream_batch_sources(("first", "second", "first", "second")) +def test_stream_batch_keeps_first_projection_error_and_prior_callback_effects( + monkeypatch, +): + arrays, paths, records = _stream_batch_sources( + ("first", "second", "first", "second") + ) observed = [] failure = ValueError("actual projector failure") original_project = StreamOutputBatch.project_item @@ -2689,7 +2778,9 @@ def project(request): monkeypatch.setattr(StreamOutputBatch, "project_item", staticmethod(project)) with pytest.raises(ValueError) as raised: StreamOutputBatch.from_projection_groups( - parser=ImageXpressFilenameParser(), payloads=arrays, paths=paths, + parser=ImageXpressFilenameParser(), + payloads=arrays, + paths=paths, produced_outputs=records, ) assert raised.value is failure @@ -2702,23 +2793,37 @@ def test_stream_batch_validates_cardinality_and_routes_before_projecting(monkeyp def forbidden_project(request): raise AssertionError("invalid inputs reached projection") - monkeypatch.setattr(StreamOutputBatch, "project_item", staticmethod(forbidden_project)) + monkeypatch.setattr( + StreamOutputBatch, "project_item", staticmethod(forbidden_project) + ) parser = ImageXpressFilenameParser() with pytest.raises(ValueError, match="payload/path cardinality mismatch"): StreamOutputBatch.from_projection_groups( - parser=parser, payloads=arrays, paths=paths[:1], produced_outputs=(), + parser=parser, + payloads=arrays, + paths=paths[:1], + produced_outputs=(), ) with pytest.raises(ValueError, match="payload/output-record cardinality mismatch"): StreamOutputBatch.from_projection_groups( - parser=parser, payloads=arrays, paths=paths, produced_outputs=records[:1], + parser=parser, + payloads=arrays, + paths=paths, + produced_outputs=records[:1], ) with pytest.raises(ValueError, match="at least one produced output record"): StreamOutputBatch.from_projection_groups( - parser=parser, payloads=(), paths=(), produced_outputs=(), + parser=parser, + payloads=(), + paths=(), + produced_outputs=(), ) with pytest.raises(ValueError, match="cannot mix producer projections"): StreamOutputBatch.from_projection( - parser=parser, payloads=arrays, paths=paths, produced_outputs=records, + parser=parser, + payloads=arrays, + paths=paths, + produced_outputs=records, ) @@ -2780,7 +2885,10 @@ def test_whole_volume_checkpoint_publication_retains_exact_producer_scope( ) assert str(contextualized.source_component_metadata["timepoint"]) == "1" assert "z_index" not in contextualized.source_component_metadata - assert contextualized.source_image_provenance_planes == metadata.source_image_provenance_planes + assert ( + contextualized.source_image_provenance_planes + == metadata.source_image_provenance_planes + ) manager = FileManager( {"disk": DiskStorageBackend(), "memory": MemoryStorageBackend()} ) @@ -2850,9 +2958,7 @@ def test_whole_volume_checkpoint_publication_retains_exact_producer_scope( path=output_plan.path, backend=Backend.MEMORY.value, ) - materializations = ArtifactMaterializationTargetPlan.materialize( - context, plan - ) + materializations = ArtifactMaterializationTargetPlan.materialize(context, plan) assert { Path(output.path) for artifact in materializations @@ -2866,19 +2972,14 @@ def test_whole_volume_checkpoint_publication_retains_exact_producer_scope( assert projection.address is None assert projection.source_alias == "DNA" assert ( - projection.execution_scope.value_text_for_component(Microscopy.Channel) - == "2" - ) - assert ( - projection.execution_scope.value_text_for_component(Microscopy.Site) == "1" + projection.execution_scope.value_text_for_component(Microscopy.Channel) == "2" ) + assert projection.execution_scope.value_text_for_component(Microscopy.Site) == "1" assert ( - projection.execution_scope.value_text_for_component(Microscopy.Timepoint) - == "1" + projection.execution_scope.value_text_for_component(Microscopy.Timepoint) == "1" ) assert ( - projection.execution_scope.value_text_for_component(Microscopy.ZIndex) - is None + projection.execution_scope.value_text_for_component(Microscopy.ZIndex) is None ) assert ( projection.image_metadata.source_image_provenance_planes.component_metadata From 24bfafede5922fbcf46d65a477497e6de65a2166 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 04:57:48 -0400 Subject: [PATCH 47/84] B6: debug-boundary guard reads the debug methods from FunctionCoreDebugMixin Co-Authored-By: Claude Opus 5.5 --- tests/unit/test_debug_runtime.py | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/tests/unit/test_debug_runtime.py b/tests/unit/test_debug_runtime.py index e12751fe23..3baee48e0d 100644 --- a/tests/unit/test_debug_runtime.py +++ b/tests/unit/test_debug_runtime.py @@ -578,10 +578,15 @@ def test_debug_artifact_boundary_has_no_name_key_reconstruction(): "DebugArtifactRefProjection", {"from_artifact_plans"}, ), + ( + repository_root / "openhcs/core/steps/function_runtime.py", + "FunctionCoreDebugMixin", + {"debug_artifacts", "debug_event"}, + ), ( repository_root / "openhcs/core/steps/function_runtime.py", "FunctionCoreExecutor", - {"debug_artifacts", "debug_event", "invoke"}, + {"invoke"}, ), ) violations: list[tuple[str, int, str]] = [] From 7c1c188181cf0094aae1022a5602c0fd71b73720 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 04:58:05 -0400 Subject: [PATCH 48/84] B6: re-pin documentation authorities after the OMERO metadata and test fixes Co-Authored-By: Claude Opus 5.5 --- docs/audits/architecture_n_z_development.json | 6 ++++-- docs/audits/user_docs.json | 3 ++- 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/docs/audits/architecture_n_z_development.json b/docs/audits/architecture_n_z_development.json index 8d9aadee84..fb3dad8897 100644 --- a/docs/audits/architecture_n_z_development.json +++ b/docs/audits/architecture_n_z_development.json @@ -627,7 +627,7 @@ }, { "path": "openhcs/core/steps/function_outputs.py", - "sha256": "89405feaa9092caa7ac3699de649eca5516c7bc54a757a24ce881264c21af262", + "sha256": "fc6176fddfcddfa6801091ce0d861b3ed5ea57d06f07227ed634cc706436e91f", "role": "OpenHCSMetadataTarget declarations, typed projection persistence, and completed-plate deduplication across axes." }, { @@ -645,6 +645,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/pipeline_compilation_system.rst against B6's later changes to openhcs/core/steps/function_outputs.py: metadata documents are written only for backends that host local files (not virtual backends such as OMERO); the image-source guide test reads axis-family spellings. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/pipeline_compilation_system.rst against B6's later changes to openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/compiled_step_plan.py, openhcs/core/callable_contract.py, openhcs/core/function_patterns.py, openhcs/processing/backends/lib_registry/registry_service.py, openhcs/core/orchestrator/bundle_execution.py, openhcs/core/steps/function_outputs.py, tests/unit/test_compilation_session.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/pipeline_compilation_system.rst against the later B6 changes to openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/compiled_step_plan.py, openhcs/core/callable_contract.py, openhcs/core/function_patterns.py, openhcs/processing/backends/lib_registry/registry_service.py, openhcs/core/orchestrator/bundle_execution.py, openhcs/core/steps/function_outputs.py, tests/unit/test_compilation_session.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/pipeline_compilation_system.rst in full and compared it against the changed authorities: pipeline/compiler.py, compilation_session.py, compiled_execution.py, context/processing_context.py, test_compiled_execution.py, compiled_plate_execution.py (renamed bundle_execution.py), steps/function_outputs.py, test_compilation_session.py, callable_contract.py, compiled_step_plan.py, framework_device_assignment.py; verified each named class/function exists (FunctionStepTransportAuthority, RegistryService.registered_callable, AxisCompilationRequest, CompiledRuntimeEnvironmentPlan, RuntimeArtifactBatch, OpenHCSMetadataTarget, FunctionStepExecutionScope AXIS/PLATE); rewrote the resolve-once, preparation, session and plan-inventory passages that named deleted StepSnapshot, prepare_compiled_context_callables and source load plans. Dropped the framework-device resolver/assignment files and test_framework_device_resolver.py as authorities since the page states only that bindings stay on step plans.", @@ -3346,7 +3347,7 @@ }, { "path": "tests/unit/agent/test_knowledge_base_service.py", - "sha256": "ce4636580893a8736fd4607c6ed0f431d25c97e0c44e2b95912af3fc36b1d750", + "sha256": "b814db1827c3d4cc3dab479ef7e84c6c406f324ccaddee794690f3b387626411", "role": "Regression proof for document IDs, search, bounded reads, and section selection." }, { @@ -3358,6 +3359,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/mcp_knowledge_base.rst against B6's later changes to tests/unit/agent/test_knowledge_base_service.py: metadata documents are written only for backends that host local files (not virtual backends such as OMERO); the image-source guide test reads axis-family spellings. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/development/mcp_knowledge_base.rst against B6's later changes to tests/unit/agent/test_knowledge_base_service.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/development/mcp_knowledge_base.rst against the later B6 changes to openhcs/agent/capabilities.py, tests/unit/agent/test_knowledge_base_service.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/mcp_knowledge_base.rst in full and compared it against the changed authorities: knowledge_base_service.py (comparison-manifest parsing moved to openhcs.agent.knowledge_manifest_schema.ComparisonManifestSnapshot; per-case module inventories are now cached by source mtime), knowledge_manifest.py (comparison_manifest_type_for_root) and both test files. Confirmed the generated Python conversion is still recomputed per request and never stored (the new cache holds only module-name inventories), the 12,000 default bound, the four MCP surface names and dev-client commands with --limit, --max-chars and --section-id, the four document ids in mcp_knowledge_base_manifest.json, benchmark/manifests/official30_portable_axis1.json, the first-use capability names, and every cited documentation path; no page change was needed. Added openhcs/agent/capabilities.py for the surface names.", diff --git a/docs/audits/user_docs.json b/docs/audits/user_docs.json index ea93833a69..2fc8ae0f18 100644 --- a/docs/audits/user_docs.json +++ b/docs/audits/user_docs.json @@ -2279,7 +2279,7 @@ }, { "path": "tests/unit/agent/test_knowledge_base_service.py", - "sha256": "ce4636580893a8736fd4607c6ed0f431d25c97e0c44e2b95912af3fc36b1d750", + "sha256": "b814db1827c3d4cc3dab479ef7e84c6c406f324ccaddee794690f3b387626411", "role": "Proves discovery and retrieval of the example corpus map and other maintained knowledge documents." } ], @@ -2288,6 +2288,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/guides/example_corpus_map.rst against B6's later changes to tests/unit/agent/test_knowledge_base_service.py: metadata documents are written only for backends that host local files (not virtual backends such as OMERO); the image-source guide test reads axis-family spellings. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/guides/example_corpus_map.rst against B6's later changes to openhcs/processing/presets/pipelines/loose_operaphenix_neurite_outgrowth.py, tests/unit/agent/test_knowledge_base_service.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guides/example_corpus_map.rst against the later B6 changes to openhcs/processing/presets/pipelines/loose_operaphenix_neurite_outgrowth.py, openhcs/agent/capabilities.py, tests/unit/agent/test_knowledge_base_service.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guides/example_corpus_map.rst in full and compared it against the changed authorities: knowledge_base_service.py, mfd_specs.py, benchmark/datasets/manifest.py, test_knowledge_base_service.py; confirmed every repository path, the thirty scoped native reference rows, the four MFD wrapper modules, the cellprofiler_translation topic, the authoring-context and plate inspection tool names and the Native Example Source Index; replaced 'parity receipts' with 'parity evidence'.", From 359e31d72398b6ed9282ae4c6182989962829d2d Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 04:59:01 -0400 Subject: [PATCH 49/84] B6: read the validated case name like the loader does (no new string subscript) Co-Authored-By: Claude Opus 5.5 --- openhcs/agent/knowledge_manifest_schema.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/openhcs/agent/knowledge_manifest_schema.py b/openhcs/agent/knowledge_manifest_schema.py index 845b93794d..e71f892b53 100644 --- a/openhcs/agent/knowledge_manifest_schema.py +++ b/openhcs/agent/knowledge_manifest_schema.py @@ -158,7 +158,7 @@ def native_source_relative_path(self, case: Mapping[str, object]) -> Path: installed resource within ``PACKAGED_KNOWLEDGE_PATH_BUDGET``. """ path = Path(str(case["cppipe_path"])) - name = str(case["name"]) + name = str(case.get("name")) if path.is_absolute() or ".." in path.parts or path.suffix != ".cppipe": raise ValueError("Native knowledge source must be a relative .cppipe path.") if not _PORTABLE_CASE_NAME.fullmatch(name): From 433b26fef7481dc690267375cb233385d4e31f38 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 05:06:46 -0400 Subject: [PATCH 50/84] B6: Windows read-only datasets and Ctrl+Break interrupt Initialization decides in-place vs mirrored workspace by an actual file creation (anonymous / delete-on-close temp file), not os.access, which on Windows ignores ACLs and the directory read-only attribute. The measured benchmark CLI treats Ctrl+Break (SIGBREAK, the console interrupt deliverable to one process group on Windows) as Ctrl+C: cancelled row, exit 130. Tests make a genuinely non-writable tree on Windows (inherited icacls deny for Everyone) and deliver CTRL_BREAK_EVENT to a CREATE_NEW_PROCESS_GROUP child. Co-Authored-By: Claude Opus 5.5 --- benchmark/cellprofiler_benchmark_cli.py | 13 +- openhcs/authoring/session/session.py | 114 +++++++++++++----- .../integration/test_measured_cli_session.py | 30 ++++- tests/unit/test_dataset_access.py | 22 ++++ 4 files changed, 147 insertions(+), 32 deletions(-) create mode 100644 tests/unit/test_dataset_access.py diff --git a/benchmark/cellprofiler_benchmark_cli.py b/benchmark/cellprofiler_benchmark_cli.py index 7a149cf0d0..2d5825a0e8 100644 --- a/benchmark/cellprofiler_benchmark_cli.py +++ b/benchmark/cellprofiler_benchmark_cli.py @@ -6,6 +6,7 @@ import json import math import os +import signal import sys from abc import ABC, abstractmethod from collections.abc import Sequence @@ -321,7 +322,9 @@ def run(self, args: argparse.Namespace) -> int: ) return 130 except MeasuredRunEnded as ended: - print(json.dumps(to_jsonable(ended.row), sort_keys=True), file=sys.stderr) + print( + json.dumps(to_jsonable(ended.row), sort_keys=True), file=sys.stderr + ) raise RuntimeError( f"Ordinary pipeline run did not complete: {ended}" ) from ended @@ -824,7 +827,9 @@ def configure( required=True, help="MODE_LABEL=qualified_summary.csv; repeat for measured well/worker modes.", ) - parser.add_argument("--scope", choices=("execution", "total", "amortization"), required=True) + parser.add_argument( + "--scope", choices=("execution", "total", "amortization"), required=True + ) parser.add_argument("--output-dir", type=Path, required=True) return parser @@ -1068,6 +1073,10 @@ def create_benchmark_argument_parser() -> argparse.ArgumentParser: def main(argv: Sequence[str] | None = None) -> int: """Run the registered benchmark CLI.""" + if sys.platform == "win32": + # Ctrl+Break is the Windows console interrupt a launcher can deliver + # to this process alone; it stops a run exactly as Ctrl+C does. + signal.signal(signal.SIGBREAK, signal.default_int_handler) arguments = tuple(sys.argv[1:] if argv is None else argv) parser = create_benchmark_argument_parser() parsed = parser.parse_args(arguments) diff --git a/openhcs/authoring/session/session.py b/openhcs/authoring/session/session.py index 01671cacae..c01ccb89a8 100644 --- a/openhcs/authoring/session/session.py +++ b/openhcs/authoring/session/session.py @@ -13,7 +13,7 @@ import asyncio import inspect import logging -import os +import tempfile import threading import time from abc import ABC, abstractmethod @@ -196,6 +196,21 @@ def call(self, callback: Callable[[], ResultT]) -> ResultT: return self._dispatcher.call(callback) +def _creates_files_in(directory: Path) -> bool: + """Whether the filesystem lets this process create a file in ``directory``. + + Permission bits (``os.access``) do not see Windows ACLs or read-only + shares; an actual creation does. The probe file is anonymous or deleted on + close, so it leaves nothing behind. + """ + + try: + with tempfile.TemporaryFile(dir=directory): + return True + except OSError: + return False + + class DatasetAccess(ABC): """Which dataset roots this session may read and initialize.""" @@ -210,7 +225,7 @@ def initializes_in_place(self, root: Path) -> bool: """ return all( - os.access(directory, os.W_OK) + _creates_files_in(directory) for directory in { Path(root), *(path.parent for path in METADATA_CONFIG.managed_paths(root)), @@ -310,7 +325,9 @@ def __init__( self.global_config = global_config self.client = ZMQClientService( config=transport_config, - status_callback=lambda status: self.publish(ServerConnectionChanged(status)), + status_callback=lambda status: self.publish( + ServerConnectionChanged(status) + ), compatibility_callback=lambda compatibility: self.publish( ServerCompatibilityObserved(compatibility) ), @@ -345,7 +362,9 @@ def __init__( self.submission = ExecutionSubmission(self) self.execution_control = ExecutionControl(self) self.debug_runs = DebugRuns(self) - self.progress = ExecutionProgress(self, interval_seconds=progress_interval_seconds) + self.progress = ExecutionProgress( + self, interval_seconds=progress_interval_seconds + ) self._work = AsyncOperationExecutor() self._closed = False @@ -357,7 +376,9 @@ def __init__( def publish(self, event: SessionEvent) -> SessionEventRecord: return self.event_log.publish(event) - def subscribe(self, listener: Callable[[SessionEventRecord], None]) -> Callable[[], None]: + def subscribe( + self, listener: Callable[[SessionEventRecord], None] + ) -> Callable[[], None]: return self.event_log.subscribe(listener) def events_after( @@ -507,7 +528,9 @@ def add_dataset_roots( on that separately prepared root. """ - roots = tuple(self.dataset_access.require_readable(Path(root)) for root in roots) + roots = tuple( + self.dataset_access.require_readable(Path(root)) for root in roots + ) if execution_root is not None: if len(roots) != 1: raise ValueError("A prepared execution root needs exactly one root.") @@ -522,7 +545,9 @@ def add_dataset_roots( ) else: offers = tuple( - offer for root in roots for offer in DatasetScopeKind.offers_for_root(root) + offer + for root in roots + for offer in DatasetScopeKind.offers_for_root(root) ) added = self._add_scopes( tuple(offer.scope for offer in offers), @@ -636,7 +661,9 @@ def _split_dataset_scopes(self) -> None: def delete_datasets(self, scope_ids: Iterable[str]) -> None: doomed = {str(scope_id) for scope_id in scope_ids} - active = sorted(scope_id for scope_id in doomed if self.has_active_work(scope_id)) + active = sorted( + scope_id for scope_id in doomed if self.has_active_work(scope_id) + ) if active: raise RuntimeError( "Cannot delete a dataset with active initialization, compilation, " @@ -704,7 +731,9 @@ def set_pipeline_document(self, scope_id: str, document: PipelineDocument) -> No PipelineObjectStateBinding.commit_plate_state(scope_id) self.pipeline_changed(scope_id) self.publish( - StatusReported(f"Pipeline updated with {len(document.pipeline_steps)} steps") + StatusReported( + f"Pipeline updated with {len(document.pipeline_steps)} steps" + ) ) def set_pipeline_source(self, scope_id: str, source: str) -> None: @@ -748,7 +777,9 @@ def replace_step( edited_step: FunctionStep, ) -> None: self.require_definition_mutation_allowed(scope_id) - PipelineObjectStateBinding.replace_plate_step(scope_id, current_step, edited_step) + PipelineObjectStateBinding.replace_plate_step( + scope_id, current_step, edited_step + ) self.pipeline_changed(scope_id) self.publish(StatusReported(f"Updated step: {edited_step.name}")) @@ -780,7 +811,9 @@ def delete_steps(self, scope_id: str, step_scope_ids: Iterable[str]) -> None: doomed = set(step_scope_ids) self.require_definition_mutation_allowed(scope_id) steps = self.pipeline_steps(scope_id) - kept = [step for step in steps if self.step_scope_id(scope_id, step) not in doomed] + kept = [ + step for step in steps if self.step_scope_id(scope_id, step) not in doomed + ] names = [step.name for step in steps if step not in kept] with ObjectStateRegistry.atomic(f"delete steps {', '.join(names)}"): for step_scope_id in doomed: @@ -883,7 +916,9 @@ def require_definition_mutation_allowed(self, scope_id: str | None = None) -> No "active initialization or compilation. Other datasets remain editable." ) - def require_definition_mutation_allowed_for_object_scope(self, scope_id: str) -> None: + def require_definition_mutation_allowed_for_object_scope( + self, scope_id: str + ) -> None: """Authorize an ObjectState mutation that may belong to a dataset.""" if scope_id == "": @@ -1024,10 +1059,13 @@ async def initialize(scope_id: str) -> None: self.publish(DatasetsChanged()) def run() -> InputWorkspacePreparationResult | None: - ensure_global_config_context(GlobalPipelineConfig, self.global_config) + ensure_global_config_context( + GlobalPipelineConfig, self.global_config + ) workspace = scope.prepare_input_workspace() - if workspace is None and not self.dataset_access.initializes_in_place( - scope.root + if ( + workspace is None + and not self.dataset_access.initializes_in_place(scope.root) ): workspace = self.mirrored_workspace(scope.root) if workspace is not None: @@ -1055,9 +1093,7 @@ def run() -> InputWorkspacePreparationResult | None: ) self.set_dataset_state(scope_id, OrchestratorState.READY) if scope_id == self.current_scope_id or not self.current_scope_id: - self.main_thread.call( - lambda: self._reselect(scope_id) - ) + self.main_thread.call(lambda: self._reselect(scope_id)) done += 1 self.publish(ProgressAdvanced(done)) @@ -1122,7 +1158,9 @@ def _arm_run_deadline( timer = threading.Timer( wait_timeout_ms / 1000, lambda: self.main_thread.post( - lambda: self._run_deadline_passed(batch_number, scope_ids, wait_timeout_ms) + lambda: self._run_deadline_passed( + batch_number, scope_ids, wait_timeout_ms + ) ), ) timer.daemon = True @@ -1207,7 +1245,9 @@ def observation_export_params( return None path = self.dataset_access.require_writable(Path(export_path)) if path.exists(): - raise FileExistsError(f"Runtime observation export path already exists: {path}") + raise FileExistsError( + f"Runtime observation export path already exists: {path}" + ) return ZMQAuxiliaryExecutionParams( runtime_observation_export_path=path, runtime_observation_export_scope=scope, @@ -1241,7 +1281,9 @@ def finish_dataset_execution( ) -> None: """Record one dataset's terminal status (from any thread).""" - self.main_thread.post(lambda: self._finish_dataset_execution(completion, scope_id)) + self.main_thread.post( + lambda: self._finish_dataset_execution(completion, scope_id) + ) def _finish_dataset_execution( self, @@ -1273,7 +1315,9 @@ def _finish_dataset_execution( if status.emit_failure: detail = completion.traceback_text or completion.message separator = ":\n\n" if completion.traceback_text else ": " - self.publish(ErrorReported(f"Execution failed for {scope_id}{separator}{detail}")) + self.publish( + ErrorReported(f"Execution failed for {scope_id}{separator}{detail}") + ) def _add_output_dataset( self, @@ -1309,12 +1353,17 @@ def _add_output_dataset( def finish_batch(self, completed_count: int, failed_count: int) -> None: self.execution_state = ManagerExecutionState.IDLE summary = f"All done: {completed_count} completed, {failed_count} failed" - if completed_count > 1 and self.global_config.analysis_consolidation_config.enabled: + if ( + completed_count > 1 + and self.global_config.analysis_consolidation_config.enabled + ): try: consolidate_batch_results(self) summary += ". Global summary created." except Exception as error: - logger.error("Failed to create global summary: %s", error, exc_info=True) + logger.error( + "Failed to create global summary: %s", error, exc_info=True + ) summary += ". Global summary failed." self.publish(StatusReported(summary)) self.publish(BatchFinished(completed_count, failed_count)) @@ -1347,7 +1396,9 @@ async def run_debug( return self.require_execution_allowed((scope_id,)) - debug_session = DebugSession.create(subject_id=scope_id, command_type=command_type) + debug_session = DebugSession.create( + subject_id=scope_id, command_type=command_type + ) self.debug_sessions[scope_id] = debug_session root = DatasetScope.parse(scope_id).root snapshot_root = (root if root.is_dir() else root.parent) / ".openhcs_debug" @@ -1368,7 +1419,9 @@ async def run_debug( replay_mode=DebugReplayMode.PERSISTENT_PAUSED_WORKER, ) request = dataset_pipeline_request(scope_id, self.global_config) - artifact_id = await self.debug_runs.compile_artifact_id(request, debug_request) + artifact_id = await self.debug_runs.compile_artifact_id( + request, debug_request + ) self.publish(StatusReported(f"Submitting debug run for {scope_id}...")) await self.debug_runs.submit( request, @@ -1430,7 +1483,9 @@ def inspect_debug_snapshot( snapshot_store_backend=context.snapshot_store_backend, axis_id=event.axis_id, ).with_cursor(context.cursor) - elif summary is not None and summary.debug_session_id == context.debug_session_id: + elif ( + summary is not None and summary.debug_session_id == context.debug_session_id + ): self.inspected_debug_sessions.pop(scope_id, None) if snapshot is not None: self.debug_terminal_summaries[scope_id] = summary.with_snapshot( @@ -1480,7 +1535,9 @@ def debug_context(self, scope_id: str) -> DebugSessionProjectionContext: return DebugSessionProjectionContext( target=DebugSessionTargetState( current_plate_scope_id=scope_id, - pipeline_scope_id=PipelineScopeIdentity.from_plate_scope(scope_id).scope_id, + pipeline_scope_id=PipelineScopeIdentity.from_plate_scope( + scope_id + ).scope_id, initialized=self.is_initialized(scope_id), compiled=scope_id in self.compiled, terminal_status=( @@ -1506,4 +1563,3 @@ def current_debug_context(self) -> DebugSessionProjectionContext | None: if not self.current_scope_id: return None return self.debug_context(self.current_scope_id) - diff --git a/tests/integration/test_measured_cli_session.py b/tests/integration/test_measured_cli_session.py index 230ff057b5..e51ca2096d 100644 --- a/tests/integration/test_measured_cli_session.py +++ b/tests/integration/test_measured_cli_session.py @@ -78,7 +78,34 @@ def _source_state(root: Path) -> dict[str, tuple]: return state +# Everyone may not create, change or delete entries; inherited by the tree. +# Write rights are named one by one: the generic W also denies SYNCHRONIZE, +# which every open (reads included) requests. +WINDOWS_READ_ONLY_ACE = "*S-1-1-0:(OI)(CI)(WD,AD,WEA,WA,D,DC)" + +if sys.platform == "win32": + # A console's Ctrl+C cannot be sent to one process; Ctrl+Break can, to a + # child started in its own process group. + INTERRUPT_LAUNCH = {"creationflags": subprocess.CREATE_NEW_PROCESS_GROUP} + INTERRUPT_SIGNAL = signal.CTRL_BREAK_EVENT +else: + INTERRUPT_LAUNCH = {} + INTERRUPT_SIGNAL = signal.SIGINT + + def _set_writable(root: Path, writable: bool) -> None: + """Make ``root`` and its tree writable again, or genuinely not writable. + + Windows ignores the read-only attribute on directories (and os.access + ignores ACLs), so there an inherited deny ACE is what refuses writes. + """ + + if sys.platform == "win32": + change = ( + ("/remove:d", "*S-1-1-0") if writable else ("/deny", WINDOWS_READ_ONLY_ACE) + ) + subprocess.run(("icacls", str(root), *change), check=True, capture_output=True) + return for path in (root, *root.rglob("*")): mode = path.stat().st_mode path.chmod(mode | 0o200 if writable else mode & ~0o222) @@ -178,6 +205,7 @@ def test_ctrl_c_stops_the_run_and_its_server(tmp_path: Path) -> None: stderr=subprocess.PIPE, stdout=subprocess.PIPE, text=True, + **INTERRUPT_LAUNCH, ) stderr_lines: list[str] = [] try: @@ -188,7 +216,7 @@ def test_ctrl_c_stops_the_run_and_its_server(tmp_path: Path) -> None: else: pytest.fail("".join(stderr_lines)[-4000:]) time.sleep(1.0) - process.send_signal(signal.SIGINT) + process.send_signal(INTERRUPT_SIGNAL) _stdout, rest = process.communicate(timeout=RUN_DEADLINE_SECONDS) finally: if process.poll() is None: diff --git a/tests/unit/test_dataset_access.py b/tests/unit/test_dataset_access.py new file mode 100644 index 0000000000..1ed8f06e1b --- /dev/null +++ b/tests/unit/test_dataset_access.py @@ -0,0 +1,22 @@ +"""Whether a dataset initializes in place is the filesystem's answer. + +The non-writable case (POSIX mode bits, Windows ACLs) is exercised by the +read-only integration journeys, which make a genuinely non-writable tree. +""" + +from pathlib import Path + +from openhcs.authoring.session.session import UnrestrictedDatasetAccess + + +def test_writable_root_initializes_in_place_and_the_probe_leaves_nothing( + tmp_path: Path, +) -> None: + (tmp_path / "image.tif").write_bytes(b"pixels") + + assert UnrestrictedDatasetAccess().initializes_in_place(tmp_path) + assert [path.name for path in tmp_path.iterdir()] == ["image.tif"] + + +def test_missing_root_does_not_initialize_in_place(tmp_path: Path) -> None: + assert not UnrestrictedDatasetAccess().initializes_in_place(tmp_path / "absent") From ab03ba7bacfb3ef34e5db677c757657d585f215c Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 05:08:26 -0400 Subject: [PATCH 51/84] B6: Session's debug-session methods move into SessionDebugMixin (ratchet) Reformatting session.py for the Windows writable-root fix grew Session past its ratchet budget; the debug group is one cohesive concern Session inherits. Co-Authored-By: Claude Opus 5.5 --- openhcs/authoring/session/session.py | 392 ++++++++++++++------------- 1 file changed, 198 insertions(+), 194 deletions(-) diff --git a/openhcs/authoring/session/session.py b/openhcs/authoring/session/session.py index c01ccb89a8..a3fa2be760 100644 --- a/openhcs/authoring/session/session.py +++ b/openhcs/authoring/session/session.py @@ -299,7 +299,204 @@ class FinishedExecution: endpoint: "PongResponse | None" -class Session: +class SessionDebugMixin: + """Run, inspect and close one dataset's debug session.""" + + async def run_debug( + self, + scope_id: str, + *, + command_type: DebugCommandType = DebugCommandType.RUN, + snapshot_store_backend: str | None = None, + selected_source_group: str | None = None, + pause_step_indices: tuple[int, ...] = (), + start_step_index: int = 0, + start_after_invocation_key: str | None = None, + ) -> None: + """Start a debug run, or send a command to the dataset's debug worker.""" + + active = self.debug_sessions.get(scope_id) + if active is not None: + self.debug_sessions[scope_id] = active.with_command(command_type) + await self.debug_runs.send_worker_command( + debug_session_id=active.debug_session_id, + command_type=command_type, + ) + if command_type is DebugCommandType.STOP: + self.debug_sessions.pop(scope_id, None) + return + + self.require_execution_allowed((scope_id,)) + debug_session = DebugSession.create( + subject_id=scope_id, command_type=command_type + ) + self.debug_sessions[scope_id] = debug_session + root = DatasetScope.parse(scope_id).root + snapshot_root = (root if root.is_dir() else root.parent) / ".openhcs_debug" + self._begin_batch((scope_id,)) + try: + await self.connect_client() + self.progress.reset_for_new_batch() + self.publish(StatusReported(f"Compiling debug run for {scope_id}...")) + debug_request = DebugRunRequest( + debug_session_id=debug_session.debug_session_id, + snapshot_store_ref=str(snapshot_root), + snapshot_store_backend=snapshot_store_backend, + command_type=DebugCommandType(command_type), + selected_source_group=selected_source_group, + pause_step_indices=tuple(pause_step_indices), + start_step_index=start_step_index, + start_after_invocation_key=start_after_invocation_key, + replay_mode=DebugReplayMode.PERSISTENT_PAUSED_WORKER, + ) + request = dataset_pipeline_request(scope_id, self.global_config) + artifact_id = await self.debug_runs.compile_artifact_id( + request, debug_request + ) + self.publish(StatusReported(f"Submitting debug run for {scope_id}...")) + await self.debug_runs.submit( + request, + compile_artifact_id=artifact_id, + debug_request=debug_request, + ) + except Exception as error: + logger.error("Failed to execute debug run: %s", error, exc_info=True) + self.publish(ErrorReported(f"Failed to execute debug run: {error}")) + await self.execution_control.handle_execution_failure() + + def record_debug_snapshot( + self, + notification: DebugSnapshotAvailableNotification, + ) -> None: + scope_id = notification.progress_event.subject_id + if notification.snapshot is not None: + retained = tuple( + snapshot + for snapshot in self.debug_snapshots.get(scope_id, ()) + if snapshot.snapshot_id != notification.snapshot.snapshot_id + ) + self.debug_snapshots[scope_id] = (*retained, notification.snapshot) + active = self.debug_sessions.get(scope_id) + if active is not None: + context = notification.debug_context + self.debug_sessions[scope_id] = active.with_snapshot_store( + snapshot_store_ref=context.snapshot_store_ref, + snapshot_store_backend=context.snapshot_store_backend, + axis_id=notification.progress_event.axis_id, + ).with_cursor(context.cursor) + self.publish(DebugSnapshotAvailable(notification)) + + def displayed_debug_session(self, scope_id: str) -> DebugSession | None: + """The running debug session, else the one loaded from a snapshot.""" + + active = self.debug_sessions.get(scope_id) + if active is not None: + return active + if scope_id in self.debug_terminal_summaries: + return None + return self.inspected_debug_sessions.get(scope_id) + + def inspect_debug_snapshot( + self, + notification: DebugSnapshotAvailableNotification, + snapshot: DebugSnapshot | None, + ) -> None: + """Move the displayed debug cursor to a snapshot a client opened.""" + + event = notification.progress_event + context = notification.debug_context + scope_id = event.subject_id + active = self.debug_sessions.get(scope_id) + summary = self.debug_terminal_summaries.get(scope_id) + if active is not None: + self.debug_sessions[scope_id] = active.with_snapshot_store( + snapshot_store_ref=context.snapshot_store_ref, + snapshot_store_backend=context.snapshot_store_backend, + axis_id=event.axis_id, + ).with_cursor(context.cursor) + elif ( + summary is not None and summary.debug_session_id == context.debug_session_id + ): + self.inspected_debug_sessions.pop(scope_id, None) + if snapshot is not None: + self.debug_terminal_summaries[scope_id] = summary.with_snapshot( + snapshot=snapshot, + snapshot_id=context.snapshot_id, + snapshot_store_ref=context.snapshot_store_ref, + snapshot_store_backend=context.snapshot_store_backend, + ) + else: + self.inspected_debug_sessions[scope_id] = DebugSession( + debug_session_id=context.debug_session_id, + subject_id=scope_id, + axis_id=event.axis_id, + snapshot_store_ref=context.snapshot_store_ref, + snapshot_store_backend=context.snapshot_store_backend, + ).with_cursor(context.cursor) + self.refresh() + + def _close_debug_session(self, scope_id: str) -> None: + active = self.debug_sessions.pop(scope_id, None) + if active is None or active.subject_id != scope_id: + return + terminal_status = self.batch.terminal_status(scope_id) + if terminal_status is None: + self.debug_terminal_summaries.pop(scope_id, None) + else: + summary = DebugTerminalSummary.from_session( + active, + terminal_status=terminal_status.value, + completed_at_unix=time.time(), + ) + snapshots = self.debug_snapshots.get(scope_id, ()) + latest = snapshots[-1] if snapshots else None + self.debug_terminal_summaries[scope_id] = ( + summary + if latest is None + else summary.with_snapshot( + snapshot=latest, + snapshot_id=latest.snapshot_id, + snapshot_store_ref=active.snapshot_store_ref, + snapshot_store_backend=active.snapshot_store_backend, + ) + ) + self.publish(ExecutionStateChanged(self.execution_state)) + + def debug_context(self, scope_id: str) -> DebugSessionProjectionContext: + return DebugSessionProjectionContext( + target=DebugSessionTargetState( + current_plate_scope_id=scope_id, + pipeline_scope_id=PipelineScopeIdentity.from_plate_scope( + scope_id + ).scope_id, + initialized=self.is_initialized(scope_id), + compiled=scope_id in self.compiled, + terminal_status=( + None + if (status := self.batch.terminal_status(scope_id)) is None + else status.value + ), + ), + session=self.displayed_debug_session(scope_id), + terminal_summary=self.debug_terminal_summaries.get(scope_id), + pause_boundaries=DebugPauseBoundaryState( + pause_step_indices=tuple( + index + for index, step in enumerate(self.pipeline_steps(scope_id)) + if step.debug_pause + ) + ), + snapshots=self.debug_snapshots.get(scope_id, ()), + manager_execution_state=self.execution_state, + ) + + def current_debug_context(self) -> DebugSessionProjectionContext | None: + if not self.current_scope_id: + return None + return self.debug_context(self.current_scope_id) + + +class Session(SessionDebugMixin): """Datasets, their pipelines and executions, and the events they emit.""" def __init__( @@ -1370,196 +1567,3 @@ def finish_batch(self, completed_count: int, failed_count: int) -> None: self.refresh() # -- debug ---------------------------------------------------------------- - - async def run_debug( - self, - scope_id: str, - *, - command_type: DebugCommandType = DebugCommandType.RUN, - snapshot_store_backend: str | None = None, - selected_source_group: str | None = None, - pause_step_indices: tuple[int, ...] = (), - start_step_index: int = 0, - start_after_invocation_key: str | None = None, - ) -> None: - """Start a debug run, or send a command to the dataset's debug worker.""" - - active = self.debug_sessions.get(scope_id) - if active is not None: - self.debug_sessions[scope_id] = active.with_command(command_type) - await self.debug_runs.send_worker_command( - debug_session_id=active.debug_session_id, - command_type=command_type, - ) - if command_type is DebugCommandType.STOP: - self.debug_sessions.pop(scope_id, None) - return - - self.require_execution_allowed((scope_id,)) - debug_session = DebugSession.create( - subject_id=scope_id, command_type=command_type - ) - self.debug_sessions[scope_id] = debug_session - root = DatasetScope.parse(scope_id).root - snapshot_root = (root if root.is_dir() else root.parent) / ".openhcs_debug" - self._begin_batch((scope_id,)) - try: - await self.connect_client() - self.progress.reset_for_new_batch() - self.publish(StatusReported(f"Compiling debug run for {scope_id}...")) - debug_request = DebugRunRequest( - debug_session_id=debug_session.debug_session_id, - snapshot_store_ref=str(snapshot_root), - snapshot_store_backend=snapshot_store_backend, - command_type=DebugCommandType(command_type), - selected_source_group=selected_source_group, - pause_step_indices=tuple(pause_step_indices), - start_step_index=start_step_index, - start_after_invocation_key=start_after_invocation_key, - replay_mode=DebugReplayMode.PERSISTENT_PAUSED_WORKER, - ) - request = dataset_pipeline_request(scope_id, self.global_config) - artifact_id = await self.debug_runs.compile_artifact_id( - request, debug_request - ) - self.publish(StatusReported(f"Submitting debug run for {scope_id}...")) - await self.debug_runs.submit( - request, - compile_artifact_id=artifact_id, - debug_request=debug_request, - ) - except Exception as error: - logger.error("Failed to execute debug run: %s", error, exc_info=True) - self.publish(ErrorReported(f"Failed to execute debug run: {error}")) - await self.execution_control.handle_execution_failure() - - def record_debug_snapshot( - self, - notification: DebugSnapshotAvailableNotification, - ) -> None: - scope_id = notification.progress_event.subject_id - if notification.snapshot is not None: - retained = tuple( - snapshot - for snapshot in self.debug_snapshots.get(scope_id, ()) - if snapshot.snapshot_id != notification.snapshot.snapshot_id - ) - self.debug_snapshots[scope_id] = (*retained, notification.snapshot) - active = self.debug_sessions.get(scope_id) - if active is not None: - context = notification.debug_context - self.debug_sessions[scope_id] = active.with_snapshot_store( - snapshot_store_ref=context.snapshot_store_ref, - snapshot_store_backend=context.snapshot_store_backend, - axis_id=notification.progress_event.axis_id, - ).with_cursor(context.cursor) - self.publish(DebugSnapshotAvailable(notification)) - - def displayed_debug_session(self, scope_id: str) -> DebugSession | None: - """The running debug session, else the one loaded from a snapshot.""" - - active = self.debug_sessions.get(scope_id) - if active is not None: - return active - if scope_id in self.debug_terminal_summaries: - return None - return self.inspected_debug_sessions.get(scope_id) - - def inspect_debug_snapshot( - self, - notification: DebugSnapshotAvailableNotification, - snapshot: DebugSnapshot | None, - ) -> None: - """Move the displayed debug cursor to a snapshot a client opened.""" - - event = notification.progress_event - context = notification.debug_context - scope_id = event.subject_id - active = self.debug_sessions.get(scope_id) - summary = self.debug_terminal_summaries.get(scope_id) - if active is not None: - self.debug_sessions[scope_id] = active.with_snapshot_store( - snapshot_store_ref=context.snapshot_store_ref, - snapshot_store_backend=context.snapshot_store_backend, - axis_id=event.axis_id, - ).with_cursor(context.cursor) - elif ( - summary is not None and summary.debug_session_id == context.debug_session_id - ): - self.inspected_debug_sessions.pop(scope_id, None) - if snapshot is not None: - self.debug_terminal_summaries[scope_id] = summary.with_snapshot( - snapshot=snapshot, - snapshot_id=context.snapshot_id, - snapshot_store_ref=context.snapshot_store_ref, - snapshot_store_backend=context.snapshot_store_backend, - ) - else: - self.inspected_debug_sessions[scope_id] = DebugSession( - debug_session_id=context.debug_session_id, - subject_id=scope_id, - axis_id=event.axis_id, - snapshot_store_ref=context.snapshot_store_ref, - snapshot_store_backend=context.snapshot_store_backend, - ).with_cursor(context.cursor) - self.refresh() - - def _close_debug_session(self, scope_id: str) -> None: - active = self.debug_sessions.pop(scope_id, None) - if active is None or active.subject_id != scope_id: - return - terminal_status = self.batch.terminal_status(scope_id) - if terminal_status is None: - self.debug_terminal_summaries.pop(scope_id, None) - else: - summary = DebugTerminalSummary.from_session( - active, - terminal_status=terminal_status.value, - completed_at_unix=time.time(), - ) - snapshots = self.debug_snapshots.get(scope_id, ()) - latest = snapshots[-1] if snapshots else None - self.debug_terminal_summaries[scope_id] = ( - summary - if latest is None - else summary.with_snapshot( - snapshot=latest, - snapshot_id=latest.snapshot_id, - snapshot_store_ref=active.snapshot_store_ref, - snapshot_store_backend=active.snapshot_store_backend, - ) - ) - self.publish(ExecutionStateChanged(self.execution_state)) - - def debug_context(self, scope_id: str) -> DebugSessionProjectionContext: - return DebugSessionProjectionContext( - target=DebugSessionTargetState( - current_plate_scope_id=scope_id, - pipeline_scope_id=PipelineScopeIdentity.from_plate_scope( - scope_id - ).scope_id, - initialized=self.is_initialized(scope_id), - compiled=scope_id in self.compiled, - terminal_status=( - None - if (status := self.batch.terminal_status(scope_id)) is None - else status.value - ), - ), - session=self.displayed_debug_session(scope_id), - terminal_summary=self.debug_terminal_summaries.get(scope_id), - pause_boundaries=DebugPauseBoundaryState( - pause_step_indices=tuple( - index - for index, step in enumerate(self.pipeline_steps(scope_id)) - if step.debug_pause - ) - ), - snapshots=self.debug_snapshots.get(scope_id, ()), - manager_execution_state=self.execution_state, - ) - - def current_debug_context(self) -> DebugSessionProjectionContext | None: - if not self.current_scope_id: - return None - return self.debug_context(self.current_scope_id) From 7066af9f36a74edec1efd26178567104dd5fc4b0 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 05:07:47 -0400 Subject: [PATCH 52/84] B6: keep runtime-plane axes in a stream's display scope; official30 viewer test reads typed payload summaries required_scalar_components pooled every nested contributor across all runtime planes, so an axis that varies only BETWEEN planes (site on a stacked label image) looked "reduced into pixels" and was dropped from the stream's COMPONENT_ORDER. The payload still declared site as its plane axis, so Napari rejected every such batch ("component axis is not configured for stack display: 'site'") and the settle on the Napari port failed. Contributors are now grouped by the pixels they reduce into (top-level vs each runtime plane). The official30 Napari state check indexed ViewerPayloadSummary like a dict and reconstructed per-site identities by expanding aggregate payloads; it now reads the typed summary and compares the producer's expected payloads with the payloads of all of that producer's layers (image and ROI shapes). Co-Authored-By: Claude Opus 5.5 --- openhcs/core/source_image_provenance.py | 55 ++++++++------ .../test_cellprofiler_official30_zmq.py | 71 +++++++------------ tests/unit/test_aggregate_source_scope.py | 27 +++++++ 3 files changed, 88 insertions(+), 65 deletions(-) diff --git a/openhcs/core/source_image_provenance.py b/openhcs/core/source_image_provenance.py index d2f3d3d634..dc4bed5564 100644 --- a/openhcs/core/source_image_provenance.py +++ b/openhcs/core/source_image_provenance.py @@ -642,6 +642,25 @@ def contributors(self) -> tuple[SourceImageProvenanceContributor, ...]: ) ) + @property + def contributor_groups( + self, + ) -> tuple[tuple[SourceImageProvenanceContributor, ...], ...]: + """Contributors grouped by the pixels they reduce into: top-level ones + into the whole payload, nested ones into their own runtime plane.""" + return ( + tuple( + plane + for plane in self.planes + if isinstance(plane, SourceImageProvenanceContributor) + ), + *( + plane.contributors + for plane in self.planes + if not isinstance(plane, SourceImageProvenanceContributor) + ), + ) + @property def represented_source_image_names(self) -> tuple[str, ...]: """Return every alias owned by a runtime plane or pixel contributor.""" @@ -1067,33 +1086,27 @@ def required_scalar_components( """Exclude only coordinates proven reduced into pixel contributors. Contributors do not own a projectable runtime axis. A component that - varies across their complete source addresses therefore has no scalar - address on the derived image. Absent or incomplete contributor evidence - cannot exempt an acquired plane from its required coordinates. + varies across the contributors of one pixel group therefore has no + scalar address on the derived image; one that varies only between + runtime planes is that plane axis. Absent or incomplete contributor + evidence cannot exempt an acquired plane from its required coordinates. """ - contributors = self.source_image_provenance_planes.contributors scalar_components = SourceMetadataFields.component_domains( self.scalar_source_identity.component_metadata or {} ) - contributor_components = tuple( - SourceMetadataFields.component_domains(contributor.component_metadata or {}) - for contributor in contributors - ) reduced = set() - for component in AxisFamily.active().axes: - if component in scalar_components: - continue - if len(contributor_components) < 2 or not all( - len(values.get(component, ())) == 1 for values in contributor_components - ): - continue - values = frozenset( - value - for values in contributor_components - for value in values[component] + for contributors in self.source_image_provenance_planes.contributor_groups: + domains = tuple( + SourceMetadataFields.component_domains(item.component_metadata or {}) + for item in contributors ) - if len(values) > 1: - reduced.add(component.name) + for component in AxisFamily.active().axes: + if component in scalar_components or len(domains) < 2: + continue + if not all(len(values.get(component, ())) == 1 for values in domains): + continue + if len({values[component][0] for values in domains}) > 1: + reduced.add(component.name) return tuple( component for component in component_order if component not in reduced ) diff --git a/tests/integration/test_cellprofiler_official30_zmq.py b/tests/integration/test_cellprofiler_official30_zmq.py index 2b9d024417..ced0397664 100644 --- a/tests/integration/test_cellprofiler_official30_zmq.py +++ b/tests/integration/test_cellprofiler_official30_zmq.py @@ -1,7 +1,5 @@ from __future__ import annotations -from itertools import product - import json import os import shutil @@ -404,7 +402,7 @@ def _assert_napari_state_matches_runtime( runtime_artifact_viewer_component_identity(values) for values in layer.component_values ) == Counter( - runtime_artifact_viewer_component_identity(payload["components"]) + runtime_artifact_viewer_component_identity(payload.components) for payload in layer.payload_summaries ) assert all(component in layer.axis_labels for component in layer.stack_axes) @@ -416,59 +414,44 @@ def _assert_napari_state_matches_runtime( assert layer.data_shape[axis_index] == len(component_values) for expected in runtime_observation.expectation.artifact_viewer: - # One producer identity can legitimately back both its native image - # layer and its derived shapes overlay; the payload expectation below - # compares ndarray summaries, so match the image layer. - matching_layers = tuple( + # One producer identity can back both its native image layer and its + # derived shapes overlay. The producer's expected payloads are every + # payload it streamed, so compare against all of its layers' payloads. + producer_layers = tuple( layer for layer in state.layers if expected.producer_identity in layer.producer_identities - and StreamingDataType.IMAGE.value in layer.data_types ) - assert len(matching_layers) == 1 - layer = matching_layers[0] + image_layers = tuple( + layer + for layer in producer_layers + if StreamingDataType.IMAGE.value in layer.data_types + ) + assert len(image_layers) == 1 + layer = image_layers[0] assert layer.mounted - # The presentation may aggregate a varying component axis (e.g. site) - # into one layer item. That item represents both the aggregate payload - # itself and each per-coordinate payload it covers, so the actual - # identity set is the union of the summary's own identity and every - # expanded per-coordinate identity. - actual_payloads: list[RuntimeArtifactViewerPayloadExpectation] = [] - for payload in layer.payload_summaries: - aggregates = payload.get("aggregate_component_values") or {} - varying_axes = tuple(aggregates.items()) - - def summary_identity(components) -> RuntimeArtifactViewerPayloadExpectation: - return RuntimeArtifactViewerPayloadExpectation( - components=runtime_artifact_viewer_component_identity(components), - source_spatial_domain=SourceSpatialDomain.from_viewer_wire_mapping( - payload, - source_label="official30 Napari payload summary", - ), - ) - - actual_payloads.append(summary_identity(payload["components"])) - axes_values = [values for _, values in varying_axes] - for combination in product(*axes_values): - expanded = dict(payload["components"]) - expanded.update( - { - component: value - for (component, _), value in zip( - varying_axes, combination, strict=True - ) - } - ) - actual_payloads.append(summary_identity(expanded)) + actual_payloads = tuple( + RuntimeArtifactViewerPayloadExpectation( + components=runtime_artifact_viewer_component_identity( + payload.components + ), + source_spatial_domain=SourceSpatialDomain.from_viewer_wire_mapping( + payload.to_wire_mapping(), + source_label="official30 Napari payload summary", + ), + ) + for producer_layer in producer_layers + for payload in producer_layer.payload_summaries + ) assert Counter(payload.identity_key for payload in actual_payloads) == Counter( payload.identity_key for payload in expected.payloads ) - payload_paths = tuple(payload["path"] for payload in layer.payload_summaries) + payload_paths = tuple(payload.path for payload in layer.payload_summaries) assert all(isinstance(path, str) and path for path in payload_paths) payload_data_types = tuple( - dict.fromkeys(payload["data_type"] for payload in layer.payload_summaries) + dict.fromkeys(payload.data_type for payload in layer.payload_summaries) ) assert len(payload_data_types) == 1 assert layer.data_types == payload_data_types diff --git a/tests/unit/test_aggregate_source_scope.py b/tests/unit/test_aggregate_source_scope.py index 1b590e100a..dd36c0c891 100644 --- a/tests/unit/test_aggregate_source_scope.py +++ b/tests/unit/test_aggregate_source_scope.py @@ -44,6 +44,7 @@ from openhcs.core.source_image_provenance import ( RuntimeSourceImageProvenancePlane, SourceImageIdentity, + SourceImageProvenanceContributor, SourceImageProvenancePlanes, ) from openhcs.core.source_metadata import SourceVoxelSpacing @@ -180,6 +181,32 @@ def test_absence_one_equal_or_incomplete_contributors_cannot_exempt_acquired_pla adapter.filemanager_batches((output,)) +def test_axis_varying_between_runtime_planes_is_kept_not_reduced(): + common = {"well": "A01", "z_index": 1, "timepoint": 1} + planes = tuple( + RuntimeSourceImageProvenancePlane( + SourceImageIdentity(component_metadata={**common, "site": site}), + contributors=tuple( + SourceImageProvenanceContributor( + SourceImageIdentity( + component_metadata={**common, "site": site, "channel": channel} + ) + ) + for channel in (1, 2) + ), + ) + for site in (1, 2) + ) + metadata = ImagePayloadMetadata( + plane_axis=RuntimePlaneAxis.SOURCE_BINDING, + source_component_metadata=common, + source_image_provenance_planes=SourceImageProvenancePlanes(planes), + ) + assert metadata.source_provenance.required_scalar_components( + ("site", "channel", "well") + ) == ("site", "well") + + def test_present_scalar_override_remains_required_despite_varied_contributors(): metadata = (_source((1, 3))).metadata.collapse_leading_plane_axis() metadata = metadata.replace_fields( From 7f5710ae93527bf478784a74718bebb20b15e676 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 05:09:16 -0400 Subject: [PATCH 53/84] B6: black test_debug_runtime.py Co-Authored-By: Claude Opus 5.5 --- tests/unit/test_debug_runtime.py | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/tests/unit/test_debug_runtime.py b/tests/unit/test_debug_runtime.py index 3baee48e0d..a532482f76 100644 --- a/tests/unit/test_debug_runtime.py +++ b/tests/unit/test_debug_runtime.py @@ -177,6 +177,7 @@ def lane(cls) -> WorkerLaneExecutionContext: worker_slot="worker_0", worker_assignments={"worker_0": ("A01",)}, ) + SEGMENT_NAME = "segment" @staticmethod @@ -1006,8 +1007,7 @@ def test_debug_step_executes_one_function_pattern_invocation(): np.testing.assert_array_equal(result, np.array([2, 4, 6])) assert [ - DebugProgressContext.from_context(event.context).event_type - for event in emitted + DebugProgressContext.from_context(event.context).event_type for event in emitted ] == [DebugEventType.BEFORE_INVOCATION, DebugEventType.AFTER_INVOCATION] @@ -1043,7 +1043,9 @@ def test_debug_step_can_advance_past_current_function_pattern_invocation(): np.testing.assert_array_equal(result, np.array([2, 3, 4])) assert len(emitted) == 2 - assert DebugProgressContext.from_context(emitted[0].context).cursor.invocation_key.endswith(":1:plus_one") + assert DebugProgressContext.from_context( + emitted[0].context + ).cursor.invocation_key.endswith(":1:plus_one") def test_execute_chain_emits_debug_exception_event(): @@ -1122,10 +1124,7 @@ def test_projection_exception_contains_complete_invocation_context_without_debug assert "declared_cardinality=2" in message assert "selected_runtime_plane=preserved_stack" in message assert "execution_axis_cardinality=3" in message - assert ( - "image_payload_execution_mode=FullStackExecution" - in message - ) + assert "image_payload_execution_mode=FullStackExecution" in message assert "processing_contract=Pure2DContract" in message assert isinstance(exc.__cause__, RuntimeSliceProjectionDeclarationError) assert str(exc.__cause__) == "projection preflight mismatch" @@ -1311,7 +1310,8 @@ def test_progress_debug_event_sink_uses_injected_progress_emitter(): assert emitted[0].status is ProgressStatus.STARTED assert emitted[0].context is not None assert ( - emitted[0].context[DebugProgressContext.context_key]["debug_session_id"] == DebugRuntimeFixture.DEBUG_SESSION_ID + emitted[0].context[DebugProgressContext.context_key]["debug_session_id"] + == DebugRuntimeFixture.DEBUG_SESSION_ID ) assert ( emitted[0].context[DebugProgressContext.context_key]["snapshot_store_ref"] @@ -1489,9 +1489,9 @@ def test_debug_execution_policy_skips_before_start_step_index(): def test_debug_session_marks_current_cursor_dirty_nominally(): cursor = DebugRuntimeFixture.cursor() - session = DebugSession.create(subject_id=DebugRuntimeFixture.SUBJECT_ID).with_cursor( - cursor - ) + session = DebugSession.create( + subject_id=DebugRuntimeFixture.SUBJECT_ID + ).with_cursor(cursor) dirty_session = session.mark_dirty_from_cursor() From a1e3bff862423a0645859e4174667507f6fa0bc2 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 05:09:19 -0400 Subject: [PATCH 54/84] B6: re-pin documentation authorities after the Windows and viewer fixes Co-Authored-By: Claude Opus 5.5 --- docs/audits/architecture_a_m.json | 14 +++++++++----- docs/audits/architecture_n_z_development.json | 15 ++++++++++----- docs/audits/project_surface.json | 5 +++-- docs/audits/user_docs.json | 9 ++++++--- 4 files changed, 28 insertions(+), 15 deletions(-) diff --git a/docs/audits/architecture_a_m.json b/docs/audits/architecture_a_m.json index dc900a0c43..b307767d24 100644 --- a/docs/audits/architecture_a_m.json +++ b/docs/audits/architecture_a_m.json @@ -338,7 +338,7 @@ }, { "path": "openhcs/authoring/session/session.py", - "sha256": "587047b63972df89ec46a01ed2c92ff3ef9577335f58a980297db02094e4e8e0", + "sha256": "c1446fdf475503848a62086d13bc3e809dadd4958c3f763ea0347264bab5647d", "role": "Implements run_datasets (reserve, reset, connect, compile all, submit each with compile_artifact_id) and the endpoint status binding." }, { @@ -362,6 +362,7 @@ ], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/batch_workflow_service.rst against B6's later changes to openhcs/authoring/session/session.py: the session decides in-place initialization by an actual file-creation attempt that leaves nothing behind, its debug-session methods moved into an inherited mixin, the measured CLI treats Windows Ctrl+Break like Ctrl+C, and the Official30 viewer test reads typed payload summaries. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/batch_workflow_service.rst against B6's later changes to openhcs/authoring/session/operations/datasets.py, openhcs/authoring/session/compile_batch.py, openhcs/authoring/session/session.py, openhcs/authoring/session/submission.py, openhcs/authoring/session/progress.py, tests/unit/test_u1_session_guards.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/batch_workflow_service.rst against the later B6 changes to openhcs/authoring/session/operations/datasets.py, openhcs/authoring/session/compile_batch.py, openhcs/authoring/session/session.py, openhcs/authoring/session/submission.py, openhcs/authoring/session/progress.py, tests/unit/test_u1_session_guards.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/batch_workflow_service.rst in full and compared it against the changed authorities: the deleted openhcs/pyqt_gui batch workflow, compile batch, components, client service and execution-state modules and their tests (replaced by openhcs/authoring/session), openhcs/pyqt_gui/main.py, test_main_zmq_window_routing.py, endpoint_function_catalog_service.py, zmqruntime client.py and test_concurrent_initialization_progress.py; checked CompileDatasets availability, DatasetPipelineRequest/CompiledDataset, BatchSubmitWaitEngine, run_datasets ordering and compile_artifact_id submission, ExecutionProgress listener and interval rebuild, and the endpoint status binding against the session package. The page text is accurate; the record now points at the session owners.", @@ -434,7 +435,7 @@ }, { "path": "tests/integration/test_cellprofiler_official30_zmq.py", - "sha256": "3297b0e54ad3a349d35b613bd7d406e92c974320e12723be80dcfc0a90be2116", + "sha256": "e9f346e25a9fbb1ec7edf9e6e19c6d51c7d2000347ee61ef41b3017244322589", "role": "Defines the executable Official30 import, execution, equivalence, and viewer evidence boundary." }, { @@ -458,6 +459,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/cellprofiler_interop.rst against B6's later changes to tests/integration/test_cellprofiler_official30_zmq.py: the session decides in-place initialization by an actual file-creation attempt that leaves nothing behind, its debug-session methods moved into an inherited mixin, the measured CLI treats Windows Ctrl+Break like Ctrl+C, and the Official30 viewer test reads typed payload summaries. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/cellprofiler_interop.rst against B6's later changes to openhcs/interop/cellprofiler/pipeline_import.py, openhcs/interop/cellprofiler/module_declarations.py, openhcs/interop/cellprofiler/module_artifact_declarations.py, openhcs/core/pipeline/compilation_session.py, tests/unit/test_cellprofiler_artifact_declarations.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/cellprofiler_interop.rst against the later B6 changes to openhcs/interop/cellprofiler/pipeline_import.py, openhcs/interop/cellprofiler/module_declarations.py, openhcs/interop/cellprofiler/module_artifact_declarations.py, openhcs/core/pipeline/compilation_session.py, tests/unit/test_cellprofiler_artifact_declarations.py, tests/integration/test_cellprofiler_official30_zmq.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/cellprofiler_interop.rst in full and compared it against the changed authorities: pipeline_import.py, module_declarations.py, module_artifact_declarations.py, test_cellprofiler_artifact_declarations.py, the Official30 integration test, test_cellprofiler_pipeline_import.py and benchmark/cellprofiler_reference_exports.py; replaced the removed StepSnapshot with ResolvedPipelineDefinition and confirmed the import flow, interactive-module rejection, InputSource members, FunctionStepExecutionScope.PLATE, the provider's named methods, the CPA SQLite/properties route and CellProfilerReferenceExportPlan against current source.", @@ -2118,7 +2120,7 @@ }, { "path": "openhcs/authoring/session/session.py", - "sha256": "587047b63972df89ec46a01ed2c92ff3ef9577335f58a980297db02094e4e8e0", + "sha256": "c1446fdf475503848a62086d13bc3e809dadd4958c3f763ea0347264bab5647d", "role": "Defines the headless Session whose datasets and pipelines live in the MCP process's ObjectState and whose events clients follow." }, { @@ -2228,7 +2230,7 @@ }, { "path": "benchmark/cellprofiler_benchmark_cli.py", - "sha256": "4025c591644e29f3df61e476ecae2143690e014336ea9f889aadeff4e692445e", + "sha256": "2e0de28a96c57ee150e28fbbed28d876507e9281075a03f729838f3ca3610966", "role": "Owns the installed openhcs-benchmark execution, report and list-cases command surface." } ], @@ -2243,6 +2245,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/mcp_distribution.rst against B6's later changes to openhcs/authoring/session/session.py, benchmark/cellprofiler_benchmark_cli.py: the session decides in-place initialization by an actual file-creation attempt that leaves nothing behind, its debug-session methods moved into an inherited mixin, the measured CLI treats Windows Ctrl+Break like Ctrl+C, and the Official30 viewer test reads typed payload summaries. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/mcp_distribution.rst against B6's later changes to openhcs/mcp/server.py, openhcs/mcp/dev_client_core.py, openhcs/mcp/dev_client_session.py, openhcs/mcp/context.py, openhcs/authoring/session/session.py, openhcs/pyqt_gui/services/ui_bridge_windows.py, benchmark/control_service.py, benchmark/cellprofiler_benchmark_cli.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/mcp_distribution.rst against the later B6 changes to openhcs/agent/capabilities.py, openhcs/mcp/server.py, openhcs/mcp/execution.py, openhcs/mcp/dev_client_core.py, openhcs/mcp/dev_client_session.py, openhcs/mcp/context.py, openhcs/authoring/session/session.py, openhcs/pyqt_gui/services/ui_bridge_windows.py, benchmark/control_service.py, benchmark/cellprofiler_benchmark_cli.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/mcp_distribution.rst in full and compared it against the changed authorities: openhcs/agent/capabilities.py, openhcs/mcp/server.py, stdio.py, context.py, http.py, client_registration.py, dev_client*.py, the deleted openhcs/desktop_deployment.py and desktop_installation.py (moved to openhcs.desktop in 73a2aece2), ui_bridge_windows.py, synthetic-plate and benchmark control owners and tests; verified every named symbol by grep, traced McpTransportExecutor, the binder's dispatcher path, SessionJourney.run_source and skill synchronisation; corrected the PyQt/threading, session-execution, desktop-deployment, client-registration and finalisation passages.", @@ -2388,7 +2391,7 @@ }, { "path": "tests/integration/test_cellprofiler_official30_zmq.py", - "sha256": "3297b0e54ad3a349d35b613bd7d406e92c974320e12723be80dcfc0a90be2116", + "sha256": "e9f346e25a9fbb1ec7edf9e6e19c6d51c7d2000347ee61ef41b3017244322589", "role": "Implements the headless and non-persistent Napari acceptance routes over all manifest cases, plus Fiji variant tests." }, { @@ -2417,6 +2420,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/measurement_equivalence_system.rst against B6's later changes to tests/integration/test_cellprofiler_official30_zmq.py: the session decides in-place initialization by an actual file-creation attempt that leaves nothing behind, its debug-session methods moved into an inherited mixin, the measured CLI treats Windows Ctrl+Break like Ctrl+C, and the Official30 viewer test reads typed payload summaries. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/measurement_equivalence_system.rst against B6's later changes to openhcs/core/equivalence/keys.py, openhcs/core/equivalence/measurement_features.py, openhcs/core/measurement_dialect.py, benchmark/control_service.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/measurement_equivalence_system.rst against the later B6 changes to openhcs/core/equivalence/keys.py, openhcs/core/equivalence/measurement_features.py, openhcs/core/measurement_dialect.py, benchmark/adapters/openhcs.py, benchmark/openhcs_measured_run.py, benchmark/control_service.py, tests/integration/test_cellprofiler_official30_zmq.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/measurement_equivalence_system.rst in full and compared it against the changed authorities: openhcs/core/equivalence/policy.py and keys.py, the deleted openhcs/core/equivalence/report.py (moved to benchmark/equivalence in f16177ed7), benchmark/adapters/openhcs.py, openhcs_measured_run.py, timing.py, control_service.py, openhcs/runtime/zmq_execution_observation.py, measured_run_receipt.py, the Official30 manifest and integration test, reference exports and their tests; corrected the dialect name, the comparison-package boundary, archived-export readability, the session-based finalisation path and the additive-total rule.", diff --git a/docs/audits/architecture_n_z_development.json b/docs/audits/architecture_n_z_development.json index fb3dad8897..c3ed7dd648 100644 --- a/docs/audits/architecture_n_z_development.json +++ b/docs/audits/architecture_n_z_development.json @@ -675,7 +675,7 @@ "authority": [ { "path": "openhcs/authoring/session/session.py", - "sha256": "587047b63972df89ec46a01ed2c92ff3ef9577335f58a980297db02094e4e8e0", + "sha256": "c1446fdf475503848a62086d13bc3e809dadd4958c3f763ea0347264bab5647d", "role": "Session class, MainThread/DispatcherThread, DatasetAccess and Renderer ports, dataset add/delete/split and work/mutation checks." }, { @@ -740,6 +740,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/plate_manager_services.rst against B6's later changes to openhcs/authoring/session/session.py: the session decides in-place initialization by an actual file-creation attempt that leaves nothing behind, its debug-session methods moved into an inherited mixin, the measured CLI treats Windows Ctrl+Break like Ctrl+C, and the Official30 viewer test reads typed payload summaries. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/plate_manager_services.rst against B6's later changes to openhcs/authoring/session/session.py, openhcs/authoring/session/views.py, openhcs/authoring/session/events.py, openhcs/authoring/session/compile_batch.py, openhcs/authoring/session/dataset_document.py, openhcs/pyqt_gui/session_rendering.py, openhcs/mcp/server.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/plate_manager_services.rst against the later B6 changes to openhcs/authoring/session/session.py, openhcs/authoring/session/views.py, openhcs/authoring/session/events.py, openhcs/authoring/session/compile_batch.py, openhcs/authoring/session/dataset_document.py, openhcs/agent/capabilities.py, openhcs/pyqt_gui/session_rendering.py, openhcs/mcp/server.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/plate_manager_services.rst in full and compared it against the changed authorities: the page itself (rewritten in U1), deleted plate_manager_batch_workflow.py, batch_workflow_components.py, pipeline_object_state_binding.py and plate_manager_state_projection.py, changed plate_manager_workflows.py, pipeline_editor_workflows.py, endpoint_function_catalog_service.py, pyqt_gui/main.py, the Plate Manager and Pipeline Editor widget tests, and demo/synthetic_plate_pipeline.py; verified every named module in openhcs/authoring/session, SessionOperation/SessionView/SessionEvent families, DatasetListView, PipelineStepsView, DatasetDocumentScope, PipelineImporter keyed by suffix, the CellProfiler .cppipe dataset scope, QtSessionEventRelay, openhcs_session_events, DispatcherThread use in the GUI and MCP, compile-artifact ids on submission and compile-pending reservation; replaced all authorities with session-package owners.", @@ -1044,7 +1045,7 @@ }, { "path": "openhcs/authoring/session/session.py", - "sha256": "587047b63972df89ec46a01ed2c92ff3ef9577335f58a980297db02094e4e8e0", + "sha256": "c1446fdf475503848a62086d13bc3e809dadd4958c3f763ea0347264bab5647d", "role": "Session shared by the desktop application and MCP for progress, cancellation and results." }, { @@ -1067,6 +1068,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/quick_start.rst against B6's later changes to openhcs/authoring/session/session.py: the session decides in-place initialization by an actual file-creation attempt that leaves nothing behind, its debug-session methods moved into an inherited mixin, the measured CLI treats Windows Ctrl+Break like Ctrl+C, and the Official30 viewer test reads typed payload summaries. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/quick_start.rst against B6's later changes to openhcs/core/processing_contracts.py, openhcs/core/dataset_sources/source_bindings_source.py, openhcs/interop/cellprofiler/pipeline_import.py, openhcs/authoring/session/session.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/quick_start.rst against the later B6 changes to openhcs/core/processing_contracts.py, openhcs/core/source_bindings.py, openhcs/core/dataset_sources/source_bindings_source.py, openhcs/interop/cellprofiler/pipeline_import.py, openhcs/agent/capabilities.py, openhcs/authoring/session/session.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/quick_start.rst in full and compared it against the changed authorities: the page itself (edited in G5/G1), pipeline_document.py, agent/authoring_contexts.py, cellprofiler/pipeline_import.py, test_cellprofiler_pipeline_import.py, test_pipeline_document.py, agent/services/llm_context_service.py, test_progressive_authoring_context.py; verified every MCP tool name in agent/capabilities.py, the analysis-strategy section id in the knowledge manifest, the code-example imports (Microscopy.Channel, stack_percentile_normalize, Lazy*Config, PipelineDocumentCodec) and the MFD preset paths; corrected StepSnapshot, PURE_2D/PURE_3D, SourceBindingsHandler, the missing InputSource import and the progress owner. scripts/validate_docs.py reports no code-block or import finding for the page.", @@ -2202,7 +2204,7 @@ }, { "path": "openhcs/authoring/session/session.py", - "sha256": "587047b63972df89ec46a01ed2c92ff3ef9577335f58a980297db02094e4e8e0", + "sha256": "c1446fdf475503848a62086d13bc3e809dadd4958c3f763ea0347264bab5647d", "role": "Session owns the one execution client and keeps FinishedExecution (request, endpoint, ExecutionRecord) per execution." }, { @@ -2346,6 +2348,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/zmq_execution_service_extracted.rst against B6's later changes to openhcs/authoring/session/session.py: the session decides in-place initialization by an actual file-creation attempt that leaves nothing behind, its debug-session methods moved into an inherited mixin, the measured CLI treats Windows Ctrl+Break like Ctrl+C, and the Official30 viewer test reads typed payload summaries. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/zmq_execution_service_extracted.rst against B6's later changes to openhcs/runtime/zmq_execution_server.py, openhcs/runtime/zmq_execution_client.py, openhcs/authoring/session/submission.py, openhcs/authoring/session/session.py, openhcs/runtime/zmq_orchestrator_environment.py, tests/unit/test_zmq_submission_timeout.py, openhcs/processing/backends/lib_registry/registry_service.py, openhcs/processing/backends/lib_registry/unified_registry.py, openhcs/processing/backends/lib_registry/openhcs_registry.py, tests/unit/desktop/test_desktop_restart.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/zmq_execution_service_extracted.rst against the later B6 changes to openhcs/runtime/zmq_execution_server.py, external/zmqruntime/src/zmqruntime/transport_modes.py, openhcs/runtime/zmq_execution_client.py, openhcs/authoring/session/submission.py, openhcs/authoring/session/session.py, openhcs/runtime/zmq_orchestrator_environment.py, tests/unit/test_zmq_submission_timeout.py, openhcs/processing/backends/lib_registry/registry_service.py, openhcs/processing/backends/lib_registry/unified_registry.py, openhcs/processing/backends/lib_registry/openhcs_registry.py, tests/unit/test_library_registry_discovery.py, tests/unit/desktop/test_desktop_restart.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/zmq_execution_service_extracted.rst in full and compared it against the changed authorities: zmq_execution_server.py, zmq_execution_observation.py, zmq_execution_client.py, compiled_execution.py, function_catalog_preparation.py, the zmqruntime execution/messages/client/transport/timeout owners, tests/unit/test_zmq_submission_timeout.py and agent/services/execution_session_service.py (deleted, U1); verified every named class and the --prepare-capabilities mode exist, rewrote the headless job, cancellation and execution-subject paragraphs for Session/ExecutionSubmission and subject_id/execution_subject_id, moved the submission budget to ExecutionTransportConfig, and confirmed the compatibility check before submit and the benchmark execution-ID check.", @@ -2915,7 +2918,7 @@ }, { "path": "benchmark/cellprofiler_benchmark_cli.py", - "sha256": "4025c591644e29f3df61e476ecae2143690e014336ea9f889aadeff4e692445e", + "sha256": "2e0de28a96c57ee150e28fbbed28d876507e9281075a03f729838f3ca3610966", "role": "Owns registered benchmark command declarations and the derived installed parser." }, { @@ -2936,6 +2939,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/extension_workflows.rst against B6's later changes to benchmark/cellprofiler_benchmark_cli.py: the session decides in-place initialization by an actual file-creation attempt that leaves nothing behind, its debug-session methods moved into an inherited mixin, the measured CLI treats Windows Ctrl+Break like Ctrl+C, and the Official30 viewer test reads typed payload summaries. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/development/extension_workflows.rst against B6's later changes to openhcs/core/pipeline/compilation_session.py, openhcs/core/callable_contract.py, openhcs/core/compiled_step_plan.py, benchmark/cellprofiler_benchmark_cli.py, tests/unit/test_benchmark_control.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/development/extension_workflows.rst against the later B6 changes to pyproject.toml, openhcs/core/pipeline/compilation_session.py, openhcs/core/callable_contract.py, openhcs/core/compiled_step_plan.py, benchmark/cellprofiler_benchmark_cli.py, tests/unit/test_benchmark_control.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/extension_workflows.rst in full and compared it against the changed authorities: deleted openhcs/core/registry_strategies.py (strategy mixins moved to metaclass-registry, b221f8491), deleted openhcs/serialization/json.py (python-introspect codec, c0a53a1e0), compilation_session.py (StepSnapshot removed), arraybridge types.py, callable_contract.py, compiled_step_plan.py, framework_device_assignment.py, the benchmark CLI and run-receipt modules, and pyproject.toml. Corrected the StepSnapshot reference and the to_jsonable owner; confirmed ImageFileFormat, BenchmarkCliCommand, ComparisonSuiteRunDeclaration, ComparisonSuiteRunReceipt, MeasuredPipelineRunReceipt, ArtifactKey, CompiledExecutionBundle and MemoryType.import_environment exist at their owners.", @@ -3201,7 +3205,7 @@ }, { "path": "openhcs/authoring/session/session.py", - "sha256": "587047b63972df89ec46a01ed2c92ff3ef9577335f58a980297db02094e4e8e0", + "sha256": "c1446fdf475503848a62086d13bc3e809dadd4958c3f763ea0347264bab5647d", "role": "Session submission budget and runtime-observation export validation (writable root, no existing path, outcomes require a path)." }, { @@ -3277,6 +3281,7 @@ ], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/mcp_development.rst against B6's later changes to openhcs/authoring/session/session.py: the session decides in-place initialization by an actual file-creation attempt that leaves nothing behind, its debug-session methods moved into an inherited mixin, the measured CLI treats Windows Ctrl+Break like Ctrl+C, and the Official30 viewer test reads typed payload summaries. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/development/mcp_development.rst against B6's later changes to openhcs/mcp/dev_client_core.py, openhcs/agent/services/ui_bridge_service.py, openhcs/mcp/dev_client_commands/ui.py, openhcs/authoring/session/session.py, tests/unit/agent/test_mcp_server.py, openhcs/runtime/zmq_execution_client.py, tests/unit/test_zmq_submission_timeout.py, tests/unit/pyqt_gui/test_ui_agent_bridge.py, openhcs/mcp/server.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/development/mcp_development.rst against the later B6 changes to openhcs/mcp/dev_client_core.py, openhcs/agent/services/ui_bridge_service.py, openhcs/mcp/dev_client_commands/ui.py, openhcs/agent/capabilities.py, openhcs/authoring/session/session.py, tests/unit/agent/test_mcp_server.py, openhcs/runtime/zmq_execution_client.py, tests/unit/test_zmq_submission_timeout.py, tests/unit/pyqt_gui/test_ui_agent_bridge.py, openhcs/mcp/server.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/mcp_development.rst in full and compared it against the changed authorities: the page's own U1 update (session-operation tools replacing the job-status paragraph; selected-workflow initialize_datasets), dev_client.py, dev_client_commanding.py, dev_client_core.py, zmq_config.py, knowledge_pipeline.py, agent/dto/ui_bridge.py, agent/dto/viewer.py, dev_client_commands/ui.py and viewer.py, bootstrap.py, stdio.py, zmq_execution_client.py, ui_bridge_transport.py, ui_bridge_server.py, progress_stream.py, agent/dto/execution.py, server.py, the deleted execution_session_service.py and their tests. Confirmed every dev-client command and option in the examples, the full/desktop/core/authoring surface names, initialize_datasets as a UiSelectedPlateWorkflowKind value, the revision-token and idempotent-initialization wait criteria, the 5000 ms UI and viewer control bounds, the 30000 ms submission default and pre-dispatch timeout error, the openhcs_ session tools, event_sequence, and the runtime-observation export checks; replaced the deleted and no-longer-relevant authorities.", diff --git a/docs/audits/project_surface.json b/docs/audits/project_surface.json index dbd4d2b422..f991362d51 100644 --- a/docs/audits/project_surface.json +++ b/docs/audits/project_surface.json @@ -82,12 +82,12 @@ }, { "path": "tests/integration/test_cellprofiler_official30_zmq.py", - "sha256": "3297b0e54ad3a349d35b613bd7d406e92c974320e12723be80dcfc0a90be2116", + "sha256": "e9f346e25a9fbb1ec7edf9e6e19c6d51c7d2000347ee61ef41b3017244322589", "role": "Exercises all 30 declared CellProfiler comparison cases over ZMQ and validates live Napari and Fiji projections." }, { "path": "benchmark/cellprofiler_benchmark_cli.py", - "sha256": "4025c591644e29f3df61e476ecae2143690e014336ea9f889aadeff4e692445e", + "sha256": "2e0de28a96c57ee150e28fbbed28d876507e9281075a03f729838f3ca3610966", "role": "Declares the openhcs-benchmark subcommands and options, including run-measured through the OpenHCS session." }, { @@ -121,6 +121,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared README.md against B6's later changes to tests/integration/test_cellprofiler_official30_zmq.py, benchmark/cellprofiler_benchmark_cli.py: the session decides in-place initialization by an actual file-creation attempt that leaves nothing behind, its debug-session methods moved into an inherited mixin, the measured CLI treats Windows Ctrl+Break like Ctrl+C, and the Official30 viewer test reads typed payload summaries. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared README.md against B6's later changes to openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/progress/types.py, openhcs/core/artifacts.py, openhcs/core/dataset_sources/source.py, openhcs/processing/backends/lib_registry/unified_registry.py, openhcs/core/viewer_streaming_service.py, openhcs/interop/cellprofiler/pipeline_import.py, benchmark/cellprofiler_benchmark_cli.py, tests/unit/test_benchmark_control.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared README.md against the later B6 changes to pyproject.toml, openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/progress/types.py, openhcs/core/artifacts.py, openhcs/core/dataset_sources/source.py, openhcs/processing/backends/lib_registry/unified_registry.py, openhcs/core/viewer_streaming_service.py, openhcs/agent/capabilities.py, openhcs/interop/cellprofiler/pipeline_import.py, tests/integration/test_cellprofiler_official30_zmq.py, benchmark/cellprofiler_benchmark_cli.py, tests/unit/test_benchmark_control.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read README.md in full and compared it against the changed authorities: pyproject.toml, openhcs/desktop_installation.py (moved to openhcs/desktop/installation.py), openhcs/core/pipeline_document.py, openhcs/core/pipeline/compiler.py, openhcs/core/artifacts.py, openhcs/constants/constants.py, unified_registry.py, viewer_streaming_service.py, openhcs/agent/capabilities.py, openhcs/mcp/client_registration.py, pipeline_import.py, the Official30 integration test and the benchmark CLI/receipt/control modules; installed a progress queue in the CellProfiler example and renamed its root to dataset_root, rewrote the run-measured and finalizer paragraphs for the session-backed run, replaced microscope-handler wording with dataset sources, replaced the deleted StepSnapshot with ResolvedPipelineDefinition in the validation card and compilation diagram, and confirmed extras, console scripts, benchmark subcommands/options, MCP tool names, the full profile, client targets, and the eight submodules against current source.", diff --git a/docs/audits/user_docs.json b/docs/audits/user_docs.json index 2fc8ae0f18..b618740463 100644 --- a/docs/audits/user_docs.json +++ b/docs/audits/user_docs.json @@ -205,7 +205,7 @@ }, { "path": "benchmark/cellprofiler_benchmark_cli.py", - "sha256": "4025c591644e29f3df61e476ecae2143690e014336ea9f889aadeff4e692445e", + "sha256": "2e0de28a96c57ee150e28fbbed28d876507e9281075a03f729838f3ca3610966", "role": "Owns installed benchmark execution and report commands." }, { @@ -227,6 +227,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/appendices/research_impact.rst against B6's later changes to benchmark/cellprofiler_benchmark_cli.py: the session decides in-place initialization by an actual file-creation attempt that leaves nothing behind, its debug-session methods moved into an inherited mixin, the measured CLI treats Windows Ctrl+Break like Ctrl+C, and the Official30 viewer test reads typed payload summaries. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/appendices/research_impact.rst against B6's later changes to benchmark/cellprofiler_benchmark_cli.py, tests/unit/test_benchmark_control.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/appendices/research_impact.rst against the later B6 changes to pyproject.toml, benchmark/cellprofiler_benchmark_cli.py, tests/unit/test_benchmark_control.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/appendices/research_impact.rst in full and compared it against the changed authorities: pyproject.toml, benchmark/cellprofiler_benchmark_cli.py, benchmark/contracts/run_receipt.py, benchmark/control.py, tests/unit/test_benchmark_control.py; the CLI now runs through the OpenHCS session and receipts name a subject_id, but the openhcs-benchmark entry point, typed receipts, read-only inspect/report service and the archive path the page names all still exist, so no claim changed. Dropped benchmark/contracts/pipeline.py, which the page does not rely on.", @@ -1529,7 +1530,7 @@ }, { "path": "benchmark/cellprofiler_benchmark_cli.py", - "sha256": "4025c591644e29f3df61e476ecae2143690e014336ea9f889aadeff4e692445e", + "sha256": "2e0de28a96c57ee150e28fbbed28d876507e9281075a03f729838f3ca3610966", "role": "Owns packaged comparison execution and reporting commands." }, { @@ -1568,6 +1569,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/domain_expert_onboarding.rst against B6's later changes to benchmark/cellprofiler_benchmark_cli.py: the session decides in-place initialization by an actual file-creation attempt that leaves nothing behind, its debug-session methods moved into an inherited mixin, the measured CLI treats Windows Ctrl+Break like Ctrl+C, and the Official30 viewer test reads typed payload summaries. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/domain_expert_onboarding.rst against B6's later changes to openhcs/interop/cellprofiler/pipeline_import.py, benchmark/cellprofiler_benchmark_cli.py, tests/unit/test_benchmark_control.py, openhcs/interop/cellprofiler/module_artifact_declarations.py, openhcs/authoring/session/views.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/domain_expert_onboarding.rst against the later B6 changes to openhcs/agent/capabilities.py, openhcs/interop/cellprofiler/pipeline_import.py, benchmark/cellprofiler_benchmark_cli.py, tests/unit/test_benchmark_control.py, openhcs/interop/cellprofiler/module_artifact_declarations.py, openhcs/authoring/session/views.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/domain_expert_onboarding.rst in full and compared it against the changed authorities: agent/capabilities.py, knowledge_base_service.py, interop/cellprofiler/pipeline_import.py, test_architecture_projection_service.py, test_cellprofiler_pipeline_import.py, benchmark/cellprofiler_benchmark_cli.py, benchmark/contracts/run_receipt.py, benchmark/control.py, test_benchmark_control.py; confirmed openhcs_explain_architecture with topic cellprofiler_translation, the 50,000-character knowledge document bound, the interactive-module import rejection, the full MCP profile, the openhcs-benchmark entry point and read-only receipt/artifact inspection in benchmark/control.py; renamed the Plate Manager reference to the Datasets pane.", @@ -4318,7 +4320,7 @@ }, { "path": "benchmark/cellprofiler_benchmark_cli.py", - "sha256": "4025c591644e29f3df61e476ecae2143690e014336ea9f889aadeff4e692445e", + "sha256": "2e0de28a96c57ee150e28fbbed28d876507e9281075a03f729838f3ca3610966", "role": "Declares the list-cases, inspect-run, inspect-measured, and run-measured commands, including the required --wait-timeout-ms." }, { @@ -4369,6 +4371,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/user_guide/mcp_clients.rst against B6's later changes to benchmark/cellprofiler_benchmark_cli.py: the session decides in-place initialization by an actual file-creation attempt that leaves nothing behind, its debug-session methods moved into an inherited mixin, the measured CLI treats Windows Ctrl+Break like Ctrl+C, and the Official30 viewer test reads typed payload summaries. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/mcp_clients.rst against B6's later changes to openhcs/authoring/session/operations/datasets.py, benchmark/cellprofiler_benchmark_cli.py, benchmark/control_service.py, tests/unit/agent/test_mcp_server.py, tests/unit/test_benchmark_control.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/mcp_clients.rst against the later B6 changes to openhcs/agent/capabilities.py, openhcs/authoring/session/operations/datasets.py, benchmark/cellprofiler_benchmark_cli.py, benchmark/control_service.py, tests/unit/agent/test_mcp_server.py, tests/unit/test_benchmark_control.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/user_guide/mcp_clients.rst in full and compared it against the changed authorities: the page's own session-tool edit, openhcs/mcp/bootstrap.py, client_registration.py, openhcs/agent/capabilities.py, the mcpb manifest, test_mcp_server.py, gallery_catalog.py, the benchmark CLI, contracts, control and control_service, openhcs/agent/path_policy.py, and test_benchmark_control.py; confirmed every named session tool derives from a declared headless operation, the four surface names and desktop default, the benchmark subcommands and flags, the path-policy variables, and the packaging paths; restored the --wait-timeout-ms requirement and renamed the window.", From e1a1719d3a3bf7d9c16eb056ebbe502713933889 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 05:34:59 -0400 Subject: [PATCH 55/84] B6: Windows read-only fixture denies adding entries to the root only The inherited deny over the whole tree also refused reading the source files on the CI runner (PermissionError reading plate.HTD). Co-Authored-By: Claude Opus 5.5 --- tests/integration/test_measured_cli_session.py | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/tests/integration/test_measured_cli_session.py b/tests/integration/test_measured_cli_session.py index e51ca2096d..27ae7a7509 100644 --- a/tests/integration/test_measured_cli_session.py +++ b/tests/integration/test_measured_cli_session.py @@ -81,7 +81,9 @@ def _source_state(root: Path) -> dict[str, tuple]: # Everyone may not create, change or delete entries; inherited by the tree. # Write rights are named one by one: the generic W also denies SYNCHRONIZE, # which every open (reads included) requests. -WINDOWS_READ_ONLY_ACE = "*S-1-1-0:(OI)(CI)(WD,AD,WEA,WA,D,DC)" +# Deny adding files and folders to the root itself. An inherited deny over the +# tree (OI)(CI) also refused reading the source files on the CI runner. +WINDOWS_READ_ONLY_ACE = "*S-1-1-0:(WD,AD)" if sys.platform == "win32": # A console's Ctrl+C cannot be sent to one process; Ctrl+Break can, to a @@ -97,7 +99,8 @@ def _set_writable(root: Path, writable: bool) -> None: """Make ``root`` and its tree writable again, or genuinely not writable. Windows ignores the read-only attribute on directories (and os.access - ignores ACLs), so there an inherited deny ACE is what refuses writes. + ignores ACLs), so there a deny ACE on the root refuses creating files in it; + the before/after source state still catches any write below it. """ if sys.platform == "win32": From e82ae339843b455c29fe32857ebac69c1c122a3f Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 05:45:12 -0400 Subject: [PATCH 56/84] B6: wheel-integration prepares the declared Fiji bundle its Bio-Formats tests read The test session forbids ImageJ downloads (tests/conftest.py), and the wheel job installs the desktop extras whose Bio-Formats source tests need Fiji; the job now restores and prewarms the bundle as the real-viewer job does. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/integration-tests.yml | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/.github/workflows/integration-tests.yml b/.github/workflows/integration-tests.yml index 1d004578b0..e95d694a86 100644 --- a/.github/workflows/integration-tests.yml +++ b/.github/workflows/integration-tests.yml @@ -2443,6 +2443,30 @@ jobs: --extras "dev,dev-gui,cellprofiler-compat,$desktop_extras" \ --requirement pytest-split==0.11.0 + # The test session forbids ImageJ downloads (tests/conftest.py); the + # Bio-Formats source tests read the declared Fiji bundle from this cache. + - name: Cache managed Java and ImageJ artifacts + uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 + with: + path: | + ~/.cache/polystore/imagej + ~/.cache/cjdk + ~/.jgo + ~/.m2/repository + key: ${{ runner.os }}-openhcs-fiji-${{ hashFiles('pyproject.toml', 'external/PolyStore/src/polystore/imagej_distribution.py') }} + + - name: Prewarm managed Fiji runtime + run: | + source test_wheel/bin/activate + python - <<'PY' + import imagej + import scyjava as sj + from polystore.imagej_runtime import FIJI_IMAGEJ_RUNTIME + + ij = FIJI_IMAGEJ_RUNTIME.initialize(imagej, sj, mode="headless") + FIJI_IMAGEJ_RUNTIME.shutdown(ij, sj) + PY + - name: Smoke installed measured MCP protocol run: | source test_wheel/bin/activate From 5de70ef71ecd178429398efa96dbedca30d4ea45 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 05:46:02 -0400 Subject: [PATCH 57/84] B6: slice-aligned labels match a scalar image to its singleton root; demo failure names the execution error The installed neurite demo failed at PerNeuronNeuriteTopology: a scalar skeleton image with no plane projection received seed labels carrying a singleton runtime-slice axis, and plane_split refused it. The singleton projection (#477) lived only on MatchImageStackLabels; it now lives on ObjectLabelInputExecutionMode, so slice-aligned labels get it too. FullStackLabels keeps its declared label domain. DatasetRowState carries finished_execution_error (the server's error for the dataset's latest finished execution) and the installed demo puts it in InstalledDemoFailure, so CI shows why an execution failed. Co-Authored-By: Claude Opus 5.5 --- openhcs/agent/dto/session.py | 2 + openhcs/authoring/session/views.py | 40 +++-- openhcs/core/pipeline/function_contracts.py | 71 ++++---- openhcs/mcp/installed_demo.py | 3 +- ...test_singleton_rgb_label_invocation_477.py | 158 ++++++++++++++---- 5 files changed, 191 insertions(+), 83 deletions(-) diff --git a/openhcs/agent/dto/session.py b/openhcs/agent/dto/session.py index 9de3fa92a5..d7c743e5da 100644 --- a/openhcs/agent/dto/session.py +++ b/openhcs/agent/dto/session.py @@ -147,6 +147,8 @@ class DatasetRowState: debug_session_id: str | None = None finished_execution_id: str | None = None """The dataset's latest finished ordinary execution (its evidence id).""" + finished_execution_error: str | None = None + """The server's error for that execution, when it did not complete.""" scope_accent_color: str | None = None diff --git a/openhcs/authoring/session/views.py b/openhcs/authoring/session/views.py index 4f52d57c13..46dc17ee42 100644 --- a/openhcs/authoring/session/views.py +++ b/openhcs/authoring/session/views.py @@ -174,7 +174,11 @@ def orchestrator_state(self): if terminal is not None: return terminal.orchestrator_state orchestrator = ObjectStateRegistry.get_object(self.scope_id) - return orchestrator.state if isinstance(orchestrator, PipelineOrchestrator) else None + return ( + orchestrator.state + if isinstance(orchestrator, PipelineOrchestrator) + else None + ) @property def execution_id(self) -> str | None: @@ -255,7 +259,9 @@ def state_of(cls, session: "Session") -> DatasetListState: selected = set(session.selected_scope_ids) return DatasetListState( rows=tuple( - cls.row_state(session, row, relations[row.scope_id], row.scope_id in selected) + cls.row_state( + session, row, relations[row.scope_id], row.scope_id in selected + ) for row in rows ), selected_scope_ids=session.selected_scope_ids, @@ -281,6 +287,16 @@ def row_state( terminal = activity.terminal_status runtime = activity.runtime debug_phase = activity.debug_phase + finished_id, finished = next( + ( + (execution_id, finished) + for execution_id, finished in reversed( + session.finished_executions.items() + ) + if finished.scope_id == row.scope_id + ), + (None, None), + ) return DatasetRowState( scope_id=row.scope_id, name=row.name, @@ -307,15 +323,9 @@ def row_state( source_root=relation.source_root, debug_phase=None if debug_phase is None else debug_phase.value, debug_session_id=activity.debug_session_id, - finished_execution_id=next( - ( - execution_id - for execution_id, finished in reversed( - session.finished_executions.items() - ) - if finished.scope_id == row.scope_id - ), - None, + finished_execution_id=finished_id, + finished_execution_error=( + None if finished is None else finished.record.error ), ) @@ -328,7 +338,9 @@ def row_state( def _function_entries(function_spec) -> tuple: if function_spec is None: return () - entries = tuple(function_spec) if isinstance(function_spec, list) else (function_spec,) + entries = ( + tuple(function_spec) if isinstance(function_spec, list) else (function_spec,) + ) functions = [] for entry in entries: function = entry[0] if isinstance(entry, tuple) else entry @@ -393,7 +405,9 @@ def state_of( selected=step_scope_id in selected, dirty=bool(state.dirty_fields) if state is not None else False, default_diff=( - bool(state.signature_diff_fields) if state is not None else False + bool(state.signature_diff_fields) + if state is not None + else False ), description=step.description, debug_pause=step.debug_pause, diff --git a/openhcs/core/pipeline/function_contracts.py b/openhcs/core/pipeline/function_contracts.py index c79d486e06..30c0177151 100644 --- a/openhcs/core/pipeline/function_contracts.py +++ b/openhcs/core/pipeline/function_contracts.py @@ -80,9 +80,16 @@ def resolved_callable_type_hints(func: Callable) -> dict[str, Any]: from typing import _strip_annotations return { - **{name: _strip_annotations(parameter.annotation) for name, parameter in signature.parameters.items() - if parameter.annotation is not inspect.Parameter.empty}, - **({} if signature.return_annotation is inspect.Signature.empty else {"return": _strip_annotations(signature.return_annotation)}), + **{ + name: _strip_annotations(parameter.annotation) + for name, parameter in signature.parameters.items() + if parameter.annotation is not inspect.Parameter.empty + }, + **( + {} + if signature.return_annotation is inspect.Signature.empty + else {"return": _strip_annotations(signature.return_annotation)} + ), } @@ -175,16 +182,28 @@ def invocation_kwargs( runtime_projection: RuntimePlaneAxisValueProjection | None, semantic_controls: Mapping[str, Any], ) -> dict[str, Any]: - """Return the invocation kwargs once the image's execution mode is known.""" - del execution_mode, image_projection, runtime_projection + """Match a scalar image to its declared singleton runtime root. + + Labels can consume a singleton root only after the image's final + execution mode and retained plane projection have been resolved. + """ + if ( + execution_mode.per_plane + and image_projection is None + and runtime_projection is not None + and runtime_projection.axis_size == 1 + ): + from openhcs.core.runtime_slice_projection import RuntimeSliceProjection + + kwargs = RuntimeSliceProjection.kwargs_for_slice( + kwargs, runtime_projection.selected_plane(0) + ) return {**kwargs, **semantic_controls} @classmethod def semantic_label_payload(cls, source_projected_payload, completion_payload): """Return the label payload that represents this measurement domain.""" - raise TypeError( - f"{cls.__name__} declares no object-measurement label domain." - ) + raise TypeError(f"{cls.__name__} declares no object-measurement label domain.") @classmethod def image_execution_mode( @@ -247,16 +266,6 @@ def image_execution_mode(cls, labels, default, *, runtime_slice_count=None): del labels, default, runtime_slice_count return FullStackExecution - -class MatchImageStackLabels(ObjectLabelInputExecutionMode): - """Labels follow the image: a stack when the image needs one.""" - - key = "match_image_stack" - - @classmethod - def preserves_full_stack(cls, *, image_stack_required: bool) -> bool: - return image_stack_required - @classmethod def invocation_kwargs( cls, @@ -267,23 +276,19 @@ def invocation_kwargs( runtime_projection, semantic_controls, ): - """Match scalar images to their declared singleton runtime root. + """Full-stack label consumers keep their declared label domain.""" + del execution_mode, image_projection, runtime_projection + return {**kwargs, **semantic_controls} - Matching labels can consume a singleton root only after the image's - final execution mode and retained plane projection have been resolved. - """ - if ( - execution_mode.per_plane - and image_projection is None - and runtime_projection is not None - and runtime_projection.axis_size == 1 - ): - from openhcs.core.runtime_slice_projection import RuntimeSliceProjection - kwargs = RuntimeSliceProjection.kwargs_for_slice( - kwargs, runtime_projection.selected_plane(0) - ) - return {**kwargs, **semantic_controls} +class MatchImageStackLabels(ObjectLabelInputExecutionMode): + """Labels follow the image: a stack when the image needs one.""" + + key = "match_image_stack" + + @classmethod + def preserves_full_stack(cls, *, image_stack_required: bool) -> bool: + return image_stack_required def _artifact_spec_from_output_declaration( diff --git a/openhcs/mcp/installed_demo.py b/openhcs/mcp/installed_demo.py index 04ae0a01c6..c3139782da 100644 --- a/openhcs/mcp/installed_demo.py +++ b/openhcs/mcp/installed_demo.py @@ -398,7 +398,8 @@ def _execute_pipeline( (row,) = (row for row in datasets.rows if row.root == str(plate_path)) if row.terminal_status != TerminalExecutionStatus.COMPLETE.value: raise InstalledDemoFailure( - f"Portable neurite execution ended with {row.terminal_status}: {row}" + f"Portable neurite execution ended with {row.terminal_status}: " + f"{row.finished_execution_error}; row={row}" ) return row diff --git a/tests/unit/test_singleton_rgb_label_invocation_477.py b/tests/unit/test_singleton_rgb_label_invocation_477.py index 7ee07aea04..f658e092c4 100644 --- a/tests/unit/test_singleton_rgb_label_invocation_477.py +++ b/tests/unit/test_singleton_rgb_label_invocation_477.py @@ -1,15 +1,32 @@ """Scalar RGB invocations consume only an explicitly declared singleton root.""" + import numpy as np import pytest -from openhcs.core.artifacts import ArtifactSpec, ImageArtifactType, ObjectLabelsArtifactType +from openhcs.core.artifacts import ( + ArtifactSpec, + ImageArtifactType, + ObjectLabelsArtifactType, +) from openhcs.core.function_contract_metadata import FunctionContractAttribute from openhcs.core.runtime_image_values import ImageMetadataPayload, ImagePayloadMetadata -from openhcs.core.runtime_object_label_domains import ObjectLabelDomain, ObjectLabelDomainScope -from openhcs.core.runtime_object_labels import ObjectLabelSet, ObjectLabelVariantData, object_label_dense_array -from openhcs.core.runtime_plane_projection import RuntimePlaneAxis, RuntimePlaneProjection +from openhcs.core.runtime_object_label_domains import ( + ObjectLabelDomain, + ObjectLabelDomainScope, +) +from openhcs.core.runtime_object_labels import ( + ObjectLabelSet, + ObjectLabelVariantData, + object_label_dense_array, +) +from openhcs.core.runtime_plane_projection import ( + RuntimePlaneAxis, + RuntimePlaneProjection, +) from openhcs.core.runtime_slice_projection import RuntimeSliceProjectionDeclarationError -from openhcs.interop.cellprofiler.runtime.function_contract_execution import CellProfilerFunctionContractExecutor +from openhcs.interop.cellprofiler.runtime.function_contract_execution import ( + CellProfilerFunctionContractExecutor, +) from openhcs.processing.backends.cellprofiler.outlines import OverlayOutlinesModule from tests.unit.test_cellprofiler_module_execution import ( _FakeCellProfilerRuntime, @@ -25,64 +42,106 @@ ) from openhcs.core.pipeline.function_contracts import ( FullStackLabels, + MatchImageStackLabels, + SliceAlignedLabels, ) def _invocation_case(*, root_count=1, label_count=1): - image_spec = ArtifactSpec.input('GFPandDNA', ImageArtifactType) - labels_spec = ArtifactSpec.input('Nuclei', ObjectLabelsArtifactType) + image_spec = ArtifactSpec.input("GFPandDNA", ImageArtifactType) + labels_spec = ArtifactSpec.input("Nuclei", ObjectLabelsArtifactType) raw = OverlayOutlinesModule.require_callable() - contract = _compiled_callable_contract(raw, artifact_inputs=(image_spec, labels_spec)) + contract = _compiled_callable_contract( + raw, artifact_inputs=(image_spec, labels_spec) + ) data = np.linspace(0, 1, 8 * 9 * 3, dtype=np.float32).reshape(8, 9, 3) - image = ImageMetadataPayload(data, ImagePayloadMetadata(axes=PayloadAxes.colour_samples(2), source_image_names=('GFPandDNA',))) + image = ImageMetadataPayload( + data, + ImagePayloadMetadata( + axes=PayloadAxes.colour_samples(2), source_image_names=("GFPandDNA",) + ), + ) label_data = np.zeros((label_count, 8, 9), dtype=np.int32) label_data[:, 2:5, 3:6] = 1 labels = ObjectLabelSet( - name='Nuclei', variant_data=ObjectLabelVariantData(labels=label_data), + name="Nuclei", + variant_data=ObjectLabelVariantData(labels=label_data), plane_axis=RuntimePlaneAxis.RUNTIME_SLICE, - domain=ObjectLabelDomain(declared_object_id_domains=((1,),) * label_count, scope=ObjectLabelDomainScope.PLANE), + domain=ObjectLabelDomain( + declared_object_id_domains=((1,),) * label_count, + scope=ObjectLabelDomainScope.PLANE, + ), ) adapter = _FakeCellProfilerRuntime( - {'GFPandDNA': image}, objects={'Nuclei': labels}, callable_contract=contract, - artifact_input_edges=tuple(_artifact_input_edge_for_test(spec) for spec in (image_spec, labels_spec)), + {"GFPandDNA": image}, + objects={"Nuclei": labels}, + callable_contract=contract, + artifact_input_edges=tuple( + _artifact_input_edge_for_test(spec) for spec in (image_spec, labels_spec) + ), plane_projection=RuntimePlaneProjection.stack(root_count), ) executor = _module_executor(contract) - request = executor._image_request(image, adapter, module_type=OverlayOutlinesModule, active_input_specs=(image_spec, labels_spec)) + request = executor._image_request( + image, + adapter, + module_type=OverlayOutlinesModule, + active_input_specs=(image_spec, labels_spec), + ) return raw, contract, image, labels, adapter, executor, request def _bind(case): raw, contract, image, labels, adapter, executor, request = case return executor._invocation_request( - image_request=request, adapter=adapter, current_image=image, - kwargs={'outline_colors': ('#21FFFF',)}, module_type=OverlayOutlinesModule, + image_request=request, + adapter=adapter, + current_image=image, + kwargs={"outline_colors": ("#21FFFF",)}, + module_type=OverlayOutlinesModule, ) def _execute(case, invocation): raw, contract, *_ = case return CellProfilerFunctionContractExecutor().execute( - contract, raw, invocation.payload, invocation.kwargs, - execution_mode=invocation.execution_mode, plane_projection=invocation.plane_projection, + contract, + raw, + invocation.payload, + invocation.kwargs, + execution_mode=invocation.execution_mode, + plane_projection=invocation.plane_projection, ) -def test_match_image_labels_consume_declared_singleton_root_for_scalar_rgb(): +@pytest.mark.parametrize("label_mode", (MatchImageStackLabels, SliceAlignedLabels)) +def test_plane_label_modes_consume_declared_singleton_root_for_scalar_rgb( + monkeypatch, label_mode +): + raw = OverlayOutlinesModule.require_callable() + monkeypatch.setattr( + raw, FunctionContractAttribute.object_label_input_execution_mode, label_mode + ) case = _invocation_case() raw, contract, image, labels, *_ = case image_before = image.data.copy() labels_before = object_label_dense_array(labels).copy() invocation = _bind(case) - selected = invocation.kwargs['object_labels'][0] + selected = invocation.kwargs["object_labels"][0] assert invocation.execution_mode is NaturalExecution assert invocation.plane_projection is None assert invocation.payload.metadata.axis_position(ColourAxis) == 2 assert selected.plane_axis is None assert object_label_dense_array(selected).shape == (8, 9) output = _execute(case, invocation) - scalar_labels = ObjectLabelSet(name='Nuclei', variant_data=ObjectLabelVariantData(labels=labels_before[0]), domain=ObjectLabelDomain(declared_object_ids=(1,), scope=ObjectLabelDomainScope.PAYLOAD)) - expected = raw(image, object_labels=(scalar_labels,), outline_colors=('#21FFFF',)) + scalar_labels = ObjectLabelSet( + name="Nuclei", + variant_data=ObjectLabelVariantData(labels=labels_before[0]), + domain=ObjectLabelDomain( + declared_object_ids=(1,), scope=ObjectLabelDomainScope.PAYLOAD + ), + ) + expected = raw(image, object_labels=(scalar_labels,), outline_colors=("#21FFFF",)) np.testing.assert_array_equal(output.data, expected.data) np.testing.assert_array_equal(image.data, image_before) np.testing.assert_array_equal(object_label_dense_array(labels), labels_before) @@ -90,49 +149,76 @@ def test_match_image_labels_consume_declared_singleton_root_for_scalar_rgb(): assert image.metadata.plane_axis is None -@pytest.mark.parametrize('root_count', (None, 2)) +@pytest.mark.parametrize("root_count", (None, 2)) def test_scalar_rgb_cannot_authorize_unknown_or_multiple_root_planes(root_count): case = _invocation_case(root_count=root_count) invocation = _bind(case) - assert invocation.kwargs['object_labels'][0].plane_axis is RuntimePlaneAxis.RUNTIME_SLICE - with pytest.raises(RuntimeSliceProjectionDeclarationError, match='Kwargs cannot create image-axis execution semantics'): + assert ( + invocation.kwargs["object_labels"][0].plane_axis + is RuntimePlaneAxis.RUNTIME_SLICE + ) + with pytest.raises( + RuntimeSliceProjectionDeclarationError, + match="Kwargs cannot create image-axis execution semantics", + ): _execute(case, invocation) -def test_explicit_full_stack_label_contract_preserves_scalar_boundary_refusal(monkeypatch): +def test_explicit_full_stack_label_contract_preserves_scalar_boundary_refusal( + monkeypatch, +): raw = OverlayOutlinesModule.require_callable() - monkeypatch.setattr(raw, FunctionContractAttribute.object_label_input_execution_mode, FullStackLabels) + monkeypatch.setattr( + raw, + FunctionContractAttribute.object_label_input_execution_mode, + FullStackLabels, + ) case = _invocation_case() invocation = _bind(case) - assert invocation.kwargs['object_labels'][0].plane_axis is RuntimePlaneAxis.RUNTIME_SLICE - with pytest.raises(RuntimeSliceProjectionDeclarationError, match='Kwargs cannot create image-axis execution semantics'): + assert ( + invocation.kwargs["object_labels"][0].plane_axis + is RuntimePlaneAxis.RUNTIME_SLICE + ) + with pytest.raises( + RuntimeSliceProjectionDeclarationError, + match="Kwargs cannot create image-axis execution semantics", + ): _execute(case, invocation) def test_declared_singleton_root_refuses_mismatched_label_cardinality(): case = _invocation_case(label_count=2) - with pytest.raises(ValueError, match='Object-label runtime plane-axis cardinality mismatch'): + with pytest.raises( + ValueError, match="Object-label runtime plane-axis cardinality mismatch" + ): _bind(case) def test_final_module_mode_sees_full_labels_before_scalar_projection(monkeypatch): case = _invocation_case() + def mode(cls, default, *, image, kwargs, variable_components): - assert kwargs['object_labels'][0].plane_axis is RuntimePlaneAxis.RUNTIME_SLICE - assert object_label_dense_array(kwargs['object_labels'][0]).shape == (1, 8, 9) + assert kwargs["object_labels"][0].plane_axis is RuntimePlaneAxis.RUNTIME_SLICE + assert object_label_dense_array(kwargs["object_labels"][0]).shape == (1, 8, 9) return FullStackExecution - monkeypatch.setattr(OverlayOutlinesModule, 'execution_mode', classmethod(mode)) + + monkeypatch.setattr(OverlayOutlinesModule, "execution_mode", classmethod(mode)) invocation = _bind(case) assert invocation.execution_mode is FullStackExecution - assert invocation.kwargs['object_labels'][0].plane_axis is RuntimePlaneAxis.RUNTIME_SLICE + assert ( + invocation.kwargs["object_labels"][0].plane_axis + is RuntimePlaneAxis.RUNTIME_SLICE + ) def test_module_mode_failure_precedes_projection_guard(monkeypatch): case = _invocation_case(label_count=2) - error = RuntimeError('module image-domain selection failed') + error = RuntimeError("module image-domain selection failed") + def mode(cls, default, *, image, kwargs, variable_components): raise error - monkeypatch.setattr(OverlayOutlinesModule, 'execution_mode', classmethod(mode)) + + monkeypatch.setattr(OverlayOutlinesModule, "execution_mode", classmethod(mode)) with pytest.raises(RuntimeError) as caught: _bind(case) assert caught.value is error From fbe17e02deb0cf35926a46483742e347d5348f2c Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 05:59:33 -0400 Subject: [PATCH 58/84] B6: check out tracked files as committed bytes on every platform Official30 parity on Windows failed because autocrlf rewrote the checked-in reference-export .cppipe files, so their sha256 no longer matched the sidecars. Command scripts keep platform line endings. Co-Authored-By: Claude Opus 5.5 --- .gitattributes | 7 +++++++ 1 file changed, 7 insertions(+) create mode 100644 .gitattributes diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000000..afe469b493 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,7 @@ +# Tracked files are checked out as their committed bytes on every platform. +# Benchmark corpora, reference-export sidecars and documentation audits are +# compared by sha256, which Windows end-of-line conversion would change. +* -text +# Windows command scripts keep the platform's line endings. +*.cmd text +*.bat text From f322e395b24314af19f7491edbf722523ee232ed Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 06:06:38 -0400 Subject: [PATCH 59/84] B6: the test session creates its QApplication first; MCP transport keeps adopting it Replaces deleting the transport's QCoreApplication on close (df75920fa): destroying the application while other QObjects and pool threads still lived is a plausible source of the later xcb segfault in pyqt_gui teardown. A process holds one Qt application; the session now creates the QApplication before any test, and in-process MCP transports adopt it, so later tests can still build widgets. Co-Authored-By: Claude Opus 5.5 --- openhcs/mcp/execution.py | 13 ++----------- tests/conftest.py | 11 +++++++++++ .../agent/test_mcp_process_resource_lifetime.py | 16 ---------------- 3 files changed, 13 insertions(+), 27 deletions(-) diff --git a/openhcs/mcp/execution.py b/openhcs/mcp/execution.py index a90b3a6428..094fe20495 100644 --- a/openhcs/mcp/execution.py +++ b/openhcs/mcp/execution.py @@ -9,7 +9,6 @@ import threading from typing import Any, ParamSpec, TypeVar -from PyQt6 import sip from PyQt6.QtCore import QCoreApplication, QEventLoop, QThread from polystore import cleanup_backend_connections from pyqt_reactive.core.future_completion import FutureCompletion @@ -88,11 +87,7 @@ def __init__(self) -> None: raise RuntimeError( "MCP transport must be started on the process main thread." ) - existing_application = QCoreApplication.instance() - # A process holds one Qt application; one this transport created is - # its own and goes with it, so a later GUI can create a QApplication. - self._created_application = existing_application is None - self._application = existing_application or QCoreApplication([]) + self._application = QCoreApplication.instance() or QCoreApplication([]) if QThread.currentThread() != self._application.thread(): raise RuntimeError("MCP transport requires the Qt application main thread.") super().__init__(max_workers=1) @@ -137,8 +132,4 @@ def close(self) -> None: try: super().close() finally: - try: - cleanup_backend_connections(include_process_resources=True) - finally: - if self._created_application: - sip.delete(self._application) + cleanup_backend_connections(include_process_resources=True) diff --git a/tests/conftest.py b/tests/conftest.py index ea7aecaf00..0a295dded3 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -68,6 +68,17 @@ def pytest_sessionstart(session): CPU_ONLY_MODE = OpenHCSProcessEnvironment.cpu_only_mode() if not CPU_ONLY_MODE: pytest_plugins = ["pytestqt"] + + @pytest.fixture(scope="session", autouse=True) + def session_qapplication(qapp): + """Create the session's one QApplication before any test runs. + + A process holds one Qt application. In-process MCP transports adopt + an existing one and otherwise create a QCoreApplication, after which + no test in the session could create a widget. + """ + yield qapp + else: pytest_plugins = [] diff --git a/tests/unit/agent/test_mcp_process_resource_lifetime.py b/tests/unit/agent/test_mcp_process_resource_lifetime.py index d351041c20..59b43a08c1 100644 --- a/tests/unit/agent/test_mcp_process_resource_lifetime.py +++ b/tests/unit/agent/test_mcp_process_resource_lifetime.py @@ -282,19 +282,3 @@ async def journey(): assert calls == ["process resource closed"] transport.execution.close() assert calls == ["process resource closed"] - - -def test_transport_releases_only_the_qt_application_it_created(): - from PyQt6.QtCore import QCoreApplication - - existing = QCoreApplication.instance() - executor = McpTransportExecutor() - application = QCoreApplication.instance() - executor.close() - - if existing is None: - # A later GUI in this process must be able to create its QApplication. - assert QCoreApplication.instance() is None - else: - assert application is existing - assert QCoreApplication.instance() is existing From 9630f350379a13a8b8d1f4b3affaef47cc20ff81 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 06:07:49 -0400 Subject: [PATCH 60/84] B6: re-pin documentation authorities after the demo, workflow and test-session fixes Co-Authored-By: Claude Opus 5.5 --- docs/audits/architecture_a_m.json | 9 ++++--- docs/audits/architecture_n_z_development.json | 24 ++++++++++++------- docs/audits/user_docs.json | 23 +++++++++++------- 3 files changed, 37 insertions(+), 19 deletions(-) diff --git a/docs/audits/architecture_a_m.json b/docs/audits/architecture_a_m.json index b307767d24..62a80af721 100644 --- a/docs/audits/architecture_a_m.json +++ b/docs/audits/architecture_a_m.json @@ -260,7 +260,7 @@ }, { "path": "openhcs/core/pipeline/function_contracts.py", - "sha256": "13bcaf7c7a62654f512d2cc739edccd2deeff35d99970ca493f3e827afbda91a", + "sha256": "f5e7c74ea3d2948c75388184263cd9f1f17905f820b180f634dfaa7514e1d526", "role": "Defines the special_inputs, artifact_inputs and artifact_outputs decorators." }, { @@ -290,6 +290,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/artifact_contract_system.rst against B6's later changes to openhcs/core/pipeline/function_contracts.py: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/artifact_contract_system.rst against B6's later changes to openhcs/core/artifacts.py, openhcs/core/callable_contract.py, openhcs/core/function_patterns.py, openhcs/core/pipeline/function_contracts.py, openhcs/core/pipeline/path_planner.py, tests/unit/test_implicit_main_flow_compilation.py, tests/unit/test_callable_contract.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/artifact_contract_system.rst against the later B6 changes to openhcs/core/artifacts.py, openhcs/core/callable_contract.py, openhcs/core/function_patterns.py, openhcs/core/pipeline/function_contracts.py, openhcs/core/pipeline/path_planner.py, tests/unit/test_implicit_main_flow_compilation.py, tests/unit/test_callable_contract.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/artifact_contract_system.rst in full and compared it against the changed authorities: openhcs/core/artifacts.py, openhcs/core/callable_contract.py, openhcs/core/pipeline/path_planner.py, tests/unit/test_callable_contract.py, tests/unit/test_artifact_input_projection_plan.py; replaced the removed consumes_main_flow flag with main_flow_projection in the occurrence and implicit main-flow sections and the regression description, tidied the special_inputs sentence, and confirmed every other named declaration (ArtifactType families, relation leaves, SourceImageProvenance, ArtifactPlanKeySelector, SelectedPlaneImageOutput, RuntimeValueStore, StepMaterializationConfig) exists with the stated role.", @@ -1139,7 +1140,7 @@ }, { "path": "openhcs/authoring/session/views.py", - "sha256": "8ce6cac27293ac9105f4d2ebc76d10eebeba1ef90e0e8954c0a117910e17e3b3", + "sha256": "c772d515dfa738363ab403a3e19c9e438c17b0748a3a9ed65c44c416f579a378", "role": "Declares the SessionView family, each view's title, and the operations its window binds." }, { @@ -1158,6 +1159,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/declarative_window_system.rst against B6's later changes to openhcs/authoring/session/views.py: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/declarative_window_system.rst against B6's later changes to openhcs/authoring/session/views.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/declarative_window_system.rst against the later B6 changes to openhcs/authoring/session/views.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/declarative_window_system.rst in full and compared it against the changed authorities: ui_bridge_identities.py, test_build_website.py; confirmed the pyqt-reactive window-manager and scope-window-factory pages and sources exist, traced UiSessionViewWidgetIdentityDeclaration to SessionView.rendered_in, and revised the OpenHCS-retained ownership sentence.", @@ -2080,7 +2082,7 @@ }, { "path": "openhcs/mcp/execution.py", - "sha256": "e1e3b4c649e8b62e873c10b92f9e27eedb5196e8e4890700929e40ab406c2170", + "sha256": "7e94fd4c876944803bdd7d5b93300cda1376326be848ff5636fe2de1b71db096", "role": "Runs the MCP SDK loop on a worker thread while a Qt core application and McpMainThreadDispatcher own main-thread calls." }, { @@ -2245,6 +2247,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/mcp_distribution.rst against B6's later changes to openhcs/mcp/execution.py: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/mcp_distribution.rst against B6's later changes to openhcs/authoring/session/session.py, benchmark/cellprofiler_benchmark_cli.py: the session decides in-place initialization by an actual file-creation attempt that leaves nothing behind, its debug-session methods moved into an inherited mixin, the measured CLI treats Windows Ctrl+Break like Ctrl+C, and the Official30 viewer test reads typed payload summaries. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/mcp_distribution.rst against B6's later changes to openhcs/mcp/server.py, openhcs/mcp/dev_client_core.py, openhcs/mcp/dev_client_session.py, openhcs/mcp/context.py, openhcs/authoring/session/session.py, openhcs/pyqt_gui/services/ui_bridge_windows.py, benchmark/control_service.py, benchmark/cellprofiler_benchmark_cli.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/mcp_distribution.rst against the later B6 changes to openhcs/agent/capabilities.py, openhcs/mcp/server.py, openhcs/mcp/execution.py, openhcs/mcp/dev_client_core.py, openhcs/mcp/dev_client_session.py, openhcs/mcp/context.py, openhcs/authoring/session/session.py, openhcs/pyqt_gui/services/ui_bridge_windows.py, benchmark/control_service.py, benchmark/cellprofiler_benchmark_cli.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", diff --git a/docs/audits/architecture_n_z_development.json b/docs/audits/architecture_n_z_development.json index c3ed7dd648..de332793c5 100644 --- a/docs/audits/architecture_n_z_development.json +++ b/docs/audits/architecture_n_z_development.json @@ -685,7 +685,7 @@ }, { "path": "openhcs/authoring/session/views.py", - "sha256": "8ce6cac27293ac9105f4d2ebc76d10eebeba1ef90e0e8954c0a117910e17e3b3", + "sha256": "c772d515dfa738363ab403a3e19c9e438c17b0748a3a9ed65c44c416f579a378", "role": "SessionView family with DatasetListView and PipelineStepsView." }, { @@ -740,6 +740,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/plate_manager_services.rst against B6's later changes to openhcs/authoring/session/views.py: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/plate_manager_services.rst against B6's later changes to openhcs/authoring/session/session.py: the session decides in-place initialization by an actual file-creation attempt that leaves nothing behind, its debug-session methods moved into an inherited mixin, the measured CLI treats Windows Ctrl+Break like Ctrl+C, and the Official30 viewer test reads typed payload summaries. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/plate_manager_services.rst against B6's later changes to openhcs/authoring/session/session.py, openhcs/authoring/session/views.py, openhcs/authoring/session/events.py, openhcs/authoring/session/compile_batch.py, openhcs/authoring/session/dataset_document.py, openhcs/pyqt_gui/session_rendering.py, openhcs/mcp/server.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/plate_manager_services.rst against the later B6 changes to openhcs/authoring/session/session.py, openhcs/authoring/session/views.py, openhcs/authoring/session/events.py, openhcs/authoring/session/compile_batch.py, openhcs/authoring/session/dataset_document.py, openhcs/agent/capabilities.py, openhcs/pyqt_gui/session_rendering.py, openhcs/mcp/server.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -1460,7 +1461,7 @@ }, { "path": "openhcs/core/pipeline/function_contracts.py", - "sha256": "13bcaf7c7a62654f512d2cc739edccd2deeff35d99970ca493f3e827afbda91a", + "sha256": "f5e7c74ea3d2948c75388184263cd9f1f17905f820b180f634dfaa7514e1d526", "role": "special_inputs decorator: records parameter names as runtime-managed callable metadata and excludes them from user parameters; no special_outputs decorator exists." }, { @@ -1485,6 +1486,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/special_io_system.rst against B6's later changes to openhcs/core/pipeline/function_contracts.py: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/special_io_system.rst against B6's later changes to openhcs/core/callable_contract.py, openhcs/core/pipeline/function_contracts.py, openhcs/core/runtime_stores.py, openhcs/interop/cellprofiler/module_artifact_contracts.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/special_io_system.rst against the later B6 changes to openhcs/core/callable_contract.py, openhcs/core/pipeline/function_contracts.py, openhcs/core/runtime_stores.py, openhcs/interop/cellprofiler/module_artifact_contracts.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/special_io_system.rst in full, including its unreviewed special_outputs removal, and compared it against the changed authorities: callable_contract.py, pipeline/artifact_planning.py, runtime_stores.py, interop/cellprofiler/module_artifact_contracts.py; confirmed each named class and the special_inputs decorator exist with the roles the page assigns and that no special_outputs declaration remains; fixed the edited sentence.", @@ -2513,7 +2515,7 @@ }, { "path": "openhcs/mcp/installed_demo.py", - "sha256": "5debdb5ba24fbe29f48f065f9b3f473022c17209724e618f2a0b36190fb82913", + "sha256": "fc5149e135ce528c7704028434011e347df8b42ffdec8bc2be333d9fc2d0bd37", "role": "Declares the deterministic fixture request (1x1 grid, 192 px tiles, 10% overlap, 1 px stage error, 2 wavelengths, 1 Z, 12 cells, 0.95 shared, A01, ImageXpress, seed 1)." }, { @@ -2535,6 +2537,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/agent_workflow_validation.rst against B6's later changes to openhcs/mcp/installed_demo.py: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/development/agent_workflow_validation.rst against B6's later changes to tests/unit/test_installed_mcp_demo.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/development/agent_workflow_validation.rst against the later B6 changes to tests/unit/test_installed_mcp_demo.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/agent_workflow_validation.rst in full and compared it against the changed authorities: openhcs/mcp/client_registration.py, synthetic_plate_service.py, tests/unit/test_installed_mcp_demo.py, openhcs/demo/synthetic_data.py, openhcs/mcp/installed_demo.py; the client skill sync and session-based execute path do not touch the protocol, and the fixture request values, the registry the page names, the time-derived HTD identifier, the ZMQ Server Manager and Live Results windows and the retained baseline record all still match; updated the test's role because its old progress-polling watchdog tests were replaced by the session journey.", @@ -2606,7 +2609,7 @@ }, { "path": "openhcs/core/pipeline/function_contracts.py", - "sha256": "13bcaf7c7a62654f512d2cc739edccd2deeff35d99970ca493f3e827afbda91a", + "sha256": "f5e7c74ea3d2948c75388184263cd9f1f17905f820b180f634dfaa7514e1d526", "role": "Declares artifact_inputs, artifact_outputs, special_inputs, execution_scope and runtime_bound_parameters." }, { @@ -2653,6 +2656,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/callable_artifact_authoring.rst against B6's later changes to openhcs/core/pipeline/function_contracts.py: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/development/callable_artifact_authoring.rst against B6's later changes to openhcs/core/callable_contract.py, openhcs/core/artifacts.py, openhcs/core/processing_contracts.py, openhcs/core/pipeline/function_contracts.py, openhcs/core/runtime_stores.py, openhcs/core/orchestrator/bundle_execution.py, openhcs/processing/materialization/core.py, openhcs/interop/cellprofiler/module_artifact_contracts.py, tests/unit/test_callable_contract.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/development/callable_artifact_authoring.rst against the later B6 changes to openhcs/core/callable_contract.py, openhcs/core/artifacts.py, openhcs/core/processing_contracts.py, openhcs/core/pipeline/function_contracts.py, openhcs/core/runtime_stores.py, openhcs/core/orchestrator/bundle_execution.py, openhcs/processing/materialization/core.py, openhcs/interop/cellprofiler/module_artifact_contracts.py, tests/unit/test_callable_contract.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/callable_artifact_authoring.rst in full and compared it against the changed authorities: the page, openhcs/core/callable_contract.py, openhcs/core/artifacts.py, module_artifact_contracts.py, tests/unit/test_artifact_spec_collection.py, the ArrayBridge types and decorators, tests/unit/test_callable_contract.py; grepped every API the page names (SettingToKeywordBinding, SelectedPlaneImageOutput, canonical/trailing return specs, RuntimeArtifactBatch.records, PointROIOptions features, the custom-function capability and guide names) and the quoted error texts, confirmed the plate executor's single SpecialArtifactType rule and the empty-Points rejection, corrected the axis-role and processing-contract names, and added processing_contracts.py, function_contracts.py, runtime_stores.py, bundle_execution.py and materialization/core.py as owners while dropping arraybridge/types.py.", @@ -2995,7 +2999,7 @@ }, { "path": "tests/conftest.py", - "sha256": "82d01adebe952964b61c0cad85024d0b3ba60a4a6dba1a11f6aa715a9f046664", + "sha256": "7bf3dc16e833053599ad02eb2e5a0a4744e0754f8008d827afc565a198717e79", "role": "Test-harness parsing of OPENHCS_CPU_ONLY, which accepts only 'true' when skipping pytest-qt." } ], @@ -3004,6 +3008,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/git_worktree_testing.rst against B6's later changes to tests/conftest.py: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/development/git_worktree_testing.rst against the later B6 changes to pyproject.toml: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/git_worktree_testing.rst in full and compared it against the changed authorities: pyproject.toml (first-party floors moved to zmqruntime 0.6, objectstate 1.3.1, python-introspect 0.2.5, metaclass-registry 0.3, arraybridge 0.4.1, polystore 0.6, pyqt-reactive 0.4; JAX extra now jax[cuda12-local]; openhcs-basicpy added); the page states no version floors. Checked the eight editable installs against .gitmodules, the dev and gui extras, tests/integration/test_main.py, and every --it-* option and value against tests/pytest_integration_options.py; corrected OPENHCS_CPU_ONLY=1 to OPENHCS_CPU_ONLY=true to match tests/conftest.py.", "Re-read docs/source/development/git_worktree_testing.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -3210,7 +3215,7 @@ }, { "path": "openhcs/agent/dto/session.py", - "sha256": "cfad720d060c36c2985d956fbf1f0ca838476ba1521acbdc0c95b4e8c7a746c6", + "sha256": "6d954139237672eddae155f3ece1dfefef4d4e4ce20b1934ae388860538627bc", "role": "Typed run request fields runtime_observation_export_path, runtime_observation_export_scope and submit_timeout_ms, and event_sequence results." }, { @@ -3281,6 +3286,7 @@ ], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/mcp_development.rst against B6's later changes to openhcs/agent/dto/session.py: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/development/mcp_development.rst against B6's later changes to openhcs/authoring/session/session.py: the session decides in-place initialization by an actual file-creation attempt that leaves nothing behind, its debug-session methods moved into an inherited mixin, the measured CLI treats Windows Ctrl+Break like Ctrl+C, and the Official30 viewer test reads typed payload summaries. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/development/mcp_development.rst against B6's later changes to openhcs/mcp/dev_client_core.py, openhcs/agent/services/ui_bridge_service.py, openhcs/mcp/dev_client_commands/ui.py, openhcs/authoring/session/session.py, tests/unit/agent/test_mcp_server.py, openhcs/runtime/zmq_execution_client.py, tests/unit/test_zmq_submission_timeout.py, tests/unit/pyqt_gui/test_ui_agent_bridge.py, openhcs/mcp/server.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/development/mcp_development.rst against the later B6 changes to openhcs/mcp/dev_client_core.py, openhcs/agent/services/ui_bridge_service.py, openhcs/mcp/dev_client_commands/ui.py, openhcs/agent/capabilities.py, openhcs/authoring/session/session.py, tests/unit/agent/test_mcp_server.py, openhcs/runtime/zmq_execution_client.py, tests/unit/test_zmq_submission_timeout.py, tests/unit/pyqt_gui/test_ui_agent_bridge.py, openhcs/mcp/server.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -3451,7 +3457,7 @@ }, { "path": ".github/workflows/integration-tests.yml", - "sha256": "5017eb98536006a0cec4f34573a35c1a547c3ad89889e276e88a8fb6f992991c", + "sha256": "781bd20677f309152ad4041bd7d62d5aca9ea802bd9a0a201a96ad187bb7b05f", "role": "Cross-platform source-candidate and published-dependency readiness, installed-wheel live GUI/MCP operation with retained native screenshots, three-stage native-installer workflow proof, signing-helper parse, and staged-update validation matrix." }, { @@ -3546,6 +3552,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/mcp_release.rst against B6's later changes to .github/workflows/integration-tests.yml: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/development/mcp_release.rst against B6's later changes to tests/unit/test_mcp_release_workflow.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/development/mcp_release.rst against the later B6 changes to tests/unit/test_mcp_release_workflow.py, scripts/smoke_installed_gui.py, tests/unit/test_smoke_installed_gui.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/mcp_release.rst in full and compared it against the changed authorities: openhcs/__init__.py (version 0.8.7, microscopy axis-family activation; the version-authority claim holds), .github/workflows/publish.yml and tests/unit/test_mcp_release_workflow.py (new build-native-wheels job, sdist built separately, per-wheel validation, Linux x86_64 smoke selection), .github/workflows/integration-tests.yml (parity-scope gate, Official30 parity on Python 3.12 Linux and 3.14 on all three OSes, run_installed_tests.py coverage), scripts/resolve_successful_job_baseline.py (branch filter removed; ancestor rule unchanged), scripts/smoke_installed_gui.py (typed MCP payloads, ShowDatasetCode), tests/installer/test_windows_simple_installer.py, tests/installer/test_installer_signing.py, tests/unit/test_validate_local_release_floors.py and tests/unit/test_ci_package_boundaries.py. Corrected the single-wheel publication and recovery descriptions and the GUI smoke action name; confirmed the Python 3.14 wheel-candidate lane still excludes the Centrosome oracle and the remaining gate, recovery and signing claims.", @@ -3672,13 +3679,14 @@ }, { "path": ".github/workflows/integration-tests.yml", - "sha256": "5017eb98536006a0cec4f34573a35c1a547c3ad89889e276e88a8fb6f992991c", + "sha256": "781bd20677f309152ad4041bd7d62d5aca9ea802bd9a0a201a96ad187bb7b05f", "role": "Runs explicitly selected live OMERO variants and preserves component and table-service diagnostics on failure." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/omero_testing.rst against B6's later changes to .github/workflows/integration-tests.yml: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/development/omero_testing.rst against the later B6 changes to external/PolyStore/src/polystore/omero_local.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/omero_testing.rst in full and compared it against the changed authorities: tests/integration/test_main.py (partition-filter config renames, Microscopy axis family, DatasetSourceChoice, per-test allocated ports, execute_compiled_bundle, subject_id, PipelineDocumentCodec) and .github/workflows/integration-tests.yml (parity-scope gate, run_installed_tests.py coverage). The page names none of the renamed test internals; confirmed the harness still registers OMEROLocalBackend and runs direct and ZMQ modes without opening a browser, the omero-tests-linux lane still builds submodule candidate wheels without a publication dependency and keeps Tables-0 and component diagnostics, and OMEROWellAddress, OMEROPlaneAddress, OMEROPlaneFilenameTemplate and FilenameParseResult exist at their owners.", "Re-read docs/source/development/omero_testing.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", diff --git a/docs/audits/user_docs.json b/docs/audits/user_docs.json index b618740463..acfb0930f4 100644 --- a/docs/audits/user_docs.json +++ b/docs/audits/user_docs.json @@ -1002,7 +1002,7 @@ }, { "path": "openhcs/core/pipeline/function_contracts.py", - "sha256": "13bcaf7c7a62654f512d2cc739edccd2deeff35d99970ca493f3e827afbda91a", + "sha256": "f5e7c74ea3d2948c75388184263cd9f1f17905f820b180f634dfaa7514e1d526", "role": "Declares the @artifact_outputs decorator named by the page." }, { @@ -1041,6 +1041,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/concepts/storage_system.rst against B6's later changes to openhcs/core/pipeline/function_contracts.py: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/concepts/storage_system.rst against B6's later changes to openhcs/core/config.py, openhcs/core/runtime_stores.py, openhcs/core/steps/function_artifact_materialization.py, openhcs/core/pipeline/function_contracts.py, tests/unit/test_runtime_value_store.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/concepts/storage_system.rst against the later B6 changes to openhcs/core/config.py, openhcs/core/runtime_stores.py, openhcs/core/steps/function_artifact_materialization.py, openhcs/core/pipeline/function_contracts.py, tests/unit/test_runtime_value_store.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/concepts/storage_system.rst in full and compared it against the changed authorities: the page, openhcs/core/config.py, openhcs/core/runtime_stores.py, function_artifact_materialization.py, PolyStore base/config/disk, ArrayBridge decorators, and the runtime-store and TIFF compression tests; confirmed the three VFSConfig backends and AUTO default, the fieldless ZarrConfig/TiffConfig subtypes, the TIFF NONE/DEFLATE codecs, the @artifact_outputs decorator and RuntimeValueStore replace/observation behaviour, and added openhcs/core/pipeline/function_contracts.py as the decorator owner.", @@ -1080,7 +1081,7 @@ }, { "path": "openhcs/mcp/installed_demo.py", - "sha256": "5debdb5ba24fbe29f48f065f9b3f473022c17209724e618f2a0b36190fb82913", + "sha256": "fc5149e135ce528c7704028434011e347df8b42ffdec8bc2be333d9fc2d0bd37", "role": "Implements the installed-package MCP demonstration used by the getting-started verification flow." }, { @@ -1140,7 +1141,7 @@ }, { "path": ".github/workflows/integration-tests.yml", - "sha256": "5017eb98536006a0cec4f34573a35c1a547c3ad89889e276e88a8fb6f992991c", + "sha256": "781bd20677f309152ad4041bd7d62d5aca9ea802bd9a0a201a96ad187bb7b05f", "role": "Exercises Windows and macOS native installers, staged updates, and cellprofiler-compat installation on Python 3.14." }, { @@ -1155,6 +1156,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/getting_started/getting_started.rst against B6's later changes to openhcs/mcp/installed_demo.py, .github/workflows/integration-tests.yml: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/getting_started/getting_started.rst against B6's later changes to tests/unit/test_installed_mcp_demo.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/getting_started/getting_started.rst against the later B6 changes to pyproject.toml, tests/unit/test_installed_mcp_demo.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/getting_started/getting_started.rst in full and compared it against the changed authorities: pyproject.toml, openhcs/cli.py, openhcs/mcp/installed_demo.py, render_installer_contract.py, both native installers, smoke_installed_desktop.py, installer tests, integration-tests.yml, test_installed_mcp_demo.py, and the desktop modules moved into openhcs/desktop by 73a2aece2; confirmed the extras, entry points (openhcs, openhcs-mcp-demo), --log-level handling in pyqt_gui/launch.py, the demo's generated plate / neurite preset / nonzero Napari payload check / output_root report, the desktop profile extras and pip/uv override suppression, and sibling-environment update with a preserved previous environment; removed the false Python 3.13 limit on cellprofiler-compat and replaced the moved authorities.", @@ -1342,7 +1344,7 @@ }, { "path": "openhcs/authoring/session/views.py", - "sha256": "8ce6cac27293ac9105f4d2ebc76d10eebeba1ef90e0e8954c0a117910e17e3b3", + "sha256": "c772d515dfa738363ab403a3e19c9e438c17b0748a3a9ed65c44c416f579a378", "role": "DatasetListView and PipelineStepsView titles and the operations each pane binds." }, { @@ -1362,6 +1364,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/basic_interface.rst against B6's later changes to openhcs/authoring/session/views.py: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/basic_interface.rst against B6's later changes to openhcs/pyqt_gui/main.py, openhcs/pyqt_gui/widgets/pipeline_editor.py, openhcs/agent/services/ui_bridge_service.py, tests/unit/pyqt_gui/test_ui_agent_bridge.py, tests/unit/pyqt_gui/test_pipeline_code_document_transactionality.py, openhcs/pyqt_gui/widgets/shared/services/pipeline_editor_workflows.py, tests/unit/pyqt_gui/test_pipeline_editor_widget.py, openhcs/pyqt_gui/services/ui_bridge_windows.py, tests/unit/pyqt_gui/test_main_config_propagation.py, openhcs/authoring/session/views.py, openhcs/authoring/session/operations/datasets.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/basic_interface.rst against the later B6 changes to openhcs/pyqt_gui/main.py, openhcs/pyqt_gui/widgets/pipeline_editor.py, openhcs/agent/services/ui_bridge_service.py, tests/unit/pyqt_gui/test_ui_agent_bridge.py, tests/unit/pyqt_gui/test_pipeline_code_document_transactionality.py, openhcs/pyqt_gui/widgets/shared/services/pipeline_editor_workflows.py, tests/unit/pyqt_gui/test_pipeline_editor_widget.py, openhcs/pyqt_gui/services/ui_bridge_windows.py, tests/unit/pyqt_gui/test_main_config_propagation.py, openhcs/authoring/session/views.py, openhcs/authoring/session/operations/datasets.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/basic_interface.rst in full and compared it against the changed authorities: pyqt_gui/main.py, plate_manager.py, pipeline_editor.py, ui_bridge_service.py, ui_bridge_windows.py, pipeline_document.py, pipeline_editor_workflows.py, endpoint_function_catalog_service.py, gallery_catalog.py and their tests; confirmed the dock layout, menus (Image Browser, Log Viewer, Global Configuration, Custom Functions, Analysis Consolidation), Connect/Connecting/Compile/Run/Stop rules in the session operations, selected/all dataset code documents, steps-only pipeline documents, background catalogue preparation, the ui-function-selector gallery scenario and unsupported-navigation reporting; renamed the Plate Manager pane to its current Datasets title and pointed agent step 6 at session events.", @@ -1560,7 +1563,7 @@ }, { "path": "openhcs/authoring/session/views.py", - "sha256": "8ce6cac27293ac9105f4d2ebc76d10eebeba1ef90e0e8954c0a117910e17e3b3", + "sha256": "c772d515dfa738363ab403a3e19c9e438c17b0748a3a9ed65c44c416f579a378", "role": "DatasetListView title for the desktop pane that UI-owned work populates." } ], @@ -1569,6 +1572,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/domain_expert_onboarding.rst against B6's later changes to openhcs/authoring/session/views.py: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/domain_expert_onboarding.rst against B6's later changes to benchmark/cellprofiler_benchmark_cli.py: the session decides in-place initialization by an actual file-creation attempt that leaves nothing behind, its debug-session methods moved into an inherited mixin, the measured CLI treats Windows Ctrl+Break like Ctrl+C, and the Official30 viewer test reads typed payload summaries. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/domain_expert_onboarding.rst against B6's later changes to openhcs/interop/cellprofiler/pipeline_import.py, benchmark/cellprofiler_benchmark_cli.py, tests/unit/test_benchmark_control.py, openhcs/interop/cellprofiler/module_artifact_declarations.py, openhcs/authoring/session/views.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/domain_expert_onboarding.rst against the later B6 changes to openhcs/agent/capabilities.py, openhcs/interop/cellprofiler/pipeline_import.py, benchmark/cellprofiler_benchmark_cli.py, tests/unit/test_benchmark_control.py, openhcs/interop/cellprofiler/module_artifact_declarations.py, openhcs/authoring/session/views.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -2746,7 +2750,7 @@ }, { "path": ".github/workflows/integration-tests.yml", - "sha256": "5017eb98536006a0cec4f34573a35c1a547c3ad89889e276e88a8fb6f992991c", + "sha256": "781bd20677f309152ad4041bd7d62d5aca9ea802bd9a0a201a96ad187bb7b05f", "role": "Defines the operating-system integration matrix, the Python 3.14 core job, the scoped four-platform Official30 parity lane and its required check, installed-wheel GUI/MCP operation, installer screenshot artifacts, and source-gate coverage artifacts used by CI." }, { @@ -2881,6 +2885,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/guides/testing_guide.rst against B6's later changes to .github/workflows/integration-tests.yml: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/guides/testing_guide.rst against the later B6 changes to scripts/smoke_installed_gui.py, tests/unit/test_smoke_installed_gui.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guides/testing_guide.rst in full and compared it against the changed authorities: pytest.ini, integration-tests.yml, resolve_successful_job_baseline.py, test_windows_simple_installer.py, website-pages.yml, test_ci_package_boundaries.py, smoke_installed_desktop.py, test_installed_desktop_smoke.py, smoke_installed_gui.py, xdg_paths.py, tests/integration/conftest.py; confirmed the default discovery options, --it-* flags, artifact names, concurrency, actionlint, code-action probe and log-directory paths, and added the scoped Official30 parity lane and the installed-wheel test runner.", "Re-read docs/source/guides/testing_guide.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -3487,7 +3492,7 @@ }, { "path": "openhcs/authoring/session/views.py", - "sha256": "8ce6cac27293ac9105f4d2ebc76d10eebeba1ef90e0e8954c0a117910e17e3b3", + "sha256": "c772d515dfa738363ab403a3e19c9e438c17b0748a3a9ed65c44c416f579a378", "role": "Owns the session view titles that the plate-manager and pipeline-editor window identities present." }, { @@ -3524,6 +3529,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/reference/ui_windows.rst against B6's later changes to openhcs/authoring/session/views.py: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/reference/ui_windows.rst against B6's later changes to openhcs/authoring/session/views.py, openhcs/pyqt_gui/main.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/reference/ui_windows.rst against the later B6 changes to openhcs/authoring/session/views.py, openhcs/pyqt_gui/main.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/reference/ui_windows.rst in full and compared it against the changed authorities: openhcs/agent/ui_bridge_identities.py, openhcs/pyqt_gui/main.py, openhcs/pyqt_gui/services/main_window_workflows.py, scripts/gallery_catalog.py, tests/unit/test_build_website.py, and tests/unit/test_capture_media_gallery.py; the U1 session refactor moved two window titles onto session views but the inventory, titles, and capture targets are still projected from the registered identity declarations, no OMERO window is registered, and all linked pages exist; added openhcs/authoring/session/views.py as the title owner; no page change was needed.", @@ -3621,7 +3627,7 @@ }, { "path": "openhcs/authoring/session/views.py", - "sha256": "8ce6cac27293ac9105f4d2ebc76d10eebeba1ef90e0e8954c0a117910e17e3b3", + "sha256": "c772d515dfa738363ab403a3e19c9e438c17b0748a3a9ed65c44c416f579a378", "role": "Declares the Datasets and Pipeline Editor view titles shown for the two code-document owners." }, { @@ -3742,6 +3748,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/user_guide/code_ui_editing.rst against B6's later changes to openhcs/authoring/session/views.py: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/code_ui_editing.rst against B6's later changes to openhcs/authoring/session/dataset_document.py, openhcs/authoring/session/views.py, openhcs/pyqt_gui/widgets/pipeline_editor.py, openhcs/pyqt_gui/services/ui_bridge_object_state.py, openhcs/pyqt_gui/services/ui_agent_bridge.py, openhcs/agent/services/ui_bridge_service.py, openhcs/pyqt_gui/widgets/shared/services/pipeline_editor_workflows.py, tests/unit/pyqt_gui/test_ui_agent_bridge.py, tests/unit/pyqt_gui/test_plate_manager_widget.py, tests/unit/pyqt_gui/test_pipeline_editor_widget.py, tests/unit/test_pycodify_formatters.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/code_ui_editing.rst against the later B6 changes to openhcs/authoring/session/dataset_document.py, openhcs/authoring/session/views.py, openhcs/pyqt_gui/widgets/pipeline_editor.py, openhcs/pyqt_gui/services/ui_bridge_object_state.py, openhcs/pyqt_gui/services/ui_agent_bridge.py, openhcs/agent/services/ui_bridge_service.py, openhcs/pyqt_gui/widgets/shared/services/pipeline_editor_workflows.py, tests/unit/pyqt_gui/test_ui_agent_bridge.py, tests/unit/pyqt_gui/test_plate_manager_widget.py, tests/unit/pyqt_gui/test_pipeline_editor_widget.py, tests/unit/test_pycodify_formatters.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/user_guide/code_ui_editing.rst in full and compared it against the changed authorities: pipeline_document.py, ui_bridge_object_state.py, ui_bridge_service.py, plate_manager_workflows.py and pipeline_editor_workflows.py, the deleted pipeline_object_state_binding.py (now authoring/session/pipelines.py), step_parameter_editor.py, pycodify_formatters.py, gallery_catalog.py, and the changed tests; checked selected/all document scopes, selected-dataset writability, global-draft protection while work runs, catalogue imports and wildcard rejection, unchanged no-op results, and saved-baseline commit; renamed Plate Manager to the Datasets window.", From dd8709f3f0c06212425491bd19e9e417ecd2339a Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 06:09:10 -0400 Subject: [PATCH 61/84] B6: receipt - causes, fixes, guards and lockstep library PRs Co-Authored-By: Claude Opus 5.5 --- .../consolidation-20261009/B6-ci-green.md | 62 ++++++++++++------- 1 file changed, 39 insertions(+), 23 deletions(-) diff --git a/docs/refactor/consolidation-20261009/B6-ci-green.md b/docs/refactor/consolidation-20261009/B6-ci-green.md index 0938dd5f17..38d3ec4f19 100644 --- a/docs/refactor/consolidation-20261009/B6-ci-green.md +++ b/docs/refactor/consolidation-20261009/B6-ci-green.md @@ -1,34 +1,50 @@ # B6: main's CI green -**Head audited:** `openhcs` `main` at `84e7a2904` (#1193). **Rules:** [00-RULES.md](00-RULES.md). **Input:** the last completed runs on main: Integration Tests 38116770859 (`b468e7359`, W), Website 38110269836, Documentation 38116770933. - -## What is wrong - -**Three workflows are red on main, and Documentation has not passed on main in the last 1,000 runs. Most jobs fail for a handful of shared causes that only CI exposes (wheel install, `--import-mode=importlib`, Windows, path filters), so local runs never see them.** - -Causes measured on the W run, before B5 merged (to be re-measured on this PR's CI): - -- **Website:** the job installs a hand-written roster of five first-party packages instead of OpenHCS, then imports `openhcs.authoring.session.operations`, whose graph needs `numba` (and the rest of OpenHCS's dependencies). The `paths:` filter lists `openhcs/__init__.py` only, so refactors that broke the import never triggered it. -- **unit-tests:** collection stops at 17 errors. 14 test modules import a sibling test module by bare name (`from test_function_outputs import …`); CI runs `--import-mode=importlib` from the installed wheel, where the test directory is not on `sys.path`. The rest were fixed by B5. -- **Documentation:** `scripts/validate_docs.py docs/source` reports 950 findings: 828 "authority changed after review", 78 "authority file does not exist", 34 "source changed after its editorial audit" over 140 pages in `docs/audits/*.json`, and 10 missing `openhcs/skills/use-openhcs/references/*.md` paths in the MCP knowledge-base manifest. -- **Integration (other jobs):** listed per job under Remaining, with owner, as measured on this PR. - -## Target - -- Each workflow installs what it imports through the one candidate installer (`scripts/install_ci_candidate.py`), never a roster. -- Test helpers are imported through the `tests` package. -- Root causes fixed in production, packaging or workflows; no skip, xfail, `continue-on-error` or loosened check. +**Head audited:** `openhcs` `main` at `84e7a2904` (#1193). **Rules:** [00-RULES.md](00-RULES.md). **Input:** the last completed runs on main: Integration Tests 38116770859 (`b468e7359`), Website 38110269836 and Documentation 38116770933. Integration Tests last passed on main at `fc1f94632` (2026-09-24). Documentation had not passed in the last 1,000 main runs. + +## What was wrong + +**Three workflows were red on main because of about twenty independent causes.** Most of them only CI exposes: the installed wheel with `--import-mode=importlib`, Windows, Intel macOS, xcb under Xvfb, path filters, and jobs nobody watched. The table gives each cause, its fix, and its certificate where something was deleted. + +| Job(s) | First cause | Fix | +|---|---|---| +| Website | The job installed a five-package roster, then imported `openhcs`, whose import graph needs numba. The `paths:` filter only listed `openhcs/__init__.py`. | Install the candidate through `install_ci_candidate` with the `dev` extra, because `pytest.ini`'s addopts need pytest-xdist. Trigger on `openhcs/**` and `external/**`. | +| Documentation (`validate_docs`) | 950 findings: 141 of 155 pages had stale editorial audits (828 changed and 78 deleted authorities), plus 10 false "missing path" findings. | The path regex now matches whole repository paths and covers `packaging/`. All 141 pages were re-read against the diff of each stale authority since its reviewed version, wrong claims were corrected (see the commit), and digests were pinned after review. B6's own later edits were compared and re-pinned. | +| Documentation (sphinx `-W`) | PolyStore and pyqt-reactive had agent run logs outside every toctree. zmqruntime's API page listed members removed by L3. | Deleted the logs (**contract**: Q4, validation logs nothing links to). zmqruntime's page derives its members with plain `:members:`. | +| unit-tests (collection) | 14 test modules imported a sibling test module by bare name, which fails under importlib mode. | Import through the `tests` package; guard in `test_ci_import_guards.py`. | +| unit-tests (abort, exit 134) | An in-process MCP transport created a `QCoreApplication`, so a later `QWidget` aborted. | The session creates its `QApplication` first (`tests/conftest.py`). | +| code-quality | 412 unformatted files since the last success. A test used Python 3.12-only f-string quote reuse, which is a syntax error on 3.11. | Black over this PR's changed files; quote fixed. Main's baseline becomes this PR's green head. | +| structural ratchet | Black wrapping and B6 fixes grew six classes past their budget. | Cohesive method groups moved into inherited mixins: `MainOutputSliceProjectionMixin`, `FunctionCoreDebugMixin`, `CustomFunctionSourceFilesMixin`, `SessionDebugMixin`. Catalog policy moved to module functions. Four unused main-window theme methods and `SourceImageProvenance.source_component_identity` were deleted (**unused**: no caller, menu, bridge or string reference). | +| Windows (every ZMQ job) | `tests/conftest.py` used `os.getpgid`. zmqruntime's `startup_owner` reopened the lock file that portalocker holds (mandatory locks give PermissionError). | Process-group reaping only on POSIX. zmqruntime reads and writes the owner record through the held handle (0.6.1). | +| Windows | NumPy exports uint32 as format `'L'`; custom-function sources were written in text mode, which changes their hashes; SIGINT can't be sent to a child; chmod doesn't make a directory read-only and `os.access` ignores ACLs; autocrlf rewrote checksummed `.cppipe` files; packaged knowledge paths were longer than MAX_PATH. | Native granularity accepts both formats. Sources are persisted as exact bytes. The CLI treats Ctrl+Break as Ctrl+C. Writability is decided by an actual file-creation attempt, and the test denies writes with an ACL. `.gitattributes` keeps committed bytes (`* -text`). Recipes are placed by case name within a declared path budget. | +| Intel macOS | `openhcs-basicpy` is excluded by marker, but the catalog imported its module. The kernel cache compiled serially because macOS has no `sched_getaffinity`. | Modules declare `REQUIRED_DISTRIBUTIONS`, and the catalog excludes those whose package markers leave them out. Workers come from `os.cpu_count()` where affinity is unavailable. | +| python-314 | ObjectState read a class's own annotations through `__dict__`, which 3.14 no longer populates. mahotas sat only in an extra, but core catalog modules need it. | ObjectState uses `inspect.get_annotations` (1.3.2). mahotas is a core dependency. | +| gui-tests (13) | Stale tests after U1/U5. | Rewritten against the session-backed plate manager and the current stubs. | +| backend-microscope, boundary, OMERO | STEP_INPUT filters matched virtual names instead of stored source paths. Passthrough outputs dropped their producer's filename qualifier. BaSiC read `.metadata` off plain arrays. `OMEROLocalBackend` had no `is_dir`. The metadata document was written to local disk for a virtual backend. Viewer tests used `ViewerType` members. The GUI journey test ran in headless jobs. Several integration tests had gone stale. | Fixed at each owner: `SourceImageIdentity.filter_paths`, producer qualifiers, `ImagePayload.of`, `OMEROLocalBackend.is_dir`/`is_file` (PolyStore 0.6.1), `OpenHCSMetadataTarget.hosts_metadata_document` (not for a `VirtualBackend`), viewer families, and the GUI journey moved to gui-tests. | +| Official30 real-viewer smoke | `required_scalar_components` pooled contributors across runtime planes and dropped `site`, so napari rejected the stack. | Contributors are grouped per plane (`contributor_groups`). | +| Installed demo (Windows, Intel macOS) | The one-plane label projection (#477) lived only on `MatchImageStackLabels`; the skeleton step used `SliceAlignedLabels`. | The projection is owned by `ObjectLabelInputExecutionMode` (**derived**: `MatchImageStackLabels`' copy deleted). Dataset rows carry the finished execution's error. | +| desktop live MCP | The smoke read the catalog before the GUI's endpoint was up. Requestless UI-bridge operations received the connection in the request slot. | `execution_services_prepared` future; connection passed by keyword. | +| wheel-integration | The MCP smoke sent `wells` (G2 renamed it `partition_values`). Bio-Formats tests ran without the declared Fiji bundle. | `partition_values`; the job restores and prewarms the bundle. | +| pypi-dependency-readiness | Pinned submodule sources differ from their release tags. | Version bumps: pycodify 0.1.4, zmqruntime 0.6.1, PolyStore 0.6.1, ObjectState 1.3.2. Floors raised. **The job stays red until the owner merges and tags the four library PRs.** | ## Guards - `tests/unit/test_ci_import_guards.py`: no test module imports a test-tree module by bare name. -- `tests/unit/test_ci_package_boundaries.py`: the website installs the OpenHCS candidate through `install_ci_candidate`. -- `tests/unit/test_build_website.py`: the website workflow triggers on `openhcs/**`. +- `tests/unit/test_ci_package_boundaries.py`: the website job installs the candidate with the dev extra. +- `tests/unit/test_build_website.py`: the website job triggers on `openhcs/**`. +- `tests/unit/test_validate_documentation_audit.py`: repository paths match only whole paths. +- `tests/unit/test_build_mcp_knowledge_assets.py`: every packaged resource fits the Windows path budget. +- `tests/unit/test_library_registry_discovery.py`: marker-excluded distributions are left out of the catalog, while a required but missing one fails. +- `tests/unit/test_dataset_access.py`; `external/zmqruntime` held-handle owner-record test; `external/PolyStore/tests/test_omero_virtual_paths.py`. ## Done when -Website, Documentation and Integration Tests are green on this PR's head, or each remaining red job names its exact cause and owning surface below. +Website, Documentation and Integration Tests are green on this PR's head, or each remaining red job names its exact cause and owner (see the PR's final status). -## Remaining +## Lockstep library PRs (owner merges and tags) -(filled from this PR's CI) +- OpenHCSDev/pycodify#3 (0.1.4) +- OpenHCSDev/ZMQRuntime#29 (0.6.1; docs; startup lock) +- OpenHCSDev/PolyStore#38 (0.6.1; OMERO `is_dir`; docs logs) +- OpenHCSDev/ObjectState#26 (1.3.2) +- OpenHCSDev/PyQT-reactive#21 (docs logs) From a5ce88b63014e442b4da4cd537c3111f409ca4c5 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 06:19:41 -0400 Subject: [PATCH 62/84] B6: only the unit session pre-creates its QApplication; GUI journey waits for each batch to end The workflow suite in tests/pyqt_gui owns its QApplication lifecycle, so the pre-created application moves to tests/unit/conftest.py. The GUI journey asserted the next button enabled while the previous batch was still closing. Co-Authored-By: Claude Opus 5.5 --- tests/conftest.py | 11 ----------- tests/pyqt_gui/test_session_gui_journey.py | 3 +++ tests/unit/conftest.py | 17 +++++++++++++++++ 3 files changed, 20 insertions(+), 11 deletions(-) create mode 100644 tests/unit/conftest.py diff --git a/tests/conftest.py b/tests/conftest.py index 0a295dded3..ea7aecaf00 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -68,17 +68,6 @@ def pytest_sessionstart(session): CPU_ONLY_MODE = OpenHCSProcessEnvironment.cpu_only_mode() if not CPU_ONLY_MODE: pytest_plugins = ["pytestqt"] - - @pytest.fixture(scope="session", autouse=True) - def session_qapplication(qapp): - """Create the session's one QApplication before any test runs. - - A process holds one Qt application. In-process MCP transports adopt - an existing one and otherwise create a QCoreApplication, after which - no test in the session could create a widget. - """ - yield qapp - else: pytest_plugins = [] diff --git a/tests/pyqt_gui/test_session_gui_journey.py b/tests/pyqt_gui/test_session_gui_journey.py index 76c6b25a92..aad333283c 100644 --- a/tests/pyqt_gui/test_session_gui_journey.py +++ b/tests/pyqt_gui/test_session_gui_journey.py @@ -94,6 +94,9 @@ def wait_until(condition, label) -> None: "connect", ) manager.update_button_states() + # The previous stage's batch ends after its row reports done. + wait_until(lambda: not session.execution_state.busy, "batch finished") + manager.update_button_states() assert manager.buttons[operation.operation_id].isEnabled(), operation manager.handle_button_action(operation.operation_id) deadline = time.monotonic() + STAGE_TIMEOUT_SECONDS diff --git a/tests/unit/conftest.py b/tests/unit/conftest.py new file mode 100644 index 0000000000..7e7f38f10f --- /dev/null +++ b/tests/unit/conftest.py @@ -0,0 +1,17 @@ +"""Unit-suite process setup.""" + +import pytest + +from openhcs.utils.environment import OpenHCSProcessEnvironment + +if not OpenHCSProcessEnvironment.cpu_only_mode(): + + @pytest.fixture(scope="session", autouse=True) + def session_qapplication(qapp): + """Create the unit session's one QApplication before any test runs. + + A process holds one Qt application. In-process MCP transports adopt an + existing one and otherwise create a QCoreApplication, after which no + later unit test could create a widget. + """ + yield qapp From 88e22432ed77f7c12b416d248df722f7510aaca0 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 06:26:38 -0400 Subject: [PATCH 63/84] B6: installed demo reports the viewer's wire name through ViewerType.value ViewerType is the derived enum view whose member value is the wire name; it has no wire_value attribute. Co-Authored-By: Claude Opus 5.5 --- openhcs/mcp/installed_demo.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/openhcs/mcp/installed_demo.py b/openhcs/mcp/installed_demo.py index c3139782da..e5f2628438 100644 --- a/openhcs/mcp/installed_demo.py +++ b/openhcs/mcp/installed_demo.py @@ -610,7 +610,7 @@ def run_installed_demo( viewer_type = ( None if viewer_payload is None or viewer_payload.viewer is None - else viewer_payload.viewer.viewer_type.wire_value + else viewer_payload.viewer.viewer_type.value ) result = InstalledDemoResult( openhcs_version=distribution("openhcs").version, From c74b2bd5517d8a8e0ad3dc0abd2159aae2cc92b1 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 06:39:02 -0400 Subject: [PATCH 64/84] B6: dataset write probe makes one creation attempt tempfile.TemporaryFile on Windows reads a PermissionError as a name collision whenever os.access passes (it ignores ACLs) and retries up to os.TMP_MAX (2**31 - 1) times, so a source root denying new entries made initialization spin instead of mirroring. The probe now creates one uniquely named file with O_CREAT|O_EXCL and removes it. Co-Authored-By: Claude Opus 5.5 --- openhcs/authoring/session/session.py | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/openhcs/authoring/session/session.py b/openhcs/authoring/session/session.py index a3fa2be760..d2a439c25b 100644 --- a/openhcs/authoring/session/session.py +++ b/openhcs/authoring/session/session.py @@ -13,9 +13,10 @@ import asyncio import inspect import logging -import tempfile +import os import threading import time +import uuid from abc import ABC, abstractmethod from dataclasses import dataclass from collections.abc import Callable, Iterable @@ -200,15 +201,19 @@ def _creates_files_in(directory: Path) -> bool: """Whether the filesystem lets this process create a file in ``directory``. Permission bits (``os.access``) do not see Windows ACLs or read-only - shares; an actual creation does. The probe file is anonymous or deleted on - close, so it leaves nothing behind. + shares; an actual creation does. One attempt answers: ``tempfile`` on + Windows reads a refused creation as a name collision whenever ``os.access`` + passes and retries up to ``os.TMP_MAX`` (2**31 - 1) times. The probe file is + removed at once, so it leaves nothing behind. """ + probe = Path(directory) / f".openhcs-write-probe-{uuid.uuid4().hex}" try: - with tempfile.TemporaryFile(dir=directory): - return True + os.close(os.open(probe, os.O_CREAT | os.O_EXCL | os.O_WRONLY, 0o600)) except OSError: return False + probe.unlink() + return True class DatasetAccess(ABC): From 2aaceeacd28f7daae5b6a4b802a419fa9fa289d7 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 06:39:58 -0400 Subject: [PATCH 65/84] B6: re-pin documentation authorities after the probe and test-session fixes Co-Authored-By: Claude Opus 5.5 --- docs/audits/architecture_a_m.json | 6 ++++-- docs/audits/architecture_n_z_development.json | 18 ++++++++++++------ docs/audits/user_docs.json | 3 ++- 3 files changed, 18 insertions(+), 9 deletions(-) diff --git a/docs/audits/architecture_a_m.json b/docs/audits/architecture_a_m.json index 62a80af721..a9a4fec356 100644 --- a/docs/audits/architecture_a_m.json +++ b/docs/audits/architecture_a_m.json @@ -339,7 +339,7 @@ }, { "path": "openhcs/authoring/session/session.py", - "sha256": "c1446fdf475503848a62086d13bc3e809dadd4958c3f763ea0347264bab5647d", + "sha256": "8d74425f65ce9f39f88439ca90bd09c02678ae3aef8d3cf6795570986444add0", "role": "Implements run_datasets (reserve, reset, connect, compile all, submit each with compile_artifact_id) and the endpoint status binding." }, { @@ -363,6 +363,7 @@ ], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/batch_workflow_service.rst against B6's later changes to openhcs/authoring/session/session.py: the writability probe makes one exclusive create attempt, the installed demo reports ViewerType.value, the unit session pre-creates its QApplication, and the GUI journey waits for each batch to end. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/batch_workflow_service.rst against B6's later changes to openhcs/authoring/session/session.py: the session decides in-place initialization by an actual file-creation attempt that leaves nothing behind, its debug-session methods moved into an inherited mixin, the measured CLI treats Windows Ctrl+Break like Ctrl+C, and the Official30 viewer test reads typed payload summaries. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/batch_workflow_service.rst against B6's later changes to openhcs/authoring/session/operations/datasets.py, openhcs/authoring/session/compile_batch.py, openhcs/authoring/session/session.py, openhcs/authoring/session/submission.py, openhcs/authoring/session/progress.py, tests/unit/test_u1_session_guards.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/batch_workflow_service.rst against the later B6 changes to openhcs/authoring/session/operations/datasets.py, openhcs/authoring/session/compile_batch.py, openhcs/authoring/session/session.py, openhcs/authoring/session/submission.py, openhcs/authoring/session/progress.py, tests/unit/test_u1_session_guards.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -2122,7 +2123,7 @@ }, { "path": "openhcs/authoring/session/session.py", - "sha256": "c1446fdf475503848a62086d13bc3e809dadd4958c3f763ea0347264bab5647d", + "sha256": "8d74425f65ce9f39f88439ca90bd09c02678ae3aef8d3cf6795570986444add0", "role": "Defines the headless Session whose datasets and pipelines live in the MCP process's ObjectState and whose events clients follow." }, { @@ -2247,6 +2248,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/mcp_distribution.rst against B6's later changes to openhcs/authoring/session/session.py: the writability probe makes one exclusive create attempt, the installed demo reports ViewerType.value, the unit session pre-creates its QApplication, and the GUI journey waits for each batch to end. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/mcp_distribution.rst against B6's later changes to openhcs/mcp/execution.py: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/mcp_distribution.rst against B6's later changes to openhcs/authoring/session/session.py, benchmark/cellprofiler_benchmark_cli.py: the session decides in-place initialization by an actual file-creation attempt that leaves nothing behind, its debug-session methods moved into an inherited mixin, the measured CLI treats Windows Ctrl+Break like Ctrl+C, and the Official30 viewer test reads typed payload summaries. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/mcp_distribution.rst against B6's later changes to openhcs/mcp/server.py, openhcs/mcp/dev_client_core.py, openhcs/mcp/dev_client_session.py, openhcs/mcp/context.py, openhcs/authoring/session/session.py, openhcs/pyqt_gui/services/ui_bridge_windows.py, benchmark/control_service.py, benchmark/cellprofiler_benchmark_cli.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", diff --git a/docs/audits/architecture_n_z_development.json b/docs/audits/architecture_n_z_development.json index de332793c5..0048a1eec1 100644 --- a/docs/audits/architecture_n_z_development.json +++ b/docs/audits/architecture_n_z_development.json @@ -675,7 +675,7 @@ "authority": [ { "path": "openhcs/authoring/session/session.py", - "sha256": "c1446fdf475503848a62086d13bc3e809dadd4958c3f763ea0347264bab5647d", + "sha256": "8d74425f65ce9f39f88439ca90bd09c02678ae3aef8d3cf6795570986444add0", "role": "Session class, MainThread/DispatcherThread, DatasetAccess and Renderer ports, dataset add/delete/split and work/mutation checks." }, { @@ -740,6 +740,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/plate_manager_services.rst against B6's later changes to openhcs/authoring/session/session.py: the writability probe makes one exclusive create attempt, the installed demo reports ViewerType.value, the unit session pre-creates its QApplication, and the GUI journey waits for each batch to end. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/plate_manager_services.rst against B6's later changes to openhcs/authoring/session/views.py: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/plate_manager_services.rst against B6's later changes to openhcs/authoring/session/session.py: the session decides in-place initialization by an actual file-creation attempt that leaves nothing behind, its debug-session methods moved into an inherited mixin, the measured CLI treats Windows Ctrl+Break like Ctrl+C, and the Official30 viewer test reads typed payload summaries. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/plate_manager_services.rst against B6's later changes to openhcs/authoring/session/session.py, openhcs/authoring/session/views.py, openhcs/authoring/session/events.py, openhcs/authoring/session/compile_batch.py, openhcs/authoring/session/dataset_document.py, openhcs/pyqt_gui/session_rendering.py, openhcs/mcp/server.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", @@ -1046,7 +1047,7 @@ }, { "path": "openhcs/authoring/session/session.py", - "sha256": "c1446fdf475503848a62086d13bc3e809dadd4958c3f763ea0347264bab5647d", + "sha256": "8d74425f65ce9f39f88439ca90bd09c02678ae3aef8d3cf6795570986444add0", "role": "Session shared by the desktop application and MCP for progress, cancellation and results." }, { @@ -1069,6 +1070,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/quick_start.rst against B6's later changes to openhcs/authoring/session/session.py: the writability probe makes one exclusive create attempt, the installed demo reports ViewerType.value, the unit session pre-creates its QApplication, and the GUI journey waits for each batch to end. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/quick_start.rst against B6's later changes to openhcs/authoring/session/session.py: the session decides in-place initialization by an actual file-creation attempt that leaves nothing behind, its debug-session methods moved into an inherited mixin, the measured CLI treats Windows Ctrl+Break like Ctrl+C, and the Official30 viewer test reads typed payload summaries. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/quick_start.rst against B6's later changes to openhcs/core/processing_contracts.py, openhcs/core/dataset_sources/source_bindings_source.py, openhcs/interop/cellprofiler/pipeline_import.py, openhcs/authoring/session/session.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/quick_start.rst against the later B6 changes to openhcs/core/processing_contracts.py, openhcs/core/source_bindings.py, openhcs/core/dataset_sources/source_bindings_source.py, openhcs/interop/cellprofiler/pipeline_import.py, openhcs/agent/capabilities.py, openhcs/authoring/session/session.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -2206,7 +2208,7 @@ }, { "path": "openhcs/authoring/session/session.py", - "sha256": "c1446fdf475503848a62086d13bc3e809dadd4958c3f763ea0347264bab5647d", + "sha256": "8d74425f65ce9f39f88439ca90bd09c02678ae3aef8d3cf6795570986444add0", "role": "Session owns the one execution client and keeps FinishedExecution (request, endpoint, ExecutionRecord) per execution." }, { @@ -2350,6 +2352,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/zmq_execution_service_extracted.rst against B6's later changes to openhcs/authoring/session/session.py: the writability probe makes one exclusive create attempt, the installed demo reports ViewerType.value, the unit session pre-creates its QApplication, and the GUI journey waits for each batch to end. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/zmq_execution_service_extracted.rst against B6's later changes to openhcs/authoring/session/session.py: the session decides in-place initialization by an actual file-creation attempt that leaves nothing behind, its debug-session methods moved into an inherited mixin, the measured CLI treats Windows Ctrl+Break like Ctrl+C, and the Official30 viewer test reads typed payload summaries. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/zmq_execution_service_extracted.rst against B6's later changes to openhcs/runtime/zmq_execution_server.py, openhcs/runtime/zmq_execution_client.py, openhcs/authoring/session/submission.py, openhcs/authoring/session/session.py, openhcs/runtime/zmq_orchestrator_environment.py, tests/unit/test_zmq_submission_timeout.py, openhcs/processing/backends/lib_registry/registry_service.py, openhcs/processing/backends/lib_registry/unified_registry.py, openhcs/processing/backends/lib_registry/openhcs_registry.py, tests/unit/desktop/test_desktop_restart.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/zmq_execution_service_extracted.rst against the later B6 changes to openhcs/runtime/zmq_execution_server.py, external/zmqruntime/src/zmqruntime/transport_modes.py, openhcs/runtime/zmq_execution_client.py, openhcs/authoring/session/submission.py, openhcs/authoring/session/session.py, openhcs/runtime/zmq_orchestrator_environment.py, tests/unit/test_zmq_submission_timeout.py, openhcs/processing/backends/lib_registry/registry_service.py, openhcs/processing/backends/lib_registry/unified_registry.py, openhcs/processing/backends/lib_registry/openhcs_registry.py, tests/unit/test_library_registry_discovery.py, tests/unit/desktop/test_desktop_restart.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -2515,7 +2518,7 @@ }, { "path": "openhcs/mcp/installed_demo.py", - "sha256": "fc5149e135ce528c7704028434011e347df8b42ffdec8bc2be333d9fc2d0bd37", + "sha256": "10fccd5b2d4382dca19857c0f7b9eaefe4eccbc34b4ff1898e8fced689a90223", "role": "Declares the deterministic fixture request (1x1 grid, 192 px tiles, 10% overlap, 1 px stage error, 2 wavelengths, 1 Z, 12 cells, 0.95 shared, A01, ImageXpress, seed 1)." }, { @@ -2537,6 +2540,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/agent_workflow_validation.rst against B6's later changes to openhcs/mcp/installed_demo.py: the writability probe makes one exclusive create attempt, the installed demo reports ViewerType.value, the unit session pre-creates its QApplication, and the GUI journey waits for each batch to end. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/development/agent_workflow_validation.rst against B6's later changes to openhcs/mcp/installed_demo.py: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/development/agent_workflow_validation.rst against B6's later changes to tests/unit/test_installed_mcp_demo.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/development/agent_workflow_validation.rst against the later B6 changes to tests/unit/test_installed_mcp_demo.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -2999,7 +3003,7 @@ }, { "path": "tests/conftest.py", - "sha256": "7bf3dc16e833053599ad02eb2e5a0a4744e0754f8008d827afc565a198717e79", + "sha256": "82d01adebe952964b61c0cad85024d0b3ba60a4a6dba1a11f6aa715a9f046664", "role": "Test-harness parsing of OPENHCS_CPU_ONLY, which accepts only 'true' when skipping pytest-qt." } ], @@ -3008,6 +3012,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/git_worktree_testing.rst against B6's later changes to tests/conftest.py: the writability probe makes one exclusive create attempt, the installed demo reports ViewerType.value, the unit session pre-creates its QApplication, and the GUI journey waits for each batch to end. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/development/git_worktree_testing.rst against B6's later changes to tests/conftest.py: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/development/git_worktree_testing.rst against the later B6 changes to pyproject.toml: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/git_worktree_testing.rst in full and compared it against the changed authorities: pyproject.toml (first-party floors moved to zmqruntime 0.6, objectstate 1.3.1, python-introspect 0.2.5, metaclass-registry 0.3, arraybridge 0.4.1, polystore 0.6, pyqt-reactive 0.4; JAX extra now jax[cuda12-local]; openhcs-basicpy added); the page states no version floors. Checked the eight editable installs against .gitmodules, the dev and gui extras, tests/integration/test_main.py, and every --it-* option and value against tests/pytest_integration_options.py; corrected OPENHCS_CPU_ONLY=1 to OPENHCS_CPU_ONLY=true to match tests/conftest.py.", @@ -3210,7 +3215,7 @@ }, { "path": "openhcs/authoring/session/session.py", - "sha256": "c1446fdf475503848a62086d13bc3e809dadd4958c3f763ea0347264bab5647d", + "sha256": "8d74425f65ce9f39f88439ca90bd09c02678ae3aef8d3cf6795570986444add0", "role": "Session submission budget and runtime-observation export validation (writable root, no existing path, outcomes require a path)." }, { @@ -3286,6 +3291,7 @@ ], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/mcp_development.rst against B6's later changes to openhcs/authoring/session/session.py: the writability probe makes one exclusive create attempt, the installed demo reports ViewerType.value, the unit session pre-creates its QApplication, and the GUI journey waits for each batch to end. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/development/mcp_development.rst against B6's later changes to openhcs/agent/dto/session.py: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/development/mcp_development.rst against B6's later changes to openhcs/authoring/session/session.py: the session decides in-place initialization by an actual file-creation attempt that leaves nothing behind, its debug-session methods moved into an inherited mixin, the measured CLI treats Windows Ctrl+Break like Ctrl+C, and the Official30 viewer test reads typed payload summaries. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/development/mcp_development.rst against B6's later changes to openhcs/mcp/dev_client_core.py, openhcs/agent/services/ui_bridge_service.py, openhcs/mcp/dev_client_commands/ui.py, openhcs/authoring/session/session.py, tests/unit/agent/test_mcp_server.py, openhcs/runtime/zmq_execution_client.py, tests/unit/test_zmq_submission_timeout.py, tests/unit/pyqt_gui/test_ui_agent_bridge.py, openhcs/mcp/server.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", diff --git a/docs/audits/user_docs.json b/docs/audits/user_docs.json index acfb0930f4..674c48d09d 100644 --- a/docs/audits/user_docs.json +++ b/docs/audits/user_docs.json @@ -1081,7 +1081,7 @@ }, { "path": "openhcs/mcp/installed_demo.py", - "sha256": "fc5149e135ce528c7704028434011e347df8b42ffdec8bc2be333d9fc2d0bd37", + "sha256": "10fccd5b2d4382dca19857c0f7b9eaefe4eccbc34b4ff1898e8fced689a90223", "role": "Implements the installed-package MCP demonstration used by the getting-started verification flow." }, { @@ -1156,6 +1156,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/getting_started/getting_started.rst against B6's later changes to openhcs/mcp/installed_demo.py: the writability probe makes one exclusive create attempt, the installed demo reports ViewerType.value, the unit session pre-creates its QApplication, and the GUI journey waits for each batch to end. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/getting_started/getting_started.rst against B6's later changes to openhcs/mcp/installed_demo.py, .github/workflows/integration-tests.yml: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/getting_started/getting_started.rst against B6's later changes to tests/unit/test_installed_mcp_demo.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/getting_started/getting_started.rst against the later B6 changes to pyproject.toml, tests/unit/test_installed_mcp_demo.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", From b14856fbd3f6fe2a144bd9519ce073004320378e Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 06:50:06 -0400 Subject: [PATCH 66/84] B6: observation-export test reads the partition count by its declared message field zmqruntime 0.6.0 (L3) renamed the results-summary key well_count to MessageFields.PARTITION_COUNT. Co-Authored-By: Claude Opus 5.5 --- .../test_agent_observation_export_zmq.py | 31 +++++++++++++------ 1 file changed, 21 insertions(+), 10 deletions(-) diff --git a/tests/integration/test_agent_observation_export_zmq.py b/tests/integration/test_agent_observation_export_zmq.py index 9fc672778f..66796e2a08 100644 --- a/tests/integration/test_agent_observation_export_zmq.py +++ b/tests/integration/test_agent_observation_export_zmq.py @@ -11,7 +11,7 @@ import pytest from objectstate.object_state import ObjectStateRegistry from zmqruntime import DataControlPortPairAuthority -from zmqruntime.messages import ExecutionStatus +from zmqruntime.messages import ExecutionStatus, MessageFields from benchmark.cellprofiler_benchmark_cli import create_benchmark_argument_parser from benchmark.contracts.measured_run_receipt import MeasuredPipelineRunReceipt @@ -113,14 +113,21 @@ def _start_run(built, plate: Path, pipeline, export_path: Path) -> tuple[str, in _call( built, "openhcs_set_dataset_pipeline", - {"scope_id": scope_id, "pipeline_source": PipelineDocumentCodec.render(pipeline)}, - ) - _wait_for_row(built, _call( - built, "openhcs_initialize_datasets", {"scope_ids": [scope_id]} - ), lambda row: row["initialized"] and not row["init_pending"]) - _wait_for_row(built, _call( - built, "openhcs_compile_datasets", {"scope_ids": [scope_id]} - ), lambda row: row["compiled"] and not row["compile_pending"]) + { + "scope_id": scope_id, + "pipeline_source": PipelineDocumentCodec.render(pipeline), + }, + ) + _wait_for_row( + built, + _call(built, "openhcs_initialize_datasets", {"scope_ids": [scope_id]}), + lambda row: row["initialized"] and not row["init_pending"], + ) + _wait_for_row( + built, + _call(built, "openhcs_compile_datasets", {"scope_ids": [scope_id]}), + lambda row: row["compiled"] and not row["compile_pending"], + ) started = _call( built, "openhcs_run_datasets", @@ -335,7 +342,11 @@ def test_ordinary_execution_can_export_outcomes_without_value_observation( exported = ZMQRuntimeExecutionOutcomeExport.read(export_path) exported.require_successful_axes() - assert completed.results_summary["well_count"] == exported.axis_count == 1 + assert ( + completed.results_summary[MessageFields.PARTITION_COUNT] + == exported.axis_count + == 1 + ) assert completed.results_summary["runtime_observation_export_scope"] == "outcomes" assert exported.successful_axis_count == 1 assert exported.output_roots From 2ed28bb325637f6b45e4ccba790e8ae6288aefb1 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 12:16:00 -0400 Subject: [PATCH 67/84] B6: black after merging main Co-Authored-By: Claude Opus 5.5 --- openhcs/core/pipeline/function_contracts.py | 4 +++- openhcs/pyqt_gui/main.py | 5 ++++- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/openhcs/core/pipeline/function_contracts.py b/openhcs/core/pipeline/function_contracts.py index 549406edb4..ca0ebfee14 100644 --- a/openhcs/core/pipeline/function_contracts.py +++ b/openhcs/core/pipeline/function_contracts.py @@ -404,7 +404,9 @@ def special_inputs(*parameter_names: str) -> Callable[[F], F]: ) def decorator(func: F) -> F: - vars(func)[FunctionContractAttribute.artifact_input_parameter_names] = normalized + vars(func)[ + FunctionContractAttribute.artifact_input_parameter_names + ] = normalized add_parameter_exclusions(func, normalized) return func diff --git a/openhcs/pyqt_gui/main.py b/openhcs/pyqt_gui/main.py index f3ec9dd2c9..799366c37e 100644 --- a/openhcs/pyqt_gui/main.py +++ b/openhcs/pyqt_gui/main.py @@ -860,7 +860,10 @@ def setup_status_bar(self): # Add time-travel controls to the ordinary left lane. Transient # QStatusBar messages hide ordinary widgets, so application status is # rendered by a permanent right-lane label below instead. - from pyqt_reactive.widgets.time_travel import TimeTravelCommands, TimeTravelWidget + from pyqt_reactive.widgets.time_travel import ( + TimeTravelCommands, + TimeTravelWidget, + ) from openhcs.authoring.session.datasets import DATASET_LIST_SCOPE_ID From b2fb1bcd286ebe48d78a90db642b594b0a98f47c Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 12:21:34 -0400 Subject: [PATCH 68/84] B6: re-review 126 pages again after main's U3, K4, K5, K6, L4 and L6 merges Pages naming moved or deleted mechanisms now name their replacements (SourceSetPairing, ProgressPhase/Status classes, DebugBoundary, pyqt-reactive config windows and shortcut lifecycle, objectstate.codegen, ScopeKey); records point at the current owners; digests pinned after review. Co-Authored-By: Claude Opus 5.5 --- docs/audits/architecture_a_m.json | 426 ++++++----- docs/audits/architecture_n_z_development.json | 645 ++++++++-------- docs/audits/project_surface.json | 26 +- docs/audits/user_docs.json | 718 +++++++++--------- .../architecture/code_ui_interconversion.rst | 6 +- .../source/architecture/concurrency_model.rst | 4 +- .../architecture/external_foundations.rst | 15 +- .../pattern_grouping_and_special_outputs.rst | 2 +- .../pipeline_compilation_system.rst | 2 +- .../progress_runtime_projection_system.rst | 38 +- .../architecture/serialization_boundaries.rst | 7 +- docs/source/architecture/source_model.rst | 7 +- docs/source/architecture/system_overview.rst | 7 +- .../architecture/ui_services_architecture.rst | 9 +- docs/source/concepts/core_model.rst | 5 +- docs/source/concepts/function_library.rst | 5 +- docs/source/concepts/module_structure.rst | 9 +- .../callable_artifact_authoring.rst | 6 +- .../development/source_binding_extension.rst | 5 +- .../guide_for_biologists/image_sources.rst | 6 +- 20 files changed, 1014 insertions(+), 934 deletions(-) diff --git a/docs/audits/architecture_a_m.json b/docs/audits/architecture_a_m.json index a9a4fec356..35f5c2be22 100644 --- a/docs/audits/architecture_a_m.json +++ b/docs/audits/architecture_a_m.json @@ -10,7 +10,7 @@ "authority": [ { "path": "external/pyqt-reactive/src/pyqt_reactive/widgets/shared/abstract_manager_widget.py", - "sha256": "92f8ede35a48392112c6c1aa17e959c3cc4a9450e69f5b1447cbc4ef5498dd5b", + "sha256": "befd5b285e4f64780a87609e0872dad84a05f399b96e0ccf684104a3e4de4cc6", "role": "Defines the extracted abstract manager widget, lifecycle hooks, selection, and item-management mechanism." }, { @@ -20,10 +20,11 @@ } ], "findings": [ - "The transition page linked only to a package-level overview instead of the exact owner narrative and OpenHCS integration page." + "The page is a transition stub for a mechanism now owned and documented by pyqt-reactive; its targets (pyqt-reactive architecture/abstract_manager_widget, ui_services_architecture, external_foundations) all still exist, so it stays a redirect." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/abstract_manager_widget.rst in full and compared it against the changed authorities: external/pyqt-reactive/src/pyqt_reactive/widgets/shared/abstract_manager_widget.py (U3 pyqt-reactive 0.5 advance); the class still lives at that path, pyqt-reactive still ships docs/architecture/abstract_manager_widget.rst, and both OpenHCS link targets exist, so the redirect remains accurate.", "Read the complete page and challenged accuracy, freshness, duplication, boundary, flow, and navigation against the listed current authorities.", "python scripts/validate_docs.py docs/source/architecture passed, and .venv Sphinx -E -W --keep-going rendered all 152 active pages without warnings.", "2026-09-09: Reviewed the page sections affected by exact manager-item navigation, selection recovery guidance, and the pyqt-reactive dependency floor change from 0.3.20 to 0.3.21. Generic selection and deferred dispatch remain in pyqt-reactive; domain window identities stay in OpenHCS. Navigation acceptance does not replace action revision guards. Installation extras, Python support, code-document apply, and experimental/runtime semantics are unchanged by this batch. Updated the MCP architecture and basic-interface navigation explanations; retained existing setup/ownership links rather than duplicating a version catalogue. Only the reviewed navigation/dependency authorities were refreshed, excluding the scientific threshold patch. Final parent gates: 176 OpenHCS and 463 pyqt-reactive tests passed, including deferred target removal; full fresh-agent trial remains separate acceptance evidence.", @@ -81,17 +82,17 @@ }, { "path": "openhcs/interop/cellprofiler/module_declarations.py", - "sha256": "6bbf41c158496a02439be2ee5aef12d7712fb632419c226a64c9259a55438b95", + "sha256": "b0195b66126c31f71dc0d4bb281a128e0164bb7dfb3b314ee8fe744ebc55d626", "role": "Defines CellProfilerModule leaves and require_module as owners of module identity, lowering, headless-import validation and artifact semantics." }, { "path": "openhcs/interop/cellprofiler/module_artifact_declarations.py", - "sha256": "f70b52114fb5d5e2c2fc49cd9a2b46986bde48ce02d82f1a34a67f072ca2ab20", + "sha256": "ef233be43002d0f16b53c78c202b9203b6261847708d311089671a6b037db894", "role": "Defines the nominal interactive-module family that rejects headless import." }, { "path": "openhcs/core/callable_contract.py", - "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", + "sha256": "a8e73c5df804d78baf72710e210cb2ae63156657a074071e11fd868f4b19d3b4", "role": "Declares CallableContract, the declaration-owned callable metadata at the nominal boundary." }, { @@ -101,12 +102,12 @@ }, { "path": "openhcs/core/dataset_sources/source.py", - "sha256": "097fde4d2b74031eacf53a88e79f499417e8c1c97ba0753e8a9df831b307efa6", + "sha256": "350bf5691a03522e9f54b1d8ab712c6beba1683aac4fa5159ea4e80c6041530d", "role": "Defines the DatasetSource registry and its layout detection and workspace projection boundary." }, { "path": "openhcs/core/dataset_sources/source_bindings_source.py", - "sha256": "fccda877f39e7b2f99d6cc95143480e5955186264e316e832ca9a69c2ae90b8c", + "sha256": "63e4a3f849ab5122ab7af8767a6288135b7330027dff962c80add93d5fd820f1", "role": "Defines SourceBindingsSource, which composes a dataset source with SourceBindingsConfig declarations." }, { @@ -126,16 +127,19 @@ }, { "path": "tests/unit/test_cellprofiler_pipeline_import.py", - "sha256": "6b1d201ec5c4ba620fa2367a9cbef1f6fbdd25edbdda9bd7b723ebe3a3363412", + "sha256": "0ce4efd85d819c7e9f7fa84ec6f1ac7ec4bda024c9a8d5c7f264fda2730264f0", "role": "Verifies that importing an enabled interactive-only module fails explicitly." + }, + { + "path": "openhcs/core/config.py", + "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", + "role": "Declares SourceBindingsConfig, the source-binding owner the page contrasts with DatasetSource (moved here from source_bindings.py by K4)." } ], - "findings": [ - "The page named MicroscopeHandler and SourceBindingsHandler; G4 replaced them with the DatasetSource family and SourceBindingsSource.", - "The strategy selection mixins moved from the deleted openhcs/core/registry_strategies.py into metaclass-registry; the page now says where they come from." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/abstraction_lattices.rst in full and compared it against the changed authorities: module_declarations.py, module_artifact_declarations.py and the pipeline-import test (K4 import moves, K5 flattened CallableContract metadata, PairByOrder), callable_contract.py (K5), dataset_sources/source.py and source_bindings_source.py (SourceBindingsConfig and source_bindings_defaults_to_base now imported from openhcs.core.config). The page names classes, not modules, for these owners; CellProfilerModule.require_module, CallableContract, ArtifactSpecRelation, DatasetSource, SourceBindingsConfig, SourceBindingsSource, the three metaclass-registry strategy mixins, RuntimePlaneAxisStrategy, AgentCapabilityDeclaration and CapabilityTransport all still exist, so no claim changed. Added openhcs/core/config.py as the current SourceBindingsConfig owner.", "On 2026-10-11 (B6 final), compared docs/source/architecture/abstraction_lattices.rst against B6's later changes to openhcs/interop/cellprofiler/module_declarations.py, openhcs/interop/cellprofiler/module_artifact_declarations.py, openhcs/core/callable_contract.py, openhcs/core/artifacts.py, openhcs/core/dataset_sources/source.py, openhcs/core/dataset_sources/source_bindings_source.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/abstraction_lattices.rst against the later B6 changes to openhcs/interop/cellprofiler/module_declarations.py, openhcs/interop/cellprofiler/module_artifact_declarations.py, openhcs/core/callable_contract.py, openhcs/core/artifacts.py, openhcs/core/dataset_sources/source.py, openhcs/core/dataset_sources/source_bindings_source.py, openhcs/agent/capabilities.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/abstraction_lattices.rst in full and compared it against the changed authorities: openhcs/core/registry_strategies.py (deleted; mixins now in metaclass_registry.strategies), openhcs/microscopes/microscope_base.py (deleted; DatasetSource family in openhcs/core/dataset_sources), module_declarations.py, module_artifact_declarations.py, callable_contract.py, openhcs/agent/capabilities.py and the two CellProfiler tests; replaced the microscope-handler owner and SourceBindingsHandler with DatasetSource and SourceBindingsSource, attributed the strategy mixins to metaclass-registry, and confirmed CellProfilerModule.require_module, ArtifactSpecRelation, RuntimePlaneAxisStrategy, for_enum_member and CapabilityTransport against current source. Historical commit references are left as history.", @@ -200,7 +204,7 @@ }, { "path": "tests/unit/test_function_artifact_materialization.py", - "sha256": "e92bd682c105778d8bde9cb0f63ff1baed9f4c082a05d2c7f8f04bf539bd699f", + "sha256": "6cb3fcbad366d303be1f0846c4497ae4b0bd4a95082aedac3801041907c5dc71", "role": "Proves the hook consumes saved CSV content, uses the compiled backend destination, skips a disabled config, and propagates failures." }, { @@ -209,12 +213,10 @@ "role": "Executes a native plate with separate analysis destinations and verifies per-directory and merged summaries without writing into the input plate." } ], - "findings": [ - "The page named consolidate_analysis_outputs in the deleted openhcs/core/orchestrator/analysis_consolidation.py; consolidation is now AnalysisConsolidationHook, a microscopy PostExecuteHook bound from the global config.", - "The page said consolidation selection is passed into runtime materialisation so writers can skip rendering; no caller passes an output path filter now, and the hook instead records the CSV text the writer already saved, filtered by the inclusion selection." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/analysis_consolidation_system.rst in full and compared it against the changed authorities: tests/unit/test_function_artifact_materialization.py (the streaming-filter test now uses the IncludePartitions class instead of PartitionFilterMode.INCLUDE, from K6/L-surface family changes); the page names no filter mode, and AnalysisConsolidationHook, PostExecuteHook, AnalysisTableSource, AnalysisSummaryWriter and DataSink.contextual_save_kwargs still exist at the listed owners, so no claim changed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/analysis_consolidation_system.rst against B6's later changes to openhcs/domains/microscopy/analysis_consolidation.py, openhcs/core/post_execute.py, tests/unit/test_function_artifact_materialization.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/analysis_consolidation_system.rst against the later B6 changes to openhcs/domains/microscopy/analysis_consolidation.py, openhcs/core/post_execute.py, tests/unit/test_function_artifact_materialization.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/analysis_consolidation_system.rst in full and compared it against the changed authorities: openhcs/core/orchestrator/analysis_consolidation.py (moved to openhcs/domains/microscopy/analysis_consolidation.py as a post-execute hook), consolidate_analysis_results.py, openhcs/core/config.py (consolidation config moved to openhcs/domains/microscopy/config.py), the three tests and openhcs/processing/materialization/core.py; rewrote the flow paragraph for AnalysisConsolidationHook and PostExecuteHook, removed the obsolete output-selection/skip-rendering paragraph, and confirmed the writer, DataSink save context, output-root destination and failure propagation against current source. Dropped test_orchestrator_lane_planning.py and materialization/core.py, which no longer own a stated claim.", @@ -250,22 +252,22 @@ }, { "path": "openhcs/core/callable_contract.py", - "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", + "sha256": "a8e73c5df804d78baf72710e210cb2ae63156657a074071e11fd868f4b19d3b4", "role": "Defines CallableContract and its ordered artifact input and output collections." }, { "path": "openhcs/core/function_patterns.py", - "sha256": "95b9056c6980e4f7943f909aeae8262252c9aa175247a534befc8d4cc459fdd2", + "sha256": "cc8c35f37ea0a3bd939905fe317652a34b3129ee6f47f6f9023b8c459e62df70", "role": "Defines InvocationArtifactInputEdgePlan, its main_flow_projection and storage plan, and MainFlowInputProjection." }, { "path": "openhcs/core/pipeline/function_contracts.py", - "sha256": "f5e7c74ea3d2948c75388184263cd9f1f17905f820b180f634dfaa7514e1d526", + "sha256": "ffc21dd3021a2316cfbd100c2d655867ce4bec619f2db7288e82aeb11c1930e8", "role": "Defines the special_inputs, artifact_inputs and artifact_outputs decorators." }, { "path": "openhcs/core/pipeline/path_planner.py", - "sha256": "677d106969aae4d818745bff29b6f84e487ec79a044c8eaf26bea1916236fc6c", + "sha256": "6ff14c63768ed5cbd77e84236b3980d62f616465e9ecfb7cdf27c63d8070a405", "role": "Implements artifact satisfaction and exact compiled path planning." }, { @@ -275,7 +277,7 @@ }, { "path": "tests/unit/test_callable_contract.py", - "sha256": "cc220f53107a08b0406b6e0dd49896e3e4c3c0af6e527b0ef00d4fdaba8f0b2c", + "sha256": "9170f8694f388dbed5bfd36d882a5c0a4cdf28f50f51f7fdeaf4a955a9f3b412", "role": "Verifies callable contract construction, validation, and artifact semantics." }, { @@ -284,12 +286,10 @@ "role": "Verifies occurrence-specific artifact input projection plans." } ], - "findings": [ - "The page described a consumes_main_flow flag on invocation input edges; the field is gone and an edge consumes main flow exactly when it carries a main_flow_projection.", - "The regression description said the test asserts consumes_main_flow=True; it now asserts a main-flow projection on the edge." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/artifact_contract_system.rst in full and compared it against the changed authorities: callable_contract.py and test_callable_contract.py (K5: CallableMetadata became CallableDeclaration and the metadata wrapper was flattened into direct ArtifactSpecCollection fields), function_patterns.py (K5 pattern consolidation: AuthoredFunctionPattern, kwarg merge removed), function_contracts.py (special_inputs now stores artifact_input_parameter_names) and path_planner.py (metadata injection over the ArtifactGraph, source_bindings.compiled_plan). The page's claims still hold: special_inputs only names ABI slots, CallableContract carries ordered ArtifactSpecCollection inputs/outputs, InvocationArtifactInputEdgePlan keeps spec, storage_plan and main_flow_projection with the two-member MainFlowInputProjection enum, and every other named class (ArtifactPlanKeySelector, ArtifactDeclarationStepContext, MainFlowStackOutputSpec, SelectedPlaneImageOutput, SourceProjectedImageOutput, the object subject relations and binding, SourceImageProvenance, StepMaterializationConfig) still exists, so no claim changed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/artifact_contract_system.rst against B6's later changes to openhcs/core/pipeline/function_contracts.py: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/artifact_contract_system.rst against B6's later changes to openhcs/core/artifacts.py, openhcs/core/callable_contract.py, openhcs/core/function_patterns.py, openhcs/core/pipeline/function_contracts.py, openhcs/core/pipeline/path_planner.py, tests/unit/test_implicit_main_flow_compilation.py, tests/unit/test_callable_contract.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/artifact_contract_system.rst against the later B6 changes to openhcs/core/artifacts.py, openhcs/core/callable_contract.py, openhcs/core/function_patterns.py, openhcs/core/pipeline/function_contracts.py, openhcs/core/pipeline/path_planner.py, tests/unit/test_implicit_main_flow_compilation.py, tests/unit/test_callable_contract.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -339,7 +339,7 @@ }, { "path": "openhcs/authoring/session/session.py", - "sha256": "8d74425f65ce9f39f88439ca90bd09c02678ae3aef8d3cf6795570986444add0", + "sha256": "0c4e0f0979e1793862859e47eefeb6dd9d4b51c99fba20c2ebc6adf11619c3b5", "role": "Implements run_datasets (reserve, reset, connect, compile all, submit each with compile_artifact_id) and the endpoint status binding." }, { @@ -358,11 +358,10 @@ "role": "Guards that session state lives in the Qt-free session package and its operations are declared classes." } ], - "findings": [ - "The page had already been rewritten for the session in U1, but its audit record still named the deleted Plate Manager batch workflow, compile batch, execution-state and client-service modules and their tests; the audience and user need also still described Plate Manager." - ], + "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/batch_workflow_service.rst in full and compared it against the changed authorities: openhcs/authoring/session/session.py (K6 debug command and replay mode became classes: RunDebugCommand, StopDebugCommand, PersistentPausedWorkerReplay); the page does not describe debug sessions, and CompileDatasets, DatasetPipelineRequest, CompiledDataset, BatchSubmitWaitEngine, compile_artifact_id submission, ExecutionProgress, Session.close and the endpoint_status binding all still exist, so no claim changed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/batch_workflow_service.rst against B6's later changes to openhcs/authoring/session/session.py: the writability probe makes one exclusive create attempt, the installed demo reports ViewerType.value, the unit session pre-creates its QApplication, and the GUI journey waits for each batch to end. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/batch_workflow_service.rst against B6's later changes to openhcs/authoring/session/session.py: the session decides in-place initialization by an actual file-creation attempt that leaves nothing behind, its debug-session methods moved into an inherited mixin, the measured CLI treats Windows Ctrl+Break like Ctrl+C, and the Official30 viewer test reads typed payload summaries. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/batch_workflow_service.rst against B6's later changes to openhcs/authoring/session/operations/datasets.py, openhcs/authoring/session/compile_batch.py, openhcs/authoring/session/session.py, openhcs/authoring/session/submission.py, openhcs/authoring/session/progress.py, tests/unit/test_u1_session_guards.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", @@ -402,27 +401,27 @@ "authority": [ { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "3439d96f3fbfcb104a53019dc150049021d9a8132bd27096f1ccf510eefbe37b", + "sha256": "b96f33b866fe6c9f8118ca70fcb7dff8e6a02cb29d79df67230a5e8b97df8002", "role": "Implements .cppipe parsing, declaration-owned headless validation, and lowering into PipelineConfig and FunctionStep declarations." }, { "path": "openhcs/interop/cellprofiler/module_declarations.py", - "sha256": "6bbf41c158496a02439be2ee5aef12d7712fb632419c226a64c9259a55438b95", + "sha256": "b0195b66126c31f71dc0d4bb281a128e0164bb7dfb3b314ee8fe744ebc55d626", "role": "Owns module-specific callable, setting, headless-import, scope, artifact, and function-catalogue declarations." }, { "path": "openhcs/interop/cellprofiler/module_artifact_declarations.py", - "sha256": "f70b52114fb5d5e2c2fc49cd9a2b46986bde48ce02d82f1a34a67f072ca2ab20", + "sha256": "ef233be43002d0f16b53c78c202b9203b6261847708d311089671a6b037db894", "role": "Defines the nominal interactive-module family that rejects headless import." }, { "path": "openhcs/interop/cellprofiler/compile_time_contracts.py", - "sha256": "75d24fb50658de135b4d777e78aacda84d15769a76d31043343f18645cb92f3a", + "sha256": "3b1879d23635b1e9f7998522e812c738417de85fe53f0201501981a1780a21c3", "role": "Defines CellProfilerInvocationContractProviderFactory, which walks the ResolvedPipelineDefinition steps with a forward artifact context." }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", + "sha256": "0f9cc4819b53d55b7e9446429b8c8ac1ff1a59f4902020ea1658495ce01d464c", "role": "Declares ResolvedPipelineDefinition, the resolved step sequence the provider walks." }, { @@ -432,7 +431,7 @@ }, { "path": "tests/unit/test_cellprofiler_artifact_declarations.py", - "sha256": "fe894cd2b592d0755c106be3363eeaf078bc26d0b698a7ccfe6662ef0c0c475b", + "sha256": "ba791ec36bbc4159c09a4ea8d950e05785a539b5a30a85394098bba26aa11f5a", "role": "Verifies typed image, object, measurement, relationship, table, grid, graph, and special artifact declarations." }, { @@ -442,7 +441,7 @@ }, { "path": "tests/unit/test_cellprofiler_pipeline_import.py", - "sha256": "6b1d201ec5c4ba620fa2367a9cbef1f6fbdd25edbdda9bd7b723ebe3a3363412", + "sha256": "0ce4efd85d819c7e9f7fa84ec6f1ac7ec4bda024c9a8d5c7f264fda2730264f0", "role": "Verifies that direct import rejects an enabled interactive-only module explicitly." }, { @@ -456,11 +455,10 @@ "role": "Proves exact inventory selection, generated-pipeline digest validation, exporter derivation, and the declared image and label comparison contracts." } ], - "findings": [ - "The compile-time reconstruction section said the provider walks a resolved StepSnapshot sequence; StepSnapshot and openhcs/core/pipeline/step_snapshot.py are gone, and the provider walks the steps of the ResolvedPipelineDefinition." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/cellprofiler_interop.rst in full and compared it against the changed authorities: pipeline_import.py, module_declarations.py, module_artifact_declarations.py, compile_time_contracts.py (K4 moved SourceBindingsConfig/StepSourceBindingsConfig into openhcs.core.config; K5 flattened CallableContract.metadata into direct artifact_inputs/artifact_outputs/runtime_adapter fields), compilation_session.py, and the two CellProfiler unit tests (SourceBindingMatchMethod.ORDER became PairByOrder). The page names SourceBindingsConfig, CallableContract and the module pipeline without import paths or the metadata wrapper; every class and function it names (CPPipeParser, SettingsBinder, CellProfilerModuleArtifactContracts, SettingToKeywordBinding, CellProfilerInvocationContractProviderFactory, ArtifactDeclarationStepContext, FunctionStepExecutionScope.PLATE, FileBundleOptions, CPASQLiteRenderer, CellProfilerReferenceExportPlan, numpy_avx512_skx_svml_symbol_available, the two-member InputSource enum) still exists, so no claim changed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/cellprofiler_interop.rst against B6's later changes to tests/integration/test_cellprofiler_official30_zmq.py: the session decides in-place initialization by an actual file-creation attempt that leaves nothing behind, its debug-session methods moved into an inherited mixin, the measured CLI treats Windows Ctrl+Break like Ctrl+C, and the Official30 viewer test reads typed payload summaries. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/cellprofiler_interop.rst against B6's later changes to openhcs/interop/cellprofiler/pipeline_import.py, openhcs/interop/cellprofiler/module_declarations.py, openhcs/interop/cellprofiler/module_artifact_declarations.py, openhcs/core/pipeline/compilation_session.py, tests/unit/test_cellprofiler_artifact_declarations.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/cellprofiler_interop.rst against the later B6 changes to openhcs/interop/cellprofiler/pipeline_import.py, openhcs/interop/cellprofiler/module_declarations.py, openhcs/interop/cellprofiler/module_artifact_declarations.py, openhcs/core/pipeline/compilation_session.py, tests/unit/test_cellprofiler_artifact_declarations.py, tests/integration/test_cellprofiler_official30_zmq.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -479,7 +477,7 @@ }, { "path": "docs/source/architecture/code_ui_interconversion.rst", - "source_sha256": "e92fe7d1580b8cbc121dbf13023c64faf85150de3e4bca9a13f99513ef6e2cea", + "source_sha256": "e1a2607666a806193e4abe240ed21cb189326d5e1d54da761511e2f70fd895f0", "audience": [ "OpenHCS UI and agent-surface maintainers", "advanced attached-agent users" @@ -489,12 +487,12 @@ "authority": [ { "path": "openhcs/core/pipeline_document.py", - "sha256": "8c554c8275c665f308eb098955bda517aa804eeb34418b5e0f41d412650ce7b5", + "sha256": "9a77a08367f901594859c8b6cae9609c9230a551f4f4e9ec6ea6cd2ed233c0f4", "role": "Defines the nominal pipeline code document and its rendering, parsing, and application boundary." }, { "path": "openhcs/pyqt_gui/services/ui_bridge_object_state.py", - "sha256": "e548b367507b3fb6830ce9ecb844e3b756243af96fe38e0d35de5da0b974c1fd", + "sha256": "e64b444a74fdde1d79f82c1361c9b67d8e8916b9bebced092762ce51eabac52b", "role": "Projects live ObjectState scopes, fields, revisions, document capabilities, and mutation receipts through the UI bridge." }, { @@ -514,7 +512,7 @@ }, { "path": "external/zmqruntime/src/zmqruntime/messages.py", - "sha256": "ba7b9bac8f0202690f1e2bd17f445fe1741f43642f0a36e4a806bb577c2d1aa1", + "sha256": "fb257a7ac25a47d8c604fbfbd88c96b92fac16f914a61dab81c784fe99d95378", "role": "Owns generic exact endpoint-application compatibility and the typed mismatch error." }, { @@ -529,17 +527,17 @@ }, { "path": "external/pyqt-reactive/src/pyqt_reactive/services/window_code_document.py", - "sha256": "d0ab321af0eb2b14fac3ab2577c8c61672763d2f13a70a7ce8a35ee7bfab7c75", + "sha256": "56d92d665a7b3bf6acde28292679cad13388793e1acb5bea74926f4330150d42", "role": "Owns generic window-document writability and declaration-normalised source comparison." }, { "path": "tests/unit/pyqt_gui/test_ui_agent_bridge.py", - "sha256": "de6758859d9743e826369162855327e4a84056cafdd7ddfdc33868d2a3093353", + "sha256": "67cdad5133dc83ccb005b58cc56535ae89e77fc23d3ebd76e616c0da5f567b44", "role": "Verifies read-only rejection, semantic no-op application, and branch restoration through the UI bridge." }, { "path": "openhcs/pyqt_gui/services/function_step_code_document.py", - "sha256": "67b00a1964680d65a371e238c862775069bb882da2ccb75773adfed4981b90f7", + "sha256": "f89732e608ddaae8b44dac635a80e86a95eb6c953194558471ee6eed7ca75b65", "role": "Defines the step-editor code-document projection and apply semantics." }, { @@ -549,12 +547,12 @@ }, { "path": "openhcs/pyqt_gui/widgets/step_parameter_editor.py", - "sha256": "d7d6a2af7d3dafbd3edd57d4a3c52a1f40812e21b95820d1e2190a57e9a97c0c", + "sha256": "534f33acc5e295bfda4fbd73cbec49c4fbf662953e14842bf04ce59550f2dea3", "role": "Applies edited function patterns through generic reconciliation before rebuilding parameter panes." }, { "path": "openhcs/serialization/pycodify_formatters.py", - "sha256": "fe06f3ab873b537b7b2c0b13599ffd11030ae6a8ac504caea9e3dcfa1d4695b4", + "sha256": "18b744cdeb8d9ce22801efe6f3f81fec346194019293e68f542dc13c11724a2d", "role": "Supplies OpenHCS semantic equality and clean/full choices to shared parameter projection." }, { @@ -564,7 +562,7 @@ }, { "path": "external/pyqt-reactive/src/pyqt_reactive/services/function_pattern_code_document.py", - "sha256": "bdf41b58bff009890e6e696cfc4f18313adfb1669566b2566f9e7b93d863c071", + "sha256": "77b0bc021efaaac47bc5fb8c29aa50536162a9630dafd48c6ece42790837b94d", "role": "Projects child parameter states into function-pattern code documents, retaining a child only while its function authority and editable surface match." }, { @@ -579,18 +577,31 @@ }, { "path": "tests/unit/test_pycodify_formatters.py", - "sha256": "2f0c6683a5255d01442ae33f676f601d5e3f1e3e04d1b2c2fc945b3c564796ce", + "sha256": "78d695333a74b4a75a07ca7de87a5c685a6fd429c67412b43375d16a298a9c80", "role": "Verifies clean/full parameter order and semantic default handling." }, { "path": "external/pyqt-reactive/tests/test_numeric_roundtrip.py", "sha256": "5975b0ce8bd7bfcf1b21bf03593b72882c775a7982d834894081b6012c556566", "role": "Verifies numeric value preservation and compact presentation." + }, + { + "path": "external/ObjectState/src/objectstate/codegen.py", + "sha256": "347d2697c061a35e6e440e373fe4e8bfce1ad92b500e98da07faa26f6a3a1356", + "role": "Owns lazy-dataclass source formatting and field_needs_source, the clean-view decision that leaves unset lazy fields out of generated code." + }, + { + "path": "external/ObjectState/src/objectstate/inspection.py", + "sha256": "aba713b8bba961c2fc53e31ea181ed807d8190f7290e8fb4647f0044c5ae771f", + "role": "Defines FieldInspection, the raw/resolved/dirty/provenance field view the UI bridge projection inherits." } ], - "findings": [], - "disposition": "keep", + "findings": [ + "The round-trip section implied OpenHCS owns the semantic equality rule and lazy-field omission for clean code views; after L6, pycodify_formatters.py passes ObjectState's semantic_values_equal and the lazy dataclass formatting and field_needs_source decision live in objectstate.codegen. Reworded and added objectstate/codegen.py and objectstate/inspection.py (field projection now inherits FieldInspection) as authorities." + ], + "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/code_ui_interconversion.rst in full and compared it against the changed authorities: openhcs/core/pipeline_document.py (renders through pycodify.generate_python_source), ui_bridge_object_state.py (field projection now extends objectstate.inspection.FieldInspection), zmqruntime messages.py and pyqt-reactive window_code_document.py / function_pattern_code_document.py (U3 advance; config code-document driver and external-editor preference moved into pyqt-reactive), function_step_code_document.py and step_parameter_editor.py (patch_lazy_constructors from ObjectState, CodeEditorFormUpdater from pyqt-reactive), pycodify_formatters.py and its test (L6: lazy-dataclass, enum, path and import formatters deleted in favour of objectstate.codegen and pycodify imported_name), and test_ui_agent_bridge.py (K6 debug/progress classes, DockPane). Corrected the equality/omission ownership sentence and added the ObjectState codegen and inspection owners; document, revision, field-mutation, bridge identity and parameter-order claims still hold.", "On 2026-10-11 (B6 final), compared docs/source/architecture/code_ui_interconversion.rst against B6's later changes to openhcs/pyqt_gui/services/ui_bridge_object_state.py, openhcs/agent/dto/ui_bridge.py, openhcs/pyqt_gui/services/ui_agent_bridge.py, tests/unit/pyqt_gui/test_ui_agent_bridge.py, tests/unit/pyqt_gui/test_pipeline_code_document_transactionality.py, tests/unit/test_pycodify_formatters.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/code_ui_interconversion.rst against the later B6 changes to openhcs/pyqt_gui/services/ui_bridge_object_state.py, openhcs/agent/dto/ui_bridge.py, openhcs/pyqt_gui/services/ui_agent_bridge.py, tests/unit/pyqt_gui/test_ui_agent_bridge.py, tests/unit/pyqt_gui/test_pipeline_code_document_transactionality.py, tests/unit/test_pycodify_formatters.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/code_ui_interconversion.rst in full and compared it against the changed authorities: openhcs/core/pipeline_document.py (PipelineDocumentAuthority renamed PipelineDocumentCodec; PipelineDocument unchanged), ui_bridge_object_state.py, openhcs/agent/dto/ui_bridge.py, ui_bridge_transport.py, ui_bridge_server.py, zmqruntime messages.py, python-introspect annotation_types.py, ui_agent_bridge.py and its test, function_step_code_document.py (now FunctionStepDocumentCodec), the code-document transactionality test, step_parameter_editor.py (the pickled .step settings files are gone, which the page never described), pycodify_formatters.py and its test; checked the schema/bridge-protocol/application identity checks on both bridge ends, the field-mutation request token without base revision, enum values in field help, code-document revision apply, and Plate Manager naming. Every claim the page makes still holds; no text change.", @@ -624,12 +635,12 @@ "authority": [ { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "sha256": "4a33908fb42874e6d57eeea3f5305793ddbda41d13356f0a78501af38dcdfeb4", "role": "Lowers declarations into typed step plans, including declaration-derived framework-device assignments." }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", + "sha256": "0f9cc4819b53d55b7e9446429b8c8ac1ff1a59f4902020ea1658495ce01d464c", "role": "Defines the per-context CompilationSession and ResolvedPipelineDefinition." }, { @@ -639,10 +650,11 @@ } ], "findings": [ - "The old compilation-service URL no longer owns compiler facts and is intentionally a concise transition to the current model." + "The old compilation-service URL owns no compiler facts and stays a concise transition to pipeline_compilation_system; no claim on it became false." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/compilation_service.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compiler.py (ScopeKey-built scope ids, CompileProgressPhase/RunningProgressStatus classes, CompiledSourceBindingPlan() default) and openhcs/core/pipeline/compilation_session.py (strip_disabled_functions removed, invocation validation moved onto the graph); CompilationSession, ResolvedPipelineDefinition and CompiledExecutionBundle still exist and the redirect target exists, so the transition remains accurate.", "On 2026-10-11 (B6 final), compared docs/source/architecture/compilation_service.rst against B6's later changes to openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/compilation_session.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/compilation_service.rst against the later B6 changes to openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/compilation_session.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/compilation_service.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/compiled_execution.py; the page only names the session, typed-plan and execution-bundle model and redirects to pipeline_compilation_system, all of which still exist (CompilationSession, CompiledStepPlan, CompiledExecutionBundle), so the redirect stays unchanged.", @@ -665,17 +677,17 @@ "authority": [ { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "sha256": "4a33908fb42874e6d57eeea3f5305793ddbda41d13356f0a78501af38dcdfeb4", "role": "Implements the current compiler stages, including declaration-derived framework-device assignment." }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", + "sha256": "0f9cc4819b53d55b7e9446429b8c8ac1ff1a59f4902020ea1658495ce01d464c", "role": "Defines ResolvedPipelineDefinition, which replaced resolved step snapshots, and CompilationSession." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", + "sha256": "358d3eaf5952170f9e9e2dc9056fb08707e8fcf74682063c074b367a83c2648f", "role": "Defines typed step-plan fields, exact framework footprints, and framework-local device bindings." }, { @@ -685,10 +697,11 @@ } ], "findings": [ - "The redirect named StepSnapshot as part of the current architecture; openhcs/core/pipeline/step_snapshot.py was deleted and compilation now reads ResolvedPipelineDefinition." + "The old detailed-compiler URL is a transition to pipeline_compilation_system; every class it names (ResolvedPipelineDefinition, CompilationSession, CompiledExecutionBundle) still exists, so it stays a redirect with no false claim." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/compilation_system_detailed.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compiler.py (ScopeKey scope ids, progress phase/status classes), openhcs/core/pipeline/compilation_session.py (disabled-function stripping removed, graph-level invocation validation) and openhcs/core/compiled_step_plan.py (source binding and universe plans default-construct instead of .empty()); the named classes and the redirect target still exist, so the transition remains accurate.", "On 2026-10-11 (B6 final), compared docs/source/architecture/compilation_system_detailed.rst against B6's later changes to openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/compiled_step_plan.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/compilation_system_detailed.rst against the later B6 changes to openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/compiled_step_plan.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/compilation_system_detailed.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/step_snapshot.py (deleted in 0adb68108; resolved steps now live on ResolvedPipelineDefinition in compilation_session.py), openhcs/core/compiled_step_plan.py; replaced StepSnapshot with ResolvedPipelineDefinition in the redirect text and confirmed the redirect target page exists.", @@ -711,12 +724,12 @@ "authority": [ { "path": "external/ObjectState/src/objectstate/lazy_factory.py", - "sha256": "4f86ec64d975b354203539581943f36a75e34fbf2da076a3ef9b32036e54bff9", + "sha256": "b9bd1547f5f78a6e15399006c085b1459f3172e1d31637233a93200846e07c37", "role": "Implements generic lazy dataclass creation and resolution owned by ObjectState." }, { "path": "openhcs/core/config.py", - "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", "role": "Declares the kernel OpenHCS configuration dataclasses that consume ObjectState mechanics." }, { @@ -726,10 +739,11 @@ } ], "findings": [ - "The transition did not link the exact ObjectState architecture owner or the component-system integration page." + "The page is a transition stub for lazy configuration now owned by ObjectState; its targets (ObjectState docs/architecture.rst, component_system_integration, external_foundations) still exist, so it stays a redirect." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/component_configuration_framework.rst in full and compared it against the changed authorities: external/ObjectState/src/objectstate/lazy_factory.py (ObjectState 2.x advance) and openhcs/core/config.py (K4 source-binding classes, K6/L-surface family moves); the page states no configuration fact beyond ownership, ObjectState still owns lazy dataclass creation in lazy_factory.py and ships docs/architecture.rst, and both OpenHCS link targets exist, so the redirect remains accurate.", "On 2026-10-11 (B6 final), compared docs/source/architecture/component_configuration_framework.rst against B6's later changes to openhcs/core/config.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/component_configuration_framework.rst against the later B6 changes to external/ObjectState/src/objectstate/lazy_factory.py, openhcs/core/config.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/component_configuration_framework.rst in full and compared it against the changed authorities: external/ObjectState/src/objectstate/lazy_factory.py, openhcs/core/config.py (domain sections such as analysis consolidation moved to openhcs/domains/microscopy/config.py, now added as an authority); the page only states that ObjectState owns lazy configuration and that OpenHCS documents its domain declarations, and redirects to component_system_integration and external_foundations, both of which exist, so the redirect stays unchanged.", @@ -806,7 +820,7 @@ }, { "path": "openhcs/core/config.py", - "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", "role": "Declares the step-level variable_components and group_by fields and the pipeline-level sequential_components field." }, { @@ -826,16 +840,14 @@ }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "sha256": "4a33908fb42874e6d57eeea3f5305793ddbda41d13356f0a78501af38dcdfeb4", "role": "Consumes the axis declarations to construct execution-axis contexts, stacks, grouping, and typed plans." } ], - "findings": [ - "The page listed ``multiprocessing_axis`` as a component declaration; no such declaration exists. The parallel axis is the family's single PartitionAxis member (well in the microscopy family).", - "The constants.py and component_set.py authorities no longer own component declarations after G1 deleted the component enums; replaced them with openhcs/core/axes.py and the config fields' owner, and added the dataset-source interface that defines MetadataComponentValueSet." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/component_system_integration.rst in full and compared it against the changed authorities: openhcs/core/config.py (PartitionFilterMode became an IncludePartitions/ExcludePartitions class family; SourceBindingsConfig/StepSourceBindingsConfig moved in by K4) and openhcs/core/pipeline/compiler.py (ScopeKey scope ids, progress classes); the page names no filter mode or source-binding class, and variable_components, group_by, sequential_components, PartitionAxis, AxisValues, FilenameParseResult and MetadataComponentValueSet are unchanged, so no claim changed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/component_system_integration.rst against B6's later changes to openhcs/core/axes.py, openhcs/core/config.py, openhcs/core/dataset_sources/interfaces.py, openhcs/core/pipeline/compiler.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/component_system_integration.rst against the later B6 changes to openhcs/core/axes.py, openhcs/core/config.py, openhcs/core/dataset_sources/interfaces.py, openhcs/core/pipeline/compiler.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/component_system_integration.rst in full and compared it against the changed authorities: constants.py, component_set.py, component_values.py, parser_metaprogramming.py, compiler.py; verified AxisValues, FilenameParseResult, MetadataComponentValueSet, variable_components/group_by/sequential_components and AxisFamily.partition_axis in current source, and replaced the nonexistent multiprocessing_axis declaration with the family's PartitionAxis.", @@ -875,22 +887,19 @@ }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "sha256": "4a33908fb42874e6d57eeea3f5305793ddbda41d13356f0a78501af38dcdfeb4", "role": "Rejects pipeline sequential axes that are also a step's variable axes." }, { "path": "tests/unit/test_funcstep_contract_validator.py", - "sha256": "a208ca59a60b2d5bbcc9955e99592fd480368995336f6ef86a0a06f8f6991747", + "sha256": "65d9b010d43c85e20fb4ef5c2f8f4c7363496384635ec64d906c7138649d1900", "role": "Covers accepted and rejected grouping, required-role, and callable-contract combinations." } ], - "findings": [ - "The page told consumers to use nominal enum members; G1 deleted the component enums and consumers now use the axis classes.", - "The check list was imprecise: variable axes are checked against the family's variable axes (not a source universe), dictionary-pattern keys against dataset values, callable requirements are axis roles, and the sequential check rejects overlap with a step's variable axes; there is no separate multiprocessing topology check.", - "The opening framed the declarations as microscope dimensions; the kernel is domain-blind and the microscopy family is one declaration." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/component_validation_system.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compiler.py (ScopeKey scope ids, CompileProgressPhase/RunningProgressStatus progress classes, CompiledSourceBindingPlan() default) and tests/unit/test_funcstep_contract_validator.py (CallableDeclaration, admitted_graph); confirmed AxisFamily.named, variable_axes, Ungrouped, GenericValidator.validate_step/validate_dict_pattern_keys, validate_allowed_group_by, validate_required_axis_roles and validate_sequential_components_compatibility still exist and behave as stated; no claim changed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/component_validation_system.rst against B6's later changes to openhcs/core/axes.py, openhcs/core/pipeline/funcstep_contract_validator.py, openhcs/core/pipeline/compiler.py, tests/unit/test_funcstep_contract_validator.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/component_validation_system.rst against the later B6 changes to openhcs/core/axes.py, openhcs/core/pipeline/funcstep_contract_validator.py, openhcs/core/pipeline/compiler.py, tests/unit/test_funcstep_contract_validator.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/component_validation_system.rst in full and compared it against the changed authorities: validation.py, funcstep_contract_validator.py, test_funcstep_contract_validator.py; traced GenericValidator.validate_step, validate_dict_pattern_keys, validate_allowed_group_by, validate_required_axis_roles and the compiler's sequential-components check, and rewrote the check list and consumer guidance to match.", @@ -903,7 +912,7 @@ }, { "path": "docs/source/architecture/concurrency_model.rst", - "source_sha256": "6c3bd9dbb4c18735e7bcccf48777ddf0f8ea646ae8ab9d3e5a6d2ce8fd0120bb", + "source_sha256": "2277aa8bbedf0052770b78bca2424ef45c580eeac121973ad39ba9807a14b0c9", "audience": [ "OpenHCS runtime and multiprocessing maintainers" ], @@ -922,7 +931,7 @@ }, { "path": "openhcs/core/orchestrator/bundle_execution.py", - "sha256": "f5ba09a2ec7c688b1ca08167ab398cc976a9e2c39b301ae98f3c5fd560a25c85", + "sha256": "bc3a2d0a50c3db3be81b03f6af4a139c3db2f779e969aad608e59be66508ee4b", "role": "Runs one compiled bundle, then the parent-owned dataset-scoped steps and post-execute hooks." }, { @@ -932,7 +941,7 @@ }, { "path": "openhcs/core/orchestrator/worker_execution.py", - "sha256": "23a3ef2a70378dd1bb5c413a56e36bbb3fa013142741c9b45407bfc47de8c4a4", + "sha256": "26b9586890e7117b24b6b85660d3f417455d55546401739f91146d18e4d21134", "role": "Owns executor construction, the WorkerCommand family served by process workers, result collection, cancellation, and cleanup." }, { @@ -947,23 +956,26 @@ }, { "path": "tests/unit/test_orchestrator_lane_planning.py", - "sha256": "3d5c2f3662b8d4c330582af0c4a649b8f4d9582e064d9cf3159798c8934a2bbb", + "sha256": "85a73ddf071552fc88d454c9de857d066fd62f45492b74945ee22ec908b85289", "role": "Verifies that worker-lane grouping remains correct when context mapping keys do not encode axis identity." }, { "path": "tests/unit/test_worker_start_policy.py", "sha256": "5a6f5e1ef54c49bc7f49c8b7baa530e685a8b76daf37679488ec50d9fb5fc37c", "role": "Verifies platform, server, override, and GPU start-method decisions." + }, + { + "path": "openhcs/core/progress/types.py", + "sha256": "c46e825cd32801caae9fc107e0759976da054eadc56adc0d13354b3a8b7db209", + "role": "Declares the ProgressPhase and ProgressStatus class families, including AxisCompletedProgressPhase and SuccessProgressPhase." } ], "findings": [ - "The page named ``compiled_plate_execution``, which G7 renamed to ``bundle_execution``.", - "The execution-unit text said the configured multiprocessing axis yields one context per well and that workers receive one context at a time; the partition axis is the family's PartitionAxis, and each worker executes a lane of the contexts it owns.", - "Process workers now serve a typed WorkerCommand family and forward progress over their parent connection; the page described only a shared queue.", - "Output consolidation is a microscopy-domain post-execute hook, and terminal steps are dataset-scoped; the page called them plate steps and core consolidation." + "K6 replaced the ProgressPhase enum members with phase classes; the page still named the AXIS_COMPLETED and SUCCESS enum members, now AxisCompletedProgressPhase and SuccessProgressPhase." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/concurrency_model.rst in full and compared it against the changed authorities: openhcs/core/orchestrator/bundle_execution.py (progress phase/status classes, lane results reordered to compiled axis order), openhcs/core/orchestrator/worker_execution.py (progress classes) and tests/unit/test_orchestrator_lane_planning.py; renamed the AXIS_COMPLETED and SUCCESS enum references to AxisCompletedProgressPhase and SuccessProgressPhase from openhcs/core/progress/types.py; the lane, worker-command, start-method and cancellation claims still hold.", "On 2026-10-11 (B6 final), compared docs/source/architecture/concurrency_model.rst against B6's later changes to openhcs/core/orchestrator/bundle_execution.py, openhcs/core/orchestrator/worker_execution.py, tests/unit/test_orchestrator_lane_planning.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/concurrency_model.rst against the later B6 changes to openhcs/core/orchestrator/bundle_execution.py, openhcs/core/orchestrator/worker_execution.py, tests/unit/test_orchestrator_lane_planning.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/concurrency_model.rst in full and compared it against the changed authorities: compiled_execution.py, processing_context.py, worker_lanes.py, test_orchestrator_lane_planning.py, worker_execution.py, test_worker_start_policy.py; confirmed CompiledExecutionBundle, WorkerLaneExecutionContext, resolve_worker_start_context, the cancellation signal and AXIS_COMPLETED, renamed compiled_plate_execution to bundle_execution, and corrected the lane, worker-command, progress and post-execute claims.", @@ -988,25 +1000,26 @@ "authority": [ { "path": "external/ObjectState/src/objectstate/lazy_factory.py", - "sha256": "4f86ec64d975b354203539581943f36a75e34fbf2da076a3ef9b32036e54bff9", + "sha256": "b9bd1547f5f78a6e15399006c085b1459f3172e1d31637233a93200846e07c37", "role": "Implements reusable lazy dataclass configuration mechanics." }, { "path": "external/ObjectState/src/objectstate/context_manager.py", - "sha256": "25b1a7aa1dc6abc96b6ff91dd14e27fc3804c06f8c45a2c7a3ca6a3dd01b35a0", + "sha256": "412adb2bba3cf8f91a9c1e103ab1cebfd5d1d541a7d7787a3032d2ac8ab5cb9b", "role": "Implements generic scoped context resolution." }, { "path": "openhcs/core/config.py", - "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", "role": "Declares GlobalPipelineConfig and the step and standalone configuration dataclasses OpenHCS composes over ObjectState." } ], "findings": [ - "The changes to lazy_factory.py, context_manager.py and config.py (axis-family typing, partition filter, viewer slot modes) do not touch the ownership the redirect states; the page stays a redirect, and the config.py role now names the declarations it actually holds instead of a separate pipeline config class, which the injection system generates." + "The page is a redirect stub; its link target still exists and owns the content (checked 2026-10-11, B6)." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/configuration_framework.rst in full and compared it against the changed authorities: external/ObjectState/src/objectstate/lazy_factory.py, external/ObjectState/src/objectstate/context_manager.py (current files) and openhcs/core/config.py (PartitionFilterMode became a class family, K4 source-binding declarations imported from openhcs.core.source_bindings); the redirect's only claims (ObjectState owns lazy configuration and resolution, OpenHCS owns its domain declarations such as GlobalPipelineConfig) still hold, and the linked ObjectState architecture page and system_overview exist.", "On 2026-10-11 (B6 final), compared docs/source/architecture/configuration_framework.rst against B6's later changes to openhcs/core/config.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/configuration_framework.rst against the later B6 changes to external/ObjectState/src/objectstate/lazy_factory.py, openhcs/core/config.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/configuration_framework.rst in full and compared it against the changed authorities: lazy_factory.py, context_manager.py, openhcs/core/config.py; confirmed ObjectState still owns lazy configuration and context resolution, the linked system_overview and external_foundations pages exist, and no page edit was needed.", @@ -1029,20 +1042,21 @@ "authority": [ { "path": "external/ObjectState/src/objectstate/context_manager.py", - "sha256": "25b1a7aa1dc6abc96b6ff91dd14e27fc3804c06f8c45a2c7a3ca6a3dd01b35a0", + "sha256": "412adb2bba3cf8f91a9c1e103ab1cebfd5d1d541a7d7787a3032d2ac8ab5cb9b", "role": "Implements reusable configuration-context registration and resolution." }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", + "sha256": "0f9cc4819b53d55b7e9446429b8c8ac1ff1a59f4902020ea1658495ce01d464c", "role": "Consumes an already resolved GlobalPipelineConfig in a scoped compiler session." } ], "findings": [ - "The changes to context_manager.py and compilation_session.py do not alter the ownership the redirect states; the compiler session still receives resolved configuration, so the page stays a redirect without edits." + "The page is a redirect stub; its link target still exists and owns the content (checked 2026-10-11, B6)." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/context_system.rst in full and compared it against the changed authorities: external/ObjectState/src/objectstate/context_manager.py (current file) and openhcs/core/pipeline/compilation_session.py (invocation-declaration validation moved onto ArtifactGraph.validate_invocation_declarations, disabled-function stripping removed from step normalization); ObjectState still owns context registration and resolution and the compiler session still consumes a resolved GlobalPipelineConfig, so the redirect stands unchanged.", "On 2026-10-11 (B6 final), compared docs/source/architecture/context_system.rst against B6's later changes to openhcs/core/pipeline/compilation_session.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/context_system.rst against the later B6 changes to openhcs/core/pipeline/compilation_session.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/context_system.rst in full and compared it against the changed authorities: context_manager.py, compilation_session.py; confirmed ObjectState owns context management and the compiler session consumes a resolved GlobalPipelineConfig; no page edit was needed.", @@ -1126,7 +1140,7 @@ "authority": [ { "path": "external/pyqt-reactive/src/pyqt_reactive/services/window_manager.py", - "sha256": "84fff75ff74e59c5d7080f103ad1a4baa2c884aff3d2f87891a0afdae9a5c332", + "sha256": "782472813aebe2b2ad7586db6ffb9809b68ac82a98d41f95ab9f25a22a7dbc21", "role": "Implements live top-level window discovery plus generic registered window lifecycle, reuse, parentage, and cleanup." }, { @@ -1155,11 +1169,10 @@ "role": "Verifies that the stable UI window reference derives its identities and titles from the identity registry." } ], - "findings": [ - "Since U1, the plate-manager and pipeline-editor windows render session views and take their titles from them, and pipeline and execution actions are session operations rather than widget actions; the retained-ownership sentence now says so." - ], - "disposition": "redirect", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/declarative_window_system.rst in full and compared it against the changed authority external/pyqt-reactive/src/pyqt_reactive/services/window_manager.py (current file): WindowManager still provides visible top-level window discovery, register/unregister, show_or_focus reuse by scope, focus_and_navigate, and close cleanup; the linked pyqt-reactive window_manager_usage and scope_window_factory pages exist; no claim changed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/declarative_window_system.rst against B6's later changes to openhcs/authoring/session/views.py: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/declarative_window_system.rst against B6's later changes to openhcs/authoring/session/views.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/declarative_window_system.rst against the later B6 changes to openhcs/authoring/session/views.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -1184,20 +1197,21 @@ "authority": [ { "path": "openhcs/core/function_patterns.py", - "sha256": "95b9056c6980e4f7943f909aeae8262252c9aa175247a534befc8d4cc459fdd2", + "sha256": "cc8c35f37ea0a3bd939905fe317652a34b3129ee6f47f6f9023b8c459e62df70", "role": "Defines callable, sequential-chain, and dictionary-fan-out patterns and their grouped source keys." }, { "path": "tests/unit/test_function_patterns.py", - "sha256": "d0160652d548748901a8b95d21abde9dca050259fd2f74d4a85cb60a50441175", + "sha256": "296c5ab1f5044b04ce631c92a1e8510fe9dd0e21a776d7a30877766154a74b18", "role": "Verifies dictionary fan-out, composition, and invalid pattern rejection." } ], "findings": [ - "The function_patterns.py and test changes (axis-typed grouping, payload-axis proofs) do not affect the redirect; dictionary fan-out still exists and concepts/function_patterns remains its owner page." + "The page is a redirect stub; its link target still exists and owns the content (checked 2026-10-11, B6)." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/dict_pattern_case_study.rst in full and compared it against the changed authorities: openhcs/core/function_patterns.py (invocation-artifact imports and FunctionInvocationKey.from_callable removed, K5 callable-contract patterns) and tests/unit/test_function_patterns.py; dictionary fan-out keyed by group is still part of FunctionPatternSyntax and covered by test_invocation_positions_are_renumbered_per_dict_group, and concepts/function_patterns still exists, so the redirect stands.", "On 2026-10-11 (B6 final), compared docs/source/architecture/dict_pattern_case_study.rst against B6's later changes to openhcs/core/function_patterns.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/dict_pattern_case_study.rst against the later B6 changes to openhcs/core/function_patterns.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/dict_pattern_case_study.rst in full and compared it against the changed authorities: function_patterns.py, test_function_patterns.py; confirmed NormalizedFunctionPattern still carries grouped dictionary keys and the redirect target exists; no page edit was needed.", @@ -1219,7 +1233,7 @@ "authority": [ { "path": "external/ObjectState/src/objectstate/lazy_factory.py", - "sha256": "4f86ec64d975b354203539581943f36a75e34fbf2da076a3ef9b32036e54bff9", + "sha256": "b9bd1547f5f78a6e15399006c085b1459f3172e1d31637233a93200846e07c37", "role": "Implements dynamic lazy dataclass generation, validation, and resolution." }, { @@ -1229,10 +1243,11 @@ } ], "findings": [ - "The lazy_factory.py change does not move lazy-dataclass generation out of ObjectState; the redirect is still accurate and needed no edit." + "The page is a redirect stub; its link target still exists and owns the content (checked 2026-10-11, B6)." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/dynamic_dataclass_factory.rst in full and compared it against the changed authority external/ObjectState/src/objectstate/lazy_factory.py (current file); lazy-dataclass generation and resolution remain in ObjectState and the linked pages exist, so the redirect stands.", "On 2026-10-11 (B6 final), compared docs/source/architecture/dynamic_dataclass_factory.rst against the later B6 changes to external/ObjectState/src/objectstate/lazy_factory.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/dynamic_dataclass_factory.rst in full and compared it against the changed authority external/ObjectState/src/objectstate/lazy_factory.py; confirmed ObjectState still owns lazy dataclass generation and the linked pages exist; no page edit was needed.", "Read the complete page and challenged accuracy, freshness, duplication, boundary, flow, and navigation against the listed current authorities.", @@ -1271,7 +1286,7 @@ }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "01dcb82b5850f3cdb1c300a2aa360ba330e7f15dffb5a24e37504311095090f9", + "sha256": "3cdb7819f09604046ba69e0b4f57b73e4e98d8f68af80158501a54bb42e701a2", "role": "Routes the desktop analysis action through ExperimentalAnalysisEngine.run_directory with the default configuration." }, { @@ -1280,12 +1295,10 @@ "role": "Verifies configured path projection, exclusions, normalization-method delegation, and an end-to-end MetaXpress directory run." } ], - "findings": [ - "openhcs/formats/experimental_result_formats.py was moved (G2) to openhcs/processing/backends/experimental_analysis/result_formats.py, and ExperimentalAnalysisConfig and NormalizationMethod moved from core/config.py into the microscopy domain config; the page now says the config is a domain declaration.", - "The page contrasted the desktop path with a deprecated compatibility wrapper that no longer exists; the clause was removed." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/experimental_analysis_system.rst in full and compared it against the changed authority openhcs/pyqt_gui/main.py (U3 moved dock layout, shortcuts and time-travel navigation to pyqt-reactive); the desktop analysis action still constructs ExperimentalAnalysisEngine() with its default configuration and calls run_directory, and ExperimentalAnalysisConfig and NormalizationMethod are still declared in openhcs/domains/microscopy/config.py; no claim changed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/experimental_analysis_system.rst against B6's later changes to openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py, openhcs/processing/backends/experimental_analysis/result_formats.py, openhcs/processing/backends/experimental_analysis/analysis.py, openhcs/pyqt_gui/main.py, tests/unit/test_experimental_analysis_engine.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/experimental_analysis_system.rst against the later B6 changes to openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py, openhcs/processing/backends/experimental_analysis/result_formats.py, openhcs/processing/backends/experimental_analysis/analysis.py, openhcs/pyqt_gui/main.py, tests/unit/test_experimental_analysis_engine.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/experimental_analysis_system.rst in full and compared it against the changed authorities: core/config.py, unified_analysis_engine.py, the moved formats/experimental_result_formats.py, pyqt_gui/main.py, test_experimental_analysis_engine.py; traced ExperimentalAnalysisConfig and NormalizationMethod to domains/microscopy/config.py, the result-format strategies to result_formats.py and layout parsing to analysis.py, confirmed the desktop action calls ExperimentalAnalysisEngine().run_directory, and removed the stale deprecated-wrapper clause.", @@ -1305,7 +1318,7 @@ }, { "path": "docs/source/architecture/external_foundations.rst", - "source_sha256": "032486c75f7c51f03c8868814044e42624517ff2c8e6f785cf9e88d27066ef4d", + "source_sha256": "25610fe738d81d8911b99aa902c7e000945141d80424cb3f67b47dbdd93ca6fc", "audience": [ "OpenHCS maintainers", "first-party package integrators" @@ -1315,7 +1328,7 @@ "authority": [ { "path": "pyproject.toml", - "sha256": "c3a2bea22aa6872940125f4aaac72c9eac72084fefc6c2cefdb54b5273361e4a", + "sha256": "b5d4d752704b886da20d860743b81dc3d2ce3a3c402208246fadc1c6cb71f597", "role": "Pins the eight first-party package constraints and documents their installed integration surface." }, { @@ -1330,7 +1343,7 @@ }, { "path": "external/ObjectState/README.md", - "sha256": "e13c149f1130f43eab46cca59986ca0e2de5ce29f5b7d48f8e8aebb9fc9e127a", + "sha256": "e3c31ab46f9e4a754b9bfebe496a5c84a06461fe10c69a203fc04cff49dc56b5", "role": "States ObjectState ownership of lazy state, contexts, provenance, and editing mechanics." }, { @@ -1390,7 +1403,7 @@ }, { "path": "external/python-introspect/src/python_introspect/jsonable.py", - "sha256": "8fe06533a6dce179b7d77792b6ac768c2a64a1e9e79bbe926893498fae035bb4", + "sha256": "0148e51fb2fc4c7e1c625b088cb0ca56fecd58f9836b9e5da58b75b5eb2d40ee", "role": "Owns the to_jsonable encoder paired with the dataclass_from_mapping decoder." }, { @@ -1417,16 +1430,41 @@ "path": "external/pycodify/README.md", "sha256": "f32db990f5e1feb01ee3d7adbe1195fd691c8964acd789759901d00170cecf7e", "role": "States pycodify ownership of Python-source serialisation and formatter composition." + }, + { + "path": "external/ObjectState/src/objectstate/scope_key.py", + "sha256": "d0dd6e56cebf413d6869b07c7b9d8668ba954ed5aeead36c70288920a5b412b2", + "role": "Owns the ScopeKey scope grammar (child, parent, root, ancestors, containment) moved into ObjectState by U3." + }, + { + "path": "external/ObjectState/src/objectstate/codegen.py", + "sha256": "347d2697c061a35e6e440e373fe4e8bfce1ad92b500e98da07faa26f6a3a1356", + "role": "Owns LazyResolutionFormatter and field_needs_source, the pycodify formatting of lazy configurations moved out of OpenHCS by L6." + }, + { + "path": "external/pycodify/src/pycodify/core.py", + "sha256": "f5ae16c3bc62d2b2dc24f2701fb7fa936c6dd64fc602d3e64aabc8a38730ef18", + "role": "Owns imported_name, PythonSourceLiteral and PathRendering, the qualified-name, value-owned literal and path-factoring rules." + }, + { + "path": "external/python-introspect/src/python_introspect/field_label.py", + "sha256": "8c2d9d94dd235181aee2097764cc7199948cde2b4de74e2a9bd5f58f062452b4", + "role": "Owns the FieldLabel annotation marker moved out of openhcs/core/field_label.py." + }, + { + "path": "external/pyqt-reactive/src/pyqt_reactive/widgets/shared/dock_layout.py", + "sha256": "6643f7eb404ffc6eba0f7b57cf7d5ef1631efd089d56d9c905f2cf8a743ef13a", + "role": "Owns DockPane and DockLayoutStore, part of the generic Qt machinery (with editable_table, grid_selector, time_travel and config_window) that U3 moved into pyqt-reactive." } ], "findings": [ - "L1 moved the registry strategy/selection mixins and the cache family into metaclass-registry; the page still said OpenHCS owned the shared selection mixins.", - "L2 moved the to_jsonable encoder and the public-API/lazy-export helpers into python-introspect; the catalogue did not list them.", - "L3 moved the background-process launch policy into ZMQRuntime, and L5 added ArrayBridge's per-memory-type array operations; neither was in the catalogue.", - "The pyqt-reactive window_snapshot.rst authority was a documentation file; it was dropped in favour of the source owner already listed." + "U3 moved the ScopeKey scope grammar and field inspection into ObjectState and the editable table, grid selector, time-travel widgets and navigation, configuration window, and dock and shortcut layout into pyqt-reactive; the catalogue listed none of them.", + "L6 moved lazy-configuration source formatting into objectstate.codegen and the qualified-name, value-owned literal and path-factoring rules into pycodify; the ObjectState and pycodify entries did not say so.", + "U3 moved FieldLabel into python-introspect; its entry did not list field labels." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/external_foundations.rst in full and compared it against the changed authorities: pyproject.toml (pycodify 0.2, objectstate 2.1, pyqt-reactive 0.5, zmqruntime and python-introspect floors; still eight first-party packages), external/ObjectState/README.md (objectstate.codegen) and external/python-introspect/src/python_introspect/jsonable.py (to_jsonable unchanged), plus the U3 and L6 surface notes; added the moved ScopeKey, inspection, codegen, field-label, Qt machinery and pycodify rules to the catalogue and confirmed OpenHCS still keeps its domain formatters in openhcs/serialization/pycodify_formatters.py.", "On 2026-10-11 (B6 final), compared docs/source/architecture/external_foundations.rst against the later B6 changes to pyproject.toml: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/external_foundations.rst in full and compared it against the changed authorities: pyproject.toml (zmqruntime 0.6, polystore 0.6, objectstate 1.3.1, python-introspect 0.2.5, metaclass-registry 0.3.0, arraybridge 0.4.1, pyqt-reactive 0.4 bounds; still eight first-party packages), the package READMEs, window_snapshot.py, client.py and arraybridge types.py; added the L1, L2, L3 and L5 ownership moves to the catalogue and confirmed the PolyStore, OMERO and pyqt-reactive claims against current source.", "Re-read docs/source/architecture/external_foundations.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -1477,7 +1515,7 @@ "authority": [ { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "3439d96f3fbfcb104a53019dc150049021d9a8132bd27096f1ccf510eefbe37b", + "sha256": "b96f33b866fe6c9f8118ca70fcb7dff8e6a02cb29d79df67230a5e8b97df8002", "role": "Implements CellProfiler lowering to FunctionStep and PipelineConfig declarations with declaration-owned headless validation." }, { @@ -1492,17 +1530,17 @@ }, { "path": "openhcs/interop/cellprofiler/module_declarations.py", - "sha256": "6bbf41c158496a02439be2ee5aef12d7712fb632419c226a64c9259a55438b95", + "sha256": "b0195b66126c31f71dc0d4bb281a128e0164bb7dfb3b314ee8fe744ebc55d626", "role": "Defines the nominal module validation hook queried by the importer." }, { "path": "openhcs/interop/cellprofiler/module_artifact_declarations.py", - "sha256": "f70b52114fb5d5e2c2fc49cd9a2b46986bde48ce02d82f1a34a67f072ca2ab20", + "sha256": "ef233be43002d0f16b53c78c202b9203b6261847708d311089671a6b037db894", "role": "Defines the interactive-module family that rejects headless import." }, { "path": "openhcs/core/dataset_sources/source.py", - "sha256": "097fde4d2b74031eacf53a88e79f499417e8c1c97ba0753e8a9df831b307efa6", + "sha256": "350bf5691a03522e9f54b1d8ab712c6beba1683aac4fa5159ea4e80c6041530d", "role": "Defines the registered DatasetSource family that microscope handlers implement." }, { @@ -1532,16 +1570,14 @@ }, { "path": "tests/unit/test_cellprofiler_pipeline_import.py", - "sha256": "6b1d201ec5c4ba620fa2367a9cbef1f6fbdd25edbdda9bd7b723ebe3a3363412", + "sha256": "0ce4efd85d819c7e9f7fa84ec6f1ac7ec4bda024c9a8d5c7f264fda2730264f0", "role": "Verifies explicit rejection of an interactive-only module during direct import." } ], - "findings": [ - "The page named ``MicroscopeHandler`` implementations; G4 deleted microscope_base.py and handlers now implement the kernel DatasetSource family.", - "The .cppipe importer is now a member of the kernel's PipelineImporter family registered from interop; the page did not say how the generic code reaches it." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/external_integrations_overview.rst in full and compared it against the changed authorities: openhcs/interop/cellprofiler/pipeline_import.py, module_declarations.py and module_artifact_declarations.py (K4 SourceBindingsConfig/StepSourceBindingsConfig now imported from openhcs.core.config; K5 contract fields read directly instead of through metadata), openhcs/core/dataset_sources/source.py (import move) and tests/unit/test_cellprofiler_pipeline_import.py (PairByOrder); the CppipeImporter is still a PipelineImporter family member, lowering still produces PipelineConfig plus FunctionStep declarations, interactive-only modules still fail, and DatasetSource remains the source family; no claim changed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/external_integrations_overview.rst against B6's later changes to openhcs/interop/cellprofiler/pipeline_import.py, openhcs/interop/cellprofiler/module_declarations.py, openhcs/interop/cellprofiler/module_artifact_declarations.py, openhcs/core/dataset_sources/source.py, openhcs/runtime/zmq_server_hooks.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/external_integrations_overview.rst against the later B6 changes to openhcs/interop/cellprofiler/pipeline_import.py, openhcs/interop/cellprofiler/module_declarations.py, openhcs/interop/cellprofiler/module_artifact_declarations.py, openhcs/core/dataset_sources/source.py, openhcs/runtime/zmq_server_hooks.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/external_integrations_overview.rst in full and compared it against the changed authorities: pipeline_import.py, module_declarations.py, module_artifact_declarations.py, the deleted microscope_base.py, polystore/base.py, zmqruntime execution/server.py, zmq_server_hooks.py, test_cellprofiler_pipeline_import.py; replaced MicroscopeHandler with the DatasetSource family, named the PipelineImporter registration, and confirmed the CellProfiler, OMERO and cancellation claims and every linked page.", @@ -1632,7 +1668,7 @@ "authority": [ { "path": "external/pyqt-reactive/src/pyqt_reactive/animation/flash_mixin.py", - "sha256": "5b44c084784499a03c8eccffe37dbf31efc0e3379a6c99907272507f9100a492", + "sha256": "53e77aa4d0a40285dc33b1a7a533e0cdef9985342eebc1fca07ebe05b9e51c3c", "role": "Implements reusable flash propagation, widget-geometry masking, label padding derived from animation config, and paint-acknowledged animation coordination." }, { @@ -1647,10 +1683,11 @@ } ], "findings": [ - "The pyqt-reactive flash changes since the last review (pending registrations dropped on widget destruction, label padding derived from animation config) stay inside the generic owner; the redirect's owner link and OpenHCS boundary links remain correct, so only evidence roles changed." + "The page is a redirect stub; its link target still exists and owns the content (checked 2026-10-11, B6)." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/flash_animation_system.rst in full and compared it against the changed authority external/pyqt-reactive/src/pyqt_reactive/animation/flash_mixin.py (current file); the flash mechanism stays in pyqt-reactive, its architecture/flash_animation_system page exists, and the OpenHCS boundary links still resolve, so the redirect stands.", "On 2026-10-11 (B6), re-read docs/source/architecture/flash_animation_system.rst in full and compared it against the changed authorities: external/pyqt-reactive/src/pyqt_reactive/animation/flash_mixin.py and external/pyqt-reactive/tests/test_flash_destruction_lifetime.py; confirmed the pyqt-reactive flash page and the ui_services_architecture and external_foundations targets still exist and updated the evidence roles; the page text needed no change.", "Read the complete page and challenged accuracy, freshness, duplication, boundary, flow, and navigation against the listed current authorities.", "python scripts/validate_docs.py docs/source/architecture passed, and .venv Sphinx -E -W --keep-going rendered all 152 active pages without warnings.", @@ -1670,20 +1707,21 @@ "authority": [ { "path": "openhcs/core/function_patterns.py", - "sha256": "95b9056c6980e4f7943f909aeae8262252c9aa175247a534befc8d4cc459fdd2", + "sha256": "cc8c35f37ea0a3bd939905fe317652a34b3129ee6f47f6f9023b8c459e62df70", "role": "Normalizes callable, (callable, kwargs), list-chain, and dict patterns into named invocations and compiles them with their memory and artifact plans." }, { "path": "tests/unit/test_function_patterns.py", - "sha256": "d0160652d548748901a8b95d21abde9dca050259fd2f74d4a85cb60a50441175", + "sha256": "296c5ab1f5044b04ce631c92a1e8510fe9dd0e21a776d7a30877766154a74b18", "role": "Verifies invocation identity for list and dict patterns, normalization, tuple-leaf validation, and compiled pattern semantics." } ], "findings": [ - "The redirect still points at the current concept page; function_patterns.py changed only in compiled-invocation detail. Evidence roles updated to the current module's normalized and compiled pattern classes." + "The page is a redirect stub; its link target still exists and owns the content (checked 2026-10-11, B6)." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/function_pattern_system.rst in full and compared it against the changed authorities: openhcs/core/function_patterns.py (K5: invocation-artifact imports and FunctionInvocationKey.from_callable removed, compile_function_pattern takes an admitted artifact graph) and tests/unit/test_function_patterns.py; normalize_function_pattern and compile_function_pattern still own callable, chain and dict pattern semantics and concepts/function_patterns exists, so the redirect stands.", "On 2026-10-11 (B6 final), compared docs/source/architecture/function_pattern_system.rst against B6's later changes to openhcs/core/function_patterns.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/function_pattern_system.rst against the later B6 changes to openhcs/core/function_patterns.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/function_pattern_system.rst in full and compared it against the changed authorities: openhcs/core/function_patterns.py and tests/unit/test_function_patterns.py; confirmed the module still accepts callable, (callable, kwargs), list and dict patterns and that docs/source/concepts/function_patterns.rst exists; the page text needed no change.", @@ -1715,20 +1753,21 @@ }, { "path": "openhcs/core/callable_contract.py", - "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", - "role": "Adds OpenHCS processing, runtime, artifact, scope, and ArrayBridge memory semantics on top of python-introspect annotation helpers." + "sha256": "a8e73c5df804d78baf72710e210cb2ae63156657a074071e11fd868f4b19d3b4", + "role": "Defines CallableDeclaration and CallableContract, which add OpenHCS processing, runtime, artifact, scope, and ArrayBridge memory semantics on top of python-introspect annotation helpers." }, { "path": "tests/unit/test_callable_contract.py", - "sha256": "cc220f53107a08b0406b6e0dd49896e3e4c3c0af6e527b0ef00d4fdaba8f0b2c", - "role": "Verifies OpenHCS callable contracts projected from declaration-owned metadata, including execution memory and runtime batching." + "sha256": "9170f8694f388dbed5bfd36d882a5c0a4cdf28f50f51f7fdeaf4a955a9f3b412", + "role": "Verifies OpenHCS callable contracts projected from declared callable fields, including execution memory and runtime batching." } ], "findings": [ - "Re-checked the redirect against the rewritten callable_contract.py: generic signature analysis and declaration projection still come from python-introspect (SignatureAnalyzer, callable_declaration_kwargs, annotation helpers), and OpenHCS-specific callable semantics stay in callable_contract. No page text was wrong; evidence roles were refreshed." + "The page is a redirect stub; its link target still exists and owns the content (checked 2026-10-11, B6)." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/function_reference_pattern.rst in full and compared it against the changed authorities: openhcs/core/callable_contract.py (K5: CallableMetadata replaced by CallableDeclaration, which CallableContract now subclasses) and tests/unit/test_callable_contract.py; generic signature analysis and callable_declaration_kwargs still come from python-introspect and OpenHCS callable semantics stay in callable_contract, so the redirect stands; refreshed the callable_contract and test evidence roles.", "On 2026-10-11 (B6 final), compared docs/source/architecture/function_reference_pattern.rst against B6's later changes to openhcs/core/callable_contract.py, tests/unit/test_callable_contract.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/function_reference_pattern.rst against the later B6 changes to openhcs/core/callable_contract.py, tests/unit/test_callable_contract.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/function_reference_pattern.rst in full and compared it against the changed authorities: external/python-introspect/src/python_introspect/signature_analyzer.py, openhcs/core/callable_contract.py and tests/unit/test_callable_contract.py; grepped OpenHCS imports of python_introspect and confirmed docs/source/architecture/processing_semantics.rst exists; the page text needed no change.", @@ -1799,7 +1838,7 @@ }, { "path": "openhcs/core/callable_contract.py", - "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", + "sha256": "a8e73c5df804d78baf72710e210cb2ae63156657a074071e11fd868f4b19d3b4", "role": "Projects declaration-owned input, output, and execution memory into immutable callable contracts." }, { @@ -1814,17 +1853,17 @@ }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "sha256": "4a33908fb42874e6d57eeea3f5305793ddbda41d13356f0a78501af38dcdfeb4", "role": "Calls assign_framework_devices on each context's step plans before bundle assembly." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", + "sha256": "358d3eaf5952170f9e9e2dc9056fb08707e8fcf74682063c074b367a83c2648f", "role": "Owns FrameworkDeviceAssignment bindings, merged cleanup footprints, and the requires_gpu projection." }, { "path": "openhcs/core/orchestrator/worker_execution.py", - "sha256": "23a3ef2a70378dd1bb5c413a56e36bbb3fa013142741c9b45407bfc47de8c4a4", + "sha256": "26b9586890e7117b24b6b85660d3f417455d55546401739f91146d18e4d21134", "role": "Releases each axis's compiled footprint after process and inline contexts and the merged footprint only after thread-backed lanes join." }, { @@ -1844,7 +1883,7 @@ }, { "path": "tests/unit/test_framework_execution_scope.py", - "sha256": "ff92362a71f31b4b893a3816b1fdd396ed30307f45c7084270b4929c1aaa7aa4", + "sha256": "b61d2df837d570f99031b29cadf75d2ac7381617cdd845b2fb67914eb5015a3f", "role": "Verifies execution-memory device scopes and exact per-lane allocator cleanup." }, { @@ -1853,12 +1892,10 @@ "role": "Verifies GPU-sensitive worker start policy and explicit unsafe overrides." } ], - "findings": [ - "The page described parent/child framework import consistency with the jargon word 'admission'; reworded to plain terms.", - "Re-checked every ownership claim against the consolidation changes to callable_contract, compiler, compiled_step_plan and worker_execution: device resolution, per-step bindings, CPU-only rejection, subprocess_environment use by catalogue preparation (registry_service) and the execution-server client, worker-start policy and per-lane cleanup all remain as described. Added openhcs/utils/environment.py as the owner of the CPU-only switch." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/gpu_resource_management.rst in full and compared it against the changed authorities: openhcs/core/callable_contract.py (K5: input, output and execution memory are now declared CallableDeclaration fields read from the same ArrayBridge MemoryContractAttribute keys; declared_memory_types unchanged), openhcs/core/pipeline/compiler.py (ScopeKey scope ids, progress classes), openhcs/core/compiled_step_plan.py (source plan default constructors), openhcs/core/orchestrator/worker_execution.py (progress classes) and tests/unit/test_framework_execution_scope.py (admitted_graph); FrameworkDeviceResolver, WorkerStartExecutionFacts, MemoryType.subprocess_environment and per-lane cleanup are unchanged; no claim changed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/gpu_resource_management.rst against B6's later changes to openhcs/core/callable_contract.py, openhcs/core/pipeline/compiler.py, openhcs/core/compiled_step_plan.py, openhcs/core/orchestrator/worker_execution.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/gpu_resource_management.rst against the later B6 changes to openhcs/core/callable_contract.py, openhcs/core/pipeline/compiler.py, openhcs/core/compiled_step_plan.py, openhcs/core/orchestrator/worker_execution.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/gpu_resource_management.rst in full and compared it against the changed authorities: arraybridge decorators.py and types.py, openhcs/core/callable_contract.py, openhcs/core/pipeline/framework_device_assignment.py, openhcs/core/pipeline/compiler.py, openhcs/core/compiled_step_plan.py, openhcs/core/orchestrator/worker_execution.py and the worker-start and execution-scope tests; traced assign_framework_devices from the compiler, FrameworkDeviceResolver's gpu_imports_disabled check, MemoryType.subprocess_environment() callers, _release_runtime_resources per axis and after threaded lanes, and resolve_worker_start_context; replaced one jargon phrase and refreshed evidence roles.", @@ -1924,22 +1961,22 @@ }, { "path": "openhcs/core/config.py", - "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", "role": "Declares GlobalPipelineConfig and the step variable_components and group_by fields, with group_by defaulting from the active AxisFamily." }, { "path": "openhcs/core/processing_contracts.py", - "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", + "sha256": "eb0f7fd99ded1c4d5957152ccf6abe1468436c13574a1d1bb83ce368d24b9f69", "role": "Defines the ProcessingContract family named as an independent declaration in the invariants." }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", + "sha256": "0f9cc4819b53d55b7e9446429b8c8ac1ff1a59f4902020ea1658495ce01d464c", "role": "Defines CompilationSession, the scoped compiler state named by the invariants." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", + "sha256": "358d3eaf5952170f9e9e2dc9056fb08707e8fcf74682063c074b367a83c2648f", "role": "Defines typed CompiledStepPlan fields, including framework footprints and device bindings." }, { @@ -1951,6 +1988,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/index.rst in full and compared it against the changed authorities: openhcs/core/config.py (PartitionFilterMode family, K4 source-binding declarations), openhcs/core/processing_contracts.py (CallableMetadata renamed CallableDeclaration), openhcs/core/pipeline/compilation_session.py (graph-owned invocation validation) and openhcs/core/compiled_step_plan.py (source plan default constructors); the invariants (PipelineConfig plus FunctionStep, CompilationSession and typed CompiledStepPlan, separate variable_components, group_by and ProcessingContract declarations, CompiledExecutionBundle for workers) still hold; no claim changed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/index.rst against B6's later changes to openhcs/core/config.py, openhcs/core/processing_contracts.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/compiled_step_plan.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/index.rst against the later B6 changes to openhcs/core/config.py, openhcs/core/processing_contracts.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/compiled_step_plan.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/index.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compilation_session.py, openhcs/core/compiled_step_plan.py and openhcs/core/compiled_execution.py; verified each architecture invariant against the current declarations and that every toctree target file exists; the page text needed no change.", @@ -1963,7 +2001,7 @@ }, { "path": "docs/source/architecture/list_item_preview_system.rst", - "source_sha256": "fb9177bbbabb4168c1284df0209497cfb23fa9aaf981a248be6e0a15d33134c8", + "source_sha256": "a1ddaf8e8135bea0564aa5e85713f6abe254447dacb24bf4dc586858f48e0782", "audience": [ "OpenHCS UI maintainers following the extracted list-preview URL" ], @@ -1982,7 +2020,7 @@ }, { "path": "openhcs/core/config.py", - "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", "role": "Declares PartitionFilterConfig's show_preview_without_partition_filter policy and PartitionFilterMode members' preview_prefix operators." }, { @@ -1992,7 +2030,7 @@ }, { "path": "openhcs/pyqt_gui/services/reactor_providers.py", - "sha256": "aecf1c77636095408486cab53a95cb0fc044fcf2ead0d7b72ac674842cf69abe", + "sha256": "04ad362d7bb31641f5a4ba0bfcc8670cd2e330a6eadc231a0ca5df7abed8d2b4", "role": "Registers format_partition_filter_config for PartitionFilterConfig with the generic preview service." }, { @@ -2002,7 +2040,7 @@ }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "01dcb82b5850f3cdb1c300a2aa360ba330e7f15dffb5a24e37504311095090f9", + "sha256": "3cdb7819f09604046ba69e0b4f57b73e4e98d8f68af80158501a54bb42e701a2", "role": "_apply_list_preview_config applies list_previews to Plate Manager and Pipeline Editor at startup and on UI config change." }, { @@ -2042,11 +2080,11 @@ } ], "findings": [ - "After the axis-family rename (WellFilterConfig to PartitionFilterConfig), the page still said an enabled member stays visible 'without an explicit well selection'; the declaration is show_preview_without_partition_filter. Corrected to 'partition filter'.", - "Evidence roles named the deleted WellFilterConfig/WellFilterMode; updated to the partition-filter declarations and the current formatter name." + "PartitionFilterMode became a class family; the page's 'members' wording was already corrected to 'subclasses' in the page change under review, which this review confirms against openhcs/core/config.py." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/list_item_preview_system.rst in full and compared it against the page change (PartitionFilterMode members to subclasses) and the changed authorities: openhcs/core/config.py (PartitionFilterMode subclasses declare preview_prefix), openhcs/pyqt_gui/services/reactor_providers.py (still registers the one PartitionFilterConfig preview formatter; code documents now use pycodify generate_python_source) and openhcs/pyqt_gui/main.py (U3 moves; still applies UIConfig.list_previews to each manager via set_preview_config); every claim holds.", "On 2026-10-11 (B6 final), compared docs/source/architecture/list_item_preview_system.rst against B6's later changes to openhcs/core/config.py, openhcs/pyqt_gui/services/reactor_providers.py, openhcs/pyqt_gui/main.py, tests/unit/pyqt_gui/test_main_config_propagation.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/list_item_preview_system.rst against the later B6 changes to openhcs/core/config.py, openhcs/pyqt_gui/services/reactor_providers.py, openhcs/pyqt_gui/main.py, tests/unit/pyqt_gui/test_main_config_propagation.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/list_item_preview_system.rst in full and compared it against the changed authorities: openhcs/core/config.py, openhcs/pyqt_gui/widgets/config_preview_formatters.py, openhcs/pyqt_gui/services/reactor_providers.py, openhcs/pyqt_gui/main.py and tests/unit/pyqt_gui/test_main_config_propagation.py; confirmed PartitionFilterConfig/PartitionFilterMode preview declarations, the single formatter registration, UIConfig.list_previews, _apply_list_preview_config for both managers, and TogglePreviewAction; corrected one stale well-selection phrase.", @@ -2123,7 +2161,7 @@ }, { "path": "openhcs/authoring/session/session.py", - "sha256": "8d74425f65ce9f39f88439ca90bd09c02678ae3aef8d3cf6795570986444add0", + "sha256": "0c4e0f0979e1793862859e47eefeb6dd9d4b51c99fba20c2ebc6adf11619c3b5", "role": "Defines the headless Session whose datasets and pipelines live in the MCP process's ObjectState and whose events clients follow." }, { @@ -2133,7 +2171,7 @@ }, { "path": "openhcs/core/config_document.py", - "sha256": "37f66fd971806e059da6194c003e20a7ad08f69066eec4af7ebb11b4b70a08db", + "sha256": "9695a6f300d626ceddaf4e0cd18961fd5600b63becf9a133cfa7f0b654b32cfb", "role": "Derives one typed nested field from a configuration Python document and its dataclass schema." }, { @@ -2183,7 +2221,7 @@ }, { "path": "openhcs/pyqt_gui/services/ui_bridge_windows.py", - "sha256": "e76298f78e3cbd949e8426a2b42cd6768cc54770a5f45c9dd6c9f788b257785e", + "sha256": "29265ed04137392757a6d515020920120e29fd556f433575956ae38cedff0158", "role": "Resolves authorised OpenHCS windows, projects snapshot results, and distinguishes focus from accepted target navigation." }, { @@ -2237,17 +2275,10 @@ "role": "Owns the installed openhcs-benchmark execution, report and list-cases command surface." } ], - "findings": [ - "The local-lane section said the MCP server does not import PyQt; the stdio transport now keeps a QCoreApplication on the main thread and runs the SDK loop on a worker thread (McpTransportExecutor). Corrected, keeping the true claim that it hosts no GUI.", - "The liveness section said source-backed orchestrator-session creation runs on the main thread and that the event loop cannot emit progress meanwhile; that capability is gone (U1). Non-worker-safe capabilities now run through the main-thread dispatcher while the off-main SDK loop keeps emitting liveness. Rewrote the paragraph.", - "execute-source no longer creates a session and submits it for status polling; it adds the dataset to the headless session, sets its pipeline, and initializes, compiles and runs it, following openhcs_session_events (--no-wait returns on acceptance). Rewrote the session/execution paragraph around the per-operation session tools.", - "DesktopDeploymentAuthority was renamed DesktopDeployment and moved to openhcs.desktop.deployment (73a2aece2); corrected the name and replaced the three deleted desktop authorities.", - "Client registration now also synchronises the OpenHCS skill bundle into clients with a documented skill directory (Codex), reported separately; added this and narrowed 'agent instructions' to 'server instructions'.", - "The headless-session sentence implied no ObjectState state; the session keeps its datasets and pipelines in the MCP process's own ObjectState. Clarified that it creates no rows or history in a running desktop application.", - "Measured-run finalisation consumes the session's finished execution; corrected. Removed 'canonical' filler (three places)." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/mcp_distribution.rst in full and compared it against the changed authorities: authoring/session/session.py (K6 debug command/replay classes), core/config_document.py (L6: rendering through pycodify.generate_python_source; project_dataclass_field_from_source unchanged) and pyqt_gui/services/ui_bridge_windows.py (U3: dirty/signature markers from objectstate.inspection, DockPane from pyqt-reactive); none changes the headless-session, saved-endpoint projection, window-capture, navigation or row-action claims, so no prose change was needed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/mcp_distribution.rst against B6's later changes to openhcs/authoring/session/session.py: the writability probe makes one exclusive create attempt, the installed demo reports ViewerType.value, the unit session pre-creates its QApplication, and the GUI journey waits for each batch to end. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/mcp_distribution.rst against B6's later changes to openhcs/mcp/execution.py: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/mcp_distribution.rst against B6's later changes to openhcs/authoring/session/session.py, benchmark/cellprofiler_benchmark_cli.py: the session decides in-place initialization by an actual file-creation attempt that leaves nothing behind, its debug-session methods moved into an inherited mixin, the measured CLI treats Windows Ctrl+Break like Ctrl+C, and the Official30 viewer test reads typed payload summaries. No claim this page makes changes.", @@ -2481,12 +2512,12 @@ }, { "path": "openhcs/core/callable_contract.py", - "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", - "role": "Reads ArrayBridge metadata through MemoryContractAttribute into immutable callable input, output, and execution memory contracts." + "sha256": "a8e73c5df804d78baf72710e210cb2ae63156657a074071e11fd868f4b19d3b4", + "role": "CallableDeclaration declares the input, output and execution memory types under ArrayBridge's MemoryContractAttribute keys; its namespace codec is derived from those field declarations." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", + "sha256": "358d3eaf5952170f9e9e2dc9056fb08707e8fcf74682063c074b367a83c2648f", "role": "Owns FrameworkDeviceAssignment framework-local device bindings and footprint cleanup." }, { @@ -2496,10 +2527,11 @@ } ], "findings": [ - "Re-checked the redirect against the consolidation changes to callable_contract.py and compiled_step_plan.py and the current ArrayBridge declarations; ArrayBridge still owns memory types, conversion and the input/output/execution metadata keys, and OpenHCS still owns contract projection, compiled framework-device plans and runtime adapters. No page text was wrong; evidence roles were refreshed." + "K5 replaced callable_contract.py's hand-written MemoryContractAttribute read/write code with field declarations on CallableDeclaration; the callable_contract.py evidence role was updated. The page's ownership split (ArrayBridge: memory types, conversion, metadata keys; OpenHCS: contract projection, compiled framework-device plans, runtime adapters) is still accurate, so no prose change was needed." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/memory_type_system.rst in full and compared it against the changed authorities: openhcs/core/callable_contract.py (K5 declared-field CallableDeclaration) and openhcs/core/compiled_step_plan.py (CompiledSourceBindingPlan defaults); confirmed CallableDeclaration still keys memory types by arraybridge MemoryContractAttribute, FrameworkDeviceAssignment still owns footprint cleanup, and runtime_adapters.py, gpu_resource_management and external_foundations exist; updated one authority role.", "On 2026-10-11 (B6 final), compared docs/source/architecture/memory_type_system.rst against B6's later changes to openhcs/core/callable_contract.py, openhcs/core/compiled_step_plan.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/memory_type_system.rst against the later B6 changes to openhcs/core/callable_contract.py, openhcs/core/compiled_step_plan.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/memory_type_system.rst in full and compared it against the changed authorities: external/arraybridge/src/arraybridge/decorators.py, types.py, openhcs/core/callable_contract.py and openhcs/core/compiled_step_plan.py; confirmed MemoryContractAttribute's INPUT/OUTPUT/EXECUTION keys, FrameworkDeviceAssignment, openhcs/core/runtime_adapters.py and the gpu_resource_management and external_foundations targets; the page text needed no change.", @@ -2528,12 +2560,12 @@ "authority": [ { "path": "openhcs/core/dataset_sources/source.py", - "sha256": "097fde4d2b74031eacf53a88e79f499417e8c1c97ba0753e8a9df831b307efa6", + "sha256": "350bf5691a03522e9f54b1d8ab712c6beba1683aac4fa5159ea4e80c6041530d", "role": "Defines the DatasetSource family keyed by source_name, its selection-role mixins, detection_order, create hook, workspace backend registration, and virtual-workspace metadata recording." }, { "path": "openhcs/core/dataset_sources/choice.py", - "sha256": "7fb3c7ce701f0d629f288b79090044f43ac8ce1b51ca4f50937534407f226f1a", + "sha256": "497eb028aabb80283be8a621ea94e09881dfcd51b2e34e2bbecafa09e004c1f6", "role": "Defines DatasetSourceChoice, AutoDetectedSource, and open(), which selects the source class, applies declared-file selection, calls create, and binds dataset_root." }, { @@ -2548,17 +2580,17 @@ }, { "path": "openhcs/core/dataset_sources/openhcs_format.py", - "sha256": "a8fa6a597104075d888a5b14cc4b24a9676d1a3918c30b5df2bad4a9abb26078", + "sha256": "0c1241f2d729e724e076f315acd042afa9c4e9022cc5046c3255753c408f3c2a", "role": "Reopens a persisted workspace by looking up the recorded source_name and re-registering that source's workspace backends." }, { "path": "openhcs/core/config.py", - "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", - "role": "Declares PipelineConfig.dataset_source as a DatasetSourceChoice class defaulting to AutoDetectedSource." + "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", + "role": "Declares PipelineConfig.dataset_source as a DatasetSourceChoice class defaulting to AutoDetectedSource, and the SourceBindingsConfig declaration the source-binding source consumes." }, { "path": "openhcs/microscopes/bioformats.py", - "sha256": "bda05411746852ea12ffe11b49e5298cb475e2137cd1a09cf9659b52a89f40e4", + "sha256": "3b4ba677bebe178d0613f65ffdcadee349e25af82837fd29a1b33898f493d966", "role": "Shows a decoder-owned source metadata handler replaced by the workspace metadata handler after the workspace is written." }, { @@ -2572,14 +2604,10 @@ "role": "Guards that the kernel imports no domain package and that the deleted microscope-handler names do not return." } ], - "findings": [ - "The page documented MicroscopeHandler, create_microscope_handler, MicroscopeHandler.__registry__, the Microscope enum with AUTO, and a factory binding plate_folder; G4 (c6d9a52ce) replaced these with the DatasetSource family keyed by source_name, PipelineConfig.dataset_source holding a DatasetSourceChoice (AutoDetectedSource or a registered source), selection-role mixins, and DatasetSourceChoice.open binding dataset_root. Rewrote the page in the current vocabulary and retitled it 'Dataset source integration' (URL unchanged).", - "Discovery is no longer plain package discovery of openhcs/microscopes: the kernel discovers its own members and imports the active AxisFamily's extension_modules. Added this and placed the extension boundary on those modules.", - "Removed 'canonical' and 'original' filler wording.", - "Four listed authorities were deleted by G4 (microscope_base.py, microscopes/openhcs.py, microscope_interfaces.py, test_microscope_handler_identity.py); replaced with the openhcs/core/dataset_sources owners and the family and guard tests." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/microscope_handler_integration.rst in full and compared it against the changed authorities: dataset_sources/source.py, choice.py, openhcs_format.py and microscopes/bioformats.py (K4: SourceBindingsConfig and source_bindings_defaults_to_base now imported from core/config.py) and core/config.py (K4: SourceBindingsConfig declared there; PartitionFilterMode became a class family); the page names SourceBindingsConfig without a module path, and DatasetSourceChoice.choices/open, detection_order, extension_modules discovery and PipelineConfig.dataset_source are unchanged, so no prose change was needed; the config.py evidence role now also covers SourceBindingsConfig.", "On 2026-10-11 (B6 final), compared docs/source/architecture/microscope_handler_integration.rst against B6's later changes to openhcs/core/dataset_sources/source.py, openhcs/core/dataset_sources/choice.py, openhcs/core/dataset_sources/interfaces.py, openhcs/core/dataset_sources/openhcs_format.py, openhcs/core/config.py, openhcs/microscopes/bioformats.py, tests/unit/test_dataset_source_family.py, tests/unit/test_dataset_source_guards.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/microscope_handler_integration.rst against the later B6 changes to openhcs/core/dataset_sources/source.py, openhcs/core/dataset_sources/choice.py, openhcs/core/dataset_sources/interfaces.py, openhcs/core/dataset_sources/openhcs_format.py, openhcs/core/config.py, openhcs/microscopes/bioformats.py, tests/unit/test_dataset_source_family.py, tests/unit/test_dataset_source_guards.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/microscope_handler_integration.rst in full and compared it against the changed authorities: the deleted openhcs/microscopes/microscope_base.py, openhcs.py and microscope_interfaces.py (moved to openhcs/core/dataset_sources in c6d9a52ce), openhcs/core/source_bindings.py and the Bio-Formats and plate-inspection tests; traced DatasetSource registration and discovery, DatasetSourceChoice.open, detection_order, PipelineConfig.dataset_source, component_value_set, and workspace replay through the recorded source name; rewrote the page in DatasetSource vocabulary.", @@ -2606,12 +2634,12 @@ "authority": [ { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "sha256": "4a33908fb42874e6d57eeea3f5305793ddbda41d13356f0a78501af38dcdfeb4", "role": "Discovers partition-axis values through the active AxisFamily and compiles one frozen ProcessingContext per value." }, { "path": "openhcs/core/orchestrator/bundle_execution.py", - "sha256": "f5ba09a2ec7c688b1ca08167ab398cc976a9e2c39b301ae98f3c5fd560a25c85", + "sha256": "bc3a2d0a50c3db3be81b03f6af4a139c3db2f779e969aad608e59be66508ee4b", "role": "Validates compiled contexts and worker-safe steps before worker setup and builds the worker-lane execution plan." }, { @@ -2621,7 +2649,7 @@ }, { "path": "openhcs/core/orchestrator/worker_execution.py", - "sha256": "23a3ef2a70378dd1bb5c413a56e36bbb3fa013142741c9b45407bfc47de8c4a4", + "sha256": "26b9586890e7117b24b6b85660d3f417455d55546401739f91146d18e4d21134", "role": "Submits one task per active lane, collects per-axis ExecutionResult values, ships or inherits the bundle, and terminates owned process workers on cancellation." }, { @@ -2640,14 +2668,10 @@ "role": "Selects one multiprocessing context from the compiled GPU footprint." } ], - "findings": [ - "Step 5 said the executor submits one task per compiled axis; PooledWorkerLaneRunner submits one task per active worker lane, and a lane can own several axis contexts. Corrected.", - "The page attributed axis discovery to a 'microscope/source model' and component identities to 'nominal component declarations'; after G1/G4 the dataset source discovers the active AxisFamily's partition-axis values. Corrected the vocabulary.", - "'Worker-lane authority' and 'the execution server's worker ownership boundary' were vague; named ForkInheritedWorkerExecutionState (fork-inherited or shipped bundle) and the executor resources that terminate their own processes.", - "Bundle validation lives in orchestrator/bundle_execution.py, not compiled_execution.py; added it and worker_lanes.py as authorities." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/multiprocessing_coordination_system.rst in full and compared it against the changed authorities: pipeline/compiler.py (U3 ScopeKey step scopes, K6 progress phase/status classes, CompiledSourceBindingPlan() empty value), orchestrator/bundle_execution.py (lane results now reordered to compiled axis order) and orchestrator/worker_execution.py (K6 progress classes); every numbered step, ForkInheritedWorkerExecutionState, the per-lane task submission, per-axis ExecutionResult collection and the cancellation split still hold; no prose change needed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/multiprocessing_coordination_system.rst against B6's later changes to openhcs/core/pipeline/compiler.py, openhcs/core/orchestrator/bundle_execution.py, openhcs/core/orchestrator/worker_execution.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/multiprocessing_coordination_system.rst against the later B6 changes to openhcs/core/pipeline/compiler.py, openhcs/core/orchestrator/bundle_execution.py, openhcs/core/orchestrator/worker_execution.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/multiprocessing_coordination_system.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compiler.py, openhcs/core/compiled_execution.py and openhcs/core/orchestrator/worker_execution.py; traced _axis_values_to_process through AxisFamily.active().partition_axis(), validate_bundle_execution, CompiledContextLanePlanner, PooledWorkerLaneRunner._submit_lanes, ExecuteShippedLane and ProcessWorkerExecutorResources.cancel_execution; corrected the task granularity, axis vocabulary, bundle installation and cancellation wording.", diff --git a/docs/audits/architecture_n_z_development.json b/docs/audits/architecture_n_z_development.json index 0048a1eec1..ecd15cca57 100644 --- a/docs/audits/architecture_n_z_development.json +++ b/docs/audits/architecture_n_z_development.json @@ -102,7 +102,7 @@ }, { "path": "openhcs/core/processing_contracts.py", - "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", + "sha256": "eb0f7fd99ded1c4d5957152ccf6abe1468436c13574a1d1bb83ce368d24b9f69", "role": "ProcessingContract subclass family owning per-callable execution over the plane axis." }, { @@ -112,7 +112,7 @@ }, { "path": "openhcs/interop/cellprofiler/module_declarations.py", - "sha256": "6bbf41c158496a02439be2ee5aef12d7712fb632419c226a64c9259a55438b95", + "sha256": "b0195b66126c31f71dc0d4bb281a128e0164bb7dfb3b314ee8fe744ebc55d626", "role": "CellProfilerModule declarations owning callable names, catalogue placement, and headless-import eligibility." }, { @@ -121,12 +121,10 @@ "role": "Proves semantic-control parameters and strategy iteration derive from their nominal declaration owners." } ], - "findings": [ - "The example 'RuntimeSliceProjectionStrategy selects behavior by nominal runtime type' named a class deleted by G3; owned runtime values now subclass RuntimeSliceProjectableValue and RuntimeSliceProjection handles only foreign values. Example rewritten.", - "The selection mixins were described as OpenHCS models while openhcs/core/registry_strategies.py had been deleted (b221f8491); they are now imported from metaclass_registry.strategies, which the page now says." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/nominal_ownership.rst in full and compared it against the changed authorities: openhcs/core/processing_contracts.py (K5: CallableMetadata renamed CallableDeclaration for signature lookup) and interop/cellprofiler/module_declarations.py (K4: SourceBindingsConfig imported from core/config.py; K5: contracts carry artifact inputs/outputs directly); re-checked that the three metaclass_registry.strategies mixins, RegistryConfig/RegistryFamily, RuntimeSliceProjectableValue, RuntimeSliceProjection, ProcessingContract and CellProfilerModule exist with the described roles; the page names none of the renamed symbols, so no prose change was needed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/nominal_ownership.rst against B6's later changes to openhcs/core/processing_contracts.py, openhcs/processing/backends/lib_registry/openhcs_registry.py, openhcs/interop/cellprofiler/module_declarations.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/nominal_ownership.rst against the later B6 changes to openhcs/core/processing_contracts.py, openhcs/processing/backends/lib_registry/openhcs_registry.py, openhcs/interop/cellprofiler/module_declarations.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/nominal_ownership.rst in full and compared it against the changed authorities: deleted openhcs/core/registry_strategies.py (now metaclass_registry/strategies.py), metaclass_registry/core.py, tests/unit/test_registry_owner_iteration.py, lib_registry/openhcs_registry.py, interop/cellprofiler/module_declarations.py; verified ArtifactType, ProcessingContract, CellProfilerModule, the three mixins and the equivalence MostDerivedContextStrategyMixin users exist; replaced the deleted RuntimeSliceProjectionStrategy example and named the mixins' new home.", @@ -227,12 +225,12 @@ "authority": [ { "path": "openhcs/core/orchestrator/bundle_execution.py", - "sha256": "f5ba09a2ec7c688b1ca08167ab398cc976a9e2c39b301ae98f3c5fd560a25c85", + "sha256": "bc3a2d0a50c3db3be81b03f6af4a139c3db2f779e969aad608e59be66508ee4b", "role": "Parent compiled-bundle execution: executor shutdown, BrokenProcessPool handling, parent resource release, reverse-order viewer bootstrap rollback, and viewer settlement ordering." }, { "path": "openhcs/core/orchestrator/worker_execution.py", - "sha256": "23a3ef2a70378dd1bb5c413a56e36bbb3fa013142741c9b45407bfc47de8c4a4", + "sha256": "26b9586890e7117b24b6b85660d3f417455d55546401739f91146d18e4d21134", "role": "Inline, process and thread lane resources: per-axis image-cache release, process-local runtime/framework cleanup, and thread-lane deferral to the parent." }, { @@ -247,7 +245,7 @@ }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", + "sha256": "358d3eaf5952170f9e9e2dc9056fb08707e8fcf74682063c074b367a83c2648f", "role": "Framework/device assignment merge and cleanup_loaded used for the compiled framework/device footprint." }, { @@ -262,7 +260,7 @@ }, { "path": "tests/unit/test_orchestrator_lane_planning.py", - "sha256": "3d5c2f3662b8d4c330582af0c4a649b8f4d9582e064d9cf3159798c8934a2bbb", + "sha256": "85a73ddf071552fc88d454c9de857d066fd62f45492b74945ee22ec908b85289", "role": "Proves partial viewer acquisition and readiness failures roll back every acquired viewer." }, { @@ -271,11 +269,10 @@ "role": "Tests context-local image-cache lifetime across success and failure." } ], - "findings": [ - "Authority openhcs/core/orchestrator/compiled_plate_execution.py was renamed to bundle_execution.py by G7 (4cd901c9f); the page's 'compiled plate execution boundary' now reads 'compiled-bundle execution boundary' to match the current owner. Cleanup ordering, lane ownership and viewer rollback claims were re-verified and remain accurate." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/orchestrator_cleanup_guarantees.rst in full and compared it against the changed authorities: orchestrator/bundle_execution.py (K6 progress classes; lane results reordered to compiled axis order), orchestrator/worker_execution.py (K6 progress classes), compiled_step_plan.py (empty plan defaults) and tests/unit/test_orchestrator_lane_planning.py (progress phase name instead of value); none touches executor shutdown, viewer rollback, lane cleanup ownership, image-cache release or the terminal ordering, so no prose change was needed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/orchestrator_cleanup_guarantees.rst against B6's later changes to openhcs/core/orchestrator/bundle_execution.py, openhcs/core/orchestrator/worker_execution.py, openhcs/runtime/viewer_protocol.py, openhcs/core/compiled_step_plan.py, tests/unit/test_orchestrator_lane_planning.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/orchestrator_cleanup_guarantees.rst against the later B6 changes to openhcs/core/orchestrator/bundle_execution.py, openhcs/core/orchestrator/worker_execution.py, openhcs/runtime/viewer_protocol.py, openhcs/core/compiled_step_plan.py, tests/unit/test_orchestrator_lane_planning.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/orchestrator_cleanup_guarantees.rst in full and compared it against the changed authorities: compiled_plate_execution.py (renamed to bundle_execution.py, 4cd901c9f), worker_execution.py, orchestrator.py, viewer_protocol.py, arraybridge types.py, test_orchestrator_lane_planning.py, processing_context.py, test_runtime_stack_cache_lifetime.py; checked executor shutdown and BrokenProcessPool handling, reverse-order viewer rollback with external-client handling, thread-lane deferral of runtime release, per-axis image-cache release and cancellation-scope finish; renamed the boundary in the opening sentence. Dropped orchestrator.py and test_framework_device_resolver.py as authorities; added compiled_step_plan.py for the device-footprint cleanup.", @@ -303,30 +300,29 @@ "authority": [ { "path": "openhcs/core/orchestrator/orchestrator.py", - "sha256": "a61658b107f170c192eea505966c268ab6826826f03586bd64562a44800e1ccc", + "sha256": "99870516da6d64e1858f4dd8799d6e38ab6911b7b02bc29b252d2a23f7687410", "role": "PipelineOrchestrator: __objectstate_delegate__ to pipeline_config, dataset_root identity, and initialize enforcing the source role's availability check." }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "sha256": "4a33908fb42874e6d57eeea3f5305793ddbda41d13356f0a78501af38dcdfeb4", "role": "Compiler consuming ObjectState-resolved steps and the resolved global configuration once per compilation." }, { "path": "openhcs/core/dataset_sources/source.py", - "sha256": "097fde4d2b74031eacf53a88e79f499417e8c1c97ba0753e8a9df831b307efa6", + "sha256": "350bf5691a03522e9f54b1d8ab712c6beba1683aac4fa5159ea4e80c6041530d", "role": "DatasetSource family and SourceSelectionRole.require_available_source, which owns the local-directory availability contract." }, { "path": "external/ObjectState/src/objectstate/object_state.py", - "sha256": "03f818c3045b6a07024444ab2b5552fb6be9eab32dd151bb165b6f904d2863a3", + "sha256": "413a7bcd27695a823805cbc16a393569e30293a70c8d60d66dc6d56d3b589a4a", "role": "Generic editable ObjectState scope, resolution and snapshot mechanics." } ], - "findings": [ - "The availability paragraph named a 'microscope handler's source role' and a 'plate path'; after G4 the orchestrator opens a DatasetSource from configuration, whose SourceSelectionRole.require_available_source owns availability, and the orchestrator identity is dataset_root. The paragraph now names DatasetSource, SourceSelectionRole and the dataset root." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/orchestrator_configuration_management.rst in full and compared it against the changed authorities: orchestrator.py (K4 import of source_bindings_defaults_to_base from core/config.py), pipeline/compiler.py (U3 ScopeKey compiler scope ids, K6 progress classes), dataset_sources/source.py (K4 import move) and objectstate/object_state.py (U3 library release); confirmed __objectstate_delegate__ = 'pipeline_config', initialize opening the configured DatasetSource and calling its SourceSelectionRole.require_available_source on dataset_root, and once-per-compilation resolution; no prose change needed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/orchestrator_configuration_management.rst against B6's later changes to openhcs/core/orchestrator/orchestrator.py, openhcs/core/pipeline/compiler.py, openhcs/core/dataset_sources/source.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/orchestrator_configuration_management.rst against the later B6 changes to openhcs/core/orchestrator/orchestrator.py, openhcs/core/pipeline/compiler.py, openhcs/core/dataset_sources/source.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/orchestrator_configuration_management.rst in full and compared it against the changed authorities: orchestrator.py, pipeline/compiler.py, objectstate/object_state.py; checked the pipeline_config ObjectState delegation, once-per-compilation resolution, and the initialize availability check (now dataset_source.open plus SourceSelectionRole.require_available_source); corrected the availability paragraph's handler/plate-path vocabulary and added dataset_sources/source.py as authority.", @@ -349,7 +345,7 @@ "authority": [ { "path": "external/pyqt-reactive/src/pyqt_reactive/forms/parameter_form_manager.py", - "sha256": "a561e62e29af42630fa107b0929fc87b74007f7ed14ec8f0a6881c49070dc923", + "sha256": "2c59c3dc2f95a349970815a33c2afcf5158c97f4627973a8a0c6826f8ca9ae6c", "role": "Reusable parameter-form lifecycle and value projection." }, { @@ -369,10 +365,11 @@ } ], "findings": [ - "The transition page linked the same OpenHCS boundary twice instead of linking the extracted owner narrative." + "The page carries no product claim beyond the redirect; after U3 moved more generic Qt machinery into pyqt-reactive, parameter_form_manager.py, deferred_callback.py and the owner page still exist at the linked targets, so the redirect stays correct." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/parameter_form_lifecycle.rst in full and compared it against the changed authorities: pyqt_reactive/forms/parameter_form_manager.py (U3 library release); confirmed ParameterFormManager still owns form build, completion/failure callbacks and value projection, and that the pyqt-reactive owner page and external_foundations target exist; no prose change needed.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", "Sphinx resolved the pyqt-reactive intersphinx target backed by its current owner source", "Re-reviewed the public form-build completion, failure, and signal lifecycle against the complete dependent page and the updated pyqt-reactive owner documentation.", @@ -420,7 +417,7 @@ "authority": [ { "path": "external/pyqt-reactive/src/pyqt_reactive/forms/parameter_form_manager.py", - "sha256": "a561e62e29af42630fa107b0929fc87b74007f7ed14ec8f0a6881c49070dc923", + "sha256": "2c59c3dc2f95a349970815a33c2afcf5158c97f4627973a8a0c6826f8ca9ae6c", "role": "Reusable parameter-form lifecycle and value projection." }, { @@ -430,10 +427,11 @@ } ], "findings": [ - "The transition page duplicated the OpenHCS boundary link and omitted the extracted owner document." + "The page carries no product claim beyond the redirect; after U3, parameter_form_manager.py and the pyqt-reactive parametric_widget_creation owner page still exist at the linked targets, so the redirect stays correct." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/parametric_widget_creation.rst in full and compared it against the changed authorities: pyqt_reactive/forms/parameter_form_manager.py (U3 library release); confirmed ParameterFormManager still builds widgets from parameter declarations (build_form/setup_ui) and that the owner page and external_foundations target exist; no prose change needed.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", "Sphinx resolved the current owner-document target", "Re-reviewed the public form-build completion, failure, and signal lifecycle against the complete dependent page and the updated pyqt-reactive owner documentation.", @@ -467,15 +465,16 @@ }, { "path": "openhcs/processing/backends/lib_registry/registry_service.py", - "sha256": "daef8925338fddf8504a9faa066af42456a13fe4602976bf64be06547a45fcc8", + "sha256": "f2e981157632adb5020828720ec26bfa6444ab52f545e1e71b1524242eaed6a3", "role": "OpenHCS consumer: imports SignatureAnalyzer from python-introspect for callable catalogue preparation instead of keeping its own signature parser." } ], "findings": [ - "The record still carried disposition revise from an earlier prose fix; the page is a pure redirect to the python-introspect owner, so its disposition is redirect." + "The page carries no product claim beyond the redirect; the only owner change is registry_service.py reading processing_contract from the callable declaration instead of its metadata (K5), which this page does not describe." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/parser_metaprogramming_system.rst in full and compared it against the changed authorities: lib_registry/registry_service.py (K5: processing_contract read from reference.declaration); confirmed registry_service.py still imports SignatureAnalyzer from python-introspect and that external_foundations exists; no prose change needed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/parser_metaprogramming_system.rst against B6's later changes to openhcs/processing/backends/lib_registry/registry_service.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/parser_metaprogramming_system.rst against the later B6 changes to openhcs/processing/backends/lib_registry/registry_service.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/parser_metaprogramming_system.rst in full and compared it against the changed authorities: python_introspect/signature_analyzer.py, python-introspect docs index, lib_registry/registry_service.py; confirmed python-introspect still owns signature analysis and declaration projection, registry_service.py now imports SignatureAnalyzer from it, and the external_foundations target exists; no prose change needed. Disposition corrected from revise to redirect, since the page is only a route.", @@ -502,25 +501,26 @@ "authority": [ { "path": "openhcs/core/source_bindings.py", - "sha256": "1dfbd51ae4c64d6a6a8e5ff5ab2ba82465c5e0adacb98dddaf819e0fea05e241", - "role": "Public typed source-binding declaration vocabulary (SourceBindingsConfig and binding kinds)." + "sha256": "2defaf02ac0e214bb63f74d6eb56f294fbb5db4e06ecd9cc4479b16848f09854", + "role": "Typed source-binding declaration vocabulary: bindings, selectors, metadata rules, the SourceSetPairing family (PairByOrder, PairByMetadata) and the compiled plan." }, { "path": "openhcs/core/source_binding_selection.py", - "sha256": "f97a4475aa26e465679b89ff9a1fe96690ab7631649d71650608fc7a8aa4bf5a", + "sha256": "9610bc6112c068cf6c4b0ff7a1265ee4086291042e2dc422d8a90cce3c560eb7", "role": "Source candidate matching, anchor-pattern policies, and source-universe requests that replaced pattern detection." }, { "path": "openhcs/core/dataset_sources/source_bindings_source.py", - "sha256": "fccda877f39e7b2f99d6cc95143480e5955186264e316e832ca9a69c2ae90b8c", + "sha256": "63e4a3f849ab5122ab7af8767a6288135b7330027dff962c80add93d5fd820f1", "role": "SourceBindingsSource: the dataset source that ingests arbitrary image folders through source-binding declarations." } ], "findings": [ - "Authority openhcs/microscopes/source_bindings_handler.py was deleted by G4 (dataset-source family); arbitrary-folder ingestion now lives in openhcs/core/dataset_sources/source_bindings_source.py. The page names no module, so only the record changed." + "The record's source_bindings.py role named SourceBindingsConfig, which K4 moved to openhcs/core/config.py, and the binding kinds' pairing is now the SourceSetPairing family (SourceBindingMatchPlan/SourceBindingMatchMethod deleted). The page itself names no module or class, so only the record changed." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/pattern_detection_system.rst in full and compared it against the changed authorities: source_bindings.py (K4: config classes moved to core/config.py, SourceSetPairing replaces SourceBindingMatchPlan), source_binding_selection.py, dataset_sources/source_bindings_source.py; the page only links to source_model, which exists; corrected the authority role only.", "On 2026-10-11 (B6 final), compared docs/source/architecture/pattern_detection_system.rst against B6's later changes to openhcs/core/source_binding_selection.py, openhcs/core/dataset_sources/source_bindings_source.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/pattern_detection_system.rst against the later B6 changes to openhcs/core/source_bindings.py, openhcs/core/source_binding_selection.py, openhcs/core/dataset_sources/source_bindings_source.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/pattern_detection_system.rst in full and compared it against the changed authorities: source_bindings.py, source_binding_selection.py, and the deleted microscopes/source_bindings_handler.py (G4, c6d9a52ce); confirmed typed source matching still owns the model and the source_model target page exists; replaced the deleted handler authority with dataset_sources/source_bindings_source.py; no prose change needed.", @@ -530,7 +530,7 @@ }, { "path": "docs/source/architecture/pattern_grouping_and_special_outputs.rst", - "source_sha256": "42dcebbbb001891b323a38fb786c50ad1def8d13f52ec0b77750fd8b846ab293", + "source_sha256": "ebcb6664cbc3b3e80fb6dfa26e3f03c5b8990a00fef5c499052fb3aad5ef6f1b", "audience": [ "OpenHCS maintainers migrating older grouping and special-output concepts" ], @@ -539,30 +539,31 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", "role": "ProcessingConfig field docstrings: variable_components is the stack-axis authority and group_by partitions already assembled values, used for dictionary-pattern branch selection." }, { "path": "openhcs/core/processing_contracts.py", - "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", + "sha256": "eb0f7fd99ded1c4d5957152ccf6abe1468436c13574a1d1bb83ce368d24b9f69", "role": "ProcessingContract family, which moved here from the library registry module (G5)." }, { "path": "openhcs/core/pipeline/artifact_planning.py", - "sha256": "492024a74d5ce40dc326b162b3ac45edb0e55515a526a4f6512a71a8e4b11a77", + "sha256": "aea5f172c469aae05b473eebf52fe280367c606cd82c45e5821289eb7ab1ec90", "role": "Artifact producer/consumer graph and output materialization planning behind the artifact-contract route." }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "b6a7231bc2964d472070731018f6a4935e5f23cf8f238a232045c32244e6545a", + "sha256": "74b591773b7f653c957b2db33a1262925e5c02b71074695b6fc18d6120ed5934", "role": "RuntimeValueStore and runtime artifact queries behind the runtime-value route." } ], "findings": [ - "ProcessingContract moved from lib_registry/unified_registry.py to openhcs/core/processing_contracts.py (G5); the authority now points at the new owner. The page names only the class, so its prose remains correct." + "Removed 'canonical' filler from 'Use these canonical pages'. The page's claims are otherwise unaffected: config.py (K4 source-binding config classes, PartitionFilterMode family), processing_contracts.py (CallableDeclaration rename), artifact_planning.py (config.compiled_plan_for_contracts, invocation validation) and runtime_stores.py (hand-written to_dict/from_dict deleted) change nothing it states." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/pattern_grouping_and_special_outputs.rst in full and compared it against the changed authorities: core/config.py, core/processing_contracts.py, pipeline/artifact_planning.py and core/runtime_stores.py; confirmed the ProcessingConfig docstrings still make variable_components the assembled-array axis and group_by a partition of already assembled values, ProcessingContract and RuntimeValueStore exist, and the four linked pages exist; removed one filler word.", "On 2026-10-11 (B6 final), compared docs/source/architecture/pattern_grouping_and_special_outputs.rst against B6's later changes to openhcs/core/config.py, openhcs/core/processing_contracts.py, openhcs/core/runtime_stores.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/pattern_grouping_and_special_outputs.rst against the later B6 changes to openhcs/core/config.py, openhcs/core/processing_contracts.py, openhcs/core/runtime_stores.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/pattern_grouping_and_special_outputs.rst in full and compared it against the changed authorities: core/config.py, pipeline/artifact_planning.py, core/runtime_stores.py, lib_registry/unified_registry.py; checked the group_by/variable_components claims against the ProcessingConfig docstrings, confirmed ProcessingContract still exists (now in core/processing_contracts.py) and that all four target pages exist; replaced unified_registry.py with processing_contracts.py as authority; no prose change needed.", @@ -578,7 +579,7 @@ }, { "path": "docs/source/architecture/pipeline_compilation_system.rst", - "source_sha256": "17eed1d9929c9a983d8a8b3f69c6862fea341ba5485e1f166a59accc2ff7fbc5", + "source_sha256": "aa4e93269f118b794c42b4d42270dda228308194ec33b9d7170c4d373714b293", "audience": [ "OpenHCS compiler, runtime, and extension maintainers" ], @@ -587,17 +588,17 @@ "authority": [ { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "sha256": "4a33908fb42874e6d57eeea3f5305793ddbda41d13356f0a78501af38dcdfeb4", "role": "_resolve_pipeline_once ObjectState resolution, AxisCompilationRequest context projection, stage order, worker assignment, and bundle construction." }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", + "sha256": "0f9cc4819b53d55b7e9446429b8c8ac1ff1a59f4902020ea1658495ce01d464c", "role": "ResolvedPipelineDefinition scope/provenance validation and CompilationSession fields and missing-plan errors." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", + "sha256": "358d3eaf5952170f9e9e2dc9056fb08707e8fcf74682063c074b367a83c2648f", "role": "CompiledStepPlan typed field inventory and per-step framework-device assignment." }, { @@ -607,22 +608,22 @@ }, { "path": "openhcs/core/callable_contract.py", - "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", + "sha256": "a8e73c5df804d78baf72710e210cb2ae63156657a074071e11fd868f4b19d3b4", "role": "CallableContract.from_prepared_callable, process-local resolve_runtime_callable cache, and AXIS/PLATE FunctionStepExecutionScope." }, { "path": "openhcs/core/function_patterns.py", - "sha256": "95b9056c6980e4f7943f909aeae8262252c9aa175247a534befc8d4cc459fdd2", + "sha256": "cc8c35f37ea0a3bd939905fe317652a34b3129ee6f47f6f9023b8c459e62df70", "role": "CompiledFunctionInvocation binds its runtime callable at construction and rebuilds it after process transport." }, { "path": "openhcs/processing/backends/lib_registry/registry_service.py", - "sha256": "daef8925338fddf8504a9faa066af42456a13fe4602976bf64be06547a45fcc8", + "sha256": "f2e981157632adb5020828720ec26bfa6444ab52f545e1e71b1524242eaed6a3", "role": "prepare_in_current_process endpoint catalogue preparation and registered_callable owner projection." }, { "path": "openhcs/core/orchestrator/bundle_execution.py", - "sha256": "f5ba09a2ec7c688b1ca08167ab398cc976a9e2c39b301ae98f3c5fd560a25c85", + "sha256": "bc3a2d0a50c3db3be81b03f6af4a139c3db2f779e969aad608e59be66508ee4b", "role": "Parent ordering of plate-scoped steps, completed-dataset metadata finalization, and viewer settlement." }, { @@ -632,19 +633,16 @@ }, { "path": "tests/unit/test_compilation_session.py", - "sha256": "381adf3a87729928a225fd1db5a35f8c802340fb2a1e583501884dc5500f6b9d", + "sha256": "82f7227730a5285524dc9da39eb2d09f7af7ce1ee13285e5ebb0e7983d7e393b", "role": "Regression proof that axis sessions share one resolved pipeline and own their axis plans." } ], "findings": [ - "The 'Resolve once' section described a StepSnapshot tuple and an ObjectState map on ResolvedPipelineDefinition; StepSnapshot no longer exists and the definition now holds resolved steps with per-index scope ids and saved provenance, resolved without registering editor states.", - "The warm-up paragraph named prepare_compiled_context_callables(), deleted by db8b3650b; registered catalogue preparation now happens at endpoint readiness through RegistryService.prepare_in_current_process(), and compilation binds callables through CallableContract.from_prepared_callable() and CompiledFunctionInvocation.", - "The session field list named a step ObjectState map, snapshots and 'plate' facts, and the session was said to validate one-to-one step/state/snapshot identity; the session now holds the shared ResolvedPipelineDefinition, dataset scope and path resolver, and the checks are scope/provenance completeness plus missing-plan errors.", - "The typed-plan inventory listed source load plans and execution memory roles; CompiledStepPlan no longer has a source load plan and records only input/output memory types.", - "Authority openhcs/core/orchestrator/compiled_plate_execution.py was renamed to bundle_execution.py (G7, 4cd901c9f)." + "Replaced 'duplicate canonical names' with 'duplicate callable names' (filler word). The K5/K4/K6/U3 changes to the authorities (CallableDeclaration split, function-pattern walking moved to AuthoredFunctionPattern, artifact-declaration validation moved onto ArtifactGraph, ScopeKey compiler scopes, progress classes, empty plan defaults) do not change any claim the page makes." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/pipeline_compilation_system.rst in full and compared it against the changed authorities: pipeline/compiler.py, compilation_session.py (graph.validate_invocation_declarations; disabled-function stripping dropped from normalization), compiled_step_plan.py, callable_contract.py (K5), function_patterns.py (AuthoredFunctionPattern, graph-resolved invocation compile), lib_registry/registry_service.py, orchestrator/bundle_execution.py and tests/unit/test_compilation_session.py; re-verified _resolve_pipeline_once, ResolvedPipelineDefinition, CompilationSession, AxisCompilationRequest, FunctionStepTransportAuthority, RegistryService.registered_callable/prepare_in_current_process, CallableContract.from_prepared_callable, resolve_runtime_callable, CompiledFunctionInvocation.runtime_callable, CompiledRuntimeEnvironmentPlan, input/output memory types on CompiledStepPlan, AXIS/PLATE scopes and OpenHCSMetadataTarget; removed one filler word.", "On 2026-10-11 (B6 final), compared docs/source/architecture/pipeline_compilation_system.rst against B6's later changes to openhcs/core/steps/function_outputs.py: metadata documents are written only for backends that host local files (not virtual backends such as OMERO); the image-source guide test reads axis-family spellings. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/pipeline_compilation_system.rst against B6's later changes to openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/compiled_step_plan.py, openhcs/core/callable_contract.py, openhcs/core/function_patterns.py, openhcs/processing/backends/lib_registry/registry_service.py, openhcs/core/orchestrator/bundle_execution.py, openhcs/core/steps/function_outputs.py, tests/unit/test_compilation_session.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/pipeline_compilation_system.rst against the later B6 changes to openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/compiled_step_plan.py, openhcs/core/callable_contract.py, openhcs/core/function_patterns.py, openhcs/processing/backends/lib_registry/registry_service.py, openhcs/core/orchestrator/bundle_execution.py, openhcs/core/steps/function_outputs.py, tests/unit/test_compilation_session.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -675,7 +673,7 @@ "authority": [ { "path": "openhcs/authoring/session/session.py", - "sha256": "8d74425f65ce9f39f88439ca90bd09c02678ae3aef8d3cf6795570986444add0", + "sha256": "0c4e0f0979e1793862859e47eefeb6dd9d4b51c99fba20c2ebc6adf11619c3b5", "role": "Session class, MainThread/DispatcherThread, DatasetAccess and Renderer ports, dataset add/delete/split and work/mutation checks." }, { @@ -700,12 +698,12 @@ }, { "path": "openhcs/authoring/session/dataset_document.py", - "sha256": "305da3c0bf7bb15a0f251a101696b721f9151d8620450a6b3e12413b79218574", + "sha256": "50dc7b601b6a2343c8aaa1ebdaae92d9b4641f7c7ca6a02ade207809e5bdd172", "role": "DatasetDocumentScope family and payload scope-id check for selected code documents." }, { "path": "openhcs/authoring/session/pipelines.py", - "sha256": "245bed768c49ae17b26252d41a1040e59c46d59e0894b553309cd5ec8a9a1793", + "sha256": "febf9f9f464b2600554256f16929c408d4bdcaedd6ca99c71810bcb7be519519", "role": "Pipeline declaration reconciliation and saved-baseline commits." }, { @@ -734,12 +732,10 @@ "role": "MCP binds the session to DispatcherThread over McpMainThreadDispatcher." } ], - "findings": [ - "U1 (0631adc7c) replaced the Plate Manager service stack with the openhcs/authoring/session package and rewrote this page without a review; every Plate Manager authority (plate_manager_batch_workflow.py, batch_workflow_components.py, pipeline_object_state_binding.py, plate_manager_state_projection.py) was deleted. Authorities, audience and user need now follow the session package.", - "Re-checked the rewritten page's module map and invariants against the session package; replaced the jargon 'admission checks' with 'work and mutation checks' (require_work_allowed / require_definition_mutation_allowed). The file name still says plate_manager_services while the page documents the session; a rename is left to the lead." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/plate_manager_services.rst in full and compared it against the changed authorities: authoring/session/session.py (K6: debug command and replay mode are now classes such as RunDebugCommand and PersistentPausedWorkerReplay), dataset_document.py (L6: retain_only_authored_paths now from objectstate) and pipelines.py (U3: function scope ids built with ScopeKey); re-checked every module in the ownership map, MainThread/DispatcherThread/DatasetAccess/Renderer, QtSessionEventRelay, openhcs_session_events, McpMainThreadDispatcher, require_work_allowed/require_definition_mutation_allowed and PipelineImporter; none of the changed names appears on the page, so no prose change was needed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/plate_manager_services.rst against B6's later changes to openhcs/authoring/session/session.py: the writability probe makes one exclusive create attempt, the installed demo reports ViewerType.value, the unit session pre-creates its QApplication, and the GUI journey waits for each batch to end. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/plate_manager_services.rst against B6's later changes to openhcs/authoring/session/views.py: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/plate_manager_services.rst against B6's later changes to openhcs/authoring/session/session.py: the session decides in-place initialization by an actual file-creation attempt that leaves nothing behind, its debug-session methods moved into an inherited mixin, the measured CLI treats Windows Ctrl+Break like Ctrl+C, and the Official30 viewer test reads typed payload summaries. No claim this page makes changes.", @@ -849,18 +845,18 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", - "role": "ProcessingConfig variable_components, group_by and input_source field semantics." + "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", + "role": "ProcessingConfig variable_components, group_by and input_source field semantics, and StepMaterializationConfig." }, { "path": "openhcs/core/processing_contracts.py", - "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", + "sha256": "eb0f7fd99ded1c4d5957152ccf6abe1468436c13574a1d1bb83ce368d24b9f69", "role": "ProcessingContract family: Pure2DContract, Pure3DContract, FlexibleContract, VolumetricToSliceContract and collapses_input_plane_axis." }, { "path": "openhcs/core/callable_contract.py", - "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", - "role": "CallableContract memory types, allowed group_by roles, variable-component requirement, execution scope, runtime adapter, image mode and ImagePayloadConsumption." + "sha256": "a8e73c5df804d78baf72710e210cb2ae63156657a074071e11fd868f4b19d3b4", + "role": "CallableDeclaration fields (memory types, allowed group_by roles, stack requirement, processing contract, execution scope, runtime adapter, image mode, ImagePayloadConsumption, artifact declarations) inherited by CallableContract." }, { "path": "openhcs/core/payload_axes.py", @@ -879,18 +875,16 @@ }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", + "sha256": "358d3eaf5952170f9e9e2dc9056fb08707e8fcf74682063c074b367a83c2648f", "role": "CompiledStepPlan storage of resolved callable facts and framework-local device bindings." } ], "findings": [ - "The intro said image data is transported as 3D arrays; after G3 payloads are tensors with declared axes (PayloadAxes: plane axis, spatial domain, other declared axes). Rewritten.", - "The processing-contract section and common-mistakes list used enum-member names PURE_2D/PURE_3D/FLEXIBLE/VOLUMETRIC_TO_SLICE; G5 made contracts ProcessingContract subclasses (Pure2DContract, Pure3DContract, FlexibleContract, VolumetricToSliceContract) declared by class on the decorator. FLEXIBLE was said to delegate to the others; FlexibleContract now subclasses both and switches on its slice-by-slice control.", - "The runtime-projection section said RuntimeSliceProjection selects a registered strategy per value family; owned values now implement RuntimeSliceProjectableValue themselves and RuntimeSliceProjection handles only foreign values.", - "ProcessingContract moved from lib_registry/unified_registry.py to core/processing_contracts.py; the authority list follows it." + "K5 split callable_contract.py into CallableDeclaration (the declared fields) and CallableContract (declaration bound to the callable, inheriting those fields); the page's statement that a callable's CallableContract declares memory roles, grouping, processing contract, scope, adapter, image mode and artifacts still holds through inheritance, so only the evidence role was updated." ], - "disposition": "revise", + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/processing_semantics.rst in full and compared it against the changed authorities: core/config.py (K4 source-binding config classes moved in, PartitionFilterMode family; variable_components, group_by, input_source and StepMaterializationConfig unchanged), processing_contracts.py (CallableMetadata renamed CallableDeclaration), callable_contract.py (K5 declared-field CallableDeclaration, CallableContract subclass) and compiled_step_plan.py (empty plan defaults); confirmed InputSource still has two members, the four contract subclasses and FlexibleContract's bases, FunctionStepExecutionScope and ImagePayloadConsumption; no prose change needed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/processing_semantics.rst against B6's later changes to openhcs/core/config.py, openhcs/core/processing_contracts.py, openhcs/core/callable_contract.py, openhcs/core/compiled_step_plan.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/processing_semantics.rst against the later B6 changes to openhcs/core/config.py, openhcs/core/processing_contracts.py, openhcs/core/callable_contract.py, openhcs/core/compiled_step_plan.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/processing_semantics.rst in full and compared it against the changed authorities: core/config.py, lib_registry/unified_registry.py, callable_contract.py, runtime_slice_projection.py, compiled_step_plan.py, framework_device_assignment.py; verified the ProcessingConfig field semantics, the four ProcessingContract subclasses and keys, contract=Pure2DContract decorator use, CallableContract fields, FunctionStepExecutionScope and ImagePayloadConsumption; rewrote the payload-rank intro, contract names and the runtime-projection paragraph. Replaced unified_registry.py and framework_device_assignment.py with processing_contracts.py, payload_axes.py and runtime_plane_projection.py as authorities.", @@ -908,7 +902,7 @@ }, { "path": "docs/source/architecture/progress_runtime_projection_system.rst", - "source_sha256": "c659560a49b0afd79636c3209940a51209db5c9662a696d48ce7e08cb076a147", + "source_sha256": "76768c7b6ef484ede75b7f3ffbd5bdaad2f930fb04ad7675941b84a5d8d49635", "audience": [ "OpenHCS runtime, debugger, and UI maintainers" ], @@ -917,28 +911,33 @@ "authority": [ { "path": "openhcs/core/progress/projection.py", - "sha256": "0c931a096020b5622bcfbf92c856c7cb3ead193805b7276f5fb89bfb5613f8cb", + "sha256": "a643ebfa722014a0f03ff5bc168daf14e23a81bbc58f3159bc3bd6e8ba191617", "role": "SubjectRuntimeState family (labels, terminal and server-lifecycle policy), SubjectRuntimeIdentity, state_counts and ExecutionRuntimeProjection heartbeat reconciliation." }, { "path": "openhcs/core/progress/runtime_tree.py", - "sha256": "016279f4135e2c9b10cb841e57ba0297c4197a195623f8a746ed90ab06533ce5", + "sha256": "75599ae64cf21ebbc2496c0a77b2e3b40b465f867cebc290066fc65c4513dff3", "role": "Runtime tree node kinds subject, worker, partition, step and compilation." }, { "path": "openhcs/core/progress/types.py", - "sha256": "beabbfeb8a6ed727b4bb09f6155de60c4e8b89e71e7ed053bb85574aa1675419", - "role": "ProgressEvent, set_progress_queue and the PATTERN_GROUP, AXIS_COMPLETED and SUCCESS phases." + "sha256": "c46e825cd32801caae9fc107e0759976da054eadc56adc0d13354b3a8b7db209", + "role": "ProgressStream, ProgressPhase and ProgressStatus class families (PatternGroupProgressPhase on the step stream, AxisCompletedProgressPhase and SuccessProgressPhase), ProgressEvent and set_progress_queue." + }, + { + "path": "openhcs/core/progress/registry.py", + "sha256": "4172a15b1f1f5fab4c0490aa5f454f48678bdd477fa1c06d28c37ac0621a01e3", + "role": "ProgressRegistry keys retained events by subject, axis and stream." }, { "path": "openhcs/core/progress/debug_projection.py", - "sha256": "f3fc8a8628b3d6a739efdd9449fd71d56ea3ba5780c3d72f3344ed56516aeed4", - "role": "RuntimeProjectionBuilder and DebugRuntimeProjectionBuilder over one event snapshot." + "sha256": "82666d4afc2703b47cf954b2f74b36824142131844bc2c6a7645556381780531", + "role": "RuntimeProjectionBuilder and DebugRuntimeProjectionBuilder over one event snapshot; DebugRuntimeProjection current and last frames." }, { "path": "openhcs/core/debug.py", - "sha256": "95d365afefc46ee50da5abb8d5d246b3980feb32b85839f91e874cb7e689caaa", - "role": "Debug command, boundary-event, cursor, session, snapshot, artifact identity, snapshot store, warm-replay and paused-worker declarations." + "sha256": "4a1ed9fb0dcd40beeda9a2a3c90cdb5b2541252590d28281a9fb6f1aa021c510", + "role": "DebugCommandType and DebugBoundary families, command-keyed execution and stop strategies, cursor, session, snapshot, artifact identity, snapshot store, warm-replay and paused-worker declarations." }, { "path": "openhcs/core/debug_session_projection.py", @@ -957,12 +956,12 @@ }, { "path": "openhcs/core/orchestrator/bundle_execution.py", - "sha256": "f5ba09a2ec7c688b1ca08167ab398cc976a9e2c39b301ae98f3c5fd560a25c85", - "role": "execute_dataset_scoped_steps and the parent ordering before the pipeline SUCCESS event." + "sha256": "bc3a2d0a50c3db3be81b03f6af4a139c3db2f779e969aad608e59be66508ee4b", + "role": "execute_dataset_scoped_steps and the parent ordering before the pipeline SuccessProgressPhase event." }, { "path": "openhcs/runtime/zmq_progress.py", - "sha256": "740f8167a93565fb595b8b184776901457bb455c52eff68a89fd5974e602bc24", + "sha256": "7f182cb0832ccaa1bbc8f5d78034eda73d1eb2fb572b360b5eddeefb6751a0e5", "role": "ZMQCompilerProgressQueue and ZMQProgressEmitter for execution-server events." }, { @@ -972,13 +971,14 @@ } ], "findings": [ - "G7 renamed the progress vocabulary from plates to subjects: PlateRuntimeStateDeclarationBase is now the SubjectRuntimeState family, runtime identity is SubjectRuntimeIdentity(execution_id, subject_id), and the tree path is subject -> worker -> partition -> step (was plate -> worker -> well -> step). Invariants corrected.", - "Consumption steps 6-7 and the background-callback invariant described a Plate Manager GUI-thread Qt timer; after U1 the session's ExecutionProgress marks the projection dirty and rebuilds it on the main thread at an interval, publishing session events. Corrected.", - "execute_plate_scoped_steps() no longer exists; the parent calls execute_dataset_scoped_steps(). Corrected.", - "Authority openhcs/agent/services/execution_session_service.py was deleted (U1, 880cb2c55); the headless path is now the session (authoring/session/progress.py, agent/capabilities.py). Dropped the agent DTO, progress_stream and agent-services test authorities the page no longer relies on." + "K6 turned the progress enums into class families: channels are now the ProgressStream family and the registry keys by subject, axis and stream; PATTERN_GROUP, AXIS_COMPLETED and SUCCESS are PatternGroupProgressPhase, AxisCompletedProgressPhase and SuccessProgressPhase. The page named the old channel vocabulary and enum members; corrected.", + "K6 merged DebugCommandDeclarationBase into the DebugCommandType family (each command owns apply_to_worker; strategies key by command class) and replaced DebugBoundaryEventDeclarationBase with the DebugBoundary family; boundary outcome and timeline state were deleted as dead. The debugger-ownership bullets named both deleted classes and the deleted outcome/timeline facts; rewritten.", + "DebugRuntimeProjection no longer derives timeline frames (DebugRuntimeProjection.timeline deleted in K6); the projection-path paragraph now says current and last frames.", + "Removed 'Canonical' from the owner-docs heading. Added progress/registry.py as authority for the stream keying the page now states." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/progress_runtime_projection_system.rst in full and compared it against the changed authorities: core/progress/projection.py, runtime_tree.py, types.py, debug_projection.py, core/debug.py (K6 families), orchestrator/bundle_execution.py and runtime/zmq_progress.py (progress classes); checked ProgressStream/Phase/Status classes, ProgressRegistry's (subject, axis, stream) key, DebugCommandType.apply_to_worker, the command-keyed DebugInvocationExecutionStrategy/DebugStepStopStrategy, DebugBoundary members, DebugRuntimeProjection fields, and that SubjectRuntimeState, state_counts, node kinds, ExecutionProgress, execute_dataset_scoped_steps and the zmqruntime owner pages exist; corrected the progress vocabulary, the debugger-ownership bullets and the frame list.", "On 2026-10-11 (B6 final), compared docs/source/architecture/progress_runtime_projection_system.rst against B6's later changes to openhcs/core/progress/projection.py, openhcs/core/progress/runtime_tree.py, openhcs/core/progress/types.py, openhcs/core/debug.py, openhcs/authoring/session/progress.py, openhcs/core/orchestrator/bundle_execution.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/progress_runtime_projection_system.rst against the later B6 changes to openhcs/core/progress/projection.py, openhcs/core/progress/runtime_tree.py, openhcs/core/progress/types.py, openhcs/core/debug.py, openhcs/authoring/session/progress.py, openhcs/agent/capabilities.py, openhcs/core/orchestrator/bundle_execution.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/progress_runtime_projection_system.rst in full and compared it against the changed authorities: the page itself (edited in U1), core/progress/projection.py, core/debug.py, zmqruntime progress/projection.py and execution/progress_stream.py, deleted agent/services/execution_session_service.py, agent/dto/execution.py, tests/unit/agent/test_agent_services.py; verified every named class/function (all debug declarations, RuntimeProjectionBuilder, ExecutionProgress, LiveMeasurementProgressPayload, ZMQ progress emitters, phases) and corrected the plate-to-subject vocabulary, tree path, Qt-timer coalescing description and the dataset-scoped step function name.", @@ -1007,32 +1007,37 @@ "authority": [ { "path": "openhcs/core/pipeline_document.py", - "sha256": "8c554c8275c665f308eb098955bda517aa804eeb34418b5e0f41d412650ce7b5", + "sha256": "9a77a08367f901594859c8b6cae9609c9230a551f4f4e9ec6ea6cd2ed233c0f4", "role": "PipelineDocumentCodec.from_values and the optional pipeline_config / required pipeline_steps rule." }, { "path": "openhcs/core/steps/abstract.py", - "sha256": "63734f0c5198272b08f128ae974ada725c8201cb44efd0ce32500fd280b879bf", + "sha256": "9e710b3db635730f88258e5a6e18aed607876e296783022c06798f2bcd03d5e4", "role": "Step constructor accepting processing_config and source_bindings nested declarations." }, { "path": "openhcs/core/processing_contracts.py", - "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", + "sha256": "eb0f7fd99ded1c4d5957152ccf6abe1468436c13574a1d1bb83ce368d24b9f69", "role": "Pure2DContract and Pure3DContract named in the minimal example and the locality summary." }, + { + "path": "openhcs/core/config.py", + "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", + "role": "SourceBindingsConfig, StepSourceBindingsConfig and their lazy forms used in the folder example (declared here since K4)." + }, { "path": "openhcs/core/source_bindings.py", - "sha256": "1dfbd51ae4c64d6a6a8e5ff5ab2ba82465c5e0adacb98dddaf819e0fea05e241", - "role": "SourceBindingsConfig fields and the selector, filter and metadata-rule types used in the folder example." + "sha256": "2defaf02ac0e214bb63f74d6eb56f294fbb5db4e06ecd9cc4479b16848f09854", + "role": "Selector, filter, metadata-rule, component-selector and NamedSourceBinding types imported by the folder example." }, { "path": "openhcs/core/dataset_sources/source_bindings_source.py", - "sha256": "fccda877f39e7b2f99d6cc95143480e5955186264e316e832ca9a69c2ae90b8c", + "sha256": "63e4a3f849ab5122ab7af8767a6288135b7330027dff962c80add93d5fd820f1", "role": "SourceBindingsSource, selected when a non-empty source-binding declaration is present." }, { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "3439d96f3fbfcb104a53019dc150049021d9a8132bd27096f1ccf510eefbe37b", + "sha256": "b96f33b866fe6c9f8118ca70fcb7dff8e6a02cb29d79df67230a5e8b97df8002", "role": "import_cellprofiler_pipeline signature and headless-import rejection of interactive modules." }, { @@ -1047,29 +1052,24 @@ }, { "path": "openhcs/authoring/session/session.py", - "sha256": "8d74425f65ce9f39f88439ca90bd09c02678ae3aef8d3cf6795570986444add0", + "sha256": "0c4e0f0979e1793862859e47eefeb6dd9d4b51c99fba20c2ebc6adf11619c3b5", "role": "Session shared by the desktop application and MCP for progress, cancellation and results." }, { "path": "tests/unit/test_pipeline_document.py", - "sha256": "4df57a9c4f12a009222981bf1ac28554405874f3533656b61344ee94056d1bab", + "sha256": "4fe05636d5a9eb93f576904a774d30653a3686c7c8f61031ec0d316854cf5303", "role": "Proves missing pipeline_config selects PipelineConfig() while explicit invalid values and missing steps remain rejected." }, { "path": "tests/unit/test_cellprofiler_pipeline_import.py", - "sha256": "6b1d201ec5c4ba620fa2367a9cbef1f6fbdd25edbdda9bd7b723ebe3a3363412", + "sha256": "0ce4efd85d819c7e9f7fa84ec6f1ac7ec4bda024c9a8d5c7f264fda2730264f0", "role": "Proves interactive CellProfiler modules fail headless import instead of disappearing." } ], - "findings": [ - "The declaration-path diagram named StepSnapshot, which no longer exists; it now names ResolvedPipelineDefinition.", - "The ProcessingContract summary used the enum spellings PURE_2D/PURE_3D while the page's own example uses Pure2DContract (G5 class family); corrected to the class names.", - "The unsupported-folder section named SourceBindingsHandler and a 'registered microscope handler'; G4 replaced it with SourceBindingsSource in the DatasetSource family.", - "The source-binding code example used InputSource.PIPELINE_START without importing InputSource; the import is added.", - "The page said 'MCP execution services' own progress and cancellation; after U1 the shared Session does, for both the desktop application and MCP." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/quick_start.rst in full and compared it against the changed authorities: pipeline_document.py (rendering now through pycodify generate_python_source, L6), steps/abstract.py and source_bindings.py (K4: SourceBindingsConfig/StepSourceBindingsConfig now declared in core/config.py, SourceSetPairing family), processing_contracts.py (K5 CallableDeclaration rename), dataset_sources/source_bindings_source.py, cellprofiler/pipeline_import.py, authoring/session/session.py (K6 debug command classes) and both tests; imported every name in the page's code examples from the current tree (Lazy*Config from core.config, the selector/filter/metadata types from core.source_bindings, PipelineDocumentCodec.from_values, Pure2DContract, import_cellprofiler_pipeline signature, InputSource.PIPELINE_START, Microscopy.Channel, stack_percentile_normalize, SourceBindingsSource, Session). No claim on the page changed; the authority list now names core/config.py for the source-binding config classes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/quick_start.rst against B6's later changes to openhcs/authoring/session/session.py: the writability probe makes one exclusive create attempt, the installed demo reports ViewerType.value, the unit session pre-creates its QApplication, and the GUI journey waits for each batch to end. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/quick_start.rst against B6's later changes to openhcs/authoring/session/session.py: the session decides in-place initialization by an actual file-creation attempt that leaves nothing behind, its debug-session methods moved into an inherited mixin, the measured CLI treats Windows Ctrl+Break like Ctrl+C, and the Official30 viewer test reads typed payload summaries. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/quick_start.rst against B6's later changes to openhcs/core/processing_contracts.py, openhcs/core/dataset_sources/source_bindings_source.py, openhcs/interop/cellprofiler/pipeline_import.py, openhcs/authoring/session/session.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", @@ -1157,7 +1157,7 @@ }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "b6a7231bc2964d472070731018f6a4935e5f23cf8f238a232045c32244e6545a", + "sha256": "74b591773b7f653c957b2db33a1262925e5c02b71074695b6fc18d6120ed5934", "role": "RuntimeValueStore recording, replacement, RuntimeArtifactQuery resolution, and observation semantics." }, { @@ -1192,7 +1192,7 @@ }, { "path": "openhcs/core/callable_contract.py", - "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", + "sha256": "a8e73c5df804d78baf72710e210cb2ae63156657a074071e11fd868f4b19d3b4", "role": "CallableContract.resolve_returned_plan_values binding returned positions to exact output plan refs." }, { @@ -1202,8 +1202,8 @@ }, { "path": "tests/unit/test_function_artifact_outputs.py", - "sha256": "4c2b42cd9f31cc3cd63682039bb691cdd0af5bec3ca5b6f23f2ebea0fdbf0fac", - "role": "Proves image-output contextualization and that named artifacts bypass unrelated source-axis projection." + "sha256": "cba628c3eeff1176f634ccb388b8aa698b336c4a6ce408a9ec48f09e9bf85510", + "role": "Proves image-output contextualization and that named artifacts bypass unrelated source-axis projection, using admitted artifact graphs." }, { "path": "tests/unit/test_runtime_slice_projection.py", @@ -1211,14 +1211,10 @@ "role": "Proves undeclared value types are rejected and declared slices project without implicit device-to-host conversion." } ], - "findings": [ - "RuntimeReturnedOutputMatcher was deleted; CallableContract.resolve_returned_plan_values now binds returned positions to plan refs.", - "RuntimeSliceProjectionStrategy (a registry of per-type strategies) was replaced by the RuntimeSliceProjectableValue ABC that each value family implements; RuntimeSliceProjection only lists foreign types. The section described the deleted registry.", - "FunctionOutputContextStrategy, UnchangedFunctionOutputContextStrategy and ImageOutputSourceContextStrategy were deleted (G3/G5); contextualization is now ArtifactType.contextualize_output with image dispatch to the source value's contextualize_image_output.", - "SelectedPlaneImageOutput preserves source-plane identity, not specifically channel identity; ImagePayload now also carries declared payload axes (G3)." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/runtime_value_system.rst in full and compared it against the changed authorities: openhcs/core/runtime_stores.py, openhcs/core/callable_contract.py, tests/unit/test_function_artifact_outputs.py; runtime_stores dropped to_dict/from_dict on RuntimeArtifactLocation/Address (no page claim relies on them); K5 restructured callable metadata into CallableDeclaration but resolve_returned_plan_values remains on CallableContract; the test changes are fixture-only (admitted_graph, CompiledSourceBindingPlan()). Grepped every class and method the page names (RuntimeArrayPayload through ObjectLabelValue, RuntimeArtifactQuery, RuntimeExecutionAxisScope, ComponentGroupScope, source_context_source); all exist. No correction needed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/runtime_value_system.rst against B6's later changes to openhcs/core/runtime_stores.py, openhcs/core/artifacts.py, openhcs/core/runtime_object_labels.py, openhcs/core/callable_contract.py, tests/unit/test_function_artifact_outputs.py, tests/unit/test_runtime_slice_projection.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/runtime_value_system.rst against the later B6 changes to openhcs/core/runtime_stores.py, openhcs/core/artifacts.py, openhcs/core/runtime_object_labels.py, openhcs/core/callable_contract.py, tests/unit/test_function_artifact_outputs.py, tests/unit/test_runtime_slice_projection.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/runtime_value_system.rst in full and compared it against the changed authorities: runtime_artifact_values.py, runtime_stores.py, runtime_slice_projection.py, runtime_image_values.py, steps/function_runtime.py and the two test files; verified every named class/function exists, replaced the deleted output matcher, slice-projection strategy registry and function-output context strategies with their current owners (resolve_returned_plan_values, RuntimeSliceProjectableValue, ArtifactType.contextualize_output and contextualize_image_output), and confirmed the persisted-metadata reload fallback in source_workspace_projection.py still holds.", @@ -1244,7 +1240,7 @@ "authority": [ { "path": "external/pyqt-reactive/src/pyqt_reactive/forms/parameter_form_manager.py", - "sha256": "a561e62e29af42630fa107b0929fc87b74007f7ed14ec8f0a6881c49070dc923", + "sha256": "2c59c3dc2f95a349970815a33c2afcf5158c97f4627973a8a0c6826f8ca9ae6c", "role": "Reusable parameter-form lifecycle and value projection." }, { @@ -1254,10 +1250,11 @@ } ], "findings": [ - "The transition page duplicated the OpenHCS boundary link and omitted the owner document." + "Redirect still correct after U3: the pyqt-reactive owner document exists and OpenHCS holds no scope-feedback mechanism; no page change needed." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/scope_visual_feedback_system.rst in full and compared it against the changed authorities: pyqt_reactive/forms/parameter_form_manager.py; the page states only that pyqt-reactive owns scope visual feedback; U3 moved more generic Qt machinery into pyqt-reactive but did not move this mechanism back, and the linked owner document external/pyqt-reactive/docs/architecture/scope_visual_feedback_system.rst still exists. No correction needed.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", "Sphinx resolved the current pyqt-reactive owner target", "Re-reviewed the public form-build completion, failure, and signal lifecycle against the complete dependent page and the updated pyqt-reactive owner documentation.", @@ -1296,7 +1293,7 @@ }, { "path": "docs/source/architecture/serialization_boundaries.rst", - "source_sha256": "5af8b42af40d836966f05ac2cf7cc943c7e34535687cd78ceb711a62e032bd56", + "source_sha256": "f3f6ee89037b43f7babe9c4c9342a2255894210517c295afb668e9b69aae03eb", "audience": [ "OpenHCS serialization and transport maintainers" ], @@ -1305,12 +1302,12 @@ "authority": [ { "path": "external/python-introspect/src/python_introspect/jsonable.py", - "sha256": "8fe06533a6dce179b7d77792b6ac768c2a64a1e9e79bbe926893498fae035bb4", + "sha256": "0148e51fb2fc4c7e1c625b088cb0ca56fecd58f9836b9e5da58b75b5eb2d40ee", "role": "to_jsonable singledispatch encoder: scalars, mappings, sequences, paths, enums, callables by import path, dataclasses, and AutoRegisterMeta types by registry key." }, { "path": "external/python-introspect/src/python_introspect/dataclass_projection.py", - "sha256": "e77d4f22b02e9d636e401c27643051a436b41b2f222bf17a0784e1c18a8e8425", + "sha256": "7a8edc205fae5c663f2bfd70017c96492008171b40700bb9b41e3f52e4dcf9c6", "role": "dataclass_from_mapping decoder paired with to_jsonable." }, { @@ -1320,15 +1317,21 @@ }, { "path": "external/pycodify/src/pycodify/core.py", - "sha256": "b32dea495ceb8745c1b363f2e42aa62eaa28a1d54198a5d6541e0b1546facfb4", - "role": "Executable Python rendering and formatter composition." + "sha256": "f5ae16c3bc62d2b2dc24f2701fb7fa936c6dd64fc602d3e64aabc8a38730ef18", + "role": "generate_python_source, FormatContext (clean mode, PathRendering), formatter registration, and the PythonSourceLiteral ABC." + }, + { + "path": "external/ObjectState/src/objectstate/codegen.py", + "sha256": "347d2697c061a35e6e440e373fe4e8bfce1ad92b500e98da07faa26f6a3a1356", + "role": "LazyResolutionFormatter and LazyDataclassFormatter: source formatting for lazy configuration dataclasses." } ], "findings": [ - "Authority openhcs/serialization/json.py was deleted (L2, c0a53a1e0); the JSON codec now lives only in python_introspect.jsonable and dataclass_projection, which the page already names. Authority replaced; prose unchanged." + "The Python-source section did not name pycodify's PythonSourceLiteral and path factoring or objectstate.codegen, which L6 made the owners of value-owned source spelling, path factoring, and lazy-config formatting." ], - "disposition": "keep", + "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/serialization_boundaries.rst in full and compared it against the changed authorities: python_introspect/jsonable.py, python_introspect/dataclass_projection.py, pycodify/core.py; to_jsonable still dispatches scalars, mappings, sequences, paths, enums, callables by import path, dataclasses and AutoRegisterMeta types by registry key, and dataclass_from_mapping is its decoder. L6 moved PythonSourceLiteral and path factoring into pycodify 0.2 (FormatContext.paths) and lazy-config formatting into objectstate.codegen; added those owners to the Python-source section. Unformattable values now raise in pycodify (FallbackFormatter deleted), consistent with the extension rule.", "On 2026-10-11 (B6 final), compared docs/source/architecture/serialization_boundaries.rst against B6's later changes to openhcs/core/image_file_serialization.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/serialization_boundaries.rst against the later B6 changes to openhcs/core/image_file_serialization.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/serialization_boundaries.rst in full and compared it against the changed authorities: openhcs/serialization/json.py (deleted, L2) and openhcs/core/image_file_serialization.py; checked every to_jsonable dispatch case in python_introspect/jsonable.py, the dataclass_from_mapping decoder, OpenHCS to_jsonable.register users, and ImageFileFormat suffix/prepare/read/write/pixel-semantics members; no prose change was needed.", @@ -1396,36 +1399,41 @@ }, { "path": "docs/source/architecture/source_model.rst", - "source_sha256": "8d0e4c61c920be48ae0de60aea6ba29eb850d4cdb564646ba53422129aa69384", + "source_sha256": "73cae6b1f57916e13fcdfd69eb97c09dbe0d518b4e9980f8b42743f245ff50db", "audience": [ "Pipeline authors and OpenHCS source/compiler maintainers" ], "user_need": "Understand how arbitrary and native image sources become typed compiled inputs.", "diataxis": "explanation", "authority": [ + { + "path": "openhcs/core/config.py", + "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", + "role": "SourceBindingsConfig and StepSourceBindingsConfig declarations and source_bindings_defaults_to_base." + }, { "path": "openhcs/core/source_bindings.py", - "sha256": "1dfbd51ae4c64d6a6a8e5ff5ab2ba82465c5e0adacb98dddaf819e0fea05e241", - "role": "Public source-binding declaration vocabulary and compiled binding/universe plans." + "sha256": "2defaf02ac0e214bb63f74d6eb56f294fbb5db4e06ecd9cc4479b16848f09854", + "role": "Declaration vocabulary (ImagePlaneSource, selectors, filters, MetadataExtractionRule, NamedSourceBinding, SourceSetPairing/PairByOrder/PairByMetadata) and the compiled binding and universe plans." }, { "path": "openhcs/core/source_binding_workspace.py", - "sha256": "4d28066a9f73b283fc637662e05fb2632c6fa18af164476c2899fce58cd647fe", - "role": "Source-binding workspace projector and materialization of the virtual workspace." + "sha256": "5e7ac439d7ef6cd35a5660d2ad9db24f1890801d8bdf1d82c13b12f8baaa9eb0", + "role": "Source-binding workspace projector, pairing-keyed source-set assemblers and materialization of the virtual workspace." }, { "path": "openhcs/core/dataset_sources/source_bindings_source.py", - "sha256": "fccda877f39e7b2f99d6cc95143480e5955186264e316e832ca9a69c2ae90b8c", + "sha256": "63e4a3f849ab5122ab7af8767a6288135b7330027dff962c80add93d5fd820f1", "role": "SourceBindingsSource: recursive folder inventory and declared source-binding projection." }, { "path": "openhcs/core/dataset_sources/choice.py", - "sha256": "7fb3c7ce701f0d629f288b79090044f43ac8ce1b51ca4f50937534407f226f1a", + "sha256": "497eb028aabb80283be8a621ea94e09881dfcd51b2e34e2bbecafa09e004c1f6", "role": "AutoDetectedSource falls back to the registered declared-file source only when detection fails and bindings are non-empty." }, { "path": "openhcs/core/source_projection.py", - "sha256": "deee0bf8b0f5c19bcbbf511fd5355767e65cfa0666c4c2b38cf5b709ae342919", + "sha256": "c51f7c16a9174ed1fda972fc020d2ce0f14dd2005d32929df0bae174878bb735", "role": "SourcePlaneProjection, SourceArtifactProjection, SourceProjectionSet and typed SourceDatasetDiagnostic." }, { @@ -1435,10 +1443,11 @@ } ], "findings": [ - "SourceBindingsHandler and its module openhcs/microscopes/source_bindings_handler.py were replaced by SourceBindingsSource in the G4 dataset-source family; the fallback is chosen by AutoDetectedSource, not a handler factory." + "The declaration vocabulary named SourceBindingMatchPlan, which K4 deleted; pairing is now the SourceSetPairing family (PairByOrder, the default when no method is declared, and PairByMetadata)." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/source_model.rst in full and compared it against the changed authorities: core/source_bindings.py (K4: config classes moved to core/config.py, SourceBindingMatchPlan/SourceBindingMatchMethod replaced by SourceSetPairing, plans never None), source_binding_workspace.py (pairing-keyed assemblers, path identity), dataset_sources/source_bindings_source.py and choice.py (import moves only), source_projection.py (source_path_posix); grepped every other named declaration, projection, diagnostic, serializer and PolyStore method in the current tree; replaced the SourceBindingMatchPlan entry with SourceSetPairing and added core/config.py as the config-class authority.", "On 2026-10-11 (B6 final), compared docs/source/architecture/source_model.rst against B6's later changes to openhcs/core/source_binding_workspace.py, openhcs/core/dataset_sources/source_bindings_source.py, openhcs/core/dataset_sources/choice.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/source_model.rst against the later B6 changes to openhcs/core/source_bindings.py, openhcs/core/source_binding_workspace.py, openhcs/core/dataset_sources/source_bindings_source.py, openhcs/core/dataset_sources/choice.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/source_model.rst in full and compared it against the changed authorities: openhcs/core/source_bindings.py, openhcs/core/source_binding_workspace.py, openhcs/microscopes/source_bindings_handler.py (deleted, G4) and PolyStore filemanager.py; checked every named declaration, compiled plan, projection, diagnostic and PolyStore method exists, corrected the arbitrary-folder fallback to AutoDetectedSource/SourceBindingsSource, and confirmed CompiledStepPlan carries no separate source-binding enabled flag.", @@ -1458,36 +1467,36 @@ "authority": [ { "path": "openhcs/core/callable_contract.py", - "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", + "sha256": "a8e73c5df804d78baf72710e210cb2ae63156657a074071e11fd868f4b19d3b4", "role": "CallableContract input/output declarations and runtime-owned parameter projection." }, { "path": "openhcs/core/pipeline/function_contracts.py", - "sha256": "f5e7c74ea3d2948c75388184263cd9f1f17905f820b180f634dfaa7514e1d526", + "sha256": "ffc21dd3021a2316cfbd100c2d655867ce4bec619f2db7288e82aeb11c1930e8", "role": "special_inputs decorator: records parameter names as runtime-managed callable metadata and excludes them from user parameters; no special_outputs decorator exists." }, { "path": "openhcs/core/pipeline/artifact_planning.py", - "sha256": "492024a74d5ce40dc326b162b3ac45edb0e55515a526a4f6512a71a8e4b11a77", + "sha256": "aea5f172c469aae05b473eebf52fe280367c606cd82c45e5821289eb7ab1ec90", "role": "ArtifactGraph and ArtifactSpecRef planning during compilation." }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "b6a7231bc2964d472070731018f6a4935e5f23cf8f238a232045c32244e6545a", + "sha256": "74b591773b7f653c957b2db33a1262925e5c02b71074695b6fc18d6120ed5934", "role": "RuntimeValueStore recording, replacement, query, and observation during execution." }, { "path": "openhcs/interop/cellprofiler/module_artifact_contracts.py", - "sha256": "982680cd36409ea6d90a07d4e60893b162847f5525ed516891e995877dc32fd1", + "sha256": "4d521778e83fee62a4b164d6a3934d0824f2c5e7375bc7bea1d74256e67d61fe", "role": "CellProfilerModuleArtifactContracts: per-invocation artifact bindings from CellProfiler setting declarations." } ], "findings": [ - "The string-keyed special-I/O architecture is superseded; this URL exists only to route legacy terminology.", - "The unreviewed page edit that dropped special_outputs left a broken sentence ('It names Python input parameters; they do not own ...'); fixed the grammar and stated that special_outputs no longer exists. Added the special_inputs declaration owner as an authority." + "Redirect still correct after K5: every owner the page names still exists with the stated role; no page change needed." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/special_io_system.rst in full and compared it against the changed authorities: callable_contract.py, function_contracts.py, artifact_planning.py, runtime_stores.py, module_artifact_contracts.py; K5 replaced CallableMetadata with CallableDeclaration (CallableContract now subclasses it); CallableContract, special_inputs, ArtifactGraph, RuntimeValueStore and CellProfilerModuleArtifactContracts all still exist with the roles the page assigns, and no special_outputs decorator exists. No correction needed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/special_io_system.rst against B6's later changes to openhcs/core/pipeline/function_contracts.py: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/special_io_system.rst against B6's later changes to openhcs/core/callable_contract.py, openhcs/core/pipeline/function_contracts.py, openhcs/core/runtime_stores.py, openhcs/interop/cellprofiler/module_artifact_contracts.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/special_io_system.rst against the later B6 changes to openhcs/core/callable_contract.py, openhcs/core/pipeline/function_contracts.py, openhcs/core/runtime_stores.py, openhcs/interop/cellprofiler/module_artifact_contracts.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -1512,30 +1521,29 @@ "authority": [ { "path": "external/pyqt-reactive/src/pyqt_reactive/forms/parameter_form_manager.py", - "sha256": "a561e62e29af42630fa107b0929fc87b74007f7ed14ec8f0a6881c49070dc923", + "sha256": "2c59c3dc2f95a349970815a33c2afcf5158c97f4627973a8a0c6826f8ca9ae6c", "role": "Reusable parameter-form lifecycle and value projection." }, { "path": "openhcs/pyqt_gui/widgets/step_parameter_editor.py", - "sha256": "d7d6a2af7d3dafbd3edd57d4a3c52a1f40812e21b95820d1e2190a57e9a97c0c", + "sha256": "534f33acc5e295bfda4fbd73cbec49c4fbf662953e14842bf04ce59550f2dea3", "role": "FunctionStep editor widget: config-tree, code-document round trip, and OpenHCS parameter policy over the pyqt-reactive form." }, { "path": "openhcs/authoring/session/step_scopes.py", - "sha256": "2d34e0765fb3fb3e20f33d23b3c2dcb9e9b213abd0c5d3ea7238a0adf3e16926", + "sha256": "6caab26c963c958bf65aec69c03e49e1a4e84a46005c6d61a5301e67618be875", "role": "StepEditorScope: FunctionStep scope identity used by the editor." }, { "path": "openhcs/pyqt_gui/services/window_handlers.py", - "sha256": "1523b0874cbd4f7048d713afef84311c18291918c432bbbe7305488c1baa03a6", + "sha256": "6f05c7945f41b25e0b783231d51b508729ba0713911224bf46f84f89ea24823f", "role": "OpenHCS window-route registration, including definition-mutation guards and compiled-inspection actions through the plate session." } ], - "findings": [ - "Authority changes (U1 session interface, D1 dead UI code) moved StepEditorScope to openhcs/authoring/session/step_scopes.py, routed mutation/inspection guards through plate_manager.session, and deleted the pickled .step load/save controller. The page names none of these details, so its boundary claims stay true; added step_scopes.py as the scope-registration owner." - ], + "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/step-editor-generalization.rst in full and compared it against the changed authorities: parameter_form_manager.py, step_parameter_editor.py, step_scopes.py, window_handlers.py; the diffs move CodeEditorFormUpdater, use_external_code_editor and ConfigWindow into pyqt-reactive (OpenHCSConfigWindow subclasses it), take SourceBindingsConfig from openhcs.core.config, and replace '::' joins with objectstate.ScopeKey. The page already assigns editor-window lifecycle to pyqt-reactive and keeps FunctionStep scope registration, code documents and workflow composition in OpenHCS, which still holds; both linked pyqt-reactive owner documents exist. No correction needed.", "On 2026-10-11 (B6), re-read docs/source/architecture/step-editor-generalization.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/widgets/step_parameter_editor.py and openhcs/pyqt_gui/services/window_handlers.py; confirmed scope registration, code-document actions and compile/inspection routes still live in OpenHCS while form mechanics stay in pyqt-reactive; no prose change was needed.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", "Current imports place generic form/window mechanics in pyqt-reactive and OpenHCS actions in the named adapters", @@ -1569,13 +1577,14 @@ }, { "path": "pyproject.toml", - "sha256": "c3a2bea22aa6872940125f4aaac72c9eac72084fefc6c2cefdb54b5273361e4a", + "sha256": "b5d4d752704b886da20d860743b81dc3d2ce3a3c402208246fadc1c6cb71f597", "role": "Declares polystore and arraybridge as separate runtime dependencies of OpenHCS." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/storage_and_memory_system.rst in full and compared it against the changed authorities: pyproject.toml; the diff only raises dependency pins (pyqt-reactive 0.5, pycodify 0.2, objectstate 2.1, zmqruntime 0.6.2, python-introspect 0.2.7); polystore and arraybridge remain separate runtime dependencies, and FileManager and the arraybridge converters still exist. No correction needed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/storage_and_memory_system.rst against the later B6 changes to pyproject.toml: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/storage_and_memory_system.rst in full and compared it against the changed authorities: external/PolyStore/src/polystore/filemanager.py (submodule advance) and pyproject.toml (polystore 0.6, arraybridge 0.4.1, JAX and openhcs-basicpy dependency changes); the redirect makes no version, backend or API claim those changes affect, and both packages remain separate dependencies with intersphinx targets in docs/source/conf.py; no prose change was needed.", "Re-read docs/source/architecture/storage_and_memory_system.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -1622,7 +1631,7 @@ }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", + "sha256": "358d3eaf5952170f9e9e2dc9056fb08707e8fcf74682063c074b367a83c2648f", "role": "CompiledStepPlan.producer_identity_for_main_flow and producer_identity_for_artifact." }, { @@ -1632,7 +1641,7 @@ }, { "path": "openhcs/core/orchestrator/bundle_execution.py", - "sha256": "f5ba09a2ec7c688b1ca08167ab398cc976a9e2c39b301ae98f3c5fd560a25c85", + "sha256": "bc3a2d0a50c3db3be81b03f6af4a139c3db2f779e969aad608e59be66508ee4b", "role": "Viewer bootstrap with rollback of the acquired set, settle-then-read-state order, and stopping of execution-owned viewers." }, { @@ -1662,7 +1671,7 @@ }, { "path": "openhcs/runtime/zmq_execution_server.py", - "sha256": "e24bdcb3b086cc60197ceb63f29a6f6b639a93cad175218941148a207eb285f2", + "sha256": "12e96bcd1fb1aaccb43aa3fe90697b5d1cdda8998721a556005e95f344c9b2b3", "role": "Releases execution-session process resources when the execution server stops." }, { @@ -1697,18 +1706,14 @@ }, { "path": "tests/unit/test_orchestrator_lane_planning.py", - "sha256": "3d5c2f3662b8d4c330582af0c4a649b8f4d9582e064d9cf3159798c8934a2bbb", + "sha256": "85a73ddf071552fc88d454c9de857d066fd62f45492b74945ee22ec908b85289", "role": "Proves multi-viewer bootstrap rollback after partial acquisition and readiness failure." } ], - "findings": [ - "FunctionStepOutputProducerIdentityAuthority no longer exists; producer identity is derived by CompiledStepPlan.producer_identity_for_main_flow/producer_identity_for_artifact.", - "ObjectLabelImageOutputSourceContextStrategy was deleted; the label-rendered image rule is now ObjectLabelValue.contextualize_image_output.", - "openhcs/core/orchestrator/compiled_plate_execution.py was renamed bundle_execution.py (G7); the page's 'compiled plate execution' wording and the authority were updated.", - "Dropped authorities the page does not rely on (FileManager, zmqruntime execution client, execution_visualizer.py) and added PolyStore's StreamProducerIdentity owner." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/streaming_boundary_and_wrappers.rst in full and compared it against the changed authorities: compiled_step_plan.py (CompiledSourceBindingPlan/CompiledSourceUniversePlan default factories), orchestrator/bundle_execution.py (K6 ProgressPhase/ProgressStatus classes, lane results ordered by compiled axis), runtime/zmq_execution_server.py (K6 debug replay predicate) and test_orchestrator_lane_planning.py; verified StreamComponentMessageExtraAuthority, StepOutputManifestStore, ArtifactMaterializationTargetPlan, producer_identity_for_main_flow/_for_artifact, StreamProducerIdentity.projection_key, ObjectLabelValue.contextualize_image_output, ViewerFamily/ViewerType, ViewerControlAction, ViewerProcessPlatform, ViewerSettleProgress, EndpointProcessGroup, the viewer control/projection option classes, settle-then-read-state order, server stop cleanup and the bootstrap rollback test all still exist as described; no claim needed correction.", "On 2026-10-11 (B6 final), compared docs/source/architecture/streaming_boundary_and_wrappers.rst against B6's later changes to openhcs/core/viewer_streaming_service.py, openhcs/core/compiled_step_plan.py, openhcs/core/runtime_object_labels.py, openhcs/core/orchestrator/bundle_execution.py, openhcs/runtime/viewer_protocol.py, openhcs/runtime/zmq_execution_server.py, tests/unit/test_viewer_result_element_selection.py, tests/unit/test_napari_streaming_handlers.py, tests/unit/test_napari_accepted_work_settlement.py, tests/unit/test_orchestrator_lane_planning.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/streaming_boundary_and_wrappers.rst against the later B6 changes to openhcs/core/viewer_streaming_service.py, openhcs/core/compiled_step_plan.py, openhcs/core/runtime_object_labels.py, openhcs/core/orchestrator/bundle_execution.py, openhcs/runtime/viewer_protocol.py, openhcs/runtime/zmq_execution_server.py, tests/unit/test_viewer_result_element_selection.py, tests/unit/test_napari_streaming_handlers.py, tests/unit/test_napari_accepted_work_settlement.py, tests/unit/test_orchestrator_lane_planning.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/streaming_boundary_and_wrappers.rst in full and compared it against the changed authorities: viewer_streaming_service.py, streaming_config_declarations.py, viewer_protocol.py, viewer_controls.py, napari_viewer_server.py, viewer_window_service.py, zmq_execution_server.py, compiled_plate_execution.py (renamed bundle_execution.py, G7), zmqruntime client.py and the two viewer tests; checked every named class exists, corrected the producer-identity and label-image context owners, and confirmed the bootstrap rollback, settle/read-state/stop order, Linux XCB/MIT-SHM/vblank policy and settle phases in current code.", @@ -1749,12 +1754,12 @@ "authority": [ { "path": "openhcs/core/pipeline_document.py", - "sha256": "8c554c8275c665f308eb098955bda517aa804eeb34418b5e0f41d412650ce7b5", + "sha256": "9a77a08367f901594859c8b6cae9609c9230a551f4f4e9ec6ea6cd2ed233c0f4", "role": "PipelineDocument and PipelineDocumentCodec: the declaration entry point of the end-to-end flow." }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "sha256": "4a33908fb42874e6d57eeea3f5305793ddbda41d13356f0a78501af38dcdfeb4", "role": "PipelineCompiler phases from declaration resolution to framework device assignment." }, { @@ -1764,10 +1769,11 @@ } ], "findings": [ - "The previous broad integration narrative was replaced by the current typed system overview; this URL only routes to it. B6 re-check: the target page and the authorities still exist (PipelineDocumentAuthority was renamed PipelineDocumentCodec, which this redirect does not name)." + "Redirect still correct: the target page covers the current flow; no page change needed." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/system_integration.rst in full and compared it against the changed authorities: openhcs/core/pipeline_document.py, openhcs/core/pipeline/compiler.py; PipelineDocument now renders through pycodify generate_python_source and the compiler uses ScopeKey and the K6 progress classes; the redirect to system_overview still describes the current flow. No correction needed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/system_integration.rst against B6's later changes to openhcs/core/pipeline/compiler.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/system_integration.rst against the later B6 changes to openhcs/core/pipeline/compiler.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/system_integration.rst in full and compared it against the changed authorities: openhcs/core/pipeline_document.py (PipelineDocumentAuthority renamed PipelineDocumentCodec), openhcs/core/pipeline/compiler.py and openhcs/core/compiled_execution.py; the redirect states no API detail and its target system_overview.rst exists; no prose change was needed.", @@ -1781,7 +1787,7 @@ }, { "path": "docs/source/architecture/system_overview.rst", - "source_sha256": "cff7b61a5cdcf7f49a883725ccb25da48ded119783c7758952d3015eba85e362", + "source_sha256": "30b4399b81c0557f010d3723c24aac7752551b3fc03b9b3e3d250679b6cfb1da", "audience": [ "OpenHCS users, maintainers, and integrators" ], @@ -1790,17 +1796,17 @@ "authority": [ { "path": "openhcs/core/pipeline_document.py", - "sha256": "8c554c8275c665f308eb098955bda517aa804eeb34418b5e0f41d412650ce7b5", + "sha256": "9a77a08367f901594859c8b6cae9609c9230a551f4f4e9ec6ea6cd2ed233c0f4", "role": "PipelineDocument: PipelineConfig plus the FunctionStep list that all front ends converge on." }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", + "sha256": "0f9cc4819b53d55b7e9446429b8c8ac1ff1a59f4902020ea1658495ce01d464c", "role": "ResolvedPipelineDefinition (resolved steps, scope ids, field provenance) and the per-axis CompilationSession." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", + "sha256": "358d3eaf5952170f9e9e2dc9056fb08707e8fcf74682063c074b367a83c2648f", "role": "CompiledStepPlan typed fields, including framework-local device assignment." }, { @@ -1815,7 +1821,7 @@ }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "b6a7231bc2964d472070731018f6a4935e5f23cf8f238a232045c32244e6545a", + "sha256": "74b591773b7f653c957b2db33a1262925e5c02b71074695b6fc18d6120ed5934", "role": "RuntimeValueStore recording, replacement, query, and observation semantics." }, { @@ -1825,7 +1831,7 @@ }, { "path": "external/python-introspect/src/python_introspect/jsonable.py", - "sha256": "8fe06533a6dce179b7d77792b6ac768c2a64a1e9e79bbe926893498fae035bb4", + "sha256": "0148e51fb2fc4c7e1c625b088cb0ca56fecd58f9836b9e5da58b75b5eb2d40ee", "role": "python-introspect's to_jsonable JSON encoder." }, { @@ -1842,15 +1848,29 @@ "path": "external/metaclass-registry/src/metaclass_registry/discovery.py", "sha256": "757958272d11633e7b854c7104c07f0d90845c37fee6ab0c3646fb14c2ed8daa", "role": "Registry discovery imports that preserve host-owned root logging." + }, + { + "path": "external/ObjectState/src/objectstate/scope_key.py", + "sha256": "d0dd6e56cebf413d6869b07c7b9d8668ba954ed5aeead36c70288920a5b412b2", + "role": "ScopeKey: the one scope-key grammar and separator." + }, + { + "path": "external/ObjectState/src/objectstate/codegen.py", + "sha256": "347d2697c061a35e6e440e373fe4e8bfce1ad92b500e98da07faa26f6a3a1356", + "role": "Lazy-configuration source formatters used when pipelines are saved as Python." + }, + { + "path": "external/pyqt-reactive/src/pyqt_reactive/windows/config_window.py", + "sha256": "2ee4f5c173b30b0a94a4288243d82ae66b6d65d12aa59493ad9363559966c1cd", + "role": "Generic configuration window that OpenHCSConfigWindow subclasses." } ], "findings": [ - "StepSnapshot was deleted: ResolvedPipelineDefinition now holds resolved steps with step scope ids and per-field provenance, and CompilationSession holds that definition instead of snapshot tuples and an ObjectState map. The flow diagram and the resolution/compilation sections described the deleted type and its index invariants.", - "Runtime slice projection is owned by each value family (RuntimeSliceProjectableValue), not by projection strategies.", - "python-introspect now also owns the JSON codec after openhcs/serialization/json.py was deleted (L2); the owner list omitted it." + "The package-owner list predated U3 and L6: it omitted ObjectState's ScopeKey grammar and lazy-config source formatting and pyqt-reactive's configuration windows, editable tables, grid selection and time-travel views." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/system_overview.rst in full and compared it against the changed authorities: pipeline_document.py, compilation_session.py, compiled_step_plan.py, runtime_stores.py, python_introspect/jsonable.py; the flow, ResolvedPipelineDefinition, CompilationSession, CompiledStepPlan typed fields and RuntimeValueStore claims still hold (diffs: generate_python_source, graph.validate_invocation_declarations, CompiledSourceBindingPlan() defaults). Updated the package-owner list for U3/L6: ObjectState now owns ScopeKey and objectstate.codegen, and pyqt-reactive owns configuration windows, editable tables, grid selection and time travel.", "On 2026-10-11 (B6 final), compared docs/source/architecture/system_overview.rst against B6's later changes to openhcs/core/pipeline/compilation_session.py, openhcs/core/compiled_step_plan.py, openhcs/core/runtime_stores.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/system_overview.rst against the later B6 changes to openhcs/core/pipeline/compilation_session.py, openhcs/core/compiled_step_plan.py, openhcs/core/runtime_stores.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/system_overview.rst in full and compared it against the changed authorities: pipeline_document.py, pipeline/compilation_session.py, compiled_execution.py, runtime_stores.py, compiled_step_plan.py, framework_device_assignment.py, pyproject.toml and the python-introspect/arraybridge/metaclass-registry owners; replaced StepSnapshot with ResolvedPipelineDefinition, restated the session invariant to what CompilationSession enforces, moved slice projection to value families, added the python-introspect JSON codec, and confirmed the CompiledStepPlan fields and bundle environment facts.", @@ -1890,25 +1910,26 @@ "authority": [ { "path": "pyproject.toml", - "sha256": "c3a2bea22aa6872940125f4aaac72c9eac72084fefc6c2cefdb54b5273361e4a", + "sha256": "b5d4d752704b886da20d860743b81dc3d2ce3a3c402208246fadc1c6cb71f597", "role": "No Textual dependency or entry point; openhcs-gui gui-script points at the PyQt application." }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "01dcb82b5850f3cdb1c300a2aa360ba330e7f15dffb5a24e37504311095090f9", + "sha256": "3cdb7819f09604046ba69e0b4f57b73e4e98d8f68af80158501a54bb42e701a2", "role": "PyQt desktop main window composition." }, { "path": "tests/unit/pyqt_gui/test_ui_config_lifecycle.py", - "sha256": "5fd4194e70de5210a6b51283693a1d25c6a6a1278b8a869c7deebcbd583503e0", + "sha256": "78c9d42d91a384d0b9bce1139dfcd171a2d87a8781be4c078e2f9703bf302043", "role": "Guards that openhcs/textual_tui does not return." } ], "findings": [ - "The deprecated Textual implementation remains removed; the page routes readers to the PyQt desktop. B6 re-check found the claims unchanged by the pyproject and main-window changes." + "Redirect still correct: no Textual interface in source or distribution; no page change needed." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/tui_system.rst in full and compared it against the changed authorities: pyproject.toml, openhcs/pyqt_gui/main.py, tests/unit/pyqt_gui/test_ui_config_lifecycle.py; pyproject.toml still has no Textual dependency and the openhcs-gui gui-script points at openhcs.pyqt_gui.__main__:main; main.py changes are U3 moves of generic Qt machinery; the lifecycle test still guards that openhcs/textual_tui does not return (its shortcut tests moved to pyqt-reactive); docs/archive still exists. No correction needed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/tui_system.rst against B6's later changes to openhcs/pyqt_gui/main.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/tui_system.rst against the later B6 changes to pyproject.toml, openhcs/pyqt_gui/main.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/tui_system.rst in full and compared it against the changed authorities: pyproject.toml, openhcs/pyqt_gui/main.py and tests/unit/pyqt_gui/test_ui_config_lifecycle.py; confirmed no Textual dependency, package or entry point exists, openhcs-gui still launches openhcs.pyqt_gui, the textual_tui guard test remains and docs/archive still exists; no prose change was needed.", @@ -1940,7 +1961,7 @@ }, { "path": "docs/source/architecture/ui_services_architecture.rst", - "source_sha256": "207497e08bf32257edc1eb223a012340d5e2b09e3772c6c9498b5413e599decc", + "source_sha256": "abc8843ed2629474f2d541424a237731f8da1ebd38eb21348e9740264e3c149b", "audience": [ "OpenHCS desktop, ObjectState, and UI-service maintainers" ], @@ -1959,7 +1980,7 @@ }, { "path": "openhcs/pyqt_gui/services/window_handlers.py", - "sha256": "1523b0874cbd4f7048d713afef84311c18291918c432bbbe7305488c1baa03a6", + "sha256": "6f05c7945f41b25e0b783231d51b508729ba0713911224bf46f84f89ea24823f", "role": "Registers OpenHCS domain window routes, including declaration-owned config ObjectState scopes and stable configuration-window presentation, over pyqt-reactive." }, { @@ -1969,8 +1990,8 @@ }, { "path": "tests/unit/pyqt_gui/test_ui_config_lifecycle.py", - "sha256": "5fd4194e70de5210a6b51283693a1d25c6a6a1278b8a869c7deebcbd583503e0", - "role": "Proves one absolute environment-selected UIConfig cache path and rejects relative paths." + "sha256": "78c9d42d91a384d0b9bce1139dfcd171a2d87a8781be4c078e2f9703bf302043", + "role": "Proves one absolute environment-selected UIConfig cache path and that UIConfig reaches the session's progress interval and transport." }, { "path": "external/pyqt-reactive/src/pyqt_reactive/services/ui_thread_dispatch.py", @@ -2009,7 +2030,7 @@ }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "01dcb82b5850f3cdb1c300a2aa360ba330e7f15dffb5a24e37504311095090f9", + "sha256": "3cdb7819f09604046ba69e0b4f57b73e4e98d8f68af80158501a54bb42e701a2", "role": "OpenHCSMainWindow.set_ui_config consumer reconciliation and rollback, and preview-policy propagation to both managers." }, { @@ -2064,18 +2085,33 @@ }, { "path": "openhcs/pyqt_gui/widgets/plate_manager.py", - "sha256": "bf77012db6c0984deebf09bc5d696d8ea8f273ff535ad964354b6ddbfe839791", + "sha256": "e816cdafd4795c07b5d177a04472221057230416311d1bf83c5bf65188d2aa00", "role": "Plate Manager renders the Session dataset list, holds no session state, and forwards transport and progress settings to the session." + }, + { + "path": "external/pyqt-reactive/src/pyqt_reactive/windows/config_window.py", + "sha256": "2ee4f5c173b30b0a94a4288243d82ae66b6d65d12aa59493ad9363559966c1cd", + "role": "ConfigWindow.save_config: validation and mutation guard before checkpoint, participant compensation in reverse, global-context then ObjectState restoration, post-commit notifications." + }, + { + "path": "openhcs/pyqt_gui/windows/config_window.py", + "sha256": "f72c0e481e35819f49749a7b82729e2f99ff200ebfd5bd6031efdb6f0585b24b", + "role": "OpenHCSConfigWindow supplies only the OpenHCS configuration-document format and window ids." + }, + { + "path": "external/pyqt-reactive/src/pyqt_reactive/services/application_shortcuts.py", + "sha256": "73ccd1ddebc60e90514c0b2b561e8e84cb3b8b5743a50318911a947823b0b2a5", + "role": "ShortcutLifecycle rejects duplicate keys before rebinding QActions and the application shortcut filter." } ], "findings": [ - "MainWindowAction and openhcs/agent/ui_bridge_actions.py were deleted (U1); main-window actions are SessionOperation classes projected by SessionOperationActionProvider.", - "openhcs/pyqt_gui/services/desktop_restart.py was deleted (D1/U5); DesktopRestartSession now lives in openhcs/desktop/update.py. Removed the 'accepted receipt' wording.", - "OpenHCSZMQConfig now subclasses zmqruntime's ExecutionTransportConfig, not ZMQConfig.", - "Plate Manager no longer owns selection or orchestrator delegation; it renders the Session and forwards transport/progress settings to it." + "The main-window shortcut lifecycle moved to pyqt-reactive (U3): MainWindowShortcutLifecycle was replaced by the generic ShortcutLifecycle and its application shortcut filter; the page now names that owner.", + "The configuration window moved to pyqt-reactive (U3): ConfigWindow there owns the two-tab save transaction and OpenHCSConfigWindow only supplies the OpenHCS document format and window ids; the page now says so and the authorities name both files.", + "test_ui_config_lifecycle.py no longer holds the shortcut-lifecycle tests (moved with the lifecycle); its role was narrowed to the UIConfig cache path and session transport/progress projection." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/ui_services_architecture.rst in full and compared it against the changed authorities: pyqt_gui/services/window_handlers.py (ScopeKey separator, OpenHCSConfigWindow over pyqt-reactive ConfigWindow, session event connection), tests/unit/pyqt_gui/test_ui_config_lifecycle.py (shortcut tests removed), pyqt_gui/main.py (pyqt-reactive ShortcutLifecycle, DockLayoutStore/DockPane, TimeTravelCommands/TimeTravelNavigator) and widgets/plate_manager.py (GuiEventBusBroadcaster removed, PathCacheKey and external-editor policy from pyqt-reactive); corrected the shortcut-lifecycle and configuration-window transaction owners and checked the save transaction order in pyqt-reactive ConfigWindow.save_config against the page's rollback description.", "On 2026-10-11 (B6 final), compared docs/source/architecture/ui_services_architecture.rst against B6's later changes to openhcs/pyqt_gui/services/ui_bridge_session_actions.py, openhcs/pyqt_gui/main.py, tests/unit/pyqt_gui/test_main_config_propagation.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/ui_services_architecture.rst against the later B6 changes to openhcs/pyqt_gui/services/ui_bridge_session_actions.py, openhcs/pyqt_gui/main.py, tests/unit/pyqt_gui/test_main_config_propagation.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/ui_services_architecture.rst in full and compared it against the changed authorities: runtime/zmq_config.py, services/window_handlers.py, pyqt_gui/main.py, ui_thread_dispatch.py, the two UI config tests, agent/ui_bridge_actions.py (deleted, U1) and services/desktop_restart.py (deleted); verified every named class exists, replaced MainWindowAction with session operations, moved DesktopRestartSession to openhcs/desktop/update.py, corrected the OpenHCSZMQConfig base, and restated Plate Manager as a renderer of Session state; UIConfig fields, set_ui_config ordering and application shutdown were confirmed in code.", @@ -2106,7 +2142,7 @@ "authority": [ { "path": "external/pyqt-reactive/src/pyqt_reactive/forms/parameter_form_manager.py", - "sha256": "a561e62e29af42630fa107b0929fc87b74007f7ed14ec8f0a6881c49070dc923", + "sha256": "2c59c3dc2f95a349970815a33c2afcf5158c97f4627973a8a0c6826f8ca9ae6c", "role": "Reusable parameter-form lifecycle and value projection." }, { @@ -2116,10 +2152,11 @@ } ], "findings": [ - "The transition page duplicated the OpenHCS boundary link and omitted the extracted owner document." + "Redirect still correct after U3: the pyqt-reactive owner document exists; no page change needed." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/widget_protocol_system.rst in full and compared it against the changed authorities: pyqt_reactive/forms/parameter_form_manager.py; the page states only that pyqt-reactive owns widget protocols; the linked owner document external/pyqt-reactive/docs/architecture/widget_protocol_system.rst still exists. No correction needed.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", "Sphinx resolved the current pyqt-reactive owner target", "Re-reviewed the public form-build completion, failure, and signal lifecycle against the complete dependent page and the updated pyqt-reactive owner documentation.", @@ -2143,7 +2180,7 @@ }, { "path": "openhcs/runtime/zmq_execution_server.py", - "sha256": "e24bdcb3b086cc60197ceb63f29a6f6b639a93cad175218941148a207eb285f2", + "sha256": "12e96bcd1fb1aaccb43aa3fe90697b5d1cdda8998721a556005e95f344c9b2b3", "role": "OpenHCS compiled-execution adapter over the generic execution server." }, { @@ -2158,7 +2195,7 @@ }, { "path": "external/zmqruntime/src/zmqruntime/messages.py", - "sha256": "ba7b9bac8f0202690f1e2bd17f445fe1741f43642f0a36e4a806bb577c2d1aa1", + "sha256": "fb257a7ac25a47d8c604fbfbd88c96b92fac16f914a61dab81c784fe99d95378", "role": "Declares endpoint application identity, generic exact compatibility, and member-owned execution lifecycle semantics." }, { @@ -2193,7 +2230,7 @@ }, { "path": "openhcs/runtime/zmq_execution_client.py", - "sha256": "147fb663d3ea60b1348063440e05b77d54b40634d1ae81cf4fb4f3b5457fbc99", + "sha256": "5df650f60c4d48cbf47c275d4e46b870b35c915ca8baa8e06e77f321578ca6d7", "role": "Spawns the execution server with the ArrayBridge-projected child environment and carries typed execution transport." }, { @@ -2208,7 +2245,7 @@ }, { "path": "openhcs/authoring/session/session.py", - "sha256": "8d74425f65ce9f39f88439ca90bd09c02678ae3aef8d3cf6795570986444add0", + "sha256": "0c4e0f0979e1793862859e47eefeb6dd9d4b51c99fba20c2ebc6adf11619c3b5", "role": "Session owns the one execution client and keeps FinishedExecution (request, endpoint, ExecutionRecord) per execution." }, { @@ -2248,7 +2285,7 @@ }, { "path": "openhcs/processing/backends/lib_registry/registry_service.py", - "sha256": "daef8925338fddf8504a9faa066af42456a13fe4602976bf64be06547a45fcc8", + "sha256": "f2e981157632adb5020828720ec26bfa6444ab52f545e1e71b1524242eaed6a3", "role": "Prepares the canonical catalogue in a child environment projected by ArrayBridge declarations." }, { @@ -2268,7 +2305,7 @@ }, { "path": "tests/unit/test_cellprofiler_processing_backend.py", - "sha256": "d5d53e859e04cafd3c815296e9c58c3642f9e788a45d5604ab54a9ce0c892c27", + "sha256": "111733bb0df819bb8bd9d170a1a6e2b48bd3acd343750a299937ead4d15539a8", "role": "Proves distinct persistent cache identities for CPU-only and GPU-capable function-catalogue admission contexts." }, { @@ -2342,16 +2379,10 @@ "role": "Owns the framework child environment consumed by catalogue preparation and server startup." } ], - "findings": [ - "openhcs/agent/services/execution_session_service.py and its headless job store were deleted (U1); headless MCP and the desktop now submit through Session/ExecutionSubmission. The retained-client, job-status and per-job cancellation paragraphs described the deleted service.", - "The per-job headless cancellation capability no longer exists; Session stop shuts down the endpoint and cancels active datasets, while the client keeps a cancel_execution control request.", - "plate_path/execution_plate_path were renamed subject_id/execution_subject_id (G7); the selected-pipeline rejection and read-path checks belonged to the deleted service and were dropped.", - "The submission budget is declared by zmqruntime's ExecutionTransportConfig, which OpenHCSZMQConfig now only extends with application names.", - "Replaced 'receipt' wording with plain 'evidence'.", - "Desktop restart and version-mismatch restart modules and their tests moved from openhcs/pyqt_gui/services to openhcs/desktop (D1/U5); authorities were remapped, and the agent-services test that covered the deleted job store was dropped." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/zmq_execution_service_extracted.rst in full and compared it against the changed authorities: runtime/zmq_execution_server.py and zmq_execution_client.py (K6 debug request codecs and replay predicate), external/zmqruntime messages.py (current file: EndpointApplication.compatibility_with, ExecutionRecord, ExecutionStatus), authoring/session/session.py (debug command classes), lib_registry/registry_service.py (contract read from reference.declaration) and test_cellprofiler_processing_backend.py (runtime_bound_parameter_names); verified EndpointConnectionAttempt, EndpointConnectionCancelledError, TransportEndpoint, OperationDeadline, ExecutionSubmission, ZMQAuxiliaryExecutionParams, FunctionCatalogPreparation, --prepare-capabilities, OperationCancellation and execution_subject_id still exist as described; no claim needed correction.", "On 2026-10-11 (B6 final), compared docs/source/architecture/zmq_execution_service_extracted.rst against B6's later changes to openhcs/authoring/session/session.py: the writability probe makes one exclusive create attempt, the installed demo reports ViewerType.value, the unit session pre-creates its QApplication, and the GUI journey waits for each batch to end. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/zmq_execution_service_extracted.rst against B6's later changes to openhcs/authoring/session/session.py: the session decides in-place initialization by an actual file-creation attempt that leaves nothing behind, its debug-session methods moved into an inherited mixin, the measured CLI treats Windows Ctrl+Break like Ctrl+C, and the Official30 viewer test reads typed payload summaries. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/zmq_execution_service_extracted.rst against B6's later changes to openhcs/runtime/zmq_execution_server.py, openhcs/runtime/zmq_execution_client.py, openhcs/authoring/session/submission.py, openhcs/authoring/session/session.py, openhcs/runtime/zmq_orchestrator_environment.py, tests/unit/test_zmq_submission_timeout.py, openhcs/processing/backends/lib_registry/registry_service.py, openhcs/processing/backends/lib_registry/unified_registry.py, openhcs/processing/backends/lib_registry/openhcs_registry.py, tests/unit/desktop/test_desktop_restart.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", @@ -2433,7 +2464,7 @@ }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "01dcb82b5850f3cdb1c300a2aa360ba330e7f15dffb5a24e37504311095090f9", + "sha256": "3cdb7819f09604046ba69e0b4f57b73e4e98d8f68af80158501a54bb42e701a2", "role": "Projects the configured endpoint snapshot into its dedicated right-hand indicator, and routes endpoint termination to the client lifecycle owner." }, { @@ -2463,16 +2494,14 @@ }, { "path": "openhcs/core/progress/runtime_tree.py", - "sha256": "016279f4135e2c9b10cb841e57ba0297c4197a195623f8a746ed90ab06533ce5", + "sha256": "75599ae64cf21ebbc2496c0a77b2e3b40b465f867cebc290066fc65c4513dff3", "role": "RuntimeExecutionTopology, RuntimeTreeProjectionBuilder and the subject/worker/partition/step/compilation node kinds with mean or explicit percent policies." } ], - "findings": [ - "openhcs/pyqt_gui/widgets/shared/services/zmq_client_service.py was moved into the session as openhcs/authoring/session/execution_client.py (U1); the progress subscriber is the Session's ZMQClientService, not the Plate Manager's.", - "G7 renamed the progress tree nodes: the execution path is subject -> worker -> partition -> step, not plate -> worker -> well -> step; aggregation policies were updated to match." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/zmq_server_browser_system.rst in full and compared it against the changed authorities: pyqt_gui/main.py (shortcut, dock and time-travel machinery moved to pyqt-reactive; endpoint indicator projection unchanged) and core/progress/runtime_tree.py (K6 ProgressStream classes replace channel helpers); verified every named class (ZMQServerBrowserWidgetABC, EndpointObservationSnapshot, RuntimeExecutionTopology, RuntimeTreeProjectionBuilder, ProgressTreeBuilder, ExecutionProgressProjection, ServerRowPresenter, LiveServerTreeSync, EndpointShutdownService, ZMQClientService, ProgressUIConfig, EndpointClientSession), the primary module paths, and mean/explicit percent node policies on subject/worker vs partition/step/compilation nodes; no claim needed correction.", "On 2026-10-11 (B6 final), compared docs/source/architecture/zmq_server_browser_system.rst against B6's later changes to openhcs/pyqt_gui/main.py, openhcs/core/progress/runtime_tree.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/zmq_server_browser_system.rst against the later B6 changes to openhcs/pyqt_gui/main.py, openhcs/core/progress/runtime_tree.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/zmq_server_browser_system.rst in full and compared it against the changed authorities: server_browser/progress_projection.py, server_browser/live_tree_sync.py, pyqt_gui/main.py, tests/unit/pyqt_gui/test_main_zmq_window_routing.py, zmqruntime client.py and services/zmq_client_service.py (moved, U1); verified every named class and module path exists, corrected the progress subscriber owner and the subject/worker/partition/step tree path and aggregation kinds from core/progress/runtime_tree.py, and confirmed the main window still projects the endpoint snapshot into its indicator and manager button states.", @@ -2589,7 +2618,7 @@ }, { "path": "docs/source/development/callable_artifact_authoring.rst", - "source_sha256": "06b84f60a5bb09683e64772fd564ad3478286fe6104747fa6c2410ddcb0ff3a8", + "source_sha256": "7a7502b1ad080497bf5e1490c6a1159e87a94a4b479250c9b02d7dd3cd7fbaa8", "audience": [ "OpenHCS callable and artifact extension authors" ], @@ -2598,8 +2627,8 @@ "authority": [ { "path": "openhcs/core/callable_contract.py", - "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", - "role": "Declares CallableContract fields (required_axis_roles, allowed_group_by_roles, memory roles), resolve_returned_output, the return partition and the artifact-parameter binding errors quoted by the page." + "sha256": "a8e73c5df804d78baf72710e210cb2ae63156657a074071e11fd868f4b19d3b4", + "role": "Declares CallableDeclaration (inherited by CallableContract) with required_axis_roles, allowed_group_by_roles, memory roles and prepared signatures, processing_prepare, resolve_returned_output, the return partition and the artifact-parameter binding errors quoted by the page." }, { "path": "openhcs/core/artifacts.py", @@ -2608,22 +2637,22 @@ }, { "path": "openhcs/core/processing_contracts.py", - "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", + "sha256": "eb0f7fd99ded1c4d5957152ccf6abe1468436c13574a1d1bb83ce368d24b9f69", "role": "Declares Pure2DContract and Pure3DContract used in the examples." }, { "path": "openhcs/core/pipeline/function_contracts.py", - "sha256": "f5e7c74ea3d2948c75388184263cd9f1f17905f820b180f634dfaa7514e1d526", - "role": "Declares artifact_inputs, artifact_outputs, special_inputs, execution_scope and runtime_bound_parameters." + "sha256": "ffc21dd3021a2316cfbd100c2d655867ce4bec619f2db7288e82aeb11c1930e8", + "role": "Declares artifact_inputs, artifact_outputs, special_inputs (which writes the artifact-fed parameter names), execution_scope and runtime_bound_parameters." }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "b6a7231bc2964d472070731018f6a4935e5f23cf8f238a232045c32244e6545a", + "sha256": "74b591773b7f653c957b2db33a1262925e5c02b71074695b6fc18d6120ed5934", "role": "Declares RuntimeArtifactBatch.records and StoredRuntimeValue for plate-scope callables." }, { "path": "openhcs/core/orchestrator/bundle_execution.py", - "sha256": "f5ba09a2ec7c688b1ca08167ab398cc976a9e2c39b301ae98f3c5fd560a25c85", + "sha256": "bc3a2d0a50c3db3be81b03f6af4a139c3db2f779e969aad608e59be66508ee4b", "role": "Requires a plate-scope callable to declare exactly one SpecialArtifactType output." }, { @@ -2633,7 +2662,7 @@ }, { "path": "openhcs/interop/cellprofiler/module_artifact_contracts.py", - "sha256": "982680cd36409ea6d90a07d4e60893b162847f5525ed516891e995877dc32fd1", + "sha256": "4d521778e83fee62a4b164d6a3934d0824f2c5e7375bc7bea1d74256e67d61fe", "role": "Declares CellProfilerModuleArtifactContracts resolving setting-backed specs into the invocation CallableContract." }, { @@ -2648,18 +2677,16 @@ }, { "path": "tests/unit/test_callable_contract.py", - "sha256": "cc220f53107a08b0406b6e0dd49896e3e4c3c0af6e527b0ef00d4fdaba8f0b2c", + "sha256": "9170f8694f388dbed5bfd36d882a5c0a4cdf28f50f51f7fdeaf4a955a9f3b412", "role": "Proves exact enum validation and contract-derived runtime-owned parameter exclusions." } ], "findings": [ - "The contract summary said contracts carry required variable components and allowed grouping; after G1 they carry required_axis_roles and allowed_group_by_roles.", - "Prose still named the deleted enum members PURE_2D and PURE_3D although the code examples already use Pure2DContract; after G5 the prose names the contract classes.", - "The rewritten special_inputs sentence was an overlong unwrapped line; rewrapped.", - "Replaced the jargon \"registration receipt\" with \"registration result\"." + "The contract summary listed a preparation hook among the contract's fields; K5 removed prepare from the declaration, so the hook stays on the callable (processing_prepare) and the contract carries only the prepared signatures. Reworded." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/callable_artifact_authoring.rst in full and compared it against the changed authorities: callable_contract.py (K5 CallableDeclaration, prepare removed, repeated_return_slots), processing_contracts.py, function_contracts.py (special_inputs now writes artifact_input_parameter_names), runtime_stores.py (K6 codec removal), bundle_execution.py (progress classes, lane results in compiled axis order), module_artifact_contracts.py (UniqueReturnSlots) and tests/unit/test_callable_contract.py; corrected the preparation-hook claim, confirmed by a direct call that special_inputs('labels') alone still fails with 'has no exact artifact declaration binding', and re-checked canonical/trailing_return_output_specs, RuntimeArtifactBatch.records, StoredRuntimeValue and the single SpecialArtifactType plate output.", "On 2026-10-11 (B6 final), compared docs/source/development/callable_artifact_authoring.rst against B6's later changes to openhcs/core/pipeline/function_contracts.py: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/development/callable_artifact_authoring.rst against B6's later changes to openhcs/core/callable_contract.py, openhcs/core/artifacts.py, openhcs/core/processing_contracts.py, openhcs/core/pipeline/function_contracts.py, openhcs/core/runtime_stores.py, openhcs/core/orchestrator/bundle_execution.py, openhcs/processing/materialization/core.py, openhcs/interop/cellprofiler/module_artifact_contracts.py, tests/unit/test_callable_contract.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/development/callable_artifact_authoring.rst against the later B6 changes to openhcs/core/callable_contract.py, openhcs/core/artifacts.py, openhcs/core/processing_contracts.py, openhcs/core/pipeline/function_contracts.py, openhcs/core/runtime_stores.py, openhcs/core/orchestrator/bundle_execution.py, openhcs/processing/materialization/core.py, openhcs/interop/cellprofiler/module_artifact_contracts.py, tests/unit/test_callable_contract.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -2689,22 +2716,22 @@ "authority": [ { "path": "openhcs/interop/cellprofiler/module_declarations.py", - "sha256": "6bbf41c158496a02439be2ee5aef12d7712fb632419c226a64c9259a55438b95", + "sha256": "b0195b66126c31f71dc0d4bb281a128e0164bb7dfb3b314ee8fe744ebc55d626", "role": "CellProfiler module registry, catalogue placement, and headless-import eligibility." }, { "path": "openhcs/interop/cellprofiler/module_artifact_contracts.py", - "sha256": "982680cd36409ea6d90a07d4e60893b162847f5525ed516891e995877dc32fd1", + "sha256": "4d521778e83fee62a4b164d6a3934d0824f2c5e7375bc7bea1d74256e67d61fe", "role": "Invocation-specific CellProfiler artifact contract derivation." }, { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "3439d96f3fbfcb104a53019dc150049021d9a8132bd27096f1ccf510eefbe37b", + "sha256": "b96f33b866fe6c9f8118ca70fcb7dff8e6a02cb29d79df67230a5e8b97df8002", "role": "Public CellProfiler pipeline lowering and declaration-owned headless-import validation." }, { "path": "tests/unit/test_cellprofiler_pipeline_import.py", - "sha256": "6b1d201ec5c4ba620fa2367a9cbef1f6fbdd25edbdda9bd7b723ebe3a3363412", + "sha256": "0ce4efd85d819c7e9f7fa84ec6f1ac7ec4bda024c9a8d5c7f264fda2730264f0", "role": "Proves interactive CellProfiler modules fail headless import instead of disappearing from the lowered pipeline." }, { @@ -2726,6 +2753,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/cellprofiler_module_authoring.rst in full and compared it against the changed authorities: module_declarations.py, module_artifact_contracts.py, pipeline_import.py and tests/unit/test_cellprofiler_pipeline_import.py (K4 moved SourceBindingsConfig/StepSourceBindingsConfig imports to openhcs.core.config and replaced SourceBindingMatchMethod.ORDER with PairByOrder; K5 put artifact_inputs/outputs directly on CallableContract and added repeated_return_slots through UniqueReturnSlots); the page names only SourceBindingsConfig, CellProfilerModule hooks, CellProfilerModuleArtifactContracts, SettingToKeywordBinding.input()/output(), module_blocks_for_invocation() and ModuleBlock, all re-checked in the current tree, and its claim that outputs are declared independently of return position still holds.", "On 2026-10-11 (B6 final), compared docs/source/development/cellprofiler_module_authoring.rst against B6's later changes to openhcs/interop/cellprofiler/module_declarations.py, openhcs/interop/cellprofiler/module_artifact_contracts.py, openhcs/interop/cellprofiler/pipeline_import.py, openhcs/processing/backends/lib_registry/openhcs_registry.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/development/cellprofiler_module_authoring.rst against the later B6 changes to openhcs/interop/cellprofiler/module_declarations.py, openhcs/interop/cellprofiler/module_artifact_contracts.py, openhcs/interop/cellprofiler/pipeline_import.py, openhcs/processing/backends/lib_registry/openhcs_registry.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/cellprofiler_module_authoring.rst in full and compared it against the changed authorities: module_declarations.py (axis-family imports, MeasurementDialect, lazy registry preparation, private _require_declared_callable with object-identity check, prepare_declared_callable), module_artifact_contracts.py (target_units_for_invocation replaces target_contract_for_invocation), pipeline_import.py, tests/unit/test_cellprofiler_pipeline_import.py and openhcs_registry.py (LazyDiscoveryDict declaration discovery, declared_callable_contract). Confirmed every name the page relies on still exists at its owner: AutoRegisterMeta, CellProfilerModule.__registry__, for_callable_contract (import identity then object identity), declared_function_names and the lazy CellProfilerBackendModule projection, validate_pipeline_import called by pipeline_import before lowering, CellProfilerModuleArtifactContracts, SettingToKeywordBinding.input/output, module_blocks_for_invocation, SourceBindingsConfig, ModuleBlock and CallableContract; no page change was needed.", @@ -2751,22 +2779,22 @@ "authority": [ { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", + "sha256": "0f9cc4819b53d55b7e9446429b8c8ac1ff1a59f4902020ea1658495ce01d464c", "role": "ResolvedPipelineDefinition and CompilationSession fields, one-time artifact admission before axis fan-out, and the missing-plan failure of plan(index)." }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "sha256": "4a33908fb42874e6d57eeea3f5305793ddbda41d13356f0a78501af38dcdfeb4", "role": "Compilation order, declaration resolution, callable warm-up, framework-local device assignment, and bundle construction." }, { "path": "openhcs/core/pipeline/path_planner.py", - "sha256": "677d106969aae4d818745bff29b6f84e487ec79a044c8eaf26bea1916236fc6c", + "sha256": "6ff14c63768ed5cbd77e84236b3980d62f616465e9ecfb7cdf27c63d8070a405", "role": "Compiled source satisfaction, artifact edge, path, grouping, and plan decisions." }, { "path": "tests/unit/test_compilation_session.py", - "sha256": "381adf3a87729928a225fd1db5a35f8c802340fb2a1e583501884dc5500f6b9d", + "sha256": "82f7227730a5285524dc9da39eb2d09f7af7ce1ee13285e5ebb0e7983d7e393b", "role": "Regression proof that axis sessions share one resolved pipeline definition while owning their own plans." }, { @@ -2776,7 +2804,7 @@ }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", + "sha256": "358d3eaf5952170f9e9e2dc9056fb08707e8fcf74682063c074b367a83c2648f", "role": "Typed per-step framework-device binding authority." }, { @@ -2795,11 +2823,10 @@ "role": "Regression proof for independent framework namespaces and exact selection." } ], - "findings": [ - "The Invariants section described StepSnapshot and a CompilationSession that validates steps, snapshots, states and plate scope; StepSnapshot was deleted. ResolvedPipelineDefinition now carries the enabled steps with ObjectState scope ids and provenance and admits artifact contracts once before axis fan-out, and CompilationSession binds it to one ProcessingContext, its plan map, source projection and dataset scope (CompilationDatasetScope). Rewrote the section to match." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/compiler_extension.rst in full and compared it against the changed authorities: compilation_session.py (disabled items dropped by normalisation; invocation declarations validated by the artifact graph), compiler.py (ScopeKey scope ids, progress phase/status classes, an empty CompiledSourceBindingPlan instead of None), path_planner.py (inject_metadata over the ArtifactGraph, source_bindings.compiled_plan), compiled_step_plan.py (empty plan defaults) and tests/unit/test_compilation_session.py (PairByOrder, admitted_graph); the page's ResolvedPipelineDefinition/CompilationSession invariants, plan(index) failure, typed-plan guidance and ArtifactSpecRef identity still hold.", "On 2026-10-11 (B6 final), compared docs/source/development/compiler_extension.rst against B6's later changes to openhcs/core/pipeline/compilation_session.py, openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/path_planner.py, tests/unit/test_compilation_session.py, openhcs/core/compiled_step_plan.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/development/compiler_extension.rst against the later B6 changes to openhcs/core/pipeline/compilation_session.py, openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/path_planner.py, tests/unit/test_compilation_session.py, openhcs/core/compiled_step_plan.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/compiler_extension.rst in full and compared it against the changed authorities: compilation_session.py (StepSnapshot and snapshot validation deleted; ResolvedPipelineDefinition and CompilationDatasetScope added), compiler.py, path_planner.py, tests/unit/test_compilation_session.py, arraybridge types.py, compiled_step_plan.py and framework_device_assignment.py. Rewrote the Invariants section for the current session model; confirmed CompiledStepPlan, ArtifactSpecRef, assign_framework_devices, FrameworkDeviceResolver and the absence of a global GPU scheduler.", @@ -2886,7 +2913,7 @@ }, { "path": "pyproject.toml", - "sha256": "c3a2bea22aa6872940125f4aaac72c9eac72084fefc6c2cefdb54b5273361e4a", + "sha256": "b5d4d752704b886da20d860743b81dc3d2ce3a3c402208246fadc1c6cb71f597", "role": "Current Python support, dependency, package discovery, and entry-point authority." }, { @@ -2896,12 +2923,12 @@ }, { "path": "external/python-introspect/src/python_introspect/jsonable.py", - "sha256": "8fe06533a6dce179b7d77792b6ac768c2a64a1e9e79bbe926893498fae035bb4", + "sha256": "0148e51fb2fc4c7e1c625b088cb0ca56fecd58f9836b9e5da58b75b5eb2d40ee", "role": "singledispatch to_jsonable projection for dataclasses, mappings, sequences, callables, registered types, enums, paths and scalars, paired with dataclass_from_mapping." }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", + "sha256": "0f9cc4819b53d55b7e9446429b8c8ac1ff1a59f4902020ea1658495ce01d464c", "role": "ResolvedPipelineDefinition and CompilationSession consumed by compiler phases." }, { @@ -2911,12 +2938,12 @@ }, { "path": "openhcs/core/callable_contract.py", - "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", + "sha256": "a8e73c5df804d78baf72710e210cb2ae63156657a074071e11fd868f4b19d3b4", "role": "OpenHCS callable declaration projection for input, output, and execution frameworks." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", + "sha256": "358d3eaf5952170f9e9e2dc9056fb08707e8fcf74682063c074b367a83c2648f", "role": "Typed per-step framework-device binding authority." }, { @@ -2940,13 +2967,10 @@ "role": "Proves registered CLI derivation and declaration-derived receipt construction." } ], - "findings": [ - "The Compiler/runtime split said compiler phases consume steps through StepSnapshot, which was deleted; they now consume ResolvedPipelineDefinition and CompilationSession.", - "The Serialization boundaries section still attributed to_jsonable to OpenHCS after openhcs/serialization/json.py was deleted in favour of python-introspect's to_jsonable/dataclass_from_mapping pair (L2); the owner table had already been updated but the prose contradicted it.", - "The record cited the deleted openhcs/core/registry_strategies.py and openhcs/serialization/json.py; their owners are now metaclass_registry/strategies.py and python_introspect/jsonable.py." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/extension_workflows.rst in full and compared it against the changed authorities: pyproject.toml (first-party floors only), external/python-introspect/src/python_introspect/jsonable.py (current file: singledispatch registrations for dataclasses, mappings, sequences, callables, AutoRegisterMeta families, enums, paths and scalars, plus registered_class_from_key), openhcs/core/pipeline/compilation_session.py (validation moved onto ArtifactGraph), openhcs/core/callable_contract.py (K5: CallableMetadata folded into CallableContract declared fields; input/output/execution memory types remain) and openhcs/core/compiled_step_plan.py (CompiledSourceBindingPlan() default). Rechecked every class the owner table and prose names (CallableContract, CellProfilerModule, ImageFileFormat, ResolvedPipelineDefinition, CompilationSession, CompiledExecutionBundle, CompiledStepPlan device bindings, BenchmarkCliCommand, ComparisonSuiteRunDeclaration/Receipt, MeasuredPipelineRunReceipt) and that pycodify still owns Python source generation after L6 (ObjectState adds only the lazy-config formatter); no claim changes.", "On 2026-10-11 (B6 final), compared docs/source/development/extension_workflows.rst against B6's later changes to benchmark/cellprofiler_benchmark_cli.py: the session decides in-place initialization by an actual file-creation attempt that leaves nothing behind, its debug-session methods moved into an inherited mixin, the measured CLI treats Windows Ctrl+Break like Ctrl+C, and the Official30 viewer test reads typed payload summaries. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/development/extension_workflows.rst against B6's later changes to openhcs/core/pipeline/compilation_session.py, openhcs/core/callable_contract.py, openhcs/core/compiled_step_plan.py, benchmark/cellprofiler_benchmark_cli.py, tests/unit/test_benchmark_control.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/development/extension_workflows.rst against the later B6 changes to pyproject.toml, openhcs/core/pipeline/compilation_session.py, openhcs/core/callable_contract.py, openhcs/core/compiled_step_plan.py, benchmark/cellprofiler_benchmark_cli.py, tests/unit/test_benchmark_control.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -2993,7 +3017,7 @@ }, { "path": "pyproject.toml", - "sha256": "c3a2bea22aa6872940125f4aaac72c9eac72084fefc6c2cefdb54b5273361e4a", + "sha256": "b5d4d752704b886da20d860743b81dc3d2ce3a3c402208246fadc1c6cb71f597", "role": "Current Python support, dependency, package discovery, and entry-point authority." }, { @@ -3002,16 +3026,17 @@ "role": "Executable authority for accepted integration-test option names and values." }, { - "path": "tests/conftest.py", - "sha256": "82d01adebe952964b61c0cad85024d0b3ba60a4a6dba1a11f6aa715a9f046664", - "role": "Test-harness parsing of OPENHCS_CPU_ONLY, which accepts only 'true' when skipping pytest-qt." + "path": "openhcs/utils/environment.py", + "sha256": "2d8738e593b3e92c966089772624f47d17d38bb27fb90974707af659c91e8ad8", + "role": "OpenHCSProcessEnvironment.flag_enabled, which tests/conftest.py uses to read OPENHCS_CPU_ONLY (1, true, yes or on) before deciding whether to load pytest-qt." } ], "findings": [ - "The focused-test and comparison commands set OPENHCS_CPU_ONLY=1; tests/conftest.py only recognises the value 'true' when deciding whether to load pytest-qt, so the commands now use OPENHCS_CPU_ONLY=true, which both the test harness and OpenHCSProcessEnvironment accept." + "The record's tests/conftest.py role said the harness accepts only 'true' for OPENHCS_CPU_ONLY; conftest now reads the flag through OpenHCSProcessEnvironment.cpu_only_mode, which accepts 1, true, yes or on. The page's OPENHCS_CPU_ONLY=true commands remain correct; the authority now names openhcs/utils/environment.py, which owns that parsing." ], - "disposition": "revise", + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/git_worktree_testing.rst in full and compared it against the changed authorities: pyproject.toml (first-party floors moved to zmqruntime 0.6.2, pycodify 0.2, objectstate 2.1.1, python-introspect 0.2.7, pyqt-reactive 0.5); the page states no version floors. Rechecked the eight editable installs against .gitmodules, the dev and gui extras, and the integration options file; corrected the stale conftest role (OPENHCS_CPU_ONLY parsing now lives in OpenHCSProcessEnvironment).", "On 2026-10-11 (B6 final), compared docs/source/development/git_worktree_testing.rst against B6's later changes to tests/conftest.py: the writability probe makes one exclusive create attempt, the installed demo reports ViewerType.value, the unit session pre-creates its QApplication, and the GUI journey waits for each batch to end. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/development/git_worktree_testing.rst against B6's later changes to tests/conftest.py: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/development/git_worktree_testing.rst against the later B6 changes to pyproject.toml: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -3050,7 +3075,7 @@ "authority": [ { "path": "pyproject.toml", - "sha256": "c3a2bea22aa6872940125f4aaac72c9eac72084fefc6c2cefdb54b5273361e4a", + "sha256": "b5d4d752704b886da20d860743b81dc3d2ce3a3c402208246fadc1c6cb71f597", "role": "Current Python support, dependency, package discovery, and entry-point authority." }, { @@ -3059,11 +3084,10 @@ "role": "Recorded first-party submodule set behind the recursive-clone and install-extracted-packages-first instruction." } ], - "findings": [ - "The record cited openhcs/core/compiled_execution.py and the page itself as authorities although the page makes no compiled-bundle claim and a page cannot prove its own claims; replaced them with .gitmodules, which owns the recursive clone and extracted-package statement. Page text needed no change." - ], + "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/index.rst in full and compared it against the changed authorities: pyproject.toml (first-party dependency floors only); the page states no versions. Confirmed every toctree target and related-architecture link exists and the recursive-clone guidance matches .gitmodules; no claim changes.", "On 2026-10-11 (B6 final), compared docs/source/development/index.rst against the later B6 changes to pyproject.toml: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/index.rst in full and compared it against the changed authorities: pyproject.toml (first-party dependency floors, JAX extra, openhcs-basicpy) and openhcs/core/compiled_execution.py; the page states neither versions nor compiled-bundle facts. Confirmed every toctree target exists and the recursive-clone guidance matches .gitmodules; dropped the unrelated compiled_execution.py and self-referential page authorities.", "Re-read docs/source/development/index.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -3215,7 +3239,7 @@ }, { "path": "openhcs/authoring/session/session.py", - "sha256": "8d74425f65ce9f39f88439ca90bd09c02678ae3aef8d3cf6795570986444add0", + "sha256": "0c4e0f0979e1793862859e47eefeb6dd9d4b51c99fba20c2ebc6adf11619c3b5", "role": "Session submission budget and runtime-observation export validation (writable root, no existing path, outcomes require a path)." }, { @@ -3250,7 +3274,7 @@ }, { "path": "openhcs/runtime/zmq_execution_client.py", - "sha256": "147fb663d3ea60b1348063440e05b77d54b40634d1ae81cf4fb4f3b5457fbc99", + "sha256": "5df650f60c4d48cbf47c275d4e46b870b35c915ca8baa8e06e77f321578ca6d7", "role": "Owns the total headless submission budget and the pre-dispatch timeout boundary documented for MCP developers." }, { @@ -3270,7 +3294,7 @@ }, { "path": "tests/unit/pyqt_gui/test_ui_agent_bridge.py", - "sha256": "de6758859d9743e826369162855327e4a84056cafdd7ddfdc33868d2a3093353", + "sha256": "67cdad5133dc83ccb005b58cc56535ae89e77fc23d3ebd76e616c0da5f567b44", "role": "Proves stale protocol and application versions fail before UI operation dispatch." }, { @@ -3284,13 +3308,10 @@ "role": "Declares OPENHCS_UI_BRIDGE_DESCRIPTOR, OPENHCS_UI_BRIDGE_DESCRIPTOR_DIR and the UIConfigCacheEnvironment override forwarded to the development MCP child." } ], - "findings": [ - "The record cited openhcs/agent/services/execution_session_service.py, deleted when headless MCP became a session (880cb2c55, U1); the page had already been updated to describe SessionOperation tools, event_sequence, openhcs_session_events and openhcs_session_datasets, so their owners (capabilities.py, authoring/session/session.py, agent/dto/session.py) replace it.", - "The record still cited the asynchronous-job progress-observation owners (zmqruntime progress_stream.py, agent/dto/execution.py, tests/unit/agent/test_agent_services.py) after the page dropped its openhcs_get_execution_status paragraph; they were removed from the authorities.", - "The execution-submission deadline the page attributes to OpenHCSZMQConfig is now declared on the inherited zmqruntime ExecutionTransportConfig; the zmqruntime config was added as its owner. Page text needed no further change." - ], + "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/mcp_development.rst in full and compared it against the changed authorities: openhcs/authoring/session/session.py, openhcs/runtime/zmq_execution_client.py and tests/unit/pyqt_gui/test_ui_agent_bridge.py (K6: debug commands, boundaries and progress phases/statuses became class families; debug control requests send request.to_dict() directly; U3: DockPane and ConfigCodeDocumentDriver moved to pyqt-reactive). The page names none of those types; rechecked the openhcs-mcp-dev entry point, every dev-client command and option it shows, the SessionOperation operation_ids behind the six named tools, openhcs_session_events/openhcs_session_datasets, execution_submission_timeout_ms on OpenHCSZMQConfig's zmqruntime base, UIConfigCacheEnvironment, the bridge/cache environment variables and OPENHCS_MCP_VERBOSE; no claim changes.", "On 2026-10-11 (B6 final), compared docs/source/development/mcp_development.rst against B6's later changes to openhcs/authoring/session/session.py: the writability probe makes one exclusive create attempt, the installed demo reports ViewerType.value, the unit session pre-creates its QApplication, and the GUI journey waits for each batch to end. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/development/mcp_development.rst against B6's later changes to openhcs/agent/dto/session.py: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/development/mcp_development.rst against B6's later changes to openhcs/authoring/session/session.py: the session decides in-place initialization by an actual file-creation attempt that leaves nothing behind, its debug-session methods moved into an inherited mixin, the measured CLI treats Windows Ctrl+Break like Ctrl+C, and the Official30 viewer test reads typed payload summaries. No claim this page makes changes.", @@ -3755,13 +3776,14 @@ }, { "path": "external/pyqt-reactive/src/pyqt_reactive/forms/parameter_form_manager.py", - "sha256": "a561e62e29af42630fa107b0929fc87b74007f7ed14ec8f0a6881c49070dc923", + "sha256": "2c59c3dc2f95a349970815a33c2afcf5158c97f4627973a8a0c6826f8ca9ae6c", "role": "Reusable parameter-form lifecycle and value projection." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/parameter_analysis_audit.rst in full and compared it against the changed authority external/pyqt-reactive/src/pyqt_reactive/forms/parameter_form_manager.py (current file; ParameterFormManager and FormManagerConfig still own the generic Qt form lifecycle in pyqt-reactive). Confirmed signature and callable-declaration analysis remain in python-introspect, no Textual import remains in openhcs/, and the archived audit path exists; no claim changes.", "On 2026-10-11 (B6), re-read docs/source/development/parameter_analysis_audit.rst in full and compared it against the changed authorities: pyproject.toml (dependency floors only) and external/python-introspect/src/python_introspect/signature_analyzer.py (current file); signature and docstring analysis still lives in python-introspect, generic forms in pyqt-reactive, no Textual UI remains in openhcs/, and the archived audit path exists. Dropped pyproject.toml because the orphan redirect states no package facts.", "Re-read docs/source/development/parameter_analysis_audit.rst in full against repository base bb4218fae and the current shared worktree; challenged its reference user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", "Compared every factual claim directly with the current hash-pinned authorities: pyproject.toml; external/python-introspect/src/python_introspect/signature_analyzer.py; external/pyqt-reactive/src/pyqt_reactive/forms/parameter_form_manager.py.", @@ -3801,7 +3823,7 @@ "authority": [ { "path": "openhcs/core/debug.py", - "sha256": "95d365afefc46ee50da5abb8d5d246b3980feb32b85839f91e874cb7e689caaa", + "sha256": "4a1ed9fb0dcd40beeda9a2a3c90cdb5b2541252590d28281a9fb6f1aa021c510", "role": "Nominal debug command, boundary, snapshot, replay, and paused-worker declarations." }, { @@ -3811,18 +3833,19 @@ }, { "path": "openhcs/agent/services/runtime_server_service.py", - "sha256": "ec521df2fbea765dedda5a64dbdbde9537cfa931823eace795d3aca86905fff1", + "sha256": "f34c26b3c8110e85b4322591a43dff9d81da4c5cf3f48422c065b0d069ee6d6d", "role": "Bounded runtime-server and paused-debug inspection service." }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "sha256": "4a33908fb42874e6d57eeea3f5305793ddbda41d13356f0a78501af38dcdfeb4", "role": "Compilation order, declaration resolution, callable warm-up, framework-local device assignment, and bundle construction." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/pipeline_debugging_guide.rst in full and compared it against the changed authorities: openhcs/core/debug.py (K6: debug commands, event boundaries and replay modes became class families such as StepDebugCommand and AfterInvocationDebugBoundary; control payload wrappers deleted), openhcs/agent/services/runtime_server_service.py (command decoded through registered_class_from_key, worker state read from the typed status) and openhcs/core/pipeline/compiler.py (ScopeKey scope ids, progress phase/status classes, CompiledSourceBindingPlan() default). The page names none of the changed types; rechecked debug_pause on AbstractStep, the 'Run to Pause', Step and 'Inspect Runtime' action labels, pipeline_debug_toolbar.session, openhcs_inspect_pipeline_source_artifact_plan, openhcs_inspect_debug_runtime_values / runtime-debug-values, SettingToKeywordBinding and StepMaterializationConfig; no claim changes.", "On 2026-10-11 (B6 final), compared docs/source/development/pipeline_debugging_guide.rst against B6's later changes to openhcs/core/debug.py, openhcs/core/pipeline/compiler.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/development/pipeline_debugging_guide.rst against the later B6 changes to openhcs/core/debug.py, openhcs/core/pipeline/compiler.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/pipeline_debugging_guide.rst in full and compared it against the changed authorities: openhcs/core/debug.py (DebugJsonCodec removed, DebugProgressContext now a declared progress-context payload, plate_id renamed subject_id), openhcs/agent/services/runtime_server_service.py and openhcs/core/pipeline/compiler.py (CompilationPlateScope and plate_path renamed to dataset scope and dataset_root). The page names none of the renamed fields; confirmed openhcs_inspect_pipeline_source_artifact_plan, openhcs_inspect_debug_runtime_values and its runtime-debug-values command, debug_session_id, pipeline_debug_toolbar, FunctionStep.debug_pause, the Run to Pause, Step and Inspect Runtime actions, StepMaterializationConfig, RuntimeValueStore, CompiledStepPlan, CompilationSession and CompiledExecutionBundle still exist with the described roles.", @@ -3875,12 +3898,12 @@ }, { "path": "openhcs/core/processing_contracts.py", - "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", + "sha256": "eb0f7fd99ded1c4d5957152ccf6abe1468436c13574a1d1bb83ce368d24b9f69", "role": "ProcessingContract class family including Pure3DContract used in the example." }, { "path": "openhcs/core/callable_contract.py", - "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", + "sha256": "a8e73c5df804d78baf72710e210cb2ae63156657a074071e11fd868f4b19d3b4", "role": "Compiler-visible projection of all declared callable memory roles." }, { @@ -3890,7 +3913,7 @@ }, { "path": "openhcs/core/steps/function_runtime.py", - "sha256": "b40ebb069565573c340e4fd95ce220474d27f200396ec6d9c797bf716b2c0893", + "sha256": "fa9486cddf3db943a218f276bb43d079f505bf84b18d9f7bd074f7a4d86332b3", "role": "Runtime conversion and invocation inside the compiled execution-framework device scope." }, { @@ -3900,13 +3923,14 @@ }, { "path": "tests/unit/test_unified_registry_runtime_payloads.py", - "sha256": "76e02cc26b40f5af984d31e874eab1b18d334259fb5cd44bc9fe5945c474b139", + "sha256": "a817850fc2d84f82da3c5e60d2ae33d461d795b36d65816955a4c8b18bd9b852", "role": "Proves pyclesperanto output buffers and device handles remain runtime-owned while algorithm parameters stay authored." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/pyclesperanto_simple_implementation.rst in full and compared it against the changed authorities: openhcs/core/processing_contracts.py and openhcs/core/callable_contract.py (K5: CallableMetadata replaced by CallableDeclaration, which CallableContract inherits), openhcs/core/steps/function_runtime.py (K6 debug boundary classes; main-flow source-binding plan no longer None) and tests/unit/test_unified_registry_runtime_payloads.py. The page names none of the changed types; rechecked the pyclesperanto decorator re-exported from openhcs.core.memory, Pure3DContract, and PyclesperantoRegistry's runtime-owned output_image, output_image_matrix and device parameters; no claim changes.", "On 2026-10-11 (B6 final), compared docs/source/development/pyclesperanto_simple_implementation.rst against B6's later changes to openhcs/processing/backends/processors/pyclesperanto_processor.py, openhcs/core/memory/decorators.py, openhcs/core/processing_contracts.py, openhcs/core/callable_contract.py, openhcs/core/steps/function_runtime.py, tests/unit/test_unified_registry_runtime_payloads.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/development/pyclesperanto_simple_implementation.rst against the later B6 changes to openhcs/processing/backends/processors/pyclesperanto_processor.py, openhcs/core/memory/decorators.py, openhcs/core/processing_contracts.py, openhcs/core/callable_contract.py, openhcs/core/steps/function_runtime.py, tests/unit/test_unified_registry_runtime_payloads.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/pyclesperanto_simple_implementation.rst in full and compared it against the changed authorities: the page itself (example already moved from ProcessingContract.PURE_3D to the Pure3DContract class, G5), pyclesperanto_processor.py (shared operations moved to image_stack_operations), arraybridge types.py and decorators.py, openhcs/core/memory/decorators.py, callable_contract.py, function_runtime.py and test_unified_registry_runtime_payloads.py. Confirmed the example imports (openhcs.core.memory.pyclesperanto, openhcs.core.processing_contracts.Pure3DContract), that the OpenHCS decorator accepts a ProcessingContract subclass as contract, and the three runtime-owned PyclesperantoRegistry parameters; added processing_contracts.py as the owner of the example contract.", @@ -3938,13 +3962,14 @@ }, { "path": "pyproject.toml", - "sha256": "c3a2bea22aa6872940125f4aaac72c9eac72084fefc6c2cefdb54b5273361e4a", + "sha256": "b5d4d752704b886da20d860743b81dc3d2ce3a3c402208246fadc1c6cb71f597", "role": "Current Python support, dependency, package discovery, and entry-point authority." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/repository_setup.rst in full and compared it against the changed authorities: pyproject.toml (first-party floors moved to zmqruntime 0.6.2, pycodify 0.2, objectstate 2.1.1, python-introspect 0.2.7, pyqt-reactive 0.5); the page states no version floors. Rechecked the eight submodules against .gitmodules, requires-python >=3.11,<3.15, the named extras (dev, gui, dev-gui, cellprofiler-compat, viz, gpu, docs), and that OPENHCS_CPU_ONLY=1 is accepted by OpenHCSProcessEnvironment.flag_enabled; no claim changes.", "On 2026-10-11 (B6 final), compared docs/source/development/repository_setup.rst against the later B6 changes to pyproject.toml: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/repository_setup.rst in full and compared it against the changed authorities: pyproject.toml (first-party floors moved to zmqruntime 0.6, objectstate 1.3.1, python-introspect 0.2.5, metaclass-registry 0.3, arraybridge 0.4.1, polystore 0.6, pyqt-reactive 0.4; JAX GPU extra now jax[cuda12-local]; openhcs-basicpy added as an ordinary dependency); challenged the eight-submodule count against .gitmodules, the 3.11-3.14 range against requires-python, the centrosome parity oracle's python_version < 3.14 marker, the named extras (dev, gui, dev-gui, cellprofiler-compat, viz, gpu, docs), setup.py deferring metadata to pyproject.toml, and the CUDA 12 statement for the gpu extra; all remain accurate and the page states no version floors.", "Re-read docs/source/development/repository_setup.rst in full against repository base bb4218fae and the current shared worktree; challenged its how-to user need for accuracy, freshness, duplication, owner boundaries, flow, and navigation.", @@ -3996,12 +4021,12 @@ }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", + "sha256": "0f9cc4819b53d55b7e9446429b8c8ac1ff1a59f4902020ea1658495ce01d464c", "role": "Resolved step/session identity and typed plan assembly invariants." }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "b6a7231bc2964d472070731018f6a4935e5f23cf8f238a232045c32244e6545a", + "sha256": "74b591773b7f653c957b2db33a1262925e5c02b71074695b6fc18d6120ed5934", "role": "RuntimeValueStore recording, replacement, query, and observation semantics." }, { @@ -4010,11 +4035,10 @@ "role": "Deletion gate proving obsolete parallel configuration and browser modules remain absent." } ], - "findings": [ - "The registry-discipline section pointed generic consumers at openhcs.core.registry_strategies, which commit b221f8491 deleted; the enum, nominal-type and most-derived strategy mixins are now imported from metaclass_registry.strategies, and the page names that module." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/respecting_codebase_architecture.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compilation_session.py (per-invocation validation moved onto ArtifactGraph.validate_invocation_declarations; strip_disabled_functions no longer applied) and openhcs/core/runtime_stores.py (hand-written to_dict/from_dict transport on RuntimeArtifactLocation and RuntimeArtifactAddress deleted). Both moves agree with the page's single-owner guidance; rechecked that the three strategy mixins, AutoRegisterMeta, RegistryConfig and RegistryFamily still exist in metaclass-registry; no claim changes.", "On 2026-10-11 (B6 final), compared docs/source/development/respecting_codebase_architecture.rst against B6's later changes to openhcs/core/pipeline/compilation_session.py, openhcs/core/runtime_stores.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/development/respecting_codebase_architecture.rst against the later B6 changes to openhcs/core/pipeline/compilation_session.py, openhcs/core/runtime_stores.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/respecting_codebase_architecture.rst in full and compared it against the changed authorities: openhcs/core/registry_strategies.py (deleted in b221f8491), openhcs/core/pipeline/compilation_session.py, openhcs/core/runtime_stores.py, tests/unit/test_configuration_mirror_deletion.py; confirmed the three strategy mixins and the registry types named in the search command exist in metaclass-registry, replaced the deleted openhcs.core.registry_strategies reference with metaclass_registry.strategies, and found the typed-plan, RuntimeValueStore and mirror-deletion claims still accurate.", @@ -4036,7 +4060,7 @@ "authority": [ { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", + "sha256": "0f9cc4819b53d55b7e9446429b8c8ac1ff1a59f4902020ea1658495ce01d464c", "role": "Declares ResolvedPipelineDefinition and CompilationSession, the resolved declaration view and per-context compiler boundary named in the assembly order." }, { @@ -4046,7 +4070,7 @@ }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "b6a7231bc2964d472070731018f6a4935e5f23cf8f238a232045c32244e6545a", + "sha256": "74b591773b7f653c957b2db33a1262925e5c02b71074695b6fc18d6120ed5934", "role": "RuntimeValueStore recording, replacement, query, and observation semantics." }, { @@ -4056,12 +4080,12 @@ }, { "path": "openhcs/core/callable_contract.py", - "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", + "sha256": "a8e73c5df804d78baf72710e210cb2ae63156657a074071e11fd868f4b19d3b4", "role": "Callable ABI and input/output/execution memory-type declaration authority." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", + "sha256": "358d3eaf5952170f9e9e2dc9056fb08707e8fcf74682063c074b367a83c2648f", "role": "Typed per-step source-binding plan and framework-device decisions." }, { @@ -4070,13 +4094,10 @@ "role": "Compiler projection from declared framework footprint to exact step bindings." } ], - "findings": [ - "The assembly order named StepSnapshot, which no longer exists; compilation now runs from ResolvedPipelineDefinition into CompilationSession.", - "The CompiledExecutionBundle entry listed worker-safe steps and plans; the bundle now holds runtime contexts, pickle-safe transport contexts, worker assignments and the runtime environment.", - "The RuntimeSliceProjection entry described a nominal strategy registry with aggregation; the strategy family was removed, owned values declare slice behaviour through RuntimeSliceProjectableValue, and no aggregation exists there." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/runtime_system_assembly_rules.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compilation_session.py (per-invocation validation moved onto ArtifactGraph), openhcs/core/runtime_stores.py (hand-written address/location to_dict/from_dict deleted), openhcs/core/callable_contract.py (K5: CallableContract now inherits CallableDeclaration and its declared fields directly; memory roles, artifact inputs/outputs and execution scope remain on the contract) and openhcs/core/compiled_step_plan.py (CompiledSourceBindingPlan() default). Rechecked CellProfilerModuleArtifactContracts, CompiledExecutionBundle's runtime/transport contexts, worker assignments and runtime environment, SourceBindingsConfig, ArtifactSpecRef and ArtifactKey; no claim changes.", "On 2026-10-11 (B6 final), compared docs/source/development/runtime_system_assembly_rules.rst against B6's later changes to openhcs/core/pipeline/compilation_session.py, openhcs/core/runtime_stores.py, openhcs/core/callable_contract.py, openhcs/core/compiled_step_plan.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/development/runtime_system_assembly_rules.rst against the later B6 changes to openhcs/core/pipeline/compilation_session.py, openhcs/core/runtime_stores.py, openhcs/core/callable_contract.py, openhcs/core/compiled_step_plan.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/runtime_system_assembly_rules.rst in full and compared it against the changed authorities: compilation_session.py, compiled_execution.py, runtime_stores.py, runtime_slice_projection.py, callable_contract.py, compiled_step_plan.py, framework_device_assignment.py; replaced the deleted StepSnapshot with ResolvedPipelineDefinition, corrected the CompiledExecutionBundle field summary and the RuntimeSliceProjection description, and confirmed CallableContract memory roles, CellProfilerModuleArtifactContracts, SourceBindingsConfig, CompiledSourceUniversePlan, ArtifactSpecRef and ArtifactKey still exist as described.", @@ -4115,21 +4136,19 @@ }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "b6a7231bc2964d472070731018f6a4935e5f23cf8f238a232045c32244e6545a", + "sha256": "74b591773b7f653c957b2db33a1262925e5c02b71074695b6fc18d6120ed5934", "role": "RuntimeValueStore recording, replacement, query, and observation semantics." }, { "path": "tests/unit/test_runtime_value_store.py", - "sha256": "4ea189341b761bda993eaa2b6e24b83917568d047f4834d9f44410f7545d1452", + "sha256": "d20ae0d5160c79ba7b7c3291a052a606b3ed7dee3c0cd84a6cf013e17a127a23", "role": "Regression proof for typed runtime recording, replacement, queries, and failure behaviour." } ], - "findings": [ - "Step 3 told authors to register a RuntimeSliceProjectionStrategy; that strategy family was deleted. Payloads now subclass RuntimeSliceProjectableValue (or RuntimeSliceIndexedValue / RuntimeSliceInvariantValue) and RuntimeSliceProjection handles only foreign values.", - "The page claimed projection strategies are chosen by MRO specificity and that slices are aggregated; neither holds for the current projection boundary." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/runtime_value_extension.rst in full and compared it against the changed authorities: openhcs/core/runtime_stores.py and tests/unit/test_runtime_value_store.py (hand-written to_dict/from_dict on RuntimeArtifactLocation and RuntimeArtifactAddress deleted; round trips now use python_introspect to_jsonable and dataclass_from_mapping; CompiledSourceBindingPlan() replaces .empty()). The page names none of those methods and its 'add transport only where needed' guidance still holds; rechecked RuntimeSliceProjectableValue and its indexed/invariant subclasses, RuntimeSliceProjection and RuntimeValueStore; no claim changes.", "On 2026-10-11 (B6 final), compared docs/source/development/runtime_value_extension.rst against B6's later changes to openhcs/core/runtime_stores.py, tests/unit/test_runtime_value_store.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/development/runtime_value_extension.rst against the later B6 changes to openhcs/core/runtime_stores.py, tests/unit/test_runtime_value_store.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/runtime_value_extension.rst in full and compared it against the changed authorities: runtime_artifact_values.py, runtime_stores.py, runtime_slice_projection.py, tests/unit/test_runtime_value_store.py; replaced the deleted RuntimeSliceProjectionStrategy registration step with the RuntimeSliceProjectableValue base classes from runtime_plane_projection.py, removed the MRO-selection and aggregation claims, and confirmed RuntimeValue, ArtifactKey, separate RuntimeArtifactLocation and fail-closed store queries remain as described.", @@ -4140,7 +4159,7 @@ }, { "path": "docs/source/development/source_binding_extension.rst", - "source_sha256": "65a1cd3ee98bed4974a17d20d1998e325dc9685eb044e071393189c680c8df79", + "source_sha256": "1f33644ddd081d2659cd5417f2607189a049bab6e07d26e198e65f452de52659", "audience": [ "OpenHCS source-ingestion and compiler extension authors" ], @@ -4149,28 +4168,33 @@ "authority": [ { "path": "openhcs/core/source_bindings.py", - "sha256": "1dfbd51ae4c64d6a6a8e5ff5ab2ba82465c5e0adacb98dddaf819e0fea05e241", - "role": "Declares SourceFilterClause, MetadataExtractionRule, NamedSourceBinding, ImagePlaneSource, SourceBindingsConfig, StepSourceBindingsConfig and the compiled source universe and binding plans." + "sha256": "2defaf02ac0e214bb63f74d6eb56f294fbb5db4e06ecd9cc4479b16848f09854", + "role": "Declares SourceFilterClause, MetadataExtractionRule, NamedSourceBinding, ImagePlaneSource, the SourceSetPairing family (PairByOrder, PairByMetadata) and the compiled source universe and binding plans." + }, + { + "path": "openhcs/core/config.py", + "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", + "role": "Declares SourceBindingsConfig and StepSourceBindingsConfig as global_pipeline_config sections, where the step config compiles its own binding plan." }, { "path": "openhcs/core/source_binding_selection.py", - "sha256": "f97a4475aa26e465679b89ff9a1fe96690ab7631649d71650608fc7a8aa4bf5a", + "sha256": "9610bc6112c068cf6c4b0ff7a1265ee4086291042e2dc422d8a90cce3c560eb7", "role": "SourceUniverseRequest family that selects source universes." }, { "path": "openhcs/core/source_matching.py", - "sha256": "c608880341d5c40d745ebf4254d8a2931ae611c354a6fe5bb5b3d18611679c63", + "sha256": "f32db4e25a0b6b7c1a334ee27c7206044e1ba645717ee12f6acd69d28d9b30a8", "role": "SourceFilterMatcher and SourceFilterTargetResolver matching families." }, { "path": "openhcs/core/source_workspace_projection.py", - "sha256": "d833421b963550aa5fe428c68c0286216437aec97834e4204b0b88def011f98b", + "sha256": "2d2b6bf5a331cb61411b293b333980a9951f9ec1cd8537874d6529949180f3ec", "role": "VirtualWorkspaceSourceProjection mapping virtual paths to SourcePixelRef and source metadata." }, { "path": "openhcs/core/source_binding_workspace.py", - "sha256": "4d28066a9f73b283fc637662e05fb2632c6fa18af164476c2899fce58cd647fe", - "role": "Compiled virtual source workspace and binding projection." + "sha256": "5e7ac439d7ef6cd35a5660d2ad9db24f1890801d8bdf1d82c13b12f8baaa9eb0", + "role": "Compiled virtual source workspace and the source-set assemblers keyed by SourceSetPairing class." }, { "path": "external/PolyStore/src/polystore/virtual_workspace.py", @@ -4179,11 +4203,12 @@ } ], "findings": [ - "The strategy-selection section asked authors to declare an enum/type/context key on an unnamed strategy root; source families are now AutoRegisterMeta subclass families such as SourceUniverseRequest and SourceFilterMatcher, and the page names them and asks for the family's registry key.", - "The virtual-workspace paragraph said OpenHCS owns microscope metadata; the projection is built from dataset source metadata, so the page now says that." + "K4 replaced the SourceBindingMatchMethod enum and SourceBindingMatchPlan with the SourceSetPairing class family (PairByOrder default, PairByMetadata); the strategy-selection paragraph named only SourceUniverseRequest and SourceFilterMatcher, so it now also names SourceSetPairing as the family that decides source-set pairing.", + "The record's source_bindings.py role said it declares SourceBindingsConfig and StepSourceBindingsConfig; K4 moved both to openhcs/core/config.py, which is now a separate authority." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/source_binding_extension.rst in full and compared it against the changed authorities: openhcs/core/source_bindings.py, source_binding_selection.py, source_matching.py, source_workspace_projection.py and source_binding_workspace.py (K4: config classes moved to core/config.py, SourceSetPairing family replaces the match-method enum, CompiledSourceBindingPlan()/CompiledSourceUniversePlan() replace .empty(), one lexical path identity). Confirmed every class the page names exists at its current owner, added SourceSetPairing to the strategy families, and added core/config.py as the config-class authority.", "On 2026-10-11 (B6 final), compared docs/source/development/source_binding_extension.rst against B6's later changes to openhcs/core/source_binding_selection.py, openhcs/core/source_workspace_projection.py, openhcs/core/source_binding_workspace.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/development/source_binding_extension.rst against the later B6 changes to openhcs/core/source_bindings.py, openhcs/core/source_binding_selection.py, openhcs/core/source_workspace_projection.py, openhcs/core/source_binding_workspace.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/source_binding_extension.rst in full and compared it against the changed authorities: source_bindings.py (runtime-context and path-lookup classes removed), source_binding_workspace.py, pipeline/path_planner.py, PolyStore filemanager.py; confirmed every named declaration and compiled plan class still exists, rewrote the strategy-selection paragraph around the current SourceUniverseRequest / SourceFilterMatcher families, changed microscope metadata to dataset source metadata, and moved the PolyStore authority to virtual_workspace.py where SourcePixelRef is declared; path_planner.py and filemanager.py were dropped as the page states no fact they own.", diff --git a/docs/audits/project_surface.json b/docs/audits/project_surface.json index f991362d51..d52180346a 100644 --- a/docs/audits/project_surface.json +++ b/docs/audits/project_surface.json @@ -12,7 +12,7 @@ "authority": [ { "path": "pyproject.toml", - "sha256": "c3a2bea22aa6872940125f4aaac72c9eac72084fefc6c2cefdb54b5273361e4a", + "sha256": "b5d4d752704b886da20d860743b81dc3d2ce3a3c402208246fadc1c6cb71f597", "role": "Declares the published README, Python range, package dependencies, optional extras, console commands, and project links." }, { @@ -27,22 +27,22 @@ }, { "path": "openhcs/core/pipeline_document.py", - "sha256": "8c554c8275c665f308eb098955bda517aa804eeb34418b5e0f41d412650ce7b5", + "sha256": "9a77a08367f901594859c8b6cae9609c9230a551f4f4e9ec6ea6cd2ed233c0f4", "role": "Owns the validated Python pipeline document shared by visual, code, and agent authoring." }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "sha256": "4a33908fb42874e6d57eeea3f5305793ddbda41d13356f0a78501af38dcdfeb4", "role": "Owns compile-time resolution across selected axes, compile progress emission, and construction of the execution bundle." }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", + "sha256": "0f9cc4819b53d55b7e9446429b8c8ac1ff1a59f4902020ea1658495ce01d464c", "role": "Declares ResolvedPipelineDefinition and CompilationSession named in the compilation diagram." }, { "path": "openhcs/core/progress/types.py", - "sha256": "beabbfeb8a6ed727b4bb09f6155de60c4e8b89e71e7ed053bb85574aa1675419", + "sha256": "c46e825cd32801caae9fc107e0759976da054eadc56adc0d13354b3a8b7db209", "role": "Declares set_progress_queue, which compilation requires before it emits progress events." }, { @@ -52,7 +52,7 @@ }, { "path": "openhcs/core/dataset_sources/source.py", - "sha256": "097fde4d2b74031eacf53a88e79f499417e8c1c97ba0753e8a9df831b307efa6", + "sha256": "350bf5691a03522e9f54b1d8ab712c6beba1683aac4fa5159ea4e80c6041530d", "role": "Declares the auto-registered DatasetSource family that the ImageXpress, Opera Phenix, Bio-Formats, OMERO and OpenHCS-format sources join." }, { @@ -77,7 +77,7 @@ }, { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "3439d96f3fbfcb104a53019dc150049021d9a8132bd27096f1ccf510eefbe37b", + "sha256": "b96f33b866fe6c9f8118ca70fcb7dff8e6a02cb29d79df67230a5e8b97df8002", "role": "Declares import_cellprofiler_pipeline and its returned FunctionStep list and PipelineConfig." }, { @@ -111,16 +111,10 @@ "role": "Proves the installed command derivation, typed receipts, MCP projection, path policy, and inspect-only import boundary." } ], - "findings": [ - "The Python example compiled without installing a progress queue; compilation now emits typed progress events and raises ProgressError when none is installed.", - "The run-measured paragraph said the command submits through ordinary headless job control; it now runs through the OpenHCS session and mirrors the never-written source plate into an execution workspace (EVIDENCE/workspace or --workspace-root).", - "The finalizer paragraph described an ordinary headless job and server completion record; the finalizer now takes the submission and server record from the OpenHCS session.", - "The metaclass-registry row and source table named microscope handlers; sources are now the DatasetSource family.", - "Authority openhcs/desktop_installation.py was deleted (moved to openhcs/desktop/installation.py), and openhcs/constants/constants.py no longer declares microscope sources or axes.", - "The compile-time validation card and the compilation diagram named step snapshots/StepSnapshot, which were deleted; compilation resolves a ResolvedPipelineDefinition and runs per-context CompilationSession objects." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read README.md in full and compared it against the changed authorities: pyproject.toml (zmqruntime 0.6.2, pycodify 0.2, objectstate 2.1.1, python-introspect 0.2.7, pyqt-reactive 0.5 floors), openhcs/core/pipeline_document.py (renders through pycodify.generate_python_source), openhcs/core/pipeline/compiler.py and compilation_session.py (ScopeKey ids, progress classes, disabled-function stripping removed), openhcs/core/progress/types.py (K6: ProgressPhase/ProgressStatus enums became class families; set_progress_queue unchanged and still exported from openhcs.core.progress), openhcs/core/dataset_sources/source.py (SourceBindingsConfig import moved) and pipeline_import.py (K4 import moves). The README states no dependency version or phase name; the CellProfiler example's imports (ensure_global_config_context, set_progress_queue, import_cellprofiler_pipeline with source_root, PipelineOrchestrator.initialize/compile_pipelines) and the compilation diagram's class names all still resolve, so no claim changed.", "On 2026-10-11 (B6 final), compared README.md against B6's later changes to tests/integration/test_cellprofiler_official30_zmq.py, benchmark/cellprofiler_benchmark_cli.py: the session decides in-place initialization by an actual file-creation attempt that leaves nothing behind, its debug-session methods moved into an inherited mixin, the measured CLI treats Windows Ctrl+Break like Ctrl+C, and the Official30 viewer test reads typed payload summaries. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared README.md against B6's later changes to openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/progress/types.py, openhcs/core/artifacts.py, openhcs/core/dataset_sources/source.py, openhcs/processing/backends/lib_registry/unified_registry.py, openhcs/core/viewer_streaming_service.py, openhcs/interop/cellprofiler/pipeline_import.py, benchmark/cellprofiler_benchmark_cli.py, tests/unit/test_benchmark_control.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared README.md against the later B6 changes to pyproject.toml, openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/progress/types.py, openhcs/core/artifacts.py, openhcs/core/dataset_sources/source.py, openhcs/processing/backends/lib_registry/unified_registry.py, openhcs/core/viewer_streaming_service.py, openhcs/agent/capabilities.py, openhcs/interop/cellprofiler/pipeline_import.py, tests/integration/test_cellprofiler_official30_zmq.py, benchmark/cellprofiler_benchmark_cli.py, tests/unit/test_benchmark_control.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", diff --git a/docs/audits/user_docs.json b/docs/audits/user_docs.json index 674c48d09d..760cf16b92 100644 --- a/docs/audits/user_docs.json +++ b/docs/audits/user_docs.json @@ -11,7 +11,7 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", "role": "Declares PipelineConfig, GlobalPipelineConfig, ProcessingConfig and the lazy processing configuration used by the step example." }, { @@ -26,12 +26,12 @@ }, { "path": "openhcs/core/orchestrator/orchestrator.py", - "sha256": "a61658b107f170c192eea505966c268ab6826826f03586bd64562a44800e1ccc", + "sha256": "99870516da6d64e1858f4dd8799d6e38ab6911b7b02bc29b252d2a23f7687410", "role": "Declares the PipelineOrchestrator constructor, compile_pipelines, progress_context and execute_compiled_bundle signatures." }, { "path": "openhcs/core/progress/types.py", - "sha256": "beabbfeb8a6ed727b4bb09f6155de60c4e8b89e71e7ed053bb85574aa1675419", + "sha256": "c46e825cd32801caae9fc107e0759976da054eadc56adc0d13354b3a8b7db209", "role": "Declares ProgressQueue, set_progress_queue and ProgressExecutionContext, which compilation and execution emit through." }, { @@ -41,21 +41,19 @@ }, { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "3439d96f3fbfcb104a53019dc150049021d9a8132bd27096f1ccf510eefbe37b", + "sha256": "b96f33b866fe6c9f8118ca70fcb7dff8e6a02cb29d79df67230a5e8b97df8002", "role": "Declares import_cellprofiler_pipeline and its returned steps and PipelineConfig." }, { "path": "tests/unit/test_cellprofiler_pipeline_import.py", - "sha256": "6b1d201ec5c4ba620fa2367a9cbef1f6fbdd25edbdda9bd7b723ebe3a3363412", + "sha256": "0ce4efd85d819c7e9f7fa84ec6f1ac7ec4bda024c9a8d5c7f264fda2730264f0", "role": "Proves direct import rejects modules requiring interactive desktop input instead of omitting them." } ], - "findings": [ - "The execution example called the removed PipelineOrchestrator.execute_compiled_plate with a hand-built progress dict keyed by plate_id; execution is now execute_compiled_bundle with a typed ProgressExecutionContext from orchestrator.progress_context().", - "Compilation emits progress events and fails without a queue installed by set_progress_queue; the example compiled before creating any queue." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/api/index.rst in full and compared it against the changed authorities: openhcs/core/config.py (K4 source-binding classes moved in, PartitionFilterMode became a class family), orchestrator.py (source_bindings_defaults_to_base now imported from config), progress/types.py (K6 phase/status/stream class families), pipeline_import.py and its test (K4 import moves, PairByOrder). The page's two examples still match: ProcessingConfig(variable_components=(Microscopy.Site,)), LazyProcessingConfig.from_config, import_cellprofiler_pipeline(..., source_root=...), set_progress_queue, compile_pipelines and execute_compiled_bundle(execution_bundle=, progress_queue=, progress_context=) with ProgressQueue and ProgressExecutionContext unchanged, so no claim changed.", "On 2026-10-11 (B6 final), compared docs/source/api/index.rst against B6's later changes to openhcs/core/config.py, openhcs/domains/microscopy/axes.py, openhcs/core/orchestrator/orchestrator.py, openhcs/core/progress/types.py, openhcs/interop/cellprofiler/pipeline_import.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/api/index.rst against the later B6 changes to openhcs/core/config.py, openhcs/domains/microscopy/axes.py, openhcs/core/orchestrator/orchestrator.py, openhcs/core/progress/types.py, openhcs/interop/cellprofiler/pipeline_import.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/api/index.rst in full and compared it against the changed authorities: openhcs/core/config.py, openhcs/core/orchestrator/orchestrator.py, openhcs/core/compiled_execution.py, tests/unit/test_compiled_execution.py, openhcs/interop/cellprofiler/pipeline_import.py, tests/unit/test_cellprofiler_pipeline_import.py; rewrote the compile/execute example to install a progress queue, call execute_compiled_bundle with orchestrator.progress_context(), and name the constructor argument dataset_root, matching tests/unit/remote_sensing_witness_pipeline.py; confirmed the FunctionStep/ProcessingConfig example, the import_cellprofiler_pipeline signature and the interactive-module rejection against current source.", @@ -87,7 +85,7 @@ }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", + "sha256": "358d3eaf5952170f9e9e2dc9056fb08707e8fcf74682063c074b367a83c2648f", "role": "Declares the per-step compiled plan read by runtime code." }, { @@ -97,27 +95,27 @@ }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", + "sha256": "0f9cc4819b53d55b7e9446429b8c8ac1ff1a59f4902020ea1658495ce01d464c", "role": "Owns compilation-scoped source, artifact and plan state." }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "b6a7231bc2964d472070731018f6a4935e5f23cf8f238a232045c32244e6545a", + "sha256": "74b591773b7f653c957b2db33a1262925e5c02b71074695b6fc18d6120ed5934", "role": "Implements scoped typed runtime-value storage and lookup for compiled execution." }, { "path": "openhcs/core/source_bindings.py", - "sha256": "1dfbd51ae4c64d6a6a8e5ff5ab2ba82465c5e0adacb98dddaf819e0fea05e241", - "role": "Declares source bindings that connect callable inputs to sources or artifact plans." + "sha256": "2defaf02ac0e214bb63f74d6eb56f294fbb5db4e06ecd9cc4479b16848f09854", + "role": "Declares NamedSourceBinding and the source-assignment family that connect callable inputs to sources, plus CompiledSourceUniversePlan." }, { "path": "openhcs/core/callable_contract.py", - "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", - "role": "Declares the compiler-visible callable metadata defined by the glossary." + "sha256": "a8e73c5df804d78baf72710e210cb2ae63156657a074071e11fd868f4b19d3b4", + "role": "Declares CallableDeclaration and CallableContract fields: memory types, artifact inputs/outputs, runtime-bound parameters, required axis roles and execution scope." }, { "path": "openhcs/core/processing_contracts.py", - "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", + "sha256": "eb0f7fd99ded1c4d5957152ccf6abe1468436c13574a1d1bb83ce368d24b9f69", "role": "Declares the ProcessingContract family with Pure2DContract and Pure3DContract." }, { @@ -127,17 +125,14 @@ }, { "path": "openhcs/core/dataset_sources/source.py", - "sha256": "097fde4d2b74031eacf53a88e79f499417e8c1c97ba0753e8a9df831b307efa6", + "sha256": "350bf5691a03522e9f54b1d8ab712c6beba1683aac4fa5159ea4e80c6041530d", "role": "Declares DatasetSource with detection, parsing, metadata, virtual-workspace and backend behaviour." } ], - "findings": [ - "The Processing contract entry named PURE_2D/PURE_3D enum members; contracts are now classes of the ProcessingContract family (Pure2DContract, Pure3DContract).", - "The Microscope handler entry and the Source workspace projection entry named the removed microscope-handler concept; the registered owner is now DatasetSource.", - "The Execution axis entry and the Compiled step plan entry used the old runtime vocabulary; the axis family now names a PartitionAxis role, and compiled bundles are consumed by bundle execution rather than plate execution." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/appendices/glossary.rst in full and compared it against the changed authorities: compiled_step_plan.py and runtime_stores.py (source plans default-construct; RuntimeArtifactLocation/Address dict codecs removed), compilation_session.py, source_bindings.py (K4: SourceBindingsConfig moved to config.py, match methods became the SourceSetPairing family), callable_contract.py (K5: metadata wrapper flattened into CallableDeclaration fields), processing_contracts.py and dataset_sources/source.py. The glossary defines terms without naming moved classes; Pure2DContract, Pure3DContract, PartitionAxis, DatasetSource, CompiledSourceUniversePlan and the callable-contract fields it lists still exist, so no claim changed. Sharpened the source_bindings and callable_contract roles to what they now declare.", "On 2026-10-11 (B6 final), compared docs/source/appendices/glossary.rst against B6's later changes to openhcs/core/artifacts.py, openhcs/core/compiled_step_plan.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/runtime_stores.py, openhcs/core/callable_contract.py, openhcs/core/processing_contracts.py, openhcs/core/axes.py, openhcs/core/dataset_sources/source.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/appendices/glossary.rst against the later B6 changes to openhcs/core/artifacts.py, openhcs/core/compiled_step_plan.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/runtime_stores.py, openhcs/core/source_bindings.py, openhcs/core/callable_contract.py, openhcs/core/processing_contracts.py, openhcs/core/axes.py, openhcs/core/dataset_sources/source.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/appendices/glossary.rst in full and compared it against the changed authorities: openhcs/core/artifacts.py, compiled_step_plan.py, pipeline/compilation_session.py, runtime_stores.py, source_bindings.py, callable_contract.py, openhcs/processing/backends/lib_registry/unified_registry.py, plus the G1/G4/G5/G7 owners; rewrote the Processing contract, Dataset source (was Microscope handler), Partition axis (was Execution axis), Compiled step plan, Compiled execution bundle and Source workspace projection entries, and confirmed the remaining terms (FunctionStep, CompiledExecutionBundle, SourceUniverse, RuntimeAdapterSpec, CellProfilerModuleArtifactContracts, variable_components, group_by) against current source.", @@ -200,7 +195,7 @@ "authority": [ { "path": "pyproject.toml", - "sha256": "c3a2bea22aa6872940125f4aaac72c9eac72084fefc6c2cefdb54b5273361e4a", + "sha256": "b5d4d752704b886da20d860743b81dc3d2ce3a3c402208246fadc1c6cb71f597", "role": "Declares the packaged openhcs-benchmark console script entry point." }, { @@ -227,6 +222,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/appendices/research_impact.rst in full and compared it against the changed authorities: pyproject.toml (dependency floors for zmqruntime, pycodify 0.2, objectstate 2.1, python-introspect, pyqt-reactive 0.5); the openhcs-benchmark entry point, benchmark CLI, run receipts, read-only inspection service and the archived page path all still exist, and the page states no version, so no claim changed.", "On 2026-10-11 (B6 final), compared docs/source/appendices/research_impact.rst against B6's later changes to benchmark/cellprofiler_benchmark_cli.py: the session decides in-place initialization by an actual file-creation attempt that leaves nothing behind, its debug-session methods moved into an inherited mixin, the measured CLI treats Windows Ctrl+Break like Ctrl+C, and the Official30 viewer test reads typed payload summaries. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/appendices/research_impact.rst against B6's later changes to benchmark/cellprofiler_benchmark_cli.py, tests/unit/test_benchmark_control.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/appendices/research_impact.rst against the later B6 changes to pyproject.toml, benchmark/cellprofiler_benchmark_cli.py, tests/unit/test_benchmark_control.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -269,22 +265,22 @@ "authority": [ { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "sha256": "4a33908fb42874e6d57eeea3f5305793ddbda41d13356f0a78501af38dcdfeb4", "role": "Compiles pipeline declarations into validated step plans and execution bundles." }, { "path": "openhcs/core/source_bindings.py", - "sha256": "1dfbd51ae4c64d6a6a8e5ff5ab2ba82465c5e0adacb98dddaf819e0fea05e241", - "role": "Source-binding declarations that name pipeline inputs from files and metadata." + "sha256": "2defaf02ac0e214bb63f74d6eb56f294fbb5db4e06ecd9cc4479b16848f09854", + "role": "Declares NamedSourceBinding and the SourceSetPairing family that turn files and metadata into named inputs." }, { "path": "openhcs/core/dataset_sources/source.py", - "sha256": "097fde4d2b74031eacf53a88e79f499417e8c1c97ba0753e8a9df831b307efa6", + "sha256": "350bf5691a03522e9f54b1d8ab712c6beba1683aac4fa5159ea4e80c6041530d", "role": "DatasetSource family that owns recognized dataset layouts." }, { "path": "openhcs/core/processing_contracts.py", - "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", + "sha256": "eb0f7fd99ded1c4d5957152ccf6abe1468436c13574a1d1bb83ce368d24b9f69", "role": "ProcessingContract family deciding per-plane versus whole-stack execution." }, { @@ -296,14 +292,19 @@ "path": "external/arraybridge/README.md", "sha256": "2fe1a330ca101dadb6cea5ac6f0482a5ffe8c7e0e4b9bd7bc8bc2220e106415a", "role": "States ArrayBridge ownership of array conversion, callable memory metadata, and framework device mechanics." + }, + { + "path": "openhcs/core/config.py", + "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", + "role": "Declares SourceBindingsConfig, the pipeline-level source-binding configuration (moved here by K4)." } ], "findings": [ - "G4 replaced microscope handlers with the DatasetSource family; the owner checklist named 'microscope integration'.", - "Swapped the unified_registry.py authority (the page states nothing about the catalogue) for processing_contracts.py, which owns the ProcessingContract claim." + "Audit record only: after K4 the SourceBindingsConfig class lives in openhcs/core/config.py, so the source_bindings.py role was narrowed to NamedSourceBinding and the SourceSetPairing family and config.py was added; the page prose names no moved class and needed no change." ], - "disposition": "revise", + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/concepts/building_intuition.rst in full and compared it against the changed authorities: compiler.py (ScopeKey scope ids, progress classes, empty CompiledSourceBindingPlan), source_bindings.py (K4: config class moved to config.py, SourceSetPairing family replacing SourceBindingMatchMethod), dataset_sources/source.py and processing_contracts.py (CallableDeclaration rename); the page's owner checklist (DatasetSource, PolyStore, ArrayBridge, pyqt-reactive) and its no force_disk_output claim still hold.", "On 2026-10-11 (B6 final), compared docs/source/concepts/building_intuition.rst against B6's later changes to openhcs/core/pipeline/compiler.py, openhcs/core/dataset_sources/source.py, openhcs/core/processing_contracts.py, openhcs/core/artifacts.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/concepts/building_intuition.rst against the later B6 changes to openhcs/core/pipeline/compiler.py, openhcs/core/source_bindings.py, openhcs/core/dataset_sources/source.py, openhcs/core/processing_contracts.py, openhcs/core/artifacts.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/concepts/building_intuition.rst in full and compared it against the changed authorities: pipeline/compiler.py, source_bindings.py, artifacts.py, lib_registry/unified_registry.py and external/arraybridge/README.md; confirmed FunctionStep, variable_components, group_by and ProcessingContract exist, that force_disk_output is absent, and renamed the source-integration owner to DatasetSource.", @@ -346,17 +347,14 @@ }, { "path": "tests/unit/test_axis_family_guards.py", - "sha256": "a81f12f9057e2ed7393e6daeb30f1c086f5c867222289a2bb4af3cf297b59c5c", + "sha256": "2c70f7e629eb5c67978aa389b632ea3a806d693a261f93d87660b52ffd178b03", "role": "Proves the component enums are deleted and kernel modules name no microscopy member." } ], - "findings": [ - "The authority record still named openhcs/constants/constants.py, which no longer declares any axis or component identity after G1; the owners are openhcs/core/axes.py and openhcs/domains/microscopy/axes.py.", - "\"Site declares the stack axis\" read as if Site carried the StackAxis role, which belongs to ZIndex; reworded to say Site is the axis the step assembles its stacks along.", - "tests/unit/test_configuration_mirror_deletion.py does not support this page's claims and was dropped." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/concepts/component_system_concepts.rst in full and compared it against the changed authorities: tests/unit/test_axis_family_guards.py (U3 deleted openhcs/core/path_cache.py, so the kernel microscopy-reference ceiling dropped from 72 to 70); the page names no pinned reference, and Axis, AxisFamily.active()/with_role(), the seven roles, Ungrouped, ComponentGroupScope, RuntimeExecutionAxisScope and LazyProcessingConfig.from_config were re-checked against the current tree.", "On 2026-10-11 (B6 final), compared docs/source/concepts/component_system_concepts.rst against B6's later changes to openhcs/core/axes.py, openhcs/domains/microscopy/axes.py, tests/unit/test_axis_family_guards.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/concepts/component_system_concepts.rst against the later B6 changes to openhcs/core/axes.py, openhcs/domains/microscopy/axes.py, tests/unit/test_axis_family_guards.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/concepts/component_system_concepts.rst in full and compared it against the changed authorities: openhcs/constants/constants.py, openhcs/core/component_group_scope.py, tests/unit/test_configuration_mirror_deletion.py; the page had already been rewritten for the G1 axis family, so checked every named class, role, module path and the ProcessingConfig/LazyProcessingConfig.from_config example against openhcs/core/axes.py, openhcs/domains/microscopy/axes.py, openhcs/core/config.py and the ObjectState lazy factory, corrected the Site sentence, and moved the authorities to the axis owners.", @@ -368,7 +366,7 @@ }, { "path": "docs/source/concepts/core_model.rst", - "source_sha256": "7e81dd24da51a195a7e41c056773e84658f8c57ee4fe88f47cb88bc42b465cad", + "source_sha256": "8b4e5a46e424780bc2bf6e8d1609c667b24f42f6dd946b89e146d3c5eb45aca1", "audience": [ "imaging scientists", "integrators", @@ -379,17 +377,17 @@ "authority": [ { "path": "openhcs/core/pipeline_document.py", - "sha256": "8c554c8275c665f308eb098955bda517aa804eeb34418b5e0f41d412650ce7b5", + "sha256": "9a77a08367f901594859c8b6cae9609c9230a551f4f4e9ec6ea6cd2ed233c0f4", "role": "PipelineDocument holds a PipelineConfig plus an ordered step list; no Pipeline wrapper class exists." }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "sha256": "4a33908fb42874e6d57eeea3f5305793ddbda41d13356f0a78501af38dcdfeb4", "role": "Creates one CompilationSession per partition-axis context and emits the compiled bundle after validation." }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", + "sha256": "0f9cc4819b53d55b7e9446429b8c8ac1ff1a59f4902020ea1658495ce01d464c", "role": "Declares CompilationSession with the ResolvedPipelineDefinition, source workspace projection and CompiledStepPlan map." }, { @@ -399,32 +397,38 @@ }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "b6a7231bc2964d472070731018f6a4935e5f23cf8f238a232045c32244e6545a", + "sha256": "74b591773b7f653c957b2db33a1262925e5c02b71074695b6fc18d6120ed5934", "role": "Declares RuntimeValueStore keyed by typed artifact keys and execution scope." }, { "path": "openhcs/core/callable_contract.py", - "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", - "role": "Declares the CallableContract fields: memory types, required axis roles, allowed group-by roles, processing contract, execution scope, runtime adapter and artifact specs." + "sha256": "a8e73c5df804d78baf72710e210cb2ae63156657a074071e11fd868f4b19d3b4", + "role": "Declares CallableDeclaration (inherited by CallableContract) with the memory types, required axis roles, allowed group-by roles, processing contract, execution scope, runtime adapter and artifact spec fields." }, { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "3439d96f3fbfcb104a53019dc150049021d9a8132bd27096f1ccf510eefbe37b", + "sha256": "b96f33b866fe6c9f8118ca70fcb7dff8e6a02cb29d79df67230a5e8b97df8002", "role": "Lowers parsed .cppipe modules into ordinary source bindings and FunctionStep declarations." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", + "sha256": "358d3eaf5952170f9e9e2dc9056fb08707e8fcf74682063c074b367a83c2648f", "role": "Declares the source, artifact, memory, materialisation, streaming and worker fields of CompiledStepPlan." + }, + { + "path": "openhcs/core/dataset_sources/source.py", + "sha256": "350bf5691a03522e9f54b1d8ab712c6beba1683aac4fa5159ea4e80c6041530d", + "role": "Declares the DatasetSource family that interprets plate layouts and metadata." } ], "findings": [ - "The compilation section named StepSnapshot objects, which were deleted when the compiler moved to compiling from the resolved pipeline definition (commit 0adb68108); the session now carries a ResolvedPipelineDefinition.", - "\"Each execution axis receives a CompilationSession\" misstated the scope: one session is created per partition-axis value (context).", - "Callable contracts now declare required axis roles (required_axis_roles) rather than required axes, after G1." + "The Sources section still credited \"microscope handlers\" with interpreting plate layouts; G4 replaced them with the DatasetSource family. Reworded and added dataset_sources/source.py as an authority.", + "The diagram still read \"snapshots + compilation sessions\" after step snapshots were deleted; it now reads \"resolved pipeline + compilation sessions\".", + "K5 moved the callable fields from CallableMetadata to CallableDeclaration, which CallableContract inherits; the callable_contract.py role was updated (the page's field list still holds)." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/concepts/core_model.rst in full and compared it against the changed authorities: pipeline_document.py (pycodify generate_python_source), compiler.py (ScopeKey, progress classes, empty source plans), compilation_session.py (graph-level declaration validation, disabled items dropped by normalisation), runtime_stores.py (K6 codec removal), callable_contract.py (K5 CallableDeclaration with artifact fields and repeated_return_slots), pipeline_import.py (K4 imports) and compiled_step_plan.py (empty plan defaults); corrected the microscope-handler and snapshot wording and re-checked the callable-contract field list, CompilationSession per partition value, CompiledExecutionBundle and RuntimeValueStore claims.", "On 2026-10-11 (B6 final), compared docs/source/concepts/core_model.rst against B6's later changes to openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/runtime_stores.py, openhcs/core/callable_contract.py, openhcs/interop/cellprofiler/pipeline_import.py, openhcs/core/compiled_step_plan.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/concepts/core_model.rst against the later B6 changes to openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/runtime_stores.py, openhcs/core/callable_contract.py, openhcs/interop/cellprofiler/pipeline_import.py, openhcs/core/compiled_step_plan.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/concepts/core_model.rst in full and compared it against the changed authorities: openhcs/core/pipeline_document.py, openhcs/core/pipeline/compiler.py, openhcs/core/compiled_execution.py, openhcs/core/runtime_stores.py, openhcs/interop/cellprofiler/pipeline_import.py, openhcs/core/compiled_step_plan.py; grepped every named class (CompilationSession, CompiledExecutionBundle, RuntimeValueStore, CellProfilerModule, SourceBindingsConfig, ArtifactSpec, CompiledStepPlan, ProcessingContract), replaced the deleted StepSnapshot, corrected session scope and contract axis wording, and added compilation_session.py and callable_contract.py as owners.", @@ -455,7 +459,7 @@ }, { "path": "openhcs/core/processing_contracts.py", - "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", + "sha256": "eb0f7fd99ded1c4d5957152ccf6abe1468436c13574a1d1bb83ce368d24b9f69", "role": "Declares the Pure2DContract, Pure3DContract, FlexibleContract and VolumetricToSliceContract subclasses of ProcessingContract." }, { @@ -470,7 +474,7 @@ }, { "path": "openhcs/processing/backends/cellprofiler/watershed.py", - "sha256": "41c8cb9cb139f8b4041a25e1a7a2d19aa85cf0427422fc87b9551c8141069625", + "sha256": "2c8c352bdc5fedad6baf14e4c1a3cd5347a1f862fd841472241680294c2762ca", "role": "Declares Watershed with FullStackExecution so it consumes the complete payload." }, { @@ -480,7 +484,7 @@ }, { "path": "openhcs/processing/backends/cellprofiler/secondary.py", - "sha256": "fc9d9b1bbc3baff7216c6abe3a7135ffddfc9f7e1212d4a391fa8534038a354c", + "sha256": "f731f95f874ba3368828ca3a539fac4e5c6eb0ec7049e546f45fb874e0888e95", "role": "Declares IdentifySecondaryObjects and IdentifyTertiaryObjects with Pure2DContract." }, { @@ -489,13 +493,10 @@ "role": "Binds the published dimensionality boundary to the current callable declarations." } ], - "findings": [ - "The processing-locality list named enum members PURE_2D, PURE_3D, FLEXIBLE and VOLUMETRIC_TO_SLICE; after G5 ProcessingContract is a class family (Pure2DContract, Pure3DContract, FlexibleContract, VolumetricToSliceContract).", - "The page called the well axis the configured multiprocessing execution axis; after G1 the active axis family declares a single partition axis.", - "openhcs/constants/constants.py no longer declares any axis and was replaced by openhcs/core/axes.py; unified_registry.py, variable_component_stack_requirement.py and tests/unit/test_callable_contract.py supported no claim the page makes and were dropped." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/concepts/data_dimensions.rst in full and compared it against the changed authorities: openhcs/core/processing_contracts.py (CallableMetadata renamed CallableDeclaration at three call sites), openhcs/processing/backends/cellprofiler/watershed.py (StepSourceBindingsConfig imported from openhcs.core.config after K4) and openhcs/processing/backends/cellprofiler/secondary.py (repeated_return_slots for identical tertiary inputs); the page names none of these, and the four contract classes, the PAYLOAD/PLANE label domains, the per-plane Identify modules and the ProcessingConfig example still hold.", "On 2026-10-11 (B6 final), compared docs/source/concepts/data_dimensions.rst against B6's later changes to openhcs/core/axes.py, openhcs/core/processing_contracts.py, openhcs/processing/backends/cellprofiler/watershed.py, openhcs/processing/backends/cellprofiler/primary_objects.py, openhcs/processing/backends/cellprofiler/secondary.py, tests/unit/test_documented_dimensionality.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/concepts/data_dimensions.rst against the later B6 changes to openhcs/core/axes.py, openhcs/core/processing_contracts.py, openhcs/processing/backends/cellprofiler/watershed.py, openhcs/processing/backends/cellprofiler/primary_objects.py, openhcs/processing/backends/cellprofiler/secondary.py, tests/unit/test_documented_dimensionality.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/concepts/data_dimensions.rst in full and compared it against the changed authorities: openhcs/constants/constants.py, openhcs/core/component_group_scope.py, openhcs/processing/backends/lib_registry/unified_registry.py, openhcs/core/runtime_object_label_domains.py, the three CellProfiler backends, tests/unit/test_callable_contract.py and tests/unit/test_documented_dimensionality.py; confirmed Watershed still declares FullStackExecution and the Identify modules Pure2DContract, the PAYLOAD/PLANE scopes are unchanged, and corrected the processing-contract names and partition-axis wording.", @@ -520,12 +521,12 @@ "authority": [ { "path": "openhcs/core/dataset_sources/source.py", - "sha256": "097fde4d2b74031eacf53a88e79f499417e8c1c97ba0753e8a9df831b307efa6", + "sha256": "350bf5691a03522e9f54b1d8ab712c6beba1683aac4fa5159ea4e80c6041530d", "role": "Declares the registered DatasetSource family that projects each microscope format into axis values, backends and metadata." }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "sha256": "4a33908fb42874e6d57eeea3f5305793ddbda41d13356f0a78501af38dcdfeb4", "role": "Resolves inputs, axes, locality, memory conversion and artifact dependencies before emitting compiled plans." }, { @@ -534,12 +535,10 @@ "role": "Declares ArtifactKey and RuntimeValue, the typed runtime result carrier." } ], - "findings": [ - "Authority openhcs/microscopes/microscope_interfaces.py was deleted by G4 (c6d9a52ce); the source interface now lives in openhcs/core/dataset_sources/source.py as the DatasetSource family.", - "The page said \"registered microscope handlers\"; reworded to registered dataset sources, matching the G4 family name." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/concepts/domain_fundamentals.rst in full and compared it against the changed authorities: openhcs/core/dataset_sources/source.py (SourceBindingsConfig now imported from openhcs.core.config after K4) and openhcs/core/pipeline/compiler.py (ScopeKey scope ids, CompileProgressPhase/RunningProgressStatus classes after K6, an empty CompiledSourceBindingPlan instead of None after K4); none changes the page's explanation of dataset sources, compile-time resolution or typed results.", "On 2026-10-11 (B6 final), compared docs/source/concepts/domain_fundamentals.rst against B6's later changes to openhcs/core/dataset_sources/source.py, openhcs/core/pipeline/compiler.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/concepts/domain_fundamentals.rst against the later B6 changes to openhcs/core/dataset_sources/source.py, openhcs/core/pipeline/compiler.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/concepts/domain_fundamentals.rst in full and compared it against the changed authorities: openhcs/microscopes/microscope_interfaces.py (deleted in G4), openhcs/core/pipeline/compiler.py, openhcs/core/runtime_artifact_values.py; replaced the deleted source interface with openhcs/core/dataset_sources/source.py, updated the dataset-source wording, and confirmed the compilation and typed-result claims still hold.", @@ -563,12 +562,12 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", "role": "Declares DtypeConfig as a DtypeConversionConfig subclass with NATIVE_OUTPUT as its default_dtype_conversion." }, { "path": "openhcs/core/steps/abstract.py", - "sha256": "63734f0c5198272b08f128ae974ada725c8201cb44efd0ce32500fd280b879bf", + "sha256": "9e710b3db635730f88258e5a6e18aed607876e296783022c06798f2bcd03d5e4", "role": "Declares the step-level dtype_config field separate from processing_config." }, { @@ -583,13 +582,14 @@ }, { "path": "tests/unit/test_callable_contract.py", - "sha256": "cc220f53107a08b0406b6e0dd49896e3e4c3c0af6e527b0ef00d4fdaba8f0b2c", + "sha256": "9170f8694f388dbed5bfd36d882a5c0a4cdf28f50f51f7fdeaf4a955a9f3b412", "role": "Proves dtype_config is the overridable runtime-bound parameter on a decorated callable's contract." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/concepts/dtype_config_system.rst in full and compared it against the changed authorities: openhcs/core/config.py (K4 source-binding classes, PartitionFilterMode family; DtypeConfig unchanged with NATIVE_OUTPUT default), openhcs/core/steps/abstract.py (import regrouping only; dtype_config still a separate FunctionStep field) and tests/unit/test_callable_contract.py (CallableMetadata renamed CallableDeclaration, namespace round-trip test); dtype_config is still the one config-bound, overridable runtime parameter on a decorated callable's contract, and the DtypeConfig example still resolves.", "On 2026-10-11 (B6 final), compared docs/source/concepts/dtype_config_system.rst against B6's later changes to openhcs/core/config.py, openhcs/core/memory/decorators.py, tests/unit/test_callable_contract.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/concepts/dtype_config_system.rst against the later B6 changes to openhcs/core/config.py, openhcs/core/memory/decorators.py, tests/unit/test_callable_contract.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/concepts/dtype_config_system.rst in full and compared it against the changed authorities: openhcs/core/config.py, openhcs/core/memory/decorators.py, external/arraybridge/src/arraybridge/decorators.py, tests/unit/test_callable_contract.py; checked the DtypeConversion members, the NATIVE_OUTPUT default, the import paths in the example, the step-level dtype_config field (now owned by openhcs/core/steps/abstract.py, added as an authority) and the runtime-bound dtype_config contract parameter; no claim changed.", @@ -605,7 +605,7 @@ }, { "path": "docs/source/concepts/function_library.rst", - "source_sha256": "d1cc56563fe60736d196ac23358cc4f4f26ee57572478351bddbae1c17c624d1", + "source_sha256": "4af7d9f51f08b2ef9573882b50f1e5c7c2f70093ae1d37fcd8869b0ab8bb60ec", "audience": [ "imaging scientists", "callable authors" @@ -615,12 +615,12 @@ "authority": [ { "path": "openhcs/core/callable_contract.py", - "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", - "role": "Declares the CallableContract fields the page lists: memory types, artifact specs, runtime-bound parameters, required_axis_roles, allowed_group_by_roles, processing contract, runtime adapter, execution scope and prepare hook." + "sha256": "a8e73c5df804d78baf72710e210cb2ae63156657a074071e11fd868f4b19d3b4", + "role": "Declares the CallableDeclaration fields CallableContract inherits (memory types, artifact specs, runtime-bound parameters, required_axis_roles, allowed_group_by_roles, processing contract, runtime adapter, execution scope, prepared signatures) and processing_prepare, which attaches the preparation hook to the callable." }, { "path": "openhcs/core/processing_contracts.py", - "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", + "sha256": "eb0f7fd99ded1c4d5957152ccf6abe1468436c13574a1d1bb83ce368d24b9f69", "role": "Declares the ProcessingContract class family selecting per-slice, full-stack, flexible or volumetric-to-slice execution." }, { @@ -630,7 +630,7 @@ }, { "path": "openhcs/processing/backends/lib_registry/registry_service.py", - "sha256": "daef8925338fddf8504a9faa066af42456a13fe4602976bf64be06547a45fcc8", + "sha256": "f2e981157632adb5020828720ec26bfa6444ab52f545e1e71b1524242eaed6a3", "role": "Owns backend discovery, availability filtering and the process catalogue projection." }, { @@ -640,7 +640,7 @@ }, { "path": "openhcs/agent/services/function_catalog_service.py", - "sha256": "211c81f3747dcbc5cad2bde36191bf7ac90fab007f44c960c729ac33c2737df1", + "sha256": "6cd88eca73ca960f32a1f46002cea805f8ffe723ec5e9e1bcd224991a806e164", "role": "Serves catalogue search and callable detail to MCP and GUI authoring from the execution endpoint." }, { @@ -660,7 +660,7 @@ }, { "path": "tests/unit/test_unified_registry_runtime_payloads.py", - "sha256": "76e02cc26b40f5af984d31e874eab1b18d334259fb5cd44bc9fe5945c474b139", + "sha256": "a817850fc2d84f82da3c5e60d2ae33d461d795b36d65816955a4c8b18bd9b852", "role": "Proves external library runtime controls stay out of authored function parameters." }, { @@ -670,11 +670,11 @@ } ], "findings": [ - "The callable-contract list still said contracts declare required variable components and allowed grouping; after G1 they declare required_axis_roles and allowed_group_by_roles, and after G5 the processing contract is a ProcessingContract subclass rather than an enum member.", - "Replaced the jargon \"admitted\" with \"catalogued\" for native and custom function inclusion." + "The contract list claimed CallableContract declares compiler preparation hooks; K5 removed prepare from the declaration, so the hook lives on the callable (processing_prepare) and the contract carries only the prepared signatures. The bullet was corrected and a sentence added; the callable_contract.py role was updated to name CallableDeclaration." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/concepts/function_library.rst in full and compared it against the changed authorities: callable_contract.py (K5: CallableMetadata merged into CallableDeclaration, prepare left the declaration, runtime_bound_parameter_names), processing_contracts.py (CallableDeclaration rename), registry_service.py and function_catalog_service.py (declaration/runtime_bound_parameter_names renames) and tests/unit/test_unified_registry_runtime_payloads.py (admitted_graph helper); corrected the preparation-hook claim and re-checked the remaining contract fields, the ProcessingContract family, and the catalogue and form-exclusion claims.", "On 2026-10-11 (B6 final), compared docs/source/concepts/function_library.rst against B6's later changes to openhcs/core/callable_contract.py, openhcs/core/processing_contracts.py, openhcs/processing/backends/lib_registry/unified_registry.py, openhcs/processing/backends/lib_registry/registry_service.py, openhcs/processing/backends/lib_registry/openhcs_registry.py, openhcs/agent/services/function_catalog_service.py, tests/unit/test_unified_registry_runtime_payloads.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/concepts/function_library.rst against the later B6 changes to openhcs/core/callable_contract.py, openhcs/core/processing_contracts.py, openhcs/processing/backends/lib_registry/unified_registry.py, openhcs/processing/backends/lib_registry/registry_service.py, openhcs/processing/backends/lib_registry/openhcs_registry.py, openhcs/agent/services/function_catalog_service.py, tests/unit/test_library_registry_discovery.py, tests/unit/test_unified_registry_runtime_payloads.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/concepts/function_library.rst in full and compared it against the changed authorities: unified_registry.py, registry_service.py, openhcs/core/callable_contract.py, function_catalog_service.py, openhcs_registry.py, the ArrayBridge types and decorators, and the three registry/contract tests; the G5 processing-contract move and catalogue-service signature/doc projection changes leave the page's catalogue, exclusion and framework claims intact, corrected the contract field names, and added openhcs/core/processing_contracts.py as an owner.", @@ -710,25 +710,24 @@ }, { "path": "openhcs/core/function_patterns.py", - "sha256": "95b9056c6980e4f7943f909aeae8262252c9aa175247a534befc8d4cc459fdd2", - "role": "Normalises callable, chain and dictionary patterns into CompiledFunctionPattern." + "sha256": "cc8c35f37ea0a3bd939905fe317652a34b3129ee6f47f6f9023b8c459e62df70", + "role": "Declares AuthoredFunctionPattern (the single walker over callable, chain and dictionary syntax) and normalises patterns into CompiledFunctionPattern, whose execution_scope requires one uniform scope across invocations." }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "sha256": "4a33908fb42874e6d57eeea3f5305793ddbda41d13356f0a78501af38dcdfeb4", "role": "Validates per-invocation callable contracts and dictionary keys against compiled grouping." }, { "path": "tests/unit/test_function_patterns.py", - "sha256": "d0160652d548748901a8b95d21abde9dca050259fd2f74d4a85cb60a50441175", + "sha256": "296c5ab1f5044b04ce631c92a1e8510fe9dd0e21a776d7a30877766154a74b18", "role": "Proves callable, chain and dictionary patterns compile with their declared semantics." } ], - "findings": [ - "The callable-contract paragraph said invocations satisfy declared axes; after G1 contracts declare required axis roles." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/concepts/function_patterns.rst in full and compared it against the changed authorities: function_patterns.py (K5 replaced the seven raw pattern walkers, strip_disabled_functions, inject_kwargs_into_pattern and inject_artifact_input_values with AuthoredFunctionPattern and NormalizedFunctionPattern methods), compiler.py (ScopeKey, progress classes) and tests/unit/test_function_patterns.py (renamed tests for normalisation dropping disabled items and artifact input values); the page names none of the removed helpers, and the pattern forms, dictionary routing example, uniform execution scope (CompiledFunctionPattern.execution_scope) and declaration-identity claims still hold.", "On 2026-10-11 (B6 final), compared docs/source/concepts/function_patterns.rst against B6's later changes to openhcs/core/function_patterns.py, openhcs/core/pipeline/compiler.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/concepts/function_patterns.rst against the later B6 changes to openhcs/core/function_patterns.py, openhcs/core/pipeline/compiler.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/concepts/function_patterns.rst in full and compared it against the changed authorities: the page itself (example already moved to Microscopy axes), openhcs/core/pipeline/compiler.py, tests/unit/test_function_patterns.py; confirmed the example imports, CompiledFunctionPattern, and the grouping-versus-stacking claims, corrected the axis-role wording, and added openhcs/core/function_patterns.py as the pattern owner.", @@ -753,23 +752,24 @@ "authority": [ { "path": "openhcs/core/pipeline_document.py", - "sha256": "8c554c8275c665f308eb098955bda517aa804eeb34418b5e0f41d412650ce7b5", + "sha256": "9a77a08367f901594859c8b6cae9609c9230a551f4f4e9ec6ea6cd2ed233c0f4", "role": "Declares the pipeline document (config plus ordered steps) that the core-building-blocks pages explain." }, { "path": "openhcs/core/config.py", - "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", "role": "Declares the processing, dtype and storage configuration the processing-and-storage pages explain." }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "sha256": "4a33908fb42874e6d57eeea3f5305793ddbda41d13356f0a78501af38dcdfeb4", "role": "Compiles declarations into plans, the step the core-model and dimensions pages describe." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/concepts/index.rst in full and compared it against the changed authorities: openhcs/core/pipeline_document.py (source rendered through pycodify generate_python_source after L6), openhcs/core/config.py (source-binding config classes now declared there after K4) and openhcs/core/pipeline/compiler.py (ScopeKey scope ids, progress classes); the page only routes readers to the concept pages and every toctree entry still exists.", "On 2026-10-11 (B6 final), compared docs/source/concepts/index.rst against B6's later changes to openhcs/core/config.py, openhcs/core/pipeline/compiler.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/concepts/index.rst against the later B6 changes to openhcs/core/config.py, openhcs/core/pipeline/compiler.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/concepts/index.rst in full and compared it against the changed authorities: openhcs/core/pipeline_document.py, openhcs/core/config.py, openhcs/core/pipeline/compiler.py; the page is a reading-order toctree whose every entry still exists under docs/source/concepts and makes no product claim affected by the G1/G4/G5 renames; no change.", @@ -784,7 +784,7 @@ }, { "path": "docs/source/concepts/module_structure.rst", - "source_sha256": "20d7d199be94986d9ed0cbb3de76059339c8e0d5f3ee78505f7fdd78d0d612da", + "source_sha256": "2dbec5d897d4384d25fed93a72d83c95ba527b9885dc82b4e7dd31643ca83a7f", "audience": [ "contributors", "integrators" @@ -794,7 +794,7 @@ "authority": [ { "path": "pyproject.toml", - "sha256": "c3a2bea22aa6872940125f4aaac72c9eac72084fefc6c2cefdb54b5273361e4a", + "sha256": "b5d4d752704b886da20d860743b81dc3d2ce3a3c402208246fadc1c6cb71f597", "role": "Declares the eight first-party foundation distributions as OpenHCS dependencies." }, { @@ -861,16 +861,34 @@ "path": "benchmark/contracts/run_receipt.py", "sha256": "4cd61bd33f8f6199b3cc4130ed091dddde9c3a23975e132b1bee06c3c6a43cd7", "role": "Declares the typed benchmark run receipts." + }, + { + "path": "external/ObjectState/src/objectstate/codegen.py", + "sha256": "347d2697c061a35e6e440e373fe4e8bfce1ad92b500e98da07faa26f6a3a1356", + "role": "Registers the pycodify formatters that write lazy dataclasses from their raw field values." + }, + { + "path": "external/ObjectState/src/objectstate/scope_key.py", + "sha256": "d0dd6e56cebf413d6869b07c7b9d8668ba954ed5aeead36c70288920a5b412b2", + "role": "Declares ScopeKey, the scope grammar ObjectState and pyqt-reactive share." + }, + { + "path": "external/pycodify/src/pycodify/paths.py", + "sha256": "f74983cd801215505bbd94f33d42718fa8ba8efdebbbdbfa5174bc56496af2e1", + "role": "Implements document-level factoring of absolute paths in generated source." + }, + { + "path": "external/pyqt-reactive/src/pyqt_reactive/widgets/__init__.py", + "sha256": "ddcf678b0dc5c03aaf11d2b6f12bb9cda681a29f0105925c51a5b9263d751716", + "role": "Exports the generic widget set, including the editable table, grid selector and time-travel widgets." } ], "findings": [ - "The OpenHCS-owned area map omitted openhcs/domains (G1: domain declarations such as the microscopy axis family) and openhcs/authoring (U1: the session every client renders).", - "openhcs/core was described without the axis family, dataset sources and callable/processing contracts it now owns, and openhcs/processing was credited with callable decorators/contracts that live in openhcs/core and ArrayBridge.", - "openhcs/microscopes now holds dataset sources (G4); the description was updated.", - "Dropped config.py, unified_registry.py, pipeline_import.py, viewer_protocol.py and the benchmark CLI from the authorities; the page's claims do not rest on them." + "The foundations list predated U3 and L6: pyqt-reactive now owns the editable table, grid selector, time-travel widgets and windows moved out of openhcs/pyqt_gui; ObjectState owns ScopeKey and the lazy-dataclass source formatters (objectstate.codegen); pycodify owns document-level path factoring. The three lines were updated and the owners added as authorities." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/concepts/module_structure.rst in full and compared it against the changed authority pyproject.toml (zmqruntime 0.6.2, pycodify 0.2.0, objectstate 2.1.1, python-introspect 0.2.7, pyqt-reactive 0.5 floors) and the U3/L6 moves behind them; confirmed the openhcs/ area map, the eight foundations and the OMERO classes, and updated the ObjectState, pyqt-reactive and pycodify ownership lines.", "On 2026-10-11 (B6 final), compared docs/source/concepts/module_structure.rst against B6's later changes to openhcs/microscopes/omero.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/concepts/module_structure.rst against the later B6 changes to pyproject.toml, openhcs/microscopes/omero.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/concepts/module_structure.rst in full and compared it against the changed authorities: pyproject.toml, external/python-introspect/README.md, openhcs/core/config.py, unified_registry.py, pipeline_import.py, openhcs/runtime/viewer_protocol.py, external/zmqruntime client, ArrayBridge README and types, metaclass-registry discovery, benchmark CLI, run_receipt.py and control.py; listed the current openhcs/ top-level packages, confirmed the OMERO address and parser classes, the eight foundation distributions and the benchmark MCP capabilities (now session-backed), and added the domains and authoring areas.", @@ -917,7 +935,7 @@ "authority": [ { "path": "openhcs/core/pipeline_document.py", - "sha256": "8c554c8275c665f308eb098955bda517aa804eeb34418b5e0f41d412650ce7b5", + "sha256": "9a77a08367f901594859c8b6cae9609c9230a551f4f4e9ec6ea6cd2ed233c0f4", "role": "PipelineDocument holds a PipelineConfig and an ordered FunctionStep list." }, { @@ -927,12 +945,12 @@ }, { "path": "openhcs/core/config.py", - "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", "role": "Declares ProcessingConfig (variable_components, group_by, input_source) and PipelineConfig." }, { "path": "openhcs/core/orchestrator/orchestrator.py", - "sha256": "a61658b107f170c192eea505966c268ab6826826f03586bd64562a44800e1ccc", + "sha256": "99870516da6d64e1858f4dd8799d6e38ab6911b7b02bc29b252d2a23f7687410", "role": "Declares PipelineOrchestrator.compile_pipelines over selected partition values, returning CompiledExecutionBundle." }, { @@ -947,22 +965,19 @@ }, { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "3439d96f3fbfcb104a53019dc150049021d9a8132bd27096f1ccf510eefbe37b", + "sha256": "b96f33b866fe6c9f8118ca70fcb7dff8e6a02cb29d79df67230a5e8b97df8002", "role": "import_cellprofiler_pipeline returns the lowered steps with a populated PipelineConfig." }, { "path": "tests/unit/test_pipeline_document.py", - "sha256": "4df57a9c4f12a009222981bf1ac28554405874f3533656b61344ee94056d1bab", + "sha256": "4fe05636d5a9eb93f576904a774d30653a3686c7c8f61031ec0d316854cf5303", "role": "Proves pipeline-document construction and declaration invariants." } ], - "findings": [ - "The compile section described the CompiledExecutionBundle as holding worker steps and environment decisions; its fields are runtime and transport contexts, worker assignments and a runtime environment plan.", - "\"compiles every selected execution axis\" now reads as every selected partition-axis value (G1 vocabulary); required axes in contracts are now required axis roles.", - "After U1 the GUI no longer compiles through its own path: GUI and MCP clients compile the selected datasets through the OpenHCS session, which reaches the same orchestrator boundary." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/concepts/pipelines_and_steps.rst in full and compared it against the changed authorities: openhcs/core/pipeline_document.py (pycodify generate_python_source after L6), openhcs/core/config.py and openhcs/core/orchestrator/orchestrator.py (K4 moved the source-binding config classes and source_bindings_defaults_to_base into config.py), openhcs/interop/cellprofiler/pipeline_import.py and tests/unit/test_pipeline_document.py; the FunctionStep example, the func pattern forms, PipelineConfig source declarations, CellProfiler import result and PipelineOrchestrator.compile_pipelines returning CompiledExecutionBundle still hold.", "On 2026-10-11 (B6 final), compared docs/source/concepts/pipelines_and_steps.rst against B6's later changes to openhcs/core/config.py, openhcs/core/orchestrator/orchestrator.py, openhcs/authoring/session/compile_batch.py, openhcs/interop/cellprofiler/pipeline_import.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/concepts/pipelines_and_steps.rst against the later B6 changes to openhcs/core/config.py, openhcs/core/orchestrator/orchestrator.py, openhcs/authoring/session/compile_batch.py, openhcs/interop/cellprofiler/pipeline_import.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/concepts/pipelines_and_steps.rst in full and compared it against the changed authorities: the page (example already on Microscopy axes), openhcs/core/pipeline_document.py, openhcs/core/config.py, openhcs/core/pipeline/compiler.py, tests/unit/test_pipeline_document.py; checked the example imports, compile_pipelines signature, CompiledExecutionBundle fields, the session compile path and the CellProfiler import return, and corrected the compile section and axis wording.", @@ -987,12 +1002,12 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", "role": "Declares VFSConfig read/intermediate/materialization backends and the fieldless OpenHCS ZarrConfig and TiffConfig subtypes." }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "b6a7231bc2964d472070731018f6a4935e5f23cf8f238a232045c32244e6545a", + "sha256": "74b591773b7f653c957b2db33a1262925e5c02b71074695b6fc18d6120ed5934", "role": "Declares RuntimeValueStore with explicit replace and retained observation history." }, { @@ -1002,7 +1017,7 @@ }, { "path": "openhcs/core/pipeline/function_contracts.py", - "sha256": "f5e7c74ea3d2948c75388184263cd9f1f17905f820b180f634dfaa7514e1d526", + "sha256": "ffc21dd3021a2316cfbd100c2d655867ce4bec619f2db7288e82aeb11c1930e8", "role": "Declares the @artifact_outputs decorator named by the page." }, { @@ -1027,7 +1042,7 @@ }, { "path": "tests/unit/test_runtime_value_store.py", - "sha256": "4ea189341b761bda993eaa2b6e24b83917568d047f4834d9f44410f7545d1452", + "sha256": "d20ae0d5160c79ba7b7c3291a052a606b3ed7dee3c0cd84a6cf013e17a127a23", "role": "Proves scoped typed runtime-value insertion, replacement and lookup." }, { @@ -1036,11 +1051,10 @@ "role": "Proves main-flow and named-artifact disk-TIFF writes round-trip exactly." } ], - "findings": [ - "The @artifact_outputs sentence (rewritten when special_outputs was removed) was left as one overlong unwrapped line; rewrapped to the page's line width." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/concepts/storage_system.rst in full and compared it against the changed authorities: openhcs/core/config.py (PartitionFilterMode family and the source-binding config classes after K4; VFSConfig, ZarrConfig and TiffConfig unchanged), openhcs/core/runtime_stores.py and tests/unit/test_runtime_value_store.py (hand-written RuntimeArtifactLocation/RuntimeArtifactAddress to_dict/from_dict replaced by python-introspect to_jsonable/dataclass_from_mapping after K6), openhcs/core/pipeline/function_contracts.py (CallableDeclaration rename, artifact_input_parameter_names); the page names none of the removed methods, and its backend intents, RuntimeValueStore replace/history and @artifact_outputs claims still hold.", "On 2026-10-11 (B6 final), compared docs/source/concepts/storage_system.rst against B6's later changes to openhcs/core/pipeline/function_contracts.py: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/concepts/storage_system.rst against B6's later changes to openhcs/core/config.py, openhcs/core/runtime_stores.py, openhcs/core/steps/function_artifact_materialization.py, openhcs/core/pipeline/function_contracts.py, tests/unit/test_runtime_value_store.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/concepts/storage_system.rst against the later B6 changes to openhcs/core/config.py, openhcs/core/runtime_stores.py, openhcs/core/steps/function_artifact_materialization.py, openhcs/core/pipeline/function_contracts.py, tests/unit/test_runtime_value_store.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -1071,7 +1085,7 @@ "authority": [ { "path": "pyproject.toml", - "sha256": "c3a2bea22aa6872940125f4aaac72c9eac72084fefc6c2cefdb54b5273361e4a", + "sha256": "b5d4d752704b886da20d860743b81dc3d2ce3a3c402208246fadc1c6cb71f597", "role": "Declares the package version, Python requirement, dependency extras, scripts, and distribution metadata." }, { @@ -1150,12 +1164,10 @@ "role": "Proves the documented installed MCP verification flow." } ], - "findings": [ - "The manual-installation note said the cellprofiler-compat extra requires Python 3.13 or earlier because of its parity dependency; that extra declares only mahotas and matplotlib, Centrosome is a development-only dependency, and CI installs dev,cellprofiler-compat on Python 3.14 on every OS. The note was removed.", - "Seven authorities were moved by commit 73a2aece2 into the openhcs.desktop package and tests/unit/desktop; the record now names openhcs/desktop/installation.py, update.py, update_worker.py, deployment.py and their moved tests." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/getting_started/getting_started.rst in full and compared it against the changed authorities: pyproject.toml (first-party floors moved to zmqruntime 0.6.2, pycodify 0.2, objectstate 2.1.1, python-introspect 0.2.7, pyqt-reactive 0.5). The page states no dependency floors; rechecked requires-python (3.11-3.14) and every named extra (gui, viz, bioformats, mcp, cellprofiler-compat, napari, fiji, gpu) against pyproject.toml; no claim changes.", "On 2026-10-11 (B6 final), compared docs/source/getting_started/getting_started.rst against B6's later changes to openhcs/mcp/installed_demo.py: the writability probe makes one exclusive create attempt, the installed demo reports ViewerType.value, the unit session pre-creates its QApplication, and the GUI journey waits for each batch to end. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/getting_started/getting_started.rst against B6's later changes to openhcs/mcp/installed_demo.py, .github/workflows/integration-tests.yml: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/getting_started/getting_started.rst against B6's later changes to tests/unit/test_installed_mcp_demo.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", @@ -1240,17 +1252,17 @@ "authority": [ { "path": "openhcs/pyqt_gui/main.py", - "sha256": "01dcb82b5850f3cdb1c300a2aa360ba330e7f15dffb5a24e37504311095090f9", + "sha256": "3cdb7819f09604046ba69e0b4f57b73e4e98d8f68af80158501a54bb42e701a2", "role": "Dock layout (system monitor above, Datasets and ZMQ server manager left, Pipeline Editor right), View/Tools/Help menus, and preview-policy application to both panes." }, { "path": "openhcs/pyqt_gui/widgets/plate_manager.py", - "sha256": "bf77012db6c0984deebf09bc5d696d8ea8f273ff535ad964354b6ddbfe839791", + "sha256": "e816cdafd4795c07b5d177a04472221057230416311d1bf83c5bf65188d2aa00", "role": "Implements plate import, selection, pipeline assignment, and execution controls." }, { "path": "openhcs/pyqt_gui/widgets/pipeline_editor.py", - "sha256": "8e418631fd9ba4f8ac3d6b257f9af127f9e8711281608516ea2617d7855d6f36", + "sha256": "7b3b58f3685d5a636a102cae7d1e31c663d5d574bcdc71d4a289b69a9eee6535", "role": "Implements step editing, pipeline compilation controls, and declaration feedback in the main UI." }, { @@ -1260,12 +1272,12 @@ }, { "path": "tests/unit/pyqt_gui/test_ui_agent_bridge.py", - "sha256": "de6758859d9743e826369162855327e4a84056cafdd7ddfdc33868d2a3093353", + "sha256": "67cdad5133dc83ccb005b58cc56535ae89e77fc23d3ebd76e616c0da5f567b44", "role": "Proves bidirectional UI-bridge inspection and mutation of live declarations, plus truthful rejection of unsupported window-navigation targets." }, { "path": "openhcs/core/pipeline_document.py", - "sha256": "8c554c8275c665f308eb098955bda517aa804eeb34418b5e0f41d412650ce7b5", + "sha256": "9a77a08367f901594859c8b6cae9609c9230a551f4f4e9ec6ea6cd2ed233c0f4", "role": "PipelineDocumentCodec rule that pipeline_steps is required and a missing pipeline_config selects PipelineConfig()." }, { @@ -1275,7 +1287,7 @@ }, { "path": "openhcs/pyqt_gui/widgets/shared/services/pipeline_editor_workflows.py", - "sha256": "c704ef7e6c5f8a5745c741e923bc4e620a321fdce069e1cdcdf521c0176e4a4f", + "sha256": "06e54377bc594beef653baa9cfaf4de0031e754da1e7b1ab3a22ac67a922b68f", "role": "Applies Pipeline Editor documents and commits the reconciled editor state graph." }, { @@ -1285,7 +1297,7 @@ }, { "path": "tests/unit/pyqt_gui/test_pipeline_editor_widget.py", - "sha256": "0ce88271aa7f5f55a8fc4daec8279d732844e03dfd437c11e6e6265ca196b4a0", + "sha256": "2024d9560dec38faee1fa922d2f8b7f547dd53d35e0b0e79376e45233445589a", "role": "Proves Pipeline Editor code apply leaves the reconciled root, step, and nested function states saved." }, { @@ -1295,7 +1307,7 @@ }, { "path": "openhcs/pyqt_gui/services/ui_bridge_windows.py", - "sha256": "e76298f78e3cbd949e8426a2b42cd6768cc54770a5f45c9dd6c9f788b257785e", + "sha256": "29265ed04137392757a6d515020920120e29fd556f433575956ae38cedff0158", "role": "Projects embedded manager navigation through the generic declared driver." }, { @@ -1359,12 +1371,10 @@ "role": "plate_manager, pipeline_editor and main_window widget identities and stable window titles used by MCP navigation." } ], - "findings": [ - "The page called the left dataset pane Plate Manager; since the session interface (U1) the pane and its View menu entry are titled Datasets (DatasetListView), while MCP still identifies the window as plate_manager. The page now uses the visible name and gives the MCP identity once.", - "Agent navigation step 6 said to poll operation/state surfaces; openhcs_ui_invoke_action now returns an event_sequence and work is followed through openhcs_ui_session_events, so the step names session events." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/basic_interface.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/main.py, widgets/plate_manager.py, widgets/pipeline_editor.py, shared/services/pipeline_editor_workflows.py, services/ui_bridge_windows.py, openhcs/core/pipeline_document.py and their tests (U3: DockPane, DockLayoutStore, ShortcutLifecycle, time-travel navigation, path cache and the external-editor switch moved to pyqt-reactive, GlobalEventBus broadcasts replaced by session events, ScopeKey scope ids; L6: pipeline documents render through pycodify generate_python_source; K6 debug command classes). The page describes behaviour, not those classes; rechecked the dock arrangement (system monitor above, Datasets and ZMQ server manager stacked left, Pipeline Editor right), the plate_manager window identity, list_previews on UIConfig, and that PipelineDocument still requires pipeline_steps; no claim changes.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/basic_interface.rst against B6's later changes to openhcs/authoring/session/views.py: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/basic_interface.rst against B6's later changes to openhcs/pyqt_gui/main.py, openhcs/pyqt_gui/widgets/pipeline_editor.py, openhcs/agent/services/ui_bridge_service.py, tests/unit/pyqt_gui/test_ui_agent_bridge.py, tests/unit/pyqt_gui/test_pipeline_code_document_transactionality.py, openhcs/pyqt_gui/widgets/shared/services/pipeline_editor_workflows.py, tests/unit/pyqt_gui/test_pipeline_editor_widget.py, openhcs/pyqt_gui/services/ui_bridge_windows.py, tests/unit/pyqt_gui/test_main_config_propagation.py, openhcs/authoring/session/views.py, openhcs/authoring/session/operations/datasets.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/basic_interface.rst against the later B6 changes to openhcs/pyqt_gui/main.py, openhcs/pyqt_gui/widgets/pipeline_editor.py, openhcs/agent/services/ui_bridge_service.py, tests/unit/pyqt_gui/test_ui_agent_bridge.py, tests/unit/pyqt_gui/test_pipeline_code_document_transactionality.py, openhcs/pyqt_gui/widgets/shared/services/pipeline_editor_workflows.py, tests/unit/pyqt_gui/test_pipeline_editor_widget.py, openhcs/pyqt_gui/services/ui_bridge_windows.py, tests/unit/pyqt_gui/test_main_config_propagation.py, openhcs/authoring/session/views.py, openhcs/authoring/session/operations/datasets.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -1411,12 +1421,12 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", "role": "Declares the supported OpenHCS configuration dataclasses, nested scopes, defaults, and field help metadata." }, { "path": "external/ObjectState/src/objectstate/object_state.py", - "sha256": "03f818c3045b6a07024444ab2b5552fb6be9eab32dd151bb165b6f904d2863a3", + "sha256": "413a7bcd27695a823805cbc16a393569e30293a70c8d60d66dc6d56d3b589a4a", "role": "Implements ObjectState provenance, inheritance resolution, and field-change propagation used by nested configuration objects." }, { @@ -1431,12 +1441,12 @@ }, { "path": "openhcs/pyqt_gui/services/ui_bridge_object_state.py", - "sha256": "e548b367507b3fb6830ce9ecb844e3b756243af96fe38e0d35de5da0b974c1fd", + "sha256": "e64b444a74fdde1d79f82c1361c9b67d8e8916b9bebced092762ce51eabac52b", "role": "Projects declaration-derived accepted enum inputs through live ObjectState field help." }, { "path": "tests/unit/pyqt_gui/test_ui_agent_bridge.py", - "sha256": "de6758859d9743e826369162855327e4a84056cafdd7ddfdc33868d2a3093353", + "sha256": "67cdad5133dc83ccb005b58cc56535ae89e77fc23d3ebd76e616c0da5f567b44", "role": "Proves live field help reports the exact accepted inputs for enum-backed configuration fields." }, { @@ -1451,12 +1461,12 @@ }, { "path": "openhcs/core/pipeline_document.py", - "sha256": "8c554c8275c665f308eb098955bda517aa804eeb34418b5e0f41d412650ce7b5", + "sha256": "9a77a08367f901594859c8b6cae9609c9230a551f4f4e9ec6ea6cd2ed233c0f4", "role": "Owns default PipelineConfig construction for source documents that provide pipeline_steps only." }, { "path": "tests/unit/test_pipeline_document.py", - "sha256": "4df57a9c4f12a009222981bf1ac28554405874f3533656b61344ee94056d1bab", + "sha256": "4fe05636d5a9eb93f576904a774d30653a3686c7c8f61031ec0d316854cf5303", "role": "Proves the default-config compatibility rule and the remaining source-document validation constraints." }, { @@ -1470,13 +1480,10 @@ "role": "Declares AnalysisConsolidationConfig, PlateMetadataConfig and the standalone ExperimentalAnalysisConfig named in the post-run section." } ], - "findings": [ - "The mental-model diagram and inheritance paragraph ended in immutable step snapshots; StepSnapshot no longer exists and compilation runs from the resolved step declarations, so both now say resolved declarations.", - "AnalysisConsolidationConfig was described as including a well pattern; the declaration has no such field and instead carries MetaXpress-style formatting, file extensions, exclude patterns and output filenames.", - "The earlier WellFilterConfig to PartitionFilterConfig rename edited the page without review and left the inheritance diagram misaligned; the class and field names were checked against config.py and the diagram realigned." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/configuration_reference.rst in full and compared it against the changed authorities: openhcs/core/config.py (K4: SourceBindingsConfig and StepSourceBindingsConfig now declared here; PartitionFilterMode became the IncludePartitions/ExcludePartitions class family offered through RegisteredClassChoices), external/ObjectState/src/objectstate/object_state.py (current file), openhcs/pyqt_gui/services/ui_bridge_object_state.py (field inspection and provenance moved to objectstate.inspection), openhcs/core/pipeline_document.py (pycodify generate_python_source) and their tests. The page states include/exclude selection without naming enum members, and still-accurate omitted pipeline_config -> PipelineConfig() behaviour; rechecked every config class it names, TiffConfig's PolyStore DEFLATE codec, the microscopy-domain AnalysisConsolidationConfig/PlateMetadataConfig/ExperimentalAnalysisConfig, and openhcs_describe_config_schema; no claim changes.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/configuration_reference.rst against B6's later changes to openhcs/core/config.py, openhcs/pyqt_gui/services/ui_bridge_object_state.py, tests/unit/pyqt_gui/test_ui_agent_bridge.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/configuration_reference.rst against the later B6 changes to openhcs/core/config.py, openhcs/pyqt_gui/services/ui_bridge_object_state.py, tests/unit/pyqt_gui/test_ui_agent_bridge.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/configuration_reference.rst in full and compared it against the changed authorities: the page's own unreviewed WellFilter-to-PartitionFilter rename, openhcs/core/config.py, ObjectState object_state.py, config_service.py, python-introspect annotation_types.py, ui_bridge_object_state.py, pipeline_document.py, gallery_catalog.py and their tests; imported GlobalPipelineConfig/PipelineConfig and AbstractStep to confirm every named config field and the step-overridable families, checked the PartitionFilterConfig inheritance tree, InputSource values, main_flow_axis_persistence_enabled, openhcs_describe_config_schema and the four gallery scenarios; replaced the step-snapshot wording, corrected the consolidation field summary and realigned the diagram.", @@ -1519,7 +1526,7 @@ }, { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "3439d96f3fbfcb104a53019dc150049021d9a8132bd27096f1ccf510eefbe37b", + "sha256": "b96f33b866fe6c9f8118ca70fcb7dff8e6a02cb29d79df67230a5e8b97df8002", "role": "Imports CellProfiler modules into ordinary OpenHCS pipeline declarations through the interop boundary." }, { @@ -1529,7 +1536,7 @@ }, { "path": "tests/unit/test_cellprofiler_pipeline_import.py", - "sha256": "6b1d201ec5c4ba620fa2367a9cbef1f6fbdd25edbdda9bd7b723ebe3a3363412", + "sha256": "0ce4efd85d819c7e9f7fa84ec6f1ac7ec4bda024c9a8d5c7f264fda2730264f0", "role": "Proves direct import rejects modules requiring interactive desktop input instead of omitting them." }, { @@ -1554,12 +1561,12 @@ }, { "path": "openhcs/agent/services/architecture_projection_service.py", - "sha256": "b209098d78c5520fd34d744d17b10c1ef0375a9bc2655449f6f3f9908decce0e", + "sha256": "66e19059af19bdbd9e0f06b8bb5dad43e34d57210b3fb230f6c998538d2fea4d", "role": "Declares the cellprofiler_translation architecture topic served by openhcs_explain_architecture." }, { "path": "openhcs/interop/cellprofiler/module_artifact_declarations.py", - "sha256": "f70b52114fb5d5e2c2fc49cd9a2b46986bde48ce02d82f1a34a67f072ca2ab20", + "sha256": "ef233be43002d0f16b53c78c202b9203b6261847708d311089671a6b037db894", "role": "Raises the import error for CellProfiler modules that require interactive desktop input." }, { @@ -1568,11 +1575,10 @@ "role": "DatasetListView title for the desktop pane that UI-owned work populates." } ], - "findings": [ - "The MCP section said a headless draft does not create Plate Manager state; the desktop pane is now titled Datasets (session DatasetListView), so the sentence names the Datasets pane." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/domain_expert_onboarding.rst in full and compared it against the changed authorities: openhcs/interop/cellprofiler/pipeline_import.py, module_artifact_declarations.py, openhcs/agent/services/architecture_projection_service.py and tests/unit/test_cellprofiler_pipeline_import.py (K4: SourceBindingsConfig and StepSourceBindingsConfig now imported from openhcs.core.config, order matching is PairByOrder). The page names neither import location; rechecked the cellprofiler_translation topic id, the interactive-input import error, and the DatasetListView 'Datasets' title; no claim changes.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/domain_expert_onboarding.rst against B6's later changes to openhcs/authoring/session/views.py: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/domain_expert_onboarding.rst against B6's later changes to benchmark/cellprofiler_benchmark_cli.py: the session decides in-place initialization by an actual file-creation attempt that leaves nothing behind, its debug-session methods moved into an inherited mixin, the measured CLI treats Windows Ctrl+Break like Ctrl+C, and the Official30 viewer test reads typed payload summaries. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/domain_expert_onboarding.rst against B6's later changes to openhcs/interop/cellprofiler/pipeline_import.py, benchmark/cellprofiler_benchmark_cli.py, tests/unit/test_benchmark_control.py, openhcs/interop/cellprofiler/module_artifact_declarations.py, openhcs/authoring/session/views.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", @@ -1612,12 +1618,12 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", "role": "ProcessingConfig declares variable_components as the stack-axis authority and group_by as post-assembly partitioning." }, { "path": "openhcs/core/processing_contracts.py", - "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", + "sha256": "eb0f7fd99ded1c4d5957152ccf6abe1468436c13574a1d1bb83ce368d24b9f69", "role": "ProcessingContract family (for example Pure2DContract and WholeStackInputContract) declares whether a callable runs per plane or on the whole stack." }, { @@ -1629,6 +1635,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/glossary.rst in full and compared it against the changed authorities: openhcs/core/config.py (PartitionFilterMode subclass family, source-binding configs moved in by K4) and openhcs/core/processing_contracts.py (CallableDeclaration rename); variable_components and group_by remain on the processing config and the ProcessingContract family (Pure2DContract, WholeStackInputContract) still states plane-local versus whole-stack, so no correction was needed.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/glossary.rst against B6's later changes to openhcs/core/config.py, openhcs/core/processing_contracts.py, openhcs/core/axes.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/glossary.rst against the later B6 changes to openhcs/core/config.py, openhcs/core/processing_contracts.py, openhcs/core/axes.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/glossary.rst in full and compared it against the changed authorities: openhcs/constants/constants.py (component enums removed by the axis-family refactor), lib_registry/unified_registry.py, openhcs/core/config.py; confirmed variable_components and group_by docstrings in ProcessingConfig and that ProcessingContract still exists as the per-plane versus whole-stack family in processing_contracts.py; the three distinctions remain accurate. Replaced constants.py and unified_registry.py, which no longer own these facts, with processing_contracts.py and core/axes.py.", @@ -1644,7 +1651,7 @@ }, { "path": "docs/source/guide_for_biologists/image_sources.rst", - "source_sha256": "6ba98557f6404d739a12fbf152167b23066920504f76b497d67d1ae0527c2d1b", + "source_sha256": "38ad499634f2423f4360a3f9b3bfdd96d39dca0327c9668ab5e5fd801ae4288a", "audience": [ "imaging scientists", "source-integration operators" @@ -1654,27 +1661,27 @@ "authority": [ { "path": "openhcs/core/source_bindings.py", - "sha256": "1dfbd51ae4c64d6a6a8e5ff5ab2ba82465c5e0adacb98dddaf819e0fea05e241", - "role": "Declares source bindings that map logical pipeline inputs to microscope-backed workspaces." + "sha256": "2defaf02ac0e214bb63f74d6eb56f294fbb5db4e06ecd9cc4479b16848f09854", + "role": "Declares NamedSourceBinding (required defaults to True, component_identity), SourceSelector, ComponentSelector and the SourceFilterClause/Subject/MatchType types imported by the code examples." }, { "path": "openhcs/core/source_workspace_projection.py", - "sha256": "d833421b963550aa5fe428c68c0286216437aec97834e4204b0b88def011f98b", + "sha256": "2d2b6bf5a331cb61411b293b333980a9951f9ec1cd8537874d6529949180f3ec", "role": "Projects source bindings and persisted semantic metadata into the concrete workspace identities compiled for execution." }, { "path": "tests/unit/test_source_bindings.py", - "sha256": "3d2bca343054aa8aa2d66ce1136ad23d1805d2c800134ee5682fac91feabaadd", + "sha256": "28eefba0de5b84f9a0f928225590d70f8ed87d5048e7f4597cdb5e1e61b3fe45", "role": "Proves source-binding validation and projection into concrete workspace identities." }, { "path": "openhcs/core/source_binding_workspace.py", - "sha256": "4d28066a9f73b283fc637662e05fb2632c6fa18af164476c2899fce58cd647fe", + "sha256": "5e7ac439d7ef6cd35a5660d2ad9db24f1890801d8bdf1d82c13b12f8baaa9eb0", "role": "Selects raw candidates and projects binding-owned component identity." }, { "path": "tests/unit/test_source_binding_workspace.py", - "sha256": "2d3582e109bfd2541a1c217cea4209674e7603dd0443f06733644d35ed80a55a", + "sha256": "c11a1e40bebc3b025ecad2faeeb1fb1e5421f59042cb91d2428dfbf550112374", "role": "Proves raw selector coordinates are retained as original metadata while the projected address uses semantic component identity." }, { @@ -1709,12 +1716,12 @@ }, { "path": "tests/unit/test_plane_store_sources.py", - "sha256": "f0f795651e4e3d48da30a6352088234d3ac8dcdce3a90c93e86c420491d02c1f", + "sha256": "806ca01b3d5aae89c1601956f1f5f0cb556abc81d899cea25b5489a4fb6759f5", "role": "Proves both physical NGFF formats retain axis cardinalities, pixels, source identities, bindings, materialisation, and reinitialisation through OpenHCS." }, { "path": "tests/integration/test_source_store_code_mode_zmq.py", - "sha256": "2bc8c0ae1d365e6d62febab95890daf8641bb12c7a07e4634f5a3178851837fb", + "sha256": "1330cbcbc6c2a9cc9e320db91e9bb7ff28c2768c8b9537fb3d6f5eb09861331e", "role": "Exercises mixed-store code documents, source projection and the opt-in compile-then-execute ZMQ path for both NGFF formats." }, { @@ -1734,27 +1741,27 @@ }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "b6a7231bc2964d472070731018f6a4935e5f23cf8f238a232045c32244e6545a", + "sha256": "74b591773b7f653c957b2db33a1262925e5c02b71074695b6fc18d6120ed5934", "role": "RuntimeArtifactInput resolves exact artifact addresses and checks execution context through the declared image-set relation; it does not select arbitrary same-shape labels." }, { "path": "openhcs/core/source_matching.py", - "sha256": "c608880341d5c40d745ebf4254d8a2931ae611c354a6fe5bb5b3d18611679c63", + "sha256": "f32db4e25a0b6b7c1a334ee27c7206044e1ba645717ee12f6acd69d28d9b30a8", "role": "SourceImageSetIdentityPolicy derives plane membership and retained sample identity from source declarations." }, { "path": "openhcs/interop/cellprofiler/module_artifact_declarations.py", - "sha256": "f70b52114fb5d5e2c2fc49cd9a2b46986bde48ce02d82f1a34a67f072ca2ab20", + "sha256": "ef233be43002d0f16b53c78c202b9203b6261847708d311089671a6b037db894", "role": "The existing object-measurement declaration ancestor derives label input image-set relations from its selected image specs." }, { "path": "openhcs/core/dataset_sources/source.py", - "sha256": "097fde4d2b74031eacf53a88e79f499417e8c1c97ba0753e8a9df831b307efa6", + "sha256": "350bf5691a03522e9f54b1d8ab712c6beba1683aac4fa5159ea4e80c6041530d", "role": "DatasetSource family with format-specific, broad structured-store and declared-file selection roles and their detection order." }, { "path": "openhcs/core/dataset_sources/source_bindings_source.py", - "sha256": "fccda877f39e7b2f99d6cc95143480e5955186264e316e832ca9a69c2ae90b8c", + "sha256": "63e4a3f849ab5122ab7af8767a6288135b7330027dff962c80add93d5fd820f1", "role": "SourceBindingsSource, the declared-file fallback for unrecognized folders driven by source_bindings_config." }, { @@ -1771,16 +1778,20 @@ "path": "openhcs/core/axes.py", "sha256": "8d93133639c576050f089fb8aab99937c8982aff07e742fea44f8a12e62bb7b2", "role": "Ungrouped grouping declaration that overlapping non-dict group_by normalizes to." + }, + { + "path": "openhcs/core/config.py", + "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", + "role": "Declares SourceBindingsConfig and StepSourceBindingsConfig with their lazy forms LazySourceBindingsConfig and LazyStepSourceBindingsConfig, and PipelineConfig.source_bindings_config, used by the code-mode examples." } ], "findings": [ - "The page named SourceBindingsHandler and MicroscopeHandler, both deleted by the G4 dataset-source refactor (c6d9a52ce); the fallback is now SourceBindingsSource and ingestion owners are DatasetSource subclasses with selection roles. Handler wording was changed to dataset source throughout.", - "openhcs_inspect_plate_path no longer takes microscope_type; the request field is source_format (\"auto\", \"opera_phenix\", ...).", - "Prose used the deleted component enum spellings SITE, CHANNEL, WELL, Z_INDEX, TIMEPOINT and group_by=NONE; they now read Microscopy.Site/Channel/Well/ZIndex/Timepoint and Ungrouped, matching the axis family and the already-updated code example.", - "The page told users to add directories to the Plate Manager; the desktop pane is now titled Datasets." + "The page said 'A failed receipt remains preserved'; 'receipt' is internal jargon, so it now says to keep the record of a failed run.", + "K4 moved SourceBindingsConfig, StepSourceBindingsConfig and their lazy forms into openhcs/core/config.py; the page already imports the lazy forms from openhcs.core.config, so its claims hold, but config.py is added as the authority for them and the source_bindings.py role is narrowed to the binding, selector and filter types the examples import." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/image_sources.rst in full and compared it against the changed authorities: openhcs/core/source_bindings.py (K4: SourceBindingsConfig/StepSourceBindingsConfig moved to openhcs/core/config.py, match plan became the SourceSetPairing family PairByOrder/PairByMetadata), openhcs/core/source_workspace_projection.py, openhcs/core/source_binding_workspace.py, openhcs/core/runtime_stores.py, openhcs/core/source_matching.py, openhcs/interop/cellprofiler/module_artifact_declarations.py, openhcs/core/dataset_sources/source.py, openhcs/core/dataset_sources/source_bindings_source.py and their tests; confirmed every symbol the code blocks import still exists at its import path (validate_docs reports no import or code-block finding for the page), NamedSourceBinding.required defaults to True, and InputImageSetContextSourceRelation and SourceImageSetIdentityPolicy are unchanged; replaced the 'failed receipt' wording and added openhcs/core/config.py as an authority.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/image_sources.rst against B6's later changes to openhcs/core/source_workspace_projection.py, tests/unit/test_source_bindings.py, openhcs/core/source_binding_workspace.py, tests/unit/test_source_binding_workspace.py, tests/unit/test_plane_store_sources.py, openhcs/core/artifacts.py, openhcs/core/runtime_stores.py, openhcs/interop/cellprofiler/module_artifact_declarations.py, openhcs/core/dataset_sources/source.py, openhcs/core/dataset_sources/source_bindings_source.py, openhcs/agent/dto/plate.py, openhcs/domains/microscopy/axes.py, openhcs/core/axes.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/image_sources.rst against the later B6 changes to openhcs/core/source_bindings.py, openhcs/core/source_workspace_projection.py, tests/unit/test_source_bindings.py, openhcs/core/source_binding_workspace.py, tests/unit/test_source_binding_workspace.py, tests/unit/test_plane_store_sources.py, openhcs/core/artifacts.py, openhcs/core/runtime_stores.py, openhcs/interop/cellprofiler/module_artifact_declarations.py, openhcs/core/dataset_sources/source.py, openhcs/core/dataset_sources/source_bindings_source.py, openhcs/agent/dto/plate.py, openhcs/domains/microscopy/axes.py, openhcs/core/axes.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/image_sources.rst in full and compared it against the changed authorities: the page's own unreviewed axis-family edit, microscope_interfaces.py and source_bindings_handler.py (deleted in c6d9a52ce), source_bindings.py, source_workspace_projection.py, pipeline/compiler.py, source_binding_workspace.py, bioformats_adapter.py, PolyStore zarr.py and config.py, artifacts.py, runtime_stores.py, source_matching.py, module_artifact_declarations.py and their tests; replaced the deleted handler classes with DatasetSource / SourceBindingsSource, microscope_type with source_format, deleted component enum spellings with Microscopy axes and Ungrouped, and Plate Manager with the Datasets pane; confirmed source_stack_components, ImportedMetadataTable, InputImageSetContextSourceRelation, SourceFilterSubject/MatchType, viewer route keys and both code blocks' imports still exist.", @@ -1811,7 +1822,7 @@ "authority": [ { "path": "openhcs/pyqt_gui/main.py", - "sha256": "01dcb82b5850f3cdb1c300a2aa360ba330e7f15dffb5a24e37504311095090f9", + "sha256": "3cdb7819f09604046ba69e0b4f57b73e4e98d8f68af80158501a54bb42e701a2", "role": "Composes the main desktop window, menus, task windows, and registered UI services." }, { @@ -1828,6 +1839,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/index.rst in full and compared it against the changed authority openhcs/pyqt_gui/main.py (U3: dock layout, shortcut lifecycle and time-travel navigation now come from pyqt-reactive); the page only routes readers and names no main-window mechanism, so no correction was needed.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/index.rst against B6's later changes to openhcs/pyqt_gui/main.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/index.rst against the later B6 changes to openhcs/pyqt_gui/main.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/index.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/main.py, openhcs/core/pipeline_document.py, openhcs/microscopes/microscope_interfaces.py (deleted in c6d9a52ce); the page states no fact owned by the deleted microscope interface or by the pipeline document, so both were replaced by the synthetic-plate generator window and the eight-step synthetic_plate_pipeline.py that back its only product claim; the toctree and links remain correct.", @@ -1852,7 +1864,7 @@ "authority": [ { "path": "pyproject.toml", - "sha256": "c3a2bea22aa6872940125f4aaac72c9eac72084fefc6c2cefdb54b5273361e4a", + "sha256": "b5d4d752704b886da20d860743b81dc3d2ce3a3c402208246fadc1c6cb71f597", "role": "Declares the package version, Python requirement, dependency extras, scripts, and distribution metadata." }, { @@ -1872,10 +1884,11 @@ } ], "findings": [ - "The page is still only a redirect to getting_started; it restates no installer, package or entry-point fact, so the changed pyproject.toml, cli.py and native installer scripts do not make it wrong. A second installation procedure here would duplicate volatile installer facts." + "The page is still only a redirect to getting_started; the pyproject.toml dependency-floor changes (zmqruntime 0.6.2, pycodify 0.2, objectstate 2.1.1, python-introspect 0.2.7, pyqt-reactive 0.5) touch no claim it makes." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/installation_and_setup.rst in full and compared it against the changed authorities: pyproject.toml (dependency floors only); the page restates no version or installer fact, so the redirect disposition stands.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/installation_and_setup.rst against the later B6 changes to pyproject.toml: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/installation_and_setup.rst in full and compared it against the changed authorities: pyproject.toml, openhcs/cli.py, packaging/installers/windows/Install-OpenHCS.ps1, packaging/installers/macos/install-openhcs.sh; the page contains only the redirect sentence and its target getting_started.rst was re-reviewed in this batch, so the redirect disposition stands.", "On 2026-08-27, re-read docs/source/guide_for_biologists/installation_and_setup.rst in full after the python-introspect dependency floor advanced to 0.1.12; confirmed that its redirect still reaches the single maintained installation how-to.", @@ -1919,35 +1932,34 @@ "authority": [ { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "sha256": "4a33908fb42874e6d57eeea3f5305793ddbda41d13356f0a78501af38dcdfeb4", "role": "Compiles pipeline declarations into validated step plans and execution bundles." }, { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "3439d96f3fbfcb104a53019dc150049021d9a8132bd27096f1ccf510eefbe37b", + "sha256": "b96f33b866fe6c9f8118ca70fcb7dff8e6a02cb29d79df67230a5e8b97df8002", "role": "Imports CellProfiler modules into ordinary OpenHCS pipeline declarations through the interop boundary." }, { "path": "openhcs/processing/backends/lib_registry/registry_service.py", - "sha256": "daef8925338fddf8504a9faa066af42456a13fe4602976bf64be06547a45fcc8", + "sha256": "f2e981157632adb5020828720ec26bfa6444ab52f545e1e71b1524242eaed6a3", "role": "Owns backend discovery, availability filtering, and the canonical process catalogue projection." }, { "path": "openhcs/pyqt_gui/widgets/plate_manager.py", - "sha256": "bf77012db6c0984deebf09bc5d696d8ea8f273ff535ad964354b6ddbfe839791", + "sha256": "e816cdafd4795c07b5d177a04472221057230416311d1bf83c5bf65188d2aa00", "role": "Implements dataset import, selection, initialization, compilation and run controls in the Datasets pane." }, { "path": "openhcs/core/dataset_sources/choice.py", - "sha256": "7fb3c7ce701f0d629f288b79090044f43ac8ce1b51ca4f50937534407f226f1a", + "sha256": "497eb028aabb80283be8a621ea94e09881dfcd51b2e34e2bbecafa09e004c1f6", "role": "DatasetSourceChoices projects the registered dataset sources, plus auto, as the dataset_source field's choices." } ], - "findings": [ - "The page pointed readers to the Plate Manager as the catalogue of available microscope formats; the pane is now titled Datasets and lists datasets, while the registered formats are the dataset_source choices (DatasetSourceChoices) shown in the configuration form. The sentence now names those choices." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/intro.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compiler.py (ScopeKey scope ids, K6 progress classes), openhcs/interop/cellprofiler/pipeline_import.py and openhcs/core/dataset_sources/choice.py (source-binding config imports from openhcs.core.config), openhcs/processing/backends/lib_registry/registry_service.py (reference.declaration replaces reference.metadata) and openhcs/pyqt_gui/widgets/plate_manager.py (U3: path cache and config window from pyqt-reactive, GlobalEventBus broadcast removed); the page's capability, dataset_source choice and compile-before-run claims still hold, so no correction was needed.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/intro.rst against B6's later changes to openhcs/core/pipeline/compiler.py, openhcs/interop/cellprofiler/pipeline_import.py, openhcs/processing/backends/lib_registry/registry_service.py, openhcs/core/dataset_sources/choice.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/intro.rst against the later B6 changes to openhcs/core/pipeline/compiler.py, openhcs/interop/cellprofiler/pipeline_import.py, openhcs/processing/backends/lib_registry/registry_service.py, openhcs/core/dataset_sources/choice.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/intro.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compiler.py, openhcs/interop/cellprofiler/pipeline_import.py, lib_registry/registry_service.py, pyqt_gui/widgets/plate_manager.py; the fit list, CellProfiler import claim and compile-before-run flow remain accurate (Run requires a compiled selection in the session operations); replaced the Plate Manager format-catalogue pointer with the dataset_source choices declared in openhcs/core/config.py and dataset_sources/choice.py.", @@ -1996,7 +2008,7 @@ }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "01dcb82b5850f3cdb1c300a2aa360ba330e7f15dffb5a24e37504311095090f9", + "sha256": "3cdb7819f09604046ba69e0b4f57b73e4e98d8f68af80158501a54bb42e701a2", "role": "Places Generate Synthetic Plate in the View menu." }, { @@ -2005,12 +2017,10 @@ "role": "Owns the documentation-enabled synthetic generator, plate metadata, and live measurement result capture declarations used by the first-workflow tutorial." } ], - "findings": [ - "The page sent readers to an unnamed 'metadata viewer'; after U1 the dataset action is the Viewer operation (ShowDatasetImages) with a Metadata tab, so the step now names that button and tab.", - "The end-to-end GUI foundation test no longer adds evidence for this tutorial's claims beyond the generator and session operations; it was replaced as an authority by the operation declarations and the menu owner." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/intro_stitching.rst in full and compared it against the changed authority openhcs/pyqt_gui/main.py (U3 dock, shortcut and time-travel moves to pyqt-reactive); the View menu still holds the Generate Synthetic Plate action, so no correction was needed.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/intro_stitching.rst against B6's later changes to openhcs/authoring/session/operations/datasets.py, openhcs/pyqt_gui/main.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/intro_stitching.rst against the later B6 changes to openhcs/authoring/session/operations/datasets.py, openhcs/pyqt_gui/main.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/intro_stitching.rst in full and compared it against the changed authorities: openhcs/demo/synthetic_data.py, openhcs/demo/synthetic_plate_pipeline.py, tests/pyqt_gui/integration/test_end_to_end_workflow_foundation.py, scripts/gallery_catalog.py; confirmed the eight step names, the View menu entry, the generator dialog labels, the Init/Compile/Run/Results labels and the four gallery scenario ids, and replaced the unnamed metadata viewer with the Viewer button and Metadata tab.", @@ -2046,7 +2056,7 @@ }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "sha256": "4a33908fb42874e6d57eeea3f5305793ddbda41d13356f0a78501af38dcdfeb4", "role": "Stops before execution on unsatisfied declarations and applies materialize_runtime_artifacts to automatic terminal outputs." }, { @@ -2066,7 +2076,7 @@ }, { "path": "openhcs/runtime/zmq_execution_client.py", - "sha256": "147fb663d3ea60b1348063440e05b77d54b40634d1ae81cf4fb4f3b5457fbc99", + "sha256": "5df650f60c4d48cbf47c275d4e46b870b35c915ca8baa8e06e77f321578ca6d7", "role": "Implements execution-server connection and retains the .startup.jsonl journal for a server that fails before readiness." }, { @@ -2076,7 +2086,7 @@ }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "01dcb82b5850f3cdb1c300a2aa360ba330e7f15dffb5a24e37504311095090f9", + "sha256": "3cdb7819f09604046ba69e0b4f57b73e4e98d8f68af80158501a54bb42e701a2", "role": "Wires endpoint compatibility to the version-restart workflow and gates the restart operation until initialisation, compilation and execution have finished." }, { @@ -2130,14 +2140,10 @@ "role": "Advertises the current application identity and rejects incompatible MCP requests before UI dispatch." } ], - "findings": [ - "The desktop restart, version-restart and restart-worker modules and their tests moved from openhcs/pyqt_gui/services and tests/unit/pyqt_gui to openhcs/desktop and tests/unit/desktop (D1/U5); the record's authorities pointed at deleted paths.", - "The restart paragraph described an 'action receipt' and 'operation receipt'; the restart operation now states that it returns before the old UI exits and that the old process's operation status does not survive, so the wording follows it.", - "Plate Manager actions are now session operations (U1); Connect, Code, Init, Compile and Run keep the labels the page uses, so no step changed.", - "The isolated-session paragraph called the selected UI configuration 'canonical'; it now says 'shared'." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/troubleshooting_FAQ.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compiler.py (ScopeKey scope ids, K6 progress classes), openhcs/runtime/zmq_execution_client.py (K6: debug requests serialize themselves, command types are DebugCommandType classes; the .startup.jsonl journal beside the server log is unchanged) and openhcs/pyqt_gui/main.py (U3 moves to pyqt-reactive); the restart-and-restore action, OPENHCS_UI_CONFIG_CACHE_FILE, submit_timeout_ms and materialize_runtime_artifacts still exist, so no correction was needed.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/troubleshooting_FAQ.rst against B6's later changes to openhcs/authoring/session/operations/datasets.py, openhcs/core/pipeline/compiler.py, openhcs/runtime/viewer_protocol.py, openhcs/runtime/zmq_execution_client.py, openhcs/pyqt_gui/main.py, tests/unit/desktop/test_desktop_restart.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/troubleshooting_FAQ.rst against the later B6 changes to openhcs/authoring/session/operations/datasets.py, openhcs/core/pipeline/compiler.py, openhcs/runtime/viewer_protocol.py, openhcs/runtime/zmq_execution_client.py, openhcs/pyqt_gui/main.py, tests/unit/desktop/test_desktop_restart.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/troubleshooting_FAQ.rst in full and compared it against the changed authorities: plate_manager.py, compiler.py, environment.py, viewer_protocol.py, zmq_execution_client.py, pyqt_gui/main.py, the moved desktop restart modules and tests, smoke_installed_desktop.py, the catalogue service and UI bridge transport/server; confirmed every named option, environment variable, journal file, button and action label in current source, replaced the receipt wording of the restart paragraph, and repointed restart authorities to openhcs/desktop.", @@ -2196,7 +2202,7 @@ }, { "path": "openhcs/core/processing_contracts.py", - "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", + "sha256": "eb0f7fd99ded1c4d5957152ccf6abe1468436c13574a1d1bb83ce368d24b9f69", "role": "Declares Pure2DContract used in the minimal declaration example." }, { @@ -2206,7 +2212,7 @@ }, { "path": "openhcs/processing/presets/pipelines/loose_operaphenix_neurite_outgrowth.py", - "sha256": "1997c15cf71b8c65c8b49f238fced11b2d7a6c8bd5725ea75bdcd8cf7ab14314", + "sha256": "ad42221c3a03e9bad8bdffce92b56b10a43fd4bb12f12a8dfa947cdb0f63937f", "role": "Declares the example_inputs fields, including source_voxel_spacing, and the UnifiedNeurons result of the loose Opera Phenix example." }, { @@ -2226,17 +2232,14 @@ }, { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "3439d96f3fbfcb104a53019dc150049021d9a8132bd27096f1ccf510eefbe37b", + "sha256": "b96f33b866fe6c9f8118ca70fcb7dff8e6a02cb29d79df67230a5e8b97df8002", "role": "Defines import_cellprofiler_pipeline, which lowers a .cppipe into steps and a pipeline config." } ], - "findings": [ - "The page told readers to use the 'Microscope.OPERAPHENIX handler'; the Microscope enum was removed with the dataset-source family (G4), so the page now names OperaPhenixHandler selected through dataset_source detection.", - "The loose Opera Phenix example gained a source_voxel_spacing input for calibration the copied TIFFs lose with Index.xml; the list of fields to edit now includes it.", - "The Official30 paragraph said 'canonical importer'; it now says 'public importer', matching the CellProfiler import section." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guides/complete_examples.rst in full and compared it against the changed authorities: openhcs/core/processing_contracts.py (CallableMetadata renamed CallableDeclaration internally), openhcs/processing/presets/pipelines/loose_operaphenix_neurite_outgrowth.py and openhcs/interop/cellprofiler/pipeline_import.py (source-binding configs now imported from openhcs.core.config, AuthoredFunctionPattern.first_callable replaces get_core_callable); the page names none of the moved symbols, and Pure2DContract, the openhcs.core.memory.decorators numpy decorator, source_voxel_spacing and import_cellprofiler_pipeline still exist, so no correction was needed.", "On 2026-10-11 (B6 final), compared docs/source/guides/complete_examples.rst against B6's later changes to openhcs/core/processing_contracts.py, openhcs/processing/presets/pipelines/loose_operaphenix_neurite_outgrowth.py, openhcs/microscopes/opera_phenix.py, scripts/master_multi_plate_demo.py, scripts/mcp_assay_showcase.py, openhcs/interop/cellprofiler/pipeline_import.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guides/complete_examples.rst against the later B6 changes to openhcs/core/processing_contracts.py, openhcs/processing/presets/pipelines/loose_operaphenix_neurite_outgrowth.py, openhcs/microscopes/opera_phenix.py, scripts/master_multi_plate_demo.py, scripts/mcp_assay_showcase.py, openhcs/interop/cellprofiler/pipeline_import.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guides/complete_examples.rst in full and compared it against the changed authorities: knowledge_base_service.py, mfd_specs.py, loose_operaphenix_neurite_outgrowth.py, master_multi_plate_demo.py, test_official30_knowledge_examples.py; confirmed the code block's imports, the Official30 document and section ids, the MFD owner names and template path, the master demo flags, port, blueprint count and contributor callables, replaced Microscope.OPERAPHENIX with OperaPhenixHandler and added the source_voxel_spacing input.", @@ -2271,7 +2274,7 @@ }, { "path": "openhcs/processing/presets/pipelines/loose_operaphenix_neurite_outgrowth.py", - "sha256": "1997c15cf71b8c65c8b49f238fced11b2d7a6c8bd5725ea75bdcd8cf7ab14314", + "sha256": "ad42221c3a03e9bad8bdffce92b56b10a43fd4bb12f12a8dfa947cdb0f63937f", "role": "Declares the loose Opera Phenix example's exact Hoechst, MAP2 and SMI312 sources, one-well bound and unified neuron result." }, { @@ -2290,11 +2293,10 @@ "role": "Proves discovery and retrieval of the example corpus map and other maintained knowledge documents." } ], - "findings": [ - "The page called the benchmark's parity evidence 'parity receipts'; changed to 'parity evidence'." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guides/example_corpus_map.rst in full and compared it against the changed authority openhcs/processing/presets/pipelines/loose_operaphenix_neurite_outgrowth.py (source-binding config imports moved to openhcs.core.config, AuthoredFunctionPattern.first_callable replaces get_core_callable); the page states only the example's sources, one-well bound and review evidence, which are unchanged, so no correction was needed.", "On 2026-10-11 (B6 final), compared docs/source/guides/example_corpus_map.rst against B6's later changes to tests/unit/agent/test_knowledge_base_service.py: metadata documents are written only for backends that host local files (not virtual backends such as OMERO); the image-source guide test reads axis-family spellings. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/guides/example_corpus_map.rst against B6's later changes to openhcs/processing/presets/pipelines/loose_operaphenix_neurite_outgrowth.py, tests/unit/agent/test_knowledge_base_service.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guides/example_corpus_map.rst against the later B6 changes to openhcs/processing/presets/pipelines/loose_operaphenix_neurite_outgrowth.py, openhcs/agent/capabilities.py, tests/unit/agent/test_knowledge_base_service.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -2334,12 +2336,12 @@ }, { "path": "external/zmqruntime/src/zmqruntime/__init__.py", - "sha256": "8d6315480e060a02ca247179eef2ecae71727a4b30bea3496c3c32ad48bf3640", + "sha256": "394b1071d1bf74c6a4fc5d3785c868072a0da1d11f8bdb4b048d91638053b36c", "role": "Exports the generic ZMQ runtime contracts used by viewer and execution endpoints." }, { "path": "pyproject.toml", - "sha256": "c3a2bea22aa6872940125f4aaac72c9eac72084fefc6c2cefdb54b5273361e4a", + "sha256": "b5d4d752704b886da20d860743b81dc3d2ce3a3c402208246fadc1c6cb71f597", "role": "Declares the fiji and viz extras named in setup." }, { @@ -2348,12 +2350,10 @@ "role": "Proves Fiji endpoint lifecycle, streamed image updates, settlement and control handling." } ], - "findings": [ - "The page said Fiji's display path is synchronous and SETTLE returns terminal progress immediately; Fiji tracks queued and active debounced display batches and SETTLE reports running, complete or failed progress through the shared settle action.", - "The per-viewer Fiji control plans for STATE and PAYLOADS were removed; Fiji registers no such action and the shared unknown-control action now answers them with an unsupported-control error. The page states that instead of naming dedicated error plans." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guides/fiji_viewer_management.rst in full and compared it against the changed authorities: external/zmqruntime/src/zmqruntime/__init__.py (current lazy export surface of client, messages and execution contracts) and pyproject.toml (dependency floors only; fiji and viz extras unchanged); the page's ZMQRuntime ownership and setup claims still hold, so no correction was needed.", "On 2026-10-11 (B6 final), compared docs/source/guides/fiji_viewer_management.rst against B6's later changes to openhcs/runtime/fiji_viewer_server.py, openhcs/runtime/viewer_protocol.py, tests/unit/test_fiji_viewer_server.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guides/fiji_viewer_management.rst against the later B6 changes to openhcs/runtime/fiji_viewer_server.py, openhcs/runtime/viewer_protocol.py, pyproject.toml, tests/unit/test_fiji_viewer_server.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guides/fiji_viewer_management.rst in full and compared it against the changed authorities: fiji_viewer_server.py, viewer_protocol.py, zmqruntime __init__.py, test_fiji_viewer_server.py, plus the new shared viewer_control_actions.py; rewrote the settlement section for batch-tracked settle progress and the unsupported STATE/PAYLOADS answer, and confirmed the extras, FIJI_IMAGEJ_RUNTIME initialization and shutdown delegation.", @@ -2379,7 +2379,7 @@ "authority": [ { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "sha256": "4a33908fb42874e6d57eeea3f5305793ddbda41d13356f0a78501af38dcdfeb4", "role": "Compiles pipeline declarations into validated step plans and execution bundles." }, { @@ -2411,6 +2411,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guides/index.rst in full and compared it against the changed authority openhcs/core/pipeline/compiler.py (ScopeKey-built compiler scope ids from U3, CompileProgressPhase/RunningProgressStatus progress classes from K6, CompiledSourceBindingPlan() default); the page only routes to the integration explanations and states no compiler detail, so no correction was needed.", "On 2026-10-11 (B6 final), compared docs/source/guides/index.rst against B6's later changes to openhcs/core/pipeline/compiler.py, openhcs/core/memory/decorators.py, openhcs/runtime/viewer_protocol.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guides/index.rst against the later B6 changes to openhcs/core/pipeline/compiler.py, openhcs/core/memory/decorators.py, openhcs/runtime/viewer_protocol.py, external/PolyStore/src/polystore/omero_local.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guides/index.rst in full and compared it against the changed authorities: compiler.py, core/memory/decorators.py, viewer_protocol.py, arraybridge types.py; the five toctree targets exist, ArrayBridge still declares input, output and execution memory attributes, framework device assignment still plans worker scope, and the viewer and OMERO routes name current owners; no correction was needed.", @@ -2458,17 +2459,17 @@ }, { "path": "openhcs/core/processing_contracts.py", - "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", + "sha256": "eb0f7fd99ded1c4d5957152ccf6abe1468436c13574a1d1bb83ce368d24b9f69", "role": "Declares the ProcessingContract family and Pure2DContract used in the example." }, { "path": "openhcs/core/callable_contract.py", - "sha256": "5b8e85dc1708d5727c73ac1007150219683fb9fb75c4ff2ec2c0bc676445738c", + "sha256": "a8e73c5df804d78baf72710e210cb2ae63156657a074071e11fd868f4b19d3b4", "role": "Reads input, output and execution memory types through MemoryContractAttribute." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", + "sha256": "358d3eaf5952170f9e9e2dc9056fb08707e8fcf74682063c074b367a83c2648f", "role": "Declares FrameworkDeviceAssignment and the typed per-step conversion decisions recorded by compilation." }, { @@ -2483,17 +2484,17 @@ }, { "path": "openhcs/core/steps/function_runtime.py", - "sha256": "b40ebb069565573c340e4fd95ce220474d27f200396ec6d9c797bf716b2c0893", + "sha256": "fa9486cddf3db943a218f276bb43d079f505bf84b18d9f7bd074f7a4d86332b3", "role": "Enters the invocation's execution device scope around the function body." }, { "path": "openhcs/core/orchestrator/worker_execution.py", - "sha256": "23a3ef2a70378dd1bb5c413a56e36bbb3fa013142741c9b45407bfc47de8c4a4", + "sha256": "26b9586890e7117b24b6b85660d3f417455d55546401739f91146d18e4d21134", "role": "Cleans the exact compiled framework-device footprint during worker teardown." }, { "path": "openhcs/core/config.py", - "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", "role": "Declares variable_components and group_by on the step processing configuration." }, { @@ -2503,15 +2504,14 @@ }, { "path": "tests/unit/test_framework_execution_scope.py", - "sha256": "ff92362a71f31b4b893a3816b1fdd396ed30307f45c7084270b4929c1aaa7aa4", + "sha256": "b61d2df837d570f99031b29cadf75d2ac7381617cdd845b2fb67914eb5015a3f", "role": "Proves execution memory metadata selects the runtime framework device scope independently of boundary types." } ], - "findings": [ - "ProcessingContract is now an ABC family with subclasses such as Pure2DContract (G5), not an enum; the paragraph now names the family and its subclass rather than describing ProcessingContract as a single value." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guides/memory_type_integration.rst in full and compared it against the changed authorities: openhcs/core/processing_contracts.py and openhcs/core/callable_contract.py (K5: CallableMetadata became CallableDeclaration, which CallableContract inherits and which reads the input, output and execution keys through MemoryContractAttribute), openhcs/core/compiled_step_plan.py (empty-plan defaults), openhcs/core/steps/function_runtime.py (execution_device_scope still wraps the function body), openhcs/core/orchestrator/worker_execution.py (K6 progress classes; compiled GPU footprint cleanup unchanged), openhcs/core/config.py (variable_components and group_by still on the processing config) and tests/unit/test_framework_execution_scope.py; confirmed openhcs.constants.MemoryType and openhcs.core.memory.MemoryType are arraybridge.MemoryType and that MemoryType.subprocess_environment exists; no correction was needed.", "On 2026-10-11 (B6 final), compared docs/source/guides/memory_type_integration.rst against B6's later changes to openhcs/core/memory/decorators.py, openhcs/core/processing_contracts.py, openhcs/core/callable_contract.py, openhcs/core/compiled_step_plan.py, openhcs/core/steps/function_runtime.py, openhcs/core/orchestrator/worker_execution.py, openhcs/core/config.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guides/memory_type_integration.rst against the later B6 changes to openhcs/core/memory/decorators.py, openhcs/core/processing_contracts.py, openhcs/core/callable_contract.py, openhcs/core/compiled_step_plan.py, openhcs/core/steps/function_runtime.py, openhcs/core/orchestrator/worker_execution.py, openhcs/core/config.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guides/memory_type_integration.rst in full and compared it against the changed authorities: arraybridge types.py and decorators.py, constants.py, core/memory/__init__.py and decorators.py, callable_contract.py, compiled_step_plan.py, test_callable_contract.py, framework_device_assignment.py, function_runtime.py, worker_execution.py, test_framework_execution_scope.py; confirmed the MemoryType re-exports, MemoryContractAttribute reads, FrameworkDeviceAssignment, execution device scope, subprocess_environment consumers and variable_components/group_by, and reworded the processing-contract paragraph for the subclass family.", @@ -2574,7 +2574,7 @@ }, { "path": "openhcs/core/dataset_sources/source.py", - "sha256": "097fde4d2b74031eacf53a88e79f499417e8c1c97ba0753e8a9df831b307efa6", + "sha256": "350bf5691a03522e9f54b1d8ab712c6beba1683aac4fa5159ea4e80c6041530d", "role": "Declares the DatasetSource interface the OMERO source implements for workspace and metadata access." }, { @@ -2604,7 +2604,7 @@ }, { "path": "pyproject.toml", - "sha256": "c3a2bea22aa6872940125f4aaac72c9eac72084fefc6c2cefdb54b5273361e4a", + "sha256": "b5d4d752704b886da20d860743b81dc3d2ce3a3c402208246fadc1c6cb71f597", "role": "Includes the complete openhcs/omero deployment and plugin bundle in installed distributions." }, { @@ -2623,12 +2623,10 @@ "role": "Declares the desktop-window family, which contains no OMERO management or credential window." } ], - "findings": [ - "Two authorities were deleted by G4: openhcs/microscopes/microscope_interfaces.py (now the DatasetSource family in openhcs/core/dataset_sources/source.py) and openhcs/core/orchestrator/analysis_consolidation.py (now openhcs/domains/microscopy/analysis_consolidation.py); the page's claims hold against the new owners.", - "The PolyStore paragraph called the OMERO addresses 'canonical'; the word was dropped." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guides/omero_integration.rst in full and compared it against the changed authorities: openhcs/core/dataset_sources/source.py (SourceBindingsConfig type import now from openhcs.core.config, K4) and pyproject.toml (dependency floors only); the page names neither, and its DatasetSource and OMERO ownership claims still hold, so no correction was needed.", "On 2026-10-11 (B6 final), compared docs/source/guides/omero_integration.rst against B6's later changes to openhcs/domains/microscopy/analysis_consolidation.py, openhcs/core/dataset_sources/source.py, openhcs/microscopes/omero.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guides/omero_integration.rst against the later B6 changes to external/PolyStore/src/polystore/omero_local.py, openhcs/domains/microscopy/analysis_consolidation.py, openhcs/core/dataset_sources/source.py, openhcs/microscopes/omero.py, pyproject.toml: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guides/omero_integration.rst in full and compared it against the changed authorities: PolyStore filemanager.py and base.py, microscopes/omero.py, test_omero_microscope_metadata.py, pyproject.toml, tests/integration/test_main.py, ui_bridge_identities.py, plus the moved analysis consolidation and dataset-source owners; confirmed OMEROFilenameParser, FilenameParseResult, the PolyStore address types, DataSink contextual saves, ledger-based consolidation through FileManager and OMEROInstanceManager, and dropped 'canonical'.", @@ -2678,22 +2676,22 @@ "authority": [ { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "sha256": "4a33908fb42874e6d57eeea3f5305793ddbda41d13356f0a78501af38dcdfeb4", "role": "Defines compile_pipelines over an orchestrator and list[AbstractStep] and returns a CompiledExecutionBundle." }, { "path": "openhcs/core/orchestrator/orchestrator.py", - "sha256": "a61658b107f170c192eea505966c268ab6826826f03586bd64562a44800e1ccc", + "sha256": "99870516da6d64e1858f4dd8799d6e38ab6911b7b02bc29b252d2a23f7687410", "role": "Supplies the initialized orchestrator, its DatasetSource workspace and selected partition values to compilation." }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "8f691504ea5983189c8ff1984b6125866449d0d68f6077138780398773f2c6e0", + "sha256": "0f9cc4819b53d55b7e9446429b8c8ac1ff1a59f4902020ea1658495ce01d464c", "role": "Declares CompilationSession, which owns cross-step compile state." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "5bce281e8291a3434b57ba44f97008e4095cc95998a0805f0369fd212c3a7eec", + "sha256": "358d3eaf5952170f9e9e2dc9056fb08707e8fcf74682063c074b367a83c2648f", "role": "Declares CompiledStepPlan and its typed per-step memory, conversion, device and runtime fields." }, { @@ -2712,11 +2710,10 @@ "role": "Proves compilation produces immutable step plans and execution bundles with the expected resolved values." } ], - "findings": [ - "The inputs list named a 'microscope/source workspace'; after G4 the orchestrator holds a DatasetSource, so the item now says dataset-source workspace." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guides/pipeline_compilation_workflow.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compiler.py (ScopeKey scope ids, K6 progress classes), openhcs/core/orchestrator/orchestrator.py (source_bindings_defaults_to_base imported from openhcs.core.config), openhcs/core/pipeline/compilation_session.py (strip_disabled_functions removed, per-graph invocation validation moved onto the graph) and openhcs/core/compiled_step_plan.py (empty-plan defaults now plain constructors); CompilationSession, CompiledStepPlan and CompiledExecutionBundle still own the stated responsibilities, so no correction was needed.", "On 2026-10-11 (B6 final), compared docs/source/guides/pipeline_compilation_workflow.rst against B6's later changes to openhcs/core/pipeline/compiler.py, openhcs/core/orchestrator/orchestrator.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/compiled_step_plan.py, tests/unit/test_compiled_execution.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guides/pipeline_compilation_workflow.rst against the later B6 changes to openhcs/core/pipeline/compiler.py, openhcs/core/orchestrator/orchestrator.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/compiled_step_plan.py, tests/unit/test_compiled_execution.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guides/pipeline_compilation_workflow.rst in full and compared it against the changed authorities: compiler.py, compilation_session.py, compiled_step_plan.py, compiled_execution.py, test_compiled_execution.py, framework_device_assignment.py; confirmed CompilationSession, CompiledStepPlan, CompiledExecutionBundle and the compile_pipelines inputs, and renamed the microscope/source workspace to the dataset-source workspace.", @@ -2856,7 +2853,7 @@ }, { "path": "openhcs/core/xdg_paths.py", - "sha256": "d015a57a46b0052a717d03febe00065041200c7adf079639aa1ac887c24cf45c", + "sha256": "f7aa72330de80a1a4dce33a56f3f9d2cdb57a4257c03db4e3e641950fe146d3b", "role": "Resolves OpenHCS data, cache, state, and log locations from the platform and XDG environment." }, { @@ -2880,12 +2877,10 @@ "role": "Owns the table-service readiness contract required before OMERO storage tests proceed." } ], - "findings": [ - "The CI section did not mention the Official30 headless parity lane, which now runs on Python 3.12 Linux and Python 3.14 Linux, Windows and macOS behind a path-scope check whose skip a required job accepts only for proven-irrelevant pull requests.", - "The CI section did not say that test jobs now run the checkout's tests against the installed candidate wheel through scripts/run_installed_tests.py; one paragraph now states it." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guides/testing_guide.rst in full and compared it against the changed authority openhcs/core/xdg_paths.py (legacy ~/.openhcs migration helpers deleted; get_*_file_path now plain path joins); the page's log-location claim (get_openhcs_data_dir() / \"logs\", honouring XDG_DATA_HOME, default ~/.local/share/openhcs/logs) still holds and the page never described the deleted migration, so no correction was needed.", "On 2026-10-11 (B6 final), compared docs/source/guides/testing_guide.rst against B6's later changes to .github/workflows/integration-tests.yml: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/guides/testing_guide.rst against the later B6 changes to scripts/smoke_installed_gui.py, tests/unit/test_smoke_installed_gui.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guides/testing_guide.rst in full and compared it against the changed authorities: pytest.ini, integration-tests.yml, resolve_successful_job_baseline.py, test_windows_simple_installer.py, website-pages.yml, test_ci_package_boundaries.py, smoke_installed_desktop.py, test_installed_desktop_smoke.py, smoke_installed_gui.py, xdg_paths.py, tests/integration/conftest.py; confirmed the default discovery options, --it-* flags, artifact names, concurrency, actionlint, code-action probe and log-directory paths, and added the scoped Official30 parity lane and the installed-wheel test runner.", @@ -2965,7 +2960,7 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", "role": "Declares StreamingConfig, NapariStreamingConfig, FijiStreamingConfig and StepMaterializationConfig, including the partition filter inherited by streaming configuration." }, { @@ -3039,12 +3034,10 @@ "role": "Proves isolation preflights every requested route and selected-route axis before changing live Napari state." } ], - "findings": [ - "The streaming-config entry said well selection belongs to streaming configuration; after G1 the field is the partition filter, so the entry now says partition (well) selection.", - "The viewer QA paragraph used 'scene/payload custody'; reworded as retained scene and payload state." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guides/viewer_management.rst in full and compared it against the changed authority openhcs/core/config.py (PartitionFilterMode became an IncludePartitions/ExcludePartitions subclass family, K4 moved SourceBindingsConfig/StepSourceBindingsConfig into config.py); StreamingConfig, NapariStreamingConfig, FijiStreamingConfig and StepMaterializationConfig still exist with partition selection inherited by streaming, and the page names no partition-mode member, so no correction was needed.", "On 2026-10-11 (B6 final), compared docs/source/guides/viewer_management.rst against B6's later changes to openhcs/core/config.py, openhcs/runtime/viewer_protocol.py, openhcs/core/artifacts.py, tests/unit/test_napari_roi_manager_integration.py, tests/unit/test_viewer_result_element_selection.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guides/viewer_management.rst against the later B6 changes to openhcs/core/config.py, openhcs/runtime/viewer_protocol.py, openhcs/core/artifacts.py, tests/unit/test_napari_roi_manager_integration.py, tests/unit/test_viewer_result_element_selection.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guides/viewer_management.rst in full and compared it against the changed authorities: napari_viewer_server.py, viewer_protocol.py, viewer_controls.py, viewer_window_service.py, napari.yaml, the ROI manager, streaming handler and element-selection tests, and napari_streaming_handlers.py; confirmed the streaming and materialization config classes, SHAPES boundary, ROI toolbar labels, plugin menu entries, license file, spatial-graph options, max_rois, layer isolation and transport-thread settle, renamed well selection to partition selection, and replaced 'custody'.", @@ -3074,17 +3067,17 @@ "authority": [ { "path": "openhcs/core/pipeline_document.py", - "sha256": "8c554c8275c665f308eb098955bda517aa804eeb34418b5e0f41d412650ce7b5", + "sha256": "9a77a08367f901594859c8b6cae9609c9230a551f4f4e9ec6ea6cd2ed233c0f4", "role": "Declares PipelineDocument and PipelineDocumentCodec, the one pipeline config-plus-steps document shared by the GUI, Python and agents." }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "01dcb82b5850f3cdb1c300a2aa360ba330e7f15dffb5a24e37504311095090f9", + "sha256": "3cdb7819f09604046ba69e0b4f57b73e4e98d8f68af80158501a54bb42e701a2", "role": "Composes the main desktop window, menus, task windows, and registered UI services." }, { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "3439d96f3fbfcb104a53019dc150049021d9a8132bd27096f1ccf510eefbe37b", + "sha256": "b96f33b866fe6c9f8118ca70fcb7dff8e6a02cb29d79df67230a5e8b97df8002", "role": "Imports CellProfiler modules into ordinary OpenHCS pipeline declarations through the interop boundary." }, { @@ -3101,6 +3094,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/index.rst in full and compared it against the changed authorities: pipeline_document.py (render now via pycodify.generate_python_source, L6; PipelineDocument/Codec unchanged), pyqt_gui/main.py (dock layout, shortcuts and time-travel navigation moved to pyqt-reactive, U3), cellprofiler/pipeline_import.py (SourceBindingsConfig now imported from openhcs.core.config, K4); the landing page names no module, class or window behaviour these touched, and still routes to existing pages; no edit needed.", "On 2026-10-11 (B6 final), compared docs/source/index.rst against B6's later changes to openhcs/pyqt_gui/main.py, openhcs/interop/cellprofiler/pipeline_import.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/index.rst against the later B6 changes to openhcs/pyqt_gui/main.py, openhcs/interop/cellprofiler/pipeline_import.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/index.rst in full and compared it against the changed authorities: pipeline_document.py (PipelineDocumentAuthority renamed PipelineDocumentCodec), pyqt_gui/main.py, cellprofiler pipeline_import.py, mcp/bootstrap.py, gallery_catalog.py; the page names none of the renamed symbols, the multi-plate-overview scenario still exists, and the routing claims and toctree remain accurate; no correction was needed.", @@ -3128,7 +3122,7 @@ "authority": [ { "path": "pyproject.toml", - "sha256": "c3a2bea22aa6872940125f4aaac72c9eac72084fefc6c2cefdb54b5273361e4a", + "sha256": "b5d4d752704b886da20d860743b81dc3d2ce3a3c402208246fadc1c6cb71f597", "role": "Declares the package version, Python requirement, dependency extras, scripts, and distribution metadata the redirected guide covers." }, { @@ -3148,11 +3142,11 @@ } ], "findings": [ - "Maintaining installation instructions here would duplicate package extras and installer behaviour; the page correctly redirects to the maintained task.", - "The redirect sentence called the Getting Started instructions 'canonical'; it now says 'maintained'." + "Maintaining installation instructions here would duplicate package extras and installer behaviour; the page correctly redirects to the maintained task. The later dependency floor changes (zmqruntime 0.6.2, pycodify 0.2, objectstate 2.1.1, python-introspect 0.2.7, pyqt-reactive 0.5) are not copied here, so nothing on the page became wrong." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/installation.rst in full and compared it against the changed authorities: pyproject.toml (zmqruntime, pycodify, objectstate, python-introspect and pyqt-reactive floor changes after U3/K4/K5/K6/L4/L6); the page states no version or extra and still redirects to getting_started; no edit needed.", "On 2026-10-11 (B6 final), compared docs/source/installation.rst against the later B6 changes to pyproject.toml: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/installation.rst in full and compared it against the changed authorities: pyproject.toml (dependency floors, JAX extra), openhcs/cli.py (skills subcommand), the Windows and macOS installers (skill sync); the page copies none of those facts and still redirects to getting_started; replaced 'canonical' with 'maintained'.", "On 2026-08-27, re-read docs/source/installation.rst in full after the python-introspect dependency floor advanced to 0.1.12; confirmed that its compatibility redirect still reaches the canonical installation procedure.", @@ -3198,7 +3192,7 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", "role": "Declares GlobalPipelineConfig, PipelineConfig and the step configuration families with their defaults and field help." }, { @@ -3240,6 +3234,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/reference/configuration.rst in full and compared it against the changed authority openhcs/core/config.py (PartitionFilterMode became a registered subclass family with RegisteredClassChoices, SourceBindingsConfig/StepSourceBindingsConfig declared in config.py, K4); the page copies no field, type or default and renders them through the openhcs-config-reference directive from live declarations, so the new family spellings appear automatically; confirmed UIConfig still holds list_previews, logging, shortcuts, progress, zmq transport and agent_bridge, openhcs_describe_config_schema and ConfigPatch still exist; no edit needed.", "On 2026-10-11 (B6 final), compared docs/source/reference/configuration.rst against B6's later changes to openhcs/core/config.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/reference/configuration.rst against the later B6 changes to openhcs/core/config.py, openhcs/agent/capabilities.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/reference/configuration.rst in full and compared it against the changed authorities: core/config.py, test_config_reference_service.py, test_user_facing_help_surfaces.py; the four directive names still resolve through config_service aliases, the named config classes, openhcs_describe_config_schema and ConfigPatch exist, and the UIConfig overview matches its fields; no correction was needed. The generated field tables carry the G1/G4 renames without page edits.", @@ -3267,7 +3262,7 @@ "authority": [ { "path": "openhcs/core/processing_contracts.py", - "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", + "sha256": "eb0f7fd99ded1c4d5957152ccf6abe1468436c13574a1d1bb83ce368d24b9f69", "role": "Declares the ProcessingContract family: Pure2D, Pure3D, Flexible, and VolumetricToSlice contracts." }, { @@ -3282,7 +3277,7 @@ }, { "path": "openhcs/processing/backends/cellprofiler/watershed.py", - "sha256": "41c8cb9cb139f8b4041a25e1a7a2d19aa85cf0427422fc87b9551c8141069625", + "sha256": "2c8c352bdc5fedad6baf14e4c1a3cd5347a1f862fd841472241680294c2762ca", "role": "Declares the full-stack payload route for CellProfiler-compatible Watershed." }, { @@ -3292,7 +3287,7 @@ }, { "path": "openhcs/processing/backends/cellprofiler/secondary.py", - "sha256": "fc9d9b1bbc3baff7216c6abe3a7135ffddfc9f7e1212d4a391fa8534038a354c", + "sha256": "f731f95f874ba3368828ca3a539fac4e5c6eb0ec7049e546f45fb874e0888e95", "role": "Declares the current plane-local IdentifySecondaryObjects and IdentifyTertiaryObjects routes." }, { @@ -3332,20 +3327,21 @@ }, { "path": "tests/unit/test_cellprofiler_module_execution.py", - "sha256": "f5d678e29026578687a68f6ce7abc3184eec5223a7e4e2337038fb2079a98ea2", + "sha256": "203501b194710b747f6a791bce8fb46d0298e09ffdc20fb0c22e4a96296c1a64", "role": "Proves payload-scoped size-and-shape rows use the 3D volume schema." }, { "path": "tests/unit/test_cellprofiler_pipeline_import.py", - "sha256": "6b1d201ec5c4ba620fa2367a9cbef1f6fbdd25edbdda9bd7b723ebe3a3363412", + "sha256": "0ce4efd85d819c7e9f7fa84ec6f1ac7ec4bda024c9a8d5c7f264fda2730264f0", "role": "Proves CellProfiler import preserves 3D image, object, and morphology variants from declarations." } ], "findings": [ - "The page's claims still hold after G3/G5: ProcessingContract is now a class family (Pure2DContract, Pure3DContract, FlexibleContract, VolumetricToSliceContract) and image execution modes moved to openhcs/core/image_payload_execution_mode.py. The recorded authorities aligned_image_payload.py and unified_registry.py no longer own the stated facts and were replaced by processing_contracts.py and image_payload_execution_mode.py." + "The page's claims still hold after K4/K5: processing_contracts.py now reads signatures through CallableDeclaration (CallableMetadata folded into it), watershed.py imports StepSourceBindingsConfig from openhcs.core.config, and IdentifyTertiaryObjects gained repeated_return_slots for identical larger/smaller inputs. None of these changes the plane-local or payload-scoped domains, the 3D measurement schemas, or the Official30 evidence the page states; the page names no renamed class." ], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/reference/dimensionality_and_measurements.rst in full and compared it against the changed authorities: processing_contracts.py (CallableMetadata -> CallableDeclaration), cellprofiler/watershed.py (StepSourceBindingsConfig import from config.py), cellprofiler/secondary.py (tertiary repeated_return_slots), tests/unit/test_cellprofiler_module_execution.py and test_cellprofiler_pipeline_import.py (flattened CallableDeclaration fields, admitted_graph helper); the segmentation table, volumetric measurement fields and validation boundary remain accurate; no edit needed.", "On 2026-10-11 (B6 final), compared docs/source/reference/dimensionality_and_measurements.rst against B6's later changes to openhcs/core/processing_contracts.py, openhcs/core/image_payload_execution_mode.py, openhcs/processing/backends/cellprofiler/watershed.py, openhcs/processing/backends/cellprofiler/primary_objects.py, openhcs/processing/backends/cellprofiler/secondary.py, openhcs/processing/backends/cellprofiler/intensity.py, openhcs/processing/backends/cellprofiler/shape.py, openhcs/processing/backends/cellprofiler/area_occupied.py, tests/unit/test_documented_dimensionality.py, tests/unit/test_cellprofiler_library_loading.py, tests/unit/test_cellprofiler_module_execution.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/reference/dimensionality_and_measurements.rst against the later B6 changes to openhcs/core/processing_contracts.py, openhcs/core/image_payload_execution_mode.py, openhcs/processing/backends/cellprofiler/watershed.py, openhcs/processing/backends/cellprofiler/primary_objects.py, openhcs/processing/backends/cellprofiler/secondary.py, openhcs/processing/backends/cellprofiler/intensity.py, openhcs/processing/backends/cellprofiler/shape.py, openhcs/processing/backends/cellprofiler/area_occupied.py, tests/unit/test_documented_dimensionality.py, tests/unit/test_cellprofiler_library_loading.py, tests/unit/test_cellprofiler_module_execution.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/reference/dimensionality_and_measurements.rst in full and compared it against the changed authorities: aligned_image_payload.py, runtime_object_label_domains.py, unified_registry.py, the six CellProfiler backends, the Official30 axis1 manifest, and the four tests; confirmed primary/secondary/tertiary still declare Pure2DContract, Watershed and intensity use FullStackExecution, shape and area-occupied use FullStackLabels, occupied-volume callables use Pure3DContract, PAYLOAD/PLANE scopes remain, and every listed measurement family has a backend module; no page change was needed.", @@ -3436,7 +3432,7 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", "role": "Declares the global, pipeline, step, and desktop configuration dataclasses that the generated configuration reference lists." }, { @@ -3455,11 +3451,10 @@ "role": "Declares the artifact types named by the shared terminology the index points to." } ], - "findings": [ - "The page's navigation and descriptions remain accurate; its recorded authority openhcs/formats/experimental_analysis.py was deleted (moved to openhcs/processing/backends/experimental_analysis/analysis.py) and was repointed." - ], + "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/reference/index.rst in full and compared it against the changed authority openhcs/core/config.py (PartitionFilterMode became a registered subclass family, IncludePartitions/ExcludePartitions; SourceBindingsConfig and StepSourceBindingsConfig declared directly in config.py, K4); the index names no field, class or mode, and every listed reference page still exists; no edit needed.", "On 2026-10-11 (B6 final), compared docs/source/reference/index.rst against B6's later changes to openhcs/core/config.py, openhcs/processing/backends/experimental_analysis/analysis.py, openhcs/core/artifacts.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/reference/index.rst against the later B6 changes to openhcs/core/config.py, openhcs/processing/backends/experimental_analysis/analysis.py, openhcs/core/artifacts.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/reference/index.rst in full and compared it against the changed authorities: openhcs/core/config.py, the deleted openhcs/formats/experimental_analysis.py, openhcs/agent/services/knowledge_base_service.py, and openhcs/core/artifacts.py; checked that every toctree target exists and each one-line description still matches its page and owner; no page change was needed.", @@ -3498,12 +3493,12 @@ }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "01dcb82b5850f3cdb1c300a2aa360ba330e7f15dffb5a24e37504311095090f9", + "sha256": "3cdb7819f09604046ba69e0b4f57b73e4e98d8f68af80158501a54bb42e701a2", "role": "Composes the main desktop window and its stable docked surfaces from the registered identities." }, { "path": "openhcs/pyqt_gui/services/main_window_workflows.py", - "sha256": "883fc3ddd1c406719ba75439ef9f3330bbbab1ff85e3476f6e343111a8f3c8fc", + "sha256": "8aca9e8a94b6a3ee36dbd454c2acd3147d96c767fccbf8149f0bf05287e66312", "role": "Projects registered window identities into WindowManager specifications for stable task windows." }, { @@ -3530,6 +3525,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/reference/ui_windows.rst in full and compared it against the changed authorities: pyqt_gui/main.py and main_window_workflows.py (U3 moved dock layout, shortcut lifecycle and time-travel navigation to pyqt-reactive DockLayoutStore/DockPanes, ShortcutLifecycle, TimeTravelNavigator); build_main_window_specs still projects UiStableWindowIdentityDeclaration subclasses into WindowManager specs and the openhcs-ui-window-reference directive still renders the registry, so the page (which keeps no window list of its own) remains accurate; no edit needed.", "On 2026-10-11 (B6 final), compared docs/source/reference/ui_windows.rst against B6's later changes to openhcs/authoring/session/views.py: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/reference/ui_windows.rst against B6's later changes to openhcs/authoring/session/views.py, openhcs/pyqt_gui/main.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/reference/ui_windows.rst against the later B6 changes to openhcs/authoring/session/views.py, openhcs/pyqt_gui/main.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -3555,7 +3551,7 @@ "authority": [ { "path": "openhcs/pyqt_gui/main.py", - "sha256": "01dcb82b5850f3cdb1c300a2aa360ba330e7f15dffb5a24e37504311095090f9", + "sha256": "3cdb7819f09604046ba69e0b4f57b73e4e98d8f68af80158501a54bb42e701a2", "role": "Declares the four Tools > Analysis Consolidation actions and the experimental-analysis directory check." }, { @@ -3579,12 +3575,10 @@ "role": "Owns the post-execute hook that consolidates execution-saved tables per directory and writes the merged summary beneath the compiled output plate root." } ], - "findings": [ - "The page said AnalysisConsolidationConfig selects well extraction; the config declares no well field (the well comes from the partition axis value), so the sentence now lists the declared choices: MetaXpress-style layout, file extensions, exclusion patterns, and output names. It also placed the config 'on the pipeline'; it is a global pipeline configuration section.", - "Recorded authorities named the deleted openhcs/formats/experimental_analysis.py and openhcs/core/orchestrator/analysis_consolidation.py; consolidation moved to openhcs/domains/microscopy/analysis_consolidation.py and both configs to openhcs/domains/microscopy/config.py." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/user_guide/analysis_consolidation.rst in full and compared it against the changed authority openhcs/pyqt_gui/main.py (U3: dock layout, shortcuts and time-travel navigation moved to pyqt-reactive); the Tools > Analysis Consolidation submenu still declares the four actions (Consolidate Results Directory, Merge MetaXpress Summaries, Concatenate MetaXpress Summaries (Keep Headers), Run Experimental Analysis) and the experimental-analysis directory dialog; no edit needed.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/analysis_consolidation.rst against B6's later changes to openhcs/pyqt_gui/main.py, openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py, tests/unit/test_experimental_analysis_engine.py, openhcs/domains/microscopy/analysis_consolidation.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/analysis_consolidation.rst against the later B6 changes to openhcs/pyqt_gui/main.py, openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py, tests/unit/test_experimental_analysis_engine.py, openhcs/domains/microscopy/analysis_consolidation.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/user_guide/analysis_consolidation.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/main.py, openhcs/core/config.py, the deleted openhcs/formats/experimental_analysis.py and openhcs/core/orchestrator/analysis_consolidation.py (now openhcs/domains/microscopy/analysis_consolidation.py), unified_analysis_engine.py, and the engine test; checked the menu actions, directory filenames, consolidation config fields, and per-directory plus merged summary behaviour; corrected the config description.", @@ -3613,17 +3607,17 @@ "authority": [ { "path": "openhcs/core/pipeline_document.py", - "sha256": "8c554c8275c665f308eb098955bda517aa804eeb34418b5e0f41d412650ce7b5", + "sha256": "9a77a08367f901594859c8b6cae9609c9230a551f4f4e9ec6ea6cd2ed233c0f4", "role": "PipelineDocumentCodec validates, parses, and renders the pipeline_steps and optional pipeline_config document." }, { "path": "openhcs/authoring/session/dataset_document.py", - "sha256": "305da3c0bf7bb15a0f251a101696b721f9151d8620450a6b3e12413b79218574", + "sha256": "50dc7b601b6a2343c8aaa1ebdaae92d9b4641f7c7ca6a02ade207809e5bdd172", "role": "Owns the selected and all-datasets document scopes, selected-scope ID enforcement, and the global-draft and per-dataset change checks on apply." }, { "path": "openhcs/authoring/session/pipelines.py", - "sha256": "245bed768c49ae17b26252d41a1040e59c46d59e0894b553309cd5ec8a9a1793", + "sha256": "febf9f9f464b2600554256f16929c408d4bdcaedd6ca99c71810bcb7be519519", "role": "PipelineObjectStateBinding updates and commits a dataset's step ObjectState graph after a complete pipeline replacement." }, { @@ -3633,27 +3627,37 @@ }, { "path": "openhcs/pyqt_gui/widgets/pipeline_editor.py", - "sha256": "8e418631fd9ba4f8ac3d6b257f9af127f9e8711281608516ea2617d7855d6f36", + "sha256": "7b3b58f3685d5a636a102cae7d1e31c663d5d574bcdc71d4a289b69a9eee6535", "role": "Makes the Pipeline Editor document writable only while a dataset is selected." }, { - "path": "openhcs/pyqt_gui/services/config_window_code_document.py", - "sha256": "bdbf33b254451945239bd1c1f256dedbed40d2a8c0e598c4426bcba939ab5d2d", - "role": "Owns typed configuration source projection and applies parsed updates." + "path": "openhcs/pyqt_gui/windows/config_window.py", + "sha256": "f72c0e481e35819f49749a7b82729e2f99ff200ebfd5bd6031efdb6f0585b24b", + "role": "OpenHCSConfigWindow plugs the OpenHCS configuration document format into pyqt-reactive's ConfigWindow, rendering and parsing typed config source." + }, + { + "path": "openhcs/core/config_document.py", + "sha256": "9695a6f300d626ceddaf4e0cd18961fd5600b63becf9a133cfa7f0b654b32cfb", + "role": "ConfigDocumentAuthority renders a typed configuration as source and validates parsed source against the expected config type." + }, + { + "path": "external/pyqt-reactive/src/pyqt_reactive/windows/config_window.py", + "sha256": "2ee4f5c173b30b0a94a4288243d82ae66b6d65d12aa59493ad9363559966c1cd", + "role": "ConfigCodeDocumentDriver applies parsed configuration source to the window's edit session." }, { "path": "external/pyqt-reactive/src/pyqt_reactive/services/window_code_document.py", - "sha256": "d0ab321af0eb2b14fac3ab2577c8c61672763d2f13a70a7ce8a35ee7bfab7c75", + "sha256": "56d92d665a7b3bf6acde28292679cad13388793e1acb5bea74926f4330150d42", "role": "Owns generic window-document writability and declaration-normalised source comparison." }, { "path": "external/pyqt-reactive/src/pyqt_reactive/services/scope_token_service.py", - "sha256": "f84056a6b542d1d9a7df9e05c2ec344d6acb7cafd3301054158e89f04eb32b17", + "sha256": "a4913835f35ae1b773eab817224410786cd569e459f633e9949ac4e3c3e724ba", "role": "Reconciles submitted step occurrences with existing scopes, giving ambiguous duplicates new scopes." }, { "path": "openhcs/pyqt_gui/services/ui_bridge_object_state.py", - "sha256": "e548b367507b3fb6830ce9ecb844e3b756243af96fe38e0d35de5da0b974c1fd", + "sha256": "e64b444a74fdde1d79f82c1361c9b67d8e8916b9bebced092762ce51eabac52b", "role": "Enforces driver writability, returns unchanged for equivalent documents, and applies documents against revision tokens." }, { @@ -3668,13 +3672,18 @@ }, { "path": "external/pycodify/src/pycodify/__init__.py", - "sha256": "5869747d68ca64e6497f55e09b53cbbaa87741d39ae922edb035997c53f19c49", + "sha256": "083feeadebf55cc3437b44fa23aaccc51e1cef7b2cf169339031e7ca3a3e3abe", "role": "Exports the PyCodify source-generation and reconstruction API used for declaration round-tripping." }, { "path": "openhcs/serialization/pycodify_formatters.py", - "sha256": "fe06f3ab873b537b7b2c0b13599ffd11030ae6a8ac504caea9e3dcfa1d4695b4", - "role": "Supplies OpenHCS semantic equality, function-reference import rendering, and clean/full choices to parameter projection." + "sha256": "18b744cdeb8d9ce22801efe6f3f81fec346194019293e68f542dc13c11724a2d", + "role": "Formats function patterns and steps: signature-ordered parameters, semantic default omission in clean mode, and function-reference imports." + }, + { + "path": "external/ObjectState/src/objectstate/codegen.py", + "sha256": "347d2697c061a35e6e440e373fe4e8bfce1ad92b500e98da07faa26f6a3a1356", + "role": "Writes raw lazy dataclass field values, omitting inherited and default fields in clean mode and writing every field in full mode." }, { "path": "external/python-introspect/src/python_introspect/callable_declaration.py", @@ -3683,7 +3692,7 @@ }, { "path": "external/pyqt-reactive/src/pyqt_reactive/services/function_pattern_code_document.py", - "sha256": "bdf41b58bff009890e6e696cfc4f18313adfb1669566b2566f9e7b93d863c071", + "sha256": "77b0bc021efaaac47bc5fb8c29aa50536162a9630dafd48c6ece42790837b94d", "role": "Retains a child parameter state only while its function and editable surface match." }, { @@ -3693,7 +3702,7 @@ }, { "path": "openhcs/pyqt_gui/widgets/shared/services/pipeline_editor_workflows.py", - "sha256": "c704ef7e6c5f8a5745c741e923bc4e620a321fdce069e1cdcdf521c0176e4a4f", + "sha256": "06e54377bc594beef653baa9cfaf4de0031e754da1e7b1ab3a22ac67a922b68f", "role": "Applies a complete Pipeline Editor document and commits the reconciled live state graph." }, { @@ -3703,27 +3712,27 @@ }, { "path": "tests/unit/pyqt_gui/test_code_document_catalog_imports.py", - "sha256": "7391f26f23c396c974fb0f7eada5ae5a0646a5ec2beb5515388fe7367efe954c", + "sha256": "adfcec931eaad3efbde3495585bb8e184e840e107ce998ecc3a607be9fd80595", "role": "Proves catalogue imports such as skimage.filters.edges.sobel keep the registry wrapper and wildcard or unlisted imports are rejected." }, { "path": "tests/unit/pyqt_gui/test_ui_agent_bridge.py", - "sha256": "de6758859d9743e826369162855327e4a84056cafdd7ddfdc33868d2a3093353", + "sha256": "67cdad5133dc83ccb005b58cc56535ae89e77fc23d3ebd76e616c0da5f567b44", "role": "Proves bidirectional UI-bridge inspection, capability enforcement, semantic no-op handling, and mutation of live declarations." }, { "path": "tests/unit/pyqt_gui/test_plate_manager_widget.py", - "sha256": "4daa91c0de965dc2dd9cd77ea4969f4fa0aff90fc5b69cbcadaba3bd2a1cfc6b", + "sha256": "91ed9468677fee730d67e5d997b7250285e8443f44897e5aa67beaaabe67a7e7", "role": "Proves selected code-document application preserves unselected dataset graphs." }, { "path": "tests/unit/test_pipeline_document.py", - "sha256": "4df57a9c4f12a009222981bf1ac28554405874f3533656b61344ee94056d1bab", + "sha256": "4fe05636d5a9eb93f576904a774d30653a3686c7c8f61031ec0d316854cf5303", "role": "Proves omitted pipeline_config becomes PipelineConfig() and rendering restores the explicit assignment." }, { "path": "tests/unit/pyqt_gui/test_pipeline_editor_widget.py", - "sha256": "0ce88271aa7f5f55a8fc4daec8279d732844e03dfd437c11e6e6265ca196b4a0", + "sha256": "2024d9560dec38faee1fa922d2f8b7f547dd53d35e0b0e79376e45233445589a", "role": "Proves selected-dataset writability and that code apply advances saved baselines for the editor, step, and nested function states." }, { @@ -3733,7 +3742,7 @@ }, { "path": "tests/unit/test_pycodify_formatters.py", - "sha256": "2f0c6683a5255d01442ae33f676f601d5e3f1e3e04d1b2c2fc945b3c564796ce", + "sha256": "78d695333a74b4a75a07ca7de87a5c685a6fd429c67412b43375d16a298a9c80", "role": "Verifies clean/full parameter order and semantic default handling." }, { @@ -3743,12 +3752,12 @@ } ], "findings": [ - "The page called the multi-dataset window Plate Manager; after U1 the window renders the session's DatasetListView titled Datasets, so the four references now name the Datasets window and Datasets document.", - "Recorded authority openhcs/pyqt_gui/services/pipeline_object_state_binding.py was deleted in U1; pipeline replacement now goes through openhcs/authoring/session/pipelines.py and dataset document scopes through openhcs/authoring/session/dataset_document.py. The selected/all scope, global-draft protection, writability, and reconciliation claims were rechecked against them and still hold.", - "Other pages still name the window Plate Manager (24 pages under docs/source); reported to the lead rather than edited here." + "Recorded authority openhcs/pyqt_gui/services/config_window_code_document.py was deleted in U3; configuration code documents now go through openhcs/pyqt_gui/windows/config_window.py (OpenHCSConfigWindow over pyqt-reactive's ConfigWindow) and openhcs/core/config_document.py, which replace it as authorities.", + "L6 moved lazy-dataclass clean/full emission out of openhcs/serialization/pycodify_formatters.py into objectstate.codegen; the page's sparse clean-document claim now rests on external/ObjectState/src/objectstate/codegen.py, added as an authority, and the pycodify_formatters role was narrowed to function-pattern and step formatting. The page prose (clean vs resolved views, pycodify owning generic serialization) remains accurate, so it was not edited." ], - "disposition": "revise", + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/user_guide/code_ui_editing.rst in full and compared it against the changed authorities: pipeline_document.py (generate_python_source), dataset_document.py (objectstate.retain_only_authored_paths), pipelines.py and pipeline_editor.py (ScopeKey child scopes, GlobalEventBus broadcast removed, use_external_code_editor), deleted config_window_code_document.py (U3), ui_bridge_object_state.py (field markers now from objectstate.inspection FieldInspection; '*' dirty and '_' signature-diff unchanged), pycodify_formatters.py (lazy-dataclass formatters moved to objectstate.codegen, L6), pipeline_editor_workflows.py (K6 debug command classes), pyqt-reactive window_code_document/scope_token_service/function_pattern_code_document (current files), and the changed tests; selected/all scope apply, writability, unchanged no-op, revision tokens, occurrence reconciliation, signature parameter order and clean/resolved views all still hold; replaced and added authorities; no page edit needed.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/code_ui_editing.rst against B6's later changes to openhcs/authoring/session/views.py: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/code_ui_editing.rst against B6's later changes to openhcs/authoring/session/dataset_document.py, openhcs/authoring/session/views.py, openhcs/pyqt_gui/widgets/pipeline_editor.py, openhcs/pyqt_gui/services/ui_bridge_object_state.py, openhcs/pyqt_gui/services/ui_agent_bridge.py, openhcs/agent/services/ui_bridge_service.py, openhcs/pyqt_gui/widgets/shared/services/pipeline_editor_workflows.py, tests/unit/pyqt_gui/test_ui_agent_bridge.py, tests/unit/pyqt_gui/test_plate_manager_widget.py, tests/unit/pyqt_gui/test_pipeline_editor_widget.py, tests/unit/test_pycodify_formatters.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/code_ui_editing.rst against the later B6 changes to openhcs/authoring/session/dataset_document.py, openhcs/authoring/session/views.py, openhcs/pyqt_gui/widgets/pipeline_editor.py, openhcs/pyqt_gui/services/ui_bridge_object_state.py, openhcs/pyqt_gui/services/ui_agent_bridge.py, openhcs/agent/services/ui_bridge_service.py, openhcs/pyqt_gui/widgets/shared/services/pipeline_editor_workflows.py, tests/unit/pyqt_gui/test_ui_agent_bridge.py, tests/unit/pyqt_gui/test_plate_manager_widget.py, tests/unit/pyqt_gui/test_pipeline_editor_widget.py, tests/unit/test_pycodify_formatters.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -3793,7 +3802,7 @@ }, { "path": "openhcs/processing/backends/lib_registry/registry_service.py", - "sha256": "daef8925338fddf8504a9faa066af42456a13fe4602976bf64be06547a45fcc8", + "sha256": "f2e981157632adb5020828720ec26bfa6444ab52f545e1e71b1524242eaed6a3", "role": "Skips registries that do not support CPU-only mode during function discovery." }, { @@ -3803,13 +3812,14 @@ }, { "path": "pyproject.toml", - "sha256": "c3a2bea22aa6872940125f4aaac72c9eac72084fefc6c2cefdb54b5273361e4a", + "sha256": "b5d4d752704b886da20d860743b81dc3d2ce3a3c402208246fadc1c6cb71f597", "role": "Declares the optional gpu extra and the openhcs console entry point." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/user_guide/cpu_only_mode.rst in full and compared it against the changed authorities: registry_service.py (processing contract now read from reference.declaration instead of reference.metadata, no change to CPU-only registry skipping) and pyproject.toml (dependency floors only; gpu extra and openhcs entry point unchanged); confirmed OPENHCS_CPU_ONLY and --no-gpu are still declared; no edit needed.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/cpu_only_mode.rst against B6's later changes to openhcs/processing/backends/lib_registry/registry_service.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/cpu_only_mode.rst against the later B6 changes to openhcs/processing/backends/lib_registry/registry_service.py, tests/unit/test_library_registry_discovery.py, pyproject.toml: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/user_guide/cpu_only_mode.rst in full and compared it against the changed authorities: openhcs/utils/environment.py, registry_service.py, test_library_registry_discovery.py, and pyproject.toml; confirmed --no-gpu and OPENHCS_CPU_ONLY still select the same mode, discovery still filters registries by CPU-only support, and the gpu extra still exists; the new import-policy defaults (JAX platform, NumPy huge pages, worker profiling) do not contradict the page; no page change was needed.", @@ -3869,7 +3879,7 @@ }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "01dcb82b5850f3cdb1c300a2aa360ba330e7f15dffb5a24e37504311095090f9", + "sha256": "3cdb7819f09604046ba69e0b4f57b73e4e98d8f68af80158501a54bb42e701a2", "role": "Declares the Tools > Custom Functions > Manage Functions action." }, { @@ -3909,7 +3919,7 @@ }, { "path": "tests/unit/test_custom_function_lifecycle.py", - "sha256": "70c87f0dbefd2c1a204d060c2211ec57f4f84023f7c6931d2cb7d81eb82e9252", + "sha256": "e4f6405ff18dac865141586579ed95b92e143e3672f94e6fca6352476fa0b32d", "role": "Proves transactional create, load, replace, delete, collision, and source-revision behaviour." }, { @@ -3923,12 +3933,10 @@ "role": "Verifies native receiver destruction, delivery thread, garbage collection and cancellation." } ], - "findings": [ - "The page said the manager requires a supported memory decorator; it now requires exactly one function with a declared callable contract per source (the contract may name an execution scope rather than a memory type), so the sentence now says one decorated processing function per source.", - "openhcs/pyqt_gui/config.py was dropped as an authority; the page makes no UI-configuration claim and the MCP catalogue now comes from the session's endpoint catalogue in openhcs/mcp/context.py." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/user_guide/custom_function_management.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/main.py (U3 moved dock layout, shortcuts and time-travel navigation to pyqt-reactive; Tools > Custom Functions > Manage Functions unchanged) and tests/unit/test_custom_function_lifecycle.py (function references expose .declaration instead of .metadata, K5); CustomFunctionManager.register_from_code and its package export, and CustomFunctionRuntimeRegistry publishing FunctionMetadata, are unchanged, so the code example and prose remain accurate; no edit needed.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/custom_function_management.rst against B6's later changes to openhcs/processing/custom_functions/manager.py, openhcs/pyqt_gui/main.py, openhcs/mcp/context.py, tests/unit/test_custom_function_lifecycle.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/custom_function_management.rst against the later B6 changes to openhcs/processing/custom_functions/manager.py, openhcs/pyqt_gui/main.py, openhcs/mcp/context.py, tests/unit/test_custom_function_lifecycle.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/user_guide/custom_function_management.rst in full and compared it against the changed authorities: custom_functions/manager.py and runtime_registry.py, custom_function_manager_dialog.py, openhcs/pyqt_gui/main.py, endpoint_function_catalog_service.py, openhcs/mcp/context.py, gallery_catalog.py, and the catalogue and lifecycle tests; checked the menu path, register_from_code signature and package export, source validation rules, ephemeral retention on reload, and catalogue invalidation; corrected the decorator requirement.", @@ -3970,7 +3978,7 @@ }, { "path": "openhcs/core/processing_contracts.py", - "sha256": "d37b8a0645d69cb502a7da943b4cf86a87e2daa1cc5323b66b80323caf468f7c", + "sha256": "eb0f7fd99ded1c4d5957152ccf6abe1468436c13574a1d1bb83ce368d24b9f69", "role": "Declares the ProcessingContract classes Pure2DContract, Pure3DContract, VolumetricToSliceContract, and FlexibleContract." }, { @@ -3980,7 +3988,7 @@ }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "01dcb82b5850f3cdb1c300a2aa360ba330e7f15dffb5a24e37504311095090f9", + "sha256": "3cdb7819f09604046ba69e0b4f57b73e4e98d8f68af80158501a54bb42e701a2", "role": "Declares the Tools > Custom Functions > Create New Function action." }, { @@ -4000,7 +4008,7 @@ }, { "path": "tests/unit/test_custom_function_lifecycle.py", - "sha256": "70c87f0dbefd2c1a204d060c2211ec57f4f84023f7c6931d2cb7d81eb82e9252", + "sha256": "e4f6405ff18dac865141586579ed95b92e143e3672f94e6fca6352476fa0b32d", "role": "Proves validated custom declarations publish atomically without mirrored catalogue state." }, { @@ -4009,12 +4017,10 @@ "role": "Proves persisted custom functions propagate through registry discovery and the execution-server catalogue." } ], - "findings": [ - "After G5 made processing contracts a class family, the code example already imported Pure2DContract, but the contract list still named enum members PURE_2D, PURE_3D, VOLUMETRIC_TO_SLICE, and FLEXIBLE that no longer exist; the list now names Pure2DContract, Pure3DContract, VolumetricToSliceContract, and FlexibleContract.", - "unified_registry.py no longer owns the processing contract and was replaced as an authority by openhcs/core/processing_contracts.py." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/user_guide/custom_functions.rst in full and compared it against the changed authorities: processing_contracts.py (signature lookup through CallableDeclaration, K5), pyqt_gui/main.py (U3; Tools > Custom Functions > Create New Function unchanged) and tests/unit/test_custom_function_lifecycle.py (reference.declaration); the example's numpy decorator still accepts contract= with ProcessingContract subclasses (default FlexibleContract), and Pure2DContract, Pure3DContract, VolumetricToSliceContract and FlexibleContract still exist; no edit needed.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/custom_functions.rst against B6's later changes to openhcs/core/memory/decorators.py, openhcs/core/processing_contracts.py, openhcs/pyqt_gui/main.py, openhcs/processing/custom_functions/manager.py, tests/unit/test_custom_function_lifecycle.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/custom_functions.rst against the later B6 changes to openhcs/core/memory/decorators.py, openhcs/core/processing_contracts.py, openhcs/pyqt_gui/main.py, openhcs/processing/custom_functions/manager.py, tests/unit/test_custom_function_lifecycle.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/user_guide/custom_functions.rst in full and compared it against the changed authorities: the page's own earlier contract-import edit, openhcs/core/memory/decorators.py, unified_registry.py, custom_functions/manager.py and runtime_registry.py, external/arraybridge decorators.py, and the catalogue and lifecycle tests; checked the menu path, the decorator export, and the four contract classes in openhcs/core/processing_contracts.py; corrected the contract names in the list.", @@ -4041,7 +4047,7 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", "role": "Declares DtypeConfig with default_dtype_conversion (NATIVE_OUTPUT default) and StepMaterializationConfig, from which the lazy step forms are generated." }, { @@ -4071,18 +4077,19 @@ }, { "path": "tests/unit/test_callable_contract.py", - "sha256": "cc220f53107a08b0406b6e0dd49896e3e4c3c0af6e527b0ef00d4fdaba8f0b2c", + "sha256": "9170f8694f388dbed5bfd36d882a5c0a4cdf28f50f51f7fdeaf4a955a9f3b412", "role": "Proves callable contract derivation for dimensions, memory types, dtype policy, and outputs." }, { "path": "openhcs/core/steps/abstract.py", - "sha256": "63734f0c5198272b08f128ae974ada725c8201cb44efd0ce32500fd280b879bf", + "sha256": "9e710b3db635730f88258e5a6e18aed607876e296783022c06798f2bcd03d5e4", "role": "Declares the step-level dtype_config and step_materialization_config lazy overrides used by the examples." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/user_guide/dtype_conversion.rst in full and compared it against the changed authorities: openhcs/core/config.py (K4 source-binding classes moved in, PartitionFilterMode family, source-metadata field dtypes; DtypeConfig, LazyDtypeConfig and LazyStepMaterializationConfig unchanged), openhcs/core/steps/abstract.py (lazy config imports consolidated) and tests/unit/test_callable_contract.py (CallableMetadata folded into CallableDeclaration, K5; dtype_config remains a config-bound runtime parameter); both code examples' imports still resolve; no edit needed.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/dtype_conversion.rst against B6's later changes to openhcs/core/config.py, openhcs/core/memory/decorators.py, openhcs/processing/backends/processors/numpy_processor.py, tests/unit/test_callable_contract.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/dtype_conversion.rst against the later B6 changes to openhcs/core/config.py, openhcs/core/memory/decorators.py, openhcs/processing/backends/processors/numpy_processor.py, tests/unit/test_callable_contract.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/user_guide/dtype_conversion.rst in full and compared it against the changed authorities: openhcs/core/config.py, openhcs/core/memory/decorators.py, external/arraybridge decorators.py and array_operations.py, numpy_processor.py, external/PolyStore disk.py, test_callable_contract.py, and openhcs/core/steps/abstract.py; confirmed DtypeConfig.default_dtype_conversion and its NATIVE_OUTPUT default, the PRESERVE_INPUT and UINT8 members, LazyDtypeConfig and LazyStepMaterializationConfig imports from openhcs.core.config, create_projection, and step dtype_config/step_materialization_config parameters; no page change was needed.", @@ -4108,7 +4115,7 @@ "authority": [ { "path": "openhcs/pyqt_gui/main.py", - "sha256": "01dcb82b5850f3cdb1c300a2aa360ba330e7f15dffb5a24e37504311095090f9", + "sha256": "3cdb7819f09604046ba69e0b4f57b73e4e98d8f68af80158501a54bb42e701a2", "role": "Declares the Tools > Analysis Consolidation > Run Experimental Analysis action and its directory check for the config and results files." }, { @@ -4137,11 +4144,10 @@ "role": "Runs the documented MetaXpress directory task end to end." } ], - "findings": [ - "The page's claims still hold, but its recorded authorities named the deleted openhcs/formats/experimental_analysis.py and openhcs/core/config.py, which no longer declares ExperimentalAnalysisConfig; they were repointed to processing/backends/experimental_analysis/analysis.py and openhcs/domains/microscopy/config.py." - ], + "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/user_guide/experimental_layouts.rst in full and compared it against the changed authority openhcs/pyqt_gui/main.py (U3: dock layout, shortcuts and time-travel navigation moved to pyqt-reactive); the Run Experimental Analysis action and its directory dialog are unchanged, and ExperimentalAnalysisConfig.normalization_method still defaults to fold change in openhcs/domains/microscopy/config.py; no edit needed.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/experimental_layouts.rst against B6's later changes to openhcs/pyqt_gui/main.py, openhcs/processing/backends/experimental_analysis/analysis.py, openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py, tests/unit/test_experimental_analysis_engine.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/experimental_layouts.rst against the later B6 changes to openhcs/pyqt_gui/main.py, openhcs/processing/backends/experimental_analysis/analysis.py, openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py, tests/unit/test_experimental_analysis_engine.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/user_guide/experimental_layouts.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/main.py, the deleted openhcs/formats/experimental_analysis.py (moved to processing/backends/experimental_analysis/analysis.py), openhcs/core/config.py, unified_analysis_engine.py, and both experimental-analysis tests; checked the menu path, the five default filenames, normalization_method, and that variable_components and group_by remain step-config fields; no page change was needed.", @@ -4169,33 +4175,34 @@ "authority": [ { "path": "openhcs/pyqt_gui/main.py", - "sha256": "01dcb82b5850f3cdb1c300a2aa360ba330e7f15dffb5a24e37504311095090f9", + "sha256": "3cdb7819f09604046ba69e0b4f57b73e4e98d8f68af80158501a54bb42e701a2", "role": "Composes the PyQt desktop main window that the openhcs command launches." }, { "path": "tests/unit/pyqt_gui/test_ui_config_lifecycle.py", - "sha256": "5fd4194e70de5210a6b51283693a1d25c6a6a1278b8a869c7deebcbd583503e0", + "sha256": "78c9d42d91a384d0b9bce1139dfcd171a2d87a8781be4c078e2f9703bf302043", "role": "Prevents the removed Textual package and local UI mirrors from returning." }, { "path": "openhcs/core/pipeline_document.py", - "sha256": "8c554c8275c665f308eb098955bda517aa804eeb34418b5e0f41d412650ce7b5", + "sha256": "9a77a08367f901594859c8b6cae9609c9230a551f4f4e9ec6ea6cd2ed233c0f4", "role": "Declares the pipeline document as pipeline_steps plus PipelineConfig." }, { "path": "openhcs/core/config.py", - "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", "role": "Declares the nested processing, materialization, and streaming configuration dataclasses on PipelineConfig and steps." }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "2dc739b39ef2b8a468dca8cef2ab1900e2d0f1091c15ef54c24215a4abefc991", + "sha256": "4a33908fb42874e6d57eeea3f5305793ddbda41d13356f0a78501af38dcdfeb4", "role": "Compiles pipeline declarations into typed step plans and a CompiledExecutionBundle before execution." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/user_guide/index.rst in full and compared it against the changed authorities: pyqt_gui/main.py (U3 dock/shortcut/time-travel machinery moved to pyqt-reactive), tests/unit/pyqt_gui/test_ui_config_lifecycle.py (shortcut lifecycle tests moved with it; the textual_tui non-recurrence check remains), pipeline_document.py (generate_python_source), core/config.py (PartitionFilterMode family, K4 source-binding classes) and compiler.py (ScopeKey scope ids, K6 progress phase/status classes; still returns CompiledExecutionBundle); the PyQt-only interface, toctree and declaration-model claims remain accurate; no edit needed.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/index.rst against B6's later changes to openhcs/pyqt_gui/main.py, openhcs/core/config.py, openhcs/core/pipeline/compiler.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/index.rst against the later B6 changes to openhcs/pyqt_gui/main.py, openhcs/core/config.py, openhcs/core/pipeline/compiler.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/user_guide/index.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/main.py, test_ui_config_lifecycle.py, pipeline_document.py, and openhcs/core/pipeline/compiler.py; confirmed the Textual interface is still absent, the declaration is pipeline_steps plus PipelineConfig, nested step configuration still owns processing, input source, materialization and streaming options, and compilation still returns a CompiledExecutionBundle; all toctree targets exist; added openhcs/core/config.py for the nested-configuration claim; no page change was needed.", @@ -4235,8 +4242,8 @@ }, { "path": "openhcs/pyqt_gui/services/reactor_providers.py", - "sha256": "aecf1c77636095408486cab53a95cb0fc044fcf2ead0d7b72ac674842cf69abe", - "role": "Registers the log, process, status, and execution reactors consumed by desktop widgets." + "sha256": "04ad362d7bb31641f5a4ba0bfcc8670cd2e330a6eadc231a0ca5df7abed8d2b4", + "role": "Registers the OpenHCS log-discovery and server-scan providers that supply the Log Viewer's log files and live server list." }, { "path": "openhcs/pyqt_gui/config.py", @@ -4245,7 +4252,7 @@ }, { "path": "openhcs/runtime/zmq_execution_client.py", - "sha256": "147fb663d3ea60b1348063440e05b77d54b40634d1ae81cf4fb4f3b5457fbc99", + "sha256": "5df650f60c4d48cbf47c275d4e46b870b35c915ca8baa8e06e77f321578ca6d7", "role": "Spawns the execution server with its own per-port log file and startup status file and reports connection state." }, { @@ -4254,9 +4261,12 @@ "role": "Owns the execution-log-viewer capture, caption, accessibility text, and dimensions." } ], - "findings": [], + "findings": [ + "The audit role for reactor_providers.py described 'log, process, status, and execution reactors'; the module registers provider objects (log discovery, server scan, codegen, selection), so the role now names the log-discovery and server-scan providers the page relies on. The page prose itself remains accurate." + ], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/user_guide/log_viewer.rst in full and compared it against the changed authorities: reactor_providers.py (codegen provider now calls pycodify.generate_python_source; external-editor env var) and zmq_execution_client.py (debug requests sent directly via request.to_dict(), K6 DebugCommandType class); neither touches log discovery, per-port server logs or live server scanning; confirmed the execution-log-viewer gallery scenario and the pipeline_debugging_guide link still exist; corrected the reactor_providers role.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/log_viewer.rst against B6's later changes to openhcs/pyqt_gui/services/reactor_providers.py, openhcs/runtime/zmq_execution_client.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/log_viewer.rst against the later B6 changes to openhcs/pyqt_gui/services/reactor_providers.py, openhcs/runtime/zmq_execution_client.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/user_guide/log_viewer.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/services/reactor_providers.py, openhcs/runtime/zmq_execution_client.py, and scripts/gallery_catalog.py; the reactor changes (axis-family grouping, session scope) and the execution server's per-port log and startup-status files are consistent with the page's claims that subprocesses write dedicated logs reported by the launching process and that readiness is recorded separately; no page change was needed.", @@ -4435,12 +4445,12 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "57add27eeb590c61324df6bafe488435d3177e7022e39e1827ae6b1a8d26adde", + "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", "role": "Declares StreamingDefaults (persistent, host, transport_mode, enabled), partition_filter, and the Napari and Fiji streaming configs with their ports." }, { "path": "openhcs/core/steps/abstract.py", - "sha256": "63734f0c5198272b08f128ae974ada725c8201cb44efd0ce32500fd280b879bf", + "sha256": "9e710b3db635730f88258e5a6e18aed607876e296783022c06798f2bcd03d5e4", "role": "Declares the per-step napari_streaming_config, fiji_streaming_config, and step_materialization_config fields." }, { @@ -4450,7 +4460,7 @@ }, { "path": "pyproject.toml", - "sha256": "c3a2bea22aa6872940125f4aaac72c9eac72084fefc6c2cefdb54b5273361e4a", + "sha256": "b5d4d752704b886da20d860743b81dc3d2ce3a3c402208246fadc1c6cb71f597", "role": "Declares the gui, napari, fiji, and viz extras." }, { @@ -4509,12 +4519,10 @@ "role": "Owns the Napari ROI-navigation and Fiji ROI-review scenarios, captions, and alt text." } ], - "findings": [ - "Step 3 listed batching and well filters among viewer options; streaming configs declare no batching option, and the filter field is partition_filter (G1), so the step now lists persistence, partition filters, transport, and display options.", - "max_rois is declared on openhcs/agent/dto/viewer.py, not on the recorded viewer authorities, so that file was added as an authority." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/user_guide/real_time_visualization.rst in full and compared it against the changed authorities: openhcs/core/config.py (PartitionFilterMode became a subclass family, IncludePartitions/ExcludePartitions; partition_filter, persistent and the streaming configs unchanged; K4 source-binding classes moved in), openhcs/core/steps/abstract.py (lazy-config imports consolidated; napari_streaming_config, fiji_streaming_config and step_materialization_config still declared) and pyproject.toml (dependency floors only; gui, napari, fiji and viz extras unchanged); no edit needed.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/real_time_visualization.rst against B6's later changes to openhcs/core/config.py, openhcs/runtime/viewer_protocol.py, openhcs/runtime/fiji_viewer_server.py, tests/unit/test_napari_streaming_handlers.py, tests/unit/test_viewer_result_element_selection.py, tests/unit/test_fiji_viewer_server.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/real_time_visualization.rst against the later B6 changes to openhcs/core/config.py, pyproject.toml, openhcs/runtime/viewer_protocol.py, openhcs/runtime/fiji_viewer_server.py, tests/unit/test_napari_streaming_handlers.py, tests/unit/test_viewer_result_element_selection.py, tests/unit/test_fiji_viewer_server.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/user_guide/real_time_visualization.rst in full and compared it against the changed authorities: the page's own partition_filter edit, openhcs/core/config.py, viewer_protocol.py, viewer_controls.py, viewer_window_service.py, napari_viewer_server.py, fiji_viewer_server.py, napari_streaming_handlers.py, the three viewer tests, and gallery_catalog.py; confirmed the extras, the per-step streaming fields, persistent and partition_filter, max_rois, the selected-layer overlay controller, and forward-progress settlement; corrected the step-3 option list.", diff --git a/docs/source/architecture/code_ui_interconversion.rst b/docs/source/architecture/code_ui_interconversion.rst index 371046e64f..2d492c0196 100644 --- a/docs/source/architecture/code_ui_interconversion.rst +++ b/docs/source/architecture/code_ui_interconversion.rst @@ -161,8 +161,10 @@ Round-trip fidelity means: Function parameter order is projected by python-introspect from the function signature for both clean and full code views. The OpenHCS formatter supplies -its semantic equality rule and whether signature defaults should be omitted; -extra keywords retain their supplied order after the declared parameters. +ObjectState's semantic equality rule and whether signature defaults should be +omitted; extra keywords retain their supplied order after the declared +parameters. ObjectState's code generation decides which lazy configuration +fields a clean view emits. When a nested function ObjectState exists, pyqt-reactive projects its current values into the function pattern, including groups whose panes are not visible. diff --git a/docs/source/architecture/concurrency_model.rst b/docs/source/architecture/concurrency_model.rst index a48f743858..d76cc2eaec 100644 --- a/docs/source/architecture/concurrency_model.rst +++ b/docs/source/architecture/concurrency_model.rst @@ -55,11 +55,11 @@ owned worker processes. A callable already in progress reaches its next safe boundary before stopping. Cancellation never becomes success or failure when late worker completion or an exception reaches the server, and every path still runs cleanup. -Worker ``AXIS_COMPLETED`` events close only the axis-scoped portion. Terminal +Worker ``AxisCompletedProgressPhase`` events close only the axis-scoped portion. Terminal dataset-scoped steps, post-execute hooks such as the microscopy domain's output consolidation, viewer settlement, and final state projection remain parent-owned; :doc:`progress_runtime_projection_system` defines the -separate final ``SUCCESS`` boundary. +separate final ``SuccessProgressPhase`` boundary. Invariants ---------- diff --git a/docs/source/architecture/external_foundations.rst b/docs/source/architecture/external_foundations.rst index e7c319eeb2..98d5483d77 100644 --- a/docs/source/architecture/external_foundations.rst +++ b/docs/source/architecture/external_foundations.rst @@ -11,7 +11,9 @@ Ownership catalog ObjectState Owns lazy configuration types, context resolution, ObjectState editing, - snapshots, provenance, undo, and generic projection/composition. OpenHCS owns + snapshots, provenance, undo, generic projection/composition, the + ``ScopeKey`` scope grammar, field inspection, and the Python-source + formatting of lazy configurations (``objectstate.codegen``). OpenHCS owns its configuration hierarchy, compiler resolution boundary, and UI/code use. See :external+objectstate:doc:`ObjectState documentation `. @@ -46,15 +48,17 @@ PolyStore pyqt-reactive Owns generic reactive forms, widget protocols and strategies, managers, - previews, reusable services, live Qt-window discovery and infrastructure, and - Qt-rendered window snapshots. OpenHCS owns concrete workflows, Plate Manager, + previews, reusable services, live Qt-window discovery and infrastructure, + Qt-rendered window snapshots, editable tables, grid selection, time-travel + widgets and navigation, configuration windows, and dock and shortcut layout. + OpenHCS owns concrete workflows, Plate Manager, pipeline editors, application window identity and authorisation, and domain transport adapters. See :external+pyqt-reactive:doc:`pyqt-reactive documentation `. python-introspect Owns callable/signature analysis, wrapped-target resolution, parameter - metadata, callable-declaration projection, analyzer extensions, the + metadata, field labels, callable-declaration projection, analyzer extensions, the ``to_jsonable`` encoder paired with the ``dataclass_from_mapping`` decoder, and declared public-API and lazy-export helpers. OpenHCS owns domain exclusions, callable contracts, and step-editor consumption. See @@ -71,7 +75,8 @@ ZMQRuntime `_. pycodify - Owns Python-source serialization, formatter registration, import construction, + Owns Python-source serialization, formatter registration, qualified-name + import construction, value-owned source literals, absolute-path factoring, and collision handling. OpenHCS owns formatters and round-trip policies for its domain declarations. See the `pycodify repository `_. diff --git a/docs/source/architecture/pattern_grouping_and_special_outputs.rst b/docs/source/architecture/pattern_grouping_and_special_outputs.rst index 19f907298b..8c2e47af0d 100644 --- a/docs/source/architecture/pattern_grouping_and_special_outputs.rst +++ b/docs/source/architecture/pattern_grouping_and_special_outputs.rst @@ -7,7 +7,7 @@ Grouping and output terminology (transition) output-slot names, artifact declarations, and VFS paths. Those are independent systems in the current architecture. -Use these canonical pages: +Use these pages: - :doc:`processing_semantics` for ``variable_components``, ``group_by``, dictionary pattern fan-out, and ``ProcessingContract``; diff --git a/docs/source/architecture/pipeline_compilation_system.rst b/docs/source/architecture/pipeline_compilation_system.rst index f7c1b2c9b0..d618d44767 100644 --- a/docs/source/architecture/pipeline_compilation_system.rst +++ b/docs/source/architecture/pipeline_compilation_system.rst @@ -41,7 +41,7 @@ wrapper identity rather than create a second callable projection. Native module families may declare both their owned callable names and the catalogue module that exposes them. Catalogue discovery projects those nominal -claims, rejects duplicate canonical names, and does not maintain a parallel +claims, rejects duplicate callable names, and does not maintain a parallel module-to-function list. An execution endpoint prepares the registered catalogue before it reports diff --git a/docs/source/architecture/progress_runtime_projection_system.rst b/docs/source/architecture/progress_runtime_projection_system.rst index e7a98f18c7..be1f7b1ae3 100644 --- a/docs/source/architecture/progress_runtime_projection_system.rst +++ b/docs/source/architecture/progress_runtime_projection_system.rst @@ -16,8 +16,9 @@ OpenHCS Implementation Path OpenHCS-specific modules: -- ``openhcs.core.progress.types`` owns the registered channel, phase, and status - semantics carried by ``ProgressEvent``. +- ``openhcs.core.progress.types`` owns the ``ProgressStream``, ``ProgressPhase``, + and ``ProgressStatus`` class families carried by ``ProgressEvent``; each phase + owns its stream and terminal semantics. - ``openhcs.core.progress.registry`` owns OpenHCS semantic keying over the generic registry. - ``openhcs.core.progress.projection`` and @@ -31,7 +32,7 @@ Consumption path: 1. Emit typed ``ProgressEvent`` values from compile/execute flows. 2. Route events through explicit queue wiring via ``set_progress_queue(...)``. -3. Register events in ``ProgressRegistry`` with semantic channel keying. +3. Register events in ``ProgressRegistry`` keyed by subject, axis, and stream. 4. Reconcile typed server-heartbeat running/queued entries by execution and subject identity in ``ExecutionRuntimeProjection``. 5. Build runtime and execution-tree projections for status derivation. @@ -45,7 +46,7 @@ OpenHCS Invariants ------------------ - The execution tree path is ``subject -> worker -> partition -> step``. -- ``PATTERN_GROUP`` is step-detail only and does not set partition pipeline +- ``PatternGroupProgressPhase`` is step-detail only and does not set partition pipeline percent. - Subject/server status is derived from projection snapshots, not mutable cache. - Event retention is monotonic per semantic key. Stale or duplicate timestamps @@ -67,10 +68,11 @@ Debugger ownership Debugger execution uses the normal progress transport but does not reduce its retained state to status text: -- ``DebugCommandDeclarationBase`` owns command semantics, while the registered - execution and stop strategies implement command-specific runtime behavior. -- ``DebugBoundaryEventDeclarationBase`` owns the progress status, boundary - outcome, timeline state, and artifact-reporting policy for each boundary event. +- Each ``DebugCommandType`` subclass is one command and owns what it does to a + paused worker; the execution and stop strategy families, keyed by command + class, implement command-specific bounded-run behaviour. +- Each ``DebugBoundary`` subclass is one boundary event and owns its progress + status and whether it reports output artifacts or carries an exception. - ``DebugCursor`` and ``DebugSession`` retain the exact invocation and controller identity. ``DebugSnapshot`` retains boundary evidence, and ``DebugArtifactRef`` plus ``DebugArtifactIdentity`` retain typed artifact provenance. @@ -88,7 +90,7 @@ Debugger projection path ``ProgressEvent``. ``RuntimeProjectionBuilder`` then builds the normal execution projection and the debugger projection from the same immutable event snapshot. ``DebugRuntimeProjectionBuilder`` filters events by debug session, resolves their -snapshots, and derives current, last, and timeline frames while retaining the +snapshots, and derives the current and last frames while retaining the normal execution projection beside them. The runtime projection is not the session-state machine. Given the selected @@ -132,22 +134,22 @@ command declarations, debugger snapshots, or debugger session phase state. Axis completion is not plate completion --------------------------------------- -Workers execute only axis-scoped steps. ``AXIS_COMPLETED`` reports the number -of completed axis positions and deliberately stops at the first terminal -plate-scoped step. It is a successful terminal event for that worker/axis -channel, but it does not assert that the plate execution has finished. +Workers execute only axis-scoped steps. ``AxisCompletedProgressPhase`` reports +the number of completed axis positions and deliberately stops at the first +terminal plate-scoped step. It is a successful terminal event for that +worker/axis key, but it does not assert that the plate execution has finished. After every worker result succeeds, the parent merges runtime records and calls ``execute_dataset_scoped_steps()``, which runs each terminal plate-scoped step once. The parent then materializes those outputs, runs analysis consolidation, settles viewer state, and projects the orchestrator result. Only after those stages succeed -does it emit the pipeline ``SUCCESS`` event at 100 percent. Failure, -cancellation, viewer settlement, or terminal-step errors therefore cannot be -hidden behind an earlier worker ``AXIS_COMPLETED`` event. +does it emit the pipeline ``SuccessProgressPhase`` event at 100 percent. +Failure, cancellation, viewer settlement, or terminal-step errors therefore +cannot be hidden behind an earlier worker ``AxisCompletedProgressPhase`` event. -Canonical Abstraction Docs --------------------------- +Abstraction Docs +---------------- See the published ``zmqruntime`` owner documentation for abstraction internals: diff --git a/docs/source/architecture/serialization_boundaries.rst b/docs/source/architecture/serialization_boundaries.rst index 995729ad35..ef835c2806 100644 --- a/docs/source/architecture/serialization_boundaries.rst +++ b/docs/source/architecture/serialization_boundaries.rst @@ -8,8 +8,11 @@ Python source ------------- pycodify owns generic executable-Python generation, import construction, -formatter registration, collision handling, and clean versus explicit output. -OpenHCS owns formatters and round-trip policy for its domain declarations. Code +formatter registration, collision handling, absolute-path factoring, and clean +versus explicit output; a value that owns its source spelling subclasses +pycodify's ``PythonSourceLiteral``. ``objectstate.codegen`` owns source +formatting for lazy configuration dataclasses. OpenHCS owns formatters and +round-trip policy for its domain declarations. Code documents are user-editable source and must pass the normal validation boundary before application. diff --git a/docs/source/architecture/source_model.rst b/docs/source/architecture/source_model.rst index b76573e06e..be4966e911 100644 --- a/docs/source/architecture/source_model.rst +++ b/docs/source/architecture/source_model.rst @@ -68,9 +68,10 @@ Declaration vocabulary A regex over file or folder names with named capture groups. The captured values become typed source metadata used for matching and grouping. -``SourceBindingMatchPlan`` - The explicit order- or metadata-based rule for pairing multiple aliases into - one logical source set. +``SourceSetPairing`` + The rule for pairing multiple aliases into one logical source set: + ``PairByOrder`` (the default when no method is declared) or + ``PairByMetadata`` over shared metadata keys. Compilation ----------- diff --git a/docs/source/architecture/system_overview.rst b/docs/source/architecture/system_overview.rst index 7695fa2aec..6975aa95d2 100644 --- a/docs/source/architecture/system_overview.rst +++ b/docs/source/architecture/system_overview.rst @@ -101,12 +101,15 @@ OpenHCS owns domain wiring: microscopy sources, pipeline declarations, compiler plans, processing contracts, artifacts, runtime values, and integrations. The extracted packages own their generic mechanisms: -- ObjectState: lazy configuration and edit/snapshot machinery +- ObjectState: lazy configuration, edit/snapshot machinery, the scope-key + grammar (``ScopeKey``), and lazy-configuration source formatting + (``objectstate.codegen``) - ArrayBridge: memory types, callable memory-role metadata, conversion, child import environments, and framework-local device/execution mechanics - PolyStore: storage, formats, ROI, and virtual-workspace primitives - metaclass-registry: generic nominal registration and logging-safe discovery -- pyqt-reactive: generic reactive forms and widgets +- pyqt-reactive: generic reactive forms and widgets, configuration windows, + editable tables, grid selection, and time-travel views - python-introspect: callable/signature analysis, declaration projection, and the JSON codec (``to_jsonable``/``dataclass_from_mapping``) - ZMQRuntime: generic process and transport protocols diff --git a/docs/source/architecture/ui_services_architecture.rst b/docs/source/architecture/ui_services_architecture.rst index 5df3ea327e..0f2371e83c 100644 --- a/docs/source/architecture/ui_services_architecture.rst +++ b/docs/source/architecture/ui_services_architecture.rst @@ -149,8 +149,9 @@ first reconciles every live consumer and only then publishes the runtime-context value and ``ui_config_changed`` signal. The system monitor applies ``performance_monitor`` through its idempotent update owner. Plate Manager passes the execution transport declaration and progress-update interval to the -session it renders. The main-window shortcut lifecycle validates the complete key -set before rebinding its concrete QActions and time-travel event filter. ZMQ +session it renders. pyqt-reactive's ``ShortcutLifecycle`` validates the complete +key set before rebinding the main window's QActions and its application shortcut +filter, which carries the time-travel commands. ZMQ Manager rebuilds its scan inputs from the current transport declaration and the bridge's actual running binding, never from an unstarted configured port. The logging lifecycle rebuilds its owned console and rotating-file handlers from @@ -188,7 +189,9 @@ boundary before the ordinary XDG path is chosen; it does not add another configuration value or persistence implementation. The two-tab configuration window is one transaction, not two independent Save -callbacks. It validates all materialized forms and runs mutation guards before +callbacks. pyqt-reactive's ``ConfigWindow`` owns that transaction; +``OpenHCSConfigWindow`` only supplies the OpenHCS configuration-document format +and window identities. It validates all materialized forms and runs mutation guards before reconstruction. It then checkpoints every affected ObjectState plus the independent saved/live global contexts, commits all states, and invokes explicit reversible persistence/live participants. A participant or saved-state failure diff --git a/docs/source/concepts/core_model.rst b/docs/source/concepts/core_model.rst index f258517df3..521a14e8b0 100644 --- a/docs/source/concepts/core_model.rst +++ b/docs/source/concepts/core_model.rst @@ -19,7 +19,7 @@ The model in one diagram | | resolve configuration once v - snapshots + compilation sessions + resolved pipeline + compilation sessions | | validate sources, artifacts, | memory, scope, and workers @@ -43,7 +43,8 @@ produce the same declarations. Sources ------- -Microscope handlers interpret plate layouts and metadata. Source bindings give +Dataset sources (one ``DatasetSource`` per format) interpret plate layouts and +metadata. Source bindings give semantic aliases to selected images and metadata. The compiler resolves those bindings against an explicit source universe and virtual-workspace projection; runtime adapters do not rescan directories or reinterpret filenames. diff --git a/docs/source/concepts/function_library.rst b/docs/source/concepts/function_library.rst index fdb9f12bd2..f5fbaffb6d 100644 --- a/docs/source/concepts/function_library.rst +++ b/docs/source/concepts/function_library.rst @@ -18,7 +18,10 @@ declare: - a ``ProcessingContract`` subclass; - runtime adapter and image execution mode; - axis- or plate-level execution scope; -- compiler preparation hooks. +- the signatures a compiler preparation hook produced. + +The preparation hook itself is attached to the callable +(``processing_prepare``), not carried in the contract. Image transport --------------- diff --git a/docs/source/concepts/module_structure.rst b/docs/source/concepts/module_structure.rst index 9dcc734147..53cd057f3d 100644 --- a/docs/source/concepts/module_structure.rst +++ b/docs/source/concepts/module_structure.rst @@ -53,16 +53,19 @@ Extracted foundations The submodules under ``external/`` own reusable behavior: -- ObjectState — lazy config and state/provenance +- ObjectState — lazy config, state/provenance, scope keys, and Python source + for lazy dataclasses - ArrayBridge — array conversion, callable memory metadata, child import environments, and framework-local device and execution mechanics - PolyStore — storage, formats, ROI, and virtual workspace - metaclass-registry — generic nominal registry mechanics and logging-safe discovery imports -- pyqt-reactive — generic forms/widgets/services +- pyqt-reactive — generic forms, widgets (editable tables, grid selector, + time travel), windows, and services - python-introspect — callable/signature analysis and declaration projection - ZMQRuntime — generic execution and transport protocols -- pycodify — Python-source serialization +- pycodify — Python-source serialization, including document-level path + factoring The root ``pyproject.toml`` declares their published distributions. Development checkouts install the submodules explicitly; see diff --git a/docs/source/development/callable_artifact_authoring.rst b/docs/source/development/callable_artifact_authoring.rst index 8b06c44fc9..5907324f56 100644 --- a/docs/source/development/callable_artifact_authoring.rst +++ b/docs/source/development/callable_artifact_authoring.rst @@ -10,8 +10,10 @@ Callable contract The contract can carry input, output, and execution memory types, artifact inputs and outputs, runtime-bound parameters, required axis roles, allowed -group-by roles, processing contract, execution scope, runtime adapter, -preparation hook, and a callable request binding. The execution memory role is +group-by roles, processing contract, execution scope, runtime adapter, the +signatures a preparation hook produced, and a callable request binding. The +hook itself stays on the callable (``processing_prepare``); it is not part of +the transported contract. The execution memory role is the framework whose device scope must be active while the callable runs; it may differ from an input or output conversion boundary. Use the existing decorators and declaration helpers; do not teach compiler phases to inspect backend names. diff --git a/docs/source/development/source_binding_extension.rst b/docs/source/development/source_binding_extension.rst index 52e0573d28..30df7368aa 100644 --- a/docs/source/development/source_binding_extension.rst +++ b/docs/source/development/source_binding_extension.rst @@ -23,8 +23,9 @@ Strategy selection ------------------ Existing families select source universes, matching behavior, component -projection, and workspace projection, for example ``SourceUniverseRequest`` -and ``SourceFilterMatcher``. Add a concrete subclass to the correct family root +projection, and workspace projection, for example ``SourceUniverseRequest``, +``SourceFilterMatcher``, and the ``SourceSetPairing`` family (``PairByOrder``, +``PairByMetadata``) that decides how matched sources form source sets. Add a concrete subclass to the correct family root and declare its registry key. Generic consumers should continue to use the root registry or its strategy mixin. diff --git a/docs/source/guide_for_biologists/image_sources.rst b/docs/source/guide_for_biologists/image_sources.rst index 5c1fccffd8..75e0c3ce9b 100644 --- a/docs/source/guide_for_biologists/image_sources.rst +++ b/docs/source/guide_for_biologists/image_sources.rst @@ -271,9 +271,9 @@ relation from their selected image inputs; a custom callable's input names and Check the compiled image and artifact scopes before the first execution, then read back each requested source's own values and label/object identity. Stack assembly, image-set context and intensity-unit conversion are separate -decisions: fixing one does not prove the others. A failed receipt remains -preserved; validate a separately recorded declaration repair rather than replay -an uncertain operation. +decisions: fixing one does not prove the others. Keep the record of a failed +run; validate a separately recorded declaration repair rather than replay an +uncertain operation. Executable code-mode declarations --------------------------------- From f419cb3bbcfd73703a61a062300a192f6dc032e9 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 12:21:51 -0400 Subject: [PATCH 69/84] B6: receipt - library versions after merging main Co-Authored-By: Claude Opus 5.5 --- docs/refactor/consolidation-20261009/B6-ci-green.md | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/docs/refactor/consolidation-20261009/B6-ci-green.md b/docs/refactor/consolidation-20261009/B6-ci-green.md index 38d3ec4f19..250e389d9e 100644 --- a/docs/refactor/consolidation-20261009/B6-ci-green.md +++ b/docs/refactor/consolidation-20261009/B6-ci-green.md @@ -15,17 +15,17 @@ | unit-tests (abort, exit 134) | An in-process MCP transport created a `QCoreApplication`, so a later `QWidget` aborted. | The session creates its `QApplication` first (`tests/conftest.py`). | | code-quality | 412 unformatted files since the last success. A test used Python 3.12-only f-string quote reuse, which is a syntax error on 3.11. | Black over this PR's changed files; quote fixed. Main's baseline becomes this PR's green head. | | structural ratchet | Black wrapping and B6 fixes grew six classes past their budget. | Cohesive method groups moved into inherited mixins: `MainOutputSliceProjectionMixin`, `FunctionCoreDebugMixin`, `CustomFunctionSourceFilesMixin`, `SessionDebugMixin`. Catalog policy moved to module functions. Four unused main-window theme methods and `SourceImageProvenance.source_component_identity` were deleted (**unused**: no caller, menu, bridge or string reference). | -| Windows (every ZMQ job) | `tests/conftest.py` used `os.getpgid`. zmqruntime's `startup_owner` reopened the lock file that portalocker holds (mandatory locks give PermissionError). | Process-group reaping only on POSIX. zmqruntime reads and writes the owner record through the held handle (0.6.1). | +| Windows (every ZMQ job) | `tests/conftest.py` used `os.getpgid`. zmqruntime's `startup_owner` reopened the lock file that portalocker holds (mandatory locks give PermissionError). | Process-group reaping only on POSIX. zmqruntime reads and writes the owner record through the held handle (0.6.2). | | Windows | NumPy exports uint32 as format `'L'`; custom-function sources were written in text mode, which changes their hashes; SIGINT can't be sent to a child; chmod doesn't make a directory read-only and `os.access` ignores ACLs; autocrlf rewrote checksummed `.cppipe` files; packaged knowledge paths were longer than MAX_PATH. | Native granularity accepts both formats. Sources are persisted as exact bytes. The CLI treats Ctrl+Break as Ctrl+C. Writability is decided by an actual file-creation attempt, and the test denies writes with an ACL. `.gitattributes` keeps committed bytes (`* -text`). Recipes are placed by case name within a declared path budget. | | Intel macOS | `openhcs-basicpy` is excluded by marker, but the catalog imported its module. The kernel cache compiled serially because macOS has no `sched_getaffinity`. | Modules declare `REQUIRED_DISTRIBUTIONS`, and the catalog excludes those whose package markers leave them out. Workers come from `os.cpu_count()` where affinity is unavailable. | -| python-314 | ObjectState read a class's own annotations through `__dict__`, which 3.14 no longer populates. mahotas sat only in an extra, but core catalog modules need it. | ObjectState uses `inspect.get_annotations` (1.3.2). mahotas is a core dependency. | +| python-314 | ObjectState read a class's own annotations through `__dict__`, which 3.14 no longer populates. mahotas sat only in an extra, but core catalog modules need it. | ObjectState uses `inspect.get_annotations` (2.1.1). mahotas is a core dependency. | | gui-tests (13) | Stale tests after U1/U5. | Rewritten against the session-backed plate manager and the current stubs. | | backend-microscope, boundary, OMERO | STEP_INPUT filters matched virtual names instead of stored source paths. Passthrough outputs dropped their producer's filename qualifier. BaSiC read `.metadata` off plain arrays. `OMEROLocalBackend` had no `is_dir`. The metadata document was written to local disk for a virtual backend. Viewer tests used `ViewerType` members. The GUI journey test ran in headless jobs. Several integration tests had gone stale. | Fixed at each owner: `SourceImageIdentity.filter_paths`, producer qualifiers, `ImagePayload.of`, `OMEROLocalBackend.is_dir`/`is_file` (PolyStore 0.6.1), `OpenHCSMetadataTarget.hosts_metadata_document` (not for a `VirtualBackend`), viewer families, and the GUI journey moved to gui-tests. | | Official30 real-viewer smoke | `required_scalar_components` pooled contributors across runtime planes and dropped `site`, so napari rejected the stack. | Contributors are grouped per plane (`contributor_groups`). | | Installed demo (Windows, Intel macOS) | The one-plane label projection (#477) lived only on `MatchImageStackLabels`; the skeleton step used `SliceAlignedLabels`. | The projection is owned by `ObjectLabelInputExecutionMode` (**derived**: `MatchImageStackLabels`' copy deleted). Dataset rows carry the finished execution's error. | | desktop live MCP | The smoke read the catalog before the GUI's endpoint was up. Requestless UI-bridge operations received the connection in the request slot. | `execution_services_prepared` future; connection passed by keyword. | | wheel-integration | The MCP smoke sent `wells` (G2 renamed it `partition_values`). Bio-Formats tests ran without the declared Fiji bundle. | `partition_values`; the job restores and prewarms the bundle. | -| pypi-dependency-readiness | Pinned submodule sources differ from their release tags. | Version bumps: pycodify 0.1.4, zmqruntime 0.6.1, PolyStore 0.6.1, ObjectState 1.3.2. Floors raised. **The job stays red until the owner merges and tags the four library PRs.** | +| pypi-dependency-readiness | Pinned submodule sources differ from their release tags. | Version bumps: zmqruntime 0.6.2, PolyStore 0.6.1, ObjectState 2.1.1 (pycodify's change shipped in main's 0.2.0). Floors raised. **The job stays red until the owner merges and tags the three library PRs.** | ## Guards @@ -43,8 +43,7 @@ Website, Documentation and Integration Tests are green on this PR's head, or eac ## Lockstep library PRs (owner merges and tags) -- OpenHCSDev/pycodify#3 (0.1.4) -- OpenHCSDev/ZMQRuntime#29 (0.6.1; docs; startup lock) +- OpenHCSDev/ZMQRuntime#29 (0.6.2; docs; startup lock) - OpenHCSDev/PolyStore#38 (0.6.1; OMERO `is_dir`; docs logs) -- OpenHCSDev/ObjectState#26 (1.3.2) +- OpenHCSDev/ObjectState#26 (2.1.1) - OpenHCSDev/PyQT-reactive#21 (docs logs) From d43533f9003f60672f30accfe1cd7f0bfe67de01 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 12:26:42 -0400 Subject: [PATCH 70/84] B6: unit session asks for pytest-qt's QApplication only where the plugin is loaded The website job runs tests/unit with --confcutdir=tests/unit, which loads no pytest-qt; the fixture now depends on the plugin being present. Co-Authored-By: Claude Opus 5.5 --- tests/unit/conftest.py | 21 ++++++++++----------- 1 file changed, 10 insertions(+), 11 deletions(-) diff --git a/tests/unit/conftest.py b/tests/unit/conftest.py index 7e7f38f10f..53cfa68a4f 100644 --- a/tests/unit/conftest.py +++ b/tests/unit/conftest.py @@ -2,16 +2,15 @@ import pytest -from openhcs.utils.environment import OpenHCSProcessEnvironment -if not OpenHCSProcessEnvironment.cpu_only_mode(): +@pytest.fixture(scope="session", autouse=True) +def session_qapplication(request): + """Create the unit session's one QApplication before any test runs. - @pytest.fixture(scope="session", autouse=True) - def session_qapplication(qapp): - """Create the unit session's one QApplication before any test runs. - - A process holds one Qt application. In-process MCP transports adopt an - existing one and otherwise create a QCoreApplication, after which no - later unit test could create a widget. - """ - yield qapp + A process holds one Qt application. In-process MCP transports adopt an + existing one and otherwise create a QCoreApplication, after which no later + unit test could create a widget. Sessions without pytest-qt (CPU-only, or a + narrowed ``--confcutdir``) run no widget tests. + """ + if request.config.pluginmanager.hasplugin("pytestqt"): + request.getfixturevalue("qapp") From 0250ad848436dfae54ddb20ab740d1b679fa9d5e Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 15:42:33 -0400 Subject: [PATCH 71/84] B6: keep main's python-introspect 0.2.8 pin after the merge Co-Authored-By: Claude Opus 5.5 --- external/python-introspect | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/external/python-introspect b/external/python-introspect index 046a1f6be7..a88fe56f78 160000 --- a/external/python-introspect +++ b/external/python-introspect @@ -1 +1 @@ -Subproject commit 046a1f6be7713ac05efa7ccac4199250e5fa9ee3 +Subproject commit a88fe56f78b523ad5c791e3f868bb35d7d6b3f12 From 297a423d5b53f6e1a17b9f3e6dd55e23ec9fc62c Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 15:55:52 -0400 Subject: [PATCH 72/84] B6: re-review 127 pages after main's K2, K3, N1 and E1 merges Pages and examples naming deleted enum members or the old artifacts module now name the class families and the artifacts package; digests pinned after review. Co-Authored-By: Claude Opus 5.5 --- docs/audits/architecture_a_m.json | 389 ++++++----- docs/audits/architecture_n_z_development.json | 540 ++++++++------- docs/audits/project_surface.json | 29 +- docs/audits/user_docs.json | 637 ++++++++++-------- .../architecture/artifact_contract_system.rst | 2 +- .../architecture/cellprofiler_interop.rst | 2 +- .../architecture/code_ui_interconversion.rst | 3 +- .../experimental_analysis_system.rst | 4 +- .../architecture/external_foundations.rst | 4 +- .../measurement_equivalence_system.rst | 16 +- .../architecture/processing_semantics.rst | 2 +- .../progress_runtime_projection_system.rst | 2 +- docs/source/architecture/quick_start.rst | 22 +- .../architecture/runtime_value_system.rst | 13 +- .../development/source_binding_extension.rst | 7 +- .../guide_for_biologists/image_sources.rst | 18 +- docs/source/guides/complete_examples.rst | 4 +- docs/source/guides/example_corpus_map.rst | 4 +- 18 files changed, 942 insertions(+), 756 deletions(-) diff --git a/docs/audits/architecture_a_m.json b/docs/audits/architecture_a_m.json index 35f5c2be22..005b416ed2 100644 --- a/docs/audits/architecture_a_m.json +++ b/docs/audits/architecture_a_m.json @@ -82,27 +82,22 @@ }, { "path": "openhcs/interop/cellprofiler/module_declarations.py", - "sha256": "b0195b66126c31f71dc0d4bb281a128e0164bb7dfb3b314ee8fe744ebc55d626", + "sha256": "6ff4f4bc3eaf6b4c9fc42752493c2b9ba2e11c8994d7edaea81f586ac57c3b89", "role": "Defines CellProfilerModule leaves and require_module as owners of module identity, lowering, headless-import validation and artifact semantics." }, { "path": "openhcs/interop/cellprofiler/module_artifact_declarations.py", - "sha256": "ef233be43002d0f16b53c78c202b9203b6261847708d311089671a6b037db894", + "sha256": "5442b56c7dd7c913d38d18c5a76660ad6006e147ffc8f790712fb79b301ff035", "role": "Defines the nominal interactive-module family that rejects headless import." }, { "path": "openhcs/core/callable_contract.py", - "sha256": "a8e73c5df804d78baf72710e210cb2ae63156657a074071e11fd868f4b19d3b4", + "sha256": "6089f63e72e1a47fc5f6bc7b443028ac8ec40d934898589fa03b5592c2273ab3", "role": "Declares CallableContract, the declaration-owned callable metadata at the nominal boundary." }, - { - "path": "openhcs/core/artifacts.py", - "sha256": "65eb0ec5c903366243ca8bd5ea07412e8d34bdb81e6015fa76b3c4c59838733f", - "role": "Declares the ArtifactSpecRelation family for cross-artifact semantics." - }, { "path": "openhcs/core/dataset_sources/source.py", - "sha256": "350bf5691a03522e9f54b1d8ab712c6beba1683aac4fa5159ea4e80c6041530d", + "sha256": "b3758a878fdf8ec28b6d9818ebfd96c5c470895575f7a79500c85f1490679281", "role": "Defines the DatasetSource registry and its layout detection and workspace projection boundary." }, { @@ -112,7 +107,7 @@ }, { "path": "openhcs/core/runtime_plane_projection.py", - "sha256": "3c88ed042a7f1c65c9495a1c62777e008f1e59c4742ec0563b2849d77ca87cdd", + "sha256": "cf644451b6649dcec398fecbf961a4a29ea74f0bb2e0bdab052ec1be8a427f95", "role": "Defines RuntimePlaneAxis and the RuntimePlaneAxisStrategy family used as the strategy example." }, { @@ -122,23 +117,29 @@ }, { "path": "tests/unit/test_cellprofiler_static_deletion_gates.py", - "sha256": "785ea0bd5da907137672735ca631007ee1273393605f3d5e263ac929fbe36c5a", + "sha256": "54ad0f53d664d72af92af4836948d56ab5ec1a61eadbc3337756eea4ab88af7e", "role": "Guards deletion of superseded CellProfiler mirrors and compatibility surfaces." }, { "path": "tests/unit/test_cellprofiler_pipeline_import.py", - "sha256": "0ce4efd85d819c7e9f7fa84ec6f1ac7ec4bda024c9a8d5c7f264fda2730264f0", + "sha256": "67576f37b7cbe0595acea633265f8e14677ca3c1b8204630413330273dc4e58a", "role": "Verifies that importing an enabled interactive-only module fails explicitly." }, { "path": "openhcs/core/config.py", - "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", + "sha256": "9625ca054fac6c1e460216cfe0e85621f5306592a004d1bdc28a6b1ce2099366", "role": "Declares SourceBindingsConfig, the source-binding owner the page contrasts with DatasetSource (moved here from source_bindings.py by K4)." + }, + { + "path": "openhcs/core/artifacts/relations.py", + "sha256": "b1ae8be613fbabc372bd6692b41c5f925d4f968291f0a715bd1473d85cbed000", + "role": "Declares the ArtifactSpecRelation family for cross-artifact semantics." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/abstraction_lattices.rst in full and compared it against the changed authorities: openhcs/interop/cellprofiler/module_declarations.py, openhcs/interop/cellprofiler/module_artifact_declarations.py, openhcs/core/callable_contract.py, openhcs/core/artifacts.py (K2 split it into the openhcs/core/artifacts package), openhcs/core/dataset_sources/source.py, openhcs/core/runtime_plane_projection.py, tests/unit/test_cellprofiler_static_deletion_gates.py, tests/unit/test_cellprofiler_pipeline_import.py, openhcs/core/config.py; CellProfilerModule.require_module, ArtifactSpecRelation, SourceBindingsConfig, RuntimePlaneAxis and RuntimePlaneAxisStrategy, and the three metaclass-registry strategy mixins with for_enum_member all still exist as the page describes, so no claim changed. Replaced the deleted artifacts.py authority with openhcs/core/artifacts/relations.py.", "On 2026-10-11 (B6), re-read docs/source/architecture/abstraction_lattices.rst in full and compared it against the changed authorities: module_declarations.py, module_artifact_declarations.py and the pipeline-import test (K4 import moves, K5 flattened CallableContract metadata, PairByOrder), callable_contract.py (K5), dataset_sources/source.py and source_bindings_source.py (SourceBindingsConfig and source_bindings_defaults_to_base now imported from openhcs.core.config). The page names classes, not modules, for these owners; CellProfilerModule.require_module, CallableContract, ArtifactSpecRelation, DatasetSource, SourceBindingsConfig, SourceBindingsSource, the three metaclass-registry strategy mixins, RuntimePlaneAxisStrategy, AgentCapabilityDeclaration and CapabilityTransport all still exist, so no claim changed. Added openhcs/core/config.py as the current SourceBindingsConfig owner.", "On 2026-10-11 (B6 final), compared docs/source/architecture/abstraction_lattices.rst against B6's later changes to openhcs/interop/cellprofiler/module_declarations.py, openhcs/interop/cellprofiler/module_artifact_declarations.py, openhcs/core/callable_contract.py, openhcs/core/artifacts.py, openhcs/core/dataset_sources/source.py, openhcs/core/dataset_sources/source_bindings_source.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/abstraction_lattices.rst against the later B6 changes to openhcs/interop/cellprofiler/module_declarations.py, openhcs/interop/cellprofiler/module_artifact_declarations.py, openhcs/core/callable_contract.py, openhcs/core/artifacts.py, openhcs/core/dataset_sources/source.py, openhcs/core/dataset_sources/source_bindings_source.py, openhcs/agent/capabilities.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -204,18 +205,19 @@ }, { "path": "tests/unit/test_function_artifact_materialization.py", - "sha256": "6cb3fcbad366d303be1f0846c4497ae4b0bd4a95082aedac3801041907c5dc71", + "sha256": "69b548e9eb1c60d2224b02b6f53932f580e0e12c6b808f9a69998541d3a28adf", "role": "Proves the hook consumes saved CSV content, uses the compiled backend destination, skips a disabled config, and propagates failures." }, { "path": "tests/unit/test_routed_channel_selection.py", - "sha256": "3e4df60b48b699f18e0de29920be778bd03dc98f767fca8bb64a42d6536c4f41", + "sha256": "68f646ca37428ab983f02517409f8a811fa724d6777eb1ae975371b23d73f234", "role": "Executes a native plate with separate analysis destinations and verifies per-directory and merged summaries without writing into the input plate." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/analysis_consolidation_system.rst in full and compared it against the changed authorities: tests/unit/test_function_artifact_materialization.py, tests/unit/test_routed_channel_selection.py; the tests moved to the artifacts package imports, the MainFlowTarget output target and the family spellings of observation mode, filename identity and input source, and still prove the saved-CSV, compiled-destination and separate-destination behaviour the page states, so no claim changed.", "On 2026-10-11 (B6), re-read docs/source/architecture/analysis_consolidation_system.rst in full and compared it against the changed authorities: tests/unit/test_function_artifact_materialization.py (the streaming-filter test now uses the IncludePartitions class instead of PartitionFilterMode.INCLUDE, from K6/L-surface family changes); the page names no filter mode, and AnalysisConsolidationHook, PostExecuteHook, AnalysisTableSource, AnalysisSummaryWriter and DataSink.contextual_save_kwargs still exist at the listed owners, so no claim changed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/analysis_consolidation_system.rst against B6's later changes to openhcs/domains/microscopy/analysis_consolidation.py, openhcs/core/post_execute.py, tests/unit/test_function_artifact_materialization.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/analysis_consolidation_system.rst against the later B6 changes to openhcs/domains/microscopy/analysis_consolidation.py, openhcs/core/post_execute.py, tests/unit/test_function_artifact_materialization.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -237,7 +239,7 @@ }, { "path": "docs/source/architecture/artifact_contract_system.rst", - "source_sha256": "270407772ecb71fad671363045561f7f736a535fc0b500410005687a04497b72", + "source_sha256": "576ad13f18601dd0dfc61c9ab24131eb7193caeee11138c2e63c42949025e43e", "audience": [ "OpenHCS compiler maintainers", "callable and artifact extension authors" @@ -245,34 +247,29 @@ "user_need": "Understand how artifact declarations become exact compiled input, output, relation, and materialisation plans.", "diataxis": "explanation", "authority": [ - { - "path": "openhcs/core/artifacts.py", - "sha256": "65eb0ec5c903366243ca8bd5ea07412e8d34bdb81e6015fa76b3c4c59838733f", - "role": "Defines the ArtifactType families, ArtifactSpec/ArtifactSpecRef, ArtifactSpecRelation leaves and materialisation declarations." - }, { "path": "openhcs/core/callable_contract.py", - "sha256": "a8e73c5df804d78baf72710e210cb2ae63156657a074071e11fd868f4b19d3b4", + "sha256": "6089f63e72e1a47fc5f6bc7b443028ac8ec40d934898589fa03b5592c2273ab3", "role": "Defines CallableContract and its ordered artifact input and output collections." }, { "path": "openhcs/core/function_patterns.py", - "sha256": "cc8c35f37ea0a3bd939905fe317652a34b3129ee6f47f6f9023b8c459e62df70", + "sha256": "c8004ae1e3f241c4ac6ce0a61c69e3c0d1ecfb5021e1bef5dd0fb1fd1251ca50", "role": "Defines InvocationArtifactInputEdgePlan, its main_flow_projection and storage plan, and MainFlowInputProjection." }, { "path": "openhcs/core/pipeline/function_contracts.py", - "sha256": "ffc21dd3021a2316cfbd100c2d655867ce4bec619f2db7288e82aeb11c1930e8", + "sha256": "67a654102b1d60c0a0f90e64ccaf3a8c177035cd11a0346734f872b329783d74", "role": "Defines the special_inputs, artifact_inputs and artifact_outputs decorators." }, { "path": "openhcs/core/pipeline/path_planner.py", - "sha256": "6ff14c63768ed5cbd77e84236b3980d62f616465e9ecfb7cdf27c63d8070a405", + "sha256": "cf31311ffa462d0cbd8eb8cf426d75d5ee2f5668de7b7291ca6787b5d5be118a", "role": "Implements artifact satisfaction and exact compiled path planning." }, { "path": "tests/unit/test_implicit_main_flow_compilation.py", - "sha256": "9f8f3c44f096ce0318612df7d51affbb21a7bbc837c9846d599dbe20c0a8998b", + "sha256": "5fc152553e9a13e023d21a0e297e9b2b9015755cb88eb391423b033f5986eb35", "role": "Compiles grouped native output into a CellProfiler Threshold consumer and asserts the main-flow edge, group scope and execution scope." }, { @@ -282,13 +279,41 @@ }, { "path": "tests/unit/test_artifact_input_projection_plan.py", - "sha256": "45e9b6d379c6b2f95bfe9bc5821de66f157d4f828b00ddb74ea3c65a7d64bbd7", + "sha256": "1c4453a0c636f5a5ba9e9c1c240279170d34a19e29a64e9ba64fd37ae9e752b9", "role": "Verifies occurrence-specific artifact input projection plans." + }, + { + "path": "openhcs/core/artifacts/kinds.py", + "sha256": "216b913efad9ec76707e9f61be445fb6fa1c6601c36ef53e296acd4928604515", + "role": "Defines the registered ArtifactType families the Declarations section lists." + }, + { + "path": "openhcs/core/artifacts/specs.py", + "sha256": "c4c50309a195bf85e1c0641413544d9581fc0ce1197043130126f7a4c70d8c4e", + "role": "Defines ArtifactSpec, ArtifactSpecCollection and MainFlowStackOutputSpec." + }, + { + "path": "openhcs/core/artifacts/plans.py", + "sha256": "0d6a10a78f7177273deca4ad07b9bc729a2eb99254164f2606922674b7286b22", + "role": "Defines ArtifactSpecRef and the ArtifactInputPlan and ArtifactOutputPlan compiled plans with their relations and materialisation fields." + }, + { + "path": "openhcs/core/artifacts/relations.py", + "sha256": "b1ae8be613fbabc372bd6692b41c5f925d4f968291f0a715bd1473d85cbed000", + "role": "Defines the ArtifactSpecRelation leaves, including the object measurement and member subject relations, and ObjectArtifactSubjectBinding." + }, + { + "path": "openhcs/constants/input_source.py", + "sha256": "4a7929e66d11d5ad81c40f080ba8cd23d466797bfa780d59eae0e1aea3ffd9a4", + "role": "Declares ReadPipelineStart, the input source a step selects to read the pipeline-start flow." } ], - "findings": [], - "disposition": "keep", + "findings": [ + "The forward-planning section named the removed enum member PIPELINE_START; E1 made InputSource a class family, so the page now names ReadPipelineStart." + ], + "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/artifact_contract_system.rst in full and compared it against the changed authorities: openhcs/core/artifacts.py (K2 split it into the openhcs/core/artifacts package), openhcs/core/callable_contract.py, openhcs/core/function_patterns.py, openhcs/core/pipeline/function_contracts.py, openhcs/core/pipeline/path_planner.py, tests/unit/test_implicit_main_flow_compilation.py, tests/unit/test_artifact_input_projection_plan.py; grepped every class, constant and field the page names (the ArtifactType kinds, MainFlowInputProjection members, PIPELINE_INPUT_ARTIFACT, unnamed_main_flow_artifact_name, MainFlowStackOutputSpec, the invocation edge and projection-key classes, SelectedPlaneImageOutput, SourceProjectedImageOutput, ObjectArtifactSubjectBinding) and all still exist; corrected the PIPELINE_START spelling to ReadPipelineStart and replaced the deleted artifacts.py authority with the package modules that own the named declarations.", "On 2026-10-11 (B6), re-read docs/source/architecture/artifact_contract_system.rst in full and compared it against the changed authorities: callable_contract.py and test_callable_contract.py (K5: CallableMetadata became CallableDeclaration and the metadata wrapper was flattened into direct ArtifactSpecCollection fields), function_patterns.py (K5 pattern consolidation: AuthoredFunctionPattern, kwarg merge removed), function_contracts.py (special_inputs now stores artifact_input_parameter_names) and path_planner.py (metadata injection over the ArtifactGraph, source_bindings.compiled_plan). The page's claims still hold: special_inputs only names ABI slots, CallableContract carries ordered ArtifactSpecCollection inputs/outputs, InvocationArtifactInputEdgePlan keeps spec, storage_plan and main_flow_projection with the two-member MainFlowInputProjection enum, and every other named class (ArtifactPlanKeySelector, ArtifactDeclarationStepContext, MainFlowStackOutputSpec, SelectedPlaneImageOutput, SourceProjectedImageOutput, the object subject relations and binding, SourceImageProvenance, StepMaterializationConfig) still exists, so no claim changed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/artifact_contract_system.rst against B6's later changes to openhcs/core/pipeline/function_contracts.py: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/artifact_contract_system.rst against B6's later changes to openhcs/core/artifacts.py, openhcs/core/callable_contract.py, openhcs/core/function_patterns.py, openhcs/core/pipeline/function_contracts.py, openhcs/core/pipeline/path_planner.py, tests/unit/test_implicit_main_flow_compilation.py, tests/unit/test_callable_contract.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", @@ -329,7 +354,7 @@ }, { "path": "openhcs/authoring/session/compile_batch.py", - "sha256": "0cbe5a6b86a0fb5e9b92c4fc45f858c979031ac93594b317b9761c28f5aaf9ab", + "sha256": "3e7397caa5ea1b38c1abc47d363e37fc01b0b601ab1e6b37ed00226f7aa51f42", "role": "Runs compile-only batches and compile-before-execution through BatchSubmitWaitEngine." }, { @@ -339,12 +364,12 @@ }, { "path": "openhcs/authoring/session/session.py", - "sha256": "0c4e0f0979e1793862859e47eefeb6dd9d4b51c99fba20c2ebc6adf11619c3b5", + "sha256": "ced615b580e4625e09873449b29ac67cdea87c8b8065c205f32e6d5921cf31fc", "role": "Implements run_datasets (reserve, reset, connect, compile all, submit each with compile_artifact_id) and the endpoint status binding." }, { "path": "openhcs/authoring/session/submission.py", - "sha256": "710ac218d1db01f231b67bf0fe229dc03fb428bc6655b23671f3a026be91c673", + "sha256": "e88dba60a818ce44d0ecf744b918f8a82421ccd0d9a6a5aebce7c15bb331f1dc", "role": "Submits each execution with its compile_artifact_id and auxiliary observation-export parameters." }, { @@ -361,6 +386,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/batch_workflow_service.rst in full and compared it against the changed authorities: openhcs/authoring/session/compile_batch.py, openhcs/authoring/session/session.py, openhcs/authoring/session/submission.py; E1 replaced the OrchestratorState and terminal-status enum members with classes and the session now takes the observation export scope as a class, but the compile-only and run flows, compile_artifact_id submission and lifecycle the page describes are unchanged and it names no state member, so no claim changed.", "On 2026-10-11 (B6), re-read docs/source/architecture/batch_workflow_service.rst in full and compared it against the changed authorities: openhcs/authoring/session/session.py (K6 debug command and replay mode became classes: RunDebugCommand, StopDebugCommand, PersistentPausedWorkerReplay); the page does not describe debug sessions, and CompileDatasets, DatasetPipelineRequest, CompiledDataset, BatchSubmitWaitEngine, compile_artifact_id submission, ExecutionProgress, Session.close and the endpoint_status binding all still exist, so no claim changed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/batch_workflow_service.rst against B6's later changes to openhcs/authoring/session/session.py: the writability probe makes one exclusive create attempt, the installed demo reports ViewerType.value, the unit session pre-creates its QApplication, and the GUI journey waits for each batch to end. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/batch_workflow_service.rst against B6's later changes to openhcs/authoring/session/session.py: the session decides in-place initialization by an actual file-creation attempt that leaves nothing behind, its debug-session methods moved into an inherited mixin, the measured CLI treats Windows Ctrl+Break like Ctrl+C, and the Official30 viewer test reads typed payload summaries. No claim this page makes changes.", @@ -391,7 +417,7 @@ }, { "path": "docs/source/architecture/cellprofiler_interop.rst", - "source_sha256": "462d00bb6421b6de9772e6a8953288b45ee096aab92f630fcbc6a592c11ac8d2", + "source_sha256": "6674c881ea67c190d8d5d7ea3fc5aa280273dfbfdb2701d7a9332abe629dd367", "audience": [ "CellProfiler users", "OpenHCS interoperability and compiler maintainers" @@ -401,37 +427,37 @@ "authority": [ { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "b96f33b866fe6c9f8118ca70fcb7dff8e6a02cb29d79df67230a5e8b97df8002", + "sha256": "5aa8895c9f533603b7ef972202141e2439473f0604b97d66f5a027b8b3cd1c83", "role": "Implements .cppipe parsing, declaration-owned headless validation, and lowering into PipelineConfig and FunctionStep declarations." }, { "path": "openhcs/interop/cellprofiler/module_declarations.py", - "sha256": "b0195b66126c31f71dc0d4bb281a128e0164bb7dfb3b314ee8fe744ebc55d626", + "sha256": "6ff4f4bc3eaf6b4c9fc42752493c2b9ba2e11c8994d7edaea81f586ac57c3b89", "role": "Owns module-specific callable, setting, headless-import, scope, artifact, and function-catalogue declarations." }, { "path": "openhcs/interop/cellprofiler/module_artifact_declarations.py", - "sha256": "ef233be43002d0f16b53c78c202b9203b6261847708d311089671a6b037db894", + "sha256": "5442b56c7dd7c913d38d18c5a76660ad6006e147ffc8f790712fb79b301ff035", "role": "Defines the nominal interactive-module family that rejects headless import." }, { "path": "openhcs/interop/cellprofiler/compile_time_contracts.py", - "sha256": "3b1879d23635b1e9f7998522e812c738417de85fe53f0201501981a1780a21c3", + "sha256": "2528a13067bd2d24f8592c430e5f83f84ddcab560c07ba0299b47f734c5ecfb5", "role": "Defines CellProfilerInvocationContractProviderFactory, which walks the ResolvedPipelineDefinition steps with a forward artifact context." }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "0f9cc4819b53d55b7e9446429b8c8ac1ff1a59f4902020ea1658495ce01d464c", + "sha256": "4946c221e34ec19957b5af2b1a54359d454ee840fb1f5e1fe2b3b306ac8e5426", "role": "Declares ResolvedPipelineDefinition, the resolved step sequence the provider walks." }, { "path": "openhcs/constants/input_source.py", - "sha256": "fccc1854051fbe47371df2cfb737080bdca14d932825ef7ddc9a0e1ed1cf0dc9", - "role": "Declares the InputSource members PREVIOUS_STEP and PIPELINE_START." + "sha256": "4a7929e66d11d5ad81c40f080ba8cd23d466797bfa780d59eae0e1aea3ffd9a4", + "role": "Declares the InputSource family and its two members, ReadPreviousStep and ReadPipelineStart." }, { "path": "tests/unit/test_cellprofiler_artifact_declarations.py", - "sha256": "ba791ec36bbc4159c09a4ea8d950e05785a539b5a30a85394098bba26aa11f5a", + "sha256": "9b45af5b1638781add722578b9b76efbf8ae4bea2bae9ad7425215ed22c3a228", "role": "Verifies typed image, object, measurement, relationship, table, grid, graph, and special artifact declarations." }, { @@ -441,12 +467,12 @@ }, { "path": "tests/unit/test_cellprofiler_pipeline_import.py", - "sha256": "0ce4efd85d819c7e9f7fa84ec6f1ac7ec4bda024c9a8d5c7f264fda2730264f0", + "sha256": "67576f37b7cbe0595acea633265f8e14677ca3c1b8204630413330273dc4e58a", "role": "Verifies that direct import rejects an enabled interactive-only module explicitly." }, { "path": "benchmark/cellprofiler_reference_exports.py", - "sha256": "58cdcb7572266d562314ccae5bf9d98b5f3a4b19944e8a068f89b617219952f3", + "sha256": "3f175ad2173cffb2181246b6d5cc498c64dfe90b5f38f87265c1effafeb60e6f", "role": "Derives digest-bound benchmark-only exporter pipelines from terminal importer contracts and owns numeric-image and categorical-label comparison semantics." }, { @@ -455,9 +481,12 @@ "role": "Proves exact inventory selection, generated-pipeline digest validation, exporter derivation, and the declared image and label comparison contracts." } ], - "findings": [], - "disposition": "keep", + "findings": [ + "The page called InputSource an enum; E1 made it a class family with the members ReadPreviousStep and ReadPipelineStart, so the page now says family." + ], + "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/cellprofiler_interop.rst in full and compared it against the changed authorities: openhcs/interop/cellprofiler/pipeline_import.py, module_declarations.py, module_artifact_declarations.py, compile_time_contracts.py, openhcs/core/pipeline/compilation_session.py, openhcs/constants/input_source.py, tests/unit/test_cellprofiler_artifact_declarations.py, tests/unit/test_cellprofiler_pipeline_import.py, benchmark/cellprofiler_reference_exports.py; the importer still chooses previous-step or pipeline-start main flow from source and producer evidence (now by returning ReadPreviousStep or ReadPipelineStart), and the artifact families the page lists match the kinds in openhcs/core/artifacts/kinds.py; corrected the one sentence that called InputSource an enum.", "On 2026-10-11 (B6), re-read docs/source/architecture/cellprofiler_interop.rst in full and compared it against the changed authorities: pipeline_import.py, module_declarations.py, module_artifact_declarations.py, compile_time_contracts.py (K4 moved SourceBindingsConfig/StepSourceBindingsConfig into openhcs.core.config; K5 flattened CallableContract.metadata into direct artifact_inputs/artifact_outputs/runtime_adapter fields), compilation_session.py, and the two CellProfiler unit tests (SourceBindingMatchMethod.ORDER became PairByOrder). The page names SourceBindingsConfig, CallableContract and the module pipeline without import paths or the metadata wrapper; every class and function it names (CPPipeParser, SettingsBinder, CellProfilerModuleArtifactContracts, SettingToKeywordBinding, CellProfilerInvocationContractProviderFactory, ArtifactDeclarationStepContext, FunctionStepExecutionScope.PLATE, FileBundleOptions, CPASQLiteRenderer, CellProfilerReferenceExportPlan, numpy_avx512_skx_svml_symbol_available, the two-member InputSource enum) still exists, so no claim changed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/cellprofiler_interop.rst against B6's later changes to tests/integration/test_cellprofiler_official30_zmq.py: the session decides in-place initialization by an actual file-creation attempt that leaves nothing behind, its debug-session methods moved into an inherited mixin, the measured CLI treats Windows Ctrl+Break like Ctrl+C, and the Official30 viewer test reads typed payload summaries. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/cellprofiler_interop.rst against B6's later changes to openhcs/interop/cellprofiler/pipeline_import.py, openhcs/interop/cellprofiler/module_declarations.py, openhcs/interop/cellprofiler/module_artifact_declarations.py, openhcs/core/pipeline/compilation_session.py, tests/unit/test_cellprofiler_artifact_declarations.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", @@ -477,7 +506,7 @@ }, { "path": "docs/source/architecture/code_ui_interconversion.rst", - "source_sha256": "e1a2607666a806193e4abe240ed21cb189326d5e1d54da761511e2f70fd895f0", + "source_sha256": "d2afa247a25b0b07128b0527d04f65eb83d86e135c748da798c905dcc73b0ebd", "audience": [ "OpenHCS UI and agent-surface maintainers", "advanced attached-agent users" @@ -492,12 +521,12 @@ }, { "path": "openhcs/pyqt_gui/services/ui_bridge_object_state.py", - "sha256": "e64b444a74fdde1d79f82c1361c9b67d8e8916b9bebced092762ce51eabac52b", + "sha256": "f615718186e4aaebdc12eec6bec6a97d47953322075c5b067974e5a16ecc8e80", "role": "Projects live ObjectState scopes, fields, revisions, document capabilities, and mutation receipts through the UI bridge." }, { "path": "openhcs/agent/dto/ui_bridge.py", - "sha256": "918b30f34e7d096b19ce04017d9a991d0a10e4088c310797e497fb2444fc37c0", + "sha256": "9b754a28e367e910bee35b6d900f54daf7121e224cac32e43908d147e7faea00", "role": "Carries generic endpoint application and operation-lifecycle declarations through UI bridge descriptors and request/response envelopes." }, { @@ -507,7 +536,7 @@ }, { "path": "openhcs/pyqt_gui/services/ui_bridge_server.py", - "sha256": "4b0664fa62d24557294324b631ee3e39052c84e578d40ec408197bce5adcedac", + "sha256": "f93a3570a834eca8596139676629dda48c3f22991fd8cb58cd03427af8b24a38", "role": "Advertises the shared application identity and rejects incompatible requests before dispatching them into live UI state." }, { @@ -518,11 +547,11 @@ { "path": "external/python-introspect/src/python_introspect/annotation_types.py", "sha256": "004a3ce2e214a184aaa3230ceaa89ccaa5f188c1c3d885aacad5222726c34b7a", - "role": "Owns declaration-derived enum discovery, accepted input strings, and member coercion across transparent annotation wrappers." + "role": "Owns enum_input_values: the accepted input strings for an enum or a declared choice family, and enum member coercion across transparent annotation wrappers." }, { "path": "openhcs/pyqt_gui/services/ui_agent_bridge.py", - "sha256": "ce650c52475764657bbc0221109879594f9cf1482de8245d6c141f00474d5d23", + "sha256": "55a21b2746e3ba27eff1a3315604725d340dbddce6645c8457f707484a2f1f0c", "role": "Projects ObjectState branch and snapshot restoration through the attached-agent surface." }, { @@ -532,7 +561,7 @@ }, { "path": "tests/unit/pyqt_gui/test_ui_agent_bridge.py", - "sha256": "67cdad5133dc83ccb005b58cc56535ae89e77fc23d3ebd76e616c0da5f567b44", + "sha256": "422be9298e5120c898bd7110884ca770f1f033f07fd69be4ac0caa978d3014f4", "role": "Verifies read-only rejection, semantic no-op application, and branch restoration through the UI bridge." }, { @@ -542,7 +571,7 @@ }, { "path": "tests/unit/pyqt_gui/test_pipeline_code_document_transactionality.py", - "sha256": "0bb9b43d867a4e26f13b51531a2b7a847c6c3685841f8e002e209200a37366b6", + "sha256": "6604f4cc9ff8d1f3824f0479c4947c7d264aff59c7b3e139d670b659b63b39ec", "role": "Verifies revision checks, validation, snapshots, rollback, and transactional application." }, { @@ -597,10 +626,11 @@ } ], "findings": [ - "The round-trip section implied OpenHCS owns the semantic equality rule and lazy-field omission for clean code views; after L6, pycodify_formatters.py passes ObjectState's semantic_values_equal and the lazy dataclass formatting and field_needs_source decision live in objectstate.codegen. Reworded and added objectstate/codegen.py and objectstate/inspection.py (field projection now inherits FieldInspection) as authorities." + "The field-projection list said only enum inputs are exposed; after E1 many fields hold a registered class family, whose choice labels enum_input_values exposes the same way, so the page now names both." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/code_ui_interconversion.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/services/ui_bridge_object_state.py, openhcs/agent/dto/ui_bridge.py, openhcs/pyqt_gui/services/ui_bridge_server.py, openhcs/pyqt_gui/services/ui_agent_bridge.py, tests/unit/pyqt_gui/test_ui_agent_bridge.py, tests/unit/pyqt_gui/test_pipeline_code_document_transactionality.py; E1 made the bridge operation status, field filter and action status class families with unchanged wire names, and the Plate Manager document assignments a typed record; the code-document, revision, field-mutation and identity claims are unchanged, and the accepted-inputs bullet now covers registered-family choices as well as enums.", "On 2026-10-11 (B6), re-read docs/source/architecture/code_ui_interconversion.rst in full and compared it against the changed authorities: openhcs/core/pipeline_document.py (renders through pycodify.generate_python_source), ui_bridge_object_state.py (field projection now extends objectstate.inspection.FieldInspection), zmqruntime messages.py and pyqt-reactive window_code_document.py / function_pattern_code_document.py (U3 advance; config code-document driver and external-editor preference moved into pyqt-reactive), function_step_code_document.py and step_parameter_editor.py (patch_lazy_constructors from ObjectState, CodeEditorFormUpdater from pyqt-reactive), pycodify_formatters.py and its test (L6: lazy-dataclass, enum, path and import formatters deleted in favour of objectstate.codegen and pycodify imported_name), and test_ui_agent_bridge.py (K6 debug/progress classes, DockPane). Corrected the equality/omission ownership sentence and added the ObjectState codegen and inspection owners; document, revision, field-mutation, bridge identity and parameter-order claims still hold.", "On 2026-10-11 (B6 final), compared docs/source/architecture/code_ui_interconversion.rst against B6's later changes to openhcs/pyqt_gui/services/ui_bridge_object_state.py, openhcs/agent/dto/ui_bridge.py, openhcs/pyqt_gui/services/ui_agent_bridge.py, tests/unit/pyqt_gui/test_ui_agent_bridge.py, tests/unit/pyqt_gui/test_pipeline_code_document_transactionality.py, tests/unit/test_pycodify_formatters.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/code_ui_interconversion.rst against the later B6 changes to openhcs/pyqt_gui/services/ui_bridge_object_state.py, openhcs/agent/dto/ui_bridge.py, openhcs/pyqt_gui/services/ui_agent_bridge.py, tests/unit/pyqt_gui/test_ui_agent_bridge.py, tests/unit/pyqt_gui/test_pipeline_code_document_transactionality.py, tests/unit/test_pycodify_formatters.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -635,12 +665,12 @@ "authority": [ { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "4a33908fb42874e6d57eeea3f5305793ddbda41d13356f0a78501af38dcdfeb4", + "sha256": "40b07b7f40aca39b19b95a8f61c1b0c8270f0ab0a29e8c36caa13f04d8fa539f", "role": "Lowers declarations into typed step plans, including declaration-derived framework-device assignments." }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "0f9cc4819b53d55b7e9446429b8c8ac1ff1a59f4902020ea1658495ce01d464c", + "sha256": "4946c221e34ec19957b5af2b1a54359d454ee840fb1f5e1fe2b3b306ac8e5426", "role": "Defines the per-context CompilationSession and ResolvedPipelineDefinition." }, { @@ -650,10 +680,11 @@ } ], "findings": [ - "The old compilation-service URL owns no compiler facts and stays a concise transition to pipeline_compilation_system; no claim on it became false." + "The page is a redirect stub; its link target still exists and owns the content (checked 2026-10-11, B6)." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/compilation_service.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/compilation_session.py; the stub states no compiler fact beyond its link to pipeline_compilation_system, which still exists.", "On 2026-10-11 (B6), re-read docs/source/architecture/compilation_service.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compiler.py (ScopeKey-built scope ids, CompileProgressPhase/RunningProgressStatus classes, CompiledSourceBindingPlan() default) and openhcs/core/pipeline/compilation_session.py (strip_disabled_functions removed, invocation validation moved onto the graph); CompilationSession, ResolvedPipelineDefinition and CompiledExecutionBundle still exist and the redirect target exists, so the transition remains accurate.", "On 2026-10-11 (B6 final), compared docs/source/architecture/compilation_service.rst against B6's later changes to openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/compilation_session.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/compilation_service.rst against the later B6 changes to openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/compilation_session.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -677,17 +708,17 @@ "authority": [ { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "4a33908fb42874e6d57eeea3f5305793ddbda41d13356f0a78501af38dcdfeb4", + "sha256": "40b07b7f40aca39b19b95a8f61c1b0c8270f0ab0a29e8c36caa13f04d8fa539f", "role": "Implements the current compiler stages, including declaration-derived framework-device assignment." }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "0f9cc4819b53d55b7e9446429b8c8ac1ff1a59f4902020ea1658495ce01d464c", + "sha256": "4946c221e34ec19957b5af2b1a54359d454ee840fb1f5e1fe2b3b306ac8e5426", "role": "Defines ResolvedPipelineDefinition, which replaced resolved step snapshots, and CompilationSession." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "358d3eaf5952170f9e9e2dc9056fb08707e8fcf74682063c074b367a83c2648f", + "sha256": "6525348d097bda8d0265bc7c3f312dd8839ca9f65dd05f797fc82ba873e04547", "role": "Defines typed step-plan fields, exact framework footprints, and framework-local device bindings." }, { @@ -697,10 +728,11 @@ } ], "findings": [ - "The old detailed-compiler URL is a transition to pipeline_compilation_system; every class it names (ResolvedPipelineDefinition, CompilationSession, CompiledExecutionBundle) still exists, so it stays a redirect with no false claim." + "The page is a redirect stub; its link target still exists and owns the content (checked 2026-10-11, B6)." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/compilation_system_detailed.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/compiled_step_plan.py; ResolvedPipelineDefinition, CompilationSession and CompiledExecutionBundle still exist and the link target pipeline_compilation_system still exists.", "On 2026-10-11 (B6), re-read docs/source/architecture/compilation_system_detailed.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compiler.py (ScopeKey scope ids, progress phase/status classes), openhcs/core/pipeline/compilation_session.py (disabled-function stripping removed, graph-level invocation validation) and openhcs/core/compiled_step_plan.py (source binding and universe plans default-construct instead of .empty()); the named classes and the redirect target still exist, so the transition remains accurate.", "On 2026-10-11 (B6 final), compared docs/source/architecture/compilation_system_detailed.rst against B6's later changes to openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/compiled_step_plan.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/compilation_system_detailed.rst against the later B6 changes to openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/compiled_step_plan.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -729,7 +761,7 @@ }, { "path": "openhcs/core/config.py", - "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", + "sha256": "9625ca054fac6c1e460216cfe0e85621f5306592a004d1bdc28a6b1ce2099366", "role": "Declares the kernel OpenHCS configuration dataclasses that consume ObjectState mechanics." }, { @@ -739,10 +771,11 @@ } ], "findings": [ - "The page is a transition stub for lazy configuration now owned by ObjectState; its targets (ObjectState docs/architecture.rst, component_system_integration, external_foundations) still exist, so it stays a redirect." + "The page is a redirect stub; its link target still exists and owns the content (checked 2026-10-11, B6)." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/component_configuration_framework.rst in full and compared it against the changed authorities: openhcs/core/config.py; the stub states no configuration field, and its targets component_system_integration and external_foundations still exist.", "On 2026-10-11 (B6), re-read docs/source/architecture/component_configuration_framework.rst in full and compared it against the changed authorities: external/ObjectState/src/objectstate/lazy_factory.py (ObjectState 2.x advance) and openhcs/core/config.py (K4 source-binding classes, K6/L-surface family moves); the page states no configuration fact beyond ownership, ObjectState still owns lazy dataclass creation in lazy_factory.py and ships docs/architecture.rst, and both OpenHCS link targets exist, so the redirect remains accurate.", "On 2026-10-11 (B6 final), compared docs/source/architecture/component_configuration_framework.rst against B6's later changes to openhcs/core/config.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/component_configuration_framework.rst against the later B6 changes to external/ObjectState/src/objectstate/lazy_factory.py, openhcs/core/config.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -820,7 +853,7 @@ }, { "path": "openhcs/core/config.py", - "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", + "sha256": "9625ca054fac6c1e460216cfe0e85621f5306592a004d1bdc28a6b1ce2099366", "role": "Declares the step-level variable_components and group_by fields and the pipeline-level sequential_components field." }, { @@ -840,13 +873,14 @@ }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "4a33908fb42874e6d57eeea3f5305793ddbda41d13356f0a78501af38dcdfeb4", + "sha256": "40b07b7f40aca39b19b95a8f61c1b0c8270f0ab0a29e8c36caa13f04d8fa539f", "role": "Consumes the axis declarations to construct execution-axis contexts, stacks, grouping, and typed plans." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/component_system_integration.rst in full and compared it against the changed authorities: openhcs/core/config.py, openhcs/core/pipeline/compiler.py; ProcessingConfig still declares variable_components and group_by and the pipeline config still declares sequential_components; the E1 changes to the materialization backend, input source and orchestrator state families touch nothing the page states, so no claim changed.", "On 2026-10-11 (B6), re-read docs/source/architecture/component_system_integration.rst in full and compared it against the changed authorities: openhcs/core/config.py (PartitionFilterMode became an IncludePartitions/ExcludePartitions class family; SourceBindingsConfig/StepSourceBindingsConfig moved in by K4) and openhcs/core/pipeline/compiler.py (ScopeKey scope ids, progress classes); the page names no filter mode or source-binding class, and variable_components, group_by, sequential_components, PartitionAxis, AxisValues, FilenameParseResult and MetadataComponentValueSet are unchanged, so no claim changed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/component_system_integration.rst against B6's later changes to openhcs/core/axes.py, openhcs/core/config.py, openhcs/core/dataset_sources/interfaces.py, openhcs/core/pipeline/compiler.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/component_system_integration.rst against the later B6 changes to openhcs/core/axes.py, openhcs/core/config.py, openhcs/core/dataset_sources/interfaces.py, openhcs/core/pipeline/compiler.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -882,23 +916,24 @@ }, { "path": "openhcs/core/pipeline/funcstep_contract_validator.py", - "sha256": "4a167f620be9ff6e6524101652b6dfa25984b3a389b19da3f5ce78c14d5947d4", + "sha256": "d6144cff25772f57ede73d67635d0ed0fe2ffa1ec7d8b3913efbce2bd3eb2ed6", "role": "Validates allowed group_by roles, required callable axis roles, and declared memory contracts per callable." }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "4a33908fb42874e6d57eeea3f5305793ddbda41d13356f0a78501af38dcdfeb4", + "sha256": "40b07b7f40aca39b19b95a8f61c1b0c8270f0ab0a29e8c36caa13f04d8fa539f", "role": "Rejects pipeline sequential axes that are also a step's variable axes." }, { "path": "tests/unit/test_funcstep_contract_validator.py", - "sha256": "65d9b010d43c85e20fb4ef5c2f8f4c7363496384635ec64d906c7138649d1900", + "sha256": "fff3ca054398acd7794dbb1f2e70e9905933c9c8e438c5792a2627d60aa94eba", "role": "Covers accepted and rejected grouping, required-role, and callable-contract combinations." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/component_validation_system.rst in full and compared it against the changed authorities: openhcs/core/pipeline/funcstep_contract_validator.py, openhcs/core/pipeline/compiler.py, tests/unit/test_funcstep_contract_validator.py; the validator now tests the plate-scoped main-input dependency by its NoMainFlowInput class and the test imports moved to the artifacts package; the grouping, required-role and sequential-axis checks the page lists are unchanged, so no claim changed.", "On 2026-10-11 (B6), re-read docs/source/architecture/component_validation_system.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compiler.py (ScopeKey scope ids, CompileProgressPhase/RunningProgressStatus progress classes, CompiledSourceBindingPlan() default) and tests/unit/test_funcstep_contract_validator.py (CallableDeclaration, admitted_graph); confirmed AxisFamily.named, variable_axes, Ungrouped, GenericValidator.validate_step/validate_dict_pattern_keys, validate_allowed_group_by, validate_required_axis_roles and validate_sequential_components_compatibility still exist and behave as stated; no claim changed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/component_validation_system.rst against B6's later changes to openhcs/core/axes.py, openhcs/core/pipeline/funcstep_contract_validator.py, openhcs/core/pipeline/compiler.py, tests/unit/test_funcstep_contract_validator.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/component_validation_system.rst against the later B6 changes to openhcs/core/axes.py, openhcs/core/pipeline/funcstep_contract_validator.py, openhcs/core/pipeline/compiler.py, tests/unit/test_funcstep_contract_validator.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -931,17 +966,17 @@ }, { "path": "openhcs/core/orchestrator/bundle_execution.py", - "sha256": "bc3a2d0a50c3db3be81b03f6af4a139c3db2f779e969aad608e59be66508ee4b", + "sha256": "25446d6c6a43c22ff7104c6712c143f0276bc4749c2642fffac10db1455ef707", "role": "Runs one compiled bundle, then the parent-owned dataset-scoped steps and post-execute hooks." }, { "path": "openhcs/core/orchestrator/worker_lanes.py", - "sha256": "d915c01a0ae2dd010d6b972e275669fef63bb3aefd100f1b66a1e34653673db0", + "sha256": "4273af5b358c7f4cf01a504bb151fdd0117fcc724cf121913cb9ebb5b7695115", "role": "Defines WorkerLaneExecutionContext, the fork-inherited bundle state, and lane grouping from context-owned axis identity." }, { "path": "openhcs/core/orchestrator/worker_execution.py", - "sha256": "26b9586890e7117b24b6b85660d3f417455d55546401739f91146d18e4d21134", + "sha256": "83ade0271c9b24ca1ba61c708d9e6f60a432b630ec6a8a2fac6d1018458e6be5", "role": "Owns executor construction, the WorkerCommand family served by process workers, result collection, cancellation, and cleanup." }, { @@ -956,7 +991,7 @@ }, { "path": "tests/unit/test_orchestrator_lane_planning.py", - "sha256": "85a73ddf071552fc88d454c9de857d066fd62f45492b74945ee22ec908b85289", + "sha256": "4af8c5e5a6d68a64f862a0bcb98f398a0a898948e6f8b5e462a4a11381f19524", "role": "Verifies that worker-lane grouping remains correct when context mapping keys do not encode axis identity." }, { @@ -970,11 +1005,10 @@ "role": "Declares the ProgressPhase and ProgressStatus class families, including AxisCompletedProgressPhase and SuccessProgressPhase." } ], - "findings": [ - "K6 replaced the ProgressPhase enum members with phase classes; the page still named the AXIS_COMPLETED and SUCCESS enum members, now AxisCompletedProgressPhase and SuccessProgressPhase." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/concurrency_model.rst in full and compared it against the changed authorities: openhcs/core/orchestrator/bundle_execution.py, worker_lanes.py, worker_execution.py and tests/unit/test_orchestrator_lane_planning.py (E1: OrchestratorState members became OrchestratorExecuting/Ready/ExecFailed/Completed classes, RuntimeObservationMode members became MergeAllRuntimeRecords/OmitRuntimeRecords classes, ViewerSettlePhase became settlement classes; K2: artifact imports from openhcs.core.artifacts submodules); the page names none of those members, and its bundle, lane, worker-command, start-method, progress-phase and cancellation claims still hold.", "On 2026-10-11 (B6), re-read docs/source/architecture/concurrency_model.rst in full and compared it against the changed authorities: openhcs/core/orchestrator/bundle_execution.py (progress phase/status classes, lane results reordered to compiled axis order), openhcs/core/orchestrator/worker_execution.py (progress classes) and tests/unit/test_orchestrator_lane_planning.py; renamed the AXIS_COMPLETED and SUCCESS enum references to AxisCompletedProgressPhase and SuccessProgressPhase from openhcs/core/progress/types.py; the lane, worker-command, start-method and cancellation claims still hold.", "On 2026-10-11 (B6 final), compared docs/source/architecture/concurrency_model.rst against B6's later changes to openhcs/core/orchestrator/bundle_execution.py, openhcs/core/orchestrator/worker_execution.py, tests/unit/test_orchestrator_lane_planning.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/concurrency_model.rst against the later B6 changes to openhcs/core/orchestrator/bundle_execution.py, openhcs/core/orchestrator/worker_execution.py, tests/unit/test_orchestrator_lane_planning.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -1010,7 +1044,7 @@ }, { "path": "openhcs/core/config.py", - "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", + "sha256": "9625ca054fac6c1e460216cfe0e85621f5306592a004d1bdc28a6b1ce2099366", "role": "Declares GlobalPipelineConfig and the step and standalone configuration dataclasses OpenHCS composes over ObjectState." } ], @@ -1019,6 +1053,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/configuration_framework.rst in full and compared it against the changed authorities: openhcs/core/config.py (E1: MaterializationBackend and InputSource became registered class families; K2: artifact import path); the stub states no configuration fact, and its targets (ObjectState docs/architecture.rst, system_overview, external_foundations) still exist.", "On 2026-10-11 (B6), re-read docs/source/architecture/configuration_framework.rst in full and compared it against the changed authorities: external/ObjectState/src/objectstate/lazy_factory.py, external/ObjectState/src/objectstate/context_manager.py (current files) and openhcs/core/config.py (PartitionFilterMode became a class family, K4 source-binding declarations imported from openhcs.core.source_bindings); the redirect's only claims (ObjectState owns lazy configuration and resolution, OpenHCS owns its domain declarations such as GlobalPipelineConfig) still hold, and the linked ObjectState architecture page and system_overview exist.", "On 2026-10-11 (B6 final), compared docs/source/architecture/configuration_framework.rst against B6's later changes to openhcs/core/config.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/configuration_framework.rst against the later B6 changes to external/ObjectState/src/objectstate/lazy_factory.py, openhcs/core/config.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -1047,7 +1082,7 @@ }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "0f9cc4819b53d55b7e9446429b8c8ac1ff1a59f4902020ea1658495ce01d464c", + "sha256": "4946c221e34ec19957b5af2b1a54359d454ee840fb1f5e1fe2b3b306ac8e5426", "role": "Consumes an already resolved GlobalPipelineConfig in a scoped compiler session." } ], @@ -1056,6 +1091,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/context_system.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compilation_session.py (K2: ArtifactSpecRef imported from openhcs.core.artifacts.plans); the stub states no compiler fact, and its targets (ObjectState docs/architecture.rst, system_overview, external_foundations) still exist.", "On 2026-10-11 (B6), re-read docs/source/architecture/context_system.rst in full and compared it against the changed authorities: external/ObjectState/src/objectstate/context_manager.py (current file) and openhcs/core/pipeline/compilation_session.py (invocation-declaration validation moved onto ArtifactGraph.validate_invocation_declarations, disabled-function stripping removed from step normalization); ObjectState still owns context registration and resolution and the compiler session still consumes a resolved GlobalPipelineConfig, so the redirect stands unchanged.", "On 2026-10-11 (B6 final), compared docs/source/architecture/context_system.rst against B6's later changes to openhcs/core/pipeline/compilation_session.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/context_system.rst against the later B6 changes to openhcs/core/pipeline/compilation_session.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -1105,7 +1141,7 @@ "authority": [ { "path": "openhcs/processing/custom_functions/manager.py", - "sha256": "0ffa6fc2a18566600dcff33d4b27466d8a5684662fa1c696cb5ea52ea459a1e4", + "sha256": "77612ea5ed53888c36e8ed31f763d8eaf5e0af9e4f32b4beaaa7cff09012b130", "role": "Owns persisted custom-function lifecycle and projects validated declarations through the custom runtime registry." }, { @@ -1115,10 +1151,11 @@ } ], "findings": [ - "The manager.py changes keep the persisted lifecycle the redirect points to; no claim was invalidated, and the architecture URL remains a transition to the user workflow page." + "The page is a redirect stub; its link target still exists and owns the content (checked 2026-10-11, B6)." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/custom_function_registration_system.rst in full and compared it against the changed authorities: openhcs/processing/custom_functions/manager.py (E1: CustomFunctionLifetime.PERSISTED became the PersistedLifetime class); the stub names no lifetime, and its target user_guide/custom_function_management still exists.", "On 2026-10-11 (B6 final), compared docs/source/architecture/custom_function_registration_system.rst against B6's later changes to openhcs/processing/custom_functions/manager.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/custom_function_registration_system.rst against the later B6 changes to openhcs/processing/custom_functions/manager.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/custom_function_registration_system.rst in full and compared it against the changed authority manager.py; confirmed the user_guide/custom_function_management target exists and CustomFunctionManager still owns persisted custom functions through the runtime registry; no page edit was needed.", @@ -1155,7 +1192,7 @@ }, { "path": "openhcs/authoring/session/views.py", - "sha256": "c772d515dfa738363ab403a3e19c9e438c17b0748a3a9ed65c44c416f579a378", + "sha256": "296adbceb7db7a7f7ac3cb09dbe8bb6620dec588a3b593ef2239108298bee5c8", "role": "Declares the SessionView family, each view's title, and the operations its window binds." }, { @@ -1172,6 +1209,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/declarative_window_system.rst in full and compared it against the changed authorities: openhcs/authoring/session/views.py (E1: debug-phase status prefix read from the phase class; orchestrator state, terminal status and debug phase projected by name); the dataset, pipeline and execution session views still exist and the page states nothing about those projections; the pyqt-reactive window-manager and scope-window-factory targets still exist.", "On 2026-10-11 (B6), re-read docs/source/architecture/declarative_window_system.rst in full and compared it against the changed authority external/pyqt-reactive/src/pyqt_reactive/services/window_manager.py (current file): WindowManager still provides visible top-level window discovery, register/unregister, show_or_focus reuse by scope, focus_and_navigate, and close cleanup; the linked pyqt-reactive window_manager_usage and scope_window_factory pages exist; no claim changed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/declarative_window_system.rst against B6's later changes to openhcs/authoring/session/views.py: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/declarative_window_system.rst against B6's later changes to openhcs/authoring/session/views.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", @@ -1197,12 +1235,12 @@ "authority": [ { "path": "openhcs/core/function_patterns.py", - "sha256": "cc8c35f37ea0a3bd939905fe317652a34b3129ee6f47f6f9023b8c459e62df70", + "sha256": "c8004ae1e3f241c4ac6ce0a61c69e3c0d1ecfb5021e1bef5dd0fb1fd1251ca50", "role": "Defines callable, sequential-chain, and dictionary-fan-out patterns and their grouped source keys." }, { "path": "tests/unit/test_function_patterns.py", - "sha256": "296c5ab1f5044b04ce631c92a1e8510fe9dd0e21a776d7a30877766154a74b18", + "sha256": "2eb64da61d0ae1ef2aedacddafc2f81b886515e9c500ac80b2cbcd49c48f5d9d", "role": "Verifies dictionary fan-out, composition, and invalid pattern rejection." } ], @@ -1211,6 +1249,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/dict_pattern_case_study.rst in full and compared it against the changed authorities: openhcs/core/function_patterns.py and tests/unit/test_function_patterns.py (K2: artifact imports split across openhcs.core.artifacts plans, specs, kinds, relations and input_projection); the stub states no pattern fact, and its target concepts/function_patterns still exists.", "On 2026-10-11 (B6), re-read docs/source/architecture/dict_pattern_case_study.rst in full and compared it against the changed authorities: openhcs/core/function_patterns.py (invocation-artifact imports and FunctionInvocationKey.from_callable removed, K5 callable-contract patterns) and tests/unit/test_function_patterns.py; dictionary fan-out keyed by group is still part of FunctionPatternSyntax and covered by test_invocation_positions_are_renumbered_per_dict_group, and concepts/function_patterns still exists, so the redirect stands.", "On 2026-10-11 (B6 final), compared docs/source/architecture/dict_pattern_case_study.rst against B6's later changes to openhcs/core/function_patterns.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/dict_pattern_case_study.rst against the later B6 changes to openhcs/core/function_patterns.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -1256,7 +1295,7 @@ }, { "path": "docs/source/architecture/experimental_analysis_system.rst", - "source_sha256": "5fb63b58a0a9d3c63672167141dccdcb3d11ec786b044386d136b4cc7d5730a6", + "source_sha256": "b4454a8c8396ac2861f2b8bbb8d9918c00a9aca7a249b475e904b8aa42bf0363", "audience": [ "OpenHCS experimental-analysis maintainers", "desktop analysis integrators" @@ -1271,22 +1310,22 @@ }, { "path": "openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py", - "sha256": "2705d459204557c4f7092df3fd21ed687dcf56449ed361485382b1c0ce196444", + "sha256": "782f462451aec3f98fc09956492cf4882531cdc41a459ea3c2a539353ed244b9", "role": "Coordinates the workbook-declared format strategy, replicate-local normalization, exports, and the configuration-projected run_directory boundary." }, { "path": "openhcs/processing/backends/experimental_analysis/result_formats.py", - "sha256": "6951f41a2ff2bdac1945c7e56584a578c111c078cac1d684030c00f95a23f528", - "role": "Defines the scope-keyed CX5 and MetaXpress ExperimentalResultFormatStrategy leaves for result reading and plate population." + "sha256": "7581999983e5acc930733451dddc84f69bf5dfbb94cfd9954f35690b7a6dd885", + "role": "Defines the CX5 and MetaXpress ExperimentalResultFormatStrategy leaves, registered by the scope_name a workbook declares, for result reading and plate population." }, { "path": "openhcs/processing/backends/experimental_analysis/analysis.py", - "sha256": "17a701e0740eac3e11319fe73f10302bbcab603218349bfba4e07014aabc27bb", + "sha256": "96b29239552724475c9fb010248bb266340fa0bbf06ae890f6074b8620c6183b", "role": "Owns layout parsing, exclusion projection, replicate averaging, and table/plate-grid export." }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "3cdb7819f09604046ba69e0b4f57b73e4e98d8f68af80158501a54bb42e701a2", + "sha256": "2d10806e310cb0d47ee7e47b879a7a95b28dcf5c688d9ce2a54b81ae1a041c47", "role": "Routes the desktop analysis action through ExperimentalAnalysisEngine.run_directory with the default configuration." }, { @@ -1295,9 +1334,12 @@ "role": "Verifies configured path projection, exclusions, normalization-method delegation, and an end-to-end MetaXpress directory run." } ], - "findings": [], - "disposition": "keep", + "findings": [ + "E1 deleted the ExperimentalAnalysisScope enum; the page still said result readers were selected by the workbook's ExperimentalAnalysisScope declaration. Each ExperimentalResultFormatStrategy leaf now names the workbook scope it serves as scope_name." + ], + "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/experimental_analysis_system.rst in full and compared it against the changed authorities: unified_analysis_engine.py, result_formats.py and analysis.py (E1: ExperimentalAnalysisScope deleted, formats resolved by ExperimentalResultFormatStrategy.for_scope over the scope_name registry) and openhcs/pyqt_gui/main.py (selection and update-check classes); replaced the ExperimentalAnalysisScope sentence with the scope_name selection, and confirmed NormalizationMethod, ExperimentalAnalysisConfig and the desktop ExperimentalAnalysisEngine call are unchanged.", "On 2026-10-11 (B6), re-read docs/source/architecture/experimental_analysis_system.rst in full and compared it against the changed authority openhcs/pyqt_gui/main.py (U3 moved dock layout, shortcuts and time-travel navigation to pyqt-reactive); the desktop analysis action still constructs ExperimentalAnalysisEngine() with its default configuration and calls run_directory, and ExperimentalAnalysisConfig and NormalizationMethod are still declared in openhcs/domains/microscopy/config.py; no claim changed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/experimental_analysis_system.rst against B6's later changes to openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py, openhcs/processing/backends/experimental_analysis/result_formats.py, openhcs/processing/backends/experimental_analysis/analysis.py, openhcs/pyqt_gui/main.py, tests/unit/test_experimental_analysis_engine.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/experimental_analysis_system.rst against the later B6 changes to openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py, openhcs/processing/backends/experimental_analysis/result_formats.py, openhcs/processing/backends/experimental_analysis/analysis.py, openhcs/pyqt_gui/main.py, tests/unit/test_experimental_analysis_engine.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -1318,7 +1360,7 @@ }, { "path": "docs/source/architecture/external_foundations.rst", - "source_sha256": "25610fe738d81d8911b99aa902c7e000945141d80424cb3f67b47dbdd93ca6fc", + "source_sha256": "d07de11d351f9e9920a8d56961e370a079266f547b2fbd877dd7a0fcf84590b3", "audience": [ "OpenHCS maintainers", "first-party package integrators" @@ -1328,7 +1370,7 @@ "authority": [ { "path": "pyproject.toml", - "sha256": "b5d4d752704b886da20d860743b81dc3d2ce3a3c402208246fadc1c6cb71f597", + "sha256": "75f92010b3c18ede61cf7296064079590c143283871e372a1fd4546417d4e0f9", "role": "Pins the eight first-party package constraints and documents their installed integration surface." }, { @@ -1403,8 +1445,8 @@ }, { "path": "external/python-introspect/src/python_introspect/jsonable.py", - "sha256": "0148e51fb2fc4c7e1c625b088cb0ca56fecd58f9836b9e5da58b75b5eb2d40ee", - "role": "Owns the to_jsonable encoder paired with the dataclass_from_mapping decoder." + "sha256": "c845361d0d14914798d9a1b5329d681300511fc7e95c67508943425716930ebd", + "role": "Owns the to_jsonable encoder paired with the dataclass_from_mapping decoder, registered_class_from_key and RegisteredClassChoices." }, { "path": "external/python-introspect/src/python_introspect/public_api.py", @@ -1458,12 +1500,11 @@ } ], "findings": [ - "U3 moved the ScopeKey scope grammar and field inspection into ObjectState and the editable table, grid selector, time-travel widgets and navigation, configuration window, and dock and shortcut layout into pyqt-reactive; the catalogue listed none of them.", - "L6 moved lazy-configuration source formatting into objectstate.codegen and the qualified-name, value-owned literal and path-factoring rules into pycodify; the ObjectState and pycodify entries did not say so.", - "U3 moved FieldLabel into python-introspect; its entry did not list field labels." + "E1 put the registry-key spelling of registered class families and the RegisteredClassChoices field annotation into python-introspect's jsonable module; the python-introspect entry did not list them." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/external_foundations.rst in full and compared it against the changed authorities: pyproject.toml (python-introspect floor 0.2.8; still eight first-party packages) and external/python-introspect/src/python_introspect/jsonable.py (registered_class_from_key and RegisteredClassChoices added beside to_jsonable); added the registered-class spelling and choices to the python-introspect entry and confirmed dataclass_from_mapping still lives in python-introspect and the strategy mixins still live in metaclass-registry.", "On 2026-10-11 (B6), re-read docs/source/architecture/external_foundations.rst in full and compared it against the changed authorities: pyproject.toml (pycodify 0.2, objectstate 2.1, pyqt-reactive 0.5, zmqruntime and python-introspect floors; still eight first-party packages), external/ObjectState/README.md (objectstate.codegen) and external/python-introspect/src/python_introspect/jsonable.py (to_jsonable unchanged), plus the U3 and L6 surface notes; added the moved ScopeKey, inspection, codegen, field-label, Qt machinery and pycodify rules to the catalogue and confirmed OpenHCS still keeps its domain formatters in openhcs/serialization/pycodify_formatters.py.", "On 2026-10-11 (B6 final), compared docs/source/architecture/external_foundations.rst against the later B6 changes to pyproject.toml: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/external_foundations.rst in full and compared it against the changed authorities: pyproject.toml (zmqruntime 0.6, polystore 0.6, objectstate 1.3.1, python-introspect 0.2.5, metaclass-registry 0.3.0, arraybridge 0.4.1, pyqt-reactive 0.4 bounds; still eight first-party packages), the package READMEs, window_snapshot.py, client.py and arraybridge types.py; added the L1, L2, L3 and L5 ownership moves to the catalogue and confirmed the PolyStore, OMERO and pyqt-reactive claims against current source.", @@ -1515,7 +1556,7 @@ "authority": [ { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "b96f33b866fe6c9f8118ca70fcb7dff8e6a02cb29d79df67230a5e8b97df8002", + "sha256": "5aa8895c9f533603b7ef972202141e2439473f0604b97d66f5a027b8b3cd1c83", "role": "Implements CellProfiler lowering to FunctionStep and PipelineConfig declarations with declaration-owned headless validation." }, { @@ -1530,17 +1571,17 @@ }, { "path": "openhcs/interop/cellprofiler/module_declarations.py", - "sha256": "b0195b66126c31f71dc0d4bb281a128e0164bb7dfb3b314ee8fe744ebc55d626", + "sha256": "6ff4f4bc3eaf6b4c9fc42752493c2b9ba2e11c8994d7edaea81f586ac57c3b89", "role": "Defines the nominal module validation hook queried by the importer." }, { "path": "openhcs/interop/cellprofiler/module_artifact_declarations.py", - "sha256": "ef233be43002d0f16b53c78c202b9203b6261847708d311089671a6b037db894", + "sha256": "5442b56c7dd7c913d38d18c5a76660ad6006e147ffc8f790712fb79b301ff035", "role": "Defines the interactive-module family that rejects headless import." }, { "path": "openhcs/core/dataset_sources/source.py", - "sha256": "350bf5691a03522e9f54b1d8ab712c6beba1683aac4fa5159ea4e80c6041530d", + "sha256": "b3758a878fdf8ec28b6d9818ebfd96c5c470895575f7a79500c85f1490679281", "role": "Defines the registered DatasetSource family that microscope handlers implement." }, { @@ -1570,13 +1611,14 @@ }, { "path": "tests/unit/test_cellprofiler_pipeline_import.py", - "sha256": "0ce4efd85d819c7e9f7fa84ec6f1ac7ec4bda024c9a8d5c7f264fda2730264f0", + "sha256": "67576f37b7cbe0595acea633265f8e14677ca3c1b8204630413330273dc4e58a", "role": "Verifies explicit rejection of an interactive-only module during direct import." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/external_integrations_overview.rst in full and compared it against the changed authorities: openhcs/interop/cellprofiler/pipeline_import.py, module_declarations.py, module_artifact_declarations.py (E1: InputSource became ReadPipelineStart/ReadPreviousStep classes; K2 artifact import paths; K3 object-count feature naming), openhcs/core/dataset_sources/source.py (get_required_backend removed, absolute source paths) and tests/unit/test_cellprofiler_pipeline_import.py; the page names none of those members and its importer, DatasetSource, ZMQRuntime and OMERO ownership claims still hold.", "On 2026-10-11 (B6), re-read docs/source/architecture/external_integrations_overview.rst in full and compared it against the changed authorities: openhcs/interop/cellprofiler/pipeline_import.py, module_declarations.py and module_artifact_declarations.py (K4 SourceBindingsConfig/StepSourceBindingsConfig now imported from openhcs.core.config; K5 contract fields read directly instead of through metadata), openhcs/core/dataset_sources/source.py (import move) and tests/unit/test_cellprofiler_pipeline_import.py (PairByOrder); the CppipeImporter is still a PipelineImporter family member, lowering still produces PipelineConfig plus FunctionStep declarations, interactive-only modules still fail, and DatasetSource remains the source family; no claim changed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/external_integrations_overview.rst against B6's later changes to openhcs/interop/cellprofiler/pipeline_import.py, openhcs/interop/cellprofiler/module_declarations.py, openhcs/interop/cellprofiler/module_artifact_declarations.py, openhcs/core/dataset_sources/source.py, openhcs/runtime/zmq_server_hooks.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/external_integrations_overview.rst against the later B6 changes to openhcs/interop/cellprofiler/pipeline_import.py, openhcs/interop/cellprofiler/module_declarations.py, openhcs/interop/cellprofiler/module_artifact_declarations.py, openhcs/core/dataset_sources/source.py, openhcs/runtime/zmq_server_hooks.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -1707,12 +1749,12 @@ "authority": [ { "path": "openhcs/core/function_patterns.py", - "sha256": "cc8c35f37ea0a3bd939905fe317652a34b3129ee6f47f6f9023b8c459e62df70", + "sha256": "c8004ae1e3f241c4ac6ce0a61c69e3c0d1ecfb5021e1bef5dd0fb1fd1251ca50", "role": "Normalizes callable, (callable, kwargs), list-chain, and dict patterns into named invocations and compiles them with their memory and artifact plans." }, { "path": "tests/unit/test_function_patterns.py", - "sha256": "296c5ab1f5044b04ce631c92a1e8510fe9dd0e21a776d7a30877766154a74b18", + "sha256": "2eb64da61d0ae1ef2aedacddafc2f81b886515e9c500ac80b2cbcd49c48f5d9d", "role": "Verifies invocation identity for list and dict patterns, normalization, tuple-leaf validation, and compiled pattern semantics." } ], @@ -1721,6 +1763,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/function_pattern_system.rst in full and compared it against the changed authorities: openhcs/core/function_patterns.py and tests/unit/test_function_patterns.py (K2: artifact imports split across openhcs.core.artifacts submodules); the stub states no pattern fact, and its target concepts/function_patterns still exists.", "On 2026-10-11 (B6), re-read docs/source/architecture/function_pattern_system.rst in full and compared it against the changed authorities: openhcs/core/function_patterns.py (K5: invocation-artifact imports and FunctionInvocationKey.from_callable removed, compile_function_pattern takes an admitted artifact graph) and tests/unit/test_function_patterns.py; normalize_function_pattern and compile_function_pattern still own callable, chain and dict pattern semantics and concepts/function_patterns exists, so the redirect stands.", "On 2026-10-11 (B6 final), compared docs/source/architecture/function_pattern_system.rst against B6's later changes to openhcs/core/function_patterns.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/function_pattern_system.rst against the later B6 changes to openhcs/core/function_patterns.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -1753,7 +1796,7 @@ }, { "path": "openhcs/core/callable_contract.py", - "sha256": "a8e73c5df804d78baf72710e210cb2ae63156657a074071e11fd868f4b19d3b4", + "sha256": "6089f63e72e1a47fc5f6bc7b443028ac8ec40d934898589fa03b5592c2273ab3", "role": "Defines CallableDeclaration and CallableContract, which add OpenHCS processing, runtime, artifact, scope, and ArrayBridge memory semantics on top of python-introspect annotation helpers." }, { @@ -1767,6 +1810,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/function_reference_pattern.rst in full and compared it against the changed authorities: openhcs/core/callable_contract.py (K2: artifact imports from openhcs.core.artifacts plans, specs and kinds; artifact type spelled by value); the stub states no callable-contract fact, and its targets processing_semantics and external_foundations still exist.", "On 2026-10-11 (B6), re-read docs/source/architecture/function_reference_pattern.rst in full and compared it against the changed authorities: openhcs/core/callable_contract.py (K5: CallableMetadata replaced by CallableDeclaration, which CallableContract now subclasses) and tests/unit/test_callable_contract.py; generic signature analysis and callable_declaration_kwargs still come from python-introspect and OpenHCS callable semantics stay in callable_contract, so the redirect stands; refreshed the callable_contract and test evidence roles.", "On 2026-10-11 (B6 final), compared docs/source/architecture/function_reference_pattern.rst against B6's later changes to openhcs/core/callable_contract.py, tests/unit/test_callable_contract.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/function_reference_pattern.rst against the later B6 changes to openhcs/core/callable_contract.py, tests/unit/test_callable_contract.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -1838,7 +1882,7 @@ }, { "path": "openhcs/core/callable_contract.py", - "sha256": "a8e73c5df804d78baf72710e210cb2ae63156657a074071e11fd868f4b19d3b4", + "sha256": "6089f63e72e1a47fc5f6bc7b443028ac8ec40d934898589fa03b5592c2273ab3", "role": "Projects declaration-owned input, output, and execution memory into immutable callable contracts." }, { @@ -1853,17 +1897,17 @@ }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "4a33908fb42874e6d57eeea3f5305793ddbda41d13356f0a78501af38dcdfeb4", + "sha256": "40b07b7f40aca39b19b95a8f61c1b0c8270f0ab0a29e8c36caa13f04d8fa539f", "role": "Calls assign_framework_devices on each context's step plans before bundle assembly." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "358d3eaf5952170f9e9e2dc9056fb08707e8fcf74682063c074b367a83c2648f", + "sha256": "6525348d097bda8d0265bc7c3f312dd8839ca9f65dd05f797fc82ba873e04547", "role": "Owns FrameworkDeviceAssignment bindings, merged cleanup footprints, and the requires_gpu projection." }, { "path": "openhcs/core/orchestrator/worker_execution.py", - "sha256": "26b9586890e7117b24b6b85660d3f417455d55546401739f91146d18e4d21134", + "sha256": "83ade0271c9b24ca1ba61c708d9e6f60a432b630ec6a8a2fac6d1018458e6be5", "role": "Releases each axis's compiled footprint after process and inline contexts and the merged footprint only after thread-backed lanes join." }, { @@ -1873,7 +1917,7 @@ }, { "path": "openhcs/utils/environment.py", - "sha256": "2d8738e593b3e92c966089772624f47d17d38bb27fb90974707af659c91e8ad8", + "sha256": "1985ffbe42b5dfb283b78b6282f9b56e08cd51d5a9e79f2dd4eebffcb6da0bf5", "role": "Owns OPENHCS_CPU_ONLY and the CPU-only mode that --no-gpu enables before framework imports." }, { @@ -1895,6 +1939,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/gpu_resource_management.rst in full and compared it against the changed authorities: openhcs/core/callable_contract.py and compiled_step_plan.py (K2 artifact import paths, UnresolvedStepInput default), openhcs/core/pipeline/compiler.py (E1: orchestrator state and MaterializeToZarr classes, PrimaryPlaneRole), openhcs/core/orchestrator/worker_execution.py (runtime observation mode classes) and openhcs/utils/environment.py (AgentRuntimePlatform rename); none touches the memory-declaration, device-assignment, worker-start or cleanup claims, which still hold.", "On 2026-10-11 (B6), re-read docs/source/architecture/gpu_resource_management.rst in full and compared it against the changed authorities: openhcs/core/callable_contract.py (K5: input, output and execution memory are now declared CallableDeclaration fields read from the same ArrayBridge MemoryContractAttribute keys; declared_memory_types unchanged), openhcs/core/pipeline/compiler.py (ScopeKey scope ids, progress classes), openhcs/core/compiled_step_plan.py (source plan default constructors), openhcs/core/orchestrator/worker_execution.py (progress classes) and tests/unit/test_framework_execution_scope.py (admitted_graph); FrameworkDeviceResolver, WorkerStartExecutionFacts, MemoryType.subprocess_environment and per-lane cleanup are unchanged; no claim changed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/gpu_resource_management.rst against B6's later changes to openhcs/core/callable_contract.py, openhcs/core/pipeline/compiler.py, openhcs/core/compiled_step_plan.py, openhcs/core/orchestrator/worker_execution.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/gpu_resource_management.rst against the later B6 changes to openhcs/core/callable_contract.py, openhcs/core/pipeline/compiler.py, openhcs/core/compiled_step_plan.py, openhcs/core/orchestrator/worker_execution.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -1961,7 +2006,7 @@ }, { "path": "openhcs/core/config.py", - "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", + "sha256": "9625ca054fac6c1e460216cfe0e85621f5306592a004d1bdc28a6b1ce2099366", "role": "Declares GlobalPipelineConfig and the step variable_components and group_by fields, with group_by defaulting from the active AxisFamily." }, { @@ -1971,12 +2016,12 @@ }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "0f9cc4819b53d55b7e9446429b8c8ac1ff1a59f4902020ea1658495ce01d464c", + "sha256": "4946c221e34ec19957b5af2b1a54359d454ee840fb1f5e1fe2b3b306ac8e5426", "role": "Defines CompilationSession, the scoped compiler state named by the invariants." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "358d3eaf5952170f9e9e2dc9056fb08707e8fcf74682063c074b367a83c2648f", + "sha256": "6525348d097bda8d0265bc7c3f312dd8839ca9f65dd05f797fc82ba873e04547", "role": "Defines typed CompiledStepPlan fields, including framework footprints and device bindings." }, { @@ -1988,6 +2033,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/index.rst in full and compared it against the changed authorities: openhcs/core/config.py (E1: MaterializationBackend and InputSource families), openhcs/core/pipeline/compilation_session.py and openhcs/core/compiled_step_plan.py (K2 artifact import paths, UnresolvedStepInput); every toctree target still exists and the architecture invariants (PipelineConfig plus FunctionStep, CompilationSession, CompiledStepPlan, ProcessingContract, CompiledExecutionBundle) still hold.", "On 2026-10-11 (B6), re-read docs/source/architecture/index.rst in full and compared it against the changed authorities: openhcs/core/config.py (PartitionFilterMode family, K4 source-binding declarations), openhcs/core/processing_contracts.py (CallableMetadata renamed CallableDeclaration), openhcs/core/pipeline/compilation_session.py (graph-owned invocation validation) and openhcs/core/compiled_step_plan.py (source plan default constructors); the invariants (PipelineConfig plus FunctionStep, CompilationSession and typed CompiledStepPlan, separate variable_components, group_by and ProcessingContract declarations, CompiledExecutionBundle for workers) still hold; no claim changed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/index.rst against B6's later changes to openhcs/core/config.py, openhcs/core/processing_contracts.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/compiled_step_plan.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/index.rst against the later B6 changes to openhcs/core/config.py, openhcs/core/processing_contracts.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/compiled_step_plan.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -2020,7 +2066,7 @@ }, { "path": "openhcs/core/config.py", - "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", + "sha256": "9625ca054fac6c1e460216cfe0e85621f5306592a004d1bdc28a6b1ce2099366", "role": "Declares PartitionFilterConfig's show_preview_without_partition_filter policy and PartitionFilterMode members' preview_prefix operators." }, { @@ -2030,17 +2076,17 @@ }, { "path": "openhcs/pyqt_gui/services/reactor_providers.py", - "sha256": "04ad362d7bb31641f5a4ba0bfcc8670cd2e330a6eadc231a0ca5df7abed8d2b4", + "sha256": "ca5fe97ba932805dd0f8cfa3bc85352cb428b12a46fcae83751382092d77475c", "role": "Registers format_partition_filter_config for PartitionFilterConfig with the generic preview service." }, { "path": "openhcs/pyqt_gui/config.py", - "sha256": "a3eece2e5b2926fc6fc0edd197221da451ab7c0d0d6f4ffc8bb2ff865ad74bd4", + "sha256": "d092cb20206127484fbbc80a38a5152462b9b8991e848a814c85f564d51f4db8", "role": "Embeds FormattingConfig as UIConfig.list_previews." }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "3cdb7819f09604046ba69e0b4f57b73e4e98d8f68af80158501a54bb42e701a2", + "sha256": "2d10806e310cb0d47ee7e47b879a7a95b28dcf5c688d9ce2a54b81ae1a041c47", "role": "_apply_list_preview_config applies list_previews to Plate Manager and Pipeline Editor at startup and on UI config change." }, { @@ -2079,11 +2125,10 @@ "role": "Native Qt regressions prove row wrapping, disclosure, refresh/reorder preservation, graphical guides, structural action parity, and delegate paint acknowledgement." } ], - "findings": [ - "PartitionFilterMode became a class family; the page's 'members' wording was already corrected to 'subclasses' in the page change under review, which this review confirms against openhcs/core/config.py." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/list_item_preview_system.rst in full and compared it against the changed authorities: openhcs/core/config.py (E1: MaterializationBackend and InputSource families; PartitionFilterMode subclasses and preview_prefix unchanged), openhcs/pyqt_gui/services/reactor_providers.py (OrchestratorCreated), openhcs/pyqt_gui/config.py (log-level helper, AgentRuntimePlatform rename; UIConfig.list_previews unchanged) and openhcs/pyqt_gui/main.py (selection and update-check classes); the formatter, shared-policy and wrapping claims still hold and the pyqt-reactive target page still exists.", "On 2026-10-11 (B6), re-read docs/source/architecture/list_item_preview_system.rst in full and compared it against the page change (PartitionFilterMode members to subclasses) and the changed authorities: openhcs/core/config.py (PartitionFilterMode subclasses declare preview_prefix), openhcs/pyqt_gui/services/reactor_providers.py (still registers the one PartitionFilterConfig preview formatter; code documents now use pycodify generate_python_source) and openhcs/pyqt_gui/main.py (U3 moves; still applies UIConfig.list_previews to each manager via set_preview_config); every claim holds.", "On 2026-10-11 (B6 final), compared docs/source/architecture/list_item_preview_system.rst against B6's later changes to openhcs/core/config.py, openhcs/pyqt_gui/services/reactor_providers.py, openhcs/pyqt_gui/main.py, tests/unit/pyqt_gui/test_main_config_propagation.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/list_item_preview_system.rst against the later B6 changes to openhcs/core/config.py, openhcs/pyqt_gui/services/reactor_providers.py, openhcs/pyqt_gui/main.py, tests/unit/pyqt_gui/test_main_config_propagation.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -2116,7 +2161,7 @@ }, { "path": "openhcs/mcp/server.py", - "sha256": "e1f236011df715da3d17574c0b80cbf50fcae45e3f82491c22fb57e98c6bb662", + "sha256": "c3dd3a396a3c304341b06c7c813036e779c921fa915060d2d3d2ed594b00b697", "role": "Binds the selected declaration registry into FastMCP with outputContract metadata and structured output, runs non-worker-safe capabilities through the main-thread dispatcher, and binds the session to the main thread." }, { @@ -2136,17 +2181,17 @@ }, { "path": "tests/unit/agent/test_mcp_distribution_runtime.py", - "sha256": "9ddc7dd1386813dc8199ff64ab1ecb0941527fbf62a9cab6577679730f94ea36", + "sha256": "3f443a32c1ba1549e22275fee7a0d05fc26a89cc14cdbdad230a05ae20be42a2", "role": "Proves building the local and hosted MCP servers imports no PyQt6." }, { "path": "openhcs/mcp/dev_client_core.py", - "sha256": "41739b744b0e4287ca7d14cd133371d669e9bff12f4b622313996ae9bb5597da", + "sha256": "1bb405ee541cdb48b27c355aeb71ad680ec8c14f9fd31089814e7ac01751b89b", "role": "Resolves tool identity and compatible profiles from capability declarations and renews the command deadline on each response or progress notification." }, { "path": "openhcs/mcp/dev_client_session.py", - "sha256": "2dceca7937087d05f72af1d037122ea393721d61d690803628a7b14ca70fd980", + "sha256": "16a508501214f9a393636320fad49d8015936c30bd8c85737d277aecf602876a", "role": "Implements execute-source as add-datasets, set-pipeline, then initialize, compile and run stages followed through session events." }, { @@ -2161,12 +2206,12 @@ }, { "path": "openhcs/authoring/session/session.py", - "sha256": "0c4e0f0979e1793862859e47eefeb6dd9d4b51c99fba20c2ebc6adf11619c3b5", + "sha256": "ced615b580e4625e09873449b29ac67cdea87c8b8065c205f32e6d5921cf31fc", "role": "Defines the headless Session whose datasets and pipelines live in the MCP process's ObjectState and whose events clients follow." }, { "path": "openhcs/pyqt_gui/config.py", - "sha256": "a3eece2e5b2926fc6fc0edd197221da451ab7c0d0d6f4ffc8bb2ff865ad74bd4", + "sha256": "d092cb20206127484fbbc80a38a5152462b9b8991e848a814c85f564d51f4db8", "role": "Declares UIConfig and projects its unique execution endpoint without constructing unrelated UI configuration." }, { @@ -2176,17 +2221,17 @@ }, { "path": "openhcs/mcp/http.py", - "sha256": "c9de5befe7bbbcd6948749e4eafccb1364c0eec78930d55381ac9be0569a3618", + "sha256": "cac880ea5dfee3cc545c80d09c4a20ab960f40c1104fe1cb724ff34a56d93956", "role": "Builds the stateless hosted Streamable HTTP server and rejects mutating hosted declarations." }, { "path": "openhcs/mcp/http_auth.py", - "sha256": "1c2801842259c91a764940264e07ad5a05cc6492e42f04a841b0ceeb026d73e2", + "sha256": "54f5291960610c0aef4ec7aa6c4afa781d9828c78e62dd604a34e57c0128b3c5", "role": "Declares public read-only and OAuth deployment policy, URL, origin, host, and audit constraints." }, { "path": "openhcs/mcp/client_registration.py", - "sha256": "55b701bb77a3de19243fa9a6276f784fad8329e2d435fb26fcc7fdce1e060731", + "sha256": "4d6fc5f744617ab1d0e9b8317d07965496aaa6b098a4d179ef95cc6baec35671", "role": "Projects one stable launcher into registered local-client configurations and synchronises the skill bundle into clients that declare a skill directory." }, { @@ -2196,12 +2241,12 @@ }, { "path": "openhcs/desktop/deployment.py", - "sha256": "4c6a7d63bf5ea759cc3eecd46e93fa335304113873869474f5fe6823f60c298f", + "sha256": "aafb7c77756cd3cd9de71bec7a6360058cd14e3aebc27b5e014c3b74271307cf", "role": "Defines DesktopDeployment and its Windows and macOS leaves, including bounded deferral of the live Windows launcher." }, { "path": "tests/unit/desktop/test_desktop_deployment.py", - "sha256": "6730f5346c91dc8b4f80656b9ab02012124b5ec9bdc4311f3f8d4932ac951ede", + "sha256": "6063b676685472fd6431b8ad90329552ce84f953aa938bb964c3b0eefaf11fb0", "role": "Proves live-launcher sharing violations defer only the cache projection and all unsafe failures remain fatal." }, { @@ -2221,7 +2266,7 @@ }, { "path": "openhcs/pyqt_gui/services/ui_bridge_windows.py", - "sha256": "29265ed04137392757a6d515020920120e29fd556f433575956ae38cedff0158", + "sha256": "a10489e1bae296bc224b339ee32e2e99537175f891b86f901b4d7503d6f38f13", "role": "Resolves authorised OpenHCS windows, projects snapshot results, and distinguishes focus from accepted target navigation." }, { @@ -2251,7 +2296,7 @@ }, { "path": "openhcs/agent/services/llm_context_service.py", - "sha256": "ca5efbd7bfbd6a8aded1695b05abecba160fb817c04a14bc7c4a2df449ac5eaa", + "sha256": "a6511e95142361137fdfd2c50f0d251db14a8509b99c14f3883f48cbee18bca5", "role": "Composes operating-guide sections from AuthoringContextSection facets and bounds returned guidance." }, { @@ -2261,7 +2306,7 @@ }, { "path": "benchmark/control.py", - "sha256": "83fe9016625d7d8721905b8a153a28c3dcaaf2b9e925a3c97711c634868156b7", + "sha256": "b10dd87d7bc44063566e0c53d849091c0fc43a345b434c13467df1ede86ac892", "role": "Implements contained read-only receipt, observation-progress, and artifact inspection without importing benchmark execution modules." }, { @@ -2271,13 +2316,14 @@ }, { "path": "benchmark/cellprofiler_benchmark_cli.py", - "sha256": "2e0de28a96c57ee150e28fbbed28d876507e9281075a03f729838f3ca3610966", + "sha256": "0c2cab4d989d50149cb314e908baace8aa794953c86ebc572fa120beaa695780", "role": "Owns the installed openhcs-benchmark execution, report and list-cases command surface." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/mcp_distribution.rst in full and compared it against the changed authorities: openhcs/mcp/server.py (registered-family defaults in tool schemas, recovery reason by name), dev_client_core.py and dev_client_session.py (poll status, field filter and orchestrator state classes), http.py and http_auth.py (McpHttpAuthenticationMode became PublicReadOnlyAccess/OAuthIntrospectionAccess classes), client_registration.py and openhcs/desktop/deployment.py (status classes; platform leaves keyed by AgentRuntimePlatform class), openhcs/authoring/session/session.py (current file: E1 state families; events_after, set_transport_config and FinishedExecution unchanged), ui_bridge_windows.py (error_dialog flag), llm_context_service.py, benchmark/control.py and benchmark/cellprofiler_benchmark_cli.py (export scope classes) and the two changed tests; CapabilityTransport is still an Enum with HOSTED_STREAMABLE_HTTP, the Windows and macOS registration and deployment leaves, the public or OAuth-introspection policy and the benchmark capability claims still hold, and the page names no renamed member.", "On 2026-10-11 (B6), re-read docs/source/architecture/mcp_distribution.rst in full and compared it against the changed authorities: authoring/session/session.py (K6 debug command/replay classes), core/config_document.py (L6: rendering through pycodify.generate_python_source; project_dataclass_field_from_source unchanged) and pyqt_gui/services/ui_bridge_windows.py (U3: dirty/signature markers from objectstate.inspection, DockPane from pyqt-reactive); none changes the headless-session, saved-endpoint projection, window-capture, navigation or row-action claims, so no prose change was needed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/mcp_distribution.rst against B6's later changes to openhcs/authoring/session/session.py: the writability probe makes one exclusive create attempt, the installed demo reports ViewerType.value, the unit session pre-creates its QApplication, and the GUI journey waits for each batch to end. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/mcp_distribution.rst against B6's later changes to openhcs/mcp/execution.py: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", @@ -2337,7 +2383,7 @@ }, { "path": "docs/source/architecture/measurement_equivalence_system.rst", - "source_sha256": "55ba2895f65352f0b5404bb1730d42ff8a755b3e495777f66c9e592e407bcf96", + "source_sha256": "1ccad7b80c8dd238b8318071f1d96fdc968e15fa277578f9161bc3cc35803102", "audience": [ "OpenHCS equivalence maintainers", "CellProfiler parity reviewers" @@ -2352,22 +2398,22 @@ }, { "path": "openhcs/core/equivalence/keys.py", - "sha256": "0ae79e4cc4e70804f9d2e948078ade7c271e142473821731b38eeccaef064b5d", + "sha256": "78f2f9851bade6dad6307f5520a46789af31144240e7f55b9cda7f65c7abadc4", "role": "Defines RuntimeMeasurementFeatureKey and the semantic row, source, and aggregate identities." }, { "path": "openhcs/core/equivalence/measurement_features.py", - "sha256": "18ff75cdd4dad0f88870b90b9a91f2ea352372773bcc9a74cee288d4c110f05b", + "sha256": "5593fc6a55967f35effeb1e5b3a55a93dcbfbf59f0baecc06c2ac641036d33cc", "role": "Defines the RuntimeMeasurementFeatureSemanticProfile most-derived strategy family." }, { "path": "openhcs/core/equivalence/relationships.py", - "sha256": "fa257ac5280204b3f749fb581f98e80afea203dbd11fdb39c3cc4186d42625b5", - "role": "Defines the registered object-instance-key plane alignment strategies used for relationship comparison." + "sha256": "9d4d85b89918efc19e5b8a0349f8b9d4097cd7aacf93d31bf503d5829148fd49", + "role": "Defines the relationship aggregate-feature semantics that production measurement execution keeps." }, { "path": "openhcs/core/measurement_dialect.py", - "sha256": "96cac83e09e22eeaa02dbd14da2f9553c1f19f51962e83731936ba33429c4578", + "sha256": "4eb11b861af78a3b86108a95bd07295ad8efbcb3e125cfbda429ef9839e9ad47", "role": "Defines the registered MeasurementDialect family and PlainMeasurementDialect." }, { @@ -2382,12 +2428,12 @@ }, { "path": "benchmark/adapters/openhcs.py", - "sha256": "8621fb5adcab09e7e376d2c5f4eddafc5f485a9328669985f7a36b0646b3a561", + "sha256": "fbfc303a156081a5646993e41429fa1459cef7776df12f29f593dd1641222ad8", "role": "Builds benchmark runtime snapshots, the strict CellProfiler policy at 1e-6 tolerances, and digest-bound reference-export comparisons." }, { "path": "benchmark/openhcs_measured_run.py", - "sha256": "58916c818b713cc6388a6941ce33cbbdf7160837792475ce76292f16bbe442b6", + "sha256": "cf05a4e08f0a24a68b9d11fa06f1ae362f81de1a959fffcfa02437557acc026d", "role": "Wraps the ordinary compile-and-execute submission with observation validation, phase timing from completed server records, endpoint provenance, and the completed-run receipt." }, { @@ -2407,17 +2453,17 @@ }, { "path": "benchmark/control.py", - "sha256": "83fe9016625d7d8721905b8a153a28c3dcaaf2b9e925a3c97711c634868156b7", + "sha256": "b10dd87d7bc44063566e0c53d849091c0fc43a345b434c13467df1ede86ac892", "role": "Derives unreceipted_artifacts and retained_evidence_valid during measured-run inspection." }, { "path": "openhcs/runtime/zmq_execution_observation.py", - "sha256": "3cc43d3eee6c4701c23fec183ecd895a71acdfcc14acfe2afca62625ccb7381b", + "sha256": "db40a0d1e82bdbe778f948619e62a93d74c16d3155ebc701a067bbba164f0efa", "role": "Declares value and outcome exports with required compiled axis membership and rejects exports from other schema versions." }, { "path": "benchmark/contracts/measured_run_receipt.py", - "sha256": "c04b69926f1994fd7528eadf172a31ef33cf409380fef60b659d5f27f522f065", + "sha256": "e19f8a8f3448027fc8ad9139cd843b4cea75369890ae950230a20528aff35128", "role": "Declares the typed single-pipeline receipt, accepting archived receipts without retained-artifact digests and requiring them on current receipts." }, { @@ -2432,7 +2478,7 @@ }, { "path": "benchmark/cellprofiler_reference_exports.py", - "sha256": "58cdcb7572266d562314ccae5bf9d98b5f3a4b19944e8a068f89b617219952f3", + "sha256": "3f175ad2173cffb2181246b6d5cc498c64dfe90b5f38f87265c1effafeb60e6f", "role": "Owns terminal-artifact exporter derivation, exact output-inventory selection, numeric-image tolerance, categorical-label equality, and explicit single-plane label projection." }, { @@ -2444,18 +2490,30 @@ "path": "tests/unit/test_benchmark_timing.py", "sha256": "a6d64635ab230da628c417a08e482f95b1c2a90ba6b9389abb21a459ce42da89", "role": "Proves completed-job duration validation and the additive-total exclusions." + }, + { + "path": "benchmark/equivalence/exported_relationships.py", + "sha256": "8df617edf9fd1865e78f22a9089da2e8c44ab1204422514151a3e159586730ce", + "role": "Defines the registered object-instance-key plane alignment strategies used for relationship comparison." + }, + { + "path": "benchmark/equivalence/measurement_rows.py", + "sha256": "a3da752a21b0ce57a527c1d7be5975d951ef82bcb48d48f09658c2b163ef3eba", + "role": "Owns row identity and wide/long-form row projection for comparison, outside the product package." + }, + { + "path": "benchmark/equivalence/native_shape_rows.py", + "sha256": "759d9dc2816778df2aed5c8bec6809b2c0cbabf791521c03f587c7305db66a7f", + "role": "Moves positionally written native AreaShape values back onto their objects before comparison." } ], "findings": [ - "The page named RuntimeMeasurementDialect, which no longer exists; dialects are now the registered MeasurementDialect family (PlainMeasurementDialect in the kernel, CellProfilerMeasurementDialect in interop). Corrected.", - "RuntimeEquivalenceReport and the comparison half moved from openhcs/core/equivalence to benchmark/equivalence (f16177ed7); the page did not say where comparison lives. Added the product/benchmark boundary and replaced the deleted report.py authority.", - "The page said archived outcome-only exports without compiled axis membership remain readable and archived observation exports keep an all-axis expectation; the legacy readers were deleted and exports from other schema versions are now rejected. Corrected both passages.", - "Agent finalisation is supplied by the OpenHCS session's finished execution (U1), not an execution service and submission. Corrected.", - "Additive phase totals now also exclude benchmark validation phases; added.", - "'The two integration tests' was imprecise now that the module also holds Fiji variant tests; named the headless and Napari tests. Removed 'canonical' filler." + "K3 moved measurement_rows.py, measurement_facts.py and the object-instance plane-alignment strategies from openhcs.core.equivalence to benchmark/equivalence; the page still listed rows among what the product package keeps.", + "K3 keeps AreaShape values on the object they measured and reads native CellProfiler reference tables through benchmark/equivalence/native_shape_rows.py; the reference-comparison section did not say the native values are moved back to their objects before comparison." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/measurement_equivalence_system.rst in full and compared it against the changed authorities: openhcs/core/equivalence/keys.py, measurement_features.py and relationships.py (K3: row and fact projection and plane alignment moved to benchmark/equivalence, aggregate-feature semantics stay), openhcs/core/measurement_dialect.py (SourceNameEncoding family; row qualifiers in openhcs/core/measurement_row_qualifiers.py), benchmark/adapters/openhcs.py (shape_rows_on_their_labels on reference snapshots, MaterializeToDisk), benchmark/openhcs_measured_run.py, benchmark/control.py, benchmark/contracts/measured_run_receipt.py and openhcs/runtime/zmq_execution_observation.py (E1: export scope and axis status classes) and benchmark/cellprofiler_reference_exports.py (K2 import path); corrected the product-package module list, added the native AreaShape row placement, and confirmed that no openhcs module imports benchmark and that the dialect, policy, receipt, timing and Official30 claims still hold.", "On 2026-10-11 (B6 final), compared docs/source/architecture/measurement_equivalence_system.rst against B6's later changes to tests/integration/test_cellprofiler_official30_zmq.py: the session decides in-place initialization by an actual file-creation attempt that leaves nothing behind, its debug-session methods moved into an inherited mixin, the measured CLI treats Windows Ctrl+Break like Ctrl+C, and the Official30 viewer test reads typed payload summaries. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/measurement_equivalence_system.rst against B6's later changes to openhcs/core/equivalence/keys.py, openhcs/core/equivalence/measurement_features.py, openhcs/core/measurement_dialect.py, benchmark/control_service.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/measurement_equivalence_system.rst against the later B6 changes to openhcs/core/equivalence/keys.py, openhcs/core/equivalence/measurement_features.py, openhcs/core/measurement_dialect.py, benchmark/adapters/openhcs.py, benchmark/openhcs_measured_run.py, benchmark/control_service.py, tests/integration/test_cellprofiler_official30_zmq.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -2512,12 +2570,12 @@ }, { "path": "openhcs/core/callable_contract.py", - "sha256": "a8e73c5df804d78baf72710e210cb2ae63156657a074071e11fd868f4b19d3b4", + "sha256": "6089f63e72e1a47fc5f6bc7b443028ac8ec40d934898589fa03b5592c2273ab3", "role": "CallableDeclaration declares the input, output and execution memory types under ArrayBridge's MemoryContractAttribute keys; its namespace codec is derived from those field declarations." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "358d3eaf5952170f9e9e2dc9056fb08707e8fcf74682063c074b367a83c2648f", + "sha256": "6525348d097bda8d0265bc7c3f312dd8839ca9f65dd05f797fc82ba873e04547", "role": "Owns FrameworkDeviceAssignment framework-local device bindings and footprint cleanup." }, { @@ -2527,10 +2585,11 @@ } ], "findings": [ - "K5 replaced callable_contract.py's hand-written MemoryContractAttribute read/write code with field declarations on CallableDeclaration; the callable_contract.py evidence role was updated. The page's ownership split (ArrayBridge: memory types, conversion, metadata keys; OpenHCS: contract projection, compiled framework-device plans, runtime adapters) is still accurate, so no prose change was needed." + "The page is a redirect stub; its link target still exists and owns the content (checked 2026-10-11, B6)." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/memory_type_system.rst in full and compared it against the changed authorities: openhcs/core/callable_contract.py and openhcs/core/compiled_step_plan.py (K2: imports from the openhcs/core/artifacts package modules; E1: UnresolvedStepInput default); confirmed neither change touches the memory-type keys or FrameworkDeviceAssignment, and that gpu_resource_management, external_foundations and openhcs/core/runtime_adapters.py still exist; the page text needed no change.", "On 2026-10-11 (B6), re-read docs/source/architecture/memory_type_system.rst in full and compared it against the changed authorities: openhcs/core/callable_contract.py (K5 declared-field CallableDeclaration) and openhcs/core/compiled_step_plan.py (CompiledSourceBindingPlan defaults); confirmed CallableDeclaration still keys memory types by arraybridge MemoryContractAttribute, FrameworkDeviceAssignment still owns footprint cleanup, and runtime_adapters.py, gpu_resource_management and external_foundations exist; updated one authority role.", "On 2026-10-11 (B6 final), compared docs/source/architecture/memory_type_system.rst against B6's later changes to openhcs/core/callable_contract.py, openhcs/core/compiled_step_plan.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/memory_type_system.rst against the later B6 changes to openhcs/core/callable_contract.py, openhcs/core/compiled_step_plan.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -2560,7 +2619,7 @@ "authority": [ { "path": "openhcs/core/dataset_sources/source.py", - "sha256": "350bf5691a03522e9f54b1d8ab712c6beba1683aac4fa5159ea4e80c6041530d", + "sha256": "b3758a878fdf8ec28b6d9818ebfd96c5c470895575f7a79500c85f1490679281", "role": "Defines the DatasetSource family keyed by source_name, its selection-role mixins, detection_order, create hook, workspace backend registration, and virtual-workspace metadata recording." }, { @@ -2585,7 +2644,7 @@ }, { "path": "openhcs/core/config.py", - "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", + "sha256": "9625ca054fac6c1e460216cfe0e85621f5306592a004d1bdc28a6b1ce2099366", "role": "Declares PipelineConfig.dataset_source as a DatasetSourceChoice class defaulting to AutoDetectedSource, and the SourceBindingsConfig declaration the source-binding source consumes." }, { @@ -2607,6 +2666,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/microscope_handler_integration.rst in full and compared it against the changed authorities: openhcs/core/dataset_sources/source.py (get_required_backend deleted, virtual-workspace root compared through absolute_source_path) and openhcs/core/config.py (E1: MaterializationBackend and InputSource became class families); the page states compatible storage backends without naming get_required_backend or MaterializationBackend, and DatasetSourceChoice, detection_order, the selection-role mixins, PipelineConfig.dataset_source and SourceBindingsConfig are unchanged, so no prose change was needed.", "On 2026-10-11 (B6), re-read docs/source/architecture/microscope_handler_integration.rst in full and compared it against the changed authorities: dataset_sources/source.py, choice.py, openhcs_format.py and microscopes/bioformats.py (K4: SourceBindingsConfig and source_bindings_defaults_to_base now imported from core/config.py) and core/config.py (K4: SourceBindingsConfig declared there; PartitionFilterMode became a class family); the page names SourceBindingsConfig without a module path, and DatasetSourceChoice.choices/open, detection_order, extension_modules discovery and PipelineConfig.dataset_source are unchanged, so no prose change was needed; the config.py evidence role now also covers SourceBindingsConfig.", "On 2026-10-11 (B6 final), compared docs/source/architecture/microscope_handler_integration.rst against B6's later changes to openhcs/core/dataset_sources/source.py, openhcs/core/dataset_sources/choice.py, openhcs/core/dataset_sources/interfaces.py, openhcs/core/dataset_sources/openhcs_format.py, openhcs/core/config.py, openhcs/microscopes/bioformats.py, tests/unit/test_dataset_source_family.py, tests/unit/test_dataset_source_guards.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/microscope_handler_integration.rst against the later B6 changes to openhcs/core/dataset_sources/source.py, openhcs/core/dataset_sources/choice.py, openhcs/core/dataset_sources/interfaces.py, openhcs/core/dataset_sources/openhcs_format.py, openhcs/core/config.py, openhcs/microscopes/bioformats.py, tests/unit/test_dataset_source_family.py, tests/unit/test_dataset_source_guards.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -2634,22 +2694,22 @@ "authority": [ { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "4a33908fb42874e6d57eeea3f5305793ddbda41d13356f0a78501af38dcdfeb4", + "sha256": "40b07b7f40aca39b19b95a8f61c1b0c8270f0ab0a29e8c36caa13f04d8fa539f", "role": "Discovers partition-axis values through the active AxisFamily and compiles one frozen ProcessingContext per value." }, { "path": "openhcs/core/orchestrator/bundle_execution.py", - "sha256": "bc3a2d0a50c3db3be81b03f6af4a139c3db2f779e969aad608e59be66508ee4b", + "sha256": "25446d6c6a43c22ff7104c6712c143f0276bc4749c2642fffac10db1455ef707", "role": "Validates compiled contexts and worker-safe steps before worker setup and builds the worker-lane execution plan." }, { "path": "openhcs/core/orchestrator/worker_lanes.py", - "sha256": "d915c01a0ae2dd010d6b972e275669fef63bb3aefd100f1b66a1e34653673db0", + "sha256": "4273af5b358c7f4cf01a504bb151fdd0117fcc724cf121913cb9ebb5b7695115", "role": "Plans deterministic worker lanes from compiled axis contexts and installs the bundle in workers through ForkInheritedWorkerExecutionState." }, { "path": "openhcs/core/orchestrator/worker_execution.py", - "sha256": "26b9586890e7117b24b6b85660d3f417455d55546401739f91146d18e4d21134", + "sha256": "83ade0271c9b24ca1ba61c708d9e6f60a432b630ec6a8a2fac6d1018458e6be5", "role": "Submits one task per active lane, collects per-axis ExecutionResult values, ships or inherits the bundle, and terminates owned process workers on cancellation." }, { @@ -2671,6 +2731,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/multiprocessing_coordination_system.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compiler.py, orchestrator/bundle_execution.py, worker_lanes.py and worker_execution.py (E1: OrchestratorState and RuntimeObservationMode became class families; K2: artifacts package imports); the page names no orchestrator state or observation mode; PipelineCompiler, ProcessingContext, ExecutionResult, CompiledExecutionBundle and ForkInheritedWorkerExecutionState.require_current are unchanged, so no prose change was needed.", "On 2026-10-11 (B6), re-read docs/source/architecture/multiprocessing_coordination_system.rst in full and compared it against the changed authorities: pipeline/compiler.py (U3 ScopeKey step scopes, K6 progress phase/status classes, CompiledSourceBindingPlan() empty value), orchestrator/bundle_execution.py (lane results now reordered to compiled axis order) and orchestrator/worker_execution.py (K6 progress classes); every numbered step, ForkInheritedWorkerExecutionState, the per-lane task submission, per-axis ExecutionResult collection and the cancellation split still hold; no prose change needed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/multiprocessing_coordination_system.rst against B6's later changes to openhcs/core/pipeline/compiler.py, openhcs/core/orchestrator/bundle_execution.py, openhcs/core/orchestrator/worker_execution.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/multiprocessing_coordination_system.rst against the later B6 changes to openhcs/core/pipeline/compiler.py, openhcs/core/orchestrator/bundle_execution.py, openhcs/core/orchestrator/worker_execution.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", diff --git a/docs/audits/architecture_n_z_development.json b/docs/audits/architecture_n_z_development.json index ecd15cca57..13d3550a27 100644 --- a/docs/audits/architecture_n_z_development.json +++ b/docs/audits/architecture_n_z_development.json @@ -10,12 +10,12 @@ "authority": [ { "path": "openhcs/runtime/napari_viewer_server.py", - "sha256": "038d7a76eeac8d69079591cf6eb7736af5415b28a7a067d6677bd15d115d11c7", - "role": "Detached Napari endpoint: accepted stream batches, layer routing, variable-size display strategies, and dimension-label overlays." + "sha256": "ac54f8bfdce33150c31b38bcd5cf6069b5fd66fb7c05d5a6951610ec3d895bfd", + "role": "Detached Napari endpoint: accepted stream batches, layer routing, the NapariLayerKind family, and dimension-label overlays with their route-source family." }, { "path": "openhcs/runtime/napari_streaming_handlers.py", - "sha256": "be585484075783f7717776d2b8ba163aab8a69aa24c560ca3844214485bd9eef", + "sha256": "b43f29163cef420c77f30e46e92d94145520f7534077d54636a0ab141da32e7e", "role": "Napari route handlers that project typed received batches onto native viewer state." }, { @@ -25,10 +25,11 @@ } ], "findings": [ - "The Napari endpoint and handler rewrites (domain-blind viewers, G6) did not move ownership: the transport contract stays in zmqruntime and the OpenHCS wrappers stay in openhcs/runtime, so the redirect to streaming_boundary_and_wrappers and external_foundations remains correct. Only the endpoint evidence role was refreshed." + "The page is a redirect stub; its link targets still exist and own the content (checked 2026-10-11, B6). The record's napari_viewer_server.py role named variable-size display strategies, which E1 moved to the NapariVariableSizeHandling family in openhcs/runtime/viewer_display.py; the role now names what the endpoint still owns." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/napari_integration_architecture.rst in full and compared it against the changed authorities: openhcs/runtime/napari_viewer_server.py and openhcs/runtime/napari_streaming_handlers.py (E1: NapariDimensionLabelRouteSource and NapariLayerKind became class families in the endpoint; the variable-size display strategies moved onto the NapariVariableSizeHandling family in openhcs/runtime/viewer_display.py); confirmed both redirect targets exist and the endpoint and handlers remain the OpenHCS Napari wrappers; the page text needed no change, and the endpoint evidence role no longer claims the variable-size strategies.", "On 2026-10-11 (B6), re-read docs/source/architecture/napari_integration_architecture.rst in full and compared it against the changed authorities: openhcs/runtime/napari_viewer_server.py and openhcs/runtime/napari_streaming_handlers.py; confirmed both redirect targets exist and the modules remain the OpenHCS Napari wrappers; the page text needed no change.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", "tests/unit/test_napari_streaming_handlers.py and tests/unit/test_viewer_protocol.py exercise the named OpenHCS and transport boundaries", @@ -48,7 +49,7 @@ "authority": [ { "path": "openhcs/core/viewer_streaming_service.py", - "sha256": "c96eca530f767002cfcc9aca4fe75c93d7a5848a7f42df9e5d336bf6712ed51e", + "sha256": "50742cdcb589dc9fba5a177c0107c046dd63c518c5aa77bd8a80e4f69e898e07", "role": "Compiled output streaming parametrized by viewer type, viewer settlement, and lifecycle cleanup." }, { @@ -63,10 +64,11 @@ } ], "findings": [ - "The streaming service and Napari visualiser changes since the last review do not move the transport or wrapper boundary; the redirect to streaming_boundary_and_wrappers and external_foundations remains correct. Evidence roles refreshed only." + "The page is a redirect stub; its link target still exists and owns the content (checked 2026-10-11, B6)." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/napari_streaming_system.rst in full and compared it against the changed authorities: openhcs/core/viewer_streaming_service.py (E1: viewer lifecycle is assigned as a ViewerLifecycle class, ViewerConnectedExternal); confirmed both redirect targets exist and the module remains the OpenHCS streaming service; the page text needed no change.", "On 2026-10-11 (B6 final), compared docs/source/architecture/napari_streaming_system.rst against B6's later changes to openhcs/core/viewer_streaming_service.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/napari_streaming_system.rst against the later B6 changes to openhcs/core/viewer_streaming_service.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/napari_streaming_system.rst in full and compared it against the changed authorities: openhcs/core/viewer_streaming_service.py and openhcs/runtime/napari_stream_visualizer.py; confirmed both redirect targets exist and the modules remain the OpenHCS streaming service and Napari adapter; the page text needed no change.", @@ -97,7 +99,7 @@ }, { "path": "openhcs/core/runtime_plane_projection.py", - "sha256": "3c88ed042a7f1c65c9495a1c62777e008f1e59c4742ec0563b2849d77ca87cdd", + "sha256": "cf644451b6649dcec398fecbf961a4a29ea74f0bb2e0bdab052ec1be8a427f95", "role": "RuntimeSliceProjectableValue ABC on which owned runtime values declare their slice projection." }, { @@ -112,7 +114,7 @@ }, { "path": "openhcs/interop/cellprofiler/module_declarations.py", - "sha256": "b0195b66126c31f71dc0d4bb281a128e0164bb7dfb3b314ee8fe744ebc55d626", + "sha256": "6ff4f4bc3eaf6b4c9fc42752493c2b9ba2e11c8994d7edaea81f586ac57c3b89", "role": "CellProfilerModule declarations owning callable names, catalogue placement, and headless-import eligibility." }, { @@ -124,6 +126,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/nominal_ownership.rst in full and compared it against the changed authorities: openhcs/core/runtime_plane_projection.py (K2/K3: stored_slices, match_alignment and contextualized_output added to RuntimeSliceProjectableValue) and openhcs/interop/cellprofiler/module_declarations.py (E1: InputSource class family); confirmed owned values still subclass RuntimeSliceProjectableValue while RuntimeSliceProjection handles foreign values, ArtifactType is still the artifact-kind root (now in openhcs/core/artifacts/kinds.py), and OpenHCS still imports all three selection mixins from metaclass_registry.strategies after E1; no prose change was needed.", "On 2026-10-11 (B6), re-read docs/source/architecture/nominal_ownership.rst in full and compared it against the changed authorities: openhcs/core/processing_contracts.py (K5: CallableMetadata renamed CallableDeclaration for signature lookup) and interop/cellprofiler/module_declarations.py (K4: SourceBindingsConfig imported from core/config.py; K5: contracts carry artifact inputs/outputs directly); re-checked that the three metaclass_registry.strategies mixins, RegistryConfig/RegistryFamily, RuntimeSliceProjectableValue, RuntimeSliceProjection, ProcessingContract and CellProfilerModule exist with the described roles; the page names none of the renamed symbols, so no prose change was needed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/nominal_ownership.rst against B6's later changes to openhcs/core/processing_contracts.py, openhcs/processing/backends/lib_registry/openhcs_registry.py, openhcs/interop/cellprofiler/module_declarations.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/nominal_ownership.rst against the later B6 changes to openhcs/core/processing_contracts.py, openhcs/processing/backends/lib_registry/openhcs_registry.py, openhcs/interop/cellprofiler/module_declarations.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -225,12 +228,12 @@ "authority": [ { "path": "openhcs/core/orchestrator/bundle_execution.py", - "sha256": "bc3a2d0a50c3db3be81b03f6af4a139c3db2f779e969aad608e59be66508ee4b", + "sha256": "25446d6c6a43c22ff7104c6712c143f0276bc4749c2642fffac10db1455ef707", "role": "Parent compiled-bundle execution: executor shutdown, BrokenProcessPool handling, parent resource release, reverse-order viewer bootstrap rollback, and viewer settlement ordering." }, { "path": "openhcs/core/orchestrator/worker_execution.py", - "sha256": "26b9586890e7117b24b6b85660d3f417455d55546401739f91146d18e4d21134", + "sha256": "83ade0271c9b24ca1ba61c708d9e6f60a432b630ec6a8a2fac6d1018458e6be5", "role": "Inline, process and thread lane resources: per-axis image-cache release, process-local runtime/framework cleanup, and thread-lane deferral to the parent." }, { @@ -240,12 +243,12 @@ }, { "path": "openhcs/runtime/viewer_protocol.py", - "sha256": "c4bdae4bae6dee412f0d27a9cf75a95db53b3bd95c03cee7d046be16b52fe5b8", - "role": "rollback_failed_bootstrap distinguishing connected external viewers from OpenHCS-owned viewer processes." + "sha256": "347a65b63c149f2f437014d537cfaf481a33801d860742f4083b7377494e792f", + "role": "ViewerLifecycle family: ViewerConnectedExternal and ViewerOwnedProcess each implement rollback_failed_bootstrap, releasing an external client or stopping an OpenHCS-owned viewer process." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "358d3eaf5952170f9e9e2dc9056fb08707e8fcf74682063c074b367a83c2648f", + "sha256": "6525348d097bda8d0265bc7c3f312dd8839ca9f65dd05f797fc82ba873e04547", "role": "Framework/device assignment merge and cleanup_loaded used for the compiled framework/device footprint." }, { @@ -260,18 +263,19 @@ }, { "path": "tests/unit/test_orchestrator_lane_planning.py", - "sha256": "85a73ddf071552fc88d454c9de857d066fd62f45492b74945ee22ec908b85289", + "sha256": "4af8c5e5a6d68a64f862a0bcb98f398a0a898948e6f8b5e462a4a11381f19524", "role": "Proves partial viewer acquisition and readiness failures roll back every acquired viewer." }, { "path": "tests/unit/test_runtime_stack_cache_lifetime.py", - "sha256": "a63226b25afc995817b496c1cf153ddd324786cb5e746c49399082b0ad2f573c", + "sha256": "c92b69339b57d51498f2e879ab0d2bf0059efa2cae4fdc1ce91663eb95e4bd31", "role": "Tests context-local image-cache lifetime across success and failure." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/orchestrator_cleanup_guarantees.rst in full and compared it against the changed authorities: openhcs/core/orchestrator/bundle_execution.py and worker_execution.py (E1 class families for orchestrator state and observation mode), openhcs/runtime/viewer_protocol.py (E1: ViewerLifecycleState replaced by the ViewerLifecycle class family), openhcs/core/compiled_step_plan.py and the two test files; traced rollback_failed_bootstrap onto ViewerConnectedExternal (releases the client, leaves the endpoint) and ViewerOwnedProcess (stops the process), which is what the page states; parent and worker cleanup ordering is unchanged; no prose change was needed, and the viewer_protocol.py evidence role now names the lifecycle family.", "On 2026-10-11 (B6), re-read docs/source/architecture/orchestrator_cleanup_guarantees.rst in full and compared it against the changed authorities: orchestrator/bundle_execution.py (K6 progress classes; lane results reordered to compiled axis order), orchestrator/worker_execution.py (K6 progress classes), compiled_step_plan.py (empty plan defaults) and tests/unit/test_orchestrator_lane_planning.py (progress phase name instead of value); none touches executor shutdown, viewer rollback, lane cleanup ownership, image-cache release or the terminal ordering, so no prose change was needed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/orchestrator_cleanup_guarantees.rst against B6's later changes to openhcs/core/orchestrator/bundle_execution.py, openhcs/core/orchestrator/worker_execution.py, openhcs/runtime/viewer_protocol.py, openhcs/core/compiled_step_plan.py, tests/unit/test_orchestrator_lane_planning.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/orchestrator_cleanup_guarantees.rst against the later B6 changes to openhcs/core/orchestrator/bundle_execution.py, openhcs/core/orchestrator/worker_execution.py, openhcs/runtime/viewer_protocol.py, openhcs/core/compiled_step_plan.py, tests/unit/test_orchestrator_lane_planning.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -300,17 +304,17 @@ "authority": [ { "path": "openhcs/core/orchestrator/orchestrator.py", - "sha256": "99870516da6d64e1858f4dd8799d6e38ab6911b7b02bc29b252d2a23f7687410", + "sha256": "a7097eec86c5966552bcb546a2161d1a08e6a4c630e6b156de3b8354b0026b54", "role": "PipelineOrchestrator: __objectstate_delegate__ to pipeline_config, dataset_root identity, and initialize enforcing the source role's availability check." }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "4a33908fb42874e6d57eeea3f5305793ddbda41d13356f0a78501af38dcdfeb4", + "sha256": "40b07b7f40aca39b19b95a8f61c1b0c8270f0ab0a29e8c36caa13f04d8fa539f", "role": "Compiler consuming ObjectState-resolved steps and the resolved global configuration once per compilation." }, { "path": "openhcs/core/dataset_sources/source.py", - "sha256": "350bf5691a03522e9f54b1d8ab712c6beba1683aac4fa5159ea4e80c6041530d", + "sha256": "b3758a878fdf8ec28b6d9818ebfd96c5c470895575f7a79500c85f1490679281", "role": "DatasetSource family and SourceSelectionRole.require_available_source, which owns the local-directory availability contract." }, { @@ -322,6 +326,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/orchestrator_configuration_management.rst in full and compared it against the changed authorities: openhcs/core/orchestrator/orchestrator.py and openhcs/core/pipeline/compiler.py (E1: OrchestratorState became a class family) and openhcs/core/dataset_sources/source.py (get_required_backend deleted); confirmed __objectstate_delegate__ is still pipeline_config, initialize still calls source_selection_role().require_available_source and a failed check sets OrchestratorInitFailed without marking the orchestrator initialized; the page names no state member, so no prose change was needed.", "On 2026-10-11 (B6), re-read docs/source/architecture/orchestrator_configuration_management.rst in full and compared it against the changed authorities: orchestrator.py (K4 import of source_bindings_defaults_to_base from core/config.py), pipeline/compiler.py (U3 ScopeKey compiler scope ids, K6 progress classes), dataset_sources/source.py (K4 import move) and objectstate/object_state.py (U3 library release); confirmed __objectstate_delegate__ = 'pipeline_config', initialize opening the configured DatasetSource and calling its SourceSelectionRole.require_available_source on dataset_root, and once-per-compilation resolution; no prose change needed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/orchestrator_configuration_management.rst against B6's later changes to openhcs/core/orchestrator/orchestrator.py, openhcs/core/pipeline/compiler.py, openhcs/core/dataset_sources/source.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/orchestrator_configuration_management.rst against the later B6 changes to openhcs/core/orchestrator/orchestrator.py, openhcs/core/pipeline/compiler.py, openhcs/core/dataset_sources/source.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -501,12 +506,12 @@ "authority": [ { "path": "openhcs/core/source_bindings.py", - "sha256": "2defaf02ac0e214bb63f74d6eb56f294fbb5db4e06ecd9cc4479b16848f09854", + "sha256": "890473715373d1adc6b12fec2b57eab6f179cbca7d837920c0388d620ce8b8c2", "role": "Typed source-binding declaration vocabulary: bindings, selectors, metadata rules, the SourceSetPairing family (PairByOrder, PairByMetadata) and the compiled plan." }, { "path": "openhcs/core/source_binding_selection.py", - "sha256": "9610bc6112c068cf6c4b0ff7a1265ee4086291042e2dc422d8a90cce3c560eb7", + "sha256": "e837296accf4dd3353956a8e5f267d3fdd4581fb01db1227bc592a2ed6b4d0d3", "role": "Source candidate matching, anchor-pattern policies, and source-universe requests that replaced pattern detection." }, { @@ -516,10 +521,11 @@ } ], "findings": [ - "The record's source_bindings.py role named SourceBindingsConfig, which K4 moved to openhcs/core/config.py, and the binding kinds' pairing is now the SourceSetPairing family (SourceBindingMatchPlan/SourceBindingMatchMethod deleted). The page itself names no module or class, so only the record changed." + "The page is a redirect stub; its link target still exists and owns the content (checked 2026-10-11, B6)." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/pattern_detection_system.rst in full and compared it against the changed authorities: openhcs/core/source_bindings.py and openhcs/core/source_binding_selection.py (E1: source-binding kinds became class families; anchor and match status enums deleted; SourceUniverseRequest keyed by binding origin class); confirmed the source_model target exists and that SourceSetPairing, PairByOrder and PairByMetadata are still declared in source_bindings.py; the page names no module or class, so its text needed no change.", "On 2026-10-11 (B6), re-read docs/source/architecture/pattern_detection_system.rst in full and compared it against the changed authorities: source_bindings.py (K4: config classes moved to core/config.py, SourceSetPairing replaces SourceBindingMatchPlan), source_binding_selection.py, dataset_sources/source_bindings_source.py; the page only links to source_model, which exists; corrected the authority role only.", "On 2026-10-11 (B6 final), compared docs/source/architecture/pattern_detection_system.rst against B6's later changes to openhcs/core/source_binding_selection.py, openhcs/core/dataset_sources/source_bindings_source.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/pattern_detection_system.rst against the later B6 changes to openhcs/core/source_bindings.py, openhcs/core/source_binding_selection.py, openhcs/core/dataset_sources/source_bindings_source.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -539,7 +545,7 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", + "sha256": "9625ca054fac6c1e460216cfe0e85621f5306592a004d1bdc28a6b1ce2099366", "role": "ProcessingConfig field docstrings: variable_components is the stack-axis authority and group_by partitions already assembled values, used for dictionary-pattern branch selection." }, { @@ -549,20 +555,21 @@ }, { "path": "openhcs/core/pipeline/artifact_planning.py", - "sha256": "aea5f172c469aae05b473eebf52fe280367c606cd82c45e5821289eb7ab1ec90", + "sha256": "377558d738198216e9f240af6e19dfd71b5a81d826a2044ec957fa410abcdf92", "role": "Artifact producer/consumer graph and output materialization planning behind the artifact-contract route." }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "74b591773b7f653c957b2db33a1262925e5c02b71074695b6fc18d6120ed5934", + "sha256": "35fe71906bfd16842de347776a0c5cc846550ad490cc3628885129d746ec6f31", "role": "RuntimeValueStore and runtime artifact queries behind the runtime-value route." } ], "findings": [ - "Removed 'canonical' filler from 'Use these canonical pages'. The page's claims are otherwise unaffected: config.py (K4 source-binding config classes, PartitionFilterMode family), processing_contracts.py (CallableDeclaration rename), artifact_planning.py (config.compiled_plan_for_contracts, invocation validation) and runtime_stores.py (hand-written to_dict/from_dict deleted) change nothing it states." + "The page is a redirect stub; its link target still exists and owns the content (checked 2026-10-11, B6)." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/pattern_grouping_and_special_outputs.rst in full and compared it against the changed authorities: openhcs/core/config.py (E1: MaterializationBackend and InputSource class families), openhcs/core/pipeline/artifact_planning.py (K2 artifacts package imports, E1 StepInputDependency subclasses) and openhcs/core/runtime_stores.py (K2 imports); confirmed the ProcessingConfig variable_components and group_by docstrings are unchanged and all four link targets exist; the page text needed no change.", "On 2026-10-11 (B6), re-read docs/source/architecture/pattern_grouping_and_special_outputs.rst in full and compared it against the changed authorities: core/config.py, core/processing_contracts.py, pipeline/artifact_planning.py and core/runtime_stores.py; confirmed the ProcessingConfig docstrings still make variable_components the assembled-array axis and group_by a partition of already assembled values, ProcessingContract and RuntimeValueStore exist, and the four linked pages exist; removed one filler word.", "On 2026-10-11 (B6 final), compared docs/source/architecture/pattern_grouping_and_special_outputs.rst against B6's later changes to openhcs/core/config.py, openhcs/core/processing_contracts.py, openhcs/core/runtime_stores.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/pattern_grouping_and_special_outputs.rst against the later B6 changes to openhcs/core/config.py, openhcs/core/processing_contracts.py, openhcs/core/runtime_stores.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -588,17 +595,17 @@ "authority": [ { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "4a33908fb42874e6d57eeea3f5305793ddbda41d13356f0a78501af38dcdfeb4", + "sha256": "40b07b7f40aca39b19b95a8f61c1b0c8270f0ab0a29e8c36caa13f04d8fa539f", "role": "_resolve_pipeline_once ObjectState resolution, AxisCompilationRequest context projection, stage order, worker assignment, and bundle construction." }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "0f9cc4819b53d55b7e9446429b8c8ac1ff1a59f4902020ea1658495ce01d464c", + "sha256": "4946c221e34ec19957b5af2b1a54359d454ee840fb1f5e1fe2b3b306ac8e5426", "role": "ResolvedPipelineDefinition scope/provenance validation and CompilationSession fields and missing-plan errors." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "358d3eaf5952170f9e9e2dc9056fb08707e8fcf74682063c074b367a83c2648f", + "sha256": "6525348d097bda8d0265bc7c3f312dd8839ca9f65dd05f797fc82ba873e04547", "role": "CompiledStepPlan typed field inventory and per-step framework-device assignment." }, { @@ -608,12 +615,12 @@ }, { "path": "openhcs/core/callable_contract.py", - "sha256": "a8e73c5df804d78baf72710e210cb2ae63156657a074071e11fd868f4b19d3b4", + "sha256": "6089f63e72e1a47fc5f6bc7b443028ac8ec40d934898589fa03b5592c2273ab3", "role": "CallableContract.from_prepared_callable, process-local resolve_runtime_callable cache, and AXIS/PLATE FunctionStepExecutionScope." }, { "path": "openhcs/core/function_patterns.py", - "sha256": "cc8c35f37ea0a3bd939905fe317652a34b3129ee6f47f6f9023b8c459e62df70", + "sha256": "c8004ae1e3f241c4ac6ce0a61c69e3c0d1ecfb5021e1bef5dd0fb1fd1251ca50", "role": "CompiledFunctionInvocation binds its runtime callable at construction and rebuilds it after process transport." }, { @@ -623,25 +630,24 @@ }, { "path": "openhcs/core/orchestrator/bundle_execution.py", - "sha256": "bc3a2d0a50c3db3be81b03f6af4a139c3db2f779e969aad608e59be66508ee4b", + "sha256": "25446d6c6a43c22ff7104c6712c143f0276bc4749c2642fffac10db1455ef707", "role": "Parent ordering of plate-scoped steps, completed-dataset metadata finalization, and viewer settlement." }, { "path": "openhcs/core/steps/function_outputs.py", - "sha256": "fc6176fddfcddfa6801091ce0d861b3ed5ea57d06f07227ed634cc706436e91f", + "sha256": "a8a8fd0297c421a8a9fa6dc0d6d3b8be952a5ba426e7c7811faf70bfc21fe38d", "role": "OpenHCSMetadataTarget declarations, typed projection persistence, and completed-plate deduplication across axes." }, { "path": "tests/unit/test_compilation_session.py", - "sha256": "82f7227730a5285524dc9da39eb2d09f7af7ce1ee13285e5ebb0e7983d7e393b", + "sha256": "f7d5483997bf604f2c4fccaf08c165ae1a7c881079a250add3c98ce02250e2c0", "role": "Regression proof that axis sessions share one resolved pipeline and own their axis plans." } ], - "findings": [ - "Replaced 'duplicate canonical names' with 'duplicate callable names' (filler word). The K5/K4/K6/U3 changes to the authorities (CallableDeclaration split, function-pattern walking moved to AuthoredFunctionPattern, artifact-declaration validation moved onto ArtifactGraph, ScopeKey compiler scopes, progress classes, empty plan defaults) do not change any claim the page makes." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/pipeline_compilation_system.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compiler.py, compilation_session.py, compiled_step_plan.py, callable_contract.py, function_patterns.py, orchestrator/bundle_execution.py, steps/function_outputs.py and tests/unit/test_compilation_session.py (K2: artifacts package imports; E1: OrchestratorState, MaterializationBackend, InputSource, StepInputDependency and RuntimeObservationMode class families); the page names none of the replaced enum members; AXIS/PLATE FunctionStepExecutionScope, _resolve_pipeline_once, AxisCompilationRequest, from_prepared_callable, prepare_in_current_process and registered_callable are unchanged, so no prose change was needed.", "On 2026-10-11 (B6), re-read docs/source/architecture/pipeline_compilation_system.rst in full and compared it against the changed authorities: pipeline/compiler.py, compilation_session.py (graph.validate_invocation_declarations; disabled-function stripping dropped from normalization), compiled_step_plan.py, callable_contract.py (K5), function_patterns.py (AuthoredFunctionPattern, graph-resolved invocation compile), lib_registry/registry_service.py, orchestrator/bundle_execution.py and tests/unit/test_compilation_session.py; re-verified _resolve_pipeline_once, ResolvedPipelineDefinition, CompilationSession, AxisCompilationRequest, FunctionStepTransportAuthority, RegistryService.registered_callable/prepare_in_current_process, CallableContract.from_prepared_callable, resolve_runtime_callable, CompiledFunctionInvocation.runtime_callable, CompiledRuntimeEnvironmentPlan, input/output memory types on CompiledStepPlan, AXIS/PLATE scopes and OpenHCSMetadataTarget; removed one filler word.", "On 2026-10-11 (B6 final), compared docs/source/architecture/pipeline_compilation_system.rst against B6's later changes to openhcs/core/steps/function_outputs.py: metadata documents are written only for backends that host local files (not virtual backends such as OMERO); the image-source guide test reads axis-family spellings. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/pipeline_compilation_system.rst against B6's later changes to openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/compiled_step_plan.py, openhcs/core/callable_contract.py, openhcs/core/function_patterns.py, openhcs/processing/backends/lib_registry/registry_service.py, openhcs/core/orchestrator/bundle_execution.py, openhcs/core/steps/function_outputs.py, tests/unit/test_compilation_session.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", @@ -673,7 +679,7 @@ "authority": [ { "path": "openhcs/authoring/session/session.py", - "sha256": "0c4e0f0979e1793862859e47eefeb6dd9d4b51c99fba20c2ebc6adf11619c3b5", + "sha256": "ced615b580e4625e09873449b29ac67cdea87c8b8065c205f32e6d5921cf31fc", "role": "Session class, MainThread/DispatcherThread, DatasetAccess and Renderer ports, dataset add/delete/split and work/mutation checks." }, { @@ -683,22 +689,22 @@ }, { "path": "openhcs/authoring/session/views.py", - "sha256": "c772d515dfa738363ab403a3e19c9e438c17b0748a3a9ed65c44c416f579a378", + "sha256": "296adbceb7db7a7f7ac3cb09dbe8bb6620dec588a3b593ef2239108298bee5c8", "role": "SessionView family with DatasetListView and PipelineStepsView." }, { "path": "openhcs/authoring/session/events.py", - "sha256": "6f0b27e3c6d43de6f301de4c9bb80b713b433c7e10c351eda5b35c57d91ece98", + "sha256": "3df85cff1c283becb247d9c5cb24fe1441c8dcaf8d2810b560b7398d41834449", "role": "SessionEvent family and event log with subscribe and events-after reads." }, { "path": "openhcs/authoring/session/compile_batch.py", - "sha256": "0cbe5a6b86a0fb5e9b92c4fc45f858c979031ac93594b317b9761c28f5aaf9ab", + "sha256": "3e7397caa5ea1b38c1abc47d363e37fc01b0b601ab1e6b37ed00226f7aa51f42", "role": "Compile batches that check work is allowed and mark datasets compile-pending before connecting." }, { "path": "openhcs/authoring/session/dataset_document.py", - "sha256": "50dc7b601b6a2343c8aaa1ebdaae92d9b4641f7c7ca6a02ade207809e5bdd172", + "sha256": "ddb55e52107819b22d7bb7d18e6310d98eb0600471a722cb789a5a68028d26f3", "role": "DatasetDocumentScope family and payload scope-id check for selected code documents." }, { @@ -728,13 +734,14 @@ }, { "path": "openhcs/mcp/server.py", - "sha256": "e1f236011df715da3d17574c0b80cbf50fcae45e3f82491c22fb57e98c6bb662", + "sha256": "c3dd3a396a3c304341b06c7c813036e779c921fa915060d2d3d2ed594b00b697", "role": "MCP binds the session to DispatcherThread over McpMainThreadDispatcher." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/plate_manager_services.rst in full and compared it against the changed authorities: openhcs/authoring/session/session.py (read in its current form), views.py, events.py, compile_batch.py and dataset_document.py (E1: orchestrator state, terminal status, debug phase and selection scope became class families spelled by name) and openhcs/mcp/server.py (tool schemas spell family defaults by registry key); confirmed Session, MainThread, DispatcherThread, DatasetAccess, Renderer, the SessionView and SessionOperation roots, DatasetDocumentScope, every listed session module, core/dataset_sources/dataset_scopes.py, QtSessionEventRelay and the DispatcherThread binding in MCP and the GUI; no prose change was needed.", "On 2026-10-11 (B6), re-read docs/source/architecture/plate_manager_services.rst in full and compared it against the changed authorities: authoring/session/session.py (K6: debug command and replay mode are now classes such as RunDebugCommand and PersistentPausedWorkerReplay), dataset_document.py (L6: retain_only_authored_paths now from objectstate) and pipelines.py (U3: function scope ids built with ScopeKey); re-checked every module in the ownership map, MainThread/DispatcherThread/DatasetAccess/Renderer, QtSessionEventRelay, openhcs_session_events, McpMainThreadDispatcher, require_work_allowed/require_definition_mutation_allowed and PipelineImporter; none of the changed names appears on the page, so no prose change was needed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/plate_manager_services.rst against B6's later changes to openhcs/authoring/session/session.py: the writability probe makes one exclusive create attempt, the installed demo reports ViewerType.value, the unit session pre-creates its QApplication, and the GUI journey waits for each batch to end. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/plate_manager_services.rst against B6's later changes to openhcs/authoring/session/views.py: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", @@ -836,7 +843,7 @@ }, { "path": "docs/source/architecture/processing_semantics.rst", - "source_sha256": "67a1c1bccacef1b3ccdbc97cfa9a95fec6b52dc0b6dd535d92db3e11876aca59", + "source_sha256": "74a5847ebc8c9e7245fd81c44ec3eaca4229edbd31afa3b81863f5fad926afc0", "audience": [ "Pipeline authors and OpenHCS processing maintainers" ], @@ -845,8 +852,8 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", - "role": "ProcessingConfig variable_components, group_by and input_source field semantics, and StepMaterializationConfig." + "sha256": "9625ca054fac6c1e460216cfe0e85621f5306592a004d1bdc28a6b1ce2099366", + "role": "ProcessingConfig variable_components, group_by and input_source field semantics (input_source chooses an InputSource class), and StepMaterializationConfig." }, { "path": "openhcs/core/processing_contracts.py", @@ -855,7 +862,7 @@ }, { "path": "openhcs/core/callable_contract.py", - "sha256": "a8e73c5df804d78baf72710e210cb2ae63156657a074071e11fd868f4b19d3b4", + "sha256": "6089f63e72e1a47fc5f6bc7b443028ac8ec40d934898589fa03b5592c2273ab3", "role": "CallableDeclaration fields (memory types, allowed group_by roles, stack requirement, processing contract, execution scope, runtime adapter, image mode, ImagePayloadConsumption, artifact declarations) inherited by CallableContract." }, { @@ -865,25 +872,26 @@ }, { "path": "openhcs/core/runtime_plane_projection.py", - "sha256": "3c88ed042a7f1c65c9495a1c62777e008f1e59c4742ec0563b2849d77ca87cdd", + "sha256": "cf644451b6649dcec398fecbf961a4a29ea74f0bb2e0bdab052ec1be8a427f95", "role": "RuntimeSliceProjectableValue: owned values declare slice count and per-slice projection." }, { "path": "openhcs/core/runtime_slice_projection.py", - "sha256": "8f45a17d6aabd4fae1df55e8d3cda2efdd8df2156357c69133f06ccf1cfb2d96", + "sha256": "f59785d901d0d7bcc3ac1fc1be606bd1728a727ea7347cdc1dd4f549cc3193fa", "role": "RuntimeSliceProjection foreign-value handling and the missing-declaration error." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "358d3eaf5952170f9e9e2dc9056fb08707e8fcf74682063c074b367a83c2648f", + "sha256": "6525348d097bda8d0265bc7c3f312dd8839ca9f65dd05f797fc82ba873e04547", "role": "CompiledStepPlan storage of resolved callable facts and framework-local device bindings." } ], "findings": [ - "K5 split callable_contract.py into CallableDeclaration (the declared fields) and CallableContract (declaration bound to the callable, inheriting those fields); the page's statement that a callable's CallableContract declares memory roles, grouping, processing contract, scope, adapter, image mode and artifacts still holds through inheritance, so only the evidence role was updated." + "E1 turned InputSource into a class family (ReadPreviousStep, ReadPipelineStart); the common-mistakes list still said 'a third InputSource enum value'. Now 'a third InputSource choice'." ], - "disposition": "keep", + "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/processing_semantics.rst in full and compared it against the changed authorities: openhcs/core/config.py (E1: InputSource is a class family, ReadPreviousStep and ReadPipelineStart; MaterializationBackend family), callable_contract.py and compiled_step_plan.py (K2 imports), runtime_plane_projection.py and runtime_slice_projection.py (stored_slices, match_alignment, contextualized_output added); corrected the common-mistakes bullet that called InputSource an enum; confirmed the two input-source choices, the four contract classes, RuntimeSliceProjectableValue ownership and the foreign-value role of RuntimeSliceProjection.", "On 2026-10-11 (B6), re-read docs/source/architecture/processing_semantics.rst in full and compared it against the changed authorities: core/config.py (K4 source-binding config classes moved in, PartitionFilterMode family; variable_components, group_by, input_source and StepMaterializationConfig unchanged), processing_contracts.py (CallableMetadata renamed CallableDeclaration), callable_contract.py (K5 declared-field CallableDeclaration, CallableContract subclass) and compiled_step_plan.py (empty plan defaults); confirmed InputSource still has two members, the four contract subclasses and FlexibleContract's bases, FunctionStepExecutionScope and ImagePayloadConsumption; no prose change needed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/processing_semantics.rst against B6's later changes to openhcs/core/config.py, openhcs/core/processing_contracts.py, openhcs/core/callable_contract.py, openhcs/core/compiled_step_plan.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/processing_semantics.rst against the later B6 changes to openhcs/core/config.py, openhcs/core/processing_contracts.py, openhcs/core/callable_contract.py, openhcs/core/compiled_step_plan.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -902,7 +910,7 @@ }, { "path": "docs/source/architecture/progress_runtime_projection_system.rst", - "source_sha256": "76768c7b6ef484ede75b7f3ffbd5bdaad2f930fb04ad7675941b84a5d8d49635", + "source_sha256": "fab87f05006b4b1ce13788ca20e608d6a2d388b38474bd50b1af68bda3a87991", "audience": [ "OpenHCS runtime, debugger, and UI maintainers" ], @@ -936,13 +944,13 @@ }, { "path": "openhcs/core/debug.py", - "sha256": "4a1ed9fb0dcd40beeda9a2a3c90cdb5b2541252590d28281a9fb6f1aa021c510", + "sha256": "311b003ce50b69abbbdd934e1ff53cade27105605b6c3d7c980fc4ac81b8c2a4", "role": "DebugCommandType and DebugBoundary families, command-keyed execution and stop strategies, cursor, session, snapshot, artifact identity, snapshot store, warm-replay and paused-worker declarations." }, { "path": "openhcs/core/debug_session_projection.py", - "sha256": "0a630eb03a83d741a4ed55838adb8ee65dd4fde9be07878aeba5e7a8c86c7378", - "role": "DebugSessionPhaseDeclarationBase phase selection." + "sha256": "a088647748f3f3dace29ace81a6efacf3af32d66fd846417eedd5755b3545868", + "role": "DebugSessionPhase family: for_context picks the matching phase class by priority." }, { "path": "openhcs/authoring/session/progress.py", @@ -956,7 +964,7 @@ }, { "path": "openhcs/core/orchestrator/bundle_execution.py", - "sha256": "bc3a2d0a50c3db3be81b03f6af4a139c3db2f779e969aad608e59be66508ee4b", + "sha256": "25446d6c6a43c22ff7104c6712c143f0276bc4749c2642fffac10db1455ef707", "role": "execute_dataset_scoped_steps and the parent ordering before the pipeline SuccessProgressPhase event." }, { @@ -971,13 +979,11 @@ } ], "findings": [ - "K6 turned the progress enums into class families: channels are now the ProgressStream family and the registry keys by subject, axis and stream; PATTERN_GROUP, AXIS_COMPLETED and SUCCESS are PatternGroupProgressPhase, AxisCompletedProgressPhase and SuccessProgressPhase. The page named the old channel vocabulary and enum members; corrected.", - "K6 merged DebugCommandDeclarationBase into the DebugCommandType family (each command owns apply_to_worker; strategies key by command class) and replaced DebugBoundaryEventDeclarationBase with the DebugBoundary family; boundary outcome and timeline state were deleted as dead. The debugger-ownership bullets named both deleted classes and the deleted outcome/timeline facts; rewritten.", - "DebugRuntimeProjection no longer derives timeline frames (DebugRuntimeProjection.timeline deleted in K6); the projection-path paragraph now says current and last frames.", - "Removed 'Canonical' from the owner-docs heading. Added progress/registry.py as authority for the stream keying the page now states." + "E1 merged DebugSessionPhaseDeclarationBase and the DebugSessionPhase enum into one DebugSessionPhase class family whose for_context selects the phase; the projection-path paragraph named the deleted class. Corrected." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/progress_runtime_projection_system.rst in full and compared it against the changed authorities: openhcs/core/debug.py (K2 imports), openhcs/core/debug_session_projection.py (E1: DebugSessionPhaseDeclarationBase and the DebugSessionPhase enum merged into the DebugSessionPhase class family) and openhcs/core/orchestrator/bundle_execution.py (E1 class families); replaced the deleted DebugSessionPhaseDeclarationBase with DebugSessionPhase.for_context; confirmed the debugger names in openhcs/core/debug.py, execute_dataset_scoped_steps and LiveMeasurementProgressPayload.", "On 2026-10-11 (B6), re-read docs/source/architecture/progress_runtime_projection_system.rst in full and compared it against the changed authorities: core/progress/projection.py, runtime_tree.py, types.py, debug_projection.py, core/debug.py (K6 families), orchestrator/bundle_execution.py and runtime/zmq_progress.py (progress classes); checked ProgressStream/Phase/Status classes, ProgressRegistry's (subject, axis, stream) key, DebugCommandType.apply_to_worker, the command-keyed DebugInvocationExecutionStrategy/DebugStepStopStrategy, DebugBoundary members, DebugRuntimeProjection fields, and that SubjectRuntimeState, state_counts, node kinds, ExecutionProgress, execute_dataset_scoped_steps and the zmqruntime owner pages exist; corrected the progress vocabulary, the debugger-ownership bullets and the frame list.", "On 2026-10-11 (B6 final), compared docs/source/architecture/progress_runtime_projection_system.rst against B6's later changes to openhcs/core/progress/projection.py, openhcs/core/progress/runtime_tree.py, openhcs/core/progress/types.py, openhcs/core/debug.py, openhcs/authoring/session/progress.py, openhcs/core/orchestrator/bundle_execution.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/progress_runtime_projection_system.rst against the later B6 changes to openhcs/core/progress/projection.py, openhcs/core/progress/runtime_tree.py, openhcs/core/progress/types.py, openhcs/core/debug.py, openhcs/authoring/session/progress.py, openhcs/agent/capabilities.py, openhcs/core/orchestrator/bundle_execution.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -998,7 +1004,7 @@ }, { "path": "docs/source/architecture/quick_start.rst", - "source_sha256": "8785a34f2a2723bb302210d4da339932e3c956c808b30476267903ab5f2ce5bf", + "source_sha256": "089e7b6a3d66efef0ac3aa8cbbcf9eaf79198a8ad3e51c88799bfef7deab77b0", "audience": [ "New OpenHCS users, developers, and coding agents" ], @@ -1012,7 +1018,7 @@ }, { "path": "openhcs/core/steps/abstract.py", - "sha256": "9e710b3db635730f88258e5a6e18aed607876e296783022c06798f2bcd03d5e4", + "sha256": "2bffdfe02664c4974797bb6cefbf7f84ebb6912fcc11e026a3f2d8b3da5262b8", "role": "Step constructor accepting processing_config and source_bindings nested declarations." }, { @@ -1022,13 +1028,13 @@ }, { "path": "openhcs/core/config.py", - "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", + "sha256": "9625ca054fac6c1e460216cfe0e85621f5306592a004d1bdc28a6b1ce2099366", "role": "SourceBindingsConfig, StepSourceBindingsConfig and their lazy forms used in the folder example (declared here since K4)." }, { "path": "openhcs/core/source_bindings.py", - "sha256": "2defaf02ac0e214bb63f74d6eb56f294fbb5db4e06ecd9cc4479b16848f09854", - "role": "Selector, filter, metadata-rule, component-selector and NamedSourceBinding types imported by the folder example." + "sha256": "890473715373d1adc6b12fec2b57eab6f179cbca7d837920c0388d620ce8b8c2", + "role": "Selector, NamedSourceBinding, component-selector and metadata-rule types, and the filter-subject, match-type and metadata-source class families (SourceFileName, SourceExtension, TextContains, PathIsImage, FromFileName) the folder example imports." }, { "path": "openhcs/core/dataset_sources/source_bindings_source.py", @@ -1037,7 +1043,7 @@ }, { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "b96f33b866fe6c9f8118ca70fcb7dff8e6a02cb29d79df67230a5e8b97df8002", + "sha256": "5aa8895c9f533603b7ef972202141e2439473f0604b97d66f5a027b8b3cd1c83", "role": "import_cellprofiler_pipeline signature and headless-import rejection of interactive modules." }, { @@ -1052,7 +1058,7 @@ }, { "path": "openhcs/authoring/session/session.py", - "sha256": "0c4e0f0979e1793862859e47eefeb6dd9d4b51c99fba20c2ebc6adf11619c3b5", + "sha256": "ced615b580e4625e09873449b29ac67cdea87c8b8065c205f32e6d5921cf31fc", "role": "Session shared by the desktop application and MCP for progress, cancellation and results." }, { @@ -1062,13 +1068,21 @@ }, { "path": "tests/unit/test_cellprofiler_pipeline_import.py", - "sha256": "0ce4efd85d819c7e9f7fa84ec6f1ac7ec4bda024c9a8d5c7f264fda2730264f0", + "sha256": "67576f37b7cbe0595acea633265f8e14677ca3c1b8204630413330273dc4e58a", "role": "Proves interactive CellProfiler modules fail headless import instead of disappearing." + }, + { + "path": "openhcs/constants/input_source.py", + "sha256": "4a7929e66d11d5ad81c40f080ba8cd23d466797bfa780d59eae0e1aea3ffd9a4", + "role": "InputSource class family; ReadPipelineStart is the choice the folder example assigns." } ], - "findings": [], - "disposition": "keep", + "findings": [ + "E1 replaced the enums the unsupported-folder example used with class families: InputSource.PIPELINE_START, MetadataSource.FILE_NAME, SourceFilterSubject.FILE/EXTENSION and SourceFilterMatchType.CONTAINS/IS_IMAGE no longer exist, so the example raised AttributeError. It now uses ReadPipelineStart, FromFileName, SourceFileName, SourceExtension, TextContains and PathIsImage." + ], + "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/quick_start.rst in full and compared it against the changed authorities: openhcs/core/steps/abstract.py, openhcs/core/config.py, openhcs/core/source_bindings.py, openhcs/interop/cellprofiler/pipeline_import.py, openhcs/authoring/session/session.py and tests/unit/test_cellprofiler_pipeline_import.py (E1: InputSource, MetadataSource, SourceFilterSubject and SourceFilterMatchType became class families); rewrote the folder example to the family classes and executed both self-contained Python blocks against the worktree (they construct without error); confirmed import_cellprofiler_pipeline's signature and return order, the preset paths and the first_use context.", "On 2026-10-11 (B6), re-read docs/source/architecture/quick_start.rst in full and compared it against the changed authorities: pipeline_document.py (rendering now through pycodify generate_python_source, L6), steps/abstract.py and source_bindings.py (K4: SourceBindingsConfig/StepSourceBindingsConfig now declared in core/config.py, SourceSetPairing family), processing_contracts.py (K5 CallableDeclaration rename), dataset_sources/source_bindings_source.py, cellprofiler/pipeline_import.py, authoring/session/session.py (K6 debug command classes) and both tests; imported every name in the page's code examples from the current tree (Lazy*Config from core.config, the selector/filter/metadata types from core.source_bindings, PipelineDocumentCodec.from_values, Pure2DContract, import_cellprofiler_pipeline signature, InputSource.PIPELINE_START, Microscopy.Channel, stack_percentile_normalize, SourceBindingsSource, Session). No claim on the page changed; the authority list now names core/config.py for the source-binding config classes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/quick_start.rst against B6's later changes to openhcs/authoring/session/session.py: the writability probe makes one exclusive create attempt, the installed demo reports ViewerType.value, the unit session pre-creates its QApplication, and the GUI journey waits for each batch to end. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/quick_start.rst against B6's later changes to openhcs/authoring/session/session.py: the session decides in-place initialization by an actual file-creation attempt that leaves nothing behind, its debug-session methods moved into an inherited mixin, the measured CLI treats Windows Ctrl+Break like Ctrl+C, and the Official30 viewer test reads typed payload summaries. No claim this page makes changes.", @@ -1095,23 +1109,28 @@ "authority": [ { "path": "openhcs/napari_roi_manager/widgets/_roi_manager.py", - "sha256": "841a7ead6e0b577ac224526e1bc1c6ca2fcef42ced5420c632e5501befb45ea3", + "sha256": "3234a3735245f225c8035a8d8f12af085cc4e2653e4a7fd49f124808cf01d358", "role": "ROI manager bound to the active native Shapes layer through connect_layer, with observer release on widget destruction." }, { "path": "openhcs/processing/materialization/core.py", - "sha256": "744573f1bafa67b7a81be5af425012ada47741292519863e250b79f1c778b265", + "sha256": "79db4b008483dfb5ac752f3a5da692a941cba7caae25a6ec26085bbdb4bf278f", "role": "ROI output writers, polyline graph-edge writer with its 2D-only rejection, and projected filename resolution." }, { "path": "openhcs/processing/materialization/options.py", - "sha256": "271346eda98bfcabe2614696e1d2499327b84dcd8245f3ad61025cc18ddcae0f", + "sha256": "e7487e9ad44276d4f2880f0d9b03fd6396f8053b9bda5775ab25caaf023554ca", "role": "SpatialGraphROIOptions and SWCOptions declarations." }, { - "path": "openhcs/core/artifacts.py", - "sha256": "65eb0ec5c903366243ca8bd5ea07412e8d34bdb81e6015fa76b3c4c59838733f", - "role": "ArtifactOutputPlan, ArtifactSpecRelation and ObjectArtifactSubjectBinding." + "path": "openhcs/core/artifacts/plans.py", + "sha256": "0d6a10a78f7177273deca4ad07b9bc729a2eb99254164f2606922674b7286b22", + "role": "ArtifactOutputPlan: artifact type, producer, execution scope, materialization options and object_subject_binding." + }, + { + "path": "openhcs/core/artifacts/relations.py", + "sha256": "b1ae8be613fbabc372bd6692b41c5f925d4f968291f0a715bd1473d85cbed000", + "role": "ArtifactSpecRelation family and ObjectArtifactSubjectBinding feature metadata." }, { "path": "external/PolyStore/src/polystore/roi.py", @@ -1120,15 +1139,16 @@ }, { "path": "tests/unit/test_napari_roi_manager_integration.py", - "sha256": "a4b7dee556f0ceccaa212751e29eb8160d3d0c08c741760715ee8372b0c0e927", + "sha256": "15b0417fc6388f932ac6cb8b35102721918b987b68e52caef04b1c6bc5ea9e81", "role": "Regression proof for the native Shapes-layer ROI manager seam." } ], "findings": [ - "Flow defect: 'Spatial graphs have two independent registered projections:' was followed by the unrelated projected-filename paragraph before the two options it introduced; the paragraph now precedes the object-label sentence. Ownership and naming claims were re-verified against the changed widget, materialization core and PolyStore ROI owner and remain accurate." + "K2 split openhcs/core/artifacts.py into a package; the record's authority now names artifacts/plans.py and artifacts/relations.py. The page text names the classes without a module path and is unchanged." ], - "disposition": "revise", + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/roi_system.rst in full and compared it against the changed authorities: openhcs/napari_roi_manager/widgets/_roi_manager.py (E1: RoiTableColumn became a class family), openhcs/processing/materialization/core.py and options.py (E1: MaterializedFilenameIdentity class family; output_plan is now required), the deleted openhcs/core/artifacts.py (K2: split into the openhcs/core/artifacts package) and tests/unit/test_napari_roi_manager_integration.py; confirmed connect_layer, SpatialGraphROIOptions, SWCOptions, ArtifactOutputPlan (artifacts/plans.py) and ArtifactSpecRelation with ObjectArtifactSubjectBinding (artifacts/relations.py); the page names classes without module paths, so no prose change was needed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/roi_system.rst against B6's later changes to openhcs/processing/materialization/core.py, openhcs/core/artifacts.py, tests/unit/test_napari_roi_manager_integration.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/roi_system.rst against the later B6 changes to openhcs/processing/materialization/core.py, openhcs/core/artifacts.py, tests/unit/test_napari_roi_manager_integration.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/roi_system.rst in full and compared it against the changed authorities: napari_roi_manager/widgets/_roi_manager.py (observer release on destroy), processing/materialization/core.py (new ROIOutput/PointROIOutput writers), polystore/roi.py, test_napari_roi_manager_integration.py; verified SpatialGraphROIOptions, SWCOptions, SpatialGraph, ArtifactOutputPlan, ArtifactSpecRelation, ObjectArtifactSubjectBinding and connect_layer exist and that the polyline writer rejects 3D graphs; reordered one misplaced paragraph. Added options.py and core/artifacts.py as authorities for the named declarations.", @@ -1143,7 +1163,7 @@ }, { "path": "docs/source/architecture/runtime_value_system.rst", - "source_sha256": "43f47fa2b2c534958a47c42c887badc4492f83165772503e98ae2a994a18f3e5", + "source_sha256": "26890bf40926be1ebe4cf728c31b565a0b7c5b854896b082e4da78a7fd32040f", "audience": [ "OpenHCS runtime and artifact maintainers" ], @@ -1152,47 +1172,52 @@ "authority": [ { "path": "openhcs/core/runtime_artifact_values.py", - "sha256": "930752071d37ef1a8669812e7a629e151690b3f33bf4a29dba92d1d479b9074d", + "sha256": "ce42e38f893459b9cb0ed6d779fabd5818204301ab1acc13ab0a7df32e6e3a2d", "role": "RuntimeValue, ArtifactKey, execution scope, and the output-normalization entry that calls the artifact type's contextualize_output_from_projector." }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "74b591773b7f653c957b2db33a1262925e5c02b71074695b6fc18d6120ed5934", + "sha256": "35fe71906bfd16842de347776a0c5cc846550ad490cc3628885129d746ec6f31", "role": "RuntimeValueStore recording, replacement, RuntimeArtifactQuery resolution, and observation semantics." }, { "path": "openhcs/core/runtime_plane_projection.py", - "sha256": "3c88ed042a7f1c65c9495a1c62777e008f1e59c4742ec0563b2849d77ca87cdd", - "role": "RuntimeSliceProjectableValue ABC and its indexed, invariant and identity-projectable subclasses." + "sha256": "cf644451b6649dcec398fecbf961a4a29ea74f0bb2e0bdab052ec1be8a427f95", + "role": "RuntimeSliceProjectableValue ABC, its indexed, invariant and identity-projectable subclasses, and the default contextualized_output hook selection." }, { "path": "openhcs/core/runtime_slice_projection.py", - "sha256": "8f45a17d6aabd4fae1df55e8d3cda2efdd8df2156357c69133f06ccf1cfb2d96", - "role": "RuntimeSliceProjection foreign-value list and rejection of undeclared owned values." + "sha256": "f59785d901d0d7bcc3ac1fc1be606bd1728a727ea7347cdc1dd4f549cc3193fa", + "role": "RuntimeSliceProjection foreign-value list, rejection of undeclared owned values, and contextualized_output for foreign returned values." }, { - "path": "openhcs/core/artifacts.py", - "sha256": "65eb0ec5c903366243ca8bd5ea07412e8d34bdb81e6015fa76b3c4c59838733f", - "role": "ArtifactType.contextualize_output root (unchanged) and its image, object-label, measurement, spatial-grid and spatial-graph overrides." + "path": "openhcs/core/artifacts/kinds.py", + "sha256": "216b913efad9ec76707e9f61be445fb6fa1c6601c36ef53e296acd4928604515", + "role": "ArtifactType.contextualize_output root with its per-value hooks, and the image, object-label, measurement, spatial-grid and spatial-graph overrides." + }, + { + "path": "openhcs/core/artifacts/plans.py", + "sha256": "0d6a10a78f7177273deca4ad07b9bc729a2eb99254164f2606922674b7286b22", + "role": "ArtifactSpecRef identity and the source_context_source relation an output plan resolves." }, { "path": "openhcs/core/runtime_image_values.py", - "sha256": "4801bdad5dd2bde9998e108836479923b2bb87023785eeff395936f3471fdf03", + "sha256": "4dd5173cbeaba217bffd28b04c05f6bac3f49711bfd9b5bc7f791364bfbc87c8", "role": "ImagePayload metadata (mask, axes, plane axis, provenance) and contextualize_image_output for image sources." }, { "path": "openhcs/core/runtime_object_labels.py", - "sha256": "a8ca5cc37f7049bb0ed06f1600afab9943d515e5ccb43dac0fb39f8f1bc7fbf7", + "sha256": "0e6f92699f87013550e6426a70bd2e458fab881c4108aa1559cb17121110c284", "role": "ObjectLabelValue.contextualize_image_output plane-axis rule for label-rendered images." }, { "path": "openhcs/core/projected_image_output.py", - "sha256": "84ade707b65fab8b4b66006b8f01dd045920252f5b5cabe58bea8626165aae83", + "sha256": "3870623dae8a400a8f6f0ab9a2b1d4f82ee9c2c71f1e0528d35986e75e13a0c5", "role": "SourceProjectedImageOutput and SelectedPlaneImageOutput source-plane selection." }, { "path": "openhcs/core/callable_contract.py", - "sha256": "a8e73c5df804d78baf72710e210cb2ae63156657a074071e11fd868f4b19d3b4", + "sha256": "6089f63e72e1a47fc5f6bc7b443028ac8ec40d934898589fa03b5592c2273ab3", "role": "CallableContract.resolve_returned_plan_values binding returned positions to exact output plan refs." }, { @@ -1202,7 +1227,7 @@ }, { "path": "tests/unit/test_function_artifact_outputs.py", - "sha256": "cba628c3eeff1176f634ccb388b8aa698b336c4a6ce408a9ec48f09e9bf85510", + "sha256": "6be6e5ad7669f4e10d59f593c52926cc46c4e39ea2294ea67fe6a94996ea877f", "role": "Proves image-output contextualization and that named artifacts bypass unrelated source-axis projection, using admitted artifact graphs." }, { @@ -1211,9 +1236,12 @@ "role": "Proves undeclared value types are rejected and declared slices project without implicit device-to-host conversion." } ], - "findings": [], - "disposition": "keep", + "findings": [ + "After K2 (artifacts package), ArtifactType.contextualize_output no longer goes straight from the artifact type to the source: it asks the returned value's contextualized_output which of the type's hooks applies (contextualize_value, contextualize_image, contextualize_projected_image, contextualize_aligned), and ImageArtifactType.contextualize_image then calls the source's contextualize_image_output. The page described this as a two-stage dispatch on artifact type and source; the paragraph now names the returned value's part." + ], + "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/runtime_value_system.rst in full and compared it against the changed authorities: openhcs/core/artifacts.py (now the openhcs/core/artifacts package, K2), runtime_artifact_values.py, runtime_stores.py, runtime_plane_projection.py, runtime_slice_projection.py, runtime_image_values.py, runtime_object_labels.py, projected_image_output.py, callable_contract.py, tests/unit/test_function_artifact_outputs.py; read ArtifactType.contextualize_output and ImageArtifactType.contextualize_image in artifacts/kinds.py and corrected the function-output paragraph to include the returned value's contextualized_output step; confirmed the five overriding artifact types, source_context_source (artifacts/plans.py), resolve_returned_plan_values, SourceProjectedImageOutput.resolve_source_context and every named class still exist; the other diffs are import moves, new helper methods and the object-label storage enum dropping its payload-shape member data, none of which the page states.", "On 2026-10-11 (B6), re-read docs/source/architecture/runtime_value_system.rst in full and compared it against the changed authorities: openhcs/core/runtime_stores.py, openhcs/core/callable_contract.py, tests/unit/test_function_artifact_outputs.py; runtime_stores dropped to_dict/from_dict on RuntimeArtifactLocation/Address (no page claim relies on them); K5 restructured callable metadata into CallableDeclaration but resolve_returned_plan_values remains on CallableContract; the test changes are fixture-only (admitted_graph, CompiledSourceBindingPlan()). Grepped every class and method the page names (RuntimeArrayPayload through ObjectLabelValue, RuntimeArtifactQuery, RuntimeExecutionAxisScope, ComponentGroupScope, source_context_source); all exist. No correction needed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/runtime_value_system.rst against B6's later changes to openhcs/core/runtime_stores.py, openhcs/core/artifacts.py, openhcs/core/runtime_object_labels.py, openhcs/core/callable_contract.py, tests/unit/test_function_artifact_outputs.py, tests/unit/test_runtime_slice_projection.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/runtime_value_system.rst against the later B6 changes to openhcs/core/runtime_stores.py, openhcs/core/artifacts.py, openhcs/core/runtime_object_labels.py, openhcs/core/callable_contract.py, tests/unit/test_function_artifact_outputs.py, tests/unit/test_runtime_slice_projection.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -1302,7 +1330,7 @@ "authority": [ { "path": "external/python-introspect/src/python_introspect/jsonable.py", - "sha256": "0148e51fb2fc4c7e1c625b088cb0ca56fecd58f9836b9e5da58b75b5eb2d40ee", + "sha256": "c845361d0d14914798d9a1b5329d681300511fc7e95c67508943425716930ebd", "role": "to_jsonable singledispatch encoder: scalars, mappings, sequences, paths, enums, callables by import path, dataclasses, and AutoRegisterMeta types by registry key." }, { @@ -1326,11 +1354,10 @@ "role": "LazyResolutionFormatter and LazyDataclassFormatter: source formatting for lazy configuration dataclasses." } ], - "findings": [ - "The Python-source section did not name pycodify's PythonSourceLiteral and path factoring or objectstate.codegen, which L6 made the owners of value-owned source spelling, path factoring, and lazy-config formatting." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/serialization_boundaries.rst in full and compared it against the changed authorities: external/python-introspect/src/python_introspect/jsonable.py (current file read; no reviewed version in history); confirmed to_jsonable is a singledispatch function with registrations for mappings, sequences and sets, callables, AutoRegisterMeta registered types (projected through the registry key), Enum, Path and JSON scalars plus the dataclass projection, that dataclass_from_mapping is the decoder in dataclass_projection.py, and that PythonSourceLiteral and ImageFileFormat exist; no claim changed.", "On 2026-10-11 (B6), re-read docs/source/architecture/serialization_boundaries.rst in full and compared it against the changed authorities: python_introspect/jsonable.py, python_introspect/dataclass_projection.py, pycodify/core.py; to_jsonable still dispatches scalars, mappings, sequences, paths, enums, callables by import path, dataclasses and AutoRegisterMeta types by registry key, and dataclass_from_mapping is its decoder. L6 moved PythonSourceLiteral and path factoring into pycodify 0.2 (FormatContext.paths) and lazy-config formatting into objectstate.codegen; added those owners to the Python-source section. Unformattable values now raise in pycodify (FallbackFormatter deleted), consistent with the extension rule.", "On 2026-10-11 (B6 final), compared docs/source/architecture/serialization_boundaries.rst against B6's later changes to openhcs/core/image_file_serialization.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/serialization_boundaries.rst against the later B6 changes to openhcs/core/image_file_serialization.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -1408,17 +1435,17 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", + "sha256": "9625ca054fac6c1e460216cfe0e85621f5306592a004d1bdc28a6b1ce2099366", "role": "SourceBindingsConfig and StepSourceBindingsConfig declarations and source_bindings_defaults_to_base." }, { "path": "openhcs/core/source_bindings.py", - "sha256": "2defaf02ac0e214bb63f74d6eb56f294fbb5db4e06ecd9cc4479b16848f09854", + "sha256": "890473715373d1adc6b12fec2b57eab6f179cbca7d837920c0388d620ce8b8c2", "role": "Declaration vocabulary (ImagePlaneSource, selectors, filters, MetadataExtractionRule, NamedSourceBinding, SourceSetPairing/PairByOrder/PairByMetadata) and the compiled binding and universe plans." }, { "path": "openhcs/core/source_binding_workspace.py", - "sha256": "5e7ac439d7ef6cd35a5660d2ad9db24f1890801d8bdf1d82c13b12f8baaa9eb0", + "sha256": "e242d9b932d52c1099756881870f926a273f5ec08663938347580cb4a70e8283", "role": "Source-binding workspace projector, pairing-keyed source-set assemblers and materialization of the virtual workspace." }, { @@ -1433,7 +1460,7 @@ }, { "path": "openhcs/core/source_projection.py", - "sha256": "c51f7c16a9174ed1fda972fc020d2ce0f14dd2005d32929df0bae174878bb735", + "sha256": "a922f3e66d44bc60ad0a31039128c72321f324ae9ed81070e3efe175e2e12414", "role": "SourcePlaneProjection, SourceArtifactProjection, SourceProjectionSet and typed SourceDatasetDiagnostic." }, { @@ -1442,11 +1469,10 @@ "role": "BackendBase.resolve_listed_address and optional physical_source_path versus DataSource.source_path." } ], - "findings": [ - "The declaration vocabulary named SourceBindingMatchPlan, which K4 deleted; pairing is now the SourceSetPairing family (PairByOrder, the default when no method is declared, and PairByMetadata)." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/source_model.rst in full and compared it against the changed authorities: openhcs/core/config.py, openhcs/core/source_bindings.py, openhcs/core/source_binding_workspace.py, openhcs/core/source_projection.py; E1 turned SourceProjectionRole, SourceSetRole and MaterializationBackend into class families (PrimaryPlaneRole, SourceArtifactRole, BroadcastSource, MaterializeToZarr and siblings) and source roots now use absolute_source_path; the page names only SourceProjectionRole as the declared role a reader reconstructs projections from, which still holds, and every other named class, method and field was grepped and exists; no page change.", "On 2026-10-11 (B6), re-read docs/source/architecture/source_model.rst in full and compared it against the changed authorities: core/source_bindings.py (K4: config classes moved to core/config.py, SourceBindingMatchPlan/SourceBindingMatchMethod replaced by SourceSetPairing, plans never None), source_binding_workspace.py (pairing-keyed assemblers, path identity), dataset_sources/source_bindings_source.py and choice.py (import moves only), source_projection.py (source_path_posix); grepped every other named declaration, projection, diagnostic, serializer and PolyStore method in the current tree; replaced the SourceBindingMatchPlan entry with SourceSetPairing and added core/config.py as the config-class authority.", "On 2026-10-11 (B6 final), compared docs/source/architecture/source_model.rst against B6's later changes to openhcs/core/source_binding_workspace.py, openhcs/core/dataset_sources/source_bindings_source.py, openhcs/core/dataset_sources/choice.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/source_model.rst against the later B6 changes to openhcs/core/source_bindings.py, openhcs/core/source_binding_workspace.py, openhcs/core/dataset_sources/source_bindings_source.py, openhcs/core/dataset_sources/choice.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -1467,35 +1493,36 @@ "authority": [ { "path": "openhcs/core/callable_contract.py", - "sha256": "a8e73c5df804d78baf72710e210cb2ae63156657a074071e11fd868f4b19d3b4", + "sha256": "6089f63e72e1a47fc5f6bc7b443028ac8ec40d934898589fa03b5592c2273ab3", "role": "CallableContract input/output declarations and runtime-owned parameter projection." }, { "path": "openhcs/core/pipeline/function_contracts.py", - "sha256": "ffc21dd3021a2316cfbd100c2d655867ce4bec619f2db7288e82aeb11c1930e8", + "sha256": "67a654102b1d60c0a0f90e64ccaf3a8c177035cd11a0346734f872b329783d74", "role": "special_inputs decorator: records parameter names as runtime-managed callable metadata and excludes them from user parameters; no special_outputs decorator exists." }, { "path": "openhcs/core/pipeline/artifact_planning.py", - "sha256": "aea5f172c469aae05b473eebf52fe280367c606cd82c45e5821289eb7ab1ec90", + "sha256": "377558d738198216e9f240af6e19dfd71b5a81d826a2044ec957fa410abcdf92", "role": "ArtifactGraph and ArtifactSpecRef planning during compilation." }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "74b591773b7f653c957b2db33a1262925e5c02b71074695b6fc18d6120ed5934", + "sha256": "35fe71906bfd16842de347776a0c5cc846550ad490cc3628885129d746ec6f31", "role": "RuntimeValueStore recording, replacement, query, and observation during execution." }, { "path": "openhcs/interop/cellprofiler/module_artifact_contracts.py", - "sha256": "4d521778e83fee62a4b164d6a3934d0824f2c5e7375bc7bea1d74256e67d61fe", + "sha256": "cefc83a740f917730f336b7283c6ea2d9280d27bf077e030d41b7a8a9add41af", "role": "CellProfilerModuleArtifactContracts: per-invocation artifact bindings from CellProfiler setting declarations." } ], "findings": [ - "Redirect still correct after K5: every owner the page names still exists with the stated role; no page change needed." + "The page is a redirect stub; its link target still exists and owns the content (checked 2026-10-11, B6)." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/special_io_system.rst in full and compared it against the changed authorities: openhcs/core/callable_contract.py, openhcs/core/pipeline/function_contracts.py, openhcs/core/pipeline/artifact_planning.py, openhcs/core/runtime_stores.py, openhcs/interop/cellprofiler/module_artifact_contracts.py; the diffs are imports from the openhcs/core/artifacts package (K2), artifact_type.value replacing require_value(), and step-input dependency classes; CallableContract, CellProfilerModuleArtifactContracts, ArtifactGraph, RuntimeValueStore and the special_inputs decorator still exist, and artifact_contract_system.rst, the redirect target, exists.", "On 2026-10-11 (B6), re-read docs/source/architecture/special_io_system.rst in full and compared it against the changed authorities: callable_contract.py, function_contracts.py, artifact_planning.py, runtime_stores.py, module_artifact_contracts.py; K5 replaced CallableMetadata with CallableDeclaration (CallableContract now subclasses it); CallableContract, special_inputs, ArtifactGraph, RuntimeValueStore and CellProfilerModuleArtifactContracts all still exist with the roles the page assigns, and no special_outputs decorator exists. No correction needed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/special_io_system.rst against B6's later changes to openhcs/core/pipeline/function_contracts.py: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/special_io_system.rst against B6's later changes to openhcs/core/callable_contract.py, openhcs/core/pipeline/function_contracts.py, openhcs/core/runtime_stores.py, openhcs/interop/cellprofiler/module_artifact_contracts.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", @@ -1577,13 +1604,14 @@ }, { "path": "pyproject.toml", - "sha256": "b5d4d752704b886da20d860743b81dc3d2ce3a3c402208246fadc1c6cb71f597", + "sha256": "75f92010b3c18ede61cf7296064079590c143283871e372a1fd4546417d4e0f9", "role": "Declares polystore and arraybridge as separate runtime dependencies of OpenHCS." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/storage_and_memory_system.rst in full and compared it against the changed authorities: pyproject.toml (current file read; no reviewed version in history); polystore and arraybridge remain declared dependencies, and external_foundations.rst, the page this one points to, exists; no claim changed.", "On 2026-10-11 (B6), re-read docs/source/architecture/storage_and_memory_system.rst in full and compared it against the changed authorities: pyproject.toml; the diff only raises dependency pins (pyqt-reactive 0.5, pycodify 0.2, objectstate 2.1, zmqruntime 0.6.2, python-introspect 0.2.7); polystore and arraybridge remain separate runtime dependencies, and FileManager and the arraybridge converters still exist. No correction needed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/storage_and_memory_system.rst against the later B6 changes to pyproject.toml: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/architecture/storage_and_memory_system.rst in full and compared it against the changed authorities: external/PolyStore/src/polystore/filemanager.py (submodule advance) and pyproject.toml (polystore 0.6, arraybridge 0.4.1, JAX and openhcs-basicpy dependency changes); the redirect makes no version, backend or API claim those changes affect, and both packages remain separate dependencies with intersphinx targets in docs/source/conf.py; no prose change was needed.", @@ -1621,7 +1649,7 @@ "authority": [ { "path": "openhcs/core/viewer_streaming_service.py", - "sha256": "c96eca530f767002cfcc9aca4fe75c93d7a5848a7f42df9e5d336bf6712ed51e", + "sha256": "50742cdcb589dc9fba5a177c0107c046dd63c518c5aa77bd8a80e4f69e898e07", "role": "Compiled output streaming through StreamComponentMessageExtraAuthority and viewer settlement." }, { @@ -1631,22 +1659,22 @@ }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "358d3eaf5952170f9e9e2dc9056fb08707e8fcf74682063c074b367a83c2648f", + "sha256": "6525348d097bda8d0265bc7c3f312dd8839ca9f65dd05f797fc82ba873e04547", "role": "CompiledStepPlan.producer_identity_for_main_flow and producer_identity_for_artifact." }, { "path": "openhcs/core/runtime_object_labels.py", - "sha256": "a8ca5cc37f7049bb0ed06f1600afab9943d515e5ccb43dac0fb39f8f1bc7fbf7", + "sha256": "0e6f92699f87013550e6426a70bd2e458fab881c4108aa1559cb17121110c284", "role": "ObjectLabelValue.contextualize_image_output plane-axis and provenance rule for label-rendered images." }, { "path": "openhcs/core/orchestrator/bundle_execution.py", - "sha256": "bc3a2d0a50c3db3be81b03f6af4a139c3db2f779e969aad608e59be66508ee4b", + "sha256": "25446d6c6a43c22ff7104c6712c143f0276bc4749c2642fffac10db1455ef707", "role": "Viewer bootstrap with rollback of the acquired set, settle-then-read-state order, and stopping of execution-owned viewers." }, { "path": "openhcs/runtime/viewer_protocol.py", - "sha256": "c4bdae4bae6dee412f0d27a9cf75a95db53b3bd95c03cee7d046be16b52fe5b8", + "sha256": "347a65b63c149f2f437014d537cfaf481a33801d860742f4083b7377494e792f", "role": "ManagedViewerLifecycleMixin, ViewerProcessPlatform Linux policy, ViewerSettleProgress phases, and the execution-owned EndpointProcessGroup." }, { @@ -1656,12 +1684,12 @@ }, { "path": "openhcs/runtime/napari_viewer_server.py", - "sha256": "038d7a76eeac8d69079591cf6eb7736af5415b28a7a067d6677bd15d115d11c7", + "sha256": "ac54f8bfdce33150c31b38bcd5cf6069b5fd66fb7c05d5a6951610ec3d895bfd", "role": "NapariImagePayload intake validation, bounded shape projection and preflighted layer isolation in the Napari Qt process." }, { "path": "openhcs/runtime/napari_streaming_handlers.py", - "sha256": "be585484075783f7717776d2b8ba163aab8a69aa24c560ca3844214485bd9eef", + "sha256": "b43f29163cef420c77f30e46e92d94145520f7534077d54636a0ab141da32e7e", "role": "Accepted intake-cycle and route error retention through settlement." }, { @@ -1671,7 +1699,7 @@ }, { "path": "openhcs/runtime/zmq_execution_server.py", - "sha256": "12e96bcd1fb1aaccb43aa3fe90697b5d1cdda8998721a556005e95f344c9b2b3", + "sha256": "709193dcb39298d3ad8bdc7908150740993da161547b6499be9923cb2317f3f1", "role": "Releases execution-session process resources when the execution server stops." }, { @@ -1686,7 +1714,7 @@ }, { "path": "tests/unit/test_viewer_protocol.py", - "sha256": "04aa9e7e8bdc2ce6f3cd0705e531b35672c2369d5c46bb319b2eec17ee21dc75", + "sha256": "3aaf1fea097807f3bcd82c11abf58c134303c499bda49206ad9328900b68f0af", "role": "Proves execution cleanup stops exact non-persistent processes while persistent viewers remain available." }, { @@ -1696,23 +1724,24 @@ }, { "path": "tests/unit/test_napari_streaming_handlers.py", - "sha256": "93244d4d09c5cd6d7a2645fca2482578ad52646f3bb9e6de5ebe505b157fbfea", + "sha256": "c08363837db3efd13534c966a6192b785cb85ccca742b574ac9bd0e74600b80e", "role": "Proves one global ROI-member budget and metadata-only shape projection." }, { "path": "tests/unit/test_napari_accepted_work_settlement.py", - "sha256": "50cde8ceede31c3ef07db0c0aed69865d058617fd77f34d834bb48252a47fcf1", + "sha256": "32c52a4807fddceb25839b75c6a937dcc0c7212437c2c3072066b976cf894306", "role": "Tests accepted display failures, settlement reporting and recovery." }, { "path": "tests/unit/test_orchestrator_lane_planning.py", - "sha256": "85a73ddf071552fc88d454c9de857d066fd62f45492b74945ee22ec908b85289", + "sha256": "4af8c5e5a6d68a64f862a0bcb98f398a0a898948e6f8b5e462a4a11381f19524", "role": "Proves multi-viewer bootstrap rollback after partial acquisition and readiness failure." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/streaming_boundary_and_wrappers.rst in full and compared it against the changed authorities: openhcs/core/viewer_streaming_service.py, compiled_step_plan.py, runtime_object_labels.py, orchestrator/bundle_execution.py, openhcs/runtime/viewer_protocol.py, napari_viewer_server.py, napari_streaming_handlers.py, zmq_execution_server.py and the four changed tests; E1 turned ViewerSettlePhase (SettlementRunning/Complete/Failed), the viewer lifecycle (ViewerStopped/ViewerConnectedExternal/ViewerOwnedProcess), the launch session, NapariLayerKind and OrchestratorState into class families; the page states settlement by its wire phase names complete and failed, which the phase classes keep, and describes owned versus external viewer rollback, which the lifecycle classes implement; confirmed ViewerType is still the enum derived from the ViewerFamily registry, and that ViewerShapePayloadProjection.SUMMARY, ViewerProcessPlatform, ViewerSlotFamily, ViewerControlAction, DeclaredAxis and NapariImagePayload exist; no page change.", "On 2026-10-11 (B6), re-read docs/source/architecture/streaming_boundary_and_wrappers.rst in full and compared it against the changed authorities: compiled_step_plan.py (CompiledSourceBindingPlan/CompiledSourceUniversePlan default factories), orchestrator/bundle_execution.py (K6 ProgressPhase/ProgressStatus classes, lane results ordered by compiled axis), runtime/zmq_execution_server.py (K6 debug replay predicate) and test_orchestrator_lane_planning.py; verified StreamComponentMessageExtraAuthority, StepOutputManifestStore, ArtifactMaterializationTargetPlan, producer_identity_for_main_flow/_for_artifact, StreamProducerIdentity.projection_key, ObjectLabelValue.contextualize_image_output, ViewerFamily/ViewerType, ViewerControlAction, ViewerProcessPlatform, ViewerSettleProgress, EndpointProcessGroup, the viewer control/projection option classes, settle-then-read-state order, server stop cleanup and the bootstrap rollback test all still exist as described; no claim needed correction.", "On 2026-10-11 (B6 final), compared docs/source/architecture/streaming_boundary_and_wrappers.rst against B6's later changes to openhcs/core/viewer_streaming_service.py, openhcs/core/compiled_step_plan.py, openhcs/core/runtime_object_labels.py, openhcs/core/orchestrator/bundle_execution.py, openhcs/runtime/viewer_protocol.py, openhcs/runtime/zmq_execution_server.py, tests/unit/test_viewer_result_element_selection.py, tests/unit/test_napari_streaming_handlers.py, tests/unit/test_napari_accepted_work_settlement.py, tests/unit/test_orchestrator_lane_planning.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/streaming_boundary_and_wrappers.rst against the later B6 changes to openhcs/core/viewer_streaming_service.py, openhcs/core/compiled_step_plan.py, openhcs/core/runtime_object_labels.py, openhcs/core/orchestrator/bundle_execution.py, openhcs/runtime/viewer_protocol.py, openhcs/runtime/zmq_execution_server.py, tests/unit/test_viewer_result_element_selection.py, tests/unit/test_napari_streaming_handlers.py, tests/unit/test_napari_accepted_work_settlement.py, tests/unit/test_orchestrator_lane_planning.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -1759,7 +1788,7 @@ }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "4a33908fb42874e6d57eeea3f5305793ddbda41d13356f0a78501af38dcdfeb4", + "sha256": "40b07b7f40aca39b19b95a8f61c1b0c8270f0ab0a29e8c36caa13f04d8fa539f", "role": "PipelineCompiler phases from declaration resolution to framework device assignment." }, { @@ -1769,10 +1798,11 @@ } ], "findings": [ - "Redirect still correct: the target page covers the current flow; no page change needed." + "The page is a redirect stub; its link target still exists and owns the content (checked 2026-10-11, B6)." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/system_integration.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compiler.py; the diff swaps enum members for the E1 class families (OrchestratorCompiled, MaterializeToZarr, PrimaryPlaneRole); the page states nothing about them, and system_overview.rst, the redirect target, exists.", "On 2026-10-11 (B6), re-read docs/source/architecture/system_integration.rst in full and compared it against the changed authorities: openhcs/core/pipeline_document.py, openhcs/core/pipeline/compiler.py; PipelineDocument now renders through pycodify generate_python_source and the compiler uses ScopeKey and the K6 progress classes; the redirect to system_overview still describes the current flow. No correction needed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/system_integration.rst against B6's later changes to openhcs/core/pipeline/compiler.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/system_integration.rst against the later B6 changes to openhcs/core/pipeline/compiler.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -1801,12 +1831,12 @@ }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "0f9cc4819b53d55b7e9446429b8c8ac1ff1a59f4902020ea1658495ce01d464c", + "sha256": "4946c221e34ec19957b5af2b1a54359d454ee840fb1f5e1fe2b3b306ac8e5426", "role": "ResolvedPipelineDefinition (resolved steps, scope ids, field provenance) and the per-axis CompilationSession." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "358d3eaf5952170f9e9e2dc9056fb08707e8fcf74682063c074b367a83c2648f", + "sha256": "6525348d097bda8d0265bc7c3f312dd8839ca9f65dd05f797fc82ba873e04547", "role": "CompiledStepPlan typed fields, including framework-local device assignment." }, { @@ -1821,17 +1851,17 @@ }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "74b591773b7f653c957b2db33a1262925e5c02b71074695b6fc18d6120ed5934", + "sha256": "35fe71906bfd16842de347776a0c5cc846550ad490cc3628885129d746ec6f31", "role": "RuntimeValueStore recording, replacement, query, and observation semantics." }, { "path": "openhcs/core/runtime_plane_projection.py", - "sha256": "3c88ed042a7f1c65c9495a1c62777e008f1e59c4742ec0563b2849d77ca87cdd", + "sha256": "cf644451b6649dcec398fecbf961a4a29ea74f0bb2e0bdab052ec1be8a427f95", "role": "RuntimeSliceProjectableValue: value families own their slice projection." }, { "path": "external/python-introspect/src/python_introspect/jsonable.py", - "sha256": "0148e51fb2fc4c7e1c625b088cb0ca56fecd58f9836b9e5da58b75b5eb2d40ee", + "sha256": "c845361d0d14914798d9a1b5329d681300511fc7e95c67508943425716930ebd", "role": "python-introspect's to_jsonable JSON encoder." }, { @@ -1865,11 +1895,10 @@ "role": "Generic configuration window that OpenHCSConfigWindow subclasses." } ], - "findings": [ - "The package-owner list predated U3 and L6: it omitted ObjectState's ScopeKey grammar and lazy-config source formatting and pyqt-reactive's configuration windows, editable tables, grid selection and time-travel views." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/system_overview.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compilation_session.py, compiled_step_plan.py, runtime_stores.py, runtime_plane_projection.py, external/python-introspect/src/python_introspect/jsonable.py; the diffs are imports from the openhcs/core/artifacts package, UnresolvedStepInput as the default step-input dependency, and new slice-projection helper methods; CompilationSession, the CompiledStepPlan fields, CompiledExecutionBundle, RuntimeValueStore and to_jsonable/dataclass_from_mapping match the page; no page change.", "On 2026-10-11 (B6), re-read docs/source/architecture/system_overview.rst in full and compared it against the changed authorities: pipeline_document.py, compilation_session.py, compiled_step_plan.py, runtime_stores.py, python_introspect/jsonable.py; the flow, ResolvedPipelineDefinition, CompilationSession, CompiledStepPlan typed fields and RuntimeValueStore claims still hold (diffs: generate_python_source, graph.validate_invocation_declarations, CompiledSourceBindingPlan() defaults). Updated the package-owner list for U3/L6: ObjectState now owns ScopeKey and objectstate.codegen, and pyqt-reactive owns configuration windows, editable tables, grid selection and time travel.", "On 2026-10-11 (B6 final), compared docs/source/architecture/system_overview.rst against B6's later changes to openhcs/core/pipeline/compilation_session.py, openhcs/core/compiled_step_plan.py, openhcs/core/runtime_stores.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/system_overview.rst against the later B6 changes to openhcs/core/pipeline/compilation_session.py, openhcs/core/compiled_step_plan.py, openhcs/core/runtime_stores.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -1910,12 +1939,12 @@ "authority": [ { "path": "pyproject.toml", - "sha256": "b5d4d752704b886da20d860743b81dc3d2ce3a3c402208246fadc1c6cb71f597", + "sha256": "75f92010b3c18ede61cf7296064079590c143283871e372a1fd4546417d4e0f9", "role": "No Textual dependency or entry point; openhcs-gui gui-script points at the PyQt application." }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "3cdb7819f09604046ba69e0b4f57b73e4e98d8f68af80158501a54bb42e701a2", + "sha256": "2d10806e310cb0d47ee7e47b879a7a95b28dcf5c688d9ce2a54b81ae1a041c47", "role": "PyQt desktop main window composition." }, { @@ -1925,10 +1954,11 @@ } ], "findings": [ - "Redirect still correct: no Textual interface in source or distribution; no page change needed." + "The page is a redirect stub; its link target still exists and owns the content (checked 2026-10-11, B6)." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/tui_system.rst in full and compared it against the changed authorities: pyproject.toml, openhcs/pyqt_gui/main.py; pyproject.toml declares no Textual dependency, main.py only swapped selection and update-check enum members for class families, and docs/source/user_guide/index.rst and docs/archive exist.", "On 2026-10-11 (B6), re-read docs/source/architecture/tui_system.rst in full and compared it against the changed authorities: pyproject.toml, openhcs/pyqt_gui/main.py, tests/unit/pyqt_gui/test_ui_config_lifecycle.py; pyproject.toml still has no Textual dependency and the openhcs-gui gui-script points at openhcs.pyqt_gui.__main__:main; main.py changes are U3 moves of generic Qt machinery; the lifecycle test still guards that openhcs/textual_tui does not return (its shortcut tests moved to pyqt-reactive); docs/archive still exists. No correction needed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/tui_system.rst against B6's later changes to openhcs/pyqt_gui/main.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/tui_system.rst against the later B6 changes to pyproject.toml, openhcs/pyqt_gui/main.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -1970,7 +2000,7 @@ "authority": [ { "path": "openhcs/pyqt_gui/config.py", - "sha256": "a3eece2e5b2926fc6fc0edd197221da451ab7c0d0d6f4ffc8bb2ff865ad74bd4", + "sha256": "d092cb20206127484fbbc80a38a5152462b9b8991e848a814c85f564d51f4db8", "role": "Embeds the persisted generic list-preview policy in UIConfig." }, { @@ -2005,7 +2035,7 @@ }, { "path": "openhcs/pyqt_gui/app.py", - "sha256": "3257f148dfddac3f1ff7409881db8100273adac5c3564b9cd3745e8f75313086", + "sha256": "3816c79689a29c22d2e4098622c0d3242c334517b201f159f30a728ea0fb322f", "role": "Owns application execution and native window-tree destruction before QApplication teardown." }, { @@ -2025,12 +2055,12 @@ }, { "path": "openhcs/desktop/update.py", - "sha256": "7bd5ab7abffdc20a9f6f85fb2e5f169ecda462e50b210b175d918453dbebfb3c", + "sha256": "23ffce8fe2362d8651ea90d2ef9601d193658e4fcef6aef8964e888f43014a3c", "role": "DesktopRestartSession capture and restoration shared by restart, update and version-mismatch recovery." }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "3cdb7819f09604046ba69e0b4f57b73e4e98d8f68af80158501a54bb42e701a2", + "sha256": "2d10806e310cb0d47ee7e47b879a7a95b28dcf5c688d9ce2a54b81ae1a041c47", "role": "OpenHCSMainWindow.set_ui_config consumer reconciliation and rollback, and preview-policy propagation to both managers." }, { @@ -2085,7 +2115,7 @@ }, { "path": "openhcs/pyqt_gui/widgets/plate_manager.py", - "sha256": "e816cdafd4795c07b5d177a04472221057230416311d1bf83c5bf65188d2aa00", + "sha256": "b6e56da089cff746c9f0f41ca4b4d947942d4a71d1b290a1d8b76c3b7732c31e", "role": "Plate Manager renders the Session dataset list, holds no session state, and forwards transport and progress settings to the session." }, { @@ -2104,13 +2134,10 @@ "role": "ShortcutLifecycle rejects duplicate keys before rebinding QActions and the application shortcut filter." } ], - "findings": [ - "The main-window shortcut lifecycle moved to pyqt-reactive (U3): MainWindowShortcutLifecycle was replaced by the generic ShortcutLifecycle and its application shortcut filter; the page now names that owner.", - "The configuration window moved to pyqt-reactive (U3): ConfigWindow there owns the two-tab save transaction and OpenHCSConfigWindow only supplies the OpenHCS document format and window ids; the page now says so and the authorities name both files.", - "test_ui_config_lifecycle.py no longer holds the shortcut-lifecycle tests (moved with the lifecycle); its role was narrowed to the UIConfig cache path and session transport/progress projection." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/ui_services_architecture.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/config.py, openhcs/pyqt_gui/app.py, openhcs/desktop/update.py, openhcs/pyqt_gui/main.py, openhcs/pyqt_gui/widgets/plate_manager.py; E1 turned the startup-readiness state, the update-check origin (ExplicitUpdateCheck/StartupUpdateCheck), the selection scope (SelectedItems/AllItems) and OrchestratorState into class families; the page names none of them and its statements on OpenHCSPyQtApp shutdown, UIConfig fields, set_ui_config, the update-check preference and Plate Manager's session rendering are unaffected; every named class was grepped and exists; no page change.", "On 2026-10-11 (B6), re-read docs/source/architecture/ui_services_architecture.rst in full and compared it against the changed authorities: pyqt_gui/services/window_handlers.py (ScopeKey separator, OpenHCSConfigWindow over pyqt-reactive ConfigWindow, session event connection), tests/unit/pyqt_gui/test_ui_config_lifecycle.py (shortcut tests removed), pyqt_gui/main.py (pyqt-reactive ShortcutLifecycle, DockLayoutStore/DockPane, TimeTravelCommands/TimeTravelNavigator) and widgets/plate_manager.py (GuiEventBusBroadcaster removed, PathCacheKey and external-editor policy from pyqt-reactive); corrected the shortcut-lifecycle and configuration-window transaction owners and checked the save transaction order in pyqt-reactive ConfigWindow.save_config against the page's rollback description.", "On 2026-10-11 (B6 final), compared docs/source/architecture/ui_services_architecture.rst against B6's later changes to openhcs/pyqt_gui/services/ui_bridge_session_actions.py, openhcs/pyqt_gui/main.py, tests/unit/pyqt_gui/test_main_config_propagation.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/ui_services_architecture.rst against the later B6 changes to openhcs/pyqt_gui/services/ui_bridge_session_actions.py, openhcs/pyqt_gui/main.py, tests/unit/pyqt_gui/test_main_config_propagation.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -2180,12 +2207,12 @@ }, { "path": "openhcs/runtime/zmq_execution_server.py", - "sha256": "12e96bcd1fb1aaccb43aa3fe90697b5d1cdda8998721a556005e95f344c9b2b3", + "sha256": "709193dcb39298d3ad8bdc7908150740993da161547b6499be9923cb2317f3f1", "role": "OpenHCS compiled-execution adapter over the generic execution server." }, { "path": "openhcs/runtime/zmq_execution_observation.py", - "sha256": "3cc43d3eee6c4701c23fec183ecd895a71acdfcc14acfe2afca62625ccb7381b", + "sha256": "db40a0d1e82bdbe778f948619e62a93d74c16d3155ebc701a067bbba164f0efa", "role": "Owns the ordinary value/outcome export schemas, producing execution identity, and exclusive evidence-file creation." }, { @@ -2230,7 +2257,7 @@ }, { "path": "openhcs/runtime/zmq_execution_client.py", - "sha256": "5df650f60c4d48cbf47c275d4e46b870b35c915ca8baa8e06e77f321578ca6d7", + "sha256": "ad7ea01599493dbefce1f3c91e6b082e932c6a5183d0a85a186f7bd7b62779a9", "role": "Spawns the execution server with the ArrayBridge-projected child environment and carries typed execution transport." }, { @@ -2240,17 +2267,17 @@ }, { "path": "openhcs/authoring/session/submission.py", - "sha256": "710ac218d1db01f231b67bf0fe229dc03fb428bc6655b23671f3a026be91c673", + "sha256": "e88dba60a818ce44d0ecf744b918f8a82421ccd0d9a6a5aebce7c15bb331f1dc", "role": "ExecutionSubmission records each accepted submission with its request and endpoint and follows status to terminal." }, { "path": "openhcs/authoring/session/session.py", - "sha256": "0c4e0f0979e1793862859e47eefeb6dd9d4b51c99fba20c2ebc6adf11619c3b5", + "sha256": "ced615b580e4625e09873449b29ac67cdea87c8b8065c205f32e6d5921cf31fc", "role": "Session owns the one execution client and keeps FinishedExecution (request, endpoint, ExecutionRecord) per execution." }, { "path": "openhcs/authoring/session/execution_control.py", - "sha256": "1ee26c917eb106809092f9cb9f906ee8f5e32bbb53700826b369646879eb2544", + "sha256": "40e8e6bf11dc90707ea8f11ebbae4f54d2cec4721ee341fdcad394db24dddf5e", "role": "Session stop: endpoint shutdown (graceful or forced) and cancellation of active datasets." }, { @@ -2355,7 +2382,7 @@ }, { "path": "openhcs/processing/custom_functions/runtime_registry.py", - "sha256": "160fb2fbad1b4d2f1ffaf784bacc5d7355d78c2720e718083cfc8c05748384df", + "sha256": "d2311617cad39d96ceb3ea6e13b6ad89f3b846e95d5dc3d692e7a9a46dd9c739", "role": "Atomic process-local projection of persisted custom-function declarations without a copied global catalogue." }, { @@ -2382,6 +2409,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/zmq_execution_service_extracted.rst in full and compared it against the changed authorities: openhcs/runtime/zmq_execution_server.py, zmq_execution_observation.py, zmq_execution_client.py, openhcs/authoring/session/submission.py, session.py, execution_control.py, openhcs/processing/custom_functions/runtime_registry.py; E1 made the runtime-observation export scope a class family that builds its own export, and execution status, terminal status and custom-function lifetime became families; the page describes the scope as a typed value-or-outcome scope carried by ZMQAuxiliaryExecutionParams and both exports as carrying the execution ID, which still holds; no page change.", "On 2026-10-11 (B6), re-read docs/source/architecture/zmq_execution_service_extracted.rst in full and compared it against the changed authorities: runtime/zmq_execution_server.py and zmq_execution_client.py (K6 debug request codecs and replay predicate), external/zmqruntime messages.py (current file: EndpointApplication.compatibility_with, ExecutionRecord, ExecutionStatus), authoring/session/session.py (debug command classes), lib_registry/registry_service.py (contract read from reference.declaration) and test_cellprofiler_processing_backend.py (runtime_bound_parameter_names); verified EndpointConnectionAttempt, EndpointConnectionCancelledError, TransportEndpoint, OperationDeadline, ExecutionSubmission, ZMQAuxiliaryExecutionParams, FunctionCatalogPreparation, --prepare-capabilities, OperationCancellation and execution_subject_id still exist as described; no claim needed correction.", "On 2026-10-11 (B6 final), compared docs/source/architecture/zmq_execution_service_extracted.rst against B6's later changes to openhcs/authoring/session/session.py: the writability probe makes one exclusive create attempt, the installed demo reports ViewerType.value, the unit session pre-creates its QApplication, and the GUI journey waits for each batch to end. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/zmq_execution_service_extracted.rst against B6's later changes to openhcs/authoring/session/session.py: the session decides in-place initialization by an actual file-creation attempt that leaves nothing behind, its debug-session methods moved into an inherited mixin, the measured CLI treats Windows Ctrl+Break like Ctrl+C, and the Official30 viewer test reads typed payload summaries. No claim this page makes changes.", @@ -2464,7 +2492,7 @@ }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "3cdb7819f09604046ba69e0b4f57b73e4e98d8f68af80158501a54bb42e701a2", + "sha256": "2d10806e310cb0d47ee7e47b879a7a95b28dcf5c688d9ce2a54b81ae1a041c47", "role": "Projects the configured endpoint snapshot into its dedicated right-hand indicator, and routes endpoint termination to the client lifecycle owner." }, { @@ -2501,6 +2529,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/architecture/zmq_server_browser_system.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/main.py; the diff swaps selection and update-check enum members for class families and touches no endpoint-snapshot, status-bar or server-browser code; the listed module paths exist and every named class was grepped; no page change.", "On 2026-10-11 (B6), re-read docs/source/architecture/zmq_server_browser_system.rst in full and compared it against the changed authorities: pyqt_gui/main.py (shortcut, dock and time-travel machinery moved to pyqt-reactive; endpoint indicator projection unchanged) and core/progress/runtime_tree.py (K6 ProgressStream classes replace channel helpers); verified every named class (ZMQServerBrowserWidgetABC, EndpointObservationSnapshot, RuntimeExecutionTopology, RuntimeTreeProjectionBuilder, ProgressTreeBuilder, ExecutionProgressProjection, ServerRowPresenter, LiveServerTreeSync, EndpointShutdownService, ZMQClientService, ProgressUIConfig, EndpointClientSession), the primary module paths, and mean/explicit percent node policies on subject/worker vs partition/step/compilation nodes; no claim needed correction.", "On 2026-10-11 (B6 final), compared docs/source/architecture/zmq_server_browser_system.rst against B6's later changes to openhcs/pyqt_gui/main.py, openhcs/core/progress/runtime_tree.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/zmq_server_browser_system.rst against the later B6 changes to openhcs/pyqt_gui/main.py, openhcs/core/progress/runtime_tree.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -2532,7 +2561,7 @@ "authority": [ { "path": "openhcs/mcp/client_registration.py", - "sha256": "55b701bb77a3de19243fa9a6276f784fad8329e2d435fb26fcc7fdce1e060731", + "sha256": "4d6fc5f744617ab1d0e9b8317d07965496aaa6b098a4d179ef95cc6baec35671", "role": "Declares McpClientRegistrationTarget.__registry__ keyed by target_id." }, { @@ -2547,7 +2576,7 @@ }, { "path": "openhcs/mcp/installed_demo.py", - "sha256": "10fccd5b2d4382dca19857c0f7b9eaefe4eccbc34b4ff1898e8fced689a90223", + "sha256": "f89ba094072c46c1ed5504c7b4e786aa3806776615330f4e32f2b07f19555100", "role": "Declares the deterministic fixture request (1x1 grid, 192 px tiles, 10% overlap, 1 px stage error, 2 wavelengths, 1 Z, 12 cells, 0.95 shared, A01, ImageXpress, seed 1)." }, { @@ -2569,6 +2598,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/agent_workflow_validation.rst in full and compared it against the changed authorities: openhcs/mcp/client_registration.py (E1: ClientRegistrationStatus and the runtime platform became class families and the report is projected with to_jsonable; McpClientRegistrationTarget.__registry__ is still keyed by target_id) and openhcs/mcp/installed_demo.py (compares the terminal status with ExecutionCompleted.name; the deterministic fixture request is unchanged); the fixture table, registry rule and evidence schema still hold.", "On 2026-10-11 (B6 final), compared docs/source/development/agent_workflow_validation.rst against B6's later changes to openhcs/mcp/installed_demo.py: the writability probe makes one exclusive create attempt, the installed demo reports ViewerType.value, the unit session pre-creates its QApplication, and the GUI journey waits for each batch to end. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/development/agent_workflow_validation.rst against B6's later changes to openhcs/mcp/installed_demo.py: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/development/agent_workflow_validation.rst against B6's later changes to tests/unit/test_installed_mcp_demo.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", @@ -2618,7 +2648,7 @@ }, { "path": "docs/source/development/callable_artifact_authoring.rst", - "source_sha256": "7a7502b1ad080497bf5e1490c6a1159e87a94a4b479250c9b02d7dd3cd7fbaa8", + "source_sha256": "6b216b639ff38eeebcf8bd72e40e75fe8508d445689fbd55f0dbfcfb7e6b869f", "audience": [ "OpenHCS callable and artifact extension authors" ], @@ -2627,13 +2657,23 @@ "authority": [ { "path": "openhcs/core/callable_contract.py", - "sha256": "a8e73c5df804d78baf72710e210cb2ae63156657a074071e11fd868f4b19d3b4", + "sha256": "6089f63e72e1a47fc5f6bc7b443028ac8ec40d934898589fa03b5592c2273ab3", "role": "Declares CallableDeclaration (inherited by CallableContract) with required_axis_roles, allowed_group_by_roles, memory roles and prepared signatures, processing_prepare, resolve_returned_output, the return partition and the artifact-parameter binding errors quoted by the page." }, { - "path": "openhcs/core/artifacts.py", - "sha256": "65eb0ec5c903366243ca8bd5ea07412e8d34bdb81e6015fa76b3c4c59838733f", - "role": "Declares ArtifactSpec, MainFlowStackOutputSpec, the artifact types, ObjectMeasurementSubjectRelation and MaterializationSourceIdentityRelation." + "path": "openhcs/core/artifacts/specs.py", + "sha256": "c4c50309a195bf85e1c0641413544d9581fc0ce1197043130126f7a4c70d8c4e", + "role": "Declares ArtifactSpec with its input() and output() constructors and MainFlowStackOutputSpec." + }, + { + "path": "openhcs/core/artifacts/kinds.py", + "sha256": "216b913efad9ec76707e9f61be445fb6fa1c6601c36ef53e296acd4928604515", + "role": "Declares the ArtifactType members used in the examples (image, object labels, measurements, special) and runtime_parameter_types()." + }, + { + "path": "openhcs/core/artifacts/relations.py", + "sha256": "b1ae8be613fbabc372bd6692b41c5f925d4f968291f0a715bd1473d85cbed000", + "role": "Declares ObjectMeasurementSubjectRelation and MaterializationSourceIdentityRelation." }, { "path": "openhcs/core/processing_contracts.py", @@ -2642,27 +2682,27 @@ }, { "path": "openhcs/core/pipeline/function_contracts.py", - "sha256": "ffc21dd3021a2316cfbd100c2d655867ce4bec619f2db7288e82aeb11c1930e8", + "sha256": "67a654102b1d60c0a0f90e64ccaf3a8c177035cd11a0346734f872b329783d74", "role": "Declares artifact_inputs, artifact_outputs, special_inputs (which writes the artifact-fed parameter names), execution_scope and runtime_bound_parameters." }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "74b591773b7f653c957b2db33a1262925e5c02b71074695b6fc18d6120ed5934", + "sha256": "35fe71906bfd16842de347776a0c5cc846550ad490cc3628885129d746ec6f31", "role": "Declares RuntimeArtifactBatch.records and StoredRuntimeValue for plate-scope callables." }, { "path": "openhcs/core/orchestrator/bundle_execution.py", - "sha256": "bc3a2d0a50c3db3be81b03f6af4a139c3db2f779e969aad608e59be66508ee4b", + "sha256": "25446d6c6a43c22ff7104c6712c143f0276bc4749c2642fffac10db1455ef707", "role": "Requires a plate-scope callable to declare exactly one SpecialArtifactType output." }, { "path": "openhcs/processing/materialization/core.py", - "sha256": "744573f1bafa67b7a81be5af425012ada47741292519863e250b79f1c778b265", + "sha256": "79db4b008483dfb5ac752f3a5da692a941cba7caae25a6ec26085bbdb4bf278f", "role": "Point ROI writer: rejects missing fields, duplicate labels, non-finite coordinates and an empty archive." }, { "path": "openhcs/interop/cellprofiler/module_artifact_contracts.py", - "sha256": "4d521778e83fee62a4b164d6a3934d0824f2c5e7375bc7bea1d74256e67d61fe", + "sha256": "cefc83a740f917730f336b7283c6ea2d9280d27bf077e030d41b7a8a9add41af", "role": "Declares CellProfilerModuleArtifactContracts resolving setting-backed specs into the invocation CallableContract." }, { @@ -2672,7 +2712,7 @@ }, { "path": "tests/unit/test_artifact_spec_collection.py", - "sha256": "60ba79abffde80987809e2ef0621983e798780ad21cd4e98db6b29f21372c697", + "sha256": "46b3367620866438509896fc33bf196f262078f84a6b976612f104ce228148e4", "role": "Proves role-neutral artifact specs, exact refs, relations and collection validation." }, { @@ -2682,10 +2722,11 @@ } ], "findings": [ - "The contract summary listed a preparation hook among the contract's fields; K5 removed prepare from the declaration, so the hook stays on the callable (processing_prepare) and the contract carries only the prepared signatures. Reworded." + "K2 replaced the module openhcs/core/artifacts.py with the package openhcs/core/artifacts/. The four code blocks that imported from the old module were repointed by K2 to openhcs.core.artifacts.kinds, .specs and .relations after the last review, and the audit record still named the deleted module as an authority; the record now names specs.py, kinds.py and relations.py." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/callable_artifact_authoring.rst in full and compared it against the changed authorities: openhcs/core/artifacts.py (deleted by K2; replaced as authority by openhcs/core/artifacts/specs.py, kinds.py and relations.py), openhcs/core/callable_contract.py, openhcs/core/pipeline/function_contracts.py, openhcs/core/runtime_stores.py, openhcs/core/orchestrator/bundle_execution.py, openhcs/processing/materialization/core.py, openhcs/interop/cellprofiler/module_artifact_contracts.py and tests/unit/test_artifact_spec_collection.py; executed all six code blocks in order against this tree (imports from the artifacts package modules, the direct-call assertions, the points, diagnostic, input and plate declarations) and they pass; confirmed the quoted 'does not accept object_labels artifact payloads', 'no exact artifact declaration' and 'trailing return count' messages, canonical_return_output_specs, trailing_return_output_specs, runtime_parameter_types and the plate rule of exactly one SpecialArtifactType output still exist. No prose edit was needed in this round.", "On 2026-10-11 (B6), re-read docs/source/development/callable_artifact_authoring.rst in full and compared it against the changed authorities: callable_contract.py (K5 CallableDeclaration, prepare removed, repeated_return_slots), processing_contracts.py, function_contracts.py (special_inputs now writes artifact_input_parameter_names), runtime_stores.py (K6 codec removal), bundle_execution.py (progress classes, lane results in compiled axis order), module_artifact_contracts.py (UniqueReturnSlots) and tests/unit/test_callable_contract.py; corrected the preparation-hook claim, confirmed by a direct call that special_inputs('labels') alone still fails with 'has no exact artifact declaration binding', and re-checked canonical/trailing_return_output_specs, RuntimeArtifactBatch.records, StoredRuntimeValue and the single SpecialArtifactType plate output.", "On 2026-10-11 (B6 final), compared docs/source/development/callable_artifact_authoring.rst against B6's later changes to openhcs/core/pipeline/function_contracts.py: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/development/callable_artifact_authoring.rst against B6's later changes to openhcs/core/callable_contract.py, openhcs/core/artifacts.py, openhcs/core/processing_contracts.py, openhcs/core/pipeline/function_contracts.py, openhcs/core/runtime_stores.py, openhcs/core/orchestrator/bundle_execution.py, openhcs/processing/materialization/core.py, openhcs/interop/cellprofiler/module_artifact_contracts.py, tests/unit/test_callable_contract.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", @@ -2716,22 +2757,22 @@ "authority": [ { "path": "openhcs/interop/cellprofiler/module_declarations.py", - "sha256": "b0195b66126c31f71dc0d4bb281a128e0164bb7dfb3b314ee8fe744ebc55d626", + "sha256": "6ff4f4bc3eaf6b4c9fc42752493c2b9ba2e11c8994d7edaea81f586ac57c3b89", "role": "CellProfiler module registry, catalogue placement, and headless-import eligibility." }, { "path": "openhcs/interop/cellprofiler/module_artifact_contracts.py", - "sha256": "4d521778e83fee62a4b164d6a3934d0824f2c5e7375bc7bea1d74256e67d61fe", + "sha256": "cefc83a740f917730f336b7283c6ea2d9280d27bf077e030d41b7a8a9add41af", "role": "Invocation-specific CellProfiler artifact contract derivation." }, { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "b96f33b866fe6c9f8118ca70fcb7dff8e6a02cb29d79df67230a5e8b97df8002", + "sha256": "5aa8895c9f533603b7ef972202141e2439473f0604b97d66f5a027b8b3cd1c83", "role": "Public CellProfiler pipeline lowering and declaration-owned headless-import validation." }, { "path": "tests/unit/test_cellprofiler_pipeline_import.py", - "sha256": "0ce4efd85d819c7e9f7fa84ec6f1ac7ec4bda024c9a8d5c7f264fda2730264f0", + "sha256": "67576f37b7cbe0595acea633265f8e14677ca3c1b8204630413330273dc4e58a", "role": "Proves interactive CellProfiler modules fail headless import instead of disappearing from the lowered pipeline." }, { @@ -2753,6 +2794,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/cellprofiler_module_authoring.rst in full and compared it against the changed authorities: openhcs/interop/cellprofiler/module_declarations.py and pipeline_import.py (E1: the main-flow anchor is an InputSource class, ReadPreviousStep or ReadPipelineStart), module_artifact_contracts.py (K2: artifact_type.value in derived names and messages) and tests/unit/test_cellprofiler_pipeline_import.py (same renames); for_callable_contract, declared_function_names, validate_pipeline_import, module_blocks_for_invocation and SettingToKeywordBinding.input/output still exist as the page describes.", "On 2026-10-11 (B6), re-read docs/source/development/cellprofiler_module_authoring.rst in full and compared it against the changed authorities: module_declarations.py, module_artifact_contracts.py, pipeline_import.py and tests/unit/test_cellprofiler_pipeline_import.py (K4 moved SourceBindingsConfig/StepSourceBindingsConfig imports to openhcs.core.config and replaced SourceBindingMatchMethod.ORDER with PairByOrder; K5 put artifact_inputs/outputs directly on CallableContract and added repeated_return_slots through UniqueReturnSlots); the page names only SourceBindingsConfig, CellProfilerModule hooks, CellProfilerModuleArtifactContracts, SettingToKeywordBinding.input()/output(), module_blocks_for_invocation() and ModuleBlock, all re-checked in the current tree, and its claim that outputs are declared independently of return position still holds.", "On 2026-10-11 (B6 final), compared docs/source/development/cellprofiler_module_authoring.rst against B6's later changes to openhcs/interop/cellprofiler/module_declarations.py, openhcs/interop/cellprofiler/module_artifact_contracts.py, openhcs/interop/cellprofiler/pipeline_import.py, openhcs/processing/backends/lib_registry/openhcs_registry.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/development/cellprofiler_module_authoring.rst against the later B6 changes to openhcs/interop/cellprofiler/module_declarations.py, openhcs/interop/cellprofiler/module_artifact_contracts.py, openhcs/interop/cellprofiler/pipeline_import.py, openhcs/processing/backends/lib_registry/openhcs_registry.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -2779,22 +2821,22 @@ "authority": [ { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "0f9cc4819b53d55b7e9446429b8c8ac1ff1a59f4902020ea1658495ce01d464c", + "sha256": "4946c221e34ec19957b5af2b1a54359d454ee840fb1f5e1fe2b3b306ac8e5426", "role": "ResolvedPipelineDefinition and CompilationSession fields, one-time artifact admission before axis fan-out, and the missing-plan failure of plan(index)." }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "4a33908fb42874e6d57eeea3f5305793ddbda41d13356f0a78501af38dcdfeb4", + "sha256": "40b07b7f40aca39b19b95a8f61c1b0c8270f0ab0a29e8c36caa13f04d8fa539f", "role": "Compilation order, declaration resolution, callable warm-up, framework-local device assignment, and bundle construction." }, { "path": "openhcs/core/pipeline/path_planner.py", - "sha256": "6ff14c63768ed5cbd77e84236b3980d62f616465e9ecfb7cdf27c63d8070a405", + "sha256": "cf31311ffa462d0cbd8eb8cf426d75d5ee2f5668de7b7291ca6787b5d5be118a", "role": "Compiled source satisfaction, artifact edge, path, grouping, and plan decisions." }, { "path": "tests/unit/test_compilation_session.py", - "sha256": "82f7227730a5285524dc9da39eb2d09f7af7ce1ee13285e5ebb0e7983d7e393b", + "sha256": "f7d5483997bf604f2c4fccaf08c165ae1a7c881079a250add3c98ce02250e2c0", "role": "Regression proof that axis sessions share one resolved pipeline definition while owning their own plans." }, { @@ -2804,7 +2846,7 @@ }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "358d3eaf5952170f9e9e2dc9056fb08707e8fcf74682063c074b367a83c2648f", + "sha256": "6525348d097bda8d0265bc7c3f312dd8839ca9f65dd05f797fc82ba873e04547", "role": "Typed per-step framework-device binding authority." }, { @@ -2826,6 +2868,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/compiler_extension.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compilation_session.py (K2 import only), openhcs/core/pipeline/compiler.py (E1 class families), openhcs/core/pipeline/path_planner.py and openhcs/core/compiled_step_plan.py (E1: StepInputDependency is a class family, UnresolvedStepInput/PipelineStartInput/StepOutputInput, whose members answer the directory defaults) and tests/unit/test_compilation_session.py (same renames); the session invariants, plan(index) failure and exact ArtifactSpecRef planning are unchanged, and the page names no dependency-kind member.", "On 2026-10-11 (B6), re-read docs/source/development/compiler_extension.rst in full and compared it against the changed authorities: compilation_session.py (disabled items dropped by normalisation; invocation declarations validated by the artifact graph), compiler.py (ScopeKey scope ids, progress phase/status classes, an empty CompiledSourceBindingPlan instead of None), path_planner.py (inject_metadata over the ArtifactGraph, source_bindings.compiled_plan), compiled_step_plan.py (empty plan defaults) and tests/unit/test_compilation_session.py (PairByOrder, admitted_graph); the page's ResolvedPipelineDefinition/CompilationSession invariants, plan(index) failure, typed-plan guidance and ArtifactSpecRef identity still hold.", "On 2026-10-11 (B6 final), compared docs/source/development/compiler_extension.rst against B6's later changes to openhcs/core/pipeline/compilation_session.py, openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/path_planner.py, tests/unit/test_compilation_session.py, openhcs/core/compiled_step_plan.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/development/compiler_extension.rst against the later B6 changes to openhcs/core/pipeline/compilation_session.py, openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/path_planner.py, tests/unit/test_compilation_session.py, openhcs/core/compiled_step_plan.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -2851,7 +2894,7 @@ "authority": [ { "path": "openhcs/agent/services/object_state_field_help_service.py", - "sha256": "5aa83508fa68cb69160d34ca64b495f5a34a401f358ce69ce290e24c96371a4c", + "sha256": "5f8bbcd9ebc9d4361a7907900ce1df578ec489fe4ccc8907ea4fdb0f987158da", "role": "Declaration-reflected ObjectState field help and configuration schema projection." }, { @@ -2880,11 +2923,10 @@ "role": "Proves every scenario owns a valid representative static projection, stable-window figures derive from registered UI identities, documentation-only scenarios do not expand the website cards, and Sphinx branding resolves from package authority." } ], - "findings": [ - "The record listed docs/audits/README.md as an authority; it is documentation, and scripts/validate_docs.py already owns the enforced ledger and project.readme rules the page states, so it was dropped. Page text needed no change." - ], + "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/documentation_authoring.rst in full and compared it against the changed authorities: openhcs/agent/services/object_state_field_help_service.py (E1: the parameter supplier is a class family, SuppliedByAgent/SuppliedAtRuntime, replacing FunctionParameterSource); field help is still reflected from the declarations, and the four verification test files and openhcs_describe_config_schema still exist.", "On 2026-10-11 (B6), re-read docs/source/development/documentation_authoring.rst in full and compared it against the changed authorities: object_state_field_help_service.py (UI-bridge scope listing through UiBridgeService.invoke; JsonValue from python_introspect), knowledge_base_service.py (comparison-manifest parsing moved to openhcs.agent.knowledge_manifest_schema), scripts/validate_docs.py (path regex now accepts .jsonl), scripts/gallery_catalog.py (SessionOperation actions) and tests/unit/test_build_website.py. Confirmed openhcs_describe_config_schema, the openhcs-gallery directive, the napari-roi-navigation scenario, the documentation publication target, the project.readme gate, reference/configuration.rst and all four listed test files exist; no page claim changed. Dropped the docs/audits/README.md authority as a documentation file.", "scripts/validate_docs.py validated the final RST source and its concrete repository paths", "Config-help and knowledge-service tests provide the listed projection gates; the audit validator enforces exact active-source coverage and SHA-256 freshness.", @@ -2913,7 +2955,7 @@ }, { "path": "pyproject.toml", - "sha256": "b5d4d752704b886da20d860743b81dc3d2ce3a3c402208246fadc1c6cb71f597", + "sha256": "75f92010b3c18ede61cf7296064079590c143283871e372a1fd4546417d4e0f9", "role": "Current Python support, dependency, package discovery, and entry-point authority." }, { @@ -2923,12 +2965,12 @@ }, { "path": "external/python-introspect/src/python_introspect/jsonable.py", - "sha256": "0148e51fb2fc4c7e1c625b088cb0ca56fecd58f9836b9e5da58b75b5eb2d40ee", + "sha256": "c845361d0d14914798d9a1b5329d681300511fc7e95c67508943425716930ebd", "role": "singledispatch to_jsonable projection for dataclasses, mappings, sequences, callables, registered types, enums, paths and scalars, paired with dataclass_from_mapping." }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "0f9cc4819b53d55b7e9446429b8c8ac1ff1a59f4902020ea1658495ce01d464c", + "sha256": "4946c221e34ec19957b5af2b1a54359d454ee840fb1f5e1fe2b3b306ac8e5426", "role": "ResolvedPipelineDefinition and CompilationSession consumed by compiler phases." }, { @@ -2938,12 +2980,12 @@ }, { "path": "openhcs/core/callable_contract.py", - "sha256": "a8e73c5df804d78baf72710e210cb2ae63156657a074071e11fd868f4b19d3b4", + "sha256": "6089f63e72e1a47fc5f6bc7b443028ac8ec40d934898589fa03b5592c2273ab3", "role": "OpenHCS callable declaration projection for input, output, and execution frameworks." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "358d3eaf5952170f9e9e2dc9056fb08707e8fcf74682063c074b367a83c2648f", + "sha256": "6525348d097bda8d0265bc7c3f312dd8839ca9f65dd05f797fc82ba873e04547", "role": "Typed per-step framework-device binding authority." }, { @@ -2953,7 +2995,7 @@ }, { "path": "benchmark/cellprofiler_benchmark_cli.py", - "sha256": "2e0de28a96c57ee150e28fbbed28d876507e9281075a03f729838f3ca3610966", + "sha256": "0c2cab4d989d50149cb314e908baace8aa794953c86ebc572fa120beaa695780", "role": "Owns registered benchmark command declarations and the derived installed parser." }, { @@ -2970,6 +3012,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/extension_workflows.rst in full and compared it against the changed authorities: pyproject.toml (foundation pins raised; same eight distributions), external/python-introspect/src/python_introspect/jsonable.py (read in full at 0.2.8: singledispatch to_jsonable still registers dataclasses, mappings, sequences, callables, AutoRegisterMeta classes, enums, paths and scalars, paired with dataclass_from_mapping), openhcs/core/pipeline/compilation_session.py and openhcs/core/callable_contract.py (K2 imports), openhcs/core/compiled_step_plan.py (E1 StepInputDependency classes) and benchmark/cellprofiler_benchmark_cli.py (E1: observation scope choices come from the ZMQRuntimeObservationExportScope registry; BenchmarkCliCommand unchanged); every owner named in the table still exists and metaclass-registry still ships the enum-, type- and context-keyed strategy mixins.", "On 2026-10-11 (B6), re-read docs/source/development/extension_workflows.rst in full and compared it against the changed authorities: pyproject.toml (first-party floors only), external/python-introspect/src/python_introspect/jsonable.py (current file: singledispatch registrations for dataclasses, mappings, sequences, callables, AutoRegisterMeta families, enums, paths and scalars, plus registered_class_from_key), openhcs/core/pipeline/compilation_session.py (validation moved onto ArtifactGraph), openhcs/core/callable_contract.py (K5: CallableMetadata folded into CallableContract declared fields; input/output/execution memory types remain) and openhcs/core/compiled_step_plan.py (CompiledSourceBindingPlan() default). Rechecked every class the owner table and prose names (CallableContract, CellProfilerModule, ImageFileFormat, ResolvedPipelineDefinition, CompilationSession, CompiledExecutionBundle, CompiledStepPlan device bindings, BenchmarkCliCommand, ComparisonSuiteRunDeclaration/Receipt, MeasuredPipelineRunReceipt) and that pycodify still owns Python source generation after L6 (ObjectState adds only the lazy-config formatter); no claim changes.", "On 2026-10-11 (B6 final), compared docs/source/development/extension_workflows.rst against B6's later changes to benchmark/cellprofiler_benchmark_cli.py: the session decides in-place initialization by an actual file-creation attempt that leaves nothing behind, its debug-session methods moved into an inherited mixin, the measured CLI treats Windows Ctrl+Break like Ctrl+C, and the Official30 viewer test reads typed payload summaries. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/development/extension_workflows.rst against B6's later changes to openhcs/core/pipeline/compilation_session.py, openhcs/core/callable_contract.py, openhcs/core/compiled_step_plan.py, benchmark/cellprofiler_benchmark_cli.py, tests/unit/test_benchmark_control.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", @@ -3017,7 +3060,7 @@ }, { "path": "pyproject.toml", - "sha256": "b5d4d752704b886da20d860743b81dc3d2ce3a3c402208246fadc1c6cb71f597", + "sha256": "75f92010b3c18ede61cf7296064079590c143283871e372a1fd4546417d4e0f9", "role": "Current Python support, dependency, package discovery, and entry-point authority." }, { @@ -3027,15 +3070,14 @@ }, { "path": "openhcs/utils/environment.py", - "sha256": "2d8738e593b3e92c966089772624f47d17d38bb27fb90974707af659c91e8ad8", + "sha256": "1985ffbe42b5dfb283b78b6282f9b56e08cd51d5a9e79f2dd4eebffcb6da0bf5", "role": "OpenHCSProcessEnvironment.flag_enabled, which tests/conftest.py uses to read OPENHCS_CPU_ONLY (1, true, yes or on) before deciding whether to load pytest-qt." } ], - "findings": [ - "The record's tests/conftest.py role said the harness accepts only 'true' for OPENHCS_CPU_ONLY; conftest now reads the flag through OpenHCSProcessEnvironment.cpu_only_mode, which accepts 1, true, yes or on. The page's OPENHCS_CPU_ONLY=true commands remain correct; the authority now names openhcs/utils/environment.py, which owns that parsing." - ], + "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/git_worktree_testing.rst in full and compared it against the changed authorities: pyproject.toml (foundation pins raised; the dev and gui extras used by the install command still exist) and openhcs/utils/environment.py (AgentRuntimePlatformAuthority renamed AgentRuntimePlatform in the Numba cache helper; flag_enabled still accepts 1, true, yes or on); the install order, OPENHCS_CPU_ONLY usage and --it-* option names are unchanged.", "On 2026-10-11 (B6), re-read docs/source/development/git_worktree_testing.rst in full and compared it against the changed authorities: pyproject.toml (first-party floors moved to zmqruntime 0.6.2, pycodify 0.2, objectstate 2.1.1, python-introspect 0.2.7, pyqt-reactive 0.5); the page states no version floors. Rechecked the eight editable installs against .gitmodules, the dev and gui extras, and the integration options file; corrected the stale conftest role (OPENHCS_CPU_ONLY parsing now lives in OpenHCSProcessEnvironment).", "On 2026-10-11 (B6 final), compared docs/source/development/git_worktree_testing.rst against B6's later changes to tests/conftest.py: the writability probe makes one exclusive create attempt, the installed demo reports ViewerType.value, the unit session pre-creates its QApplication, and the GUI journey waits for each batch to end. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/development/git_worktree_testing.rst against B6's later changes to tests/conftest.py: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", @@ -3075,7 +3117,7 @@ "authority": [ { "path": "pyproject.toml", - "sha256": "b5d4d752704b886da20d860743b81dc3d2ce3a3c402208246fadc1c6cb71f597", + "sha256": "75f92010b3c18ede61cf7296064079590c143283871e372a1fd4546417d4e0f9", "role": "Current Python support, dependency, package discovery, and entry-point authority." }, { @@ -3087,6 +3129,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/index.rst in full and compared it against the changed authorities: pyproject.toml (read the current file: foundation pins raised to zmqruntime 0.6.2, objectstate 2.1.1, python-introspect 0.2.8 and polystore 0.6.1, mahotas kept in the core dependencies); the page gives no version and every toctree target still exists.", "On 2026-10-11 (B6), re-read docs/source/development/index.rst in full and compared it against the changed authorities: pyproject.toml (first-party dependency floors only); the page states no versions. Confirmed every toctree target and related-architecture link exists and the recursive-clone guidance matches .gitmodules; no claim changes.", "On 2026-10-11 (B6 final), compared docs/source/development/index.rst against the later B6 changes to pyproject.toml: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/index.rst in full and compared it against the changed authorities: pyproject.toml (first-party dependency floors, JAX extra, openhcs-basicpy) and openhcs/core/compiled_execution.py; the page states neither versions nor compiled-bundle facts. Confirmed every toctree target exists and the recursive-clone guidance matches .gitmodules; dropped the unrelated compiled_execution.py and self-referential page authorities.", @@ -3189,7 +3232,7 @@ }, { "path": "openhcs/mcp/dev_client_core.py", - "sha256": "41739b744b0e4287ca7d14cd133371d669e9bff12f4b622313996ae9bb5597da", + "sha256": "1bb405ee541cdb48b27c355aeb71ad680ec8c14f9fd31089814e7ac01751b89b", "role": "MCP dev-client session, selected-workflow argument projection, revision-token terminal-state criteria with idempotent initialization, and transport timeout." }, { @@ -3219,12 +3262,12 @@ }, { "path": "openhcs/agent/services/ui_bridge_service.py", - "sha256": "c7bdf8b5caec1c21c986cdc2224524839fadfa8764dafcd05125d90c72351899", + "sha256": "a2d682ad7841a7d85173f792961cae2dc5546fb4fcf39caa85f0bbd6ed062b01", "role": "Declares the default UI bridge connection spec whose timeout bounds UI development commands." }, { "path": "openhcs/mcp/dev_client_commands/ui.py", - "sha256": "be3bc9096e4a711cc6583667596ba25a60428711f2883248e82d5ac66d6dd28b", + "sha256": "e5eec92b43c20d91f83681e505afa94ca733e809b1c732f81952757b345d3cff", "role": "Owns declaration-derived UI development commands, including selected-workflow ordering and the typed output-directory projection for window snapshots." }, { @@ -3239,17 +3282,17 @@ }, { "path": "openhcs/authoring/session/session.py", - "sha256": "0c4e0f0979e1793862859e47eefeb6dd9d4b51c99fba20c2ebc6adf11619c3b5", + "sha256": "ced615b580e4625e09873449b29ac67cdea87c8b8065c205f32e6d5921cf31fc", "role": "Session submission budget and runtime-observation export validation (writable root, no existing path, outcomes require a path)." }, { "path": "openhcs/agent/dto/session.py", - "sha256": "6d954139237672eddae155f3ece1dfefef4d4e4ce20b1934ae388860538627bc", + "sha256": "730cbcfcda6c7d52a1c6e1d142ec2e7d721f32c51bb78c845a8a03201da70ba0", "role": "Typed run request fields runtime_observation_export_path, runtime_observation_export_scope and submit_timeout_ms, and event_sequence results." }, { "path": "tests/unit/agent/test_mcp_server.py", - "sha256": "4e6f6b1761b2c0316c99984baad6ea2c97e5c76c3f155111b8e498d7358e3175", + "sha256": "7f47844b6fc0bf2461dfdb97e4b622a8d16f0f8c00004c1e6b7ea1a9997abe82", "role": "Regression proof for selected-surface preflight, stale-row rejection, idempotent initialization, and every declared or malformed bridge-receipt continuation outcome." }, { @@ -3274,7 +3317,7 @@ }, { "path": "openhcs/runtime/zmq_execution_client.py", - "sha256": "5df650f60c4d48cbf47c275d4e46b870b35c915ca8baa8e06e77f321578ca6d7", + "sha256": "ad7ea01599493dbefce1f3c91e6b082e932c6a5183d0a85a186f7bd7b62779a9", "role": "Owns the total headless submission budget and the pre-dispatch timeout boundary documented for MCP developers." }, { @@ -3289,17 +3332,17 @@ }, { "path": "openhcs/pyqt_gui/services/ui_bridge_server.py", - "sha256": "4b0664fa62d24557294324b631ee3e39052c84e578d40ec408197bce5adcedac", + "sha256": "f93a3570a834eca8596139676629dda48c3f22991fd8cb58cd03427af8b24a38", "role": "Rejects stale MCP request contracts before they reach live UI state." }, { "path": "tests/unit/pyqt_gui/test_ui_agent_bridge.py", - "sha256": "67cdad5133dc83ccb005b58cc56535ae89e77fc23d3ebd76e616c0da5f567b44", + "sha256": "422be9298e5120c898bd7110884ca770f1f033f07fd69be4ac0caa978d3014f4", "role": "Proves stale protocol and application versions fail before UI operation dispatch." }, { "path": "openhcs/mcp/server.py", - "sha256": "e1f236011df715da3d17574c0b80cbf50fcae45e3f82491c22fb57e98c6bb662", + "sha256": "c3dd3a396a3c304341b06c7c813036e779c921fa915060d2d3d2ed594b00b697", "role": "Owns the mcp_server_stale guard, declaration-derived progress execution, and verbose delayed thread-stack diagnostics across the full invocation." }, { @@ -3311,6 +3354,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/mcp_development.rst in full and compared it against the changed authorities: openhcs/mcp/dev_client_core.py, openhcs/mcp/dev_client_commands/ui.py, openhcs/agent/services/ui_bridge_service.py, openhcs/agent/dto/session.py, openhcs/authoring/session/session.py, openhcs/mcp/server.py, openhcs/runtime/zmq_execution_client.py, openhcs/pyqt_gui/services/ui_bridge_server.py and the two test modules (E1: poll status, orchestrator state, field filter, selection scope and export scope became class families); the page names no enum member; the selected-workflow wait policy, the session-operation tool names, the submit_timeout_ms budget, and the runtime-observation export checks (writable root, no existing path, `values`/`outcomes` spellings, scope.check_export_path before submission) still hold in the current files, so nothing was changed.", "On 2026-10-11 (B6), re-read docs/source/development/mcp_development.rst in full and compared it against the changed authorities: openhcs/authoring/session/session.py, openhcs/runtime/zmq_execution_client.py and tests/unit/pyqt_gui/test_ui_agent_bridge.py (K6: debug commands, boundaries and progress phases/statuses became class families; debug control requests send request.to_dict() directly; U3: DockPane and ConfigCodeDocumentDriver moved to pyqt-reactive). The page names none of those types; rechecked the openhcs-mcp-dev entry point, every dev-client command and option it shows, the SessionOperation operation_ids behind the six named tools, openhcs_session_events/openhcs_session_datasets, execution_submission_timeout_ms on OpenHCSZMQConfig's zmqruntime base, UIConfigCacheEnvironment, the bridge/cache environment variables and OPENHCS_MCP_VERBOSE; no claim changes.", "On 2026-10-11 (B6 final), compared docs/source/development/mcp_development.rst against B6's later changes to openhcs/authoring/session/session.py: the writability probe makes one exclusive create attempt, the installed demo reports ViewerType.value, the unit session pre-creates its QApplication, and the GUI journey waits for each batch to end. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/development/mcp_development.rst against B6's later changes to openhcs/agent/dto/session.py: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", @@ -3385,7 +3429,7 @@ }, { "path": "tests/unit/agent/test_knowledge_base_service.py", - "sha256": "b814db1827c3d4cc3dab479ef7e84c6c406f324ccaddee794690f3b387626411", + "sha256": "7b30d1c1931efc1c380b2dacd743bcd4e509cafc949b01d24e156448681a0fc3", "role": "Regression proof for document IDs, search, bounded reads, and section selection." }, { @@ -3397,6 +3441,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/mcp_knowledge_base.rst in full and compared it against the changed authorities: tests/unit/agent/test_knowledge_base_service.py (MfdPresetKey renamed MfdPresetDefinition in the corpus assertions); the page does not name the preset class and its allowlist, search and section claims are untouched by the rename, so nothing was changed.", "On 2026-10-11 (B6 final), compared docs/source/development/mcp_knowledge_base.rst against B6's later changes to tests/unit/agent/test_knowledge_base_service.py: metadata documents are written only for backends that host local files (not virtual backends such as OMERO); the image-source guide test reads axis-family spellings. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/development/mcp_knowledge_base.rst against B6's later changes to tests/unit/agent/test_knowledge_base_service.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/development/mcp_knowledge_base.rst against the later B6 changes to openhcs/agent/capabilities.py, tests/unit/agent/test_knowledge_base_service.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -3691,7 +3736,7 @@ }, { "path": "tests/integration/test_main.py", - "sha256": "05331ebbe7a79554dfe7348e22ecf1ab3bbc0c58c43f9e3b9ca027f340a7dec0", + "sha256": "ce4a32452cef8b8dadea26894b923bbda88f0f88635e979c83f81a8d6ac34f29", "role": "Direct and ZMQ compile-before-execute integration harness with explicit OMERO lifecycle cleanup and no browser side effect." }, { @@ -3713,6 +3758,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/omero_testing.rst in full and compared it against the changed authorities: tests/integration/test_main.py (E1: ReadPipelineStart, PadToMaxSize and MaterializationBackend.__registry__ replace enum members); the page names none of those spellings; the harness still selects the omero_local materialization backend for OMERO runs and exercises direct and ZMQ execution, so nothing was changed.", "On 2026-10-11 (B6 final), compared docs/source/development/omero_testing.rst against B6's later changes to .github/workflows/integration-tests.yml: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/development/omero_testing.rst against the later B6 changes to external/PolyStore/src/polystore/omero_local.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/omero_testing.rst in full and compared it against the changed authorities: tests/integration/test_main.py (partition-filter config renames, Microscopy axis family, DatasetSourceChoice, per-test allocated ports, execute_compiled_bundle, subject_id, PipelineDocumentCodec) and .github/workflows/integration-tests.yml (parity-scope gate, run_installed_tests.py coverage). The page names none of the renamed test internals; confirmed the harness still registers OMEROLocalBackend and runs direct and ZMQ modes without opening a browser, the omero-tests-linux lane still builds submodule candidate wheels without a publication dependency and keeps Tables-0 and component diagnostics, and OMEROWellAddress, OMEROPlaneAddress, OMEROPlaneFilenameTemplate and FilenameParseResult exist at their owners.", @@ -3823,12 +3869,12 @@ "authority": [ { "path": "openhcs/core/debug.py", - "sha256": "4a1ed9fb0dcd40beeda9a2a3c90cdb5b2541252590d28281a9fb6f1aa021c510", + "sha256": "311b003ce50b69abbbdd934e1ff53cade27105605b6c3d7c980fc4ac81b8c2a4", "role": "Nominal debug command, boundary, snapshot, replay, and paused-worker declarations." }, { "path": "openhcs/core/debug_session_projection.py", - "sha256": "0a630eb03a83d741a4ed55838adb8ee65dd4fde9be07878aeba5e7a8c86c7378", + "sha256": "a088647748f3f3dace29ace81a6efacf3af32d66fd846417eedd5755b3545868", "role": "Typed debug-session phase selection from exact runtime context." }, { @@ -3838,13 +3884,14 @@ }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "4a33908fb42874e6d57eeea3f5305793ddbda41d13356f0a78501af38dcdfeb4", + "sha256": "40b07b7f40aca39b19b95a8f61c1b0c8270f0ab0a29e8c36caa13f04d8fa539f", "role": "Compilation order, declaration resolution, callable warm-up, framework-local device assignment, and bundle construction." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/pipeline_debugging_guide.rst in full and compared it against the changed authorities: openhcs/core/debug.py (K2 artifacts package imports), openhcs/core/debug_session_projection.py (E1: DebugSessionPhase is now the phase family itself) and openhcs/core/pipeline/compiler.py (E1: orchestrator state and materialization backend classes); the page names the debug phase only as state-surface content and names no removed class; CompiledStepPlan, CompilationSession, debug_pause and the inspection tool names are unchanged, so nothing was changed.", "On 2026-10-11 (B6), re-read docs/source/development/pipeline_debugging_guide.rst in full and compared it against the changed authorities: openhcs/core/debug.py (K6: debug commands, event boundaries and replay modes became class families such as StepDebugCommand and AfterInvocationDebugBoundary; control payload wrappers deleted), openhcs/agent/services/runtime_server_service.py (command decoded through registered_class_from_key, worker state read from the typed status) and openhcs/core/pipeline/compiler.py (ScopeKey scope ids, progress phase/status classes, CompiledSourceBindingPlan() default). The page names none of the changed types; rechecked debug_pause on AbstractStep, the 'Run to Pause', Step and 'Inspect Runtime' action labels, pipeline_debug_toolbar.session, openhcs_inspect_pipeline_source_artifact_plan, openhcs_inspect_debug_runtime_values / runtime-debug-values, SettingToKeywordBinding and StepMaterializationConfig; no claim changes.", "On 2026-10-11 (B6 final), compared docs/source/development/pipeline_debugging_guide.rst against B6's later changes to openhcs/core/debug.py, openhcs/core/pipeline/compiler.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/development/pipeline_debugging_guide.rst against the later B6 changes to openhcs/core/debug.py, openhcs/core/pipeline/compiler.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -3878,7 +3925,7 @@ }, { "path": "openhcs/processing/backends/processors/pyclesperanto_processor.py", - "sha256": "ae7b91db142b898f47831fcb3e44c67758e9e2f8c2d77ada3b347dadbc34af47", + "sha256": "ae6a2507ecfea6e592f4c9ec874e761ef82eb9a958c09e56c17ff9e3b3ef7390", "role": "OpenHCS processing callables using the pyclesperanto integration." }, { @@ -3903,7 +3950,7 @@ }, { "path": "openhcs/core/callable_contract.py", - "sha256": "a8e73c5df804d78baf72710e210cb2ae63156657a074071e11fd868f4b19d3b4", + "sha256": "6089f63e72e1a47fc5f6bc7b443028ac8ec40d934898589fa03b5592c2273ab3", "role": "Compiler-visible projection of all declared callable memory roles." }, { @@ -3913,7 +3960,7 @@ }, { "path": "openhcs/core/steps/function_runtime.py", - "sha256": "fa9486cddf3db943a218f276bb43d079f505bf84b18d9f7bd074f7a4d86332b3", + "sha256": "83d9eca90f2dca33ff8074bb768a24fadc5dfb1881855274eab9203b4cc41770", "role": "Runtime conversion and invocation inside the compiled execution-framework device scope." }, { @@ -3930,6 +3977,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/pyclesperanto_simple_implementation.rst in full and compared it against the changed authorities: openhcs/processing/backends/processors/pyclesperanto_processor.py and openhcs/core/callable_contract.py (K2: artifact imports moved to openhcs.core.artifacts.specs/kinds/plans) and the current openhcs/core/steps/function_runtime.py; the example imports only openhcs.core.memory and openhcs.core.processing_contracts, both unchanged; Pure3DContract, PyclesperantoRegistry's runtime-owned output_image/output_image_matrix/device parameters and the runtime execution_device_scope still exist, so nothing was changed.", "On 2026-10-11 (B6), re-read docs/source/development/pyclesperanto_simple_implementation.rst in full and compared it against the changed authorities: openhcs/core/processing_contracts.py and openhcs/core/callable_contract.py (K5: CallableMetadata replaced by CallableDeclaration, which CallableContract inherits), openhcs/core/steps/function_runtime.py (K6 debug boundary classes; main-flow source-binding plan no longer None) and tests/unit/test_unified_registry_runtime_payloads.py. The page names none of the changed types; rechecked the pyclesperanto decorator re-exported from openhcs.core.memory, Pure3DContract, and PyclesperantoRegistry's runtime-owned output_image, output_image_matrix and device parameters; no claim changes.", "On 2026-10-11 (B6 final), compared docs/source/development/pyclesperanto_simple_implementation.rst against B6's later changes to openhcs/processing/backends/processors/pyclesperanto_processor.py, openhcs/core/memory/decorators.py, openhcs/core/processing_contracts.py, openhcs/core/callable_contract.py, openhcs/core/steps/function_runtime.py, tests/unit/test_unified_registry_runtime_payloads.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/development/pyclesperanto_simple_implementation.rst against the later B6 changes to openhcs/processing/backends/processors/pyclesperanto_processor.py, openhcs/core/memory/decorators.py, openhcs/core/processing_contracts.py, openhcs/core/callable_contract.py, openhcs/core/steps/function_runtime.py, tests/unit/test_unified_registry_runtime_payloads.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -3962,13 +4010,14 @@ }, { "path": "pyproject.toml", - "sha256": "b5d4d752704b886da20d860743b81dc3d2ce3a3c402208246fadc1c6cb71f597", + "sha256": "75f92010b3c18ede61cf7296064079590c143283871e372a1fd4546417d4e0f9", "role": "Current Python support, dependency, package discovery, and entry-point authority." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/repository_setup.rst in full and compared it against the changed authorities: pyproject.toml (python-introspect requirement raised to 0.2.8); the page quotes no dependency version; the Python 3.11-3.14 range, the centrosome python_version < 3.14 marker, the dev/dev-gui/gui/viz/gpu/docs/cellprofiler-compat extras, the CUDA 12 gpu pins and the eight external/ submodule directories all match, so nothing was changed.", "On 2026-10-11 (B6), re-read docs/source/development/repository_setup.rst in full and compared it against the changed authorities: pyproject.toml (first-party floors moved to zmqruntime 0.6.2, pycodify 0.2, objectstate 2.1.1, python-introspect 0.2.7, pyqt-reactive 0.5); the page states no version floors. Rechecked the eight submodules against .gitmodules, requires-python >=3.11,<3.15, the named extras (dev, gui, dev-gui, cellprofiler-compat, viz, gpu, docs), and that OPENHCS_CPU_ONLY=1 is accepted by OpenHCSProcessEnvironment.flag_enabled; no claim changes.", "On 2026-10-11 (B6 final), compared docs/source/development/repository_setup.rst against the later B6 changes to pyproject.toml: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/development/repository_setup.rst in full and compared it against the changed authorities: pyproject.toml (first-party floors moved to zmqruntime 0.6, objectstate 1.3.1, python-introspect 0.2.5, metaclass-registry 0.3, arraybridge 0.4.1, polystore 0.6, pyqt-reactive 0.4; JAX GPU extra now jax[cuda12-local]; openhcs-basicpy added as an ordinary dependency); challenged the eight-submodule count against .gitmodules, the 3.11-3.14 range against requires-python, the centrosome parity oracle's python_version < 3.14 marker, the named extras (dev, gui, dev-gui, cellprofiler-compat, viz, gpu, docs), setup.py deferring metadata to pyproject.toml, and the CUDA 12 statement for the gpu extra; all remain accurate and the page states no version floors.", @@ -4021,12 +4070,12 @@ }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "0f9cc4819b53d55b7e9446429b8c8ac1ff1a59f4902020ea1658495ce01d464c", + "sha256": "4946c221e34ec19957b5af2b1a54359d454ee840fb1f5e1fe2b3b306ac8e5426", "role": "Resolved step/session identity and typed plan assembly invariants." }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "74b591773b7f653c957b2db33a1262925e5c02b71074695b6fc18d6120ed5934", + "sha256": "35fe71906bfd16842de347776a0c5cc846550ad490cc3628885129d746ec6f31", "role": "RuntimeValueStore recording, replacement, query, and observation semantics." }, { @@ -4038,6 +4087,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/respecting_codebase_architecture.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compilation_session.py and openhcs/core/runtime_stores.py (K2: artifact imports moved into the openhcs.core.artifacts package); the page names no artifact module path; EnumKeyedStrategyMixin, NominalTypeKeyedStrategyMixin, MostDerivedContextStrategyMixin and RegistryFamily still exist in metaclass_registry and EnumKeyedStrategyMixin still has OpenHCS users, so nothing was changed.", "On 2026-10-11 (B6), re-read docs/source/development/respecting_codebase_architecture.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compilation_session.py (per-invocation validation moved onto ArtifactGraph.validate_invocation_declarations; strip_disabled_functions no longer applied) and openhcs/core/runtime_stores.py (hand-written to_dict/from_dict transport on RuntimeArtifactLocation and RuntimeArtifactAddress deleted). Both moves agree with the page's single-owner guidance; rechecked that the three strategy mixins, AutoRegisterMeta, RegistryConfig and RegistryFamily still exist in metaclass-registry; no claim changes.", "On 2026-10-11 (B6 final), compared docs/source/development/respecting_codebase_architecture.rst against B6's later changes to openhcs/core/pipeline/compilation_session.py, openhcs/core/runtime_stores.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/development/respecting_codebase_architecture.rst against the later B6 changes to openhcs/core/pipeline/compilation_session.py, openhcs/core/runtime_stores.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -4060,7 +4110,7 @@ "authority": [ { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "0f9cc4819b53d55b7e9446429b8c8ac1ff1a59f4902020ea1658495ce01d464c", + "sha256": "4946c221e34ec19957b5af2b1a54359d454ee840fb1f5e1fe2b3b306ac8e5426", "role": "Declares ResolvedPipelineDefinition and CompilationSession, the resolved declaration view and per-context compiler boundary named in the assembly order." }, { @@ -4070,22 +4120,22 @@ }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "74b591773b7f653c957b2db33a1262925e5c02b71074695b6fc18d6120ed5934", + "sha256": "35fe71906bfd16842de347776a0c5cc846550ad490cc3628885129d746ec6f31", "role": "RuntimeValueStore recording, replacement, query, and observation semantics." }, { "path": "openhcs/core/runtime_slice_projection.py", - "sha256": "8f45a17d6aabd4fae1df55e8d3cda2efdd8df2156357c69133f06ccf1cfb2d96", + "sha256": "f59785d901d0d7bcc3ac1fc1be606bd1728a727ea7347cdc1dd4f549cc3193fa", "role": "RuntimeSliceProjection boundary between owned projectable values and foreign values, with source-provenance checks on stacks." }, { "path": "openhcs/core/callable_contract.py", - "sha256": "a8e73c5df804d78baf72710e210cb2ae63156657a074071e11fd868f4b19d3b4", + "sha256": "6089f63e72e1a47fc5f6bc7b443028ac8ec40d934898589fa03b5592c2273ab3", "role": "Callable ABI and input/output/execution memory-type declaration authority." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "358d3eaf5952170f9e9e2dc9056fb08707e8fcf74682063c074b367a83c2648f", + "sha256": "6525348d097bda8d0265bc7c3f312dd8839ca9f65dd05f797fc82ba873e04547", "role": "Typed per-step source-binding plan and framework-device decisions." }, { @@ -4097,6 +4147,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/runtime_system_assembly_rules.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compilation_session.py, openhcs/core/runtime_stores.py, openhcs/core/callable_contract.py, openhcs/core/compiled_step_plan.py (K2 artifact package imports; UnresolvedStepInput default) and openhcs/core/runtime_slice_projection.py (new stored_slices/match_alignment/contextualized_output delegating to the value); every owner in the owner map still exists with the stated responsibility, and the RuntimeSliceProjection entry stays true because the new methods delegate to RuntimeSliceProjectableValue, so nothing was changed.", "On 2026-10-11 (B6), re-read docs/source/development/runtime_system_assembly_rules.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compilation_session.py (per-invocation validation moved onto ArtifactGraph), openhcs/core/runtime_stores.py (hand-written address/location to_dict/from_dict deleted), openhcs/core/callable_contract.py (K5: CallableContract now inherits CallableDeclaration and its declared fields directly; memory roles, artifact inputs/outputs and execution scope remain on the contract) and openhcs/core/compiled_step_plan.py (CompiledSourceBindingPlan() default). Rechecked CellProfilerModuleArtifactContracts, CompiledExecutionBundle's runtime/transport contexts, worker assignments and runtime environment, SourceBindingsConfig, ArtifactSpecRef and ArtifactKey; no claim changes.", "On 2026-10-11 (B6 final), compared docs/source/development/runtime_system_assembly_rules.rst against B6's later changes to openhcs/core/pipeline/compilation_session.py, openhcs/core/runtime_stores.py, openhcs/core/callable_contract.py, openhcs/core/compiled_step_plan.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/development/runtime_system_assembly_rules.rst against the later B6 changes to openhcs/core/pipeline/compilation_session.py, openhcs/core/runtime_stores.py, openhcs/core/callable_contract.py, openhcs/core/compiled_step_plan.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -4121,33 +4172,34 @@ "authority": [ { "path": "openhcs/core/runtime_artifact_values.py", - "sha256": "930752071d37ef1a8669812e7a629e151690b3f33bf4a29dba92d1d479b9074d", + "sha256": "ce42e38f893459b9cb0ed6d779fabd5818204301ab1acc13ab0a7df32e6e3a2d", "role": "RuntimeValue and ArtifactKey construction from compiled output plans." }, { "path": "openhcs/core/runtime_plane_projection.py", - "sha256": "3c88ed042a7f1c65c9495a1c62777e008f1e59c4742ec0563b2849d77ca87cdd", + "sha256": "cf644451b6649dcec398fecbf961a4a29ea74f0bb2e0bdab052ec1be8a427f95", "role": "Declares RuntimeSliceProjectableValue and its indexed, invariant and identity-stamped subclasses that payload authors join." }, { "path": "openhcs/core/runtime_slice_projection.py", - "sha256": "8f45a17d6aabd4fae1df55e8d3cda2efdd8df2156357c69133f06ccf1cfb2d96", + "sha256": "f59785d901d0d7bcc3ac1fc1be606bd1728a727ea7347cdc1dd4f549cc3193fa", "role": "RuntimeSliceProjection foreign-value list and rejection of owned values without slice declarations." }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "74b591773b7f653c957b2db33a1262925e5c02b71074695b6fc18d6120ed5934", + "sha256": "35fe71906bfd16842de347776a0c5cc846550ad490cc3628885129d746ec6f31", "role": "RuntimeValueStore recording, replacement, query, and observation semantics." }, { "path": "tests/unit/test_runtime_value_store.py", - "sha256": "d20ae0d5160c79ba7b7c3291a052a606b3ed7dee3c0cd84a6cf013e17a127a23", + "sha256": "17e15a64a4e84437f6523f872bad11ea7d813c50c1c41fc86ca7822379a7ce27", "role": "Regression proof for typed runtime recording, replacement, queries, and failure behaviour." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/runtime_value_extension.rst in full and compared it against the changed authorities: openhcs/core/runtime_artifact_values.py (payload_of), openhcs/core/runtime_plane_projection.py (stored_slices, match_alignment, contextualized_output and axis-size checks on the value classes), openhcs/core/runtime_slice_projection.py, openhcs/core/runtime_stores.py and tests/unit/test_runtime_value_store.py (K2 artifact package imports); RuntimeSliceProjectableValue, RuntimeSliceIndexedValue and RuntimeSliceInvariantValue still exist, the added behaviour sits on the payload classes as the page says, and RuntimeSliceProjection.require_foreign_value still rejects undeclared values, so nothing was changed.", "On 2026-10-11 (B6), re-read docs/source/development/runtime_value_extension.rst in full and compared it against the changed authorities: openhcs/core/runtime_stores.py and tests/unit/test_runtime_value_store.py (hand-written to_dict/from_dict on RuntimeArtifactLocation and RuntimeArtifactAddress deleted; round trips now use python_introspect to_jsonable and dataclass_from_mapping; CompiledSourceBindingPlan() replaces .empty()). The page names none of those methods and its 'add transport only where needed' guidance still holds; rechecked RuntimeSliceProjectableValue and its indexed/invariant subclasses, RuntimeSliceProjection and RuntimeValueStore; no claim changes.", "On 2026-10-11 (B6 final), compared docs/source/development/runtime_value_extension.rst against B6's later changes to openhcs/core/runtime_stores.py, tests/unit/test_runtime_value_store.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/development/runtime_value_extension.rst against the later B6 changes to openhcs/core/runtime_stores.py, tests/unit/test_runtime_value_store.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -4159,7 +4211,7 @@ }, { "path": "docs/source/development/source_binding_extension.rst", - "source_sha256": "1f33644ddd081d2659cd5417f2607189a049bab6e07d26e198e65f452de52659", + "source_sha256": "9f62eb7718bf83ef1bb7ef5207e4ccbf60667b35ea30934442b5a1b443a39f12", "audience": [ "OpenHCS source-ingestion and compiler extension authors" ], @@ -4168,32 +4220,32 @@ "authority": [ { "path": "openhcs/core/source_bindings.py", - "sha256": "2defaf02ac0e214bb63f74d6eb56f294fbb5db4e06ecd9cc4479b16848f09854", - "role": "Declares SourceFilterClause, MetadataExtractionRule, NamedSourceBinding, ImagePlaneSource, the SourceSetPairing family (PairByOrder, PairByMetadata) and the compiled source universe and binding plans." + "sha256": "890473715373d1adc6b12fec2b57eab6f179cbca7d837920c0388d620ce8b8c2", + "role": "Declares SourceFilterClause, MetadataExtractionRule, NamedSourceBinding, ImagePlaneSource, the SourceFilterMatchType predicate family, the SourceSetPairing family (PairByOrder, PairByMetadata) and the compiled source universe and binding plans." }, { "path": "openhcs/core/config.py", - "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", + "sha256": "9625ca054fac6c1e460216cfe0e85621f5306592a004d1bdc28a6b1ce2099366", "role": "Declares SourceBindingsConfig and StepSourceBindingsConfig as global_pipeline_config sections, where the step config compiles its own binding plan." }, { "path": "openhcs/core/source_binding_selection.py", - "sha256": "9610bc6112c068cf6c4b0ff7a1265ee4086291042e2dc422d8a90cce3c560eb7", - "role": "SourceUniverseRequest family that selects source universes." + "sha256": "e837296accf4dd3353956a8e5f267d3fdd4581fb01db1227bc592a2ed6b4d0d3", + "role": "SourceUniverseRequest family, keyed by SourceBindingOrigin class, that selects source universes." }, { "path": "openhcs/core/source_matching.py", - "sha256": "f32db4e25a0b6b7c1a334ee27c7206044e1ba645717ee12f6acd69d28d9b30a8", - "role": "SourceFilterMatcher and SourceFilterTargetResolver matching families." + "sha256": "cea83c632056420b4e2affac25f9da37c3a4ee41878e2a1fb18b6747d8573dcf", + "role": "Applies filter clauses and metadata extraction rules to candidate paths and derives image-set identity from source declarations." }, { "path": "openhcs/core/source_workspace_projection.py", - "sha256": "2d2b6bf5a331cb61411b293b333980a9951f9ec1cd8537874d6529949180f3ec", + "sha256": "1f8785fec7919f52b69578bbd6f6b5f94be8b2f9e97d11d00f639f3f70bcfe23", "role": "VirtualWorkspaceSourceProjection mapping virtual paths to SourcePixelRef and source metadata." }, { "path": "openhcs/core/source_binding_workspace.py", - "sha256": "5e7ac439d7ef6cd35a5660d2ad9db24f1890801d8bdf1d82c13b12f8baaa9eb0", + "sha256": "e242d9b932d52c1099756881870f926a273f5ec08663938347580cb4a70e8283", "role": "Compiled virtual source workspace and the source-set assemblers keyed by SourceSetPairing class." }, { @@ -4203,11 +4255,11 @@ } ], "findings": [ - "K4 replaced the SourceBindingMatchMethod enum and SourceBindingMatchPlan with the SourceSetPairing class family (PairByOrder default, PairByMetadata); the strategy-selection paragraph named only SourceUniverseRequest and SourceFilterMatcher, so it now also names SourceSetPairing as the family that decides source-set pairing.", - "The record's source_bindings.py role said it declares SourceBindingsConfig and StepSourceBindingsConfig; K4 moved both to openhcs/core/config.py, which is now a separate authority." + "E1 deleted the SourceFilterMatcher strategy family from openhcs/core/source_matching.py; filter predicates are now the SourceFilterMatchType class family in openhcs/core/source_bindings.py. The strategy-selection paragraph still named SourceFilterMatcher as a family to extend, so it now names SourceFilterMatchType." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/development/source_binding_extension.rst in full and compared it against the changed authorities: openhcs/core/source_bindings.py, openhcs/core/source_matching.py, openhcs/core/source_binding_selection.py, openhcs/core/source_binding_workspace.py, openhcs/core/source_workspace_projection.py and openhcs/core/config.py (E1: the source-binding kinds became class families); replaced the deleted SourceFilterMatcher with SourceFilterMatchType in the strategy-selection paragraph and re-wrapped it; confirmed SourceUniverseRequest (now keyed by SourceBindingOrigin class), SourceSetPairing, the compiled plan classes and VirtualWorkspaceSourceProjection still exist at their owners.", "On 2026-10-11 (B6), re-read docs/source/development/source_binding_extension.rst in full and compared it against the changed authorities: openhcs/core/source_bindings.py, source_binding_selection.py, source_matching.py, source_workspace_projection.py and source_binding_workspace.py (K4: config classes moved to core/config.py, SourceSetPairing family replaces the match-method enum, CompiledSourceBindingPlan()/CompiledSourceUniversePlan() replace .empty(), one lexical path identity). Confirmed every class the page names exists at its current owner, added SourceSetPairing to the strategy families, and added core/config.py as the config-class authority.", "On 2026-10-11 (B6 final), compared docs/source/development/source_binding_extension.rst against B6's later changes to openhcs/core/source_binding_selection.py, openhcs/core/source_workspace_projection.py, openhcs/core/source_binding_workspace.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/development/source_binding_extension.rst against the later B6 changes to openhcs/core/source_bindings.py, openhcs/core/source_binding_selection.py, openhcs/core/source_workspace_projection.py, openhcs/core/source_binding_workspace.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", diff --git a/docs/audits/project_surface.json b/docs/audits/project_surface.json index d52180346a..6042473bb8 100644 --- a/docs/audits/project_surface.json +++ b/docs/audits/project_surface.json @@ -12,7 +12,7 @@ "authority": [ { "path": "pyproject.toml", - "sha256": "b5d4d752704b886da20d860743b81dc3d2ce3a3c402208246fadc1c6cb71f597", + "sha256": "75f92010b3c18ede61cf7296064079590c143283871e372a1fd4546417d4e0f9", "role": "Declares the published README, Python range, package dependencies, optional extras, console commands, and project links." }, { @@ -32,12 +32,12 @@ }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "4a33908fb42874e6d57eeea3f5305793ddbda41d13356f0a78501af38dcdfeb4", + "sha256": "40b07b7f40aca39b19b95a8f61c1b0c8270f0ab0a29e8c36caa13f04d8fa539f", "role": "Owns compile-time resolution across selected axes, compile progress emission, and construction of the execution bundle." }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "0f9cc4819b53d55b7e9446429b8c8ac1ff1a59f4902020ea1658495ce01d464c", + "sha256": "4946c221e34ec19957b5af2b1a54359d454ee840fb1f5e1fe2b3b306ac8e5426", "role": "Declares ResolvedPipelineDefinition and CompilationSession named in the compilation diagram." }, { @@ -45,14 +45,9 @@ "sha256": "c46e825cd32801caae9fc107e0759976da054eadc56adc0d13354b3a8b7db209", "role": "Declares set_progress_queue, which compilation requires before it emits progress events." }, - { - "path": "openhcs/core/artifacts.py", - "sha256": "65eb0ec5c903366243ca8bd5ea07412e8d34bdb81e6015fa76b3c4c59838733f", - "role": "Declares image, object, measurement, relationship, table, and file artifact semantics used by materialization." - }, { "path": "openhcs/core/dataset_sources/source.py", - "sha256": "350bf5691a03522e9f54b1d8ab712c6beba1683aac4fa5159ea4e80c6041530d", + "sha256": "b3758a878fdf8ec28b6d9818ebfd96c5c470895575f7a79500c85f1490679281", "role": "Declares the auto-registered DatasetSource family that the ImageXpress, Opera Phenix, Bio-Formats, OMERO and OpenHCS-format sources join." }, { @@ -62,7 +57,7 @@ }, { "path": "openhcs/core/viewer_streaming_service.py", - "sha256": "c96eca530f767002cfcc9aca4fe75c93d7a5848a7f42df9e5d336bf6712ed51e", + "sha256": "50742cdcb589dc9fba5a177c0107c046dd63c518c5aa77bd8a80e4f69e898e07", "role": "Owns manual and orchestrated image and ROI streaming through registered viewer declarations." }, { @@ -72,12 +67,12 @@ }, { "path": "openhcs/mcp/client_registration.py", - "sha256": "55b701bb77a3de19243fa9a6276f784fad8329e2d435fb26fcc7fdce1e060731", + "sha256": "4d6fc5f744617ab1d0e9b8317d07965496aaa6b098a4d179ef95cc6baec35671", "role": "Declares supported local MCP client targets such as ChatGPT desktop/Codex and Claude Desktop." }, { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "b96f33b866fe6c9f8118ca70fcb7dff8e6a02cb29d79df67230a5e8b97df8002", + "sha256": "5aa8895c9f533603b7ef972202141e2439473f0604b97d66f5a027b8b3cd1c83", "role": "Declares import_cellprofiler_pipeline and its returned FunctionStep list and PipelineConfig." }, { @@ -87,7 +82,7 @@ }, { "path": "benchmark/cellprofiler_benchmark_cli.py", - "sha256": "2e0de28a96c57ee150e28fbbed28d876507e9281075a03f729838f3ca3610966", + "sha256": "0c2cab4d989d50149cb314e908baace8aa794953c86ebc572fa120beaa695780", "role": "Declares the openhcs-benchmark subcommands and options, including run-measured through the OpenHCS session." }, { @@ -102,18 +97,24 @@ }, { "path": "benchmark/control.py", - "sha256": "83fe9016625d7d8721905b8a153a28c3dcaaf2b9e925a3c97711c634868156b7", + "sha256": "b10dd87d7bc44063566e0c53d849091c0fc43a345b434c13467df1ede86ac892", "role": "Owns read-only inspection of receipts, observation progress, and structured artifacts." }, { "path": "tests/unit/test_benchmark_control.py", "sha256": "093b584eeb37cbb12089d5af082ad45acd779e9d2bdd5b3feaea08234fc22183", "role": "Proves the installed command derivation, typed receipts, MCP projection, path policy, and inspect-only import boundary." + }, + { + "path": "openhcs/core/artifacts/kinds.py", + "sha256": "216b913efad9ec76707e9f61be445fb6fa1c6601c36ef53e296acd4928604515", + "role": "Declares the image, object-label, measurement, relationship, table, spatial and metadata artifact kinds used by materialization." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read README.md in full and compared it against the changed authorities: pyproject.toml (python-introspect floor 0.2.8), openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/artifacts.py (K2 split it into the openhcs/core/artifacts package), openhcs/core/dataset_sources/source.py, openhcs/core/viewer_streaming_service.py, openhcs/mcp/client_registration.py, openhcs/interop/cellprofiler/pipeline_import.py, benchmark/cellprofiler_benchmark_cli.py, benchmark/control.py; E1 turned OrchestratorState, MaterializationBackend, InputSource, the client-registration status and the observation export scope into class families, but the README names none of the old enum members and the `--observation-scope outcomes` spelling is still the registered name, so no claim changed. Replaced the deleted artifacts.py authority with openhcs/core/artifacts/kinds.py.", "On 2026-10-11 (B6), re-read README.md in full and compared it against the changed authorities: pyproject.toml (zmqruntime 0.6.2, pycodify 0.2, objectstate 2.1.1, python-introspect 0.2.7, pyqt-reactive 0.5 floors), openhcs/core/pipeline_document.py (renders through pycodify.generate_python_source), openhcs/core/pipeline/compiler.py and compilation_session.py (ScopeKey ids, progress classes, disabled-function stripping removed), openhcs/core/progress/types.py (K6: ProgressPhase/ProgressStatus enums became class families; set_progress_queue unchanged and still exported from openhcs.core.progress), openhcs/core/dataset_sources/source.py (SourceBindingsConfig import moved) and pipeline_import.py (K4 import moves). The README states no dependency version or phase name; the CellProfiler example's imports (ensure_global_config_context, set_progress_queue, import_cellprofiler_pipeline with source_root, PipelineOrchestrator.initialize/compile_pipelines) and the compilation diagram's class names all still resolve, so no claim changed.", "On 2026-10-11 (B6 final), compared README.md against B6's later changes to tests/integration/test_cellprofiler_official30_zmq.py, benchmark/cellprofiler_benchmark_cli.py: the session decides in-place initialization by an actual file-creation attempt that leaves nothing behind, its debug-session methods moved into an inherited mixin, the measured CLI treats Windows Ctrl+Break like Ctrl+C, and the Official30 viewer test reads typed payload summaries. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared README.md against B6's later changes to openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/progress/types.py, openhcs/core/artifacts.py, openhcs/core/dataset_sources/source.py, openhcs/processing/backends/lib_registry/unified_registry.py, openhcs/core/viewer_streaming_service.py, openhcs/interop/cellprofiler/pipeline_import.py, benchmark/cellprofiler_benchmark_cli.py, tests/unit/test_benchmark_control.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", diff --git a/docs/audits/user_docs.json b/docs/audits/user_docs.json index 760cf16b92..f47e81b85d 100644 --- a/docs/audits/user_docs.json +++ b/docs/audits/user_docs.json @@ -11,7 +11,7 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", + "sha256": "9625ca054fac6c1e460216cfe0e85621f5306592a004d1bdc28a6b1ce2099366", "role": "Declares PipelineConfig, GlobalPipelineConfig, ProcessingConfig and the lazy processing configuration used by the step example." }, { @@ -26,7 +26,7 @@ }, { "path": "openhcs/core/orchestrator/orchestrator.py", - "sha256": "99870516da6d64e1858f4dd8799d6e38ab6911b7b02bc29b252d2a23f7687410", + "sha256": "a7097eec86c5966552bcb546a2161d1a08e6a4c630e6b156de3b8354b0026b54", "role": "Declares the PipelineOrchestrator constructor, compile_pipelines, progress_context and execute_compiled_bundle signatures." }, { @@ -41,18 +41,19 @@ }, { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "b96f33b866fe6c9f8118ca70fcb7dff8e6a02cb29d79df67230a5e8b97df8002", + "sha256": "5aa8895c9f533603b7ef972202141e2439473f0604b97d66f5a027b8b3cd1c83", "role": "Declares import_cellprofiler_pipeline and its returned steps and PipelineConfig." }, { "path": "tests/unit/test_cellprofiler_pipeline_import.py", - "sha256": "0ce4efd85d819c7e9f7fa84ec6f1ac7ec4bda024c9a8d5c7f264fda2730264f0", + "sha256": "67576f37b7cbe0595acea633265f8e14677ca3c1b8204630413330273dc4e58a", "role": "Proves direct import rejects modules requiring interactive desktop input instead of omitting them." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/api/index.rst in full and compared it against the changed authorities: openhcs/core/config.py, openhcs/core/orchestrator/orchestrator.py, openhcs/interop/cellprofiler/pipeline_import.py, tests/unit/test_cellprofiler_pipeline_import.py; E1 made MaterializationBackend, InputSource and OrchestratorState class families, but the examples use only ProcessingConfig(variable_components=...), LazyProcessingConfig.from_config, the PipelineOrchestrator constructor, initialize, compile_pipelines, progress_context and execute_compiled_bundle, and import_cellprofiler_pipeline, whose signatures and import locations are unchanged, so no claim changed.", "On 2026-10-11 (B6), re-read docs/source/api/index.rst in full and compared it against the changed authorities: openhcs/core/config.py (K4 source-binding classes moved in, PartitionFilterMode became a class family), orchestrator.py (source_bindings_defaults_to_base now imported from config), progress/types.py (K6 phase/status/stream class families), pipeline_import.py and its test (K4 import moves, PairByOrder). The page's two examples still match: ProcessingConfig(variable_components=(Microscopy.Site,)), LazyProcessingConfig.from_config, import_cellprofiler_pipeline(..., source_root=...), set_progress_queue, compile_pipelines and execute_compiled_bundle(execution_bundle=, progress_queue=, progress_context=) with ProgressQueue and ProgressExecutionContext unchanged, so no claim changed.", "On 2026-10-11 (B6 final), compared docs/source/api/index.rst against B6's later changes to openhcs/core/config.py, openhcs/domains/microscopy/axes.py, openhcs/core/orchestrator/orchestrator.py, openhcs/core/progress/types.py, openhcs/interop/cellprofiler/pipeline_import.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/api/index.rst against the later B6 changes to openhcs/core/config.py, openhcs/domains/microscopy/axes.py, openhcs/core/orchestrator/orchestrator.py, openhcs/core/progress/types.py, openhcs/interop/cellprofiler/pipeline_import.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -78,14 +79,9 @@ "user_need": "Look up the precise meaning of shared microscopy and runtime terms.", "diataxis": "reference", "authority": [ - { - "path": "openhcs/core/artifacts.py", - "sha256": "65eb0ec5c903366243ca8bd5ea07412e8d34bdb81e6015fa76b3c4c59838733f", - "role": "Declares the nominal artefact types exchanged by compiled steps and runtime consumers." - }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "358d3eaf5952170f9e9e2dc9056fb08707e8fcf74682063c074b367a83c2648f", + "sha256": "6525348d097bda8d0265bc7c3f312dd8839ca9f65dd05f797fc82ba873e04547", "role": "Declares the per-step compiled plan read by runtime code." }, { @@ -95,22 +91,22 @@ }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "0f9cc4819b53d55b7e9446429b8c8ac1ff1a59f4902020ea1658495ce01d464c", + "sha256": "4946c221e34ec19957b5af2b1a54359d454ee840fb1f5e1fe2b3b306ac8e5426", "role": "Owns compilation-scoped source, artifact and plan state." }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "74b591773b7f653c957b2db33a1262925e5c02b71074695b6fc18d6120ed5934", + "sha256": "35fe71906bfd16842de347776a0c5cc846550ad490cc3628885129d746ec6f31", "role": "Implements scoped typed runtime-value storage and lookup for compiled execution." }, { "path": "openhcs/core/source_bindings.py", - "sha256": "2defaf02ac0e214bb63f74d6eb56f294fbb5db4e06ecd9cc4479b16848f09854", + "sha256": "890473715373d1adc6b12fec2b57eab6f179cbca7d837920c0388d620ce8b8c2", "role": "Declares NamedSourceBinding and the source-assignment family that connect callable inputs to sources, plus CompiledSourceUniversePlan." }, { "path": "openhcs/core/callable_contract.py", - "sha256": "a8e73c5df804d78baf72710e210cb2ae63156657a074071e11fd868f4b19d3b4", + "sha256": "6089f63e72e1a47fc5f6bc7b443028ac8ec40d934898589fa03b5592c2273ab3", "role": "Declares CallableDeclaration and CallableContract fields: memory types, artifact inputs/outputs, runtime-bound parameters, required axis roles and execution scope." }, { @@ -125,13 +121,19 @@ }, { "path": "openhcs/core/dataset_sources/source.py", - "sha256": "350bf5691a03522e9f54b1d8ab712c6beba1683aac4fa5159ea4e80c6041530d", + "sha256": "b3758a878fdf8ec28b6d9818ebfd96c5c470895575f7a79500c85f1490679281", "role": "Declares DatasetSource with detection, parsing, metadata, virtual-workspace and backend behaviour." + }, + { + "path": "openhcs/core/artifacts/kinds.py", + "sha256": "216b913efad9ec76707e9f61be445fb6fa1c6601c36ef53e296acd4928604515", + "role": "Declares the ArtifactType family: image, object-label, measurement, relationship, table and spatial-graph kinds." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/appendices/glossary.rst in full and compared it against the changed authorities: openhcs/core/artifacts.py (K2 split it into the openhcs/core/artifacts package), openhcs/core/compiled_step_plan.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/runtime_stores.py, openhcs/core/source_bindings.py (E1 source-binding kinds became families), openhcs/core/callable_contract.py, openhcs/core/dataset_sources/source.py; every artifact kind the Artifact entry lists is still a registered ArtifactType, NamedSourceBinding and CompiledSourceUniversePlan still exist, and no entry names a removed enum member, so no definition changed. Replaced the deleted artifacts.py authority with openhcs/core/artifacts/kinds.py.", "On 2026-10-11 (B6), re-read docs/source/appendices/glossary.rst in full and compared it against the changed authorities: compiled_step_plan.py and runtime_stores.py (source plans default-construct; RuntimeArtifactLocation/Address dict codecs removed), compilation_session.py, source_bindings.py (K4: SourceBindingsConfig moved to config.py, match methods became the SourceSetPairing family), callable_contract.py (K5: metadata wrapper flattened into CallableDeclaration fields), processing_contracts.py and dataset_sources/source.py. The glossary defines terms without naming moved classes; Pure2DContract, Pure3DContract, PartitionAxis, DatasetSource, CompiledSourceUniversePlan and the callable-contract fields it lists still exist, so no claim changed. Sharpened the source_bindings and callable_contract roles to what they now declare.", "On 2026-10-11 (B6 final), compared docs/source/appendices/glossary.rst against B6's later changes to openhcs/core/artifacts.py, openhcs/core/compiled_step_plan.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/runtime_stores.py, openhcs/core/callable_contract.py, openhcs/core/processing_contracts.py, openhcs/core/axes.py, openhcs/core/dataset_sources/source.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/appendices/glossary.rst against the later B6 changes to openhcs/core/artifacts.py, openhcs/core/compiled_step_plan.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/runtime_stores.py, openhcs/core/source_bindings.py, openhcs/core/callable_contract.py, openhcs/core/processing_contracts.py, openhcs/core/axes.py, openhcs/core/dataset_sources/source.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -195,12 +197,12 @@ "authority": [ { "path": "pyproject.toml", - "sha256": "b5d4d752704b886da20d860743b81dc3d2ce3a3c402208246fadc1c6cb71f597", + "sha256": "75f92010b3c18ede61cf7296064079590c143283871e372a1fd4546417d4e0f9", "role": "Declares the packaged openhcs-benchmark console script entry point." }, { "path": "benchmark/cellprofiler_benchmark_cli.py", - "sha256": "2e0de28a96c57ee150e28fbbed28d876507e9281075a03f729838f3ca3610966", + "sha256": "0c2cab4d989d50149cb314e908baace8aa794953c86ebc572fa120beaa695780", "role": "Owns installed benchmark execution and report commands." }, { @@ -210,7 +212,7 @@ }, { "path": "benchmark/control.py", - "sha256": "83fe9016625d7d8721905b8a153a28c3dcaaf2b9e925a3c97711c634868156b7", + "sha256": "b10dd87d7bc44063566e0c53d849091c0fc43a345b434c13467df1ede86ac892", "role": "Owns read-only measured-run inspection and bounded report rendering." }, { @@ -222,6 +224,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/appendices/research_impact.rst in full and compared it against the changed authorities: pyproject.toml (python-introspect floor 0.2.8), benchmark/cellprofiler_benchmark_cli.py, benchmark/control.py; the observation export scope became a class family read by name, while the openhcs-benchmark entry point, typed receipts, read-only inspection and report, and the archive path the page names are unchanged, so no claim changed.", "On 2026-10-11 (B6), re-read docs/source/appendices/research_impact.rst in full and compared it against the changed authorities: pyproject.toml (dependency floors for zmqruntime, pycodify 0.2, objectstate 2.1, python-introspect, pyqt-reactive 0.5); the openhcs-benchmark entry point, benchmark CLI, run receipts, read-only inspection service and the archived page path all still exist, and the page states no version, so no claim changed.", "On 2026-10-11 (B6 final), compared docs/source/appendices/research_impact.rst against B6's later changes to benchmark/cellprofiler_benchmark_cli.py: the session decides in-place initialization by an actual file-creation attempt that leaves nothing behind, its debug-session methods moved into an inherited mixin, the measured CLI treats Windows Ctrl+Break like Ctrl+C, and the Official30 viewer test reads typed payload summaries. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/appendices/research_impact.rst against B6's later changes to benchmark/cellprofiler_benchmark_cli.py, tests/unit/test_benchmark_control.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", @@ -265,17 +268,17 @@ "authority": [ { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "4a33908fb42874e6d57eeea3f5305793ddbda41d13356f0a78501af38dcdfeb4", + "sha256": "40b07b7f40aca39b19b95a8f61c1b0c8270f0ab0a29e8c36caa13f04d8fa539f", "role": "Compiles pipeline declarations into validated step plans and execution bundles." }, { "path": "openhcs/core/source_bindings.py", - "sha256": "2defaf02ac0e214bb63f74d6eb56f294fbb5db4e06ecd9cc4479b16848f09854", + "sha256": "890473715373d1adc6b12fec2b57eab6f179cbca7d837920c0388d620ce8b8c2", "role": "Declares NamedSourceBinding and the SourceSetPairing family that turn files and metadata into named inputs." }, { "path": "openhcs/core/dataset_sources/source.py", - "sha256": "350bf5691a03522e9f54b1d8ab712c6beba1683aac4fa5159ea4e80c6041530d", + "sha256": "b3758a878fdf8ec28b6d9818ebfd96c5c470895575f7a79500c85f1490679281", "role": "DatasetSource family that owns recognized dataset layouts." }, { @@ -284,8 +287,8 @@ "role": "ProcessingContract family deciding per-plane versus whole-stack execution." }, { - "path": "openhcs/core/artifacts.py", - "sha256": "65eb0ec5c903366243ca8bd5ea07412e8d34bdb81e6015fa76b3c4c59838733f", + "path": "openhcs/core/artifacts/kinds.py", + "sha256": "216b913efad9ec76707e9f61be445fb6fa1c6601c36ef53e296acd4928604515", "role": "Nominal artifact types exchanged by compiled steps and runtime consumers." }, { @@ -295,15 +298,14 @@ }, { "path": "openhcs/core/config.py", - "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", + "sha256": "9625ca054fac6c1e460216cfe0e85621f5306592a004d1bdc28a6b1ce2099366", "role": "Declares SourceBindingsConfig, the pipeline-level source-binding configuration (moved here by K4)." } ], - "findings": [ - "Audit record only: after K4 the SourceBindingsConfig class lives in openhcs/core/config.py, so the source_bindings.py role was narrowed to NamedSourceBinding and the SourceSetPairing family and config.py was added; the page prose names no moved class and needed no change." - ], + "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/concepts/building_intuition.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compiler.py, source_bindings.py, dataset_sources/source.py, openhcs/core/artifacts.py (now the openhcs/core/artifacts package, K2; the authority is its kinds.py), openhcs/core/config.py; the diffs are E1 class families replacing enum members and DatasetSource losing get_required_backend; the page states none of them, and force_disk_output is still absent from the tree as the page says; no page change.", "On 2026-10-11 (B6), re-read docs/source/concepts/building_intuition.rst in full and compared it against the changed authorities: compiler.py (ScopeKey scope ids, progress classes, empty CompiledSourceBindingPlan), source_bindings.py (K4: config class moved to config.py, SourceSetPairing family replacing SourceBindingMatchMethod), dataset_sources/source.py and processing_contracts.py (CallableDeclaration rename); the page's owner checklist (DatasetSource, PolyStore, ArrayBridge, pyqt-reactive) and its no force_disk_output claim still hold.", "On 2026-10-11 (B6 final), compared docs/source/concepts/building_intuition.rst against B6's later changes to openhcs/core/pipeline/compiler.py, openhcs/core/dataset_sources/source.py, openhcs/core/processing_contracts.py, openhcs/core/artifacts.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/concepts/building_intuition.rst against the later B6 changes to openhcs/core/pipeline/compiler.py, openhcs/core/source_bindings.py, openhcs/core/dataset_sources/source.py, openhcs/core/processing_contracts.py, openhcs/core/artifacts.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -342,18 +344,19 @@ }, { "path": "openhcs/core/component_group_scope.py", - "sha256": "9fc6006db14e4533ca0b45204a529ab23bac8c69382c8f1f1f883fa87179cd3c", + "sha256": "1d95c82d935a2441c69b4e0046eaa078dc1add7befa7f707b5e76867cf1dce39", "role": "Declares ComponentGroupScope and RuntimeExecutionAxisScope." }, { "path": "tests/unit/test_axis_family_guards.py", - "sha256": "2c70f7e629eb5c67978aa389b632ea3a806d693a261f93d87660b52ffd178b03", + "sha256": "699c109686040b978d9007f60b6124d3698a794711212b292348d61dd1d5fa2a", "role": "Proves the component enums are deleted and kernel modules name no microscopy member." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/concepts/component_system_concepts.rst in full and compared it against the changed authorities: openhcs/core/component_group_scope.py, tests/unit/test_axis_family_guards.py; ComponentGroupScope gained is_grouped, is_static, excludes and one_per_component, and the guard test's allow-list follows the artifacts package move; ComponentGroupScope and RuntimeExecutionAxisScope still carry the roles the page gives them; no page change.", "On 2026-10-11 (B6), re-read docs/source/concepts/component_system_concepts.rst in full and compared it against the changed authorities: tests/unit/test_axis_family_guards.py (U3 deleted openhcs/core/path_cache.py, so the kernel microscopy-reference ceiling dropped from 72 to 70); the page names no pinned reference, and Axis, AxisFamily.active()/with_role(), the seven roles, Ungrouped, ComponentGroupScope, RuntimeExecutionAxisScope and LazyProcessingConfig.from_config were re-checked against the current tree.", "On 2026-10-11 (B6 final), compared docs/source/concepts/component_system_concepts.rst against B6's later changes to openhcs/core/axes.py, openhcs/domains/microscopy/axes.py, tests/unit/test_axis_family_guards.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/concepts/component_system_concepts.rst against the later B6 changes to openhcs/core/axes.py, openhcs/domains/microscopy/axes.py, tests/unit/test_axis_family_guards.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -382,12 +385,12 @@ }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "4a33908fb42874e6d57eeea3f5305793ddbda41d13356f0a78501af38dcdfeb4", + "sha256": "40b07b7f40aca39b19b95a8f61c1b0c8270f0ab0a29e8c36caa13f04d8fa539f", "role": "Creates one CompilationSession per partition-axis context and emits the compiled bundle after validation." }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "0f9cc4819b53d55b7e9446429b8c8ac1ff1a59f4902020ea1658495ce01d464c", + "sha256": "4946c221e34ec19957b5af2b1a54359d454ee840fb1f5e1fe2b3b306ac8e5426", "role": "Declares CompilationSession with the ResolvedPipelineDefinition, source workspace projection and CompiledStepPlan map." }, { @@ -397,37 +400,34 @@ }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "74b591773b7f653c957b2db33a1262925e5c02b71074695b6fc18d6120ed5934", + "sha256": "35fe71906bfd16842de347776a0c5cc846550ad490cc3628885129d746ec6f31", "role": "Declares RuntimeValueStore keyed by typed artifact keys and execution scope." }, { "path": "openhcs/core/callable_contract.py", - "sha256": "a8e73c5df804d78baf72710e210cb2ae63156657a074071e11fd868f4b19d3b4", + "sha256": "6089f63e72e1a47fc5f6bc7b443028ac8ec40d934898589fa03b5592c2273ab3", "role": "Declares CallableDeclaration (inherited by CallableContract) with the memory types, required axis roles, allowed group-by roles, processing contract, execution scope, runtime adapter and artifact spec fields." }, { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "b96f33b866fe6c9f8118ca70fcb7dff8e6a02cb29d79df67230a5e8b97df8002", + "sha256": "5aa8895c9f533603b7ef972202141e2439473f0604b97d66f5a027b8b3cd1c83", "role": "Lowers parsed .cppipe modules into ordinary source bindings and FunctionStep declarations." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "358d3eaf5952170f9e9e2dc9056fb08707e8fcf74682063c074b367a83c2648f", + "sha256": "6525348d097bda8d0265bc7c3f312dd8839ca9f65dd05f797fc82ba873e04547", "role": "Declares the source, artifact, memory, materialisation, streaming and worker fields of CompiledStepPlan." }, { "path": "openhcs/core/dataset_sources/source.py", - "sha256": "350bf5691a03522e9f54b1d8ab712c6beba1683aac4fa5159ea4e80c6041530d", + "sha256": "b3758a878fdf8ec28b6d9818ebfd96c5c470895575f7a79500c85f1490679281", "role": "Declares the DatasetSource family that interprets plate layouts and metadata." } ], - "findings": [ - "The Sources section still credited \"microscope handlers\" with interpreting plate layouts; G4 replaced them with the DatasetSource family. Reworded and added dataset_sources/source.py as an authority.", - "The diagram still read \"snapshots + compilation sessions\" after step snapshots were deleted; it now reads \"resolved pipeline + compilation sessions\".", - "K5 moved the callable fields from CallableMetadata to CallableDeclaration, which CallableContract inherits; the callable_contract.py role was updated (the page's field list still holds)." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/concepts/core_model.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compiler.py, compilation_session.py, runtime_stores.py, callable_contract.py, openhcs/interop/cellprofiler/pipeline_import.py, compiled_step_plan.py, dataset_sources/source.py; the diffs are imports from the openhcs/core/artifacts package and E1 class families (input source, materialization backend, orchestrator state) replacing enum members; the page's diagram, declaration, compilation and runtime statements name none of the replaced members and every named class exists; no page change.", "On 2026-10-11 (B6), re-read docs/source/concepts/core_model.rst in full and compared it against the changed authorities: pipeline_document.py (pycodify generate_python_source), compiler.py (ScopeKey, progress classes, empty source plans), compilation_session.py (graph-level declaration validation, disabled items dropped by normalisation), runtime_stores.py (K6 codec removal), callable_contract.py (K5 CallableDeclaration with artifact fields and repeated_return_slots), pipeline_import.py (K4 imports) and compiled_step_plan.py (empty plan defaults); corrected the microscope-handler and snapshot wording and re-checked the callable-contract field list, CompilationSession per partition value, CompiledExecutionBundle and RuntimeValueStore claims.", "On 2026-10-11 (B6 final), compared docs/source/concepts/core_model.rst against B6's later changes to openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/runtime_stores.py, openhcs/core/callable_contract.py, openhcs/interop/cellprofiler/pipeline_import.py, openhcs/core/compiled_step_plan.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/concepts/core_model.rst against the later B6 changes to openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/runtime_stores.py, openhcs/core/callable_contract.py, openhcs/interop/cellprofiler/pipeline_import.py, openhcs/core/compiled_step_plan.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -464,7 +464,7 @@ }, { "path": "openhcs/core/component_group_scope.py", - "sha256": "9fc6006db14e4533ca0b45204a529ab23bac8c69382c8f1f1f883fa87179cd3c", + "sha256": "1d95c82d935a2441c69b4e0046eaa078dc1add7befa7f707b5e76867cf1dce39", "role": "Declares ComponentGroupScope and RuntimeExecutionAxisScope carried by artifact keys." }, { @@ -474,17 +474,17 @@ }, { "path": "openhcs/processing/backends/cellprofiler/watershed.py", - "sha256": "2c8c352bdc5fedad6baf14e4c1a3cd5347a1f862fd841472241680294c2762ca", + "sha256": "7fb797d16fc740c381d7dc9add3a0ff79ed2d82515850597c7995857bdacc206", "role": "Declares Watershed with FullStackExecution so it consumes the complete payload." }, { "path": "openhcs/processing/backends/cellprofiler/primary_objects.py", - "sha256": "b5b8a9b98dd9166b34a8f5cebf884e8af6da85f02fe878ef1227d6656d31ae4c", + "sha256": "4d28a02d5c25e4700df1affdbf414e587cde7b7d348917ee29e862e5135466cc", "role": "Declares IdentifyPrimaryObjects with Pure2DContract." }, { "path": "openhcs/processing/backends/cellprofiler/secondary.py", - "sha256": "f731f95f874ba3368828ca3a539fac4e5c6eb0ec7049e546f45fb874e0888e95", + "sha256": "e84db4b2bfa56492801e24fc9471e09c4ada4daf5b3541463cf3424ed227af2b", "role": "Declares IdentifySecondaryObjects and IdentifyTertiaryObjects with Pure2DContract." }, { @@ -496,6 +496,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/concepts/data_dimensions.rst in full and compared it against the changed authorities: openhcs/core/component_group_scope.py, openhcs/processing/backends/cellprofiler/watershed.py, primary_objects.py, secondary.py; Watershed's runtime family became WatershedRuntimeStrategy subclasses and IdentifySecondaryObjects declares its source artifact type as a class attribute; neither touches the callable declarations that make Watershed consume a complete 3D payload and the Identify modules run plane by plane, and the four ProcessingContract subclasses and the PAYLOAD/PLANE label-domain scopes still exist; no page change.", "On 2026-10-11 (B6), re-read docs/source/concepts/data_dimensions.rst in full and compared it against the changed authorities: openhcs/core/processing_contracts.py (CallableMetadata renamed CallableDeclaration at three call sites), openhcs/processing/backends/cellprofiler/watershed.py (StepSourceBindingsConfig imported from openhcs.core.config after K4) and openhcs/processing/backends/cellprofiler/secondary.py (repeated_return_slots for identical tertiary inputs); the page names none of these, and the four contract classes, the PAYLOAD/PLANE label domains, the per-plane Identify modules and the ProcessingConfig example still hold.", "On 2026-10-11 (B6 final), compared docs/source/concepts/data_dimensions.rst against B6's later changes to openhcs/core/axes.py, openhcs/core/processing_contracts.py, openhcs/processing/backends/cellprofiler/watershed.py, openhcs/processing/backends/cellprofiler/primary_objects.py, openhcs/processing/backends/cellprofiler/secondary.py, tests/unit/test_documented_dimensionality.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/concepts/data_dimensions.rst against the later B6 changes to openhcs/core/axes.py, openhcs/core/processing_contracts.py, openhcs/processing/backends/cellprofiler/watershed.py, openhcs/processing/backends/cellprofiler/primary_objects.py, openhcs/processing/backends/cellprofiler/secondary.py, tests/unit/test_documented_dimensionality.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -521,23 +522,24 @@ "authority": [ { "path": "openhcs/core/dataset_sources/source.py", - "sha256": "350bf5691a03522e9f54b1d8ab712c6beba1683aac4fa5159ea4e80c6041530d", + "sha256": "b3758a878fdf8ec28b6d9818ebfd96c5c470895575f7a79500c85f1490679281", "role": "Declares the registered DatasetSource family that projects each microscope format into axis values, backends and metadata." }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "4a33908fb42874e6d57eeea3f5305793ddbda41d13356f0a78501af38dcdfeb4", + "sha256": "40b07b7f40aca39b19b95a8f61c1b0c8270f0ab0a29e8c36caa13f04d8fa539f", "role": "Resolves inputs, axes, locality, memory conversion and artifact dependencies before emitting compiled plans." }, { "path": "openhcs/core/runtime_artifact_values.py", - "sha256": "930752071d37ef1a8669812e7a629e151690b3f33bf4a29dba92d1d479b9074d", + "sha256": "ce42e38f893459b9cb0ed6d779fabd5818204301ab1acc13ab0a7df32e6e3a2d", "role": "Declares ArtifactKey and RuntimeValue, the typed runtime result carrier." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/concepts/domain_fundamentals.rst in full and compared it against the changed authorities: openhcs/core/dataset_sources/source.py (E1: get_required_backend deleted, virtual-workspace root compared by absolute path), openhcs/core/pipeline/compiler.py (E1: OrchestratorState, MaterializationBackend and projection-role members are now classes) and openhcs/core/runtime_artifact_values.py (K2: imports from the artifacts package, RuntimeValue.payload_of added); the page names none of these members and its account of dataset sources, compile-time resolution and typed results still holds.", "On 2026-10-11 (B6), re-read docs/source/concepts/domain_fundamentals.rst in full and compared it against the changed authorities: openhcs/core/dataset_sources/source.py (SourceBindingsConfig now imported from openhcs.core.config after K4) and openhcs/core/pipeline/compiler.py (ScopeKey scope ids, CompileProgressPhase/RunningProgressStatus classes after K6, an empty CompiledSourceBindingPlan instead of None after K4); none changes the page's explanation of dataset sources, compile-time resolution or typed results.", "On 2026-10-11 (B6 final), compared docs/source/concepts/domain_fundamentals.rst against B6's later changes to openhcs/core/dataset_sources/source.py, openhcs/core/pipeline/compiler.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/concepts/domain_fundamentals.rst against the later B6 changes to openhcs/core/dataset_sources/source.py, openhcs/core/pipeline/compiler.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -562,12 +564,12 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", + "sha256": "9625ca054fac6c1e460216cfe0e85621f5306592a004d1bdc28a6b1ce2099366", "role": "Declares DtypeConfig as a DtypeConversionConfig subclass with NATIVE_OUTPUT as its default_dtype_conversion." }, { "path": "openhcs/core/steps/abstract.py", - "sha256": "9e710b3db635730f88258e5a6e18aed607876e296783022c06798f2bcd03d5e4", + "sha256": "2bffdfe02664c4974797bb6cefbf7f84ebb6912fcc11e026a3f2d8b3da5262b8", "role": "Declares the step-level dtype_config field separate from processing_config." }, { @@ -589,6 +591,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/concepts/dtype_config_system.rst in full and compared it against the changed authorities: openhcs/core/config.py (E1: MaterializationBackend and InputSource became class families; DtypeConfig, its NATIVE_OUTPUT default and annotation_type unchanged) and openhcs/core/steps/abstract.py (docstring now names ReadPreviousStep/ReadPipelineStart; dtype_config is still a step field separate from processing_config); no claim changed and the DtypeConfig example imports still resolve.", "On 2026-10-11 (B6), re-read docs/source/concepts/dtype_config_system.rst in full and compared it against the changed authorities: openhcs/core/config.py (K4 source-binding classes, PartitionFilterMode family; DtypeConfig unchanged with NATIVE_OUTPUT default), openhcs/core/steps/abstract.py (import regrouping only; dtype_config still a separate FunctionStep field) and tests/unit/test_callable_contract.py (CallableMetadata renamed CallableDeclaration, namespace round-trip test); dtype_config is still the one config-bound, overridable runtime parameter on a decorated callable's contract, and the DtypeConfig example still resolves.", "On 2026-10-11 (B6 final), compared docs/source/concepts/dtype_config_system.rst against B6's later changes to openhcs/core/config.py, openhcs/core/memory/decorators.py, tests/unit/test_callable_contract.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/concepts/dtype_config_system.rst against the later B6 changes to openhcs/core/config.py, openhcs/core/memory/decorators.py, tests/unit/test_callable_contract.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -615,7 +618,7 @@ "authority": [ { "path": "openhcs/core/callable_contract.py", - "sha256": "a8e73c5df804d78baf72710e210cb2ae63156657a074071e11fd868f4b19d3b4", + "sha256": "6089f63e72e1a47fc5f6bc7b443028ac8ec40d934898589fa03b5592c2273ab3", "role": "Declares the CallableDeclaration fields CallableContract inherits (memory types, artifact specs, runtime-bound parameters, required_axis_roles, allowed_group_by_roles, processing contract, runtime adapter, execution scope, prepared signatures) and processing_prepare, which attaches the preparation hook to the callable." }, { @@ -640,7 +643,7 @@ }, { "path": "openhcs/agent/services/function_catalog_service.py", - "sha256": "6cd88eca73ca960f32a1f46002cea805f8ffe723ec5e9e1bcd224991a806e164", + "sha256": "7ef506fe713be7a77d2a3eb6aa565cf313b43d7f79171014cb4e01543e63f4ae", "role": "Serves catalogue search and callable detail to MCP and GUI authoring from the execution endpoint." }, { @@ -669,11 +672,10 @@ "role": "Proves every locally available catalogue declaration materialises through the desktop function form." } ], - "findings": [ - "The contract list claimed CallableContract declares compiler preparation hooks; K5 removed prepare from the declaration, so the hook lives on the callable (processing_prepare) and the contract carries only the prepared signatures. The bullet was corrected and a sentence added; the callable_contract.py role was updated to name CallableDeclaration." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/concepts/function_library.rst in full and compared it against the changed authorities: openhcs/core/callable_contract.py (K2: artifact imports repointed to the package, artifact_type.value in one error message) and openhcs/agent/services/function_catalog_service.py (E1: search rank, signature view, summary view and parameter-supplier enums became class families); the page names none of them, and the contract field list, enum-member kwarg rule, catalogue and exclusion claims still hold.", "On 2026-10-11 (B6), re-read docs/source/concepts/function_library.rst in full and compared it against the changed authorities: callable_contract.py (K5: CallableMetadata merged into CallableDeclaration, prepare left the declaration, runtime_bound_parameter_names), processing_contracts.py (CallableDeclaration rename), registry_service.py and function_catalog_service.py (declaration/runtime_bound_parameter_names renames) and tests/unit/test_unified_registry_runtime_payloads.py (admitted_graph helper); corrected the preparation-hook claim and re-checked the remaining contract fields, the ProcessingContract family, and the catalogue and form-exclusion claims.", "On 2026-10-11 (B6 final), compared docs/source/concepts/function_library.rst against B6's later changes to openhcs/core/callable_contract.py, openhcs/core/processing_contracts.py, openhcs/processing/backends/lib_registry/unified_registry.py, openhcs/processing/backends/lib_registry/registry_service.py, openhcs/processing/backends/lib_registry/openhcs_registry.py, openhcs/agent/services/function_catalog_service.py, tests/unit/test_unified_registry_runtime_payloads.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/concepts/function_library.rst against the later B6 changes to openhcs/core/callable_contract.py, openhcs/core/processing_contracts.py, openhcs/processing/backends/lib_registry/unified_registry.py, openhcs/processing/backends/lib_registry/registry_service.py, openhcs/processing/backends/lib_registry/openhcs_registry.py, openhcs/agent/services/function_catalog_service.py, tests/unit/test_library_registry_discovery.py, tests/unit/test_unified_registry_runtime_payloads.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -710,23 +712,24 @@ }, { "path": "openhcs/core/function_patterns.py", - "sha256": "cc8c35f37ea0a3bd939905fe317652a34b3129ee6f47f6f9023b8c459e62df70", + "sha256": "c8004ae1e3f241c4ac6ce0a61c69e3c0d1ecfb5021e1bef5dd0fb1fd1251ca50", "role": "Declares AuthoredFunctionPattern (the single walker over callable, chain and dictionary syntax) and normalises patterns into CompiledFunctionPattern, whose execution_scope requires one uniform scope across invocations." }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "4a33908fb42874e6d57eeea3f5305793ddbda41d13356f0a78501af38dcdfeb4", + "sha256": "40b07b7f40aca39b19b95a8f61c1b0c8270f0ab0a29e8c36caa13f04d8fa539f", "role": "Validates per-invocation callable contracts and dictionary keys against compiled grouping." }, { "path": "tests/unit/test_function_patterns.py", - "sha256": "296c5ab1f5044b04ce631c92a1e8510fe9dd0e21a776d7a30877766154a74b18", + "sha256": "2eb64da61d0ae1ef2aedacddafc2f81b886515e9c500ac80b2cbcd49c48f5d9d", "role": "Proves callable, chain and dictionary patterns compile with their declared semantics." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/concepts/function_patterns.rst in full and compared it against the changed authorities: openhcs/core/function_patterns.py and tests/unit/test_function_patterns.py (K2: imports repointed to openhcs.core.artifacts.plans/specs/kinds/relations/input_projection only) and openhcs/core/pipeline/compiler.py (E1 class families for orchestrator state and materialization backend); the callable, chain and dictionary forms, the group_by example and the declaration-identity rule are unchanged.", "On 2026-10-11 (B6), re-read docs/source/concepts/function_patterns.rst in full and compared it against the changed authorities: function_patterns.py (K5 replaced the seven raw pattern walkers, strip_disabled_functions, inject_kwargs_into_pattern and inject_artifact_input_values with AuthoredFunctionPattern and NormalizedFunctionPattern methods), compiler.py (ScopeKey, progress classes) and tests/unit/test_function_patterns.py (renamed tests for normalisation dropping disabled items and artifact input values); the page names none of the removed helpers, and the pattern forms, dictionary routing example, uniform execution scope (CompiledFunctionPattern.execution_scope) and declaration-identity claims still hold.", "On 2026-10-11 (B6 final), compared docs/source/concepts/function_patterns.rst against B6's later changes to openhcs/core/function_patterns.py, openhcs/core/pipeline/compiler.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/concepts/function_patterns.rst against the later B6 changes to openhcs/core/function_patterns.py, openhcs/core/pipeline/compiler.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -757,18 +760,19 @@ }, { "path": "openhcs/core/config.py", - "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", + "sha256": "9625ca054fac6c1e460216cfe0e85621f5306592a004d1bdc28a6b1ce2099366", "role": "Declares the processing, dtype and storage configuration the processing-and-storage pages explain." }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "4a33908fb42874e6d57eeea3f5305793ddbda41d13356f0a78501af38dcdfeb4", + "sha256": "40b07b7f40aca39b19b95a8f61c1b0c8270f0ab0a29e8c36caa13f04d8fa539f", "role": "Compiles declarations into plans, the step the core-model and dimensions pages describe." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/concepts/index.rst in full and compared it against the changed authorities: openhcs/core/config.py (E1: MaterializationBackend and InputSource class families) and openhcs/core/pipeline/compiler.py (E1: orchestrator-state and backend classes); the page is a reading-order index that names no member of either, and every toctree target still exists.", "On 2026-10-11 (B6), re-read docs/source/concepts/index.rst in full and compared it against the changed authorities: openhcs/core/pipeline_document.py (source rendered through pycodify generate_python_source after L6), openhcs/core/config.py (source-binding config classes now declared there after K4) and openhcs/core/pipeline/compiler.py (ScopeKey scope ids, progress classes); the page only routes readers to the concept pages and every toctree entry still exists.", "On 2026-10-11 (B6 final), compared docs/source/concepts/index.rst against B6's later changes to openhcs/core/config.py, openhcs/core/pipeline/compiler.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/concepts/index.rst against the later B6 changes to openhcs/core/config.py, openhcs/core/pipeline/compiler.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -794,7 +798,7 @@ "authority": [ { "path": "pyproject.toml", - "sha256": "b5d4d752704b886da20d860743b81dc3d2ce3a3c402208246fadc1c6cb71f597", + "sha256": "75f92010b3c18ede61cf7296064079590c143283871e372a1fd4546417d4e0f9", "role": "Declares the eight first-party foundation distributions as OpenHCS dependencies." }, { @@ -854,7 +858,7 @@ }, { "path": "benchmark/control.py", - "sha256": "83fe9016625d7d8721905b8a153a28c3dcaaf2b9e925a3c97711c634868156b7", + "sha256": "b10dd87d7bc44063566e0c53d849091c0fc43a345b434c13467df1ede86ac892", "role": "Implements read-only case discovery, inspection and reporting over benchmark contracts." }, { @@ -883,11 +887,10 @@ "role": "Exports the generic widget set, including the editable table, grid selector and time-travel widgets." } ], - "findings": [ - "The foundations list predated U3 and L6: pyqt-reactive now owns the editable table, grid selector, time-travel widgets and windows moved out of openhcs/pyqt_gui; ObjectState owns ScopeKey and the lazy-dataclass source formatters (objectstate.codegen); pycodify owns document-level path factoring. The three lines were updated and the owners added as authorities." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/concepts/module_structure.rst in full and compared it against the changed authorities: pyproject.toml (pins raised to zmqruntime 0.6.2, objectstate 2.1.1, python-introspect 0.2.8, polystore 0.6.1; mahotas moved within the core dependencies; still eight first-party foundation distributions) and benchmark/control.py (the report prints the observation scope's name instead of an enum value); the ownership map, foundation list and benchmark paragraph are unchanged.", "On 2026-10-11 (B6), re-read docs/source/concepts/module_structure.rst in full and compared it against the changed authority pyproject.toml (zmqruntime 0.6.2, pycodify 0.2.0, objectstate 2.1.1, python-introspect 0.2.7, pyqt-reactive 0.5 floors) and the U3/L6 moves behind them; confirmed the openhcs/ area map, the eight foundations and the OMERO classes, and updated the ObjectState, pyqt-reactive and pycodify ownership lines.", "On 2026-10-11 (B6 final), compared docs/source/concepts/module_structure.rst against B6's later changes to openhcs/microscopes/omero.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/concepts/module_structure.rst against the later B6 changes to pyproject.toml, openhcs/microscopes/omero.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -945,12 +948,12 @@ }, { "path": "openhcs/core/config.py", - "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", + "sha256": "9625ca054fac6c1e460216cfe0e85621f5306592a004d1bdc28a6b1ce2099366", "role": "Declares ProcessingConfig (variable_components, group_by, input_source) and PipelineConfig." }, { "path": "openhcs/core/orchestrator/orchestrator.py", - "sha256": "99870516da6d64e1858f4dd8799d6e38ab6911b7b02bc29b252d2a23f7687410", + "sha256": "a7097eec86c5966552bcb546a2161d1a08e6a4c630e6b156de3b8354b0026b54", "role": "Declares PipelineOrchestrator.compile_pipelines over selected partition values, returning CompiledExecutionBundle." }, { @@ -960,12 +963,12 @@ }, { "path": "openhcs/authoring/session/compile_batch.py", - "sha256": "0cbe5a6b86a0fb5e9b92c4fc45f858c979031ac93594b317b9761c28f5aaf9ab", + "sha256": "3e7397caa5ea1b38c1abc47d363e37fc01b0b601ab1e6b37ed00226f7aa51f42", "role": "Compiles the selected datasets through the session before execution for GUI and MCP clients." }, { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "b96f33b866fe6c9f8118ca70fcb7dff8e6a02cb29d79df67230a5e8b97df8002", + "sha256": "5aa8895c9f533603b7ef972202141e2439473f0604b97d66f5a027b8b3cd1c83", "role": "import_cellprofiler_pipeline returns the lowered steps with a populated PipelineConfig." }, { @@ -977,6 +980,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/concepts/pipelines_and_steps.rst in full and compared it against the changed authorities: openhcs/core/config.py (E1: ProcessingConfig.input_source now holds an InputSource class, default ReadPreviousStep), openhcs/core/orchestrator/orchestrator.py and openhcs/authoring/session/compile_batch.py (E1: orchestrator states are classes) and openhcs/interop/cellprofiler/pipeline_import.py (same InputSource classes, K2 imports); the page describes previous-step and pipeline-start input in prose without naming the old members, its FunctionStep example still validates, and compile_pipelines still returns a CompiledExecutionBundle.", "On 2026-10-11 (B6), re-read docs/source/concepts/pipelines_and_steps.rst in full and compared it against the changed authorities: openhcs/core/pipeline_document.py (pycodify generate_python_source after L6), openhcs/core/config.py and openhcs/core/orchestrator/orchestrator.py (K4 moved the source-binding config classes and source_bindings_defaults_to_base into config.py), openhcs/interop/cellprofiler/pipeline_import.py and tests/unit/test_pipeline_document.py; the FunctionStep example, the func pattern forms, PipelineConfig source declarations, CellProfiler import result and PipelineOrchestrator.compile_pipelines returning CompiledExecutionBundle still hold.", "On 2026-10-11 (B6 final), compared docs/source/concepts/pipelines_and_steps.rst against B6's later changes to openhcs/core/config.py, openhcs/core/orchestrator/orchestrator.py, openhcs/authoring/session/compile_batch.py, openhcs/interop/cellprofiler/pipeline_import.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/concepts/pipelines_and_steps.rst against the later B6 changes to openhcs/core/config.py, openhcs/core/orchestrator/orchestrator.py, openhcs/authoring/session/compile_batch.py, openhcs/interop/cellprofiler/pipeline_import.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -1002,22 +1006,22 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", + "sha256": "9625ca054fac6c1e460216cfe0e85621f5306592a004d1bdc28a6b1ce2099366", "role": "Declares VFSConfig read/intermediate/materialization backends and the fieldless OpenHCS ZarrConfig and TiffConfig subtypes." }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "74b591773b7f653c957b2db33a1262925e5c02b71074695b6fc18d6120ed5934", + "sha256": "35fe71906bfd16842de347776a0c5cc846550ad490cc3628885129d746ec6f31", "role": "Declares RuntimeValueStore with explicit replace and retained observation history." }, { "path": "openhcs/core/steps/function_artifact_materialization.py", - "sha256": "2375f8e784d4d0a5116882ae588c0681940696289b252ec6bd4b53e503c8b20b", + "sha256": "dfa64f98f6033bb76d5a167da7d20c4bd531117dceb21a20bd1a8b14768462f6", "role": "Materialises planned function outputs and forms filenames without changing artifact metadata." }, { "path": "openhcs/core/pipeline/function_contracts.py", - "sha256": "ffc21dd3021a2316cfbd100c2d655867ce4bec619f2db7288e82aeb11c1930e8", + "sha256": "67a654102b1d60c0a0f90e64ccaf3a8c177035cd11a0346734f872b329783d74", "role": "Declares the @artifact_outputs decorator named by the page." }, { @@ -1042,18 +1046,19 @@ }, { "path": "tests/unit/test_runtime_value_store.py", - "sha256": "d20ae0d5160c79ba7b7c3291a052a606b3ed7dee3c0cd84a6cf013e17a127a23", + "sha256": "17e15a64a4e84437f6523f872bad11ea7d813c50c1c41fc86ca7822379a7ce27", "role": "Proves scoped typed runtime-value insertion, replacement and lookup." }, { "path": "tests/unit/test_tiff_output_compression.py", - "sha256": "2b08553609d3121987bfbb185bd3e5a332db76293f8ce7f25b170eb0b6d1b31f", + "sha256": "4db7b7c47ce17a9476c7af6984daaa14b2cea7c5ae972d32aa6e6ad9449d37da", "role": "Proves main-flow and named-artifact disk-TIFF writes round-trip exactly." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/concepts/storage_system.rst in full and compared it against the changed authorities: openhcs/core/config.py (E1: VFSConfig.materialization_backend now holds a MaterializationBackend class, default MaterializeToDisk; read_backend still defaults to Backend.AUTO and intermediate_backend to memory; TiffConfig and ZarrConfig still fieldless PolyStore subtypes), openhcs/core/runtime_stores.py and openhcs/core/pipeline/function_contracts.py (K2 imports only), openhcs/core/steps/function_artifact_materialization.py (K2: output_plan is required and viewer capture reads the artifact's streaming declaration; filename identity is still formed apart from artifact metadata), tests/unit/test_runtime_value_store.py and tests/unit/test_tiff_output_compression.py (K2 imports, NameByArtifactName class); the page names backends in prose (disk, Zarr) and no claim changed.", "On 2026-10-11 (B6), re-read docs/source/concepts/storage_system.rst in full and compared it against the changed authorities: openhcs/core/config.py (PartitionFilterMode family and the source-binding config classes after K4; VFSConfig, ZarrConfig and TiffConfig unchanged), openhcs/core/runtime_stores.py and tests/unit/test_runtime_value_store.py (hand-written RuntimeArtifactLocation/RuntimeArtifactAddress to_dict/from_dict replaced by python-introspect to_jsonable/dataclass_from_mapping after K6), openhcs/core/pipeline/function_contracts.py (CallableDeclaration rename, artifact_input_parameter_names); the page names none of the removed methods, and its backend intents, RuntimeValueStore replace/history and @artifact_outputs claims still hold.", "On 2026-10-11 (B6 final), compared docs/source/concepts/storage_system.rst against B6's later changes to openhcs/core/pipeline/function_contracts.py: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/concepts/storage_system.rst against B6's later changes to openhcs/core/config.py, openhcs/core/runtime_stores.py, openhcs/core/steps/function_artifact_materialization.py, openhcs/core/pipeline/function_contracts.py, tests/unit/test_runtime_value_store.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", @@ -1085,7 +1090,7 @@ "authority": [ { "path": "pyproject.toml", - "sha256": "b5d4d752704b886da20d860743b81dc3d2ce3a3c402208246fadc1c6cb71f597", + "sha256": "75f92010b3c18ede61cf7296064079590c143283871e372a1fd4546417d4e0f9", "role": "Declares the package version, Python requirement, dependency extras, scripts, and distribution metadata." }, { @@ -1095,7 +1100,7 @@ }, { "path": "openhcs/mcp/installed_demo.py", - "sha256": "10fccd5b2d4382dca19857c0f7b9eaefe4eccbc34b4ff1898e8fced689a90223", + "sha256": "f89ba094072c46c1ed5504c7b4e786aa3806776615330f4e32f2b07f19555100", "role": "Implements the installed-package MCP demonstration used by the getting-started verification flow." }, { @@ -1135,22 +1140,22 @@ }, { "path": "openhcs/desktop/update.py", - "sha256": "7bd5ab7abffdc20a9f6f85fb2e5f169ecda462e50b210b175d918453dbebfb3c", + "sha256": "23ffce8fe2362d8651ea90d2ef9601d193658e4fcef6aef8964e888f43014a3c", "role": "Builds a staged desktop-update plan from the active managed environment." }, { "path": "openhcs/desktop/update_worker.py", - "sha256": "d59781418acbcbd863cd843c6f61ddf7612d7b962136e41168033bc45dde5ed9", + "sha256": "917c5d2d4fb7693147f03c14096535d198151c65123b6d256a8b871cbecb54a7", "role": "Creates and verifies a sibling replacement environment before publishing it and preserves the previous environment." }, { "path": "openhcs/desktop/deployment.py", - "sha256": "4c6a7d63bf5ea759cc3eecd46e93fa335304113873869474f5fe6823f60c298f", + "sha256": "aafb7c77756cd3cd9de71bec7a6360058cd14e3aebc27b5e014c3b74271307cf", "role": "Owns platform-specific managed environment paths, launchers, shortcuts, and atomic desktop publication." }, { "path": "tests/unit/desktop/test_desktop_update_worker.py", - "sha256": "3a521574524aabb668bfee34c393a515c25ab98351fdf1bd46ae331e3aeda515", + "sha256": "b6b1a1d1e232c3de86db74b0ef7695a8af6225ddc48f33e2eb0a665a3dc5af93", "role": "Proves staged installation, verification, publication, rollback, and restart behaviour." }, { @@ -1167,6 +1172,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/getting_started/getting_started.rst in full and compared it against the changed authorities: pyproject.toml (python-introspect 0.2.8), openhcs/mcp/installed_demo.py, openhcs/desktop/update.py, openhcs/desktop/update_worker.py, openhcs/desktop/deployment.py and tests/unit/desktop/test_desktop_update_worker.py (E1: update-check origin, failed-update response and runtime platform became class families); the page names none of those types; the extras, Python range, openhcs and openhcs-mcp-demo scripts, and the verified-replacement update with the previous environment preserved are unchanged, so nothing was changed.", "On 2026-10-11 (B6), re-read docs/source/getting_started/getting_started.rst in full and compared it against the changed authorities: pyproject.toml (first-party floors moved to zmqruntime 0.6.2, pycodify 0.2, objectstate 2.1.1, python-introspect 0.2.7, pyqt-reactive 0.5). The page states no dependency floors; rechecked requires-python (3.11-3.14) and every named extra (gui, viz, bioformats, mcp, cellprofiler-compat, napari, fiji, gpu) against pyproject.toml; no claim changes.", "On 2026-10-11 (B6 final), compared docs/source/getting_started/getting_started.rst against B6's later changes to openhcs/mcp/installed_demo.py: the writability probe makes one exclusive create attempt, the installed demo reports ViewerType.value, the unit session pre-creates its QApplication, and the GUI journey waits for each batch to end. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/getting_started/getting_started.rst against B6's later changes to openhcs/mcp/installed_demo.py, .github/workflows/integration-tests.yml: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", @@ -1252,27 +1258,27 @@ "authority": [ { "path": "openhcs/pyqt_gui/main.py", - "sha256": "3cdb7819f09604046ba69e0b4f57b73e4e98d8f68af80158501a54bb42e701a2", + "sha256": "2d10806e310cb0d47ee7e47b879a7a95b28dcf5c688d9ce2a54b81ae1a041c47", "role": "Dock layout (system monitor above, Datasets and ZMQ server manager left, Pipeline Editor right), View/Tools/Help menus, and preview-policy application to both panes." }, { "path": "openhcs/pyqt_gui/widgets/plate_manager.py", - "sha256": "e816cdafd4795c07b5d177a04472221057230416311d1bf83c5bf65188d2aa00", + "sha256": "b6e56da089cff746c9f0f41ca4b4d947942d4a71d1b290a1d8b76c3b7732c31e", "role": "Implements plate import, selection, pipeline assignment, and execution controls." }, { "path": "openhcs/pyqt_gui/widgets/pipeline_editor.py", - "sha256": "7b3b58f3685d5a636a102cae7d1e31c663d5d574bcdc71d4a289b69a9eee6535", + "sha256": "f60e195031b99bd02d8e4cdf1b42ca213fc5669526bc30348ce3ddff14f6876b", "role": "Implements step editing, pipeline compilation controls, and declaration feedback in the main UI." }, { "path": "openhcs/agent/services/ui_bridge_service.py", - "sha256": "c7bdf8b5caec1c21c986cdc2224524839fadfa8764dafcd05125d90c72351899", + "sha256": "a2d682ad7841a7d85173f792961cae2dc5546fb4fcf39caa85f0bbd6ed062b01", "role": "Owns agent-to-UI declaration inspection and mutation, including code and form synchronisation." }, { "path": "tests/unit/pyqt_gui/test_ui_agent_bridge.py", - "sha256": "67cdad5133dc83ccb005b58cc56535ae89e77fc23d3ebd76e616c0da5f567b44", + "sha256": "422be9298e5120c898bd7110884ca770f1f033f07fd69be4ac0caa978d3014f4", "role": "Proves bidirectional UI-bridge inspection and mutation of live declarations, plus truthful rejection of unsupported window-navigation targets." }, { @@ -1282,7 +1288,7 @@ }, { "path": "tests/unit/pyqt_gui/test_pipeline_code_document_transactionality.py", - "sha256": "0bb9b43d867a4e26f13b51531a2b7a847c6c3685841f8e002e209200a37366b6", + "sha256": "6604f4cc9ff8d1f3824f0479c4947c7d264aff59c7b3e139d670b659b63b39ec", "role": "Proves Pipeline Editor code validates and applies pipeline_steps without an explicit pipeline_config." }, { @@ -1297,7 +1303,7 @@ }, { "path": "tests/unit/pyqt_gui/test_pipeline_editor_widget.py", - "sha256": "2024d9560dec38faee1fa922d2f8b7f547dd53d35e0b0e79376e45233445589a", + "sha256": "6761f5c3526167053b22edf5271f3413730d7798e8db45dbc57ec1baa52515aa", "role": "Proves Pipeline Editor code apply leaves the reconciled root, step, and nested function states saved." }, { @@ -1307,12 +1313,12 @@ }, { "path": "openhcs/pyqt_gui/services/ui_bridge_windows.py", - "sha256": "29265ed04137392757a6d515020920120e29fd556f433575956ae38cedff0158", + "sha256": "a10489e1bae296bc224b339ee32e2e99537175f891b86f901b4d7503d6f38f13", "role": "Projects embedded manager navigation through the generic declared driver." }, { "path": "openhcs/pyqt_gui/config.py", - "sha256": "a3eece2e5b2926fc6fc0edd197221da451ab7c0d0d6f4ffc8bb2ff865ad74bd4", + "sha256": "d092cb20206127484fbbc80a38a5152462b9b8991e848a814c85f564d51f4db8", "role": "Embeds the persisted generic list-preview policy in UIConfig." }, { @@ -1357,7 +1363,7 @@ }, { "path": "openhcs/authoring/session/views.py", - "sha256": "c772d515dfa738363ab403a3e19c9e438c17b0748a3a9ed65c44c416f579a378", + "sha256": "296adbceb7db7a7f7ac3cb09dbe8bb6620dec588a3b593ef2239108298bee5c8", "role": "DatasetListView and PipelineStepsView titles and the operations each pane binds." }, { @@ -1374,6 +1380,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/basic_interface.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/main.py, widgets/plate_manager.py, widgets/pipeline_editor.py, services/ui_bridge_windows.py, openhcs/pyqt_gui/config.py, openhcs/agent/services/ui_bridge_service.py, openhcs/authoring/session/views.py and the three test modules (E1: selection scope, orchestrator state, action status and update-check origin became class families; the error-dialog marker became a boolean); the page describes user-visible behaviour and names no changed type; all-plates versus selected code documents, Connect/Compile/Run/Stop control behaviour and the pane identities are unchanged, so nothing was changed.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/basic_interface.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/main.py, widgets/plate_manager.py, widgets/pipeline_editor.py, shared/services/pipeline_editor_workflows.py, services/ui_bridge_windows.py, openhcs/core/pipeline_document.py and their tests (U3: DockPane, DockLayoutStore, ShortcutLifecycle, time-travel navigation, path cache and the external-editor switch moved to pyqt-reactive, GlobalEventBus broadcasts replaced by session events, ScopeKey scope ids; L6: pipeline documents render through pycodify generate_python_source; K6 debug command classes). The page describes behaviour, not those classes; rechecked the dock arrangement (system monitor above, Datasets and ZMQ server manager stacked left, Pipeline Editor right), the plate_manager window identity, list_previews on UIConfig, and that PipelineDocument still requires pipeline_steps; no claim changes.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/basic_interface.rst against B6's later changes to openhcs/authoring/session/views.py: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/basic_interface.rst against B6's later changes to openhcs/pyqt_gui/main.py, openhcs/pyqt_gui/widgets/pipeline_editor.py, openhcs/agent/services/ui_bridge_service.py, tests/unit/pyqt_gui/test_ui_agent_bridge.py, tests/unit/pyqt_gui/test_pipeline_code_document_transactionality.py, openhcs/pyqt_gui/widgets/shared/services/pipeline_editor_workflows.py, tests/unit/pyqt_gui/test_pipeline_editor_widget.py, openhcs/pyqt_gui/services/ui_bridge_windows.py, tests/unit/pyqt_gui/test_main_config_propagation.py, openhcs/authoring/session/views.py, openhcs/authoring/session/operations/datasets.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", @@ -1421,7 +1428,7 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", + "sha256": "9625ca054fac6c1e460216cfe0e85621f5306592a004d1bdc28a6b1ce2099366", "role": "Declares the supported OpenHCS configuration dataclasses, nested scopes, defaults, and field help metadata." }, { @@ -1431,7 +1438,7 @@ }, { "path": "openhcs/agent/services/config_service.py", - "sha256": "902bb12178abf758cf0d7883c9cade8c5870f5362a3b5f7cc79e5640da9fb42d", + "sha256": "27a1d5d841b147e3ce35c3a358ca30b6d6efe6ed08a7db874648410b122a717f", "role": "Resolves and mutates configuration through the registered ObjectState-backed configuration services." }, { @@ -1441,12 +1448,12 @@ }, { "path": "openhcs/pyqt_gui/services/ui_bridge_object_state.py", - "sha256": "e64b444a74fdde1d79f82c1361c9b67d8e8916b9bebced092762ce51eabac52b", + "sha256": "f615718186e4aaebdc12eec6bec6a97d47953322075c5b067974e5a16ecc8e80", "role": "Projects declaration-derived accepted enum inputs through live ObjectState field help." }, { "path": "tests/unit/pyqt_gui/test_ui_agent_bridge.py", - "sha256": "67cdad5133dc83ccb005b58cc56535ae89e77fc23d3ebd76e616c0da5f567b44", + "sha256": "422be9298e5120c898bd7110884ca770f1f033f07fd69be4ac0caa978d3014f4", "role": "Proves live field help reports the exact accepted inputs for enum-backed configuration fields." }, { @@ -1483,6 +1490,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/configuration_reference.rst in full and compared it against the changed authorities: openhcs/core/config.py (E1: MaterializationBackend and InputSource are registered class families selected through RegisteredClassChoices), openhcs/agent/services/config_service.py (family-valued fields decode from their registry key), openhcs/pyqt_gui/services/ui_bridge_object_state.py and tests/unit/pyqt_gui/test_ui_agent_bridge.py; the page already says the schema reports enum and registry values and names no member of the changed families; the previous-step versus pipeline-start description of input_source and the config family list still hold, so nothing was changed.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/configuration_reference.rst in full and compared it against the changed authorities: openhcs/core/config.py (K4: SourceBindingsConfig and StepSourceBindingsConfig now declared here; PartitionFilterMode became the IncludePartitions/ExcludePartitions class family offered through RegisteredClassChoices), external/ObjectState/src/objectstate/object_state.py (current file), openhcs/pyqt_gui/services/ui_bridge_object_state.py (field inspection and provenance moved to objectstate.inspection), openhcs/core/pipeline_document.py (pycodify generate_python_source) and their tests. The page states include/exclude selection without naming enum members, and still-accurate omitted pipeline_config -> PipelineConfig() behaviour; rechecked every config class it names, TiffConfig's PolyStore DEFLATE codec, the microscopy-domain AnalysisConsolidationConfig/PlateMetadataConfig/ExperimentalAnalysisConfig, and openhcs_describe_config_schema; no claim changes.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/configuration_reference.rst against B6's later changes to openhcs/core/config.py, openhcs/pyqt_gui/services/ui_bridge_object_state.py, tests/unit/pyqt_gui/test_ui_agent_bridge.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/configuration_reference.rst against the later B6 changes to openhcs/core/config.py, openhcs/pyqt_gui/services/ui_bridge_object_state.py, tests/unit/pyqt_gui/test_ui_agent_bridge.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -1526,7 +1534,7 @@ }, { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "b96f33b866fe6c9f8118ca70fcb7dff8e6a02cb29d79df67230a5e8b97df8002", + "sha256": "5aa8895c9f533603b7ef972202141e2439473f0604b97d66f5a027b8b3cd1c83", "role": "Imports CellProfiler modules into ordinary OpenHCS pipeline declarations through the interop boundary." }, { @@ -1536,12 +1544,12 @@ }, { "path": "tests/unit/test_cellprofiler_pipeline_import.py", - "sha256": "0ce4efd85d819c7e9f7fa84ec6f1ac7ec4bda024c9a8d5c7f264fda2730264f0", + "sha256": "67576f37b7cbe0595acea633265f8e14677ca3c1b8204630413330273dc4e58a", "role": "Proves direct import rejects modules requiring interactive desktop input instead of omitting them." }, { "path": "benchmark/cellprofiler_benchmark_cli.py", - "sha256": "2e0de28a96c57ee150e28fbbed28d876507e9281075a03f729838f3ca3610966", + "sha256": "0c2cab4d989d50149cb314e908baace8aa794953c86ebc572fa120beaa695780", "role": "Owns packaged comparison execution and reporting commands." }, { @@ -1551,7 +1559,7 @@ }, { "path": "benchmark/control.py", - "sha256": "83fe9016625d7d8721905b8a153a28c3dcaaf2b9e925a3c97711c634868156b7", + "sha256": "b10dd87d7bc44063566e0c53d849091c0fc43a345b434c13467df1ede86ac892", "role": "Owns read-only inspection of existing benchmark directories." }, { @@ -1566,18 +1574,19 @@ }, { "path": "openhcs/interop/cellprofiler/module_artifact_declarations.py", - "sha256": "ef233be43002d0f16b53c78c202b9203b6261847708d311089671a6b037db894", + "sha256": "5442b56c7dd7c913d38d18c5a76660ad6006e147ffc8f790712fb79b301ff035", "role": "Raises the import error for CellProfiler modules that require interactive desktop input." }, { "path": "openhcs/authoring/session/views.py", - "sha256": "c772d515dfa738363ab403a3e19c9e438c17b0748a3a9ed65c44c416f579a378", + "sha256": "296adbceb7db7a7f7ac3cb09dbe8bb6620dec588a3b593ef2239108298bee5c8", "role": "DatasetListView title for the desktop pane that UI-owned work populates." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/domain_expert_onboarding.rst in full and compared it against the changed authorities: openhcs/interop/cellprofiler/pipeline_import.py, openhcs/interop/cellprofiler/module_artifact_declarations.py, tests/unit/test_cellprofiler_pipeline_import.py (E1 input-source classes, K2 artifact package), benchmark/cellprofiler_benchmark_cli.py, benchmark/control.py (export scope family) and openhcs/authoring/session/views.py; the page names none of the changed types; the CellProfiler translation list, the unsupported-interaction report and the read-only benchmark inspection routes are unchanged, so nothing was changed.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/domain_expert_onboarding.rst in full and compared it against the changed authorities: openhcs/interop/cellprofiler/pipeline_import.py, module_artifact_declarations.py, openhcs/agent/services/architecture_projection_service.py and tests/unit/test_cellprofiler_pipeline_import.py (K4: SourceBindingsConfig and StepSourceBindingsConfig now imported from openhcs.core.config, order matching is PairByOrder). The page names neither import location; rechecked the cellprofiler_translation topic id, the interactive-input import error, and the DatasetListView 'Datasets' title; no claim changes.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/domain_expert_onboarding.rst against B6's later changes to openhcs/authoring/session/views.py: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/domain_expert_onboarding.rst against B6's later changes to benchmark/cellprofiler_benchmark_cli.py: the session decides in-place initialization by an actual file-creation attempt that leaves nothing behind, its debug-session methods moved into an inherited mixin, the measured CLI treats Windows Ctrl+Break like Ctrl+C, and the Official30 viewer test reads typed payload summaries. No claim this page makes changes.", @@ -1618,7 +1627,7 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", + "sha256": "9625ca054fac6c1e460216cfe0e85621f5306592a004d1bdc28a6b1ce2099366", "role": "ProcessingConfig declares variable_components as the stack-axis authority and group_by as post-assembly partitioning." }, { @@ -1635,6 +1644,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/glossary.rst in full and compared it against the changed authorities: openhcs/core/config.py (E1: MaterializationBackend and InputSource class families); ProcessingConfig still declares variable_components and group_by with the meaning the three bullets give, and the page names neither changed family, so nothing was changed.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/glossary.rst in full and compared it against the changed authorities: openhcs/core/config.py (PartitionFilterMode subclass family, source-binding configs moved in by K4) and openhcs/core/processing_contracts.py (CallableDeclaration rename); variable_components and group_by remain on the processing config and the ProcessingContract family (Pure2DContract, WholeStackInputContract) still states plane-local versus whole-stack, so no correction was needed.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/glossary.rst against B6's later changes to openhcs/core/config.py, openhcs/core/processing_contracts.py, openhcs/core/axes.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/glossary.rst against the later B6 changes to openhcs/core/config.py, openhcs/core/processing_contracts.py, openhcs/core/axes.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -1651,7 +1661,7 @@ }, { "path": "docs/source/guide_for_biologists/image_sources.rst", - "source_sha256": "38ad499634f2423f4360a3f9b3bfdd96d39dca0327c9668ab5e5fd801ae4288a", + "source_sha256": "fec4bfc289273a852837ae9adbbfa5a9edad775d9d0af74770f662d64969479e", "audience": [ "imaging scientists", "source-integration operators" @@ -1661,27 +1671,27 @@ "authority": [ { "path": "openhcs/core/source_bindings.py", - "sha256": "2defaf02ac0e214bb63f74d6eb56f294fbb5db4e06ecd9cc4479b16848f09854", - "role": "Declares NamedSourceBinding (required defaults to True, component_identity), SourceSelector, ComponentSelector and the SourceFilterClause/Subject/MatchType types imported by the code examples." + "sha256": "890473715373d1adc6b12fec2b57eab6f179cbca7d837920c0388d620ce8b8c2", + "role": "Declares NamedSourceBinding (required defaults to True, component_identity), SourceSelector, ComponentSelector, SourceFilterClause and the SourceFileName and TextEquals family members passed by the code examples." }, { "path": "openhcs/core/source_workspace_projection.py", - "sha256": "2d2b6bf5a331cb61411b293b333980a9951f9ec1cd8537874d6529949180f3ec", + "sha256": "1f8785fec7919f52b69578bbd6f6b5f94be8b2f9e97d11d00f639f3f70bcfe23", "role": "Projects source bindings and persisted semantic metadata into the concrete workspace identities compiled for execution." }, { "path": "tests/unit/test_source_bindings.py", - "sha256": "28eefba0de5b84f9a0f928225590d70f8ed87d5048e7f4597cdb5e1e61b3fe45", + "sha256": "adcd46644c5005b9106a527b0c47d3c7d6821f4923213213c3a8f4c04c4934db", "role": "Proves source-binding validation and projection into concrete workspace identities." }, { "path": "openhcs/core/source_binding_workspace.py", - "sha256": "5e7ac439d7ef6cd35a5660d2ad9db24f1890801d8bdf1d82c13b12f8baaa9eb0", + "sha256": "e242d9b932d52c1099756881870f926a273f5ec08663938347580cb4a70e8283", "role": "Selects raw candidates and projects binding-owned component identity." }, { "path": "tests/unit/test_source_binding_workspace.py", - "sha256": "c11a1e40bebc3b025ecad2faeeb1fb1e5421f59042cb91d2428dfbf550112374", + "sha256": "98461001b494da536b8bd13e139e3ee01263b65b430b69bec8ce33ad32100dec", "role": "Proves raw selector coordinates are retained as original metadata while the projected address uses semantic component identity." }, { @@ -1691,7 +1701,7 @@ }, { "path": "openhcs/microscopes/bioformats_adapter.py", - "sha256": "ce3cd53ee704b9a28a9a108642f6d6e7434b7a8b53114c77fd60d734b59ebd5c", + "sha256": "e81247a7088748fed07ed290f33fe49194fd894c446d287138d000c2a91213f3", "role": "Projects native OME-Zarr metadata into source planes, validates named axes and single-resolution images, and keeps Java-backed CZI and OME-TIFF decoding separate." }, { @@ -1716,12 +1726,12 @@ }, { "path": "tests/unit/test_plane_store_sources.py", - "sha256": "806ca01b3d5aae89c1601956f1f5f0cb556abc81d899cea25b5489a4fb6759f5", + "sha256": "619a8bc276c116bc0e92ef9268e59fc57f9abd0f5819dffecb598fe3cdf1510c", "role": "Proves both physical NGFF formats retain axis cardinalities, pixels, source identities, bindings, materialisation, and reinitialisation through OpenHCS." }, { "path": "tests/integration/test_source_store_code_mode_zmq.py", - "sha256": "1330cbcbc6c2a9cc9e320db91e9bb7ff28c2768c8b9537fb3d6f5eb09861331e", + "sha256": "9edac8c05b1682e688825bb5b3c1a69c12cb42a46a27b0fc3dc41a917378628c", "role": "Exercises mixed-store code documents, source projection and the opt-in compile-then-execute ZMQ path for both NGFF formats." }, { @@ -1735,28 +1745,28 @@ "role": "Proves format-independent store discovery, legacy metadata precedence, declared resolution lookup, malformed-metadata rejection, and dimension-name validation." }, { - "path": "openhcs/core/artifacts.py", - "sha256": "65eb0ec5c903366243ca8bd5ea07412e8d34bdb81e6015fa76b3c4c59838733f", + "path": "openhcs/core/artifacts/relations.py", + "sha256": "b1ae8be613fbabc372bd6692b41c5f925d4f968291f0a715bd1473d85cbed000", "role": "Owns InputImageSetContextSourceRelation and its exact input-image and context-bearing target constraints." }, { "path": "openhcs/core/runtime_stores.py", - "sha256": "74b591773b7f653c957b2db33a1262925e5c02b71074695b6fc18d6120ed5934", + "sha256": "35fe71906bfd16842de347776a0c5cc846550ad490cc3628885129d746ec6f31", "role": "RuntimeArtifactInput resolves exact artifact addresses and checks execution context through the declared image-set relation; it does not select arbitrary same-shape labels." }, { "path": "openhcs/core/source_matching.py", - "sha256": "f32db4e25a0b6b7c1a334ee27c7206044e1ba645717ee12f6acd69d28d9b30a8", + "sha256": "cea83c632056420b4e2affac25f9da37c3a4ee41878e2a1fb18b6747d8573dcf", "role": "SourceImageSetIdentityPolicy derives plane membership and retained sample identity from source declarations." }, { "path": "openhcs/interop/cellprofiler/module_artifact_declarations.py", - "sha256": "ef233be43002d0f16b53c78c202b9203b6261847708d311089671a6b037db894", + "sha256": "5442b56c7dd7c913d38d18c5a76660ad6006e147ffc8f790712fb79b301ff035", "role": "The existing object-measurement declaration ancestor derives label input image-set relations from its selected image specs." }, { "path": "openhcs/core/dataset_sources/source.py", - "sha256": "350bf5691a03522e9f54b1d8ab712c6beba1683aac4fa5159ea4e80c6041530d", + "sha256": "b3758a878fdf8ec28b6d9818ebfd96c5c470895575f7a79500c85f1490679281", "role": "DatasetSource family with format-specific, broad structured-store and declared-file selection roles and their detection order." }, { @@ -1766,7 +1776,7 @@ }, { "path": "openhcs/agent/dto/plate.py", - "sha256": "689e7e720190f7abc2e0229373b88c5c60b2059efa4f7a7de14aa2d74dea3e69", + "sha256": "a54d0252c8c48db29e976222b5e8136943426c3081ed53cda06ea6eaf9826253", "role": "PlatePathInspectionRequest source_format field and format_specific_handler_candidates result field." }, { @@ -1781,16 +1791,33 @@ }, { "path": "openhcs/core/config.py", - "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", + "sha256": "9625ca054fac6c1e460216cfe0e85621f5306592a004d1bdc28a6b1ce2099366", "role": "Declares SourceBindingsConfig and StepSourceBindingsConfig with their lazy forms LazySourceBindingsConfig and LazyStepSourceBindingsConfig, and PipelineConfig.source_bindings_config, used by the code-mode examples." + }, + { + "path": "openhcs/core/artifacts/specs.py", + "sha256": "c4c50309a195bf85e1c0641413544d9581fc0ce1197043130126f7a4c70d8c4e", + "role": "ArtifactSpec.input, the constructor the input-contract fragment calls with parameter_name and relations." + }, + { + "path": "openhcs/core/artifacts/kinds.py", + "sha256": "216b913efad9ec76707e9f61be445fb6fa1c6601c36ef53e296acd4928604515", + "role": "ImageArtifactType and ObjectLabelsArtifactType, the artifact kinds the input-contract fragment declares." + }, + { + "path": "openhcs/constants/input_source.py", + "sha256": "4a7929e66d11d5ad81c40f080ba8cd23d466797bfa780d59eae0e1aea3ffd9a4", + "role": "InputSource family; ReadPipelineStart is the member the neurite and code-mode examples pass as input_source." } ], "findings": [ - "The page said 'A failed receipt remains preserved'; 'receipt' is internal jargon, so it now says to keep the record of a failed run.", - "K4 moved SourceBindingsConfig, StepSourceBindingsConfig and their lazy forms into openhcs/core/config.py; the page already imports the lazy forms from openhcs.core.config, so its claims hold, but config.py is added as the authority for them and the source_bindings.py role is narrowed to the binding, selector and filter types the examples import." + "E1 turned SourceFilterSubject, SourceFilterMatchType and InputSource into class families, so the mixed-store code example failed at SourceFilterSubject.FILE, SourceFilterMatchType.EQUALS and InputSource.PIPELINE_START; it now imports and passes SourceFileName, TextEquals and ReadPipelineStart.", + "The prose told readers to set input_source=PIPELINE_START in two places and named PIPELINE_START as a re-entry point in a third; the first two now say ReadPipelineStart and the third says the pipeline start.", + "K2 made openhcs/core/artifacts a package; the page's imports were already updated, but the audit still named the deleted openhcs/core/artifacts.py as the owner of InputImageSetContextSourceRelation. The authority is now openhcs/core/artifacts/relations.py." ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/image_sources.rst in full and compared it against the changed authorities: openhcs/core/source_bindings.py, openhcs/core/source_matching.py, openhcs/core/source_binding_workspace.py, openhcs/core/source_workspace_projection.py, openhcs/core/config.py and openhcs/agent/dto/plate.py (E1 class families), openhcs/core/artifacts.py (K2: now the openhcs/core/artifacts package), openhcs/core/runtime_stores.py, openhcs/interop/cellprofiler/module_artifact_declarations.py, openhcs/core/dataset_sources/source.py, openhcs/microscopes/bioformats_adapter.py and the four test modules; rewrote the mixed-store code example to pass SourceFileName, TextEquals and ReadPipelineStart, replaced the three prose uses of PIPELINE_START, and confirmed the K2 artifact imports in the input-contract fragment resolve to specs.py, kinds.py and relations.py; all three Python blocks were then executed against the worktree and ran.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/image_sources.rst in full and compared it against the changed authorities: openhcs/core/source_bindings.py (K4: SourceBindingsConfig/StepSourceBindingsConfig moved to openhcs/core/config.py, match plan became the SourceSetPairing family PairByOrder/PairByMetadata), openhcs/core/source_workspace_projection.py, openhcs/core/source_binding_workspace.py, openhcs/core/runtime_stores.py, openhcs/core/source_matching.py, openhcs/interop/cellprofiler/module_artifact_declarations.py, openhcs/core/dataset_sources/source.py, openhcs/core/dataset_sources/source_bindings_source.py and their tests; confirmed every symbol the code blocks import still exists at its import path (validate_docs reports no import or code-block finding for the page), NamedSourceBinding.required defaults to True, and InputImageSetContextSourceRelation and SourceImageSetIdentityPolicy are unchanged; replaced the 'failed receipt' wording and added openhcs/core/config.py as an authority.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/image_sources.rst against B6's later changes to openhcs/core/source_workspace_projection.py, tests/unit/test_source_bindings.py, openhcs/core/source_binding_workspace.py, tests/unit/test_source_binding_workspace.py, tests/unit/test_plane_store_sources.py, openhcs/core/artifacts.py, openhcs/core/runtime_stores.py, openhcs/interop/cellprofiler/module_artifact_declarations.py, openhcs/core/dataset_sources/source.py, openhcs/core/dataset_sources/source_bindings_source.py, openhcs/agent/dto/plate.py, openhcs/domains/microscopy/axes.py, openhcs/core/axes.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/image_sources.rst against the later B6 changes to openhcs/core/source_bindings.py, openhcs/core/source_workspace_projection.py, tests/unit/test_source_bindings.py, openhcs/core/source_binding_workspace.py, tests/unit/test_source_binding_workspace.py, tests/unit/test_plane_store_sources.py, openhcs/core/artifacts.py, openhcs/core/runtime_stores.py, openhcs/interop/cellprofiler/module_artifact_declarations.py, openhcs/core/dataset_sources/source.py, openhcs/core/dataset_sources/source_bindings_source.py, openhcs/agent/dto/plate.py, openhcs/domains/microscopy/axes.py, openhcs/core/axes.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -1822,7 +1849,7 @@ "authority": [ { "path": "openhcs/pyqt_gui/main.py", - "sha256": "3cdb7819f09604046ba69e0b4f57b73e4e98d8f68af80158501a54bb42e701a2", + "sha256": "2d10806e310cb0d47ee7e47b879a7a95b28dcf5c688d9ce2a54b81ae1a041c47", "role": "Composes the main desktop window, menus, task windows, and registered UI services." }, { @@ -1832,13 +1859,14 @@ }, { "path": "openhcs/demo/synthetic_plate_pipeline.py", - "sha256": "3709ac48de4d6937ae8bfe58a4f7bfe81fe5e198c719931da86dcf6b11da2fb0", + "sha256": "c175dfae6455690c554952a5f7202569fc0aa2c0aec0999c2d8112475587f80d", "role": "The included eight-step pipeline the Learn by doing entry promises." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/index.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/main.py and openhcs/demo/synthetic_plate_pipeline.py; both diffs are E1 enum-to-family renames (SelectedItems/AllItems, update-check origins, ReadPipelineStart, PadToMaxSize); the demo pipeline still declares eight steps and the page names no renamed symbol, so no correction was needed.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/index.rst in full and compared it against the changed authority openhcs/pyqt_gui/main.py (U3: dock layout, shortcut lifecycle and time-travel navigation now come from pyqt-reactive); the page only routes readers and names no main-window mechanism, so no correction was needed.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/index.rst against B6's later changes to openhcs/pyqt_gui/main.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/index.rst against the later B6 changes to openhcs/pyqt_gui/main.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -1864,7 +1892,7 @@ "authority": [ { "path": "pyproject.toml", - "sha256": "b5d4d752704b886da20d860743b81dc3d2ce3a3c402208246fadc1c6cb71f597", + "sha256": "75f92010b3c18ede61cf7296064079590c143283871e372a1fd4546417d4e0f9", "role": "Declares the package version, Python requirement, dependency extras, scripts, and distribution metadata." }, { @@ -1884,10 +1912,11 @@ } ], "findings": [ - "The page is still only a redirect to getting_started; the pyproject.toml dependency-floor changes (zmqruntime 0.6.2, pycodify 0.2, objectstate 2.1.1, python-introspect 0.2.7, pyqt-reactive 0.5) touch no claim it makes." + "The page is a redirect stub; its link target still exists and owns the content (checked 2026-10-11, B6)." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/installation_and_setup.rst in full and compared it against the changed authorities: pyproject.toml; the python-introspect floor moved to 0.2.8; the page states no dependency fact and docs/source/getting_started/getting_started.rst still exists, so no correction was needed.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/installation_and_setup.rst in full and compared it against the changed authorities: pyproject.toml (dependency floors only); the page restates no version or installer fact, so the redirect disposition stands.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/installation_and_setup.rst against the later B6 changes to pyproject.toml: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/installation_and_setup.rst in full and compared it against the changed authorities: pyproject.toml, openhcs/cli.py, packaging/installers/windows/Install-OpenHCS.ps1, packaging/installers/macos/install-openhcs.sh; the page contains only the redirect sentence and its target getting_started.rst was re-reviewed in this batch, so the redirect disposition stands.", @@ -1932,12 +1961,12 @@ "authority": [ { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "4a33908fb42874e6d57eeea3f5305793ddbda41d13356f0a78501af38dcdfeb4", + "sha256": "40b07b7f40aca39b19b95a8f61c1b0c8270f0ab0a29e8c36caa13f04d8fa539f", "role": "Compiles pipeline declarations into validated step plans and execution bundles." }, { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "b96f33b866fe6c9f8118ca70fcb7dff8e6a02cb29d79df67230a5e8b97df8002", + "sha256": "5aa8895c9f533603b7ef972202141e2439473f0604b97d66f5a027b8b3cd1c83", "role": "Imports CellProfiler modules into ordinary OpenHCS pipeline declarations through the interop boundary." }, { @@ -1947,7 +1976,7 @@ }, { "path": "openhcs/pyqt_gui/widgets/plate_manager.py", - "sha256": "e816cdafd4795c07b5d177a04472221057230416311d1bf83c5bf65188d2aa00", + "sha256": "b6e56da089cff746c9f0f41ca4b4d947942d4a71d1b290a1d8b76c3b7732c31e", "role": "Implements dataset import, selection, initialization, compilation and run controls in the Datasets pane." }, { @@ -1959,6 +1988,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/intro.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compiler.py, openhcs/interop/cellprofiler/pipeline_import.py and openhcs/pyqt_gui/widgets/plate_manager.py; the diffs replace OrchestratorState, MaterializationBackend, InputSource and selection-mode enum members with subclass families and move artifact imports into the openhcs.core.artifacts package; compile-before-run, .cppipe import and the dataset_source choices are unchanged, so no correction was needed.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/intro.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compiler.py (ScopeKey scope ids, K6 progress classes), openhcs/interop/cellprofiler/pipeline_import.py and openhcs/core/dataset_sources/choice.py (source-binding config imports from openhcs.core.config), openhcs/processing/backends/lib_registry/registry_service.py (reference.declaration replaces reference.metadata) and openhcs/pyqt_gui/widgets/plate_manager.py (U3: path cache and config window from pyqt-reactive, GlobalEventBus broadcast removed); the page's capability, dataset_source choice and compile-before-run claims still hold, so no correction was needed.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/intro.rst against B6's later changes to openhcs/core/pipeline/compiler.py, openhcs/interop/cellprofiler/pipeline_import.py, openhcs/processing/backends/lib_registry/registry_service.py, openhcs/core/dataset_sources/choice.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/intro.rst against the later B6 changes to openhcs/core/pipeline/compiler.py, openhcs/interop/cellprofiler/pipeline_import.py, openhcs/processing/backends/lib_registry/registry_service.py, openhcs/core/dataset_sources/choice.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -1998,7 +2028,7 @@ }, { "path": "openhcs/demo/synthetic_plate_pipeline.py", - "sha256": "3709ac48de4d6937ae8bfe58a4f7bfe81fe5e198c719931da86dcf6b11da2fb0", + "sha256": "c175dfae6455690c554952a5f7202569fc0aa2c0aec0999c2d8112475587f80d", "role": "Declares the eight supplied steps, from Image Enhancement Processing to Cell Counting, that the tutorial tells readers to expect." }, { @@ -2008,7 +2038,7 @@ }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "3cdb7819f09604046ba69e0b4f57b73e4e98d8f68af80158501a54bb42e701a2", + "sha256": "2d10806e310cb0d47ee7e47b879a7a95b28dcf5c688d9ce2a54b81ae1a041c47", "role": "Places Generate Synthetic Plate in the View menu." }, { @@ -2020,6 +2050,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/intro_stitching.rst in full and compared it against the changed authorities: openhcs/demo/synthetic_plate_pipeline.py and openhcs/pyqt_gui/main.py; the supplied pipeline still has eight steps running from Image Enhancement Processing to Cell Counting (only its InputSource and variable-size spellings became ReadPipelineStart and PadToMaxSize) and main.py still places Generate Synthetic Plate under View; no correction was needed.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/intro_stitching.rst in full and compared it against the changed authority openhcs/pyqt_gui/main.py (U3 dock, shortcut and time-travel moves to pyqt-reactive); the View menu still holds the Generate Synthetic Plate action, so no correction was needed.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/intro_stitching.rst against B6's later changes to openhcs/authoring/session/operations/datasets.py, openhcs/pyqt_gui/main.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/intro_stitching.rst against the later B6 changes to openhcs/authoring/session/operations/datasets.py, openhcs/pyqt_gui/main.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -2056,27 +2087,27 @@ }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "4a33908fb42874e6d57eeea3f5305793ddbda41d13356f0a78501af38dcdfeb4", + "sha256": "40b07b7f40aca39b19b95a8f61c1b0c8270f0ab0a29e8c36caa13f04d8fa539f", "role": "Stops before execution on unsatisfied declarations and applies materialize_runtime_artifacts to automatic terminal outputs." }, { "path": "openhcs/utils/environment.py", - "sha256": "2d8738e593b3e92c966089772624f47d17d38bb27fb90974707af659c91e8ad8", + "sha256": "1985ffbe42b5dfb283b78b6282f9b56e08cd51d5a9e79f2dd4eebffcb6da0bf5", "role": "Reads OPENHCS_CPU_ONLY to hide GPU-backed declarations." }, { "path": "openhcs/pyqt_gui/launch.py", - "sha256": "bf47d568e2d9bcb758342f34a5b3f030fd8abd9276c5e4fa2406666301e0561f", + "sha256": "fc018b94dc0aeaea4d1d7592973f5635419da5b53b85ecf07448bae975bb6e14", "role": "Defines the openhcs --no-gpu launch option." }, { "path": "openhcs/runtime/viewer_protocol.py", - "sha256": "c4bdae4bae6dee412f0d27a9cf75a95db53b3bd95c03cee7d046be16b52fe5b8", + "sha256": "347a65b63c149f2f437014d537cfaf481a33801d860742f4083b7377494e792f", "role": "Declares the viewer server contract whose readiness and dependency failures the viewer section diagnoses." }, { "path": "openhcs/runtime/zmq_execution_client.py", - "sha256": "5df650f60c4d48cbf47c275d4e46b870b35c915ca8baa8e06e77f321578ca6d7", + "sha256": "ad7ea01599493dbefce1f3c91e6b082e932c6a5183d0a85a186f7bd7b62779a9", "role": "Implements execution-server connection and retains the .startup.jsonl journal for a server that fails before readiness." }, { @@ -2086,7 +2117,7 @@ }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "3cdb7819f09604046ba69e0b4f57b73e4e98d8f68af80158501a54bb42e701a2", + "sha256": "2d10806e310cb0d47ee7e47b879a7a95b28dcf5c688d9ce2a54b81ae1a041c47", "role": "Wires endpoint compatibility to the version-restart workflow and gates the restart operation until initialisation, compilation and execution have finished." }, { @@ -2136,13 +2167,14 @@ }, { "path": "openhcs/pyqt_gui/services/ui_bridge_server.py", - "sha256": "4b0664fa62d24557294324b631ee3e39052c84e578d40ec408197bce5adcedac", + "sha256": "f93a3570a834eca8596139676629dda48c3f22991fd8cb58cd03427af8b24a38", "role": "Advertises the current application identity and rejects incompatible MCP requests before UI dispatch." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/troubleshooting_FAQ.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compiler.py, openhcs/utils/environment.py, openhcs/pyqt_gui/launch.py, openhcs/runtime/viewer_protocol.py, openhcs/runtime/zmq_execution_client.py, openhcs/pyqt_gui/main.py and openhcs/pyqt_gui/services/ui_bridge_server.py; confirmed --no-gpu, OPENHCS_CPU_ONLY, the .startup.jsonl journal, materialize_runtime_artifacts and submit_timeout_ms still exist; the diffs are enum-to-family renames (orchestrator states, settle phases, viewer lifecycle, update-check origins), the AgentRuntimePlatform rename and the removed QtPlatformSystem enum, none of which the page names, so no correction was needed.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/troubleshooting_FAQ.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compiler.py (ScopeKey scope ids, K6 progress classes), openhcs/runtime/zmq_execution_client.py (K6: debug requests serialize themselves, command types are DebugCommandType classes; the .startup.jsonl journal beside the server log is unchanged) and openhcs/pyqt_gui/main.py (U3 moves to pyqt-reactive); the restart-and-restore action, OPENHCS_UI_CONFIG_CACHE_FILE, submit_timeout_ms and materialize_runtime_artifacts still exist, so no correction was needed.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/troubleshooting_FAQ.rst against B6's later changes to openhcs/authoring/session/operations/datasets.py, openhcs/core/pipeline/compiler.py, openhcs/runtime/viewer_protocol.py, openhcs/runtime/zmq_execution_client.py, openhcs/pyqt_gui/main.py, tests/unit/desktop/test_desktop_restart.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/troubleshooting_FAQ.rst against the later B6 changes to openhcs/authoring/session/operations/datasets.py, openhcs/core/pipeline/compiler.py, openhcs/runtime/viewer_protocol.py, openhcs/runtime/zmq_execution_client.py, openhcs/pyqt_gui/main.py, tests/unit/desktop/test_desktop_restart.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -2182,7 +2214,7 @@ }, { "path": "docs/source/guides/complete_examples.rst", - "source_sha256": "75215edce0d3c4318b82c500c57640e4a3e6981bc28c40b6c634609ffd825576", + "source_sha256": "8344b2d8b15319bc08668fc37e5376701e101c71cfb43b61e3ea28e9d34dac28", "audience": [ "pipeline authors", "demo operators" @@ -2207,12 +2239,12 @@ }, { "path": "openhcs/processing/presets/mfd_specs.py", - "sha256": "0c1f43037c3789e502e497e6204157a1cdbfe17b1ec18954743a05ab17ee1f68", - "role": "Declares MfdPresetKey, build_mfd_preset and the relative whole-device template path." + "sha256": "a3de70444a7945a42907ffbf7da988cccdd508334f1b67747093040371073486", + "role": "Declares the MfdPresetDefinition values, build_mfd_preset and the relative whole-device template path." }, { "path": "openhcs/processing/presets/pipelines/loose_operaphenix_neurite_outgrowth.py", - "sha256": "ad42221c3a03e9bad8bdffce92b56b10a43fd4bb12f12a8dfa947cdb0f63937f", + "sha256": "5fddd2fbd70f778a21bea6ead99d88a9b2d8962897756536a163e11619f93300", "role": "Declares the example_inputs fields, including source_voxel_spacing, and the UnifiedNeurons result of the loose Opera Phenix example." }, { @@ -2222,23 +2254,26 @@ }, { "path": "scripts/master_multi_plate_demo.py", - "sha256": "82f8f6e074625118ec9a96b0f67b5aa6ad39fcc95aed2b7e877d8f442770abe3", + "sha256": "e77acafb6b10996be8daf46d7612ebc39df6cf1f3e79d590567068c8372c3516", "role": "Implements the master showcase's command-line options, base port 5900, contributor loading and output root." }, { "path": "scripts/mcp_assay_showcase.py", - "sha256": "a1ea7a9b6ffdec553931fbe5e9f2b755e63a98b92561ff9edee13b296d926ff6", + "sha256": "1afc72337ca6767f8430a3751be05dd4aeef258b409455d276f52fcba0228710", "role": "Declares the seven built-in scenario blueprints the master showcase composes." }, { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "b96f33b866fe6c9f8118ca70fcb7dff8e6a02cb29d79df67230a5e8b97df8002", + "sha256": "5aa8895c9f533603b7ef972202141e2439473f0604b97d66f5a027b8b3cd1c83", "role": "Defines import_cellprofiler_pipeline, which lowers a .cppipe into steps and a pipeline config." } ], - "findings": [], - "disposition": "keep", + "findings": [ + "Typed native presets said the MFD variants are declared by ``MfdPresetKey``; E1 deleted that enum and openhcs/processing/presets/mfd_specs.py now declares each variant as an ``MfdPresetDefinition`` value passed to ``build_mfd_preset``. Corrected." + ], + "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guides/complete_examples.rst in full and compared it against the changed authorities: openhcs/processing/presets/mfd_specs.py, openhcs/processing/presets/pipelines/loose_operaphenix_neurite_outgrowth.py, scripts/master_multi_plate_demo.py, scripts/mcp_assay_showcase.py and openhcs/interop/cellprofiler/pipeline_import.py; corrected the Typed native presets paragraph, which named the deleted MfdPresetKey enum, to name MfdPresetDefinition; the loose Opera Phenix example, showcase scripts and importer changed only enum spellings and artifact import paths, and their example_inputs, port 5900, seven scenarios and import_cellprofiler_pipeline claims still hold.", "On 2026-10-11 (B6), re-read docs/source/guides/complete_examples.rst in full and compared it against the changed authorities: openhcs/core/processing_contracts.py (CallableMetadata renamed CallableDeclaration internally), openhcs/processing/presets/pipelines/loose_operaphenix_neurite_outgrowth.py and openhcs/interop/cellprofiler/pipeline_import.py (source-binding configs now imported from openhcs.core.config, AuthoredFunctionPattern.first_callable replaces get_core_callable); the page names none of the moved symbols, and Pure2DContract, the openhcs.core.memory.decorators numpy decorator, source_voxel_spacing and import_cellprofiler_pipeline still exist, so no correction was needed.", "On 2026-10-11 (B6 final), compared docs/source/guides/complete_examples.rst against B6's later changes to openhcs/core/processing_contracts.py, openhcs/processing/presets/pipelines/loose_operaphenix_neurite_outgrowth.py, openhcs/microscopes/opera_phenix.py, scripts/master_multi_plate_demo.py, scripts/mcp_assay_showcase.py, openhcs/interop/cellprofiler/pipeline_import.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guides/complete_examples.rst against the later B6 changes to openhcs/core/processing_contracts.py, openhcs/processing/presets/pipelines/loose_operaphenix_neurite_outgrowth.py, openhcs/microscopes/opera_phenix.py, scripts/master_multi_plate_demo.py, scripts/mcp_assay_showcase.py, openhcs/interop/cellprofiler/pipeline_import.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -2254,7 +2289,7 @@ }, { "path": "docs/source/guides/example_corpus_map.rst", - "source_sha256": "d9cdad8b4acffb93f051bf435612df6550dc38eab63330d3a817319ad8baa9d3", + "source_sha256": "dcf4d0b8271ef5a9bed149bce8b29586c0ced5d65558d8b85f2f7ee2d1257956", "audience": [ "pipeline authors", "MCP agents assisting scientists" @@ -2269,12 +2304,12 @@ }, { "path": "openhcs/processing/presets/mfd_specs.py", - "sha256": "0c1f43037c3789e502e497e6204157a1cdbfe17b1ec18954743a05ab17ee1f68", - "role": "Owns MfdPresetKey, build_mfd_preset and the plate-relative whole-device template path wrapped by the four 10x_mfd modules." + "sha256": "a3de70444a7945a42907ffbf7da988cccdd508334f1b67747093040371073486", + "role": "Owns the MfdPresetDefinition values, build_mfd_preset and the plate-relative whole-device template path wrapped by the four 10x_mfd modules." }, { "path": "openhcs/processing/presets/pipelines/loose_operaphenix_neurite_outgrowth.py", - "sha256": "ad42221c3a03e9bad8bdffce92b56b10a43fd4bb12f12a8dfa947cdb0f63937f", + "sha256": "5fddd2fbd70f778a21bea6ead99d88a9b2d8962897756536a163e11619f93300", "role": "Declares the loose Opera Phenix example's exact Hoechst, MAP2 and SMI312 sources, one-well bound and unified neuron result." }, { @@ -2289,13 +2324,16 @@ }, { "path": "tests/unit/agent/test_knowledge_base_service.py", - "sha256": "b814db1827c3d4cc3dab479ef7e84c6c406f324ccaddee794690f3b387626411", + "sha256": "7b30d1c1931efc1c380b2dacd743bcd4e509cafc949b01d24e156448681a0fc3", "role": "Proves discovery and retrieval of the example corpus map and other maintained knowledge documents." } ], - "findings": [], - "disposition": "keep", + "findings": [ + "Native OpenHCS Examples said openhcs/processing/presets/mfd_specs.py owns 'the typed preset keys, variant declarations'; E1 deleted MfdPresetKey and MfdPresetFamily, and the module now owns MfdPresetDefinition values. Corrected to 'the preset definitions'." + ], + "disposition": "revise", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guides/example_corpus_map.rst in full and compared it against the changed authorities: openhcs/processing/presets/mfd_specs.py, openhcs/processing/presets/pipelines/loose_operaphenix_neurite_outgrowth.py and tests/unit/agent/test_knowledge_base_service.py; corrected Native OpenHCS Examples, which said mfd_specs.py owns typed preset keys, to say it owns the preset definitions (MfdPresetKey was deleted; the knowledge-base test now searches for MfdPresetDefinition); the loose Opera Phenix example changed only enum spellings and artifact import paths.", "On 2026-10-11 (B6), re-read docs/source/guides/example_corpus_map.rst in full and compared it against the changed authority openhcs/processing/presets/pipelines/loose_operaphenix_neurite_outgrowth.py (source-binding config imports moved to openhcs.core.config, AuthoredFunctionPattern.first_callable replaces get_core_callable); the page states only the example's sources, one-well bound and review evidence, which are unchanged, so no correction was needed.", "On 2026-10-11 (B6 final), compared docs/source/guides/example_corpus_map.rst against B6's later changes to tests/unit/agent/test_knowledge_base_service.py: metadata documents are written only for backends that host local files (not virtual backends such as OMERO); the image-source guide test reads axis-family spellings. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/guides/example_corpus_map.rst against B6's later changes to openhcs/processing/presets/pipelines/loose_operaphenix_neurite_outgrowth.py, tests/unit/agent/test_knowledge_base_service.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", @@ -2321,17 +2359,17 @@ "authority": [ { "path": "openhcs/runtime/fiji_viewer_server.py", - "sha256": "ee246d32ed828bbf4665cc8a411e807da29bd43c2475012243f27e93cb550410", + "sha256": "352ef6b773da07b8103873606cf8a438b8ec70b64cfc57526cc9b18cf9b48b40", "role": "Initializes Fiji through FIJI_IMAGEJ_RUNTIME, tracks debounced display-batch settlement, registers the run-macro action and delegates gateway shutdown." }, { "path": "openhcs/runtime/viewer_control_actions.py", - "sha256": "1bbd370347a9e417fac7ffcb42ec42eae0589dc1a77286814b7a586c8318ce35", + "sha256": "f19c8370dd9145f11ba7ceb91d2efff123a6473db9bc551474a47a37acb4041d", "role": "Gives every viewer the shared shutdown, clear-state, process-launch and settle actions and answers unregistered control types with an unsupported-control error." }, { "path": "openhcs/runtime/viewer_protocol.py", - "sha256": "c4bdae4bae6dee412f0d27a9cf75a95db53b3bd95c03cee7d046be16b52fe5b8", + "sha256": "347a65b63c149f2f437014d537cfaf481a33801d860742f4083b7377494e792f", "role": "Declares ViewerSettleProgress and the viewer control message types named on the page." }, { @@ -2341,18 +2379,19 @@ }, { "path": "pyproject.toml", - "sha256": "b5d4d752704b886da20d860743b81dc3d2ce3a3c402208246fadc1c6cb71f597", + "sha256": "75f92010b3c18ede61cf7296064079590c143283871e372a1fd4546417d4e0f9", "role": "Declares the fiji and viz extras named in setup." }, { "path": "tests/unit/test_fiji_viewer_server.py", - "sha256": "da575dfb02895ecc831b34d6bdf6b58112b986556459444e9701cbf43221ea22", + "sha256": "b19316b843cc53c21bd3e00b22694fb645dcba062610295adf54abed7b4f51df", "role": "Proves Fiji endpoint lifecycle, streamed image updates, settlement and control handling." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guides/fiji_viewer_management.rst in full and compared it against the changed authorities: openhcs/runtime/fiji_viewer_server.py, openhcs/runtime/viewer_control_actions.py, openhcs/runtime/viewer_protocol.py, pyproject.toml and tests/unit/test_fiji_viewer_server.py; ViewerSettlePhase became a subclass family (SettlementRunning, SettlementComplete, SettlementFailed) with the same running/complete/failed spellings the page describes, ViewerSettleProgress remains, SETTLE, STATE and PAYLOADS are still control types, FIJI_IMAGEJ_RUNTIME still initializes Fiji and pyproject.toml still declares the fiji and viz extras; no correction was needed.", "On 2026-10-11 (B6), re-read docs/source/guides/fiji_viewer_management.rst in full and compared it against the changed authorities: external/zmqruntime/src/zmqruntime/__init__.py (current lazy export surface of client, messages and execution contracts) and pyproject.toml (dependency floors only; fiji and viz extras unchanged); the page's ZMQRuntime ownership and setup claims still hold, so no correction was needed.", "On 2026-10-11 (B6 final), compared docs/source/guides/fiji_viewer_management.rst against B6's later changes to openhcs/runtime/fiji_viewer_server.py, openhcs/runtime/viewer_protocol.py, tests/unit/test_fiji_viewer_server.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guides/fiji_viewer_management.rst against the later B6 changes to openhcs/runtime/fiji_viewer_server.py, openhcs/runtime/viewer_protocol.py, pyproject.toml, tests/unit/test_fiji_viewer_server.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -2379,7 +2418,7 @@ "authority": [ { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "4a33908fb42874e6d57eeea3f5305793ddbda41d13356f0a78501af38dcdfeb4", + "sha256": "40b07b7f40aca39b19b95a8f61c1b0c8270f0ab0a29e8c36caa13f04d8fa539f", "role": "Compiles pipeline declarations into validated step plans and execution bundles." }, { @@ -2399,7 +2438,7 @@ }, { "path": "openhcs/runtime/viewer_protocol.py", - "sha256": "c4bdae4bae6dee412f0d27a9cf75a95db53b3bd95c03cee7d046be16b52fe5b8", + "sha256": "347a65b63c149f2f437014d537cfaf481a33801d860742f4083b7377494e792f", "role": "Declares the transport-independent viewer messages, stream configuration, and server contract." }, { @@ -2411,6 +2450,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guides/index.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compiler.py and openhcs/runtime/viewer_protocol.py; both diffs are enum-to-family renames (orchestrator states, materialization backends, settle phases, viewer launch session and lifecycle); the page only routes to the five explanations and names none of them, so no correction was needed.", "On 2026-10-11 (B6), re-read docs/source/guides/index.rst in full and compared it against the changed authority openhcs/core/pipeline/compiler.py (ScopeKey-built compiler scope ids from U3, CompileProgressPhase/RunningProgressStatus progress classes from K6, CompiledSourceBindingPlan() default); the page only routes to the integration explanations and states no compiler detail, so no correction was needed.", "On 2026-10-11 (B6 final), compared docs/source/guides/index.rst against B6's later changes to openhcs/core/pipeline/compiler.py, openhcs/core/memory/decorators.py, openhcs/runtime/viewer_protocol.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guides/index.rst against the later B6 changes to openhcs/core/pipeline/compiler.py, openhcs/core/memory/decorators.py, openhcs/runtime/viewer_protocol.py, external/PolyStore/src/polystore/omero_local.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -2444,7 +2484,7 @@ }, { "path": "openhcs/constants/constants.py", - "sha256": "83190e34eaa0f1ae690e10aac3fb4cdcd3929ee337841bc8f97e2e304e8feab0", + "sha256": "00957394953071b0fff6ef8298648351d19877a408dff5f7b538e7249fcb9247", "role": "Re-exports arraybridge.MemoryType as openhcs.constants.MemoryType without a translation." }, { @@ -2464,12 +2504,12 @@ }, { "path": "openhcs/core/callable_contract.py", - "sha256": "a8e73c5df804d78baf72710e210cb2ae63156657a074071e11fd868f4b19d3b4", + "sha256": "6089f63e72e1a47fc5f6bc7b443028ac8ec40d934898589fa03b5592c2273ab3", "role": "Reads input, output and execution memory types through MemoryContractAttribute." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "358d3eaf5952170f9e9e2dc9056fb08707e8fcf74682063c074b367a83c2648f", + "sha256": "6525348d097bda8d0265bc7c3f312dd8839ca9f65dd05f797fc82ba873e04547", "role": "Declares FrameworkDeviceAssignment and the typed per-step conversion decisions recorded by compilation." }, { @@ -2484,17 +2524,17 @@ }, { "path": "openhcs/core/steps/function_runtime.py", - "sha256": "fa9486cddf3db943a218f276bb43d079f505bf84b18d9f7bd074f7a4d86332b3", + "sha256": "83d9eca90f2dca33ff8074bb768a24fadc5dfb1881855274eab9203b4cc41770", "role": "Enters the invocation's execution device scope around the function body." }, { "path": "openhcs/core/orchestrator/worker_execution.py", - "sha256": "26b9586890e7117b24b6b85660d3f417455d55546401739f91146d18e4d21134", + "sha256": "83ade0271c9b24ca1ba61c708d9e6f60a432b630ec6a8a2fac6d1018458e6be5", "role": "Cleans the exact compiled framework-device footprint during worker teardown." }, { "path": "openhcs/core/config.py", - "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", + "sha256": "9625ca054fac6c1e460216cfe0e85621f5306592a004d1bdc28a6b1ce2099366", "role": "Declares variable_components and group_by on the step processing configuration." }, { @@ -2511,6 +2551,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guides/memory_type_integration.rst in full and compared it against the changed authorities: openhcs/constants/constants.py, openhcs/core/callable_contract.py, openhcs/core/compiled_step_plan.py, openhcs/core/steps/function_runtime.py, openhcs/core/orchestrator/worker_execution.py and openhcs/core/config.py; constants.py still re-exports arraybridge's MemoryType (its OrchestratorState enum became a class family, which the page does not mention), CallableContract still reads MemoryContractAttribute, FrameworkDeviceAssignment is still on CompiledStepPlan, function_runtime.py still enters execution_device_scope() around the function body, and variable_components and group_by remain on ProcessingConfig; no correction was needed.", "On 2026-10-11 (B6), re-read docs/source/guides/memory_type_integration.rst in full and compared it against the changed authorities: openhcs/core/processing_contracts.py and openhcs/core/callable_contract.py (K5: CallableMetadata became CallableDeclaration, which CallableContract inherits and which reads the input, output and execution keys through MemoryContractAttribute), openhcs/core/compiled_step_plan.py (empty-plan defaults), openhcs/core/steps/function_runtime.py (execution_device_scope still wraps the function body), openhcs/core/orchestrator/worker_execution.py (K6 progress classes; compiled GPU footprint cleanup unchanged), openhcs/core/config.py (variable_components and group_by still on the processing config) and tests/unit/test_framework_execution_scope.py; confirmed openhcs.constants.MemoryType and openhcs.core.memory.MemoryType are arraybridge.MemoryType and that MemoryType.subprocess_environment exists; no correction was needed.", "On 2026-10-11 (B6 final), compared docs/source/guides/memory_type_integration.rst against B6's later changes to openhcs/core/memory/decorators.py, openhcs/core/processing_contracts.py, openhcs/core/callable_contract.py, openhcs/core/compiled_step_plan.py, openhcs/core/steps/function_runtime.py, openhcs/core/orchestrator/worker_execution.py, openhcs/core/config.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guides/memory_type_integration.rst against the later B6 changes to openhcs/core/memory/decorators.py, openhcs/core/processing_contracts.py, openhcs/core/callable_contract.py, openhcs/core/compiled_step_plan.py, openhcs/core/steps/function_runtime.py, openhcs/core/orchestrator/worker_execution.py, openhcs/core/config.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -2574,7 +2615,7 @@ }, { "path": "openhcs/core/dataset_sources/source.py", - "sha256": "350bf5691a03522e9f54b1d8ab712c6beba1683aac4fa5159ea4e80c6041530d", + "sha256": "b3758a878fdf8ec28b6d9818ebfd96c5c470895575f7a79500c85f1490679281", "role": "Declares the DatasetSource interface the OMERO source implements for workspace and metadata access." }, { @@ -2604,7 +2645,7 @@ }, { "path": "pyproject.toml", - "sha256": "b5d4d752704b886da20d860743b81dc3d2ce3a3c402208246fadc1c6cb71f597", + "sha256": "75f92010b3c18ede61cf7296064079590c143283871e372a1fd4546417d4e0f9", "role": "Includes the complete openhcs/omero deployment and plugin bundle in installed distributions." }, { @@ -2626,6 +2667,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guides/omero_integration.rst in full and compared it against the changed authorities: openhcs/core/dataset_sources/source.py and pyproject.toml; DatasetSource lost get_required_backend and now compares roots through absolute_source_path, neither of which the page describes; pyproject.toml still packages the openhcs/omero bundle and only moved the python-introspect floor; no correction was needed.", "On 2026-10-11 (B6), re-read docs/source/guides/omero_integration.rst in full and compared it against the changed authorities: openhcs/core/dataset_sources/source.py (SourceBindingsConfig type import now from openhcs.core.config, K4) and pyproject.toml (dependency floors only); the page names neither, and its DatasetSource and OMERO ownership claims still hold, so no correction was needed.", "On 2026-10-11 (B6 final), compared docs/source/guides/omero_integration.rst against B6's later changes to openhcs/domains/microscopy/analysis_consolidation.py, openhcs/core/dataset_sources/source.py, openhcs/microscopes/omero.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guides/omero_integration.rst against the later B6 changes to external/PolyStore/src/polystore/omero_local.py, openhcs/domains/microscopy/analysis_consolidation.py, openhcs/core/dataset_sources/source.py, openhcs/microscopes/omero.py, pyproject.toml: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -2676,22 +2718,22 @@ "authority": [ { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "4a33908fb42874e6d57eeea3f5305793ddbda41d13356f0a78501af38dcdfeb4", + "sha256": "40b07b7f40aca39b19b95a8f61c1b0c8270f0ab0a29e8c36caa13f04d8fa539f", "role": "Defines compile_pipelines over an orchestrator and list[AbstractStep] and returns a CompiledExecutionBundle." }, { "path": "openhcs/core/orchestrator/orchestrator.py", - "sha256": "99870516da6d64e1858f4dd8799d6e38ab6911b7b02bc29b252d2a23f7687410", + "sha256": "a7097eec86c5966552bcb546a2161d1a08e6a4c630e6b156de3b8354b0026b54", "role": "Supplies the initialized orchestrator, its DatasetSource workspace and selected partition values to compilation." }, { "path": "openhcs/core/pipeline/compilation_session.py", - "sha256": "0f9cc4819b53d55b7e9446429b8c8ac1ff1a59f4902020ea1658495ce01d464c", + "sha256": "4946c221e34ec19957b5af2b1a54359d454ee840fb1f5e1fe2b3b306ac8e5426", "role": "Declares CompilationSession, which owns cross-step compile state." }, { "path": "openhcs/core/compiled_step_plan.py", - "sha256": "358d3eaf5952170f9e9e2dc9056fb08707e8fcf74682063c074b367a83c2648f", + "sha256": "6525348d097bda8d0265bc7c3f312dd8839ca9f65dd05f797fc82ba873e04547", "role": "Declares CompiledStepPlan and its typed per-step memory, conversion, device and runtime fields." }, { @@ -2706,13 +2748,14 @@ }, { "path": "tests/unit/test_compiled_execution.py", - "sha256": "cb6c7034e9534470daf99a471b27a9ae662b5d2828f01af8613534412e8eea81", + "sha256": "906c9f7eadb2faf92244e6ce22ce98cc5e0f6bd1d1b967458bbedc799e92e465", "role": "Proves compilation produces immutable step plans and execution bundles with the expected resolved values." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guides/pipeline_compilation_workflow.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compiler.py, openhcs/core/orchestrator/orchestrator.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/compiled_step_plan.py and tests/unit/test_compiled_execution.py; CompilationSession, CompiledStepPlan and CompiledExecutionBundle keep their names and roles; the diffs replace OrchestratorState, MaterializationBackend and RuntimeObservationMode enum members with subclass families and move artifact imports to openhcs.core.artifacts.plans and .kinds, none of which the page names, so no correction was needed.", "On 2026-10-11 (B6), re-read docs/source/guides/pipeline_compilation_workflow.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compiler.py (ScopeKey scope ids, K6 progress classes), openhcs/core/orchestrator/orchestrator.py (source_bindings_defaults_to_base imported from openhcs.core.config), openhcs/core/pipeline/compilation_session.py (strip_disabled_functions removed, per-graph invocation validation moved onto the graph) and openhcs/core/compiled_step_plan.py (empty-plan defaults now plain constructors); CompilationSession, CompiledStepPlan and CompiledExecutionBundle still own the stated responsibilities, so no correction was needed.", "On 2026-10-11 (B6 final), compared docs/source/guides/pipeline_compilation_workflow.rst against B6's later changes to openhcs/core/pipeline/compiler.py, openhcs/core/orchestrator/orchestrator.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/compiled_step_plan.py, tests/unit/test_compiled_execution.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guides/pipeline_compilation_workflow.rst against the later B6 changes to openhcs/core/pipeline/compiler.py, openhcs/core/orchestrator/orchestrator.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/compiled_step_plan.py, tests/unit/test_compiled_execution.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -2848,7 +2891,7 @@ }, { "path": "openhcs/pyqt_gui/config.py", - "sha256": "a3eece2e5b2926fc6fc0edd197221da451ab7c0d0d6f4ffc8bb2ff865ad74bd4", + "sha256": "d092cb20206127484fbbc80a38a5152462b9b8991e848a814c85f564d51f4db8", "role": "Declares UI configuration, including logging, theming, viewer, and execution-server settings." }, { @@ -2880,6 +2923,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guides/testing_guide.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/config.py; the diff renames AgentRuntimePlatformAuthority to AgentRuntimePlatform and adds GuiLogLevel.emits_records; LoggingConfig and its default log directory, which the debugging section relies on, are unchanged, so no correction was needed.", "On 2026-10-11 (B6), re-read docs/source/guides/testing_guide.rst in full and compared it against the changed authority openhcs/core/xdg_paths.py (legacy ~/.openhcs migration helpers deleted; get_*_file_path now plain path joins); the page's log-location claim (get_openhcs_data_dir() / \"logs\", honouring XDG_DATA_HOME, default ~/.local/share/openhcs/logs) still holds and the page never described the deleted migration, so no correction was needed.", "On 2026-10-11 (B6 final), compared docs/source/guides/testing_guide.rst against B6's later changes to .github/workflows/integration-tests.yml: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/guides/testing_guide.rst against the later B6 changes to scripts/smoke_installed_gui.py, tests/unit/test_smoke_installed_gui.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -2960,12 +3004,12 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", + "sha256": "9625ca054fac6c1e460216cfe0e85621f5306592a004d1bdc28a6b1ce2099366", "role": "Declares StreamingConfig, NapariStreamingConfig, FijiStreamingConfig and StepMaterializationConfig, including the partition filter inherited by streaming configuration." }, { "path": "openhcs/runtime/napari_viewer_server.py", - "sha256": "038d7a76eeac8d69079591cf6eb7736af5415b28a7a067d6677bd15d115d11c7", + "sha256": "ac54f8bfdce33150c31b38bcd5cf6069b5fd66fb7c05d5a6951610ec3d895bfd", "role": "Mounts the ROI Manager, owns the OpenHCS ROI selection toolbar and queued-route settlement, and applies layer isolation." }, { @@ -2975,12 +3019,12 @@ }, { "path": "openhcs/runtime/viewer_protocol.py", - "sha256": "c4bdae4bae6dee412f0d27a9cf75a95db53b3bd95c03cee7d046be16b52fe5b8", + "sha256": "347a65b63c149f2f437014d537cfaf481a33801d860742f4083b7377494e792f", "role": "Declares the SHAPES streaming data type and the settle progress fields." }, { "path": "openhcs/runtime/viewer_control_actions.py", - "sha256": "1bbd370347a9e417fac7ffcb42ec42eae0589dc1a77286814b7a586c8318ce35", + "sha256": "f19c8370dd9145f11ba7ceb91d2efff123a6473db9bc551474a47a37acb4041d", "role": "Answers settle requests on the control transport thread without waiting for Qt." }, { @@ -3005,12 +3049,12 @@ }, { "path": "openhcs/processing/materialization/options.py", - "sha256": "271346eda98bfcabe2614696e1d2499327b84dcd8245f3ad61025cc18ddcae0f", + "sha256": "e7487e9ad44276d4f2880f0d9b03fd6396f8053b9bda5775ab25caaf023554ca", "role": "Declares SWCOptions and SpatialGraphROIOptions." }, { - "path": "openhcs/core/artifacts.py", - "sha256": "65eb0ec5c903366243ca8bd5ea07412e8d34bdb81e6015fa76b3c4c59838733f", + "path": "openhcs/core/artifacts/kinds.py", + "sha256": "216b913efad9ec76707e9f61be445fb6fa1c6601c36ef53e296acd4928604515", "role": "Declares SpatialGraphArtifactType." }, { @@ -3020,12 +3064,12 @@ }, { "path": "openhcs/runtime/napari_streaming_handlers.py", - "sha256": "be585484075783f7717776d2b8ba163aab8a69aa24c560ca3844214485bd9eef", + "sha256": "b43f29163cef420c77f30e46e92d94145520f7534077d54636a0ab141da32e7e", "role": "Owns semantic display-axis alignment for images, points, shapes, translations, and exact-slice navigation." }, { "path": "tests/unit/test_napari_roi_manager_integration.py", - "sha256": "a4b7dee556f0ceccaa212751e29eb8160d3d0c08c741760715ee8372b0c0e927", + "sha256": "15b0417fc6388f932ac6cb8b35102721918b987b68e52caef04b1c6bc5ea9e81", "role": "Proves the packaged Napari ROI manager is discovered and integrated with the viewer contract." }, { @@ -3037,6 +3081,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/guides/viewer_management.rst in full and compared it against the changed authorities: openhcs/core/config.py, openhcs/runtime/napari_viewer_server.py, openhcs/runtime/viewer_protocol.py, openhcs/runtime/viewer_control_actions.py, openhcs/processing/materialization/options.py, openhcs/runtime/napari_streaming_handlers.py, tests/unit/test_napari_roi_manager_integration.py and the deleted openhcs/core/artifacts.py; K2 split openhcs/core/artifacts.py into a package, so the SpatialGraphArtifactType authority is now openhcs/core/artifacts/kinds.py; the other diffs turn settle phases, Napari layer kinds, variable-size handling, dimension-label route sources and filename identities into subclass families without changing the streaming configs, SHAPES boundary, settle progress, SWCOptions or SpatialGraphROIOptions the page describes, so the prose needed no correction.", "On 2026-10-11 (B6), re-read docs/source/guides/viewer_management.rst in full and compared it against the changed authority openhcs/core/config.py (PartitionFilterMode became an IncludePartitions/ExcludePartitions subclass family, K4 moved SourceBindingsConfig/StepSourceBindingsConfig into config.py); StreamingConfig, NapariStreamingConfig, FijiStreamingConfig and StepMaterializationConfig still exist with partition selection inherited by streaming, and the page names no partition-mode member, so no correction was needed.", "On 2026-10-11 (B6 final), compared docs/source/guides/viewer_management.rst against B6's later changes to openhcs/core/config.py, openhcs/runtime/viewer_protocol.py, openhcs/core/artifacts.py, tests/unit/test_napari_roi_manager_integration.py, tests/unit/test_viewer_result_element_selection.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/guides/viewer_management.rst against the later B6 changes to openhcs/core/config.py, openhcs/runtime/viewer_protocol.py, openhcs/core/artifacts.py, tests/unit/test_napari_roi_manager_integration.py, tests/unit/test_viewer_result_element_selection.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -3072,12 +3117,12 @@ }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "3cdb7819f09604046ba69e0b4f57b73e4e98d8f68af80158501a54bb42e701a2", + "sha256": "2d10806e310cb0d47ee7e47b879a7a95b28dcf5c688d9ce2a54b81ae1a041c47", "role": "Composes the main desktop window, menus, task windows, and registered UI services." }, { "path": "openhcs/interop/cellprofiler/pipeline_import.py", - "sha256": "b96f33b866fe6c9f8118ca70fcb7dff8e6a02cb29d79df67230a5e8b97df8002", + "sha256": "5aa8895c9f533603b7ef972202141e2439473f0604b97d66f5a027b8b3cd1c83", "role": "Imports CellProfiler modules into ordinary OpenHCS pipeline declarations through the interop boundary." }, { @@ -3094,6 +3139,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/index.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/main.py and openhcs/interop/cellprofiler/pipeline_import.py; both diffs are enum-to-family renames plus the artifacts-package import paths; the landing page routes readers and names no changed symbol, so no correction was needed.", "On 2026-10-11 (B6), re-read docs/source/index.rst in full and compared it against the changed authorities: pipeline_document.py (render now via pycodify.generate_python_source, L6; PipelineDocument/Codec unchanged), pyqt_gui/main.py (dock layout, shortcuts and time-travel navigation moved to pyqt-reactive, U3), cellprofiler/pipeline_import.py (SourceBindingsConfig now imported from openhcs.core.config, K4); the landing page names no module, class or window behaviour these touched, and still routes to existing pages; no edit needed.", "On 2026-10-11 (B6 final), compared docs/source/index.rst against B6's later changes to openhcs/pyqt_gui/main.py, openhcs/interop/cellprofiler/pipeline_import.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/index.rst against the later B6 changes to openhcs/pyqt_gui/main.py, openhcs/interop/cellprofiler/pipeline_import.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -3122,7 +3168,7 @@ "authority": [ { "path": "pyproject.toml", - "sha256": "b5d4d752704b886da20d860743b81dc3d2ce3a3c402208246fadc1c6cb71f597", + "sha256": "75f92010b3c18ede61cf7296064079590c143283871e372a1fd4546417d4e0f9", "role": "Declares the package version, Python requirement, dependency extras, scripts, and distribution metadata the redirected guide covers." }, { @@ -3142,10 +3188,11 @@ } ], "findings": [ - "Maintaining installation instructions here would duplicate package extras and installer behaviour; the page correctly redirects to the maintained task. The later dependency floor changes (zmqruntime 0.6.2, pycodify 0.2, objectstate 2.1.1, python-introspect 0.2.7, pyqt-reactive 0.5) are not copied here, so nothing on the page became wrong." + "The page is a redirect stub; its link target still exists and owns the content (checked 2026-10-11, B6)." ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/installation.rst in full and compared it against the changed authorities: pyproject.toml; the python-introspect floor moved to 0.2.8; the page states no dependency fact and docs/source/getting_started/getting_started.rst still exists, so no correction was needed.", "On 2026-10-11 (B6), re-read docs/source/installation.rst in full and compared it against the changed authorities: pyproject.toml (zmqruntime, pycodify, objectstate, python-introspect and pyqt-reactive floor changes after U3/K4/K5/K6/L4/L6); the page states no version or extra and still redirects to getting_started; no edit needed.", "On 2026-10-11 (B6 final), compared docs/source/installation.rst against the later B6 changes to pyproject.toml: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/installation.rst in full and compared it against the changed authorities: pyproject.toml (dependency floors, JAX extra), openhcs/cli.py (skills subcommand), the Windows and macOS installers (skill sync); the page copies none of those facts and still redirects to getting_started; replaced 'canonical' with 'maintained'.", @@ -3192,17 +3239,17 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", + "sha256": "9625ca054fac6c1e460216cfe0e85621f5306592a004d1bdc28a6b1ce2099366", "role": "Declares GlobalPipelineConfig, PipelineConfig and the step configuration families with their defaults and field help." }, { "path": "openhcs/pyqt_gui/config.py", - "sha256": "a3eece2e5b2926fc6fc0edd197221da451ab7c0d0d6f4ffc8bb2ff865ad74bd4", + "sha256": "d092cb20206127484fbbc80a38a5152462b9b8991e848a814c85f564d51f4db8", "role": "Declares UIConfig with list previews, logging, shortcuts, progress, execution transport and agent-bridge settings." }, { "path": "openhcs/agent/services/config_service.py", - "sha256": "902bb12178abf758cf0d7883c9cade8c5870f5362a3b5f7cc79e5640da9fb42d", + "sha256": "27a1d5d841b147e3ce35c3a358ca30b6d6efe6ed08a7db874648410b122a717f", "role": "Resolves the global, pipeline, step and ui directive names to configuration declarations." }, { @@ -3231,9 +3278,12 @@ "role": "Proves user-facing field help is sourced from declaration metadata rather than copied UI text." } ], - "findings": [], + "findings": [ + "Code issue, not edited: for fields whose value is a registered family class (vfs_config.materialization_backend, processing_config.input_source, napari variable_size_handling, partition filter mode), the generated reference prints the default as a Python class repr such as instead of the accepted spelling (disk). _default_value_repr in openhcs/agent/services/config_service.py has no branch for a registered class default; the same default_repr reaches openhcs_describe_config_schema. Accepted values render correctly." + ], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/reference/configuration.rst in full and compared it against the changed authorities: openhcs/core/config.py (E1: MaterializationBackend became a registered subclass family, MaterializeToDisk default; input_source holds an InputSource class, ReadPreviousStep default; K2 artifacts import), openhcs/pyqt_gui/config.py (AgentRuntimePlatform rename, GuiLogLevel.emits_records) and openhcs/agent/services/config_service.py (patch values coerced to registered family classes by key); the page copies no field, type or default; rendered the global root through ConfigReferenceRstRenderer and confirmed the family fields list their accepted values (auto/zarr/disk/omero_local, previous/start, separate_layers/pad_to_max) and that pipeline, step and ui roots still render; the rendered default of those fields is a class repr, a code defect recorded in findings; no page edit.", "On 2026-10-11 (B6), re-read docs/source/reference/configuration.rst in full and compared it against the changed authority openhcs/core/config.py (PartitionFilterMode became a registered subclass family with RegisteredClassChoices, SourceBindingsConfig/StepSourceBindingsConfig declared in config.py, K4); the page copies no field, type or default and renders them through the openhcs-config-reference directive from live declarations, so the new family spellings appear automatically; confirmed UIConfig still holds list_previews, logging, shortcuts, progress, zmq transport and agent_bridge, openhcs_describe_config_schema and ConfigPatch still exist; no edit needed.", "On 2026-10-11 (B6 final), compared docs/source/reference/configuration.rst against B6's later changes to openhcs/core/config.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/reference/configuration.rst against the later B6 changes to openhcs/core/config.py, openhcs/agent/capabilities.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -3277,32 +3327,32 @@ }, { "path": "openhcs/processing/backends/cellprofiler/watershed.py", - "sha256": "2c8c352bdc5fedad6baf14e4c1a3cd5347a1f862fd841472241680294c2762ca", + "sha256": "7fb797d16fc740c381d7dc9add3a0ff79ed2d82515850597c7995857bdacc206", "role": "Declares the full-stack payload route for CellProfiler-compatible Watershed." }, { "path": "openhcs/processing/backends/cellprofiler/primary_objects.py", - "sha256": "b5b8a9b98dd9166b34a8f5cebf884e8af6da85f02fe878ef1227d6656d31ae4c", + "sha256": "4d28a02d5c25e4700df1affdbf414e587cde7b7d348917ee29e862e5135466cc", "role": "Declares the current plane-local IdentifyPrimaryObjects route." }, { "path": "openhcs/processing/backends/cellprofiler/secondary.py", - "sha256": "f731f95f874ba3368828ca3a539fac4e5c6eb0ec7049e546f45fb874e0888e95", + "sha256": "e84db4b2bfa56492801e24fc9471e09c4ada4daf5b3541463cf3424ed227af2b", "role": "Declares the current plane-local IdentifySecondaryObjects and IdentifyTertiaryObjects routes." }, { "path": "openhcs/processing/backends/cellprofiler/intensity.py", - "sha256": "43bfa771bb26453f864b633a48371c0e68cf5aae0e175ed6d7a1eabb38ec5dbe", + "sha256": "35ea2c6c82b1451d1aee3accdb2e22cec2125ee6b2f19cf6655167d9a709025e", "role": "Declares volumetric object-intensity execution and its Z measurement features." }, { "path": "openhcs/processing/backends/cellprofiler/shape.py", - "sha256": "69e82de4faa46adc1b5ea543e352edf9a253c4b3bdedec61e0e5e3a7d95b3019", + "sha256": "1281d4aaa6adf5d431bea696b750832ca3b44aed0832257e60b3debf1e265a1b", "role": "Declares full-stack object-shape input and the exact 3D feature schema." }, { "path": "openhcs/processing/backends/cellprofiler/area_occupied.py", - "sha256": "8316d44a5ea2f88f855aeef1c3d4fd979fb01a95a6dc9f6d1bf005629f25b18b", + "sha256": "c3463d217d1db31e713e350a6d2000dc76ffe33c73587a187721af74e4ad696d", "role": "Declares 3D occupied-volume measurements and physical voxel-spacing input." }, { @@ -3327,20 +3377,19 @@ }, { "path": "tests/unit/test_cellprofiler_module_execution.py", - "sha256": "203501b194710b747f6a791bce8fb46d0298e09ffdc20fb0c22e4a96296c1a64", + "sha256": "93ee832a1da999d499afde2c08341f8da4f88e098fb27e4d9f3758b91b236b6c", "role": "Proves payload-scoped size-and-shape rows use the 3D volume schema." }, { "path": "tests/unit/test_cellprofiler_pipeline_import.py", - "sha256": "0ce4efd85d819c7e9f7fa84ec6f1ac7ec4bda024c9a8d5c7f264fda2730264f0", + "sha256": "67576f37b7cbe0595acea633265f8e14677ca3c1b8204630413330273dc4e58a", "role": "Proves CellProfiler import preserves 3D image, object, and morphology variants from declarations." } ], - "findings": [ - "The page's claims still hold after K4/K5: processing_contracts.py now reads signatures through CallableDeclaration (CallableMetadata folded into it), watershed.py imports StepSourceBindingsConfig from openhcs.core.config, and IdentifyTertiaryObjects gained repeated_return_slots for identical larger/smaller inputs. None of these changes the plane-local or payload-scoped domains, the 3D measurement schemas, or the Official30 evidence the page states; the page names no renamed class." - ], + "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/reference/dimensionality_and_measurements.rst in full and compared it against the changed authorities: cellprofiler/watershed.py (WatershedRuntimeFamily enum replaced by WatershedRuntimeStrategy subclasses), primary_objects.py, secondary.py, intensity.py and area_occupied.py (K2 artifacts package imports; replacement-primary relation declares source_artifact_type), shape.py (K3: label-indexed rows, radius and Feret zero for empty labels) and tests/unit/test_cellprofiler_module_execution.py and test_cellprofiler_pipeline_import.py (family class spellings); watershed_cellprofiler4 and watershed_library still run under FullStackExecution, the Identify* routes and the 3D shape, intensity and occupied-volume schemas are unchanged, and tests/unit/test_documented_dimensionality.py still imports the declarations it binds; no edit needed.", "On 2026-10-11 (B6), re-read docs/source/reference/dimensionality_and_measurements.rst in full and compared it against the changed authorities: processing_contracts.py (CallableMetadata -> CallableDeclaration), cellprofiler/watershed.py (StepSourceBindingsConfig import from config.py), cellprofiler/secondary.py (tertiary repeated_return_slots), tests/unit/test_cellprofiler_module_execution.py and test_cellprofiler_pipeline_import.py (flattened CallableDeclaration fields, admitted_graph helper); the segmentation table, volumetric measurement fields and validation boundary remain accurate; no edit needed.", "On 2026-10-11 (B6 final), compared docs/source/reference/dimensionality_and_measurements.rst against B6's later changes to openhcs/core/processing_contracts.py, openhcs/core/image_payload_execution_mode.py, openhcs/processing/backends/cellprofiler/watershed.py, openhcs/processing/backends/cellprofiler/primary_objects.py, openhcs/processing/backends/cellprofiler/secondary.py, openhcs/processing/backends/cellprofiler/intensity.py, openhcs/processing/backends/cellprofiler/shape.py, openhcs/processing/backends/cellprofiler/area_occupied.py, tests/unit/test_documented_dimensionality.py, tests/unit/test_cellprofiler_library_loading.py, tests/unit/test_cellprofiler_module_execution.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/reference/dimensionality_and_measurements.rst against the later B6 changes to openhcs/core/processing_contracts.py, openhcs/core/image_payload_execution_mode.py, openhcs/processing/backends/cellprofiler/watershed.py, openhcs/processing/backends/cellprofiler/primary_objects.py, openhcs/processing/backends/cellprofiler/secondary.py, openhcs/processing/backends/cellprofiler/intensity.py, openhcs/processing/backends/cellprofiler/shape.py, openhcs/processing/backends/cellprofiler/area_occupied.py, tests/unit/test_documented_dimensionality.py, tests/unit/test_cellprofiler_library_loading.py, tests/unit/test_cellprofiler_module_execution.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -3370,18 +3419,18 @@ "authority": [ { "path": "openhcs/processing/backends/experimental_analysis/layout_rows.py", - "sha256": "69e1428d0aefe683f1ee1cf741f83915fbaa1318d9d3f19b8bf8120583d883d2", - "role": "Declares workbook row roles (replicate count, Wells/WellsN) and the EDDU_CX5 and EDDU_metaxpress scope values." + "sha256": "9e1bd76a105c7673a1e0afb223eae262087563b5673ac05e540a40ff339ee5fe", + "role": "Declares the workbook row roles: replicate count, Wells and WellsN rows with their replicate numbers." }, { "path": "openhcs/processing/backends/experimental_analysis/analysis.py", - "sha256": "17a701e0740eac3e11319fe73f10302bbcab603218349bfba4e07014aabc27bb", + "sha256": "96b29239552724475c9fb010248bb266340fa0bbf06ae890f6074b8620c6183b", "role": "Parses the drug_curve_map and plate_groups sheets, applies positional dose/well/plate-group mapping, controls, exclusions, and Per Well Datapoints." }, { "path": "openhcs/processing/backends/experimental_analysis/result_formats.py", - "sha256": "6951f41a2ff2bdac1945c7e56584a578c111c078cac1d684030c00f95a23f528", - "role": "Owns the CX5 workbook and MetaXpress CSV result readers selected by the workbook scope." + "sha256": "7581999983e5acc930733451dddc84f69bf5dfbb94cfd9954f35690b7a6dd885", + "role": "Declares the EDDU_CX5 and EDDU_metaxpress scope names on the CX5 workbook and MetaXpress CSV result readers and rejects any other scope." }, { "path": "openhcs/domains/microscopy/config.py", @@ -3390,12 +3439,12 @@ }, { "path": "openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py", - "sha256": "2705d459204557c4f7092df3fd21ed687dcf56449ed361485382b1c0ce196444", + "sha256": "782f462451aec3f98fc09956492cf4882531cdc41a459ea3c2a539353ed244b9", "role": "Exports ExperimentalAnalysisEngine, whose run_directory projects config filenames and returns the documented result dictionary." }, { "path": "tests/unit/test_experimental_layout_rows.py", - "sha256": "c3b06c37f8f2a9d960d73238c6b38018de1de824d3e683b06dce5367e95a0adc", + "sha256": "5ead5df21c529a3d29601e9d5eeec84e24ce0130bca0af5e114ee90793ded8d4", "role": "Proves workbook row parsing, positional mapping, multi-digit groups, result-format leaves, normalization semantics, and conditional-format plate grids." }, { @@ -3405,11 +3454,11 @@ } ], "findings": [ - "The programmatic example imported ExperimentalAnalysisConfig from openhcs.core.config; G2/G4 moved it to openhcs.domains.microscopy.config. The import was corrected.", - "The workbook parser, row roles, and result formats moved from openhcs/formats/ into openhcs/processing/backends/experimental_analysis/ (layout_rows.py, analysis.py, result_formats.py) without behaviour change; the authorities were repointed and the workbook syntax claims still hold." + "E1 deleted the ExperimentalAnalysisScope enum from layout_rows.py; the accepted scope values EDDU_CX5 and EDDU_metaxpress are now each result format's scope_name in result_formats.py. The page's values still hold; the layout_rows.py and result_formats.py authority roles were corrected." ], - "disposition": "revise", + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/reference/experimental_config_syntax.rst in full and compared it against the changed authorities: experimental_analysis/layout_rows.py (ExperimentalAnalysisScope enum deleted, E1), result_formats.py (each format subclass declares scope_name; ExperimentalResultFormatStrategy.for_scope raises 'microscope ... not known' for any other value), analysis.py and unified_analysis_engine.py (call for_scope) and tests/unit/test_experimental_layout_rows.py; EDDU_CX5 and EDDU_metaxpress remain the only accepted scopes, the result dictionary still carries the format name, and the example's imports (openhcs.domains.microscopy.config.ExperimentalAnalysisConfig, openhcs.processing.backends.experimental_analysis.ExperimentalAnalysisEngine) resolve; no page edit, authority roles updated.", "On 2026-10-11 (B6 final), compared docs/source/reference/experimental_config_syntax.rst against B6's later changes to openhcs/processing/backends/experimental_analysis/analysis.py, openhcs/processing/backends/experimental_analysis/result_formats.py, openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py, tests/unit/test_experimental_analysis_engine.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/reference/experimental_config_syntax.rst against the later B6 changes to openhcs/processing/backends/experimental_analysis/analysis.py, openhcs/processing/backends/experimental_analysis/result_formats.py, openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py, tests/unit/test_experimental_analysis_engine.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", "On 2026-10-11 (B6), re-read docs/source/reference/experimental_config_syntax.rst in full and compared it against the changed authorities: the deleted openhcs/formats/experimental_{layout_rows,analysis,result_formats}.py (moved to processing/backends/experimental_analysis/), openhcs/core/config.py, unified_analysis_engine.py, and both experimental-analysis tests; checked the row roles, scope values, engine export and run_directory signature against the moved modules, and corrected the ExperimentalAnalysisConfig import to openhcs.domains.microscopy.config.", @@ -3432,12 +3481,12 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", + "sha256": "9625ca054fac6c1e460216cfe0e85621f5306592a004d1bdc28a6b1ce2099366", "role": "Declares the global, pipeline, step, and desktop configuration dataclasses that the generated configuration reference lists." }, { "path": "openhcs/processing/backends/experimental_analysis/analysis.py", - "sha256": "17a701e0740eac3e11319fe73f10302bbcab603218349bfba4e07014aabc27bb", + "sha256": "96b29239552724475c9fb010248bb266340fa0bbf06ae890f6074b8620c6183b", "role": "Parses the Excel experimental-design workbook that the syntax reference documents." }, { @@ -3446,14 +3495,17 @@ "role": "Serves the example corpus map and its native example source projection." }, { - "path": "openhcs/core/artifacts.py", - "sha256": "65eb0ec5c903366243ca8bd5ea07412e8d34bdb81e6015fa76b3c4c59838733f", + "path": "openhcs/core/artifacts/kinds.py", + "sha256": "216b913efad9ec76707e9f61be445fb6fa1c6601c36ef53e296acd4928604515", "role": "Declares the artifact types named by the shared terminology the index points to." } ], - "findings": [], + "findings": [ + "Recorded authority openhcs/core/artifacts.py was deleted when K2 split it into the openhcs/core/artifacts package; the artifact kinds named by the shared terminology are now declared in openhcs/core/artifacts/kinds.py, which replaces it. The page text needed no change." + ], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/reference/index.rst in full and compared it against the changed authorities: openhcs/core/config.py (E1 MaterializationBackend and InputSource families), experimental_analysis/analysis.py (for_scope) and the deleted openhcs/core/artifacts.py (K2 split it into the openhcs/core/artifacts package); the index names no field, class or module and every listed reference page still exists; the artifact-kind authority was repointed to openhcs/core/artifacts/kinds.py; no page edit.", "On 2026-10-11 (B6), re-read docs/source/reference/index.rst in full and compared it against the changed authority openhcs/core/config.py (PartitionFilterMode became a registered subclass family, IncludePartitions/ExcludePartitions; SourceBindingsConfig and StepSourceBindingsConfig declared directly in config.py, K4); the index names no field, class or mode, and every listed reference page still exists; no edit needed.", "On 2026-10-11 (B6 final), compared docs/source/reference/index.rst against B6's later changes to openhcs/core/config.py, openhcs/processing/backends/experimental_analysis/analysis.py, openhcs/core/artifacts.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/reference/index.rst against the later B6 changes to openhcs/core/config.py, openhcs/processing/backends/experimental_analysis/analysis.py, openhcs/core/artifacts.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -3488,12 +3540,12 @@ }, { "path": "openhcs/authoring/session/views.py", - "sha256": "c772d515dfa738363ab403a3e19c9e438c17b0748a3a9ed65c44c416f579a378", + "sha256": "296adbceb7db7a7f7ac3cb09dbe8bb6620dec588a3b593ef2239108298bee5c8", "role": "Owns the session view titles that the plate-manager and pipeline-editor window identities present." }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "3cdb7819f09604046ba69e0b4f57b73e4e98d8f68af80158501a54bb42e701a2", + "sha256": "2d10806e310cb0d47ee7e47b879a7a95b28dcf5c688d9ce2a54b81ae1a041c47", "role": "Composes the main desktop window and its stable docked surfaces from the registered identities." }, { @@ -3525,6 +3577,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/reference/ui_windows.rst in full and compared it against the changed authorities: openhcs/authoring/session/views.py (dataset rows report state, terminal status and debug phase by family name) and openhcs/pyqt_gui/main.py (SelectedItems/AllItems selection scopes, ExplicitUpdateCheck/StartupUpdateCheck); the view titles that the window identities present and the main-window composition are unchanged, and the page still lists no window itself; no edit needed.", "On 2026-10-11 (B6), re-read docs/source/reference/ui_windows.rst in full and compared it against the changed authorities: pyqt_gui/main.py and main_window_workflows.py (U3 moved dock layout, shortcut lifecycle and time-travel navigation to pyqt-reactive DockLayoutStore/DockPanes, ShortcutLifecycle, TimeTravelNavigator); build_main_window_specs still projects UiStableWindowIdentityDeclaration subclasses into WindowManager specs and the openhcs-ui-window-reference directive still renders the registry, so the page (which keeps no window list of its own) remains accurate; no edit needed.", "On 2026-10-11 (B6 final), compared docs/source/reference/ui_windows.rst against B6's later changes to openhcs/authoring/session/views.py: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/reference/ui_windows.rst against B6's later changes to openhcs/authoring/session/views.py, openhcs/pyqt_gui/main.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", @@ -3551,7 +3604,7 @@ "authority": [ { "path": "openhcs/pyqt_gui/main.py", - "sha256": "3cdb7819f09604046ba69e0b4f57b73e4e98d8f68af80158501a54bb42e701a2", + "sha256": "2d10806e310cb0d47ee7e47b879a7a95b28dcf5c688d9ce2a54b81ae1a041c47", "role": "Declares the four Tools > Analysis Consolidation actions and the experimental-analysis directory check." }, { @@ -3561,7 +3614,7 @@ }, { "path": "openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py", - "sha256": "2705d459204557c4f7092df3fd21ed687dcf56449ed361485382b1c0ce196444", + "sha256": "782f462451aec3f98fc09956492cf4882531cdc41a459ea3c2a539353ed244b9", "role": "Projects the five configured filenames and writes normalized, raw, and heatmap outputs." }, { @@ -3578,6 +3631,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/user_guide/analysis_consolidation.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/main.py (selection-scope and update-check families, E1) and experimental_analysis/unified_analysis_engine.py (result format resolved with for_scope); the four Tools > Analysis Consolidation actions, the config.xlsx and metaxpress_style_summary.csv directory contract and the three output workbooks are unchanged; no edit needed.", "On 2026-10-11 (B6), re-read docs/source/user_guide/analysis_consolidation.rst in full and compared it against the changed authority openhcs/pyqt_gui/main.py (U3: dock layout, shortcuts and time-travel navigation moved to pyqt-reactive); the Tools > Analysis Consolidation submenu still declares the four actions (Consolidate Results Directory, Merge MetaXpress Summaries, Concatenate MetaXpress Summaries (Keep Headers), Run Experimental Analysis) and the experimental-analysis directory dialog; no edit needed.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/analysis_consolidation.rst against B6's later changes to openhcs/pyqt_gui/main.py, openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py, tests/unit/test_experimental_analysis_engine.py, openhcs/domains/microscopy/analysis_consolidation.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/analysis_consolidation.rst against the later B6 changes to openhcs/pyqt_gui/main.py, openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py, tests/unit/test_experimental_analysis_engine.py, openhcs/domains/microscopy/analysis_consolidation.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -3612,7 +3666,7 @@ }, { "path": "openhcs/authoring/session/dataset_document.py", - "sha256": "50dc7b601b6a2343c8aaa1ebdaae92d9b4641f7c7ca6a02ade207809e5bdd172", + "sha256": "ddb55e52107819b22d7bb7d18e6310d98eb0600471a722cb789a5a68028d26f3", "role": "Owns the selected and all-datasets document scopes, selected-scope ID enforcement, and the global-draft and per-dataset change checks on apply." }, { @@ -3622,12 +3676,12 @@ }, { "path": "openhcs/authoring/session/views.py", - "sha256": "c772d515dfa738363ab403a3e19c9e438c17b0748a3a9ed65c44c416f579a378", + "sha256": "296adbceb7db7a7f7ac3cb09dbe8bb6620dec588a3b593ef2239108298bee5c8", "role": "Declares the Datasets and Pipeline Editor view titles shown for the two code-document owners." }, { "path": "openhcs/pyqt_gui/widgets/pipeline_editor.py", - "sha256": "7b3b58f3685d5a636a102cae7d1e31c663d5d574bcdc71d4a289b69a9eee6535", + "sha256": "f60e195031b99bd02d8e4cdf1b42ca213fc5669526bc30348ce3ddff14f6876b", "role": "Makes the Pipeline Editor document writable only while a dataset is selected." }, { @@ -3657,17 +3711,17 @@ }, { "path": "openhcs/pyqt_gui/services/ui_bridge_object_state.py", - "sha256": "e64b444a74fdde1d79f82c1361c9b67d8e8916b9bebced092762ce51eabac52b", + "sha256": "f615718186e4aaebdc12eec6bec6a97d47953322075c5b067974e5a16ecc8e80", "role": "Enforces driver writability, returns unchanged for equivalent documents, and applies documents against revision tokens." }, { "path": "openhcs/pyqt_gui/services/ui_agent_bridge.py", - "sha256": "ce650c52475764657bbc0221109879594f9cf1482de8245d6c141f00474d5d23", + "sha256": "55a21b2746e3ba27eff1a3315604725d340dbddce6645c8457f707484a2f1f0c", "role": "Rejects wildcard imports and resolves catalogue-listed external functions in submitted documents." }, { "path": "openhcs/agent/services/ui_bridge_service.py", - "sha256": "c7bdf8b5caec1c21c986cdc2224524839fadfa8764dafcd05125d90c72351899", + "sha256": "a2d682ad7841a7d85173f792961cae2dc5546fb4fcf39caa85f0bbd6ed062b01", "role": "Owns agent-to-UI document inspection and mutation and returns mutation receipts." }, { @@ -3717,12 +3771,12 @@ }, { "path": "tests/unit/pyqt_gui/test_ui_agent_bridge.py", - "sha256": "67cdad5133dc83ccb005b58cc56535ae89e77fc23d3ebd76e616c0da5f567b44", + "sha256": "422be9298e5120c898bd7110884ca770f1f033f07fd69be4ac0caa978d3014f4", "role": "Proves bidirectional UI-bridge inspection, capability enforcement, semantic no-op handling, and mutation of live declarations." }, { "path": "tests/unit/pyqt_gui/test_plate_manager_widget.py", - "sha256": "91ed9468677fee730d67e5d997b7250285e8443f44897e5aa67beaaabe67a7e7", + "sha256": "96ffc41a2fe3ef6ad959e0527bbbec2a86ac64c0a3d627385b384539e4858f81", "role": "Proves selected code-document application preserves unselected dataset graphs." }, { @@ -3732,7 +3786,7 @@ }, { "path": "tests/unit/pyqt_gui/test_pipeline_editor_widget.py", - "sha256": "2024d9560dec38faee1fa922d2f8b7f547dd53d35e0b0e79376e45233445589a", + "sha256": "6761f5c3526167053b22edf5271f3413730d7798e8db45dbc57ec1baa52515aa", "role": "Proves selected-dataset writability and that code apply advances saved baselines for the editor, step, and nested function states." }, { @@ -3751,12 +3805,10 @@ "role": "Verifies numeric value preservation and compact presentation." } ], - "findings": [ - "Recorded authority openhcs/pyqt_gui/services/config_window_code_document.py was deleted in U3; configuration code documents now go through openhcs/pyqt_gui/windows/config_window.py (OpenHCSConfigWindow over pyqt-reactive's ConfigWindow) and openhcs/core/config_document.py, which replace it as authorities.", - "L6 moved lazy-dataclass clean/full emission out of openhcs/serialization/pycodify_formatters.py into objectstate.codegen; the page's sparse clean-document claim now rests on external/ObjectState/src/objectstate/codegen.py, added as an authority, and the pycodify_formatters role was narrowed to function-pattern and step formatting. The page prose (clean vs resolved views, pycodify owning generic serialization) remains accurate, so it was not edited." - ], + "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/user_guide/code_ui_editing.rst in full and compared it against the changed authorities: authoring/session/dataset_document.py (document scopes keyed by SelectedItems/AllItems), session/views.py, pyqt_gui/widgets/pipeline_editor.py, services/ui_bridge_object_state.py (field filters are classes), services/ui_agent_bridge.py and agent/services/ui_bridge_service.py (operation and action statuses are classes; PlateManagerDocumentAssignments names the allowed document assignments) and the three pyqt_gui tests; selected and all-datasets document semantics, the writable check, the unchanged outcome for equivalent documents, wildcard-import rejection and revision-token apply are behaviourally unchanged; the page names none of the renamed enums; no edit needed.", "On 2026-10-11 (B6), re-read docs/source/user_guide/code_ui_editing.rst in full and compared it against the changed authorities: pipeline_document.py (generate_python_source), dataset_document.py (objectstate.retain_only_authored_paths), pipelines.py and pipeline_editor.py (ScopeKey child scopes, GlobalEventBus broadcast removed, use_external_code_editor), deleted config_window_code_document.py (U3), ui_bridge_object_state.py (field markers now from objectstate.inspection FieldInspection; '*' dirty and '_' signature-diff unchanged), pycodify_formatters.py (lazy-dataclass formatters moved to objectstate.codegen, L6), pipeline_editor_workflows.py (K6 debug command classes), pyqt-reactive window_code_document/scope_token_service/function_pattern_code_document (current files), and the changed tests; selected/all scope apply, writability, unchanged no-op, revision tokens, occurrence reconciliation, signature parameter order and clean/resolved views all still hold; replaced and added authorities; no page edit needed.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/code_ui_editing.rst against B6's later changes to openhcs/authoring/session/views.py: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/code_ui_editing.rst against B6's later changes to openhcs/authoring/session/dataset_document.py, openhcs/authoring/session/views.py, openhcs/pyqt_gui/widgets/pipeline_editor.py, openhcs/pyqt_gui/services/ui_bridge_object_state.py, openhcs/pyqt_gui/services/ui_agent_bridge.py, openhcs/agent/services/ui_bridge_service.py, openhcs/pyqt_gui/widgets/shared/services/pipeline_editor_workflows.py, tests/unit/pyqt_gui/test_ui_agent_bridge.py, tests/unit/pyqt_gui/test_plate_manager_widget.py, tests/unit/pyqt_gui/test_pipeline_editor_widget.py, tests/unit/test_pycodify_formatters.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", @@ -3792,12 +3844,12 @@ "authority": [ { "path": "openhcs/utils/environment.py", - "sha256": "2d8738e593b3e92c966089772624f47d17d38bb27fb90974707af659c91e8ad8", + "sha256": "1985ffbe42b5dfb283b78b6282f9b56e08cd51d5a9e79f2dd4eebffcb6da0bf5", "role": "Declares OPENHCS_CPU_ONLY and projects the CPU-only import policy (including JAX_PLATFORMS=cpu) before dependencies import their runtimes." }, { "path": "openhcs/pyqt_gui/launch.py", - "sha256": "bf47d568e2d9bcb758342f34a5b3f030fd8abd9276c5e4fa2406666301e0561f", + "sha256": "fc018b94dc0aeaea4d1d7592973f5635419da5b53b85ecf07448bae975bb6e14", "role": "Declares the desktop --no-gpu flag and enables CPU-only mode before Qt and backend startup." }, { @@ -3812,13 +3864,14 @@ }, { "path": "pyproject.toml", - "sha256": "b5d4d752704b886da20d860743b81dc3d2ce3a3c402208246fadc1c6cb71f597", + "sha256": "75f92010b3c18ede61cf7296064079590c143283871e372a1fd4546417d4e0f9", "role": "Declares the optional gpu extra and the openhcs console entry point." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/user_guide/cpu_only_mode.rst in full and compared it against the changed authorities: openhcs/utils/environment.py (AgentRuntimePlatform rename), openhcs/pyqt_gui/launch.py (QtPlatformSystem enum replaced by a setup table) and pyproject.toml; OPENHCS_CPU_ONLY and --no-gpu are still declared, and pyproject.toml still declares the gpu extra and the openhcs entry point; no edit needed.", "On 2026-10-11 (B6), re-read docs/source/user_guide/cpu_only_mode.rst in full and compared it against the changed authorities: registry_service.py (processing contract now read from reference.declaration instead of reference.metadata, no change to CPU-only registry skipping) and pyproject.toml (dependency floors only; gpu extra and openhcs entry point unchanged); confirmed OPENHCS_CPU_ONLY and --no-gpu are still declared; no edit needed.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/cpu_only_mode.rst against B6's later changes to openhcs/processing/backends/lib_registry/registry_service.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/cpu_only_mode.rst against the later B6 changes to openhcs/processing/backends/lib_registry/registry_service.py, tests/unit/test_library_registry_discovery.py, pyproject.toml: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -3859,12 +3912,12 @@ "authority": [ { "path": "openhcs/processing/custom_functions/manager.py", - "sha256": "0ffa6fc2a18566600dcff33d4b27466d8a5684662fa1c696cb5ea52ea459a1e4", + "sha256": "77612ea5ed53888c36e8ed31f763d8eaf5e0af9e4f32b4beaaa7cff09012b130", "role": "Owns persisted source under the data directory, requires one decorated processing function per source, rejects collisions, and coordinates create, load, replace, and delete." }, { "path": "openhcs/processing/custom_functions/runtime_registry.py", - "sha256": "160fb2fbad1b4d2f1ffaf784bacc5d7355d78c2720e718083cfc8c05748384df", + "sha256": "d2311617cad39d96ceb3ea6e13b6ad89f3b846e95d5dc3d692e7a9a46dd9c739", "role": "Atomically publishes process-local callable metadata and module exports, retaining ephemeral declarations when persisted source reloads." }, { @@ -3879,7 +3932,7 @@ }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "3cdb7819f09604046ba69e0b4f57b73e4e98d8f68af80158501a54bb42e701a2", + "sha256": "2d10806e310cb0d47ee7e47b879a7a95b28dcf5c688d9ce2a54b81ae1a041c47", "role": "Declares the Tools > Custom Functions > Manage Functions action." }, { @@ -3919,12 +3972,12 @@ }, { "path": "tests/unit/test_custom_function_lifecycle.py", - "sha256": "e4f6405ff18dac865141586579ed95b92e143e3672f94e6fca6352476fa0b32d", + "sha256": "81926f26051d5f40eead795c1c9edf60b84b31aab9d61bc0411929328b002de9", "role": "Proves transactional create, load, replace, delete, collision, and source-revision behaviour." }, { "path": "tests/unit/test_function_catalog_zmq.py", - "sha256": "7d10d9c80c101d97f7502cb8195cbb38f1742ac86579b3e4587f3b8a2048a1a6", + "sha256": "9398fd1442b6f793f5b6d273b2965ececd5888f12c37fd11b1e71731c9a9fd93", "role": "Proves persisted custom functions propagate through registry discovery and the execution-server catalogue." }, { @@ -3936,6 +3989,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/user_guide/custom_function_management.rst in full and compared it against the changed authorities: custom_functions/manager.py and runtime_registry.py (CustomFunctionLifetime became PersistedLifetime and EphemeralLifetime classes), openhcs/pyqt_gui/main.py (E1 families) and tests/unit/test_custom_function_lifecycle.py and test_function_catalog_zmq.py (K2 artifacts imports, parameter-source classes); reloading persisted functions still keeps declarations whose lifetime is EphemeralLifetime, CustomFunctionManager.register_from_code(persist=...) is unchanged and the Manage Functions action remains; no edit needed.", "On 2026-10-11 (B6), re-read docs/source/user_guide/custom_function_management.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/main.py (U3 moved dock layout, shortcuts and time-travel navigation to pyqt-reactive; Tools > Custom Functions > Manage Functions unchanged) and tests/unit/test_custom_function_lifecycle.py (function references expose .declaration instead of .metadata, K5); CustomFunctionManager.register_from_code and its package export, and CustomFunctionRuntimeRegistry publishing FunctionMetadata, are unchanged, so the code example and prose remain accurate; no edit needed.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/custom_function_management.rst against B6's later changes to openhcs/processing/custom_functions/manager.py, openhcs/pyqt_gui/main.py, openhcs/mcp/context.py, tests/unit/test_custom_function_lifecycle.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/custom_function_management.rst against the later B6 changes to openhcs/processing/custom_functions/manager.py, openhcs/pyqt_gui/main.py, openhcs/mcp/context.py, tests/unit/test_custom_function_lifecycle.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -3988,38 +4042,39 @@ }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "3cdb7819f09604046ba69e0b4f57b73e4e98d8f68af80158501a54bb42e701a2", + "sha256": "2d10806e310cb0d47ee7e47b879a7a95b28dcf5c688d9ce2a54b81ae1a041c47", "role": "Declares the Tools > Custom Functions > Create New Function action." }, { "path": "openhcs/processing/custom_functions/templates.py", - "sha256": "e114252a60ce5e57111519b98b44954f08733216335fab64413b643f5af61e50", + "sha256": "2e2b29c4ddf0d21464722aa918ea6ef176418dbc6d6b246af65e829157b291c7", "role": "Supplies the starting template shown by the custom-function editor." }, { "path": "openhcs/processing/custom_functions/manager.py", - "sha256": "0ffa6fc2a18566600dcff33d4b27466d8a5684662fa1c696cb5ea52ea459a1e4", + "sha256": "77612ea5ed53888c36e8ed31f763d8eaf5e0af9e4f32b4beaaa7cff09012b130", "role": "Validates and registers saved custom-function source into the function catalogue." }, { "path": "openhcs/processing/custom_functions/runtime_registry.py", - "sha256": "160fb2fbad1b4d2f1ffaf784bacc5d7355d78c2720e718083cfc8c05748384df", + "sha256": "d2311617cad39d96ceb3ea6e13b6ad89f3b846e95d5dc3d692e7a9a46dd9c739", "role": "Projects a validated custom declaration into the process-local callable catalogue." }, { "path": "tests/unit/test_custom_function_lifecycle.py", - "sha256": "e4f6405ff18dac865141586579ed95b92e143e3672f94e6fca6352476fa0b32d", + "sha256": "81926f26051d5f40eead795c1c9edf60b84b31aab9d61bc0411929328b002de9", "role": "Proves validated custom declarations publish atomically without mirrored catalogue state." }, { "path": "tests/unit/test_function_catalog_zmq.py", - "sha256": "7d10d9c80c101d97f7502cb8195cbb38f1742ac86579b3e4587f3b8a2048a1a6", + "sha256": "9398fd1442b6f793f5b6d273b2965ececd5888f12c37fd11b1e71731c9a9fd93", "role": "Proves persisted custom functions propagate through registry discovery and the execution-server catalogue." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/user_guide/custom_functions.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/main.py (E1 families), custom_functions/templates.py (commented artifact example imports from the K2 artifacts package), manager.py and runtime_registry.py (lifetime classes) and the two custom-function tests; the page's example imports only openhcs.core.memory.decorators.numpy and Pure2DContract, both unchanged, and the Create New Function action remains; no edit needed.", "On 2026-10-11 (B6), re-read docs/source/user_guide/custom_functions.rst in full and compared it against the changed authorities: processing_contracts.py (signature lookup through CallableDeclaration, K5), pyqt_gui/main.py (U3; Tools > Custom Functions > Create New Function unchanged) and tests/unit/test_custom_function_lifecycle.py (reference.declaration); the example's numpy decorator still accepts contract= with ProcessingContract subclasses (default FlexibleContract), and Pure2DContract, Pure3DContract, VolumetricToSliceContract and FlexibleContract still exist; no edit needed.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/custom_functions.rst against B6's later changes to openhcs/core/memory/decorators.py, openhcs/core/processing_contracts.py, openhcs/pyqt_gui/main.py, openhcs/processing/custom_functions/manager.py, tests/unit/test_custom_function_lifecycle.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/custom_functions.rst against the later B6 changes to openhcs/core/memory/decorators.py, openhcs/core/processing_contracts.py, openhcs/pyqt_gui/main.py, openhcs/processing/custom_functions/manager.py, tests/unit/test_custom_function_lifecycle.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -4047,7 +4102,7 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", + "sha256": "9625ca054fac6c1e460216cfe0e85621f5306592a004d1bdc28a6b1ce2099366", "role": "Declares DtypeConfig with default_dtype_conversion (NATIVE_OUTPUT default) and StepMaterializationConfig, from which the lazy step forms are generated." }, { @@ -4067,7 +4122,7 @@ }, { "path": "openhcs/processing/backends/processors/numpy_processor.py", - "sha256": "58df7297d1021decca547d9f5959b8d8e563544b4644ac4977452030ed8b30da", + "sha256": "fddb0447b60e30902a40b63d4e60ea52c44497807ab539cb03a9dde57766de39", "role": "Declares the decorated create_projection function used in the 8-bit materialisation example." }, { @@ -4082,13 +4137,14 @@ }, { "path": "openhcs/core/steps/abstract.py", - "sha256": "9e710b3db635730f88258e5a6e18aed607876e296783022c06798f2bcd03d5e4", + "sha256": "2bffdfe02664c4974797bb6cefbf7f84ebb6912fcc11e026a3f2d8b3da5262b8", "role": "Declares the step-level dtype_config and step_materialization_config lazy overrides used by the examples." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/user_guide/dtype_conversion.rst in full and compared it against the changed authorities: openhcs/core/config.py (E1 MaterializationBackend and InputSource families; DtypeConfig and the lazy configs unchanged), processors/numpy_processor.py (K2 artifacts imports) and openhcs/core/steps/abstract.py (docstring spellings ReadPreviousStep/ReadPipelineStart); LazyDtypeConfig, LazyStepMaterializationConfig, create_projection and the step dtype_config and step_materialization_config fields used by both examples are unchanged; no edit needed.", "On 2026-10-11 (B6), re-read docs/source/user_guide/dtype_conversion.rst in full and compared it against the changed authorities: openhcs/core/config.py (K4 source-binding classes moved in, PartitionFilterMode family, source-metadata field dtypes; DtypeConfig, LazyDtypeConfig and LazyStepMaterializationConfig unchanged), openhcs/core/steps/abstract.py (lazy config imports consolidated) and tests/unit/test_callable_contract.py (CallableMetadata folded into CallableDeclaration, K5; dtype_config remains a config-bound runtime parameter); both code examples' imports still resolve; no edit needed.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/dtype_conversion.rst against B6's later changes to openhcs/core/config.py, openhcs/core/memory/decorators.py, openhcs/processing/backends/processors/numpy_processor.py, tests/unit/test_callable_contract.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/dtype_conversion.rst against the later B6 changes to openhcs/core/config.py, openhcs/core/memory/decorators.py, openhcs/processing/backends/processors/numpy_processor.py, tests/unit/test_callable_contract.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -4115,12 +4171,12 @@ "authority": [ { "path": "openhcs/pyqt_gui/main.py", - "sha256": "3cdb7819f09604046ba69e0b4f57b73e4e98d8f68af80158501a54bb42e701a2", + "sha256": "2d10806e310cb0d47ee7e47b879a7a95b28dcf5c688d9ce2a54b81ae1a041c47", "role": "Declares the Tools > Analysis Consolidation > Run Experimental Analysis action and its directory check for the config and results files." }, { "path": "openhcs/processing/backends/experimental_analysis/analysis.py", - "sha256": "17a701e0740eac3e11319fe73f10302bbcab603218349bfba4e07014aabc27bb", + "sha256": "96b29239552724475c9fb010248bb266340fa0bbf06ae890f6074b8620c6183b", "role": "Parses the drug_curve_map and plate_groups sheets and applies replicate-local normalization and heatmap export." }, { @@ -4130,12 +4186,12 @@ }, { "path": "openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py", - "sha256": "2705d459204557c4f7092df3fd21ed687dcf56449ed361485382b1c0ce196444", + "sha256": "782f462451aec3f98fc09956492cf4882531cdc41a459ea3c2a539353ed244b9", "role": "Runs the directory workflow and writes the normalized, raw, and heatmap workbooks." }, { "path": "tests/unit/test_experimental_layout_rows.py", - "sha256": "c3b06c37f8f2a9d960d73238c6b38018de1de824d3e683b06dce5367e95a0adc", + "sha256": "5ead5df21c529a3d29601e9d5eeec84e24ce0130bca0af5e114ee90793ded8d4", "role": "Proves workbook row parsing, positional mapping, multi-digit groups, exclusions, normalization, and heatmap output." }, { @@ -4147,6 +4203,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/user_guide/experimental_layouts.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/main.py (E1 families), experimental_analysis/analysis.py and unified_analysis_engine.py (for_scope) and tests/unit/test_experimental_layout_rows.py; the Run Experimental Analysis action, the directory filenames, the three output workbooks and ExperimentalAnalysisConfig.normalization_method are unchanged; no edit needed.", "On 2026-10-11 (B6), re-read docs/source/user_guide/experimental_layouts.rst in full and compared it against the changed authority openhcs/pyqt_gui/main.py (U3: dock layout, shortcuts and time-travel navigation moved to pyqt-reactive); the Run Experimental Analysis action and its directory dialog are unchanged, and ExperimentalAnalysisConfig.normalization_method still defaults to fold change in openhcs/domains/microscopy/config.py; no edit needed.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/experimental_layouts.rst against B6's later changes to openhcs/pyqt_gui/main.py, openhcs/processing/backends/experimental_analysis/analysis.py, openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py, tests/unit/test_experimental_analysis_engine.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/experimental_layouts.rst against the later B6 changes to openhcs/pyqt_gui/main.py, openhcs/processing/backends/experimental_analysis/analysis.py, openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py, tests/unit/test_experimental_analysis_engine.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -4175,7 +4232,7 @@ "authority": [ { "path": "openhcs/pyqt_gui/main.py", - "sha256": "3cdb7819f09604046ba69e0b4f57b73e4e98d8f68af80158501a54bb42e701a2", + "sha256": "2d10806e310cb0d47ee7e47b879a7a95b28dcf5c688d9ce2a54b81ae1a041c47", "role": "Composes the PyQt desktop main window that the openhcs command launches." }, { @@ -4190,18 +4247,19 @@ }, { "path": "openhcs/core/config.py", - "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", + "sha256": "9625ca054fac6c1e460216cfe0e85621f5306592a004d1bdc28a6b1ce2099366", "role": "Declares the nested processing, materialization, and streaming configuration dataclasses on PipelineConfig and steps." }, { "path": "openhcs/core/pipeline/compiler.py", - "sha256": "4a33908fb42874e6d57eeea3f5305793ddbda41d13356f0a78501af38dcdfeb4", + "sha256": "40b07b7f40aca39b19b95a8f61c1b0c8270f0ab0a29e8c36caa13f04d8fa539f", "role": "Compiles pipeline declarations into typed step plans and a CompiledExecutionBundle before execution." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/user_guide/index.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/main.py (E1 families), openhcs/core/config.py (MaterializationBackend and InputSource families) and openhcs/core/pipeline/compiler.py (OrchestratorCompiled/OrchestratorCompileFailed state classes, MaterializeToZarr, PrimaryPlaneRole); the declaration model (list[FunctionStep] plus PipelineConfig, nested configs, compilation into typed plans and a bundle) is unchanged and every toctree page exists; no edit needed.", "On 2026-10-11 (B6), re-read docs/source/user_guide/index.rst in full and compared it against the changed authorities: pyqt_gui/main.py (U3 dock/shortcut/time-travel machinery moved to pyqt-reactive), tests/unit/pyqt_gui/test_ui_config_lifecycle.py (shortcut lifecycle tests moved with it; the textual_tui non-recurrence check remains), pipeline_document.py (generate_python_source), core/config.py (PartitionFilterMode family, K4 source-binding classes) and compiler.py (ScopeKey scope ids, K6 progress phase/status classes; still returns CompiledExecutionBundle); the PyQt-only interface, toctree and declaration-model claims remain accurate; no edit needed.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/index.rst against B6's later changes to openhcs/pyqt_gui/main.py, openhcs/core/config.py, openhcs/core/pipeline/compiler.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/index.rst against the later B6 changes to openhcs/pyqt_gui/main.py, openhcs/core/config.py, openhcs/core/pipeline/compiler.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -4242,17 +4300,17 @@ }, { "path": "openhcs/pyqt_gui/services/reactor_providers.py", - "sha256": "04ad362d7bb31641f5a4ba0bfcc8670cd2e330a6eadc231a0ca5df7abed8d2b4", + "sha256": "ca5fe97ba932805dd0f8cfa3bc85352cb428b12a46fcae83751382092d77475c", "role": "Registers the OpenHCS log-discovery and server-scan providers that supply the Log Viewer's log files and live server list." }, { "path": "openhcs/pyqt_gui/config.py", - "sha256": "a3eece2e5b2926fc6fc0edd197221da451ab7c0d0d6f4ffc8bb2ff865ad74bd4", + "sha256": "d092cb20206127484fbbc80a38a5152462b9b8991e848a814c85f564d51f4db8", "role": "Declares UI configuration, including logging and execution-server settings." }, { "path": "openhcs/runtime/zmq_execution_client.py", - "sha256": "5df650f60c4d48cbf47c275d4e46b870b35c915ca8baa8e06e77f321578ca6d7", + "sha256": "ad7ea01599493dbefce1f3c91e6b082e932c6a5183d0a85a186f7bd7b62779a9", "role": "Spawns the execution server with its own per-port log file and startup status file and reports connection state." }, { @@ -4261,11 +4319,10 @@ "role": "Owns the execution-log-viewer capture, caption, accessibility text, and dimensions." } ], - "findings": [ - "The audit role for reactor_providers.py described 'log, process, status, and execution reactors'; the module registers provider objects (log discovery, server scan, codegen, selection), so the role now names the log-discovery and server-scan providers the page relies on. The page prose itself remains accurate." - ], + "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/user_guide/log_viewer.rst in full and compared it against the changed authorities: pyqt_gui/services/reactor_providers.py (OrchestratorCreated state class), openhcs/pyqt_gui/config.py (AgentRuntimePlatform rename, GuiLogLevel.emits_records) and openhcs/runtime/zmq_execution_client.py (artifact inspection request sent directly); none touches log discovery, per-port server logs, level and source filtering or live server scanning; no edit needed.", "On 2026-10-11 (B6), re-read docs/source/user_guide/log_viewer.rst in full and compared it against the changed authorities: reactor_providers.py (codegen provider now calls pycodify.generate_python_source; external-editor env var) and zmq_execution_client.py (debug requests sent directly via request.to_dict(), K6 DebugCommandType class); neither touches log discovery, per-port server logs or live server scanning; confirmed the execution-log-viewer gallery scenario and the pipeline_debugging_guide link still exist; corrected the reactor_providers role.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/log_viewer.rst against B6's later changes to openhcs/pyqt_gui/services/reactor_providers.py, openhcs/runtime/zmq_execution_client.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/log_viewer.rst against the later B6 changes to openhcs/pyqt_gui/services/reactor_providers.py, openhcs/runtime/zmq_execution_client.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -4313,17 +4370,17 @@ }, { "path": "openhcs/mcp/client_registration.py", - "sha256": "55b701bb77a3de19243fa9a6276f784fad8329e2d435fb26fcc7fdce1e060731", + "sha256": "4d6fc5f744617ab1d0e9b8317d07965496aaa6b098a4d179ef95cc6baec35671", "role": "Owns detection and registration of the stable launcher (and skill sync) in supported desktop clients." }, { "path": "openhcs/mcp/lifecycle.py", - "sha256": "671859916b91d5e2f4ce9c5cd329c6d088800d3b8282e0d9505c49c183c7c68b", + "sha256": "4c2fb453798183bbc653b3ad1bf6d3e0ee0cf54383b6672cb745a325c92e6c3a", "role": "Owns MCP process lifecycle, endpoint readiness, and shutdown coordination." }, { "path": "openhcs/agent/path_policy.py", - "sha256": "73fb2d2f729487a926861bd3919ae26c92da79229db52d9c61e94ca88c7f392d", + "sha256": "cd01a3a0ab5e3c4a4e05b212f3df3d8c849586e80c6bc221532c1fbce76d8282", "role": "Owns OPENHCS_AGENT_READ_ROOTS and OPENHCS_AGENT_WRITE_ROOTS and rejection of paths outside them." }, { @@ -4338,7 +4395,7 @@ }, { "path": "benchmark/cellprofiler_benchmark_cli.py", - "sha256": "2e0de28a96c57ee150e28fbbed28d876507e9281075a03f729838f3ca3610966", + "sha256": "0c2cab4d989d50149cb314e908baace8aa794953c86ebc572fa120beaa695780", "role": "Declares the list-cases, inspect-run, inspect-measured, and run-measured commands, including the required --wait-timeout-ms." }, { @@ -4353,7 +4410,7 @@ }, { "path": "benchmark/control.py", - "sha256": "83fe9016625d7d8721905b8a153a28c3dcaaf2b9e925a3c97711c634868156b7", + "sha256": "b10dd87d7bc44063566e0c53d849091c0fc43a345b434c13467df1ede86ac892", "role": "Reads authorized existing run directories without invoking benchmark execution." }, { @@ -4368,12 +4425,12 @@ }, { "path": "tests/unit/agent/test_mcp_client_registration.py", - "sha256": "0763752fc04c32d2812f509845886f10c366e1a67ec290d50e433096a8d4debe", + "sha256": "2b40d0188317828d6198e0787bb6560b490238bcc2c63a056fe54ac8fd37f8f0", "role": "Proves supported-client discovery, registration commands, and resulting MCP configuration." }, { "path": "tests/unit/agent/test_mcp_server.py", - "sha256": "4e6f6b1761b2c0316c99984baad6ea2c97e5c76c3f155111b8e498d7358e3175", + "sha256": "7f47844b6fc0bf2461dfdb97e4b622a8d16f0f8c00004c1e6b7ea1a9997abe82", "role": "Proves MCP server tool registration, request handling, and lifecycle behaviour." }, { @@ -4382,13 +4439,10 @@ "role": "Proves benchmark capability binding, path policy, progress, artifacts, and lightweight imports." } ], - "findings": [ - "The run-measured paragraph had dropped the required --wait-timeout-ms argument; benchmark/cellprofiler_benchmark_cli.py still requires it and the session stops a run that exceeds it. The requirement was restored.", - "The page named the desktop multi-dataset window Plate Manager; after U1 it is titled Datasets, so the headless-session sentence now says Datasets window state.", - "Code issue, not edited: the openhcs-mcp --surface help text in openhcs/mcp/bootstrap.py says the default desktop surface hides headless tools, but DesktopLocalCapabilitySurfaceProfile includes CapabilityWorkflowGroup.HEADLESS_EXECUTION; the page's statement that the desktop surface includes headless-execution tools matches the declaration." - ], - "disposition": "revise", + "findings": [], + "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/user_guide/mcp_clients.rst in full and compared it against the changed authorities: openhcs/mcp/client_registration.py (registration statuses and platform keys became class families), openhcs/mcp/lifecycle.py (McpProcessRecoveryReason classes), openhcs/agent/path_policy.py, benchmark/cellprofiler_benchmark_cli.py (observation-scope choices read from the ZMQRuntimeObservationExportScope registry), benchmark/control.py and the two agent tests; the --observation-scope spellings outcomes and values (default values) are still the registered names, --wait-timeout-ms is still required, the dataset row still reports complete (ExecutionCompleted.name), and client registration and restart behaviour are unchanged; no edit needed. The --surface help text in openhcs/mcp/bootstrap.py still says the desktop surface hides headless tools although the desktop profile includes them (reported earlier, code not edited).", "On 2026-10-11 (B6 final), compared docs/source/user_guide/mcp_clients.rst against B6's later changes to benchmark/cellprofiler_benchmark_cli.py: the session decides in-place initialization by an actual file-creation attempt that leaves nothing behind, its debug-session methods moved into an inherited mixin, the measured CLI treats Windows Ctrl+Break like Ctrl+C, and the Official30 viewer test reads typed payload summaries. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/mcp_clients.rst against B6's later changes to openhcs/authoring/session/operations/datasets.py, benchmark/cellprofiler_benchmark_cli.py, benchmark/control_service.py, tests/unit/agent/test_mcp_server.py, tests/unit/test_benchmark_control.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/mcp_clients.rst against the later B6 changes to openhcs/agent/capabilities.py, openhcs/authoring/session/operations/datasets.py, benchmark/cellprofiler_benchmark_cli.py, benchmark/control_service.py, tests/unit/agent/test_mcp_server.py, tests/unit/test_benchmark_control.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -4445,27 +4499,27 @@ "authority": [ { "path": "openhcs/core/config.py", - "sha256": "6ee23916956716df52cb24893ecdd69747b7db2869f6622b534d8ae43e7bf5d7", + "sha256": "9625ca054fac6c1e460216cfe0e85621f5306592a004d1bdc28a6b1ce2099366", "role": "Declares StreamingDefaults (persistent, host, transport_mode, enabled), partition_filter, and the Napari and Fiji streaming configs with their ports." }, { "path": "openhcs/core/steps/abstract.py", - "sha256": "9e710b3db635730f88258e5a6e18aed607876e296783022c06798f2bcd03d5e4", + "sha256": "2bffdfe02664c4974797bb6cefbf7f84ebb6912fcc11e026a3f2d8b3da5262b8", "role": "Declares the per-step napari_streaming_config, fiji_streaming_config, and step_materialization_config fields." }, { "path": "openhcs/runtime/viewer_display.py", - "sha256": "c8979a3ef72676124270bfca21459b7a071b956333c72a4f070433b1dc2cba8b", + "sha256": "6af5ae4f4580a69ac2d55cc3d036aa7040173b51d86959959ccdccea7899f866", "role": "Declares the Napari and Fiji display settings selectable on a streaming config." }, { "path": "pyproject.toml", - "sha256": "b5d4d752704b886da20d860743b81dc3d2ce3a3c402208246fadc1c6cb71f597", + "sha256": "75f92010b3c18ede61cf7296064079590c143283871e372a1fd4546417d4e0f9", "role": "Declares the gui, napari, fiji, and viz extras." }, { "path": "openhcs/runtime/viewer_protocol.py", - "sha256": "c4bdae4bae6dee412f0d27a9cf75a95db53b3bd95c03cee7d046be16b52fe5b8", + "sha256": "347a65b63c149f2f437014d537cfaf481a33801d860742f4083b7377494e792f", "role": "Declares the transport-independent viewer messages and incremental settlement-progress fields." }, { @@ -4485,22 +4539,22 @@ }, { "path": "openhcs/runtime/napari_viewer_server.py", - "sha256": "038d7a76eeac8d69079591cf6eb7736af5415b28a7a067d6677bd15d115d11c7", + "sha256": "ac54f8bfdce33150c31b38bcd5cf6069b5fd66fb7c05d5a6951610ec3d895bfd", "role": "Implements Napari stream reception, layer updates, selected-layer routing, and the dimension-label overlay." }, { "path": "openhcs/runtime/napari_streaming_handlers.py", - "sha256": "be585484075783f7717776d2b8ba163aab8a69aa24c560ca3844214485bd9eef", + "sha256": "b43f29163cef420c77f30e46e92d94145520f7534077d54636a0ab141da32e7e", "role": "Owns shared semantic axis slots, singleton alignment, and per-route forward-progress records for Napari layers." }, { "path": "openhcs/runtime/fiji_viewer_server.py", - "sha256": "ee246d32ed828bbf4665cc8a411e807da29bd43c2475012243f27e93cb550410", + "sha256": "352ef6b773da07b8103873606cf8a438b8ec70b64cfc57526cc9b18cf9b48b40", "role": "Implements Fiji stream reception, image-stack updates, ROI handling, and endpoint lifecycle." }, { "path": "tests/unit/test_napari_streaming_handlers.py", - "sha256": "93244d4d09c5cd6d7a2645fca2482578ad52646f3bb9e6de5ebe505b157fbfea", + "sha256": "c08363837db3efd13534c966a6192b785cb85ccca742b574ac9bd0e74600b80e", "role": "Proves Napari streaming updates layers and carries ROI navigation through the declared protocol." }, { @@ -4510,7 +4564,7 @@ }, { "path": "tests/unit/test_fiji_viewer_server.py", - "sha256": "da575dfb02895ecc831b34d6bdf6b58112b986556459444e9701cbf43221ea22", + "sha256": "b19316b843cc53c21bd3e00b22694fb645dcba062610295adf54abed7b4f51df", "role": "Proves Fiji endpoint lifecycle, streamed image updates, and ROI message handling." }, { @@ -4522,6 +4576,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6), re-read docs/source/user_guide/real_time_visualization.rst in full and compared it against the changed authorities: openhcs/core/config.py and steps/abstract.py (E1 families), openhcs/runtime/viewer_display.py (NapariVariableSizeHandling became SeparateLayers/PadToMaxSize classes that own routing and padding), pyproject.toml, viewer_protocol.py (ViewerSettlePhase, launch-session and lifecycle classes; settle progress parsed as a WireView), napari_viewer_server.py, napari_streaming_handlers.py, fiji_viewer_server.py and their tests; settlement still reports completed/total progress and fails on a failed phase, the dimension-label overlay still follows the selected OpenHCS layer and clears for a non-OpenHCS layer, the per-step streaming fields, partition_filter and persistent are unchanged, and the gui, napari, fiji and viz extras are declared; no edit needed.", "On 2026-10-11 (B6), re-read docs/source/user_guide/real_time_visualization.rst in full and compared it against the changed authorities: openhcs/core/config.py (PartitionFilterMode became a subclass family, IncludePartitions/ExcludePartitions; partition_filter, persistent and the streaming configs unchanged; K4 source-binding classes moved in), openhcs/core/steps/abstract.py (lazy-config imports consolidated; napari_streaming_config, fiji_streaming_config and step_materialization_config still declared) and pyproject.toml (dependency floors only; gui, napari, fiji and viz extras unchanged); no edit needed.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/real_time_visualization.rst against B6's later changes to openhcs/core/config.py, openhcs/runtime/viewer_protocol.py, openhcs/runtime/fiji_viewer_server.py, tests/unit/test_napari_streaming_handlers.py, tests/unit/test_viewer_result_element_selection.py, tests/unit/test_fiji_viewer_server.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/real_time_visualization.rst against the later B6 changes to openhcs/core/config.py, pyproject.toml, openhcs/runtime/viewer_protocol.py, openhcs/runtime/fiji_viewer_server.py, tests/unit/test_napari_streaming_handlers.py, tests/unit/test_viewer_result_element_selection.py, tests/unit/test_fiji_viewer_server.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", diff --git a/docs/source/architecture/artifact_contract_system.rst b/docs/source/architecture/artifact_contract_system.rst index 57295bbd39..cf7107385f 100644 --- a/docs/source/architecture/artifact_contract_system.rst +++ b/docs/source/architecture/artifact_contract_system.rst @@ -133,7 +133,7 @@ consumer inherit the exact producer and group scope without generated-name prefix matching or a CellProfiler-specific branch. The initial raw image flow has the compiler-owned ``PIPELINE_INPUT_ARTIFACT`` -identity. A first-step consumer or a step selecting ``PIPELINE_START`` can +identity. A first-step consumer or a step selecting ``ReadPipelineStart`` can therefore inherit its actual source scope without referring to an earlier step. Explicit source bindings keep their own declared identities. diff --git a/docs/source/architecture/cellprofiler_interop.rst b/docs/source/architecture/cellprofiler_interop.rst index c8d73ec3c5..5a5dd3caca 100644 --- a/docs/source/architecture/cellprofiler_interop.rst +++ b/docs/source/architecture/cellprofiler_interop.rst @@ -79,7 +79,7 @@ exports into exact callable artifact contracts. These are executable conversion rules, not prose conventions. That does not make the importer a universal native-pipeline intent oracle. The -``InputSource`` enum has only previous-step and pipeline-start main flow; an +``InputSource`` family has only previous-step and pipeline-start main flow; an additional named image is represented by an artifact input plus source binding or prior producer. A ``.cppipe`` also does not choose arbitrary OpenHCS viewer streaming, native checkpoint, VFS, or user review policy that it never declared. diff --git a/docs/source/architecture/code_ui_interconversion.rst b/docs/source/architecture/code_ui_interconversion.rst index 2d492c0196..18974fc53a 100644 --- a/docs/source/architecture/code_ui_interconversion.rst +++ b/docs/source/architecture/code_ui_interconversion.rst @@ -112,7 +112,8 @@ ObjectState stores UI-editable values in typed scopes. Field projections expose: * dirty/default markers; * provenance and scope identity; * field descriptions from the underlying Python type; -* accepted enum inputs derived from that type declaration. +* accepted enum or registered-family choice inputs derived from that type + declaration. ObjectState field mutation is the narrow projection for one field; a code document is the revision-protected projection for a related configuration or diff --git a/docs/source/architecture/experimental_analysis_system.rst b/docs/source/architecture/experimental_analysis_system.rst index c5fe0fb095..a6c8481375 100644 --- a/docs/source/architecture/experimental_analysis_system.rst +++ b/docs/source/architecture/experimental_analysis_system.rst @@ -23,8 +23,8 @@ percentage-of-control leaf calculations. The shared analysis module traverses the experiment once and applies that declaration against each biological replicate's own control distribution; it does not infer control semantics from condition-name strings. ``ExperimentalResultFormatStrategy`` leaves own the -CX5 and MetaXpress result readers selected by the workbook's -``ExperimentalAnalysisScope`` declaration. The shared analysis module owns +CX5 and MetaXpress result readers; each leaf's ``scope_name`` is the scope a +workbook declares to select it. The shared analysis module owns layout parsing, exclusion projection, and table/plate-grid export. Pipeline boundary diff --git a/docs/source/architecture/external_foundations.rst b/docs/source/architecture/external_foundations.rst index 98d5483d77..aecd6e7e7d 100644 --- a/docs/source/architecture/external_foundations.rst +++ b/docs/source/architecture/external_foundations.rst @@ -60,7 +60,9 @@ python-introspect Owns callable/signature analysis, wrapped-target resolution, parameter metadata, field labels, callable-declaration projection, analyzer extensions, the ``to_jsonable`` encoder paired with the ``dataclass_from_mapping`` decoder, - and declared public-API and lazy-export helpers. OpenHCS + the registry-key spelling and field choices of registered class families + (``RegisteredClassChoices``), and declared public-API and lazy-export + helpers. OpenHCS owns domain exclusions, callable contracts, and step-editor consumption. See the `python-introspect repository `_. diff --git a/docs/source/architecture/measurement_equivalence_system.rst b/docs/source/architecture/measurement_equivalence_system.rst index c13e359bbd..1d96b4255c 100644 --- a/docs/source/architecture/measurement_equivalence_system.rst +++ b/docs/source/architecture/measurement_equivalence_system.rst @@ -69,10 +69,11 @@ counts, measurement features/content, tables, and images. An empty difference tuple means the compared outputs are semantically equivalent. The product package ``openhcs.core.equivalence`` keeps only the measurement -semantics that production code uses: feature keys, semantic profiles, rows, -relationships, and the policy. Output snapshots, comparison, and the report -live in ``benchmark/equivalence`` beside the benchmark code that consumes them; -no product module imports the benchmark package. +semantics that production code uses: feature keys, semantic profiles, cell +and array signatures, relationship aggregate features, and the policy. Output +snapshots, row and fact projection, relationship plane alignment, comparison, +and the report live in ``benchmark/equivalence`` beside the benchmark code that +consumes them; no product module imports the benchmark package. Official30 evidence boundary ---------------------------- @@ -186,6 +187,13 @@ runtime/output snapshots and compares: - CPA SQLite table projections - CellProfiler Analyst ``.properties`` values +CellProfiler writes most AreaShape values by position among an image's +non-empty objects, so an empty declared object shifts later values onto the +wrong ``ObjectNumber``. OpenHCS keeps each value on the object it measured. +The adapter therefore reads native reference tables through +``benchmark/equivalence/native_shape_rows.py``, which moves those values back +to their objects within each image without adding or dropping any. + The strict CellProfiler policy sets numeric and image absolute/relative tolerances to ``1e-6``, permits zero differing image fraction, disables the broad feature-specific relaxations used by less strict compatibility modes, and diff --git a/docs/source/architecture/processing_semantics.rst b/docs/source/architecture/processing_semantics.rst index 8a55ef7b27..0dc418a79f 100644 --- a/docs/source/architecture/processing_semantics.rst +++ b/docs/source/architecture/processing_semantics.rst @@ -118,7 +118,7 @@ Common mistakes --------------- - Treating ``group_by`` as the definition of the stack axis. -- Treating a named source binding as a third ``InputSource`` enum value. +- Treating a named source binding as a third ``InputSource`` choice. - Treating a value's presence in the runtime store as a persistence request. - Treating ``Pure2DContract`` as “the callable only accepts rank-2 arrays.” - Passing ``variable_components`` or ``group_by`` directly to diff --git a/docs/source/architecture/progress_runtime_projection_system.rst b/docs/source/architecture/progress_runtime_projection_system.rst index be1f7b1ae3..00735b8d06 100644 --- a/docs/source/architecture/progress_runtime_projection_system.rst +++ b/docs/source/architecture/progress_runtime_projection_system.rst @@ -95,7 +95,7 @@ normal execution projection beside them. The runtime projection is not the session-state machine. Given the selected target, active session, manager execution state, terminal summary, pause state, -and snapshots, ``DebugSessionPhaseDeclarationBase`` selects the current debugger +and snapshots, ``DebugSessionPhase.for_context`` selects the current debugger phase. Toolbar, inspector, UI-bridge, agent, and MCP-facing adapters consume that typed session context and projection instead of re-deriving phase semantics. diff --git a/docs/source/architecture/quick_start.rst b/docs/source/architecture/quick_start.rst index b854f38c06..9646e61c6d 100644 --- a/docs/source/architecture/quick_start.rst +++ b/docs/source/architecture/quick_start.rst @@ -222,7 +222,7 @@ For files such as ``A01_s1_DNA.tif`` and ``A01_s1_GFP.tif``: .. code-block:: python - from openhcs.constants.input_source import InputSource + from openhcs.constants.input_source import ReadPipelineStart from openhcs.core.config import ( LazyProcessingConfig, LazySourceBindingsConfig, @@ -231,13 +231,15 @@ For files such as ``A01_s1_DNA.tif`` and ``A01_s1_GFP.tif``: ) from openhcs.core.source_bindings import ( ComponentSelector, + FromFileName, MetadataExtractionRule, - MetadataSource, NamedSourceBinding, + PathIsImage, + SourceExtension, + SourceFileName, SourceFilterClause, - SourceFilterMatchType, - SourceFilterSubject, SourceSelector, + TextContains, ) from openhcs.core.steps.function_step import FunctionStep from openhcs.domains.microscopy.axes import Microscopy @@ -251,8 +253,8 @@ For files such as ``A01_s1_DNA.tif`` and ``A01_s1_GFP.tif``: selector=SourceSelector( filters=( SourceFilterClause( - subject=SourceFilterSubject.FILE, - match_type=SourceFilterMatchType.CONTAINS, + subject=SourceFileName, + match_type=TextContains, value=token, ), ), @@ -266,13 +268,13 @@ For files such as ``A01_s1_DNA.tif`` and ``A01_s1_GFP.tif``: source_bindings_config=LazySourceBindingsConfig( source_filters=( SourceFilterClause( - subject=SourceFilterSubject.EXTENSION, - match_type=SourceFilterMatchType.IS_IMAGE, + subject=SourceExtension, + match_type=PathIsImage, ), ), metadata_rules=( MetadataExtractionRule( - source=MetadataSource.FILE_NAME, + source=FromFileName, pattern=( r"^(?P[A-H][0-9]{2})_" r"s(?P[0-9]+)_(?P[^.]+)" @@ -291,7 +293,7 @@ For files such as ``A01_s1_DNA.tif`` and ``A01_s1_GFP.tif``: name="Normalize DNA", func=stack_percentile_normalize, processing_config=LazyProcessingConfig( - input_source=InputSource.PIPELINE_START, + input_source=ReadPipelineStart, ), source_bindings=LazyStepSourceBindingsConfig( enabled=True, diff --git a/docs/source/architecture/runtime_value_system.rst b/docs/source/architecture/runtime_value_system.rst index 6abaeee633..ee9c958077 100644 --- a/docs/source/architecture/runtime_value_system.rst +++ b/docs/source/architecture/runtime_value_system.rst @@ -89,11 +89,14 @@ Function-output contextualization A callable return does not inherit context from whichever runtime value happens to be nearby. The exact compiled output artifact type first selects behaviour -through its ``ArtifactType.contextualize_output`` method. For an image output, -``ImageArtifactType`` then calls ``contextualize_image_output`` on the source -value chosen by the output's compiled ``source_context_source`` relation, so the -source's nominal type decides. This is a two-stage owner dispatch, not a table -of callable names or payload shapes. +through its ``ArtifactType.contextualize_output`` method, which asks the +returned value's ``contextualized_output`` which of the type's hooks applies: +a plain value, an image payload, a source-projected image, or one value per +runtime slice. For an image payload, ``ImageArtifactType`` then calls +``contextualize_image_output`` on the source value chosen by the output's +compiled ``source_context_source`` relation, so the source's nominal type +decides. This is owner dispatch on the artifact type, the returned value, and +the source, not a table of callable names or payload shapes. An image result that already carries context may bypass derivation only when its source identity is complete and, for a declared artifact output, its represented diff --git a/docs/source/development/source_binding_extension.rst b/docs/source/development/source_binding_extension.rst index 30df7368aa..f53925f257 100644 --- a/docs/source/development/source_binding_extension.rst +++ b/docs/source/development/source_binding_extension.rst @@ -24,9 +24,10 @@ Strategy selection Existing families select source universes, matching behavior, component projection, and workspace projection, for example ``SourceUniverseRequest``, -``SourceFilterMatcher``, and the ``SourceSetPairing`` family (``PairByOrder``, -``PairByMetadata``) that decides how matched sources form source sets. Add a concrete subclass to the correct family root -and declare its registry key. Generic consumers should continue to use the +``SourceFilterMatchType`` (whose subclasses own the filter predicates), and the +``SourceSetPairing`` family (``PairByOrder``, ``PairByMetadata``) that decides +how matched sources form source sets. Add a concrete subclass to the correct +family root and declare its registry key. Generic consumers should continue to use the root registry or its strategy mixin. Virtual workspaces diff --git a/docs/source/guide_for_biologists/image_sources.rst b/docs/source/guide_for_biologists/image_sources.rst index dbd6372c34..3b3d2594a4 100644 --- a/docs/source/guide_for_biologists/image_sources.rst +++ b/docs/source/guide_for_biologists/image_sources.rst @@ -156,7 +156,7 @@ MAP2[channel=2], SMI312[channel=4])``. A MetaXpress neurite step has two useful declaration choices: simplest inherited full stack - Set ``input_source=PIPELINE_START`` and + Set ``input_source=ReadPipelineStart`` and ``variable_components=[Microscopy.Channel]``, but omit ``FunctionStep.source_bindings``. The step inherits the complete pipeline order, so use ``nuclear_stain.channel_index=0``, ``cell_body.channel_index=1``, and @@ -210,7 +210,7 @@ step that selects MAP2 is valid evidence for that route. None of these cases changes the zero-based positions of the selected callable stack. Step source subsets and reordering select or re-enter original declared sources, -especially at ``PIPELINE_START`` or on a branch that deliberately returns to the +especially at the pipeline start or on a branch that deliberately returns to the source universe. They do not reinterpret a previous step's output. Stitching, Z projection, channel projection, or filtering may change that output's slice count and axes; downstream code must use the current artifact provenance plus @@ -222,7 +222,7 @@ Plan image stacks and source-bound labels before authoring Naming two sources does not make both available in every invocation. Before authoring a multi-image measurement, separate its current image input from its -typed artifact inputs. ``input_source=PIPELINE_START`` returns to acquisition +typed artifact inputs. ``input_source=ReadPipelineStart`` returns to acquisition images; it does not remove the step's inherited variable-component scope. When the invocation needs an assembled channel stack, declare ``variable_components=[Microscopy.Channel]`` with the intended step bindings/order. A @@ -296,7 +296,7 @@ source-binding semantics; do not pass that concrete class to this lazy field. .. code-block:: python - from openhcs.constants.input_source import InputSource + from openhcs.constants.input_source import ReadPipelineStart from openhcs.core.config import ( LazyProcessingConfig, LazySourceBindingsConfig, @@ -305,10 +305,10 @@ source-binding semantics; do not pass that concrete class to this lazy field. ) from openhcs.core.source_bindings import ( NamedSourceBinding, + SourceFileName, SourceFilterClause, - SourceFilterMatchType, - SourceFilterSubject, SourceSelector, + TextEquals, ) from openhcs.core.steps.function_step import FunctionStep from openhcs.processing.backends.processors.numpy_processor import ( @@ -322,8 +322,8 @@ source-binding semantics; do not pass that concrete class to this lazy field. selector=SourceSelector( filters=( SourceFilterClause( - subject=SourceFilterSubject.FILE, - match_type=SourceFilterMatchType.EQUALS, + subject=SourceFileName, + match_type=TextEquals, value=filename, ), ), @@ -375,7 +375,7 @@ source-binding semantics; do not pass that concrete class to this lazy field. name="Normalize DNA", func=stack_percentile_normalize, processing_config=LazyProcessingConfig( - input_source=InputSource.PIPELINE_START, + input_source=ReadPipelineStart, ), source_bindings=LazyStepSourceBindingsConfig( enabled=True, diff --git a/docs/source/guides/complete_examples.rst b/docs/source/guides/complete_examples.rst index 9efddbdcf3..ff5ce6eede 100644 --- a/docs/source/guides/complete_examples.rst +++ b/docs/source/guides/complete_examples.rst @@ -58,8 +58,8 @@ desktop application, which uses the same declarations and compiler. Typed native presets -------------------- -The current native MFD variants are declared once by ``MfdPresetKey`` and -materialized by ``build_mfd_preset`` in +The current native MFD variants are declared once as ``MfdPresetDefinition`` +values and materialized by ``build_mfd_preset`` in ``openhcs.processing.presets.mfd_specs``. The four corresponding ``10x_mfd_*.py`` modules are thin wrappers over that owner. Consult :doc:`example_corpus_map` for their source index. Crop presets resolve diff --git a/docs/source/guides/example_corpus_map.rst b/docs/source/guides/example_corpus_map.rst index 34423f3d20..2a62f5dba9 100644 --- a/docs/source/guides/example_corpus_map.rst +++ b/docs/source/guides/example_corpus_map.rst @@ -62,8 +62,8 @@ Native OpenHCS Examples ----------------------- The checked-in MFD preset authority is -``openhcs/processing/presets/mfd_specs.py``. It owns the typed preset keys, -variant declarations, shared step templates, materializers, and +``openhcs/processing/presets/mfd_specs.py``. It owns the preset +definitions, shared step templates, materializers, and ``build_mfd_preset``. These four small modules are current materialization wrappers over that authority: From ae618be9d5ccb7178f264da2b70f460c0e9f8751 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 16:03:45 -0400 Subject: [PATCH 73/84] B6: black after merging main (K2, K3, N1, E1) Co-Authored-By: Claude Opus 5.5 --- benchmark/openhcs_measured_run.py | 5 +---- openhcs/agent/dto/session.py | 4 +++- openhcs/core/steps/function_runtime.py | 7 ++----- openhcs/pyqt_gui/main.py | 8 ++------ tests/integration/test_agent_observation_export_zmq.py | 9 ++------- 5 files changed, 10 insertions(+), 23 deletions(-) diff --git a/benchmark/openhcs_measured_run.py b/benchmark/openhcs_measured_run.py index e2d495fe99..b33e74ba6d 100644 --- a/benchmark/openhcs_measured_run.py +++ b/benchmark/openhcs_measured_run.py @@ -381,10 +381,7 @@ def retain_measured_openhcs_completion( "OpenHCS ZMQ execution completed without writing runtime observation " f"export: {observation_export_path}" ) - if ( - auxiliary_params.runtime_observation_export_scope - is ExportRuntimeOutcomes - ): + if auxiliary_params.runtime_observation_export_scope is ExportRuntimeOutcomes: observation_export = ZMQRuntimeExecutionOutcomeExport.read( observation_export_path ) diff --git a/openhcs/agent/dto/session.py b/openhcs/agent/dto/session.py index 37bb36388f..65145850e8 100644 --- a/openhcs/agent/dto/session.py +++ b/openhcs/agent/dto/session.py @@ -27,7 +27,9 @@ It keeps the MCP schema's published name and description for the field. """ -RuntimeObservationExportScopeName.__doc__ = ZMQRuntimeObservationExportScope.__doc__.splitlines()[0] +RuntimeObservationExportScopeName.__doc__ = ( + ZMQRuntimeObservationExportScope.__doc__.splitlines()[0] +) RuntimeObservationExportScopeName.scope = property( lambda member: ZMQRuntimeObservationExportScope.named(member.value) ) diff --git a/openhcs/core/steps/function_runtime.py b/openhcs/core/steps/function_runtime.py index 81cc3160de..c7855a0d55 100644 --- a/openhcs/core/steps/function_runtime.py +++ b/openhcs/core/steps/function_runtime.py @@ -906,9 +906,7 @@ def _filter_matching_files_for_source_bindings( ) -> list[str]: """Constrain the loaded main stack to declared image source bindings.""" - if ( - isinstance(self.execution_plan.main_input_dependency, StepOutputInput) - ): + if isinstance(self.execution_plan.main_input_dependency, StepOutputInput): return matching_files source_binding_plan = self.main_flow_source_binding_plan @@ -1781,8 +1779,7 @@ def loaded_source_artifact_payload( if ( binding is None or self.invocation.key.position != 0 - or not self.group_data.execution_plan.main_input_dependency - .uses_pipeline_start_anchors() + or not self.group_data.execution_plan.main_input_dependency.uses_pipeline_start_anchors() or binding.origin is not FromPipelineStart or request.context is not self.group_data.context ): diff --git a/openhcs/pyqt_gui/main.py b/openhcs/pyqt_gui/main.py index f21ec584e1..a9acc0b179 100644 --- a/openhcs/pyqt_gui/main.py +++ b/openhcs/pyqt_gui/main.py @@ -1424,9 +1424,7 @@ def restart_session(self) -> None: def check_for_updates(self) -> None: """Start an explicit, asynchronous stable-release check.""" - if not self.desktop_update_service.check_for_updates( - ExplicitUpdateCheck - ): + if not self.desktop_update_service.check_for_updates(ExplicitUpdateCheck): return self.check_for_updates_action.setEnabled(False) self.status_message.emit("Checking for OpenHCS updates…") @@ -1436,9 +1434,7 @@ def _check_for_updates_on_startup(self) -> None: if not self.runtime_context.ui_config.check_for_updates_on_startup: return - if self.desktop_update_service.check_for_updates( - StartupUpdateCheck - ): + if self.desktop_update_service.check_for_updates(StartupUpdateCheck): self.check_for_updates_action.setEnabled(False) def _on_update_check_completed( diff --git a/tests/integration/test_agent_observation_export_zmq.py b/tests/integration/test_agent_observation_export_zmq.py index 2449bfad56..784431bc3c 100644 --- a/tests/integration/test_agent_observation_export_zmq.py +++ b/tests/integration/test_agent_observation_export_zmq.py @@ -353,10 +353,7 @@ def test_ordinary_execution_can_export_outcomes_without_value_observation( assert exported.server_environment is not None assert exported.server_environment.python_executable assert completed.observation is None - assert ( - completed.receipt.observation_export_scope - is ExportRuntimeOutcomes - ) + assert completed.receipt.observation_export_scope is ExportRuntimeOutcomes assert { BenchmarkPhase.COMPILE_OPENHCS, BenchmarkPhase.EXECUTE_OPENHCS, @@ -404,9 +401,7 @@ def test_measured_repetitions_share_one_owned_ordinary_server(tmp_path: Path) -> runtime_observation_export_path=( evidence_dir / "observation.pkl.gz" ), - runtime_observation_export_scope=( - ExportRuntimeOutcomes - ), + runtime_observation_export_scope=(ExportRuntimeOutcomes), ) ) completed, _ = execute_measured_openhcs_pipeline_on_client( From 3e089a3f26a6943430c92af0377c4e8d2a118f39 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 16:05:01 -0400 Subject: [PATCH 74/84] B6: re-pin authorities reformatted after the merge (AST-equal) Co-Authored-By: Claude Opus 5.5 --- docs/audits/architecture_a_m.json | 9 +++-- docs/audits/architecture_n_z_development.json | 15 +++++--- docs/audits/user_docs.json | 36 ++++++++++++------- 3 files changed, 40 insertions(+), 20 deletions(-) diff --git a/docs/audits/architecture_a_m.json b/docs/audits/architecture_a_m.json index 005b416ed2..57fb7b1239 100644 --- a/docs/audits/architecture_a_m.json +++ b/docs/audits/architecture_a_m.json @@ -1325,7 +1325,7 @@ }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "2d10806e310cb0d47ee7e47b879a7a95b28dcf5c688d9ce2a54b81ae1a041c47", + "sha256": "817acfd0e44060440d57a2ada0ba75545dfa8dde009363143b2b86e831bd1dd7", "role": "Routes the desktop analysis action through ExperimentalAnalysisEngine.run_directory with the default configuration." }, { @@ -1339,6 +1339,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), openhcs/pyqt_gui/main.py changed by black formatting only (AST-equal) after docs/source/architecture/experimental_analysis_system.rst was reviewed; no claim changes.", "On 2026-10-11 (B6), re-read docs/source/architecture/experimental_analysis_system.rst in full and compared it against the changed authorities: unified_analysis_engine.py, result_formats.py and analysis.py (E1: ExperimentalAnalysisScope deleted, formats resolved by ExperimentalResultFormatStrategy.for_scope over the scope_name registry) and openhcs/pyqt_gui/main.py (selection and update-check classes); replaced the ExperimentalAnalysisScope sentence with the scope_name selection, and confirmed NormalizationMethod, ExperimentalAnalysisConfig and the desktop ExperimentalAnalysisEngine call are unchanged.", "On 2026-10-11 (B6), re-read docs/source/architecture/experimental_analysis_system.rst in full and compared it against the changed authority openhcs/pyqt_gui/main.py (U3 moved dock layout, shortcuts and time-travel navigation to pyqt-reactive); the desktop analysis action still constructs ExperimentalAnalysisEngine() with its default configuration and calls run_directory, and ExperimentalAnalysisConfig and NormalizationMethod are still declared in openhcs/domains/microscopy/config.py; no claim changed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/experimental_analysis_system.rst against B6's later changes to openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py, openhcs/processing/backends/experimental_analysis/result_formats.py, openhcs/processing/backends/experimental_analysis/analysis.py, openhcs/pyqt_gui/main.py, tests/unit/test_experimental_analysis_engine.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", @@ -2086,7 +2087,7 @@ }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "2d10806e310cb0d47ee7e47b879a7a95b28dcf5c688d9ce2a54b81ae1a041c47", + "sha256": "817acfd0e44060440d57a2ada0ba75545dfa8dde009363143b2b86e831bd1dd7", "role": "_apply_list_preview_config applies list_previews to Plate Manager and Pipeline Editor at startup and on UI config change." }, { @@ -2128,6 +2129,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), openhcs/pyqt_gui/main.py changed by black formatting only (AST-equal) after docs/source/architecture/list_item_preview_system.rst was reviewed; no claim changes.", "On 2026-10-11 (B6), re-read docs/source/architecture/list_item_preview_system.rst in full and compared it against the changed authorities: openhcs/core/config.py (E1: MaterializationBackend and InputSource families; PartitionFilterMode subclasses and preview_prefix unchanged), openhcs/pyqt_gui/services/reactor_providers.py (OrchestratorCreated), openhcs/pyqt_gui/config.py (log-level helper, AgentRuntimePlatform rename; UIConfig.list_previews unchanged) and openhcs/pyqt_gui/main.py (selection and update-check classes); the formatter, shared-policy and wrapping claims still hold and the pyqt-reactive target page still exists.", "On 2026-10-11 (B6), re-read docs/source/architecture/list_item_preview_system.rst in full and compared it against the page change (PartitionFilterMode members to subclasses) and the changed authorities: openhcs/core/config.py (PartitionFilterMode subclasses declare preview_prefix), openhcs/pyqt_gui/services/reactor_providers.py (still registers the one PartitionFilterConfig preview formatter; code documents now use pycodify generate_python_source) and openhcs/pyqt_gui/main.py (U3 moves; still applies UIConfig.list_previews to each manager via set_preview_config); every claim holds.", "On 2026-10-11 (B6 final), compared docs/source/architecture/list_item_preview_system.rst against B6's later changes to openhcs/core/config.py, openhcs/pyqt_gui/services/reactor_providers.py, openhcs/pyqt_gui/main.py, tests/unit/pyqt_gui/test_main_config_propagation.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", @@ -2433,7 +2435,7 @@ }, { "path": "benchmark/openhcs_measured_run.py", - "sha256": "cf05a4e08f0a24a68b9d11fa06f1ae362f81de1a959fffcfa02437557acc026d", + "sha256": "22bebf055ec452170aa59e75468760094a42c4a4688a132b383bdef33fa3a11a", "role": "Wraps the ordinary compile-and-execute submission with observation validation, phase timing from completed server records, endpoint provenance, and the completed-run receipt." }, { @@ -2513,6 +2515,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), benchmark/openhcs_measured_run.py changed by black formatting only (AST-equal) after docs/source/architecture/measurement_equivalence_system.rst was reviewed; no claim changes.", "On 2026-10-11 (B6), re-read docs/source/architecture/measurement_equivalence_system.rst in full and compared it against the changed authorities: openhcs/core/equivalence/keys.py, measurement_features.py and relationships.py (K3: row and fact projection and plane alignment moved to benchmark/equivalence, aggregate-feature semantics stay), openhcs/core/measurement_dialect.py (SourceNameEncoding family; row qualifiers in openhcs/core/measurement_row_qualifiers.py), benchmark/adapters/openhcs.py (shape_rows_on_their_labels on reference snapshots, MaterializeToDisk), benchmark/openhcs_measured_run.py, benchmark/control.py, benchmark/contracts/measured_run_receipt.py and openhcs/runtime/zmq_execution_observation.py (E1: export scope and axis status classes) and benchmark/cellprofiler_reference_exports.py (K2 import path); corrected the product-package module list, added the native AreaShape row placement, and confirmed that no openhcs module imports benchmark and that the dialect, policy, receipt, timing and Official30 claims still hold.", "On 2026-10-11 (B6 final), compared docs/source/architecture/measurement_equivalence_system.rst against B6's later changes to tests/integration/test_cellprofiler_official30_zmq.py: the session decides in-place initialization by an actual file-creation attempt that leaves nothing behind, its debug-session methods moved into an inherited mixin, the measured CLI treats Windows Ctrl+Break like Ctrl+C, and the Official30 viewer test reads typed payload summaries. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/architecture/measurement_equivalence_system.rst against B6's later changes to openhcs/core/equivalence/keys.py, openhcs/core/equivalence/measurement_features.py, openhcs/core/measurement_dialect.py, benchmark/control_service.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", diff --git a/docs/audits/architecture_n_z_development.json b/docs/audits/architecture_n_z_development.json index 13d3550a27..55f37a569f 100644 --- a/docs/audits/architecture_n_z_development.json +++ b/docs/audits/architecture_n_z_development.json @@ -1944,7 +1944,7 @@ }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "2d10806e310cb0d47ee7e47b879a7a95b28dcf5c688d9ce2a54b81ae1a041c47", + "sha256": "817acfd0e44060440d57a2ada0ba75545dfa8dde009363143b2b86e831bd1dd7", "role": "PyQt desktop main window composition." }, { @@ -1958,6 +1958,7 @@ ], "disposition": "redirect", "validation": [ + "On 2026-10-11 (B6 final), openhcs/pyqt_gui/main.py changed by black formatting only (AST-equal) after docs/source/architecture/tui_system.rst was reviewed; no claim changes.", "On 2026-10-11 (B6), re-read docs/source/architecture/tui_system.rst in full and compared it against the changed authorities: pyproject.toml, openhcs/pyqt_gui/main.py; pyproject.toml declares no Textual dependency, main.py only swapped selection and update-check enum members for class families, and docs/source/user_guide/index.rst and docs/archive exist.", "On 2026-10-11 (B6), re-read docs/source/architecture/tui_system.rst in full and compared it against the changed authorities: pyproject.toml, openhcs/pyqt_gui/main.py, tests/unit/pyqt_gui/test_ui_config_lifecycle.py; pyproject.toml still has no Textual dependency and the openhcs-gui gui-script points at openhcs.pyqt_gui.__main__:main; main.py changes are U3 moves of generic Qt machinery; the lifecycle test still guards that openhcs/textual_tui does not return (its shortcut tests moved to pyqt-reactive); docs/archive still exists. No correction needed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/tui_system.rst against B6's later changes to openhcs/pyqt_gui/main.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", @@ -2060,7 +2061,7 @@ }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "2d10806e310cb0d47ee7e47b879a7a95b28dcf5c688d9ce2a54b81ae1a041c47", + "sha256": "817acfd0e44060440d57a2ada0ba75545dfa8dde009363143b2b86e831bd1dd7", "role": "OpenHCSMainWindow.set_ui_config consumer reconciliation and rollback, and preview-policy propagation to both managers." }, { @@ -2137,6 +2138,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), openhcs/pyqt_gui/main.py changed by black formatting only (AST-equal) after docs/source/architecture/ui_services_architecture.rst was reviewed; no claim changes.", "On 2026-10-11 (B6), re-read docs/source/architecture/ui_services_architecture.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/config.py, openhcs/pyqt_gui/app.py, openhcs/desktop/update.py, openhcs/pyqt_gui/main.py, openhcs/pyqt_gui/widgets/plate_manager.py; E1 turned the startup-readiness state, the update-check origin (ExplicitUpdateCheck/StartupUpdateCheck), the selection scope (SelectedItems/AllItems) and OrchestratorState into class families; the page names none of them and its statements on OpenHCSPyQtApp shutdown, UIConfig fields, set_ui_config, the update-check preference and Plate Manager's session rendering are unaffected; every named class was grepped and exists; no page change.", "On 2026-10-11 (B6), re-read docs/source/architecture/ui_services_architecture.rst in full and compared it against the changed authorities: pyqt_gui/services/window_handlers.py (ScopeKey separator, OpenHCSConfigWindow over pyqt-reactive ConfigWindow, session event connection), tests/unit/pyqt_gui/test_ui_config_lifecycle.py (shortcut tests removed), pyqt_gui/main.py (pyqt-reactive ShortcutLifecycle, DockLayoutStore/DockPane, TimeTravelCommands/TimeTravelNavigator) and widgets/plate_manager.py (GuiEventBusBroadcaster removed, PathCacheKey and external-editor policy from pyqt-reactive); corrected the shortcut-lifecycle and configuration-window transaction owners and checked the save transaction order in pyqt-reactive ConfigWindow.save_config against the page's rollback description.", "On 2026-10-11 (B6 final), compared docs/source/architecture/ui_services_architecture.rst against B6's later changes to openhcs/pyqt_gui/services/ui_bridge_session_actions.py, openhcs/pyqt_gui/main.py, tests/unit/pyqt_gui/test_main_config_propagation.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", @@ -2492,7 +2494,7 @@ }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "2d10806e310cb0d47ee7e47b879a7a95b28dcf5c688d9ce2a54b81ae1a041c47", + "sha256": "817acfd0e44060440d57a2ada0ba75545dfa8dde009363143b2b86e831bd1dd7", "role": "Projects the configured endpoint snapshot into its dedicated right-hand indicator, and routes endpoint termination to the client lifecycle owner." }, { @@ -2529,6 +2531,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), openhcs/pyqt_gui/main.py changed by black formatting only (AST-equal) after docs/source/architecture/zmq_server_browser_system.rst was reviewed; no claim changes.", "On 2026-10-11 (B6), re-read docs/source/architecture/zmq_server_browser_system.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/main.py; the diff swaps selection and update-check enum members for class families and touches no endpoint-snapshot, status-bar or server-browser code; the listed module paths exist and every named class was grepped; no page change.", "On 2026-10-11 (B6), re-read docs/source/architecture/zmq_server_browser_system.rst in full and compared it against the changed authorities: pyqt_gui/main.py (shortcut, dock and time-travel machinery moved to pyqt-reactive; endpoint indicator projection unchanged) and core/progress/runtime_tree.py (K6 ProgressStream classes replace channel helpers); verified every named class (ZMQServerBrowserWidgetABC, EndpointObservationSnapshot, RuntimeExecutionTopology, RuntimeTreeProjectionBuilder, ProgressTreeBuilder, ExecutionProgressProjection, ServerRowPresenter, LiveServerTreeSync, EndpointShutdownService, ZMQClientService, ProgressUIConfig, EndpointClientSession), the primary module paths, and mean/explicit percent node policies on subject/worker vs partition/step/compilation nodes; no claim needed correction.", "On 2026-10-11 (B6 final), compared docs/source/architecture/zmq_server_browser_system.rst against B6's later changes to openhcs/pyqt_gui/main.py, openhcs/core/progress/runtime_tree.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", @@ -3287,7 +3290,7 @@ }, { "path": "openhcs/agent/dto/session.py", - "sha256": "730cbcfcda6c7d52a1c6e1d142ec2e7d721f32c51bb78c845a8a03201da70ba0", + "sha256": "5a261a14d3cd2880ecc14c2fc83f6e8664655799789f91a79e81f1fd40a8656d", "role": "Typed run request fields runtime_observation_export_path, runtime_observation_export_scope and submit_timeout_ms, and event_sequence results." }, { @@ -3354,6 +3357,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), openhcs/agent/dto/session.py changed by black formatting only (AST-equal) after docs/source/development/mcp_development.rst was reviewed; no claim changes.", "On 2026-10-11 (B6), re-read docs/source/development/mcp_development.rst in full and compared it against the changed authorities: openhcs/mcp/dev_client_core.py, openhcs/mcp/dev_client_commands/ui.py, openhcs/agent/services/ui_bridge_service.py, openhcs/agent/dto/session.py, openhcs/authoring/session/session.py, openhcs/mcp/server.py, openhcs/runtime/zmq_execution_client.py, openhcs/pyqt_gui/services/ui_bridge_server.py and the two test modules (E1: poll status, orchestrator state, field filter, selection scope and export scope became class families); the page names no enum member; the selected-workflow wait policy, the session-operation tool names, the submit_timeout_ms budget, and the runtime-observation export checks (writable root, no existing path, `values`/`outcomes` spellings, scope.check_export_path before submission) still hold in the current files, so nothing was changed.", "On 2026-10-11 (B6), re-read docs/source/development/mcp_development.rst in full and compared it against the changed authorities: openhcs/authoring/session/session.py, openhcs/runtime/zmq_execution_client.py and tests/unit/pyqt_gui/test_ui_agent_bridge.py (K6: debug commands, boundaries and progress phases/statuses became class families; debug control requests send request.to_dict() directly; U3: DockPane and ConfigCodeDocumentDriver moved to pyqt-reactive). The page names none of those types; rechecked the openhcs-mcp-dev entry point, every dev-client command and option it shows, the SessionOperation operation_ids behind the six named tools, openhcs_session_events/openhcs_session_datasets, execution_submission_timeout_ms on OpenHCSZMQConfig's zmqruntime base, UIConfigCacheEnvironment, the bridge/cache environment variables and OPENHCS_MCP_VERBOSE; no claim changes.", "On 2026-10-11 (B6 final), compared docs/source/development/mcp_development.rst against B6's later changes to openhcs/authoring/session/session.py: the writability probe makes one exclusive create attempt, the installed demo reports ViewerType.value, the unit session pre-creates its QApplication, and the GUI journey waits for each batch to end. No claim this page makes changes.", @@ -3960,7 +3964,7 @@ }, { "path": "openhcs/core/steps/function_runtime.py", - "sha256": "83d9eca90f2dca33ff8074bb768a24fadc5dfb1881855274eab9203b4cc41770", + "sha256": "0b5b3b025811df3b8075e51d605474b8be1e4b7f6fe873da82882e0232bf7dc2", "role": "Runtime conversion and invocation inside the compiled execution-framework device scope." }, { @@ -3977,6 +3981,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), openhcs/core/steps/function_runtime.py changed by black formatting only (AST-equal) after docs/source/development/pyclesperanto_simple_implementation.rst was reviewed; no claim changes.", "On 2026-10-11 (B6), re-read docs/source/development/pyclesperanto_simple_implementation.rst in full and compared it against the changed authorities: openhcs/processing/backends/processors/pyclesperanto_processor.py and openhcs/core/callable_contract.py (K2: artifact imports moved to openhcs.core.artifacts.specs/kinds/plans) and the current openhcs/core/steps/function_runtime.py; the example imports only openhcs.core.memory and openhcs.core.processing_contracts, both unchanged; Pure3DContract, PyclesperantoRegistry's runtime-owned output_image/output_image_matrix/device parameters and the runtime execution_device_scope still exist, so nothing was changed.", "On 2026-10-11 (B6), re-read docs/source/development/pyclesperanto_simple_implementation.rst in full and compared it against the changed authorities: openhcs/core/processing_contracts.py and openhcs/core/callable_contract.py (K5: CallableMetadata replaced by CallableDeclaration, which CallableContract inherits), openhcs/core/steps/function_runtime.py (K6 debug boundary classes; main-flow source-binding plan no longer None) and tests/unit/test_unified_registry_runtime_payloads.py. The page names none of the changed types; rechecked the pyclesperanto decorator re-exported from openhcs.core.memory, Pure3DContract, and PyclesperantoRegistry's runtime-owned output_image, output_image_matrix and device parameters; no claim changes.", "On 2026-10-11 (B6 final), compared docs/source/development/pyclesperanto_simple_implementation.rst against B6's later changes to openhcs/processing/backends/processors/pyclesperanto_processor.py, openhcs/core/memory/decorators.py, openhcs/core/processing_contracts.py, openhcs/core/callable_contract.py, openhcs/core/steps/function_runtime.py, tests/unit/test_unified_registry_runtime_payloads.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", diff --git a/docs/audits/user_docs.json b/docs/audits/user_docs.json index f47e81b85d..73e10cbb64 100644 --- a/docs/audits/user_docs.json +++ b/docs/audits/user_docs.json @@ -1258,7 +1258,7 @@ "authority": [ { "path": "openhcs/pyqt_gui/main.py", - "sha256": "2d10806e310cb0d47ee7e47b879a7a95b28dcf5c688d9ce2a54b81ae1a041c47", + "sha256": "817acfd0e44060440d57a2ada0ba75545dfa8dde009363143b2b86e831bd1dd7", "role": "Dock layout (system monitor above, Datasets and ZMQ server manager left, Pipeline Editor right), View/Tools/Help menus, and preview-policy application to both panes." }, { @@ -1380,6 +1380,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), openhcs/pyqt_gui/main.py changed by black formatting only (AST-equal) after docs/source/guide_for_biologists/basic_interface.rst was reviewed; no claim changes.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/basic_interface.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/main.py, widgets/plate_manager.py, widgets/pipeline_editor.py, services/ui_bridge_windows.py, openhcs/pyqt_gui/config.py, openhcs/agent/services/ui_bridge_service.py, openhcs/authoring/session/views.py and the three test modules (E1: selection scope, orchestrator state, action status and update-check origin became class families; the error-dialog marker became a boolean); the page describes user-visible behaviour and names no changed type; all-plates versus selected code documents, Connect/Compile/Run/Stop control behaviour and the pane identities are unchanged, so nothing was changed.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/basic_interface.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/main.py, widgets/plate_manager.py, widgets/pipeline_editor.py, shared/services/pipeline_editor_workflows.py, services/ui_bridge_windows.py, openhcs/core/pipeline_document.py and their tests (U3: DockPane, DockLayoutStore, ShortcutLifecycle, time-travel navigation, path cache and the external-editor switch moved to pyqt-reactive, GlobalEventBus broadcasts replaced by session events, ScopeKey scope ids; L6: pipeline documents render through pycodify generate_python_source; K6 debug command classes). The page describes behaviour, not those classes; rechecked the dock arrangement (system monitor above, Datasets and ZMQ server manager stacked left, Pipeline Editor right), the plate_manager window identity, list_previews on UIConfig, and that PipelineDocument still requires pipeline_steps; no claim changes.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/basic_interface.rst against B6's later changes to openhcs/authoring/session/views.py: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", @@ -1849,7 +1850,7 @@ "authority": [ { "path": "openhcs/pyqt_gui/main.py", - "sha256": "2d10806e310cb0d47ee7e47b879a7a95b28dcf5c688d9ce2a54b81ae1a041c47", + "sha256": "817acfd0e44060440d57a2ada0ba75545dfa8dde009363143b2b86e831bd1dd7", "role": "Composes the main desktop window, menus, task windows, and registered UI services." }, { @@ -1866,6 +1867,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), openhcs/pyqt_gui/main.py changed by black formatting only (AST-equal) after docs/source/guide_for_biologists/index.rst was reviewed; no claim changes.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/index.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/main.py and openhcs/demo/synthetic_plate_pipeline.py; both diffs are E1 enum-to-family renames (SelectedItems/AllItems, update-check origins, ReadPipelineStart, PadToMaxSize); the demo pipeline still declares eight steps and the page names no renamed symbol, so no correction was needed.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/index.rst in full and compared it against the changed authority openhcs/pyqt_gui/main.py (U3: dock layout, shortcut lifecycle and time-travel navigation now come from pyqt-reactive); the page only routes readers and names no main-window mechanism, so no correction was needed.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/index.rst against B6's later changes to openhcs/pyqt_gui/main.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", @@ -2038,7 +2040,7 @@ }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "2d10806e310cb0d47ee7e47b879a7a95b28dcf5c688d9ce2a54b81ae1a041c47", + "sha256": "817acfd0e44060440d57a2ada0ba75545dfa8dde009363143b2b86e831bd1dd7", "role": "Places Generate Synthetic Plate in the View menu." }, { @@ -2050,6 +2052,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), openhcs/pyqt_gui/main.py changed by black formatting only (AST-equal) after docs/source/guide_for_biologists/intro_stitching.rst was reviewed; no claim changes.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/intro_stitching.rst in full and compared it against the changed authorities: openhcs/demo/synthetic_plate_pipeline.py and openhcs/pyqt_gui/main.py; the supplied pipeline still has eight steps running from Image Enhancement Processing to Cell Counting (only its InputSource and variable-size spellings became ReadPipelineStart and PadToMaxSize) and main.py still places Generate Synthetic Plate under View; no correction was needed.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/intro_stitching.rst in full and compared it against the changed authority openhcs/pyqt_gui/main.py (U3 dock, shortcut and time-travel moves to pyqt-reactive); the View menu still holds the Generate Synthetic Plate action, so no correction was needed.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/intro_stitching.rst against B6's later changes to openhcs/authoring/session/operations/datasets.py, openhcs/pyqt_gui/main.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", @@ -2117,7 +2120,7 @@ }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "2d10806e310cb0d47ee7e47b879a7a95b28dcf5c688d9ce2a54b81ae1a041c47", + "sha256": "817acfd0e44060440d57a2ada0ba75545dfa8dde009363143b2b86e831bd1dd7", "role": "Wires endpoint compatibility to the version-restart workflow and gates the restart operation until initialisation, compilation and execution have finished." }, { @@ -2174,6 +2177,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), openhcs/pyqt_gui/main.py changed by black formatting only (AST-equal) after docs/source/guide_for_biologists/troubleshooting_FAQ.rst was reviewed; no claim changes.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/troubleshooting_FAQ.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compiler.py, openhcs/utils/environment.py, openhcs/pyqt_gui/launch.py, openhcs/runtime/viewer_protocol.py, openhcs/runtime/zmq_execution_client.py, openhcs/pyqt_gui/main.py and openhcs/pyqt_gui/services/ui_bridge_server.py; confirmed --no-gpu, OPENHCS_CPU_ONLY, the .startup.jsonl journal, materialize_runtime_artifacts and submit_timeout_ms still exist; the diffs are enum-to-family renames (orchestrator states, settle phases, viewer lifecycle, update-check origins), the AgentRuntimePlatform rename and the removed QtPlatformSystem enum, none of which the page names, so no correction was needed.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/troubleshooting_FAQ.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compiler.py (ScopeKey scope ids, K6 progress classes), openhcs/runtime/zmq_execution_client.py (K6: debug requests serialize themselves, command types are DebugCommandType classes; the .startup.jsonl journal beside the server log is unchanged) and openhcs/pyqt_gui/main.py (U3 moves to pyqt-reactive); the restart-and-restore action, OPENHCS_UI_CONFIG_CACHE_FILE, submit_timeout_ms and materialize_runtime_artifacts still exist, so no correction was needed.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/troubleshooting_FAQ.rst against B6's later changes to openhcs/authoring/session/operations/datasets.py, openhcs/core/pipeline/compiler.py, openhcs/runtime/viewer_protocol.py, openhcs/runtime/zmq_execution_client.py, openhcs/pyqt_gui/main.py, tests/unit/desktop/test_desktop_restart.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", @@ -2524,7 +2528,7 @@ }, { "path": "openhcs/core/steps/function_runtime.py", - "sha256": "83d9eca90f2dca33ff8074bb768a24fadc5dfb1881855274eab9203b4cc41770", + "sha256": "0b5b3b025811df3b8075e51d605474b8be1e4b7f6fe873da82882e0232bf7dc2", "role": "Enters the invocation's execution device scope around the function body." }, { @@ -2551,6 +2555,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), openhcs/core/steps/function_runtime.py changed by black formatting only (AST-equal) after docs/source/guides/memory_type_integration.rst was reviewed; no claim changes.", "On 2026-10-11 (B6), re-read docs/source/guides/memory_type_integration.rst in full and compared it against the changed authorities: openhcs/constants/constants.py, openhcs/core/callable_contract.py, openhcs/core/compiled_step_plan.py, openhcs/core/steps/function_runtime.py, openhcs/core/orchestrator/worker_execution.py and openhcs/core/config.py; constants.py still re-exports arraybridge's MemoryType (its OrchestratorState enum became a class family, which the page does not mention), CallableContract still reads MemoryContractAttribute, FrameworkDeviceAssignment is still on CompiledStepPlan, function_runtime.py still enters execution_device_scope() around the function body, and variable_components and group_by remain on ProcessingConfig; no correction was needed.", "On 2026-10-11 (B6), re-read docs/source/guides/memory_type_integration.rst in full and compared it against the changed authorities: openhcs/core/processing_contracts.py and openhcs/core/callable_contract.py (K5: CallableMetadata became CallableDeclaration, which CallableContract inherits and which reads the input, output and execution keys through MemoryContractAttribute), openhcs/core/compiled_step_plan.py (empty-plan defaults), openhcs/core/steps/function_runtime.py (execution_device_scope still wraps the function body), openhcs/core/orchestrator/worker_execution.py (K6 progress classes; compiled GPU footprint cleanup unchanged), openhcs/core/config.py (variable_components and group_by still on the processing config) and tests/unit/test_framework_execution_scope.py; confirmed openhcs.constants.MemoryType and openhcs.core.memory.MemoryType are arraybridge.MemoryType and that MemoryType.subprocess_environment exists; no correction was needed.", "On 2026-10-11 (B6 final), compared docs/source/guides/memory_type_integration.rst against B6's later changes to openhcs/core/memory/decorators.py, openhcs/core/processing_contracts.py, openhcs/core/callable_contract.py, openhcs/core/compiled_step_plan.py, openhcs/core/steps/function_runtime.py, openhcs/core/orchestrator/worker_execution.py, openhcs/core/config.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", @@ -3117,7 +3122,7 @@ }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "2d10806e310cb0d47ee7e47b879a7a95b28dcf5c688d9ce2a54b81ae1a041c47", + "sha256": "817acfd0e44060440d57a2ada0ba75545dfa8dde009363143b2b86e831bd1dd7", "role": "Composes the main desktop window, menus, task windows, and registered UI services." }, { @@ -3139,6 +3144,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), openhcs/pyqt_gui/main.py changed by black formatting only (AST-equal) after docs/source/index.rst was reviewed; no claim changes.", "On 2026-10-11 (B6), re-read docs/source/index.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/main.py and openhcs/interop/cellprofiler/pipeline_import.py; both diffs are enum-to-family renames plus the artifacts-package import paths; the landing page routes readers and names no changed symbol, so no correction was needed.", "On 2026-10-11 (B6), re-read docs/source/index.rst in full and compared it against the changed authorities: pipeline_document.py (render now via pycodify.generate_python_source, L6; PipelineDocument/Codec unchanged), pyqt_gui/main.py (dock layout, shortcuts and time-travel navigation moved to pyqt-reactive, U3), cellprofiler/pipeline_import.py (SourceBindingsConfig now imported from openhcs.core.config, K4); the landing page names no module, class or window behaviour these touched, and still routes to existing pages; no edit needed.", "On 2026-10-11 (B6 final), compared docs/source/index.rst against B6's later changes to openhcs/pyqt_gui/main.py, openhcs/interop/cellprofiler/pipeline_import.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", @@ -3545,7 +3551,7 @@ }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "2d10806e310cb0d47ee7e47b879a7a95b28dcf5c688d9ce2a54b81ae1a041c47", + "sha256": "817acfd0e44060440d57a2ada0ba75545dfa8dde009363143b2b86e831bd1dd7", "role": "Composes the main desktop window and its stable docked surfaces from the registered identities." }, { @@ -3577,6 +3583,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), openhcs/pyqt_gui/main.py changed by black formatting only (AST-equal) after docs/source/reference/ui_windows.rst was reviewed; no claim changes.", "On 2026-10-11 (B6), re-read docs/source/reference/ui_windows.rst in full and compared it against the changed authorities: openhcs/authoring/session/views.py (dataset rows report state, terminal status and debug phase by family name) and openhcs/pyqt_gui/main.py (SelectedItems/AllItems selection scopes, ExplicitUpdateCheck/StartupUpdateCheck); the view titles that the window identities present and the main-window composition are unchanged, and the page still lists no window itself; no edit needed.", "On 2026-10-11 (B6), re-read docs/source/reference/ui_windows.rst in full and compared it against the changed authorities: pyqt_gui/main.py and main_window_workflows.py (U3 moved dock layout, shortcut lifecycle and time-travel navigation to pyqt-reactive DockLayoutStore/DockPanes, ShortcutLifecycle, TimeTravelNavigator); build_main_window_specs still projects UiStableWindowIdentityDeclaration subclasses into WindowManager specs and the openhcs-ui-window-reference directive still renders the registry, so the page (which keeps no window list of its own) remains accurate; no edit needed.", "On 2026-10-11 (B6 final), compared docs/source/reference/ui_windows.rst against B6's later changes to openhcs/authoring/session/views.py: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", @@ -3604,7 +3611,7 @@ "authority": [ { "path": "openhcs/pyqt_gui/main.py", - "sha256": "2d10806e310cb0d47ee7e47b879a7a95b28dcf5c688d9ce2a54b81ae1a041c47", + "sha256": "817acfd0e44060440d57a2ada0ba75545dfa8dde009363143b2b86e831bd1dd7", "role": "Declares the four Tools > Analysis Consolidation actions and the experimental-analysis directory check." }, { @@ -3631,6 +3638,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), openhcs/pyqt_gui/main.py changed by black formatting only (AST-equal) after docs/source/user_guide/analysis_consolidation.rst was reviewed; no claim changes.", "On 2026-10-11 (B6), re-read docs/source/user_guide/analysis_consolidation.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/main.py (selection-scope and update-check families, E1) and experimental_analysis/unified_analysis_engine.py (result format resolved with for_scope); the four Tools > Analysis Consolidation actions, the config.xlsx and metaxpress_style_summary.csv directory contract and the three output workbooks are unchanged; no edit needed.", "On 2026-10-11 (B6), re-read docs/source/user_guide/analysis_consolidation.rst in full and compared it against the changed authority openhcs/pyqt_gui/main.py (U3: dock layout, shortcuts and time-travel navigation moved to pyqt-reactive); the Tools > Analysis Consolidation submenu still declares the four actions (Consolidate Results Directory, Merge MetaXpress Summaries, Concatenate MetaXpress Summaries (Keep Headers), Run Experimental Analysis) and the experimental-analysis directory dialog; no edit needed.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/analysis_consolidation.rst against B6's later changes to openhcs/pyqt_gui/main.py, openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py, tests/unit/test_experimental_analysis_engine.py, openhcs/domains/microscopy/analysis_consolidation.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", @@ -3932,7 +3940,7 @@ }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "2d10806e310cb0d47ee7e47b879a7a95b28dcf5c688d9ce2a54b81ae1a041c47", + "sha256": "817acfd0e44060440d57a2ada0ba75545dfa8dde009363143b2b86e831bd1dd7", "role": "Declares the Tools > Custom Functions > Manage Functions action." }, { @@ -3989,6 +3997,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), openhcs/pyqt_gui/main.py changed by black formatting only (AST-equal) after docs/source/user_guide/custom_function_management.rst was reviewed; no claim changes.", "On 2026-10-11 (B6), re-read docs/source/user_guide/custom_function_management.rst in full and compared it against the changed authorities: custom_functions/manager.py and runtime_registry.py (CustomFunctionLifetime became PersistedLifetime and EphemeralLifetime classes), openhcs/pyqt_gui/main.py (E1 families) and tests/unit/test_custom_function_lifecycle.py and test_function_catalog_zmq.py (K2 artifacts imports, parameter-source classes); reloading persisted functions still keeps declarations whose lifetime is EphemeralLifetime, CustomFunctionManager.register_from_code(persist=...) is unchanged and the Manage Functions action remains; no edit needed.", "On 2026-10-11 (B6), re-read docs/source/user_guide/custom_function_management.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/main.py (U3 moved dock layout, shortcuts and time-travel navigation to pyqt-reactive; Tools > Custom Functions > Manage Functions unchanged) and tests/unit/test_custom_function_lifecycle.py (function references expose .declaration instead of .metadata, K5); CustomFunctionManager.register_from_code and its package export, and CustomFunctionRuntimeRegistry publishing FunctionMetadata, are unchanged, so the code example and prose remain accurate; no edit needed.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/custom_function_management.rst against B6's later changes to openhcs/processing/custom_functions/manager.py, openhcs/pyqt_gui/main.py, openhcs/mcp/context.py, tests/unit/test_custom_function_lifecycle.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", @@ -4042,7 +4051,7 @@ }, { "path": "openhcs/pyqt_gui/main.py", - "sha256": "2d10806e310cb0d47ee7e47b879a7a95b28dcf5c688d9ce2a54b81ae1a041c47", + "sha256": "817acfd0e44060440d57a2ada0ba75545dfa8dde009363143b2b86e831bd1dd7", "role": "Declares the Tools > Custom Functions > Create New Function action." }, { @@ -4074,6 +4083,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), openhcs/pyqt_gui/main.py changed by black formatting only (AST-equal) after docs/source/user_guide/custom_functions.rst was reviewed; no claim changes.", "On 2026-10-11 (B6), re-read docs/source/user_guide/custom_functions.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/main.py (E1 families), custom_functions/templates.py (commented artifact example imports from the K2 artifacts package), manager.py and runtime_registry.py (lifetime classes) and the two custom-function tests; the page's example imports only openhcs.core.memory.decorators.numpy and Pure2DContract, both unchanged, and the Create New Function action remains; no edit needed.", "On 2026-10-11 (B6), re-read docs/source/user_guide/custom_functions.rst in full and compared it against the changed authorities: processing_contracts.py (signature lookup through CallableDeclaration, K5), pyqt_gui/main.py (U3; Tools > Custom Functions > Create New Function unchanged) and tests/unit/test_custom_function_lifecycle.py (reference.declaration); the example's numpy decorator still accepts contract= with ProcessingContract subclasses (default FlexibleContract), and Pure2DContract, Pure3DContract, VolumetricToSliceContract and FlexibleContract still exist; no edit needed.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/custom_functions.rst against B6's later changes to openhcs/core/memory/decorators.py, openhcs/core/processing_contracts.py, openhcs/pyqt_gui/main.py, openhcs/processing/custom_functions/manager.py, tests/unit/test_custom_function_lifecycle.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", @@ -4171,7 +4181,7 @@ "authority": [ { "path": "openhcs/pyqt_gui/main.py", - "sha256": "2d10806e310cb0d47ee7e47b879a7a95b28dcf5c688d9ce2a54b81ae1a041c47", + "sha256": "817acfd0e44060440d57a2ada0ba75545dfa8dde009363143b2b86e831bd1dd7", "role": "Declares the Tools > Analysis Consolidation > Run Experimental Analysis action and its directory check for the config and results files." }, { @@ -4203,6 +4213,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), openhcs/pyqt_gui/main.py changed by black formatting only (AST-equal) after docs/source/user_guide/experimental_layouts.rst was reviewed; no claim changes.", "On 2026-10-11 (B6), re-read docs/source/user_guide/experimental_layouts.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/main.py (E1 families), experimental_analysis/analysis.py and unified_analysis_engine.py (for_scope) and tests/unit/test_experimental_layout_rows.py; the Run Experimental Analysis action, the directory filenames, the three output workbooks and ExperimentalAnalysisConfig.normalization_method are unchanged; no edit needed.", "On 2026-10-11 (B6), re-read docs/source/user_guide/experimental_layouts.rst in full and compared it against the changed authority openhcs/pyqt_gui/main.py (U3: dock layout, shortcuts and time-travel navigation moved to pyqt-reactive); the Run Experimental Analysis action and its directory dialog are unchanged, and ExperimentalAnalysisConfig.normalization_method still defaults to fold change in openhcs/domains/microscopy/config.py; no edit needed.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/experimental_layouts.rst against B6's later changes to openhcs/pyqt_gui/main.py, openhcs/processing/backends/experimental_analysis/analysis.py, openhcs/processing/backends/experimental_analysis/unified_analysis_engine.py, tests/unit/test_experimental_analysis_engine.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", @@ -4232,7 +4243,7 @@ "authority": [ { "path": "openhcs/pyqt_gui/main.py", - "sha256": "2d10806e310cb0d47ee7e47b879a7a95b28dcf5c688d9ce2a54b81ae1a041c47", + "sha256": "817acfd0e44060440d57a2ada0ba75545dfa8dde009363143b2b86e831bd1dd7", "role": "Composes the PyQt desktop main window that the openhcs command launches." }, { @@ -4259,6 +4270,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), openhcs/pyqt_gui/main.py changed by black formatting only (AST-equal) after docs/source/user_guide/index.rst was reviewed; no claim changes.", "On 2026-10-11 (B6), re-read docs/source/user_guide/index.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/main.py (E1 families), openhcs/core/config.py (MaterializationBackend and InputSource families) and openhcs/core/pipeline/compiler.py (OrchestratorCompiled/OrchestratorCompileFailed state classes, MaterializeToZarr, PrimaryPlaneRole); the declaration model (list[FunctionStep] plus PipelineConfig, nested configs, compilation into typed plans and a bundle) is unchanged and every toctree page exists; no edit needed.", "On 2026-10-11 (B6), re-read docs/source/user_guide/index.rst in full and compared it against the changed authorities: pyqt_gui/main.py (U3 dock/shortcut/time-travel machinery moved to pyqt-reactive), tests/unit/pyqt_gui/test_ui_config_lifecycle.py (shortcut lifecycle tests moved with it; the textual_tui non-recurrence check remains), pipeline_document.py (generate_python_source), core/config.py (PartitionFilterMode family, K4 source-binding classes) and compiler.py (ScopeKey scope ids, K6 progress phase/status classes; still returns CompiledExecutionBundle); the PyQt-only interface, toctree and declaration-model claims remain accurate; no edit needed.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/index.rst against B6's later changes to openhcs/pyqt_gui/main.py, openhcs/core/config.py, openhcs/core/pipeline/compiler.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", From f8062d2cdfcf41a812fb365208e8fc8876b19a68 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 16:07:17 -0400 Subject: [PATCH 75/84] B6: unit-tests job runs file-isolated xdist workers to fit its 45-minute budget The job was cancelled at 45 minutes with 63% of 11,586 tests run (33 minutes for the whole suite on 2026-09-24). Local verification already runs --dist loadfile. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/integration-tests.yml | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.github/workflows/integration-tests.yml b/.github/workflows/integration-tests.yml index e95d694a86..e5801f5027 100644 --- a/.github/workflows/integration-tests.yml +++ b/.github/workflows/integration-tests.yml @@ -233,11 +233,14 @@ jobs: context.doneCurrent() PY + # 11.6k tests no longer fit the 45-minute budget in one process; files + # are isolated per worker (--dist loadfile), as local verification runs. - name: Run unit tests run: >- xvfb-run --auto-servernum --server-args="-screen 0 1920x1080x24 +extension GLX" python scripts/run_installed_tests.py --coverage --repo-root "$GITHUB_WORKSPACE" -- tests/unit tests/core -p no:randomly -vv --tb=short + -n 3 --dist loadfile -o faulthandler_timeout=120 --cov-report= From 2e0deafe6135aaa1e6c8291194716dc8bab8eba9 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 16:07:20 -0400 Subject: [PATCH 76/84] B6: re-pin pages citing the integration workflow Co-Authored-By: Claude Opus 5.5 --- docs/audits/architecture_n_z_development.json | 6 ++++-- docs/audits/user_docs.json | 6 ++++-- 2 files changed, 8 insertions(+), 4 deletions(-) diff --git a/docs/audits/architecture_n_z_development.json b/docs/audits/architecture_n_z_development.json index 55f37a569f..f83464af13 100644 --- a/docs/audits/architecture_n_z_development.json +++ b/docs/audits/architecture_n_z_development.json @@ -3533,7 +3533,7 @@ }, { "path": ".github/workflows/integration-tests.yml", - "sha256": "781bd20677f309152ad4041bd7d62d5aca9ea802bd9a0a201a96ad187bb7b05f", + "sha256": "5b8e9a8e150c07e2a003b1d7ada05711f6a2936ece1833c2a421edcb2802fddd", "role": "Cross-platform source-candidate and published-dependency readiness, installed-wheel live GUI/MCP operation with retained native screenshots, three-stage native-installer workflow proof, signing-helper parse, and staged-update validation matrix." }, { @@ -3628,6 +3628,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/mcp_release.rst against .github/workflows/integration-tests.yml: the unit-tests job now runs three file-isolated xdist workers; no claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/development/mcp_release.rst against B6's later changes to .github/workflows/integration-tests.yml: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/development/mcp_release.rst against B6's later changes to tests/unit/test_mcp_release_workflow.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", "On 2026-10-11 (B6 final), compared docs/source/development/mcp_release.rst against the later B6 changes to tests/unit/test_mcp_release_workflow.py, scripts/smoke_installed_gui.py, tests/unit/test_smoke_installed_gui.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", @@ -3755,13 +3756,14 @@ }, { "path": ".github/workflows/integration-tests.yml", - "sha256": "781bd20677f309152ad4041bd7d62d5aca9ea802bd9a0a201a96ad187bb7b05f", + "sha256": "5b8e9a8e150c07e2a003b1d7ada05711f6a2936ece1833c2a421edcb2802fddd", "role": "Runs explicitly selected live OMERO variants and preserves component and table-service diagnostics on failure." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/omero_testing.rst against .github/workflows/integration-tests.yml: the unit-tests job now runs three file-isolated xdist workers; no claim this page makes changes.", "On 2026-10-11 (B6), re-read docs/source/development/omero_testing.rst in full and compared it against the changed authorities: tests/integration/test_main.py (E1: ReadPipelineStart, PadToMaxSize and MaterializationBackend.__registry__ replace enum members); the page names none of those spellings; the harness still selects the omero_local materialization backend for OMERO runs and exercises direct and ZMQ execution, so nothing was changed.", "On 2026-10-11 (B6 final), compared docs/source/development/omero_testing.rst against B6's later changes to .github/workflows/integration-tests.yml: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/development/omero_testing.rst against the later B6 changes to external/PolyStore/src/polystore/omero_local.py: Python changes that are not formatting-only (AST-unequal) were read (dependency floors, mahotas moved to core, startup-owner record through the held lock, OMERO is_dir, catalog exclusion by declared distribution, source-path filter matching, GUI catalog readiness, MCP application release); none changes a claim this page makes.", diff --git a/docs/audits/user_docs.json b/docs/audits/user_docs.json index 73e10cbb64..313b9ae0d8 100644 --- a/docs/audits/user_docs.json +++ b/docs/audits/user_docs.json @@ -1160,7 +1160,7 @@ }, { "path": ".github/workflows/integration-tests.yml", - "sha256": "781bd20677f309152ad4041bd7d62d5aca9ea802bd9a0a201a96ad187bb7b05f", + "sha256": "5b8e9a8e150c07e2a003b1d7ada05711f6a2936ece1833c2a421edcb2802fddd", "role": "Exercises Windows and macOS native installers, staged updates, and cellprofiler-compat installation on Python 3.14." }, { @@ -1172,6 +1172,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/getting_started/getting_started.rst against .github/workflows/integration-tests.yml: the unit-tests job now runs three file-isolated xdist workers; no claim this page makes changes.", "On 2026-10-11 (B6), re-read docs/source/getting_started/getting_started.rst in full and compared it against the changed authorities: pyproject.toml (python-introspect 0.2.8), openhcs/mcp/installed_demo.py, openhcs/desktop/update.py, openhcs/desktop/update_worker.py, openhcs/desktop/deployment.py and tests/unit/desktop/test_desktop_update_worker.py (E1: update-check origin, failed-update response and runtime platform became class families); the page names none of those types; the extras, Python range, openhcs and openhcs-mcp-demo scripts, and the verified-replacement update with the previous environment preserved are unchanged, so nothing was changed.", "On 2026-10-11 (B6), re-read docs/source/getting_started/getting_started.rst in full and compared it against the changed authorities: pyproject.toml (first-party floors moved to zmqruntime 0.6.2, pycodify 0.2, objectstate 2.1.1, python-introspect 0.2.7, pyqt-reactive 0.5). The page states no dependency floors; rechecked requires-python (3.11-3.14) and every named extra (gui, viz, bioformats, mcp, cellprofiler-compat, napari, fiji, gpu) against pyproject.toml; no claim changes.", "On 2026-10-11 (B6 final), compared docs/source/getting_started/getting_started.rst against B6's later changes to openhcs/mcp/installed_demo.py: the writability probe makes one exclusive create attempt, the installed demo reports ViewerType.value, the unit session pre-creates its QApplication, and the GUI journey waits for each batch to end. No claim this page makes changes.", @@ -2796,7 +2797,7 @@ }, { "path": ".github/workflows/integration-tests.yml", - "sha256": "781bd20677f309152ad4041bd7d62d5aca9ea802bd9a0a201a96ad187bb7b05f", + "sha256": "5b8e9a8e150c07e2a003b1d7ada05711f6a2936ece1833c2a421edcb2802fddd", "role": "Defines the operating-system integration matrix, the Python 3.14 core job, the scoped four-platform Official30 parity lane and its required check, installed-wheel GUI/MCP operation, installer screenshot artifacts, and source-gate coverage artifacts used by CI." }, { @@ -2928,6 +2929,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/guides/testing_guide.rst against .github/workflows/integration-tests.yml: the unit-tests job now runs three file-isolated xdist workers; no claim this page makes changes.", "On 2026-10-11 (B6), re-read docs/source/guides/testing_guide.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/config.py; the diff renames AgentRuntimePlatformAuthority to AgentRuntimePlatform and adds GuiLogLevel.emits_records; LoggingConfig and its default log directory, which the debugging section relies on, are unchanged, so no correction was needed.", "On 2026-10-11 (B6), re-read docs/source/guides/testing_guide.rst in full and compared it against the changed authority openhcs/core/xdg_paths.py (legacy ~/.openhcs migration helpers deleted; get_*_file_path now plain path joins); the page's log-location claim (get_openhcs_data_dir() / \"logs\", honouring XDG_DATA_HOME, default ~/.local/share/openhcs/logs) still holds and the page never described the deleted migration, so no correction was needed.", "On 2026-10-11 (B6 final), compared docs/source/guides/testing_guide.rst against B6's later changes to .github/workflows/integration-tests.yml: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", From 7379deb901212b5948866ead5032cd7652a9a510 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 20:27:19 -0400 Subject: [PATCH 77/84] B6 WIP: export scope crosses the request boundary as its enum view and reaches the session as the family; smoke reads the action status family E1 made the export scope and UI action status families: the CLI passed the family class into DatasetRunRequest (an enum-view field), RunDatasets passed the enum member to the session (which expects the family), and the GUI smoke read UiActionInvocationStatus.ACCEPTED. Co-Authored-By: Claude Opus 5.5 --- benchmark/cellprofiler_benchmark_cli.py | 10 ++++---- .../authoring/session/operations/datasets.py | 23 +++++++++++++------ scripts/smoke_installed_gui.py | 3 +-- 3 files changed, 22 insertions(+), 14 deletions(-) diff --git a/benchmark/cellprofiler_benchmark_cli.py b/benchmark/cellprofiler_benchmark_cli.py index 55d6136443..cc5f259faa 100644 --- a/benchmark/cellprofiler_benchmark_cli.py +++ b/benchmark/cellprofiler_benchmark_cli.py @@ -232,7 +232,10 @@ def run(self, args: argparse.Namespace) -> int: MeasuredRunEnded, SessionMeasuredRun, ) - from openhcs.agent.dto.session import DatasetRunRequest + from openhcs.agent.dto.session import ( + DatasetRunRequest, + RuntimeObservationExportScopeName, + ) from openhcs.agent.path_policy import AgentPathPolicy from openhcs.agent.session_access import AgentPathDatasetAccess from openhcs.authoring.session.session import CallerThread, Session @@ -242,9 +245,6 @@ def run(self, args: argparse.Namespace) -> int: mirror_input_workspace, ) from openhcs.runtime.zmq_config import OpenHCSZMQConfig - from openhcs.runtime.zmq_execution_signature import ( - ZMQRuntimeObservationExportScope, - ) from python_introspect import to_jsonable output_dir = args.output_dir.expanduser().resolve() @@ -282,7 +282,7 @@ def run(self, args: argparse.Namespace) -> int: runtime_observation_export_path=str( MeasuredPipelineRunArtifact.RUNTIME_OBSERVATION.path_in(output_dir) ), - runtime_observation_export_scope=ZMQRuntimeObservationExportScope.named( + runtime_observation_export_scope=RuntimeObservationExportScopeName( args.observation_scope ), submit_timeout_ms=args.submit_timeout_ms, diff --git a/openhcs/authoring/session/operations/datasets.py b/openhcs/authoring/session/operations/datasets.py index d9030c452f..af97138844 100644 --- a/openhcs/authoring/session/operations/datasets.py +++ b/openhcs/authoring/session/operations/datasets.py @@ -148,7 +148,9 @@ class DeleteDatasets(IdleTargetsOperation, HeadlessOperation): operation_id = "delete_datasets" label = "Del" tooltip = "Delete selected datasets" - description = "Removes datasets and their configuration and pipeline from the session." + description = ( + "Removes datasets and their configuration and pipeline from the session." + ) side_effects = ("mutates_dataset_list",) @classmethod @@ -170,7 +172,9 @@ def available(cls, session, request) -> AgentError | None: error = super().available(session, request) if error is not None: return error - pending = [s for s in request.scope_ids if session.has_pending_definition_work(s)] + pending = [ + s for s in request.scope_ids if session.has_pending_definition_work(s) + ] if pending: return AgentError( code="dataset_definition_pending", @@ -179,7 +183,9 @@ def available(cls, session, request) -> AgentError | None: return None -class InitializeDatasets(IdleTargetsOperation, DatasetWorkflowOperation, HeadlessOperation): +class InitializeDatasets( + IdleTargetsOperation, DatasetWorkflowOperation, HeadlessOperation +): operation_id = "initialize_datasets" workflow_name = "INIT" label = "Init" @@ -330,7 +336,9 @@ def available(cls, session, request) -> AgentError | None: code="execution_batch_active", message="An execution batch is already running.", ) - pending = [s for s in request.scope_ids if session.has_pending_definition_work(s)] + pending = [ + s for s in request.scope_ids if session.has_pending_definition_work(s) + ] if pending: return AgentError( code="dataset_definition_pending", @@ -357,7 +365,7 @@ def run(cls, session, request) -> SessionOperationResult: tuple(request.scope_ids), auxiliary_params=session.observation_export_params( request.runtime_observation_export_path, - request.runtime_observation_export_scope, + request.runtime_observation_export_scope.scope, ), submit_timeout_ms=request.submit_timeout_ms, wait_timeout_ms=request.wait_timeout_ms, @@ -388,7 +396,9 @@ def button_label(cls, session) -> str: def available(cls, session, request) -> AgentError | None: state = session.execution_state if not state.busy: - return AgentError(code="no_execution_running", message="Nothing is running.") + return AgentError( + code="no_execution_running", message="Nothing is running." + ) if not state.run_button_enabled(True): return AgentError( code="execution_stopping", @@ -461,4 +471,3 @@ class ShowDatasetImages(InitializedTargetsOperation, RendererOperation): tooltip = "Browse dataset images and metadata" description = "Opens the image and metadata browser of the selected datasets." side_effects = ("opens_dataset_viewer_window",) - diff --git a/scripts/smoke_installed_gui.py b/scripts/smoke_installed_gui.py index e9d1837b5d..62fe595382 100644 --- a/scripts/smoke_installed_gui.py +++ b/scripts/smoke_installed_gui.py @@ -566,7 +566,6 @@ def run_installed_live_mcp_smoke( ) from openhcs.agent.dto.mcp import McpServerHealthResult from openhcs.agent.dto.ui_bridge import ( - UiActionInvocationStatus, UiActionInvokeResult, UiBridgeCatalog, UiBridgeStatus, @@ -705,7 +704,7 @@ def run_installed_live_mcp_smoke( capability=agent_capabilities.ui_invoke_action, payload_type=UiActionInvokeResult, ) - if action_result.status != UiActionInvocationStatus.ACCEPTED.value: + if not action_result.status.accepted: raise AssertionError( f"Installed UI action was not accepted: {action_result}" ) From 542f7ffeba120ba7264547e12330e031b86f3caf Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 20:30:57 -0400 Subject: [PATCH 78/84] B6: unit-tests job runs as three serial groups within the 45-minute budget Replaces the xdist attempt (a worker died in test_mcp_server under xcb): the suite's 11.6k tests are split by pytest-split into three groups, each one process, each uploading its coverage as coverage-unit-. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/integration-tests.yml | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/.github/workflows/integration-tests.yml b/.github/workflows/integration-tests.yml index e5801f5027..0a06f4b235 100644 --- a/.github/workflows/integration-tests.yml +++ b/.github/workflows/integration-tests.yml @@ -152,6 +152,12 @@ jobs: runs-on: ubuntu-latest timeout-minutes: 45 if: github.event_name == 'push' || github.event_name == 'pull_request' || github.event_name == 'workflow_dispatch' + # 11.6k tests no longer fit the 45-minute budget in one job; each group + # runs its share serially in one process. + strategy: + fail-fast: false + matrix: + group: [1, 2, 3] env: QT_QPA_PLATFORM: xcb QT_XCB_GL_INTEGRATION: xcb_glx @@ -201,7 +207,8 @@ jobs: python -m scripts.install_ci_candidate \ --dependency-source submodules \ --wheel-directory "$RUNNER_TEMP/openhcs-candidate-wheels" \ - --extras dev,dev-gui,gui,cellprofiler-compat,fiji,mcp-http,napari + --extras dev,dev-gui,gui,cellprofiler-compat,fiji,mcp-http,napari \ + --requirement pytest-split==0.11.0 - name: Materialize tracked Official30 unit fixture run: | @@ -233,21 +240,19 @@ jobs: context.doneCurrent() PY - # 11.6k tests no longer fit the 45-minute budget in one process; files - # are isolated per worker (--dist loadfile), as local verification runs. - name: Run unit tests run: >- xvfb-run --auto-servernum --server-args="-screen 0 1920x1080x24 +extension GLX" python scripts/run_installed_tests.py --coverage --repo-root "$GITHUB_WORKSPACE" -- tests/unit tests/core -p no:randomly -vv --tb=short - -n 3 --dist loadfile + --splits 3 --group ${{ matrix.group }} -o faulthandler_timeout=120 --cov-report= - name: Upload unit coverage artifact uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 with: - name: coverage-unit + name: coverage-unit-${{ matrix.group }} path: .coverage include-hidden-files: true if-no-files-found: error From 5721db4ae7bb4675da189871db99558581d68567 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 20:31:05 -0400 Subject: [PATCH 79/84] B6: re-pin authorities after the E1 boundary fixes Co-Authored-By: Claude Opus 5.5 --- docs/audits/architecture_a_m.json | 6 ++-- docs/audits/architecture_n_z_development.json | 11 +++++--- docs/audits/project_surface.json | 3 +- docs/audits/user_docs.json | 28 ++++++++++++------- 4 files changed, 31 insertions(+), 17 deletions(-) diff --git a/docs/audits/architecture_a_m.json b/docs/audits/architecture_a_m.json index 57fb7b1239..e32c1aa6d0 100644 --- a/docs/audits/architecture_a_m.json +++ b/docs/audits/architecture_a_m.json @@ -344,7 +344,7 @@ "authority": [ { "path": "openhcs/authoring/session/operations/datasets.py", - "sha256": "208274effc2e790775d90528211fa4f640eb01f7ca3fdd9c927c3bf71e39b632", + "sha256": "630a74c9e4fb026a7c47e78b3a07427c4d16c9630447a523b36dd8c73e17e430", "role": "Declares CompileDatasets and its initialised, idle and non-empty-pipeline availability checks." }, { @@ -386,6 +386,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/batch_workflow_service.rst against openhcs/authoring/session/operations/datasets.py: the export scope crosses the request boundary as its enum view and reaches the session as the family, the GUI smoke reads the action-status family, and the unit job runs as three serial groups. No claim this page makes changes.", "On 2026-10-11 (B6), re-read docs/source/architecture/batch_workflow_service.rst in full and compared it against the changed authorities: openhcs/authoring/session/compile_batch.py, openhcs/authoring/session/session.py, openhcs/authoring/session/submission.py; E1 replaced the OrchestratorState and terminal-status enum members with classes and the session now takes the observation export scope as a class, but the compile-only and run flows, compile_artifact_id submission and lifecycle the page describes are unchanged and it names no state member, so no claim changed.", "On 2026-10-11 (B6), re-read docs/source/architecture/batch_workflow_service.rst in full and compared it against the changed authorities: openhcs/authoring/session/session.py (K6 debug command and replay mode became classes: RunDebugCommand, StopDebugCommand, PersistentPausedWorkerReplay); the page does not describe debug sessions, and CompileDatasets, DatasetPipelineRequest, CompiledDataset, BatchSubmitWaitEngine, compile_artifact_id submission, ExecutionProgress, Session.close and the endpoint_status binding all still exist, so no claim changed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/batch_workflow_service.rst against B6's later changes to openhcs/authoring/session/session.py: the writability probe makes one exclusive create attempt, the installed demo reports ViewerType.value, the unit session pre-creates its QApplication, and the GUI journey waits for each batch to end. No claim this page makes changes.", @@ -2318,13 +2319,14 @@ }, { "path": "benchmark/cellprofiler_benchmark_cli.py", - "sha256": "0c2cab4d989d50149cb314e908baace8aa794953c86ebc572fa120beaa695780", + "sha256": "56a9b4e8bb7d8195095eb99a849f717e79ca37aa429c612804584cbd7939f420", "role": "Owns the installed openhcs-benchmark execution, report and list-cases command surface." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/architecture/mcp_distribution.rst against benchmark/cellprofiler_benchmark_cli.py: the export scope crosses the request boundary as its enum view and reaches the session as the family, the GUI smoke reads the action-status family, and the unit job runs as three serial groups. No claim this page makes changes.", "On 2026-10-11 (B6), re-read docs/source/architecture/mcp_distribution.rst in full and compared it against the changed authorities: openhcs/mcp/server.py (registered-family defaults in tool schemas, recovery reason by name), dev_client_core.py and dev_client_session.py (poll status, field filter and orchestrator state classes), http.py and http_auth.py (McpHttpAuthenticationMode became PublicReadOnlyAccess/OAuthIntrospectionAccess classes), client_registration.py and openhcs/desktop/deployment.py (status classes; platform leaves keyed by AgentRuntimePlatform class), openhcs/authoring/session/session.py (current file: E1 state families; events_after, set_transport_config and FinishedExecution unchanged), ui_bridge_windows.py (error_dialog flag), llm_context_service.py, benchmark/control.py and benchmark/cellprofiler_benchmark_cli.py (export scope classes) and the two changed tests; CapabilityTransport is still an Enum with HOSTED_STREAMABLE_HTTP, the Windows and macOS registration and deployment leaves, the public or OAuth-introspection policy and the benchmark capability claims still hold, and the page names no renamed member.", "On 2026-10-11 (B6), re-read docs/source/architecture/mcp_distribution.rst in full and compared it against the changed authorities: authoring/session/session.py (K6 debug command/replay classes), core/config_document.py (L6: rendering through pycodify.generate_python_source; project_dataclass_field_from_source unchanged) and pyqt_gui/services/ui_bridge_windows.py (U3: dirty/signature markers from objectstate.inspection, DockPane from pyqt-reactive); none changes the headless-session, saved-endpoint projection, window-capture, navigation or row-action claims, so no prose change was needed.", "On 2026-10-11 (B6 final), compared docs/source/architecture/mcp_distribution.rst against B6's later changes to openhcs/authoring/session/session.py: the writability probe makes one exclusive create attempt, the installed demo reports ViewerType.value, the unit session pre-creates its QApplication, and the GUI journey waits for each batch to end. No claim this page makes changes.", diff --git a/docs/audits/architecture_n_z_development.json b/docs/audits/architecture_n_z_development.json index f83464af13..99fd75bc02 100644 --- a/docs/audits/architecture_n_z_development.json +++ b/docs/audits/architecture_n_z_development.json @@ -2998,7 +2998,7 @@ }, { "path": "benchmark/cellprofiler_benchmark_cli.py", - "sha256": "0c2cab4d989d50149cb314e908baace8aa794953c86ebc572fa120beaa695780", + "sha256": "56a9b4e8bb7d8195095eb99a849f717e79ca37aa429c612804584cbd7939f420", "role": "Owns registered benchmark command declarations and the derived installed parser." }, { @@ -3015,6 +3015,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/extension_workflows.rst against benchmark/cellprofiler_benchmark_cli.py: the export scope crosses the request boundary as its enum view and reaches the session as the family, the GUI smoke reads the action-status family, and the unit job runs as three serial groups. No claim this page makes changes.", "On 2026-10-11 (B6), re-read docs/source/development/extension_workflows.rst in full and compared it against the changed authorities: pyproject.toml (foundation pins raised; same eight distributions), external/python-introspect/src/python_introspect/jsonable.py (read in full at 0.2.8: singledispatch to_jsonable still registers dataclasses, mappings, sequences, callables, AutoRegisterMeta classes, enums, paths and scalars, paired with dataclass_from_mapping), openhcs/core/pipeline/compilation_session.py and openhcs/core/callable_contract.py (K2 imports), openhcs/core/compiled_step_plan.py (E1 StepInputDependency classes) and benchmark/cellprofiler_benchmark_cli.py (E1: observation scope choices come from the ZMQRuntimeObservationExportScope registry; BenchmarkCliCommand unchanged); every owner named in the table still exists and metaclass-registry still ships the enum-, type- and context-keyed strategy mixins.", "On 2026-10-11 (B6), re-read docs/source/development/extension_workflows.rst in full and compared it against the changed authorities: pyproject.toml (first-party floors only), external/python-introspect/src/python_introspect/jsonable.py (current file: singledispatch registrations for dataclasses, mappings, sequences, callables, AutoRegisterMeta families, enums, paths and scalars, plus registered_class_from_key), openhcs/core/pipeline/compilation_session.py (validation moved onto ArtifactGraph), openhcs/core/callable_contract.py (K5: CallableMetadata folded into CallableContract declared fields; input/output/execution memory types remain) and openhcs/core/compiled_step_plan.py (CompiledSourceBindingPlan() default). Rechecked every class the owner table and prose names (CallableContract, CellProfilerModule, ImageFileFormat, ResolvedPipelineDefinition, CompilationSession, CompiledExecutionBundle, CompiledStepPlan device bindings, BenchmarkCliCommand, ComparisonSuiteRunDeclaration/Receipt, MeasuredPipelineRunReceipt) and that pycodify still owns Python source generation after L6 (ObjectState adds only the lazy-config formatter); no claim changes.", "On 2026-10-11 (B6 final), compared docs/source/development/extension_workflows.rst against B6's later changes to benchmark/cellprofiler_benchmark_cli.py: the session decides in-place initialization by an actual file-creation attempt that leaves nothing behind, its debug-session methods moved into an inherited mixin, the measured CLI treats Windows Ctrl+Break like Ctrl+C, and the Official30 viewer test reads typed payload summaries. No claim this page makes changes.", @@ -3533,7 +3534,7 @@ }, { "path": ".github/workflows/integration-tests.yml", - "sha256": "5b8e9a8e150c07e2a003b1d7ada05711f6a2936ece1833c2a421edcb2802fddd", + "sha256": "b2bcf966b13447f4f584dd27ea250d8c0adbf41faed37b61c5a1a30aec2d7aed", "role": "Cross-platform source-candidate and published-dependency readiness, installed-wheel live GUI/MCP operation with retained native screenshots, three-stage native-installer workflow proof, signing-helper parse, and staged-update validation matrix." }, { @@ -3578,7 +3579,7 @@ }, { "path": "scripts/smoke_installed_gui.py", - "sha256": "a018e473245cee30747f86e85ab0410680bbab248753d20252cff727b22d381a", + "sha256": "2c08452db78f38d06aac4b78b4ffffb88d031dbb414894048646e7b294e4278e", "role": "Owns the installed-GUI acceptance timing and single terminal completion, applies the declared logging configuration, constructs the application on declaration-allocated endpoints, proves the background function catalogue and advertised endpoint log are usable, drives the GUI through packaged MCP, validates retained window snapshots, and cleans up exact owned resources outside the checkout." }, { @@ -3628,6 +3629,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/mcp_release.rst against .github/workflows/integration-tests.yml, scripts/smoke_installed_gui.py: the export scope crosses the request boundary as its enum view and reaches the session as the family, the GUI smoke reads the action-status family, and the unit job runs as three serial groups. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/development/mcp_release.rst against .github/workflows/integration-tests.yml: the unit-tests job now runs three file-isolated xdist workers; no claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/development/mcp_release.rst against B6's later changes to .github/workflows/integration-tests.yml: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/development/mcp_release.rst against B6's later changes to tests/unit/test_mcp_release_workflow.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", @@ -3756,13 +3758,14 @@ }, { "path": ".github/workflows/integration-tests.yml", - "sha256": "5b8e9a8e150c07e2a003b1d7ada05711f6a2936ece1833c2a421edcb2802fddd", + "sha256": "b2bcf966b13447f4f584dd27ea250d8c0adbf41faed37b61c5a1a30aec2d7aed", "role": "Runs explicitly selected live OMERO variants and preserves component and table-service diagnostics on failure." } ], "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/development/omero_testing.rst against .github/workflows/integration-tests.yml: the export scope crosses the request boundary as its enum view and reaches the session as the family, the GUI smoke reads the action-status family, and the unit job runs as three serial groups. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/development/omero_testing.rst against .github/workflows/integration-tests.yml: the unit-tests job now runs three file-isolated xdist workers; no claim this page makes changes.", "On 2026-10-11 (B6), re-read docs/source/development/omero_testing.rst in full and compared it against the changed authorities: tests/integration/test_main.py (E1: ReadPipelineStart, PadToMaxSize and MaterializationBackend.__registry__ replace enum members); the page names none of those spellings; the harness still selects the omero_local materialization backend for OMERO runs and exercises direct and ZMQ execution, so nothing was changed.", "On 2026-10-11 (B6 final), compared docs/source/development/omero_testing.rst against B6's later changes to .github/workflows/integration-tests.yml: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", diff --git a/docs/audits/project_surface.json b/docs/audits/project_surface.json index 6042473bb8..3df00aad34 100644 --- a/docs/audits/project_surface.json +++ b/docs/audits/project_surface.json @@ -82,7 +82,7 @@ }, { "path": "benchmark/cellprofiler_benchmark_cli.py", - "sha256": "0c2cab4d989d50149cb314e908baace8aa794953c86ebc572fa120beaa695780", + "sha256": "56a9b4e8bb7d8195095eb99a849f717e79ca37aa429c612804584cbd7939f420", "role": "Declares the openhcs-benchmark subcommands and options, including run-measured through the OpenHCS session." }, { @@ -114,6 +114,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared README.md against benchmark/cellprofiler_benchmark_cli.py: the export scope crosses the request boundary as its enum view and reaches the session as the family, the GUI smoke reads the action-status family, and the unit job runs as three serial groups. No claim this page makes changes.", "On 2026-10-11 (B6), re-read README.md in full and compared it against the changed authorities: pyproject.toml (python-introspect floor 0.2.8), openhcs/core/pipeline/compiler.py, openhcs/core/pipeline/compilation_session.py, openhcs/core/artifacts.py (K2 split it into the openhcs/core/artifacts package), openhcs/core/dataset_sources/source.py, openhcs/core/viewer_streaming_service.py, openhcs/mcp/client_registration.py, openhcs/interop/cellprofiler/pipeline_import.py, benchmark/cellprofiler_benchmark_cli.py, benchmark/control.py; E1 turned OrchestratorState, MaterializationBackend, InputSource, the client-registration status and the observation export scope into class families, but the README names none of the old enum members and the `--observation-scope outcomes` spelling is still the registered name, so no claim changed. Replaced the deleted artifacts.py authority with openhcs/core/artifacts/kinds.py.", "On 2026-10-11 (B6), re-read README.md in full and compared it against the changed authorities: pyproject.toml (zmqruntime 0.6.2, pycodify 0.2, objectstate 2.1.1, python-introspect 0.2.7, pyqt-reactive 0.5 floors), openhcs/core/pipeline_document.py (renders through pycodify.generate_python_source), openhcs/core/pipeline/compiler.py and compilation_session.py (ScopeKey ids, progress classes, disabled-function stripping removed), openhcs/core/progress/types.py (K6: ProgressPhase/ProgressStatus enums became class families; set_progress_queue unchanged and still exported from openhcs.core.progress), openhcs/core/dataset_sources/source.py (SourceBindingsConfig import moved) and pipeline_import.py (K4 import moves). The README states no dependency version or phase name; the CellProfiler example's imports (ensure_global_config_context, set_progress_queue, import_cellprofiler_pipeline with source_root, PipelineOrchestrator.initialize/compile_pipelines) and the compilation diagram's class names all still resolve, so no claim changed.", "On 2026-10-11 (B6 final), compared README.md against B6's later changes to tests/integration/test_cellprofiler_official30_zmq.py, benchmark/cellprofiler_benchmark_cli.py: the session decides in-place initialization by an actual file-creation attempt that leaves nothing behind, its debug-session methods moved into an inherited mixin, the measured CLI treats Windows Ctrl+Break like Ctrl+C, and the Official30 viewer test reads typed payload summaries. No claim this page makes changes.", diff --git a/docs/audits/user_docs.json b/docs/audits/user_docs.json index 313b9ae0d8..b929ac6f26 100644 --- a/docs/audits/user_docs.json +++ b/docs/audits/user_docs.json @@ -202,7 +202,7 @@ }, { "path": "benchmark/cellprofiler_benchmark_cli.py", - "sha256": "0c2cab4d989d50149cb314e908baace8aa794953c86ebc572fa120beaa695780", + "sha256": "56a9b4e8bb7d8195095eb99a849f717e79ca37aa429c612804584cbd7939f420", "role": "Owns installed benchmark execution and report commands." }, { @@ -224,6 +224,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/appendices/research_impact.rst against benchmark/cellprofiler_benchmark_cli.py: the export scope crosses the request boundary as its enum view and reaches the session as the family, the GUI smoke reads the action-status family, and the unit job runs as three serial groups. No claim this page makes changes.", "On 2026-10-11 (B6), re-read docs/source/appendices/research_impact.rst in full and compared it against the changed authorities: pyproject.toml (python-introspect floor 0.2.8), benchmark/cellprofiler_benchmark_cli.py, benchmark/control.py; the observation export scope became a class family read by name, while the openhcs-benchmark entry point, typed receipts, read-only inspection and report, and the archive path the page names are unchanged, so no claim changed.", "On 2026-10-11 (B6), re-read docs/source/appendices/research_impact.rst in full and compared it against the changed authorities: pyproject.toml (dependency floors for zmqruntime, pycodify 0.2, objectstate 2.1, python-introspect, pyqt-reactive 0.5); the openhcs-benchmark entry point, benchmark CLI, run receipts, read-only inspection service and the archived page path all still exist, and the page states no version, so no claim changed.", "On 2026-10-11 (B6 final), compared docs/source/appendices/research_impact.rst against B6's later changes to benchmark/cellprofiler_benchmark_cli.py: the session decides in-place initialization by an actual file-creation attempt that leaves nothing behind, its debug-session methods moved into an inherited mixin, the measured CLI treats Windows Ctrl+Break like Ctrl+C, and the Official30 viewer test reads typed payload summaries. No claim this page makes changes.", @@ -1160,7 +1161,7 @@ }, { "path": ".github/workflows/integration-tests.yml", - "sha256": "5b8e9a8e150c07e2a003b1d7ada05711f6a2936ece1833c2a421edcb2802fddd", + "sha256": "b2bcf966b13447f4f584dd27ea250d8c0adbf41faed37b61c5a1a30aec2d7aed", "role": "Exercises Windows and macOS native installers, staged updates, and cellprofiler-compat installation on Python 3.14." }, { @@ -1172,6 +1173,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/getting_started/getting_started.rst against .github/workflows/integration-tests.yml: the export scope crosses the request boundary as its enum view and reaches the session as the family, the GUI smoke reads the action-status family, and the unit job runs as three serial groups. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/getting_started/getting_started.rst against .github/workflows/integration-tests.yml: the unit-tests job now runs three file-isolated xdist workers; no claim this page makes changes.", "On 2026-10-11 (B6), re-read docs/source/getting_started/getting_started.rst in full and compared it against the changed authorities: pyproject.toml (python-introspect 0.2.8), openhcs/mcp/installed_demo.py, openhcs/desktop/update.py, openhcs/desktop/update_worker.py, openhcs/desktop/deployment.py and tests/unit/desktop/test_desktop_update_worker.py (E1: update-check origin, failed-update response and runtime platform became class families); the page names none of those types; the extras, Python range, openhcs and openhcs-mcp-demo scripts, and the verified-replacement update with the previous environment preserved are unchanged, so nothing was changed.", "On 2026-10-11 (B6), re-read docs/source/getting_started/getting_started.rst in full and compared it against the changed authorities: pyproject.toml (first-party floors moved to zmqruntime 0.6.2, pycodify 0.2, objectstate 2.1.1, python-introspect 0.2.7, pyqt-reactive 0.5). The page states no dependency floors; rechecked requires-python (3.11-3.14) and every named extra (gui, viz, bioformats, mcp, cellprofiler-compat, napari, fiji, gpu) against pyproject.toml; no claim changes.", @@ -1369,7 +1371,7 @@ }, { "path": "openhcs/authoring/session/operations/datasets.py", - "sha256": "208274effc2e790775d90528211fa4f640eb01f7ca3fdd9c927c3bf71e39b632", + "sha256": "630a74c9e4fb026a7c47e78b3a07427c4d16c9630447a523b36dd8c73e17e430", "role": "Compile/Connect/Connecting slot, Run availability (compiled, endpoint accepting requests) and Stop while a batch runs." }, { @@ -1381,6 +1383,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/basic_interface.rst against openhcs/authoring/session/operations/datasets.py: the export scope crosses the request boundary as its enum view and reaches the session as the family, the GUI smoke reads the action-status family, and the unit job runs as three serial groups. No claim this page makes changes.", "On 2026-10-11 (B6 final), openhcs/pyqt_gui/main.py changed by black formatting only (AST-equal) after docs/source/guide_for_biologists/basic_interface.rst was reviewed; no claim changes.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/basic_interface.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/main.py, widgets/plate_manager.py, widgets/pipeline_editor.py, services/ui_bridge_windows.py, openhcs/pyqt_gui/config.py, openhcs/agent/services/ui_bridge_service.py, openhcs/authoring/session/views.py and the three test modules (E1: selection scope, orchestrator state, action status and update-check origin became class families; the error-dialog marker became a boolean); the page describes user-visible behaviour and names no changed type; all-plates versus selected code documents, Connect/Compile/Run/Stop control behaviour and the pane identities are unchanged, so nothing was changed.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/basic_interface.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/main.py, widgets/plate_manager.py, widgets/pipeline_editor.py, shared/services/pipeline_editor_workflows.py, services/ui_bridge_windows.py, openhcs/core/pipeline_document.py and their tests (U3: DockPane, DockLayoutStore, ShortcutLifecycle, time-travel navigation, path cache and the external-editor switch moved to pyqt-reactive, GlobalEventBus broadcasts replaced by session events, ScopeKey scope ids; L6: pipeline documents render through pycodify generate_python_source; K6 debug command classes). The page describes behaviour, not those classes; rechecked the dock arrangement (system monitor above, Datasets and ZMQ server manager stacked left, Pipeline Editor right), the plate_manager window identity, list_previews on UIConfig, and that PipelineDocument still requires pipeline_steps; no claim changes.", @@ -1551,7 +1554,7 @@ }, { "path": "benchmark/cellprofiler_benchmark_cli.py", - "sha256": "0c2cab4d989d50149cb314e908baace8aa794953c86ebc572fa120beaa695780", + "sha256": "56a9b4e8bb7d8195095eb99a849f717e79ca37aa429c612804584cbd7939f420", "role": "Owns packaged comparison execution and reporting commands." }, { @@ -1588,6 +1591,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/domain_expert_onboarding.rst against benchmark/cellprofiler_benchmark_cli.py: the export scope crosses the request boundary as its enum view and reaches the session as the family, the GUI smoke reads the action-status family, and the unit job runs as three serial groups. No claim this page makes changes.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/domain_expert_onboarding.rst in full and compared it against the changed authorities: openhcs/interop/cellprofiler/pipeline_import.py, openhcs/interop/cellprofiler/module_artifact_declarations.py, tests/unit/test_cellprofiler_pipeline_import.py (E1 input-source classes, K2 artifact package), benchmark/cellprofiler_benchmark_cli.py, benchmark/control.py (export scope family) and openhcs/authoring/session/views.py; the page names none of the changed types; the CellProfiler translation list, the unsupported-interaction report and the read-only benchmark inspection routes are unchanged, so nothing was changed.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/domain_expert_onboarding.rst in full and compared it against the changed authorities: openhcs/interop/cellprofiler/pipeline_import.py, module_artifact_declarations.py, openhcs/agent/services/architecture_projection_service.py and tests/unit/test_cellprofiler_pipeline_import.py (K4: SourceBindingsConfig and StepSourceBindingsConfig now imported from openhcs.core.config, order matching is PairByOrder). The page names neither import location; rechecked the cellprofiler_translation topic id, the interactive-input import error, and the DatasetListView 'Datasets' title; no claim changes.", "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/domain_expert_onboarding.rst against B6's later changes to openhcs/authoring/session/views.py: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", @@ -2036,7 +2040,7 @@ }, { "path": "openhcs/authoring/session/operations/datasets.py", - "sha256": "208274effc2e790775d90528211fa4f640eb01f7ca3fdd9c927c3bf71e39b632", + "sha256": "630a74c9e4fb026a7c47e78b3a07427c4d16c9630447a523b36dd8c73e17e430", "role": "Declares the Init, Compile, Run, Viewer, and Results dataset operations and their button labels named in the tutorial steps." }, { @@ -2053,6 +2057,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/intro_stitching.rst against openhcs/authoring/session/operations/datasets.py: the export scope crosses the request boundary as its enum view and reaches the session as the family, the GUI smoke reads the action-status family, and the unit job runs as three serial groups. No claim this page makes changes.", "On 2026-10-11 (B6 final), openhcs/pyqt_gui/main.py changed by black formatting only (AST-equal) after docs/source/guide_for_biologists/intro_stitching.rst was reviewed; no claim changes.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/intro_stitching.rst in full and compared it against the changed authorities: openhcs/demo/synthetic_plate_pipeline.py and openhcs/pyqt_gui/main.py; the supplied pipeline still has eight steps running from Image Enhancement Processing to Cell Counting (only its InputSource and variable-size spellings became ReadPipelineStart and PadToMaxSize) and main.py still places Generate Synthetic Plate under View; no correction was needed.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/intro_stitching.rst in full and compared it against the changed authority openhcs/pyqt_gui/main.py (U3 dock, shortcut and time-travel moves to pyqt-reactive); the View menu still holds the Generate Synthetic Plate action, so no correction was needed.", @@ -2081,7 +2086,7 @@ "authority": [ { "path": "openhcs/authoring/session/operations/datasets.py", - "sha256": "208274effc2e790775d90528211fa4f640eb01f7ca3fdd9c927c3bf71e39b632", + "sha256": "630a74c9e4fb026a7c47e78b3a07427c4d16c9630447a523b36dd8c73e17e430", "role": "Declares the Init, Compile, Run, Connect, and Code dataset operations and the submit_timeout_ms request field that the recovery steps name." }, { @@ -2178,6 +2183,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/guide_for_biologists/troubleshooting_FAQ.rst against openhcs/authoring/session/operations/datasets.py: the export scope crosses the request boundary as its enum view and reaches the session as the family, the GUI smoke reads the action-status family, and the unit job runs as three serial groups. No claim this page makes changes.", "On 2026-10-11 (B6 final), openhcs/pyqt_gui/main.py changed by black formatting only (AST-equal) after docs/source/guide_for_biologists/troubleshooting_FAQ.rst was reviewed; no claim changes.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/troubleshooting_FAQ.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compiler.py, openhcs/utils/environment.py, openhcs/pyqt_gui/launch.py, openhcs/runtime/viewer_protocol.py, openhcs/runtime/zmq_execution_client.py, openhcs/pyqt_gui/main.py and openhcs/pyqt_gui/services/ui_bridge_server.py; confirmed --no-gpu, OPENHCS_CPU_ONLY, the .startup.jsonl journal, materialize_runtime_artifacts and submit_timeout_ms still exist; the diffs are enum-to-family renames (orchestrator states, settle phases, viewer lifecycle, update-check origins), the AgentRuntimePlatform rename and the removed QtPlatformSystem enum, none of which the page names, so no correction was needed.", "On 2026-10-11 (B6), re-read docs/source/guide_for_biologists/troubleshooting_FAQ.rst in full and compared it against the changed authorities: openhcs/core/pipeline/compiler.py (ScopeKey scope ids, K6 progress classes), openhcs/runtime/zmq_execution_client.py (K6: debug requests serialize themselves, command types are DebugCommandType classes; the .startup.jsonl journal beside the server log is unchanged) and openhcs/pyqt_gui/main.py (U3 moves to pyqt-reactive); the restart-and-restore action, OPENHCS_UI_CONFIG_CACHE_FILE, submit_timeout_ms and materialize_runtime_artifacts still exist, so no correction was needed.", @@ -2797,7 +2803,7 @@ }, { "path": ".github/workflows/integration-tests.yml", - "sha256": "5b8e9a8e150c07e2a003b1d7ada05711f6a2936ece1833c2a421edcb2802fddd", + "sha256": "b2bcf966b13447f4f584dd27ea250d8c0adbf41faed37b61c5a1a30aec2d7aed", "role": "Defines the operating-system integration matrix, the Python 3.14 core job, the scoped four-platform Official30 parity lane and its required check, installed-wheel GUI/MCP operation, installer screenshot artifacts, and source-gate coverage artifacts used by CI." }, { @@ -2887,7 +2893,7 @@ }, { "path": "scripts/smoke_installed_gui.py", - "sha256": "a018e473245cee30747f86e85ab0410680bbab248753d20252cff727b22d381a", + "sha256": "2c08452db78f38d06aac4b78b4ffffb88d031dbb414894048646e7b294e4278e", "role": "Owns the installed-GUI acceptance timing and single terminal completion, applies the declared logging configuration, constructs the application on declaration-allocated endpoints, proves the background function catalogue and advertised endpoint log are usable, drives the GUI through packaged MCP, validates retained window snapshots, and cleans up exact owned resources outside the checkout." }, { @@ -2929,6 +2935,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/guides/testing_guide.rst against .github/workflows/integration-tests.yml, scripts/smoke_installed_gui.py: the export scope crosses the request boundary as its enum view and reaches the session as the family, the GUI smoke reads the action-status family, and the unit job runs as three serial groups. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/guides/testing_guide.rst against .github/workflows/integration-tests.yml: the unit-tests job now runs three file-isolated xdist workers; no claim this page makes changes.", "On 2026-10-11 (B6), re-read docs/source/guides/testing_guide.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/config.py; the diff renames AgentRuntimePlatformAuthority to AgentRuntimePlatform and adds GuiLogLevel.emits_records; LoggingConfig and its default log directory, which the debugging section relies on, are unchanged, so no correction was needed.", "On 2026-10-11 (B6), re-read docs/source/guides/testing_guide.rst in full and compared it against the changed authority openhcs/core/xdg_paths.py (legacy ~/.openhcs migration helpers deleted; get_*_file_path now plain path joins); the page's log-location claim (get_openhcs_data_dir() / \"logs\", honouring XDG_DATA_HOME, default ~/.local/share/openhcs/logs) still holds and the page never described the deleted migration, so no correction was needed.", @@ -4379,7 +4386,7 @@ }, { "path": "openhcs/authoring/session/operations/datasets.py", - "sha256": "208274effc2e790775d90528211fa4f640eb01f7ca3fdd9c927c3bf71e39b632", + "sha256": "630a74c9e4fb026a7c47e78b3a07427c4d16c9630447a523b36dd8c73e17e430", "role": "Declares the add, set-pipeline, initialize, compile, run, and stop dataset operations behind the session tools." }, { @@ -4409,7 +4416,7 @@ }, { "path": "benchmark/cellprofiler_benchmark_cli.py", - "sha256": "0c2cab4d989d50149cb314e908baace8aa794953c86ebc572fa120beaa695780", + "sha256": "56a9b4e8bb7d8195095eb99a849f717e79ca37aa429c612804584cbd7939f420", "role": "Declares the list-cases, inspect-run, inspect-measured, and run-measured commands, including the required --wait-timeout-ms." }, { @@ -4456,6 +4463,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), compared docs/source/user_guide/mcp_clients.rst against openhcs/authoring/session/operations/datasets.py, benchmark/cellprofiler_benchmark_cli.py: the export scope crosses the request boundary as its enum view and reaches the session as the family, the GUI smoke reads the action-status family, and the unit job runs as three serial groups. No claim this page makes changes.", "On 2026-10-11 (B6), re-read docs/source/user_guide/mcp_clients.rst in full and compared it against the changed authorities: openhcs/mcp/client_registration.py (registration statuses and platform keys became class families), openhcs/mcp/lifecycle.py (McpProcessRecoveryReason classes), openhcs/agent/path_policy.py, benchmark/cellprofiler_benchmark_cli.py (observation-scope choices read from the ZMQRuntimeObservationExportScope registry), benchmark/control.py and the two agent tests; the --observation-scope spellings outcomes and values (default values) are still the registered names, --wait-timeout-ms is still required, the dataset row still reports complete (ExecutionCompleted.name), and client registration and restart behaviour are unchanged; no edit needed. The --surface help text in openhcs/mcp/bootstrap.py still says the desktop surface hides headless tools although the desktop profile includes them (reported earlier, code not edited).", "On 2026-10-11 (B6 final), compared docs/source/user_guide/mcp_clients.rst against B6's later changes to benchmark/cellprofiler_benchmark_cli.py: the session decides in-place initialization by an actual file-creation attempt that leaves nothing behind, its debug-session methods moved into an inherited mixin, the measured CLI treats Windows Ctrl+Break like Ctrl+C, and the Official30 viewer test reads typed payload summaries. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/user_guide/mcp_clients.rst against B6's later changes to openhcs/authoring/session/operations/datasets.py, benchmark/cellprofiler_benchmark_cli.py, benchmark/control_service.py, tests/unit/agent/test_mcp_server.py, tests/unit/test_benchmark_control.py: black formatting (AST-equal) and the ratchet refactor (method groups moved into inherited mixins, unused main-window theme methods deleted, catalog policy moved to module functions); none changes a claim this page makes.", From f0498a630d6102f4ebc5dc7a692ae676b77f9d09 Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 20:31:54 -0400 Subject: [PATCH 80/84] B6: GUI smoke compares the action status to ActionAccepted nominally Co-Authored-By: Claude Opus 5.5 --- scripts/smoke_installed_gui.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/scripts/smoke_installed_gui.py b/scripts/smoke_installed_gui.py index 62fe595382..be1bb8abec 100644 --- a/scripts/smoke_installed_gui.py +++ b/scripts/smoke_installed_gui.py @@ -566,6 +566,7 @@ def run_installed_live_mcp_smoke( ) from openhcs.agent.dto.mcp import McpServerHealthResult from openhcs.agent.dto.ui_bridge import ( + ActionAccepted, UiActionInvokeResult, UiBridgeCatalog, UiBridgeStatus, @@ -704,7 +705,7 @@ def run_installed_live_mcp_smoke( capability=agent_capabilities.ui_invoke_action, payload_type=UiActionInvokeResult, ) - if not action_result.status.accepted: + if action_result.status is not ActionAccepted: raise AssertionError( f"Installed UI action was not accepted: {action_result}" ) From 8b31dc02f56d1f8d07f9a6b1fa276c463036ea9f Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 20:31:59 -0400 Subject: [PATCH 81/84] B6: re-pin pages citing the GUI smoke Co-Authored-By: Claude Opus 5.5 --- docs/audits/architecture_n_z_development.json | 3 ++- docs/audits/user_docs.json | 3 ++- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/audits/architecture_n_z_development.json b/docs/audits/architecture_n_z_development.json index 99fd75bc02..892706e141 100644 --- a/docs/audits/architecture_n_z_development.json +++ b/docs/audits/architecture_n_z_development.json @@ -3579,7 +3579,7 @@ }, { "path": "scripts/smoke_installed_gui.py", - "sha256": "2c08452db78f38d06aac4b78b4ffffb88d031dbb414894048646e7b294e4278e", + "sha256": "c6cca9ad8a248e63949c3556562ec4760d6aaeeaba215bb21ef556c8a8cb726d", "role": "Owns the installed-GUI acceptance timing and single terminal completion, applies the declared logging configuration, constructs the application on declaration-allocated endpoints, proves the background function catalogue and advertised endpoint log are usable, drives the GUI through packaged MCP, validates retained window snapshots, and cleans up exact owned resources outside the checkout." }, { @@ -3629,6 +3629,7 @@ ], "disposition": "revise", "validation": [ + "On 2026-10-11 (B6 final), scripts/smoke_installed_gui.py changed after docs/source/development/mcp_release.rst was reviewed: the GUI smoke compares the action status to ActionAccepted; no claim changes.", "On 2026-10-11 (B6 final), compared docs/source/development/mcp_release.rst against .github/workflows/integration-tests.yml, scripts/smoke_installed_gui.py: the export scope crosses the request boundary as its enum view and reaches the session as the family, the GUI smoke reads the action-status family, and the unit job runs as three serial groups. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/development/mcp_release.rst against .github/workflows/integration-tests.yml: the unit-tests job now runs three file-isolated xdist workers; no claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/development/mcp_release.rst against B6's later changes to .github/workflows/integration-tests.yml: one-plane label projection now owned by every slice-aligned label mode, dataset rows carry the finished execution's error, the wheel job prewarms the Fiji bundle, the test session creates its QApplication first, and tracked files keep committed bytes. No claim this page makes changes.", diff --git a/docs/audits/user_docs.json b/docs/audits/user_docs.json index b929ac6f26..20bc2613db 100644 --- a/docs/audits/user_docs.json +++ b/docs/audits/user_docs.json @@ -2893,7 +2893,7 @@ }, { "path": "scripts/smoke_installed_gui.py", - "sha256": "2c08452db78f38d06aac4b78b4ffffb88d031dbb414894048646e7b294e4278e", + "sha256": "c6cca9ad8a248e63949c3556562ec4760d6aaeeaba215bb21ef556c8a8cb726d", "role": "Owns the installed-GUI acceptance timing and single terminal completion, applies the declared logging configuration, constructs the application on declaration-allocated endpoints, proves the background function catalogue and advertised endpoint log are usable, drives the GUI through packaged MCP, validates retained window snapshots, and cleans up exact owned resources outside the checkout." }, { @@ -2935,6 +2935,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), scripts/smoke_installed_gui.py changed after docs/source/guides/testing_guide.rst was reviewed: the GUI smoke compares the action status to ActionAccepted; no claim changes.", "On 2026-10-11 (B6 final), compared docs/source/guides/testing_guide.rst against .github/workflows/integration-tests.yml, scripts/smoke_installed_gui.py: the export scope crosses the request boundary as its enum view and reaches the session as the family, the GUI smoke reads the action-status family, and the unit job runs as three serial groups. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/guides/testing_guide.rst against .github/workflows/integration-tests.yml: the unit-tests job now runs three file-isolated xdist workers; no claim this page makes changes.", "On 2026-10-11 (B6), re-read docs/source/guides/testing_guide.rst in full and compared it against the changed authorities: openhcs/pyqt_gui/config.py; the diff renames AgentRuntimePlatformAuthority to AgentRuntimePlatform and adds GuiLogLevel.emits_records; LoggingConfig and its default log directory, which the debugging section relies on, are unchanged, so no correction was needed.", From ce529e45904b7fad6217efa1097d5f98ffb1345a Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 20:54:00 -0400 Subject: [PATCH 82/84] B6: website concurrency group is per ref With one shared 'pages' group, every push to main cancelled pull requests' Website validation. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/website-pages.yml | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/.github/workflows/website-pages.yml b/.github/workflows/website-pages.yml index e05d73bcbd..5dc97e8d30 100644 --- a/.github/workflows/website-pages.yml +++ b/.github/workflows/website-pages.yml @@ -41,8 +41,9 @@ on: permissions: contents: read +# One group per ref: a push to main must not cancel a pull request's validation. concurrency: - group: pages + group: pages-${{ github.ref }} cancel-in-progress: true jobs: From abf42ae1bfe4e37ff7608c6acc9a4bb98e5299fd Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 20:54:08 -0400 Subject: [PATCH 83/84] B6: re-pin pages citing the website workflow Co-Authored-By: Claude Opus 5.5 --- docs/audits/user_docs.json | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/audits/user_docs.json b/docs/audits/user_docs.json index 20bc2613db..7b30caffa3 100644 --- a/docs/audits/user_docs.json +++ b/docs/audits/user_docs.json @@ -2843,7 +2843,7 @@ }, { "path": ".github/workflows/website-pages.yml", - "sha256": "577e0747692e7738bdb22f1b579c08984d6dffa8e709e0e33ed9edf4110b3be6", + "sha256": "d4fdfdf27f02a9c9cab67fa876f7a06844fa8b7298acce0e25a352682a34925f", "role": "Installs the OpenHCS candidate from submodule sources with install_ci_candidate before validating gallery generation, then deploys the checked-in dependency-free website projection." }, { @@ -2935,6 +2935,7 @@ "findings": [], "disposition": "keep", "validation": [ + "On 2026-10-11 (B6 final), .github/workflows/website-pages.yml changed after docs/source/guides/testing_guide.rst was reviewed: the Website workflow's concurrency group is per ref; no claim changes.", "On 2026-10-11 (B6 final), scripts/smoke_installed_gui.py changed after docs/source/guides/testing_guide.rst was reviewed: the GUI smoke compares the action status to ActionAccepted; no claim changes.", "On 2026-10-11 (B6 final), compared docs/source/guides/testing_guide.rst against .github/workflows/integration-tests.yml, scripts/smoke_installed_gui.py: the export scope crosses the request boundary as its enum view and reaches the session as the family, the GUI smoke reads the action-status family, and the unit job runs as three serial groups. No claim this page makes changes.", "On 2026-10-11 (B6 final), compared docs/source/guides/testing_guide.rst against .github/workflows/integration-tests.yml: the unit-tests job now runs three file-isolated xdist workers; no claim this page makes changes.", From 852504769f894829d43bf7ba13afbfeb58a725ab Mon Sep 17 00:00:00 2001 From: Tristan Simas Date: Sun, 11 Oct 2026 22:01:37 -0400 Subject: [PATCH 84/84] B6: receipt - handoff Co-Authored-By: Claude Opus 5.5 --- .../consolidation-20261009/B6-ci-green.md | 81 +++++++++++++++++++ 1 file changed, 81 insertions(+) diff --git a/docs/refactor/consolidation-20261009/B6-ci-green.md b/docs/refactor/consolidation-20261009/B6-ci-green.md index 250e389d9e..301fca9327 100644 --- a/docs/refactor/consolidation-20261009/B6-ci-green.md +++ b/docs/refactor/consolidation-20261009/B6-ci-green.md @@ -47,3 +47,84 @@ Website, Documentation and Integration Tests are green on this PR's head, or eac - OpenHCSDev/PolyStore#38 (0.6.1; OMERO `is_dir`; docs logs) - OpenHCSDev/ObjectState#26 (2.1.1) - OpenHCSDev/PyQT-reactive#21 (docs logs) + +## Handoff + +**Branch `refactor/B6-ci-green`, head `abf42ae1b` plus this receipt commit, merged with main `625150118`. PR #1195. GitHub Actions stalled before any job ran on this head, so nothing after `f419cb3bb` has a completed CI run.** + +### Last completed CI runs + +- **`f419cb3bb` (merged with main `eb64b3c13`): 36 jobs passed, 7 failed.** Website, Documentation and Refactor Guardrails were green. This is the last trustworthy picture. +- **`2e0deafe6` (merged with main `44980dfe2`): 27 jobs failed.** Fixed since, unverified (see below). + +### Root causes fixed, with commits + +| Cause | Commit | +|---|---| +| Docs path regex matched inside longer paths | `c9de5fab0` | +| Website installed a package roster; needs dev extra; path filter; shared `pages` concurrency group | `6bccc4d6d`, `5544c445f`, `ce529e459` | +| Sibling test-module imports under importlib mode | `6bccc4d6d` | +| `os.getpgid` on Windows; top-level import in GUI smoke | `62752cf33` | +| 141/126/127 stale documentation pages (three rounds, after each main merge) | `e59901e76`, `b2fb1bcd2`, `297a423d5` | +| Library docs: orphaned logs, removed zmqruntime members | `6669b3401`, `f3eb57aa9` | +| zmqruntime startup-owner lock on Windows | `333d3e0ab` (library PR ZMQRuntime#29) | +| Windows uint32 format; custom-function bytes | `47724ccd4`, `d5172f398` | +| Stale GUI tests; catalog race; requestless UI-bridge call; `partition_values` | `f59da9b8f`, `03ca14cc9`, `c3ab99415`, `87e4df3ff` | +| Marker-excluded distributions in the catalog (Intel macOS basicpy) | `c1013dd51` | +| BaSiC `.metadata`; STEP_INPUT filter paths; viewer families; GUI journey job | `c00a652d4`, `35abb00d6`, `4b1dec1e2`, `2eb48fb9d`, `dbae3e0b4` | +| mahotas core dependency; measured CLI child environment | `5e519401f`, `42f51b7a7` | +| OMERO `is_dir` (PolyStore#38); metadata document for virtual backends | `54e87d5b4`, `956326e3e` | +| Passthrough filename qualifier; adapter config; watchdog; stale synthetic tests | `89269b9a8`, `0f0ad9f7c`, `0d83e1e16`, `34556a1ac` | +| ObjectState own annotations on 3.14 (ObjectState#26) | `fac0519e5`, re-based to 2.1.1 in the first main merge | +| Ratchet: mixins, dead theme methods, catalog policy functions | `239953c4c`, `ab03ba7ba` | +| Packaged recipe paths over MAX_PATH; macOS serial kernel preparation | `35833f3a5`, `ded24279e` | +| Runtime-plane axes dropped from streams (napari settle) | `7066af9f3` | +| Windows read-only datasets, Ctrl+Break, one-shot write probe | `433b26fef`, `e1a1719d3`, `c74b2bd55` | +| Wheel job Fiji bundle; `.gitattributes` exact bytes | `e82ae3398`, `fbe17e02d` | +| One-plane label projection; demo error surfaced; `ViewerType.value` | `5de70ef71`, `88e22432e` | +| Unit session QApplication before MCP transports | `f322e395b`, `a5ce88b63`, `d43533f90` | +| `PARTITION_COUNT` in observation-export test | `b14856fbd` | +| E1 boundary: export scope and UI action status | `7379deb90`, `f0498a630` | +| Unit job split into three serial groups | `542f7ffeb` | + +### Still red at the last completed run (`f419cb3bb`) + +| Job | First cause | Owner | +|---|---|---| +| pypi-dependency-readiness | Pinned ObjectState 2.1.1, PolyStore 0.6.1 and zmqruntime 0.6.2 have no release tags | owner: merge and tag ObjectState#26, PolyStore#38, ZMQRuntime#29 | +| unit-tests | Cancelled at the 45-minute limit with 63% run. `test_custom_function_events.py` alone took 9 minutes (200 `gc.collect()` calls on a large heap). Of 48 failures before the cut, 29 are in `known.fail`; 18 pass locally on the merged tree; `test_declaration_selected_discovery.py::test_examplehuman_selected_source_uses_same_declaration_owner` fails on main `44980dfe2` too and is not in `known.fail` | job time: B6 (split unverified); the test: unowned, needs an owner | +| official30-headless-parity (macOS 3.14) | ExampleHuman `cells/zernike_*` values differ from the native reference; Linux 3.12 and 3.14 pass | parity owner (Zernike numerics off Linux) | +| official30-headless-parity (Windows 3.14) | ExampleImagingFlowCytometryObjectsInGrid `ssc/zernike_phase_*` values differ | same | +| official30-parity-required | Aggregates the two above | same | +| desktop-candidate-live-mcp and desktop-installer-source-test (macos-15-intel) | Cold function-catalog preparation runs past the smoke's 300-second deadline (it passed once at about 200 s) | owner decision: performance work or the budget | + +### Unverified because CI stalled + +- The E1 boundary fixes (`7379deb90`, `f0498a630`); locally the four integration tests and the GUI journey pass. +- The three-group unit job (`542f7ffeb`). The earlier xdist attempt (`f8062d2cd`) lost a worker in `test_mcp_server.py` under xcb and was replaced. +- The Website per-ref concurrency group (`ce529e459`). +- The third documentation round and re-pins after `44980dfe2` and `625150118`; `validate_docs.py` passes locally on every target and the OpenHCS Sphinx `-W` build passed locally after round two. +- Windows-only fixes confirmed by one run each but not on this head: read-only probe (`c74b2bd55` never ran on Windows), `.gitattributes`. +- An earlier CI-only segfault in `tests/unit/pyqt_gui` teardown under xcb was not reproduced or root-caused; it has not recurred since `f322e395b`. + +### Why 27 jobs failed after merging main `44980dfe2` + +**E1 left two boundaries half-migrated on main, and every job that runs a dataset or the GUI smoke hit one of them:** + +- `DatasetRunRequest.runtime_observation_export_scope` is an enum-view field, but the measured CLI passed the family class into it, and `RunDatasets` passed the enum member to `Session.observation_export_params`, which expects the family. Every `run_datasets` call failed: four integration tests in all 17 backend, boundary and OMERO jobs, the wheel MCP smoke and the GUI journey. +- `scripts/smoke_installed_gui.py` read `UiActionInvocationStatus.ACCEPTED`, which E1 replaced with the `ActionAccepted` class: all desktop installer and live-MCP jobs. +- The unit job failed separately on the xdist worker crash described above. + +Main's own run at `625150118` had not started, so "main is red the same way" is inferred from the code being identical on main, not observed. + +### Standing problem for the owner + +**The documentation audit gate goes red on every surface merge.** Each merge changes authority files, which invalidates about 125 pages; B6 re-reviewed them three times in one day. Unless each surface re-reviews its dependent pages in its own PR, Documentation will be red again after the next merge. + +### Also found, not fixed + +- `openhcs/agent/services/config_service.py` `_default_value_repr` prints family-class defaults as `` in the generated config reference. +- `openhcs/mcp/bootstrap.py` `--surface` help says the desktop surface hides headless tools; it does not. +- `scripts/check_basicpy_source.py` expects a `jax_func` decorator the module no longer uses. +- `polystore/atomic.py` `_try_acquire_lock` retries `PermissionError` until its 30-second timeout. +- About 20 pages still say "Plate Manager" where the window title is now "Datasets".