Skip to content

Wait for terminal progress callbacks before completing client execution - #19

Merged
trissim merged 1 commit into
mainfrom
fix/terminal-progress-delivery-20261004
Oct 4, 2026
Merged

trissim merged 1 commit into
mainfrom
fix/terminal-progress-delivery-20261004

Conversation

@trissim

@trissim trissim commented Oct 4, 2026

Copy link
Copy Markdown
Contributor

A terminal STATUS reply can arrive before queued progress callbacks finish. In the retained OpenHCS Speckles run, execution and all three CSV exports succeeded, but disconnect discarded the final step/export/axis notifications and the worker-coverage check failed.

ExecutionRecord now owns the sequence of admitted progress and its latest detached event. Registration establishes a producer baseline before the listener starts. Each published event carries that sequence, and wait_for_completion waits on the existing subscriber condition until callbacks finish through the terminal record’s watermark. FAILED and CANCELLED receipts retain their actual record too. Socket polling wakes immediately for available events while keeping listener shutdown bounded.

Late subscribers start at the registration baseline without fabricated historical callbacks. Missing tails time out; sequence gaps and callback failures produce explicit delivery errors, preserved across reconnects. This detects PUB loss; it does not add a journal or guarantee replay, and it does not change the server execution outcome. No additional STATUS request, sleep-based completion delay, or coverage waiver is introduced.

Validation: 79 passing tests across test_execution.py, test_progress_stream.py, and test_messages.py, including blocked callback tails for COMPLETE/FAILED/CANCELLED, late registration, missing/gapped events, callback failure, reconnect error retention, and existing listener ownership. git diff --check passes. Existing repository lint debt remains outside this change. The original scientific workload has not been rerun with this patch yet.

Closes #18
Refs OpenHCSDev/openhcs#573
Refs OpenHCSDev/openhcs#394

Copilot AI balanced review requested due to automatic review settings October 4, 2026 07:19

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@trissim
trissim merged commit 04d813f into main Oct 4, 2026
1 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Terminal completion can discard queued progress callbacks

2 participants