Repository navigation
Reset child signal ownership to preserve parent ZMQ endpoints - #160
Conversation
Dependency review: a second reproducible unbounded-wait path belongs in the investigationReviewed plan head The plan correctly avoids inventing a missing FAILED branch. There is another concrete failure mode in the dependency beyond the already-noted alive-process exemption: A provider-free reproduction using the actual Proper ownership: let the declared status snapshot boundary determine whether a usable status observation succeeded, and reset the waiter error state only after that decode/admission. Keep terminal execution state on the existing lifecycle/record owner; a successful socket receive or a living PID is not successful observation of a job. Avoid a benchmark-local timeout/status cache as the repair (BOUND-1/2, identity of transport/process versus job observation). Add this malformed-status case to the dependency-focused failure probes before the end-to-end injected worker failure. Preserve the separate healthy-long-job, cancellation and lost-process cases. The eventual implementation belongs with ZMQRuntime's waiter policy, followed by the OpenHCS integration test and validated gitlink, not another polling algorithm in OpenHCS. |
Assigned waiter finding implemented in paired dependency draftNamed owner: H003/Codex waiter worker. Paired ZMQRuntime PR6, head Production change is confined to Real-decoder bounded provider-free no-sleep tests reproduce the base failure: 4 failed/10 passed, malformed polls repeatedly logged attempt1/3. Repair:14 passed. With five existing waiter/client controls:19 passed,52 deselected (latest0.18s). Healthy long job, progressing retries, known-live timeout tolerance, cancellation, failure, server error and lost-process diagnostics remain covered. Receipt: This fixes the additional reviewed accounting path, not a proven cause of the originally reported issue140 hang; do not close issue140 solely on these tests. Draft/source-validated only, not merged/installed/live verified. CI was queued at exact pushed head; no CI result is claimed. Coordination: parent owns #159 ViewerStateManager reuse/admission/acquisition and shared gitlink acquisition, plus separate #218 published metadata write policy; these are disjoint from this wait_policy change. Existing PR160 remains the OpenHCS integration umbrella; no competing OpenHCS implementation was created. No frozen installed submodule, environment or scientific artifact was modified. Actual ordinary-ZMQ/OpenHCS injected-failure integration is deferred under Confucius's runtime lock until parent schedules the slot. |
Public cancellation of real fork workers can remove the live parent server's IPC endpoints. Fork children inherit
serve_forever's SIGTERM/SIGINT handler, which captures the server and callsserver.stop(). Terminating those children runs copied server cleanup: the parent process remains alive, but fresh clients cannot reach it.Existing
WorkerExecutorResourcesnow owns child signal initialization. Both the actual fork entry and existing pooled-process initializer reset SIGTERM/SIGINT to SIG_DFL. Parent/inline/thread execution does not call it. Fork catalog, kernels, logging, environment, progress and resources stay inherited; no spawn preparation, cancellation-specific branch, timeout change or extra status authority is introduced.Closes #140.
Validation completed: 38 existing lane/resource controls pass, including a real fork/SIGTERM regression that checks child defaults, actual signal exit, no parent cleanup invocation and unchanged parent handlers. Genuine ordinary diagnostics on clean
56c3da776with installed ZMQRuntime0.4.1 establish original fork failure propagation, finite ten-second healthy work, real server-process loss and recovery from transient transport/malformed status observations. They are correctness diagnostics, not pipeline timing measurements.The causal cancellation packet
actual-cancel-ipc-witness-v4retains exact original server/worker incarnations, inherited handler names and both IPC inodes before cancellation; both paths disappear afterward while the original server is alive. Original waiter and terminal progress agree on CANCELLED. Earlier observer/protocol-scope mistakes and negative packets remain retained. Actual candidate cancellation and same-server recovery are qualified below.Historical investigation and current exact evidence are in
docs/plans/issue140_execution_failure_receipt_20260929.md; private artifacts are under/home/ts/.local/state/openhcs-maintenance/20261006/issue140-fork-acceptance-preparation-v1. The original failure's lifecycle/finalization work is already merged in ZMQRuntime21/OpenHCS697, and malformed status accounting in ZMQRuntime6. This change repairs the separately reproduced inherited-signal defect; it does not add another FAILED path or claim to reconstruct the original reported hang's exact cause.Actual candidate qualification: normal native extension construction passed in the owned worktree, without copied extensions or installed-environment changes. On clean source
b0cf6e9cb, public cancellation kills two genuine fork workers; the original waiter, terminal progress and a fresh attached status client agree on CANCELLED. Both IPC addresses retain their original inodes while the original server remains alive. A new ordinary compile request then completes on the same server, and the exact newly returned artifact ID feeds a new execution which completes finite ten-second work in two new fork children. All four children have SIGTERM/SIGINT defaults; all five owned process incarnations exit after normal disconnect and both runtime leases are free.The actual v3 helper remains terminal 1 because its private observer passed a frozen nested progress mapping to a wire-dictionary parser after saving the real COMPLETE outcomes. Separate offline acceptance passes using existing
ExecutionProgressObservation.terminal_status_response(), checking exact IDs/statuses/PIDs, both IPC inodes, child defaults, ten-second healthy work, the original failed cancellation CSV, and cleanup. Every original packet SHA remains unchanged; no terminal status was fabricated and no additional runtime was needed. Earlier negative fixture attempts (consumed artifact reuse and compile-only plus artifact ID) are preserved explicitly. The concrete endpoint defect is closed; the original reported hang's exact historical cause remains unclaimed. This is correctness acceptance, not performance evidence.