Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
9ac6b39
fix(video): only enable videocomposer wraparound for cues that repeat…
ibiltari Oct 1, 2026
6dad8cc
test(follow): the controller's run counter and follow relay (869fc8yt…
ibiltari Oct 6, 2026
16a6d6a
feat(follow): the controller relays cross-node Auto follow messages (…
ibiltari Oct 6, 2026
fd52615
test(follow): node side of the cross-node Auto follow (869fc8ytz) (red)
ibiltari Oct 6, 2026
8236720
fix(follow): an Auto follow into a cue on another node starts it (869…
ibiltari Oct 6, 2026
2c358f4
docs(follow): the cross-node Auto follow relay in CLAUDE.md (869fc8ytz)
ibiltari Oct 6, 2026
a90adf9
test(audio): wire a player only once it reports RUNNING; check and re…
ibiltari Oct 6, 2026
58315af
fix(audio): wire an audio player only once it reports RUNNING (869fby…
ibiltari Oct 6, 2026
a4eeebc
fix(audio): check and repair a player's wiring at GO edge by edge (86…
ibiltari Oct 6, 2026
58869e6
fix(arm): say whether a GO waited for an arm in progress or fell back…
ibiltari Oct 6, 2026
bd8381b
docs: audio wiring waits for the player's RUNNING line and repairs ed…
ibiltari Oct 6, 2026
d749925
test(arm): make the arm-wait message tests self-contained (869fbyjzx)
ibiltari Oct 6, 2026
ee3b060
docs(audio): state the known limit and plan B of the ready gate (869f…
ibiltari Oct 7, 2026
9d23f88
fix(audio): review fixes for the ready gate (869fbyjzx)
ibiltari Oct 7, 2026
0118d1e
style: black (the CI lint is black --check, not ruff format) (869fbyjzx)
ibiltari Oct 7, 2026
7ff98d5
Merge pull request #27 from stagesoft/fix/869fbyjzx-audio-ready-gate
ibiltari Oct 7, 2026
df3e796
test(status): cue names on the :9190 channel — red (869fedahu)
ibiltari Oct 8, 2026
5ffb1b8
fix(status): broadcast cue names after load on the :9190 channel (869…
ibiltari Oct 8, 2026
ce41208
test(orphan-kill): sweeps never abort a load and match only own playe…
ibiltari Oct 8, 2026
674aefb
fix(orphan-kill): an orphan sweep never aborts its caller (869evtdf7)
ibiltari Oct 8, 2026
32dae0d
fix(orphan-kill): sweep only own-user processes of the known binaries…
ibiltari Oct 8, 2026
f57f154
fix(comms): log the command callback traceback (869evtdf7)
ibiltari Oct 8, 2026
6ae3291
fix(load): the zombie JACK client cleanup never aborts a load (869evt…
ibiltari Oct 8, 2026
0d197f3
docs(orphan-kill): sweeps never abort a load and depend on the binary…
ibiltari Oct 8, 2026
81058b7
Merge pull request #29 from stagesoft/fix/869evtdf7-orphan-kill-eperm
ibiltari Oct 8, 2026
bee4918
Merge pull request #23 from stagesoft/fix/869fa89uh-video-loop-wrap
ibiltari Oct 8, 2026
1e36548
Merge branch 'rc_1' into fix/869fc8ytz-cross-node-follow
ibiltari Oct 8, 2026
173301e
test(audio): count only the test thread's sleeps in the AudioMixer wa…
ibiltari Oct 8, 2026
68af3d1
Merge pull request #30 from stagesoft/fix/869fbyjzx-test-sleep-thread…
ibiltari Oct 8, 2026
4154318
Merge branch 'rc_1' into fix/869fc8ytz-cross-node-follow
ibiltari Oct 8, 2026
f68eaee
Merge pull request #25 from stagesoft/fix/869fc8ytz-cross-node-follow
ibiltari Oct 8, 2026
98866d7
Merge branch 'rc_1' into fix/869fedahu-cue-names-broadcast
ibiltari Oct 8, 2026
b6d8570
Merge pull request #28 from stagesoft/fix/869fedahu-cue-names-broadcast
ibiltari Oct 8, 2026
44d1a46
Merge branch 'rc_1' into debian/bookworm (release/0.1.0rc6-3)
ibiltari Oct 8, 2026
e71aca8
release: cuems-engine 0.1.0rc6-3
ibiltari Oct 8, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,11 @@ Terminology: **body / duration** = media playback length (`media.duration` for A
- **A stop takes its scheduled chain with it.** `stop`/`pause`/`fade_out` actions call `cancel_pending_descendants`: walks `_target_object` past the target, cancelling every local+enabled cue that has **not** produced output yet (`_revealed`), transitively, stopping at a chain break. A cue already on stage is never touched and the walk continues past it. Each node runs it on its own graph (ActionCues are local everywhere). Cancelled cues also get `remove_cue` — their threads exit through the generation guard, upstream of the normal one.
- **Enable/disable mid-pass are honoured.** Nothing downstream re-reads `enabled` once a cue is dispatched, so: disabling a parked cue **cancels** it (`cancel_parked`), and enabling one **rejoins** it (`rejoin_chain`) at the anchor stamped on it when the walk skipped it (`_chain_pass`), dispatched with `unroll=False` so the running chain is not re-dispatched. Refused if its slot already passed (a loud journal line — firing late by surprise is worse), if a STOP ended that pass, or if a stop action already claimed it.
- **Auto-follow / auto-pause postwait = an MTC-gated tail after the body**: when `loop_cue` returns, the engine blanks the video (`blank_cue`: `/visible 0`, cue stays armed → STOP-reachable), waits until `body_end + postwait` on the MTC timeline, then ends illumination (`remove_cue`) and — for follow — fires the target via `go_from`. A manual GO during the tail **preempts** the auto-fire. Follow fires at `body_end + postwait`, not at media end.
- **An Auto follow into a cue on another node is relayed** (`cues/FollowRelay.py`, ClickUp 869fc8ytz). Only the nodes where the source is local run it and fire its follow, so a target on another node used to never start, silently. For an **AudioCue or VideoCue** source (ActionCue, FadeCue, CueList and DmxCue are local on every node, so every node already fires their follow) the owning node **announces** the follow instant (`_follow_instant_ms`: the same expression its own fire uses) right after the reveal, or at the start of the last loop for `loop > 1`; never for an endless loop. The controller relays it to every node (`_relay_follow`); the others pre-arm what they would start, wait for the instant on MTC and dispatch it there through `go_from`. If the follow does not happen the owner **cancels**: reason `stop` (stop/pause/fade-out action), `restart` (the source was started again: `go()`'s commit section sends it, the replaced thread notices too late), `error`; nothing for a STOP, a load or the yield, which every node handles itself. A cancel that reaches a target already on stage stops it only within `FOLLOW_CANCEL_GRACE_S` (2 s) and only for `stop`/`restart` -- a cue is never cut for an internal error.
- **Run counter.** The controller's `run_seq` (from the clock) travels in every GO; STOP, load and unload bump it before forwarding. A node records it, and empties its pending follows, in the **receive hook** (`NodeCommunications.set_receive_hook`), i.e. in bus order, before the command threads race for `_command_lock`. Follow messages are applied there too and never take that lock. A message of another run is dropped at the controller and at the node. Between a STOP/load's receipt and the moment the node has run it (`run_command` marks it applied), the node's epochs still belong to the ending run: it announces nothing and reads no epochs for incoming follows until then.
- **`go()` has a `commit_gate`**, evaluated inside the commit section after every other refusal: the follow's dispatch and its cancel are decided in one place. Order inside that section: refusals → gate → restart cancel → assignments.
- **`disarm_if_unchanged`** is the one other place that may disarm a published cue from a background thread (the stopped follow target): it re-checks generation, STOP/load epoch and arm stamp in one `_lock` section and releases the player while holding the cue's **arm claim**. Extend it, don't bypass it (publish-or-abandon contract, 869f9wqpn).
- Version skew: an old controller sends no `run_seq`, so nothing is announced (the bug stays); an old node logs `Command follow not found` for every relayed message. Upgrade the whole cluster together.
- `prewait` is applied at exactly one point (`start = arrival + prewait`), never as a wall-clock sleep. The postwait `sleep` in `go_threaded` is **auto-continue only** and no longer paces anything (the chain left at the trigger): it holds the thread past the body when `post > body`, and **it is the sole mechanism producing auto-continue's `prewait + max(body, postwait)` illumination** — `loop_cue` never reads `postwait` and `remove_cue` fires only after it returns — so removing it would silently shorten the highlight to `prewait + body`. Guarded by a test.

Cross-cutting invariants: **never auto-stop a running project**; `_reveal_wait` exits on `_stop_requested` (STOP) or a changed `_go_generation` (newer GO/reload) and does **not** bail on a recoverable MTC stall (reveal fires late on resume); anchor comparisons use wrap-accumulated `milliseconds_exact` (24h-safe). See `Plans/postwait-postgo-chain-semantics.md` and `Plans/postwait-engine-only-implementation.md` in the cuems-RELATIONS index repo for the full design/diagnosis.
Expand All @@ -100,7 +105,8 @@ Cross-cutting invariants: **never auto-stop a running project**; `_reveal_wait`

Engine → VideoComposer and engine → DmxPlayer are inter-component protocols documented once in the cuems-RELATIONS CLAUDE.md ("Video/DMX Cue Lifecycle"). The engine side: arm → send load/`/frame` bundle → run (set offset/visible/mtcfollow, or scene bundle) → reveal on MTC. See the videocomposer and dmxplayer CLAUDE.md for the player halves.

- **Audio mixer wiring waits for the player's JACK port** (`AudioMixer.connect_player_to_outputs`, ~15 s `port_exists` loop): a fresh `cuems-audioplayer` does OSC bind + RtMidi + FFmpeg probe before RtAudio registers `outport 0/1`, so connecting immediately after spawn hits jackd "Unknown source port" and the cue plays **green but silent** (2026-07 isil multilingual incident). The function returns success/failure; `new_audio_output` and the GO-time repair in `run_cue` log a loud "will be SILENT" error on failure.
- **Audio mixer wiring waits for the player to report itself started** (`AudioMixer.connect_player_to_outputs`, `ready=Player.ready`, 869fbyjzx). A fresh `cuems-audioplayer` does OSC bind + RtMidi + FFmpeg probe, then registers `outport 0/1` (`openStream`), then RtAudio **auto-connects them to `system:playback_*`** (`startStream`), then finishes its constructor and prints `[OK] RUNNING!` (`main.cpp`); the stdout reader in `Player.call_subprocess` sets `Player.ready` on that line. The wiring waits for it, then disconnects whatever the outports are connected to (the auto-connects) and connects the mixer. Wiring on port visibility alone (before 869fbyjzx) either hit jackd "Unknown source port" (green but **silent**, 2026-07 isil) or ran before the auto-connect, which then stayed: the cue sounded through the mixer **and** straight to the outputs, doubled and outside mixer volume/mute (869fcvz85, ~3.5 % of arms). One 14.5 s budget for the ready wait and the port wait; a dead player ends it; a player that never prints the line is wired on port presence with a WARNING. **At GO** `run_cue._verify_audio_wiring_at_go` checks with `player_connection_diff` (missing + stray edges, one `get_connections` per outport) and repairs **edge by edge** (`repair_player_connections`) — never a full rewire, because a GO can reach a cue that is still playing. Failures log a loud "will be SILENT" error (or "stray edge(s) still connected"). **Known limit:** the arm still takes ~0.47 s cold (0.35 s warm; the player's own start-up), so a GO that reaches the cue sooner than that after its arm began waits and fires LATE (test2, node audio behind another node's pause: second GO at 0.3 s → ~0.2 s late; 0.5 s and later → on time). Only arming **before** the hand-off removes it — **plan B / Part B**, not built: arm the first local segment at load even behind another node's pause and the next after each GO (measured cost ~15 MB per held audio player, ~40 MB GPU per held 1080p layer; or pre-launch players). Deferred because no real show hits it (sala1's node segments are video only); revisit if a venue reports a late node audio cue after a fast double GO. See `Plans/2026-10-06-engine-audio-arm-port-wait.md` (cuems-RELATIONS).
- **Orphan sweeps never abort their caller and see only our own players** (869evtdf7). `kill_orphaned_audio_processes` runs in the middle of `_load_project_inner` and `kill_orphaned_mixer_processes` before the mixer starts; an EPERM from either used to abort the load half-way through its teardown (old project torn down, new one never armed, only `Error executing command callback for load: [Errno 1]`) or leave the node without a mixer. Both now go through module-level helpers in `PlayerHandler.py` (`_orphan_pids`, `_kill_orphan`) that log and go on: EPERM names the PID and its owner, pgrep rc ≥ 2 is a WARNING, not silence. `cleanup_zombie_jack_clients` is guarded the same way, and `run_command` logs a callback failure with its traceback. They match **own-user** processes only (`pgrep -u <euid>`), the audioplayer one anchored on argv[0] (`^([^ ]*/)?cuems-audioplayer( |$)`), so an operator's `tail -f …cuems-audioplayer.log` is no longer killed. ⚠ The cleanup now depends on the **binary name**: a configured `audioplayer`/`audiomixer`/`dmxplayer` path with another basename escapes these sweeps **and** the node-engine unit's `pkill` (anchored the same way in cuems-common ≥ 1.3.0-24); only the cgroup kill of a `systemctl restart` remains. `set_players` warns at start when that happens (`check_orphan_binary_names`). See `Plans/2026-10-07-engine-orphan-kill-eperm-aborts-load.md` (cuems-RELATIONS).
- **`enable`/`disable` actions arm/disarm their target** via `NodeEngine._action_result_sink` → `_apply_cue_enabled_side_effects` (same helper as the Controller's `cue_enabled` toggle): enable spawns an async `ReArm:<id>` (`_arm_with_enabled_guard` re-checks enabled + project generation after the arm); disable disarms only when the cue is idle — guarded by the `_playing` lifecycle flag (set by `go()`, cleared by `disarm()`/`stop_all_cues()`; `_go_generation` is increment-only and NOT a playing signal). Handlers themselves only flip the flag. Without this, runtime-enabled cues always hit the go()-time fallback re-arm (inline load at GO). CueList targets react on their first enabled child; note CueList children have **no** go()-fallback of their own (run_cueList bypasses `go()`).
- **FadeCue duration must be > 0** — invariant enforced at the top of `_build_fade_payload` (`ActionHandler.py`): duration `None`/zero/non-numeric raises `ValueError` → clean `failed` action result at reveal, **before** any OSC dispatch, `record_value` mirror write, or `_start_mtc`/`_end_mtc` stamping. Rationale: parser-built FadeCues bypass the cuemsutils setter, and gradient-motiond silently drops `dur <= 0` on its UDP input — without the guard a zero fade reported "applied" and poisoned the next fade's `start_value` via the mirror. XML load stays lenient on purpose (legacy projects with a zero still load; they fail loudly only at reveal). The editor rejects such durations at save time (cuems-editor `validate_fade_durations_in_contents`).

Expand Down
29 changes: 29 additions & 0 deletions debian/changelog
Original file line number Diff line number Diff line change
@@ -1,3 +1,32 @@
cuems-engine (0.1.0rc6-3) bookworm; urgency=medium

* Merge rc_1 into debian/bookworm:
- 869fa89uh (#23): a play-once video cue no longer wraps to frame 0 at
its end; videocomposer wraparound is enabled only for cues that
repeat.
- 869fc8ytz (#25): an Auto follow into a cue on another node starts
it; the controller relays the follow to the owning node. Every GO
value is now a dict carrying run_seq: the WHOLE CLUSTER must run
this engine (controller and every node).
- 869fbyjzx, 869fcvz85 (#27): an audio player is wired to the mixer
only once it reports RUNNING, so RtAudio's auto-connect to
system:playback_* is always removed (no more doubled audio outside
mixer volume); the GO-time check repairs the wiring edge by edge;
arm-wait log lines say whether a GO waited or fell back.
- 869fedahu (#28): the controller broadcasts each cue's name on the
:9190 channel after every load (/engine/status/cue_name/<uuid>) and
replays it to late clients. Additive; older UIs ignore it.
- 869evtdf7 (#29): orphan sweeps never abort a project load. An
orphaned player the node could not kill (EPERM) used to abort the
load after the old project was torn down; both sweeps and the zombie
JACK client cleanup now log and go on, match only own-user players
by argv[0], and warn at start about unknown player binary names. A
failing command callback is logged with its traceback.
- #30: AudioMixer wait tests count only the test thread's sleeps
(test-only).

-- Ion Reguera <ion@stagelab.coop> Thu, 08 Oct 2026 21:12:33 +0200

cuems-engine (0.1.0rc6-2) bookworm; urgency=medium

* Merge rc_1 into debian/bookworm (PR #22):
Expand Down
Loading