refactor(run): extract shared queued tail + state-init + load helpers (BE-3265)

[mattmillerai]

ELI-5

comfy run has two doors — one for your local ComfyUI server, one for Comfy
Cloud. Both doors did the exact same thing at the end after handing off a job:
write a little status file, start a background watcher, print the pretty "⏳
queued" panel, emit the machine-readable JSON line, and briefly tail the job.
That ~45-line "queued tail" was copy-pasted verbatim in both, and the cloud path
even repeated part of it a third time. Copy-paste drift had already started (the
two JSON envelopes had quietly diverged). This PR moves that shared tail into one
helper so there's a single place to change it, without altering what any user or
agent actually sees.

What changed

Pure refactor of comfy_cli/command/run/__init__.py — no behavior change. Three
small module-level helpers replace the duplication:

• _load_workflow_or_exit() — the byte-identical preloaded / _load_workflow_file
  prefix that opens both execute() and execute_cloud().
• _init_queued_state() — jobs_state.new → stash item_map → write,
  reused by the local async path, the cloud non-wait path, and (the third copy)
  the cloud --wait path.
• _emit_queued() — the queued tail: pretty status panel + watcher-failure
  warning + queued envelope + live state tail.

Each function keeps its own submit/wait logic; only the shared mechanics moved.

Preserving the public JSON envelope (the careful part)

The queued envelope is a public agent-mode contract, so field membership is
kept verbatim per target — the call site owns the field set, _emit_queued
just renders it:

• local → host, port, state_file, watcher_spawned (in that order)
• cloud → base_url, state_file (no watcher_spawned, by design)

Other deliberately-preserved divergences: _journal_run still fires earlier in
the local path and inside the block in the cloud path; the watch hint keeps its
--where cloud suffix on cloud; _spawn_watcher still gets host/port only on
local. Field order was preserved exactly, not just the set.

Tests

• Added two golden-envelope tests (test_run_json.py) that pin the exact
  queued field set for local and cloud, so the shared helper can never
  silently drop or add a key.
• Full suite green: 2575 passed, 37 skipped. The run suites
  (test_run.py + test_run_json.py) are 181 passed. ruff check +
  ruff format --check clean.

Judgment call — _prepare_workflow scoped down (please note)

The ticket also proposed a _prepare_workflow() over the full
load/convert/classify/dry-run/preflight prefix. I extracted only the genuinely
identical part of that prefix (_load_workflow_or_exit) and consciously did
not force a single shared helper over the rest, because the two prefixes are
parallel-but-divergent, not verbatim:

• object_info source differs (live server fetch_object_info vs cloud snapshot
  _load_from_target(mode="cloud"));
• UI-conversion error codes differ (cloud adds cql_no_graph; hint text differs);
• classify semantics differ (local splits empty vs invalid; cloud collapses
  both to workflow_not_api_format, and pops compose_meta unconditionally);
• print_prompt differs (local return + emits preview only in the earlier
  prompt_preview event; cloud emits its own prompt_preview event then
  raise typer.Exit(0));
• partner-node credential preflight is local-only.

A unifying helper would be a where-branching tree that's larger and riskier
than the duplication it removes, against the "small, conservative" guardrail. The
headline of the ticket — the verbatim ~45-line queued tail — is fully extracted;
the prefix is addressed for its truly-shared slice. Happy to take the fuller
prefix extraction as a follow-up if a reviewer prefers it.

Self-review

• No capability is denied and no dead-end/throw path is added — this is a pure
  extraction (negative-claim falsification trigger does not fire).
• The local --wait completed path was intentionally left untouched: it
  writes state once as completed, so routing it through _init_queued_state
  (which writes once as queued) would add a second write — a behavior change.

CI status

The tomlkit CI break (previously flagged here) is resolved on main. The one
red check this PR had — test_queued_envelope_field_set_local_golden patching
a stale HTTP target (comfy_cli.command.run.request.urlopen instead of
comfy_cli.http._AUTHED_OPENER.open, per skishore23's review) — is fixed.
test_run.py + test_run_json.py = 295 passed; full suite green;
ruff check + ruff format --check clean on the changed files.

test / "Windows Specific Commands" (ImportError: cannot import name '__version__' from 'pydantic_core') is repo-wide and unrelated to this diff;
tracked as BE-3289.

[coderabbitai]

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in: 5 seconds

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 3df7a03c-e111-45d7-894c-d4688a613de5

📥 Commits

Reviewing files that changed from the base of the PR and between 87f0414 and dc1f842.

📒 Files selected for processing (2)

• comfy_cli/command/run/__init__.py
• tests/comfy_cli/command/test_run_json.py

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch matt/be-3265-dedup-run-queued-tail

✨ Simplify code

• Create PR with simplified code
• Commit simplified code in branch matt/be-3265-dedup-run-queued-tail

---

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

🔍 Cursor Review — Consolidated panel

Triggered by @mattmillerai.

✅ No high-signal findings.

Panel: 6/8 reviewers contributed findings.

Reviewers that did not contribute: kimi-k2.5:adversarial (empty), kimi-k2.5:edge-case (empty)

[mattmillerai]

Triage pass — no code changes were needed; both red checks are external to this diff.

Merge conflict: none, PR is MERGEABLE against main.

Review feedback: nothing outstanding. The Cursor panel returned "✅ No high-signal findings" (6/8 reviewers contributed), and there are zero open review threads. CodeRabbit did not review (label-gated: "Review skipped: no matching review label", plus a rate limit).

CI — both failures are pre-existing and reproduce on unrelated branches:

• build / "Run pytest" — 9 failures in test_config_parser.py / test_node_init.py, all ValueError: Comment cannot contain line breaks. tomlkit is unpinned in pyproject.toml and CI installs via pip install -e ., so it resolves tomlkit fresh; tomlkit 0.15.1 now rejects multi-line comments. This is exactly what fix(registry): emit pyproject hint blocks as single-line tomlkit comments #533 fixes. Against the locked tomlkit 0.13.3 those 83 tests pass locally, and fix(registry): emit pyproject hint blocks as single-line tomlkit comments #533 already shows build green.
• test / "Windows Specific Commands" — ImportError: cannot import name __version__ from pydantic_core during the ComfyUI install step. Failing on the last 6+ runs across every branch, including fix(registry): emit pyproject hint blocks as single-line tomlkit comments #533. Repo-wide and previously untracked; I have proposed a follow-up ticket for it (likely an unpinned mixpanel that now hard-depends on pydantic).

So this PR should go green on build once #533 lands and this branch rebases; the Windows check is red fleet-wide and not a signal on this PR.

Self-review: re-read the extraction against all three call sites. Order of operations is preserved in each path (state write → _spawn_watcher → panel → emit → tail; _journal_run still fires earlier on local and inside the block on cloud), and the queued envelope field set and order stay verbatim per target — now pinned by the two new golden tests. _emit_queued re-fetches the renderer via get_renderer() rather than taking the callers; that is the same module-level singleton both execute() and execute_cloud() already obtain from, so it is the identical object. test_run.py + test_run_json.py = 181 passed; ruff format --check clean (the 14 local UP038 hits are pre-existing in unrelated files under a newer local ruff than CIs).

One note for the reviewer: the 287-line uv.lock hunk is additive-only and not part of the refactor — #490 added the bench / anthropic>=0.40 extra without re-locking, so uv lock --check currently fails on main. This branch carries the regenerated lock (adds only the anthropic tree, touches no existing package or version). It has no effect on CI, which never reads uv.lock. I have documented this in the PR description; happy to split it out if preferred.

[mattmillerai]

🤖 The reviews loop filed Linear follow-up ticket(s) for review thread(s) deferred as out of scope for this PR:

• https://linear.app/comfyorg/issue/BE-3289|BE-3289 — Fix repo-wide Windows CI failure: pydantic_core ImportError during ComfyUI install

[skishore23]

The refactor itself is good and I'd like it in — but one of the new golden tests is broken, and it's the only thing failing CI right now. One-line fix.

Blocker: test_queued_envelope_field_set_local_golden patches a stale target

test_run_json.py:309 patches:

patch("comfy_cli.command.run.request.urlopen") as mock_open,
mock_open.return_value.read.return_value = json.dumps({"prompt_id": "pL"}).encode()

Every other test in this file patches comfy_cli.http._AUTHED_OPENER.open — I counted ~15 of them, including the sibling test_no_wait_emits_prompt_preview_then_queued 20 lines above. run/__init__.py:15 even flags from urllib import request as a legacy patch target (# noqa: F401 — patch target for tests), but submission now goes through comfy_cli.http, so the patch never intercepts and the test makes a real connection:

{"ok": false, "error": {"code": "connection_error",
 "message": "Failed to submit workflow: [Errno 61] Connection refused"}}

Fix (verified — the whole file goes to 98 passed):

patch("comfy_cli.http._AUTHED_OPENER.open") as mock_open,
mock_open.return_value.__enter__.return_value.read.return_value = json.dumps({"prompt_id": "pL"}).encode()

Note both halves are needed: the opener is used as a context manager, so __enter__.return_value.read is the right attribute (swapping only the patch target still fails).

This is the same stale-HTTP-target class as #521 — worth a quick grep across the other open PRs in this stack for request.urlopen / urllib.request.urlopen patch targets.

The CI triage in the description is out of date

build / "Run pytest" — 9 failures in test_config_parser.py / test_node_init.py, all ValueError: Comment cannot contain line breaks … Fixed by #533

That was true when written, but the tomlkit fix has since landed. The current build run (2026-07-30) reports:

FAILED tests/comfy_cli/command/test_run_json.py::TestSuccessfulRun::test_queued_envelope_field_set_local_golden - assert 1 == 0
===== 1 failed, 3649 passed, 36 skipped =====

So build is red because of this PR's own new test, not a pre-existing issue. Everything else in the suite is green — which is actually good news for the refactor.

(Separately, GitHub currently reports this PR as CONFLICTING/DIRTY. It merged cleanly for me against current main with no conflict markers, so that metadata looks stale — a rebase should clear it.)

The refactor itself checks out

With the test fixed, all 98 pass, and I agree with the substantive judgment calls:

• Keeping the queued envelope field set owned by the call site rather than the helper is the right shape for a public agent-mode contract — and pinning order, not just membership, is the detail that makes the golden tests actually load-bearing.
• Scoping _prepare_workflow down was correct. The five divergences you list (object_info source, UI-conversion error codes, classify semantics, print_prompt behavior, local-only partner preflight) are real, and a where-branching unifier would trade duplication for a worse conditional. Right call not to force it.
• Leaving the local --wait completed path alone is a genuinely sharp catch — routing it through _init_queued_state would have turned one completed write into a queued write plus a second write. That's the kind of thing a "pure refactor" quietly breaks.
• uv.lock is no longer in the diff, so that section of the description can go too.

Push the one-line test fix and I'll approve.

[mattmillerai]

Pushed the one-line fix (ce45f62): test_queued_envelope_field_set_local_golden now patches comfy_cli.http._AUTHED_OPENER.open with the context-manager __enter__.return_value.read shape, matching every other test in the file. Also rebased onto current main (resolved a real merge conflict in _load_workflow_or_exit — main's concurrent checkpoint_user_set addition to the preloaded tuple needed threading through the new helper) and trimmed the two now-stale PR description sections (uv.lock is no longer in the diff; CI status updated). Full suite green (295 passed in the run suites), ruff check/ruff format --check clean on the changed files. @skishore23 ready for another look.