Skip to content

Surface submission failure diagnostics to agents - #76

Merged
msaroufim merged 1 commit into
mainfrom
agent/surface-submission-diagnostics
Jul 29, 2026
Merged

Surface submission failure diagnostics to agents#76
msaroufim merged 1 commit into
mainfrom
agent/surface-submission-diagnostics

Conversation

@msaroufim

Copy link
Copy Markdown
Member

What changed

  • Parse owner-visible diagnostics returned for public submission runs.
  • Print structured failure rows plus compiler/runtime stderr in popcorn submissions show.
  • Surface the same diagnostics from plain popcorn submit --no-tui polling, including public failures during leaderboard submissions.
  • Return a nonzero CLI exit status when a completed public test or benchmark run failed.
  • Preserve secret-run redaction in every formatter.

Why

A benchmark exception could complete the KernelBot job while failing the submitted kernel. Popcorn previously printed only run metadata in the history view and returned exit code 0 for direct test/benchmark failures, leaving autonomous agents without either the traceback or a reliable failure signal.

Impact

Autonomous loops can now consume the actual Python traceback or compiler error directly from the CLI and branch reliably on the process exit status. Existing stored submissions also become diagnosable after the API dependency is deployed.

Depends on gpu-mode/kernelbot#524.

Validation

  • cargo test — 50 passed
  • cargo fmt --check
  • cargo clippy --all-targets -- -D warnings

@msaroufim
msaroufim marked this pull request as ready for review July 29, 2026 15:57
@msaroufim
msaroufim merged commit 6f56863 into main Jul 29, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant