Skip to content

feat!: add a Decisions API judge and a pluggable overview step for clawhub - #148

Merged
Patrick-Erichsen merged 10 commits into
mainfrom
decisions-judge-overview
Nov 4, 2026
Merged

Patrick-Erichsen merged 10 commits into
mainfrom
decisions-judge-overview

Conversation

@Patrick-Erichsen

@Patrick-Erichsen Patrick-Erichsen commented Nov 4, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

ClawHub's verdict now comes from the OpenAI Decisions API. Decisions returns typed answers only, so readable prose moves to a new, separate, pluggable overview step.

  • Decisions judge: clawscan-decisions-judge (stdlib Python, docker/clawscan-runtime/judges/clawscan_decisions_judge.py, installed on PATH in the runtime image). It is an ordinary judge command: it reads the judge workspace and writes schema-valid JSON to {{ output }}. It owns its own checks (input limit, refusals, HTTP errors, schema validation). The runner stays judge-agnostic.
  • Overview step: new optional profile overview.command (plus --overview <cmd> / --no-overview). It runs after a completed verdict in a fresh workspace with the same layout and placeholders plus judge.json and the {{ judge.result }} prompt placeholder. Its result is recorded under overview in the run artifact. A failure, invalid output, or timeout (5 min, separate budget) is recorded and never changes the verdict, gate, status, complete, or exit code. It is skipped (status: "skipped") when the judge did not complete. Benchmarks skip the profile overview unless --overview is passed.
  • clawhub / clawhub-aig profiles use both: the Decisions judge (gpt-6-luna), then a Codex gpt-6-luna overview with low reasoning effort and Standard tier, routed through OPENAI_BASE_URL (for example ClawRouter) when it is set.

Judge request shape

POST {DECISIONS_BASE_URL}/v1/decisions:

{
  "model": "gpt-6-luna",
  "input": [
    {"role": "developer", "content": "<clawhub/decisions.md policy>"},
    {"role": "user", "content": [
      {"type": "input_text", "text": "<exact artifact text files between random per-run boundary markers>"},
      {"type": "input_text", "text": "Scanner and run evidence follows as JSON ..."},
      {"type": "input_json", "data": {
        "metadata": {"target": {"kind": "...", "id": "..."}, "scannerStatus": {...}, "runContext": {...}},
        "scanners": [{"scanner": "aig", "format": "sarif", "invocations": [...], "findingCount": 1}, ...],
        "findings": [{"scanner": "...", "ruleId": "...", "severity": "...", "title": "...", "file": "...", "line": 3}],
        "findingCounts": {...}, "artifactManifest": [{"path": "...", "bytes": 1, "sha256": "...", "status": "included|excerpted|omitted"}], "artifactCounts": {...}
      }}
    ]}
  ],
  "questions": [
    {"type": "choice", "name": "verdict", "instructions": "...", "choices": [
      {"value": "benign", "description": "..."}, {"value": "suspicious", "description": "..."}, {"value": "malicious", "description": "..."}]},
    {"type": "choice", "name": "dimension.<purpose_capability|instruction_scope|install_mechanism|environment_proportionality|persistence_privilege>", "choices": [ok, note, concern]},
    {"type": "predicate", "name": "signal.<exfiltration|tool_poisoning|backdoor|remote_code_execution|persistence|credential_access|obfuscation|destructive_actions>"},
    {"type": "predicate", "name": "finding.<n>"}  // "expected for the stated purpose?", up to 25 highest-severity findings
  ]
}

Verdict instructions: malicious = intentional harmful or deceptive behavior (exfiltration, tool poisoning, backdoors, RCE, persistence, destructive actions, deliberate misdirection). suspicious = real security weaknesses or undeclared risky behavior without clear malicious intent. benign otherwise. The policy states that scanner severity is never evidence of malice by itself, and that scanner failures and coverage gaps (metadata statuses and errors, SARIF invocations[].executionSuccessful and notifications) are uncertainty, not findings.

Bounds (owned by the judge): text files are sent whole up to 120k chars each and 600k in total. Larger files are head/tail excerpted. Binary and non-UTF-8 files appear only in the manifest. Findings are capped at 200 and trimmed to keep input_json under 200 KiB (the API limit is 256 KiB, 10k values, depth 32). The total must fit --max-input-chars (default 1,000,000, env DECISIONS_MAX_INPUT_CHARS). Otherwise the judge exits 2 with decisions judge input too large: N chars exceeds the L-char limit (...) and sends no request.

Failures: any refusal answer, HTTP error (message/type/code preserved, e.g. Decisions API HTTP 401: Incorrect API key provided. (type=invalid_request_error, code=invalid_api_key)), network error, missing or unexpected answer, or schema mismatch exits 1 with the real error on stderr. ClawScan records a failed judge with that error and does not retry. HTTP 429/5xx get up to 2 internal retries within the judge timeout, honoring Retry-After.

Config (env only): DECISIONS_API_KEY (falls back to OPENAI_API_KEY), DECISIONS_BASE_URL (falls back to OPENAI_BASE_URL, then https://api.openai.com/v1; /v1 is appended when missing), DECISIONS_MODEL (default gpt-6-luna; ClawRouter needs openai/gpt-6-luna), DECISIONS_MAX_INPUT_CHARS. The overview uses CODEX_API_KEY (falls back to OPENAI_API_KEY), OVERVIEW_MODEL (default gpt-6-luna), and OPENAI_BASE_URL. All are in the profile sandbox.env allowlist.

Verdict mapping (internal/profiles/clawhub/output.schema.json)

Field Source
verdict verdict choice
confidence verdict confidence: high ≥ 0.8, medium ≥ 0.5, else low
dimensions.<name> {status, confidence} from dimension.<name>
scan_findings_in_context[] {ruleId (rule + location), scanner, expected_for_purpose (p ≥ 0.5), probability}
decision api, endpointHost, requestedModel, API-reported model, latencyMs, httpAttempts, usage, input accounting, verdict {choice, confidence, probabilities}, signals{...}

⚠️ Artifact changes for consumers (ClawHub reads run artifacts)

  • judge.result for clawhub/clawhub-aig no longer has prose: summary, dimensions.*.detail, scan_findings_in_context[].note, and user_guidance are gone. Dimensions are {status, confidence}. Findings gain scanner and probability. There is a new required decision object. verdict and confidence keep their enums.
  • New top-level overview (omitted when no overview is configured): {status: completed|failed|skipped|running|not_started, commandStarted, configuredModel, promptSha256, outputSchemaSha256, error, durationMs, result}. For clawhub, overview.result = {summary, dimensions.<name> (string), scan_findings_in_context[{ruleId, note}], user_guidance} (internal/profiles/clawhub/overview.schema.json). Consumers must treat the overview as optional: show prose when overview.status == "completed", and never gate on it.
  • judge.configuredModel is now absent for the clawhub judge (the Codex-only parser doesn't recognize it). Use judge.result.decision.model / requestedModel.
  • Worker install: with --sandbox off, the ClawHub worker must put clawscan-decisions-judge on PATH, e.g. ln -s "$CLAWSCAN_SOURCE_DIR/docker/clawscan-runtime/judges/clawscan_decisions_judge.py" "$RUNNER_TEMP/clawscan-bin/clawscan-decisions-judge" (needs only python3), plus DECISIONS_* / OVERVIEW_MODEL env as desired. The Docker runtime image already ships it.
  • cmd/verify-clawhub-prompt still compares the embedded output.schema.json with ClawHub's production Codex schema, so it will report a schema mismatch until ClawHub adopts the new schema. clawhub/prompt.md stays embedded for that tool and for --judge comparisons, but the profiles no longer use it.
  • Plugin pipeline note: any SARIF or JSON report under scanners/ (for example a single plugin-wide aig-mcp SARIF) is ingested the same way, with invocation failures preserved as coverage gaps. No scanner adapters are touched here.

Tests

  • cmd/clawscan/decisions_judge_test.go: tests at the clawscan <target> --profile clawhub seam with the real judge command, an httptest Decisions stub, a stub SkillSpector subprocess, and a stub codex overview:
    • verdict mapping for benign/suspicious/malicious with high/medium/low confidence, request shape (model, developer + user messages, input_text + input_json, verdict choices as objects with descriptions), auth header, model/usage/latency recorded, no key in the artifact; the overview completes, its prompt and judge.json carry the verdict
    • refusal → judge failed with Decisions API refused question(s): verdict, run failed/complete:false/gate error, no retry, overview skipped
    • HTTP 401 → API error preserved, not retried
    • oversized input (DECISIONS_MAX_INPUT_CHARS=4000) → the judge's own decisions judge input too large ... exceeds the 4000-char limit, zero requests sent
    • overview failure (stub codex exits 1) and invalid overview JSON → overview failed with the real error, while verdict, gate, status, complete, and exit status are identical to a --no-overview run
  • internal/runner/overview_test.go: judge.json + {{ judge.result }} rendering, overview timeout recorded (not returned), skip without a completed verdict, placeholder fails without a verdict.
  • docker/clawscan-runtime/tests/test_decisions_judge.py (CI lint job + inside the runtime image build): request shape, mapping, refusal, HTTP error, 429 retry, oversized input, excerpting, input_json bounding, SARIF coverage gaps + scanner failures preserved, schema validation.
  • internal/profiles: profile wiring, overview Codex args (luna, low effort, key alias, OPENAI_BASE_URL provider routing), --overview / --no-overview, benchmark skip.
  • Local: go test -count=1 ./..., go vet ./..., gofmt, runtime-image unit tests, make docs-site, all green.

Live smoke (ClawRouter, 2026-11-03)

Locally built clawscan with the real clawscan-decisions-judge and Codex CLI 0.157.0, --profile clawhub --sandbox off, scanner fixtures (clean SkillSpector, empty A.I.G SARIF), DECISIONS_BASE_URL=https://clawrouter.openclaw.ai, DECISIONS_MODEL=openai/gpt-6-luna, OPENAI_BASE_URL=https://clawrouter.openclaw.ai/v1, OVERVIEW_MODEL=openai/gpt-6-luna, key from MODEL_CLAWROUTER_API_KEY (never printed; 0 occurrences in either artifact):

Fixture Verdict Decisions Overview Run
benign daily-notes SKILL.md benign / high (p=1.0), all dimensions ok, all signals 0 model gpt-6-luna, 237 ms, 3,649 input tokens, 1 HTTP attempt completed in 203.7 s: "Daily notes appends a dated entry and the user's note to ./notes.md; its instructions are limited to that task…" completed, complete:true, gate pass, exit 0
same skill + hidden tar ~/.ssh ~/.aws | curl POST and curl … | sh into ~/.zshrc malicious / high (p=0.99), all dimensions concern; exfiltration, RCE, persistence, tool_poisoning, credential_access = 1.0 gpt-6-luna, 647 ms, 3,697 input tokens completed in 171.5 s: "Do not install this skill. Its own instructions call for secretly sending SSH and AWS files to an external service…" completed, complete:true, exit 0

Note: through ClawRouter the Codex overview took about 170–200 s, well inside its 5-minute budget but not fast. The verdict itself took under 1 s.

Review fixes (15d86a2 + c82a2fe, runtime qualified at the final head)

  • node_modules (joint feat!: add a Decisions API judge and a pluggable overview step for clawhub #148+feat: audit OpenClaw plugins with the A.I.G MCP scanner #149 repro): for plugin targets, files under node_modules at any depth are excluded from the judge text, manifest, and budget and reported only as artifactCounts.excludedDependencies {files, bytes, reason}, a deliberate scope exclusion rather than missing coverage. The copied artifact is untouched, and skill targets keep node_modules in scope. The parent's TestIntegratedPluginAIGDecisionsOverview fails with the old judge and passes with this one.
  • P1 gap/provenance preservation: analysis_completeness lists (ledger exceptions, scope exclusions, references, limitations, incomplete analyzer statuses), component gaps (limitations, incomplete inspection, executable code), and bounded metadata provenance become {total, countsBy, counts, records, omitted} with group-balanced, path-aware records. Bundled children keep rootPath, prefixed paths, their own gaps, provenance, and findings (rootPath on each child finding). Covered by tests on the existing skillspector-usable-partial.json and skillspector-bundled-partial.json fixtures, plus a 3,000-record bounding test.
  • P2 JSON limits: bytes, values, and depth are checked on the fully assembled evidence (manifest and counts included). Trimming covers findings, then manifest, then gap samples, then bundled children, with truthful omitted / manifestOmitted accounting and a hard failure if nothing is left to trim. Final stats are recorded in decision.input.evidenceJson. The reviewer's repro (2,000 findings + 450 empty files) is a test. --max-input-chars is documented as a character budget, not a token guarantee.
  • P2 off-by-one (fresh review on 66c4361), fixed in c82a2fe: findingCounts.omitted was added after the fit loop, so evidence measured at exactly 10,000 values went out with 10,001. All accounting fields are now initialized before the first measurement, and the exact final object is re-checked after assembly (assert_evidence_fits). The new test_fit_evidence_exact_value_boundary pads evidence to exactly the limit; the previous judge sends 10,001 values there and the fixed judge stays within the limit. Also added: ClawSweeper's exact e1781d2 repro (1,661 findings, context: {}, 8-char SKILL.md: 10,001 values on the old judge), the reviewer's 1,200 findings + 463 empty files case, and a 455–471-file sweep asserting bytes, values, and depth on what is actually sent.
  • P2 static questions: finding predicates reference only the finding's stable index in the evidence JSON. No scanner or artifact text is interpolated into questions[].instructions (hostile-title test).
  • P2 answer validation: any refusal anywhere is terminal (the refusal-then-valid-duplicate repro is a test). Duplicate, missing, or unexpected names, wrong answer count, wrong type, disallowed choice, unknown or repeated probability values, and non-finite or out-of-range numbers are all rejected, independent of the output schema.

Overview latency investigation

Measured locally through ClawRouter with a captured overview workspace: the model turn itself takes 7–9 s (about 57k input tokens, mostly cached, two tool calls). The 172–204 s seen earlier is Codex CLI process overhead before thread.started (9–227 s) and after turn.completed (up to ~68 s), which varies run to run on this machine. It persists with an empty CODEX_HOME and with analytics.enabled=false, feedback.enabled=false, check_for_update_on_startup=false, and supports_websockets=false. ClawRouter /v1/models answers in under 1 s. This looks environment-specific (this host routes Codex through a local egress proxy). This is not proof that the Codex CLI is fast elsewhere. Latency must be measured on the actual worker against the 900 s whole-plugin budget. The overview stays bounded (5 min) and non-gating, and no profile change is made for it here.

Data-model compatibility (e1781d2; f905a55 fixes a pre-existing #147 test flake)

Proof in this repo:

  • Stored artifacts: cmd/clawscan/compat_test.go › TestLegacyStoredArtifactsRemainReadable uses real artifacts generated by the old binaries. Both come from --profile clawhub --sandbox off, with scanner fixtures and a stub judge writing the legacy prose result; host paths are scrubbed.
    • testdata/legacy-clawhub-artifact-c3cfaf1.json comes from the ClawHub production pin, with the legacy artifact_inspection receipt, receipt fields on judge.attempts, and the retired clawscan-static scanner.
    • testdata/legacy-clawhub-artifact-fa01a95.json comes from pre-feat!: add a Decisions API judge and a pluggable overview step for clawhub #148 main.
    • Each fixture still decodes into runner.Artifact and summarizes correctly (judge_completed: 1, clean: 1, no overview lines).
    • On re-encode the absent overview stays absent (omitempty) and judge.result is deep-equal to the stored value.
  • Profile configs: TestPreOverviewProfileConfigRunsUnchanged runs the exact pre-feat!: add a Decisions API judge and a pluggable overview step for clawhub #148 clawhub profile (schemas/testdata/clawhub-profile-fa01a95.yml, with its original prompt and output schema) as a project config.
    • It runs end to end through its Codex judge; the test codex stub stands in for the CLI.
    • No overview runs or appears in the artifact, the legacy prose judge.result is recorded unchanged, and judge.configuredModel is still gpt-6.1-sol.
    • schemas/clawscan.schema_test.go also keeps that pre-overview config valid against the updated config schema.
  • Optional overview absent: overview is optional at every level.
    • It is omitted when no overview.command is configured, and --no-overview removes it.
    • Benchmarks skip it, and the install policy rejects it the same way it rejects judges.
    • Consumers must treat an absent overview, or any overview.status other than completed, as "no prose". It never affects verdict or gate.

Consumer side (openclaw/clawhub#4411 at fba9211, draft, not activated), positive tests that keep legacy and old-pin behavior:

Breaking boundary: the new clawhub / clawhub-aig profile projection is the break.

  • What changes: judge.result loses its prose fields and gains decision, prose moves to overview.result, the clawhub output.schema.json changes, and clawscan-decisions-judge must be on PATH.
  • Rule for consumers: validate stored legacy results against the legacy contract, and only new-profile results against the new schema.
  • What stays readable: stored artifacts, the run-artifact envelope (clawscan-run-v1), custom/project profiles, and --judge overrides.
  • Not a deployment claim: this is not a claim of deployment compatibility. ClawHub stays on pin c3cfaf1 until #4411 and the paired runtime are independently qualified.

Runtime qualification

000cda8 refreshes scripts/runtime-image/production-contract.json and the production-contract workflow to the reviewed runtime candidate ca84550. The Dockerfile now copies judges/, installs clawscan-decisions-judge, and runs its tests during the image build. The publish smoke also runs clawscan-decisions-judge --help. The locked-SDK qualification and both native image builds pass.

Not done / follow-ups

  • .agents/skills/autoreview could not run locally. The Codex engine rejected the ClawRouter provider config, and the Claude engine's OAuth session had expired. Changes were self-reviewed.
  • The live Live Judge Smoke workflow now also asserts a completed overview. It uses the repository OPENAI_API_KEY and was not dispatched from this PR.

…awhub

The clawhub and clawhub-aig profiles now take their verdict from the OpenAI
Decisions API through clawscan-decisions-judge, a stdlib Python command shipped
in the runtime image. It builds one bounded request from the judge workspace
(exact artifact text plus an input_json part with run metadata, scanner
summaries, SARIF coverage gaps, and normalized findings), asks a verdict choice
plus dimension choices, risk-signal predicates, and per-finding predicates,
maps the typed answers onto the clawhub output schema, and records model,
usage, and latency. It enforces its own input limit; refusals, HTTP errors, and
malformed answers fail the judge with the real error.

Add an optional profile overview.command (--overview, --no-overview). It runs
after a completed verdict with the judge workspace plus judge.json and the
{{ judge.result }} prompt placeholder, records overview in the run artifact,
and never changes the verdict, gate, status, completeness, or exit status. The
clawhub overview runs Codex gpt-6-luna with low effort, routed through
OPENAI_BASE_URL when set.

BREAKING CHANGE: clawhub judge.result now holds typed fields only plus a
decision record; summary, dimension details, finding notes, and user_guidance
move to overview.result.
@Patrick-Erichsen
Patrick-Erichsen requested a review from a team as a code owner November 4, 2026 06:26
@clawsweeper

clawsweeper Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

@clawsweeper clawsweeper Bot added P2 Normal priority bug or improvement with limited blast radius. merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. proof: sufficient Contributor real behavior proof is sufficient. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. labels Nov 4, 2026
@clawsweeper

clawsweeper Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

Codex review: needs maintainer review before merge.

What this changes

This PR adds a Decisions API judge and an optional overview command that explains completed verdicts without affecting the gate.

Example: Scan the daily-notes skill using the clawhub profile.

  • Before: A Codex judge returns both the security verdict and explanatory prose.
  • After: Decisions returns a typed benign verdict, and the optional overview provides the readable explanation.

Review scores

Measure Result What it means
Overall readiness 🐚 platinum hermit (4/6) Useful, supported by live CLI evidence and legacy compatibility coverage, with prior findings resolved and reviewed-head validation passing.
Proof confidence 🐚 platinum hermit (4/6) Sufficient (live_output): The supplied live CLI results exercise the actual Decisions judge and Codex overview through ClawRouter with fixture scanners, producing expected benign and malicious verdicts and completed explanations.
Patch quality 🐚 platinum hermit (4/6) No actionable review findings were identified.

Product

Kind: Feature · Worth it: Yes · Fix scope: Complete
User problem: ClawHub needs typed security verdicts with readable explanations that can fail independently.
Reason: The maintainer-authored direction follows the merged judge-agnostic design and explicitly accepts the profile contract transition.

Merge readiness

✅ Ready for maintainer review

Keep this maintainer-authored PR open: the prior findings are resolved, and review found no remaining actionable merge blocker.

Priority: P2
Reviewed head: 9ed036bd25766cd63e668a4c6f5374ef8e85f2a6

Before merge

None.

Findings

None.

Agent review details

How this fits together

The runner passes artifact files and scanner evidence to an external judge, records its verdict, and optionally runs a separate explanatory command.

flowchart LR
A[CLI and profile] --> B[Scanner adapters]
B --> C[Artifact and raw evidence]
C --> D[Decisions judge]
D --> E[Typed verdict]
E --> F[Optional overview]
E --> G[Run artifact and gate]
F --> G
Loading

Technical review

Best possible solution:

Land the external Decisions judge and advisory overview while retaining legacy artifact readability and the documented downstream deployment boundary.

Do we have a high-confidence way to reproduce the issue?

This is a feature PR; source inspection confirms the earlier limit and gap-preservation findings are addressed, without executing target code during this review.

Is this the best way to solve the issue?

The implementation fits the established external-judge boundary and reuses the shared harness for overview execution while keeping overview failures outside verdict completeness.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning medium; reviewed against fa01a95bcd18.

Provenance checked

  • External judge execution keeps the original intent (fa01a95: The merged predecessor assigns judge-specific checks to the configured external command and keeps the runner judge-agnostic.)
  • Bundled judge model and output projection changes intended behavior with a stated reason (5be7f8e: Configure the bundled Sol judge while preserving operator overrides; this PR explains replacing its verdict with typed Decisions answers and separating prose.)
  • Partial SkillSpector evidence keeps the original intent (a5301cc: Permit independent judgment of usable partial reports while retaining material gaps, child evidence, and uncertainty.)
  • Runtime production qualification keeps the original intent (e036a88: Bind production qualification to reviewed runtime source and locked dependencies; this PR refreshes that binding for the new command.)
  • Input-write protection keeps the original intent (7343e0e: Prevent result writes from replacing consumed configuration and auxiliary inputs; overview prompt and schema inputs receive the same protection.)

Testing

Proof path: shipped entry point.

Security

None.

Evidence

What I checked:

  • Current main and introduced change: Current main remains fa01a95, whose bundled profiles use the Codex judge; the pinned introduced diff adds the Decisions judge and overview capability. (internal/profiles/clawhub/clawscan.yml:32, 9ed036bd2576)
  • Previous boundary finding resolved: Omission counters now exist before the fit loop, and the exact final evidence object is checked afterward and before request construction; regressions include the earlier 1,661-finding reproduction. (docker/clawscan-runtime/judges/clawscan_decisions_judge.py:950, 9ed036bd2576)
  • Earlier partial-evidence finding resolved: The projection retains bounded completeness gaps, component limitations, provenance, bundled-child paths, and truthful omission counts; tests exercise existing partial-report fixtures. (docker/clawscan-runtime/judges/clawscan_decisions_judge.py:604, 9ed036bd2576)
  • Overview failure isolation: Overview execution uses a separate timeout and command-runner scope; lifecycle completeness depends on scanners and judge rather than overview status. (internal/runner/overview.go:64, 9ed036bd2576)
  • Legacy artifact and configuration compatibility: Compatibility coverage decodes stored artifacts from the production pin and preceding main, preserves judge.result during re-encoding, and exercises the pre-overview profile without introducing an overview. (cmd/clawscan/compat_test.go:28, 9ed036bd2576)
  • Supplied live behavior and accepted cutover: The full PR body supplies actual CLI results through Decisions and Codex using ClawRouter with fixture scanners: daily-notes was benign and its credential-exfiltration variant malicious, both with completed overviews; the maintainer explicitly accepts the profile projection break while deferring downstream activation. (9ed036bd2576)

Likely related people:

  • Patrick Erichsen: Raw commit 5be7f8e adds internal/profiles/resolver.go:35 relative to its recorded parents. This identifies author metadata, not feature responsibility or a PR merger. (role: source-line author; confidence: high; commits: 5be7f8e97fa4; files: internal/profiles/resolver.go)
  • Jesse Merhi: Raw commit 656e558 adds internal/profiles/resolver.go:42 relative to its recorded parents. This identifies author metadata, not feature responsibility or a PR merger. (role: source-line author; confidence: medium; commits: 656e558b4ba5; files: internal/profiles/resolver.go)

Labels

Label changes:

  • add rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
  • add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR.
  • remove status: ⏳ waiting on author: Current PR status label is status: 👀 ready for maintainer look.
  • remove rating: 🦪 silver shellfish: Current PR rating is rating: 🐚 platinum hermit, so this older rating label is no longer current.

Label justifications:

  • P2: This is an intentional feature and profile transition without evidence of an urgent released-user regression.
  • merge-risk: 🚨 compatibility: The bundled profile intentionally changes judge.result fields and requires a new host command; the maintainer accepts that boundary and keeps downstream deployment pinned pending separate qualification.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR.
  • proof: sufficient: Contributor real behavior proof is sufficient.

Rating scale

6/6 🦀 challenger crab · 5/6 🦞 diamond lobster · 4/6 🐚 platinum hermit · 3/6 🦐 gold shrimp · 2/6 🦪 silver shellfish · 1/6 🧂 unranked krab. Overall follows the weaker of proof and patch quality; ✨ marks media proof (a screenshot, video, or linked artifact) that directly shows the changed behavior.

Workflow

ClawSweeper edits this one comment on every review. Comment @clawsweeper re-review for a fresh review only. Repair and merge execution lanes are retired.

History

Review history (2 earlier review cycles)
  • reviewed 2026-11-04T06:52:35.493Z sha 000cda8 :: blocked before merge. :: [P1] Preserve material partial-scan gaps in the Decisions evidence | [P2] Recheck JSON value limits after adding the artifact manifest
  • reviewed 2026-11-04T08:54:17.680Z sha e1781d2 :: needs changes before merge. :: [P2] Include omission accounting before the final JSON-limit check

Reviewed November 4, 2026, 4:13 AM ET / 09:13 UTC (Revision 3).

…ictly

- Exclude node_modules at every depth from plugin judge text, manifest, and
  budget, reporting only artifactCounts.excludedDependencies as a deliberate
  scope exclusion. Skill targets and the copied artifact are unchanged.
- Project analysis_completeness lists (ledger exceptions, scope exclusions,
  references, limitations, incomplete analyzer statuses), component gaps, and
  bounded metadata provenance into bounded, path-aware records with counts and
  truthful omission accounting, for root reports and bundled children (with
  rootPath and prefixed paths).
- Check byte, JSON value, and depth limits on the fully assembled evidence,
  including the manifest and counts, trimming with recorded omissions.
- Keep finding text out of questions: finding predicates reference a stable
  evidence index.
- Reject duplicate, missing, unexpected, mistyped, out-of-range, or
  non-finite answers; any refusal is terminal.
…able

Add real stored artifacts from the ClawHub production pin (c3cfaf1) and pre-#148 main (fa01a95) and check they decode, summarize, and re-encode with judge.result unchanged and no overview. Run the exact pre-#148 clawhub profile as a project config end to end, and keep it valid against the config schema.
@Patrick-Erichsen

Copy link
Copy Markdown
Collaborator Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

🦞👀
Exact review queued.

Re-review progress:

The fixture had no CompletedAt, so checkedAt fell back to the current millisecond and two renders could differ (pre-existing flake on main since #147).
findingCounts.omitted was added after the fit loop, so evidence measured at exactly 10,000 values was sent with 10,001. Initialize all accounting fields before measuring, re-check the exact final object, and add exact-boundary regressions (the old judge sends 10,001 values on the new test).
1,661 findings with context {} and an eight-character SKILL.md measure exactly 10,000 values untrimmed; the previous judge sends 10,001.
@Patrick-Erichsen

Copy link
Copy Markdown
Collaborator Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

🦞👀
Exact review queued.

Re-review progress:

@clawsweeper clawsweeper Bot added rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. and removed status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. labels Nov 4, 2026
@Patrick-Erichsen
Patrick-Erichsen merged commit 22d2f69 into main Nov 4, 2026
18 of 19 checks passed
@Patrick-Erichsen
Patrick-Erichsen deleted the decisions-judge-overview branch November 4, 2026 10:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. P2 Normal priority bug or improvement with limited blast radius. proof: sufficient Contributor real behavior proof is sufficient. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant