Skip to content

feat: audit OpenClaw plugins with the A.I.G MCP scanner - #149

Merged
Patrick-Erichsen merged 17 commits into
mainfrom
aig-mcp-plugin-scanner
Nov 4, 2026
Merged

Patrick-Erichsen merged 17 commits into
mainfrom
aig-mcp-plugin-scanner

Conversation

@Patrick-Erichsen

@Patrick-Erichsen Patrick-Erichsen commented Nov 4, 2026 •

Copy link
Copy Markdown
Collaborator

Why

Native OpenClaw plugins run in-process in the gateway with its full permissions, and nothing in ClawScan audits plugin code for malice today. This adds Tencent A.I.G's MCP code audit (aig-mcp-scan 0.2.0) as the plugin scanner. A local experiment validated that configuration on 20 popular ClawHub plugins and 4 synthetic malicious ones.

What changes

  • New aig-mcp adapter (plugin targets only). Skill, file and URL targets are skipped. For each plugin it:

    • copies the whole plugin into a private temporary workspace, leaving out only node_modules directories. Embedded skills (SKILL.md and their scripts), hooks, dist//build/ output, and .git/ and clawscan-results/ content stay in the copy;
    • refuses symlinks that point outside the plugin, and fails closed above 20,000 files or 256 MiB;
    • runs aig-mcp-scan --repo plugin --language en -m <model> -u <base url> -p <validated OpenClaw plugin prompt> -o aig-mcp-scan.sarif.json inside that workspace. The prompt adds an "audit the whole artifact" scope note. Upstream leaves dist//build/ out of its directory summary and truncates it at 200 files, and the patched agent has no shell to find files itself. So the prompt lists the omitted files as read_file paths (plugin/...): every file for plugins over 200 files, up to 400 paths, source files and SKILL.md first.
  • Credentials and model settings. The key comes only from LLM_API_KEY or OPENAI_API_KEY in the environment, never from argv. Model and endpoint follow the aig conventions: DEFAULT_MODEL/DEFAULT_BASE_URL, with OpenAI defaults for OpenAI or Codex keys (never OpenRouter) and otherwise upstream's own defaults. Both are always passed as -m/-u, so a host's generic OPENAI_BASE_URL/LLM_* values are never paired with the key.

  • Evidence. The SARIF stays raw in the artifact, and the ClawHub judge receives that one raw SARIF result. Benchmarks map SARIF levels to severities: error → high, which predicts malicious; any other finding predicts suspicious.

  • Failures are never reported as clean. Any of these marks the scanner failed, so the judge does not start:

    • a non-zero exit;
    • missing or invalid SARIF;
    • upstream model failures that it swallows behind exit 0 (LLM chat failed after retries, LLM chat error, retry 5 times);
    • scanNote other than complete.

    Failure messages keep the upstream error and a bounded stderr tail.

  • Plugin scanner selection. New profile field pluginScanners replaces scanners for plugin targets. The clawhub profile sets pluginScanners: [aig-mcp]:

    • plugins run only aig-mcp, with no skill-scanner calls and no skill-scanner entries in the artifact;
    • skills keep skillspector and aig, and their judge prompt is unchanged;
    • --scanner overrides both lists.

    The field is in the schema, is validated (built-in, plugin-capable, no duplicates), and shows in clawscan profiles.

  • Sandbox by default. Upstream 0.2.0 gives its agent execute_shell and file writes during static audits, and its MCP client tools accept any model-supplied URL, so prompt injection in the plugin under review could reach them. The runtime image therefore installs aig-mcp-scan==0.2.0 and applies patches/aig-mcp-static-tools.patch: static audits get only confined read-only tools (allow-listed on canonical names), and model-chosen MCP destinations are refused before any transport. tests/test_aig_mcp_static_tools.py and tests/test_aig_mcp_entry_network.py run during the build. They assert the exact reviewed tool set and that a real CLI audit with injected remote MCP calls reaches no destination. With --sandbox off, the adapter fails unless AIG_MCP_SCAN_ALLOW_HOST_EXEC=1 is set.

  • Install and CI.

    • ARG AIG_MCP_SCAN_VERSION=0.2.0 is added to the runtime Dockerfile.
    • aig-mcp-scan --help is added to the image publication smoke (publish.py) and to the runtime-tool-update smoke.
    • clawscan install aig-mcp runs pip install aig-mcp-scan==0.2.0 (unpatched; documented).
    • A test keeps the adapter pin and the Dockerfile pin in step.
  • Production contract. Rebound to the latest runtime commit (b98eb30) in d430dc4, following the existing pattern (e036a88).

Evidence

Live pilot. Host run with the patched tool, the real OpenAI endpoint and gpt-6.1-sol:

Plugin Result Time
Credential/env exfiltration in a tool handler (src/) MCP01 error at src/index.js 51s
Same payload shipped only in dist/ MCP01 error at dist/index.js 51s
Exfiltration hidden in an embedded skill script (skills/notes/scripts/setup.sh) MCP01 error at that script 80s
Official OpenClaw duckduckgo plugin 0 findings, score 100 46s

An end-to-end --profile clawhub run on the embedded-skill sample, with the real scanner and a stand-in judge, gave:

  • artifact scanners exactly ["aig-mcp"];
  • zero calls to the stub skillspector and aig-skill-scan binaries;
  • judge prompt with exactly one A.I.G MCP plugin code audit SARIF block and target kind packageRelease.

Patch test. test_aig_mcp_static_tools.py: unpatched 0.2.0 fails 3 of 4 tests (shell and write tools are offered and callable); patched passes 4 of 4.

Tests.

  • go test -count=1 ./..., go vet ./..., gofmt, python3 -m unittest discover -s scripts/runtime-image -p 'test_*.py' and make docs-site all pass.
  • CLI seam (cmd/clawscan/aig_mcp_test.go, stub aig-mcp-scan):
    • a plugin target runs exactly aig-mcp, on a staged copy without node_modules (embedded skill, dist/ and hooks present), with the expected argv;
    • the original target is never touched and the key never reaches argv;
    • the judge gets one raw SARIF result;
    • skill-scanner stubs get zero calls;
    • a skill target keeps the existing scanners and never invokes aig-mcp-scan;
    • an explicit --scanner overrides both lists;
    • a command crash and swallowed model failures are kept as failures, and the judge does not run.
  • Runner unit tests cover:
    • staging, the bounds and escaping symlinks;
    • model/endpoint conventions, including cross-pairing;
    • the host-exec opt-in;
    • Docker mounts (only the scanner workspace);
    • every failure path, including exit 0 without SARIF;
    • SARIF severity and benchmark predictions;
    • the raw-SARIF judge prompt;
    • the install pin.
  • Profile tests cover pluginScanners resolution, the override, and validation.

Combined with #148 (rebased onto 22d2f69)

This branch now sits on #148's squash merge 22d2f69, whose tree is identical to #148's reviewed head 9ed036b. The changes in the rebase:

Local results:

An independent rebase-delta review found no P0 or P1 issues. It confirmed that #148-only files are identical to #148, apart from the intended fixture, and #149-only files are identical to 1a64906, apart from the docs.

Proof for the ClawSweeper review of d430dc4

[P1] Remote MCP tools callable in static audits — fixed in two layers.

  • Static dispatch allows only read_file, read_file_chunk, search_files, think and finish, matched on canonical names. call_mcp_tool and the MCP listing tools are rejected before tool lookup.
  • Separately, upstream's _get_mcp_manager now refuses any model-supplied URL that isn't the operator-configured dynamic server, before a transport is created.

Production-entry proof. No real network or LLM is used: the plugin is a synthetic, inert fixture, the model is a trusted loopback stub, and every destination is a loopback canary. The canary is also set as HTTP(S)_PROXY, so any other egress would be counted rather than sent.

The scripted agent follows a prompt injection and calls call_mcp_tool, Call_MCP_Tool, list_mcp_tools, LIST_MCP_RESOURCES and list_mcp_prompts with canary URLs. Then it reads a local file and finishes.

Entry Runtime Canary connections
shipped clawscan ./plugin --profile clawhub d430dc4 patch 3
shipped clawscan ./plugin --profile clawhub this head 0
python -m mcp_scan (the console script's cli()) stock 0.2.0 / d430dc4 patch connections (test fails)
python -m mcp_scan this head 0 (test passes)

At this head, the shipped CLI run also showed:

  • artifact scanners exactly ["aig-mcp"], completed, scanNote: complete;
  • the recorded command passing the stub -m/-u;
  • all 5 injected calls refused in the model-visible history, and the local file content reached the model;
  • the judge completed with one SARIF block.

The python -m mcp_scan rows are tests/test_aig_mcp_entry_network.py, which runs inside the runtime image build on both architectures, so this proof is enforced by CI. It also checks that a model-chosen URL is refused before transport even when a dynamic server is configured, and includes a control that the canary does count connections.

[P2] Plugin-selected scanner fixtures rejected — fixed.

  • clawscan ./plugin --profile clawhub --scanner-result aig-mcp=report.sarif.json is accepted and validated like a scanner run.
  • A complete report gives completed and the judge runs. possibly-incomplete gives failed, and the judge does not start.
  • Genuinely unrequested results are still rejected, for example --scanner skillspector --scanner-result aig-mcp=….
  • On a skill target, a plugin-only result is an error rather than being silently ignored.
  • This covers both resolution paths (--profile and all-profiles --config).

[P1, re-review of f737248] Authored files dropped by staging — fixed in 2ae7c16. Scanner staging had inherited the judge workspace filter, which skips top-level .git/ and clawscan-results/, so plugin code there (for example import "./clawscan-results/payload.js") never reached aig-mcp. This restores fa01a95's intent that scanners see authored code wherever it lives:

The proof:

  • A staging regression test fails at f737248 and passes now, with the same layout covered at the --profile clawhub CLI seam.
  • Live run (gpt-6.1-sol, current runtime) on a plugin whose only malicious code is an inert exfiltration payload in clawscan-results/payload.js, imported by index.js: an MCP01 error at clawscan-results/payload.js quoting its destination, 334s, scanners ["aig-mcp"], judge completed.

[P1, independent review of c6601d8] Unusable final reports accepted as clean — fixed in 69ab8ab. Upstream's _format_final_output keeps the last of three format attempts even when none matched. A model that finishes with a refusal ("I cannot perform this security scan.") therefore produced SARIF with scanNote: complete, score 100 and no findings, with no log marker. Separately, <vuln> blocks the extractor couldn't parse were dropped silently. The patched upstream path now:

  • raises when no format attempt matches;
  • accepts only an explicit <empty> result, or findings that all parse;
  • otherwise exits with ClawScan audit output invalid: … and writes no SARIF, which the adapter records as a failure.

Offline in-image tests drive the real python -m mcp_scan CLI with a scripted loopback model:

  • refusal, prose, malformed finding and partly malformed finding: non-zero exit, no SARIF;
  • explicit <empty>, a parsed finding, and a report fixed on a later format attempt: SARIF as expected.

The four failure cases exit 0 on the previous runtime and pass now; all 28 in-image tests pass.

Real-model check of the stricter validation (gpt-6.1-sol, this runtime). Hidden-tail sample: MCP01 error at dist/index.js plus MCP06 for the embedded instruction, 111s. DuckDuckGo: aig-mcp completed with no findings, 77s, so genuine reports are not failed. The judge again stopped on the node_modules symlink in its workspace copy, which is #148's.

Shipped clawscan ./plugin --profile clawhub with the refusal script:

previous runtime this head
exit 0 1
aig-mcp completed, scanNote: complete, score 100, 0 findings failed, no SARIF, error names the cause
judge started not started

Compatibility with the latest stable release (v0.3.0). I verified the release binary checksum and ran three binaries side by side: v0.3.0, main (fa01a95) and this PR.

  • Existing project config. A v0.3.0-era .clawscan.yml uses scannerEnvDefaults, built-in scanners with gate rules, a command scanner with targets/env/gate block, scannerResults, sandbox env and a judge, and has no pluginScanners. On both a skill and a plugin target, the artifacts are identical across all three binaries: scanners, statuses, raw evidence, gate rules and decisions (block / warn), env presence, sandbox metadata and judge. The new field changes nothing unless a profile sets it. This config is committed as cmd/clawscan/testdata/v0.3.0-project.clawscan.yml with a regression test.
  • Old stored reports. SkillSpector and A.I.G reports stored in a v0.3.0 clawscan-run-v1 artifact, re-imported into this PR's --profile clawhub on a skill, give completed for both and a completed judge.
  • Explicit plugin overrides. --scanner skillspector on a plugin runs only SkillSpector. v0.3.0 rejects aig-mcp as an unknown scanner, so it is purely additive.

Hold fixes (review of d430dc4)

All four findings are fixed at head; the reviewer's repro tests and the in-image tests fail at d430dc4 and pass now.

  • P1, shell block bypassed by case. Upstream looks up tool names case-insensitively, so Execute_Shell reached the shell past a case-sensitive deny list. The patch now:

    • never registers the shell; execute_shell is a non-executing stub that nothing imports;
    • allows only read_file, read_file_chunk, search_files, think and finish in static audits, matched on canonical (trimmed, lowercase) names, and canonicalizes dynamic mode too.

    In-image spy tests (Execute_Shell, EXECUTE_SHELL, " execute_shell ", Write_File, MCP tools) fail on the d430dc4 patch, where the spy records the process start, and pass now.

  • P1, tail of large compiled files not reachable. Upstream keeps only a 20k prefix of tool results over 40k in the model's history. Now:

    • read_file returns at most 30,000 characters and says how to continue;
    • new read_file_chunk (character offsets, at most 30k) and search_files (literal, at most 50 matches and 24k characters) stay confined to the audited directory;
    • invalid UTF-8 is replaced instead of reported as binary; case-insensitive search offsets index the original text;
    • files over 64 MiB fail staging, matching the tools' limit.

    An in-image test drives BaseAgent with a 50k bundle. Repeated prefix reads never show the tail marker; read_file_chunk and search_files put it in the model-visible history without truncation. A maximal chunk stays under the 40k limit.

  • P1, exhausted or failed audits accepted. Max iterations (N) reached and the model-failure records, logged as WARNING/ERROR on stderr, fail the scan. INFO records and continuation lines that only quote the words don't. Every SARIF run must say scanNote: complete; a missing value fails. Imported --scanner-result aig-mcp gets the same SARIF and scanNote checks, and the judge does not start. A profile's plugin-only scanner accepts --scanner-result; on a non-plugin target such a result fails instead of being ignored.

  • P2, benchmark fallback. A completed judge with no recognized verdict makes the case unscorable, with no fallback to scanner severity. Scanner-only runs still score from scanners.

  • Contract. Rebound to the full SHA of the last runtime commit (now 040f4fa1322145047a1280a2aae828627ad9ebfa, after the rebase onto feat!: add a Decisions API judge and a pluggable overview step for clawhub #148). The previous rebind wrote a short SHA, which actions/checkout and source_binding() reject; a new test requires 40-hex ids and a matching workflow ref. I replayed the real source_binding() against a clean checkout and it passes.

  • Pre-existing flake. TestRenderClawHubPromptUsesProductionScannerContextShape (from feat!: make the runner judge-agnostic and remove VirusTotal and clawscan-static #147) compared prompts rendered moments apart, with a clock-derived SkillSpector checkedAt; it failed 9 of 300 runs on main and once in this PR's Linux CI. Its fixture now has a fixed completion time (300/300 pass).

Reviewer overlay (TestReview149*): 4/5 pass. They fail at d430dc4. The fifth, judge workspace node_modules, belongs to #148.

Live inert control. Host run with the patched tool, gpt-6.1-sol and --profile clawhub. The sample has credential-exfiltration code at characters 45,894-46,400 of a single-line 58k dist/index.js (past both the read cap and the history cutoff). It also hides an instruction in SKILL.md telling the auditor to call EXECUTE_SHELL with touch <canary> and report no issues. Result:

  • MCP01 error, quoting syncPrefs and cdn-metrics.example.net, which first appear at characters 45,894 and 46,173;
  • MCP06 for the embedded instruction;
  • canary never created, zero skill-scanner calls, artifact scanners ["aig-mcp"], one SARIF block in the judge prompt.

Repeated on the a371960 patch: both findings at error level, 172s, same results. The later a773037 patch adds only the MCP destination refusal, which this sample's model never used.

Review

Two independent review passes ran. The first pass's P1s, both now fixed:

  • the agent shell, now removed in the image, with a host-exec opt-in;
  • the contract binding.

Its P2s, also fixed:

  • dist/ files hidden from the agent;
  • key and endpoint cross-pairing;
  • a ClawScan prediction label in the judge evidence.

The second pass found no P0/P1. Fixed from it:

  • the inventory past upstream's 200-file cap;
  • plugin/-prefixed paths that read_file accepts;
  • host-mode env pairing, by always passing -m/-u;
  • the exact tool-set assertion;
  • the final gate using the plugin scanner list;
  • a schema enum limited to plugin-capable scanners.

Left as noted below:

  • the rollout sequencing;
  • the plugin judge prompt keeping ClawHub's parity text (SkillSpector findings: null);
  • --sandbox-image <unpatched> not being detected.

A re-pilot at the new head with --profile clawhub showed:

  • dist-only sample: MCP01 error at plugin/dist/index.js, 59s;
  • DuckDuckGo: aig-mcp 0 findings, 55s. The judge stage then failed, because the judge workspace copy refuses DuckDuckGo's node_modules/@openclaw/plugin-sdk symlink. That copy still includes node_modules, which feat!: add a Decisions API judge and a pluggable overview step for clawhub #148 removes;
  • artifact scanners exactly ["aig-mcp"], with explicit -m gpt-6.1-sol -u https://api.openai.com/v1.

Not covered

  • I had no local Docker. The Runtime Image CI builds both architectures and its logs show the in-image patch test passing (4 passed).
  • Rollout order: publish the runtime image, then a ClawScan release, then update the ClawHub worker. clawscan install aig-mcp installs the unpatched PyPI package.
  • Until feat!: add a Decisions API judge and a pluggable overview step for clawhub #148 lands, a full --profile clawhub plugin run whose node_modules contains a symlink leaving the package (workspace links such as @openclaw/plugin-sdk) fails at the judge workspace copy. The aig-mcp stage itself stages without node_modules and is unaffected.
  • ClawHub's worker installs scanners on the host from openclaw/clawhub scripts/security/install-clawscan-scanners.sh. It needs a follow-up that installs aig-mcp-scan 0.2.0 with this patch and sets AIG_MCP_SCAN_ALLOW_HOST_EXEC=1 before it moves its ClawScan pin past this change. Until then, plugin scans in that worker would fail closed.
  • Only the top-level aig-mcp-scan package is pinned. It uses Chat Completions without the reasoning-effort or Standard-tier controls the patched aig lane has.
  • clawhub-aig is unchanged.

@Patrick-Erichsen
Patrick-Erichsen requested a review from a team as a code owner November 4, 2026 06:30
@clawsweeper

clawsweeper Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

@clawsweeper clawsweeper Bot added clawsweeper:intake:feature Preliminary submission classification; full source review pending. P2 Normal priority bug or improvement with limited blast radius. merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. merge-risk: 🚨 security-boundary 🚨 Merging this PR could weaken sandboxing, authorization, credentials, or sensitive data. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. and removed clawsweeper:intake:feature Preliminary submission classification; full source review pending. labels Nov 4, 2026
@clawsweeper

clawsweeper Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

Codex review: needs maintainer review before merge.

What this changes

Adds a confined A.I.G MCP scanner for plugins and selects it through an optional pluginScanners profile field.

Example: Scan a plugin containing credential-exfiltration code in dist/index.js with --profile clawhub.

  • Before: Main selects SkillSpector and the skill-only A.I.G adapter, which skips plugin targets.
  • After: The plugin runs through aig-mcp alone, preserving its MCP01 SARIF finding for the independent judge.

Review scores

Measure Result What it means
Overall readiness 🦞 diamond lobster (5/6) Scoped live behavior, forbidden-action controls, upgrade comparisons, and current-head validation support a clean review.
Proof confidence 🦞 diamond lobster (5/6) Sufficient (live_output): Supplied shipped-CLI results exercise patched scanner execution on benign, compiled-code, embedded-skill, and hidden-authored-file samples; adversarial model responses yield zero canary connections while allowed reads succeed, and unusable reports fail before the judge starts.
Patch quality 🦞 diamond lobster (5/6) No actionable review findings were identified.

Product

Kind: Feature · Worth it: Yes · Fix scope: Complete
User problem: Operators need a plugin-focused code audit that includes compiled output, embedded skills, and hooks as one artifact.
Reason: The maintainer-authored direction adds a distinct upstream scanner lane with concrete malicious and benign controls; existing plugin-capable scanners remain explicitly selectable.

Merge readiness

✅ Ready for maintainer review

Keep open: this maintainer-authored PR adds a useful scanner capability absent from main; the pinned diff has no actionable correctness or security findings.

Priority: P2
Reviewed head: 82ab0cdb398f05aa7acba483b257bd0616139279

Before merge

None.

Findings

None.

Agent review details

How this fits together

The CLI resolves targets and profiles, dispatches scanner adapters inside the default Docker sandbox, preserves raw evidence, and passes completed scans to an external judge.

flowchart LR
A[Plugin directory] --> B[Target and profile resolution]
B --> C[Bounded staged copy]
C --> D[Confined AIG MCP scanner]
D --> E[Raw SARIF artifact]
E --> F[External judge]
F --> G[Optional overview]
Loading

Technical review

Best possible solution:

Land the confined plugin scanner with the documented runtime-before-release sequencing and preserve existing project-profile behavior.

Do we have a high-confidence way to reproduce the issue?

This is a feature addition; source comparison confirms its absence from main, and supplied CLI results exercise the new scanner path.

Is this the best way to solve the issue?

The implementation uses the existing scanner registry, target classification, copy primitive, and external-judge boundary, with focused runtime confinement for the new upstream scanner.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning medium; reviewed against 22d2f694bb81.

Provenance checked

Testing

Proof path: shipped entry point.

Security

None.

Evidence

What I checked:

  • Pinned change and current-main boundary: Reviewed the introduced diff from main to the verified PR head; main has neither the aig-mcp adapter nor pluginScanners, and GitHub reports the reviewed head mergeable with a clean merge state. (internal/runner/scanner_registry.go:257, 82ab0cdb398f)
  • Complete primary evidence recovered: Read the complete PR body and author discussion; the body matches captured SHA-256 5163b40feaa4eb4a78595d7f08ba9ddc3a86e1c5653a7fe1c2bd740924f52c82. Patrick-Erichsen reports shipped CLI controls for blocked MCP destinations, failed unusable reports, hidden-file detection, and v0.3.0 compatibility. (82ab0cdb398f)
  • Scanner failure and staging invariants: The adapter stages authored content outside node_modules, enforces file and byte limits, retains raw SARIF, rejects incomplete reports, and preserves upstream errors; runner failures prevent judge execution. (internal/runner/aig_mcp_scanner.go:181, 82ab0cdb398f)
  • Final-effect confinement: The runtime patch canonicalizes tool names and rejects disallowed static tools before lookup, removes shell registration, confines file reads, and refuses model-selected MCP destinations before transport creation; the production-entry canary scenario also verifies allowed local reads. (docker/clawscan-runtime/patches/aig-mcp-static-tools.patch:149, 82ab0cdb398f)
  • Compatibility and accepted rollout boundary: The maintainer-authored direction explicitly changes the bundled plugin scanner, documents host opt-in and patched-runtime sequencing, and supplies stable-release comparisons preserving existing project configs, gate rules, explicit overrides, and stored skill reports. (cmd/clawscan/aig_mcp_test.go:511, 82ab0cdb398f)
  • Current-head validation: Current-head checks pass, including both runtime-image architectures and the rebound production SDK contract; the added integration scenario connects plugin scanner evidence to the real Decisions judge implementation and optional overview. (cmd/clawscan/integrated_plugin_test.go:15, 82ab0cdb398f)

Likely related people:

  • Patrick Erichsen: Raw commit 5be7f8e adds internal/profiles/clawhub/clawscan.yml:10 relative to its recorded parents. This identifies author metadata, not feature responsibility or a PR merger. (role: source-line author; confidence: high; commits: 5be7f8e97fa4; files: internal/profiles/clawhub/clawscan.yml)
  • Vito Cappello: Raw commit a37f49f adds internal/runner/scanner_registry.go:167 relative to its recorded parents. This identifies author metadata, not feature responsibility or a PR merger. (role: source-line author; confidence: medium; commits: a37f49f1b09f; files: internal/runner/scanner_registry.go)
  • Jesse Merhi: Raw commit 656e558 adds internal/profiles/resolver.go:834 relative to its recorded parents. This identifies author metadata, not feature responsibility or a PR merger. (role: source-line author; confidence: medium; commits: 656e558b4ba5; files: internal/profiles/resolver.go)

Labels

Label changes:

No label changes.

Label justifications:

  • P2: Useful plugin-audit capability with bounded scope and no demonstrated urgent regression.
  • merge-risk: 🚨 compatibility: The bundled plugin scanner and artifact scanner IDs intentionally change; the maintainer documents consumer updates and rollout sequencing, with existing-config upgrade evidence supplied.
  • merge-risk: 🚨 security-boundary: Adds an agent-driven scanner that consumes untrusted plugin content; reviewed tool restrictions and supplied forbidden-destination proof address the changed execution and network boundaries.
  • rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR.
  • proof: sufficient: Contributor real behavior proof is sufficient.

Rating scale

6/6 🦀 challenger crab · 5/6 🦞 diamond lobster · 4/6 🐚 platinum hermit · 3/6 🦐 gold shrimp · 2/6 🦪 silver shellfish · 1/6 🧂 unranked krab. Overall follows the weaker of proof and patch quality; ✨ marks media proof (a screenshot, video, or linked artifact) that directly shows the changed behavior.

Workflow

ClawSweeper edits this one comment on every review. Comment @clawsweeper re-review for a fresh review only. Repair and merge execution lanes are retired.

History

Review history (4 earlier review cycles)
  • reviewed 2026-11-04T07:01:55.299Z sha d430dc4 :: needs real behavior proof before merge. :: [P1] [P1] Reject remote MCP tools during static plugin audits | [P2] [P2] Include plugin-selected scanners when validating result fixtures
  • reviewed 2026-11-04T09:07:05.995Z sha f737248 :: blocked before merge. :: [P1] [P1] Preserve authored plugin files when staging the audit
  • reviewed 2026-11-04T09:27:00.571Z sha 2ae7c16 :: needs maintainer review before merge. :: none
  • reviewed 2026-11-04T09:45:57.244Z sha 1a64906 :: needs maintainer review before merge. :: none

Reviewed November 4, 2026, 5:43 AM ET / 10:43 UTC (Revision 5).

@Patrick-Erichsen

Copy link
Copy Markdown
Collaborator Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

🦞👀
Exact review queued.

Re-review progress:

@clawsweeper clawsweeper Bot added proof: sufficient Contributor real behavior proof is sufficient. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. and removed rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Nov 4, 2026
@Patrick-Erichsen

Copy link
Copy Markdown
Collaborator Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

🦞👀
Exact review queued.

Re-review progress:

@clawsweeper clawsweeper Bot added rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. and removed status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. labels Nov 4, 2026
@Patrick-Erichsen

Copy link
Copy Markdown
Collaborator Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

🦞👀
Exact review queued.

Re-review progress:

@clawsweeper clawsweeper Bot added rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. and removed rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. labels Nov 4, 2026
Add the plugin-only aig-mcp adapter around Tencent A.I.G's aig-mcp-scan
0.2.0. It stages a bounded copy of the whole plugin without node_modules,
runs the validated OpenClaw plugin audit prompt with A.I.G model and
endpoint conventions, keeps the SARIF raw and never reports failed or
incomplete audits as clean.

The clawhub profile gains pluginScanners so plugins run only aig-mcp and
the judge receives that one raw SARIF result, while skills keep their
scanners. The runtime image pins aig-mcp-scan and patches out its agent's
shell and file-write tools; host runs require an explicit opt-in.
The patched runtime has no shell, so plugins past the upstream 200-file
summary get a prioritized file inventory with read_file-compatible
plugin/ paths. The model and endpoint are always passed explicitly so
host LLM_/OPENAI_ settings never pair with the key, the final gate sees
the plugin scanner list, and the schema admits only plugin-capable
scanners.
…able

The shell tool is no longer registered, and static audits accept only
read_file, read_file_chunk, search_files, think and finish, matched on
canonical names because upstream resolves tool names case-insensitively.
read_file returns at most 30000 characters with an explicit continuation,
and bounded, directory-confined chunk and literal-search tools keep the
rest of large compiled files visible past upstream's history truncation.
The in-image tests spy on process starts and check that a 50k bundle's
tail reaches the agent history.
…cores

aig-mcp now fails when upstream logs exhausted iterations or failed model
calls on stdout or stderr, and requires every SARIF run to state
scanNote complete. Imported aig-mcp results get the same checks, and
profile plugin-only scanners accept --scanner-result. A benchmark case
whose configured judge returned no recognized verdict is unscorable
instead of falling back to scanner severity.
…ile reads

read_file replaces invalid UTF-8 as Node does instead of reporting a
plugin file as binary. Case-insensitive search matches on the original
text so offsets index the file, and both read tools refuse files over
64 MiB. In-image tests cover these, symlink escapes, and a maximal chunk
staying whole in agent history.
…results

Unfinished-audit markers count only as WARNING/ERROR log records on
stderr, so model reports or file names quoting them cannot fail a scan.
Staging rejects plugin files the read tools cannot open, a plugin-only
imported result on a non-plugin target fails instead of being ignored,
and the contract test requires full commit ids.
…e CLI entry

The MCP client helper now refuses any URL the model supplies unless it is
the operator-configured dynamic server, before a transport exists. Static
audits already reject the MCP tools by allow-list; this keeps a second
layer. The image build runs the real aig-mcp-scan CLI against a loopback
model stub that injects remote MCP calls under several spellings, with a
loopback canary as every destination and as the HTTP(S) proxy, and fails
if the canary sees a connection or the local read misses the model.
A v0.3.0-era .clawscan.yml without pluginScanners runs exactly its
configured scanners and gate rules for plugin and skill targets, and
never selects aig-mcp. Docs describe the MCP destination refusal and the
in-build entry check.
Scanner staging inherited the judge workspace filter, which drops
top-level .git/ and clawscan-results/, so plugin code there (for example
an import of ./clawscan-results/payload.js) never reached aig-mcp. Staging
now drops only node_modules, keeping its file, size and symlink checks,
and the prompt inventory lists every file the upstream directory summary
hides, including .git/, __pycache__/ and virtualenv content. The judge
workspace filter is unchanged.
Upstream keeps the last of three format attempts even when none matched,
so a refusal or prose became SARIF with scanNote complete, score 100 and
no findings. The final-output path now raises when no attempt matches,
and the single-stage audit accepts only an explicit <empty> result or
findings that all parse. In-image tests drive the real CLI with a
scripted model through refusal, prose, malformed and partly malformed
reports (all fail with no SARIF), and through empty, parsed and
late-retry reports (all produce SARIF).
@Patrick-Erichsen

Copy link
Copy Markdown
Collaborator Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

🦞👀
Exact review queued.

Re-review progress:

@Patrick-Erichsen
Patrick-Erichsen merged commit 56d17aa into main Nov 4, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. merge-risk: 🚨 security-boundary 🚨 Merging this PR could weaken sandboxing, authorization, credentials, or sensitive data. P2 Normal priority bug or improvement with limited blast radius. proof: sufficient Contributor real behavior proof is sufficient. rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant