Skip to content

feat(malo-browser): add the agent browser runtime and demo recorder plugin (0.1.11) - #168

Open
Lumos-789 wants to merge 1 commit into
zai-org:mainfrom
Lumos-789:feat/malo-browser
Open

Lumos-789 wants to merge 1 commit into
zai-org:mainfrom
Lumos-789:feat/malo-browser

Conversation

@Lumos-789

@Lumos-789 Lumos-789 commented Nov 4, 2026 •

Copy link
Copy Markdown

Plugin type (required) / 插件类型(必填)

  • UI Plugin / 是 UI 插件 — provides MCP Apps interactive pages / 提供 MCP Apps 交互页面
  • Standard plugin / 普通插件(非 UI) — does not provide MCP Apps interactive pages / 不提供 MCP Apps 交互页面
  • Not applicable / 不适用 — repository-only changes; no plugin content changed / 仅仓库级改动,不涉及插件内容

Affected plugin name(s) / 涉及的插件名称:malo-browser

What / 改动内容

EN — Adds malo-browser, which is two things in one Electron app: a scripted browser runtime for agents, and a demo recorder.

  • A daemon hands out port leases so several agents can drive isolated, profile-backed sessions concurrently; each session exposes 24 tools (navigate, act, observe, screenshot, batch steps, network_capture / responses).
  • A scene-driven stage records headless 1920×1080@30 demos with an evidence JSON per take (frames, fps, duration, black-frame ratio, hashes), plus a director console and a loopback control API.
  • network_capture subscribes a session to XHR/fetch responses matching URL prefixes and returns bodies by sequence number. Subscribe before navigating: requests sent earlier are missed, and response bodies never reach logs, evidence or run reports.
  • Deliberate boundary: the debugger attaches to the page target, so OOPIF (cross-process iframe) and Service Worker requests are not captured. Documented in the skill and both READMEs.

The 0.1.6 review pass fixed 33 reproduced defects; the security-relevant ones are listed under testing below.

中文 — 新增 malo-browser:一个 Electron 应用同时充当 agent 浏览器运行时与演示录制器。守护进程按端口发放租约,支持多 agent 并发、隔离且可复用登录态的 session(24 个工具,含网络抓包);场景化舞台可无头录制 1920×1080@30 演示并产出证据 JSON。0.1.6 修掉 33 个已复现缺陷。

Visible behavior and testing / 可见行为与测试

  • npx vitest run → 254 passed / 22 files (Node 24.15.0).
  • Real Electron end-to-end (Electron 45.1.0 with a real daemon): lease endpoint listed 24 tools; subscribe-then-navigate captured a real XHR JSON body; navigate-then-subscribe returned empty (the documented order constraint holds); an unreachable page produced a status: failed evidence record.
  • Real recording produced 8 s / 1920×1080@30 / 240 frames / 0 black frames.
  • Security fixes from the review pass were reproduced before being fixed: ${env:NAME} is limited to SCENE_* (a third-party scene could otherwise interpolate any process.env value into a URL); demo:// validates the scene id instead of using the URL segment as a containment root; the stage blocks server-side redirects as well as navigations; a password input is rejected as an assert target.

Not covered by this round / 本轮未覆盖:

  • macOS-only in practice (packaged with electron-builder for mac dir); Windows and Linux were not run. OOPIF / Service Worker capture is out of scope by design.

Version and marketplace registration / 版本与市场注册

  • New plugin directory external_plugins/malo-browser/ (15 files, 135 KiB), manifest and marketplace.json entry both at 0.1.6.
  • Category developer-tools; icon added at the repo-root path the entry uses (assets/malo-browser/icon.png, 512×512).
  • description_i18n carries en and zh-CN, identical in both files; the description now covers the agent runtime, not only the recorder.

Dependencies, network, permissions, side effects / 依赖、网络、权限与副作用

  • Node 22.12+ / 24.x / 26+ (23 and 25 are excluded by engines), git, and an Electron/Chromium runtime; ffmpeg/ffprobe for the recording checks.
  • The skill guides the agent to clone and build the main repository — the plugin package itself ships the skill and docs, not a prebuilt binary.
  • Network: the browser visits whatever the user's scene tells it to visit. No telemetry; response bodies stay in daemon memory and are never persisted.
  • Writes takes, evidence JSON and failure screenshots under its data directory (reports/screenshots 0600, session dirs 0700). Uses SCENE_* environment variables for scene interpolation and nothing else.

Licensing / provenance / 许可与来源

  • MIT (LICENSE ships in the plugin directory); code written for this submission.
  • Electron and its transitive dependencies are installed at run time under their own licences; nothing is vendored into the payload.

Self-check / 自查清单

  • Plugin name is unique kebab-case; new external plugin directory is external_plugins/malo-browser/; existing fixes retain ownership / 插件名唯一且 kebab-case,外部新增插件位于 external_plugins/malo-browser/,已有插件修复保留原归属
  • Usage category is set independently of publisher ownership / 用途分类 category 已填写,与维护归属分开:developer-tools
  • Plugin type and affected names are declared above / 已在上方声明插件类型与涉及的插件名称
  • version bumped in BOTH plugin.json and marketplace.json (artifacts are immutable) / plugin.json 与 marketplace.json 两处版本号已同步升级:0.1.6
  • README.md (EN) and README_CN.md (中文) both updated and equivalent / 中英文 README 均已更新且语义一致
  • description_i18n has both en and zh-CN / description_i18n 包含 en 与 zh-CN
  • No secrets, machine-specific paths, or private dependencies / 不含密钥、机器专属路径或私有依赖
  • Model/API/network dependencies documented / 模型、API、网络依赖已写明(见上方依赖小节与两个 README)
  • python3 scripts/validate.py passes on this branch, and the plugin artifact was built through the same build_zip() code path CI uses (size and sha256 in the comment below). The full python3 scripts/build_dist.py needs the repo's UI-plugin pnpm build and fails identically on upstream main (excalidraw: incomplete build), so CI runs it here — see the note above. / 本地 validate.py 通过,本插件产物用 CI 同一条 build_zip() 路径打出(体积与 sha256 见下方评论);完整 build_dist.py 依赖仓库自带 UI 插件的 pnpm build,在 main 上同样失败,交由 CI 执行

@Lumos-789

Copy link
Copy Markdown
Author

0.1.3 — verification (2026-11-04)

Environment: ZCode CLI 0.17.1, macOS 27.0.1 arm64, Node 24.15.0.

Installed from a local marketplace → malo-browser@0.1.3; the skill resolves from the plugin cache (malo-browser/0.1.3/skills/malo-browser/SKILL.md), not from a repo checkout.

Repo-side for the same tree:

Gate Result
npx vitest run 202 passed (18 files)
npm run typecheck clean (tsc main + renderer)
npm run build main + fixture + director bundles built
plugin_acceptance.py PASS malo-browser@0.1.3

What changed in this revision: only deletions — three internal phase reports under docs/ and scripts/baseline_chrome.py (it rebuilt a Chrome baseline from the author's private ~/recorder-pipe virtualenv, so it could never run on another machine). No runtime code changed.

Not covered here: a live Electron capture run. That needs a GUI session and real screen content; it is exercised by npm run smoke:control / smoke:toolbar locally, not in this verification.

@panda920

panda920 commented Nov 4, 2026

Copy link
Copy Markdown
Collaborator

✅ Internal validation passed. A ZCode maintainer will review the change next; after acceptance, release checks run separately and this PR is closed automatically.

@Lumos-789 Lumos-789 changed the title feat: add malo-browser (scripted browser demo recorder) feat: add malo-browser (agent browser runtime and demo recorder) Nov 4, 2026
@Lumos-789

Copy link
Copy Markdown
Author

Third round — review & simplification (2026-11-05)

No new features. The most important outcome is that the plugin description was selling the smaller half of the product: it described a demo recorder, while the daemon (port leases, isolated sessions, saved login profiles, 24 session tools) was not mentioned at all. The Codex manifest had it right; the ZCode one did not. Both now describe the runtime and the recorder.

network_capture / responses are documented. They were added in 0.1.5 but appeared in no skill or README. Now stated, together with the two things a caller gets wrong: subscribe before navigating (requests sent earlier are missed), and OOPIF / Service Worker requests are never captured because the CDP session attaches to the page target — the same boundary that was only in docs/malo-browser.md.

Documentation consolidated (nothing user-facing was dropped):

Change Lines
docs/control-api.md folded into the README — its only unique content was the {url, token, pid} record shape and how the token is compared −59
docs/malo-web-compat.md compressed — kept the three conclusions, the repro commands and the "not doing" list 148 → 55
Orphan fixtures/pages/agent-form.html and fixtures/scenes/report-review/notes.md deleted −90
Dead exports overlayClear and stepStatusTone removed −19

One factual correction inside the compat doc: it claimed the app sets a MaloBrowser/0.1.0 Chrome/156… user agent. Nothing in src sets a user agent — the observed value is Electron's, and it moves with the Electron version. The doc now says to read it from a real observe call instead of quoting a string that will be stale.

Verification: 232 tests pass (I re-ran the suite myself), npm run typecheck and npm run build are green.

Payload: re-synced, 11 files, verified blob-by-blob against Lumos-789/malo-browser@adc64c9.

Not verified this round: headed window modes and a live Electron capture — the change touched documentation, two dead exports and two orphan fixtures only.

@panda920

panda920 commented Nov 4, 2026

Copy link
Copy Markdown
Collaborator

✅ Internal validation passed. A ZCode maintainer will review the change next; after acceptance, release checks run separately and this PR is closed automatically.

@Lumos-789

Copy link
Copy Markdown
Author

Real-Electron verification of network_capture / responses (2026-11-05)

The previous comment listed "headed window modes and a live Electron capture" as not verified. The capture path is now verified against a real browser: daemon → lease → session → subscribe → navigate → read the body.

Setup: a live daemon on 127.0.0.1:7410 (Electron 45.1.0, Chromium 156.0.8078.49), a lease at 127.0.0.1:7411, and a local fixture site serving /index.html plus an XHR to /api/data that returns {"ok": true, "path": …, "n": 42}.

Step Result
GET /v1/tools on the lease 24 tools, including network_capture and responses
POST …/tools/network_capture {"url_prefixes":["http://127.0.0.1:4799/api/"]} subscribed, captured: 0
POST …/tools/navigate to the fixture page page loaded, the fetched JSON rendered in the DOM
POST …/tools/responses 2 records: the HTML document with body_error: "not_subscribed", and the XHR with the full body {"ok": true, "path": "/api/data?tag=boot", "n": 42}

The documented ordering constraint holds in practice. A second session navigated first and subscribed afterwards: responses returned [] — the page-load requests were never subscribed. Navigating again after the subscription produced the document (not_subscribed) plus the XHR with its body.

Failure path also holds. Recording against a page that cannot be reached wrote status: "failed" evidence (e2e.evidence.json) instead of leaving an empty directory — the 0.1.4 fix, re-confirmed on 0.1.5.

A real 8 s take was also produced: 1920×1080, 30 fps, 240 frames, H.264, 0 % black frames, overlay visible in the video, and the frame at 2.5 s is the actual fixture UI with the filtered row highlighted.

One environment note for anyone reproducing this: if ELECTRON_RUN_AS_NODE=1 is set in the shell, every Electron binary runs as plain Node and exits 0 with no output. Unset it (env -u ELECTRON_RUN_AS_NODE …) or nothing appears to happen.

Still not verified: headed/visible window modes (all runs above were headless and took no focus).

@Lumos-789

Copy link
Copy Markdown
Author

Payload refreshed to 6a191cf — the only diff is one CHANGELOG.md bullet.

That file is part of the ZCode payload (package.json files), and it fell one line behind after this repo grew scripts/pack-dsh-bundle.sh for the DSH submission. tools/verify_pr_payload.py reported the drift, so I re-cut the payload: all 11 files are byte-identical to the repo at 6a191cf again.

No change to the skill, plugin.json, marketplace.json or any code.

@Lumos-789

Copy link
Copy Markdown
Author

Resubmitted at 0.1.6 — head 5dcf563, 15 files, tools/verify_pr_payload.py reports MATCH against Lumos-789/malo-browser@1a53964.

0.1.6 is a full review pass: 33 fixes, each reproduced before it was fixed. Tests 232 → 254. The ones a reviewer can check without a browser:

  • Two sessions sharing a profile no longer cross-wire downloads or network-failure tracking — Electron keeps one webRequest listener per session, and will-download fires for every webContents in the partition, so setSavePath used to race between sessions. Handlers are now installed once per session and dispatched by webContents id; an unowned download is cancelled.
  • The stage blocks redirects, not just navigations: a server-side 3xx emits will-redirect, which was never hooked, so an allowlisted origin could 302 somewhere else and the remaining steps ran there.
  • goto consults the scene allowlist first — webContents.loadURL emits no will-navigate, so tool-driven navigation was unchecked.
  • ${env:NAME} is limited to SCENE_*; a third-party scene could otherwise interpolate any process.env value (cloud keys, tokens) into a URL or a form field, with the allowlist declared by that same scene.
  • demo:// validates the scene id instead of using the URL segment as the containment root (demo://scene/..%2Fprivate%2Fcredential.json escaped one level up), and a 404 no longer echoes a filesystem path to the page.
  • A rejected take no longer claims "no video was written" while a playable mp4 sits on disk; ffmpeg/ffprobe run with timeouts and are killed instead of hanging stop() forever.
  • Invalid selectors are E_BAD_REQUEST on every tool instead of a misleading E_STEP_TIMEOUT; assert value on a password input is rejected (comparing the redacted literal made value: "[redacted]" pass); run_steps rejects login, as the docs always claimed.
  • schemas/scene.schema.json accepts the same step fields as the runtime, with a test that fails when the two drift — that drift is how target_text, modifiers and coordinate targets became documented-but-rejected.

Payload and text changes:

  • version 0.1.5 → 0.1.6 in .zcode-plugin/plugin.json, the payload's marketplace.json entry and the main repo. Description and description_i18n are unchanged and still byte-identical between the manifest and the marketplace entry.
  • The PR title said "scripted browser demo recorder" while the description and the shipped skills cover the agent runtime too — retitled to "agent browser runtime and demo recorder", and this body now carries a 0.1.6 section.

Kept from the earlier round: the skill is install/config/troubleshooting only (no source copy in the payload); network_capture / responses stay documented with the subscribe-before-navigate ordering and the OOPIF/Service-Worker boundary; response bodies never reach logs, evidence or run reports (run_steps refuses both tools).

Not covered: this payload round ran no Electron itself. The 0.1.6 fixes were verified in the main repo during the fix round — --mode=record self-checks, a multi-step autorun scene walk, five scene cases (redirect blocked, goto outside the allowlist refused, ${env:} outside SCENE_ refused) and a 14-probe daemon run covering shared-profile isolation plus 0700/0600 permissions. The payload is byte-identical to that tree (verify_pr_payload.py MATCH), so the code under review is the code that ran.

@Lumos-789 Lumos-789 changed the title feat: add malo-browser (agent browser runtime and demo recorder) feat(malo-browser): add the agent browser runtime and demo recorder plugin (0.1.6) Nov 5, 2026
@Lumos-789

Copy link
Copy Markdown
Author

Re-pushed this branch as b34714a2 and fixed two classes of defect that were ours, not the repository's. Nothing about the plugin's behaviour changed.

1. The PR now follows .github/PULL_REQUEST_TEMPLATE.md. The body had drifted into hand-written prose and lost the two parts a reviewer checks first: the Plugin type declaration and the Self-check list. Both are back, the five sections CONTRIBUTING asks for are filled in (problem / visible behaviour and testing / version and marketplace registration / dependencies and side effects / licensing), and the title carries the (<plugin-name>) scope the template requires for the squash commit on main.

2. The entry named the repo-root assets/malo-browser/icon.png, but that file was never uploaded with the branch — the catalogue would have shown a broken icon. It is committed now (512×512) and the entry uses the same absolute CDN form as every other icon-bearing entry in marketplace.json.

Re-verified against the branch itself (not a local checkout):

check result
python3 scripts/validate.py @ b34714a2 OK: 53 plugin(s) validated
plugin artifact via build_zip() — the same code path CI runs 69266 B, sha256 432eee9758c2721bd0e0e52628c6af1b67e046fbf6026188f43074a3b39cd621
assets/malo-browser/icon.png present, 512×512
README.md + README_CN.md in the payload both present

Request: the earlier validate status was bound to 5dcf563a, so it is stale after this push. Could you re-run the external validation on b34714a2? The full build_dist.py still needs the repository's UI-plugin pnpm build and fails identically on main (excalidraw: incomplete build; run pnpm build first), which is why I am relying on CI for that step.

中文摘要:本轮只补元数据与载荷完整性 —— 恢复了模板要求的插件类型声明与自查清单、标题加上 (插件名) scope(模板说明该标题会成为 main 上的 squash commit)、把条目引用的根级图标真正提交并以 CDN 绝对地址引用。分支上 validate.py 通过,插件产物按 CI 同一代码路径打包成功(体积与 sha256 见上表)。旧的 validate 状态绑定在 5dcf563a 上已失效,请在 b34714a2 上重跑一次校验。

@Lumos-789 Lumos-789 changed the title feat(malo-browser): add the agent browser runtime and demo recorder plugin (0.1.6) feat(malo-browser): add the agent browser runtime and demo recorder plugin (0.1.11) Nov 8, 2026
@Lumos-789

Copy link
Copy Markdown
Author

Update: payload bumped to 0.1.11 (head e8217e7). Payload refreshed to the current main repo (byte-identical, verified MATCH). This round is a behavior-preserving cleanup: redundant comments and dead code removed, docs re-aligned with the code. Full test suite, packaging gates (plugin acceptance / simplify / public release) and the install lifecycle matrix all pass locally.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants