Add bounded selector actions with checked target identity

CHANGELOG.md

@@ -3,6 +3,7 @@
 ## Unreleased
 - Added `bun run dev:deploy` for validated extension builds, backups, atomic receipt replacement, rollback, and bounded readiness checks. Reject symlinked writable paths and unsupported default reload platforms before deployment. Extension health checks now require a ready host and an actual extension round-trip.
 - Added a DOM editable fallback to accessibility snapshots. When Chrome's debugger-backed accessibility tree omits an editable element, for example a framework's pre-hydration composer textarea, full-tree snapshots lead with up to 40 `dom_fallback` textbox nodes minted from DOM query results, scanning past accessibility-covered and unsupported matches until the limit of accepted editables or a bounded inspection budget is reached. Each carries a normal revisioned ref that the existing fill, type, and click paths already accept; password inputs are skipped, partial (`root_ref`) snapshots are unchanged, the `max_nodes` budget and `truncated` flag now cover the combined result, and a fallback capture failure never fails the snapshot itself.
+- Added unique CSS selector targets for Standard `click`, `type`, and `fill`, retaining sensitive-field and Commit checks and executing against the checked backend node. Normalize and truncate selector text in operation displays.
 - Replaced blocking human handoffs with task-owned attention notices. Agents can request, inspect, resolve, or dismiss a notice without popup acknowledgement; open notices do not block ordinary commands or task cleanup. Legacy handoffs migrate without restoring a lock, and stale notice IDs cannot clear newer requests. Explicit Pause, ownership, sensitive-field restrictions, and configured Commit review remain independent.
 - Enabled YOLO mode for new and legacy default state so recognizable consequential controls execute in the original `browser_act` call. Turning YOLO mode off restores staged Commit review. Task ownership, origin policy, revision checks, credential isolation, and all other Standard boundaries remain enforced.
 - Preserved existing permissive-mode settings and managed 1Password defaults during the notice cutover, including GUI-host executable discovery and legacy created-tab provenance sanitization.

docs/mcp.md

@@ -72,6 +72,15 @@ For MCP, the capability store namespace is `mcp`; OMP uses `omp`; Pi uses `pi`.
 | `browser_credentials` | `prepare` requires a task tab and expected page revision, then returns an opaque short-lived token when the default-enabled 1Password broker is available and one through three Login items match the host-derived current origin. Owner-only policy can disable it. `fill` consumes that token and selected username, password, or one-time-code field refs without returning any value. `next` advances to another bounded candidate. It never submits the form. |
 | `browser_finish` | Accepts `disposition: "auto" | "close" | "keep"` and optional task-owned `keep_tab_ids`. Automatic mode follows the popup cleanup policy: close task-created tabs while retaining adopted tabs, ask for confirmation, or retain all tabs. Successful finalization ungroups retained tabs, releases ownership, closes the Core connection, and returns closed and retained tab IDs. An open handoff notice never defers finalization; staged Commit review and other in-flight work defer it without destroying resumability. |
 
+For `click`, `type`, and `fill`, supply exactly one of `ref` or `selector`.
+A CSS selector is limited to 1–2048 characters and must match exactly one element;
+missing and ambiguous targets fail before mutation. The same task ownership,
+revision, sensitive-field, and Commit checks apply to either target form.
+Before Commit, the selector is resolved again and checked against the staged
+element identity. Execute against that checked element, never a later selector
+match. Existing `upload_file` selector first-match behavior is unchanged.
+
+
 Every existing-page mutation carries its expected page revision. If navigation or document replacement makes that revision stale, AgentTab rejects the operation rather than selecting a new target.
 
 `automation_route` is `full` for ordinary HTTP, HTTPS, and `about:blank` tabs. It is `tab_only` with `route_reason: "browser_restricted_origin"` for Chrome system pages, extension pages, DevTools, the Chrome Web Store, malformed URLs, and unknown schemes. Page inspection or interaction requested on a `tab_only` tab returns `browser_restricted_origin` with `outcome: "not_started"` and recovery that explicitly says not to retry the same AgentTab route. This is a browser platform boundary, not a policy denial and not permission that can be granted through AgentTab.

host-rs/crates/agenttab-protocol/src/lib.rs

@@ -449,14 +449,23 @@ pub struct BrowserActParams {
 #[serde(tag = "kind", rename_all = "snake_case", deny_unknown_fields)]
 pub enum BrowserAction {
     Click {
-        r#ref: String,
+        #[serde(default, skip_serializing_if = "Option::is_none")]
+        r#ref: Option<String>,
+        #[serde(default, skip_serializing_if = "Option::is_none")]
+        selector: Option<String>,
     },
     Type {
-        r#ref: String,
+        #[serde(default, skip_serializing_if = "Option::is_none")]
+        r#ref: Option<String>,
+        #[serde(default, skip_serializing_if = "Option::is_none")]
+        selector: Option<String>,
         text: String,
     },
     Fill {
-        r#ref: String,
+        #[serde(default, skip_serializing_if = "Option::is_none")]
+        r#ref: Option<String>,
+        #[serde(default, skip_serializing_if = "Option::is_none")]
+        selector: Option<String>,
         text: String,
     },
     Select {
@@ -893,9 +902,53 @@ impl MethodParams {
 
 fn validate_action(method: RpcMethod, action: &BrowserAction) -> Result<(), ProtocolError> {
     match action {
-        BrowserAction::Click { r#ref } => require_ref(method, r#ref),
-        BrowserAction::Type { r#ref, text } | BrowserAction::Fill { r#ref, text } => {
-            require_ref(method, r#ref)?;
+        BrowserAction::Click { r#ref, selector } => match (r#ref, selector) {
+            (Some(value), None) => require_ref(method, value),
+            (None, Some(value)) => require_len(method, value, 1, MAX_SELECTOR_CHARS, "selector"),
+            _ => require(
+                method,
+                false,
+                "click requires exactly one of ref or selector",
+            ),
+        },
+        BrowserAction::Type {
+            r#ref,
+            selector,
+            text,
+        } => {
+            match (r#ref, selector) {
+                (Some(value), None) => require_ref(method, value)?,
+                (None, Some(value)) => {
+                    require_len(method, value, 1, MAX_SELECTOR_CHARS, "selector")?;
+                }
+                _ => {
+                    return require(
+                        method,
+                        false,
+                        "type requires exactly one of ref or selector",
+                    )
+                }
+            }
+            require_len(method, text, 0, MAX_ACTION_TEXT_CHARS, "text")
+        }
+        BrowserAction::Fill {
+            r#ref,
+            selector,
+            text,
+        } => {
+            match (r#ref, selector) {
+                (Some(value), None) => require_ref(method, value)?,
+                (None, Some(value)) => {
+                    require_len(method, value, 1, MAX_SELECTOR_CHARS, "selector")?;
+                }
+                _ => {
+                    return require(
+                        method,
+                        false,
+                        "fill requires exactly one of ref or selector",
+                    )
+                }
+            }
             require_len(method, text, 0, MAX_ACTION_TEXT_CHARS, "text")
         }
         BrowserAction::Select { r#ref, value } => {
@@ -2266,6 +2319,53 @@ mod tests {
             .is_err());
         }
 
+        let (_, act_selectors) = RpcRequest::parse(request(
+            "browser_act",
+            json!({
+                "tab_id": 7,
+                "expected_page_revision": 1,
+                "actions": [
+                    {"kind": "click", "selector": "button.submit"},
+                    {"kind": "type", "selector": "input.search", "text": "agenttab"},
+                    {"kind": "fill", "selector": "input.email", "text": "test@example.com"}
+                ]
+            }),
+            true,
+        ))
+        .unwrap();
+        assert!(matches!(
+            act_selectors,
+            MethodParams::Act(BrowserActParams { actions, .. })
+                if matches!(
+                    actions.as_slice(),
+                    [
+                        BrowserAction::Click { selector: Some(sel1), r#ref: None },
+                        BrowserAction::Type { selector: Some(sel2), r#ref: None, text: t1 },
+                        BrowserAction::Fill { selector: Some(sel3), r#ref: None, text: t2 },
+                    ] if sel1 == "button.submit" && sel2 == "input.search" && t1 == "agenttab" && sel3 == "input.email" && t2 == "test@example.com"
+                )
+        ));
+
+        for action in [
+            json!({"kind": "click"}),
+            json!({"kind": "click", "ref": "e1", "selector": "button"}),
+            json!({"kind": "type", "text": "hello"}),
+            json!({"kind": "type", "ref": "e1", "selector": "input", "text": "hello"}),
+            json!({"kind": "fill", "text": "hello"}),
+            json!({"kind": "fill", "ref": "e1", "selector": "input", "text": "hello"}),
+        ] {
+            assert!(RpcRequest::parse(request(
+                "browser_act",
+                json!({
+                    "tab_id": 7,
+                    "expected_page_revision": 1,
+                    "actions": [action]
+                }),
+                true,
+            ))
+            .is_err());
+        }
+
         let (_, close) = RpcRequest::parse(request("agenttab.close", json!({}), false)).unwrap();
         assert!(matches!(close, MethodParams::Close(_)));