Skip to content

Target background scrolling at observed native areas - #10

Merged
max1874 merged 5 commits into
mainfrom
max/scroll-targets
Nov 5, 2026
Merged

max1874 merged 5 commits into
mainfrom
max/scroll-targets

Conversation

@max1874

@max1874 max1874 commented Nov 4, 2026 •

Copy link
Copy Markdown
Owner

Background scrolling previously sent a synthetic wheel event to the largest area, leaving System Settings sidebar controls unreachable. Scroll operations now select an observed AXScrollArea and press its enabled vertical scrollbar page control through the existing process, identity and delivery guards. The wheel path and largest-area guess are removed; unsupported containers stop explicitly.

Native list descriptions identify their scroll area, and container filtering preserves its scroll choice. Two general policy sentences continue scrolling when a needed control is offscreen. Settings acceptance reads the actual pane title or recognized detail heading, never the sidebar selection, and its existing approval callback holds unrelated preference changes. The coding-agent skill now documents the required scroll target.

Validation on the final implementation:

  • Before: two wheel operations left the sidebar at zero and Keyboard unavailable. After: main-pane down/up and sidebar scrolling changed only the selected container, then native selection opened Keyboard.
  • Real deepseek-flash runs completed Keyboard navigation and Calculator 12 × 34 = 408 with DONE, independent false → true result readings, zero activation events from setup through cleanup, and confirmed owned-process termination.
  • Integrated with the newly merged CLI without changing the native bridge, Agent, action table or model code. Its observe/act path also reached Keyboard with zero activation events and cleanup; this scripted CLI path is integration evidence, not a model-accuracy result. A preceding interactive CLI run reached Keyboard but contained unrelated activation events and is not counted as passed. A stale fingerprint refused input before a fresh decision.
  • Evidence and screenshots remain outside Git in ~/AI/Artifacts/Open Computer Use Goal/: Scroll Model Final Policy, Scroll Calculator Final, CLI Final, 05-Acceptance-Calculator.png, 12-Model-Keyboard.png and 14-CLI-Keyboard-Final.png. Earlier invalid targets, policy loops and activation-interrupted samples remain failures.
  • Existing CI passed for final head 1a50c6b (run 42035579565): Python contracts, native-source guards and the macOS Swift build. No local test/check suites ran, and CI definitions are unchanged. Earlier attempts on the pre-integration head never acquired a hosted macOS runner.

One successful sample per final real-model task is not a full benchmark or an accuracy-improvement claim. No additional executor, activation, pointer, screenshot fallback or planner was introduced. Paid batches remain 79/80, 20/20 and 7/10 calls; integration used no paid API calls.

@max1874 max1874 changed the title Target native scrolling at observed areas Target background scrolling at observed native areas Nov 4, 2026
@max1874
max1874 marked this pull request as ready for review November 5, 2026 13:27
@max1874
max1874 merged commit 169489b into main Nov 5, 2026
2 checks passed
@max1874
max1874 deleted the max/scroll-targets branch November 5, 2026 13:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant