feat(webdriver): spoof navigator.webdriver to false in every realm

[vringar]

What this is

navigator.webdriver reads as false in every realm the page can reach. New BrowserParams.spoof_webdriver, default True.

This is deliberately the smallest useful slice of #1154. navigator.webdriver is the one detection vector in that PR's D1–D10 suite that is independent of instrumentation — it is true whether or not OpenWPM instruments anything, because Selenium sets it. Splitting it out makes it possible to measure how much of a crawl's differential treatment by sites is attributable to this one flag versus the rest of the stealth instrument's surface.

It also serves a second purpose: it is a complete, reproducible demonstration of what "cover every realm" actually costs on current Firefox. Ported from #526, open since 2019.

The mypy fix that used to ride along here is now #1236.

Closes #526.

Mechanism

A privileged window actor replaces the page's Navigator.prototype.webdriver getter:

const spoofedGetter = Cu.exportFunction(() => false, pageWindow.Object.create(null), {
  defineAs: "get webdriver",
});
Object.defineProperty(navigatorPrototype, "webdriver", { ...descriptor, get: spoofedGetter });

The getter is compiled inside the page's compartment, so it reports [native code] under the native accessor name. The native descriptor is reused apart from get, so nothing else moves. Nothing in the content realm is hooked — no patched HTMLIFrameElement.prototype, no proxy — so the spoof adds no page-observable surface of its own.

Why the prototype and not the instance

Object.defineProperty(navigator, "webdriver", {value: false}) adds an own property the navigator instance does not normally have and reorders enumeration — the tell that led #526 to proxy the whole navigator object. Replacing the accessor where Firefox defines it avoids both.

Why an actor and not a content script

An earlier revision of this PR used a content script. It covers almost everything, but not a frame's uncommitted initial about:blank — the placeholder that exists between appendChild and the real document committing. Firefox deliberately never injects content scripts there (bug 1415539, cited in ExtensionContent.sys.mjs), and no configuration changes it: matchAboutBlank does not reach it, and matchOriginAsFallback disables matchAboutBlank while requiring a wildcard path glob <all_urls> does not provide.

That document is not merely transient — it is observable. A page can append a srcdoc iframe and read frame.contentWindow.navigator.webdriver in the same task and get the uncommitted about:blank, uninstrumented. I verified this by reading back the realm the parent actually sees: in both the plain-iframe and srcdoc same-task cases location.href is about:blank, and only the srcdoc one is unpatched.

The distinction that matters:

Firefox guarantees that content scripts run before the document's own scripts.
It does not guarantee that a realm is unreachable from another realm until its content scripts have run.

Those differ exactly here, and only the second is sufficient for a measurement tool. The actor works below the content-script layer: its child observes content-document-global-created, which fires synchronously for every window global, uncommitted ones included.

What full coverage costs

Documented because the stealth work will hit all of it:

• resource: is the only scheme the system ESM loader accepts for an actor module. moz-extension: and the XPI's own jar: URL are both silently declined — no error, the actor simply never runs.
• A resource: substitution is per-process. Registering it only in the parent leaves content processes unable to resolve the module, reported as a bare Failed to load <uri>. bootstrap.js re-registers it per process.
• safeForUntrustedWebProcess: true is mandatory, or registerWindowActor succeeds, reports nothing, and the actor is never instantiated for web content.
• Preallocated content processes. Firefox keeps a pool of blank processes and claims one when a navigation needs a new process. Those launched before the extension started never ran the bootstrap, so a page landing in one was unhooked — the source of every intermittent failure while developing this. Observing process-type-set and pushing a script when a process is claimed does not fix it: that fires in the parent while the process is already loading, and the IPC can lose. Draining the stale pool once at startup does, and preallocation stays enabled for the rest of the crawl.

Verification

test_spoof_webdriver_reaches_other_realms checks every reachable realm against a spoof_webdriver=False baseline: window.open(""), window.open("about:blank"), an iframe appended and read in the same task, about:blank and srcdoc frames, indexed frame access via window[n], and calling a frame's own Navigator.prototype getter directly. All spoofed — including the uncommitted-about:blank case no content script can reach.

test_spoof_webdriver_is_invisible_to_the_page pins that nothing else moves: descriptor.get.toString(), .name, the descriptor flags, the absent setter, hasOwnProperty(navigator, "webdriver"), and the full Object.getOwnPropertyNames(Navigator.prototype) array are byte-identical to baseline.

Both escapes named in the earlier literature are closed on Firefox 155 — #526's window.open("") case and the synchronous-iframe race from #1154's D10 notes — measured rather than inherited.

Run green locally against the unbranded Firefox 155 build this repo pins: test_webdriver_spoof.py + test_js_instrument.py (10), test_update_script.py (15), test_dataclass_validations.py (8), plus npm run lint and black/isort/mypy over all tracked Python files.

Bonus test: instrumentation reaches srcdoc frames

Chasing the above raised a broader question — do OpenWPM's content scripts reach srcdoc frames at all? A gap there would be a silent measurement hole in every crawl, since ad frames and embeds use them heavily. They do: a <script> inside the srcdoc attribute is captured under about:srcdoc, so injection lands before the frame's own code. TestSrcdocFrameInstrumentation pins it. This tests pre-existing behaviour rather than anything this PR changes, so it is easy to drop if you would rather it landed separately.

---

Findings

Investigation notes, kept so they are not lost. Measured on Firefox 155.0.1
(unbranded add-on-devel) and Chromium 152.0.7977.64, Linux x86_64.

Two loaders, two gates

	subscript loader	system ESM loader
gate	IsTrustedScheme or (allowUnsafeURL || security.allow_unsafe_subscript_loads) for file/jar/moz-extension	IsTrustedScheme only
official bypass	two: a per-call option and the pref	none

IsTrustedScheme is resource:, chrome:, moz-src: only
(dom/security/nsContentSecurityUtils.cpp). Actors reach it through
JSActorManager.cpp calling ImportESModule for esModuleURI. An
esModuleURI of moz-extension: or the XPI's own jar: URL is silently
declined — JSWindowActorProtocol::Observe uses GetActor(..., IgnoreErrors()),
so the actor simply never instantiates and nothing is logged.

So the resource://openwpm/ substitution does not bypass that gate, it satisfies
it by aliasing a trusted scheme onto the XPI.

Why the bootstrap and the pool drain are needed

SubstitutingProtocolHandler does propagate a substitution to content
processes (SendSubstitution, and CollectSubstitutions for new ones), and the
shipped webcompat about-compat API relies on exactly that: parent-only
setSubstitution, then a process script loaded over resource://webcompat/.

But propagation is asynchronous with respect to actor registration. Registering
the actor immediately after setSubstitution was measured to fail with a bare
Failed to load resource://openwpm/...: a content process is asked for the
module before the mapping arrives. webcompat avoids this by doing its setup in
onStartup(), long before anything needs the mapping. Having each content
process register the mapping itself, before it hosts a document, is the ordering
guarantee.

Separately, content processes that already exist when the bootstrap is
registered do not run it, so the preallocated pool is replaced rather than
patched by toggling dom.ipc.processPrelaunch.enabled once. That pref is
therefore a requirement, enforced as a ConfigError.

Upstream state

• bug 1976115 — move
  experiment_apis scripts to moz-extension:. Its TODO, and 1974691's, sit
  directly in CheckAllowedURI.
• bug 1974691 —
  restrict moz-extension: subscript loads to privileged extensions. Rob--W
  notes such loads "can happen from extension experiments only", so the branch
  is already privileged-only in practice and the pref is belt-and-braces.
• bug 1771341 —
  canUseAPIExperiment() allows experiment_apis when the add-on is privileged
  or AddonSettings.EXPERIMENTS_ENABLED. OpenWPM is the second case. A
  future check keyed on WebExtensionPolicy::IsPrivileged() would lock us out;
  one keyed on canUseAPIExperiment() would not.

The one case a content script cannot reach

A frame's uncommitted initial about:blank — the placeholder between
appendChild and the real document committing. Firefox never injects content
scripts there (bug 1415539),
and no option changes it. It is observable: a page that appends a srcdoc iframe
and reads frame.contentWindow.navigator.webdriver in the same task reads that
document, and re-checking the saved document object later shows it is never
injected, not merely injected late.

Chromium 152 behaves identically on every case measured, so this is not a
Firefox bug or a compat issue; it is
w3c/webextensions#1009
territory, where the general form
(#513) is closed and
opposed: by all three vendors.

srcdoc origin semantics, measured: it inherits the opener's origin and is fully
scriptable. sandbox="allow-scripts" makes it opaque and unreachable;
allow-scripts allow-same-origin restores same-origin and is still never
injected — so the gap is a property of srcdoc, not of origin opacity, which
also rules out match_origin_as_fallback.

A narrower mechanism exists for this one flag

Navigator::Webdriver() reads Marionette/RemoteAgent state out of sharedData,
which the parent can clear — leaving the native getter untouched entirely. It is
deliberately not used here: it can only move that one flag, while the instrument
this is a slice of needs to overwrite arbitrary properties and to run at realm
creation. Matching the native accessor exactly is the part that generalises,
which is why the tests pin property order and wrong-receiver behaviour.

Tried and rejected

• Parent-only setSubstitution without the per-process bootstrap — loses the
  propagation race, measured.
• matchOriginAsFallback on a content script — per its documentation it
  disables matchAboutBlank and needs a wildcard path glob <all_urls> does
  not provide.
• moz-extension: and jar: as esModuleURI — silently declined.
• Installing the extension unpacked for a file: root — CheckAllowedURI gates
  file: identically.
• A process-type-set observer and an acknowledgement handshake, both of which
  measured unnecessary once the pool is drained.

[Copilot]

🟡 Changes recommended

The new webdriver spoof test doesn’t reliably clean up the TaskManager on failure paths (missing try/finally), which can leak browser processes and destabilize the test suite.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds a small, standalone anti-detection mitigation to OpenWPM by spoofing navigator.webdriver to false (default-on) via a Firefox content script injected at document_start, while also tightening mypy’s execution environment so type-checking reflects the real conda-installed dependencies.

Changes:

• Add BrowserParams.spoof_webdriver (default True) and wire it through to the WebExtension, which registers a new document_start content script that patches Navigator.prototype.webdriver via Xray waiver + exportFunction.
• Add a targeted regression test page + Python test to verify the spoof flips only the value while preserving descriptor shape and prototype property order, and update JS-instrument expectations accordingly.
• Make mypy run as a language: system pre-commit hook and add scripts/update.py automation to keep [tool.mypy].python_version synced to the conda Python version.

File summaries

File	Description
openwpm/config.py	Adds spoof_webdriver BrowserParam and documents behavior/escapes.
Extension/src/webdriver-spoof.ts	Implements the page-visible spoof by replacing Navigator.prototype.webdriver getter.
Extension/src/background/webdriver-spoof.ts	Registers the spoof content script across all frames at document_start.
Extension/src/feature.ts	Wires spoof_webdriver into extension startup ordering (before JS instrument).
Extension/src/types/xray.d.ts	Declares Firefox-only wrappedJSObject/exportFunction for TS typechecking.
Extension/webpack.config.js	Adds a webpack entry to bundle the new content script.
Extension/eslint.config.mjs	Ignores the new bundled output file.
Extension/.prettierignore	Ignores the new bundled output file.
Extension/.gitignore	Ignores the new bundled output file.
test/test_pages/webdriver_spoof.html	Adds an inline, parse-time probe page used for verification.
test/test_webdriver_spoof.py	Adds end-to-end test validating spoof value flip without detectable shape changes.
test/test_js_instrument.py	Updates expectation: instrument should record spoofed navigator.webdriver value.
docs/Configuration.md	Documents spoof_webdriver, default, mechanism, and known escapes.
.pre-commit-config.yaml	Switches mypy to a system hook (conda env) with rationale.
pyproject.toml	Updates mypy python_version to match conda Python feature level.
scripts/update.py	Drops mypy rev syncing and adds sync_mypy_python_version() to prevent drift.
test/test_update_script.py	Updates/update-tests for mypy hook changes and new python_version syncing.

Review details

• Files reviewed: 16/17 changed files
• Comments generated: 1
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[codecov]

Codecov Report

❌ Patch coverage is 17.46032% with 52 lines in your changes missing coverage. Please review.
✅ Project coverage is 61.59%. Comparing base (f655dfa) to head (f7f90ce).

Files with missing lines	Patch %	Lines
test/test_webdriver_spoof.py	7.14%	52 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1230      +/-   ##
==========================================
- Coverage   62.34%   61.59%   -0.75%     
==========================================
  Files          40       41       +1     
  Lines        3930     3992      +62     
==========================================
+ Hits         2450     2459       +9     
- Misses       1480     1533      +53     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[vringar]

Updated: the coverage claims in the original description were inherited from #526 (2019) and the stealth instrument's D10 notes rather than measured. I tested them on Firefox 155 and both are stale.

probe	baseline	spoofed
window.open("")	true	false
window.open("about:blank")	true	false
iframe appended and read in the same task	true	false
about:blank iframe	true	false
indexed frame access, window[n]	true	false
frame's own Navigator.prototype getter, called directly	true	false
srcdoc frame, read in the frame's load handler	true	false
srcdoc frame, read after a macrotask	true	false
srcdoc frame, read in the same task as appendChild	true	true

So the window.open("") escape #526 documented is closed on current Firefox, and so is the synchronous-iframe race. The one real gap is narrower and different from either: a srcdoc document is parsed on a later task than the appendChild that creates it, so a parent reading frame.contentWindow.navigator.webdriver in that same task wins the race. From inside the frame — where page code actually runs — and once the frame has loaded, it reads false. test_spoof_webdriver_reaches_other_realms pins all of this, including the known-true case, so a future Firefox change surfaces.

I also tried matchOriginAsFallback: true and reverted it. Per the API's type documentation it disables matchAboutBlank and requires match patterns with a wildcard path glob, which <all_urls> does not provide; it changed nothing in the probe and risked losing the about:srcdoc coverage matchAboutBlank already gives.

Unrelated finding worth keeping

Chasing the above raised the question of whether OpenWPM's content scripts reach srcdoc frames at all — a gap there would be a silent measurement hole in every crawl, since ad frames and embed widgets use them heavily. They do:

window.navigator.userAgent   about:srcdoc            <- <script> inside the srcdoc attribute
window.navigator.userAgent   .../srcdoc_child.html   <- control, ordinary same-origin frame
window.navigator.userAgent   .../srcdoc_probe.html   <- top level

The about:srcdoc row comes from script running during that frame's parse, so the content script lands before the frame's own code. matchAboutBlank: true covers it. Added as TestSrcdocFrameInstrumentation in test/test_js_instrument.py — it tests existing behaviour rather than anything this PR changes, so it is easy to drop if you would rather it went in separately.

Docs, the webdriver-spoof.ts module comment, and the commit message now state the measured behaviour instead of the inherited claims. The BrowserParams.spoof_webdriver docstring was also cut from 18 lines to 7 — it was restating docs/Configuration.md rather than pointing at it.