From d9d7bcef0bd7a33d964cfd9be443661cde7e9555 Mon Sep 17 00:00:00 2001 From: Abdullah Hashim Date: Tue, 16 Jun 2026 12:54:03 -0400 Subject: [PATCH 1/3] Position axe-cuprite as Cuprite-only (#31) Drop the "works with any real-browser Capybara driver" claim from the README, gemspec description, and AGENTS.md. Cuprite is the only supported and tested driver; the driver-neutral JS API is now framed as the implementation detail that makes the gem work on Cuprite and avoid Selenium internals, not a promise of broad driver support. The best-effort non-Ferrum fallback in Injector#evaluate_axe is kept but documented as unsupported/untested. Add a unit test pinning that fallback: with no Ferrum page, evaluate_axe routes through evaluate_async_script under the explicit timeout rather than Capybara.default_max_wait_time, so the path can't silently break. --- AGENTS.md | 6 ++++-- CHANGELOG.md | 17 +++++++++++++++++ README.md | 18 +++++++++++------- axe-cuprite.gemspec | 4 ++-- spec/injector_spec.rb | 44 +++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 78 insertions(+), 11 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 6a79281..e251874 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -15,8 +15,10 @@ The whole reason this gem exists: Deque's official `axe-core-capybara` reaches i Capybara driver's **Selenium** `browser` object to inject and run axe, so it breaks on **Cuprite** (the Ferrum/CDP headless-Chrome driver, which has no Selenium browser). This gem drives axe exclusively through Capybara's **driver-neutral** JS API (`execute_script`, -`evaluate_async_script`) and Ferrum's public API — **never** Selenium internals. Cuprite is the -primary must-pass target; staying driver-agnostic is a bonus, not the goal. +`evaluate_async_script`) and Ferrum's public API — **never** Selenium internals. **Cuprite is +the only supported and tested driver.** The driver-neutral JS approach is an implementation +detail that makes the gem work on Cuprite — not a promise of broad driver support. Other +real-browser drivers may work via a best-effort fallback, but they are unsupported and untested. **The only runtime dependency is Capybara.** Do not add a runtime dependency on cuprite, ferrum, selenium, or rspec — consumers bring their own driver/framework. cuprite/ferrum/rspec are diff --git a/CHANGELOG.md b/CHANGELOG.md index 3b66aa6..5e8c1fb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,23 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Changed +- Repositioned as **Cuprite-only**: the README, gemspec `description`, and + `AGENTS.md` no longer advertise support for "any real-browser Capybara driver." + Cuprite is the only supported and tested driver; the driver-neutral JS approach + is described as the implementation detail that makes the gem work on Cuprite + (and avoid Selenium internals), not a promise of broad driver support. The + best-effort non-Ferrum fallback in `Injector#evaluate_axe` is kept but is now + documented as unsupported/untested + ([#31](https://github.com/Guided-Rails/axe-cuprite/issues/31)). + +### Added +- A unit test pinning the best-effort non-Ferrum fallback in + `Injector#evaluate_axe`: with no Ferrum page it routes through + `evaluate_async_script` under the explicit timeout rather than + `Capybara.default_max_wait_time`, so the path can't silently break + ([#31](https://github.com/Guided-Rails/axe-cuprite/issues/31)). + ## [0.2.0] - 2026-06-10 ### Added diff --git a/README.md b/README.md index 7b3c237..427a854 100644 --- a/README.md +++ b/README.md @@ -23,9 +23,12 @@ they break. just a JavaScript file — a driver-agnostic engine — so we drive it exclusively through Capybara's **driver-neutral** JavaScript API (`execute_script`, `evaluate_async_script`), which Cuprite fully implements. That single decision is -what makes this gem work where the official one doesn't. As a bonus it stays -driver-agnostic (it works on any real-browser Capybara driver), but **Cuprite is -the primary, must-pass target.** +what makes this gem work where the official one doesn't. + +**Cuprite is the only supported and tested driver.** Driving axe through the +driver-neutral JS API is what lets the gem avoid Selenium internals — not a promise +of broad driver support. Other real-browser drivers may happen to work (a +best-effort fallback exists), but they are unsupported and untested. There is no runtime dependency on Selenium, Cuprite, or Ferrum — the only runtime dependency is Capybara. You bring your own driver. @@ -197,10 +200,11 @@ page routinely exceeds. axe-cuprite avoids this trap: on Cuprite it calls Ferrum's `page.evaluate_async(script, explicit_wait, *args)` **directly**, with its own timeout (default **30s**, configurable) that is completely **decoupled from -`default_max_wait_time`**. On non-Ferrum drivers it falls back to -`evaluate_async_script` under a temporarily-raised wait time. If axe still -doesn't finish, you get a clear `AxeCuprite::TimeoutError` telling you to raise -the timeout or scope the run with `.within`. +`default_max_wait_time`**. (On a non-Ferrum driver there is a best-effort, +untested fallback to `evaluate_async_script` under a temporarily-raised wait time — +but Cuprite is the only supported driver.) If axe still doesn't finish, you get a +clear `AxeCuprite::TimeoutError` telling you to raise the timeout or scope the run +with `.within`. Tune it globally (`c.timeout = 60`) or per assertion (`.with_timeout(60)`). diff --git a/axe-cuprite.gemspec b/axe-cuprite.gemspec index 8583fd5..dbc96d5 100644 --- a/axe-cuprite.gemspec +++ b/axe-cuprite.gemspec @@ -14,8 +14,8 @@ Gem::Specification.new do |spec| Capybara system/feature tests and exposes the results as RSpec matchers (be_axe_clean / be_accessible). Unlike Deque's official axe-core-capybara gem, it never touches Selenium-specific driver internals: axe is driven entirely - through Capybara's driver-neutral JavaScript API, so it works with Cuprite - and any other real-browser driver. + through Capybara's driver-neutral JavaScript API, which is what makes it work + on Cuprite. Cuprite is the only supported and tested driver. DESC spec.homepage = "https://github.com/Guided-Rails/axe-cuprite" diff --git a/spec/injector_spec.rb b/spec/injector_spec.rb index b399f45..c9e998b 100644 --- a/spec/injector_spec.rb +++ b/spec/injector_spec.rb @@ -50,4 +50,48 @@ def ferrum_js_error(message) expect(classify(StandardError.new("the request timed out somewhere"))).to be(false) end end + + # The non-Ferrum fallback in #evaluate_axe is best-effort and unsupported (the + # suite runs under Cuprite, where ferrum_page is always truthy), but it should + # not silently break. A driver with no #browser makes the real ferrum_page + # return nil, forcing the fallback; we then assert it routes through + # evaluate_async_script under our explicit timeout rather than + # Capybara.default_max_wait_time. + describe "#evaluate_axe non-Ferrum fallback" do + subject(:injector) { described_class.new(page) } + + let(:page) { instance_double(Capybara::Session, driver: non_ferrum_driver) } + let(:non_ferrum_driver) { Object.new } # does not respond to #browser + let(:context) { { include: [["#main"]] } } + let(:options) { { runOnly: "color-contrast" } } + let(:timeout) { 17 } + + def evaluate + injector.send(:evaluate_axe, context, options, timeout) + end + + it "falls back to evaluate_async_script with the run JS and args" do + result = { "violations" => [], "incomplete" => [] } + allow(page).to receive(:evaluate_async_script).and_return(result) + + expect(evaluate).to eq(result) + expect(page).to have_received(:evaluate_async_script) + .with(described_class::RUN_JS, context, options) + end + + it "runs under the explicit timeout, not Capybara.default_max_wait_time" do + Capybara.default_max_wait_time = 2 + observed_wait = nil + allow(page).to receive(:evaluate_async_script) do + observed_wait = Capybara.default_max_wait_time + { "violations" => [], "incomplete" => [] } + end + + evaluate + + expect(observed_wait).to eq(timeout) + # And the temporary bump is restored afterwards. + expect(Capybara.default_max_wait_time).to eq(2) + end + end end From 6422f4bf9616468bd8d10276eadeb8de58b652cd Mon Sep 17 00:00:00 2001 From: Abdullah Hashim Date: Tue, 16 Jun 2026 13:19:40 -0400 Subject: [PATCH 2/3] wip --- spec/injector_spec.rb | 65 +++++++++++++++++++++++-------------------- 1 file changed, 35 insertions(+), 30 deletions(-) diff --git a/spec/injector_spec.rb b/spec/injector_spec.rb index c9e998b..c39365c 100644 --- a/spec/injector_spec.rb +++ b/spec/injector_spec.rb @@ -3,51 +3,50 @@ require "spec_helper" RSpec.describe AxeCuprite::Injector do - # timeout_error? decides whether an evaluation failure is rewritten as a - # helpful AxeCuprite::TimeoutError or re-raised untouched. It is private; we - # drive it directly because the distinction is subtle and class-driven. - describe "#timeout_error? classification" do - subject(:injector) { described_class.new(nil) } - - def classify(error) - injector.send(:timeout_error?, error) - end + describe "#timeout_error?" do + it "treats Ferrum::TimeoutError as a timeout" do + error = Ferrum::TimeoutError.new + injector = described_class.new(nil) - # A Ferrum::JavaScriptError carrying a given message. We stub #message rather - # than drive Ferrum's constructor: how it derives a message from a CDP - # response is a private detail that varies across ferrum patch releases, and - # all timeout_error? looks at is the class name and the message. - def ferrum_js_error(message) - error = Ferrum::JavaScriptError.allocate - allow(error).to receive(:message).and_return(message) - error - end + result = injector.send(:timeout_error?, error) - it "treats Ferrum::TimeoutError as a timeout (the Cuprite fast-path)" do - # Its own message even mentions a timeout, but the class is what counts. - expect(classify(Ferrum::TimeoutError.new)).to be(true) + expect(result).to be(true) end - it "treats Ferrum::ScriptTimeoutError as a timeout (evaluate_async's own timeout)" do - expect(classify(Ferrum::ScriptTimeoutError.new)).to be(true) + it "treats Ferrum::ScriptTimeoutError as a timeout" do + error = Ferrum::ScriptTimeoutError.new + injector = described_class.new(nil) + + result = injector.send(:timeout_error?, error) + + expect(result).to be(true) end it "treats a Ferrum::JavaScriptError carrying 'timed out promise' as a timeout" do - # This is how Ferrum surfaces its async-evaluation (evaluate_async) timeout. error = ferrum_js_error("Error: timed out promise") - expect(classify(error)).to be(true) + injector = described_class.new(nil) + + result = injector.send(:timeout_error?, error) + + expect(result).to be(true) end it "does NOT treat a real JS error whose message merely contains 'timeout' as a timeout" do - # The regression this issue is about: a genuine page-side error from axe or - # the app whose text happens to mention a timeout must propagate as-is, not - # get rewritten with misleading 'increase the timeout' guidance. error = ferrum_js_error("TypeError: Cannot read properties of null (reading 'timeout')") - expect(classify(error)).to be(false) + injector = described_class.new(nil) + + result = injector.send(:timeout_error?, error) + + expect(result).to be(false) end it "does NOT treat an arbitrary StandardError mentioning 'timed out' as a timeout" do - expect(classify(StandardError.new("the request timed out somewhere"))).to be(false) + error = StandardError.new("the request timed out somewhere") + injector = described_class.new(nil) + + result = injector.send(:timeout_error?, error) + + expect(result).to be(false) end end @@ -94,4 +93,10 @@ def evaluate expect(Capybara.default_max_wait_time).to eq(2) end end + + def ferrum_js_error(message) + error = Ferrum::JavaScriptError.allocate + allow(error).to receive(:message).and_return(message) + error + end end From 00c1effa106a4b04eff087823ce9dbbee07251a8 Mon Sep 17 00:00:00 2001 From: Abdullah Hashim Date: Tue, 16 Jun 2026 13:54:36 -0400 Subject: [PATCH 3/3] wip --- spec/injector_spec.rb | 55 ++++++++++++++++++++++--------------------- 1 file changed, 28 insertions(+), 27 deletions(-) diff --git a/spec/injector_spec.rb b/spec/injector_spec.rb index c39365c..98e2358 100644 --- a/spec/injector_spec.rb +++ b/spec/injector_spec.rb @@ -50,46 +50,47 @@ end end - # The non-Ferrum fallback in #evaluate_axe is best-effort and unsupported (the - # suite runs under Cuprite, where ferrum_page is always truthy), but it should - # not silently break. A driver with no #browser makes the real ferrum_page - # return nil, forcing the fallback; we then assert it routes through - # evaluate_async_script under our explicit timeout rather than - # Capybara.default_max_wait_time. - describe "#evaluate_axe non-Ferrum fallback" do - subject(:injector) { described_class.new(page) } - - let(:page) { instance_double(Capybara::Session, driver: non_ferrum_driver) } - let(:non_ferrum_driver) { Object.new } # does not respond to #browser - let(:context) { { include: [["#main"]] } } - let(:options) { { runOnly: "color-contrast" } } - let(:timeout) { 17 } - - def evaluate - injector.send(:evaluate_axe, context, options, timeout) - end - + describe "#evaluate_axe" do it "falls back to evaluate_async_script with the run JS and args" do - result = { "violations" => [], "incomplete" => [] } - allow(page).to receive(:evaluate_async_script).and_return(result) - - expect(evaluate).to eq(result) + page = instance_double(Capybara::Session, driver: Object.new) + expected_result = { "violations" => [], "incomplete" => [] } + allow(page).to receive(:evaluate_async_script).and_return(expected_result) + injector = described_class.new(page) + + result = injector.send( + :evaluate_axe, + { include: [["#main"]] }, + { runOnly: "color-contrast" }, + 17 + ) + + expect(result).to eq(expected_result) expect(page).to have_received(:evaluate_async_script) - .with(described_class::RUN_JS, context, options) + .with( + described_class::RUN_JS, + { include: [["#main"]] }, + { runOnly: "color-contrast" } + ) end it "runs under the explicit timeout, not Capybara.default_max_wait_time" do + page = instance_double(Capybara::Session, driver: Object.new) Capybara.default_max_wait_time = 2 observed_wait = nil allow(page).to receive(:evaluate_async_script) do observed_wait = Capybara.default_max_wait_time { "violations" => [], "incomplete" => [] } end + injector = described_class.new(page) - evaluate + injector.send( + :evaluate_axe, + { include: [["#main"]] }, + { runOnly: "color-contrast" }, + 17 + ) - expect(observed_wait).to eq(timeout) - # And the temporary bump is restored afterwards. + expect(observed_wait).to eq(17) expect(Capybara.default_max_wait_time).to eq(2) end end