feat(discovery): Lever, Ashby, and Workday ATS adapters - #32
Merged
Conversation
Extend the Fetcher/BrowserFetcher protocols and default_fetcher with three optional, GET-defaulted keyword params — method, json_body, headers — so an ATS adapter can issue a JSON POST (Workday's CxS API) over the same seam. Backward compatible by construction: the existing GET call sites pass none of these and are unchanged, and runtime_checkable checks method presence not signature. default_fetcher gains an httpx.request branch (merging headers over the default User-Agent) and stays # pragma: no cover. The test FakeFetcher now records method/json_body/headers on each FetchCall and accepts an optional results sequence (replayed in order, raising when exhausted) so the paginated Workday poll can script one page per offset — the fake carries the whole test load. Full suite green at 100% line+branch, mypy --strict clean (incl. win32).
The second ATS discovery source (PROJECT.md §5.4-A): Lever's public Postings API. A drop-in on the existing registry — the poller/service/CLI are generic over the ats_type string, so only the adapter + one _ADAPTERS entry are new. - LeverAdapter.detect recognizes both the public board URL (jobs.lever.co/<site>, jobs.eu.lever.co) and the raw API URL (api.lever.co/v0/postings/<site>, api.eu.lever.co) — token = the site. - list_postings hits GET api.lever.co/v0/postings/<site>?mode=json, which returns a RAW JSON ARRAY (so the shape guard is 'not a list' rather than a missing 'jobs' key), and maps id/text/hostedUrl(or applyUrl)/categories/workplaceType/ descriptionPlain(or stripped HTML)/createdAt onto a DiscoveredPosting, skipping malformed postings and propagating FetchError. Registered in _ADAPTERS (ATS_TYPES auto-derives to greenhouse+lever). The registry and poller tests that hard-coded 'lever' as the unknown provider now use 'smartrecruiters' (documented but unregistered). Tests mirror the Greenhouse suite; 100% line+branch, mypy --strict clean (incl. win32). Known limitation: list_postings always polls the US base; an EU-only board may 404 — encoding the region in board_ref is a follow-up.
The third ATS discovery source (PROJECT.md §5.4-A): Ashby's public Job Posting
API — another drop-in on the registry.
- AshbyAdapter.detect recognizes the public board URL (jobs.ashbyhq.com/<name>)
and the raw API URL (api.ashbyhq.com/posting-api/job-board/<name>).
- list_postings hits GET api.ashbyhq.com/posting-api/job-board/<name>?includeCompensation=false
({apiVersion, jobs: [...]}), and maps title/location/employmentType/workplaceType/
descriptionPlain(or stripped descriptionHtml)/publishedAt onto a DiscoveredPosting.
- Ashby jobs carry NO top-level id, so the external id is derived from jobUrl's
last path segment (a UUID), falling back to applyUrl — isolated in a helper with
its own branch coverage. Unlisted postings (isListed false) are skipped; apply_url
prefers applyUrl then jobUrl.
Registered in _ADAPTERS (ATS_TYPES → ashby+greenhouse+lever); registry test
extended. Tests mirror the Greenhouse suite plus the id-derivation and
isListed-filter branches. 100% line+branch, mypy --strict clean (incl. win32).
The fourth ATS discovery source (PROJECT.md §5.4-A): Workday's per-tenant CxS
API — the first adapter to use the POST Fetcher seam.
- WorkdayAdapter.detect recognizes the public board URL
(<tenant>.<wdN>.myworkdayjobs.com/<locale?>/<site>) and the raw CxS API URL
(.../wday/cxs/<tenant>/<site>/jobs), validating the wd-datacenter label and
skipping a leading locale segment. Because the registry stores one board_ref
string but Workday needs three values, detect emits a compound
'<tenant>:<wd>:<site>' token that list_postings parses back apart (guarding the
part count).
- list_postings POSTs {appliedFacets, limit, offset, searchText} with
Accept: application/json and paginates offset by _LIMIT until len >= total, an
empty page, or a _MAX_PAGES cap (≤200) — logging a warning when capped (no
silent truncation, AGENTS.md). Maps title/externalPath/locationsText/postedOn,
deriving external_id from the trailing _JR id and the apply URL from the base +
externalPath.
Registered in _ADAPTERS (ATS_TYPES → ashby+greenhouse+lever+workday). Tests cover
detection (board + CxS URL + rejections), the POST body/headers, pagination
(offsets 0,20), the cap warning (caplog), empty-page early stop, malformed
board_ref / non-JSON / unexpected-shape errors, JR-id parsing, and skipped
malformed jobs. 100% line+branch, mypy --strict clean (incl. win32).
Known limitation: the apply URL omits any locale; a tenant that requires one may
404 — carrying an optional locale in board_ref is a follow-up.
Update the docs for the three new ATS adapters (Definition of Done, AGENTS.md §8): - STATUS.md: bump 'Last updated' + 'Current phase'; add the 'What has landed' entry for the Lever/Ashby/Workday adapters + Fetcher POST extension; drop the 'More ATS adapters' item from 'Next up' (add small ATS follow-ups) and re-order the rest (aggregators, multiple profiles, IPC); tick the Phase-2 table row. - PROJECT.md §15: check off the company-watchlist + ATS-adapters item (all four). - CHANGELOG.md (Unreleased/Added): the three adapters + the Fetcher POST seam. - README.md: list all four supported providers (board + API URL forms).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
Broadens Phase-2 discovery from Greenhouse-only to all four ATS providers the roadmap
names — Lever, Ashby, and Workday (STATUS.md "Next up" #1, PROJECT.md §5.4-A) — so the
watchlist and daemon cover far more companies. Each is a new adapter dropped onto the existing
atlas.discovery.atsregistry; the poller, watchlist service, andatlas company addarealready generic over the
ats_typestring, so no downstream code changed.detect(url)accepts both the public board URL and the raw API URL for each provider.
Key commits
feat(scrape): support POST + JSON body on the Fetcher seam— the one shared change: optional,GET-defaulted
method/json_body/headersonFetcher/BrowserFetcher/default_fetcherso Workday can POST over the same boundary. Backward compatible (existing GET callers
unchanged); the test
FakeFetcherrecords them and can replay a page sequence.feat(ats): add lever board adapter—GET api.lever.co/v0/postings/<site>?mode=json, a rawJSON array; maps id/text/hostedUrl/categories/workplaceType/descriptionPlain/createdAt.
feat(ats): add ashby board adapter—GET api.ashbyhq.com/posting-api/job-board/<name>; Ashbyjobs carry no id, so the external id is derived from the job URL, and unlisted postings are
skipped.
feat(ats): add workday board adapter— the per-tenant CxS API, a POST that paginates;a compound
<tenant>:<wd>:<site>board reference (detect emits it, list_postings parses it),offset paging to a
_MAX_PAGEScap with a warning if capped (no silent truncation).docs: record lever/ashby/workday ATS discovery— STATUS / PROJECT §15 / CHANGELOG / README.Each adapter's registry entry lands in its own commit (a module not in
_ADAPTERSis dead code),keeping every commit self-contained and green.
Testing
Fully hermetic (AGENTS.md §6.2): inline recorded-JSON fixtures replayed through
FakeFetcher(no real network). Per-adapter detection tables (board + API URL → token; cross-provider + non-ATS
→
None), happy-path normalization, empty board, malformed JSON, missing/wrong array,skipped-malformed-job, and
FetchErrorpropagation — plus each adapter's quirks: Lever raw-arrayisListedfilter, and Workday POST body/headers,pagination (offsets 0/20), cap warning (
caplog), empty-page early stop, malformed-board-ref /unexpected-shape errors, and JR-id parsing. Gates green:
ruff format --check,ruff check,mypy --strict(incl.--platform win32), and 959 tests at 100% line+branch coverage. Nonew dependency (
httpx+ stdlib).Design notes for review
kwargs, so the two existing GET call sites and the poller's local sequenced fake needed only a
signature widening (no behavior change).
board_ref(<tenant>:<wd>:<site>) is how a single registry stringcarries the tenant + datacenter + site the CxS endpoint needs;
atlas company listrenders itliterally (a
--nameoverrides the display).boards may 404 — encode region in
board_ref); Workday apply URLs omit any locale segment (sometenants 404 — carry an optional locale in
board_ref).smartrecruitersreplacedleveras the "still-unknown provider" in the registry/pollernegative tests once Lever registered.