test(0120): endpoint conformance suite for 20 major assets - #226
Merged
stkrolikiewicz merged 10 commits intoAug 20, 2026
Conversation
Derived from production volume ranking plus the six s9 majors; covers all three identifier forms. USDT excluded (bug 0172). Records four preliminary findings from the derivation probes for the suite run to verify.
Validates all 7 route groups for the fixed 20-asset list against the live OpenAPI spec plus sanity assertions the schema cannot express. Run 2026-08-19: 752 pass, 55 fail, zero schema failures - every failure is a data defect, spawned as tasks 0207-0211.
The conformance run's USDC failures confirm what okarcz already diagnosed at the store level: 0178 (current_prices groups on the base leg) and 0170 (/ohlcv self-pair). Cross-references updated; 0207/0209 now point at the enrichment-family neighbours as diagnostic starts.
…every failure class Cross-check against the backlog showed all four conformance failure classes are okarcz's tasks with decided contracts or deeper diagnoses. Probed on prod: the 'empty OHLCV windows' are 0170's blast radius, not missing data - all five assets return real buckets in XLM mode. Fresh run evidence appended to 0135 and 0170 histories.
Three crash paths left a run with no report at all: a failed spec fetch passed null to ajv.addSchema, and the pagination walk both validated against a possibly-null $ref and iterated a data array whose shape it had just rejected. Every early exit now goes through the report writer. volume_24h_usd is parse-only (a tracked asset can go a day without a trade) and updated_at tolerates 5 min of producer clock skew - both would otherwise flake on the 0128 re-run.
Re-running the committed suite via the npm entry point 16 minutes after the previous run moved the fail count 54 -> 42. The membership churns in both directions: the sources-empty set lost five assets and gained XRP. 0170 and 0178's counts held exactly stable across all three runs while 0135's three classes moved, which corroborates the root-cause split from the outside. Corrects an earlier claim that counts drift by one or two.
Develop independently allocated 0207-0209 to unrelated tasks after this branch retired its own spawns under those ids. References in 0120, 0135 and 0170 now describe the retired spawns without bare ids so they cannot be read as pointing at the new tasks. 0210/0211 remain the next free ids on the shared sequence.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
tools/scripts/conformance-assets.json) — §9 six majors + top-volume fill from the store, all three identifier forms covered; USDT excluded per 0172. Shared with 0121/0123/0127/0128.tools/scripts/conformance-0120.mjs(npm run conformance:0120): exercises all 7 route groups per asset against production, validates every response (errors included) against the live/api-docs-jsonspec (ajv, JSON Schema 2020-12), then layers sanity assertions the schema cannot express — sentinels, OHLCV invariants, exhaustive pagination walk, batch-vs-single agreement. Paced ≤1 rps for the free usage plan.