docs: publish the compatibility matrix as machine-readable data - #146
Merged
Merged
Conversation
docs/compatibility.json carries, for each of the 53 entries, the per-OS result with whatever the cell says beside the verdict, and the quirks the harness actually applies to that database. Nobody else publishes verified per-database/per-driver/per-OS capability data, and a table only a human can read cannot be consumed by anything else. Generated by scripts/gen_compatibility_json.py from docs/COMPATIBILITY.md and the DBS dict in tests/compat/test_matrix.py, so the quirks come from the code that runs rather than from prose. Connection strings and fixture plumbing are left out: they are local detail, and publishing a connection string invites copy-paste. Two things the generator refuses to guess, both learned the hard way while writing it: - The two tables in COMPATIBILITY.md are in different orders, so joining them by position put QuestDB's results under Microsoft Access's name. Matching the entry id against the display name looked safer but silently paired "db2" with "Db2 for i" and "ibmi" with "IBM Informix" - each had exactly one match, so an "exactly one candidate" check passed both. The mapping is now written out for all 53 entries and asserted to be a bijection, so a row added to one table and not the other fails the build. - The per-OS totals are counted, then checked against the 53/45/48 the README and the site quote. The script exits rather than emit a number that disagrees with its source, which is the failure this project has had before in hand-written copy. CI regenerates the file and diffs it, so the published data cannot drift from the table.
Two things CI caught that local runs could not. Importing tests/compat/test_matrix.py pulls in pyarrow, which the version-agreement job does not have and should not need - it is the cheap job that runs plain python3 scripts. The DBS dict is now read with ast instead, so nothing is imported and no test module runs for its data. Values that are not literals are kept as their source text, which also reads better in the output: "pa.bool_(): pa.int8()" rather than "DataType(bool)", and "str.upper" rather than "<method 'upper' of 'str' objects>". The generated file also carried a generation date and the short commit, which made it different on every run. CI regenerates and diffs it, so the output has to be reproducible: both fields are gone, and git already records when the file changed and at what commit. Verified: two consecutive runs produce identical bytes, and the script runs on a python3 with no pyarrow installed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
docs/compatibility.json: for each of the 53 entries, the per-OS result with whatever the cell says beside the verdict, plus the driver quirks the harness actually applies.The reasoning: verified per-database, per-driver, per-OS capability data for ODBC drivers does not exist anywhere else, and a table only a human can read cannot be consumed by anything else. This makes the project useful to people who never install it.
Generated, never edited
scripts/gen_compatibility_json.pyreadsdocs/COMPATIBILITY.mdand theDBSdict intests/compat/test_matrix.py, so the quirks come from the code that runs rather than from prose. Connection strings and fixture plumbing are excluded — local detail, and a published connection string invites copy-paste.Two things it refuses to guess, both learned while writing it
The two tables are in different orders. Joining them by position put QuestDB's results under Microsoft Access's name. Falling back to matching the entry id against the display name looked safe and was worse: it silently paired
db2with "Db2 for i" andibmiwith "IBM Informix", and because each had exactly one candidate, an "exactly one match" check passed both. The mapping is now written out for all 53 entries and asserted to be a bijection, so a row added to one table and not the other fails the build instead of producing plausible nonsense.The per-OS totals are counted and then checked against the 53 / 45 / 48 that
README.md,docs/index.mdand the landing page quote. The generator exits rather than emit a figure that disagrees with its source. It also forced me to fix my own result parser: an early version counted macOS at 33 because it only understoodPASS (detail)and notPASS (detail) — proseorPASS with <driver> …, while treatingdriver unavailable: …andserver not runnable here: …as their own statuses rather than failures. With that right, the count matches the published figures exactly.Drift protection
CI regenerates the file and diffs it, alongside the existing version-agreement and
rust/csrcchecks. So the data cannot fall behind the table, and a matrix change that would alter the published counts fails the build rather than going out quietly.docs/COMPATIBILITY.mdand the README both point at it.