Skip to content

[DOCS] Consolidate the Python docs into SCHEMA_GUIDE.md - #684

Merged
Victor Schappert (vcschapp) merged 5 commits into
mainfrom
docs-consolidation
Aug 26, 2026
Merged

[DOCS] Consolidate the Python docs into SCHEMA_GUIDE.md#684
Victor Schappert (vcschapp) merged 5 commits into
mainfrom
docs-consolidation

Conversation

@sethfitz

@sethfitz Seth Fitzsimmons (sethfitz) commented Aug 19, 2026

Copy link
Copy Markdown
Collaborator

Summary

Replaces #672. Same consolidation — PYDANTIC_GUIDE.md and README.pydantic.md become a single SCHEMA_GUIDE.md, linked from README.md and CONTRIBUTING.md, and GLOSSARY.md grows from 6 to 22 terms — rebased onto the documentation fixes in #683, with the corrections carried into the new guide. Dana Bauer (@danabauer)'s commit is preserved with its authorship; the second commit is the port.

#672 can be closed in favour of this; it does not rebase cleanly, because the files it deletes are the files #683 edits.

Why this replaces it rather than stacking on it

#672 was written against main before the doc fixes landed, so it did two things this branch undoes:

  1. It carried the stale content forward. [DOCS] PYDANTIC_GUIDE.md documents a package layout the repo no longer uses #604's models.py / enums.py / types.py file-type rules, the per-type-subdirectory prescription, and # In buildings/models.py all survive verbatim into SCHEMA_GUIDE.md. Those sections are now taken from the corrected PYDANTIC_GUIDE.md — extracted from the file programmatically rather than retyped, so the two cannot disagree.

  2. It catalogued the defects instead of fixing them. The guide gained a "Docs in the repo that are currently wrong" section listing eight items. Seven are fixed in [BUG] Correct the Python package documentation #683. Only pip install survives, and it now points at the section explaining why. A standing list of known-wrong docs is a workaround for not having a check; [BUG] Correct the Python package documentation #683 adds the check.

Other corrections

The transcribed overture-schema list-types output (two copies) and the --group-by output were regenerated by running the commands, so they show the overture tag that #683 adds. The ModelKey examples were showing BuildingModel / PlaceModel, which are not the names of anything.

GLOSSARY.md's Tag entry said "Seven tags exist today ... There is no plain overture tag, despite what some help text suggests." Accurate when written, and the observation is what prompted #683 to add the tag rather than delete the help that referenced it. Now eight, with a note on what the tag does and does not assert — see the open question in #683, which is the place to weigh in on it.

What importing the guide exposed

SCHEMA_GUIDE.md broke the drift detector twice on arrival, which is a fair argument for having it:

  • A >>> REPL block failed to parse and dropped out of the sweep whole, taking its imports with it. Transcripts are unwrapped now.
  • The guide deliberately shows from overture.schema import Building # ✗ ImportError as a counter-example, and the detector read it as a defect. A line carrying the counter-example glyph is exempt.

Both behaviours have unit fixtures and a mutation that fails them.

Review notes

  • Dana Bauer (@danabauer)'s own checks still hold. Every code block in the guide parses, every overture.* import resolves, every internal link resolves. Two of those are now enforced by tests/test_documented_imports.py on every run rather than checked once.
  • The SCHEMA_CONVENTIONS.md staleness flags in README.md were spot-checked and are accurate: it does describe JSON Schema as the definition mechanism (line 3) and does spell subType (line 37).
  • CONTRIBUTING.md still does not mention the DCO sign-off requirement. Carried over unaddressed, as in [DOCS] test, update, revise, and consolidate schema guides #672.

Closes #685

@github-actions

github-actions Bot commented Aug 19, 2026

Copy link
Copy Markdown

🗺️ Schema reference docs preview is live!

🌍 Preview https://staging.overturemaps.org/schema/pr/684/schema/index.html
🕐 Updated Aug 26, 2026 16:23 UTC
📝 Commit fd6a409
🔧 env SCHEMA_PREVIEW true

Note

♻️ This preview updates automatically with each push to this PR.

@lowlydba

John McCall (lowlydba) commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Suggestion:

With really wide-reaching/complex docs like this (several audiences, several different use cases) I really like to cornerstone them with Diataxis, which is roughly just:

Diátaxis identifies four distinct needs, and four corresponding forms of documentation - tutorials, how-to guides, technical reference and explanation. It places them in a systematic relationship, and proposes that documentation should itself be organised around the structures of those needs.

One of the bonuses of this is models seem very adept at following/implementing it just by name reference with "Do the docs in Diataxis", so you can almost completely offload the information theory cognitive load of organizing big documentation, while still being consistent.

@sethfitz
Seth Fitzsimmons (sethfitz) changed the base branch from main to docs-accuracy-fixes August 24, 2026 20:15
Base automatically changed from docs-accuracy-fixes to main August 26, 2026 15:47
@sethfitz
Seth Fitzsimmons (sethfitz) marked this pull request as ready for review August 26, 2026 15:49
Dana Bauer (danabauer) and others added 2 commits August 26, 2026 08:51
Signed-off-by: Dana Bauer <dana.bauer@gmail.com>
Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>
The consolidation was written against main before the doc fixes landed, so
it brought the stale content forward and catalogued the rest.

Ported from the corrected PYDANTIC_GUIDE.md, extracted from that file
rather than retyped: the module-organization section, the
import-organization example, and the container-to-mixin example built on
the real `Named` / `Appearance` mixins. Section 3.2 is reframed too --
Overture publishes one shape, and GeoJSON is the representation the models
support for compatibility and that the generated JSON Schema describes;
the renamed heading's anchor is chased through GLOSSARY.md and two
in-guide links.

"Docs in the repo that are currently wrong" listed eight items; seven are
fixed, so only `pip install` survives, pointing at the section that
explains it. GLOSSARY.md's Tag entry said there is no plain `overture`
tag, which was true when written and is what prompted adding one; it now
reads eight tags, with a note on what the tag does and does not assert.
The transcribed `list-types` output was regenerated by running the
command.

The imported guide also exposed two holes in the drift detector. A `>>>`
REPL block failed to parse and dropped out of the sweep whole, taking its
imports with it. And the guide deliberately shows `from overture.schema
import Building` as a counter-example, which the detector read as a
defect; a line carrying the counter-example glyph is exempt. Both have unit
fixtures and a mutation that fails them. With those handled and the
validation README's block fixed, no block in the repo is excused.

Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>
Dana Bauer (danabauer) and others added 3 commits August 26, 2026 09:01
The guide had grown to 4,803 lines across four parts, mixing rationale,
consumer instructions, contributor reference, and troubleshooting. The
consolidation issue asked for consumer and contributor material to be
separated rather than interleaved; this promotes that separation from
parts within one file to pages.

  SCHEMA_GUIDE.md   1,743  using the schema (was Part I)
  AUTHORING.md      1,687  extending and authoring it (was Part II + templates)
  CONCEPTS.md         803  why the schema is built this way (was Part 0 + digressions)
  TROUBLESHOOTING.md  370  symptom-indexed errors and model gotchas (was section 12)

Guide sections 4-6 now delegate per-command reference to the package
READMEs under packages/, which are versioned with the code they document,
rather than restating it. SDK and CLI construction moved from Part II into
the guide as section 8 -- that is consumer work, not authoring.

Section 2.5 rewritten around the queries people run rather than an
explanation of JSON Schema: 454 lines to 194. The required-field
derivation, the tag model, and the example-file walkthrough moved to
CONCEPTS.md.

All `uv run python <<'PY'` heredocs converted to plain python blocks, which
also brings them under tests/test_documented_imports.py -- that raised the
checked-block count from 227 to 241.

REVIEW STATE: guide sections 1 through 2.6 have had a full pass. Sections
2.6-8, CONCEPTS.md, and AUTHORING.md are not yet reviewed as prose;
AUTHORING.md is a near-verbatim move of Part II. All code blocks are
parsed, import-checked, and enum-checked by the test suite, and all
internal links and anchors resolve.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>
Signed-off-by: Dana Bauer <dana.bauer@gmail.com>
Phase 3 (#638) replaced publish-python-packages.yaml with
main-publish.yaml and release-publish.yaml, and moved the PySpark
codegen step out of the workflow into a package-owned
packages/<package>/scripts/prebuild.sh. Describe that convention and the
two publish targets (CodeArtifact for interim .postN builds, public PyPI
via Trusted Publishing for version bumps) instead of the workflow that
no longer exists.

Correct the README package count: packages/ holds thirteen packages, as
GLOSSARY.md and SCHEMA_GUIDE.md already said. extensions-operating-hours
has never been on main.

Run the markdown code blocks through ruff format, which make check has
been failing on independently of this rebase.

Signed-off-by: Seth Fitzsimmons <seth@mojodna.net>
@vcschapp
Victor Schappert (vcschapp) merged commit cc025c5 into main Aug 26, 2026
25 checks passed
@vcschapp
Victor Schappert (vcschapp) deleted the docs-consolidation branch August 26, 2026 16:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

change type - documentation - member 📝 Documentation change by Overture member

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[DOCS] Python documentation is unreachable and conflates audiences

5 participants