Skip to content

Fix the Read the Docs build and remove the numpy<1.23 requirement - #33

Merged
danielreardon merged 4 commits into
masterfrom
fix-docs-build
Jul 31, 2026
Merged

Fix the Read the Docs build and remove the numpy<1.23 requirement#33
danielreardon merged 4 commits into
masterfrom
fix-docs-build

Conversation

@danielreardon

@danielreardon danielreardon commented Jul 31, 2026

Copy link
Copy Markdown
Owner

Why

https://scintools.readthedocs.io still shows the old placeholder text ("blah blah this is how you do things") even though the documentation source on master has been correct since the docs rewrite. The source was never the problem — every Read the Docs build was failing, so RTD kept serving the last build that succeeded, which predates the rewrite.

I found four independent failures, each of which had to be fixed before the next became visible.

The four build failures

  1. No .readthedocs.yaml. RTD has required a config file since September 2023; without one the build fails outright. This alone explains the frozen page.

  2. jupyter_sphinx was never installed. conf.py does extensions += ['jupyter_sphinx'], so Sphinx aborted with an ExtensionError before rendering anything.

  3. scintools was not importable during the build. conf.py added docs/source to sys.path (via os.path.abspath('.')) rather than the repository root, so autodoc in reference.rst failed. Now resolved relative to __file__, so it no longer depends on the working directory. Separately, the jupyter-execute cells run in a separate IPython kernel subprocess that does not inherit sys.path, so conf.py now also exports PYTHONPATH.

  4. The build exceeded the RTD time limit. With the imports fixed, the build reached the theta-theta tutorials and ran for over 25 minutes without finishing, because jupyter-execute runs a full theta-theta analysis on every build. Excluding those two tutorials takes the build to 7 seconds, exit 0, with every other page rendering correctly.

Removing the numpy pin

setup.py pinned numpy<1.23.0. This was not harmless: running pip install . downgraded numpy 2.4.6 → 1.22.4 in a working environment, which pulled astropy back with it and left scipy/pandas binary-incompatible. numpy 1.22 also has no wheels for Python 3.11+.

The incompatibility the pin worked around has already been fixed (np.complex_ → np.complex128, np.float128 → np.longdouble); no version-conditional code remains. Verified on numpy 2.4.6: the full test suite passes (74 tests), the ACF class and analytic 2D ACF path work, and a complete Dynspec pipeline (trim_edges / refill / correct_dyn / calc_sspec / calc_acf / get_scint_params) runs end to end.

Also included

  • The theta-theta tutorials added unseeded noise (np.random.normal(0, 20, ...)) and then fit an arc to it, making the build non-deterministic — it could fail outright with ValueError: Fit returned a forward parabola. Both are now seeded. I confirmed this flakiness predates the recent bug-fix work: the same code path on pre-rewrite master returns bit-identical results.
  • docs/build/ was committed (48 stale artifacts, including the HTML containing the old placeholder). Now untracked and gitignored.
  • language = None → 'en' (deprecated in modern Sphinx).

Verification

Built exactly as RTD will: scintools uninstalled, PYTHONPATH cleared.

SPHINX EXIT: 0    ELAPSED: 7s
acf.html dynspec.html genindex.html index.html py-modindex.html
reference.html scint_models.html scint_utils.html search.html
simulation.html thetatheta.html

dynspec.html contains no placeholder text and includes the new content; reference.html shows autodoc is working (it picks up calc_sspec, prep_thetatheta, and svd_reconstruct).

Notes for review

  • Excluding the tutorials is the one judgement call here. Nothing is lost relative to what is published today — the last successful RTD build predates these tutorials and never contained them. Their sources stay in the repository and thetatheta.rst now points readers at them. To publish them again, pre-render their output rather than executing on every build.
  • 27 non-fatal Sphinx warnings remain, mostly docstring formatting ("Unexpected indentation" in dynspec.py, "Missing matching underline" in ththmod.errString). These were invisible before because autodoc could never import the package. They do not fail the build; worth a follow-up.
  • After merging: RTD → Builds → Build version to trigger, and check Admin → Integrations that the GitHub webhook exists, or pushes will never trigger builds.

🤖 Generated with Claude Code

Generated by Claude Code

claude added 4 commits July 31, 2026 01:05
@claude
Fix the Read the Docs build so published docs update again
4dfa853 
The documentation source on master has been correct since the docs
rewrite, but https://scintools.readthedocs.io still served an old build
(complete with the "blah blah this is how you do things" placeholder)
because every Read the Docs build was failing. Four separate problems:

1. No .readthedocs.yaml. Read the Docs has required a config file since
   September 2023; without one the build fails and the last successful
   build keeps being served. Added, pinning Python 3.11 and installing
   both the doc requirements and the package itself.

2. jupyter_sphinx was not installed. conf.py does
   `extensions += ['jupyter_sphinx']`, so Sphinx aborted with an
   ExtensionError before rendering anything. Added docs/requirements.txt
   with sphinx, sphinx_rtd_theme, jupyter_sphinx and ipykernel.

3. scintools was not importable during the build, which broke autodoc in
   reference.rst. conf.py added docs/source to sys.path (via
   os.path.abspath('.')) rather than the repository root. Now resolved
   relative to __file__ so it does not depend on the working directory.
   The jupyter-execute cells run in a separate kernel and do not inherit
   this, which is why .readthedocs.yaml also installs the package.

4. The theta-theta tutorials added unseeded noise
   (np.random.normal(0, 20, ...)) and then fit an arc to it, so the build
   was non-deterministic and could fail outright with "Fit returned a
   forward parabola". Seeded the noise in both tutorials. Verified this
   failure predates the recent bug-fix work: the same code path on the
   pre-rewrite master returns bit-identical results.

Also: stop tracking docs/build (48 stale artifacts, including the
committed HTML containing the old placeholder text) and add it to
.gitignore, and replace the deprecated `language = None` with 'en'.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EG6cv3jHNLYv1TN2YWDqi3
@claude
docs: import scintools without installing it, avoiding the numpy pin
f4e45cf 
The jupyter-execute cells in the theta-theta tutorials run in a separate
IPython kernel subprocess, which does not inherit the sys.path entry that
conf.py adds for autodoc, so they still failed with
"ModuleNotFoundError: No module named 'scintools'".

Installing the package on Read the Docs would fix the import but pulls in
setup.py's `numpy<1.23.0` pin. Verified locally that `pip install .`
silently downgrades numpy (2.4.6 -> 1.22.4); numpy 1.22 has no wheels for
Python 3.11, so that would break the docs environment.

Instead, conf.py now also exports PYTHONPATH so the kernel subprocess can
import scintools from the source tree, and docs/requirements.txt lists the
runtime dependencies (numpy, scipy, matplotlib, astropy, lmfit) explicitly
rather than installing the package. .readthedocs.yaml no longer performs a
pip install of the project.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EG6cv3jHNLYv1TN2YWDqi3
@claude
Remove the numpy<1.23.0 version pin
f1671fd 
The pin dated from before the package supported NumPy 2. It forced users
onto a three-year-old NumPy that has no wheels for Python 3.11+, and
installing scintools would silently downgrade a working environment
(verified: pip install . took numpy 2.4.6 -> 1.22.4, which in turn
downgraded astropy and left scipy/pandas binary-incompatible).

The incompatibility the pin was working around has since been fixed:
np.complex_ (removed in NumPy 2.0) is now np.complex128, and np.float128
is now np.longdouble. No version-conditional code remains.

Verified on numpy 2.4.6: the full test suite passes (74 tests), the ACF
class and the analytic 2D ACF path work, and a complete Dynspec pipeline
(trim_edges / refill / correct_dyn / calc_sspec / calc_acf /
get_scint_params) runs end to end. CI additionally covers Python
3.8-3.10, where the resolver picks the newest NumPy valid for each.

Also updates the docs build comments, which previously cited the pin as
the reason for not installing the package on Read the Docs; the remaining
reason is simply to keep the docs environment small.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EG6cv3jHNLYv1TN2YWDqi3
@claude
docs: exclude the theta-theta tutorials from the build (build timeout)
3658599 
With the import problems fixed, the build got as far as executing the
tutorials and then ran for over 25 minutes without finishing. The two
theta-theta tutorials use `jupyter-execute`, so a full theta-theta
analysis of the sample data runs on every build; that exceeds the Read the
Docs build limit, so the build would still have timed out and left stale
documentation published.

Excluding them takes the build from >25 minutes (never completing) to 7
seconds, exit 0, with every other page rendered correctly. Nothing is lost
relative to what is currently published: the last successful build on Read
the Docs predates these tutorials and never contained them.

The tutorial sources stay in the repository, and thetatheta.rst now points
readers at them and explains why they are not rendered. To publish them
again, pre-render their output rather than executing on every build.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EG6cv3jHNLYv1TN2YWDqi3
@danielreardon danielreardon changed the title Fix the Read the Docs build and remove the numpy&lt;1.23 pin Fix the Read the Docs build and remove the numpy-1.23 requirement Jul 31, 2026
@danielreardon danielreardon changed the title Fix the Read the Docs build and remove the numpy-1.23 requirement Fix the Read the Docs build and remove the numpy<1.23 requirement Jul 31, 2026
danielreardon
danielreardon commented Jul 31, 2026
View reviewed changes

@danielreardon danielreardon left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All documentation changes look good

@danielreardon
danielreardon merged commit 44d2605 into master Jul 31, 2026
3 checks passed
@danielreardon
danielreardon deleted the fix-docs-build branch July 31, 2026 02:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@danielreardon @claude