Repository navigation
fix(security): pin arXiv CA everywhere, ship bundle - #576
Merged
Merged
Conversation
ARXMCP_PIN_ARXIV_CA=1 built a pinned SSLContext that no production caller used: bulk ingest and the seed, one-paper, notebook and preamble back-fill fetchers all called try_cache / fetch_eprint with the default context, so every download trusted the system store. The server printed a startup warning admitting the gap. Add server.ssl_pin.arxiv_ssl_context_from_env, which loads Config from the environment (and so runs the same fail-closed bundle check as server startup), and pass its result at all six call sites. A missing or unparseable bundle now stops the fetch instead of falling back to the system store. Drop the startup warning. The new tests drive each entry point with a recording urlopen and check the context it receives: pinned bundle with the flag on, operator bundle when overridden, None with the flag off, and no request at all when the bundle is missing. Closes #324
The pinned bundle lived at infra/ca/, outside every package, so no wheel and no image carried it. On any install that was not a source checkout, ARXMCP_PIN_ARXIV_CA=1 failed Config validation with "CA bundle not found", and pinning could not be turned on at all. Move it to server/ca/arxiv-ca-bundle.pem and ship it through a server.ca package-data glob. server.config now takes the path from server.ssl_pin instead of rebuilding it, and make refresh-arxiv-ca writes to the new location. wheel-check now requires the installed file, the packaging test names it, and the server image builds the pinned context from the installed wheel, so a release that drops it fails before it ships. Closes #343
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
arXiv CA pinning now works from an install and on every fetch path.
#324: pin on every fetch path
ARXMCP_PIN_ARXIV_CA=1built a pinnedSSLContextthat no production caller used. Every download trusted the system store, and the server printed a startup warning that admitted it.server.ssl_pin.arxiv_ssl_context_from_envloadsConfigfrom the environment, so it runs the same fail-closed bundle check as server startup. All six call sites pass its result: bulk ingest, and the seed, one-paper, notebook and preamble back-fill fetchers.urlopenand check which context it receives:Nonewith the flag off;#343: ship the bundle
infra/ca/, outside every package, so no wheel or image carried it. On any install that wasn't a source checkout, turning pinning on failed with "CA bundle not found".server/ca/arxiv-ca-bundle.pemand ships through aserver.capackage-data glob.server.configtakes the path fromserver.ssl_pininstead of rebuilding it.make refresh-arxiv-cawrites to the new location.make wheel-checknow requires the installed file, and the packaging test names it.Closes #324
Closes #343
Verification
ruff check .is clean, andmake wheel-checkpasses.5ac66b2: 5593 passed, 182 skipped, 1 xfailed.This PR and the write-safety PR both touch
ingest/bulk_ingest.py, and git merges them cleanly.Any wiki page that gives the bundle's path as
infra/ca/is now out of date.