Skip to content

fix(security): pin arXiv CA everywhere, ship bundle - #576

Merged
chris-dare-dev merged 2 commits into
mainfrom
fix/batch-j-ca-pinning
Nov 21, 2026
Merged

chris-dare-dev merged 2 commits into
mainfrom
fix/batch-j-ca-pinning

Conversation

@chris-dare-dev

Copy link
Copy Markdown
Owner

arXiv CA pinning now works from an install and on every fetch path.

#324: pin on every fetch path

  • Before, ARXMCP_PIN_ARXIV_CA=1 built a pinned SSLContext that no production caller used. Every download trusted the system store, and the server printed a startup warning that admitted it.
  • New helper server.ssl_pin.arxiv_ssl_context_from_env loads Config from the environment, so it runs the same fail-closed bundle check as server startup. All six call sites pass its result: bulk ingest, and the seed, one-paper, notebook and preamble back-fill fetchers.
  • With pinning on, a missing or unreadable bundle now stops the fetch instead of falling back to the system store.
  • The startup warning is gone.
  • The tests drive each entry point with a recording urlopen and check which context it receives:
    • the pinned bundle with the flag on;
    • the operator's bundle when it is overridden;
    • None with the flag off;
    • no request at all when the bundle is missing.

#343: ship the bundle

  • The bundle lived at infra/ca/, outside every package, so no wheel or image carried it. On any install that wasn't a source checkout, turning pinning on failed with "CA bundle not found".
  • It now lives at server/ca/arxiv-ca-bundle.pem and ships through a server.ca package-data glob.
    • server.config takes the path from server.ssl_pin instead of rebuilding it.
    • make refresh-arxiv-ca writes to the new location.
  • make wheel-check now requires the installed file, and the packaging test names it.
  • The server image builds the pinned context from the installed wheel, so a release that drops the bundle fails before it ships.

Closes #324
Closes #343

Verification

  • ruff check . is clean, and make wheel-check passes.
  • Full suite on this branch: 5560 passed, 182 skipped, 1 xfailed (Python 3.12, Ubuntu). No test touches the network.
  • This branch merged with the write-safety and server-runtime batches on 5ac66b2: 5593 passed, 182 skipped, 1 xfailed.
  • Not built: the Docker image. Its new pinned-context build step has not run yet.

This PR and the write-safety PR both touch ingest/bulk_ingest.py, and git merges them cleanly.

Any wiki page that gives the bundle's path as infra/ca/ is now out of date.

ARXMCP_PIN_ARXIV_CA=1 built a pinned SSLContext that no production
caller used: bulk ingest and the seed, one-paper, notebook and
preamble back-fill fetchers all called try_cache / fetch_eprint
with the default context, so every download trusted the system
store. The server printed a startup warning admitting the gap.

Add server.ssl_pin.arxiv_ssl_context_from_env, which loads Config
from the environment (and so runs the same fail-closed bundle check
as server startup), and pass its result at all six call sites. A
missing or unparseable bundle now stops the fetch instead of
falling back to the system store. Drop the startup warning.

The new tests drive each entry point with a recording urlopen and
check the context it receives: pinned bundle with the flag on,
operator bundle when overridden, None with the flag off, and no
request at all when the bundle is missing.

Closes #324
The pinned bundle lived at infra/ca/, outside every package, so no
wheel and no image carried it. On any install that was not a source
checkout, ARXMCP_PIN_ARXIV_CA=1 failed Config validation with
"CA bundle not found", and pinning could not be turned on at all.

Move it to server/ca/arxiv-ca-bundle.pem and ship it through a
server.ca package-data glob. server.config now takes the path from
server.ssl_pin instead of rebuilding it, and make refresh-arxiv-ca
writes to the new location.

wheel-check now requires the installed file, the packaging test
names it, and the server image builds the pinned context from the
installed wheel, so a release that drops it fails before it ships.

Closes #343
@chris-dare-dev
chris-dare-dev merged commit 96ddda1 into main Nov 21, 2026
4 checks passed
@chris-dare-dev
chris-dare-dev deleted the fix/batch-j-ca-pinning branch November 21, 2026 04:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant