Skip to content

ci: pin apt hosts through DNS over HTTPS when the runner can't resolve them - #175

Merged
merge-loop[bot] merged 2 commits into
mainfrom
cb/ci-apt-mirror-fallback
Nov 11, 2026
Merged

merge-loop[bot] merged 2 commits into
mainfrom
cb/ci-apt-mirror-fallback

Conversation

@joaquim-verges

@joaquim-verges joaquim-verges commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Problem

Since about 14:40 UTC on 2026-11-11, GitHub's Ubuntu runners can't resolve the ubuntu.com apt hosts: azure.archive.ubuntu.com, and also archive.ubuntu.com (this PR's first run proved that). Every apt step retries for minutes and then fails: playwright install --with-deps in checks and determinism / flake-check, and the runner image's apt-get in determinism / runner-image. That turns the required build check red on any PR. #174 failed three attempts in a row this way, and release.yml has the same step, so a release would fail too.

Change

  • scripts/apt-mirror-fallback.sh checks each apt host (azure.archive.ubuntu.com, archive.ubuntu.com, security.ubuntu.com). If a host doesn't resolve on the runner, it looks the host up through Cloudflare's DNS over HTTPS (falling back to Google's) and pins the answer in /etc/hosts for that job. It also exports the pins as --add-host flags for docker build, because the Playwright base image names the same mirror. While the names resolve, it does nothing.
  • The script runs before the apt steps in ci.yml, determinism.yml and release.yml.
  • images/runner/Dockerfile is unchanged, so the image digest and baselines aren't invalidated.

Verification

  • On Eng Computer, where the names resolve, the script makes no change.
  • With getent stubbed to fail, the script pins all three hosts from DNS over HTTPS and emits the --add-host list.
  • The outage is still live, so this PR's own CI runs the fallback path.

🤖 Generated with Claude Code

…n't resolve it

Since 2026-11-11 ~14:40 UTC GitHub's Ubuntu runners cannot resolve
azure.archive.ubuntu.com, so 'playwright install --with-deps' and the runner
image's apt-get retry for minutes and fail every CI run (bugpatrol#174 failed
three attempts the same way). The Ubuntu archive serves the same paths under
that host name, so scripts/apt-mirror-fallback.sh maps the name to
archive.ubuntu.com in /etc/hosts for the job, and docker build gets the same
mapping through --add-host. It does nothing while the mirror resolves, and the
runner Dockerfile is untouched, so the image digest and baselines don't change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Nebula AI Agent <277912164+nebula-ai-agent[bot]@users.noreply.github.com>
@vercel

vercel Bot commented Nov 11, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
bugpatrol Ready Ready Preview Nov 11, 2026 3:22pm UTC

Request Review

The first fallback mapped the Azure mirror to archive.ubuntu.com, but on the
runners archive.ubuntu.com does not resolve either (#175's first run exited 2
in getent), so the ubuntu.com names are gone from the runner's resolver as a
whole. Look each apt host up through Cloudflare's or Google's DNS over HTTPS
instead and pin the answer, and pass every pin to docker build.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Nebula AI Agent <277912164+nebula-ai-agent[bot]@users.noreply.github.com>
@joaquim-verges joaquim-verges changed the title ci: map the Azure apt mirror to archive.ubuntu.com when the runner can't resolve it ci: pin apt hosts through DNS over HTTPS when the runner can't resolve them Nov 11, 2026
@joaquim-verges joaquim-verges added the mergepatrol Queue this PR: Mergepatrol rebases, tests and merges it label Nov 11, 2026 — with Nebula AI Agent
@merge-loop

merge-loop Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Merged as dab7c7b.

@merge-loop
merge-loop Bot merged commit dab7c7b into main Nov 11, 2026
8 checks passed
@merge-loop merge-loop Bot removed the mergepatrol Queue this PR: Mergepatrol rebases, tests and merges it label Nov 11, 2026

This branch was successfully deployed

1 active deployment
Preview — 06209ce0 Deployed Nov 11, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant