Skip to content

Container test harness for install, plus the bugs it found - #1

Merged
jsg2021 merged 2 commits into
mainfrom
install-test-harness
Aug 22, 2026
Merged

Container test harness for install, plus the bugs it found#1
jsg2021 merged 2 commits into
mainfrom
install-test-harness

Conversation

@jsg2021

@jsg2021 jsg2021 commented Aug 22, 2026

Copy link
Copy Markdown
Owner

./install now runs end to end in a disposable Linux container, so a broken installer is caught before it touches a real machine.

./tests/run              # fast checks, no Docker (~2s)
./tests/run --container  # full install in a sandbox (~40s)

The repo is mounted read-only; the container copies it to ~/.dotfiles inside itself. Everything that downloads, installs, or needs root is shadowed by a stub that records the call, and --network none is passed so a missing stub fails loudly instead of quietly reaching the internet. Only the stubs for the container's real package manager are kept, so a Debian box cannot take the dnf branch. --real drops both for a genuine run.

Seven scenarios: fresh machine, running twice, dirty repo, a machine with hand-written dotfiles, codespaces, the README's piped bootstrap, and the whole install again with uname -m faked to the other CPU architecture.

CI runs the unit tests on Linux and macOS, the container scenarios on Debian and Fedora, and shellcheck at error severity.

Bugs it found

Every one of these is fixed in the first commit.

Install died on Debian and Ubuntu. Installers were invoked as sh some-bash-script, which overrides their #!/bin/bash shebang. On Debian sh is dash, so source failed and the whole install aborted at the first step. They are now executed directly, and the exec bit is set on the nine that were missing it.

Codespaces never linked any configs. A return outside a function ended the script before stow ran.

Fedora installed the wrong packages. The dnf alias map overwrote the shared one, so it ran dnf install rg instead of ripgrep (same for nvim, delta).

chsh -s ran with no argument on Fedora 41+, which has no which.

scripts/installers/docker never skippedif [ command -v docker ] is not valid test syntax, so the check never passed.

ghostty matched both AppImages and piped two URLs into a single curl -o.

diffnav was broken on every arm Linux box — it asked for diffnav_Linux_aarch64; the asset is named arm64.

wezterm downloaded the x86_64 AppImage on arm. Upstream ships no arm AppImage, so it now uses the arm64 .deb on apt distros and skips with a reason where no build exists.

vscode and docker hardcoded Apple Silicon URLs.

app-image-launcher assigned a download URL to PATH, so the very next command was not found. It also prefixed https://github.com onto an already absolute URL, and removed a package named AppImageLauncher when the package is appimagelauncher.

Other changes

stow now gets an explicit --dir/--target, so install works from any directory and from the codespaces checkout, not just ~/.dotfiles. The README's curl … | sh is now | bash, and install falls back to $DIR when $0 is not a real path, as it is not when piped.

Binary lookups are unified on has_binary (plus a new binary_path for when the path itself is needed), so the rule that an installer script must not shadow the binary it installs applies everywhere.

New arch_for, because projects spell architectures three different ways, and github_asset_url, which refuses when a filter matches more than one asset rather than guessing — that is exactly the ghostty bug, now impossible.

app-image-launcher moved under app/ with the other per-app installers and gained an apt/.deb path. It is deliberately not wired into apps; it stays a manual, interactive script.

How the config-link checks work

The core dotfiles are asserted by name, so a silent change to the stow rules cannot pass. Everything else is derived at run time from stow --simulate, so adding a config needs no test change, and anything stow skipped must be explained by a .stow-local-ignore rule. That is how configs/.profile turned up — a tracked symlink a hand-written list had missed.

The fake curl serves each GitHub repo its own release listing carrying every architecture a real release carries, so an installer whose filter does not pin the architecture matches two assets and fails.

Test results

Debian aarch64 126 unit + 153 container, 0 failed
Fedora aarch64 126 unit + 154 container, 0 failed
shellcheck --severity=error clean

Not verified

The real apt-get install of the AppImageLauncher .deb was never completed — Debian mirrors time out from the machine this was developed on. The asset URL selection, the package name, and the command form were all verified against the live release; an actual install was not. Same reason --platform linux/amd64 could not be exercised locally, though CI runs x86_64 natively.

🤖 Generated with Claude Code

jsg2021 and others added 2 commits August 22, 2026 12:43
Found by the new test harness running ./install in a throwaway container.

Portability
- Installers were invoked as `sh some-bash-script`, which overrides their
  #!/bin/bash shebang. On Debian sh is dash, so `source` failed and the whole
  install aborted at the first step. They are now executed directly and the
  exec bit is set on the ones that were missing it.
- `return` outside a function ended the script early in codespaces, so configs
  were never stowed. It also silently broke app-image-launcher's dnf branch.
- stow now gets an explicit --dir/--target, so install works from any cwd and
  from the codespaces checkout, not just ~/.dotfiles.
- `curl ... | sh` in the README is now `| bash`, and install falls back to $DIR
  when $0 is not a real path (as it is not when piped).
- set-zsh used `which`, absent on Fedora 41+, so it ran `chsh -s` with an empty
  argument.
- `if [ command -v docker ]` is not valid test syntax; the check never passed.

One way to look up a binary
- has_binary everywhere instead of command -v, so the rule that an installer
  script must not shadow the binary it installs applies consistently.
- New binary_path for when the path itself is needed.
- The dnf alias map overwrote the shared one, so Fedora installed `rg` and
  `nvim` rather than ripgrep and neovim. It now only overrides.

Architecture
- New arch_for, since projects spell architectures three different ways, and
  github_asset_url, which refuses when a filter matches more than one asset.
- ghostty matched both AppImages and piped two URLs into one `curl -o`.
- diffnav asked for diffnav_Linux_aarch64; the asset is named arm64.
- wezterm downloaded the x86_64 AppImage on arm. It now uses the arm64 .deb on
  apt distros and skips with a reason where upstream ships no build.
- vscode and docker hardcoded Apple Silicon URLs.

app-image-launcher
- Moved under app/ alongside the other per-app installers. Not wired into
  `apps`: it stays a manual, interactive script.
- Added an apt/.deb path, arch-aware on both branches.
- It assigned a download URL to PATH, so the very next command was not found.
- It prefixed https://github.com onto an already absolute URL.
- The package is `appimagelauncher`, not `AppImageLauncher`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
./install now runs end to end in a disposable Linux container, so a broken
installer is caught before it touches a real machine.

    ./tests/run              fast checks, no Docker
    ./tests/run --container  full install in a sandbox

The repo is mounted read-only; the container copies it to ~/.dotfiles inside
itself. Everything that downloads, installs, or needs root is shadowed by a
stub that records the call, and --network none is passed so a missing stub
fails loudly rather than quietly reaching the internet. Only the stubs for the
container's real package manager are kept, so a Debian box cannot take the dnf
branch. --real drops both for a genuine run.

Seven scenarios: fresh machine, running twice, dirty repo, a machine with
hand-written dotfiles, codespaces, the README's piped bootstrap, and the whole
install again with uname -m faked to the other CPU architecture.

Config links are checked two ways. The core dotfiles are asserted by name so a
silent change to the stow rules cannot pass. Everything else is derived at run
time from `stow --simulate`, so adding a config needs no test change, and
anything stow skipped must be explained by a .stow-local-ignore rule. That is
how configs/.profile turned up, which a hand-written list had missed.

The fake curl serves each GitHub repo its own release listing carrying every
architecture a real release carries, so an installer whose filter does not pin
the architecture matches two assets and fails.

Unit tests cover the helpers functions and lint the repo: script syntax, JSON
validity, exec bits, and the house rules (no `sh $script`, no `command -v`,
no `which`).

CI runs the unit tests on Linux and macOS, the container scenarios on Debian
and Fedora, and shellcheck at error severity.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@jsg2021
jsg2021 merged commit 82f9fa2 into main Aug 22, 2026
5 checks passed
@jsg2021
jsg2021 deleted the install-test-harness branch August 22, 2026 17:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant